자동차 소리로 고장 진단하는 오디오 ML 파이프라인 'cardiag'

cardiag — diagnose a car fault from its sound cardiag is an end-to-end audio-ML pipeline. It scrapes fault-sound clips from YouTube/TikTok, cleans the audio (isolating the mechanical sound from speech, music, and noise), embeds it with a frozen CLAP model, and trains small linear heads to triage the fault. It is exposed as a CLI and a live web app. cardiag-demo.mp4 This is a proof of concept, and honest about what that means. Diagnosing a car fault from a phone recording is genuinely hard, so cardiag is built as a calibrated triage aid rather than a diagnoser: it tells you whether something sounds wrong, roughly where in the car it is, and a ranked shortlist of likely parts. When the audio won't support a call, it says "uncertain" instead of bluffing. The real contribution is the cleaning + honest-training recipe, which is reusable on other audio datasets. The modest accuracy here reflects how hard the problem is from crude phone audio (we hit the literature ceiling); the same method reaches 0.93 AUROC on clean engine audio. See docs/DEFENSE.md . Interactive demos Two pages visualize the first two stages of the pipeline: Isolating the engine audio — an interactive look at the clean() cascade pulling a short mechanical span out of noisy YouTube audio (speech, music, road noise). CLAP, visualized — how the frozen CLAP model turns those spans into the 512-d embedding the linear heads classify. What it actually achieves Measured out-of-sample, leakage-safe (by-video grouped CV over 1,031 video groups; permutation p = 0.0005 ). These are honest numbers, not a leaderboard. Capability Result vs. chance Is something wrong? (fault/normal) AUROC 0.79 [0.76, 0.83] 0.50 Where in the car? (6 zones) right zone in top-3 ≈ 75% 2× Which part? (12+ families) right part in top-3 ≈ 45–65% 3–4× Knows when it doesn't know calibrated (ECE ≈ 0.04), returns UNCERTAIN — Full details, and the one head we demoted for failing out-of-sample (knock), are in docs/MODEL_CARD.md . Quickstart: clone to inference A fresh clone is immediately usable. A small pre-trained model ships in models/ , and a synthetic demo clip is bundled, so nothing needs to be downloaded or scraped. git clone < this-repo > && cd car-diagnosis uv venv && source .venv/bin/activate uv pip install -e " .[scrape,web,dev,viz] " # Python 3.11 cardiag doctor # preflight: what's installed cardiag train --fixtures # a working model offline in ~2s (no scrape, no 2 GB download) cardiag diagnose < clip.wav > # verdict + where-in-the-car + ranked parts cardiag serve --model models # live web app: drop a clip / paste a link, "explain why" Verify the whole thing end-to-end in an isolated worktree: bash scripts/clone_verify.sh . How it works audio ──► clean() cascade ──► CLAP embedding ──► linear heads ──► Diagnosis (isolate spans) (frozen, 512-d) (fault/region/ (calibrated, part/knock) UNCERTAIN-aware) There is one segmentation path . Scraped clips, your own recordings ( cardiag ingest , any length), and uploads at inference all flow through the same clean() cascade that isolates short mechanical spans. Spans over ~10 s are split into windows so CLAP never silently truncates them. Training and serving share one embedding contract, so there is no train/serve skew. Usage cardiag diagnose clip.wav # full model: verdict + region + ranked parts cardiag triage clip.wav # calibrated engine-vs-running-gear cardiag clean clip.wav # isolate the mechanical sound (no model needed) cardiag inspect clip.wav -o r.html # SEE/HEAR the pipeline: spans, spectrograms, scores cardiag ingest ./my_audio --kind fault --cause wheel_bearing # bring your own audio cardiag scrape youtube | tiktok # build a corpus (Reddit is deprecated — too noisy) cardiag train # train on your corpus Add --json to any inference command for machine-readable output. Documentation docs/DEFENSE.md — the honest case that a deliberately crude method earns a real triage result. docs/MODEL_CARD.md — per-head metrics, intended use, limitations. docs/architecture.md — pipeline diagrams. docs/scraping-guide.md — start-to-finish corpus building. Scope & honesty Valid for social-style / targeted-upload audio (YouTube, TikTok, or a phone clip a user records deliberately). It is not a safety-critical or standalone diagnostic. It is a triage assistant that narrows where to look and is honest about its uncertainty. Model files are joblib artifacts: load only ones you trust. License: see LICENSE .