Systems and subsystems
Inventories and the shortcut ledger
Before any model trained, scripts measured how the training and test data differ in ways that have nothing to do with the voice. Every difference found is a shortcut a model would learn instead of the task, and each one is closed somewhere in the pipeline. The ledger is the list.
In plain words
A detector with a perfect holdout score can be measuring the microphone, the file format or the loudness meter. The inventory scripts ask, for each cheap property of a file, how well it predicts the label on its own; anything that predicts too well is a trap. The ledger records the trap, the evidence and the fix, and every new detector or dataset is checked against it.
What goes in and comes out
In: a directory of audio (and labels, when known). Out: per-file probe and decode statistics, per-class format and duration tables, shortcut AUCs with the suspicious ones flagged, container facts, and hash indexes for leakage checks.
Where it lives
inventory.py (probe and decode statistics, shortcut AUCs), inventory_container.py (header facts, the container crosstab), hash_audio.py (byte and PCM hashes, duplicate groups, cross-set matches). The ledger is section 11 of the architecture document.
What each measures
- Decode statistics per file: length, peak, leading and trailing silence, voiced fraction; shortcut AUCs for duration, silence, voiced fraction, peak, sample rate and bit rate, flagged beyond a fixed margin from chance (inventory.py, lines 108–118).
- Container facts through the container detector's own probe: format, codec, rate, channels, bits, tags; embedded fields only, never filesystem timestamps or names (inventory_container.py).
- Hashes of the file bytes and of the decoded PCM, so exact and re-muxed copies are found across sets; lossy re-encodes are not caught (hash_audio.py).
Status
Run at every data drop. The leakage check across LJ Speech, the sponsor's LJ subset and In-the-Wild found no exact copy of any test file.
The ledger
From the shortcut ledger:
| Shortcut | Evidence | Closed by |
|---|---|---|
| Peak level | test peaks near 1.0source: docs/architecture.md, section 11-shortcut-ledger, training real speech about 0.5source: docs/architecture.md, section 11-shortcut-ledger; AUC 0.66source: docs/architecture.md, section 11-shortcut-ledger alone | per-input normalization in both paths; level is never a feature |
| Leading silence | LibriSpeech 0.37source: docs/architecture.md, section 11-shortcut-ledger s vs test 0.06source: docs/architecture.md, section 11-shortcut-ledger s | the same silence trim on train and test |
| Duration and tiling | training 5–9 s vs test about 3.4source: docs/architecture.md, section 11-shortcut-ledger s; repeat-padding put a seam only in test data | segment mode: no tiling, training crops drawn from the test-duration distribution |
| Container and codec | real = FLAC or 22 kHz WAV, fakes include MP3, test = 16 kHz PCM with one FFmpeg tag (also on PlayHT) | one decode path; the container detector is never learned; laundering-equalized compression training |
| The 7.2 kHzsource: docs/architecture.md, section 11-shortcut-ledger low-pass | 1,602source: docs/architecture.md, section 11-shortcut-ledger of 1,671source: docs/architecture.md, section 11-shortcut-ledger test files drop 44source: docs/architecture.md, section 11-shortcut-ledger dB between 6.5source: docs/architecture.md, section 11-shortcut-ledger and 7.5source: docs/architecture.md, section 11-shortcut-ledger kHz; no training corpus does | the band match in both paths |
| Mains hum | stable hum in 36%source: docs/architecture.md, section 11-shortcut-ledger of LJ, 21%source: docs/architecture.md, section 11-shortcut-ledger of LibriSpeech, under 0.5%source: docs/architecture.md, section 11-shortcut-ledger of nine generators | ENF is never learned |
| Crop edges | training crops start mid-word, test clips are whole utterances | no onset or offset features |
| LJ single speaker | one voice on both sides of every fold | folds group LJ by chapter; the stacker was fit on non-LJ real rows |
| Non-speech | silence and a chord score near 1.0source: docs/architecture.md, section 11-shortcut-ledger with the probe | the speech gate and the default-answer policy |
The band inventory of the test set: 981source: README.md, section orchestration-what-routing-changes files read 7,250source: README.md, section orchestration-what-routing-changes Hz, 670source: README.md, section orchestration-what-routing-changes read 7,500source: README.md, section orchestration-what-routing-changes Hz, 12 read full band and 8 sit at 5–7 kHz. The later λ̂ correction found the same class of shortcut again, in a noise-floor feature that read the stopband.
Known limits
Inventories find the differences someone thought to measure. The test-domain offset in the handcrafted features (a darker envelope even after the band match) was found by a domain classifier, not by the inventory.
Tests that pin it
The inventory scripts have no direct tests; the PCM hash is pinned through tests/test_docker_image.py, and the shortcut gate in the M5 trainer refuses a bundle whose duration, peak or silence AUC exceeds a limit (tests/test_m5_manifest.py).
Sources
Architecture §11; the CPU detectors report ("Trap 1", "Trap 2"); STATUS, "Findings so far".