Systems and subsystems

The M5 head on XLS-R

A trained head on the first twelve layers of XLS-R: learned layer weights, attentive statistics pooling and a linear output, trained on rented A100s with heavy class-blind augmentation. It failed its gate as a replacement for the probe and ships as a fifth of the fusion vote, because it errs on different files.

In plain words

The plan for M5A trained head on a truncated XLS-R: learned layer weights, attentive pooling and a linear output, trained on rented GPUs with heavy augmentation. It did not beat the plain probe on its own and ships as a minority vote in the fusion. was to fine-tune XLS-R and beat the frozen probe. Fine-tuning the kept layers fit the seen generators and transferred worse to the unseen one, so the shipped arm keeps the backbone frozen and trains only a small head over the hidden states. On its own that head lost to the probe on the holdout, on LibriSpeech real speech and on short clips, and won only on In-the-Wild false alarms. The pre-declared M5 sweep then showed that as a minority vote beside the probe and the handcrafted model it improves every readout, which is the role it ships in.

What goes in and comes out

In: trimmed, band-limited, capped and normalized audio (the deploy transform). Out: one logit, high for synthetic, mapped to a probability for the sentence "XLS-R fine-tuned head (M5, 12 layers, attentive pooling): logit +x.xx (synthetic-like, P=0.xx)". Export column m5_xlsr_ft.

Where it lives

Model in m5_model.py; data, folds and the deploy transform in m5_data.py; augmentation in augment.py; the bundle in m5_bundle.py; trainer, scorer and assembler in scripts/m5_*.py; the vast.ai launch and reaper scripts under scripts/cloud/; the runner's path in pipeline.py, lines 485–495.

Parameters and where each is set

• Architecture: 12 kept layers, a softmax-weighted sum over the hidden states, masked attentive statistics pooling with a bottleneck of 128source: src/hearsay/m5_model.py line 27 (GitHub; assumes main is public) (mean and standard deviation concatenated), a linear head (m5_model.py, lines 21–110). The CNN feature encoder is always frozen; with train_top = 0 every backbone parameter is frozen, which is the shipped arm.
• Training recipe (defaults in M5Config): layer-wise learning-rate decay 0.85source: src/hearsay/m5_model.py line 29 (GitHub; assumes main is public), weight decay 0.01source: src/hearsay/m5_model.py line 31 (GitHub; assumes main is public), warmup 0.08source: src/hearsay/m5_model.py line 32 (GitHub; assumes main is public) then cosine, label smoothing 0.05source: src/hearsay/m5_model.py line 34 (GitHub; assumes main is public), gradient clip 1.0source: src/hearsay/m5_model.py line 36 (GitHub; assumes main is public), batches of 32 crops, bf16 on the A100; 3,000source: docs/reports/2026-09-26_m5-xlsr-finetune.md steps for the full model. A direction tripwire aborts a fold model whose validation AUC is not above chance.
• Batch mix (m5_model.py, lines 43–47): real speech LJ 0.25source: src/hearsay/m5_model.py line 45 (GitHub; assumes main is public), LibriSpeech 0.35source: src/hearsay/m5_model.py line 45 (GitHub; assumes main is public), ASVspoof 2019 0.40source: src/hearsay/m5_model.py line 45 (GitHub; assumes main is public); spoof DiffSSD 0.78source: src/hearsay/m5_model.py line 45 (GitHub; assumes main is public), MLAAD 0.12source: src/hearsay/m5_model.py line 45 (GitHub; assumes main is public), ASVspoof 0.10source: src/hearsay/m5_model.py line 45 (GitHub; assumes main is public); label-balanced batches, half and half.
• Augmentation, class-blind: applied with probability 0.65source: src/hearsay/m5_model.py line 39 (GitHub; assumes main is public), one op with probability 0.6source: src/hearsay/augment.py line 127 (GitHub; assumes main is public) else two, RawBoost appended with probability 0.25source: src/hearsay/m5_model.py line 40 (GitHub; assumes main is public): white or pink noise at 5–30 dB SNR, telephony band-limits at 3,400source: src/hearsay/augment.py line 27 (GitHub; assumes main is public) or 4,000source: src/hearsay/augment.py line 27 (GitHub; assumes main is public) Hz, reverb with RT60 0.2source: src/hearsay/augment.py line 28 (GitHub; assumes main is public)–0.8source: src/hearsay/augment.py line 28 (GitHub; assumes main is public) s, gain and clipping, notch-and-nonlinearity, codec round-trips (MP3, Opus, AAC, AMR-NB, μ-law) on a share of rows and every holdout row; then the deterministic band match last (augment.py, lines 25–163).
• Fold discipline: the fold files are hash-pinned; a fold model never sees its own fold, a holdout row, or the generator families that validate on that fold (OpenVoice, XTTS, ElevenLabs); In-the-Wild paths are refused (m5_data.py, lines 28–40 and lines 166–179).
• Checkpoint pinning: the saved model carries hashes of its config, backbone and head; the loader and the runner refuse a mismatch, and a constants file that weights M5 must record those hashes (m5_model.py, lines 173–223; pipeline.py, lines 351–364).

Status

Gate not passed as a replacement; shipped at weight 0.2source: README.md, section how-it-decides since the noon switch. Twelve A100 instances were rented (six never booted), $5.42source: README.md, section what-did-not-work in total; the checkpoint is also on the Hugging Face Hub, private.

Numbers

Alone: inner 0.302source: README.md, section numbers, holdout 0.363source: README.md, section numbers against 0.159source: README.md, section what-did-not-work for the probe, In-the-Wild false alarms 0.45%source: README.md, section numbers against 0.7%source: README.md, section what-did-not-work, clips under 4 s at 0.60source: README.md, section what-did-not-work; the fine-tuned arms scored 0.38–0.41source: README.md, section what-did-not-work on the ElevenLabs fold against 0.31source: README.md, section what-did-not-work frozen. In the fusion: holdout 0.014source: README.md, section what-worked → 0.0065source: README.md, section what-worked, In-the-Wild 0.260source: README.md, section what-worked → 0.228source: README.md, section what-worked. Live CPU against the A100 export, the logit differs by at most 0.035source: README.md, section score-a-directory-on-the-mac (median 7.0e-4source: README.md, section score-a-directory-on-the-mac).

Known limits

The final encoder LayerNorm and the mask embedding were unintentionally trainable in the shipped run (a few thousand parameters); fixed afterwards and pinned by a test. The M5 addendum's first In-the-Wild run scored whole clips instead of the shared crops and was repeated crop-fair; both are reported. On MLAAD its miss rate is the in-sample one.

Tests that pin it

tests/test_m5_model.py (frozen and trainable sets with a counterfactual, padding invariance, direction, determinism, save and load hash refusal, layer-wise decay and no-decay rules, layer weights sum to one), test_m5_bundle.py, test_m5_manifest.py (no holdout row in any training set, the fold-file hashes, shortcut AUCs), test_m5_crops.py, test_m5_codecs.py, test_m5_scores.py, test_m5_train.py (smoke runs, resume, the tree-hash refusal), test_augment.py, test_cloud_scripts.py; in test_pipeline.py the identity gate and the live logit's agreement with the scorer.

Sources

The M5 report; the M5 addendum; the extra-data consult; the expense ledger.