Systems and subsystems
The fold file
One tracked CSV assigns every training clip to an inner fold or the outer holdout, grouping fakes by generator and real speech by speaker, so that no model is ever tested on a voice or a synthesis system it trained on.
In plain words
Splitting clips at random would let a model see every generator and every speaker in training and then be "tested" on more of the same. The fold file prevents that by moving whole groups. Two generators and about a fifth of the real-speech groups form a holdout that nothing is ever fitted or selected on and each model reads once; the rest forms five inner folds that each hold out whole generators. Extra training corpora are added to the inner folds only, so holdout numbers stay comparable across every model.
What goes in and comes out
In: the training manifest (path, label, generator, speaker, source). Out: splits/nsa_folds.csv with a group and a fold per row (0–4 or holdout); splits/nsa_folds_plus_asv19.csv with the ASVspoof 2019 rows added to the inner folds.
Where it lives
scripts/make_folds.py builds the base file; scripts/extend_folds.py adds extra rows; the M5 data module pins both files' hashes.
Parameters and where each is set
- Grouping: fakes by generator; LJ Speech by chapter (it is one speaker); other real speech by speaker (make_folds.py, lines 28–33).
- Holdout generators
wavegrad2andplayht; a 0.2source: scripts/make_folds.py line 41 (GitHub; assumes main is public) share of real-speech groups; five inner folds by stratified group k-fold; at least 30 clips per class in every fold; seed 0 (make_folds.py, lines 39–44). Two assertions: every fold has both classes, and no group straddles folds. - Extra rows (M1b, M5) go to the inner folds only by the same grouped split, never to the holdout; path overlap with the base file is refused; the base file is never modified (extend_folds.py, lines 28–48).
Status
Built and frozen on Friday night; hash-pinned by the M5 tests since Saturday.
Numbers
- 20,000source: README.md, section validation rows: 10 DiffSSDThe sponsor's synthetic training corpus: clips from ten text-to-speech generators, including commercial ones. generators, LJ SpeechA public read-speech corpus of a single speaker. Part of the real side of the training data; being one voice, it is a shortcut risk of its own., LibriSpeechA public multi-speaker read-speech corpus. The multi-speaker real side of the training data.. Inner folds 16,142source: README.md, section numbers rows; the holdout 3,858source: README.md, section validation rows (PlayHT, WaveGrad2 and 26 real-speech groups).
- Inner folds hold out, in turn: grad_tts + unit_speech; diffgan_tts + openvoicev2; pro_diff + xtts_v2; your_tts; ElevenLabs (README).
- M1b's extension adds 5,128source: docs/architecture.md, section 62-xls-r-backbone-with-the-m1-probe-and-the-m5-head ASVspoof 2019 real clips (40 speakers) and 2,520source: docs/architecture.md, section 62-xls-r-backbone-with-the-m1-probe-and-the-m5-head spoof anchors to the inner folds.
- The holdout's resolution floor is about ±0.07–0.10source: README.md, section validation; a gap under 0.15source: README.md, section validation is treated as noise.
Known limits
Known leakage that grouping cannot fix is printed, not hidden: several DiffSSD generators clone the same voices, and LJ is one voice on both sides of every fold. grad_tts is invisible to the handcrafted features under this scheme because no other generator shows its phase cue.
Tests that pin it
tests/test_m5_manifest.py: the fold files' hashes, the extended file agrees with the shared one on all 20,000source: README.md, section validation core rows and never puts an extra row in the holdout, no holdout row in any training set. tests/test_cloud_scripts.py: the M5 code never writes the fold files.