Systems and subsystems

The XLS-R probe (M1, M1b)

A frozen self-supervised speech model, one of its middle layers averaged over time, and a small logistic classifier on top. It is the primary detector, with the largest weight in the fusion, and the one every other rung was measured against.

In plain words

XLS-RA large self-supervised speech model from Meta, pretrained on many languages and never trained for spoof detection. Its middle layers carry the acoustic detail that separates vocoders from voices; we read one of those layers. was trained to predict masked speech in many languages, never to spot fakes; but its middle layers encode fine acoustic detail, and a vocoder leaves traces there that a voice does not. We read one layer, average it over the clip, and fit a logistic regression that says how far the clip sits toward synthetic. "Probe" because the big model is never changed: only the 1,024source: docs/architecture.md, section 61-depth-at-a-glance weights of the classifier are learned. M1b is the same probe with 40 more real speakers in the training folds.

What goes in and comes out

In: the prepared segment (band-matched, trimmed, normalized). Out: a calibrated LLRLog-likelihood ratio: the log of how much more likely the evidence is under "synthetic" than under "real". Positive means synthetic-like, negative real-like, zero is even. The XLS-R probe reports one per file., positive for synthetic-like, mapped through a Platt mapA one-dimensional logistic fit that turns a raw decision value into a probability. It is monotone, so it changes the numbers a reader sees without changing the ranking the metric scores.; the evidence sentence reads "XLS-R layer-7 probe: calibrated log-likelihood ratio +x.xx (synthetic-like, P=0.xx)". Its export column is m1b_v3.

Where it lives

Embedding in embed.py (embed_segment: one forward pass over the whole segment, the time-mean of every hidden state kept, so the layer choice happens at probe time); the classifier in probe.py; extraction, training and export in extract_embeddings.py, train_probe.py and export_probe_scores.py; the runner's path in pipeline.py, lines 466–475.

Parameters and where each is set

• Layer 7, chosen by inner cross-validation: 0.257source: README.md, section deep-detectors out of fold, with layers 6 and 8 close and layers 20 and above at 0.40–0.43source: README.md, section deep-detectors (README). The runner truncates the backbone after that layer so the hidden state stays bit-identical and about two thirds of the compute is skipped (pipeline.py, lines 380–389).
• The classifier: a standard scaler then a class-balanced logistic regression with C = 1 and at most 3,000source: src/hearsay/probe.py line 45 (GitHub; assumes main is public) iterations, fit fold-locally (probe.py, lines 43–47); the layer with the lowest out-of-fold minDCF wins, ties by EER; a Platt map is fit on the winning layer's out-of-fold scores, then the probe is refit on all inner rows (train_probe.py, lines 70–79).
• Embedding stored in float16 for the runner; the parity cost is a maximum logit difference of 5.3e-4source: src/hearsay/pipeline.py line 101 (GitHub; assumes main is public) on 50 files (pipeline.py, lines 100–101).
• M1b's extra rows: 5,128source: docs/architecture.md, section 62-xls-r-backbone-with-the-m1-probe-and-the-m5-head ASVspoof 2019 real clips from 40 VCTK speakers and 2,520source: docs/architecture.md, section 62-xls-r-backbone-with-the-m1-probe-and-the-m5-head spoof anchors, inner folds only (extend_folds.py).
• Segment mode: the earlier fixed 4-second windows tiled short test clips and put a seam only in test data; the shipped probe embeds each clip at its own length (embed.py, lines 63–68).

Status

Shipped as the 0.6source: README.md, section how-it-decides weight of the fusion. Three versions were trained: v2 before the band match, v3 band-matched, M1b v3 with the extra speakers.

Numbers

M1b v3: inner 0.301source: README.md, section numbers, holdout 0.072source: README.md, section numbers (EER 1.4%source: README.md, section numbers), In-the-Wild 0.342source: README.md, section numbers with false alarms 0.8%source: README.md, section numbers, test share above 0.5source: README.md, section numbers 26.8%source: README.md, section numbers. Band matching then the extra speakers took the holdout from 0.159source: README.md, section what-worked to 0.072source: README.md, section what-worked; before band matching the same speakers made In-the-Wild worse (0.374source: README.md, section what-worked → 0.405source: README.md, section what-worked). Under 20 dB white noise its holdout cost rises from 0.056source: README.md, section what-did-not-work to 0.271source: README.md, section what-did-not-work.

Known limits

Like every probe we trained it sits at 0.34–0.41source: README.md, section what-did-not-work on In-the-Wild against 0.07–0.16source: README.md, section what-did-not-work on the holdout: channel robustness is the gap. A runner comment about the probe's segment flag is stale; the runner uses segment mode regardless.

Tests that pin it

tests/test_pipeline.py: truncation keeps the probe layer identical on a tiny model (and, weights-gated, on the real one), M1-only mode is the probe arithmetic. tests/test_audio_and_submission.py: the segment path never tiles and crops deterministically, normalization removes level, the band match, the bulk extraction path matches the clip path (weights-gated).

Sources

README, "Deep detectors"; the M1b report; the public-probe shakedown; Architecture §6.2.