Systems and subsystems

Speaker drift

Speaker embeddings on one-second windows, compared pairwise, to ask whether the voice stays the same within a clip. It works as a detector of edits and second voices, and it points the wrong way as a detector of fakes: cloned voices are the most self-consistent voices in the data.

In plain words

The rubric's speaker-embedding consistency technique. A pretrained speaker-recognition network (ECAPAA pretrained speaker-recognition network that turns a second of speech into a vector describing the voice. Comparing vectors from successive windows shows whether the voice stays the same within a clip.) turns each second of speech into a vector describing the voice; if two windows of the same clip disagree, the voice changed. The surprise in the data: real people vary from second to second and clones do not, so the mean similarity separates classes toward fake. Fused, it would push the hard real recordings toward synthetic, which is the expensive error, so it stays evidence.

What goes in and comes out

In: the clip, trimmed and capped. Out: the number of windows, the minimum, mean and standard deviation of the pairwise cosine similarities, the adjacent-window minimum, a drift flag, and a sentence: voice drifts across the clip: minimum window-to-window speaker similarity −0.10 (mean 0.37) over 14 windows; consistent with a second voice or an unstable cloned identity, or one consistent voice: ….

Where it lives

detectors/speaker_drift.py; the weights are the SpeechBrain ECAPA model loaded from a local directory with the hub path overridden, so it runs offline.

Parameters and where each is set

• Windows of 1.0source: src/hearsay/detectors/speaker_drift.py line 34 (GitHub; assumes main is public) s every 0.5source: src/hearsay/detectors/speaker_drift.py line 34 (GitHub; assumes main is public) s over at most 8.0source: src/hearsay/detectors/speaker_drift.py line 34 (GitHub; assumes main is public) s, the last window flush with the end; unit-norm embeddings; fewer than two windows means no drift (speaker_drift.py, lines 75–101).
• Drift when the minimum pairwise cosine is below 0.05source: src/hearsay/detectors/speaker_drift.py line 35 (GitHub; assumes main is public); scores drift 0.6source: src/hearsay/detectors/speaker_drift.py line 36 (GitHub; assumes main is public), none 0.5source: src/hearsay/detectors/speaker_drift.py line 36 (GitHub; assumes main is public).
• Calibrated on synthetic two-speaker splices: 94%source: docs/reports/2026-09-26_gate-and-drift.md of splices flagged against 6.2%source: docs/reports/2026-09-26_gate-and-drift.md of single voices.

Status

Built about 06:10 on Saturday; evidence only, by the finding below.

Numbers

Mean window-to-window similarity separates classes toward fake at AUC 0.73source: README.md, section what-did-not-work. Drift flags 16%source: README.md, section what-did-not-work of test files, mostly the hard real recordings; in the ablation 180source: README.md, section orchestration-what-routing-changes holdout rows and 269source: README.md, section orchestration-what-routing-changes test files, without moving a score.

Known limits

After trimming, a mostly silent clip yields too few windows to say anything (worked example 7). The module's docstring quotes older flag rates than the report.

Tests that pin it

tests/test_speaker_drift.py: windows cover the clip flush with the end, one voice is consistent and two voices drift (with a stub encoder), a single window is neutral, registration, offline loading with an empty cache and a real two-speaker splice (both weights-gated).

Sources

The gate and drift report; README, "Speaker drift points the wrong way"; worked example 4.