Systems and subsystems

Handcrafted spectral and prosody features

Two hundred and thirty-four engineered features of the spectrum and the prosody of a clip, fed to gradient-boosted trees that are evaluated in plain numpy at inference. Interpretable by construction: every sentence it writes names the three features that moved it and by how many standard deviations from real speech.

In plain words

The handcrafted detectorEngineered spectral and prosody features (cepstral statistics, spectral shape, phase cues, pitch, loudness, pauses) fed to gradient-boosted trees, evaluated at inference in plain numpy. is the one a person can argue with. It measures things with names, such as how much the cepstral coefficients change from frame to frame, how the pitch moves, whether the phase behaves like a vocoder's, and learns which of them separate real from synthetic. It is the spectralFeatures of the frequency content of the audio over time: energy per band, roll-off, flatness, cepstral coefficients (MFCC, LFCC, CQCC) and how they change from frame to frame. and the prosodyThe rhythm and melody of speech: pitch movement, loudness changes, pauses, breaths. Synthetic speech tends to be more regular than a person. techniques of the rubric in one detector. It is excellent on generators it has seen and blind to web-sourced fakes, which is why it has a fifth of the vote and not half.

What goes in and comes out

In: the band-matched, trimmed, test-length-cropped, RMS-normalized clip. Out: a probability from the trees (the exported logit is its logit), the feature values, and a sentence of the form spectral/prosody features real-like (P=0.16): LFCC 16 frame-to-frame change 3.0 SD below real speech (toward synthetic); …, the three largest contributions in standard deviations from the real-speech mean of the inner rows.

Where it lives

The 75 base features and the augmentation in handcrafted.py; the six added families in hc_v4.py; extraction and training in extract_handcrafted.py and train_handcrafted.py; the detector wrapper in detectors/handcrafted.py over the shared learned base; the numpy tree evaluator in trees.py.

Parameters and where each is set

• The 75 base features: 64 spectral (high-band energy ratios at 4, 6 and 7 kHz, roll-offs, centroid, bandwidth, flatness, six-band spectral contrast, flux, 20 MFCC means and standard deviations) and 11 prosody (YIN pitch median, spread and slope, voiced fraction, energy dynamics, pauses, zero-crossing rate), on 32 ms frames every 10 ms (handcrafted.py, lines 72–140).
• The six added families (hc_v4.py): LFCC (40 linear filters to 7 kHz, 20 coefficients, mean, standard deviation and delta), phase (group-delay spread per voiced frame, peak phase-versus-magnitude coherence), CQCC (a constant-Q transform from 65 Hz, 24 coefficients), the modulation spectrum of the loudness envelope in four bands, breath frames between speech, and cycle-level jitter and shimmer; 234source: README.md, section the-eight-forensic-techniques columns in all.
• The per-column gate that admitted each family: a column is kept if it separates classes on grad_tts, pro_diff or ElevenLabs by a margin, does not mainly separate the two real corpora from each other, and survives a codec round-trip; the rule and the tables are in the report (the v4 report).
• Training: LightGBM (300source: scripts/train_handcrafted.py line 70 (GitHub; assumes main is public) trees, 31 leaves, class-balanced) chosen over a logistic regression by inner out-of-fold minDCF, then refit on all inner rows; the bundle stores the dumped trees and the real-speech feature statistics, and the trainer asserts the numpy evaluator matches LightGBM (train_handcrafted.py, lines 64–74 and lines 255–261).
• v5 augmentation, fit-only: codec round-trips and a random spectral tilt with a low-pass between 4,500source: src/hearsay/handcrafted.py line 153 (GitHub; assumes main is public) and 7,000source: src/hearsay/handcrafted.py line 153 (GitHub; assumes main is public) Hz, drawn per row; the augmented twins take their clean twin's fold, are excluded from their own validation fold and never enter a readout or the export, so every number is on clean rows (handcrafted.py, lines 152–197; train_handcrafted.py, lines 120–123).
• Why numpy trees: LightGBM and PyTorch each ship an OpenMP runtime that crashes together on macOS, so nothing on the inference path imports LightGBM; the evaluator also gives per-feature path contributions for the sentence (trees.py, lines 4–9).

Status

Shipped as v5b at weight 0.2source: README.md, section how-it-decides: the v4 columns with augmented twins, trained with augmentation and validated on clean rows only.

Numbers

v3 (75 features): inner 0.614source: README.md, section numbers, holdout 0.254source: README.md, section numbers. v4a (234source: README.md, section the-eight-forensic-techniques features): inner 0.434source: README.md, section numbers, holdout 0.170source: README.md, section numbers; pro_diff went from 1.00source: README.md, section what-worked to 0.32source: README.md, section what-worked out of fold. v5b: inner 0.469source: README.md, section numbers, holdout 0.137source: README.md, section numbers (EER 3.1%source: README.md, section numbers), In-the-Wild false alarms 0.40%source: README.md, section numbers, test share above 0.5source: README.md, section numbers 45.1%source: README.md, section numbers; holdout-real false alarms fell from 9.1%source: README.md, section what-worked to 6.3%source: README.md, section what-worked. Trees beat the linear model on the same features, holdout 0.374source: README.md, section what-worked against 0.661source: README.md, section what-worked.

Known limits

In-the-Wild minDCF 1.00source: README.md, section what-did-not-work: it does not transfer to web-sourced fakes. Even band-matched, the columns tell training from test audio at AUCArea under the ROC curve: the probability that a random synthetic file scores above a random real one. 1.0 is perfect separation, 0.5 is chance. 0.99source: README.md, section what-did-not-work, and pruning the shifted columns made it worse. grad_tts is invisible to it when held out. At 20 dB white noise it loses almost all discrimination (AUC 0.998source: README.md, section what-did-not-work → 0.64source: README.md, section what-did-not-work), the shipped rule's weak spot.

Tests that pin it

tests/test_handcrafted_detector.py (contract, direction, the sentence names features and direction, memoization, a missing model is an error, the bundle's families are recomputed at inference), test_hc_v4.py (key sets, offset invariance, phase coherence, LFCC tilt, modulation sums, jitter), test_hc_augment.py (tilt darkens or brightens, reproducible draws, augmentation off by default), test_trees.py (matches LightGBM, contributions sum to the raw score, agrees with SHAP on top features, NaN handling, categorical splits rejected, no LightGBM import).

Sources

The CPU detectors report; the v4 and v5 report; the feature-engineering map; Architecture §7.