Systems and subsystems

The runner and its routing rules

One script takes a directory of audio and produces the TSV, one explanation JSON per file, timings and a resumable cache. Inside it, a short list of fixed rules over measured facts decides what each detector's result may do. That list is the orchestrator.

In plain words

Orchestration here is not a learned router and not a language model. Every detector runs on every file (the whole test set is about 95 minutes of audio, so compute was never the constraint). What the rules decide is what each result is allowed to do to the score, and every decision is written to the file's routing log so it can be checked afterwards. The rules are on the How it decides page as a diagram; this page is the reference.

What goes in and comes out

In: a directory of audio files and, optionally, the sponsor's template TSV that fixes the row order. Out, under the output directory: the TSV (never overwritten; a clash gets a timestamp suffix), a results/ folder with one JSON per file, a resumable results.jsonl, timings.csv, a sidecar CSV of the per-detector values, run_meta.json, and the preflight outputs.

Where it lives

src/hearsay/pipeline.py: the constants, the Models loader, analyze_clip, routing_log, fusion_line, final_score and refuse. scripts/run_pipeline.py: the command line, the template reader, the cache, the preflight, the TSV writer and the exit codes.

The rules and where each is set

• Container → compression. The container detector's facts set how compression forensics is reported: a lossy header gets "compression forensics apply (evidence)"; a PCM file written by FFmpeg (every test file) gets "run for evidence, not score" (pipeline.py, lines 605–615). Either way the compression score never enters fusion.
• Bandwidth. The compression detector's effective bandwidth is reported against the band match: at or below 7,250source: src/hearsay/pipeline.py line 618 (GitHub; assumes main is public) Hz, or above it (pipeline.py, lines 616–619). This is the one bandwidth threshold in the router; the 7,500source: README.md, section orchestration-what-routing-changes Hz figure in the reports is an inventory reading, not a rule.
• Speech gate → abstention. A file the gate rejects skips fusion and goes to the pinned block; a gate error does not gate the file (pipeline.py, lines 636–645). See the gate and the default-answer policy.
• Spectra → suppression only. If Spectra's margin is below the constants file's threshold and the fused rank is above its threshold, the rank is halved; if Spectra is missing, nothing is suppressed (pipeline.py, lines 307–316). The thresholds themselves are in the fusion rule.
• Evidence-only detectors (ENF, splice, speaker drift, compression) are written to the log and the evidence list and never touch the score (pipeline.py, lines 620–635).
• Imputation. A fused detector that errors on a file has its column imputed at the inner-fold centre (rank 0.5source: src/hearsay/pipeline.py line 299 (GitHub; assumes main is public)), and the fusion line names it (pipeline.py, lines 299–300). A deep scorer that raises or returns a non-finite value becomes an error entry; the row is kept.
• Decode failure. No detector runs; the file is flagged and lands below the block (pipeline.py, lines 603–604).

Command line

--in, --out, --template, --team (the default is not our team name; the deliverable passes CrossExam), --fusion (default: the shipped constants file since the 12:35 switch), --rule, --policy, --flip (writes the pre-flipped twin through the same map), --detectors (cannot narrow the set the constants file needs), --m5, --threads (capped at 6source: src/hearsay/pipeline.py line 102 (GitHub; assumes main is public)), --require-offline, --no-truncate, --fresh, --compare-tsv, --preflight. Environment variables mirror the main flags (run_pipeline.py, lines 98–144).

Preflight and cache

Before scoring, four seconds of silence and a three-note chord are scored twice through the whole pipeline: both probabilities must be finite and agree, every scorer must be ok, neither clip may pass the gate, and both must land below the block (run_pipeline.py, lines 318–354). The cache header records the models, the constants file's hash and the scorer set; a mismatch moves the old cache aside; rule, policy and polarity are not part of the identity, so cached logits are re-fused in a fraction of a second (run_pipeline.py, lines 247–303).

Status

Built; the default constants file is the shipped rule since the 12:35 switch on Saturday. Live from audio on all 1,671source: README.md, section score-a-directory-on-the-mac files it reproduces the submitted TSV with Spearman 1.000000source: README.md, section score-a-directory-on-the-mac and a maximum difference of 9.3e-4source: README.md, section score-a-directory-on-the-mac.

Numbers

Speed with M5 loaded: 1.34 ssource: README.md, section score-a-directory-on-the-mac per file; peak memory 4.0 GBsource: README.md, section score-a-directory-on-the-mac. The router-on versus router-off ablation is on How it decides.

Known limits

The verdict field compares the pre-map fused probability with 0.5source: src/hearsay/pipeline.py line 652 (GitHub; assumes main is public) and ignores --flip; it is a label for the report, not the ranking the metric scores. A comment in the runner and the Docker build script still name the previous default; the code's default is the shipped file.

Tests that pin it

tests/test_pipeline.py: response shape, the map applied exactly once, the routing log reads the detector features (order and wording), lossy/seams/hum/drift wording, the default answer applied when there is no speech, a gate error does not gate, a failed fused detector is imputed and logged, a decode failure sits below the block, re-fusing switches rule, policy and polarity without models, scorers come from the constants file, the cache identity gates the resume and tolerates a torn line, the template is read in order and missing files are flagged, exit code 4 when a weighted scorer fails, the preflight refuses a failing scorer, --require-offline refuses without the environment.

Sources

README, "Orchestration"; the runner report; the worked examples; the M5-scorer run spec.