The library

Rendered from docs/scoping.md; the markdown file is the record.

Historical (pre-rescope, Fri Sep 25): the working plan is docs/plan.md; the spec is docs/nsa-challenge.md.

September 25, 2026

You are helping a four-person hackathon team scope the technical side of an audio deepfake detection project. I want a rigorous, opinionated technical scoping document, not a survey. Push back where my assumptions are wrong and tell me plainly when something is not feasible in the time we have. ## Situation Event: HackGT 13 at Georgia Tech, Sep 25 to 27, 2026. Coding starts 8 pm Friday; everything is frozen Sunday morning. That is 36 hours as a hard maximum for software. Plan for something like 24 to 30 hours of real work so we have breathing room for debugging, rehearsal and sleep; any plan that only works if all 36 hours go perfectly is a bad plan. The hardware side has closer to 48 hours because physical assembly and wiring can start before the coding window. Pretrained weights and public datasets can be downloaded before coding starts. The project is called HEARSAY, after an NSA sponsor challenge of the same name. The NSA challenge: given any audio clip (WAV, MP3, M4A, etc.), output a 0 to 100% likelihood that the voice is AI-generated, optionally naming the manipulation type (TTS, voice cloning, splicing). They provide a labeled training set and a held-out test set at the event. Submission is a CSV of predictions on the test set, scored automatically. Cloud services, open-source models, speech models and DSP toolkits are all allowed; they say they care about how techniques are combined. Unknown until the event: dataset size, which generators made the fakes, the scoring metric, whether manipulation types are labeled, and whether outside training data is allowed (we are assuming yes and asking). ## What we are building One detection model powers two things:

1. The NSA CSV submission (offline batch scoring of the test set). This path has no latency constraint beyond the submission deadline, so it can use far more compute per clip than the live path.
2. A live physical demo: a tabletop "listening head" on pan-tilt servos with a 4-mic array, a webcam and an RGB LED (possibly a class 2 laser). Two phones on the table take turns playing the same voice, one real and one a clone. The head turns toward whichever phone is talking (TDOA direction finding with GCC-PHAT, camera snap-to-source), stays green on the real voice, and turns red on the clone. Everything on the live path runs locally on a laptop, no hosted API. Second product framing we want to keep alive, aimed at the voice-clone scam call ("Grandma, I'm in jail, send money"): a small device or app that sits next to an elderly person's phone and gives a haptic buzz, a red LED, or an audible alarm when it thinks the caller's voice is synthetic. Same model, different wrapper. I want the scoping to cover what would have to be true of the model for that version to work (latency, window length, false positive tolerance, running on a phone or a microcontroller-class device vs. a laptop). ## Team and compute
   • Me: owns the detector, validation split, CSV, and the streaming scorer. My depth is ML/CV architecture, not tooling; I work through an AI coding agent for implementation.
   • One EE: mic array capture, direction finding, VAD, servo power. Has about 48 hours.
   • One CS/hardware: pan-tilt firmware, serial protocol, aiming. Has about 48 hours.
   • One frontend/full-stack: dashboard (per-second timeline shaded by synthetic score, per-detector "why" panel, upload mode, replay mode), demo video, and floater. Compute I have for free: an Apple M3 Pro laptop with 18 GB unified memory (Metal integrated GPU). I am willing to rent on vast.ai; an H100 or H200 is a few dollars an hour there. I do not want to spend more than roughly $50 to $100 on GPU time for the weekend unless the gain is clearly worth it. ## Current software plan (critique this) Model ladder, each rung measured on a held-out split, never without a valid CSV after the first rung:
   • M0: loader (any format to 16 kHz mono, chunk long clips), validation split held out by generator or speaker.
   • M1: frozen WavLM or XLS-R 300M embeddings, pooled, into logistic regression. First CSV.
   • M2: hand-crafted features (LFCC/MFCC stats, spectral flatness, high-frequency energy cutoff, pause and breathing patterns).
   • M3: off-the-shelf detector scores (e.g. an AASIST3 checkpoint) as extra features.
   • M4: stacked gradient-boosted ensemble with calibrated 0 to 100% output (Platt or isotonic).
   • M5: fine-tuned SSL front end with an AASIST-style back end (needs GPU).
   • M6: manipulation-type head, if labels exist. Live scoring: 2 to 4 second windows while VAD is active, exponential moving average per source, alert only after ~3 windows above threshold. Fastest good-enough model on the live path. Replay problem: everything the head hears passes through a phone speaker and the room air, which is known to hurt detectors badly (one study: a top model's error rate went from 4.7% to 18.2% under replay). Planned fix: a one-hour Friday-night session playing a few hundred labeled clips through the actual phones at the actual mic array and adding them to training, plus software codec augmentation (AMR, Opus). Keep a clean-only validation score because the NSA test set is probably clean. ## Questions I need answered
3. Software levels. Rewrite the model ladder if you would change it. For each rung give: expected gain, wall-clock cost on my M3 Pro vs. a rented H100, and the point at which the rung is not worth it in a 24 to 30 hour window. Tell me which rung the live path should use, and which rungs to cut first if we fall behind.
4. Extra training data. List the public datasets worth pulling before Friday night (ASVspoof 2019/2021/5, MLAAD, In-the-Wild, WaveFake, Codecfake, ADD, anything newer), with size, license, download time, which generators they cover, and which ones actually help generalization to unseen generators vs. just add volume. Flag any that are likely to leak into or overlap with an NSA-style test set. Assume outside data is allowed but give a plan for if it is not.
5. Base model. Which pretrained SSL front end should we start from (WavLM base/large, XLS-R 300M/1B, wav2vec2, HuBERT, Whisper encoder, something else), and why for this task specifically? Compare frozen-embedding use vs. partial vs. full fine-tuning. If you are not confident picking one, do not guess: instead design a bake-off we can run in the first two or three hours after the dataset drops, testing several front ends under identical conditions (same split, same pooling, same classifier), with a specific pass/fail rule for choosing the winner and an estimate of how long it takes on the M3 Pro. The first valid CSV comes before any bake-off. Separately: is there any world in which self-supervised pretraining of our own narrow model (e.g. on the NSA training set plus public data) beats starting from a public checkpoint in this time window? If yes, what is the cheapest GPU configuration on vast.ai that gets it done, with an hour and dollar estimate. If no, say so and move on.
6. Hand-crafted features. I am not historically good at designing signal features by hand, though I can probably figure it out with guidance. Is M2 worth its time at all once a strong SSL front end is in the ensemble, or does it mostly add noise? If it is worth it, name the three to five features with the best value per hour, say exactly which library computes each one (librosa, torchaudio, something else) so I am calling functions rather than designing filters, and give a 30-minute test that tells us whether they are helping before we commit further.
7. Test-time compute for the offline submission. The CSV path has hours of slack and no latency constraint. What is the best way to spend extra compute per test clip to raise the score? Cover: test-time augmentation (multiple crops, speed and pitch perturbation, codec round-trips, averaging logits), scoring every window of a long clip instead of one, ensembling several front ends or checkpoints, a larger model that is too slow for the live path but fine offline, and transductive or semi-supervised tricks on the unlabeled test set (pseudo-labeling, domain adaptation, test-set statistics) with an honest note on whether those are within the spirit of the challenge. For each, give expected gain, cost on the M3 Pro vs. H100, and the point where more compute stops helping.
8. Multilingual. Can this be made to work on non-English speech without a separate model? Which of the above front ends and datasets are language-agnostic enough, and what would we lose? What is the cheapest way to demo it on two or three languages?
9. Phone-quality audio. Can we fine-tune so the model is robust to fuzzier, through-the-air iPhone speaker output, or do we need a wired connection for the live demo to be reliable? Be specific about what augmentations and how much replay data would get us to a usable live accuracy, and what the honest ceiling is for this time window. Same question for the grandma device, where the audio comes off a phone call (codec, bandlimited, possibly speakerphone).
10. Reinforcement learning. Is any form of RL applicable to this problem at all (reward shaping on the scoring metric, RL-based augmentation policy search, anything else)? My main interest is batch efficiency, since I understand RL training loops to have much higher batch throughput. If RL does not apply, or my batch-efficiency premise is wrong, say so directly rather than finding a use for it.
11. Hardware. Two parts. First, the 4-mic array: is it worth it over 2 mics for pan-only tracking of two phones a couple of feet apart on a table, or is it complexity for no reason? Does it buy the detector anything, for example beamforming or noise suppression that cleans the audio before scoring in a loud hall? Second, the two hardware people want more to build than the head alone. Propose additional hardware features, ranked by how much they improve the live demo or the grandma product, with parts, hours, and the risk each adds. Do not pad with decorations; anything proposed should change what a judge sees or make the product story more credible. Ideas we have already considered: a servo-driven analog gauge from REAL to SYNTHETIC, an enclosure, and a laser lock on the fake phone.
12. Anything I have not asked that will bite us: calibration on a different scoring metric than we expect, class imbalance, long clips vs. short windows, silence and music handling, format conversion pitfalls, and what to do with the first two hours after the dataset drops. ## Output format Give me a scoping document with a section per question, a final "recommended plan" with a Friday/Saturday/Sunday timeline for the model owner only that fits in 24 to 30 hours, and a "things I would cut" section. Give numbers and named checkpoints, not categories. Where you are unsure, say so and give a way to test it in under 30 minutes.

Executive summary

Scope

Technical scoping for HEARSAY, a hackathon audio-deepfake-detection system (HackGT 13, Sep 25–27 2026, ~24–30 h real coding budget). Covers the M0–M6 model ladder critique, dataset/checkpoint selection, live vs. offline compute tradeoffs, replay/multilingual robustness, RL applicability, hardware demo design (mic array, gauge, enclosure, laser), risk scan, and a requirements-only feasibility discussion of an elder-scam-call detector. Assumes the NSA dataset, metric, and outside-data policy drop at the 8 pm Friday coding start.

Conclusion

The plan is feasible in 24–30 h after reordering: ship a constant-prior CSV within the first hour, a frozen $\text{XLS-R 300M}$ + logistic-regression CSV within ~3 h, then promote H100 fine-tuning of the same backbone ahead of hand-crafted features (each 3-epoch run ~1–1.5 h, a few dollars; total GPU spend well under the $50–100 cap). The live path runs the same XLS-R core on 3–4 s windows with a 2–3-window confirmation. Hand-crafted features get one 30-minute gate; the manipulation-type head, scratch SSL pretraining, RL (whose batch-efficiency premise is wrong), and the laser lock are cut outright. Replay degrades detectors severely; the honest live ceiling is ~80–90% balanced accuracy on the matched two-phone rig with a disclosed wired fallback, and open-set through-air accuracy near clean levels is not achievable this weekend.

ChatGPT and Claude independently converged on $\text{XLS-R 300M}$ as the core, first-CSV-first sequencing, demoting hand-crafted features, rejecting RL and scratch pretraining, and cutting the laser — grounded in verified benchmark and replay evidence. Gemini diverged on the base model (WavLM Base) resting on a single back-end study, and its 'frozen SSL is ineffective' claim was graded wrong against its own source; the divergence is explained and resolved via a bake-off. Confidence stays Medium because all M3 Pro wall-clock figures are analytic estimates, GPU price totals were disputed in grading, and the NSA dataset/metric/policy are unknown until the drop.

Sensitivities

• If the M3 Pro microbenchmark shows XLS-R 300M missing the 750 ms live latency gate, the live path drops to WavLM-base or off-the-shelf W2V2-AASIST while the offline path keeps XLS-R.

• If the sponsor disallows outside data or counts pretrained anti-spoof checkpoints as outside data, M5 falls back to NSA-only fine-tuning with heavy augmentation and the dataset staging is unused.

• If through-air balanced accuracy on untouched replay clips falls under 80%, the demo switches to a disclosed wired feed with mic-based localization retained.

• The bake-off can overturn the XLS-R pick: NSA data differing sharply from ASVspoof-lineage benchmarks flips the front-end ranking, which is why the ≥1.0 $\text{EER}$-point pass rule exists.

Final Verified Code

Claude's script builds the analytic FLOP model (xlsr\_fwd\_flops giving $F_c \approx 133$ GFLOPs per 4 s window), derives M3 Pro vs. H100 extraction and fine-tune wall-clock, tabulates dataset download times across bandwidth tiers, and writes datasets\_staging.csv and gpu\_ledger.csv — the most decision-useful artifact because it makes every timing claim reproducible and parameterized.

Show code

import pandas as pd, numpy as np

# ---------- 1. XLS-R 300M forward FLOPs per clip (wav2vec2-large arch: 24 layers, d=1024, ffn=4096, 50 Hz frames)
def xlsr_fwd_flops(sec=4.0, d=1024, layers=24, ffn=4096, frame_hz=50, conv_gflops_per_s=2.0):
    Tf = int(sec*frame_hz)
    attn_proj = 2*(4*d*d*Tf)      # Q,K,V,O projections; 2 FLOPs per MAC
    attn_sc   = 2*(2*Tf*Tf*d)     # QK^T + AV
    ffn_f     = 2*(2*d*ffn*Tf)    # two linear layers
    return layers*(attn_proj+attn_sc+ffn_f) + conv_gflops_per_s*1e9*sec

Fc = xlsr_fwd_flops()
print(f"F_c (XLS-R 300M forward, 4 s clip): {Fc/1e9:.0f} GFLOPs (analytic estimate)")

# ---------- 2. inference throughput: M3 Pro (MPS sustained est. 1.0-2.5 TFLOPS) vs H100 (sustained 150-350 TFLOPS bf16)
devs = [("M3 Pro low", 1.0e12), ("M3 Pro high", 2.5e12), ("H100 low", 1.5e14), ("H100 high", 3.5e14)]
rows=[]
for n,R in devs:
    t = Fc/R
    rows.append([n, f"{t*1e3:.0f} ms", f"{4.0/t:.0f}x RT", f"{50_000*t/3600:.2f} h", f"{25_000*t/3600:.2f} h"])
print(pd.DataFrame(rows, columns=["device (sustained est.)","per 4s clip (floor)","realtime","extract 50k clips","score 25k test"]).to_string(index=False))
print("Wall-clock on MPS typically 1.5-2.5x this floor (fp32, host overhead); H100 becomes I/O-bound.")

# ---------- 3. fine-tune cost (SSL front end + AASIST head, 4 s crops, fwd+bwd ~ 3x fwd)
N_ft = 120_000; epochs = 3
F_total = N_ft*Fc*3*epochs
print(f"\nFine-tune total compute: {F_total/1e15:.0f} PFLOPs ({N_ft} clips x {epochs} epochs)")
for n,R,ovh in [("H100 compute floor",2.5e14,1.0),("H100 realistic (dataloader-bound, 4x)",2.5e14,4.0),("M3 Pro realistic (2x floor)",1.5e12,2.0)]:
    hrs=F_total/R*ovh/3600
    print(f"  {n}: {hrs:.1f} h per {epochs}-epoch run")
for rate in [1.50,2.27,2.94]:
    print(f"  cost per 1.5 h run @ ${rate}/h = ${rate*1.5:.2f}")

# ---------- 4. pre-event download staging (sizes marked est. where unverified)
ds = [
 ("ASVspoof 5 full (Zenodo)",142.3,"ODC-BY 1.0 (verified)","train+dev only if separable; codec-rich"),
 ("MLAAD v10 (size UNVERIFIED, est.)",90.0,"per deepfake-total.com/mlaad","175 TTS gens, 54 langs; core generalization set"),
 ("M-AILABS bona fide (size est.)",70.0,"public; verify","bona fide counterpart for MLAAD"),
 ("ASVspoof 2019 LA (size est.)",7.0,"ODC-BY (verify)","canonical train set; recipes assume it"),
 ("In-the-Wild (Kaggle zip)",8.16,"Apache 2.0 (Kaggle listing)","HOLD OUT as generalization test only"),
]
rows=[]
for n,gb,lic,role in ds:
    r=[n,f"{gb:.1f} GB",lic]
    for mbps in [100,300,500]:
        r.append(f"{gb*8*1e9/(mbps*1e6)/3600:.1f} h")
    r.append(f"{gb*1e9/(15e6)/3600:.1f} h")
    rows.append(r)
print("\n"+pd.DataFrame(rows,columns=["dataset","size","license","@100Mbps","@300Mbps","@500Mbps","@15MB/s throttle"]).to_string(index=False))

# ---------- 5. GPU dollar ledger
led=[("Bake-off + M1 embeddings + first CSV (M3 Pro)",0.0),
     ("M5 run 1: XLS-R 300M + AASIST fine-tune (H100)",1.5),
     ("M5 run 2: augmentation variant (RawBoost+codec)",1.5),
     ("M5 run 3: +replay-capture data / second seed",1.5),
     ("Offline path: XLS-R-1B embeddings + TTA scoring",1.5),
     ("Contingency (retries, bigger NSA set)",2.0)]
tot=sum(h for _,h in led)
rows=[(n,h,f"${h*2.0:.0f}-{h*2.94:.0f}") for n,h in led]
print("\n"+pd.DataFrame(rows,columns=["item","H100 h","cost @$2.00-2.94/h"]).to_string(index=False))
print(f"TOTAL: {tot} H100-hours -> ${tot*2.0:.0f}-${tot*2.94:.0f} against $50-100 budget")

# ---------- write staging + ledger CSVs
stage = pd.DataFrame([
 ["MLAAD v10 + protocols","est. 60-120 GB (unverified)","see deepfake-total.com/mlaad","P0","train: unseen-generator generalization (175 TTS, 54 langs)","low","Thu"],
 ["M-AILABS","est. ~70 GB (unverified)","public/BSD-like (verify)","P0","bona fide counterpart to MLAAD","low","Thu"],
 ["ASVspoof 2019 LA","est. ~7 GB","ODC-BY (verify)","P0","canonical train; complementary to MLAAD","medium: VCTK-derived, widely reused","Thu"],
 ["In-the-Wild","8.16 GB","Apache 2.0 (Kaggle listing)","P0","HELD-OUT generalization test, never train","high: famous celebrities, likely overlaps public test sources","Thu"],
 ["ASVspoof 5 (train+dev)","142.3 GB full","ODC-BY 1.0","P1","codec-rich train/dev; MLS-derived","high: MLS/LibriVox overlaps SSL pretraining + possibly NSA sources","Thu night (throttle risk)"],
 ["WaveFake","est. ~30 GB (unverified)","research use (verify)","P2 skip if disk-tight","vocoder-only volume; old generators","low","Fri if idle"],
 ["ASVspoof 2021 DF keys+subset","est. tens of GB","ODC-BY (verify)","P2","extra eval stressor only","medium","skip by default"],
 ["Codecfake","size unverified","unverified","P3","codec-artifact training; overlaps ASVspoof5 codec aug","unknown","skip unless verified quickly"],
 ["ReplayDF","size unverified","research-gated (verify)","P3","replayed-deepfake eval for demo path","low","request access Thu; do not block on it"],
], columns=["dataset","size","license","priority","role","nsa_overlap_risk","when"])
stage.to_csv("datasets_staging.csv", index=False)
pd.DataFrame(rows, columns=["item","h100_hours","cost_range"]).to_csv("gpu_ledger.csv", index=False)
print("\nWrote datasets_staging.csv, gpu_ledger.csv")

Expert Summaries (3)

ChatGPT

Approach:

Built the plan around one core detector — $\text{XLS-R 300M}$ with mean pooling and partial fine-tuning of the upper four transformer blocks on a verified H100 — with measured stop gates at every rung, a structured 480-clip replay capture matrix, and a detailed hour-by-hour 26 h timeline.

Conclusion:

Stop after calibrated logistic fusion of a channel-robust partially fine-tuned XLS-R core; custom AASIST training, scratch SSL pretraining, RL, and manipulation typing do not fit the budget; live ceiling ~80–90% balanced accuracy on the matched rig.

Strengths:

• Most rigorous dataset table with licenses, verified sizes, and explicit contamination policy

• Concrete stop gates and schedule gates (e.g., freeze the frozen probe if fine-tuning has not won by Saturday 12:30)

• Structured replay capture design (2 phones × 3 positions × 2 levels × 2 classes × 20 clips = 480 captures) with source-disjoint splitting

• Verified Spectra-AASIST score-direction and MIT-license details

Weaknesses:

• All M3 Pro runtimes are self-declared planning envelopes, not measurements

• GPU rental cost figures were not verified against a live source in grading

• Several dataset byte sizes flagged as unverified

Unique:

The score-direction assertion test, separate clean/replay calibrators, the contamination/hash-dedupe policy, and the explicit two-window vs. three-window alert-latency tradeoff (6 s vs. 8 s of evidence).

Claude

Approach:

Anchored recommendations in a frozen-SSL benchmark and fine-tuned SOTA lineage, built an analytic FLOP model ($F_c \approx 133$ GFLOPs per 4 s window) to derive M3 Pro vs. H100 wall-clock, and produced a GPU dollar ledger and dataset staging CSVs.

Conclusion:

Frozen XLS-R 300M + logistic regression gives the first CSV ~3 h post-drop; background H100 full fine-tunes (~1.5 h, a few dollars each) are the biggest gain; the live path reuses the same model; M2 hand-crafted and the laser are cut; RL has no role.

Strengths:

• Quantitative FLOP-based compute model separating M3 Pro (fine-tune ~53 h, infeasible) from H100 (~0.6–1.5 h)

• Computed TDOA geometry showing 2 mics separate the two phones with large margin

• Explicit disagreement ledger (frozen vs. fine-tuned evidence, hand-crafted, beamforming, transduction) with resolution rules

• Grandma-device mockup puck as a high-story-value hardware add-on

Weaknesses:

• Many claims graded as missing citations; the $16–24 total GPU spend figure was graded wrong against its pricing sources

• M3 Pro numbers are analytic floors with assumed 1.5–2.5× overhead, never measured under MPS

• Recommends full fine-tuning where ChatGPT's partial recipe is the published, lower-risk default

Unique:

The FLOP/throughput cost model, the mic-array geometry computation ($\Delta\tau \approx 9.8$ samples), the push-to-challenge button idea, and the datasets_staging/gpu_ledger artifacts.

Gemini

Approach:

Recommended WavLM Base full fine-tuning with an off-the-shelf W2V2-AASIST baseline on the live path, MFCC-20 + LightGBM hand-crafted features, and language orthogonalization for cross-lingual robustness; strictly budgeted rungs with hard cuts.

Conclusion:

Cut RL, scratch pretraining, the laser, and the type head; replay caps live accuracy around 85–90%; downgrade to a 2-mic array; fuse M1+M2+M3 via linear logistic regression.

Strengths:

• Correctly and bluntly refuted the RL batch-efficiency premise

• Surfaced genuine cross-lingual degradation evidence (Bengali zero-shot 46.20% $\text{EER}$) and a cheap mitigation

• Simplest, most conservative hardware verdict (2-mic) with clear rationale

Weaknesses:

• Base-model pick (WavLM Base) rests on one back-end study; its claim that frozen SSL is largely ineffective was graded wrong against that source

• Live-path latency figure (~50 ms) and several cost figures are uncited

• Endorses MFCC-20 hand-crafted features more strongly than the benchmark evidence supports; the overfitting statistic was not verified

Unique:

Language orthogonalization as a target-free cross-lingual fix, the BanglaFake multilingual test option, and BOSARIS-style linear logistic fusion as the calibrated fusion recipe.

Assumptions and Limitations

Assumptions

• NSA training/test data, metric, label format, and outside-data policy are unknown until 8 pm Friday; the plan is built around a two-hour drop runbook, not an assumed dataset.

• Compute: Apple M3 Pro with 18 GB unified memory (Metal/MPS) locally; one datacenter-verified vast.ai H100 for training; total GPU budget cap ~$75 within the $50–100 envelope.

• Real-work budget of 24–30 h for the model owner (planned ~26 h: Fri 6 / Sat 15 / Sun 5).

• XLS-R 300M forward pass modeled analytically at $F_c \approx 133$ GFLOPs per 4 s window; M3 Pro sustained throughput assumed 1.0–2.5 TFLOPS with 1.5–2.5× wall-clock overhead — to be replaced by a measured 100-window microbenchmark.

• Fine-tune cost model assumes $N_{ft} \approx 120,000$ clips, 3 epochs, fwd+bwd ≈ 3× forward FLOPs.

• Live path: $T_w = 4$ s windows at 2 s hop, VAD-gated, 2-window confirmation, per-source EMA, replay-specific calibrator and $\theta$.

• Demo geometry: two phones ~±0.30 m lateral at ~0.80 m range; $d_{mic} \approx 0.10$ m; $c \approx 343$ m/s.

• Public checkpoints and datasets (XLS-R 300M, WavLM, Spectra-AASIST, ReplayDF, MLAAD, ASVspoof 2019/5, In-the-Wild) are staged before Friday; MLAAD gated access accepted Thursday.

• The NSA test set is presumed clean/codec-processed rather than physically replayed, so a clean-only validation score is maintained alongside replay metrics.

• Outside training data is assumed allowed pending the sponsor's answer; an NSA-data-only fallback with heavy augmentation is defined.

Limitations

• No M3 Pro figure in any expert analysis is a measurement; all are analytic FLOP-based or self-declared planning envelopes pending the mandated microbenchmark.

• GPU rental dollar figures were disputed or unverified in grading (Claude's $16–24 total graded wrong; ChatGPT's rate range unverified); only the order of magnitude relative to the $50–100 budget is robust.

• Several dataset artifacts could not be verified: MLAAD archive byte size, ASVspoof 2019 archive terms, Codecfake license/URL, AASIST3 released weights compatibility, and Gemini's MFCC overfitting statistic.

• Expected accuracy on the NSA test set cannot be forecast — published systems have degraded severely between progress and hidden sets, and cross-corpus transfer can approach chance.

• The frozen front-end rankings driving the XLS-R pick were measured with ASVspoof-2019-trained heads; NSA data could reorder them (mitigated by the bake-off, not eliminated).

• The grandma-device section is requirements-level only; no on-device benchmarking, codec-specific validation, or hardware sourcing was performed.

• Multilingual capability is demonstrated, not validated: no per-language accuracy guarantees, and cross-lingual degradation evidence (46.20% zero-shot Bengali EER) shows the risk is real.

Figures (2)

Claude (2 figures)

Nomenclature (17 symbols)

Narrative

Expert Comparison

All three experts agree on the load-bearing structure: constant-prior CSV in the first hour, frozen-embedding baseline before any fine-tuning, fine-tuning happens on a rented H100 and never on the M3 Pro, hand-crafted features are secondary at best, RL and scratch SSL pretraining are cut outright, the manipulation-type head is a distraction, the laser is cut, and replay through phone-speaker-and-air is the single biggest threat to the live demo. The disagreements are the base checkpoint, fine-tune depth, the mic count, and how much codec test-time augmentation is worth.

Expert Divergence

Base model. Gemini recommends WavLM Base and asserts that frozen SSL models are largely ineffective for spoof detection — a claim graded as wrong against its own cited WavLM back-end study [S8]. The broader controlled evidence points the other way: frozen XLS-R 300M is the best frozen front end across evaluation sets at 17.4% mean $\text{EER}$ versus WavLM Large 20.6% and HuBERT Large 22.7% [S13], and in a ten-test-set fine-tuning comparison XLS-R 300M averaged 4.26% $\text{EER}$ versus 7.26% for WavLM-large and 11.34% for Whisper-medium [S10]. ChatGPT and Claude independently converged on facebook/wav2vec2-xls-r-300m from different evidence lines. Resolution: XLS-R 300M is the pick, with a timed bake-off (WavLM Large and one more candidate) as the safety valve, because the frozen rankings were measured with ASVspoof-2019-trained heads and NSA data could flip them [S13].

Fine-tune depth. ChatGPT prescribes partial fine-tuning (freeze conv encoder and lower blocks, tune the upper four, backbone LR near $10^{-6}$); Claude says go full on the H100 and fall back to partial only on instability. Both are cheap on an H100. Resolution: start partial — it is the published lower-risk recipe with known hyperparameters — and escalate to full only if the partial run is stable and a locked holdout improves. This costs at most one extra background run.

Mic array. Three-way spread (keep 4 if synchronized / 2 sufficient / downgrade to 2). Claude's computed geometry shows two mics already separate the two phones with large margin (inter-source $\Delta\tau \approx 9.8$ samples at 48 kHz for phones at ±0.30 m, 0.80 m range). Resolution below under Question 9.

Cost figures. Claude's headline "$16–24 total" GPU spend was graded wrong against its pricing sources [S16][S42], and ChatGPT's H100 rate range was likewise unverified. All dollar figures here are planning estimates to be checked against live vast.ai listings Thursday; the only robust claim is that the total sits comfortably inside the $50–100 envelope even with a 2× rate error.

Technical Detail

Q1 — Model ladder, rewritten

The ladder's shape survives; the order does not. Reordered consensus ladder:

The key numbers: Claude's analytic model puts the XLS-R forward pass at $F_c \approx 133$ GFLOPs per 4 s window — a 53–133 ms compute floor per window on the M3 Pro and a ~53 h fine-tune per 3-epoch run on ~120k clips, versus ~0.6–1.5 h on an H100. These are analytic estimates, not measurements; both ChatGPT and Claude mandate a 100-window microbenchmark on the M3 Pro in the first two hours before trusting any of them. Published priors support the fine-tune promotion: WavLM-base fine-tuning improved a progress-set $\text{EER}$ from 15.82% to 7.26%, with the cited 114,000-example five-epoch run taking about 20 h on a V100 [S6] — hence the H100, not the laptop. Replace the gradient-boosted stacker with regularized logistic late fusion: correlated scores on a small validation set are an overfit trap, and logistic late fusion is the published WavLM-ensemble recipe [S6]; BOSARIS-style linear logistic regression is the standard implementation [S8].

Live rung: the best available XLS-R 300M checkpoint at demo time — a fine-tuned 300M model costs the same at inference as a frozen one. Gemini's off-the-shelf W2V2-AASIST is the fallback if the latency gate ($\lt 750$ ms 95th-percentile processing tail) fails, alongside ChatGPT's WavLM-base speed spare. Use a 2-window confirmation, not 3: two consecutive positive 4 s windows at a 2 s hop means a decision on ~6 s of evidence versus ~8 s for three — the difference between a crisp and a sluggish stage demo.

Cut order if behind: M6 → M2 → M3 → M4 degraded to mean fusion → M5 capped at one run. Never cut M0, M1 + first CSV, grouped validation, or calibration.

Q2 — Datasets to stage Thursday

Do not concatenate everything: adding another large corpus has sometimes degraded both In-the-Wild and in-domain performance [S10]. Leakage policy: hash-dedupe NSA train and test against everything public; In-the-Wild is celebrity-sourced and the highest contamination risk if the NSA set draws from public figures [S17]. ASVspoof 2021 LA/DF is evaluation data (telephony/VoIP-stressed), not new training evidence [S45][S2]. If outside data is disallowed: M1→M4 never needed it; fine-tune M5 on NSA data only with heavy augmentation (RawBoost + codec round-trips + RIR), and ask the sponsor explicitly whether pretrained anti-spoof checkpoints count as outside data — challenge organizers have policed pretraining overlap before.

Q3 — Base SSL front end

Pick facebook/wav2vec2-xls-r-300m. Converging evidence: best frozen front end at 17.4% mean $\text{EER}$ [S13]; best codec robustness frozen at 13.5% average $\text{EER}$ over ASVspoof 5 codec conditions vs. WavLM Large 18.1% and FBANK 49.8% [S13]; most resilient under babble noise + reverb at 9.4% $\text{EER}$ [S13] — directly relevant to a loud hall; and best in a controlled ten-test-set fine-tuning comparison at 4.26% $\text{EER}$ [S10]. XLS-R 1B reached 3.60% but was three times larger and slower, which is why the study's authors themselves selected the 300M model [S10]; model size alone does not ensure generalization [S11]. Fallbacks: WavLM Large (20.6% frozen [S13]) on load/compat failure; WavLM-base as the live-path speed spare.

Bake-off (after the first CSV, ~1.5–3 h on the M3 Pro): XLS-R 300M vs. WavLM Large vs. one of WavLM-base/HuBERT Large on an identical ~2–5k grouped, class-balanced NSA subsample, identical 4 s crops, identical mean pooling and logistic head. Pass rule: the challenger replaces XLS-R only if it wins by ≥1.0 absolute $\text{EER}$ point (ChatGPT adds: and wins a paired bootstrap and loses ≤2 points on the replay stress subset); ties go to XLS-R on prior evidence, or the faster model.

Scratch SSL pretraining: no, in no world. All three experts refuse it. Continual SSL training on M-AILABS/LJSpeech did not reliably improve detection [S10], and even ordinary supervised WavLM-base fine-tuning took ~20 h on a V100 for the cited run [S6]. Rent nothing for this.

Q4 — Hand-crafted features

Mostly noise once XLS-R is in the ensemble; two of three experts demote M2 to a single gated test, and that is the call. If the gate runs, use exactly these function-call features: LFCC + deltas (torchaudio.transforms.LFCC, torchaudio.functional.compute\_deltas); spectral flatness (librosa.feature.spectral\_flatness); high-frequency rolloff (librosa.feature.spectral\_rolloff, roll percentile 0.95) to catch vocoder bandwidth cutoffs; $F_0$ statistics via librosa.pyin — neural TTS/VC oversmooths $F_0$ [S35]; and pause statistics via librosa.effects.split, with the caution that detectors are known to shortcut on silence duration [S30], so pause features belong in the dashboard "why" panel more than in the model. Gemini's specific MFCC-20 recommendation and its overfitting statistic (high coefficient counts pushing $\text{EER}$ from 29.71% to 65.15%) were not independently verified in grading — treat as reported but unconfirmed. 30-minute gate: logistic fusion of SSL + hand scores on the fixed split; keep only if it adds ≥0.5 absolute point on the official metric and worsens neither the clean nor replay domain by more than 1 point. Otherwise delete.

Q5 — Offline test-time compute

Ranked by value per hour, with consensus and one resolved disagreement:

1. Score every speech window, aggregate mean + max of logits — always on, near-free, catches splices. Choose the aggregator from the sponsor's clip-label semantics (mean if the whole clip is synthetic; max/top-quartile if any spliced segment makes it positive).
2. Two-to-three-model logistic fusion (fine-tuned XLS-R + frozen-LR stream + Spectra-AASIST) — 0–1.5 points, trivial cost, only if errors are decorrelated and locked validation improves ≥0.5 point. Diversity flattens beyond 3–4 members.
3. Codec/speed TTA — the one real disagreement. ChatGPT says default off (often zero or negative because perturbation destroys forensic cues); Claude ranks codec round-trips high if the test set is codec-laden, mirroring the ASVspoof 5 eval codec list [S34]. Resolution: run the drop-time inventory; if the NSA test set shows codec/bitrate diversity, run ~5–6 codec views on the H100 (cheap there, hours on the M3 Pro); if it looks clean, skip TTA entirely.
4. XLS-R 1B as an offline-only second stream — controlled evidence suggests only ~0.66 $\text{EER}$ point average gain [S10]; H100-only, last priority.
5. Stochastic weight averaging of 3–5 neighboring fine-tune checkpoints — free at inference, do it if training is stable.
6. Pseudo-labeling / transductive adaptation on the test set — do not. All three experts land here: distribution-level calibration is standard evaluation practice, but pseudo-label fine-tuning on the hidden test set is gray-to-out-of-spirit [S34], and if the test class balance differs from train it can collapse predictions. Disclose whatever is done.

Returns flatten after all unique windows + one fusion partner + (conditionally) ~6 codec views. Four-way fusion did not prevent severe hidden-domain degradation in ASVspoof 5 per ChatGPT's evidence line — more members are not a hedge.

Q6 — Multilingual

Yes, one model, with a bounded claim. XLS-R was trained for cross-lingual representation learning — the large variant covers 128 languages [S6] — and MLAAD supplies 54-language spoof coverage across 127+ architectures [S2], with ReplayDF adding replayed material in six languages [S41]. What is lost: calibration and artifact-distribution transfer, not just lexical content — fine-tuning on a small English NSA set can narrow the multilingual representation, and MLAAD's multilingual anti-spoofing effectiveness had not been directly validated in the cited work [S2]. Gemini's caution is real: S3Ms encode language-dependent structure that classifiers confuse with spoof cues [S15], and zero-shot Bengali detection with W2V2-XLSR-53 hit 46.20% $\text{EER}$ [S12]. Its proposed fix — language orthogonalization, a target-free ridge map removing LID-projected variance [S15] — is a legitimate published technique but is a stretch goal here, not a plan item. Cheapest live demo: English + two non-English languages (Spanish/German, both in M-AILABS), 10–20 held-out genuine/fake clips per language played through the actual rig, ~1 h Saturday evening, zero training. XTTS v2 is in MLAAD's generator list [S3], which makes the fake clips defensible when a judge asks what generated them. Demonstrate that the pipeline handles multilingual audio; do not claim equal accuracy across languages.

Q7 — Replay and phone-quality audio

This is the demo's biggest risk and the experts agree on the numbers. Physical replay drives a top detector's $\text{EER}$ from 4.7% to 18.2%, and RIR-augmented retraining only partially compensates, to ~11.0% [S9][S41]. All six open detectors tested in ReplayDF degraded under replay [S41]; one W2V2-AASIST result fell from 90.0% accuracy / 10.6% $\text{EER}$ to 74.2% / 24.8% [S41]. Critically, replay failures are asymmetric — synthetic clips increasingly pass as genuine while genuine clips remain stable [S41] — so simply lowering $\theta$ buys false alarms, not safety.

Plan: structure the Friday-night capture. ChatGPT's matrix (2 phones × 3 positions × 2 levels × 2 classes × 20 clips = 480 captures, ~32 min of pure playback) supersedes an unstructured hour; split by original utterance and speaker before replaying so no source clip crosses train/validation, and hold out one phone-position cell. Train on a mixture: ~30% clean, ~40% RIR/ReplayDF-style channel simulation, ~20% codec round-trips (Opus, AMR, μ-law, AAC, MP3), ~10% combined RawBoost/noise/clipping — applied to both classes, because a cross-domain study traced a major failure to compressed bona-fide audio and showed indiscriminate 8 kHz resampling can hurt an otherwise strong recipe [S10]. Robust augmentation of this kind cut ASVspoof 5 test $\text{EER}$ from 7.68% to 3.57% in one study [S10].

Honest ceiling: matching clean accuracy through-air is not achievable this weekend. The realistic claim is ~80–90% balanced accuracy on the fixed, matched two-phone rig (the demo is a 2-way choice between known devices with temporal smoothing, not open-set detection), and 70–80% for any unmatched phone/room. Go/no-go gate: ≥80% balanced accuracy on untouched local replay clips and ≤10% genuine false alarms; below that, use a disclosed 3.5 mm wired detector feed while keeping mic-based localization — and say so, rather than implying the wired feed is replay-robust.

Grandma device (requirements only): phone-call audio is codec-limited rather than air-replayed, and codec augmentation is the documented mitigation, with XLS-R the most codec-robust frozen front end at 13.5% average codec $\text{EER}$ [S13]. Requirements: robust at $f_s = 8$–16 kHz; $T_w = 3$–6 s with a rolling decision over 10–20 s of speech; event-level alert within ~8–12 s; multi-window confirmation with hysteresis and an explicit uncertain state; and very low false-positive tolerance — on the order of one false alert per tens of hours of genuine calls at the event level, because false alarms destroy an elderly user's trust. A 300M model runs on a laptop or mains-powered hub; a phone app needs quantization/distillation; a microcontroller-class device is not feasible with this model — it requires a separately distilled compact model and a new validation program. ASVspoof 2021 LA is the right telephony stress set but is evaluation data, not product validation [S45]. Weekend scope: a mockup puck driven by the laptop.

Q8 — Reinforcement learning

Unanimous: no role, and the batch-efficiency premise is wrong. This is supervised binary classification with a differentiable loss; a cost-weighted loss plus calibration covers any DCF-like metric with no sequential decision problem for RL to solve. RL augmentation-policy search exists in the literature but costs hundreds of full train/evaluate cycles — the exact resource this weekend lacks — while a fixed published recipe (RawBoost + audiomentations [S38]) captures most of the gain for zero search cost. The throughput seen in RL systems comes from cheap simulators generating unlimited data, not from RL making gradient steps more efficient; on-policy methods are typically less sample- and compute-efficient than supervised minibatch training. Spend the GPU dollars on fine-tuning.

Q9 — Hardware

Mic array. The three experts split 4-conditional / 2-leaning / 2, and the geometry settles it. For phones at ±0.30 m and 0.80 m range, the inter-source TDOA difference is ~9.8 samples at 48 kHz with $d_{mic}=0.10$ m (angular quantum ~4°/sample, ~1–2° with parabolic interpolation) — two mics separate the sources with large margin, and front-back ambiguity is irrelevant for sources confined to the front half-plane. What four mics buy is robustness (six GCC-PHAT pairs to vote across, better behavior in a reflective hall) and ~6 dB delay-and-sum gain vs. ~3 dB. Ruling: use a 4-mic array only if it is an off-the-shelf clock-synchronized USB unit (ReSpeaker-class) the EE already has working; hand-wiring four channels, or mixing independent USB mics, is complexity for no reason — go 2-mic. Detector input: a selected raw channel by default. Beamforming/noise suppression can mask the very artifacts the detector keys on — the same channel-processing mechanism that makes replay hurt [S35] — so it feeds localization, and feeds detection only if a Saturday A/B on the replay holdout improves.

Add-ons, ranked: (1) Servo analog gauge REAL→SYNTHETIC — 2–4 h, ~$10, low risk; makes continuous $P_{synth}$ legible across a room; unanimous build. (2) Grandma-device mockup puck (Claude's proposal) — RGB LED + coin vibration motor on the same serial protocol, 3–6 h, low risk; converts the second product framing from slideware into hardware, arguably the highest story value per hour. (3) Fixed phone docks with per-dock source LEDs (ChatGPT) — 2–3 h; stabilizes replay geometry, which directly protects detector accuracy. (4) Push-to-challenge button — a judge plays any clip and sees a live score; 1–2 h, mostly software; display $P_{synth}$, not just a verdict, since an unseen generator may fool the model. (5) Minimal enclosure — 3–5 h, build only if ahead. Laser lock: cut, unanimously — alignment and servo jitter, crowd-safety optics, and zero information beyond the head pose and LED.

Q10 — Risks and the first two hours

• Metric mismatch: the metric could be AUC, $\text{EER}$, thresholded accuracy, a DCF-family cost, or a proper scoring rule — precedent challenges score calibration explicitly and weight false alarms 10× [S34]. Keep raw logits, calibrated probabilities, and hard decisions separately; submit calibrated probabilities (near-optimal under log-loss/Brier, harmless under ranking) with a rank-preserving raw-score CSV variant ready. Prefer Platt scaling; isotonic only with thousands of calibration examples per class.
• Class imbalance: anti-spoof train sets run ~90% spoofed (ASVspoof 2019 LA: 2,580 bona fide vs. 22,800 spoof [S38]; ASVspoof 5 ~8:1 [S6]). Use a weighted sampler enforcing ~50% bona fide per batch; never quote accuracy; never assume the test prior matches train.
• Split hygiene: generator- and speaker-disjoint validation; a random clip split is not decision-grade. Check the duration-by-class histogram in hour one — clip length correlating with label is a classic shortcut.
• Silence/music: some detectors decide on silence duration rather than speech [S30]. VAD-gate the live scorer; offline, define a non-speech policy (emit a prior-like score, flag internally); test one music file and one silence file before every CSV.
• Format pitfalls: one FFmpeg path to 16 kHz mono float32 for train and test; never denoise or "clean" test audio — codec artifacts are signal; avoid loudness normalization; for stereo, score channels and average logits rather than blindly downmixing. Unit-test empty, truncated, NaN, odd-sample-rate, MP3/M4A, and VBR files — every file must yield one finite score in exact row order.
• Score direction: Spectra-AASIST reports bona-fide-vs-spoof [S26]; assert with known examples that the exported value increases with synthetic likelihood before any CSV ships.

First-two-hours runbook (merged): 0:00–0:20 read metric, label semantics, outside-data policy, submission schema; send the sponsor three written questions (outside data? metric? type labels?). 0:20–0:40 inventory — $N$, formats, $f_s$ and duration histograms by class, class balance, decode failures on a stratified sample. 0:40–1:00 build grouped splits; write the constant-prior CSV through the actual test loader and save it as the rollback artifact. 1:00–1:30 run Spectra-AASIST on a labeled sample, verify score direction, fit a provisional Platt map. 1:30–2:00 launch the M3 latency microbenchmark and start frozen-embedding extraction for the bake-off. No custom architecture work in this window.

Model-owner timeline (~26 h of a 24–30 h budget)

Friday 20:00–02:00 (6 h): two-hour runbook → constant + first learned CSVs; bake-off on the M3 Pro; structured 480-clip replay capture with the EE ~23:15–00:15; spin up and verify one datacenter-verified H100 [S42]; kick full embedding extraction overnight.

Saturday ~09:00–24:00 (≈15 h): bake-off readout and front-end lock (pass rule above); launch M5 run 1 (partial fine-tune) as a background H100 job; build M4 logistic fusion + calibration + all-window scorer while it trains; run 2 with the robust augmentation mixture; ingest the replay capture, lock its untouched split, and run 3 with replay adaptation; select one core checkpoint, fit separate clean and replay calibrators, freeze a stable CSV; integrate the live scorer on the rig with the EE and tune $\theta$ (confirmed alert on ~6 s of evidence); 30-min hand-feature gate and 45-min Spectra-AASIST fusion timebox; evening: multilingual demo clips, optional conditional TTA. No new architecture after ~22:00.

Sunday 06:00–11:00 (5 h): freeze weights and calibration; final CSV — fused, all-window, calibrated, with music/silence sanity files checked; schema, row-order, finite-range, and reproducibility checks; set live $\theta$, rehearse clean/replay pairs, confirm the wired fallback; archive code, hashes, licenses, and attribution (the hackathon requires crediting every model and dataset).

Budget: roughly 8–24 H100-hours depending on how many background runs fire. Both experts' rate quotes were disputed in grading; even at a pessimistic doubling of quoted verified-tier rates, $C$ stays inside the $50–100 envelope with margin. Use datacenter-verified hosts — the unverified tier is not worth the reliability risk on a deadline [S42] — checkpoint every run, and cap spend near $75.

Things to cut (unified)

Cut now, unanimously: scratch SSL pretraining; RL of any kind; a new custom AASIST back end; the laser lock; the manipulation-type head unless labels are excellent and all binary deliverables are frozen. First cuts if behind: XLS-R 1B/2B offline experiments; more than one auxiliary fusion checkpoint; TTA; hand features on gate failure; gradient boosting and isotonic calibration; multilingual live clips beyond the preselected set; enclosure elaboration. Hard gates: no learned CSV by Friday midnight → stop the bake-off, ship calibrated Spectra-AASIST; fine-tune not beating the frozen probe by Saturday midday → freeze the probe, move to replay robustness; live model misses the 750 ms latency gate → WavLM-base or off-the-shelf W2V2-AASIST live; through-air balanced accuracy $\lt 80\%$ → disclosed wired fallback. Never cut: robust decoding, the rollback CSV, grouped validation, score-direction tests, one clean core model, local replay evaluation, metric-matched calibration, final reproducibility checks.

Scope Definition Q&A

• Q1:

  Will you actually have access to the NSA labeled training set and dataset details before Friday night, or should the plan assume dataset drop happens at the start of the coding window with unknowns about size/generators/metric?

  
  A:

  No. The NSA training set, test set, metric, label format and outside-data policy are all released at or after the 8 pm Friday coding start. Plan the first two hours around a dataset drop with unknowns. Everything public (pretrained weights, ASVspoof, MLAAD, In-the-Wild, etc.) can and will be downloaded before Friday night, so the pre-event prep should be about staging those, not the NSA data.

• Q2:

  Should the search prioritize sources from the last 1-2 years (2024-2026) for SOTA deepfake detection benchmarks and checkpoints, or is older foundational work (ASVspoof 2019-2021 era) equally relevant to include?

  
  A:

  Split the recency window by type of claim. For checkpoints, front ends, datasets and which generators made recent fakes: last 12 months, and prefer the newest thing that is actually downloadable today. For generalization findings and methodology (what transfers to unseen generators, augmentation that works, how to split validation): last 24 to 36 months is fine, since those results do not go stale the way checkpoints do. Keep ASVspoof and WaveFake-era work only as baselines or standard datasets. Where a newer checkpoint has no independent generalization result yet, say so and rank it below an older one that does.

• Q3:

  For the grandma-scam-device framing, should the research treat it as a fully separate scoped deliverable (with its own hardware/latency feasibility research) or purely as a forward-looking discussion section with no dedicated hardware sourcing research?

  
  A:

  Discussion and feasibility section only. Give the model requirements it implies (latency, window length, false positive tolerance, phone-call codec robustness, rough on-device inference feasibility) and treat it as a later product direction. No microcontroller sourcing or benchmark research.

• Q4:

  Is there a specific set of competing hackathon teams or prior HackGT/NSA HEARSAY-style submissions whose approaches you want benchmarked or differentiated against, or should the comparison stay generic to the deepfake-detection literature?

  
  A:

  No competitor benchmarking. Compare against the published literature and public checkpoints. If prior audio deepfake hackathon or Kaggle-style challenge write-ups exist and were found in passing, one paragraph on what top entries did is welcome, but do not spend effort searching for them.

• Q5:

  Should dataset and checkpoint license/legal review (e.g., research-only vs. commercial-use restrictions) be treated as a hard constraint for the hackathon submission, or is it out of scope since this is a non-commercial hackathon entry?

  
  A:

  Informational, not a hard exclusion. Flag research-only or restrictive licenses as caveats. The hackathon requires crediting every model and dataset used, so list the license and source per item anyway.

• Q6:

  Is there anything else that would help refine this analysis?

  
  A:

  Time is the constraint, not credits. Prefer one confident recommendation with a named fallback over a menu of options. Where the underlying models disagree on a recommendation, show the disagreement and the reason instead of averaging it away. Every checkpoint, dataset and library named should be something that exists and is downloadable today; if unsure, say so rather than guess. Give wall-clock estimates for an Apple M3 Pro with 18 GB unified memory, not a generic laptop.

Appendix A — Scope Definition Q&A

• Q1:

  Will you actually have access to the NSA labeled training set and dataset details before Friday night, or should the plan assume dataset drop happens at the start of the coding window with unknowns about size/generators/metric?

  
  A:

  No. The NSA training set, test set, metric, label format and outside-data policy are all released at or after the 8 pm Friday coding start. Plan the first two hours around a dataset drop with unknowns. Everything public (pretrained weights, ASVspoof, MLAAD, In-the-Wild, etc.) can and will be downloaded before Friday night, so the pre-event prep should be about staging those, not the NSA data.

• Q2:

  Should the search prioritize sources from the last 1-2 years (2024-2026) for SOTA deepfake detection benchmarks and checkpoints, or is older foundational work (ASVspoof 2019-2021 era) equally relevant to include?

  
  A:

  Split the recency window by type of claim. For checkpoints, front ends, datasets and which generators made recent fakes: last 12 months, and prefer the newest thing that is actually downloadable today. For generalization findings and methodology (what transfers to unseen generators, augmentation that works, how to split validation): last 24 to 36 months is fine, since those results do not go stale the way checkpoints do. Keep ASVspoof and WaveFake-era work only as baselines or standard datasets. Where a newer checkpoint has no independent generalization result yet, say so and rank it below an older one that does.

• Q3:

  For the grandma-scam-device framing, should the research treat it as a fully separate scoped deliverable (with its own hardware/latency feasibility research) or purely as a forward-looking discussion section with no dedicated hardware sourcing research?

  
  A:

  Discussion and feasibility section only. Give the model requirements it implies (latency, window length, false positive tolerance, phone-call codec robustness, rough on-device inference feasibility) and treat it as a later product direction. No microcontroller sourcing or benchmark research.

• Q4:

  Is there a specific set of competing hackathon teams or prior HackGT/NSA HEARSAY-style submissions whose approaches you want benchmarked or differentiated against, or should the comparison stay generic to the deepfake-detection literature?

  
  A:

  No competitor benchmarking. Compare against the published literature and public checkpoints. If prior audio deepfake hackathon or Kaggle-style challenge write-ups exist and were found in passing, one paragraph on what top entries did is welcome, but do not spend effort searching for them.

• Q5:

  Should dataset and checkpoint license/legal review (e.g., research-only vs. commercial-use restrictions) be treated as a hard constraint for the hackathon submission, or is it out of scope since this is a non-commercial hackathon entry?

  
  A:

  Informational, not a hard exclusion. Flag research-only or restrictive licenses as caveats. The hackathon requires crediting every model and dataset used, so list the license and source per item anyway.

• Q6:

  Is there anything else that would help refine this analysis?

  
  A:

  Time is the constraint, not credits. Prefer one confident recommendation with a named fallback over a menu of options. Where the underlying models disagree on a recommendation, show the disagreement and the reason instead of averaging it away. Every checkpoint, dataset and library named should be something that exists and is downloadable today; if unsure, say so rather than guess. Give wall-clock estimates for an Apple M3 Pro with 18 GB unified memory, not a generic laptop.

Appendix B — Project Metadata

Mode: Deep Think

Created: September 25, 2026

Expert models:

• ChatGPT GPT-5.6 Sol · Max
• Claude Fable 5 · Max
• Gemini 3.1 Pro · High

Generated by VeriLM