The library

Rendered from docs/consults/2026-09-26_m5-extra-data-finetune_RESPONSE_verbatim.md; the markdown file is the record.

September 26, 2026

read the attached doc and answer questions technically as an ML expert would don't hedge

Executive summary

Scope

Technical review of an XLS-R 300M fine-tune data mix and training recipe (M5) for a spoof-detection hackathon system, judged on normalized minDCF ($C_{FA}=4$, $C_{miss}=1$, $\pi=0.3$). Covers MLAAD spoof-only inclusion risk, ASVspoof 2019 LA utility, batch mixing/curriculum, and fine-tune hyperparameters (unfreeze depth, LR, epochs, augmentation, pooling, loss). Excludes deployment/Docker engineering and non-M5 pipeline components.

Conclusion

The cost function is the design spec: normalized DCF reduces to $P_{miss}+9.33\,P_{FA}$, and at the test set's 70/30 split one false alarm costs exactly four misses. Both experts therefore invert the doc's plan — real-side diversity beats spoof volume. Take all ~5.1k ASV19 bona fide (highest-EV item); cut ASV19 spoof from 45k to a ~2.5k channel anchor or zero; drop the +50k DiffSSD spoof (zero new generators); run a constant mix with no curriculum or taper; use a learned layer-weighted sum plus attentive statistics pooling with BCE, not OC-softmax. MLAAD is the one live disagreement: drop outright (Gemini) versus keep at ~28% of the spoof side gated behind a CPU corpus probe with an explicit drop rule (Claude).

Both experts independently derived the $9.33\times$ false-alarm weight and converged on ASV19 bona fide in / ASV19 spoof out, no curriculum, no OC-softmax, layer-weighted sum plus attentive statistics pooling, and class-symmetric augmentation. They split on two items: MLAAD (drop versus gate-and-cap) and unfreeze depth (freeze the whole backbone versus truncate to 12 layers and fully fine-tune). Both splits trace to identifiable causes — a premise error about synthesized audio carrying a microphone chain, and a frozen backbone collapsing M5 into M1. Nothing has been measured yet; every ratio is an engineering prior, so material uncertainty remains.

Sensitivities

• The MLAAD keep-at-28% recommendation holds only if the CPU corpus probe clears it; a positive fingerprint that projects LibriSpeech onto the MLAAD side forces a fallback to ~12% or a drop.

• The $L=12$ truncation assumes the frozen probe's layer-7 optimum survives fine-tuning; if learned layer weights concentrate at the top layer, $L=14\text{–}16$ is required.

• Every recommendation rests on the test set's 70/30 real/spoof split matching $\pi=0.3$; if the actual split differs, the clean one-FA-equals-four-misses equivalence breaks.

• The M5-versus-M1 call is unresolvable on the outer holdout alone, where the one-sigma noise on minDCF is 0.074–0.104 and gaps under ~0.15 carry no information.

Final Verified Code

A single self-contained script derives the normalized-DCF coefficient and the exact one-false-alarm-equals-four-misses equivalence, builds the break-even table for spoof-only corpus additions, solves the Gaussian minDCF operating point via brentq for the reported 0.61 and a 0.25 target, computes the binomial holdout noise floor $\sigma_{DCF}$, accounts XLS-R parameters and CPU FLOPs across $L\in\{24,14,12,9\}$, tabulates the layer-wise LR decay schedule for $\gamma\in\{0.9,0.85,0.8\}$, and compares the recommended pool against the doc's plan. Selected over the alternative, which computed only the cost weight.

Show code

import numpy as np, math, json
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

# ---------- 1. DCF normalization and cost-of-one-error ----------
C_FA, C_miss, pi = 4.0, 1.0, 0.3
num_FA  = C_FA*(1-pi)      # 2.8
num_miss= C_miss*pi        # 0.3
Cdef = min(num_FA, num_miss)
k = num_FA/num_miss        # weight on P_FA in normalized DCF
print(f"Normalized DCF = P_miss + {k:.4f} * P_FA   (C_def = {Cdef})")

N_test = 1671
N_real = round(0.70*N_test); N_spoof = N_test-N_real
dDCF_per_FA   = k/N_real
dDCF_per_miss = 1.0/N_spoof
print(f"N_real={N_real}, N_spoof={N_spoof}")
print(f"dDCF per single false alarm = {dDCF_per_FA:.5f}")
print(f"dDCF per single miss        = {dDCF_per_miss:.5f}")
print(f"=> 1 false alarm == {dDCF_per_FA/dDCF_per_miss:.2f} misses in file-count terms\n")

# ---------- 2. Break-even for a spoof-only corpus that costs FA ----------
for dPfa_pct in [0.5,1.0,2.0]:
    dPfa = dPfa_pct/100
    need_dPmiss = k*dPfa
    print(f"If a corpus addition raises P_FA by {dPfa_pct:>4.1f}% ({dPfa*N_real:5.1f} real files), "
          f"it must cut P_miss by {need_dPmiss*100:5.2f}% ({need_dPmiss*N_spoof:5.1f} spoof files) just to break even")
print()

# ---------- 3. Where does the minDCF operating point sit? (equal-var Gaussian) ----------
def gauss_mindcf(d):
    t = np.log(k)/d + d/2.0
    Pfa = 1-norm.cdf(t); Pmiss = norm.cdf(t-d)
    return t, Pfa, Pmiss, Pmiss + k*Pfa

print("d'    thr   P_FA     P_miss   minDCF   (share of DCF from FA)")
for d in [1.0,1.5,2.0,2.5,3.0,3.5,4.0]:
    t,pf,pm,dcf = gauss_mindcf(d)
    print(f"{d:3.1f} {t:6.2f} {pf:7.4f} {pm:8.4f} {dcf:7.4f}   {k*pf/dcf:5.1%}")

# invert: what d' matches the engineered-detector overall minDCF 0.61?
from scipy.optimize import brentq
d_eng = brentq(lambda d: gauss_mindcf(d)[3]-0.61, 0.2, 6)
t,pf,pm,_ = gauss_mindcf(d_eng)
print(f"\nEngineered LightGBM (minDCF=0.61) -> implied d'={d_eng:.2f}, operating at P_FA={pf:.3%}, P_miss={pm:.1%}")
d_good = brentq(lambda d: gauss_mindcf(d)[3]-0.25, 0.2, 8)
t2,pf2,pm2,_ = gauss_mindcf(d_good)
print(f"A 'good' M5 (minDCF=0.25)          -> implied d'={d_good:.2f}, operating at P_FA={pf2:.3%}, P_miss={pm2:.1%}")

# ---------- 4. Outer-holdout noise floor ----------
print("\n--- Outer-holdout decision noise (M5 vs M1) ---")
for Nb in [400, 600, 800]:
    for Ns in [400, 800]:
        pf_op, pm_op = 0.05, 0.30
        se_fa = math.sqrt(pf_op*(1-pf_op)/Nb)
        se_ms = math.sqrt(pm_op*(1-pm_op)/Ns)
        se_dcf = math.sqrt((k*se_fa)**2 + se_ms**2)
        print(f"  N_bona={Nb:4d} N_spoof={Ns:4d} (2 generators) -> 1-sigma on holdout minDCF ~ {se_dcf:.3f}"
              f"  [FA part {k*se_fa:.3f}, miss part {se_ms:.3f}]")

# ---------- 5. Truncation: params / CPU inference ----------
H, FF, L = 1024, 4096, 24
p_layer = 4*H*H + 2*H*FF
print(f"\nPer transformer layer params: {p_layer/1e6:.2f} M ; 24 layers = {24*p_layer/1e6:.0f} M")
dur_total_s = 1671*3.9          # mean ~3.9s given median 3.41, p90 4.57
frames = dur_total_s*50
flops_per_frame_layer = 2*p_layer
for Lk in [24,14,12,9]:
    tot = frames*flops_per_frame_layer*Lk
    print(f"  keep {Lk:2d} layers: {Lk*p_layer/1e6:5.0f} M params, {tot/1e12:6.1f} TFLOP total, "
          f"~{tot/60e9/60:5.1f} min @60 GFLOP/s CPU, ~{tot/150e9/60:5.1f} min @150 GFLOP/s")

# ---------- 6. Layer-wise LR decay schedule ----------
print("\n--- LLRD, 12-layer truncated backbone, top-layer LR 1e-5 ---")
for gamma in [0.9, 0.85, 0.8]:
    lrs = [1e-5*gamma**i for i in range(12)]   # i=0 is top (layer 12)
    print(f"  gamma={gamma}: L12={lrs[0]:.2e}  L9={lrs[3]:.2e}  L6={lrs[6]:.2e}  L1={lrs[11]:.2e}")

# ---------- 7. Recommended batch composition ----------
mix = {
 "REAL side (50% of batch)": {"LJ Speech (1 spk)":0.25,"LibriSpeech (~80 spk)":0.35,"ASV19-LA bona fide / VCTK (~40 spk)":0.40},
 "SPOOF side (50% of batch)":{"DiffSSD inner generators (8)":0.62,"MLAAD (150 generators)":0.28,"ASV19-LA A01-A06":0.10},
}
rows=[]
for side,d in mix.items():
    for src,sh in d.items():
        rows.append((side,src,sh,0.5*sh))
print("\n--- Recommended batch composition ---")
for side,src,sh,ov in rows:
    print(f"  {side:26s} {src:38s} {sh:5.0%} of side  = {ov:5.1%} of batch")

pool = {"NSA DiffSSD spoof (2k/gen x8)":16000,"NSA LJ+Libri real (sample+extra)":18400,
        "MLAAD spoof":6400,"ASV19 bona fide":5100,"ASV19 spoof (stratified A01-A06)":2500}
tot=sum(pool.values()); nsa=pool["NSA DiffSSD spoof (2k/gen x8)"]+pool["NSA LJ+Libri real (sample+extra)"]
print(f"\nRecommended pool total = {tot:,} clips; NSA share = {nsa/tot:.0%}")
doc_pool = {"NSA spoof (70k)":70000,"NSA real (18.4k)":18400,"MLAAD":6400,"ASV19 bona":5100,"ASV19 spoof":45000}
dt=sum(doc_pool.values()); dn=doc_pool["NSA spoof (70k)"]+doc_pool["NSA real (18.4k)"]
print(f"Doc 'add everything' pool = {dt:,} clips; NSA share = {dn/dt:.0%}; real share = {(18400+5100)/dt:.0%}")
print(f"Recommended real share of pool = {(18400+5100)/tot:.0%}")

# ---------- FIGURES ----------
fig,ax=plt.subplots(1,2,figsize=(13,4.6))
ds=np.linspace(0.5,5,300)
res=np.array([gauss_mindcf(d) for d in ds])
ax[0].plot(ds,res[:,3],'k-',lw=2,label='minDCF')
ax[0].plot(ds,res[:,1],'r-',lw=1.6,label='$P_{FA}$ at min point')
ax[0].plot(ds,res[:,2],'b-',lw=1.6,label='$P_{miss}$ at min point')
ax[0].axvline(d_eng,color='gray',ls='--'); ax[0].text(d_eng+.05,0.8,'LightGBM\n(0.61)',fontsize=8)
ax[0].axvline(d_good,color='green',ls='--'); ax[0].text(d_good+.05,0.8,'target M5\n(0.25)',fontsize=8,color='green')
ax[0].set_xlabel("discriminability d'"); ax[0].set_ylabel("value"); ax[0].set_ylim(0,1.05)
ax[0].set_title("minDCF operating point ($C_{FA}$=4, $\\pi$=0.3): the\nthreshold lives in the low-$P_{FA}$ tail")
ax[0].legend(fontsize=8); ax[0].grid(alpha=.3)

lbl=[f"{s}\n{r}" for s,r,_,_ in rows]; vals=[ov for _,_,_,ov in rows]
cols=['#2b7bba','#4a9fd4','#79c2e8','#c2453a','#e06c5a','#f0a08f']
ax[1].barh(range(len(vals)),[v*100 for v in vals],color=cols)
ax[1].set_yticks(range(len(vals)));ax[1].set_yticklabels([r for _,r,_,_ in rows],fontsize=8)
ax[1].invert_yaxis(); ax[1].set_xlabel("% of every batch"); ax[1].grid(alpha=.3,axis='x')
ax[1].set_title("Recommended per-batch source shares")
for i,v in enumerate(vals): ax[1].text(v*100+0.3,i,f"{v*100:.1f}%",va='center',fontsize=8)
plt.tight_layout(); plt.savefig("fig_dcf_and_mix.png",dpi=140)

# break-even figure
fig2,ax2=plt.subplots(figsize=(6.4,4.4))
dpfa=np.linspace(0,0.03,100)
ax2.fill_between(dpfa*100,k*dpfa*100,20,color='#cfe9cf',label='MLAAD worth keeping')
ax2.fill_between(dpfa*100,0,k*dpfa*100,color='#f5cccc',label='MLAAD net harmful')
ax2.plot(dpfa*100,k*dpfa*100,'k-',lw=2)
ax2.set_xlabel("$\\Delta P_{FA}$ induced on real speech (%)")
ax2.set_ylabel("$\\Delta P_{miss}$ reduction required (%)")
ax2.set_title("Break-even for any spoof-only corpus addition\nslope = 9.33")
ax2.set_ylim(0,20);ax2.set_xlim(0,3);ax2.legend(fontsize=8);ax2.grid(alpha=.3)
plt.tight_layout(); plt.savefig("fig_breakeven.png",dpi=140)

json.dump({"normalized_dcf":"P_miss + 9.3333*P_FA","one_FA_equals_misses":4.0,
           "recommended_pool":pool,"recommended_batch_shares":{r:ov for _,r,_,ov in rows}},
          open("m5_recommended_mix.json","w"),indent=2)
print("\nfiles written: fig_dcf_and_mix.png, fig_breakeven.png, m5_recommended_mix.json")

Expert Summaries (2)

Claude

Approach:

Derived the file-level cost equivalence and the Gaussian minDCF operating point numerically, then used the resulting break-even frontier as the decision rule for every data question; corrected three premises in the source doc before answering.

Conclusion:

Keep MLAAD gated at ~28% of the spoof side behind a CPU corpus probe with an explicit drop rule; add all ASV19 bona fide plus a ~2.5k spoof channel anchor; drop the +50k DiffSSD spoof; truncate the backbone to $L=12$ and fully fine-tune; no curriculum, no OC-softmax.

Strengths:

• Quantified the decision rules — break-even table, operating point at $P_{FA}\approx0.97\%$, and a holdout noise floor $\sigma_{DCF}\approx0.074\text{–}0.104$ that reframes the M5-versus-M1 call

• Correctly identified that MLAAD clips are synthesized output and therefore carry vocoder and resample signatures, not an M-AILABS microphone chain

• Designed a properly controlled shortcut probe (MLAAD versus DiffSSD, label held constant) rather than the confounded spoof-versus-bona probe in the doc

• Named six failure modes absent from the doc, including score–duration coupling and LJ Speech single-speaker leakage into the stacker

Weaknesses:

• The specific ratios (28% MLAAD, $L=12$, 2.5k ASV19 spoof) are engineering priors presented with more precision than the underlying evidence supports

• Stated the ASVspoof 2019 LA eval partition as 48 speakers, which is itself off from the commonly cited 67; the train+dev ≈40 correction stands but the supporting detail is imprecise

• Volume of recommendations risks exceeding the 4–6 human-readable ablation slots it correctly says are the binding constraint

Unique:

The holdout noise floor analysis — showing that ~97% of outer-holdout minDCF variance comes from the $P_{FA}$ term and that a gap under ~0.15 is not evidence — directly changes the M5-versus-M1 submission decision.

Gemini

Approach:

Applied the $9.33\times$ false-alarm penalty as a blunt risk filter, eliminating any data source or architectural choice whose $P_{FA}$ risk could not be bounded, and optimized for the shortest path to a stable model.

Conclusion:

Drop MLAAD and ASV19 spoof entirely; use ASV19 bona fide for real-side diversity; freeze the whole XLS-R backbone and train only a learned layer-weighted sum with attentive statistics pooling; skip OC-softmax and curriculum.

Strengths:

• Reached the same cost-asymmetry conclusion independently, corroborating the $9.33\times$ weight and the real-side-diversity priority

• Named the asymmetric-augmentation failure mode, matching the highest-probability implementation bug the other expert also flagged

• Gave the tightest, most executable minimal plan — valuable as a low-variance fallback under deadline

Weaknesses:

• Premise error: attributed a recording-chain shortcut to MLAAD even though MLAAD contains only synthesized audio, overstating the case for dropping it

• Freezing the entire backbone makes M5 architecturally near-identical to the M1 frozen probe it must beat, undercutting both the replacement goal and stacker diversity

• Repeated the doc's ~100-speaker figure for ASV19 LA train+dev without checking it

• Proposed diagnostic (MLAAD versus LibriSpeech) is confounded — that is the spoof-versus-bona task, so near-perfect separation is uninformative

• No concrete caps, per-source shares, or epoch-level reasoning for the mix

Unique:

Forced the question of whether M5 should differ from M1 at all in hypothesis class, and supplied the cheapest possible control arm.

Assumptions and Limitations

Assumptions

• $C_{FA}=4$, $C_{miss}=1$, $\pi=0.3$, giving $\mathrm{DCF}_{norm}=P_{miss}+9.333\,P_{FA}$

• Test set of 1,671 files at 70% real: $N_{real}=1170$, $N_{spoof}=501$, so one false alarm equals exactly four misses in file-count terms

• Equal-variance Gaussian score distributions for the operating-point analysis, threshold $t=\ln(k)/d'+d'/2$

• Outer-holdout noise floor assumes 400–800 files per class, operating point $P_{FA}\approx5\%$, $P_{miss}\approx30\%$, and binomial sampling error

• XLS-R 300M geometry: 24 layers, hidden 1024, FFN 4096, ~12.58M parameters per layer; mean test clip duration ~3.9 s at 50 frames/s

• CPU throughput assumed at 60–150 GFLOP/s for the inference-time estimates

• ASVspoof 2019 LA train+dev bona fide is ~40 VCTK speakers, not the ~100 stated in the source doc (requires protocol-file verification)

• MLAAD clips are fully synthesized output and carry no M-AILABS microphone chain; their corpus signature is resample rolloff plus reading register

• M-AILABS is unavailable and no substitute matched bona fide will be obtained

• Compute is unlimited but human ablation-reading capacity is limited to roughly 4–6 decision points

Limitations

• No empirical measurement supports any of the recommended ratios — 28% MLAAD, 2.5k ASV19 spoof, $L=12$, $p_{aug}=0.65$ are engineering priors with stated decision rules, not measured optima

• The MLAAD shortcut diagnostic has not been run; the keep-versus-drop call is therefore procedural, not empirical

• MLAAD's actual per-model sample-rate and resample provenance could not be verified from the doc, so the bandwidth-rolloff hazard is inferred rather than measured

• The ASV19 LA speaker count correction rests on recollection of the protocol rather than the protocol file itself; the supporting eval-partition figure cited by one expert is imprecise

• The Gaussian score model used for the operating point and noise floor is an idealization; real score distributions are typically skewed and heavy-tailed on the bona fide side, which would shift $P_{FA}$ at the optimum

• No expert had access to the M1 frozen-probe NSA number, so the bar M5 must clear is unknown

• The interaction between the recommended mix and the logistic fusion stacker was not modelled quantitatively

Figures (2)

Claude (2 figures)

Nomenclature (17 symbols)

Narrative

Expert Comparison

The agreement is broad and load-bearing. Both experts independently derived the same cost weight, both concluded that the doc's plan spends its budget on the cheap error class, both rejected curriculum staging, both rejected OC-softmax, both landed on a learned layer-weighted sum feeding attentive statistics pooling, and both independently named class-asymmetric augmentation as the unflagged failure mode with the highest expected cost. That convergence — reached from different starting postures — is the strongest signal in this review.

The disagreements are two: what to do with MLAAD, and how much of the backbone to train. Gemini's overall confidence rating of High is not supported by its own reasoning, since its MLAAD verdict rests on a mechanism that does not apply to fully synthesized audio and its architectural recommendation would reproduce M1.

Expert Divergence

Divergence 1 — MLAAD: drop versus gate. Gemini's case for dropping rests on the claim that MLAAD spoof carries "recording-chain acoustics... noise floors, compression artifacts, mastering chains" from the studio data used to train commercial TTS systems. Claude rejected the mechanism: MLAAD clips are 100% vocoder output and never passed through an M-AILABS microphone, so what they inherit is the text domain, the audiobook reading register, and each model's resampling history on the way to 16 kHz. This is the correct reading. Two consequences follow. First, the hazard is narrower than the doc and Gemini assume — it is a bandwidth-deficit and rolloff cue, which the existing band-limit augmentation directly attacks, and an audiobook-prosody cue, which is already contradicted inside the training set because LibriSpeech is LibriVox audiobook speech sitting in the same batches labelled bona fide. Second, the unreachable M-AILABS hosts matter much less than the doc fears; pairing bona fide from the same text corpus would not have addressed the resample axis at all.

That said, Gemini's conclusion is not unreasonable under the asymmetry — a spoof-only pool whose shortcut misfires produces weight-4 errors, and the 4:1 prior justifies structural suspicion. The resolution is that Claude's gated position strictly dominates: it applies the same risk control through a measurement rather than an assumption, costs zero GPU, and carries an explicit numeric drop rule. Gemini's answer is the correct action if the diagnostic fires; Claude's is the correct procedure. Note also that Gemini's proposed diagnostic is confounded — training LightGBM on MLAAD versus LibriSpeech is the spoof-versus-bona task itself, so near-perfect AUC is expected regardless of whether a corpus shortcut exists.

Divergence 2 — unfreeze depth. Gemini recommends freezing the entire transformer stack, reasoning that unfreezing upper layers destroys the mid-layer acoustic representations the frozen probe found best at layer 7. The premise is right about where the information lives; the conclusion does not follow. A frozen backbone with a learned layer-weighted sum and attentive pooling is, to within a pooling head, the M1 frozen probe. M5 must beat M1 on the outer holdout to be submitted, and its secondary value is supplying a decorrelated score to the fusion stacker. A model in the same hypothesis class as M1 can do neither well. Claude's truncate-to-$L=12$-and-fully-fine-tune proposal responds to the same layer-7 evidence with the opposite move: stop asking layers 13–24 to re-derive discarded information, and train the layers that actually carry the artifact signal. It also halves CPU inference (196.8 → 98.4 TFLOP for the full test set, roughly 27 minutes instead of 55 at 60 GFLOP/s), which matters under the hard CPU-deployment constraint.

Resolution: run $L=12$ full fine-tune as the primary M5 arm. Gemini's frozen configuration is worth retaining as a near-zero-cost control arm — it doubles as a sanity check that the fine-tune is not underperforming the probe, and with compute unlimited it costs one slot of wall-clock, not of judgment. Claude's own stated hedge applies: if the learned layer weights concentrate on the topmost retained layer, $L=12$ truncated too aggressively and $L=14\text{–}16$ is the fallback.

Technical Detail

The cost function is the specification, not a scoring detail

With $C_{FA}=4$, $C_{miss}=1$, $\pi=0.3$:

\mathrm{DCF}_{norm}=\frac{C_{miss}\pi P_{miss}+C_{FA}(1-\pi)P_{FA}}{\min(C_{miss}\pi,\;C_{FA}(1-\pi))}=P_{miss}+9.333\,P_{FA}

Because the test set's 70/30 real/spoof split coincides with $\pi=0.3$, the rate normalizers cancel and the statement becomes file-level and exact: one false alarm costs four misses. Claude's execution confirms this numerically — 0.00798 DCF per false alarm against 0.00200 per miss across $N_{real}=1170$ and $N_{spoof}=501$.

Two implications the source doc gestures at but does not operationalize. First, under an equal-variance Gaussian score model the minDCF threshold sits at $t=\ln(k)/d'+d'/2$; at a target $\mathrm{minDCF}=0.25$ that is $d'\approx3.33$, $P_{FA}\approx0.97\%$, $P_{miss}\approx15.9\%$ — roughly eleven false alarms out of 1,170 real files decide the score. The entire outcome lives in the right tail of the bona fide score distribution. Average-case spoof accuracy is close to irrelevant.

Second, the break-even for any spoof-only corpus addition is steep. A shortcut that raises $P_{FA}$ by one point (about 12 real files) must deliver 9.33 points of $P_{miss}$ reduction (about 47 spoof files) merely to pay for itself.

At equal shortcut magnitude, spoof-only additions are 4× more expensive than bona-fide-only additions. The doc's closing intuition that real-speech diversity "may" beat spoof diversity is not a maybe — it is a 4:1 prior.

Q1 — MLAAD (highest priority)

Recommendation: keep, gated, at roughly 28% of the spoof side (~14% of every batch), with a per-model cap of ~120 clips. The 15% first guess is too low to be useful: at 15% of a 50% spoof side, MLAAD supplies ~24 exposures per generator per epoch across ~150 generators, which is memorization territory rather than family-level artifact learning. The upside is specific — MLAAD is the only inventory item containing the commercial families (ElevenLabs Turbo, Gemini Flash TTS, Qwen3-TTS) the test plausibly contains, and the engineered detector currently scores 0.80 on ElevenLabs and 1.00 (no better than an all-bona-fide system) on grad_tts and pro_diff.

On the existing mitigations, the honest accounting is that two are load-bearing and two are decoration. Band-limit and codec augmentation applied symmetrically across classes is the real control, because it directly destroys the bandwidth-deficit cue. Silence trim plus ZMUV normalization is necessary but incomplete — ZMUV equalizes variance, not crest factor, so a peak-normalized spoof pool against a non-peak-normalized LJ (which peaks at 0.54 against the test set's 1.0) leaves an intact clipping signature. And the spoof-side cap is a compute knob, not a safety knob: a perfectly separable low-level cue is learned at 5% just as at 30%, and learned faster than the hard artifact signal because it carries less gradient noise. Do not treat the cap as protection.

On LibriSpeech as bona fide proxy: adequate for the domain axis, inadequate for the channel axis, and that is acceptable. The doc worries about "audiobook acoustics = fake," but LibriSpeech is LibriVox audiobook speech labelled bona fide in the same batches — the model cannot learn that mapping when ~35% of its real side contradicts it. What LibriSpeech does not cover is the resample/rolloff axis, which is why band-limit augmentation carries the load.

Decisive diagnostic (CPU, LightGBM, ~90 minutes, no GPU). Three parts. Probe A holds the label constant: train LightGBM on handcrafted features to separate MLAAD spoof from DiffSSD spoof, on augmented copies so it measures what survives the pipeline, then project the real pools onto that corpus axis. If AUC exceeds 0.95 and LibriSpeech lands on the MLAAD side, the shortcut points straight at the worst false-alarm source — fall back to ~12%. If AUC falls under 0.85 post-augmentation, there is no exploitable fingerprint. Probe B retrains the LightGBM NSA detector with and without MLAAD and compares inner-fold minDCF at $\pi=0.3$ with $P_{FA}$ broken out on LibriSpeech separately. Probe C exploits a rule the doc reads too conservatively: In-the-Wild is barred from training, not from read-out, so score both arms on In-the-Wild bona fide only at the inner-fold threshold and compare $P_{FA}$. Drop MLAAD if that delta exceeds +1 point — the break-even table shows it cannot repay that.

Q2 — ASVspoof 2019 LA

All 5.1k bona fide, unambiguously — this is the highest-expected-value single item in the entire plan. The current real side is one LJ speaker plus roughly 80 LibriSpeech readers, both audiobook-register read speech from two recording chains. Under $C_{FA}=4$, the bona fide manifold is what is being graded, and a two-chain manifold is a line.

On the doc's explicit question — does the VCTK microphone chain add or reduce false-alarm risk? — it reduces it, and the argument is one-directional. With two real chains, "unfamiliar chain" correlates with spoof, so a test-time phone or field recording lands in unmodelled space with no reason to be called real. With three real chains, the model is pushed toward a chain-invariant bona fide representation. The failure mode being traded into — learning "VCTK studio sound = real" and therefore missing a VCTK-derived VC attack — is a miss, cost 1. That trade is free money.

One correction to the doc that both the review and the user should act on: the ~100-speaker figure for ASV19 LA train+dev is wrong. Train and dev together are approximately 40 VCTK speakers; the larger speaker counts include the eval partition, which is not being used. The recommendation is unchanged (40 speakers is still a 40× increase over LJ's one), but the sizing of the benefit is smaller than the doc assumes. Verify against the protocol file before writing the mix.

On spoof: 45k is clearly wrong; zero is defensible; ~2.5k is better. A01–A06 are 2018-era unit-selection, WORLD/STRAIGHT vocoder and early neural systems whose artifacts — buzzy excitation, concatenation-joint phase discontinuity, over-smoothed envelopes — share little with diffusion or neural-codec artifacts. Modern SSL front ends separate them near-trivially, so after epoch one they contribute negligible gradient while consuming batch slots, and at 45k they would make an obsolete generator family the majority of all spoof evidence. The single reason to include any is channel anchoring: if VCTK appears exclusively as bona fide, speaker-and-chain identity becomes a shortcut. Roughly 2.5k stratified across A01–A06 (about 420 per attack, ~0.5:1 against the bona fide) forces the boundary to be within-channel. Gemini's zero is a reasonable simplification; the anchor is cheap insurance against a shortcut the plan otherwise creates.

Q3 — Mixing and curriculum

A concrete starting mix, stated as a recommendation rather than a settled ratio: label-balanced 50/50, with the real side split 25/35/40 across LJ / LibriSpeech / ASV19 bona fide, and the spoof side 62/28/10 across DiffSSD / MLAAD / ASV19. LJ is capped below its data share deliberately — it is one speaker, its marginal clips carry near-zero information, and over-weighting it invites "this voice = real." LibriSpeech is upweighted because it is the measured worst false-alarm source at 0.61 inner-fold minDCF overall and 0.77 on LibriSpeech specifically.

The resulting pool is ~48.4k clips at 49% real, against the doc's add-everything plan at ~144.9k clips and only 16% real. A 16%-real pool in a task where one false alarm costs four misses is the central design error, and it is worth stating plainly.

Kill the +50k DiffSSD spoof. The inner folds are generator-grouped, so within-generator volume buys nothing on the axis being validated. Going from 1k to 7k clips per generator adds zero new generators, multiplies epoch cost fivefold, and drives the real share down. Cap DiffSSD at ~2k per inner generator. This is a statistical argument, not a compute argument — compute is unlimited, and that does not make low-information samples informative.

No curriculum, no taper, constant mix. Three reasons. At 3–5 epochs a curriculum is one to two epochs per stage, untunable inside 4–6 human-readable ablation slots. Tapering is actively wrong here because the last epochs dominate the shipped weights, so the channel invariance bought from VCTK and the generator breadth bought from MLAAD are the first things forgotten at exactly the checkpoint that gets submitted. And the usual justification — anneal toward the target domain — does not apply, because NSA is not the target domain; the test is a fresh corpus with telephony and field recordings, so annealing toward NSA narrows the model in the false-alarm-generating direction.

Per-source loss weighting: use sampling ratios instead. They are equivalent in expectation with lower gradient-norm variance and a stable effective learning rate. The one exception worth a slot is a bona fide class weight of ~2.0 in BCE; going to the full 4.0 over-corrects because 50/50 sampling already supplies emphasis. This is genuinely uncertain — it is a tail-shaping intervention, and minDCF sees only ranking.

On whether the recipe becomes data-dominated at ~16k NSA clips per fold: under the doc's plan, yes and catastrophically, since NSA would be 61% of clips but real audio only 16%. Under the revised mix, NSA sits at ~71% and real at ~49%. The honest framing is that at this scale every extra-data decision is a recipe decision, which is why the ratios deserve more attention than the learning rate.

Q4 — Fine-tune recipe

Depth. Truncate to $L=12$ and fully fine-tune those layers with the CNN encoder frozen — 302M → 151M transformer parameters, and roughly 27 minutes of CPU inference on the full test set at 60 GFLOP/s instead of 55. $L=14$ is the conservative variant if the learned layer weights come back concentrated near the top. Keep Gemini's fully-frozen configuration as a cheap control arm; if it matches or beats the fine-tune, that is decision-relevant information about whether M5 can differ from M1 at all.

Representations. A learned softmax-weighted sum over the retained layers, initialized uniform, trained at $\eta_{head}$. Both experts converged here independently. The twelve extra scalars are also a free diagnostic — logging them tells you directly whether the truncation depth was right.

Learning rates and schedule. $\eta_{head}=5\times10^{-4}$ with no weight decay; $\eta_{bb}=1\times10^{-5}$ at the top retained layer with $\gamma=0.85$ decay descending and weight decay 0.01 off biases and LayerNorm; 8% warmup then cosine; bf16; grad clip 1.0. The $10^{-5}$-scale backbone rate is the well-supported part — SSL anti-spoofing fine-tunes live in $10^{-6}$ to $3\times10^{-5}$, and XLS-R representations degrade above roughly $5\times10^{-5}$. The choice between $\gamma=0.85$ and 0.9 carries little weight. Gemini's $10^{-4}$ is appropriate for its frozen-backbone arm, where only head parameters train, and should not be carried over to a fine-tune arm.

Epochs and selection. Four epochs maximum, validating every half epoch. The load-bearing detail is not the epoch count but the selection criterion: select on inner-fold minDCF at $\pi=0.3$, never on loss and never on EER. EER is the equal-cost operating point; the real operating point is at $P_{FA}\approx1\%$. Two checkpoints can tie on EER and differ by 0.1 in minDCF.

Augmentation. Keep the existing codec/noise/band-limit/RIR set at $p_{aug}=0.65$ and add only RawBoost's linear-plus-nonlinear convolutive algorithm at $p=0.25$ — that is the one axis (device and amplifier nonlinearity) the current set does not cover. The full RawBoost cascade duplicates existing additive-noise coverage and adds untunable configuration. Band-limit should be upweighted within the chain specifically because it is the mitigation for both the MLAAD rolloff hazard and real telephony false alarms. The non-negotiable constraint, which both experts flagged independently: the augmentation type and probability distribution must be identical across classes and across source pools. Sample the chain before looking at the label or source, and log realized augmentation rates per source-class cell. Precomputed codec round-trips on spoof pools with on-the-fly augmentation on real pools is the exact shape of bug that creates a codec→spoof correlation, and the test set contains codec'd real telephony.

Pooling. Attentive statistics pooling (mean + std, 128-d bottleneck). The cost function decides this: sharp attention or max-like pooling lets a single anomalous frame dominate a long real clip's embedding, producing a length-dependent false-alarm bias that lands entirely on the 5.9% of files over 5 seconds under a single global threshold. Mean+std retains within-clip variability without letting one frame dominate. AASIST is a legitimate back end with published gains, but it costs an ablation slot and brings its own hyperparameters for an expected gain smaller than getting the mix right.

Loss. BCE with label smoothing 0.05. OC-softmax is not merely unnecessary — it is contraindicated. The one-class argument works when the bona fide class is diverse enough that "compact" means "compactly covering all real speech." Here the bona fide class is one LJ speaker plus clean LibriSpeech, so a one-class objective draws a tight ball around clean audiobook read speech and every unfamiliar real clip — smartphone, telephony, field recording, all explicitly promised in the brief — falls outside it. That is a weight-4 error manufactured by design.

Failure modes the doc does not name

The doc covers level, silence, tiling and container shortcuts. Four additions, ordered by expected cost:

1. Score–duration coupling. minDCF uses a single global threshold, so any monotone drift of bona fide scores with clip length dumps its entire cost on one end of the duration distribution — potentially 10–70 files, the same order as the ~11 that decide the score. Detect with a Spearman correlation of score against log-duration on out-of-fold bona fide only; correct by fitting and subtracting a linear trend estimated on out-of-fold bona fide, which uses no test labels.
2. Augmentation–label leakage. Named independently by both experts and the highest-probability silent bug with a weight-4 payload.
3. Stacker miscalibration from LJ's single speaker. LJ Speech cannot be speaker-grouped, so it appears on both sides of every inner split and its out-of-fold bona fide scores are optimistically biased. The logistic stacker fitted on those scores will over-trust the XLS-R score on real audio and set too aggressive a fusion weight — which converts to false alarms on unfamiliar real test audio. Fit the stacker on LibriSpeech and VCTK out-of-fold bona fide only, or chapter-group LJ by recording session and treat it as weak grouping.
4. Trim-boundary and crest-factor signatures. A 35 dB relative threshold cuts precisely on clean synthetic audio and may cut nothing on noisy real audio, replacing a leading-silence cue with a trim-behaviour cue. Similarly, ZMUV leaves peak-to-RMS ratio intact. Both are ten-minute CPU histogram checks per pool; both are fixed with jitter (randomize the trim threshold in 30–40 dB, retain 0–200 ms randomly; apply random peak normalization to half of both classes before ZMUV).

The M5-versus-M1 decision is currently being set up on noise

The outer holdout has two generators and 26 bona fide groups. At a realistic operating point ($P_{FA}\approx5\%$, $P_{miss}\approx30\%$) with 400–800 files per class, $\sigma_{DCF}$ is 0.074–0.104, and roughly 97% of that variance comes from the $P_{FA}$ term because it is multiplied by 9.33. A holdout gap under ~0.15 is not evidence. With two held-out generators, the $P_{miss}$ estimate is effectively $n=2$ at the level that matters. Decide on the combination of 5-fold out-of-fold minDCF (much larger effective sample) and the holdout; if they disagree by less than 0.15, keeping M1 and routing M5 through the stacker is the lower-variance play, and the setup already supports it.

Over-engineering verdict

Over-engineered on data, proportionate on recipe, and under-invested in the quantity the metric actually measures. The recipe changes are four or five config lines and one or two runs. The data plan proposes growing the pool from 16k to ~145k clips at 16% real, in a task where one false alarm costs four misses — maximum effort along the lowest-return axis. Concrete cuts: the +50k DiffSSD spoof, ASV19 spoof from 45k to ~2.5k, curriculum and taper entirely, OC-softmax and AASIST, and the non-convolutive parts of RawBoost. Reallocate to the MLAAD CPU diagnostic, the mix A/B, and the three CPU audits, all of which fit comfortably in the remaining window.

One caution specific to the "compute is unlimited" clarification: it lifts the H100-hour constraint but not the binding one, which is that each A/B needs a human to read the result and decide. Launching twenty configs and selecting the best inner-fold minDCF across five noisy folds at $C_{FA}=4$ will return a configuration overfit to fold noise that does not reproduce on the holdout. Run few arms, pre-declare the criterion, and stop.

Scope Definition Q&A

• Q1:

  How firm is the 4-6 run ablation budget (including the 6 final runs) — can we spend more than 1 H100-hour total across all four questions combined, or must each question's experiment fit within a shared strict ceiling?

  
  A:

  unlimited

• Q2:

  Is the M1 frozen-probe outer-holdout minDCF (the bar M5 must beat) expected in time to gate whether M5 is submitted at all, or should the answer assume M5 will be submitted regardless of that comparison?

  
  A:

  M1's NSA number lands hours before the M5 runs finish, so the bar is known in time. M5 replaces M1 only if it beats M1 on the outer holdout. Its scores feed the fusion stacker either way, so a robust recipe that differs from the frozen probe still has value. Optimize expected minDCF.

• Q3:

  Should the recommendation treat the 15%-of-spoof-side cap on MLAAD and the 'NSA at 50%+ of every batch' figures as fixed constraints, or are these open to being revised upward/downward as part of the answer?

  
  A:

  All ratios are first guesses. Give concrete alternatives if they're justified.

• Q4:

  Do you want the answer to include a decision on fetching the few extra GB of M-AILABS English real audio (to pair with MLAAD spoof), given remaining wall-clock and download/setup time, or should that option be treated as effectively off the table due to time cost?

  
  A:

  Off the table. Both M-AILABS hosts (data.solak.de, caito.de) returned nothing to a header request at 02:36 EDT, and there's no verified mirror. LibriSpeech is the only audiobook bona fide we have. Tell us whether MLAAD should be dropped, capped, or kept only if a diagnostic clears it.

• Q5:

  Should the answer's runnable experiments assume access to the existing engineered-detector/LightGBM stacking pipeline for diagnostics (e.g., the MLAAD-vs-LibriSpeech shortcut probe), or should diagnostics be limited to the XLS-R fine-tune pipeline itself?

  
  A:

  Yes. The handcrafted-feature + LightGBM tooling runs on the Mac's CPU. Frozen XLS-R embeddings can only be recomputed on the rented box.

• Q6:

  Is there anything else that would help refine this analysis?

  
  A:

  be somewhat cognizant of time we have another day+ 5 hrs

Appendix A — Scope Definition Q&A

• Q1:

  How firm is the 4-6 run ablation budget (including the 6 final runs) — can we spend more than 1 H100-hour total across all four questions combined, or must each question's experiment fit within a shared strict ceiling?

  
  A:

  unlimited

• Q2:

  Is the M1 frozen-probe outer-holdout minDCF (the bar M5 must beat) expected in time to gate whether M5 is submitted at all, or should the answer assume M5 will be submitted regardless of that comparison?

  
  A:

  M1's NSA number lands hours before the M5 runs finish, so the bar is known in time. M5 replaces M1 only if it beats M1 on the outer holdout. Its scores feed the fusion stacker either way, so a robust recipe that differs from the frozen probe still has value. Optimize expected minDCF.

• Q3:

  Should the recommendation treat the 15%-of-spoof-side cap on MLAAD and the 'NSA at 50%+ of every batch' figures as fixed constraints, or are these open to being revised upward/downward as part of the answer?

  
  A:

  All ratios are first guesses. Give concrete alternatives if they're justified.

• Q4:

  Do you want the answer to include a decision on fetching the few extra GB of M-AILABS English real audio (to pair with MLAAD spoof), given remaining wall-clock and download/setup time, or should that option be treated as effectively off the table due to time cost?

  
  A:

  Off the table. Both M-AILABS hosts (data.solak.de, caito.de) returned nothing to a header request at 02:36 EDT, and there's no verified mirror. LibriSpeech is the only audiobook bona fide we have. Tell us whether MLAAD should be dropped, capped, or kept only if a diagnostic clears it.

• Q5:

  Should the answer's runnable experiments assume access to the existing engineered-detector/LightGBM stacking pipeline for diagnostics (e.g., the MLAAD-vs-LibriSpeech shortcut probe), or should diagnostics be limited to the XLS-R fine-tune pipeline itself?

  
  A:

  Yes. The handcrafted-feature + LightGBM tooling runs on the Mac's CPU. Frozen XLS-R embeddings can only be recomputed on the rented box.

• Q6:

  Is there anything else that would help refine this analysis?

  
  A:

  be somewhat cognizant of time we have another day+ 5 hrs

Appendix B — Project Metadata

Mode: Deep Think

Created: September 26, 2026

Expert models:

• Claude Opus 5 · Extra High
• Gemini 3.1 Pro · High

Generated by VeriLM