Method · 8 Jul 2026

What held-out means when the data is audio

Text benchmarks leak by being quoted. Audio leaks by being reused — the same speaker, the same room, the same script.

Keeping a set held out is not only about publishing it. Two clips recorded by the same speaker in the same room are close to the same clip, so a set that draws its development and test audio from one recording session is already compromised.

The development split is drawn from separate sessions and separate speakers to the test set. Speakers appear in one side or the other, never both, and the sector, device and acoustic condition mix is reported for each so the two are comparable without overlapping.