PERIT-STT · Travel & hospitality

Rebookings during disruption — airport noise, urgency, and three changes in one turn.

Audio comes from airline desks, hotel front office, disruption lines. Scored on word error rate, with a working travel & hospitality operator as the control.

880
Clips in the target held-out set
68h
Audio, never published
8
Languages in the set
1
Working travel & hospitality operator as control

Leaderboard

Illustrative figures. Public runs begin when the held-out sets close.
ModelWord error rate (%)Entity accuracyDiarization error
Human control · sector transcriptionistHuman control
2.8 ± 0.7
98.6%2.9%
Frontier ASR · lab AFrontier
7.6 ± 0.9
90.3%9.3%
Frontier ASR · lab BFrontier
7.7 ± 2.3
90.2%11.4%
Specialist vendor ASRSpecialist
9.1 ± 2.4
91.8%10.2%
Frontier multimodal · lab CFrontier
12.6 ± 1.3
87.4%12.5%
Open weights · 8BOpen weights
20.8 ± 1.1
71.4%16.9%
Task families

What the set actually asks for.

  • Verbatim transcription
  • Entity capture (IDs, amounts, names)
  • Speaker diarization
  • Code-switching
  • Noisy & far-field audio
  • Domain vocabulary
Method & limits

How a run on this set is produced.

  1. 01Audio is held out and never published. A 60-clip development split per sector is open for calibration.
  2. 02Reference transcripts are written by two annotators independently; a senior reviewer adjudicates every disagreement.
  3. 03Each clip is run 3 times per model. Reported intervals are bootstrapped at 95%.
  4. 04Human control is a working transcriptionist from the same sector, timed and paid at market rate.

Limits, stated up front: 8 languages only, single-channel audio, and no cross-sector transfer. The control is one operator per sector, so the human line carries its own error bar — it is a working standard, not a ceiling.