Transcription is solved on clean audio. Nobody calls from clean audio.
PERIT-STT scores models on real support recordings — mobile lines, crosstalk, accents and the domain vocabulary that decides whether a transcript is usable downstream.
- 22
- Models ranked
- 11
- Providers compared
- 1.53%
- Best word error rate · qwen3-asr-flash-2026-02-10
- 27
- Metrics reported per model
Who won what.
Accuracy is only one of the questions, so each of these is the best model on its own axis.
- Most accurate
- 1.53%Word Error Rate (WER)qwen3-asr-flash-2026-02-10Qwen
- Lowest character error
- 0.87%Character Error Rate (CER)mai-transcribe-1.5Microsoft
- Most exact matches
- 66.1%Exact Match Rateqwen3-asr-flash-2026-02-10Qwen
- Most robust
- 5.72%P90 WERqwen3-asr-flash-2026-02-10Qwen
- Fewest dropped words
- 0.26%Deletion Ratemai-transcribe-1.5Microsoft
- Fewest invented words
- 0.36%Insertion Rategpt-4o-transcribeOpenAI
- Fastest
- 0.4 sMedian Latencynova-3Deepgram
Every model, ranked by how many words it got wrong.
Lower is better. The leaderboard has the confidence interval behind each number, and every other metric.
- 01qwen3-asr-flash-2026-02-10Qwen1.53%
- 02universal-3-5-proAssemblyAI1.55%
- 03mai-transcribe-1.5Microsoft1.57%
- 04transcribe-1Fish Audio1.59%
- 05qwen3-asr-1.7bQwen1.60%
- 06gpt-transcribeOpenAI1.76%
- 07mai-transcribe-2Microsoft1.76%
- 08grok-stt-1.0xAI1.77%
- 09qwen3-asr-0.6bQwen1.84%
- 10gpt-4o-mini-transcribeOpenAI1.85%
- 11muse-voice-transcribe-1.0Meta1.86%
- 12chirp-3Google1.90%
- 13nova-3Deepgram2.12%
- 14parakeet-tdt-0.6b-v3NVIDIA2.13%
- 15voxtral-small-24b-2507-sttMistral AI2.14%
- 16voxtral-mini-transcribeMistral AI2.15%
- 17whisper-large-v3OpenAI2.38%
- 18whisper-1OpenAI2.51%
- 19whisper-large-v3-turboOpenAI2.55%
- 20gpt-4o-transcribeOpenAI3.17%
- 21nemotron-3.5-asr-streaming-multilingual-0.6bNVIDIA3.35%
- 22voxtral-mini-3b-2507Mistral AI3.89%
Word error rate: the share of words a model got wrong. Shorter bar is better.
Same audio, same rules, every model.
Both the reference and the model output go through the same normalization before a single word is counted, so a model is not penalised for writing “2” where the transcriber wrote “two”.
- Task
- Speech-to-text · English
- Models
- 22 from 11 providers
- Ranking
- Sorted by WER ascending, ties broken by CER.
- Run date
- 24 Sept 2026
- Lowercase and remove punctuation.
- Remove ground-truth annotation tags: [fp], [hn], [laughter], [bg], [artifact] and any other [...] tag.
- Remove unintelligible markers (()) and cut-off words (e.g. 'brow-'); keep the words inside ((uncertain)) and {{annotated}} markers.
- Remove non-speech markers in model output such as [music] or (laughs).
- Remove filler words (um, uh, hmm, mm, mhm, er) from both sides.
- Join spelled-out letters ('g. p. s.' and 'GPS' both become 'gps').
- Convert digits to words ('2' → 'two', '1st' → 'first', '5%' → 'five percent').
- Expand contractions on both sides (you're → you are, I'm → I am, don't → do not, there's → there is).
- Expand informal forms (gonna → going to, wanna → want to, ok → okay).
- Rejoin words split around a tag in the ground truth ('ex- [artifact] -perimenting' → 'experimenting').
The same call, ten different rooms.
A refund request in retail and a claim in insurance share a shape. They do not share vocabulary, background noise, or what counts as a wrong answer.
Plan changes, SIM swaps and outage triage, mostly over mobile lines that drop mid-sentence.
Card disputes, KYC checks and payment failures, where a misheard digit is the whole error.
First-notice-of-loss calls taken from roadsides and hospital corridors, rarely from a quiet room.
Appointment intake, triage lines and medication questions, dense with drug names and dosages.
Order status, returns and refunds, with catalogue names that no general model has seen.
Rebookings during disruption — airport noise, urgency, and three changes in one turn.
Driver and dispatch calls from vehicles, warehouses and doorsteps. Half of it is hands-free.
Admissions, fees and course guidance, often with a parent and a student on the same line.
Benefits, permits and civic helplines, where callers read reference numbers aloud, badly.
Technicians on rooftops and in basements, talking through a fix with a machine running.
How a sector run works.
- 01Audio is held out and never published. A 60-clip development split per sector is open for calibration.
- 02Reference transcripts are written by two annotators independently; a senior reviewer adjudicates every disagreement.
- 03Each clip is run 3 times per model. Reported intervals are bootstrapped at 95%.
- 04Human control is a working transcriptionist from the same sector, timed and paid at market rate.