PERIT-STT · Leaderboard
Speech recognition, measured against human experts
Ranked by word error rate. Switch tabs to see accuracy, the kind of error, robustness on hard clips, clip length, and what each model costs to run.
| Rank | Model | Word Error Rate | WER 95% CI (low) | WER 95% CI (high) | Word Accuracy | Character Error Rate | Strict WER | Match Error Rate | Word Information Lost | Word Information Preserved |
|---|---|---|---|---|---|---|---|---|---|---|
| 01 | qwen3-asr-flash-2026-02-10Qwen | 0.91%0.56–1.33 | 0.56% | 1.33% | 99.09% | 0.50% | 4.60% | 0.91% | 1.22% | 98.78% |
| 02 | transcribe-1Fish Audio | 1.28%0.79–1.83 | 0.79% | 1.83% | 98.72% | 0.74% | 4.97% | 1.27% | 1.78% | 98.22% |
| 03 | mai-transcribe-1.5Microsoft | 1.33%0.88–1.84 | 0.88% | 1.84% | 98.67% | 0.63% | 4.89% | 1.33% | 1.89% | 98.11% |
| 04 | qwen3-asr-0.6bQwen | 1.36%0.90–1.94 | 0.90% | 1.94% | 98.64% | 0.64% | 5.03% | 1.36% | 2.12% | 97.88% |
| 05 | qwen3-asr-1.7bQwen | 1.36%0.87–1.86 | 0.87% | 1.86% | 98.64% | 0.75% | 5.06% | 1.36% | 1.92% | 98.08% |
| 06 | chirp-3Google | 1.45%1.01–1.94 | 1.01% | 1.94% | 98.55% | 0.78% | 5.40% | 1.44% | 2.18% | 97.82% |
| 07 | mai-transcribe-2Microsoft | 1.45%0.99–1.94 | 0.99% | 1.94% | 98.55% | 0.80% | 5.14% | 1.44% | 2.01% | 97.99% |
| 08 | universal-3-5-proAssemblyAI | 1.48%1.01–1.94 | 1.01% | 1.94% | 98.52% | 0.70% | 5.03% | 1.47% | 2.12% | 97.88% |
| 09 | muse-voice-transcribe-1.0Meta | 1.48%1.00–2.11 | 1.00% | 2.11% | 98.52% | 0.75% | 5.03% | 1.47% | 2.09% | 97.91% |
| 10 | gpt-transcribeOpenAI | 1.59%1.07–2.15 | 1.07% | 2.15% | 98.41% | 0.91% | 5.37% | 1.59% | 2.09% | 97.91% |
| 11 | grok-stt-1.0xAI | 1.65%1.19–2.20 | 1.19% | 2.20% | 98.35% | 0.83% | 5.26% | 1.64% | 2.51% | 97.49% |
| 12 | gpt-4o-mini-transcribeOpenAI | 1.82%1.22–2.42 | 1.22% | 2.42% | 98.18% | 1.06% | 5.46% | 1.81% | 2.52% | 97.48% |
| 13 | voxtral-mini-transcribeMistral AI | 1.96%1.42–2.52 | 1.42% | 2.52% | 98.04% | 1.07% | 5.86% | 1.95% | 2.66% | 97.34% |
| 14 | parakeet-tdt-0.6b-v3NVIDIA | 1.99%1.45–2.55 | 1.45% | 2.55% | 98.01% | 1.03% | 5.46% | 1.97% | 2.84% | 97.16% |
| 15 | nova-3Deepgram | 2.06%1.55–2.64 | 1.55% | 2.64% | 97.94% | 1.19% | 5.83% | 2.06% | 2.84% | 97.16% |
| 16 | whisper-1OpenAI | 2.13%1.55–2.76 | 1.55% | 2.76% | 97.87% | 1.22% | 5.72% | 2.12% | 2.85% | 97.15% |
| 17 | voxtral-mini-3b-2507Mistral AI | 2.24%1.56–3.04 | 1.56% | 3.04% | 97.76% | 1.43% | 6.09% | 2.23% | 2.97% | 97.03% |
| 18 | voxtral-small-24b-2507-sttMistral AI | 2.27%1.60–3.08 | 1.60% | 3.08% | 97.73% | 1.32% | 6.09% | 2.27% | 3.11% | 96.89% |
| 19 | whisper-large-v3-turboOpenAI | 2.33%1.76–3.04 | 1.76% | 3.04% | 97.67% | 1.42% | 5.89% | 2.32% | 3.30% | 96.70% |
| 20 | whisper-large-v3OpenAI | 2.35%1.75–3.08 | 1.75% | 3.08% | 97.65% | 1.49% | 6.26% | 2.35% | 3.19% | 96.81% |
| 21 | nemotron-3.5-asr-streaming-multilingual-0.6bNVIDIA | 3.12%2.31–4.08 | 2.31% | 4.08% | 96.88% | 1.59% | 6.83% | 3.11% | 4.48% | 95.52% |
| 22 | gpt-4o-transcribeOpenAI | 3.94%2.64–5.75 | 2.64% | 5.75% | 96.06% | 2.92% | 7.63% | 3.93% | 4.64% | 95.36% |
Blue marks the best value in each column. Open a model for all 29 metrics and where it places on each.Showing 22 of 22
Best on each axis
The winners, one question at a time.
- Most accurate
- 0.91%Word Error Rate (WER)qwen3-asr-flash-2026-02-10Qwen
- Lowest character error
- 0.50%Character Error Rate (CER)qwen3-asr-flash-2026-02-10Qwen
- Most exact matches
- 78.0%Exact Match Rateqwen3-asr-flash-2026-02-10Qwen
- Most robust
- 4.08%P90 WERqwen3-asr-flash-2026-02-10Qwen
- Fewest dropped words
- 0.34%Deletion Ratemai-transcribe-1.5Microsoft
- Fewest invented words
- 0.17%Insertion Ratevoxtral-small-24b-2507-sttMistral AI
- Lowest price
- $0.20Price per 1,000 minnemotron-3.5-asr-streaming-multilingual-0.6bNVIDIA · tied with qwen3-asr-0.6b
- Fastest
- 5.8 sMedian Latencyqwen3-asr-1.7bQwen
Glossary
What every column means.
Rates are pooled across all clips unless the name says mean, median or P90. Every metric is computed after the same normalization on both the reference and the model output.
Accuracy
Word- and character-level agreement with the reference, after normalization — and without it.
- Word Error Rate (WER)% · lower is better
- Headline metric. (substituted + deleted + inserted words) / words in the ground truth, after normalization. Pooled across all clips.
- WER 95% CI (low)% · lower is better
- Lower bound of the 95% bootstrap confidence interval for WER.
- WER 95% CI (high)% · lower is better
- Upper bound of the 95% bootstrap confidence interval for WER.
- Word Accuracy% · higher is better
- 100 − WER (floored at 0).
- Character Error Rate (CER)% · lower is better
- Same as WER but counted on characters; more forgiving of near-miss spellings.
- Strict WER% · lower is better
- WER with minimal normalization (only lowercase and punctuation removal). Shows how closely raw output matches the ground-truth style.
- Match Error Rate (MER)% · lower is better
- Errors / (errors + correct words). Bounded at 100% even with heavy hallucination.
- Word Information Lost (WIL)% · lower is better
- Share of word-level information lost between ground truth and output.
- Word Information Preserved (WIP)% · higher is better
- 100 − WIL.
Error breakdown
What kind of mistake a model makes: the wrong word, a dropped word, or an invented one.
- Substitution Rate% · lower is better
- Wrong words, as a share of ground-truth words.
- Deletion Rate% · lower is better
- Missed words (omissions), as a share of ground-truth words.
- Insertion Rate% · lower is better
- Extra words not spoken (fabrications / hallucinations), as a share of ground-truth words.
- Sentence Error Rate (SER)% · lower is better
- Share of clips with at least one word error.
- Exact Match Rate% · higher is better
- Share of clips transcribed perfectly after normalization.
- Length Ratiox · closer to 1 is better
- Output word count / ground-truth word count. Above 1 suggests hallucination, below 1 suggests skipped speech.
Robustness
How the error is spread across clips — the typical one, the hardest tenth, the outright failures.
- Mean WER% · lower is better
- Average of per-clip WER (every clip weighted equally).
- Median WER% · lower is better
- WER of the typical clip.
- P90 WER% · lower is better
- WER on the hardest 10% of clips; measures robustness.
- Severe Error Rate% · lower is better
- Share of clips with WER of 50% or more.
- Empty Output Rate% · lower is better
- Share of clips where the model returned no words.
- Success Rate% · higher is better
- Share of requests that returned a transcript.
By clip length
WER split by clip duration. Short clips give a model the least context to recover from.
- WER - short audio (under 5 s)% · lower is better
- WER on short clips.
- WER - medium audio (5 to 15 s)% · lower is better
- WER on medium-length clips.
- WER - long audio (over 15 s)% · lower is better
- WER on long clips.
Cost & speed
Billed cost and request time. Latency was measured under concurrent load and is indicative only.
- Price per 1,000 minUSD · lower is better
- Effective cost to transcribe 1,000 minutes of audio, from actual billing.
- Total CostUSD · lower is better
- Actual billed cost for the full benchmark run.
- Median Latencys · lower is better
- Median request time. Measured under concurrent load; indicative only.
- P95 Latencys · lower is better
- 95th percentile request time. Measured under concurrent load; indicative only.
- Real-Time Factor (median)x · lower is better
- Processing time / audio duration. Measured under concurrent load; indicative only.
Want your model on this board?
We run it on the same audio, through the same normalization, and publish it next to the other 22.