How Accurate Is Whisper in 2026?

Whisper Accuracy at a Glance

Our October 2026 benchmark: 2,475 samples

We ran three Whisper deployments over three openly licensed speech corpora on 3 October 2026 and scored every utterance individually. Each figure below is the unweighted mean of the three deployments, so it describes Whisper rather than any one host. Sample sizes are given per row — several are small, and we would rather you see that than not.

By language, against the published figures

FLEURS is read speech, recorded cleanly, in 102 languages. We tested 11 and compare them here with the figures in OpenAI's own paper (arXiv:2212.04356, §D.2.4), which describe Whisper large-v2 — the same model our Replicate deployment ran, so this is a like-for-like comparison rather than a generational one.

Eight of the ten languages sit within about one point of the published figures. We read that as a check on our setup rather than a result in itself — if our numbers had come out uniformly better or worse, the methodology would be the first suspect.

Real-world speech: Common Voice

Common Voice is read aloud by volunteers on whatever microphone they own, in whatever room they happen to be in. It is the closest openly licensed proxy for the audio people actually upload, and it is where the comfortable numbers stop.

The ordering changes, not just the magnitude. German actually improves — 2.5% here against 5.6% on FLEURS — while Arabic goes from 9.6% to 22.5% and Turkish from 9.3% to 21.7%. Any single ranking of Whisper's best and worst languages is therefore wrong somewhere, because it depends which recording conditions you have in mind. Pick the table that matches your audio.

What phone-quality audio costs you

Telephone audio is band-limited and quantized, which strips exactly the high-frequency detail that distinguishes consonants. To measure the cost we took the Common Voice Turkish set and passed it through a simulated phone channel, so the only variable that changed was the channel.

About 7 points of extra error from the channel alone. The transform is a downsample to 8 kHz, a 300–3400 Hz bandpass, a G.711 μ-law round trip and an upsample back to 16 kHz — the full chain is in the methodology below so you can reproduce it. If you transcribe call recordings, budget for this and do not plan against a studio figure.

Same model, very different speed

Accuracy is a property of the model; speed is mostly a property of whoever runs it. Across the same requests, the three deployments differed by more than four times at the median.

These are per-request times on short utterances, so read them as a comparison between hosts rather than as a guide to how long your hour-long recording will take. The useful takeaway is that running Whisper is not one performance profile: the fastest and slowest deployments here differ by 4.4× at the median while scoring within 2.9 points of each other on the same audio.

Whisper Is a Family of Models, Not One Model

Whisper releases come in 7 sizes from Tiny (39M params) to Large-v3 (1.55B params). Speed and accuracy trade off significantly. Most commercial products use Large-v3 or Large-v3 Turbo (a distilled version released September 2024) for production. If you're specifically deciding between Large-v3 and Turbo, our dedicated Whisper Large-v3 vs Turbo comparison has per-dataset WER, speed benchmarks by hardware, cost math, and a decision flowchart.

Speed multipliers are relative to Large-v3, not real-time. Specific WER per model size requires reading the OpenAI Whisper paper appendix; we don't reproduce unverified per-size numbers here. Source: openai/whisper README.

Why your result will differ from the benchmark

Benchmark corpora are recorded under conditions almost nobody works in: one speaker, a good microphone, a quiet room, scripted text. Every step away from that costs accuracy. These are the conditions we measured on 3 October 2026, easiest first, with the same three Whisper deployments throughout.

Three conditions we did not isolate, and will not put a number on. Strong accents, heavy background noise and overlapping speakers all make transcription worse, and we have no controlled measurement separating them from each other in our data. Published research documents the accent effect in particular — see the accent-disparity work in JASA Express Letters (2024) — but a figure from someone else's study does not transfer cleanly to this pipeline, and inventing one would put us back where the withdrawn benchmark started. If your audio has any of these, treat the Common Voice row as a floor rather than an estimate.

Overlapping speech is a structural limit rather than a quality problem: Whisper transcribes a single stream and does not separate speakers, so crosstalk is not something a better model setting fixes. That needs diarization, which is a separate model — see Whisper diarization.

Why Whisper Sometimes Makes Things Up

A peer-reviewed study presented at ACM FAccT 2024 documented that Whisper occasionally fabricates content during silences and audio with frequent pauses. Researchers have reported hallucination rates from 1% to 80% of segments depending on conditions. The problem is most pronounced with:

• ●Long silences at segment boundaries (greater than 30 seconds)
• ●Audio with frequent pauses or speech disfluencies
• ●Recordings starting or ending with silence
• ●Background noise that resembles speech

This is not unique to Whisper — most automatic speech recognition models have this issue — but Whisper's tendency was specifically flagged in healthcare contexts (Healthcare Brew, November 2024) where transcript fabrication has serious consequences. For production use, always treat AI transcripts as drafts, not records of truth.

Sources: ACM FAccT 2024; Cornell coverage (June 2024); Calm-Whisper paper.

Correction: our July 2026 benchmark has been withdrawn

Open ASR Leaderboard: 8-Benchmark Composite

The Hugging Face Open ASR Leaderboard publishes standardized WER for open and commercial models across a fixed set of English benchmarks (LibriSpeech clean/other, TED-LIUM 3, AMI, GigaSpeech, Earnings22, CallHome, CommonVoice). The composite average is the aggregate WER across all 8. Whisper Large-v3 currently sits at ~7.44% composite; NVIDIA Parakeet-TDT 0.6B v2 leads the English-only leaderboard at ~6.05%.

Composite averages are aggregate WER across the 8 benchmark datasets on the Hugging Face Open ASR Leaderboard. Verified 2026-08-19. Numbers drift as vendors submit updated models. CallHome is included in the leaderboard composite but not shown per-column here — see the leaderboard for that dataset.

Whisper Accuracy Across 99 Languages

These are OpenAI's figures, not ours. We measured ten of these languages ourselves in October 2026 and those results are earlier on this page; where the two differ, prefer ours for those ten, because they are current and their sample sizes are stated. The table below is here for the sixty-odd languages we did not test, where the paper remains the best public source. Whisper's training data is heavily English-weighted; performance scales roughly with how much training data exists per language. The table below reports FLEURS benchmark WER for Whisper Large-v2, extracted directly from §D.2.4, Appendix D of the OpenAI Whisper paper (arXiv:2212.04356). Whisper Large-v3 (released November 2023) reduces errors by 10–20% across most of these languages per OpenAI's release note.

Notes. Whisper Large-v2 FLEURS numbers from §D.2.4, Appendix D of the OpenAI paper. Character Error Rate (CER) is used for Chinese, Japanese, and Korean per the paper's convention (word segmentation is not well-defined in these scripts). Tiers reflect production-suitability: Tier 1 (<8%) is production-ready, Tier 2 (8–20%) needs a human review pass for professional output, Tier 3 (>20%) needs domain fine-tuning to be usable.

How Whisper Compares to Commercial APIs

Vendor-published and independent WER figures are typically produced on different datasets, with different text normalization, model settings, and aggregation methods. Compare absolute WER values only when the methodologies are sufficiently similar; across published studies, absolute numbers for the same engine commonly vary by 2–3 percentage points.

The takeaway: Whisper, Deepgram, AssemblyAI, and OpenAI's GPT-4o-transcribe are all in the same general accuracy class for English. Mistral's Voxtral (released July 2025, Transcribe 2 in February 2026) is the new price-performance leader on FLEURS at $0.003/min. Differences become more pronounced for non-English languages, custom vocabulary, and specific audio conditions. For a vendor-by-vendor comparison with verified pricing, see our developer transcription API comparison (12 APIs) and the podcast transcription tools comparison (10 podcaster-focused tools).

What Whisper Doesn't Do

Whisper is a transcription model. Real production transcription tools need several things Whisper doesn't natively provide:

• Speaker diarization
  Whisper transcribes all speech but doesn't identify who said what. Pair with pyannote-audio (open source) or WhisperX (combines both) for speaker labels. Commercial tools built on Whisper — including VexaScribe — bundle diarization.
• Live streaming
  Whisper is batch-oriented. For real-time transcription, look at faster-whisper (CTranslate2 reimplementation) or commercial streaming APIs (Deepgram, Speechmatics, AssemblyAI Universal-Streaming).
• Custom vocabulary / domain terms
  Whisper has limited support for biasing toward specific terms (brand names, technical jargon). Commercial APIs like Deepgram (keyterm prompting, up to 100 terms), AssemblyAI (promptable models that accept domain context alongside the audio), and Google offer better custom vocabulary handling.
• Voice activity detection (VAD)
  Whisper struggles with empty audio (contributes to the hallucination problem). Most production setups pre-process audio with a VAD model to skip silence.
• Q&A interface over the transcript
  Whisper produces text — it doesn't let you query that text. Commercial tools layer this on top: VexaScribe's AI Chat lets users ask natural-language questions about a transcript and get answers with timestamps validated against the source text. Useful when an accurate transcript is the foundation but not the end goal.

Tools and Frameworks Built on Whisper

Whisper underpins many products and open-source projects. If you're evaluating which Whisper-based tool to use:

Should You Run Whisper Yourself?

If you're transcribing more than ~50 hours/month and you're comfortable with ML infra, self-hosting can be cheaper. For everyone else, a managed service is simpler.

Middle-ground option. If you don't want GPU infra but per-minute API pricing is unpredictable at your volume, predictable-cost bulk transcription delivers Whisper Large-v3 output at a flat monthly rate for up to 6,000 minutes — cheaper than self-hosting at that scale, more predictable than per-minute APIs.

Methodology & Sources

October 2026 benchmark — how it was run

Run on 3 October 2026. Three Whisper deployments, each over the same audio, scored per utterance:

• •Replicate openai/whisper, model parameter large-v2, version e39e3547…
• •thomasmol/cog-whisper-diarization on Replicate, version 744c4f2b… — Whisper with a pyannote diarization wrapper
• •Groq whisper-large-v3-turbo

Figures on this page are the unweighted mean of the three per-deployment means, so a dataset where one host had a few failed requests does not get weighted differently from the others. That choice makes the numbers describe Whisper rather than any single host. Per-deployment figures are given where they differ in a way that matters — see the latency table above.

Datasets and licences

Every corpus here is openly licensed and redistributable. We publish our own outputs and scores; the audio itself is downloaded from each dataset's own source under its licence, which is also why the sample sizes are modest rather than exhaustive.

LibriSpeech is distributed under CC BY 4.0; see Panayotov et al., 2015. FLEURS is distributed under CC BY 4.0; see Conneau et al., 2022. Mozilla Common Voice v17.0 is released under CC0 1.0; see Ardila et al., 2020. Total: 2,475 utterance-level records across the three deployments.

Scoring and normalisation

WER and CER were computed with jiwer. Reference and hypothesis were both lowercased, stripped of punctuation, and had repeated whitespace collapsed before comparison — so casing and punctuation differences are not counted as errors. An empty prediction scores 1.0 rather than being dropped, which matters because silently discarding failures flatters the average. No custom text normalisation, no number or date rewriting, no per-language exceptions.

The telephony transform

The phone-quality subset is Common Voice Turkish passed through a simulated G.711 channel in Python (NumPy and SciPy, not FFmpeg). Four steps, in order:

1. Peak-normalise to 0.95, mono
2. Resample to 8 kHz (scipy.signal.resample_poly)
3. 4th-order Butterworth bandpass, 300–3400 Hz, zero-phase (sosfiltfilt)
4. G.711 μ-law encode then decode (μ = 255), adding the quantisation a phone network would
5. Resample back to 16 kHz and peak-normalise again, because the models expect 16 kHz input

This is a simulation of a phone channel, not a recording of a real call: it reproduces the bandwidth limit and the μ-law quantisation, but not packet loss, jitter, codec transcoding or handset noise. Real call-centre audio is likely to be worse than our 28.8%, not better. We describe the subset as telephony-simulated for that reason.

What we did not publish, and why

• •A Japanese WER. We tested 11 FLEURS languages and publish 10. Word error rate splits text on spaces to build the word lists it compares, so on a language written without spaces an utterance becomes one token and any difference scores 100% — all three deployments returned 98–100% WER on Japanese while character error rate stayed at 7–9%. OpenAI's own paper reports CER rather than WER for Japanese, Chinese and Korean for the same reason. Japanese is excluded from every WER figure on this page; its average CER was 8.0% (n=58).
• •Results for non-Whisper models. We tested several commercial engines in the same run. This page is about Whisper, and we do not publish benchmark results for other companies' products. The Open ASR Leaderboard is the neutral place to compare engines, and several dedicated commercial models currently score ahead of Whisper on it.
• •Diarization error rates. Our diarization sample was three meetings. That is too small to publish a number from.
• •Per-hour costs. Host pricing changes and we have not re-checked it against vendor pricing pages on the date of writing, so quoting it here would be a claim we cannot stand behind.

How we define Word Error Rate (WER)

WER is the sum of word substitutions, insertions, and deletions in a hypothesis transcript divided by the total number of words in the reference. Lower is better. Reported as a percentage. Where WER figures are cited on this page, they come from published benchmarks and vendor documentation, each noted with its source.

Our July 2026 benchmark — withdrawn

This section previously described the methodology for a first-party benchmark of 14 speech-to-text models. Those results were withdrawn on August 25, 2026 following an internal review — see the correction above for what was wrong and why. The methodology description has been removed along with the results rather than left standing behind data we no longer publish. Third-party figures elsewhere on this page are cited to their original sources and are unaffected.

Primary sources cited

• •Robust Speech Recognition via Large-Scale Weak Supervision (Radford et al., OpenAI Whisper paper, arXiv:2212.04356) — §D.2.4, Appendix D for FLEURS Large-v2 numbers.
• •Hugging Face Open ASR Leaderboard — 8-benchmark composite English WER, verified 2026-08-19.
• •openai/whisper README — model sizes, parameter counts, and VRAM requirements.
• •OpenAI Whisper announcement and release notes for Large-v3 (November 2023) and Large-v3 Turbo (September 2024).
• •ACM FAccT 2024 hallucination study; Cornell coverage (June 2024); Calm-Whisper silence-handling improvements paper.
• •Xu et al., arXiv:1708.08615 — professional human transcription WER on Switchboard and CallHome benchmarks.
• •LibriSpeech, AMI Meeting Corpus, TED-LIUM 3, GigaSpeech, Earnings21, VoxConverse, FLEURS (Google), CommonVoice (Mozilla), Multilingual LibriSpeech — dataset licenses vary; check per-dataset documentation before commercial republication.

Verification & update window

Our own benchmark figures — every WER, CER and latency number in the benchmark sections above — were measured on 2026-10-03 and published on 2026-10-05. They replace the first-party benchmark withdrawn on 2026-08-25, which measured a different set of models with a methodology we no longer stand behind; the withdrawal notice above is kept rather than deleted. Open ASR Leaderboard composite figures were pulled 2026-08-19 and drift as vendors submit updated models — they are third-party public data, not ours. FLEURS per-language figures attributed to OpenAI come from §D.2.4, Appendix D of the Whisper paper (arXiv:2212.04356) and describe Large-v2. Vendor pricing referenced elsewhere on this page was checked 2026-08-19 and is older than everything else here; verify it against the vendor before relying on it. Corrections are shipped in-line and dated as we find them.

Independence and third-party names

VexaScribe (formerly NovaScribe) is an independent commercial transcription product and is not affiliated with, sponsored by, or endorsed by any of the vendors discussed here. Company, product, and model names are used solely to identify the services described and remain the property of their respective owners. Some vendors discussed compete with VexaScribe.

We run two tiers. Standard is the default and uses OpenAI Whisper Large-v3 across 99 languages at the normal minute rate. Premium is opt-in, costs two minutes of quota per minute of audio, and uses a different premium model across the 55 languages it covers; where a language is outside that set, the tier selector falls back to Standard. We do not sell a transcription API and do not resell any vendor discussed in this document.

Corrections. If you identify a reproducible error in anything published here, contact us with the specific claim and supporting evidence and we will investigate and publish a dated correction where warranted. Editorial standards at /about/editorial-standards.