How Accurate Is Deepgram? Nova-3 Benchmarks, Independently Checked

Deepgram claims a 5.26% median WER for Nova-3 batch transcription and 6.84% for streaming (launch post, February 2025). Independent measurements, including Artificial Analysis's WER Index, put Nova-3 at roughly 7–10% on diverse real-world audio. Both numbers are real: the vendor figure is a median across the vendor's own test domains, while independent indexes apply one text normalization to every engine across harder, more varied audio. On specific benchmarks Nova-3 scores 2.6% on LibriSpeech test-clean, 13.4% on AMI meetings, and 21.8% on CallHome conversational phone audio.

On standard English benchmarks, Deepgram Nova-3 leads or ties Whisper Large-v3 on all eight Open ASR Leaderboard test sets — with its biggest margins on hard audio: AMI meetings (13.4% vs 15.9%), Earnings-22 financial calls (10.2% vs 12.3%), and CallHome phone audio (21.8% vs 26.4%). However, peer-reviewed testing on other real-world audio (arXiv 2408.16287) found Whisper and AssemblyAI slightly ahead on raw accuracy. The honest summary: they are within 1–3 percentage points of each other; Deepgram is decisively faster, Whisper is free to self-host and covers 99+ languages.

They trade places depending on the test. On the Open ASR Leaderboard composite, Nova-3 edges Universal-2 on most datasets by 0.2–1.6 percentage points. In the arXiv 2408.16287 evaluation, AssemblyAI ranked among the most accurate engines while Deepgram ranked fastest. AssemblyAI's current flagship is Universal-3.5 Pro (July 2026, successor to the now-deprecated Universal-3 Pro). Practical rule: if maximum accuracy on batch recorded audio is your priority, evaluate AssemblyAI's flagship alongside Deepgram on your own audio; if streaming latency and cost per minute on English audio matter more, Deepgram Nova-3 is the stronger fit.

By Deepgram's own published numbers: streaming WER dropped from 8.4% (Nova-2) to 6.84% (Nova-3) — an 18.6% relative improvement — and batch median WER reached 5.26%. Nova-3 also added keyterm prompting (up to 100 custom terms per request) and expanded multilingual support through 2025–2026. Many products integrated before 2025 still call Nova-2, so if a tool 'powered by Deepgram' seems less accurate than these numbers, check which model generation it actually uses.

Phone audio is the hardest common scenario for every STT engine. Nova-3 scores 21.8% WER on the CallHome conversational telephone benchmark — the best result among major engines (Whisper Large-v3: 26.4%, AssemblyAI Universal-2: 23.4%), but still roughly 1 word in 5 wrong. The 8 kHz bandwidth, crosstalk, and casual speech of phone audio triple typical WER. Deepgram's strongest relative margins are on telephony, which is why call-center and voice-agent platforms disproportionately build on it.

Yes, for the vocabulary it targets. Keyterm prompting (introduced with Nova-3) lets you pass up to 100 domain-specific terms — product names, people, jargon — per request, and the model biases recognition toward them. It doesn't change headline WER on generic benchmarks, but on jargon-dense audio it prevents exactly the errors that matter most in practice: misrendered names, drug names, and technical terms. Whisper has no equivalent feature; this is one of the strongest practical reasons to choose a commercial API over self-hosted open source.

Three mechanical reasons: (1) dataset selection — vendors benchmark on domains their model was tuned for; (2) median vs pooled reporting — a median across test sets hides the worst domains, and phone audio can run 3–4× the median; (3) text normalization — how punctuation, casing, and numerals are handled before scoring can swing WER by 1–3 points. Independent indexes like Artificial Analysis and the Hugging Face Open ASR Leaderboard apply identical normalization to every engine, which makes their absolute numbers higher but their comparisons fairer.

Fastest among major engines — this is Deepgram's one claim that independent testing confirms without qualification. The arXiv 2408.16287 evaluation found Deepgram the most efficient system when speed and accuracy are considered together, and its streaming mode targets sub-300 ms latency, which is why it dominates voice-agent infrastructure. At $0.0043/min for Nova-3 pay-as-you-go it is also the cheapest major commercial API per minute (AssemblyAI ~$0.006/min, Google ~$0.016/min, AWS ~$0.024/min).

Test both on your own audio before committing. Deepgram released Nova-3 Multilingual to extend language coverage beyond the English-focused Nova-3 English model, and Deepgram's multilingual pricing is competitive. But independent multilingual results vary a great deal by language, by accent, and by recording condition — an engine that leads on clean read speech in one language can trail on accented or conversational audio in the same language, and rankings rarely transfer across languages. Because of that, published aggregate multilingual scores are weak predictors for any single language. The reliable approach is to transcribe a representative sample in each target language with both Nova-3 variants and with at least one alternative engine such as Speechmatics, AssemblyAI, or Whisper, then compare on your own audio.

Deepgram reports 5.26% median batch WER for Nova-3 on its own benchmark. Vendor-published WER figures are generally measured on the vendor's own dataset mix, with their own text normalization and aggregation method, so they are not directly comparable to figures published by other vendors or by independent evaluations. Independent evaluations of commercial engines on broader, harder audio mixes typically report higher error rates than any vendor's headline number — this is a structural property of how the figures are produced, not specific to Deepgram. For a decision that matters, run your own evaluation on 30+ minutes of audio representative of your actual use case.

They optimize for different things. On published list pricing, Speechmatics Melia-1 is the cheaper of the two for batch transcription at $0.24/hr versus Deepgram Nova-3 at $0.45/hr. Where Deepgram is distinct: it offers a public real-time streaming API targeting sub-300ms first-token latency (Speechmatics does not offer public streaming), plus keyterm prompting for up to 100 domain terms per request. Accuracy between the two depends heavily on the audio, so treat it as something to test rather than settle from a table. Rule of thumb: consider Speechmatics where batch cost is the deciding factor, and Deepgram where live streaming or voice-agent latency is required — then validate accuracy on a representative sample of your own audio.

Deepgram claims Nova-3 Medical is '63.7% better' on medical terms than generic Nova-3, but the vendor number is a relative reduction in domain-specific errors and does not translate directly to a lower headline WER. No independent benchmark has published Nova-3 Medical numbers to date. For actual medical dictation production use, treat the 63.7% figure as a domain-specific relative improvement rather than a headline accuracy number — run your own evaluation on 30+ minutes of your specific specialty audio, comparing Nova-3 Medical head-to-head with Nova-3 generic + keyterm prompting.

Less frequently, but yes on some audio. Whisper is documented to invent plausible-sounding text during silence and to get stuck repeating tokens (arXiv:2402.08021 measured hallucinations in ~1.4% of transcribed segments). Deepgram Nova-3 uses a purpose-built streaming ASR architecture (not encoder-decoder like Whisper) which structurally reduces silence-hallucination and repetition-loop failure modes. It's not immune — any neural ASR can substitute words on unclear audio — but the specific 'inventing content during silence' pattern is much rarer on Nova-3. For safety-critical use (medical, legal), always keep human review regardless of engine.