How Accurate Is Deepgram? Nova-3 Benchmarks, Independently Checked

Deepgram claims a 5.26% median WER for Nova-3 batch transcription and 6.84% for streaming (launch post, February 2025). Independent measurements, including Artificial Analysis's WER Index, put Nova-3 at roughly 7–10% on diverse real-world audio. Both numbers are real: the vendor figure is a median across vendor-selected test domains, while independent indexes apply one text normalization to every engine across harder, more varied audio. On specific benchmarks Nova-3 scores 2.6% on LibriSpeech test-clean, 13.4% on AMI meetings, and 21.8% on CallHome conversational phone audio.

On standard English benchmarks, Deepgram Nova-3 leads or ties Whisper Large-v3 on all eight Open ASR Leaderboard test sets — with its biggest margins on hard audio: AMI meetings (13.4% vs 15.9%), Earnings-22 financial calls (10.2% vs 12.3%), and CallHome phone audio (21.8% vs 26.4%). However, peer-reviewed testing on other real-world audio (arXiv 2408.16287) found Whisper and AssemblyAI slightly ahead on raw accuracy. The honest summary: they are within 1–3 percentage points of each other; Deepgram is decisively faster, Whisper is free to self-host and covers 99+ languages.

They trade places depending on the test. On the Open ASR Leaderboard composite, Nova-3 edges Universal-2 on most datasets by 0.2–1.6 percentage points. In the arXiv 2408.16287 evaluation, AssemblyAI ranked among the most accurate engines while Deepgram ranked fastest. AssemblyAI's newer Universal-3 Pro (February 2026) measured 2.3% WER on Artificial Analysis's AgentTalk benchmark. Practical rule: for maximum accuracy on recorded audio, AssemblyAI's newest model has the edge; for streaming latency and cost per minute, Deepgram wins.

By Deepgram's own published numbers: streaming WER dropped from 8.4% (Nova-2) to 6.84% (Nova-3) — an 18.6% relative improvement — and batch median WER reached 5.26%. Nova-3 also added keyterm prompting (up to 100 custom terms per request) and expanded multilingual support through 2025–2026. Many products integrated before 2025 still call Nova-2, so if a tool 'powered by Deepgram' seems less accurate than these numbers, check which model generation it actually uses.

Phone audio is the hardest common scenario for every STT engine. Nova-3 scores 21.8% WER on the CallHome conversational telephone benchmark — the best result among major engines (Whisper Large-v3: 26.4%, AssemblyAI Universal-2: 23.4%), but still roughly 1 word in 5 wrong. The 8 kHz bandwidth, crosstalk, and casual speech of phone audio triple typical WER. Deepgram's strongest relative margins are on telephony, which is why call-center and voice-agent platforms disproportionately build on it.

Yes, for the vocabulary it targets. Keyterm prompting (introduced with Nova-3) lets you pass up to 100 domain-specific terms — product names, people, jargon — per request, and the model biases recognition toward them. It doesn't change headline WER on generic benchmarks, but on jargon-dense audio it prevents exactly the errors that matter most in practice: misrendered names, drug names, and technical terms. Whisper has no equivalent feature; this is one of the strongest practical reasons to choose a commercial API over self-hosted open source.

Three mechanical reasons: (1) dataset selection — vendors benchmark on domains their model was tuned for; (2) median vs pooled reporting — a median across test sets hides the worst domains, and phone audio can run 3–4× the median; (3) text normalization — how punctuation, casing, and numerals are handled before scoring can swing WER by 1–3 points. Independent indexes like Artificial Analysis and the Hugging Face Open ASR Leaderboard apply identical normalization to every engine, which makes their absolute numbers higher but their comparisons fairer.

Fastest among major engines — this is Deepgram's one claim that independent testing confirms without qualification. The arXiv 2408.16287 evaluation found Deepgram the most efficient system when speed and accuracy are considered together, and its streaming mode targets sub-300 ms latency, which is why it dominates voice-agent infrastructure. At $0.0043/min for Nova-3 pay-as-you-go it is also the cheapest major commercial API per minute (AssemblyAI ~$0.006/min, Google ~$0.016/min, AWS ~$0.024/min).