How Accurate Is Deepgram? Nova-3 Benchmarks, Independently Checked

Deepgram claims a 5.26% median WER for Nova-3 batch transcription and 6.84% for streaming (launch post, February 2025). Independent measurements, including Artificial Analysis's WER Index, put Nova-3 at roughly 7–10% on diverse real-world audio. Both numbers are real: the vendor figure is a median across vendor-selected test domains, while independent indexes apply one text normalization to every engine across harder, more varied audio. On specific benchmarks Nova-3 scores 2.6% on LibriSpeech test-clean, 13.4% on AMI meetings, and 21.8% on CallHome conversational phone audio.

On standard English benchmarks, Deepgram Nova-3 leads or ties Whisper Large-v3 on all eight Open ASR Leaderboard test sets — with its biggest margins on hard audio: AMI meetings (13.4% vs 15.9%), Earnings-22 financial calls (10.2% vs 12.3%), and CallHome phone audio (21.8% vs 26.4%). However, peer-reviewed testing on other real-world audio (arXiv 2408.16287) found Whisper and AssemblyAI slightly ahead on raw accuracy. The honest summary: they are within 1–3 percentage points of each other; Deepgram is decisively faster, Whisper is free to self-host and covers 99+ languages.

They trade places depending on the test. On the Open ASR Leaderboard composite, Nova-3 edges Universal-2 on most datasets by 0.2–1.6 percentage points. In the arXiv 2408.16287 evaluation, AssemblyAI ranked among the most accurate engines while Deepgram ranked fastest. AssemblyAI's current flagship Universal-3.5 Pro (July 2026, successor to the now-deprecated Universal-3 Pro) measured 7.0% aggregate WER in Novascribe's July 2026 benchmark of 14 models — rank 3 overall (behind Speechmatics Melia-1 at 6.4% and Enhanced at 6.9%) and comfortably ahead of Deepgram Nova-3 English (8.9% aggregate). Practical rule: for maximum accuracy on batch recorded audio, Speechmatics Melia-1 leads and AssemblyAI Universal-3.5 Pro is second; for streaming latency and cost per minute on English audio, Deepgram Nova-3 wins.

By Deepgram's own published numbers: streaming WER dropped from 8.4% (Nova-2) to 6.84% (Nova-3) — an 18.6% relative improvement — and batch median WER reached 5.26%. Nova-3 also added keyterm prompting (up to 100 custom terms per request) and expanded multilingual support through 2025–2026. Many products integrated before 2025 still call Nova-2, so if a tool 'powered by Deepgram' seems less accurate than these numbers, check which model generation it actually uses.

Phone audio is the hardest common scenario for every STT engine. Nova-3 scores 21.8% WER on the CallHome conversational telephone benchmark — the best result among major engines (Whisper Large-v3: 26.4%, AssemblyAI Universal-2: 23.4%), but still roughly 1 word in 5 wrong. The 8 kHz bandwidth, crosstalk, and casual speech of phone audio triple typical WER. Deepgram's strongest relative margins are on telephony, which is why call-center and voice-agent platforms disproportionately build on it.

Yes, for the vocabulary it targets. Keyterm prompting (introduced with Nova-3) lets you pass up to 100 domain-specific terms — product names, people, jargon — per request, and the model biases recognition toward them. It doesn't change headline WER on generic benchmarks, but on jargon-dense audio it prevents exactly the errors that matter most in practice: misrendered names, drug names, and technical terms. Whisper has no equivalent feature; this is one of the strongest practical reasons to choose a commercial API over self-hosted open source.

Three mechanical reasons: (1) dataset selection — vendors benchmark on domains their model was tuned for; (2) median vs pooled reporting — a median across test sets hides the worst domains, and phone audio can run 3–4× the median; (3) text normalization — how punctuation, casing, and numerals are handled before scoring can swing WER by 1–3 points. Independent indexes like Artificial Analysis and the Hugging Face Open ASR Leaderboard apply identical normalization to every engine, which makes their absolute numbers higher but their comparisons fairer.

Fastest among major engines — this is Deepgram's one claim that independent testing confirms without qualification. The arXiv 2408.16287 evaluation found Deepgram the most efficient system when speed and accuracy are considered together, and its streaming mode targets sub-300 ms latency, which is why it dominates voice-agent infrastructure. At $0.0043/min for Nova-3 pay-as-you-go it is also the cheapest major commercial API per minute (AssemblyAI ~$0.006/min, Google ~$0.016/min, AWS ~$0.024/min). Novascribe's July 2026 benchmark measured Nova-2 at 2.3 seconds per short LibriSpeech clip and Nova-3 at 4.4 seconds — competitive with GPT-4o (1.2s) but faster than AssemblyAI Universal-3.5 (5.5s). Deepgram's hosted Whisper Large is the exception: 20.4 seconds per short clip, 168 seconds per AMI meeting — an order of magnitude slower than every alternative.

Neither if accuracy is your top priority — use Speechmatics Melia-1, AssemblyAI Universal-3.5, or OpenAI Whisper-1 instead. Novascribe's July 2026 benchmark measured Nova-3 English at 8.0% WER on German FLEURS (vs Universal-3.5's 2.6% and Melia-1's 2.2%) and Nova-3 Multilingual at 6.3% (still ~3× worse than Speechmatics Melia-1's 2.2%). Nova-3 Multilingual improved cost but not accuracy: 8.2% aggregate multilingual average vs Universal-3.5's 4.9% and Melia-1's 4.6%. Italian and Portuguese are the two languages where Nova-3 English beat Nova-3 Multilingual on clean FLEURS audio (Italian: 2.8% vs 3.6%; Portuguese: 9.3% vs 10.7%). One anomaly: Nova-3 English also surprisingly won accented CommonVoice Portuguese at 6.2% WER — worth testing if your Portuguese audio is heavily accented Brazilian.

No, not on real-world audio. Deepgram publishes 5.26% median batch WER for Nova-3 from its own launch benchmark on vendor-selected test domains. In Novascribe's July 2026 benchmark of 14 speech-to-text models across 6 English datasets and 904 audio files, Nova-3 English measured 12.3% average English WER — 2.3× the marketing number. This gap is not unique to Deepgram: it's structural. Vendors report medians on their tuning distribution; independent benchmarks average across a broader, harder audio mix with uniform normalization. Both numbers are technically true; the 12.3% is what you should expect on mixed real-world audio.

Speechmatics Melia-1 wins accuracy and price; Deepgram wins streaming. In Novascribe's July 2026 benchmark: Melia-1 aggregate WER 6.4% (rank 1 of 14) vs Deepgram Nova-3 English 8.9% (rank 10) and Nova-3 Multilingual 8.2% (rank 6). English AVG: Melia-1 10.5% vs Nova-3 English 12.3%. Multilingual: Melia-1 4.6% vs Nova-3 Multilingual 8.2%. Price: Melia-1 $0.24/hr vs Deepgram Nova-3 $0.45/hr (batch). Where Deepgram wins: real-time streaming API with sub-300ms first-token latency (Speechmatics does not offer public streaming), keyterm prompting (up to 100 domain terms), and marginally better AMI meeting diarization (38.2% vs 39.3%). Rule of thumb: Speechmatics for batch accuracy + cost, Deepgram for voice agents.

Deepgram claims Nova-3 Medical is '63.7% better' on medical terms than generic Nova-3, but the vendor number is a relative reduction in domain-specific errors and does not translate directly to a lower headline WER. No independent benchmark has published Nova-3 Medical numbers to date, and Novascribe's July 2026 benchmark did not include Nova-3 Medical (no medical audio datasets in the 16-dataset composite). For actual medical dictation production use, do not rely on the '63.7% better' marketing claim — run your own evaluation on 30+ minutes of your specific specialty audio, comparing Nova-3 Medical head-to-head with Nova-3 generic + keyterm prompting.

Less frequently, but yes on some audio. Whisper is documented to invent plausible-sounding text during silence and to get stuck repeating tokens (arXiv:2402.08021 measured hallucinations in ~1.4% of transcribed segments). Deepgram Nova-3 uses a purpose-built streaming ASR architecture (not encoder-decoder like Whisper) which structurally reduces silence-hallucination and repetition-loop failure modes. It's not immune — any neural ASR can substitute words on unclear audio — but the specific 'inventing content during silence' pattern is much rarer on Nova-3. For safety-critical use (medical, legal), always keep human review regardless of engine.