Quick answer
Automatic Speech Recognition (ASR) is the technology that converts spoken audio into written text. Modern ASR systems use deep neural networks — most commonly Whisper, Conformer, or wav2vec 2.0 — that take raw audio, extract features, and predict text directly. The best 2026 systems reach 2-5% word error rate on clean English audio. ASR is also called speech-to-text (STT); the two terms mean the same thing.
Definition (expanded)
Automatic Speech Recognition (ASR) is the computational process that maps an acoustic speech signal to a symbolic text representation. In practical terms: audio in, text out. The audio can be a live microphone stream (dictation, voice assistants, live captions) or a recorded file (podcast, meeting recording, interview). The output is a sequence of words — often with additional structure like timestamps, speaker labels, or confidence scores depending on the specific tool.
ASR is also called computer speech recognition or speech-to-text (STT). All three terms describe the same task. Industry conventions differ: academic and technical documentation tends to use "ASR," while product marketing and consumer-facing sites tend to use "speech-to-text" or "voice-to-text." The underlying technology is identical.
A common confusion: ASR is not the same as voice recognition. Voice recognition identifies WHO is speaking (biometric speaker identification). ASR identifies WHAT was said. Different tasks, different models. See the disambiguation table below.
ASR vs speech-to-text vs voice recognition — disambiguation
Six terms get confused with ASR. Here's what each one actually means.
| Term | What it means | Example query | Relation to ASR |
|---|---|---|---|
| ASR (Automatic Speech Recognition) | Technology that converts spoken audio to written text. | "transcribe this podcast" | Same as STT — different name preference by industry / academia. |
| STT (Speech-to-Text) | Same as ASR. Product-marketing name. | "speech to text app" | Consumer / product terminology. |
| Voice recognition | Identifying WHO is speaking (biometric speaker identification). | "is this Alice's voice?" | Distinct from ASR — different task, different models. |
| Dictation | Live user-initiated ASR into a text field or document. | "dictating an email" | A specific consumer use case of ASR. |
| Voice search | ASR + intent understanding + search retrieval. | "OK Google, what's the weather?" | ASR is one stage in the pipeline. |
| Speech synthesis / TTS | Text-to-Speech — opposite direction. Generates spoken audio from written text. | "read this article aloud" | Opposite of ASR — often confused. |
The rest of this page covers ASR / STT. For speaker identification, see /what-is-speaker-diarization. For voice search intent understanding, that's NLU (natural language understanding), a separate pipeline stage.
How ASR works — the pipeline
Classical ASR pipelines had six discrete stages. Modern end-to-end neural models collapse several into a single network, but the conceptual stages still map to what's happening inside.
1. Audio capture
Microphone input, typically sampled at 16 kHz mono for ASR (higher rates offer no accuracy benefit above human speech frequency range).
2. Feature extraction
Raw waveform gets converted to a compact representation — historically MFCCs (Mel-Frequency Cepstral Coefficients), now often mel-spectrograms or log-mel features. Neural models can also learn features directly from raw audio (wav2vec 2.0 approach).
3. Acoustic model
Maps features to phonetic units, subwords, or characters. Historically HMM+GMM; from 2012 onward, deep neural networks (DNN/HMM hybrids, then fully end-to-end).
4. Language model
Assigns probability to word sequences. N-gram statistical LMs historically; neural LMs (transformer-based) in modern systems. Often fused during decoding to favor plausible word sequences over acoustic-only best guesses.
5. Decoder
Finds the most likely word sequence given acoustic + language model scores. Beam search is standard. In end-to-end models with attention (Whisper), decoding is autoregressive — one token at a time.
6. Post-processing
Punctuation, capitalization, number formatting, sometimes speaker diarization (a separate model), sometimes filler-word removal. In production tools, this is where "raw transcript" becomes "readable document."
End-to-end models (Whisper, wav2vec 2.0) collapse stages 3-5 into a single neural network trained jointly on audio + text pairs. The pipeline concept still describes what the network is learning to do.
Architecture families
Six architecture approaches, listed chronologically. Modern systems usually pick one; some hybrids combine two.
HMM + GMM (historical)
Hidden Markov Models with Gaussian Mixture Model emissions. Dominated 1980s-2010s. Foundation of most consumer dictation before deep learning (Dragon NaturallySpeaking, phone IVR systems, early Siri). Still deployed in some low-resource / edge scenarios.
Hybrid DNN/HMM
2012 breakthrough (Hinton et al.). Replaced GMM emissions with deep neural networks while keeping HMM decoding structure. Halved WER on standard benchmarks. Still deployed in enterprise systems.
End-to-end CTC (Connectionist Temporal Classification)
Graves et al. 2006. Monotonic alignment — output tokens strictly follow input frames. DeepSpeech / DeepSpeech2 use CTC. Simpler than attention, faster training, but weaker at long-range dependencies.
End-to-end Attention (encoder-decoder)
Non-monotonic — decoder attends over all encoder outputs. Whisper uses this. Highest accuracy on complex audio (accents, code-switching) but not natively streaming — best for offline / batch transcription.
RNN-T (Recurrent Neural Network Transducer)
Graves 2012. Streaming-friendly — produces output token-by-token as audio arrives. Google's production choice for on-device Assistant and Live Caption. NVIDIA Parakeet is a modern RNN-T.
Hybrid CTC / Attention
Combines CTC and attention losses during training. Best of both — competitive accuracy with faster convergence. Common in recent research papers; several ESPnet models use this.
Major ASR models in 2026
The 2026 ASR landscape has consolidated around a handful of model families. Below is a vendor-neutral profile of each.
Whisper (OpenAI, 2022)
Encoder-decoder transformer. Trained on 680,000 hours of weakly-supervised multilingual audio. Open-source, MIT license. Supports 99 languages.
Variants: tiny / base / small / medium / large-v3 / large-v3-turbo. Best for: offline batch transcription, multilingual audio, robust to accents. Not for: tight real-time streaming (chunked processing works but has latency).
wav2vec 2.0 (Meta AI, 2020)
Self-supervised pre-training on unlabeled audio, then supervised fine-tuning. Strong at low-resource languages. Wins LibriSpeech test-clean at 1.8% WER (with LM).
Best for: data-efficient fine-tuning to new languages or domains. Note: multilingual variant (XLS-R) covers 128 languages.
Conformer (Google, 2020)
Convolution-augmented transformer. Combines local receptive fields (CNN) with global attention. Streaming-friendly.
Best for: production streaming ASR. Current status: Conformer encoders paired with LLM decoders now lead English accuracy leaderboards.
Parakeet (NVIDIA, 2024)
RNN-T architecture, efficient for real-time on-device inference. Runs on Apple Silicon via MLX.
Best for: low-latency streaming, on-device dictation.
Apple SpeechAnalyzer (iOS 26, 2026)
Apple's native on-device ASR. Three modules: SpeechTranscriber (long-form), DictationTranscriber (short utterances), SpeechDetector (voice activity).
Best for: iOS / macOS apps that want zero-download local ASR. 2026 benchmark note: competitive with Whisper large-v3 turbo per independent Apple Silicon tests.
DeepSpeech2 (Baidu, 2015) & Kaldi (2011)
Historical / infrastructure. DeepSpeech2 was a milestone for end-to-end ASR. Kaldi is the leading open-source HMM+DNN toolkit — still widely used in academic and enterprise research.
Best for: education, low-resource-language research, systems that predate the Whisper era.
WER benchmark table
Word Error Rate (WER) is the standard ASR evaluation metric: percentage of words in a reference transcript that are substituted, deleted, or inserted in the predicted transcript. Lower is better. Below is a vendor-neutral cross-source benchmark table.
| Model | LibriSpeech test-clean WER | LibriSpeech test-other WER | Common Voice WER | Note |
|---|---|---|---|---|
| Whisper large-v3 | 2.7% | 5.2% | 9.0% | OpenAI, 2023. Encoder-decoder transformer trained on 680k hours. |
| wav2vec 2.0 (large + LM) | 1.8% | 3.3% | 16.1% | Meta, 2020. Self-supervised pre-training. Leads LibriSpeech test-clean. |
| Conformer + LLM decoder | leads English tables | — | — | Google, 2020. Convolution-augmented transformer. Production-favored. |
| Parakeet (NVIDIA) | — | — | — | NVIDIA, 2024. RNN-T, efficient real-time on-device. |
| DeepSpeech2 (baseline) | ~5.3% | ~13.3% | — | Baidu, 2015. Historical significance — RNN CTC. |
How to read WER: a 5% WER means roughly 1 error per 20 words — usable for meeting notes and podcast subtitles, weak for legal or medical transcripts (which typically demand human review to reach 99%+). 15% WER means the transcript needs meaningful cleanup before use. Benchmarks vary significantly by test setup — the numbers above reflect the widely-cited primary-source values.
History timeline: 1952 → 2026
Speech recognition has been an active research area for 70+ years. Below are the milestones that changed what ASR could do.
First speech recognizer. Recognized spoken digits 0-9 for a single trained speaker.
Demonstrated at Seattle World's Fair. Understood 16 English words + digits.
Recognized 1,011 words. Milestone for continuous speech.
Voice-activated typewriter. 20,000-word vocabulary.
Hidden Markov Models with Gaussian Mixture Models dominate. Consumer dictation software matures.
Consumer ASR goes mainstream via voice assistant.
Deep neural networks replace GMMs in acoustic modeling. Large WER drops across benchmarks.
First high-profile end-to-end ASR paper.
End-to-end CTC becomes practical for production.
Far-field ASR at consumer scale.
Self-supervised pre-training paradigm. Strong low-resource fine-tuning.
Convolution-augmented transformer. Streaming-friendly.
680,000 hours training, 99 languages, open-source. GPT-3-scale ASR moment.
On-device Whisper via Apple Silicon Neural Engine. Efficient RNN-T alternatives.
Apple's native on-device ASR competitive with Whisper large-v3 turbo per independent benchmarks.
Real-world examples
ASR is everywhere in 2026. A partial list of where you interact with it.
Siri, Alexa, Google Assistant, Bixby. ASR turns your spoken query into text, which then goes to NLU + retrieval.
Zoom, Teams, Google Meet, YouTube, iOS Live Captions. Real-time ASR for accessibility + note-taking.
macOS Dictation, Windows Voice Access, Google Docs voice typing, Dragon. Live speech → text field.
Otter, Fireflies, VexaScribe, Zoom AI Companion. Full meeting transcripts with speaker labels + summaries.
"Press 1 or say billing." ASR + limited-vocabulary constraints for reliable phone-tree navigation.
Live captions for deaf and hard-of-hearing users, voice control for motor accessibility.
Nuance Dragon Medical, DAX, Suki. Clinician voice → structured chart notes.
Automatic transcript + SRT generation for episodes and video content.
Duolingo, Speak, Rosetta Stone pronunciation feedback. ASR compares learner audio to reference.
Court records, depositions, legal interviews. ASR + human review to reach the 99%+ accuracy legal work requires.
Honest limits of modern ASR
Modern ASR is much better than a decade ago but still fails predictably in specific conditions. Knowing where it breaks helps you plan for review or pick the right tool.
- Accented speech — non-native English speakers typically see a 10-15% WER penalty vs native speakers on general-purpose models. Fine-tuning on accented data narrows the gap.
- Code-switching — mixing two languages mid-sentence (common in bilingual communities) confuses most models. Whisper handles it better than most.
- Technical vocabulary — medical, legal, engineering, or brand-specific terms trigger errors. Custom vocabulary lists or fine-tuning help.
- Low-resource languages — languages with under ~100 hours of transcribed training data have significantly weaker models. Whisper covers 99, but quality varies widely across the tail.
- Overlapping speakers — ASR handles who-said-what poorly. Diarization is a separate model that runs on the audio + ASR output.
- Streaming vs batch tradeoff — real-time systems sacrifice roughly 1-2 percentage points of WER vs their batch equivalents.
- Noise + poor audio quality — background noise, distance from microphone, phone-quality audio (8 kHz sample rate) all reduce accuracy substantially.
- Homophone ambiguity — "to / too / two," "their / there / they're" — context helps but errors persist.
What's next for ASR
Whisper large-v4 expected. Meta SeamlessM4T merges speech translation + recognition in one model.
Apple Silicon Neural Engine + iOS 26 SpeechAnalyzer + WhisperKit + Parakeet + MLX are all pushing best-in-class ASR to consumer devices with zero cloud dependency.
Models that stream low-latency while maintaining long-context understanding — currently a tradeoff.
Speech-native large language models (Google Gemini speech, GPT-4o audio) blur the line between ASR and general audio understanding.
Single models that output speaker-labeled text natively, instead of running ASR + diarization as separate stages.
Verified sources
Every model claim, WER number, and historical date on this page was cross-checked on 2026-08-16 against the following primary sources:
- Whisper (Radford et al., 2022) — arXiv:2212.04356. Source of truth for 680,000 hours training data, 99 languages, LibriSpeech WER numbers.
- wav2vec 2.0 (Baevski et al., 2020) — arXiv:2006.11477. Source of self-supervised pre-training numbers and LibriSpeech test-clean 1.8% WER.
- Conformer (Gulati et al., 2020) — arXiv:2005.08100. Convolution-augmented transformer architecture.
- CTC (Graves et al., 2006) — original Connectionist Temporal Classification paper.
- RNN-T (Graves, 2012) — original transducer paper.
- Hinton et al., 2012 — IEEE Signal Processing Magazine, DNN/HMM breakthrough.
- Rabiner ASR history — UCSB Rabiner ECE 259 course reprints, comprehensive history reference.
- Computer History Museum — "Audrey, Alexa, Hal, and More" blog post on 1952 Bell Labs Audrey.
- Wikipedia — "Timeline of speech and voice recognition" cross-reference.
- LibriSpeech — openslr.org dataset for benchmark comparisons.
- Common Voice — commonvoice.mozilla.org multilingual benchmark.
- Apple developer docs — iOS 26 SpeechAnalyzer / SpeechTranscriber / DictationTranscriber APIs.
- Argmax WhisperKit — GitHub argmaxinc/WhisperKit for Apple Silicon on-device Whisper.
- IBM, NVIDIA, AssemblyAI, HuggingFace — authority explainer cross-references for consistency.
Frequently asked questions
What does "automatic speech recognition" mean?
Automatic speech recognition (ASR) is the technology that converts spoken audio into written text. Modern ASR systems use deep neural networks to take raw audio, extract features, and predict text directly. ASR is also called speech-to-text (STT) or computer speech recognition — the three terms describe the same task. It powers voice assistants, live captions, meeting transcription, dictation, and phone IVR systems.
What are the best automatic speech recognition models?
In 2026 the leading ASR models are: Whisper (OpenAI, 2022) — 680,000 hours training, 99 languages, best general-purpose open-source model; wav2vec 2.0 (Meta, 2020) — leads LibriSpeech test-clean at 1.8% WER, strong at low-resource fine-tuning; Conformer + LLM decoder (Google, 2020) — currently leads English accuracy leaderboards, streaming-friendly; Parakeet (NVIDIA, 2024) — efficient RNN-T for real-time on-device; Apple SpeechAnalyzer (iOS 26, 2026) — Apple's native on-device model, competitive with Whisper large-v3 turbo per 2026 benchmarks. Choice depends on use case: Whisper for multilingual offline batch, wav2vec 2.0 for low-resource fine-tuning, Conformer for streaming production, Parakeet or SpeechAnalyzer for on-device.
How accurate is automatic speech recognition?
The best 2026 ASR systems reach 2-5% word error rate on clean English audio: Whisper large-v3 achieves 2.7% WER on LibriSpeech test-clean, wav2vec 2.0 (with language model) reaches 1.8% on the same benchmark. Accuracy degrades predictably: accented speech typically sees a 10-15% WER penalty vs native speakers, technical vocabulary (medical, legal, engineering) adds another 5-10%, noisy audio drops accuracy to 80-90%, and phone-quality audio (8 kHz sample rate) commonly measures 75-85%. Reading WER: 5% = 1 error per 20 words (usable for meeting notes); 15% = meaningful cleanup needed.
What is an example of automatic speech recognition?
ASR powers dozens of common products: voice assistants (Siri, Alexa, Google Assistant, Bixby), live captions (Zoom, Teams, Google Meet, YouTube), dictation (macOS Dictation, Windows Voice Access, Google Docs voice typing), meeting transcription (Otter, Fireflies, VexaScribe), phone IVR ("press 1 or say billing"), accessibility tools for deaf and hard-of-hearing users, medical dictation (Nuance Dragon Medical, DAX), and language-learning apps with pronunciation feedback (Duolingo, Speak). Every time you say "OK Google" or use a live-captions feature, ASR runs on your audio.
What's the difference between ASR and speech-to-text?
No functional difference. ASR (Automatic Speech Recognition) is the term academic and technical documentation prefers; STT (speech-to-text) is what product marketing and consumer sites use. Both describe the same computational task: converting spoken audio into written text. If you see "ASR API" and "STT API" from the same vendor, they're the same product with different naming conventions for different audiences.
What's the difference between speech recognition and voice recognition?
Different tasks. Speech recognition (= ASR = STT) identifies WHAT was said — it converts audio to text. Voice recognition identifies WHO is speaking — it's biometric speaker identification. Speech recognition and voice recognition are often confused because both involve audio input, but they use different models, produce different outputs, and solve different problems. Voice recognition is what unlocks your phone with "Hey Siri." Speech recognition is what turns your dictation into text.
How does ASR handle multiple speakers?
Standard ASR does not separate speakers — it produces one transcript for whatever audio it hears. Separating speakers is a distinct task called speaker diarization, which runs as a separate model on top of or alongside ASR. Modern pipelines pair ASR (Whisper, Conformer) with diarization models (pyannote, NVIDIA NeMo) to produce speaker-labeled transcripts. Recent research explores speaker-aware ASR that outputs labeled text natively, but production systems typically use the two-stage approach.
Is ASR the same as natural language processing (NLP)?
No. ASR converts spoken audio into written text. NLP (Natural Language Processing) is the broader field of computationally understanding, generating, and manipulating human language — text is typically the input. In a voice assistant pipeline: ASR converts your speech to text → NLU (Natural Language Understanding, part of NLP) figures out your intent → retrieval or generation produces a response → TTS (Text-to-Speech) reads the response back to you. ASR is the entry point of the pipeline; NLP handles what happens after.
Can ASR work offline?
Yes, with modern on-device models. Apple's SpeechAnalyzer (iOS 26, macOS), WhisperKit-based apps on Apple Silicon Macs, Parakeet via MLX, and installed openai-whisper / faster-whisper on any machine all run 100% offline once the model weights are downloaded. Cloud APIs (Google Cloud STT, AWS Transcribe, OpenAI Whisper API, Deepgram, AssemblyAI) require internet. For confidential audio (legal, medical, HR), offline models are the safer default.
What are the main challenges in modern ASR?
Five recurring challenges even in 2026: (1) accented and dialectal speech — models trained primarily on native English underperform; (2) code-switching between languages mid-sentence; (3) technical vocabulary that wasn't well-represented in training data; (4) low-resource languages with less than 100 hours of training data; (5) noisy environments, overlapping speakers, and poor audio quality. Ongoing research addresses each: multilingual self-supervised pre-training (wav2vec XLS-R), custom vocabulary fine-tuning, better diarization for overlapping speech, and larger-scale weakly-supervised training (the Whisper approach).
Related guides
Deep-dive on Whisper WER benchmarks + real-world accuracy per audio type.
The "who is speaking" task — separate from ASR, often confused with voice recognition.
Practitioner path — install openai-whisper, faster-whisper, WhisperX, or whisper.cpp locally.
If you came here looking for a tool, the STT landing page has browser dictation and file upload.