What Is ASR (Automatic Speech Recognition)?
ASR is the technology that converts spoken audio into text. It's the engine behind every transcription tool, voice assistant, live caption, and dictation feature. This page explains what ASR actually is, how it differs from related terms, how it works under the hood, and how accurate it really is.
Supported formats:
ASR vs Related Terms
These terms get confused constantly. The differences matter when you're picking a tool or designing a feature.
| Term | What it does |
|---|---|
| ASR (Automatic Speech Recognition) | Converts spoken audio into written text. The technical/research term. |
| Speech-to-text (STT) | Same thing as ASR. More common in consumer products. |
| Voice recognition | Often used casually to mean ASR, but technically refers to identifying who is speaking — not what they said. |
| Speaker identification / diarization | Identifies which person spoke each part of the audio. Often paired with ASR but is a separate technology. |
| NLU (Natural Language Understanding) | Extracts meaning, intent, or structure from text. Often runs after ASR (audio → text via ASR, text → meaning via NLU). |
The most common confusion: "voice recognition" in everyday speech means ASR, but in technical documentation it means speaker recognition (identifying a person by their voice). When in doubt, "ASR" or "speech-to-text" are unambiguous.
How ASR Works
Modern ASR turns audio into text in roughly four stages. Different systems vary in detail, but the high-level pipeline is the same.
The old approach (2000s–2015)
Earlier ASR systems used Hidden Markov Models (HMMs) combined with Gaussian Mixture Models (GMMs) for acoustic modeling, plus a separate statistical language model. The pipeline was modular and required hand-engineered features. Building one took significant linguistic expertise.
The modern approach (2017–today)
End-to-end neural models replaced the modular pipeline. A single transformer-based network maps audio features directly to text tokens. Examples:
- OpenAI Whisper — encoder-decoder transformer trained on 680,000 hours of multilingual audio
- Google Conformer — convolution-augmented transformer used in Google Cloud Speech-to-Text
- Deepgram Nova — proprietary architecture optimized for low-latency cloud inference
- AssemblyAI Universal-2 — multi-billion-parameter model focused on English accuracy
End-to-end models train on raw audio paired with transcripts. The model learns acoustic patterns, vocabulary, and language structure together — no separate language model required (though some systems still add one for specific domains).
What Affects ASR Accuracy
ASR accuracy varies more than vendor marketing suggests. The same model can hit 4% word error rate on clean audio and 30% on noisy multi-speaker recordings.
Hurts accuracy
- • Background noise (HVAC, traffic, music)
- • Heavy compression or low bitrate audio
- • Multiple overlapping speakers (cross-talk)
- • Strong accents (especially non-native speakers of the target language)
- • Domain-specific vocabulary (medical, legal, technical jargon)
- • Sub-resourced languages (anything outside the top ~30)
- • Short utterances without context
Helps accuracy
- • Single speaker, clean microphone
- • Recorded in a quiet space
- • Well-supported language (English, Spanish, Mandarin, French)
- • Native or standard-accented speaker
- • General/everyday vocabulary
- • Domain-fine-tuned models (e.g., medical-trained ASR)
- • Higher-quality audio (48kHz uncompressed, dedicated mic)
Modern ASR Systems Worth Knowing
The major systems product teams and developers choose between in 2026. None is universally best — strengths vary by use case, language, and budget.
OpenAI Whisper
Open-source, multilingual (99 languages), runs offline. Strong general-purpose accuracy. Latency higher than commercial APIs. Free to self-host; OpenAI also offers a paid API.
Deepgram Nova-3
Cloud API. Fast, low-latency, competitive pricing. Strong for streaming use cases (live captions, call centers). English-focused with growing multilingual coverage.
Google Cloud Speech-to-Text
Mature service with broadest language support (~125 languages). Well-integrated with Google Cloud. Pricing is per-15-second increments.
AssemblyAI Universal-2
Cloud API focused on highest English accuracy. Adds features like sentiment analysis, summarization, entity detection out of the box.
Amazon Transcribe
AWS-native, integrated with S3 and other AWS services. Good for teams already on AWS. Specialty editions (Medical, Call Analytics) available.
Speechmatics Ursa
UK-based provider. Strong reputation for handling accents and dialects. Supports 50+ languages with focus on real-world conversational audio.
Common Uses for ASR
- • Transcription — meetings, interviews, podcasts, lectures turned into searchable text
- • Captions and subtitles — accessibility for video content, SRT/VTT generation
- • Voice assistants — Siri, Alexa, Google Assistant rely on ASR as the first step
- • Voice dictation — typing by voice in documents, messages, code editors
- • Call center analytics — transcribing customer calls for QA, training, sentiment analysis
- • Live captioning — real-time captions in meetings, streams, events
- • Voice search — converting spoken queries into search terms
ASR Accuracy in Real Numbers
The standard metric is Word Error Rate (WER) — the percentage of words the model gets wrong (substitutions + deletions + insertions, divided by total reference words). Lower is better.
| Audio condition | Typical WER (modern systems) |
|---|---|
| Clean read-aloud audio, single speaker, well-supported language | 4–6% |
| Conversational audio, single speaker, good microphone | 8–12% |
| Meeting recordings, multiple speakers, decent audio | 10–20% |
| Phone calls or compressed audio | 15–25% |
| Heavily accented speech in target language | 15–30% |
| Noisy environments, overlapping speakers | 25–40%+ |
Public benchmarks like the Open ASR Leaderboard rank models on standard datasets (LibriSpeech, CommonVoice, etc.). Real-world performance on your specific audio almost always differs from benchmark scores — sometimes better, often worse.
ASR FAQ
What does ASR stand for?
ASR stands for Automatic Speech Recognition. It's the underlying technology that converts spoken audio into written text — the engine behind transcription tools, voice dictation, live captions, and voice assistants.
Is ASR the same as speech-to-text?
Yes, in practice they're the same thing. ASR is the older, more technical term used in research and engineering contexts. Speech-to-text (STT) is the more common consumer-facing term. Both describe the same process: turning audio of speech into a text transcript.
How accurate is ASR today?
On clean read-aloud audio in well-supported languages like English, modern ASR systems achieve word error rates (WER) around 4–6%. On real-world conversational audio — meetings, phone calls, podcasts with multiple speakers — WER typically lands between 10% and 20%. Heavy accents, background noise, technical vocabulary, or overlapping speech can push WER above 25%.
Does ASR work offline?
Some ASR models do. OpenAI's Whisper, for example, can run entirely on your local machine without internet. Cloud-based services like Google Cloud Speech-to-Text, AWS Transcribe, and Deepgram require an internet connection because the model runs on their servers. Offline ASR is typically slower and slightly less accurate than cloud equivalents, but offers privacy and works without connectivity.
Which languages does ASR support?
Coverage varies sharply by system. OpenAI Whisper supports 99 languages with varying quality. Google Cloud Speech-to-Text covers around 125. Smaller commercial systems often focus on 30–50 languages. Quality is best for languages with abundant training data — English, Spanish, Mandarin, German, French — and degrades for low-resource languages.
Can ASR identify who is speaking?
Not by itself. ASR identifies what was said. Identifying who said what is a separate task called speaker diarization. Many transcription products combine ASR with diarization so the final transcript shows speaker labels, but the two are technically different technologies.
Looking for an ASR-powered tool you can actually use?
VexaScribe runs modern ASR models (including Whisper) so you don't have to manage infrastructure. Upload audio or video, get a transcript with timestamps and speaker labels.
See how VexaScribe handles transcription →Related Reading
How Accurate Is Whisper?
Open ASR Leaderboard data, real-world WER numbers, and how Whisper compares to commercial systems.
Whisper Transcription
Use OpenAI Whisper without the setup — VexaScribe runs it for you.
AI Transcription
How AI transcription works in production — accuracy, speaker labels, formats.
Speaker Identification
ASR identifies words; speaker diarization identifies who said them. See how it works.