Definition

What Is a Speech Recognition Model?

VideoText editorial team. Updated September 13, 2026.

Direct definition

A speech recognition model is the trained statistical or neural system that maps audio features to tokens or words. Examples include the Whisper family of checkpoints. The model is not the same thing as the product UI around it.

The sections below explain how speech recognition model is used in transcription and subtitle work, what people often mix up, and which VideoText workflow applies when the next step is a file or review pass.

Key takeaways

  • Model cards and papers report WER on named sets.
  • “Powered by Whisper” is not a WER number.
  • A model name is not a WER result until the dataset and decoding settings are named.

How models are compared

Fix the dataset, the decoding settings, and the text normalizer. Then compare WER. Changing any of those three invalidates a head-to-head claim.

What the model is doing

A modern recogniser maps an audio signal to text in one learned step, rather than chaining separate acoustic, pronunciation and language components as older systems did. It is trained on large amounts of paired audio and text.

Its output reflects what it was trained on. Accents, domains and recording conditions well represented in training are transcribed well; those that are not show noticeably higher error rates, which is why a single headline accuracy figure describes a benchmark rather than your files.

Comparing models honestly

A number quoted without its test set means little. The same model can report very different error rates on clean read speech, on telephone audio, and on spontaneous multi-speaker conversation.

When comparing, hold the audio constant and state the conditions. If the comparison is between a figure measured on clean studio audio and one measured on real meeting recordings, it is not a comparison at all.

Frequently asked questions

Does a bigger model always transcribe better?
Usually but not always, and the gain narrows on clean audio while costing more processing time. The difference is largest on difficult material.
Why does the same model vary between files?
Audio conditions dominate. Microphone distance, background noise, overlapping speech and accent all move the error rate more than model choice does.
Can a model be tuned to my vocabulary?
Some systems accept a hint list of terms or names, which materially helps with domain jargon the model has rarely seen.
Does a custom vocabulary retrain the model?
Usually not. Most systems accept a hint list that biases recognition toward supplied terms for that job, which is far cheaper than retraining and effective for names and jargon.

Related terms

Sources

  1. OpenAI: Robust Speech Recognition via Large-Scale Weak Supervision

Related VideoText workflow