Whisper
OpenAI's open-source speech recognition model
Highlights
- Strong multilingual ASR baseline
- Language ID and translation
- Rich derived ecosystem
Use cases
- Automatic video subtitles
- Meeting and podcast transcription
- Multilingual ASR services
Review
Whisper is the multilingual speech recognition model open-sourced by OpenAI in 2022, trained on 680,000 hours of multilingual data. It remains a strong baseline for open-source ASR, with robustness to accents, noise, and reverberation that exceeds most peers from the same era.
Core capabilities include multilingual speech-to-text, language identification, spoken translation (X to English), timestamp alignment, and model sizes from tiny to large. The community has spawned faster-whisper (CTranslate2 acceleration), whisper.cpp (C++ inference), and whisperX (word-level timestamps and speaker diarization).
The highlight is that model quality and language coverage remain competitive in open source, with a rich ecosystem of derived projects. The downside is that the official repo updates slowly (no release in two years), long audio needs manual segmentation to avoid hallucinations, and punctuation and timestamps are unstable on fast models.
Use cases: video subtitle generation, meeting and podcast transcription, low-cost multilingual ASR services. For word-level precision or speaker diarization, go straight to whisperX.