Word-level timestamps from Whisper: every method compared
Every interactive-transcript, caption and clipping feature rests on one primitive: knowing when each word was spoken. Whisper gives you words easily and timing reluctantly. These are the five practical ways to get per-word timestamps, from zero-extra-work to research-grade, and what each one is actually good for.
The methods, ranked by effort
1. Whisper's native segment timestamps
Free with every transcription: segment-level start/end predicted as tokens. Good enough for paragraph navigation and rough SRT files. Not per-word, and drift after silences is well documented.
2. faster-whisper's word_timestamps=True
faster-whisper (CTranslate2 engine) exposes per-word timing derived from attention alignments and token probabilities — one flag, no extra model. Accuracy is respectable on clean speech; it inherits Whisper's drift on hard audio. The pragmatic default for self-hosters.
3. whisper.cpp token-level timing
whisper.cpp emits token times with experimental word-boundary handling. On-device and dependency-free, same caveats as attention-derived timing everywhere.
4. WhisperX forced alignment
The accuracy ceiling: a per-language phoneme model re-times every word against the audio. Tens-of-milliseconds precision, the standard for karaoke captions and research. Costs a second model, per-language support, and pipeline glue — the full mechanism is in our WhisperX & forced alignment guide.
5. A hosted API that does it for you
Whipscribe's API returns word-level timestamps, speaker labels and caption formats from one upload — the alignment machinery runs server-side. The right answer when timing is a feature of your product rather than your research topic.
Accuracy expectations, honestly
| Method | Typical word-timing error | Cost |
|---|---|---|
| Native segments | ±0.5–2 s at boundaries | none |
| faster-whisper words | usually <200 ms, worse after silence | one flag |
| whisper.cpp tokens | similar, experimental | one flag |
| WhisperX alignment | tens of ms on supported languages | second model + glue |
| Hosted (Whipscribe) | alignment-grade, no setup | per-hour pricing |
Numbers are engineering rules of thumb, not benchmarks — your audio's noise, accents and silences dominate. The ranking, however, is stable.
Which one should you use?
- Rough captions, notes, search: native segments. Stop there.
- Self-hosted app with decent timing: faster-whisper's flag.
- On-device (mobile/desktop): whisper.cpp.
- Karaoke captions, clipping, dubbing, research: WhisperX.
- Shipping a product feature this week: hosted API.