NVIDIA TensorRT-LLM

by NVIDIA

NVIDIA's optimized inference for Whisper, Canary, Parakeet on Triton.

TL;DR

NVIDIA's optimized inference for Whisper, Canary, Parakeet on Triton.

Best for squeezing the last 30% of throughput out of NVIDIA GPUs for ASR. Pricing: free.

Category
Open source
License
—
Stars
—
Last push
—
Pricing
free
Platforms
Linux, Docker

What it is

The premier NVIDIA inference engine; produces engine files via TRT compilation. Apache-2.0.

Best for: Squeezing the last 30% of throughput out of NVIDIA GPUs for ASR.
Watch out for: NVIDIA-only; high engineering overhead; license is Apache-2.0 but requires NGC.

Install / use

git clone https://github.com/NVIDIA/TensorRT-LLM

Features

Speaker diarizationNo
Word-level timestampsYes
Streaming / real-timeNo
Languages supported60
HIPAA eligibleNo

NVIDIA TensorRT-LLM vs Whipscribe

FeatureNVIDIA TensorRT-LLMWhipscribe
CategoryOpen sourceTranscription APIs
Pricingfreefree beta
Speaker diarizationNoYes
Word timestampsYesYes
StreamingNoNo
Languages6099
PlatformsLinux, DockerWeb, API, MCP

Alternatives to NVIDIA TensorRT-LLM

OpenAI Whisper
OpenAI

The reference open-source multilingual ASR model from OpenAI.

OSS · MIT ★ 98.1k
whisper.cpp
Georgi Gerganov

C/C++ port of Whisper — runs on anything, from a Raspberry Pi to Apple Silicon.

OSS · MIT ★ 48.8k
faster-whisper
SYSTRAN

4× faster than reference Whisper using CTranslate2 — production sweet spot.

OSS · MIT ★ 22.3k

Whipscribe is a managed faster-whisper + whisperX service. If you want transcripts without running infrastructure, paste a URL or drop a file in the form below — you'll have a transcript in seconds.