inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Top inference alternatives

transformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, f whisper.cppPort of OpenAI's Whisper model in C/C++voiceboxThe open-source AI voice studio. Clone, dictate, create.HandyA free, open source, and extensible speech-to-text application that works completely offline.meetilyPrivacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built llamafileDistribute and run LLMs with a single file.

See the full list of inference alternatives →