Self-hosted alternatives to llama.cpp
200 self-hosted apps like llama.cpp, closest match first.
Top picks
- T

TurboLLM
LaunchRun open AI models on your own machine — no cloud, no account, fully offline. Any GGUF model on any llama.cpp-compatible engine, auto-tuned to your hardware, with real measured tokens/sec before you download, a fast browser UI, and an OpenA
FreeSelf-hosted
- L

LLMKube
LaunchLLMKube is a Kubernetes operator for running language models on your own hardware. You write a Model resource and an InferenceService resource in YAML, and the operator serves the model behind an OpenAI-compatible endpoint once it is ready.
FreeSelf-hosted
- P
Picollm
LaunchProduction-grade on-device AI SDKs for voice, language, and vision. STT, wake word, TTS, translation, OCR, and more. Runs on mobile, web, desktop, and embedded.
FreeSelf-hosted
- S
Sipp
LaunchAI inference, packed simply. A blazing-fast, zero-dependency WebGPU runtime to run GGUF models directly in the browser. Features a symmetric API for seamless local execution and cloud provider routing. Built with Rust & C++.
FreeSelf-hosted
- W
Willow Inference Server
LaunchWillow Inference Server (WIS) is an open-source server that runs speech and language models on your own hardware. It handles speech recognition, text-to-speech and language model inference, and it accepts audio over WebRTC, REST and WebSock
FreeSelf-hosted
- L
Llmaker
LaunchModels, vector DBs, embeddings, a retrieval agent, and evals: the whole modern LLM stack on your own hardware, from one command. OpenAI-compatible API + web UI per instance, GPU partitioning, no data leaving your machine.
FreeSelf-hosted
More like llama.cpp

Inference Gateway
An open-source, cloud-native, high-performance gateway unifying multiple LLM…
Self-Hosted AI Stack
Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document…
OmniKey-AI-Unified-Key-Manager
A self-hosted LLM proxy and failover gateway supporting OpenAI & Gemini…
About llama.cpp
llama.cpp
C/C++ inference engine for large language models, supporting quantization
C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
- Category
- AI
- Website
- llama.app



