Open source alternatives to llama.cpp
200 open source apps like llama.cpp, closest match first.
Top picks
- W
whatcani.run
Launchwhatcani.run helps you find the best AI models to run locally and how to run them, based on real data from trials that people ran on their own hardware. Each model is listed with its runtime, decode and prefill speed, time to first token, p
Free
- M

mlx-serve
LaunchNative Zig inference server for Apple Silicon. Runs any LLM — MLX and GGUF — faster than LM Studio. OpenAI- and Anthropic-compatible (Claude Code) APIs. No Python. Drop-in alternative to LM Studio, Ollama, and llama.cpp on macOS. Gemma 4, Q
Free
- T

TurboLLM
LaunchRun open AI models on your own machine — no cloud, no account, fully offline. Any GGUF model on any llama.cpp-compatible engine, auto-tuned to your hardware, with real measured tokens/sec before you download, a fast browser UI, and an OpenA
FreeSelf-hosted
- V

vllm-mlx
Launchvllm-mlx is an inference server for large language models on Apple Silicon. It exposes OpenAI-compatible and Anthropic-compatible APIs, so existing clients and tools can talk to a model running on your Mac. The engine is built on MLX, Apple
Free
- B
BodhiApp
LaunchUnified AI gateway combining local LLM inference, cloud API proxying, and MCP tool integration with built-in chat UI and enterprise access control.
Free
- L
Llamastash
LaunchA fast terminal-native TUI and CLI for managing local LLMs, zero-overhead on llama.cpp with a pluggable backend seam. One binary, daemon on demand, OpenAI-compatible proxy, and agent-ready JSON output.
Free
More like llama.cpp
Willow Inference Server
Open source, local, and self-hosted highly optimized language inference…
Uncensored-Local-AI-Multiplatform
A cross-platform Flutter app that runs uncensored, 100% offline open-source…
About llama.cpp
llama.cpp
C/C++ inference engine for large language models, supporting quantization
C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
- Category
- AI
- Website
- llama.app

