Skip to content
llama.cpp

Open source alternatives to llama.cpp

200 open source apps like llama.cpp, closest match first.

Top picks

  1. W

    whatcani.run

    Launch

    whatcani.run helps you find the best AI models to run locally and how to run them, based on real data from trials that people ran on their own hardware. Each model is listed with its runtime, decode and prefill speed, time to first token, p

    Free

  2. M

    mlx-serve

    Launch

    Native Zig inference server for Apple Silicon. Runs any LLM — MLX and GGUF — faster than LM Studio. OpenAI- and Anthropic-compatible (Claude Code) APIs. No Python. Drop-in alternative to LM Studio, Ollama, and llama.cpp on macOS. Gemma 4, Q

    Free

  3. T

    TurboLLM

    Launch

    Run open AI models on your own machine — no cloud, no account, fully offline. Any GGUF model on any llama.cpp-compatible engine, auto-tuned to your hardware, with real measured tokens/sec before you download, a fast browser UI, and an OpenA

    FreeSelf-hosted

  4. V

    vllm-mlx

    Launch

    vllm-mlx is an inference server for large language models on Apple Silicon. It exposes OpenAI-compatible and Anthropic-compatible APIs, so existing clients and tools can talk to a model running on your Mac. The engine is built on MLX, Apple

    Free

  5. B

    BodhiApp

    Launch

    Unified AI gateway combining local LLM inference, cloud API proxying, and MCP tool integration with built-in chat UI and enterprise access control.

    Free

  6. L

    Llamastash

    Launch

    A fast terminal-native TUI and CLI for managing local LLMs, zero-overhead on llama.cpp with a pluggable backend seam. One binary, daemon on demand, OpenAI-compatible proxy, and agent-ready JSON output.

    Free

More like llama.cpp

7
M

MLX-LoRA-Studio

A native Mac App for LLM fine-tuning on Apple Silicon

Launch
8
P

Picollm

On-device LLM Inference Powered by X-Bit Quantization

Launch
9
X

Xybrid

Cross-platform on-device AI toolkit

Launch
10
W

Web LLM

High-performance In-browser LLM Inference Engine

Launch
11
R

Reins

Local AI chat for iPhone, iPad and Mac

Launch
12
T

ToshLLM

Run large language models locally on Intel Macs with AMD GPUs

Launch
13
M

MLX Tune

Fine-tune LLMs on your Mac with Apple Silicon

Launch
14
A

Audio.cpp

An all-in-one, pure C++ inference engine for audio models, powered by ggml

Launch
15
O

OfflineLLM

Private on-device AI chat for Android

Launch
16
S

Sipp

AI inference, packed simply

Launch
17
O

OlliteRT

Turn your Android phone into an OpenAI-compatible LLM inference server

Launch
18
L

liter-llm

Universal LLM API client — 163 providers, 14 native language bindings…

Launch
19
Q

Qwen3.6-windows-server

One-click Qwen3.6-27B inference on Windows

Launch
20
N

Nativ

Local AI, native to your Mac

Launch
21
V

vLLM

Inference and serving engine for large language models, built

Launch
22
W

Willow Inference Server

Open source, local, and self-hosted highly optimized language inference…

Launch
23
F

Fox

A local LLM server built for concurrent work

Launch
24
L

Llmaker

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Launch
25
U

Uncensored-Local-AI-Multiplatform

A cross-platform Flutter app that runs uncensored, 100% offline open-source…

Launch
26
T

TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models

Launch
27
L

LocalAI

Run LLMs, speech, image generation

Launch
28
T

TheWhisper

Optimized Whisper models for streaming and on-device use

Launch
29
L

Local Llms On Android

Run local LLMs like Gemma, Qwen, and LLaMA on Android for offline, private

Launch
30
E

ETOS-LLM-Studio

A native LLM client for iOS & Apple Watch

Launch
31
O

OpenGateLLM

An open-source, unified interface for running and managing self-hosted LLMs

Launch
32
O

Ollama

Load and run large LLMs locally to use in your terminal or build your apps

Launch
33
L

LlamaChat

Create AI models on Mac, have natural conversations, and gain real-time insights

Launch
34
W

WebLLM Chat

Chat with AI large language models running natively in your browser

Launch
35
C

Camelid

A Rust-native local inference backend

Launch
36
C

ChatLLM-Web

Private local model studio, AI chat

Launch
37
A

Aikit

Fine-tune, build, and deploy open-source LLMs easily!

Launch
38
J

Janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs…

Launch
39
T

TRAE-Ollama-Bridge

Encapsulate the local Ollama into an OpenAI-compatible API

Launch
40
O

Olla

High-performance lightweight proxy and load balancer for LLM infrastructure

Launch
41
H

HilbertRaum

AI, fully local on your computer

Launch
42
O

ollama

Get up and running with large language models, locally

Launch
43
P

pooled

Peer-to-peer LLM inference in the browser: pool your devices

Launch
44
S

SMG

Engine-agnostic LLM gateway in Rust

Launch
45
L

Llamafarm

Deploy any AI model, agent, database, RAG

Launch
46
I

Inference Gateway

An open-source, cloud-native, high-performance gateway unifying multiple LLM…

Launch
47
P

Paddler

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs

Launch
48
H

Hollama

A minimal LLM chat app that runs entirely in your browser

Launch
49
C

Chipper

AI interface for tinkerers (Ollama, Haystack RAG, Python)

Launch
50
W

Whisper.cpp

Port of OpenAI's Whisper model in C/C++

Launch

About llama.cpp

L

llama.cpp

C/C++ inference engine for large language models, supporting quantization

C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.

Category
AI
Website
llama.app