Skip to content
llama.cpp

Free alternatives to llama.cpp

200 free apps like llama.cpp, closest match first.

Top picks

  1. W

    whatcani.run

    Launch

    whatcani.run helps you find the best AI models to run locally and how to run them, based on real data from trials that people ran on their own hardware. Each model is listed with its runtime, decode and prefill speed, time to first token, p

    Free

  2. M

    mlx-serve

    Launch

    Native Zig inference server for Apple Silicon. Runs any LLM — MLX and GGUF — faster than LM Studio. OpenAI- and Anthropic-compatible (Claude Code) APIs. No Python. Drop-in alternative to LM Studio, Ollama, and llama.cpp on macOS. Gemma 4, Q

    Free

  3. T

    TurboLLM

    Launch

    Run open AI models on your own machine — no cloud, no account, fully offline. Any GGUF model on any llama.cpp-compatible engine, auto-tuned to your hardware, with real measured tokens/sec before you download, a fast browser UI, and an OpenA

    FreeSelf-hosted

  4. L

    LLMKube

    Launch

    LLMKube is a Kubernetes operator for running language models on your own hardware. You write a Model resource and an InferenceService resource in YAML, and the operator serves the model behind an OpenAI-compatible endpoint once it is ready.

    FreeSelf-hosted

  5. Q

    Quail

    Launch

    Local model server for GGUF and MLX models on Apple silicon, with OpenAI- and Anthropic-compatible APIs for tools like Claude Code and Codex. [![Open-Source Software][OSS Icon]](https://github.com/adatoo/quail) ![Freeware][Freeware Icon] ![

    Free

  6. V

    vllm-mlx

    Launch

    vllm-mlx is an inference server for large language models on Apple Silicon. It exposes OpenAI-compatible and Anthropic-compatible APIs, so existing clients and tools can talk to a model running on your Mac. The engine is built on MLX, Apple

    Free

More like llama.cpp

7
B

BodhiApp

Run Open Source/Open Weight LLMs locally with OpenAI compatible APIs

Launch
8
H

huggingface-local-models

Select models to run locally

Get
9
L

Llamastash

Zero-overhead, terminal-native local-LLM runtime manager

Launch
10
M

MLX-LoRA-Studio

A native Mac App for LLM fine-tuning on Apple Silicon

Launch
11
P

Picollm

On-device LLM Inference Powered by X-Bit Quantization

Launch
12
X

Xybrid

Cross-platform on-device AI toolkit

Launch
13
W

Web LLM

High-performance In-browser LLM Inference Engine

Launch
14
R

Reins

Local AI chat for iPhone, iPad and Mac

Launch
15
T

ToshLLM

Run large language models locally on Intel Macs with AMD GPUs

Launch
16
M

MLX Tune

Fine-tune LLMs on your Mac with Apple Silicon

Launch
17
A

Audio.cpp

An all-in-one, pure C++ inference engine for audio models, powered by ggml

Launch
18
O

OfflineLLM

Private on-device AI chat for Android

Launch
19
S

Sipp

AI inference, packed simply

Launch
20
O

OlliteRT

Turn your Android phone into an OpenAI-compatible LLM inference server

Launch
21
L

liter-llm

Universal LLM API client — 163 providers, 14 native language bindings…

Launch
22
Q

Qwen3.6-windows-server

One-click Qwen3.6-27B inference on Windows

Launch
23
N

Nativ

Local AI, native to your Mac

Launch
24
W

Willow Inference Server

Open source, local, and self-hosted highly optimized language inference…

Launch
25
F

Fox

A local LLM server built for concurrent work

Launch
26
L

Llmaker

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Launch
27
U

Uncensored-Local-AI-Multiplatform

A cross-platform Flutter app that runs uncensored, 100% offline open-source…

Launch
28
T

TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models

Launch
29
L

LocalAI

Run LLMs, speech, image generation

Launch
30
A

AgentOp

Run a real LLM in your browser. No install, no server

Launch
31
T

TheWhisper

Optimized Whisper models for streaming and on-device use

Launch
32
L

Local Llms On Android

Run local LLMs like Gemma, Qwen, and LLaMA on Android for offline, private

Launch
33
E

ETOS-LLM-Studio

A native LLM client for iOS & Apple Watch

Launch
34
L

LocalAI

Unlock the power of AI with LocalAI, the ultimate AI model management tool

Launch
35
O

OpenGateLLM

An open-source, unified interface for running and managing self-hosted LLMs

Launch
36
L

LlamaChat

Create AI models on Mac, have natural conversations, and gain real-time insights

Launch
37
W

WebLLM Chat

Chat with AI large language models running natively in your browser

Launch
38
C

Camelid

A Rust-native local inference backend

Launch
39
C

ChatLLM-Web

Private local model studio, AI chat

Launch
40
A

Aikit

Fine-tune, build, and deploy open-source LLMs easily!

Launch
41
J

Janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs…

Launch
42
T

TRAE-Ollama-Bridge

Encapsulate the local Ollama into an OpenAI-compatible API

Launch
43
O

Olla

High-performance lightweight proxy and load balancer for LLM infrastructure

Launch
44
H

HilbertRaum

AI, fully local on your computer

Launch
45
O

ollama

Get up and running with large language models, locally

Launch
46
P

pooled

Peer-to-peer LLM inference in the browser: pool your devices

Launch
47
S

SMG

Engine-agnostic LLM gateway in Rust

Launch
48
L

Llamafarm

Deploy any AI model, agent, database, RAG

Launch
49
I

Inference Gateway

An open-source, cloud-native, high-performance gateway unifying multiple LLM…

Launch
50
P

Paddler

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs

Launch

About llama.cpp

L

llama.cpp

C/C++ inference engine for large language models, supporting quantization

C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.

Category
AI
Website
llama.app