Skip to content
llama.cpp

Self-hosted alternatives to llama.cpp

200 self-hosted apps like llama.cpp, closest match first.

Top picks

  1. T

    TurboLLM

    Launch

    Run open AI models on your own machine — no cloud, no account, fully offline. Any GGUF model on any llama.cpp-compatible engine, auto-tuned to your hardware, with real measured tokens/sec before you download, a fast browser UI, and an OpenA

    FreeSelf-hosted

  2. L

    LLMKube

    Launch

    LLMKube is a Kubernetes operator for running language models on your own hardware. You write a Model resource and an InferenceService resource in YAML, and the operator serves the model behind an OpenAI-compatible endpoint once it is ready.

    FreeSelf-hosted

  3. P

    Picollm

    Launch

    Production-grade on-device AI SDKs for voice, language, and vision. STT, wake word, TTS, translation, OCR, and more. Runs on mobile, web, desktop, and embedded.

    FreeSelf-hosted

  4. S

    Sipp

    Launch

    AI inference, packed simply. A blazing-fast, zero-dependency WebGPU runtime to run GGUF models directly in the browser. Features a symmetric API for seamless local execution and cloud provider routing. Built with Rust & C++.

    FreeSelf-hosted

  5. W

    Willow Inference Server

    Launch

    Willow Inference Server (WIS) is an open-source server that runs speech and language models on your own hardware. It handles speech recognition, text-to-speech and language model inference, and it accepts audio over WebRTC, REST and WebSock

    FreeSelf-hosted

  6. L

    Llmaker

    Launch

    Models, vector DBs, embeddings, a retrieval agent, and evals: the whole modern LLM stack on your own hardware, from one command. OpenAI-compatible API + web UI per instance, GPU partitioning, no data leaving your machine.

    FreeSelf-hosted

More like llama.cpp

7
L

LocalAI

Run LLMs, speech, image generation

Launch
8
O

OpenGateLLM

An open-source, unified interface for running and managing self-hosted LLMs

Launch
9
O

Ollama

Load and run large LLMs locally to use in your terminal or build your apps

Launch
10
O

Olla

High-performance lightweight proxy and load balancer for LLM infrastructure

Launch
11
I

Inference Gateway

An open-source, cloud-native, high-performance gateway unifying multiple LLM…

Launch
12
P

Paddler

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs

Launch
13
F

Faster Chat

A blazingly fast, privacy first & OPEN AI Chat Interface

Launch
14
G

Guaardvark

Self-hosted AI studio on your own GPU: chat with your files

Launch
15
G

GoModel

AI gateway written in Go with a unified OpenAI-compatible API

Launch
16
M

Minion

A tiny single-file coding agent for self-hosted models

Launch
17
S

Self-Hosted AI Stack

Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document…

Launch
18
T

Textgen

Open-source desktop app for local LLMs

Launch
19
O

Otari

Open-source, OpenAI-compatible LLM gateway you run yourself

Launch
20
R

Routerly.ai

Self-hosted LLM gateway that routes requests across AI providers

Launch
21
L

Llocal

Aiming to provide a seamless and privacy driven chatting experience

Launch
22
O

Openclaw.net

Self-hosted Enterprise and Personal AI + agent runtime in .NET

Launch
23
N

NadirClaw

Source-available LLM router and AI cost optimizer

Launch
24
O

OmniKey-AI-Unified-Key-Manager

A self-hosted LLM proxy and failover gateway supporting OpenAI & Gemini…

Launch
25
O

overtchat

A simpler self-hosted alternative to Open WebUI

Launch
26
M

Matrixhub

An Open-source, self-hosted AI model hub

Launch
27
Y

YoloRouter

A self-hosted, OpenAI-compatible LLM gateway

Launch
28
S

SourceWeft

Open-source, self-hostable alternative to ChatGPT and NotebookLM

Launch
29
1

10x

Up to 20x faster AI coding with multi-step Superpowers

Launch
30
O

OpenAgentCore

Open-source, self-hosted implementation of the OpenAI Agents API

Launch
31
V

VoidLLM

Privacy-first LLM proxy and AI gateway

Launch
32
K

Klee

A native macOS AI chat app powered by MLX

Launch
33
F

FreeLingo

Open-source, self-hosted AI language learning platform

Launch
34
L

LMRing

Open-source, self-hostable LLM arena

Launch
35
G

Ggrun

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched

Launch
36
O

ObsidianArc

A self-hosted AI platform with a Go server, multi-provider routing, usage…

Launch
37
C

CliRelay

Self-hosted AI gateway for coding CLIs

Launch
38
P

Pocketpaw

Your AI agent in 30 seconds

Launch
39
S

Spanlens

Drop-in observability platform for OpenAI, Anthropic, and Gemini

Launch
40
C

ccLoad

Self-hosted AI API gateway for Claude Code

Launch
41
O

Ollama Gui

A Web Interface for chatting with your local LLMs via the ollama API

Launch
42
D

Dodaios

Your AI assistant, built for the agentic era

Launch
43
A

Atomic-Chat

Local AI app and inference engine for agents

Launch
44
P

Project Maya

Run GLM-5.3-Flash (321B MoE) privately on your own GPU(s)…

Launch
45
M

Muxi

Deploy intelligence. Open-source infrastructure for AI agents in production

Launch
46
K

Keirouter

Your friendly, blazing-fast, self-hostable AI gateway

Launch
47
C

Cezar

Open-source orchestrator and ADE for running Claude Code, Codex, OpenCode

Launch
48
S

Streamcore Server

Open-source realtime voice agent server in Go with WebRTC

Launch
49
A

AIClient2API

Self-hosted multi-protocol AI API proxy for Antigravity, Codex, Grok, Kiro

Launch
50
F

Free-GPT4-WEB-API

Not just GPT4! Easy to use, Self-Hosted

Launch

About llama.cpp

L

llama.cpp

C/C++ inference engine for large language models, supporting quantization

C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.

Category
AI
Website
llama.app