Skip to content
L

llama.cpp

C/C++ inference engine for large language models, supporting quantization

#37in AI
0In stores

Description

C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across…

Preview

Ratings & Reviews

No ratings yet

Be the first to rate llama.cpp.

People also visit

W

whatcani.run

Find the best models and how to run them locally

AI
Launch
M

mlx-serve

Native LLM inference server for Apple Silicon

AI
Launch
T

TurboLLM

Run any local LLM engine, auto-tuned to your GPU

AI
Launch
L

LLMKube

Kubernetes operator for self-hosted LLM inference with pluggable runtimes

AI
Launch
Q

Quail

Local model server for GGUF and MLX models on Apple silicon

AI
Launch
V

vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server

AI
Launch

llama.cpp FAQ

What is llama.cpp?

C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.

What are the best llama.cpp alternatives?

Popular alternatives to llama.cpp include whatcani.run, mlx-serve and TurboLLM. See all llama.cpp alternatives

Developer Stuff

Badges