Skip to content
vLLM

Open source alternatives to vLLM

200 open source apps like vLLM, closest match first.

More like vLLM

51
Y

YoloRouter

A self-hosted, OpenAI-compatible LLM gateway

Launch
52
Q

Qwen3.6-windows-server

One-click Qwen3.6-27B inference on Windows

Launch
53
O

Ollama Gui

A Web Interface for chatting with your local LLMs via the ollama API

Launch
54
S

Self-Hosted AI Stack

Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document…

Launch
55
M

Manifest

A complete backend in a single YAML file

Launch
56
P

Picollm

On-device LLM Inference Powered by X-Bit Quantization

Launch
57
H

Hollama

A minimal LLM chat app that runs entirely in your browser

Launch
58
K

Kalavai Client

Aggregates compute from spare GPU capacity

Launch
59
T

TRAE-Ollama-Bridge

Encapsulate the local Ollama into an OpenAI-compatible API

Launch
60
N

Nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and…

Launch
61
L

LLM Harbor

Containerized LLM toolkit

Launch
62
L

Local Studio

Control panel for VLLM, Sglang, llama.cpp, exllamav3

Launch
63
M

Magnitude

Run open models as fast as your hardware allows

Launch
64
A

Anubis Oss

Local LLM Testing & Benchmarking for Apple Silicon

Launch
65
S

Sipp

AI inference, packed simply

Launch
66
T

TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models

Launch
67
L

LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models

Launch
68
M

MLX-LoRA-Studio

A native Mac App for LLM fine-tuning on Apple Silicon

Launch
69
R

Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM…

Launch
70
W

WebLLM Chat

Chat with AI large language models running natively in your browser

Launch
71
T

Transformer Lab

100% Open Source Toolkit for Large Language Models: Train, Tune

Launch
72
L

Ling

The LLMs' framework optimized for ultra-fast response times

Launch
73
L

Langtrace

Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability…

Launch
74
L

LLM LS

LSP server leveraging LLMs for code completion (and more?)

Launch
75
F

Faster Chat

A blazingly fast, privacy first & OPEN AI Chat Interface

Launch
76
R

Reins

Local AI chat for iPhone, iPad and Mac

Launch
77
O

One API

LLM API 管理 & 分发系统,支持 OpenAI、Azure、Anthropic Claude、Google

Launch
78
S

Strix Halo Llamacpp

Performance-tuned llama.cpp for AMD Strix Halo

Launch
79
L

Lollms Hub

A proxy server for multiple ollama instances with Key security

Launch
80
D

Deepseek-ocr.rs

Rust multi‑backend OCR/VLM engine (DeepSeek‑OCR-1/2, PaddleOCR‑VL, DotsOCR)…

Launch
81
L

Llamafarm

Deploy any AI model, agent, database, RAG

Launch
82
U

UnoRouter

Open-source LLM gateway: 300+ AI models behind one OpenAI-compatible key

Launch
83
G

Guaardvark

Self-hosted AI studio on your own GPU: chat with your files

Launch
84
J

Janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs…

Launch
85
O

Ollama Grid Search

A multi-platform desktop application to evaluate and compare LLM models

Launch
86
O

overtchat

A simpler self-hosted alternative to Open WebUI

Launch
87
C

Camelid

A Rust-native local inference backend

Launch
88
M

Matrixhub

An Open-source, self-hosted AI model hub

Launch
89
A

Ava

All-in-one desktop app for running LLMs locally

Launch
90
L

Llmtrim

Local proxy that compresses your LLM API requests so you pay less

Launch
91
A

Audio.cpp

An all-in-one, pure C++ inference engine for audio models, powered by ggml

Launch
92
V

Vellum

Vellum helps developers build AI systems

Launch
93
C

Corpusos

Open-source protocol suite standardizing LLM, Vector, Graph

Launch
94
K

Keras LLM Robot

A web UI Project In order to learn the large language model

Launch
95
O

Open WebUI

Run AI on your own terms

Launch
96
M

Memlayer

Plug-and-play memory for LLMs in 3 lines of code

Launch
97
G

Gerbil

A desktop app for running Large Language Models locally

Launch
98
L

LLamaTuner

Easy and Efficient Finetuning LLMs

Launch
99
P

Portkey

Full-stack LLMOps platform to monitor, manage, and improve LLM-based apps

Launch
100
P

Pulsar

SSD-streaming inference engine for giant MoE models (Rust + CUDA)

Launch