Skip to content
vLLM

Self-hosted alternatives to vLLM

200 self-hosted apps like vLLM, closest match first.

Top picks

  1. P

    Paddler

    Launch

    Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem.

    FreeSelf-hosted

  2. L

    LLMKube

    Launch

    LLMKube is a Kubernetes operator for running language models on your own hardware. You write a Model resource and an InferenceService resource in YAML, and the operator serves the model behind an OpenAI-compatible endpoint once it is ready.

    FreeSelf-hosted

  3. O

    Ollama

    Launch

    Ollama is a tool for running open AI models on your own computer, from the terminal or through apps you build. It runs on macOS, Windows and Linux. When a model is too large for your hardware, cloud models are available too, paid for with u

    Self-hosted

  4. L

    LocalAI

    Launch

    LocalAI is an open-source, MIT-licensed AI engine that runs language models, speech, vision, image and video generation and agents on your own hardware. Its API is compatible with OpenAI, and the site says it also speaks the Anthropic, Olla

    FreeSelf-hosted

  5. G

    Ggrun

    Launch

    llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

    FreeSelf-hosted

  6. I

    Inference Gateway

    Launch

    An open-source, cloud-native, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic, Cloudflare and DeepSeek.

    FreeSelf-hosted

More like vLLM

7
O

Olla

High-performance lightweight proxy and load balancer for LLM infrastructure

Launch
8
T

TurboLLM

Run any local LLM engine, auto-tuned to your GPU

Launch
9
O

OpenGateLLM

An open-source, unified interface for running and managing self-hosted LLMs

Launch
10
W

Willow Inference Server

Open source, local, and self-hosted highly optimized language inference…

Launch
11
L

Llmaker

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Launch
12
T

The OSS LLMOps Stack

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies…

Launch
13
T

Textgen

Open-source desktop app for local LLMs

Launch
14
L

Llamactl

Unified management and routing for llama.cpp, MLX and vLLM models

Launch
15
R

Routerly.ai

Self-hosted LLM gateway that routes requests across AI providers

Launch
16
V

VoidLLM

Privacy-first LLM proxy and AI gateway

Launch
17
G

GoModel

AI gateway written in Go with a unified OpenAI-compatible API

Launch
18
P

Project Maya

Run GLM-5.3-Flash (321B MoE) privately on your own GPU(s)…

Launch
19
O

OmniKey-AI-Unified-Key-Manager

A self-hosted LLM proxy and failover gateway supporting OpenAI & Gemini…

Launch
20
Y

YoloRouter

A self-hosted, OpenAI-compatible LLM gateway

Launch
21
O

Ollama Gui

A Web Interface for chatting with your local LLMs via the ollama API

Launch
22
S

Self-Hosted AI Stack

Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document…

Launch
23
P

Picollm

On-device LLM Inference Powered by X-Bit Quantization

Launch
24
K

Kalavai Client

Aggregates compute from spare GPU capacity

Launch
25
L

LLM Harbor

Containerized LLM toolkit

Launch
26
S

Sipp

AI inference, packed simply

Launch
27
L

LLM LS

LSP server leveraging LLMs for code completion (and more?)

Launch
28
F

Faster Chat

A blazingly fast, privacy first & OPEN AI Chat Interface

Launch
29
G

Guaardvark

Self-hosted AI studio on your own GPU: chat with your files

Launch
30
O

overtchat

A simpler self-hosted alternative to Open WebUI

Launch
31
M

Matrixhub

An Open-source, self-hosted AI model hub

Launch
32
O

Open WebUI

Run AI on your own terms

Launch
33
O

ObsidianArc

A self-hosted AI platform with a Go server, multi-provider routing, usage…

Launch
34
L

Llocal

Aiming to provide a seamless and privacy driven chatting experience

Launch
35
O

Otari

Open-source, OpenAI-compatible LLM gateway you run yourself

Launch
36
O

OrcaRouter-Lite

Self-hosted LLM router with a managed safety net

Launch
37
O

Oriveo

Open-source, bring-your-own-key (BYOK) multi-model AI chat client for iOS…

Launch
38
O

Octopus

One Hub All LLMs For You | 为个人打造的 LLM API 聚合网关

Launch
39
U

Unsloth

Local UI to run and train LLMs and diffusion models

Launch
40
A

Agentsystems

Self-hosted app store and runtime for AI agents

Launch
41
S

Sage

Self-hosted voice chat with LLMs

Launch
42
F

FreeLingo

Open-source, self-hosted AI language learning platform

Launch
43
S

Selfhost AI

One-command installer for a self-hosted AI stack on your own server: n8n

Launch
44
N

NadirClaw

Source-available LLM router and AI cost optimizer

Launch
45
B

Belullama

Belullama is a comprehensive AI application

Launch
46
M

Minion

A tiny single-file coding agent for self-hosted models

Launch
47
S

Spanlens

Drop-in observability platform for OpenAI, Anthropic, and Gemini

Launch
48
A

Atomic-Chat

Local AI app and inference engine for agents

Launch
49
S

SourceWeft

Open-source, self-hostable alternative to ChatGPT and NotebookLM

Launch
50
B

Beam

Run GPU inference, task queues, and sandboxes on serverless infrastructure

Launch

About vLLM

V

vLLM

Inference and serving engine for large language models, built

vLLM is an inference and serving engine for large language models, built for high throughput and efficient use of memory. It runs open models on your own hardware and exposes them through a drop-in OpenAI-compatible API, so applications that already call that API can switch to a model you serve yourself. Its main techniques are PagedAttention, which manages the attention memory in pages, and advanced scheduling with continuous batching, which keeps the GPU busy by adding new requests while others are still running. The site describes it as easy to deploy across a wide range of open-source models and hardware. Hardware support is listed for NVIDIA CUDA GPUs, AMD ROCm GPUs, Huawei Ascend NPUs, AWS Neuron, Google Cloud TPUs, Intel Gaudi and XPU, CPUs, and Apple Silicon, among others. The quick start lets you pick a stable or nightly build, a platform such as CUDA, ROCm, XPU or CPU, and a package route, then run an install command. Python 3.10 or newer is required, with 3.12 or newer recommended, and uv is suggested for a faster install. vLLM is a community project supported by donations and by compute from several companies. The site also links to docs, recipes, a performance page and a roadmap.

Category
AI
Website
vllm.ai