Skip to content
vLLM

Free alternatives to vLLM

200 free apps like vLLM, closest match first.

Top picks

  1. V

    vllm-mlx

    Launch

    vllm-mlx is an inference server for large language models on Apple Silicon. It exposes OpenAI-compatible and Anthropic-compatible APIs, so existing clients and tools can talk to a model running on your Mac. The engine is built on MLX, Apple

    Free

  2. W

    Web LLM

    Launch

    WebLLM is an inference engine that runs large language models directly inside a web browser. It uses WebGPU for hardware acceleration, so text generation runs on the visitor's device without server-side processing. The project's stated goal

    Free

  3. P

    Paddler

    Launch

    Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem.

    FreeSelf-hosted

  4. L

    LLMKube

    Launch

    LLMKube is a Kubernetes operator for running language models on your own hardware. You write a Model resource and an InferenceService resource in YAML, and the operator serves the model behind an OpenAI-compatible endpoint once it is ready.

    FreeSelf-hosted

  5. F

    A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.

    Free

  6. L

    LocalAI

    Launch

    LocalAI is an open-source, MIT-licensed AI engine that runs language models, speech, vision, image and video generation and agents on your own hardware. Its API is compatible with OpenAI, and the site says it also speaks the Anthropic, Olla

    FreeSelf-hosted

More like vLLM

7
A

Ai00 Server

The all-in-one RWKV runtime box with embed, RAG, AI agents, and more

Launch
8
M

mlx-serve

Native LLM inference server for Apple Silicon

Launch
9
G

Ggrun

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched

Launch
10
I

Inference Gateway

An open-source, cloud-native, high-performance gateway unifying multiple LLM…

Launch
11
O

ollama

Get up and running with large language models, locally

Launch
12
O

Olla

High-performance lightweight proxy and load balancer for LLM infrastructure

Launch
13
L

Llamastash

Zero-overhead, terminal-native local-LLM runtime manager

Launch
14
T

TurboLLM

Run any local LLM engine, auto-tuned to your GPU

Launch
15
S

SMG

Engine-agnostic LLM gateway in Rust

Launch
16
T

Time To First Token

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization

Launch
17
F

Fastllm-proxy

The lowest-overhead LLM router

Launch
18
T

ToshLLM

Run large language models locally on Intel Macs with AMD GPUs

Launch
19
L

Llamafile

Distribute and run LLMs with a single file

Launch
20
O

OpenGateLLM

An open-source, unified interface for running and managing self-hosted LLMs

Launch
21
W

Willow Inference Server

Open source, local, and self-hosted highly optimized language inference…

Launch
22
L

Llmaker

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Launch
23
P

pooled

Peer-to-peer LLM inference in the browser: pool your devices

Launch
24
L

liter-llm

Universal LLM API client — 163 providers, 14 native language bindings…

Launch
25
H

HyperQwen

Serve large Qwen models fast on the GPUs you actually own

Launch
26
B

BodhiApp

Run Open Source/Open Weight LLMs locally with OpenAI compatible APIs

Launch
27
L

LlamaChat

Create AI models on Mac, have natural conversations, and gain real-time insights

Launch
28
H

huggingface-local-models

Select models to run locally

Get
29
O

OlliteRT

Turn your Android phone into an OpenAI-compatible LLM inference server

Launch
30
O

OpenLLMetry

Open-source observability platform for LLMs using OpenTelemetry

Launch
31
W

whatcani.run

Find the best models and how to run them locally

Launch
32
J

jetson-inference-mem-tune

Pick the serving stack and per-runtime memory flags

Get
33
T

The OSS LLMOps Stack

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies…

Launch
34
T

Textgen

Open-source desktop app for local LLMs

Launch
35
L

Llamactl

Unified management and routing for llama.cpp, MLX and vLLM models

Launch
36
R

Routerly.ai

Self-hosted LLM gateway that routes requests across AI providers

Launch
37
L

Local Llms On Android

Run local LLMs like Gemma, Qwen, and LLaMA on Android for offline, private

Launch
38
L

Llama3.java

Llama 3+ inference in pure Java

Launch
39
A

AgentOp

Run a real LLM in your browser. No install, no server

Launch
40
V

VoidLLM

Privacy-first LLM proxy and AI gateway

Launch
41
T

TheWhisper

Optimized Whisper models for streaming and on-device use

Launch
42
G

GoModel

AI gateway written in Go with a unified OpenAI-compatible API

Launch
43
L

LLM X

LLMX; Easiest 3rd party Local LLM UI for the web!

Launch
44
J

jetson-llm-serve

Stand up vLLM or SGLang serving on Jetson

Get
45
C

ChatLLM-Web

Private local model studio, AI chat

Launch
46
P

Project Maya

Run GLM-5.3-Flash (321B MoE) privately on your own GPU(s)…

Launch
47
M

MLX Tune

Fine-tune LLMs on your Mac with Apple Silicon

Launch
48
L

Lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

Launch
49
O

OmniKey-AI-Unified-Key-Manager

A self-hosted LLM proxy and failover gateway supporting OpenAI & Gemini…

Launch
50
G

Graviton

500B+ LLMs on a Mac Mini

Launch

About vLLM

V

vLLM

Inference and serving engine for large language models, built

vLLM is an inference and serving engine for large language models, built for high throughput and efficient use of memory. It runs open models on your own hardware and exposes them through a drop-in OpenAI-compatible API, so applications that already call that API can switch to a model you serve yourself. Its main techniques are PagedAttention, which manages the attention memory in pages, and advanced scheduling with continuous batching, which keeps the GPU busy by adding new requests while others are still running. The site describes it as easy to deploy across a wide range of open-source models and hardware. Hardware support is listed for NVIDIA CUDA GPUs, AMD ROCm GPUs, Huawei Ascend NPUs, AWS Neuron, Google Cloud TPUs, Intel Gaudi and XPU, CPUs, and Apple Silicon, among others. The quick start lets you pick a stable or nightly build, a platform such as CUDA, ROCm, XPU or CPU, and a package route, then run an install command. Python 3.10 or newer is required, with 3.12 or newer recommended, and uv is suggested for a faster install. vLLM is a community project supported by donations and by compute from several companies. The site also links to docs, recipes, a performance page and a roadmap.

Category
AI
Website
vllm.ai