Alternatives to vLLM
200 apps like vLLM, closest match first.
Top picks
- V

vllm-mlx
Launchvllm-mlx is an inference server for large language models on Apple Silicon. It exposes OpenAI-compatible and Anthropic-compatible APIs, so existing clients and tools can talk to a model running on your Mac. The engine is built on MLX, Apple
Free
- W
Web LLM
LaunchWebLLM is an inference engine that runs large language models directly inside a web browser. It uses WebGPU for hardware acceleration, so text generation runs on the visitor's device without server-side processing. The project's stated goal
Free
- L

llama.cpp
LaunchC/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
- P
Paddler
LaunchOpen-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem.
FreeSelf-hosted
- L

LLMKube
LaunchLLMKube is a Kubernetes operator for running language models on your own hardware. You write a Model resource and an InferenceService resource in YAML, and the operator serves the model behind an OpenAI-compatible endpoint once it is ready.
FreeSelf-hosted
- O

Ollama
LaunchOllama is a tool for running open AI models on your own computer, from the terminal or through apps you build. It runs on macOS, Windows and Linux. When a model is too large for your hardware, cloud models are available too, paid for with u
Self-hosted
More like vLLM

Inference Gateway
An open-source, cloud-native, high-performance gateway unifying multiple LLM…
Time To First Token
A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization
Willow Inference Server
Open source, local, and self-hosted highly optimized language inference…
About vLLM
vLLM
Inference and serving engine for large language models, built
vLLM is an inference and serving engine for large language models, built for high throughput and efficient use of memory. It runs open models on your own hardware and exposes them through a drop-in OpenAI-compatible API, so applications that already call that API can switch to a model you serve yourself. Its main techniques are PagedAttention, which manages the attention memory in pages, and advanced scheduling with continuous batching, which keeps the GPU busy by adding new requests while others are still running. The site describes it as easy to deploy across a wide range of open-source models and hardware. Hardware support is listed for NVIDIA CUDA GPUs, AMD ROCm GPUs, Huawei Ascend NPUs, AWS Neuron, Google Cloud TPUs, Intel Gaudi and XPU, CPUs, and Apple Silicon, among others. The quick start lets you pick a stable or nightly build, a platform such as CUDA, ROCm, XPU or CPU, and a package route, then run an install command. Python 3.10 or newer is required, with 3.12 or newer recommended, and uv is suggested for a faster install. vLLM is a community project supported by donations and by compute from several companies. The site also links to docs, recipes, a performance page and a roadmap.
- Category
- AI
- Website
- vllm.ai



