200 open source apps like vLLM, closest match first.
YoloRouter
A self-hosted, OpenAI-compatible LLM gateway
Qwen3.6-windows-server
One-click Qwen3.6-27B inference on Windows
Ollama Gui
A Web Interface for chatting with your local LLMs via the ollama API
Self-Hosted AI Stack
Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document…
Manifest
A complete backend in a single YAML file
Picollm
On-device LLM Inference Powered by X-Bit Quantization
Hollama
A minimal LLM chat app that runs entirely in your browser
Kalavai Client
Aggregates compute from spare GPU capacity
TRAE-Ollama-Bridge
Encapsulate the local Ollama into an OpenAI-compatible API
Nobodywho
NobodyWho is an inference engine that lets you run LLMs locally and…
LLM Harbor
Containerized LLM toolkit
Local Studio
Control panel for VLLM, Sglang, llama.cpp, exllamav3
Magnitude
Run open models as fast as your hardware allows
Anubis Oss
Local LLM Testing & Benchmarking for Apple Silicon
Sipp
AI inference, packed simply
TensorSharp
A native .NET LLM inference engine and agent runtime for GGUF models
LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models
MLX-LoRA-Studio
A native Mac App for LLM fine-tuning on Apple Silicon
Rapid-MLX
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM…
WebLLM Chat
Chat with AI large language models running natively in your browser
Transformer Lab
100% Open Source Toolkit for Large Language Models: Train, Tune
Ling
The LLMs' framework optimized for ultra-fast response times
Langtrace
Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability…
LLM LS
LSP server leveraging LLMs for code completion (and more?)
Faster Chat
A blazingly fast, privacy first & OPEN AI Chat Interface
Reins
Local AI chat for iPhone, iPad and Mac
One API
LLM API 管理 & 分发系统,支持 OpenAI、Azure、Anthropic Claude、Google
Strix Halo Llamacpp
Performance-tuned llama.cpp for AMD Strix Halo
Lollms Hub
A proxy server for multiple ollama instances with Key security
Deepseek-ocr.rs
Rust multi‑backend OCR/VLM engine (DeepSeek‑OCR-1/2, PaddleOCR‑VL, DotsOCR)…
Llamafarm
Deploy any AI model, agent, database, RAG
UnoRouter
Open-source LLM gateway: 300+ AI models behind one OpenAI-compatible key
Guaardvark
Self-hosted AI studio on your own GPU: chat with your files
Janas
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs…
Ollama Grid Search
A multi-platform desktop application to evaluate and compare LLM models
overtchat
A simpler self-hosted alternative to Open WebUI
Camelid
A Rust-native local inference backend
Matrixhub
An Open-source, self-hosted AI model hub
Ava
All-in-one desktop app for running LLMs locally
Llmtrim
Local proxy that compresses your LLM API requests so you pay less
Audio.cpp
An all-in-one, pure C++ inference engine for audio models, powered by ggml
Vellum
Vellum helps developers build AI systems
Corpusos
Open-source protocol suite standardizing LLM, Vector, Graph
Keras LLM Robot
A web UI Project In order to learn the large language model
Open WebUI
Run AI on your own terms
Memlayer
Plug-and-play memory for LLMs in 3 lines of code
Gerbil
A desktop app for running Large Language Models locally
LLamaTuner
Easy and Efficient Finetuning LLMs
Portkey
Full-stack LLMOps platform to monitor, manage, and improve LLM-based apps
Pulsar
SSD-streaming inference engine for giant MoE models (Rust + CUDA)