Open source alternatives to shoehorn
200 open source apps like shoehorn, closest match first.
Top picks
- L

llama.cpp
LaunchC/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
Open source
- B
BodhiApp
LaunchUnified AI gateway combining local LLM inference, cloud API proxying, and MCP tool integration with built-in chat UI and enterprise access control.
Open source
- A
- T

TurboLLM
LaunchRun open AI models on your own machine — no cloud, no account, fully offline. Any GGUF model on any llama.cpp-compatible engine, auto-tuned to your hardware, with real measured tokens/sec before you download, a fast browser UI, and an OpenA
Open sourceSelf-hosted
- W
whatcani.run
Launchwhatcani.run helps you find the best AI models to run locally and how to run them, based on real data from trials that people ran on their own hardware. Each model is listed with its runtime, decode and prefill speed, time to first token, p
Open source
- H
HyperQwen
LaunchServe large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant
Open source
More like shoehorn
About shoehorn
shoehorn
Make any language model fit your machine
shoehorn is a tool for making a language model fit the memory your machine actually has. Preset quantizations ignore your hardware: you pick one that fits and either waste quality headroom or find out at load time that it does not fit. shoehorn starts from the memory you have, subtracts what inference needs, and solves a per-tensor mixed-precision assignment, so it spends the remaining budget on the tensors where extra precision buys the most quality. The site's example fits a Qwen3 4B model into a 519 MiB weights budget, using 99.998 percent of it. The workflow quantizes a BF16 GGUF file with an importance matrix, then runs the result with llama.cpp, which must be on your PATH. The command line tool includes a local web app, started with shoehorn ui, where you pick a model, press one button and chat. The site also has a Find models page that scans popular Hugging Face models for ones that fit a chosen machine, such as a Mac with 8 to 128 GB or a GPU with 8 to 32 GB of VRAM, and a context length setting. Installation is available through Homebrew, as release downloads for macOS, Linux and Windows, or from source with cargo. The tool is MIT licensed. Its quantizer is written from scratch in Rust, and it produces standard GGUF v3 files that llama.cpp based tools can load.
- Price
- Open source
- Category
- Finance
- Website
- notactuallytreyanastasio.github.io


