Description
shoehorn is a tool for making a language model fit the memory your machine actually has. Preset quantizations ignore your hardware: you pick one that fits and either waste quality headroom or find out at load time that it does not fit. shoehorn starts from the memory you have, subtracts what inference needs, and solves a per-tensor mixed-precision assignment, so it spends the remaining budget on the tensors where extra precision buys the most quality. The site's example fits a Qwen3 4B model into a 519 MiB weights budget, using 99.998 percent of it. The workflow quantizes a BF16 GGUF file with an importance matrix, then runs the result with llama.cpp, which must be on your PATH. The command line tool includes a local web app, started with shoehorn ui, where you pick a model, press one button and chat. The site also has a Find models page that scans popular Hugging Face models for ones that fit a chosen machine, such as a Mac with 8 to 128 GB or a GPU with 8 to 32 GB of VRAM, and a context length setting. Installation is available through Homebrew, as release downloads for macOS, Linux and Windows, or from source with cargo. The tool is MIT licensed. Its quantizer is written from scratch in Rust, and it produces standard GGUF v3 files that llama.cpp based tools can load.…
Highlights
- Fits a GGUF model to your exact memory budget
- Per-tensor mixed-precision quantization
- Local web app with a one-button run and chat
- Model finder for popular Hugging Face models
- Standard GGUF v3 output, MIT licensed
Ratings & Reviews
No ratings yet
Be the first to rate shoehorn.
People also visit

nemo-mbridge-recipe-recommender
Recommend and customize Megatron Bridge library and benchmark recipes
shoehorn FAQ
What does shoehorn do?
It quantizes a language model so it uses the memory you have as fully as possible, and then runs it with llama.cpp.
What do I need installed?
llama.cpp on your PATH as the inference backend. The Homebrew install pulls it in for you.
How do I start it?
Run shoehorn ui to open the local app, pick a model, press the button and chat.
Which platforms are supported?
macOS on Apple silicon, Linux on x86-64 with NVIDIA or AMD, and Windows on x86-64 with NVIDIA, according to the download links.
Can I see which models fit my machine?
Yes. The Find models page scans popular Hugging Face models against the hardware and context length you pick.
Is it free?
Yes. It is MIT licensed and the source is on GitHub.







