Skip to content
S

shoehorn

Make any language model fit your machine

Open source

#191in Finance
0In stores

Description

shoehorn is a tool for making a language model fit the memory your machine actually has. Preset quantizations ignore your hardware: you pick…

Highlights

  • Fits a GGUF model to your exact memory budget
  • Per-tensor mixed-precision quantization
  • Local web app with a one-button run and chat
  • Model finder for popular Hugging Face models
  • Standard GGUF v3 output, MIT licensed

Ratings & Reviews

No ratings yet

Be the first to rate shoehorn.

People also visit

H

huggingface-local-models

Select models to run locally

AI Skills
Get
H

huggingface-best

About finding the best, top, or recommended model for a task

AI Skills
Get
I

Inferencer

Run and Deeply Control Local AI Models

AI
Launch
N

nemo-mbridge-recipe-recommender

Recommend and customize Megatron Bridge library and benchmark recipes

AI Skills
Get
Q

qwencloud-model-selector

Recommend the best Qwen model and parameters

AI Skills
Get
F

fine-tuning-expert

Fine-tuning LLMs, training custom models

AI Skills
Get

shoehorn FAQ

What does shoehorn do?

It quantizes a language model so it uses the memory you have as fully as possible, and then runs it with llama.cpp.

What do I need installed?

llama.cpp on your PATH as the inference backend. The Homebrew install pulls it in for you.

How do I start it?

Run shoehorn ui to open the local app, pick a model, press the button and chat.

Which platforms are supported?

macOS on Apple silicon, Linux on x86-64 with NVIDIA or AMD, and Windows on x86-64 with NVIDIA, according to the download links.

Can I see which models fit my machine?

Yes. The Find models page scans popular Hugging Face models against the hardware and context length you pick.

Is it free?

Yes. It is MIT licensed and the source is on GitHub.

Developer Stuff

Badges