LLMfit: Find Which LLM Models Will Actually Run on Your Hardware
Hugging Face hosts over three million models. A single base model often comes in dozens of quantized variants. Downloading a 30 GB file to find out it crashes with an OOM error is a familiar frustration. LLMfit is a Rust-based terminal tool that scans your hardware and scores hundreds of models across four dimensions to tell you which ones will genuinely run well on your machine, and at what performance level.
The project is MIT-licensed, has around 27,000 GitHub stars, and is available on crates.io, Homebrew, Scoop, and Docker in addition to the install script.
The problem it solves
A model listed as "35B" could have dozens of quantized variants. Q80 gives you near-original quality but needs the most VRAM. Q2K is the most compressed but degrades output quality significantly. Somewhere between those extremes is the version that best fits your hardware.
Without tooling, figuring out which quantization fits your VRAM, how fast it will actually run, and whether the context window you need is achievable requires downloading and testing. LLMfit does this analysis before you download anything.
Installation
Install via the script:
Or via Homebrew on macOS and Linux:
Or via Scoop on Windows:
Launch the interactive TUI:
For JSON output piped to other tools:
Docker is also supported: docker run ghcr.io/alexsjones/llmfit. Build from source with cargo build --release if you prefer.
How it works
Hardware detection
LLMfit probes your system rather than reading a spec file. For NVIDIA GPUs it uses nvidia-smi and supports multi-GPU setups by aggregating VRAM across all detected cards. For AMD it uses rocm-smi. For Apple Silicon it uses system_profiler and correctly identifies unified memory, where the GPU and CPU share the same pool. It also detects which inference backend you have installed (CUDA, Metal, ROCm) since that affects speed estimation.
Dynamic quantization selection
For each model in its database, LLMfit doesn't assume a fixed quantization. It walks the quality hierarchy from Q80 down to Q2K and picks the highest quality quantization that fits in your available memory. If nothing fits at full context, it tries again at half context. This means you always see the best version of a model your hardware can support, not just whether a model exists.
Scoring across four dimensions
Each model is scored 0–100 on four dimensions:
Quality measures overall model capability: parameter count, the model family's reputation, and penalties for heavy quantization that degrades output.
Speed estimates tokens per second. Token generation is memory-bandwidth-bound: each token requires reading the full model weights from VRAM. LLMfit uses your GPU's actual memory bandwidth and the model's size to calculate throughput.
Fit measures memory utilization. The sweet spot is 50–80% of available memory. Too little means you could run a larger model; too much risks crashes.
Context evaluates the model's context window against what the intended use case needs.
These four dimensions combine into a weighted composite score, and the weights vary by use case. Chat prioritizes Speed; Reasoning prioritizes Quality. The tool reranks the list based on what you're actually trying to do.
The interface
The TUI shows detected hardware specs at the top and a ranked list of models below. Columns include the model name, provider, parameter count, composite score, estimated tok/s, the quantization LLMfit selected for your hardware, disk size, which hardware node will run it (GPU or CPU), memory percentage used, context window size, a qualitative fit rating (Perfect / Good / Too Tight), and the intended use case.
Navigation: j/k to move, / to search, f to cycle use-case filters, s to sort by column, d to download a model directly, r to refresh, m/c/x for multi-model comparison, t to cycle themes. Plan mode (p) estimates the hardware requirements for a model configuration you're considering, useful if you're thinking about a hardware upgrade.
The GitHub repository is at github.com/AlexsJones/llmfit.
Performance across different hardware
2015 MacBook Pro (i5, 8GB RAM)
LLMfit produces a usable list of smaller models flagged as Good fit, primarily under 1 billion parameters at Q8_0 quantization. Estimated performance in the 40–170 tok/s range. Filtering for coding shows a 12B Qwen model, though at 2.3 tok/s that's the hardware's limit, not the tool's.
M2 Max MacBook Pro (32GB unified RAM)
Recommendations become substantially more capable. Top-rated models show as Perfect fit. LLMfit correctly handles Apple Silicon's unified memory and provides accurate suggestions that account for the full 32GB being available to the GPU.
Custom PC (RTX 5090, 64GB DDR5)
Almost everything scores 100. LLMfit's value shifts from "what can I run?" to "what's the best model for this specific task?" Sorting by context size and filtering for coding surfaces models with the largest context windows among the many that score identically.
Raspberry Pi 1 (700MHz ARM11, 512MB RAM)
LLMfit's formulas are not optimized for hardware this far below its intended range. It recommended models with multi-hundred-billion parameter counts that would require hundreds of gigabytes of RAM. On extreme low-end hardware the estimates break down. A 19M parameter model does run on this hardware at around 0.3 tok/s, but LLMfit won't tell you that usefully.
What it doesn't do
LLMfit scores based on hardware profiles and a model database, not by actually running models. Scores are estimates. Real-world performance varies with context length, prompt complexity, and thermal throttling. The community leaderboard feature aggregates real benchmark data from users to supplement the estimates, but verifying the tool's suggestions on your specific workload before committing to a large download is still worth doing.
It also requires at least one compatible inference backend to be installed (Ollama, llama.cpp, MLX, Docker Model Runner, or LM Studio) to actually run the models it recommends. LLMfit tells you what to run; the runtime runs it.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.