TurboFieldfare: Running a 26-Billion-Parameter Model on 2 GB of RAM
Running a 26-billion-parameter LLM on a consumer machine has traditionally meant accepting either enormous VRAM requirements or crushing performance penalties from CPU offloading. TurboFieldfare, a custom Swift and Metal inference runtime built by Andrey Mikhaylov, does it on any Apple Silicon Mac with as little as 2 GB of active RAM, including 8 GB models. The full installed model takes about 14.3 GB of SSD space, but the runtime footprint during inference is roughly 2 GB.
This article covers how TurboFieldfare achieves this, the architectural properties of Gemma 4 that make it possible, why Apple Silicon's unified memory is central to the approach, and how to get it running.
How it works in brief
TurboFieldfare runs the Gemma 4 26B-A4B instruction checkpoint. Rather than loading the entire model into RAM, it keeps only the shared core and KV cache resident while streaming the specific expert sub-networks needed for each token directly from SSD. The model's Mixture-of-Experts architecture means only a small fraction of parameters are active at any given moment, so most of the model can sit on disk and be fetched on demand.
The MoE architecture that makes this possible
Conventional dense LLMs activate every parameter for every token. A 26B-parameter dense model requires the full ~14 GB to be in memory at once. Gemma 4 is a Mixture-of-Experts model, which works differently.
Each MoE layer contains 128 distinct expert sub-networks. A lightweight router examines each incoming token and selects the 8 most relevant experts to activate. The remaining 120 experts in that layer are untouched.
Across the full model, this means roughly 3.9 billion parameters are active per token while the remaining 85% sit idle. TurboFieldfare exploits this sparsity: if only 8 of 128 experts per layer are ever needed at once, there's no reason to keep all 128 loaded in RAM.
Why Apple Silicon matters
The approach depends on hardware that most PCs don't have: unified memory.
On a typical PC with a discrete GPU, the CPU and GPU have separate memory pools. Getting model weights from SSD to the GPU requires the CPU to read data into system RAM, then copy it across the PCIe bus into VRAM. For an application that needs to fetch tiny chunks of data thousands of times per second, this two-copy path with a bus hop becomes a significant bottleneck.
Apple Silicon uses a System on a Chip architecture where the CPU, GPU, Neural Engine, and memory controllers all share a single physical memory pool.
There's no separate VRAM. When the CPU reads expert weights from SSD into unified memory, the GPU can access that same data at that same memory address without any copying. This zero-copy path is what makes streaming experts from SSD practical at inference speeds.
System design
Splitting the model
During initial setup, TurboFieldfare repacks the model weights into its own .gturbo format, splitting the model into two parts based on usage patterns.
The resident core (~1.35 GB) contains everything used on every single token: attention mechanisms, the router, word embeddings, and a shared expert that always activates. This stays memory-mapped and resident throughout the session.
The expert pool (~12.9 GB) contains all 3,840 experts across 30 layers (30 × 128). This file never loads fully into RAM. It sits on SSD and individual experts are streamed into Metal buffers as the router selects them.
Hiding I/O latency behind compute
The runtime orchestrates CPU and GPU work to overlap disk reads with computation. For each token through a layer, the GPU begins running attention and the router immediately since those use only the resident core already in memory. While the router is executing, it identifies the 8 needed experts. The CPU starts reading those experts from SSD while the GPU immediately begins work on the shared expert.
By the time the GPU finishes the shared expert, the CPU has loaded the 8 selected experts into a shared Metal buffer. The GPU then processes them without waiting. The disk read is hidden behind the shared expert computation.
GPU-ready file format
Normally, model weights are stored on disk in a compressed or serialized format that requires CPU-side unpacking and conversion before the GPU can use them. TurboFieldfare's one-time repack step arranges expert weights on disk in the exact binary layout the Metal kernels expect, down to the 4-bit quantized values.
Reading from SSD is loading. There's no intermediate conversion step, so the CPU performs a direct memory copy from file into GPU-accessible memory.
LFU caching
Each layer maintains a cache of 16 experts in RAM. Expert usage in MoE models is not uniformly random: some experts are high-frequency generalists activated across a wide range of inputs, while others are low-frequency specialists used only for specific contexts. TurboFieldfare uses a Least Frequently Used eviction policy rather than Least Recently Used.
LRU would evict a popular generalist expert if a rare specialist happened to be used more recently. LFU tracks cumulative access frequency, so the most broadly useful experts remain cached even after occasional specialist activations. This maximizes cache hit rate for the experts that matter most to throughput.
Installation
Requirements: Apple Silicon Mac, macOS 26, Swift 6.2, Metal 4, Xcode 26. The runtime is arm64-only with no alternative build for Intel Macs, Windows, or Linux since it depends directly on Metal.
Clone and build:
The first build downloads Swift package dependencies. On first launch, choose Install. TurboFieldfare downloads the pinned Gemma 4 checkpoint from Hugging Face (~15 GB) and repacks it into the .gturbo streaming format. This is a one-time step. After that, click Load Model and start generating.
The app shows live stats: tokens per second, total tokens generated, and memory consumption. On an M3 Mac, one developer reported over 23 tokens per second.
Beyond the native app, TurboFieldfare also includes a CLI for instruction chat and raw completion, and an experimental OpenAI-compatible local server at http://127.0.0.1:8080/v1 that supports streaming and tools. The project is licensed under Apache 2.0. Model weights are not included and remain governed by their original terms.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.