# Running a Local LLM on a Six-Year-Old Apple Watch The hardware comparison looks modest at first. **The Apple Watch Series 6, released in 2020, has 1 GB of RAM and a dual-core 1.8 GHz processor.** A Raspberry Pi 1 from 2012 has a single-core 700 MHz processor and 512 MB of RAM. On paper, that might suggest a noticeable but not dramatic gap. In practice, the difference is much larger. On the same model, **the Apple Watch generates text at about 15 tokens per second**, while **the Raspberry Pi 1 reaches only about 0.3 tokens per second**. That is roughly a **50x performance gap**, far beyond the kind of 5x improvement a rough spec-sheet comparison might lead you to expect. That gap helps illustrate an important point. **Modern inference performance is not determined by clock speed and RAM alone.** System-on-chip design, tighter memory integration, and instruction set support all have a major impact on how efficiently a device can run local AI workloads. The result is that a smartwatch can outperform an older single-board computer by a much wider margin than the raw hardware specs seem to imply. This article explains **what it takes to run a local LLM on watchOS**, why **Apple's Core ML is not the right fit for this particular setup**, how the **`arm64_32` architecture** works in practice, and how to build and deploy the [WatchLLM app](https://github.com/andrisgauracs/Apple-Watch-Edge-AI) on your own device. ## Why Core ML doesn't work for this model The natural starting point for on-device ML on Apple platforms is Core ML. You convert a model to `.mlpackage` format and hand it to the OS, which routes computation to CPU, GPU, or Neural Engine automatically. The limitation is that Core ML needs to recognize every operation in the model's computation graph. The model used here is Falcon-H1 90M, a 90-million-parameter model that combines two architectures: standard Transformer self-attention and Mamba-2 State-Space Models. The combination is efficient: attention handles broad contextual relationships, while Mamba-2 processes sequences with linear rather than quadratic scaling by maintaining a compressed running state. The problem is the "scan" operation at the heart of Mamba-2. ![An animation illustrating "The Scan" process in the Mamba-2 architecture, showing how each new token updates the fixed-size memory state sequentially.](https://imagedelivery.net/xZXo0QFi-1_4Zimer-T0XQ/f9536032-9034-405c-f805-55415596d700/public =1920x1080) The scan processes each token sequentially, updating a fixed-size state block based on the previous state and the new token. Core ML has predefined building blocks for convolutions, dense layers, and attention, but not for State-Space Models. When the Core ML converter encounters the scan operation, it fails. The architecture is also dynamically shaped: the scan length depends on conversation length, and Core ML requires a static computation graph defined at compile time. Both problems are fundamental and not workaroundable. The solution is `llama.cpp`, a pure C/C++ inference library with minimal dependencies, hand-optimized for ARM CPUs, and capable of running models in GGUF quantized format. It has no equivalent constraints on model architecture and no requirement for static graphs. ## The `arm64_32` architecture watchOS uses an architecture called `arm64_32`. The name sounds like a 32-bit system, which would mean losing access to modern vector extensions and being stuck with 32-bit arithmetic. This is a misreading of what the name means. `arm64_32` breaks into two components: **`arm64`:** The processor uses the full ARMv8 64-bit instruction set. All 64-bit arithmetic, the NEON vector unit for batched floating-point operations, and every modern ARM instruction are available. Nothing about the computational capability is 32-bit. **`_32`:** Only the pointer size is 32-bit. Pointers are the variables that store memory addresses. A 64-bit pointer can address 16 exabytes of memory space. A 32-bit pointer can address 4 GB. Since the Apple Watch has 1 GB of RAM, there's no practical reason to waste 4 bytes per pointer when 2 bytes would cover the entire address space. Apple made this tradeoff deliberately to reduce memory overhead on a RAM-constrained device. The practical implication: 32-bit pointers mean the watch can't address more than 4 GB of memory, which isn't a constraint for a device with 1 GB. Computationally, the watch is a full 64-bit ARM processor with NEON. The performance difference from pointer size is negligible. `llama.cpp` doesn't officially support watchOS. Its build scripts cover iOS, macOS, tvOS, and visionOS, but not watchOS. The fix is a one-line source guard in `llama.cpp` that tells the compiler to recognize the `arm64_32` target as a 64-bit ARM system for the purpose of enabling NEON optimizations. ![A screenshot from a GitHub issue or pull request showing the one-line code patch required to make `llama.cpp` recognize the ARM architecture of the watch.](https://imagedelivery.net/xZXo0QFi-1_4Zimer-T0XQ/c127b022-4781-4e18-d0b9-da13c3d9d900/lg2x =1920x1080) The WatchLLM project repository includes the pre-compiled `libllama.a` static library with this patch already applied. You don't need to compile `llama.cpp` from source. ## Building the app Clone the repository: ```command git clone https://github.com/andrisgauracs/Apple-Watch-Edge-AI ``` Open `WatchLLM.xcodeproj` in Xcode. ![A screenshot of the project's GitHub repository page.](https://imagedelivery.net/xZXo0QFi-1_4Zimer-T0XQ/52ba3e04-f405-4485-cf4d-3fd940305700/lg1x =1920x1080) The project structure: - `ContentView.swift`: SwiftUI interface - `LlamaCppEngine.swift`: Swift wrapper for the `llama.cpp` C API, handles model loading and inference - `LLMRunner.swift`: Observable object managing app state, prompts, and output - `WatchLLM-Bridging-Header.h`: Exposes C functions from `llama.cpp` to Swift - `vendor/llamacpp/`: Pre-compiled `libllama.a` and headers - `models/`: Where GGUF model files go (the Falcon-H1 model is included) - `tools/`: Build scripts and the tool call configuration file Connect your Apple Watch (paired with your Mac via iPhone), select it as the build target in Xcode, configure your Apple Developer account under Signing & Capabilities, set build configuration to **Release** for performance, and build with ⌘R. ### Tool calls The app supports tool calls: when the model determines it needs external data, it fetches from a configured API endpoint before generating the response. This is configured in `tools/Tools.example`: ```json [label tools/Tools.example] [ { "name": "Weather", "triggers": ["weather", "temperature", "forecast"], "url": "https://api.open-meteo.com/v1/forecast?...", "jsonPath": "current_weather.temperature", "factTemplate": "The current temperature is {value}°C." } ] ``` Rename the file to `Tools.json` to enable it. Each tool definition specifies trigger keywords that cause the model to attempt a tool call, the API endpoint to hit, a JSON path to extract the relevant value from the response, and a template for presenting the value back to the user. Add your own API endpoints by following the same structure. ![A code snippet of the `Tools.example` JSON file, showing the structure for defining tools like "Weather" and "Bitcoin price".](https://imagedelivery.net/xZXo0QFi-1_4Zimer-T0XQ/545a6e5c-84bd-4fce-67cf-6250f6aa9700/public =1920x1080) ## Performance ![A close-up of the Apple Watch displaying performance metrics like "tokens/sec", "peak memory", and "kv cache" overlaid on the generated text.](https://imagedelivery.net/xZXo0QFi-1_4Zimer-T0XQ/f6c67f90-ebd6-404b-1769-a316e03e6900/orig =1920x1080) On an Apple Watch Series 6 with Falcon-H1 90M: **Short factual queries:** The model triggers the Wikipedia tool for "What is the capital of France?" and returns the answer at 24.8 tokens/second. At this rate, responses feel instantaneous. **SmoL M2 (135M parameters) on the same query:** 14.7 tokens/second. Noticeably slower but still comfortable for interactive use. **Long-form generation ("Explain the depth-first search algorithm"):** Starts at around 20 tokens/second and degrades to under 10 tokens/second as the response lengthens. The KV cache for long conversations consumes an increasing fraction of the 1 GB RAM, causing memory management overhead that slows down inference. This is the watch's binding constraint: computation is fast enough, but RAM runs out for long sessions. For short, interactive queries and tool-augmented factual lookups, the experience is genuinely good. For multi-paragraph explanations or extended conversations, you'll notice the slowdown. The 90M and 135M models are genuinely small. They can answer factual questions, follow instructions, and make tool calls reliably, but they won't match a 7B or 13B model on complex reasoning or nuanced writing. The appropriate expectation is a capable, fast, private assistant for short tasks, not a frontier AI.