Apfel: How to Use macOS's Built-In LLM from the Terminal

Apple Silicon Macs running macOS 26 (Tahoe) ship with an on-device language model as part of Apple Intelligence. It's a roughly 3 billion parameter model that runs on the Neural Engine, works offline, and never sends data to a third party. The catch is that it's only accessible through Apple's FoundationModels Swift framework, which means building a native macOS app just to use it. Apfel is an open-source tool that removes that barrier, exposing the model through a CLI, an interactive chat mode, and an OpenAI-compatible local server.

What Apfel is and isn't

Apfel is not a model. It's a small Swift binary that wraps the FoundationModels framework and translates between plain terminal usage and Apple's native API. Because the model itself is already on disk as part of macOS, installing Apfel doesn't involve downloading gigabytes of weights. You're only adding the wrapper.

The three interfaces it exposes are a UNIX CLI for scripting and quick queries, an interactive chat mode for conversational use, and an HTTP server that mimics the OpenAI API so existing SDKs can point at it without code changes.

Requirements and installation

You need an Apple Silicon Mac (M1 or newer) running macOS 26 with Apple Intelligence enabled. Homebrew must be installed.

Installation is a single command:

brew install apfel

It completes in seconds because only the small binary is being downloaded.

Basic CLI usage

The --stream flag prints responses token by token as they're generated, which makes longer answers readable as they arrive:

apfel --stream "What can you do as a language model?"

Because the model runs locally, it works with Wi-Fi off. Prompts and responses stay on your machine and never reach Apple's servers or any third party.

Scripting and file input

Apfel behaves like a standard UNIX command, which means its output can be piped or redirected. Generating a file directly from a prompt:

apfel "Write a simple Python function that simulates a basic calculator for a list of string operations" > calculator.py

The -f flag passes a file's contents as context alongside the prompt. To ask the model to review the file just generated:

apfel -f calculator.py "Review this code and suggest two improvements"

The model reads the file contents and responds in that context. This is useful for quick code reviews, generating docstrings, or asking questions about an existing script.

The OpenAI-compatible server

The most useful feature for application development is the local server. Starting it:

apfel server

By default it listens on http://localhost:11434, the same port Ollama uses, which makes switching between them straightforward.

With the server running, any code that uses the OpenAI Python SDK can target it by changing base_url. The API key field is required by the SDK but ignored by Apfel, so any string works:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="apple-foundation-model",
    messages=[
        {"role": "user", "content": "Explain what Apfel is in one sentence."},
    ]
)

print(response.choices[0].message.content)

With the server running in one terminal, running the script in another produces a response from the on-device model with no API key, no internet connection, and no token cost. When moving to production, changing base_url back to the real OpenAI endpoint and adding a valid key is the only required change.

Apfel vs. Ollama

Both tools provide an OpenAI-compatible local server, but they serve different purposes.

Apfel is a wrapper around a single fixed model that Apple ships with macOS. There's nothing to configure or download, no model choice, and no flexibility in how the model runs. It's instant to set up and costs nothing in disk space beyond the few-megabyte binary.

Ollama is a platform for downloading and running a library of open-weight models. You can choose between Llama 3, Mistral, and many others, configure quantization, and write custom model definitions. It's cross-platform and gives you much more control, at the cost of needing to download multi-gigabyte model files and manage them over time.

Limitations

The model is roughly 3 billion parameters. It handles straightforward language tasks well but struggles with complex multi-step reasoning, mathematics, and nuanced instructions that larger models manage without difficulty.

The context window is 4,096 tokens, covering both input and output combined. That's approximately 3,000 words total, which rules it out for long documents, large codebases, or extended conversations. This is the most significant practical constraint for typical developer use cases.

Apple's content filters apply and can't be fully disabled. Apfel includes a --permissive flag that reduces false positives, but you don't have the same level of control as you would with an open model running through Ollama.

The tool is also Apple Silicon only. There's no Linux or Windows equivalent, and the underlying model is updated on Apple's schedule rather than yours.

Given those constraints, Apfel is well suited for high-frequency, low-complexity automation tasks: generating shell commands, writing commit messages from git diff output, summarizing small text blocks, classifying content, and prototyping AI application logic before connecting to a paid API. For anything requiring deep reasoning, long context, or a specific model capability, a cloud API or Ollama with a larger model is the better choice.

Got an article suggestion? Let us know

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.