AnyLanguageModel 1.0: One Year of Local Models on Apple Platforms

AnyLanguageModel 1.0 is now available alongside swift-huggingface 1.0 and swift-xet 1.0, marking a year of development for local inference on Apple devices.

The library continues to share an API that lets developers select the right model for each task.

A drop-in compatible API

The core function remains unchanged: use the Foundation Models API with whichever model fits the application. To begin, swap the import statement:

- import FoundationModels
+ import AnyLanguageModel
The same session code then works with the system model, open-weight models via llama.cpp, MLX, and Core ML, plus hosted providers.
Major additions over the past year include:
Tool calling works with every provider.
Guided generation works with every provider, including nested @Generable types and arrays of them.
Image support lets users attach images to prompts using Foundation Models 27 syntax, including when using the system model on iOS, macOS, and visionOS 27.
On OS 27, AnyLanguageModel also works with any model conforming to Foundation Models’ LanguageModel protocol, including Core AI models.
Alongside AnyLanguageModel, the team built swift-huggingface and swift-xet to bring accelerated model downloads from the Hub to Swift apps.
Hosted models, too
Support for hosted models expanded this year. AnyLanguageModel now supports:
OpenAI’s Chat Completions and Responses APIs
Open Responses, for any provider with a compatible endpoint
Anthropic’s Messages API, with adaptive thinking and effort options
Google’s Gemini API
Ollama
Private Cloud Compute, on OS 27
Hugging Face Inference Providers, and other OpenAI-compatible endpoints such as OpenRouter, through a custom base URL
With Inference Providers, users can run open-weight models too large for the device with the same session code:
let model = OpenAILanguageModel(
    baseURL: URL(string: "https://router.huggingface.co/v1/")!,
    apiKey: ProcessInfo.processInfo.environment["HF_TOKEN"]!,
    model: "openai/gpt-oss-120b",
    apiVariant: .chatCompletions
)
Hosted providers receive the same features as local models: guided generation, tool calling with streaming, and token usage on responses.
Reasoning is now part of the transcript. When switching providers, reasoning blocks are automatically omitted from requests if the new provider cannot replay them.
API keys come from a closure instead of a string, allowing apps to fetch short-lived tokens from their own server. The README explains when to let users bring their own keys and when to route requests through a proxy.
AnyLanguageModel also runs on Linux, so the same code can work on a server.
Open-weight models got better
While local models became easier to use, the models themselves have become more capable, making small ones a good starting point. Here are some of the most downloaded models on the Hub right now:
Gemma 4 E2B from Google, in GGUF, about 3.4 GB at Q4_0 with quantization-aware training.
LFM2.5 2.6B from Liquid AI, in GGUF, about 1.7 GB at Q4_K_M.
Qwen3.8 27B for MLX from Alibaba, about 16 GB at 4 bits.
The first two fit on an iPhone, and Qwen3.8 27B runs on a Mac with 32 GB of memory or more. Selecting the MLX model takes one line:
let model = MLXLanguageModel(modelId: "mlx-community/Qwen3.8-27B-4bit")
For those with four maxed-out Mac Studios, GLM-5.3 from Z.ai takes about 1.5 TB in BF16. For the rest, it is available through Inference Providers.
Upgrading to 1.0
dependencies: [
    .package(
        url: "https://github.com/huggingface/AnyLanguageModel",
        from: "1.0.0",
        traits: ["MLX"]
    )
]
If upgrading an existing app, check the migration guidance in the AnyLanguageModel, swift-huggingface, and swift-xet release notes for help updating code.
Community feedback and contributions shaped several major features. Thank you to everyone who contributed for helping make AnyLanguageModel more useful to app developers.
What’s next
Language models have proven to be a powerful tool for building intelligence into products. They’re also one of many kinds of models developers can put to work. The team is excited to bring the same approach to more capabilities: familiar Swift APIs with a choice of models running on the device. Coming soon: models that decide, and models that listen.

Source Read original →
Related reading
Meta and Apple team up to make Muse AI models run better on Macs
Cisco Foundation AI Releases Antares: 350M and 1B Open-Weight Models That Localize Known Vulnerabilities Inside Real Codebases
Nativ: Run AI models locally on your Mac