llama.cpp v1.0.0 : A stronger foundation for the local AI ecosystem

The llama.cpp project has released version 1.0.0. This update formalises the API and ABI stability for the library, ensuring that breaking changes will only occur with a major version bump.

The software allows multi-modal large language models to run on consumer hardware and enterprise servers alike. It achieves this through quantisation, optimised backends, and a portable C/C++ implementation. The team describes the platform as the most lightweight and sensible option for building cross-platform and private AI products.

Three and a half years since the project began, many downstream applications already depend on it. The team expects this trend to continue as local AI and privacy become more important. Every new integration increases the number of developers relying on the library interfaces, making a flexible but stable API essential.

Releases

Two libraries have reached version 1.0.0:

• llama.cpp v1.0.0: Provides LLM inference in C/C++ via the libllama library and public headers in llama.cpp/include/.
• ggml v1.0.0: A tensor library for machine learning provided via libggml and public headers in ggml/include/.

Semantic version update rules

The following rules apply to all ggml-org projects using semantic versioning:

• MAJOR: A breaking change to the public C API in ./include.
• MINOR: A backward-compatible feature, backend updates, model support, or API addition.
• PATCH: A bug fix with no API change.

Distribution

The team is expanding the ways in which ggml-org software is distributed to developers and end-users. Information on adding ggml-org projects to your packaging and release pipelines is available in a discussion thread.

Available channels include:

• Pre-release artifacts: Linux, macOS, Windows, Android, iOS
• Docker: Linux
• Debian: Linux
• Homebrew: macOS, Linux
• conda-forge: Windows, macOS, Linux
• MacPorts: macOS
• Winget: Windows
• XCFramework: iOS, macOS

Development

The project has fostered an open-source community with highly-skilled contributors. With the rapid advance of AI capabilities in software engineering, the team continues to adapt and integrate LLMs into its development cycle. Maintaining a high-quality codebase, controlling the architecture, and understanding implementation details remain top priorities.

To support a faster pace of development, the team is investing more resources in the CI infrastructure behind the projects. Using HF technologies such as HF Buckets and HF Jobs, they scale capacity to guarantee correctness for every change landing on the master branches. Hardware support from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel enables the team to enrol more self-hosted runners across major platforms.

Backend and model coverage

The underlying machine learning engine of llama.cpp, ggml, is designed to run on virtually any hardware. This is achieved by implementing dedicated ggml backends. Each backend can be compiled as a dynamic library and optionally loaded at runtime. A complete list of implemented backends is available online.

The team is committed to providing continuous support for all model architectures. Currently, llama.cpp supports more than 150 architecture variants as well as all mainstream speculative decoding methods.

Thanks

The llama.cpp team at Hugging Face is thankful for the incredible support from the community. Special thanks go to the long-time maintainers of the project and all contributors. This milestone would not have been possible without joint efforts.

Additional resources

• llama.cpp examples
• ggml-org at Hugging Face
• ggml-org at Github
• Llama desktop apps for MacOS and Windows, shipped by the team behind llama.cpp and Hugging Face.