llama.cpp 1.0.0: semantic versioning for libllama and ggml, FP8 KV caches and a safer llama-server

llama.cpp, the C/C++ inference engine underneath a large share of local LLM software, released version 1.0.0 on November 13, together with a 1.0.0 of ggml, the tensor library it is built on. The announcement on the Hugging Face blog puts the project at 3.5 years old, counting from its first commit, with support for more than 150 model architecture variants.

For most users nothing looks different today. The change is a promise about the future: from this release, an incompatible change to the public C API or binary interface of either library has to arrive with a new major version number. For a project that many desktop apps, language bindings and servers link against, that is the difference between pinning a commit hash and declaring a version range.

What the version number now means

Two libraries are covered: libllama, whose public headers live in llama.cpp/include/, and libggml, with headers in ggml/include/. The ggml-org projects now follow semantic versioning with these rules:

• Major: a breaking change to the public C API in the include directory.
• Minor: backward-compatible additions, such as new API functions, backend updates or support for new models.
• Patch: bug fixes with no API change.

Until now llama.cpp was identified mostly by rolling build tags like b12359, which say when a build was cut but nothing about compatibility. Those tags still appear on nightly builds. What is new is a version line on top of them that carries a compatibility meaning. The team writes that it intends to be stricter about making API and ABI boundaries visible.

Two things are worth noting about the scope. The guarantee covers the C headers, so it does not automatically extend to the command-line flags, to llama-server’s HTTP behavior or to the internal C++ code that some downstream projects reach into. And neither the blog post nor the release notes makes a new compatibility statement about the GGUF file format.

What 1.0.0 removes and renames

A 1.0 is the natural moment to delete things, and this one does. The release notes describe a pruning of deprecated functions from llama.h, including llama_set_warmup, and ggml 1.0.0 drops its own deprecated public API. Other breaking changes called out in the notes:

• The defrag_thold context parameter and the --defrag-thold flag are gone.
• clip.h is removed, with its functionality consolidated into the mtmd multimodal library.
• The NextN API moves from the experimental llama-ext.h header into llama.h, and llama_set_embd_state_src is renamed llama_set_ctx_state_src.
• llama_context_params gains moe_cache_size and moe_cache_cpu_share, which changes the struct layout.

The release notes do not include a migration table mapping removed names to replacements, and neither page mentions specific downstream projects. Anyone maintaining a binding should diff the 1.0.0 headers against the version they currently build against instead of relying on the changelog. Code that compiled cleanly with deprecation warnings will now fail to compile, and anything that passes llama_context_params by value across a shared-library boundary needs a rebuild.

New in the release: FP8 caches, MoE expert caching and shards

1.0.0 is also a normal feature release covering everything since 0.9.0.

FP8 KV caches. The key and value caches can now be stored in 8-bit floating point (-ctk and -ctv with f8_e4m3), with optional static per-tensor scales and CUDA support for the OCP FP8 format. The KV cache is what grows with context length, so halving it relative to 16-bit storage directly raises how much context fits on a given GPU. The release notes give no quality measurements, so test on your own workload before making it a default.

MoE expert cache. For mixture-of-experts models, which activate only a few experts per token, llama.cpp can now keep experts in host memory with a configurable CPU share and a cache replacement policy. This targets the situation where a sparse model is far larger than available VRAM but only a slice of it is hot at any time, as with the 594B-parameter Inkling 2 Small we covered in our post on its open weights.

Sharded loading. New --shard and --ple flags load parts of a model from separate GGUF files.

Backends as plugins. ggml backends can be compiled as dynamic libraries and loaded at runtime, which lets a distributor ship one binary and add CUDA, Vulkan, Metal or SYCL support as separate files.

llama-server gets stricter by default

Several server changes read as a response to how llama-server is now used: as a long-running local service with tools attached, reachable from a browser.

• Tool use over non-loopback addresses is now gated behind an explicit opt-in, --allow-unauthenticated-tools.
• Fetching media from private network URLs requires --media-allow-private-urls.
• The release fixes --cors-origins handling, DNS rebinding, and server-side request forgery through the CORS proxy.
• HF_TOKEN is now sent only to huggingface.co hosts.
• MCP tool calls are cancelled when the client disconnects or times out.

If you run llama-server on a LAN address with tools enabled, expect that setup to stop working after the upgrade until you add the opt-in flags. That is the intended outcome: a server that executes tools for any device on the network is a remote code execution service, and DNS rebinding makes even a localhost-only server reachable from a malicious web page. The release also adds an OpenAI-compatible /v1/audio/speech endpoint and a --sleep-idle-seconds option.

What to watch

The value of the promise depends on how often major versions arrive. A project that tracks every new model architecture within days is under constant pressure to change its API, and a 2.0 in three months would make the stability claim thin. The first real test is how the team handles the next change that would break the ABI: with an additive function and a minor bump, or with a new major.

The second thing to watch is distribution. The blog lists packages for Homebrew, conda-forge, Debian, Winget, MacPorts and Docker. Stable ABI versions are what let Linux distributions ship libllama as a shared system library that other packages depend on, instead of every application bundling its own copy. The announcement also says continuous integration now runs on Hugging Face infrastructure, with self-hosted runners backed by hardware from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel, which is the testing base an ABI promise across that many backends needs.