FP8 Cache Quantization Doubles Kimi K2.6 Context on Cloudflare

On August 3, 2026, Cloudflare rolled out a suite of backend optimizations for its Workers AI platform designed to serve large frontier models like Moonshot AI’s Kimi K-series and Z.ai’s GLM at scale. The updates target GPU memory bottlenecks inherent in large Mixture-of-Experts architectures. By modifying how model memory is stored and verified, the company expanded the maximum context window for Kimi K2.6 from approximately 686,000 to 1.37 million tokens without requiring additional hardware.

KV Cache Quantization

The core of the context expansion relies on upstream patches Cloudflare contributed to the SGLang inference framework. Cloudflare transitioned the platform’s KV cache storage from the default 16-bit precision (BF16) to an 8-bit floating point format (FP8, e4m3).

This quantization halves the memory footprint required to store past tokens. If you manage long-context applications, this is where the hardware tradeoff usually hits. Mixture-of-Experts models require massive memory reserves just to load their parameters, leaving little VRAM for the cache. Shrinking the cache allows the GPU to hold double the context history, directly enabling Kimi K2.6 to reach its 1.37 million token capacity on existing infrastructure. You can learn more about how memory reduction works in What Is Quantization in AI?.

Multi-Tenant Hardware Density

Alongside cache quantization, Cloudflare implemented model weight compression to pack more concurrent requests onto shared hardware. This lowers the base cost of serving resource-intensive models like GLM-5.2 and Kimi K2.7 Code.

Higher concurrency in shared environments creates security risks. To safely route requests through the shared cache, Cloudflare deployed strict integrity checks. These checks isolate memory segments per user, ensuring data from one prompt does not bleed into a concurrent request running on the same GPU.

These changes operate on top of Cloudflare’s existing prefill and decode disaggregation layer. Separating the initial prompt processing from token generation maximizes utilization across the fleet, a necessary step for what AI inference is becoming at the edge.

Capacity and Pricing Adjustments

The deployment coincides with Cloudflare’s “Agents Week” and follows recent tier adjustments for frontier models. As of July 28, 2026, models including @cf/moonshotai/kimi-k2.6, @cf/moonshotai/kimi-k2.7-code, and @cf/zai-org/glm-5.2 require the Workers Paid plan, which starts at $5 per month.

This paywall manages capacity for the most memory-intensive models, prioritizing performance for paid developers. The broader release cycle also included a new agent runtime for transitioning between fast isolates and full Linux containers, direct inbound TCP/gRPC support for Workers, and a Billable Usage API to track programmatic costs.

Developers building multi-turn workflows should monitor cache hit rates on the updated SGLang backend. Adjusting your request concurrency to leverage the new FP8 cache limits will reduce latency for long-context applications running on the paid tier.