Cerebras Explains How 59 Wafers Serve a 3T Model at 2,000 Tokens Per Second

Cerebras published an engineering deep-dive on October 22 explaining how it serves trillion-parameter models on wafer-scale hardware, and it is the clearest public documentation yet of an inference architecture that is genuinely different from GPU serving. The worked example is Kimi K3, the ~3-trillion-parameter open model: at 4-bit weights with an 8-bit KV cache, the model needs about 1.5 TB just for parameters, split across 93 layers at roughly 16 GB per layer. A single Cerebras WSE-3T wafer carries 44 GB of on-chip SRAM, so the mapping lands on 59 wafers: one per dense layer (dense attention needs KV cache headroom), two per linear-attention layer (fixed-size state, no cache to store), plus compute wafers, with tokens streaming through at 1,000 to 2,000+ per second.

Weights stay still; activations move

The architectural inversion is that weights never move. On GPUs, models are sharded across accelerators and activations bounce between them over NVLink; on Cerebras, each wafer holds entire layers and the runtime routes activations between wafers over RDMA at Ethernet speeds, about 2 microseconds per hop. Serving behaves like an assembly line: wafer 1 runs layers 1-2 and passes activations to wafer 2, and so on until a token emerges. Concurrency works differently too: rather than waiting for a batch to complete, different requests occupy different pipeline stages simultaneously, and stages can take micro-batches, so requests in flight scale with stages times batch size. The concurrency ceiling is set by leftover SRAM on the dense-layer wafers: 44 GB minus the 16 GB layer, divided by per-request KV cache at 256k-token request sizes. Cerebras argues the honest comparison metric against GPUs is total token throughput per megawatt at equal generation speed, not peak FLOPS.

Where this fits in the inference-hardware month

The post lands in the middle of an unusually loud month for inference infrastructure: Intel’s D-series pitching memory-dense PCIe cards, DeepInfer open-sourcing the serving stack behind one of the busiest APIs, and open-weights models growing to sizes that stress conventional serving. Cerebras’s wafer-scale pipeline is the maximal answer to the constraint all three are circling: for trillion-parameter models, weights are the memory problem, and a device with 44 GB of on-chip SRAM per wafer makes weights stationary, which is why latency does not degrade the way GPU batching does under load. The Kimi connection is also strategic: Cerebras cannot outspend Nvidia on general-purpose silicon, but for the specific workload of serving huge open models fast, the wafer is purpose-built, and Kimi K3’s 3T-class open release is exactly the workload.

What to watch

Three questions will test the economics. First, utilization: 59 wafers is a large fixed deployment, and the pipeline model earns its keep only when request volume keeps stages full; sparse-traffic deployments waste stationary silicon. Second, whether the tokens/sec/megawatt metric gets adopted, since Cerebras benefits enormously from moving the comparison away from per-accelerator benchmarks; the month’s pricing war already pushed buyers toward cost-per-task, and energy per token is the next abstraction. Third, quantization dependence: the whole mapping assumes 4-bit weights and 8-bit KV, so quality-at-quantization becomes the load-bearing claim, and independent evaluations of heavily quantized 3T models are still scarce. The wafer-scale bet has been a decade in the making; this post is the first time the deployment math has been public, and the math finally closes.