Apple Publishes the Recipe for Desk-Side Frontier Inference: Muse Spark Open at 36.7 Tokens Per Second on Four Mac Studios

Apple’s machine learning research team published a study on October 15 that turns Muse Spark Open’s wildest launch claim into a documented recipe: the 816B-parameter model (42B active, quantized to NVFP4) runs across one to four M5 Ultra Mac Studios linked by Thunderbolt 5, reaching about 36.7 tokens per second on four machines, faster than human reading speed, with tools any developer can run (mlx.launch, mlx_lm.benchmark, mlx_lm.server). The study, Local and Distributed LLM Inference with MLX and RDMA on Mac Studio, is Apple publishing the implementation manual for private, desk-side frontier inference.

The Numbers: A Frontier Model on a Desk

The configuration: Muse Spark Open needs more than 1.6TB in BF16, which is why even a 512GB M5 Ultra cannot hold it alone; quantized to NVFP4 it needs about 493GB, split across machines. The results table: at 8,192-token context, tensor parallelism across four M5 Ultras delivers 36.7 tokens per second, 25.0 on two-machine tensor, 15.7 on two-machine pipeline. Time-to-first-token on a single M5 Ultra is up to 4.6 times faster than the prior M3 Ultra (compute-bound, thanks to the new GPU Neural Accelerators), while decode is about 1.6 times faster (memory-bandwidth bound). Peak power draw is about 300W per machine. Two Thunderbolt 5 cables per link, using a new macOS 26.2 RDMA driver, cut four-machine TTFT at 65,536-token context from 50.5 seconds to 40.6.

Pipeline Versus Tensor: The Trade-Off Table

The study’s technical contribution is the comparison of the two parallelism strategies on consumer-adjacent hardware. Pipeline parallelism (model split by depth across two machines) is simple to implement and leaves decode speed essentially unchanged, but chunked 8,192-token prefill overlap makes very large contexts up to 1.8 times faster than a single node. Tensor parallelism (model split by width across two or four machines) requires MLX’s low-latency “jaccl” collectives over the new RDMA driver and delivers the throughput numbers above. The choice between them is now a documented engineering decision with measured costs, which is what turns a research demo into infrastructure.

What It Validates About Muse Spark Open’s Launch Claim

Muse Spark Open launched on October 15 claiming to be the first model in its class to run fully on two Apple M5 Ultra Mac Studios. Apple’s research, published the same day, is the independent verification of that claim’s difficulty curve: the memory math (493GB quantized against 512GB per machine), the interconnect requirement (two cables per link for doubled bandwidth), and the concrete throughput that two versus four machines deliver. The claim was true, and now the recipe to reproduce it is public, including the experimental CPU-GPU sync flag that reduces communication overhead. The local-inference wave now spans ds4 and Strata on gaming PCs, and Apple-published MLX recipes on Mac Studios: the frontier tier is reachable from a desk.

What to Watch

Three things. First, whether Meta’s hosted-API pricing survives the desk-side alternative: 36.7 tokens per second of private, 816B-class inference at roughly 300W per machine is a real option for anyone with privacy constraints, and the local-versus-API debate now has a frontier-tier data point. Second, whether the RDMA-over-Thunderbolt driver generalizes beyond Mac Studios to other Apple silicon topologies, since the two-cables-per-link mesh is the enabling trick. Third, the agentic integration angle the study itself mentions: local servers speak OpenAI-compatible APIs and integrate with coding agents like OpenCode, which means private agentic AI without cloud infrastructure is now a documented, reproducible setup rather than an aspiration.