LFM2.5-DSpark Delivers 3.18x Faster Decoding on H100

Liquid AI released LFM2.5-DSpark on August 20, 2026, a family of open-weight draft models that accelerates decoding for three LFM2.5 targets. The release reaches up to 3.18x the baseline throughput on NVIDIA H100 GPUs and up to 2.87x on Apple Silicon, while preserving the target model’s output sequence under greedy generation.

For developers building local agents, tool-use systems, or latency-sensitive inference services, the change targets the memory bandwidth bottleneck that often dominates token-by-token generation. The technique adds a lightweight drafter beside the target model, allowing the target to verify several predicted tokens in one parallel forward pass.

Three target models and their drafters

The release provides dedicated checkpoints for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. Each drafter is a simplified attention-only model with five layers, a nine-token block size, a hidden size of 2,048, an intermediate size of 6,144, and Grouped-Query Attention with 32 attention heads and eight Key-Value heads.

The additional parameter cost is 295.7M parameters for the LFM2.5-1.2B-Instruct drafter and 327.7M parameters for the drafters paired with the other two targets. The drafter therefore consumes extra memory, but its predictions can reduce the number of expensive sequential passes through the larger model.

Target model	Drafter parameters	Hardware result	Mean or peak speedup
LFM2.5-2.6B	327.7M	H100, BF16, SGLang	2.67x mean
LFM2.5-2.6B	327.7M	Apple M4 Max, FP16 GGUF	2.27x mean
LFM2.5-8B-A1B	327.7M	H100, BF16, SGLang	3.18x peak
LFM2.5-8B-A1B	327.7M	Apple M4 Max, FP16 GGUF	1.18x mean

The measurements used single-stream latency conditions with batch size 1 and temperature 0. On H100, LFM2.5-2.6B increased from 323 tokens per second to 864, while LFM2.5-8B-A1B rose from 428 to 1,362 tokens per second on MATH500. The M4 Max results were 61 to 139 tokens per second for LFM2.5-2.6B.

How DSpark verifies draft tokens

DSpark combines three mechanisms. A parallel backbone generates hidden states for a candidate block in one pass, using features from the target model’s context. A sequential Markov head adds dependencies between neighboring draft tokens, improving acceptance later in each block. A confidence-scheduled verifier estimates token survival probabilities and prunes low-confidence suffixes when additional verification would cost more than it saves.

Lossless rejection sampling gives the acceleration an important property for deterministic workloads. When generation is greedy, accepted and rejected draft tokens resolve to the same output sequence as the unaccelerated target model. Liquid AI reports zero accuracy loss on MATH500 and the Berkeley Function Calling Leaderboard.

This distinction matters for production evaluation. The speedup comes from reducing target-model decode work, not from accepting approximate outputs. You should still benchmark your own prompts, sampling settings, and output lengths because the reported gains depend on draft-token acceptance and hardware execution paths.

Integration and agent workloads

Day-one support landed in llama.cpp through native DSpark GGUF sidecar support and in SGLang. Checkpoints are available as Safetensors and GGUF files, including Q4_K_M and F16 variants. Developers deploying local models can therefore evaluate the method alongside existing quantization and runtime choices. The distinction between compute and memory bottlenecks is also central to AI inference, particularly for batch-one applications.

Tool-use workloads benefit from shorter sequential decode loops. On LFM2.5-2.6B, Liquid AI measured a 57% average reduction in function-calling latency across multi-tool sequences. For agents that repeatedly generate a tool call, wait for its result, and continue decoding, lower token latency can shorten every loop iteration. This is directly relevant to systems built around function calling and local execution.

The checkpoints use the LFM Open License v1.0. Commercial use, downloading, and fine-tuning are free for companies with less than $10 million in annual revenue. Larger entities require an enterprise commercial license.

If you run an LFM2.5 workload, test the matching DSpark checkpoint in llama.cpp or SGLang with batch-one traces from your application, then measure end-to-end latency rather than tokens per second alone. Include drafter memory, tool wait time, acceptance behavior, and your target hardware in the comparison.