Native W4A4 Inference Arrives in Diffusers via Nunchaku

On July 23, 2026, Hugging Face announced the official integration of the Nunchaku inference engine into the diffusers library. This update brings native 4-bit weights and activations (W4A4) inference to large diffusion transformers. You can now execute models without manual CUDA compilation or external libraries. For developers building systems for local inference, this removes the severe VRAM bottlenecks previously associated with high-parameter image models.

SVDQuant and W4A4 Execution

The integration relies on SVDQuant, a post-training quantization method developed by the MIT HAN Lab. 4-bit approaches often struggle with outliers that degrade output quality. SVDQuant isolates these outliers into low-rank components, allowing the primary network to operate efficiently.

Weight-only backends like GGUF or bitsandbytes dequantize weights to higher precision during computation. Nunchaku executes the transformer layers strictly in 4-bit. This is where the tradeoff matters. By avoiding continuous dequantization overhead, the engine maximizes throughput and keeps the memory footprint constrained during active generation.

FLUX.1-dev Performance

The hardware efficiency gains are most visible on 12-billion-parameter models like FLUX.1-dev. Standard BF16 precision typically requires 24 to 30 GB of VRAM, forcing CPU offloading on consumer hardware. Nunchaku drops this requirement to approximately 6.7 GB.

On a 16GB GPU, this memory reduction yields up to an 8.7x speedup over BF16 baselines by entirely avoiding the PCIe bus bottleneck. In practical terms, an NVIDIA RTX 4090 Laptop GPU can generate a complete FLUX.1-dev image in roughly 7 seconds. The engine operates up to 3.0x faster than existing weight-only 4-bit implementations.

Hardware Support and Custom Models

You can load quantized models using the standard .from_pretrained() method within diffusers version 0.30.0 or higher. The underlying operations are handled by a companion nunchaku-kernels package, which provides pre-compiled binaries for both Linux and Windows environments.

The engine supports standard INT4 instructions for NVIDIA Turing, Ampere, and Ada architectures. If you deploy on Blackwell (50-series) hardware, Nunchaku utilizes the new NVFP4 format to maintain floating-point precision at extreme low bitrates.

Hugging Face also released Nunchaku Lite models, which provide ready-to-use optimized versions of FLUX.1-dev, SANA (1.0-1600M), and Qwen-Image-Edit-2509. If you build custom architectures, you can use the included diffuse-compressor toolkit to map your own pipelines into the Nunchaku format and publish them directly to standard model repositories.

If you provision GPU instances for image generation APIs, the transition to native W4A4 execution fundamentally alters your density calculations. Update your diffusers dependencies and evaluate the Lite models to determine if the 4-bit accuracy holds up against your specific prompt distributions.