Nunchaku Lite Brings 4-bit Diffusion Inference Natively to Diffusers

Hugging Face has integrated Nunchaku's SVDQuant-based 4-bit weight-and-activation (W4A4) quantization directly into Diffusers through a new path called Nunchaku Lite. Pre-quantized checkpoints can now be loaded with a standard from_pretrained() call, no separate inference engine or local CUDA compilation required, since kernels are fetched from the Hub via the kernels package at runtime.

Under the hood, Nunchaku Lite patches a stock model's nn.Linear modules with runtime SVDQ/AWQ layers: svdq_w4a4 (NVFP4 or INT4) handles the compute-heavy attention and MLP projections, while awq_w4a16 covers precision-sensitive normalization and modulation layers. Because the module structure stays identical to the dense model, downstream Diffusers features like LoRA loading, offloading, and torch.compile continue to work unchanged.

Without architecture-specific fused kernels, Nunchaku Lite can't match the original Nunchaku engine's peak speed, but it still delivers roughly a 30% speedup with the same VRAM savings, rising to 1.8x with torch.compile. On an RTX 5090, a 1024x1024 image generates in about 1.7 seconds using ~12 GB of memory versus ~24 GB for BF16. A companion diffuse-compressor toolkit also lets developers quantize new architectures themselves and publish them as standard Diffusers repositories.