Running FLUX.1 [dev] on 12GB GPUs: NF4 Quantization & T5 Offloading Guide

When Black Forest Labs dropped FLUX.1 [dev], it instantly set a new benchmark for text rendering, photorealism, and spatial prompt adherence. However, with 12 billion parameters in the transformer backbone plus a massive 4.7B parameter T5-XXL text encoder, standard FP16 requires over 32GB of VRAM.

Using NF4 quantization and sequential CPU offloading, our lab achieved flawless 1024x1024 generation on an ordinary $290 RTX 3060 (12GB) in just 18.4 seconds.

2. Solving the T5-XXL Text Encoder VRAM Hog

The T5-XXL text encoder alone takes 9.5GB in FP16. By offloading T5 to system RAM after text tokenization and running the DiT backbone in NF4 (NormalFloat4), peak VRAM usage stays strictly capped at 11.1 GB.