Black Forest Labs has released FLUX.3 Image Klein, available in Diffusers from day one. A single model covers text‑to‑image, multi‑reference image‑to‑image, and introduces explicit bounding‑box layout control: you can place boxes in the prompt to dictate where parts of the output should appear, or which parts of a reference image should move.
Checkpoints and defaults
The release includes two public checkpoints in the "FLUX.3 Image Klein" collection:
- black-forest-labs/flux-3-image-klein — a time‑step distilled checkpoint for fast inference; defaults: 16 steps, no classifier‑free guidance (CFG).
- black-forest-labs/flux-3-image-klein-base — the base model for fine‑tuning; defaults: 50 steps, CFG.
The distilled checkpoint produces 1024 × 1024 images quickly: on an NVIDIA H100 (80 GB) the reported end‑to‑end text‑to‑image time is about 1.93 seconds (16 steps). Image‑to‑image with a 1024×1024 reference takes roughly 3.87 seconds, roughly doubling the runtime.
Layout control (bounding boxes)
Layout control is provided by appending a JSON list of regions to the caption. Each region includes id, bbox and desc; the caption then references the id. Boxes are specified as [y_min, x_min, y_max, x_max] on a normalized 0–1000 scale so the same box can apply at different output sizes. For edits, regions can state a source bounding box in a reference image (src_bbox) and a target bounding box in the output (tgt_bbox); if the two boxes match, the region remains in place.
Getting started with Diffusers
FLUX.3 ships as a Modular Diffusers pipeline. The quick start requires up‑to‑date diffusers and transformers (Qwen3‑VL is used as the text encoder). Examples in the docs show loading the distilled checkpoint and calling pipe(prompt="a red fox") to save an image.
Multiple references and prompt syntax
When passing several reference images, refer to them in the prompt as <ref_image_0>, <ref_image_1>, … in the same order they are passed. The prompt should describe the desired target image and the intended changes. Examples include composing a fox on a bench from two references and moving elements inside a single reference using src_bbox/tgt_bbox edits.
Single‑file and quantized variants
Black Forest Labs provides the transformer as a single file in bf16 and two quantized formats: fp8r (rowwise FP8) and nvfp4. These can be loaded with Flux3Transformer2DModel.from_single_file, then used to update the pipeline components. The fp8r variant targets GPUs with compute capability 8.9+ (Ada, Hopper, Blackwell); nvfp4 is intended for Blackwell with flashinfer integrations.
LoRA fine‑tuning
FLUX.3 supports both text‑to‑image and image‑to‑image LoRA fine‑tuning. diffusers ships DreamBooth LoRA training scripts for both tasks. The documentation includes a tarot‑style example using the multimodalart/1920‑raider‑waite‑tarot‑public‑domain dataset and shows common settings (mixed_precision=bf16, resolution=1024, train_batch_size=1, max_train_steps=1500). Memory‑saving options include --gradient_checkpointing, --cache_latents and --use_8bit_adam.
Performance and memory
Reported peak memory (after load / after inference) for variants:
- nvfp4 (RTX6000): 13.4 GiB / 15.3 GiB
- fp8r (H100): 14.35 GiB / 16.27 GiB
- bf16 (H100): 18.80 GiB / 20.72 GiB
End‑to‑end timings for the distilled checkpoint (16 steps) on a single NVIDIA H100 80GB (no offloading):
- Text‑to‑image 1024×1024: 1.93 s, peak memory 23.8 GiB
- Image‑to‑image with a 1024×1024 reference: 3.87 s, peak memory 23.8 GiB
- Layout editing (one 1360×768 reference): 4.10 s, peak memory 23.8 GiB
If you have less memory, options include loading a quantized transformer, running the text encoder separately and freeing it ("encode once, generate many"), or enabling automatic CPU offload for components.
Model architecture, briefly
FLUX.3 is a latent‑space flow‑matching model: the denoiser maps latents to latents and a VAE decodes latents to pixels. A 1024×1024 image becomes a 64×64 grid of 4,096 tokens (each token covers a 16×16 pixel patch). The text encoder is Qwen3‑VL; FLUX.3 reads hidden states from eight of its 36 layers and stacks them before projecting to the transformer width.
The transformer processes tokens in two stages: first per‑mode blocks (separate blocks for text, noisy image, references), then joint blocks where the concatenated sequence [text | image | reference] undergoes shared blocks. This separation lets each modality refine independently first, then combine cross‑modal information.
Position information uses rotary embeddings (RoPE) across four axes: timestep, height, width and sequence length, which allows precise alignment between reference and output pixels — the mechanism behind src_bbox/tgt_bbox edits. The distilled checkpoint is a flow map that predicts velocities during generation; a large time shift keeps noise high for most steps (44 of 50 steps above σ = 0.5 in the base schedule).
Modular Diffusers and extensibility
The pipeline is modular: you can inspect workflows without loading weights, remove or replace blocks, precompute prompt embeddings, or insert custom blocks (for example to record intermediate x0 previews during denoising). The docs include examples for these workflows and for adding custom steps.
Try it and share
Models and documentation are available on the Hugging Face Hub in the FLUX.3 Image Klein collection. The project provides examples for prompting, LoRA training, and layout editing. Authors invite users to share LoRAs and pipelines they build with FLUX.3 on the Hub.
