Text encoder

last edited

A text encoder is the language model that turns a prompt into embeddings the image model can condition on. It sits between the text and the diffusion model. Its training data and architecture set the limits of what the conditioning signal can carry.

Text encoders

Stable Diffusion 1.5 and 2.0 relied on a single CLIP-L encoder. CLIP was trained on image-caption pairs, as described in the original CLIP paper from 2021. That pretraining ties the encoder to short captions. Stable Diffusion 3 moved to three encoders, FLUX to two, and a newer model to four.

T5-XXL comes from the T5 text-to-text architecture. It serves as a language-model option in those designs.

Newer systems use decoder language or vision-language models. A compact FLUX 3 image model reads its text encoder at eight of its 36 layers, not only the last layer. The FLUX 3 Action Base model card lists an unmodified instruct vision-language model as its shared text encoder. The FLUX 3 family was announced in July 2026.

Customization

An embedding (textual inversion) is a learned pseudo-word in the encoder's input space. It extends the vocabulary with a concept the encoder lacked.

Related methods that bypass text appear in core techniques. Stable Diffusion's history is covered in Stable Diffusion.

Sources