跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 19 天前AI 評分41

NitroFusion2 將對抗式損失引入多模態 Diffusion Transformer 的文字流

NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers

AI 導讀

NitroFusion2 在多模態 Diffusion Transformer 中,使用凍結 MMDiT 的中間文字 token 狀態作為判別器表示,提出 Text-Stream Adversarial Distillation。

正文

Hi, first author here.

In an MMDiT, text and image tokens share every attention layer, so by mid-network the text tokens have already read the image. We use these intermediate text-token states of a frozen MMDiT as the discriminator's representation (Text-Stream Adversarial Distillation). In controlled ablations, text-stream discrimination outperforms conventional image-stream discrimination, and combining both streams is best on most metrics.

Because the discriminator reads the text stream, its text condition also steers what it judges: swapping the cue (typography, objects, style, …) turns it into an expert on that aspect, with aspect-specific gains (Aspect-Cued Discrimination).

Built on Z-Image-Turbo, NitroFusion2 brings few-step distillation from 4–8 steps down to 1–2 steps.

Questions and feedback welcome!
If you find NitroFusion2 interesting or useful, an upvote here or a star on GitHub helps more people find it. Thank you🙏

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co