LILIE:用於視覺生成的語言資訊損失函數
A Language-Informed Loss for Visual Generation
LILIE 以可逆正規化流編碼器為文字生成圖像提供融入語言資訊的損失,提升相對於純像素訓練的保真度。在 GPIC 數據集上,使用 LILIE 損失訓練的 JiT 模型將 gFD 從純像素基線的 66.84 降至 53.01,改善 14 分。其程式碼、編碼器及生成模型均已提供。
Published on Oct 16
Authors:
,
,
,
,
,
Abstract
LILIE uses an invertible normalizing-flow encoder to derive language-informed supervision for text-to-image generation, improving fidelity over pixel-only training.
End-to-end visual generation with pixel-space loss has recently achieved competitive performance without the need for learned image tokenizers. In this work, we study whether language-informed supervision can provide a richer training signal than pixel loss for visual generation. We introduce LILIE (pronounced "lily"), a Language-Informed and Lossless Image Encoder. LILIE is an invertible image transform trained such that a vision-language model can recover captions from noised features. The encoder uses a normalizing flow transformer trained jointly with a vision-language model via next token prediction. We then use the encoder to extract a simple language-informed loss for text-to-image generation. We train JiT models with LILIE loss, reducing gFD by 14 points (66.84 to 53.01) against a pixel-only baseline on the GPIC dataset. Our code, encoders, and generative models are available at https://suchiragarwal.com/lilie.
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2610.23653
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 1
Updated 1 day ago • 2suchiragarwal/lilie
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.23653 in a dataset README.md to link it from this page.
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co