Text-to-Image Generative Modeling Without Language Pretraining
Text-to-Image Generative Modeling Without Language Pretraining
Halcyon 證明,多模態自回歸模型只用圖文配對資料從零開始訓練,無需大規模語言預訓練,也能實現高質素文字生成圖像。Halcyon-0.52B 以少 18.2 倍的預訓練計算量,取得優於 LlamaGen-XL/8 的 gFD 評分;Halcyon-1.6B 亦以少 275 倍的計算量優於 JiT-T2I-XXL/8++。研究亦發現,純文字預訓練不會提升文字生成圖像效果。
Published on Oct 19
Authors:
,
,
,
,
,
,
Abstract
Halcyon demonstrates that strong text-to-image generation can be achieved by training multimodal autoregressive models from scratch on image-text pairs without relying on large-scale language pretraining.
Modern text-to-image (T2I) systems use internet-scale language pretraining for text-conditioned image generation. We ask whether language pretraining is necessary for strong T2I generation. We introduce Halcyon, a family of early-fusion multimodal autoregressive models pretrained entirely from scratch on GPIC, a dataset of 100M image-text pairs. Halcyon-0.52B outperforms LlamaGen-XL/8 (48.0 vs. 63.3 gFD) with 18.2times less pretraining compute. Further, Halcyon-1.6B outperforms JiT-T2I-XXL/8++ (35.3 vs. 41.0 gFD) with 275times less pretraining compute. Controlled experiments show that text-only pretraining does not improve T2I generation. Halcyon also learns image captioning by replacing caption dropout with I2T supervision, maintaining competitive T2I generation quality under the same pretraining budget. We show that strong T2I systems can be trained from scratch without internet-scale language pretraining. Code and models are available at https://cs.stanford.edu/~keshik/halcyon
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2610.26863
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.26863 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.26863 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.26863 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co