跳到正文
Hacker News 熱門(buzzing.cc 中文翻譯)·· 21 天前AI 評分51

利用 JEPAs 證明並改進柏拉圖表徵假説

利用JEPAs证明并改进柏拉图表征假说

AI 導讀

研究提出柏拉圖表徵定理:若正樣本配對是同一場景的不同觀測,且其他情況下錯位有界,分開訓練的多模態 JEPA 可學得僅相差一次旋轉的相同表徵。研究以 145k 條圖片説明及 940M 張網頁圖片的文本驗證理論,並發現良好視角達到同等對齊所需樣本比下一 token 預測少 42 倍。

正文

View PDF HTML (experimental)

Abstract:Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a \emph{Platonic Representation Theorem}: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene--\emph{good views}--and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with \emph{bad views}, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP's text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64\% against at most 56\% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
Cite as: arXiv:2610.39302 [cs.AI]
  (or arXiv:2610.39302v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2610.39302

arXiv-issued DOI via DataCite

Submission history

From: Randall Balestriero [view email]
[v1] Thu, 29 Oct 2026 03:49:53 UTC (3,665 KB)

來源:Hacker News 熱門(buzzing.cc 中文翻譯) · arxiv.org