Zooming without Zooming v2:EvidenceConnect 如何提升可泛化的多模態感知
Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception
EvidenceConnect 透過合成訓練數據,強化模型連結視覺證據的能力,改善空間感知及多模態推理。研究者以該框架生成數據訓練基礎 VLM,取得 Zooming without Zooming v2(ZwZ2),並在細粒度與空間感知等任務上取得提升。驗證方法包括要求獨立 VLM 對參考答案達成一致,以及以區域消融測試剔除不合格問題。
Published on Oct 27
Authors:
,
,
,
,
,
,
,
,
,
,
,
Abstract
EvidenceConnect synthesizes training data to strengthen visual evidence connection, improving spatial perception and multimodal reasoning across diverse tasks.
Multimodal perception forms the foundation of multimodal intelligence. In particular, spatial perception and multimodal long-context understanding are two pivotal tasks that underpin embodied AI and multimodal agents. However, synthesizing targeted, high-quality training data for these tasks remains difficult and costly. Our key insight is that these tasks share an underlying capability, which we call connecting evidence: identifying relevant visual facts, binding them to the correct entities, recovering their relations, and composing them into an answer. Crucially, targeted training data are far easier to synthesize for this capability than for these tasks themselves. To strengthen this capability, we propose EvidenceConnect, an easy-to-implement data synthesis framework that uses zoomed views of related regions within a single image to generate questions requiring comparison, aggregation, or chaining of visual evidence. To ensure quality, a reference answer is retained only when independent VLMs agree on it, and region-ablation tests discard questions whose answer survives the removal of selected regions. We then train base VLMs on the resulting data with RLVR, obtaining Zooming without Zooming v2 (ZwZ2). Although the synthesized data are not tailored to any downstream benchmark, ZwZ2 improves fine-grained and spatial perception, with further gains on embodied spatial reasoning, multimodal long-context understanding, multimodal agent tasks, and general visual reasoning. These results establish EvidenceConnect as an effective framework for supervising evidence connection and suggest that strengthening this shared capability leads to more generalizable multimodal perception.
View arXiv page View PDF Project page GitHub 2 Add to collection
Models citing this paper 3
Image-Text-to-Text • 27B • Updated 27 minutes ago • 7 • 10inclusionAI/ZwZ2-27B
Image-Text-to-Text • 5B • Updated 30 minutes ago • 9inclusionAI/ZwZ2-4B
Image-Text-to-Text • 9B • Updated 28 minutes ago • 147 • 9inclusionAI/ZwZ2-9B
Datasets citing this paper 1
inclusionAI/ZwZ2-RL-VQA
Viewer • Updated about 2 hours ago • 20.7k • 5 • 2
Spaces citing this paper 2
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co