OmniCompass:全模態影片定位與理解綜合基準
OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding
OmniCompass 是一套評估全模態影片定位與理解的綜合基準,設有 OmniCompass-VL 和 OmniCompass-AV 兩條評測軌道,涵蓋時間定位、空間定位及問答。
Published on Oct 13
Authors:
,
,
,
,
,
,
Abstract
OmniCompass is a holistic benchmark for omni-modal video grounding and understanding that evaluates vision-language and audio-visual reasoning through temporal grounding, spatial grounding, and question answering tasks.
With the rapid development of multimodal large language models (MLLMs), evaluating their video understanding capabilities has become increasingly important. Existing video understanding benchmarks cover only part of the evaluation dimensions, limiting systematic diagnosis of omni-modal video understanding. We introduce OmniCompass, a holistic benchmark for omni-modal video grounding and understanding. OmniCompass defines two evaluation tracks: OmniCompass-VL assesses vision-language models on tasks requiring visual evidence, while OmniCompass-AV assesses omni-modal models on tasks requiring joint audio-visual evidence. Each track includes three complementary tasks: temporal grounding (TG), spatial grounding (SG), and question answering (QA), assessing when a referenced target appears, where it is, and what happens. QA covers five perception and three reasoning categories in OmniCompass-VL, and three perception and two reasoning categories in OmniCompass-AV. OmniCompass-VL contains 820 videos and 2,460 instances across domains and three duration types: short, medium, and long. A narrative- and audio-rich subset forms OmniCompass-AV with 240 videos and 720 instances. Each instance is evaluated under two paired settings: text-only and text+image, specifying the same target through text and visual-language guidance. All instances are manually annotated with multi-round quality verification. Evaluations of 50 MLLMs show that open-source models still lag behind proprietary counterparts, particularly in audio-visual understanding. Precise temporal and spatial grounding remains challenging even for leading models, and their predictions still exhibit inconsistencies across the paired target specifications.
View arXiv page View PDF Project page GitHub 54 Add to collection
Get this paper in your agent:
hf papers read 2610.18829
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.18829 in a model README.md to link it from this page.
Datasets citing this paper 1
VCLab-PolyU/OmniCompass
Viewer • Updated 2 days ago • 3.18k • 486 • 4
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.18829 in a Space README.md to link it from this page.
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co