跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-14AI 評分42

MInfoVisQA:資訊密集型視覺問答的多語言基準

MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering

AI 導讀

MInfoVisQA 是一個涵蓋 24 種語言的平行基準,評估多模態大語言模型在資訊密集型視覺問答中的跨語言穩健性。基準包含 8,960 組問答配置,研究評估了 25 個模型;兩個語言對齊問答平均準確率同為 80.4% 的模型,跨語言表現差距分別達 7.8 和 20.3 個百分點。固定使用英文提問時,25 個模型在非英語視覺素材上的得分均較低。

正文

Published on Oct 14

Authors:

,

,

,

,

Abstract

MInfoVisQA, a 24-language parallel benchmark, reveals substantial cross-language robustness gaps in multimodal large language models for information-rich visual question answering.

Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear. To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels. We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages). The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.3 percentage points across languages. Moreover, all 25 models score lower with non-English visuals under fixed English QA, underscoring the need for cross-language sensitivity metrics.

View arXiv page View PDF Project page GitHub 2 Add to collection

Get this paper in your agent:

hf papers read 2611.21644

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.21644 in a model README.md to link it from this page.

Datasets citing this paper 1

arnodjiang/MInfoVisQA

Viewer •

Updated about 8 hours ago

•

8.96k

•

570

•

1

Spaces citing this paper 1

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co