EVA-Cap:透過事件中心對齊優化視聽影片描述生成
EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment
EVA-Cap 將視聽影片描述拆解為原子視聽事件,並透過以事件為中心的對齊優化描述生成。該框架在自建 EVA-Data 數據集上以監督式微調(SFT)及羣組相對策略優化(GRPO)訓練,並針對事件完整性、事件內歸因及事件間同步進行優化。評測中,EVA-Cap 在 ChronusAV 的事件中心評測平均提升 +9.0%,在 WorldSense 提升 +3.3%。
Published on Oct 15
·
on Oct 16
Authors:
,
,
,
,
,
,
Abstract
EVA-Cap improves audiovisual video captioning by decomposing captions into atomic events and optimizing event-centric alignment through supervised fine-tuning and group relative policy optimization.
Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning. Nevertheless, existing approaches predominantly treat captioning as an unstructured text generation process, neglecting the inherent semantic structure of videos and leading to suboptimal fine-grained audiovisual alignment. To bridge this gap, we propose EVA-Cap, a novel framework that decomposes captions into atomic audiovisual events and transforms unstructured text supervision into event-centric alignment. We train EVA-Cap via Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) on a self-curated dataset, EVA-Data, which is built with event-centric quality control. During the GRPO phase, the model is explicitly optimized against three event-centric alignment targets: global event completeness, intra-event attribution, and inter-event synchronization. Extensive experiments demonstrate that EVA-Cap significantly outperforms existing methods in event-centric evaluation (e.g., +9.0% average gain on ChronusAV), while achieving competitive performance on general audiovisual captioning benchmarks (e.g., +3.3% on WorldSense).
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2610.22186
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.22186 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.22186 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.22186 in a Space README.md to link it from this page.
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co