跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-15AI 評分43

AVC-Reward:以事件為中心的音訊與影片一致性評估

AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

AI 導讀

AVC-Reward提出以事件為中心的音訊與影片一致性評估框架,並實作為可訓練的獎勵模型,用於評估複雜多事件場景。框架衡量整體敍事協調、事件層級時間同步及細緻屬性對齊。它與人類判斷的吻合度在自動評估方法中達到當前最佳,亦可定位失效原因,並作為線上強化學習訊號提升聯合生成模型的音訊與影片一致性。

正文

Published on Oct 15

Authors:

,

,

,

,

,

,

,

Abstract

An event-centric framework evaluates audio-video consistency across complex multi-event scenes by measuring global harmony, temporal synchronization, and attribute alignment via a trainable reward model.

As joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: multi-event composition, multi-layer audio mixture, and asymmetric cross-modal relations. However, current evaluations largely reuse metrics from earlier simple and unidirectional tasks. They typically evaluate an audio-video clip as a whole and rely on simple predefined relations, making them unsuitable for these increasingly complex scenes. We propose an event-centric evaluation framework that disentangles audio and visual events at the event level, analyzes each event independently, cross-validates relations across modalities, and aggregates the judgments into a coarse-to-fine, three-level consistency metric. The metric assesses global narrative harmony, event-level temporal synchronization, and fine-grained attribute alignment, providing a general formulation for audio-video consistency evaluation across diverse generation settings and cross-modal relations. We instantiate the metric as AVC-Reward, a trainable audio-video reward model. To evaluate the agreement between automatic judgments and human perception across generation paradigms and relation complexities, we construct AVC-Benchmark, which is primarily built from diverse positive and negative samples generated by various types of audio-video generation models. AVC-Reward achieves state-of-the-art alignment with human judgment among automatic methods, provides interpretable rationales that localize failures, and improves the audio-video consistency of a joint generation model when used as an online reinforcement-learning signal. Project page: https://tsuki0512.github.io/AVC-Reward.github.io/

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.22321

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

Tsuki0512/LTX-2-AVCReward-LoRA

Text-to-Video •

Updated 3 days ago

•

2

Tsuki0512/AVC-Reward

Other •

32B •

Updated 3 days ago

•

40

•

1

Datasets citing this paper 1

Tsuki0512/AVC-Benchmark

Viewer •

Updated 3 days ago

•

706

•

97

•

1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.22321 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co