跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-17AI 評分40

UniAVC:透過漸進式訓練與非對稱排程實現統一音訊與影片創作

UniAVC: Unified Audio-Video Creation via Progressive Training and Asymmetric Scheduling

AI 導讀

UniAVC提出統一音訊與影片創作模型,先以大量單模態數據訓練音訊和影片分支,再以稀缺配對數據進行對齊,支援聯合合成、跨模態生成及指令引導編輯。模型採用各自獨立的噪聲排程和非對稱監督,可用同一組權重執行多種創作任務,無需任務專用 adapter。

正文

Published on Oct 17

Authors:

,

,

,

,

,

,

,

Abstract

UniAVC trains separate audio and video branches on abundant unimodal data before aligning them with scarce paired data, using independent noise schedules and asymmetric supervision to enable joint synthesis, cross-modal generation, and instruction-guided editing.

Unified audio-video creation requires a single model capable of text-conditioned joint synthesis, bidirectional cross-modal generation, and instruction-guided editing across both modalities. Existing methods rely heavily on paired corpora, which are scarce for joint generation and even rarer for editing. Consequently, they are spent learning single-branch generative and editing capabilities that abundant unimodal data could readily teach, rather than on the cross-modal coupling. Furthermore, prevailing systems apply a single shared noise schedule to both modalities, neglecting two critical limitations. First, due to the significant discrepancy in information density, video requires prolonged high noise to organize spatial and motion structure, whereas the sparser audio signal can settle earlier during the synthesis process, the dynamics a uniform schedule might compromise. Second, lockstep denoising leaves asymmetric workflows, such as conditioning on one modality to synthesize or edit another, untrained. We present UniAVC, which addresses both obstacles. It first trains each branch on unimodal dataset to full generative and editing competence, and subsequently dedicates paired data to exploit cross-modal correspondence. Modality-decoupled schedulers meanwhile warp the noise schedule independently from a shared timestep, while subtasks with one clean modality directly supervise asymmetric configurations. These settings respectively enable intent-aware editing, which preserves one modality intact while enlisting it to constrain the other, and post-inference self-refinement, which regenerates only an unsatisfactory branch against an accepted one. UniAVC thus spans multiple creation tasks from a single set of weights without task-specific adapters.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.24791

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

TK3105/UniAVC

Text-to-Video •

23B •

Updated 2 minutes ago

•

1

•

4

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.24791 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.24791 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co