跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-15AI 評分40

OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing

OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing

AI 導讀

OmniXRef 提出統一框架,讓用戶結合不同數量的文字、圖片及影片參考,控制來源影片的多種屬性。框架採用 R^2-RoPE,並以單流擴散 Transformer 處理來源影片及來自 MLLM 的多模態上下文;團隊亦建構 OmniXRef-Data 和 OmniXRef-Bench。評估顯示,OmniXRef-Model 的表現優於特定任務基線及商業模型。

正文

Published on Oct 15

Authors:

,

,

,

,

,

,

,

,

Abstract

Creative intent in video editing can be expressed through text instructions (e.g., "remove the people"), image references (e.g., subject appearance), video references (e.g., camera movement), or a combination of the above. We introduce OmniXRef, a unified framework for multimodal multi-reference video editing that allows users to combine variable numbers of text, image, and video references to control diverse attributes of a source video. We first devise a unified input design with a reference-bank interface and introduce R^2-RoPE to support generalization to more references. Then our model, OmniXRef-Model, uses a single-stream diffusion transformer to jointly process the source video and multimodal context from an MLLM in an interleaved sequence. To address the scarcity of training data for video-reference editing, we construct OmniXRef-Data using synthesis pipelines for different editing tasks and combinations of references. We use curriculum learning to progressively increase task complexity and resolution. To support systematic evaluation of video as a control modality, we introduce OmniXRef-Bench, a benchmark for video-reference attribute transfer across Camera, Motion, Content, Style, and Visual Effects (VFX). Comprehensive evaluations show that our OmniXRef-Model outperforms task-specific baselines and commercial models.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2610.22744

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.22744 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.22744 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.22744 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co