跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-13AI 評分43

Simplifying and Improving On-Policy Distillation for Few-Step Diffusion

Simplifying and Improving On-Policy Distillation for Few-Step Diffusion

AI 導讀

研究提出簡化匹配蒸餾(SMD),以簡單的在策略速度回歸作為少步擴散生成的基礎。SMD 加入僅用於訓練的 Pivot 殘差代理,並令學生模型遠離較弱的教師預測;整套方法不需線上分數估計或對抗訓練,部署時生成器維持不變。在 4-NFE 下,SMD 勝過強力少步基線,並在多項基準測試中超越多步教師,規模可達 33 billion parameters。

正文

Published on Oct 13

Authors:

,

,

,

,

,

,

Abstract

Few-step distillation reduces the sampling cost of diffusion and flow models, but existing approaches often require auxiliary score models or specialized architectures. We show that simple on-policy velocity regression already provides a strong foundation for 4-NFE generation. To represent teacher velocity variation within each coarse transition, we introduce Pivot, a training-only residual proxy that separates local corrections from the student's interval-mean prediction. We further repel the student from weaker teacher predictions, providing a lightweight mechanism for avoiding undesirable modes. The resulting framework, Simplified Matching Distillation (SMD), remains entirely regression-based, requires neither online score estimation nor adversarial training, and leaves the deployed generator unchanged. We evaluate SMD on image generation and editing, 3D generation, video synthesis, and joint audio--video generation, scaling to 33 billion parameters. At 4-NFE, SMD outperforms strong few-step baselines and surpasses multi-step teachers on several benchmarks. It also supports cross-model, multi-teacher, and causal-video distillation.

View arXiv page View PDF Project page Add to collection

Models citing this paper 1

zjwfufu/SMD

Updated about 18 hours ago

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.18472 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.18472 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co