跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 10 天前AI 評分43

MMFlash:加速視覺語言模型的推測解碼

MMFlash: Faster Speculative Decoding for Vision-Language Models

AI 導讀

MMFlash 透過自適應提高視覺依賴 token 的權重,並在長影片輸入中採用稀疏注意力,加速視覺語言模型的多模態推測解碼。它在多項多模態基準測試中實現最高 3.58× 的無損解碼加速,較 EAGLE-3 和 DFlash 分別快至 195.4% 和 17.1%。

正文

Published on Nov 13

Authors:

,

,

,

Abstract

MMFlash improves multimodal speculative decoding by adaptively upweighting visually dependent tokens and using sparse attention for long video, yielding up to 3.58× lossless speedup.

Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to 3.58times, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.

View arXiv page View PDF Project page GitHub 13 Add to collection

Get this paper in your agent:

hf papers read 2611.18482

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.18482 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.18482 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.18482 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co