跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 28 天前AI 評分46

SPOT:透過預測提交順序提升擴散語言模型的推測解碼效率

On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction

AI 導讀

SPOT 透過預測 token 生成順序並並行驗證候選項,加速擴散語言模型的精確解碼。這個 0.62M 參數模型在 SDAR-8B 上最多減少 6.8 倍目標模型通過次數,吞吐量最高達標準解碼的 3.4 倍;SPOT-Sets 在動態閾值解碼上平均提升 SDAR-8B 吞吐量 59%。

正文

Abstract

SPOT accelerates diffusion language models by predicting token generation order and verifying candidates in parallel, reducing passes up to 6.8x while maintaining exact decoding.

Diffusion language models (DLMs) promise parallel text generation, but in practice face an unresolved speed-quality trade-off. Exact decoding fills in only one masked position per model pass, while faster multi-token decoding schemes are lossy. Speculative decoding is the standard lossless acceleration for autoregressive models. However, it does not transfer directly to DLMs, which must choose both what to generate and where. We show that this order is the missing piece: the model's own predictions from the previous pass already give the right token at over 90% of positions, but not which position comes next. We introduce Speculating Positions Order Trajectory (SPOT), a 0.62M-parameter model that reads only the target's output distributions, predicts this order and proposes a tree of candidate continuations, which the target verifies in a single pass by state equality. Coupled-noise match, which shares Gumbel noise between drafter and target, keeps this verification exact at any temperature. This approach also inspired us to introduce SPOT-Sets for dynamic threshold decoding, where a pass commits several positions: SPOT-Sets drafts whole commit sets together with their tokens. Our comprehensive experiments show that SPOT cuts target passes by up to 6.8times and reaches up to 3.4times the throughput of standard decoding on SDAR-8B, matching or beating the calibrated and training-free drafting baselines in every task and budget at T=0 and T=1. In addition, we show that SPOT's learned drafter transfers across models, demonstrating successful transfer across six different DLMs; with a light calibration, the SPOT-Sets drafter transfers as well. On top of dynamic threshold decoding, SPOT-Sets improves the throughput of SDAR-8B by 59% on average over the threshold decoder alone. Project page: https://eliezer318.github.io/SPOT-dLLM/

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.22543

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

EliezerS/SPOT-dLLM

Other •

Updated 1 day ago

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.22543 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.22543 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co