ProVe:提議者-驗證者解碼
ProVe: Proposer-Verifier Decoding
ProVe 透過微調大型模型,使其一次預測一批未來 token,再以額外一次前向傳播驗證,反轉 speculative decoding 的提議與驗證角色。使用 32B 提議模型時,ProVe-lossless 在六項 benchmark 的平均自回歸生成吞吐量達 2.5 倍,無須訓練獨立 draft head。
Published on Oct 20
·
on Nov 3
Authors:
,
,
,
,
Abstract
ProVe reverses speculative decoding roles by fine-tuning a large model to propose token blocks and verifying them with a smaller or identical model, achieving faster autoregressive generation with adjustable speed-accuracy tradeoffs.
Speculative decoding accelerates autoregressive (AR) language models by letting a small model draft tokens serially and a large model verify them in parallel. We reverse these roles. Starting from a standard AR checkpoint, we fine-tune the large model into a proposer that predicts a block of future tokens in a single forward pass, then use one additional forward pass to verify the proposed block. This gives ProVe, Proposer-Verifier decoding. The verifier can be a smaller AR model or the proposer itself; the latter gives ProVe-lossless, which reproduces the proposer's greedy AR decoding. Block proposals already provide substantial room for acceleration: with a block size of 16, the proposed block has an average AR-matching prefix of 6.6 tokens on GSM8K. Across six benchmarks with a 32B proposer, ProVe-lossless reaches 2.5x average AR throughput without training a separate draft head. With a small 3B verifier, ProVe reaches 2.7x average throughput at 2.6 accuracy points below AR on the six-task average, providing a controllable tradeoff between speed and accuracy.
View arXiv page View PDF Project page Add to collection
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.27815 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.27815 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.27815 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co