跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 13 天前AI 評分46

SteerDistill:透過閉環教師引導量化大語言模型智能體

SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering

AI 導讀

SteerDistill 以閉環教師引導訓練量化智能體,讓凍結的 BF16 教師根據執行回饋引導學生重試,並將修正內化,令學生之後無需教師或引導提示詞也能獨立運行。在 W4A16 下,三個模型於五項智能體基準測試的平均分均與 BF16 相差不超過一分;在 Qwen3-8B 的 W2A16 測試中,SteerDistill 的平均分高於受測量化基線。

正文

SteerDistill: sub-4-bit LLM agents.

A quantized agent can miss a task requirement and fail the whole task. SteerDistill trains it to avoid these mistakes through closed-loop teacher steering.

A frozen BF16 teacher guides student retries and refines its guidance using execution feedback. Context distillation internalizes the locally verified corrections. The student then runs alone, without a teacher or steering prompts. Both start from the same BF16 checkpoint.

At W4A16, average scores stay within one point of BF16 across three models and five agent benchmarks. At W2A16, SteerDistill achieves the highest average score among the tested quantized baselines on Qwen3-8B.

Project & cases · Code · Paper

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co