跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-14AI 評分49

方法更多,智能體更好?智能體強化學習中自我蒸餾的現實檢驗

More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning

AI 導讀

研究比較智能體強化學習中十種具代表性的自我蒸餾方法與 GRPO,發現沒有方法能在所有設定下持續勝出。評估涵蓋 ALFWorld、ScienceWorld、Search-QA 及 WebShop 四項任務、三種模型規模,共進行 132 次訓練,記錄 12,124.4 個 H800 GPU 小時。強勁的最終表現可能掩蓋訓練不穩定,增加訓練成本亦不一定帶來更佳表現。

正文

Published on Oct 14

Authors:

,

,

,

,

,

,

,

,

Abstract

Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseline scores under differing experimental conditions, provide few direct comparisons among contemporaneous methods, and give limited attention to training stability and computational cost. These limitations obscure the source of reported gains and the practical strengths and weaknesses of competing methods. Costly agent training further motivates a common basis for comparison. To address these gaps, we compare ten representative methods against GRPO on ALFWorld, ScienceWorld, Search-QA, and WebShop across three model sizes. Our study comprises 132 training runs totaling 12,124.4 recorded H800 GPU-hours, with repeated evaluations over eight rollout seeds for the pretrained backbones and all eleven trained methods. Beyond final task performance, we examine training progress, stability, and measured computational cost. Our evaluation yields three key findings. First, gains over GRPO remain limited in scope, with no method consistently outperforming it across settings. Second, strong final performance can mask unstable training. Third, additional training cost does not consistently translate into better performance. We establish this benchmark to support fair, comprehensive comparison and guide future research toward reliable and cost-effective improvements in agent training. Project page: https://aim-uofa.github.io/Self-Distillation-AgentRL/

View arXiv page View PDF Project page GitHub 1 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.19789 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.19789 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.19789 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co