跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 11 天前AI 評分33

用於遞歸自我改進的經驗交接強化學習

Experience-Handoff Reinforcement Learning for Recursive Self-Improvement

AI 導讀

研究提出一種經驗交接強化學習方法,讓語言模型智能體在固定框架下學習執行任務、反思經驗及整理任務專用手冊,並按這些經驗對未來執行結果的影響來評估其效用。該方法在多項智能體基準測試中的留出表現高於基線方法。

正文

Published on Nov 12

Authors:

,

,

Abstract

Language-model agents trained with reinforcement learning can improve by learning to create and curate experience artifacts that help future rollouts, not just by optimizing actions from scalar rewards.

It is common to train Large Language Models (LLMs) for agentic tasks with Reinforcement Learning (RL) from scalar rewards. However, this throws away the fine-grained information about the environment that the agent experienced during the trial (e.g., error logs). To utilize this rich information, recent works wrap the agent in a harness that refines its own inputs through reflection or compaction and hands past experience to future rollouts, but the construction of the handed-off artifacts (e.g., a playbook) is itself not learned. We want the best of both worlds: gradient updates from a scalar reward, while also utilizing environment information to hand off experience. As a concrete testbed, we fix the Agentic Context Engineering () harness and train one model to act, reflect, and curate task-local playbooks. Our method, , evaluates candidate experience-handoff artifacts by their downstream effect on future actor rollouts, and co-optimizes the role-specific behaviors (action generator, insight reflector, and playbook curator) under a fixed scaffold. Across agentic benchmarks, achieves higher held-out performance than baseline methods. The results suggest that agent training should optimize not only the actions in the current rollout, but also the model-written experience that future rollouts inherit.

View arXiv page View PDF GitHub 1 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.16409 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.16409 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.16409 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co