Harness-S:從同系 Harness 學習,訓練具泛化能力的智能體
Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training
Harness-S 以同系 Harness 挽救強化學習中失敗的 rollout,並將所得能力轉移至單一策略,使其在原有 Harness 下維持效能。自適應 Harness 路由器會在基礎 Harness 的所有初始嘗試失敗時,選出 Top-k 變體產生替代軌跡。於 AppWorld、SWE-Gym 和 DBBench,訓練所得策略在原有 Harness 下優於僅訓練策略的方法。
arxiv:2611.04277
Copy markdown
Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training
Published on Nov 3
·Submitted by 张康宁on Nov 6
Authors:
Kangning Zhang ,
Wenxiang Jiao ,
Bowen Fang ,
Shuai Shao ,
Jiachen Zhu ,
Rong Shan ,
Shijian Wang ,
Hao Wang ,
Yuan Lu ,
Weiwen Liu ,
Weinan Zhang
Abstract
A selective harness-rescue training method uses specialized sibling harnesses to replace failed reinforcement learning rollouts and transfer their benefits into a single policy that performs better under the original base harness.
Generated by deepseek-ai/DeepSeek-V4.1-Flash
Harness evolution from a common base exhibits Locality, Complementarity, and Nonstationarity: variants repair different failures, and their utility changes as the policy learns. Rather than seeking a universally superior harness, we introduce Harness-S to selectively rescue failures of a base execution system through sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original harness. Directional Harness Evolution develops specialized sibling harnesses along distinct repair directions, forming a reusable Sibling Harness Pool that remains fixed during policy training. Complementary Harness Rescue Training invokes this pool only when all initial attempts under the base harness fail. The same policy generates fresh rollouts under selected auxiliary harnesses, and these trajectories replace the failed group for reinforcement learning. An Adaptive Harness Router selects the Top-k variants using Harness Behavior Profiles and voting across multiple LLMs. Policy-aware profile re-anchoring refreshes this evidence as the policy learns. Across AppWorld, SWE-Gym, and DBBench, the trained policy substantially outperforms policy-only training when returned to the original base harness. Behavioral analyses and held-out cases support the transfer of harness-elicited behaviors into the policy itself. Reusing the router for Rank-1 inference provides additional gains at the same executor rollout count.
View arXiv pageView PDFGitHubAdd to collection
Community
Paper submitter22 minutes ago
Harness evolution from a common base exhibits Locality, Complementarity, and Nonstationarity: variants repair different failures, and their utility changes as the policy learns. Rather than seeking a universally superior harness, we introduce Harness-S to selectively rescue failures of a base execution system through sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original harness. Directional Harness Evolution develops specialized sibling harnesses along distinct repair directions, forming a reusable Sibling Harness Pool that remains fixed during policy training. Complementary Harness Rescue Training invokes this pool only when all initial attempts under the base harness fail. The same policy generates fresh rollouts under selected auxiliary harnesses, and these trajectories replace the failed group for reinforcement learning. An Adaptive Harness Router selects the Top-k variants using Harness Behavior Profiles and voting across multiple LLMs. Policy-aware profile re-anchoring refreshes this evidence as the policy learns. Across AppWorld, SWE-Gym, and DBBench, the trained policy substantially outperforms policy-only training when returned to the original base harness. Behavioral analyses and held-out cases support the transfer of harness-elicited behaviors into the policy itself. Reusing the router for Rank-1 inference provides additional gains at the same executor rollout count.
Reply
Edit Preview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Comment ·Sign up or log in to comment
Get this paper in your agent:
hf papers read 2611.04277
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.04277 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.04277 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.04277 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
System theme
Company
Website
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co