ParetoSaddler:以帕累托導向優化智能體執行框架,兼顧品質與成本
ParetoSaddler: Pareto-Guided Optimization of Agent Harnesses for Quality and Cost
ParetoSaddler 透過分開處理中間候選方案的晉級與最終部署選擇,優化 LLM 智能體執行框架,在不修改模型權重下兼顧品質與成本。於 ClawMark,品質由 70.86% 升至 81.35%,每項任務成本由 \1.17 降至 1.01;於 GAIA2,品質由 56.22% 升至 62.89%,成本由 \0.78 降至 0.45。
Published on Nov 1
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
ParetoSaddler improves LLM agent harnesses by separately promoting intermediate candidates via Bayesian bootstrap Pareto support and selecting final deployments, boosting quality while reducing cost.
Harness optimization can substantially improve LLM agents without modifying model weights, but existing methods primarily optimize response quality. Extending them to joint quality--cost optimization introduces a challenge: applying the deployment preference during intermediate updates can prematurely reject candidates that are valuable for later improvements. We introduce ParetoSaddler, which addresses this mismatch by separating intermediate candidate promotion from final deployment selection. At each iteration, an optimizer updates a parent harness to produce a candidate, and then compares the two over repeated executions and uses Bayesian bootstrap Pareto support to promote candidates showing credible progress in either quality or cost. Promoted candidates receive broader development evaluation and remain available as parents for later optimization steps. We evaluate ParetoSaddler on ClawMark and GAIA2 using domain-disjoint splits. On ClawMark, ParetoSaddler improves quality from 70.86\% to 81.35\% while reducing cost from \1.17 to 1.01 per task; on GAIA2, it improves quality from 56.22\% to 62.89\% while reducing cost from \0.78 to 0.45 per task. The Pareto-selected harnesses also achieve higher quality at lower cost than quality-only harness optimization. Analysis shows that Pareto-aware optimization retains intermediate candidates that would be discarded by deployment preference but can support later improvements, while Bayesian bootstrap screening filters out candidates whose gains are weakly supported across runs.
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2611.02079
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.02079 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.02079 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.02079 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co