跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 11 天前AI 評分47

透過行為軌跡解讀編碼智能體

Interpreting Coding Agents Through Their Behavioral Trajectories

AI 導讀

研究分析 143,209 條執行軌跡,涵蓋 125 個智能體配置及 117 個不同模型,以揭示編碼智能體的行為模式。即使解題率相近,智能體在探索、修改及驗證程式碼方面仍有顯著差異;腳手架選擇和推理投入亦會系統性地改變這些行為。研究團隊公開了確定性解析器、完整標註與分析流程,以及部分已標註軌跡。

正文

Published on Nov 12

Authors:

Abstract

Standard coding-agent benchmarks primarily evaluate binary task success, treating execution as a black box and obscuring the behaviors that produce those outcomes. We analyze a large corpus of 143,209 execution trajectories from 125 agent configurations spanning 117 unique models. To enable comparisons across heterogeneous agent interfaces, we develop a rule-based parser that maps raw execution traces to high-level actions such as repository search, file editing, and test execution, and audit its predictions against a six-model consensus reference on 500 events. We find that agents with similar solve rates can nevertheless differ substantially in how they explore, modify and verify code. Moreover, without using task outcomes, token counts, or tool names as features, aggregate behavioral profiles retrieve the corresponding agent stack across disjoint repository sets and programming languages, although task composition is not fully controlled in the cross-repository setting. Through analyses of frontier models and agent scaffolds, we further show that scaffold choice and reasoning effort systematically shift agents' search, editing, and testing behavior. We release the deterministic parser together with the full annotation and analysis pipeline alongside an annotated trajectory subset.

View arXiv page View PDF Project page GitHub 0 Add to collection

Get this paper in your agent:

hf papers read 2611.17440

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.17440 in a model README.md to link it from this page.

Datasets citing this paper 1

nebius/SWE-rebench-leaderboard-trajectories

Viewer •

Updated 22 minutes ago

•

27.3k

•

5

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.17440 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co