跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 22 天前AI 評分46

CrossSWE:編碼智能體跨倉庫協同開發基準評測

CrossSWE: Benchmarking Cross-Repository Joint Development for Coding Agents

AI 導讀

CrossSWE 推出跨倉庫協同開發基準,評估編碼智能體在多個相互依賴程式碼庫間協調修改及維持一致性的能力。基準包含 171 個真實案例,涵蓋 165 個程式碼庫、20 個技術領域及 13 種程式語言;五個 LLM 中表現最強的 GPT-6.1 Sol,Pass@1 為 58.48%。跨倉庫不一致是最常見的失敗模式,佔 44.2%。

正文

Published on Nov 1

Authors:

,

,

,

,

,

Abstract

CrossSWE introduces a multi-repository benchmark showing that current LLMs struggle with cross-repository consistency during joint software development.

As coding agents achieve increasingly high success rates on software engineering benchmarks, recent efforts have sought to increase task complexity through longer horizons and more extensive code modifications. However, real-world software development often spans multiple interdependent repositories, introducing additional challenges in dependency reasoning, change propagation, and interface consistency. Rather than further increasing complexity within individual repositories, we introduce CrossSWE, a benchmark for cross-repository joint development comprising 171 real-world instances spanning 165 repositories, 20 technical domains, and 13 programming languages. CrossSWE verifies the necessity of coordinated changes by testing whether partial repository modifications are insufficient to achieve the intended behavior. Our evaluation of five LLMs reveals substantial challenges, with the strongest model, GPT-6.1 Sol, achieving only 58.48% Pass@1. Furthermore, cross-repository inconsistencies emerge as the most prevalent failure mode (44.2%), highlighting the difficulty of maintaining consistency across interdependent repositories.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.01484

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.01484 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.01484 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.01484 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co