跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 10 天前AI 評分45

OutfitterBench:評估 LLM 智能體的長期資源分配與承諾協調能力

OutfitterBench: Evaluating LLM Agents on Long-Horizon Resource Allocation and Commitment Coordination

AI 導讀

OutfitterBench 評估 LLM 智能體管理戶外裝備租賃業務的能力:在固定資金及需求不確定的條件下,經營 200 天,處理 46 種產品和約 1,100 項預訂請求。對七個前沿 LLM 的實驗顯示,多個智能體表現不及簡單的規則基準,只有領先者持續勝過基準。分析指出,智能體難以持續維持可行承諾並重新評估資源配置;基準程式碼已公開。

正文

Published on Nov 13

Authors:

,

,

,

,

Abstract

OutfitterBench evaluates LLM agents managing a 200-day outdoor-equipment rental business under fixed capital and uncertain demand, revealing limitations in sustaining feasible commitments and reallocating resources.

In many long-horizon settings, an agent's own choices create future constraints: committing finite capacity establishes future obligations and forecloses later opportunities. Sustaining performance therefore requires both keeping evolving plans feasible and comparing the value of resource allocations with the alternatives they displace. Recent benchmarks increasingly evaluate LLM agents in continuing operations, but give less attention to how agents keep their own commitments feasible while reassessing resource allocations as opportunities change. We introduce OutfitterBench, where agents manage an outdoor-equipment rental business for 200 days, purchasing from a 46-product catalog and handling roughly 1,100 reservation requests. Under a fixed capital ceiling, reservations and uncertain walk-in demand compete for reusable equipment whose availability changes over time. Experiments with seven frontier LLMs show that several agents underperform a simple rule-based baseline, while only the leading agent consistently surpasses it. Trajectory analysis shows that some agents apply known constraints inconsistently and make new decisions that conflict with existing commitments. Agents with more reliable fulfillment can still secure low returns from comparable commitments or delay resource reallocation. The leading agent better connects preparation with economic choices but also misses opportunities for more profitable asset use. These findings highlight limitations in sustaining coherent resource plans and reassessing their value over time. The benchmark code is publicly available at https://github.com/adminblackjack/OutfitterBench.

View arXiv page View PDF GitHub 3 Add to collection

Get this paper in your agent:

hf papers read 2611.17933

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.17933 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.17933 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.17933 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co