τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
τ-Emotion-Bench 以受控情緒條件下的用戶互動,評估工具呼叫智能體在長程任務中的可靠性。Tracer 是可擴展的數據生成流程,用於構建任務及收集互動軌跡,並篩除過度依賴用戶的軌跡。實驗顯示,前沿智能體的任務成功率與穩定完成任務的能力存在明顯差距;Tracer 生成的監督數據可提升不同規模的 Qwen3 模型,並泛化至未見領域及 τ-Bench、τ-Trait。
Published on Oct 14
·
on Oct 15
Authors:
,
,
,
Abstract
In long-horizon tool-calling tasks, LLM agents can fail not only because of tool-use errors, but also because of ineffective interaction with users. Yet existing benchmarks largely assume cooperative and behaviorally stable users, leaving the effect of emotional variation on agent reliability underexplored. We introduce τ-Emotion-Bench, a ten-domain benchmark for evaluating tool-calling agents under controlled emotion-conditioned user interactions while preserving the underlying task objective. To scale both benchmark construction and agent training, we introduce Tracer, a scalable agentic data-generation pipeline that constructs grounded, policy-valid tasks and collects trajectories through an interactive agent--user--environment harness. By allowing dissatisfied users to terminate inefficient interactions, the harness provides an interaction-aware signal that filters trajectories with excessive user dependence and favors effective tool use. Experiments with two user simulators show that frontier agents remain unreliable, with substantial gaps between average task success and consistent task completion. Tracer-generated supervision consistently improves Qwen3 models across scales, generalizes to unseen domains, and transfers to τ-Bench and τ-Trait. Behavioral analyses further show that higher task success is associated with a better balance between tool execution and necessary user follow-up. Overall, τ-Emotion-Bench and Tracer provide a scalable framework for evaluating agent reliability under diverse user behaviors and improving tool-calling agents through interaction-aware supervision.
View arXiv page View PDF Project page GitHub 4 Add to collection
Get this paper in your agent:
hf papers read 2610.19560
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 6
Text Generation • 2B • Updated 35 minutes agotau-emotion-bench/Tracer-Low-1.7B
Text Generation • 4B • Updated 35 minutes agotau-emotion-bench/Tracer-Low-4B
Text Generation • 8B • Updated 35 minutes agotau-emotion-bench/Tracer-Low-8B
Text Generation • 2B • Updated 35 minutes agotau-emotion-bench/Tracer-High-1.7B
Datasets citing this paper 2
tau-emotion-bench/Tracer-Low-tau-emotion-data
Viewer • Updated 35 minutes ago • 21.4k • 2
tau-emotion-bench/Tracer-High-tau-emotion-data
Viewer • Updated 35 minutes ago • 32.7k • 1
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.19560 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co