跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 26 天前AI 評分46

SciGame:讓智能體在真實與反事實世界中進行科學發現遊戲

SciGame: Let Agents Play the Scientific Discovery Games in the Real and Counterfactual World

AI 導讀

SciGame 以結構化科學卡片和反事實環境建立基準,評估科學發現智能體,並區分預訓練先驗與實證證據。五個前沿模型的實驗顯示,熟悉的先驗在控制關係改變時會損害表現,自適應實驗則對不同能力的模型有不同影響。即使智能體能正確解讀證據,恢復精確的符號形式仍是主要瓶頸。

正文

Published on Oct 27

Authors:

,

,

Abstract

SciGame evaluates scientific discovery agents using structured cards with counterfactual environments to separate prior knowledge from empirical evidence, revealing that exact symbolic law recovery remains a key challenge.

Large language model agents are increasingly used for scientific discovery. However, existing benchmarks mix law inference with programming, score data fitting instead of prediction on unseen inputs, and do not separate empirical evidence from pretrained priors. We introduce SciGame, a controlled benchmark that converts scientific literature into structured scientific cards. Each card yields a pair of Original and dimensionally consistent Counterfactual environments with the same task interface. SciGame contains a Historical collection of canonical laws and a live Temporal collection curated from papers published after model knowledge cutoffs. It evaluates agents in candidate-guided Identification and open-ended Discovery settings. Each setting runs under Prior-only, Fixed, and Adaptive protocols. Experiments on five frontier models yield three findings. First, familiar priors hurt performance once the governing relation is altered. Second, adaptive experimentation helps the strongest models but harms weaker ones under limited budgets. Third, good explanations can hide quantitative failures. Recovering the exact symbolic form remains the main bottleneck, even when agents interpret evidence correctly. Current agents can investigate systems actively, but inducing exact laws in unfamiliar settings remains an open challenge.

View arXiv page View PDF Project page GitHub 1 Add to collection

Get this paper in your agent:

hf papers read 2610.37919

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.37919 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.37919 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.37919 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co