跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 22 天前AI 評分50

AI 評審能否看見論文改進?論文修訂軌跡大型研究

Can AI Reviewers See Your Efforts to Improve Your Paper? A Large-Scale Study of Paper Revision Trajectories

AI 導讀

一項涵蓋 6,959 篇論文修訂軌跡、五個 AI 會議及八名 AI 評審的大型研究發現,AI 評審通常不會在重大改進後調高論文評分。即使修訂取得較高的中立評分,簡短的負面用戶意見仍可抵銷或逆轉這項優勢,而且不同評審之間差異明顯。修訂維度只能有限度預測評分變化,未能為不同評審提供一致指引。

正文

Published on Nov 1

Authors:

,

,

,

,

,

,

,

,

,

Abstract

AI reviewers often fail to reflect manuscript improvements in scores, with user opinions frequently overriding observed revision gains.

AI reviewers are increasingly used to assess scientific manuscripts, yet it remains unclear whether their judgments reflect the improvements authors make through revision. We investigate this question using PaperTrajectory-2026, the 2026 release of PaperTrajectory, a dataset of 6,959 paper trajectories across five AI conferences, with automatic annotations of changes along 14 dimensions and separate judgments of overall quality. The annotations show that major improvements emphasize empirical evidence and framing, while later revisions tend toward incremental improvement. Using independent assessments of manuscript versions from eight AI reviewers, we examine whether annotated major improvements receive higher scores, which revision dimensions are associated with score changes, and whether user opinions can outweigh observed revision gains. Every reviewer leaves scores unchanged on most major improvements, and agreement on the direction of score changes is weak. Revision dimensions show some reviewer-specific associations with score increases, but provide limited predictive discrimination and no consistent guide across reviewers. Even when a revision earns a higher neutral score, a brief negative user opinion can erase or reverse that advantage, with substantial variation across reviewers. These findings show that substantive revisions are not consistently reflected in review scores and that observed scoring gains can be vulnerable to user framing, motivating assessments that connect manuscript evidence to overall judgments and remain stable when no scientific evidence changes.

View arXiv page View PDF GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.02244

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.02244 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.02244 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.02244 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co