跳到正文
TNW· Ana Maria Constantin·· 9 天前AI 評分69

Anthropic 指 AI 安全評分高度取決於測試方式

Anthropic says its AI safety scores depend heavily on how they’re measured

AI 導讀

Anthropic 發佈 AI 安全測試指南,指出同一模型的安全評分會因測試設定而大幅不同。Claude Fable 5.1 在自殘測試對話中,Claude App 加入修正後的適當應對比例為 94%,透過未附該提示詞的 Anthropic API 測試則為 60%。指南建議測試前先確定衡量項目、以人類專家核對 AI 評分,並交代結果的不確定性。

正文

Anthropic says its AI safety scores depend heavily on how they’re measured

Anthropic says the same model can score very differently on safety tests depending on how they are set up.

Anthropic has published advice on building AI safety tests. It shows Claude Fable 5.1 behaved appropriately in only 60% of self-harm test conversations via its API, against 94% in the Claude app after a fix, and warns that many headline safety scores hide wide uncertainty.

Anthropic has published advice on how to build reliable AI safety tests, and it includes some unflattering numbers about its own models. The guide, out on Thursday, says a claim that one model is “safer” is only as good as the test behind it.

Before it released Claude Fable 5.1, Anthropic found the model sometimes hinted that self-harm could help people cope. It did so even while urging users not to self-harm and pointing them to human support. Anthropic added a line to the Claude app’s hidden instructions to stop this.

With that fix, Claude behaved appropriately in 94% of the test conversations in the Claude app. Through Anthropic’s API, where developers use the model without that prompt, the figure was 60%.

That gap is the guide’s main warning: the same model can score very differently depending on how it is set up. Anthropic also says its simple one-question tests are now too easy, with every recent model scoring 94% to 98%. It relies more on longer conversations instead, where Cisco found guardrails can crack.

Even those can mislead. Claude Opus 5 and Opus 5.5 scored 98% and 94% on a romance-scam test, but Anthropic says the gap was too small to show either was safer. It advises teams to decide what to measure before testing, check AI graders against human experts, and always show how certain a result is.

The guide also says Anthropic has stopped many tests that give models internet access, after its models reached the open internet and breached three companies during cyber tests. Anthropic released Fable 5.2 this week.

Get the TNW newsletter

Get the most important tech news in your inbox each week.

來源:TNW · thenextweb.com