跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 9 天前AI 評分47

語言模型中的探測與引導,是否偏好相同方向?

Do Probing and Steering Favor the Same Directions Within and Across Language Models?

AI 導讀

研究發現,在三個不同系列的指令微調語言模型中,最適合探測概念的隱藏狀態方向,並非最適合引導模型朝該概念改變的方向。這種差異在 60 組「層級—模型—概念」組閤中有 59 組成立,引導則一貫偏向較接近平均差異的方向。轉移至其他模型的探測方向可保留目標模型高於隨機水平探測表現的 81-91%;在 PopQA 引導模型回答「unknown」,可令不確定錯誤的棄答增加 18-44 個百分點。

正文

Published on Nov 14

Authors:

Abstract

The best direction for probing a concept in a language model's hidden state is not the best direction for steering the model toward that concept, with implications for transfer and abstention.

A direction in a language model's hidden state can be used to probe whether a concept is present or to steer the model toward it. Although probe directions are often reused for steering, the direction that best detects a concept need not be the one that best changes it. We study this distinction in three instruction-tuned models from different families. We construct a one-parameter family of directions ranging from a covariance-adjusted discriminant to the mean difference, and evaluate probing and additive steering, which adds the direction to the hidden state, along the family at a fixed side effect. For truthfulness and sentiment, in every model, the best probe is not the best steering direction: steering consistently favors directions closer to the mean difference. This separation holds across 59 of 60 layer-model-concept combinations in a layer sweep, with matched probe and steering texts in all six model-concept combinations, and under a rotation-based steering operator. The exact steering optimum is less stable: for sentiment, an intermediate direction produces a 27-56% stronger steering effect than the mean difference, but this gain shrinks or disappears with matched texts and depends on the layer and the operator. The two roles also transfer differently across models. A probe transferred by a linear map fitted on neutral text retains 81-91% of the target model's above-chance probing performance, while the transferred mean difference retains less of its steering effect in 11 of 12 model pairs. This is a consistent tendency, although our predefined criteria classify only three pairs, and nine remain inconclusive. Finally, as an application on PopQA, steering toward "unknown" increases abstention by 18-44 percentage points on uncertain errors, versus 2-5 on confident errors, while rarely correcting errors in cases where a correct answer is reachable.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.19499

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.19499 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.19499 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.19499 in a Space README.md to link it from this page.

Collections including this paper 1

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co