語言模型為任何殘差流方向生成最大激活示例
Language models generate maximally activating examples for any residual direction
研究人員開發最大激活示例元模型(MAEMs),可將目標模型的殘差流方向映射為能強烈激活該方向的文本示例。MAEMs 使用無需標籤的強化學習目標,最大化輸入方向與生成文本所產生激活之間的餘弦相似度,避免受語料庫限制。生成針對性探測文本,或可令可解釋性研究涵蓋整個殘差流。
Researchers develop Maximally-Activating Example Metamodels (MAEMs), language models that map a residual-stream direction from a subject model to text exemplars strongly activating it. A label-free reinforcement-learning objective maximises cosine similarity between the supplied direction and activations produced by MAEM-generated text, avoiding corpus limits.
Why it matters
Corpus scanning cannot reveal activations for rare directions; generating examples could make interpretability cover the full residual stream.
Analysis
Interpretability may move from searching existing text to generating targeted probes for model internals.
Source: arXiv
AI disclosure: this article was produced with AI assistance and reviewed for accuracy.
來源:DEV Community · dev.to