跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-20AI 評分66

《並非所有特徵都同等重要》研究揭示視覺語言動作(VLA)模型按絕對座標生成動作

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

AI 導讀

研究發現,視覺語言動作(VLA)策略的動作程序會綁定絕對座標,並忽略視覺場景變化。把任務 A 的激活值注入動作專家後,pi0.5 在任務 B 場景中仍於 1,968 個回合的 99.6% 執行任務 A 的動作軌跡;涵蓋六種 VLA、420,000+ 次 rollout 的分析亦發現,目標選擇與動作生成在可區分時分別由 VLM 和 expert 路徑承擔。

正文

Published on Oct 20

·

Submitted by

Bryce

on Oct 21

Authors:

,

Abstract

A mechanistic interpretability study of vision-language-action policies reveals that motor programs are bound to absolute coordinates and ignores visual scene changes, with sparse autoencoder analysis separating instruction encoding and motor generation across architectures.

A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6\% of episodes (n{=}1{,}968). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10--15\,cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to six VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over 420{,}000+ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (pi0.5 per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release Action Atlas (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://github.com/CWRU-AISM/action-atlas

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2603.19233

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2603.19233 in a model README.md to link it from this page.

Datasets citing this paper 10

bag100/action-atlas-groot-activations

Viewer •

Updated 12 minutes ago

•

3

•

1.45k

bag100/action-atlas-smolvla-activations

Updated 12 minutes ago

•

1.16k

bag100/action-atlas-viz

Updated 14 minutes ago

•

765

bag100/action-atlas-oft

Viewer •

Updated 12 minutes ago

•

1.76k

•

621

Browse 10 datasets citing this paper

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2603.19233 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co