跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 25 天前AI 評分35

HiFOR:以動作引導世界錨定的 Vision-Language-Action 模型分層前瞻方法

HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model

AI 導讀

HiFOR 提出一種分層前瞻框架,透過動作引導的世界檢索,預測未來狀態並提取相關資訊,以引導 Vision-Language-Action 模型生成動作。框架在兩個模擬基準及六項真實世界機械人操作任務上的實驗,驗證了其有效性。

正文

Abstract

HiFOR improves robotic manipulation by hierarchically predicting future states and retrieving relevant future information to guide action generation.

Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. However, existing methods often model short-term dynamics and long-term task objectives separately, while directly incorporating predicted futures into action policies without sufficiently considering their relevance to current action requirements. To address these limitations, we propose HiFOR, a hierarchical foresight framework with action-guided world retrieval. Specifically, Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones, and final goal states through a Hierarchical Dependency Mask (HDM), establishing dependencies across temporal horizons. Building upon these predictions, Structure-Aware Latent World Modeling organizes future latent representations along the horizon, view, and spatial dimensions to facilitate fine-grained retrieval. Furthermore, Action-Guided Factorized World Retrieval adaptively retrieves relevant future information from the structured latent world based on evolving action representations during denoising, providing targeted guidance for action generation. Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks demonstrate the effectiveness of our approach.

View arXiv page View PDF Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.40373 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.40373 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.40373 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co