HeLM:面向非馬爾可夫操作的分層顯式語言記憶
HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation
HeLM 是一個分層 VLM–VLA 框架,透過顯式語言記憶保存與任務相關的歷史資訊,協助機械人在當前觀察不足以判斷進度時追蹤任務。記憶只會在偵測到相關事件或指令變更時更新,並可保留先前指令下取得的資訊。HeLM 在 10 項真實世界非馬爾可夫任務(56 個變體)中評估;兩階段推理令每回合子任務成功率由單階段版本的 67.7% 提升至 85.7%。
🤖 HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation
How can robots track task progress when the current observation alone is insufficient?
We introduce HeLM, a hierarchical VLM–VLA framework that maintains task-relevant history in explicit language memory.
Two key aspects of our work:
🧠 Event-triggered memory updates: HeLM separates visual event detection from memory updates. The HLP updates memory and generates the next action command only when a task-relevant event is detected or the instruction changes.
🔄 Inter-episode memory: We introduce tasks where robots must remember information acquired under previous instructions, such as a human-demonstrated button sequence.
We evaluate HeLM on 10 real-world non-Markovian tasks (56 variants). Our two-stage inference improves average intra-episode subtask success from 67.7% to 85.7% compared with a one-stage variant.
We also provide detailed failure analyses and evaluations of memory scalability.
We welcome feedback and discussions on memory for robot learning!
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co