Lluna: Collaborative Unimodal Models for Missing-Modality Inference
Lluna: Collaborative Unimodal Models for Missing-Modality Inference
Lluna 透過私有單模態預測、圖式殘差協作及漸進式一致性訓練,改善模態缺失時的多模態預測。在涵蓋情感運算、場景理解及醫療預測的八個數據集上,Lluna 的 Mean-Missing 平均表現較最強評估基線提升 4.26%。固定錨點診斷顯示,八項任務在最少輸入下的平均協作均有正向提升;程式碼已提供。
Published on Oct 14
Authors:
,
,
,
,
Abstract
Lluna improves multimodal predictions under missing inputs by combining private unimodal predictions with graph-based residual collaboration and progressive consistency training.
Pretrained unimodal models offer a flexible route to multimodal prediction through collaboration, but this collaboration can become less accurate than a surviving unimodal model when modalities are missing. We study this setting using fixed surviving-model anchors and find that collaborative gains can weaken or reverse as inputs disappear, even when the anchor's own input is unchanged. Our diagnostics further reveal reduced recovery of modality-specific information after aggregation and state-dependent collaborative representations. We propose Lluna, which combines a direct private prediction path from adapted unimodal features with a common graph-based collaboration branch that provides a residual correction. Progressive Collaboration Consistency organizes richer-to-poorer guidance along single-modality removals, aligning normalized common representations while task supervision adapts predictions to the available evidence. Across eight datasets spanning affective computing, scene understanding, and medical prediction, Lluna achieves the best evaluated average performance over non-full input states, improving Mean-Missing by 4.26\% on average relative to the strongest evaluated baseline. The fixed-anchor diagnostic also shows positive gains in mean collaboration under minimal inputs on all eight tasks. Code is available at https://github.com/AIFrontierLab/Lluna.
View arXiv page View PDF GitHub 1 Add to collection
Get this paper in your agent:
hf papers read 2610.20526
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.20526 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.20526 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.20526 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co