跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 14 天前AI 評分39

EmoHarness 如何透過 Meta-Harness Evolving 提升多模態情緒理解

EmoHarness: Boosting Multimodal Emotion Understanding with Meta-Harness Evolving

AI 導讀

EmoHarness 透過 harness 為小型多模態模型整理情緒理解所需的文字、語音及視覺證據,並以 Meta-Harness Evolving(MHE)離線修改 harness。

正文

Published on Nov 9

Authors:

,

,

,

,

,

Abstract

Small multimodal models can misjudge emotion when words, voice, and visual cues conflict. We present EmoHarness, which runs such a model, the frozen student, inside a harness: the prompts, settings, and verification code around its calls. A human-written starting harness, the scaffold, numbers the student's visual observations, transcript, and prosody descriptions as citable evidence and has the student weigh candidate emotions against it. In Meta-Harness Evolving (MHE), a stronger LLM, the Builder, reads the student's errors and execution traces offline and edits the harness; static checks and validation gates decide which edits are kept, and the Builder never sees evaluation items. On 5,507 MME-Emotion items, EmoHarness raises task-macro Recognition by 7.6-13.4 points over a single video-only call for all four students and both Builders. Transcripts and prosody descriptions supply most of this boost (+5.38 to +10.71); the scaffold changes the audio-augmented call by -0.79 to +2.54, partly through extra multi-label predictions, and evolved edits add at most +0.95, although no evolved increment survives Holm correction. The harness's emotion context also transfers to IntentQA, with small gains for three of four students, none of which survives correction. EmoHarness thus boosts small multimodal students chiefly by supplying and organizing audio-derived evidence.

View arXiv page View PDF Project page GitHub 1 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.11994 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.11994 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.11994 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co