2024年8月21日 · In this paper, we aim to enhance MLLMs' capabilities in understanding facial expressions. We first generate instruction data for five FER datasets with Gemini. We then …

Additionally, we propose Emotion-LLaMA, a model that seamlessly integrates audio, visual, and textual inputs through emotion-specific encoders. By aligning features into a shared space and …

2024年6月17日 · By aligning features into a shared space and employing a modified LLaMA model with instruction tuning, Emotion-LLaMA significantly enhances both emotional …

Official Implementation for EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning Resources

Introduction Emollama-7b is part of the EmoLLMs project, the first open-source large language model (LLM) series for comprehensive affective analysis with instruction-following capability. …

2024年8月21日 · In this paper, we aim to enhance MLLMs' capabilities in understanding facial expressions. We first generate instruction data for five FER datasets with Gemini. We then …

Emotion-LLaMA 是一款多模态情绪识别与推理模型，融合了音频、视觉和文本输入，通过特定情绪编码器整合信息。 模型基于修改版 LLaMA，经过指令调整以提升情感识别能力。

为了解决情感识别中的单模态方法在捕捉真实世界情感表达复杂性方面的局限性，我们提出了MERR数据集和Emotion-LLaMA模型，通过整合音频、视觉和文本输入，显著提高情感识别能 …

本文针对面部表情识别（fer）在泛化能力和语义信息对齐方面的不足，提出了一种新颖的多模态大型语言模型emo-llama。 通过使用预训练的面部分析网络和设计面部信息挖掘模块，实验表 …

Official Implementation for EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning - xxtars/EMO-LLaMA