Multimodal Emotion Recognition and Assistive Driving Perception

The field of multimodal emotion recognition is moving towards more nuanced and effective analysis of human emotions across diverse communication channels. Recent developments have focused on leveraging evolutionary optimization, cross-modal knowledge transfer, and multi-task learning to improve performance. Noteworthy papers in this area include the Hierarchical Adaptive Expert for Multimodal Sentiment Analysis, which achieves state-of-the-art results on multiple benchmark datasets. Another significant contribution is the Behavior-Aware MLLM-based Framework for Multimodal Emotion Recognition in Conversation, which incorporates video-derived behavior information to facilitate more accurate emotion predictions. The MMTL-UniAD framework also demonstrates impressive results by jointly learning driver behavior, emotion, vehicle behavior, and traffic context in assistive driving perception. These innovative approaches underscore the importance of capturing complex multimodal interactions and generalizing across different emotional contexts.

Sources

Hierarchical Adaptive Expert for Multimodal Sentiment Analysis

Unimodal-driven Distillation in Multimodal Emotion Recognition with Dynamic Fusion

BeMERC: Behavior-Aware MLLM-based Framework for Multimodal Emotion Recognition in Conversation

MMTL-UniAD: A Unified Framework for Multimodal and Multi-Task Learning in Assistive Driving Perception

Built with on top of