Multimodal Learning Advances

The field of multimodal learning is moving towards developing more robust and flexible models that can handle incomplete or missing data. Researchers are exploring new frameworks and techniques to improve the performance of multimodal models in real-world scenarios. A key direction is the development of methods that can learn from multiple modalities and adapt to new or missing modalities. Another area of focus is the improvement of recommendation systems, with several papers proposing new approaches to enhance the performance of multimodal recommender systems. Notable papers include Harmony, which introduces a unified framework for modality incremental learning, and SURE, which enhances multimodal pretraining with missing modalities through uncertainty estimation. Other innovative papers include CoCoRec, which uses consensus-aware contrastive learning for group recommendation, and GUIDER, which proposes a universal framework for denoising multi-modal recommender systems.

Sources

Harmony: A Unified Framework for Modality Incremental Learning

Are you SURE? Enhancing Multimodal Pretraining with Missing Modalities through Uncertainty Estimation

Consensus-aware Contrastive Learning for Group Recommendation

Teach Me How to Denoise: A Universal Framework for Denoising Multi-modal Recommender Systems via Guided Calibration

Hierarchical Attention Fusion of Visual and Textual Representations for Cross-Domain Sequential Recommendation

Disentangling and Generating Modalities for Recommendation in Missing Modality Scenarios

Modality Reliability Guided Multimodal Recommendation

MMHCL: Multi-Modal Hypergraph Contrastive Learning for Recommendation

Quadratic Interest Network for Multimodal Click-Through Rate Prediction

Built with on top of