Advances in Music and Dance Generation

The field of music and dance generation is rapidly evolving, with a focus on developing innovative methods to generate high-quality, synchronized audio and dance movements. Recent developments have seen the proposal of novel frameworks, such as those utilizing gating mechanisms, compression-based techniques, and chain-of-thought prompting, to improve the alignment and creativity of generated music and dance.

Noteworthy papers in this area include LZMidi, which introduces a lightweight symbolic music generation framework based on a Lempel-Ziv-induced sequential probability assignment, achieving competitive results with state-of-the-art diffusion models while significantly reducing computational overhead. Another notable paper is MusiCoT, which proposes a novel chain-of-thought prompting technique tailored for music generation, empowering autoregressive models to outline an overall music structure before generating audio tokens, thereby enhancing the coherence and creativity of the resulting compositions.

Sources

Align Your Rhythm: Generating Highly Aligned Dance Poses with Gating-Enhanced Rhythm-Aware Feature Representation

LZMidi: Compression-Based Symbolic Music Generation

CCMusic: An Open and Diverse Database for Chinese Music Information Retrieval Research

Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generation

A Study on the Matching Rate of Dance Movements Using 2D Skeleton Detection and 3D Pose Estimation: Why Is SEVENTEEN's Performance So Bita-Zoroi (Perfectly Synchronized)?

Vision-to-Music Generation: A Survey

Tune It Up: Music Genre Transfer and Prediction

Enhancing Dance-to-Music Generation via Negative Conditioning Latent Diffusion Model

Enhance Generation Quality of Flow Matching V2A Model via Multi-Step CoT-Like Guidance and Combined Preference Optimization

DeepSound-V1: Start to Think Step-by-Step in the Audio Generation from Videos

Built with on top of