The field of audio processing and speech technology is witnessing significant advancements, particularly in the areas of speech super-resolution, enhancement, and synthesis. A notable trend is the integration of generative models, such as GANs and diffusion models, with traditional convolutional and transformer architectures to improve the fidelity and quality of audio outputs. These models are increasingly being designed to operate in lower-dimensional latent spaces, reducing complexity and enhancing generalization capabilities across diverse and unseen scenarios. Additionally, there's a growing emphasis on joint modeling approaches that address multiple aspects of audio processing tasks simultaneously, such as sound event localization and detection, and the unification of speech enhancement and neural vocoding tasks under a broader framework of speech restoration. Another innovative direction is the development of zero-shot learning paradigms for singing voice synthesis and conversion, enabling control over various aspects of the singing voice based on speech references, thereby addressing the scarcity of singing data and improving output musicality.

Noteworthy Papers

• HiFi-SR: Introduces a unified transformer-convolutional adversarial network for high-fidelity speech super-resolution, significantly outperforming existing methods in both in-domain and out-of-domain scenarios.
• Conditional Latent Diffusion-Based Speech Enhancement: Proposes a novel approach integrating a conditional latent diffusion model with dual-context learning, demonstrating strong performance and superior generalization capability.
• An Experimental Study on Joint Modeling for Sound Event Localization and Detection: Presents innovative approaches to 3D SELD, ranking first in the DCASE 2024 Challenge Task 3.
• A2SB: Audio-to-Audio Schrodinger Bridges: Offers an end-to-end audio restoration model capable of bandwidth extension and inpainting, achieving state-of-the-art quality on out-of-distribution music test sets.
• Audio Texture Manipulation by Exemplar-Based Analogy: Introduces a model for audio texture manipulation using paired speech examples, outperforming text-conditioned baselines.
• Neural Vocoders as Speech Enhancers: Demonstrates that speech enhancement and neural vocoding can be unified under a broader framework of speech restoration.
• Everyone-Can-Sing: Proposes a zero-shot learning paradigm for singing voice synthesis and conversion, showing substantial improvements in timbre similarity and musicality.