Advancements in Speech Synthesis, Voice Detection, and Model Generalization

The recent developments in the research area focus on enhancing the capabilities of speech synthesis, voice detection, and conversion, as well as improving the generalization and robustness of models across various domains and datasets. Innovations include the adaptation of multi-modal self-supervised models for text prediction from real-time MRI, the introduction of disentanglement frameworks for domain-agnostic artifact features in AI-synthesized voice detection, and the development of speaker-adaptive TTS frameworks that leverage prosody prompting for stable synthesis. Additionally, there's a significant push towards improving model generalization in low-resource scenarios, such as human activity recognition and respiratory sound classification, through novel data augmentation techniques and transformer-based contrastive meta-learning approaches. The field is also seeing advancements in multi-band massive MIMO transmission with the extrapolation of channel fingerprints using cycle-consistent generative networks, and the generation of synthetic radio-frequency data for data augmentation. Lastly, robust COVID-19 detection from cough sounds using deep neural decision trees and forests highlights the potential of machine learning in medical diagnostics.

Noteworthy Papers

  • MRI2Speech: Introduces a novel approach for speech synthesis from articulatory movements recorded by real-time MRI, significantly improving intelligibility and generalization to unseen speakers.
  • Improving Generalization for AI-Synthesized Voice Detection: Presents an innovative disentanglement framework that enhances model generalization across different domains, outperforming state-of-the-art methods.
  • Stable-TTS: A novel speaker-adaptive TTS framework that achieves prosody consistency and effective timbre capture, demonstrating effectiveness even with limited and noisy target speech samples.
  • Transformer-Based Contrastive Meta-Learning For Low-Resource Generalizable Activity Recognition: Proposes TACO, a transformer-based approach that synthesizes virtual target domains to address distribution shifts in human activity recognition.
  • CF-CGN: Introduces a method for extrapolating channel fingerprints for multi-band massive MIMO transmission, showing excellent generalization ability and improving sum rate performance.
  • Lungmix: A data augmentation technique that significantly enhances model generalization in respiratory sound classification by blending waveforms and interpolating labels.
  • ReFormer: A generative AI model for generating synthetic radio-frequency data, demonstrating adaptability and scalability for data augmentation in real-world experiments.
  • Robust COVID-19 Detection from Cough Sounds: Leverages deep neural decision trees and forests for consistent performance across diverse cough sound datasets, highlighting the challenges and benefits of dataset integration.
  • AdaptVC: Achieves high-quality voice conversion with adaptive learning, outperforming existing models in speech quality and similarity in zero-shot scenarios.

Sources

MRI2Speech: Speech Synthesis from Articulatory Movements Recorded by Real-time MRI

Improving Generalization for AI-Synthesized Voice Detection

Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting

Transformer-Based Contrastive Meta-Learning For Low-Resource Generalizable Activity Recognition

CF-CGN: Channel Fingerprints Extrapolation for Multi-band Massive MIMO Transmission based on Cycle-Consistent Generative Networks

Lungmix: A Mixup-Based Strategy for Generalization in Respiratory Sound Classification

Ensemble of classifiers for speech evaluation

ReFormer: Generating Radio Fakes for Data Augmentation

Robust COVID-19 Detection from Cough Sounds using Deep Neural Decision Tree and Forest: A Comprehensive Cross-Datasets Evaluation

AdaptVC: High Quality Voice Conversion with Adaptive Learning

Built with on top of