The recent advancements in spatial knowledge understanding and generative models have significantly enhanced the synthesis of immersive audio and visual experiences. Researchers are increasingly integrating multi-source spatial data, such as depth images and semantic captions, to improve the realism and spatial fidelity of generated content. This trend is evident in the development of models that not only synthesize audio based on spatial and environmental conditions but also generate images and text that are spatially coherent. The use of dual learning frameworks and diffusion models has shown particular promise in achieving synergistic effects between image-to-text and text-to-image tasks, leveraging shared 3D spatial features. Additionally, the evaluation of text-to-audio generation models has become more sophisticated, incorporating both objective metrics and perceptual assessments to ensure the quality and controllability of synthesized soundscapes. These developments collectively point towards a future where immersive experiences are more seamlessly integrated and evaluated across various modalities.

Noteworthy papers include one that introduces a novel multi-source spatial knowledge understanding scheme for immersive Visual Text-to-Speech, significantly enhancing the spatial speech experience, and another that presents a generative spatial audio model capable of producing high-fidelity 3D soundscapes based on diverse user inputs.