Controllable and Interactive AI Content Generation

The recent advancements in generative AI have significantly pushed the boundaries of content creation across various dimensions, particularly in 3D and 4D scene generation, interactive video control, and gaze-driven content manipulation. The field is witnessing a shift towards more controllable and personalized content generation, enabled by innovative models that integrate real-time user interaction and multi-agent collaboration. These developments are paving the way for more intuitive and efficient content creation workflows, where user input and AI-driven creativity are seamlessly combined. Notable innovations include frameworks that allow for the generation of complex 3D and 4D scenes from minimal input, systems that enable gaze-based control over visual content, and multi-agent systems that enhance the consistency and customization of storytelling video generation. These advancements not only improve the quality and realism of generated content but also expand the possibilities for user interaction and control, making AI-generated content more accessible and adaptable to individual needs.

Sources

GameGen-X: Interactive Open-world Game Video Generation

GenXD: Generating Any 3D and 4D Scenes

GazeGen: Gaze-Driven User Interaction for Visual Content Generation

StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion

SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation

ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning

Built with on top of