The field of video generation and physical reasoning is rapidly advancing, with a focus on developing models that can generate realistic and physically plausible videos. Researchers are exploring new approaches to improve the quality and diversity of generated videos, including the use of diffusion models, kinetic codes, and retrieval mechanisms. One of the key challenges in this area is evaluating the physical plausibility of generated videos, with several papers proposing new benchmarks and evaluation metrics to address this issue. Another important direction is the development of models that can generate videos with complex motion and physical interactions, such as those involving multiple objects or characters. Overall, the field is moving towards more realistic and engaging video generation, with potential applications in fields such as robotics, autonomous driving, and scientific simulation. Noteworthy papers in this area include Morpheus, which introduces a benchmark for evaluating physical reasoning in video generation models, and RAGME, which proposes a framework for improving motion realism in generated videos through retrieval mechanisms.
Advancements in Video Generation and Physical Reasoning
Sources
Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments
How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models
Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions
Can You Count to Nine? A Human Evaluation Benchmark for Counting Limits in Modern Text-to-Video Models
Learning about the Physical World through Analytic Concepts
DanceMosaic: High-Fidelity Dance Generation with Multimodal Editability
SMF: Template-free and Rig-free Animation Transfer using Kinetic Codes
Video-Bench: Human-Aligned Video Generation Benchmark
One-Minute Video Generation with Test-Time Training
Time-adaptive Video Frame Interpolation based on Residual Diffusion
Studying Image Diffusion Features for Zero-Shot Video Object Segmentation
Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling
VC-LLM: Automated Advertisement Video Creation from Raw Footage using Multi-modal LLMs
PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning
Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling
A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model
HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance
CamContextI2V: Context-aware Controllable Video Generation
STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints
RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism
Probability Density Geodesics in Image Diffusion Latent Space
Compass Control: Multi Object Orientation Control for Text-to-Image Generation
EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation
GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography