Advances in 3D Reconstruction and Scene Generation

The recent advancements in the field of 3D reconstruction and scene generation have seen a significant shift towards leveraging diffusion models and multi-view geometry. These innovations are pushing the boundaries of what can be achieved with single-image inputs, enabling more coherent and detailed 3D scene reconstructions. The integration of probabilistic approaches and hierarchical models is allowing for more accurate and robust generation of 3D structures from 2D images. Additionally, the use of video diffusion models for tasks such as relighting and scene generation is demonstrating the potential for physically-based rendering without the need for explicit 3D asset reconstruction. The field is also witnessing a rise in automated methods for VR scene generation and 3D object reconstruction, which are streamlining the process and enhancing the realism of virtual environments. Notably, the synergy between depth-edge alignment and visibility-aware patch deformation is improving the accuracy and robustness of multi-view stereo methods. Furthermore, the introduction of novel frameworks that combine rendering and inverse rendering within a single diffusion model is paving the way for more efficient and unified solutions in both computer vision and graphics. These developments collectively highlight a trend towards more automated, efficient, and high-quality 3D content creation, driven by advancements in diffusion models and multi-view learning techniques.

Noteworthy papers include 'Probabilistic Inverse Cameras: Image to 3D via Multiview Geometry,' which introduces a geometry-driven approach to novel-view synthesis that outperforms state-of-the-art baselines, and 'Coherent 3D Scene Diffusion From a Single RGB Image,' which presents a diffusion-based method for coherent 3D scene reconstruction from a single RGB image, achieving significant improvements in accuracy.

Sources

Probabilistic Inverse Cameras: Image to 3D via Multiview Geometry

Coherent 3D Scene Diffusion From a Single RGB Image

GenLit: Reformulating Single-Image Relighting as Video Generation

AURORA: Automated Unleash of 3D Room Outlines for VR Applications

View Transformation Robustness for Multi-View 3D Object Reconstruction with Reconstruction Error-Guided View Selection

Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion

DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo

Image Gradient-Aided Photometric Stereo Network

Wonderland: Navigating 3D Scenes from a Single Image

Controllable Shadow Generation with Single-Step Diffusion Models from Synthetic Data

IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations

OpenViewer: Openness-Aware Multi-View Learning

Monocular Facial Appearance Capture in the Wild

Targeted View-Invariant Adversarial Perturbations for 3D Object Recognition

Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion

GraphicsDreamer: Image to 3D Generation with Physical Consistency

SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting

An Immersive Multi-Elevation Multi-Seasonal Dataset for 3D Reconstruction and Visualization

GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction

TDCNet: Transparent Objects Depth Completion with CNN-Transformer Dual-Branch Parallel Network

Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion

DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation

Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation

Built with on top of