Report on Current Developments in Photometric Stereo and Related Fields

General Direction of the Field

Recent advancements in photometric stereo and related areas have significantly shifted towards more efficient and physically grounded methods for material estimation, relighting, and albedo recovery. The field is witnessing a strong emphasis on integrating deep learning techniques with physically-based models to address the inherent complexities of data acquisition and processing in photometric stereo. This integration aims to enhance the accuracy and robustness of surface normal estimation, material reconstruction, and relighting, even under challenging conditions such as indirect illumination and complex scene geometries.

One of the key trends is the development of single-shot or minimal-shot approaches that reduce the dependency on elaborate data acquisition setups. These methods leverage attention-based neural networks and inverse rendering techniques to achieve high-quality results from limited input data. The use of synthetic datasets for training, combined with the ability to generalize to real-world images, is another notable direction. This approach not only simplifies the data acquisition process but also improves the adaptability of models to various lighting conditions and material properties.

In addition to these advancements, there is a growing interest in optimizing illumination planning for photometric stereo. Learning-based methods are being employed to determine optimal lighting configurations that minimize normal estimation errors, thereby enhancing the overall performance of photometric stereo systems. These methods are particularly valuable in scenarios where exhaustive sampling of lighting directions is impractical due to time and resource constraints.

Another emerging area is the recovery of albedo from aerial photogrammetric images. This is crucial for enhancing the realism of 3D models and digital twins. Recent approaches focus on developing physics-based models that leverage estimable scene geometry and natural illumination to resolve albedo information through inverse rendering. These methods not only improve the quality of 3D models but also enhance various image processing tasks such as feature matching and edge extraction.

Noteworthy Papers

• MERLiN: Introduces an attention-based hourglass network that integrates single image-based inverse rendering and relighting, demonstrating high-quality results on both synthetic and real-world images.
• LIPIDS: Proposes a learning-based approach to optimize lighting configurations for photometric stereo, showing competitive performance with minimal light setups.
• Albedo Recovery Approach: Presents a physics-based method for albedo recovery from aerial photogrammetric images, significantly outperforming existing approaches and enhancing various image processing tasks.