Advances in Simulated and Real-World Modeling of Human and Garment Interactions

The field of human and garment modeling is rapidly advancing, with a focus on developing innovative methods for sim-to-real transfer, dynamic garment deformation, and virtual try-on. Researchers are exploring the use of diffusion models, physics-based simulations, and keypoints to improve the accuracy and generalizability of these models. A key direction in this field is the development of compact and shape-agnostic representations of cloth states, which can be applied to various applications such as semantic labeling and planning. Notable papers in this area include:

  • DiSRT-In-Bed, which proposes a sim-to-real transfer framework for in-bed human mesh recovery, and
  • D-Garment, which introduces a physics-conditioned latent diffusion model for dynamic garment deformations. These advancements have the potential to significantly improve the realism and accuracy of virtual try-on, garment modeling, and human mesh recovery, with applications in healthcare, entertainment, and gaming.

Sources

DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery

D-Garment: Physics-Conditioned Latent Diffusion for Dynamic Garment Deformations

From Keypoints to Realism: A Realistic and Accurate Virtual Try-on Network from 2D Images

CloSE: A Compact Shape- and Orientation-Agnostic Cloth State Representation

Built with on top of