Advances in Image Inpainting: Quality, Consistency, and Efficiency

The field of image inpainting is witnessing significant advancements driven by innovations in diffusion models and their applications. Recent developments focus on enhancing the quality and diversity of inpainted images, addressing semantic consistency, and improving the efficiency of the inpainting process. Techniques such as variational inference, anisotropic Gaussian splatting, and feature dequantization are being employed to refine the structural integrity and detail of inpainted images. Additionally, methods for controlling latent diffusion models and optimizing inference speed through parallel processing are emerging as key areas of innovation. These advancements not only improve the visual plausibility of inpainted content but also expand the practical applications of inpainting technologies, such as in interactive and real-time scenarios. Notably, the integration of multimodal large language models for generating inpainting prompts and the development of adaptive inpainting methods that consider user input habits are pushing the boundaries of what is possible in this domain.

Noteworthy Papers:

  • A hierarchical variational inference algorithm significantly outperforms previous approaches in image inpainting, offering both plausibility and diversity.
  • A novel method combining diffusion models with anisotropic Gaussian splatting produces high-fidelity inpainting results with enhanced structural integrity.
  • An efficient training method for feature dequantization significantly enhances detail quality in generated images with minimal overhead.

Sources

VIPaint: Image Inpainting with Pre-Trained Diffusion Models via Variational Inference

I Dream My Painting: Connecting MLLMs and Diffusion Models via Prompt Generation for Text-Guided Multi-Mask Inpainting

FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpainting

Improving Detail in Pluralistic Image Inpainting with Feature Dequantization

PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control

Diffusion Models with Anisotropic Gaussian Splatting for Image Inpainting

AccDiffusion v2: Towards More Accurate Higher-Resolution Diffusion Extrapolation

Controlling the Latent Diffusion Model for Generative Image Shadow Removal via Residual Generation

Partially Conditioned Patch Parallelism for Accelerated Diffusion Model Inference

Implicit Priors Editing in Stable Diffusion via Targeted Token Adjustment

Pinco: Position-induced Consistent Adapter for Diffusion Transformer in Foreground-conditioned Inpainting

Built with on top of