NeRF-In: Free-Form Inpainting for Pretrained NeRF With RGB-D Priors
Journal
IEEE Computer Graphics and Applications
Pages
1-11
Date Issued
2023-11
Author(s)
Abstract
Neural Radiance Field (NeRF) has emerged as a versatile scene representation. However, it is still unintuitive to edit a pretrained NeRF because the network parameters and the scene appearance are often not explicitly associated. In this paper, we introduce the first framework that enables users to retouch undesired regions in a pretrained NeRF scene without accessing any training data and category-specific data prior. The user first draws a free-form mask to specify a region containing the unwanted objects over an arbitrary rendered view from the pretrained NeRF. Our framework transfers the user-drawn mask to other rendered views and estimates guiding color and depth images within transferred masked regions. Next, we formulate an optimization problem that jointly inpaints the image content in all masked regions by updating NeRF's parameters. We demonstrate our framework on diverse scenes and show it obtained visually plausible and structurally consistent results using less user manual efforts.
Subjects
Cameras | Image color analysis | Optimization | Solid modeling | Three-dimensional displays | Training | Videos
Type
journal article