Crynodeb
In this paper, we investigate the
inpainting of normal maps that were captured from a lightstage.
Occlusion of parts of the face during performance capture can be caused
by the movement of, e.g., arms, hair, or props. Inpainting is the
process of interpolating missing areas of an image with plausible data.
We build on previous works about general image inpainting that use
generative adversarial networks (GANs). We extend our previous work on
normal map inpainting to use a U-Net structured generator network. Our
method takes into account the nature of the normal map data and so
requires modification of the loss function. We use a cosine loss rather
than the more common mean squared error loss when training the
generator. Due to the small amount of training data available, even when
using synthetic datasets, we require significant augmentation, which
also needs to take account of the particular nature of the input data.
Image flipping and inplane rotations need to properly flip and rotate
the normal vectors. During training, we monitor key performance metrics
including the average loss, structural similarity index measure (SSIM),
and peak signal-to-noise ratio (PSNR) of the generator, alongside the
average loss and accuracy of the discriminator. Our analysis reveals
that the proposed model generates high-quality, realistic inpainted
normal maps, demonstrating the potential for application to performance
capture. The results of this investigation provide a baseline on which
future researchers can build with more advanced networks and comparison
with inpainting of the source images used to generate the normal maps.
Iaith wreiddiol | Saesneg |
---|---|
Rhif yr erthygl | 56 |
Nifer y tudalennau | 19 |
Cyfnodolyn | Computers |
Cyfrol | 13 |
Rhif cyhoeddi | 2 |
Dynodwyr Gwrthrych Digidol (DOIs) | |
Statws | Cyhoeddwyd - 19 Chwef 2024 |
Ôl bys
Gweld gwybodaeth am bynciau ymchwil 'A U-Net Architecture for Inpainting Lightstage Normal Maps'. Gyda’i gilydd, maen nhw’n ffurfio ôl bys unigryw.Y Wasg / Y Cyfryngau
-
Data on Computer Science Discussed by Researchers at University College London (UCL) (A U-Net Architecture for Inpainting Lightstage Normal Maps)
13 Maw 2024
1 eitem o Sylw ar y cyfryngau
Y Wasg / Cyfryngau: Sylw yn y cyfryngau