Recovering badly damaged face images is a useful yet challenging task, especially in extreme cases where the masked or damaged region is very large. One of the major challenges is the ability of the system to generalize on faces outside the training dataset. We propose to tackle
this extreme inpainting task with a conditional Generative Adversarial Network (GAN) that utilizes structural information, such as edges, as a prior condition. Edge information can be obtained from the partially masked image and a structurally similar image or a hand drawing. In our proposed
conditional GAN, we pass the conditional input in every layer of the encoder while maintaining consistency in the distributions between the learned weights and the incoming conditional input. We demonstrate the effectiveness of our method with badly damaged face examples.
Journal Title : Electronic Imaging
Publisher Name : Society for Imaging Science and Technology
Publisher Location : IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA
Nilesh Pandey, Andreas Savakis, "Extreme Face Inpainting with Sketch-Guided Conditional GAN" in Proc. IS&T Int’l. Symp. on Electronic Imaging: Computational Imaging XIX,2021,pp 23-1 - 23-6, https://doi.org/10.2352/ISSN.2470-1173.2021.15.COIMG-023
IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA
10.2352/ISSN.2470-1173.2021.15.COIMG-023
2470-1173(20210118)2021:15L.231;1-
ei_24701173_v2021n15_Input/s2.xml
/ist/ei/2021/00002021/00000015/art00002
Articles
Extreme Face Inpainting with Sketch-Guided Conditional GAN
PandeyNilesh
SavakisAndreas
18012021
2021
15
23-1
23-6
2021
Recovering badly damaged face images is a useful yet challenging task, especially in extreme cases where the masked or damaged region is very large. One of the major challenges is the ability of the system to generalize on faces outside the training dataset. We propose to tackle
this extreme inpainting task with a conditional Generative Adversarial Network (GAN) that utilizes structural information, such as edges, as a prior condition. Edge information can be obtained from the partially masked image and a structurally similar image or a hand drawing. In our proposed
conditional GAN, we pass the conditional input in every layer of the encoder while maintaining consistency in the distributions between the learned weights and the incoming conditional input. We demonstrate the effectiveness of our method with badly damaged face examples.