Back to articles
Proceedings
Volume: 36 | Article ID: IPAS-246
Image
Towards Realistic Landmark-Guided Facial Video Inpainting Based on GANs
  DOI :  10.2352/EI.2024.36.10.IPAS-246  Published OnlineJanuary 2024
Abstract
Abstract

Facial video inpainting plays a crucial role in a wide range of applications, including but not limited to the removal of obstructions in video conferencing and telemedicine, enhancement of facial expression analysis, privacy protection, integration of graphical overlays, and virtual makeup. This domain presents serious challenges due to the intricate nature of facial features and the inherent human familiarity with faces, heightening the need for accurate and persuasive completions. In addressing challenges specifically related to occlusion removal in this context, our focus is on the progressive task of generating complete images from facial data covered by masks, ensuring both spatial and temporal coherence. Our study introduces a network designed for expression-based video inpainting, employing generative adversarial networks (GANs) to handle static and moving occlusions across all frames. By utilizing facial landmarks and an occlusion-free reference image, our model maintains the users identity consistently across frames. We further enhance emotional preservation through a customized facial expression recognition (FER) loss function, ensuring detailed inpainted outputs. Our proposed framework exhibits proficiency in eliminating occlusions from facial videos in an adaptive form, whether appearing static or dynamic on the frames, while providing realistic and coherent results.

Subject Areas :
Views 45
Downloads 8
 articleview.views 45
 articleview.downloads 8
  Cite this article 

Fatemeh Ghorbani Lohesara, Karen Eguiazarian, Sebastian Knorr, "Towards Realistic Landmark-Guided Facial Video Inpainting Based on GANsin Electronic Imaging,  2024,  pp 246-1 - 246-6,  https://doi.org/10.2352/EI.2024.36.10.IPAS-246

 Copy citation
  Copyright statement 
Copyright © 2024, Society for Imaging Science and Technology 2024
ei
Electronic Imaging
2470-1173
2470-1173
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA