Back to articles
JIST-first
Volume: 32 | Article ID: art00018
Abstract

Latest trend in image sensor technology allowing submicron pixel size for high-end mobile devices comes at very high image resolutions and with irregularly sampled Quad Bayer color filter array (CFA). Sustaining image quality becomes a challenge for the image signal processor (ISP), namely for demosaicing. Inspired by the success of deep learning approach to standard Bayer demosaicing, we aim to investigate how artifacts-prone Quad Bayer array can benefit from it. We found that deeper networks are capable to improve image quality and reduce artifacts; however, deeper networks can be hardly deployed on mobile devices given very high image resolutions: 24MP, 36MP, 48MP. In this article, we propose an efficient end-to-end solution to bridge this gap—a duplex pyramid network (DPN). Deep hierarchical structure, residual learning, and linear feature map depth growth allow very large receptive field, yielding better details restoration and artifacts reduction, while staying computationally efficient. Experiments show that the proposed network outperforms state of the art for standard and Quad Bayer demosaicing. For the challenging Quad Bayer CFA, the proposed method reduces visual artifacts better than state-of-the-art deep networks including artifacts existing in conventional commercial solutions. While superior in image quality, it is 2–25 times faster than state-of-the-art deep neural networks and therefore feasible for deployment on mobile devices, paving the way for a new era of on-device deep ISPs.

Subject Areas :
Views 25
Downloads 9
 articleview.views 25
 articleview.downloads 9
  Cite this article 

Irina Kim, Seongwook Song, Soonkeun Chang, Sukhwan Lim, Kai Guo, "Deep Image Demosaicing for Submicron Image Sensorsin Proc. IS&T Int’l. Symp. on Electronic Imaging: Imaging Sensors and Systems,  2019,  pp 60410-1 - 60410-12,  https://doi.org/10.2352/J.ImagingSci.Technol.2019.63.6.060410

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2020
72010604
Electronic Imaging
2470-1173
Society for Imaging Science and Technology
7003 Kilworth Lane, Springfield, VA 22151 USA