Back to articles
Volume: 28 | Article ID: art00032
Camera ISP Modification to Enable Image De-rendering
  DOI :  10.2352/issn.2169-2629.2020.28.31  Published OnlineNovember 2020

A camera's image signal processor (ISP) is dedicated hardware that performs a series of processing steps to render a captured raw sensor image to its final display-referred output suitable for viewing and sharing. It is often desirable to be able to revert – or de-render – the ISP-processed image back to the original raw sensor image. Undoing the ISP rendering, however, is not an easy task. This is because ISPs perform many nonlinear routines in the rendering pipeline that are difficult to invert. Moreover, modern cameras often apply scene-specific image processing, resulting in a wide range of possible ISP parameters. In this paper, we propose a modification to the ISP that allows the ISP-rendered image to be reverted back to a raw image. Our approach works by appending a fixed-sampling of the raw sensor values to all captured images. The appended raw samples comprise no more than 8 rows of pixels in the full-sized image and represent a negligible overhead given that 12–16 MP sensors typically have 3000 rows of pixels or more. The appended pixels are rendered along with the captured image to the final output. From these rendered raw samples, a reverse mapping function can be computed to undo the ISP processing. We demonstrate that this method performs almost on par with competing state-ofthe-art approaches for ISP de-rendering while offering a practical solution that is integrable to current camera ISP hardware.

Subject Areas :
Views 21
Downloads 13
 articleview.views 21
 articleview.downloads 13
  Cite this article 

Abhijith Punnappurath, Michael S. Brown, "Camera ISP Modification to Enable Image De-renderingin Proc. IS&T 28th Color and Imaging Conf.,  2020,  pp 199 - 204,

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2020
Color and Imaging Conference
color imaging conf
Society for Imaging Science and Technology