Quantification of the image sensor signal and noise is essential to derive key image quality performance indicators. Image sensors in automotive cameras are predominately activated in high dynamic range (HDR) mode, however, legacy procedures to quantify image sensor noise were optimized for operation in standard dynamic range mode. This work discusses the theoretical background and the workflow of the photon-transfer curve (PTC) test. Afterwards, it presents example implementations of the PTC test and its derivatives according to legacy procedures and according to procedures that were optimized for image sensors in HDR mode.
Smartphones with 100 million pixel sensor are on the market. After that, it is expected to mount a higher resolution mobile camera module of 200 million pixels or more. In order to develop high resolution sensor products by mounting more pixels in a limited space, it is necessary to reduce the size of the pixels. There are currently sensors on the market with 0.64um pixels. It is expected that sensors with smaller pixels will be developed in the future. In terms of image quality, the smaller the pixel size, the smaller the amount of light received. Therefore, the image quality deteriorates in terms of noise and crosstalk. To overcome this limitation, various high sensitivity sensors are being developed, and it is advantageous to mount Nano Prism in the development of high sensitivity sensor. In this paper, we introduce the image quality performance of CMOS image sensor equipped with Nano Prism.
Recently, with the release of 108 mega pixel resolution image sensor, the photo quality of smartphone camera, including detail, and texture, is getting much higher. This became possible only because by utilizing the remosaic technology which re-organize color filter arrays into the Bayer patterns compatible to existing Image Signal Processor (ISP) of commodity AP. However, the optimized parameter configurations of the remosaic block require lots of efforts and long tuning period in order to secure the desired image quality level and sensor characteristics. This paper proposes a deep neural network based camera auto-tuning system for the remosaic ISP block. Firstly, considering the learning phase, big image quality database is created in the random way using reference image and tuning register. Second, the virtual ISP model has been trained in order that predicts image quality by changing sensor tuning registers. Finally, the optimization layer generates the sensor remosaic parameters in order to achieve the user’s target image quality expectation. By experiment, the proposed system has been verified to secure the image quality at the level of professionally hand-tuned photography. Especially, the remosaic artifact of false color, color desaturation and line broken artifacts are improved significantly by more than 23%, 4%, and 12%, respectively.
In camera development, because the image quality is subjective and the tuning complexity is increasing, building a correlated model with image signal processor (ISP) pipeline is very demanding task. In order to overcome those problems, this paper proposes an automatic image quality tuning framework based on Deep Neural Network (DNN). The image quality metric (IQM) have been defined to quantifies subjective image quality, which effectively represents the actual user perception. In this way, fast reproduction of the desired image has been possible through the minimized computing resource. Proposed Optimization methodology consists of Phase 1, a ISP modeling, and Phase 2, parameter optimization. Phase 1 construct a model between the parameters of ISP and the image quality metric. At phase 2, we add partially connected layer at input layer in order to optimize the parameters of ISP. Using backpropagation approach, the network selectively updates only the weights of partial connections, which allow to automatically derive the optimal parameters for high quality image. This idea has been implemented and experimented through commercial 16 Mega pixel resolution CMOS image sensor (CIS) and the state-of-the art ISP.