New, high-resolution CMOS image sensors for mobile phones have moved beyond the single-binnable Quad Bayer and RGBW-Kodak patterns to the double binnable Hexadeca Bayer pattern featuring 4x4 tiles of like-colored pixels. Pixel binning enables high-speed, low-power readout in low-resolution modes and, more importantly, a reduction of read noise via floating diffusion binning. In this paper, we present the non-intuitive result that Nona and Hexadeca Bayer can be superior to Quad Bayer in demosaicking quality due to degeneracies in the latters spectrum. However, Hexadeca Bayer suffers from the weakness of generating Quad Bayer after one round of binning. We introduce novel double binnable RGBW and LMS CFAs, composed of 2x2 tiles capable of 4:1 floating diffusion binning, that are free from spectral degeneracies and thus demosaic well in full resolution and both binned modes. RGBW offers a 6 dB low-light, read noise-limited, SNR advantage over Quad and Hexadeca Bayer in all resolution modes, while for LMS, the corresponding advantage is 4.2 dB.
Consumer color cameras employ sensors that do not mimic human cone spectral sensitivities, and more generally do not meet the Luther condition since the accompanying color correction substantially amplifies noise in the red channel. This begs the question: if cone spectral sensitivities yield low SNR, why has the Human Visual System so evolved? We answer the above question by noting that since modern ISPs - and the ancient HVS - remove virtually all chrominance noise, chrominance denoising artifacts rather than the chrominance noise itself should be considered. While sensor green, blue are reasonable analogs of the human M, S cones, the spectral sensitivity of red is much narrower than that of L and does not overlap much with green. An imager employing L instead of red suffers from increased red noise but is also more sensitive. This allows a high SNR (L + M)/2 luminance image to be reconstructed and used for denoising. Modeling the color filter array on the human retina, with a higher density of L pixels at the expense of S pixels, further improves the red SNR without the accompanying loss of blue quality being perceptible. The resulting LMS camera outperforms conventional RGB cameras in color accuracy and luminance SNR while being competitive in chrominance quality.
Modern digital cameras capture images with a subsampling method via mosaic color filter array (CFA). Of particular interest is the CFA with Cyan-Magenta-Yellow (CMY) color filters. Despite the improvement of the sensitivity, images reconstructed from CMY CFA usually suffer from lower color fidelity compared to the conventional Red-Green-Blue (RGB) color filters. In this paper, we proposed a CMY CFA with novel spectral sensitivities (CMY2.0), which were carefully designed to overcome the shortcomings of previous CMY CFA (CMY1.0). A CMY CMOS image sensor (CIS) with such optimized spectral sensitivities is then realized in order to evaluate the color performance and signal-to-noise ratio (SNR). As a result, the camera equipped with the CMY CFA with proposed spectral sensitivities (CMY2.0) features both an improved sensitivity and a high color fidelity, which is suitable for a wide range of applications, such as low-light photography, under-screen cameras and automotive cameras.