Solid state optical sensors and solid state cameras have established themselves as the imaging systems of choice for many demanding professional applications such as automotive, space, medical, scientific and industrial applications. The advantages of low-power, low-noise, high-resolution, high-geometric fidelity, broad spectral sensitivity, and extremely high quantum efficiency have led to a number of revolutionary uses. ISS focuses on image sensing for consumer, industrial, medical, and scientific applications, as well as embedded image processing, and pipeline tuning for these camera systems. This conference will serve to bring together researchers, scientists, and engineers working in these fields, and provides the opportunity for quick publication of their work. Topics can include, but are not limited to, research and applications in image sensors and detectors, camera/sensor characterization, ISP pipelines and tuning, image artifact correction and removal, image reconstruction, color calibration, image enhancement, HDR imaging, light-field imaging, multi-frame processing, computational photography, 3D imaging, 360/cinematic VR cameras, camera image quality evaluation and metrics, novel imaging applications, imaging system design, and deep learning applications in imaging.

Due to the emergence of machine vision, augmented reality (AR), virtual reality (VR), and automotive connectivity in recent years, the necessity for chip miniaturization has grown. These emerging, next-generation applications, which are centered on user experience and comfort, require their constituent chips, devices, and parts to be smaller, lighter, and more accessible. AR/VR applications, especially demand smaller components due to their primary application towards wearable technology, in which the user experience would be negatively impacted by large features and bulk. Therefore, chips and devices intended for next-generation consumer applications must be small and modular, to support module miniaturization and promote user comfort. To enable the chip miniaturization required for technological advancement and innovation, we developed a 2.2μm pixel pitch Back Side Illuminated (BSI) Voltage Domain Global Shutter (VDGS) image sensor with the three-wafer stacked technology. Each wafer is connected by Stacked Pixel Level Connection (SPLC) and the middle and logic wafers are connected using a Back side Through Silicon Via (BTSV). The separation of the sensing, charge storage, and logic functions to different wafers allows process optimization in each wafer, improving overall chip performance. The peripheral circuit region is reduced by 75% compared to the previous product without degrading image sensor performance. For the session: Processing at the Edge (joint with COIMG).

A sequential transfer-gate and photodiode optimization method for CMOS Image sensors are described in this paper, which enables the design of large-scale ultra-high-speed burst mode CMOS Image sensors in a low-cost standard CMOS Image sensor process without the need for process customization or advanced process. The sequential transfer gates also show a clear advantage in minimizing the floating diffusion capacitance and improving image sensor conversion gain in large-scale pixels.

A self-powered asynchronous sensor with a novel pixel architecture is presented. Pixels are autonomous and can harvest or sense energy independently. During the image acquisition, pixels toggle to a harvesting operation mode once they have sensed their local illumination level. With the proposed pixel architecture, most illuminated pixels provide an early contribution to power the sensor, while low-illuminated ones spend more time sensing their local illumination. Thus, the equivalent frame rate is higher than the one offered by conventional self-powered sensors that harvest and sense illumination in independent phases. The proposed sensor uses a Time-to-First-Spike readout that allows trading between image quality and data and bandwidth consumption. The device has HDR operation with a dynamic range of 80 dB. Pixel power consumption is only 70 pW. The article describes the sensor’s and pixel’s architectures in detail. Experimental results are provided and discussed. Sensor specifications are benchmarked against the art.

A camera obscura is a darkened chamber in which an image of the scene outside the chamber is projected by a pinhole or other optic onto a screen within the chamber. Early obscuras used pinhole optics, but by the 16th century obscuras with lenses became popular as aids for drawing or painting scenes with the correct perspective. By the late 19th century, the screen had largely been replaced with photo-sensitive materials, and film cameras replaced obscuras. Over the last few decades, digital cameras using electronic sensors have replaced those using film. However, large projections can have significantly different properties from small projections, and it is very difficult to build a large digital image sensor. Thus, there is interest in using a small-sensor digital camera to photograph the large screen of a obscura. For example, it is relatively easy to obtain much shallower depth of field using a large screen, but a small sensor photographing the screen essentially copies that depth of field, so obscuras have often been used as “bokeh adapters” for small-sensor digital cameras. The current work is an experimentally-grounded exploration of the issues that arise in construction of digital camera obscuras, their use, and exposure control and postprocessing of digital images captured using a small sensor to photograph the image projected on an obscura’s screen.

In recent years, one-shot cameras that integrate Multispectral Filter Arrays (MSFA) are used to acquire multispectral images. In a previous paper, we have proposed a multispectral image recognition system based on this type of camera. The images acquired with these cameras are then demosaiced. Multispectral facial images acquired with our MSFA one-shot camera present information redundancy which leads to a strong correlation between bands. A dimensionality reduction is necessary to reduce information redundancy. Dimensionality reduction is a set of techniques that allow to project an initial image of dimension n into a final image of dimension p, while preserving its relevant information. This paper proposes an improvement of facial recognition system using the Multispectral Filter Array one shot camera. A dimensionality reduction module has been added to the system. A comparison of the performance of different dimensionality reduction methods based on the eigenvalues, and VGG19 classification results are conducted. Experimental results on the EXIST database made up with our camera indicate a good decorrelation of the bands leading to the reduction of bands from eight to three with the Karhuen-Love transform an accuracy of 100% with VGG19 and a 15 % gain in processing time.

Up to now, backside illuminated (BSI) CMOS miniaturized pixels have been used and manufactured for visible (Vis) and/or near infrared (NIR) light. This work is focused on the performance under UV light, in the range of [200 nm, 400 nm], of such BSI CMOS miniaturized pixels, initially developed for the Vis and NIR spectrum in mind, which have been understudied until now. This performance evaluation is based on quantum efficiency (QE) measurements of various pixel types to examine how the good signal to noise ratio (SNR) in the Vis is modified in the UV spectrum. The pixels measured in this campaign are all miniaturized backside illuminated (BSI) CMOS because they have a better light to charge conversion compared to their frontside illuminated counterparts: the architecture offers the advantage of direct access the Si substrate and having a variety of possible thinner antireflection coating stacks (ARC) thanks to the evolution of CMOS BSI passivation techniques. Optical simulations had been performed to identify the key parameters that could play a role for the pixel’s response improvement in the UV. Despite the lack of any process optimization, we can observe significant response of the sensor under UV.

Modern digital cameras capture images with a subsampling method via mosaic color filter array (CFA). Of particular interest is the CFA with Cyan-Magenta-Yellow (CMY) color filters. Despite the improvement of the sensitivity, images reconstructed from CMY CFA usually suffer from lower color fidelity compared to the conventional Red-Green-Blue (RGB) color filters. In this paper, we proposed a CMY CFA with novel spectral sensitivities (CMY2.0), which were carefully designed to overcome the shortcomings of previous CMY CFA (CMY1.0). A CMY CMOS image sensor (CIS) with such optimized spectral sensitivities is then realized in order to evaluate the color performance and signal-to-noise ratio (SNR). As a result, the camera equipped with the CMY CFA with proposed spectral sensitivities (CMY2.0) features both an improved sensitivity and a high color fidelity, which is suitable for a wide range of applications, such as low-light photography, under-screen cameras and automotive cameras.

A reset noise reduction method using a feedback amplifier that results in an 80% noise reduction in 3-transistor (3-T) pixels is presented. 3-T pixels are useful for non-visible imaging applications because they have fewer post-processing issues than 4-T pixels and do not require charge transfer. They suffer from reset noise because correlated-double sampling cannot be realized without additional memory. Analysis of the experimental power spectral density indicates potential for further noise cancellation in future devices.

Computer vision algorithms are often burdened for embedded hardware implementation due to integration time and system complexity. Many commercial systems prevent low-level image processing customization and hardware optimization due to the largely proprietary nature of the algorithms and architectures, hindering research development by the larger community. This work presents DevCAM- an open-source multi-camera environment, targeted at hardware-software research for vision systems, specifically for co-located multi-sensor processor systems. The objective being to facilitate the integration of multiple latest generation sensors, abstracting interfacing difficulties to high-bandwidth sensors, enable user defined hybrid processing architectures on FPGA, CPU and GPU, and to unite multi-module systems with networking and high-speed storage storage. The system architecture can accommodate up to six 4-lane MIPI sensor modules which are electronically synchronized, alongside support for an RTK-GPS receiver and a 9-axis IMU. We demonstrate a number of available configurations that can be achieved for stereo, quadnocular, 360, and light-field image acquisition tasks. The development framework includes mechanical, PCB, FPGA and software components for the rapid integration into any system. System capabilities are demonstrated with the focus on opening new research frontiers such as distributed edge processing, inter system synchronization, sensor synchronization, and hybrid hardware acceleration of image processing tasks.