An extension of automotive imaging from the visible (VIS) to the near infrared (NIR) spectrum is promising for driving automation applications because the technology is readily available and offers potential benefits in low visibility conditions, in low light conditions with active illumination, and by collection of complementary data. We propose the evaluation of VIS-NIR imaging in simulation using an extended version of our camera simulation and optimization framework. Our extended framework generates realistic spectral irradiance data of synthetic scenes in the VIS and NIR spectral range and includes physically based camera models with characteristic increased NIR sensitivity of VIS-NIR CMOS imagers, modified automotive VIS-NIR color filter arrays and adapted image processing. We evaluate the reproduction of potential benefits of VIS-NIR imaging in our simulated camera images using exemplary night time and daylight traffic scenes, and discuss further extensions for creation of a well-balanced VIS-NIR dataset for quantitative evaluation.
Full driving automation imposes to date unmet performance requirements on camera and computer vision systems, in order to replace the visual system of a human driver in any conditions. So far, the individual components of an automotive camera hav mostly been optimized independently, or without taking into account the effect on the computer vision applications. We propose an end-to-end optimization of the imaging system in software, from generation of radiometric input data over physically based camera component models to the output of a computer vision system. Specifically, we present an optimization framework which extends the ISETCam and ISET3d toolboxes to create synthetic spectral data of high dynamic range, and which models a stateof-the-art automotive camera in more detail. It includes a stateof-the-art object detection system as benchmark application. We highlight in which way the framework approximates the physical image formation process. As a result, we provide guidelines for optimization experiments involving modification of the model parameters, and show how these apply to a first experiment on high dynamic range imaging.