
Optical system development requires software tools to design lenses, mechanical components, sensors, and image signal processing (ISP) pipelines. Historically, these tools are operated independently and do not provide insight into complete system performance. As a result, development teams often incur time and cost inefficiencies by designing, building, and testing hardware prototypes that either fail to meet requirements or significantly exceed them. Optical systems are therefore frequently over-designed in one or more areas—such as lens tolerances, sensor bit depth, or ISP complexity—to mitigate risk. End-to-end simulation offers a path to eliminate these inefficiencies and accelerate time-to-market. In this work, we simulate a complete imaging system and demonstrate a method for identifying a minimally viable solution that meets the performance requirements of an object detection application. Using the imaging system simulator ImSym, we model the full imaging chain, including lens behavior, detector characteristics and noise, ISP routines, and straylight effects. These elements are combined to generate simulated images that enable validation of system performance prior to hardware fabrication.

Simulation is an established tool to develop and validate camera systems. The goal of autonomous driving is pushing simulation into a more important and fundamental role for safety, validation and coverage of billions of miles. Realistic camera models are moving more and more into focus, as simulations need to be more then photo-realistic, they need to be physical-realistic, representing the actual camera system onboard the self-driving vehicle in all relevant physical aspects – and this is not only true for cameras, but also for radar and lidar. But when the camera simulations are becoming more and more realistic, how is this realism tested? Actual, physical camera samples are tested in laboratories following norms like ISO12233, EMVA1288 or the developing P2020, with test charts like dead leaves, slanted edge or OECF-charts. In this article we propose to validate the realism of camera simulations by simulating the physical test bench setup, and then comparing the synthetical simulation result with physical results from the real-world test bench using the established normative metrics and KPIs. While this procedure is used sporadically in industrial settings we are not aware of a rigorous presentation of these ideas in the context of realistic camera models for autonomous driving. After the description of the process we give concrete examples for several different measurement setups using MTF and SFR, and show how these can be used to characterize the quality of different camera models.