Back to articles
Article
Volume: 35 | Article ID: AVM-125
Image
MTF as a performance indicator for AI algorithms?
  DOI :  10.2352/EI.2023.35.16.AVM-125  Published OnlineJanuary 2023
Abstract
Abstract

The modulation-transfer function (MTF) is a fundamental optical metric to measure the optical quality of an imaging system. In the automotive industry it is used to qualify camera systems for ADAS/AD. Each modern ADAS/AD system includes evaluation algorithms for environment perception and decision making that are based on AI/ML methods and neural networks. The performance of these AI algorithms is measured by established metrics like Average Precision (AP) or precision-recall-curves. In this article we research the robustness of the link between the optical quality metric and the AI performance metric. A series of numerical experiments were performed with object detection and instance segmentation algorithms (cars, pedestrians) evaluated on image databases with varying optical quality. We demonstrate with these that for strong optical aberrations a distinct performance loss is apparent, but that for subtle optical quality differences – as might arise during production tolerances – this link does not exhibit a satisfactory correlation. This calls into question how reliable the current industry practice is where a produced camera is tested end-of-line (EOL) with the MTF, and fixed MTF thresholds are used to qualify the performance of the camera-under-test.

Subject Areas :
Views 200
Downloads 89
 articleview.views 200
 articleview.downloads 89
  Cite this article 

Patrick Müller, Alexander Braun, "MTF as a performance indicator for AI algorithms?in Electronic Imaging,  2023,  pp 125-1 - 125-7,  https://doi.org/10.2352/EI.2023.35.16.AVM-125

 Copy citation
  Copyright statement 
Copyright © 2023, Society for Imaging Science and Technology 2023
ei
Electronic Imaging
2470-1173
2470-1173
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA