Back to articles
Volume: 2 | Article ID: art00011
Spatial recall index for machine learning algorithms
  DOI :  10.2352/issn.2694-118X.2021.LIM-58  Published OnlineSeptember 2021

We present a novel metric Spatial Recall Index to assess the performance of machine-learning (ML) algorithms for automotive applications, focusing on where in the image which performance occurs. Typical metrics like intersection-over-union (IoU), precisionrecallcurves or average precision (AP) quantify the performance over a whole database of images, neglecting spatial performance variations. But as the optics of camera systems are spatially variable over the field of view, the performance of ML-based algorithms is also a function of space, which we show in simulation: A realistic objective lens based on a Cooke-triplet that exhibits typical optical aberrations like astigmatism and chromatic aberration, all variable over field, is modeled. The model is then applied to a subset of the BDD100k dataset with spatially-varying kernels. We then quantify local changes in the performance of the pre-trained Mask R-CNN algorithm. Our examples demonstrate the spatial dependence of the performance of ML-based algorithms from the optical quality over field, highlighting the need to take the spatial dimension into account when training ML-based algorithms, especially when looking forward to autonomous driving applications.

Subject Areas :
Views 160
Downloads 41
 articleview.views 160
 articleview.downloads 41
  Cite this article 

Patrick Müller, Mattis Brummel, Alexander Braun, "Spatial recall index for machine learning algorithmsin Proc. IS&T London Imaging Meeting 2021: Imaging for Deep Learning,  2021,  pp 58 - 62,

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2021
London Imaging Meeting
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA