Back to articles
Proceedings
Volume: 36 | Article ID: IMAGE-243
Image
Neuro-Symbolic Ensembles for Assistance at the Edge
  DOI :  10.2352/EI.2024.36.8.IMAGE-243  Published OnlineJanuary 2024
Abstract
Abstract

Intelligence assistance applications hold enormous potential to extend the range of tasks people can perform, increase the speed and accuracy of task performance and provide high quality documentation for record keeping. However, the computational complexity of modern perception and reasoning techniques based on massive foundation model networks cannot run on devices at the edge. A remote server can be used to offload computation but latency and security concerns often rule this out. Distillation and quantization can compress networks but we still face the challenge of obtaining sufficient training data for all possible task executions. We propose a hybrid ensemble architecture that combines intelligent switching of special purpose networks and a symbolic reasoner to provide assistance on modest hardware while still allowing robust and sophisticated reasoning. The rich reasoner representations can also be to identify mistakes in complex procedures. Since system inferences are still imperfect, users can be confused about what the system expects and get frustrated. An interface which makes the capabilities and limitations of perception and reasoning transparent to users dramatically improves the usability of the system. Importantly, our interface provides feedback without compromising situational awareness through well designed audio cues and compact icon-based feedback.

Subject Areas :
Views 8
Downloads 2
 articleview.views 8
 articleview.downloads 2
  Cite this article 

Yan-Ming Chiou, Dick Crouch, Bob Price, Jiaying Shen, Michael Youngblood, Charlie Ortiz, "Neuro-Symbolic Ensembles for Assistance at the Edgein Electronic Imaging,  2024,  pp 243-1 - 243-6,  https://doi.org/10.2352/EI.2024.36.8.IMAGE-243

 Copy citation
  Copyright statement 
Copyright This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. 2024
ei
Electronic Imaging
2470-1173
2470-1173
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA