Back to articles
Volume: 30 | Article ID: art00013
A Visual Technique to Analyze Flow of Information in a Machine Learning System
  DOI :  10.2352/ISSN.2470-1173.2018.01.VDA-380  Published OnlineJanuary 2018

Machine learning (ML) algorithms and machine learning based software systems implicitly or explicitly involve complex flow of information between various entities such as training data, feature space, validation set and results. Understanding the statistical distribution of such information and how they flow from one entity to another influence the operation and correctness of such systems, especially in large-scale applications that perform classification or prediction in real time. In this paper, we propose a visual approach to understand and analyze flow of information during model training and serving phases. We build the visualizations using a technique called Sankey Diagram - conventionally used to understand data flow among sets - to address various use cases of in a machine learning system. We demonstrate how the proposed technique, tweaked and twisted to suit a classification problem, can play a critical role in better understanding of the training data, the features, and the classifier performance. We also discuss how this technique enables diagnostic analysis of model predictions and comparative analysis of predictions from multiple classifiers. The proposed concept is illustrated with the example of categorization of millions of products in the e-commerce domain - a multi-class hierarchical classification problem.

Subject Areas :
Views 15
Downloads 1
 articleview.views 15
 articleview.downloads 1
  Cite this article 

Abon Chaudhuri, "A Visual Technique to Analyze Flow of Information in a Machine Learning Systemin Proc. IS&T Int’l. Symp. on Electronic Imaging: Visualization and Data Analysis,  2018,  pp 380-1 - 380-8,

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2018
Electronic Imaging
Society for Imaging Science and Technology