Back to articles
Volume: 33 | Article ID: art00008
Does End-to-End Trained Deep Model Always Perform Better than Non-End-to-End Counterpart?
  DOI :  10.2352/ISSN.2470-1173.2021.10.IPAS-240  Published OnlineJanuary 2021

It has been rigorously demonstrated that an end-to-end (E2E) differentiable formulation of a deep neural network can turn a complex recognition problem into a unified optimization task that can be solved by some gradient descent method. Although E2E network optimization yields a powerful fitting ability, the joint optimization of layers is known to potentially bring situations where layers co-adapt one to the other in a complex way that harms generalization ability. This work aims to numerically evaluate the generalization ability of a particular non-E2E network optimization approach known as FOCA (Feature-extractor Optimization through Classifier Anonymization) that helps to avoid complex co-adaptation, with careful hyperparameter tuning. In this report, we present intriguing empirical results where the non-E2E trained models consistently outperform the corresponding E2E trained models on three image-classification datasets. We further show that E2E network fine-tuning, applied after the feature-extractor optimization by FOCA and the following classifier optimization with the fixed feature extractor, indeed gives no improvement on the test accuracy. The source code is available at

Subject Areas :
Views 22
Downloads 1
 articleview.views 22
 articleview.downloads 1
  Cite this article 

Ikuro Sato, Guoqing Liu, Kohta Ishikawa, Teppei Suzuki, Masayuki Tanaka, "Does End-to-End Trained Deep Model Always Perform Better than Non-End-to-End Counterpart?in Proc. IS&T Int’l. Symp. on Electronic Imaging: Image Processing: Algorithms and Systems XIX,  2021,  pp 240-1 - 240-7,

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2021
Electronic Imaging
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane Springfield, VA 22151 USA