Back to articles
Articles
Volume: 33 | Article ID: art00005
Image
Transfer Learning with Style Transfer between the Photorealistic and Artistic Domain
  DOI :  10.2352/ISSN.2470-1173.2021.14.CVAA-041  Published OnlineJanuary 2021
Abstract

Transfer Learning is an important strategy in Computer Vision to tackle problems in the face of limited training data. However, this strategy still heavily depends on the amount of availabl data, which is a challenge for small heritage institutions. This paper investigates various ways of enrichingsmaller digital heritage collections to boost the performance of deep learningmodels, using the identification of musical instruments as a case study. We apply traditional data augmentation techniques as well as the use of an external, photorealistic collection, distorted by Style Transfer. Style Transfer techniques are capable of artistically stylizing images, reusing the style from any other given image. Hence, collections can be easily augmented with artificially generated images. We introduce the distinction between inner and outer style transfer and show that artificially augmented images in both scenarios consistently improve classification results, on top of traditional data augmentation techniques. However, and counter-intuitively, such artificially generated artistic depictions of works are surprisingly hard to classify. In addition, we discuss an example of negative transfer within the non-photorealistic domain.

Subject Areas :
Views 26
Downloads 6
 articleview.views 26
 articleview.downloads 6
  Cite this article 

Nikolay Banar, Matthia Sabatelli, Pierre Geurts, Walter Daelemans, Mike Kestemont, "Transfer Learning with Style Transfer between the Photorealistic and Artistic Domainin Proc. IS&T Int’l. Symp. on Electronic Imaging: Computer Vision and Image Analysis of Art,  2021,  pp 41-1 - 41-9,  https://doi.org/10.2352/ISSN.2470-1173.2021.14.CVAA-041

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2021
72010604
Electronic Imaging
2470-1173
Society for Imaging Science and Technology
IS&T 7003 Kilworth Lane, Springfield, VA 22151 USA