Multispectral imaging has been a valuable technique for discovering hidden texts in manuscripts, learning the provenance of antique books, and generally studying cultural heritage objects. Standard software used in displaying and analyzing such multispectral images are often complex and requires installation and maintenance of custom packages and libraries. We present an easy-to-use web-based multispectral imaging visualization tool that enables simultaneous interaction with the information captured in different spectral bands.

The automatic analysis of fine art paintings presents a number of novel technical challenges to artificial intelligence, computer vision, machine learning, and knowledge representation quite distinct from those arising in the analysis of traditional photographs. The most important difference is that many realist paintings depict stories or episodes in order to convey a lesson, moral, or meaning. One early step in automatic interpretation and extraction of meaning in artworks is the identifications of figures (“actors”). In Christian art, specifically, one must identify the actors in order to identify the Biblical episode or story depicted, an important step in “understanding” the artwork. We designed an auto-matic system based on deep convolutional neural net-works and simple knowledge database to identify saints throughout six centuries of Christian art based in large part upon saints’ symbols or attributes. Our work rep-resents initial steps in the broad task of automatic se- mantic interpretation of messages and meaning in fine art.

We apply generative adversarial convolutional neural networks to the problem of style transfer to underdrawings and ghost-images in x-rays of fine art paintings with a special focus on enhancing their spatial resolution. We build upon a neural architecture developed for the related problem of synthesizing high-resolution photo-realistic image from semantic label maps. Our neural architecture achieves high resolution through a hierarchy of generators and discriminator sub-networks, working throughout a range of spatial resolutions. This coarse-to-fine generator architecture can increase the effective resolution by a factor of eight in each spatial direction, or an overall increase in number of pixels by a factor of 64. We also show that even just a few examples of human-generated image segmentations can greatly improve—qualitatively and quantitatively—the generated images. We demonstrate our method on works such as Leonardo’s Madonna of the carnation and the underdrawing in his Virgin of the rocks, which pose several special problems in style transfer, including the paucity of representative works from which to learn and transfer style information.

Transfer Learning is an important strategy in Computer Vision to tackle problems in the face of limited training data. However, this strategy still heavily depends on the amount of availabl data, which is a challenge for small heritage institutions. This paper investigates various ways of enrichingsmaller digital heritage collections to boost the performance of deep learningmodels, using the identification of musical instruments as a case study. We apply traditional data augmentation techniques as well as the use of an external, photorealistic collection, distorted by Style Transfer. Style Transfer techniques are capable of artistically stylizing images, reusing the style from any other given image. Hence, collections can be easily augmented with artificially generated images. We introduce the distinction between inner and outer style transfer and show that artificially augmented images in both scenarios consistently improve classification results, on top of traditional data augmentation techniques. However, and counter-intuitively, such artificially generated artistic depictions of works are surprisingly hard to classify. In addition, we discuss an example of negative transfer within the non-photorealistic domain.

We describe the application of convolutional neural network style transfer to the problem of improved visualization of underdrawings and ghost-paintings in fine art oil paintings. Such underdrawings and hidden paintings are typically revealed by x-ray or infrared techniques which yield images that are grayscale, and thus devoid of color and full style information. Past methods for inferring color in underdrawings have been based on physical x-ray uorescence spectral imaging of pigments in ghost-paintings and are thus expensive, time consuming, and require equipment not available in most conservation studios. Our algorithmic methods do not need such expensive physical imaging devices. Our proof-ofconcept system, applied to works by Pablo Picasso and Leonardo, reveal colors and designs that respect the natural segmentation in the ghost-painting. We believe the computed images provide insight into the artist and associated oeuvre not available by other means. Our results strongly suggest that future applications based on larger corpora of paintings for training will display color schemes and designs that even more closely resemble works of the artist. For these reasons refinements to our methods should find wide use in art conservation, connoisseurship, and art analysis.

Tim’s Vermeer is a recent documentary feature film following engineer and self-described non-artist Tim Jenison’ extensive efforts to “paint a Vermeer” by means of a novel optical telescope and mirror-comparator procedure. His efforts were inspired by the controversial claim that some Western painters as early as 1420 secretly built optical devices and traced passages in projected images during the execution of some of their works, thereby achieving a novel and compelling “optical look.” The authors examine the proposed telescope optics in historical perspective, the particular visual evidence adduced in support of the comparator hypothesis, and the difficulty and efficacy of the mirror-comparator procedure as revealed by an independent artist/copyist’s attempts to replicate the procedure. Specifically, the authors find that the luminance gradient along the rear wall in the duplicate painting is far from being rare, difficult, or even “impossible” to achieve as proponents claimed; in fact, such gradients appear in numerous Old Master paintings that show no ancillary evidence of having been executed with optics. There is indeed a slight bowing of a single contour in the Vermeer original, which one would normally expect to be straight; however, the optical explanation for this bowing implies that numerous other lines would be similarly bowed, but in fact all are straight. The proposed method does not explain some ofthe most compelling “optical” evidence in Vermeer’s works suchas the small disk-shaped highlights, which appear like the blur spots that arise in an out-of-focus projected image. Likewise, the comparator-based explanations for the presence of pinprick holes at central vanishing points and the presence of underdrawings and pentimenti in several of Vermeer’s works have more plausible non-optical explanations. Finally, an independent experimentalattempt to replicate the procedure fails overall to provide support for the telescope claim. In light of these considerations and evidence, the authors conclude that it is extremely unlikely that Vermeer used the proposed mirror-comparator procedure.

It is difficult to describe facial skin color through a solidcolor as it varies from region to region. In this article, the authors utilized image analysis to identify the facial color representative region. A total of 1052 female images from Humanae project were selected as a solid color was generated for each image as their representative skin colors by the photographer. Using the open CV-based libraries, such as EOS of Surrey Face Models and DeepFace, 3448 facial landmarks together with gender and race information were detected. For an illustrative and intuitive analysis, they then re-defined 27 visually important sub-regions to cluster the landmarks. The 27 sub-region colors for each image were finally derived and recorded in L^*, a^*, and b^*. By estimating the color difference among representative color and 27 sub-regions, we discovered that sub-regions of below lips (low Labial) and central cheeks (upper Buccal) were the most representative regions across four major ethnicity groups. In future study, the methodology is expected to be applied for more image sources. c 2020 Society for Imaging Science and Technology.