In recent years, Convolutional Neural Networks (CNNs) have gained huge popularity among computer vision researchers. In this paper, we investigate how features learned by these networks in a supervised manner can be used to define a measure of self-similarity, an image feature that characterizes many images of natural scenes and patterns, and is also associated with images of artworks. Compared to a previously proposed method for measuring self-similarity based on oriented luminance gradients, our approach has two advantages. Firstly, we fully take color into account, an image feature which is crucial for vision. Secondly, by using higher-layer CNN features, we define a measure of selfsimilarity that relies more on image content than on basic local image features, such as luminance gradients.
Art experience is per definition a dynamic way of processing: While perceiving the artistic object, the film, the music play, we undergo complex affective as well as cognitive experiences interactively changing the entire processing. Elaboration, understanding, aesthetic aha-insights etc. change the view on the to-be-processed entity—psychologically interpreted, the entity becomes a part of ourselves. Most methods of measuring art experience are not able to reflect on these dynamics; most of them are just object-based, e.g. correlative approaches of bringing statically assumed object-properties together with simple ratings on these "objects". Here, I will demonstrate the limitations of such approaches, accompanied by the introduction of some simple principles to be followed when art experience is the focus of research. I will then introduce some methods which can assist in unfolding the process character of art experience without interfering too much with the experience as such: For instance, by using posturography, the Continuous Evaluation Procedure (CEP) or automatic facial expression routines. When these techniques are employed with clear rationales in mind, and by deriving concrete hypotheses from a well-grounded theoretical approach, we can come much closer to the rich experience people have when encountering and elaborating art. This will assist us in our human-history-encompassing endeavor of deciphering what and how art is processed and appreciated.
Although the current societal push for Science-Art collaboration is loud and omnipresent, its integration and practice is superficial. Science and Art disciplines offer a wealth of methodologies, processes, and outcomes relevant to understanding the fundamentals of the how and why of our behaviors, but they remain disconnected in part due to an overwhelming lack of understanding that their solid integration offers invaluable insight for major questions within the study of human cognition. In this paper I argue for a shift in perspective for empirical work in human cognition that genuinely combines and transforms elements from Science and Art to create (a) a new, hybrid discipline as well as (b) sets of new data from which to extract meaningful patterns. I specifically focus on applying integrative Science-Art investigation towards such questions as the relationship between music and language, emotion expression, and spontaneous, real-time adaptability in live, artistic contexts. I discuss a novel project within theatre with live musical improvisation to dissect the characteristics of a coherent, dramatic conversation.