Olfaction is ingrained into the fabric of our lives, altering the very perception of our favorite commodities. It plays a crucial role in the multisensory perception of our surrounding environment. Crossmodal correspondences are the non-arbitrary and consistent associations between stimulus features in different sensory modalities that are shared among most of the population [67]. These associations could be considered as sensory expectations; incongruency between the actual and expected attributes of an experience could lead to a “disconfirmation of expectation” [8], which results in the experience being perceived as less pleasant [12]. These expectations influence our decision process [12], the perceived quality of the experience, and its perceived value [72]. Sensory expectations may help induce bias (i.e., providing a red glass of white wine can bias the judgment of expert wine tasters [49], and they have been shown to bias the appearance of physical creations [33]). Semantic congruency can also increase speeded olfactory discrimination [15], identification [17], and perceived pleasantness [57, 64]. Olfaction is one of the oldest senses and plays a major role in social behavior, communication, and emotional evaluation; mood and emotional processes share a common neural substrate with the olfactory pathway, namely the limbic system [40]. Therefore, it is likely that olfactory information plays a major role in modulating the quality of our immersive multisensorial experiences.

Crossmodal interactions between smell and both vision and hearing have been well documented, for example, olfaction–audition [5, 10, 63], olfaction–color [25, 34, 41], olfaction–visual motion [36], and olfaction–angularity of shapes [29, 34]. The mechanisms underlying such correspondences are still not fully understood, but three main explanations have emerged: hedonics [10, 11, 29, 46, 71], semantics [14, 34, 46, 67, 69], and natural co-occurrence [35, 67, 69].

The practical implications for crossmodal interactions expand into the marketing of products and interactive and immersive experiences. Designing experiences that conform to the sensory expectations of the user can increase the perceived quality of the experience in products [68] and human–machine interfaces [30, 50]. In marketing and human–computer interaction, the stimulation of multiple senses creates a richer and more immersive experience, provided the sensory cues are consistent with each other [60, 62, 70]. In terms of multisensory perception, the most reliable sense dominates usually vision [66]. For example, changing the color of a drink influences our perception of the product, shaping the aroma, taste, or flavor [65]. With the prevalence of crossmodal correspondences and olfaction in the entertainment, analytic, and marketing domains, it is vital to uncover the crossmodal interactions underlying common aromatic compounds, which will eventually lead to better designs, more engaging experiences, and an enhanced perceived quality of experience (see [7] for an introduction to quality of experience). Lesur et al. [42] showed that congruent visual–olfactory stimulation using virtual reality could enhance illusory embodiment (bodily self-identification). Design spaces for conveying olfactory information have been proposed [58] and evaluated [4]. Maggioni et al. [45] identified four key features for an olfactory design space: (i) chemical, (ii) emotional, (iii) spatial, and (iv) temporal. We also believe that sensory expectations are an important building block for designing more refined olfactory based experiences, most useful when perceived pleasantness or conveying information is an important factor. For instance, it has been shown that augmenting a human–computer interface with crossmodal correspondences can enhance user performance [28].

The term quality of experience has been defined in [7] as “the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state.” In terms of olfaction-enhanced multimedia, there is an increasing interest in modeling the users’ quality of experience (i.e., [53, 55]), where the users’ sensory expectation is a crucial factor. It has been shown that congruency with the sensory expectations of an observer can increase the perceived pleasantness (e.g., [57, 64]).

The current study was designed to characterize crossmodal interactions between olfactory stimuli and other sensory modalities and explore these associations’ origin. The olfactory stimuli used in our experiments were aromas commonly used in perfumes, olfaction-enhanced multimedia (e.g., [6, 22, 54]), and were selected due to an overlap with prior literature (e.g., [14, 29, 32, 34, 47, 64]). Based on previous research, we hypothesized that crossmodal correspondences would be obtained between olfactory and visual stimuli (shapes, colors) and between odors and haptic/auditory stimuli. To explore these crossmodal associations’ origin, we hypothesized that explicit knowledge of the odor’s identity might modulate these associations (e.g., are odor–color associations contingent on the object’s known color?). Second, we tested the hypothesis that hedonic values might mediate crossmodal associations (e.g., are crossmodal interactions linked by eliciting a pleasant or unpleasant emotional reaction, such as happiness or sadness?). We use the term “hedonics” to refer to our emotional and musical dimensions. We use the term “semantics” to refer to past experiences (i.e., learned associations explained by explicit knowledge of an odor’s identity).

The present study aims to (1) explore the crossmodal associations between olfactory stimuli and the angularity of shapes, smoothness of texture, perceived pleasantness, pitch, colors, musical genres, and emotional dimensions, and (2) to characterize the origin of these associations. We hypothesized the following: First, consistent crossmodal correspondences would be obtained, including a novel modality (the smoothness of texture). Second, the knowledge of odors identity (semantics) would affect the associations. Third, hedonics would play a notable and dominating role in characterizing olfactory crossmodal correspondences.

Our results further the knowledge of multisensory interactions by demonstrating how olfaction interacts with the crossmodal perception of the angularity of shapes, smoothness of texture, perceived pleasantness, pitch, musical genres, and emotions. Our results also expand upon the knowledge of how crossmodal correspondences are mediated.

Our first hypothesis was that associations exist for common aromatic compounds, and we find evidence for such associations that are consistent with prior findings—olfactory associations between the angularity of shapes [29, 34], music [10, 43], colors [14, 32, 34], pitch [5, 10], and emotions [43, 73]. Our work supports the findings of [29, 34], where lemon is perceived to be more angular. Our results also support the findings of [43], where lemon/coffee are associated with jazz, although a larger overlap between lemon/coffee and soul is observed in our findings. Further consistency was found with [43], where both orange and lemon are associated to being pleasant odors. We also found similar color profiles compared with [14, 32, 34] between lavender, lemon, peppermint, and caramel. A novel sensory modality was explored, the smoothness of texture. The hypothesis that crossmodal associations between odors and the smoothness of texture was supported with the odor black pepper being significantly associated to being rough and caramel being associated with smooth.

Our second hypothesis was that knowledge of an odor’s identity would affect the reported associations; this hypothesis was supported in part. The knowledge of the odor’s identity did not affect our ratings for the angularity of shapes, smoothness of texture, perceived pleasantness, or pitch; it did, however, effect the color, emotional and musical dimensions. These findings corroborate previous findings [13, 19] where knowledge of the odor’s identity influenced the hedonic dimensions. We expected more semantic involvement on the perceived pleasantness of the odors [46] than what was observed; although this inconsistency could be attributed to a different approach in the statistical analysis, as we split our results into two data sets (correct versus incorrect), whereas [46] had an additional hedonic dimension (familiarity) which was used as a degree for identification.

Our third hypothesis was that hedonic values might mediate these associations (e.g., [10, 16, 29, 67]). Our PCA results suggest that there was more hedonic involvement than semantic on the “Angularity,” “Smoothness,” “Pleasantness,” and “Pitch” components. The PCA also suggested that our “Classification Rate” (knowledge of the odor’s identity) mainly influences our hedonic dimensions. These findings suggest that there is more hedonic involvement than semantics in explaining the origin of these associations and therefore supporting our third hypothesis. These results corroborate the findings by Zarzo [77], who reported that the observer’s emotional response was the most dominant underlying dimension.

The results from this study could be used as a psychophysical framework to aid in the design or development of interactive experiences involving olfaction, such as product design. It has been shown that changing the packaging of products can change the consumers’ perception of the item within [68]. A study by Piqueras-Fiszman & Spence [59] has shown that people have strong crossmodal associations with products, and incongruency with these sensory expectations “annoys” the consumer [68]. Albeit, semantic congruency can increase its perceived pleasantness [57, 61, 64]. It is well known that adding additional sensory dimensions to virtual and augmented reality applications can increase the sense of presence (see [20] for a review). However, the possibility of exploring the crossmodal interactions uncovered in this work using virtual reality remains unexplored. For example, virtually changing the color, shape, and even texture of an object to determine the impact these associations have on the perceived quality of experience, perceived pleasantness, and their effects on immersion and presence. With the recent traction of quality of experience in both olfaction-enhanced multimedia and product design, there is a need to examine the user’s perception of the multisensory components [51]. These products/applications’ success depends on the impact it has on human observers [31, 51]. Presenting olfactory information in a meaningful way can enhance reality, clarity, and enjoyment [21]. Recent work in olfaction-enhanced multimedia stems away from solely the technical challenges with more focus on enhancing the perceived quality of experience. This includes, but is not limited to, synchronization [1, 24, 52], scent type [54], and impact of information recall [2, 23]. When designing applications or packaging of a product, it is important to conform to the user’s sensory expectations to enhance the perceived quality of experience. Little work in this area considers how different sensory modalities affect each other and the impact this has on immersive and interactive experiences. Metatla et al. [47] investigated the effects of scented three-dimensional (3D) printed shapes (“bouba” and “kiki”) on children; their results did not yield any significant tendencies of associating odors to 3D shapes. Furthermore, they did find significant associations between their shapes and odors (lemon and vanilla) and an emotional dimension (arousal). Mesfin et al. [48] investigated the use of crossmodally congruent olfactory stimuli in multisensory multimedia. Their results revealed that crossmodally matched media enhances the quality of experience compared to a video only condition. Koizumi et al. [39] explored the impact audio has on augmenting food textures by crossmodally changing the auditory properties thereby, changing people’s perception of the food they are experiencing. Building on the small body of work in this area, we show an aggregate of olfactory crossmodal correspondences to aid in the design process helping to provide an enriched, consistent, and more complete set of sensory cues to eventually increase the perceived pleasantness and quality of experience in both product design and human–computer interaction.

Future work could include investigating the extent the underlying chemical properties play in explaining crossmodal correspondences (i.e., [44]). As the associations are shared across observers, further work is needed to determine these associations’ stability over time [26, 29]. Additional sensory dimensions could be added; gustatory could be explored to expand upon our framework. These interactions could be used to determine how crossmodal correspondences affect the perceived quality and pleasantness of 3D printed food. Due to the nature of olfaction, it may be the case that taste played a minor role in the associations reported in this article. For example, Ngo et al. [56] reported the bitter tastes are associated with angular shapes, whereas sweet tastes are associated with a more rounded shape. Hanson-Vaux et al. [29] reported a strong association between the angularity of shapes and the perceived sourness/bitterness of odors. It may be the case that, at least some of the associations reported in this article could be modeled without the need for psychological tests. This could be accomplished by aligning the observer’s perception to the underlying chemical features of the odor’s using electronic nose technology (i.e., [27, 74, 76]). E-noses typically consist of a fine-tuned array of gas sensors with an accompanying pattern recognition system [74]. Moreover, it has already been shown that the underlying chemical features of odors can be mapped to the perceived pleasantness [27, 76]. Therefore, it is highly likely that some aspects of crossmodal correspondences could also be predicted (i.e., between the chemical features of odors and the angularity of shapes).

In summary, our results demonstrate that there is a variety of crossmodal correspondences underlying common aromatic compounds, the odors caramel and coffee are significantly associated with being rounded, while peppermint and lemon are significantly angular. In our novel modality (smoothness of haptic texture), the odor of black pepper is significantly associated with being rough, while the caramel is perceived to be smooth. The odors lemon and orange are perceived to be pleasant, with black pepper being unpleasant. Coffee and caramel are perceived to be lower in pitch, with peppermint being associated with being higher in pitch. The participants could also consistently associate with specific musical genres, emotional values, and colors to odors. Knowledge of an odor’s identity modulates the reported associations of the color, emotional and musical dimensions, but not the angularity of shapes, smoothness of texture, perceived pleasantness, or pitch. Finally, the role of hedonics (emotional and musical) in explaining olfactory crossmodal correspondences is greater than semantics (knowledge of the odor’s identity).