Crossmodal interactions during affective picture processing

Neuroplasticity and Crossmodal Connectivity in the Normal, Healthy Brain

Objective

Neuroplasticity enables the brain to establish new crossmodal connections or reorganize old connections which are essential to perceiving a multisensorial world. The intent of this review is to identify and summarize the current developments in neuroplasticity and crossmodal connectivity, and deepen understanding of how crossmodal connectivity develops in the normal, healthy brain, highlighting novel perspectives about the principles that guide this connectivity.

Methods

To the above end, a narrative review is carried out. The data documented in prior relevant studies in neuroscience, psychology and other related fields available in a wide range of prominent electronic databases are critically assessed, synthesized, interpreted with qualitative rather than quantitative elements, and linked together to form new propositions and hypotheses about neuroplasticity and crossmodal connectivity.

Results

Three major themes are identified. First, it appears that neuroplasticity operates by following eight fundamental principles and crossmodal integration operates by following three principles. Second, two different forms of crossmodal connectivity, namely direct crossmodal connectivity and indirect crossmodal connectivity, are suggested to operate in both unisensory and multisensory perception. Third, three principles possibly guide the development of crossmodal connectivity into adulthood. These are labeled as the principle of innate crossmodality, the principle of evolution-driven 'neuromodular' reorganization and the principle of multimodal experience. These principles are combined to develop a three-factor interaction model of crossmodal connectivity.

Conclusions

The hypothesized principles and the proposed model together advance understanding of neuroplasticity, the nature of crossmodal connectivity, and how such connectivity develops in the normal, healthy brain.

Emotional Picture Perception: Repetition Effects in Free-Viewing and during an Explicit Categorization Task

Several studies have found that, despite a decrease in the overall amplitude of the late positive potential (LPP) with repeated presentation of the same picture, emotional stimuli continue to elicit a larger LPP than neutral ones. These findings seem to support the hypothesis that the affective modulation of the LPP reflects a mandatory process and does not rely on stimulus novelty. However, in these studies participants were asked to merely look at the pictures, without carrying out any additional task (free-viewing), making picture emotionality the most salient aspect of the stimulus, despite its repetition. The current study aimed to examine the impact of an explicit categorization task on the emotional processing of repeated pictures. To this purpose, ERPs to novel and repeated pictures were measured during free-viewing as well as during an explicit categorization task, where the emotional content of the pictures was task-irrelevant. The within-subject comparison between the free-viewing and task context revealed that the overall LPP habituated more rapidly in the free-viewing condition, but, more importantly, the LPP affective modulation was unaffected by task requirements during both novel and repeated presentations. These results suggest that the affective modulation of the LPP reflects an automatic engagement of cortico-limbic motivational systems, which continues to take place regardless of stimulus novelty and task context.

Evaluative ratings and attention across the life span: emotional arousal and gender

This study was designed to investigate the evolution of emotional processing over the whole adult life span as a function of stimulus arousal and participants' gender. To this end, self-reported affective evaluation and attentional capture prompted by pleasant and unpleasant pictures varying in arousal were measured in a large sample of participants (n = 211) balanced by gender and equally spread across seven decades from 20 to 90 years. Results showed age differences only for affective evaluation of pleasant stimuli, with opposite patterns depending on stimulus arousal. As age increased, low-arousing pleasant cues (e.g. images of babies) were experienced as more pleasant and arousing by both males and females, whereas high-arousing stimuli (e.g. erotic images) were experienced as less pleasant only by females. In contrast, emotional pictures (both pleasant and unpleasant) were effective at capturing attention in a similar way across participants, regardless of age and gender. Taken together, these findings suggest that specific emotional cues prompt different subjective responses across different age groups, while basic mechanisms involved in attentional engagement towards both pleasant and unpleasant stimuli are preserved in healthy ageing.

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice’s emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.