Experimental and clinical usefulness of crossmodal paradigms in psychiatry: an illustration from emotional processing in alcohol-dependence

Multisensory processing of emotional cues predicts intrusive memories after virtual reality trauma

Research has shown that high trait anxiety can alter multisensory processing of threat cues (by amplifying integration of angry faces and voices); however, it remains unknown whether differences in multisensory processing play a role in the psychological response to trauma. This study examined the relationship between multisensory emotion processing and intrusive memories over seven days following exposure to an analogue trauma in a sample of 55 healthy young adults. We used an adapted version of the trauma film paradigm, where scenes showing a car accident trauma were presented using virtual reality, rather than a conventional 2D film. Multisensory processing was assessed prior to the trauma simulation using a forced choice emotion recognition paradigm with happy, sad and angry voice-only, face-only, audiovisual congruent (face and voice expressed matching emotions) and audiovisual incongruent expressions (face and voice expressed different emotions). We found that increased accuracy in recognising anger (but not happiness and sadness) in the audiovisual condition relative to the voice- and face-only conditions was associated with more intrusions following VR trauma. Despite previous results linking trait anxiety and intrusion development, no significant influence of trait anxiety on intrusion frequency was observed. Enhanced integration of threat-related information (i.e. angry faces and voices) could lead to overly threatening appraisals of stressful life events and result in greater intrusion development after trauma.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-023-00784-1.

Do Social Cognition Deficits Recover with Abstinence in Alcohol-Dependent Patients?

Background

Despite growing evidence of the presence and clinical relevance of deficits in social cognition in individuals with alcohol use disorder (AUD), less is known about the potential of “natural” recovery with abstinence in this neurocognitive domain. This study investigated the abstinence‐based recovery of neurocognitive social abilities in alcohol‐dependent patients (ADP) using a prospective longitudinal design with follow‐up assessment under controlled conditions of abstinence during alcohol dependence inpatient treatment.

Methods

Seventy‐seven participants (42 ADP and 35 healthy controls [HC]) performed social cognition testing, including facial emotion recognition, perspective taking, and affective responsiveness twice (baseline/T1 and follow‐up/T2) during comparable follow‐up periods. Assessment of social cognition in abstinent ADP was conducted at the beginning (T1; within the first 2 weeks) and at the end (T2; within the last 2 weeks) of long‐term (2 months) abstinence‐oriented alcohol dependence inpatient treatment. Only patients abstinent for >14 days (last heavy drinking day >21 days) at baseline (T1) and who remained abstinent at follow‐up (T2) were included.

Results

ADP, who on average were nearly 2 months abstinent at T1, showed poorer social cognition in all 3 areas (emotion recognition, perspective taking, and affective responsiveness) than HC. There was no difference between groups on the change in performance over time, and group differences (ADP vs. HC) remained significant at T2, indicating persistent social cognition deficits in ADP following controlled abstinence during inpatient treatment.

Conclusions

Our findings indicate no natural recovery of social cognition impairments in ADP during an intermediate to long‐term period of abstinence (2+ months), the usual active treatment phase. Research aimed at developing interventions that focus on the improvement of social cognition deficits (e.g., social cognition training) and determining whether they benefit short‐ and long‐term clinical outcomes in AUD seems warranted.

Multiple sclerosis, emotion perception and social functioning

People with multiple sclerosis (MS) can experience problems in interpreting others' emotions from faces or voices. However, to date little is known about whether difficulties in emotion perception in MS are related to broader aspects of social functioning. Also, there are few studies reporting the effect of MS on more ecologically valid assessments of emotion perception using multimodal videos. The current study looks at (1) the effect of MS on perceiving emotions from faces, voices and multimodal videos; (2) the possible role of slowed processing and executive dysfunction in emotion perception problems in MS and (3) the relationship between emotion perception and broader social functioning in MS. 53 people with MS and 31 healthy controls completed tasks of emotion perception and cognition, and assessed their levels of social support and social participation. Participants with MS performed worse than demographically matched controls on all measures of emotion perception. Emotion perception performance was related to cognitive measures in those with MS. Also, significant associations were found between emotion perception difficulties in MS and poorer social function. In particular, people with MS who had poorer emotion perception also reported lower levels of social support from their friends, and regression analysis showed that this prediction was maintained even when disease severity and cognitive function were taken into account. These results show that problems with emotion perception in MS extend to more realistic tasks and may predict key aspects of social functioning.

Unpleasant Food Odors Modulate the Processing of Facial Expressions: An Event-Related Potential Study

In real-life situations, emotional information is often expressed through multiple sensory channels, with cross-talk between channels. Previous research has established that odor environments regulate the recognition of facial expressions. Therefore, this study combined event-related potentials (ERPs) with a facial emotion recognition task to investigate the effect of food odor context on the recognition of facial expressions and its time course. Fifty-four participants were asked to identify happy, fearful, and neutral faces in an odor context (pleasant, unpleasant or neutral). Electroencephalography (EEG) was performed to extract event-related potentials (ERPs). Behaviorally, unpleasant food odors triggered faster recognition of facial expressions, especially fearful ones. ERP results found that in the early stage, unpleasant food odors within 80–110 ms evoked a larger P100 amplitude than pleasant food odors and no odors, which showed that the unpleasant odor environment promoted the rapid processing of facial expressions. Next, the interaction between odor environment and facial expressions occurred during the middle stage, and the fearful expression evoked a smaller VPP (vertex positive potential) amplitude than the happy and neutral expressions when exposed to the unpleasant food odor environment. This result indicates that unpleasant odor environment consumed fewer cognitive resources when judging fearful expression, showing the promoting effect of mood coherence effect. These findings provided evidence for how people chose odor environments to facilitate the recognition of facial expressions, and highlighted the advantages of unpleasant food odors in communicating emotional information across the olfactory and visual pathways.

Selective visual and crossmodal impairment in the discrimination of anger and fear expressions in severe alcohol use disorder

BACKGROUND

Severe alcohol use disorder (SAUD) is associated with impaired discrimination of emotional expressions. This deficit appears increased in crossmodal settings, when simultaneous inputs from different sensory modalities are presented. However, so far, studies exploring emotional crossmodal processing in SAUD relied on static faces and unmatched face/voice pairs, thus offering limited ecological validity. Our aim was therefore to assess emotional processing using a validated and ecological paradigm relying on dynamic audio-visual stimuli, manipulating the amount of emotional information available.

METHOD

Thirty individuals with SAUD and 30 matched healthy controls performed an emotional discrimination task requiring to identify five emotions (anger, disgust, fear, happiness, sadness) expressed as visual, auditory, or auditory-visual segments of varying length. Sensitivity indices (d') were computed to get an unbiased measure of emotional discrimination and entered in a Generalized Linear Mixed Model. Incorrect emotional attributions were also scrutinized through confusion matrices.

RESULTS

Discrimination levels varied across sensory modalities and emotions, and increased with stimuli duration. Crucially, performances also improved from unimodal to crossmodal conditions in both groups, but discrimination for anger crossmodal stimuli and fear crossmodal/visual stimuli remained selectively impaired in SAUD. These deficits were not influenced by stimuli duration, suggesting that they were not modulated by the amount of emotional information available. Moreover, they were not associated with systematic emotional error patterns reflecting specific confusions between emotions.

CONCLUSIONS

These results clarify the nature and extent of crossmodal impairments in SAUD and converge with earlier findings to ascribe a specific role for anger and fear in this pathology.

Prehabilitation in Alcohol Dependence as a Treatment Model for Sustainable Outcomes. A Narrative Review of Literature on the Risks Associated With Detoxification, From Animal Models to Human Translational Research

In this review paper, we discuss how the overarching concept of prehabilitation is applicable to alcohol dependence. Central to prehabilitation are the concepts of expected harm, risks, and proactive planning to eliminate the harm or cope with the risks. We review the evidence from animal models, psychological experimental studies, as well as pharmacological studies, on the potential risks and harms associated with medically assisted alcohol detoxification and the current treatment paradigm for alcohol dependence. Animal models provide an approximation mostly of the physical aspect of alcohol withdrawal and detoxification process and make predictions about the development of the phenomena in humans. Despite their limitations, these models provide good evidence that withdrawal from chronic ethanol use induces cognitive impairment, which is worsened by repeated bouts of withdrawal and that these impairments are dependent on the duration of alcohol withdrawal. Initial clinical observations with alcohol-dependent patients confirmed increased incidence of seizures. In recent years, accumulating evidence suggests that patients who have had repeated episodes of withdrawal also show changes in their affect, increased craving, as well as significant deterioration of cognitive abilities, when compared to patients with fewer withdrawals. Alcohol dependence is associated with tolerance and withdrawal, with neuroadaptations in γ-Aminobutyric Acid-A Receptor (GABA-A) and glutamatergic N-methyl-D-aspartate (NMDA) receptors playing key roles. It is suggested that dysregulation of the NMDA receptor system underpins alcohol-related memory impairments. Finally, we discuss the Structured Preparation for Alcohol Detoxification (SPADe) as an example of how prehabilitation has been applied in clinical practice. We discuss the importance of partial control over drinking as an interim step toward abstinence and early introduction of lifestyle changes for both the patient and the immediate environment prior to detoxification and while the patient is still drinking.

Identifying Core Affect in Individuals from fMRI Responses to Dynamic Naturalistic Audiovisual Stimuli

Recent research has demonstrated that affective states elicited by viewing pictures varying in valence and arousal are identifiable from whole brain activation patterns observed with functional magnetic resonance imaging (fMRI). Identification of affective states from more naturalistic stimuli has clinical relevance, but the feasibility of identifying these states on an individual trial basis from fMRI data elicited by dynamic multimodal stimuli is unclear. The goal of this study was to determine whether affective states can be similarly identified when participants view dynamic naturalistic audiovisual stimuli. Eleven participants viewed 5s audiovisual clips in a passive viewing task in the scanner. Valence and arousal for individual trials were identified both within and across participants based on distributed patterns of activity in areas selectively responsive to audiovisual naturalistic stimuli while controlling for lower level features of the stimuli. In addition, the brain regions identified by searchlight analyses to represent valence and arousal were consistent with previously identified regions associated with emotion processing. These findings extend previous results on the distributed representation of affect to multimodal dynamic stimuli.

Olfactory-visual integration facilitates perception of subthreshold negative emotion

A fast growing literature of multisensory emotion integration notwithstanding, the chemical senses, intimately associated with emotion, have been largely overlooked. Moreover, an ecologically highly relevant principle of "inverse effectiveness", rendering maximal integration efficacy with impoverished sensory input, remains to be assessed in emotion integration. Presenting minute, subthreshold negative (vs. neutral) cues in faces and odors, we demonstrated olfactory-visual emotion integration in improved emotion detection (especially among individuals with weaker perception of unimodal negative cues) and response enhancement in the amygdala. Moreover, while perceptual gain for visual negative emotion involved the posterior superior temporal sulcus/pSTS, perceptual gain for olfactory negative emotion engaged both the associative olfactory (orbitofrontal) cortex and amygdala. Dynamic causal modeling (DCM) analysis of fMRI timeseries further revealed connectivity strengthening among these areas during crossmodal emotion integration. That multisensory (but not low-level unisensory) areas exhibited both enhanced response and region-to-region coupling favors a top-down (vs. bottom-up) account for olfactory-visual emotion integration. Current findings thus confirm the involvement of multisensory convergence areas, while highlighting unique characteristics of olfaction-related integration. Furthermore, successful crossmodal binding of subthreshold aversive cues not only supports the principle of "inverse effectiveness" in emotion integration but also accentuates the automatic, unconscious quality of crossmodal emotion synthesis.

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice’s emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.

The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities

Behavior, perception and cognition are strongly shaped by the synthesis of information across the different sensory modalities. Such multisensory integration often results in performance and perceptual benefits that reflect the additional information conferred by having cues from multiple senses providing redundant or complementary information. The spatial and temporal relationships of these cues provide powerful statistical information about how these cues should be integrated or "bound" in order to create a unified perceptual representation. Much recent work has examined the temporal factors that are integral in multisensory processing, with many focused on the construct of the multisensory temporal binding window - the epoch of time within which stimuli from different modalities is likely to be integrated and perceptually bound. Emerging evidence suggests that this temporal window is altered in a series of neurodevelopmental disorders, including autism, dyslexia and schizophrenia. In addition to their role in sensory processing, these deficits in multisensory temporal function may play an important role in the perceptual and cognitive weaknesses that characterize these clinical disorders. Within this context, focus on improving the acuity of multisensory temporal function may have important implications for the amelioration of the "higher-order" deficits that serve as the defining features of these disorders.

The neural bases of crossmodal object recognition in non-human primates and rodents: a review

The ability to integrate information from different sensory modalities to form unique multisensory object representations is a highly adaptive cognitive function. Surprisingly, non-human animal studies of the neural substrates of this form of multisensory integration have been somewhat sparse until very recently, and this may be due in part to a relative paucity of viable testing methods. Here we review the historical development and use of various "crossmodal" cognition tasks for non-human primates and rodents, focusing on tests of "crossmodal object recognition", the ability to recognize an object across sensory modalities. Such procedures have great potential to elucidate the cognitive and neural bases of object representation as it pertains to perception and memory. Indeed, these studies have revealed roles in crossmodal cognition for various brain regions (e.g., prefrontal and temporal cortices) and neurochemical systems (e.g., acetylcholine). A recent increase in behavioral and physiological studies of crossmodal cognition in rodents augurs well for the future of this research area, which should provide essential information about the basic mechanisms of object representation in the brain, in addition to fostering a better understanding of the causes of, and potential treatments for, cognitive deficits in human diseases characterized by atypical multisensory integration.

Emotional facial expression processing in depression: data from behavioral and event-related potential studies

Behavioral literature investigating emotional processes in depressive populations (i.e., unipolar and bipolar depression) states that, compared to healthy controls, depressive subjects exhibit disrupted emotional processing, indexed by lower performance and/or delayed response latencies. The development of brain imaging techniques, such as functional magnetic resonance imaging (fMRI), provided the possibility to visualize the brain regions engaged in emotional processes and how they fail to interact in psychiatric diseases. However, fMRI suffers from poor temporal resolution and cognitive function involves various steps and cognitive stages (serially or in parallel) to give rise to a normal performance. Thus, the origin of a behavioral deficit may result from the alteration of a cognitive stage differently situated along the information-processing stream, outlining the importance of access to this dynamic "temporal" information. In this paper, we will illustrate, through depression, the role that should be attributed to cognitive event-related potentials (ERPs). Indeed, owing to their optimal temporal resolution, ERPs can monitor the neural processes engaged in disrupted cognitive function and provide crucial information for its treatment, training of the impaired cognitive functions and guidelines for clinicians in the choice and monitoring of appropriate medication for the patient.

Subclinical alexithymia modulates early audio-visual perceptive and attentional event-related potentials

INTRODUCTION

Previous studies have highlighted the advantage of using audio-visual oddball tasks (instead of unimodal ones) in order to electrophysiologically index subclinical behavioral differences. Since alexithymia is highly prevalent in the general population, we investigated whether the use of various bimodal tasks could elicit emotional effects in low- vs. high-alexithymic scorers.

METHODS

Fifty students (33 females and 17 males) were split into groups based on low and high scores on the Toronto Alexithymia Scale (TAS-20). During event-related potential (ERP) recordings, they were exposed to three kinds of audio-visual oddball tasks: neutral-AVN-(geometrical forms and bips), animal-AVA-(dog and cock with their respective shouts), or emotional-AVE-(faces and voices) stimuli. In each condition, participants were asked to quickly detect deviant events occurring amongst a train of repeated and frequent matching stimuli (e.g., push a button when a sad face-voice pair appeared amongst a train of neutral face-voice pairs). P100, N100, and P300 components were analyzed: P100 refers to visual perceptive and attentional processing, N100 to auditory ones, and the P300 relates to response-related stages, involving memory processes.

RESULTS

High-alexithymic scorers presented a particular pattern of results when processing the emotional stimulations, reflected in early ERP components by increased P100 and N100 amplitudes in the emotional oddball tasks [P100: F (2, 48) = 20,319, p < 0.001; N100: F (2, 96) = 8,807, p = 0.001] as compared to the animal or neutral ones. Indeed, regarding the P100, subjects exhibited a higher amplitude in the AVE condition (8.717 μV), which was significantly different from that observed during the AVN condition (4.382 μV, p < 0.001). For the N100, the highest amplitude was found in the AVE condition (-4.035 μV) and the lowest was observed in the AVN condition (-2.687 μV, p = 0.003). However, no effect was found on the later P300 component.

CONCLUSIONS

Our findings suggest that high-alexithymic scorers require heightened early attentional resources in comparison to low scorers, particularly when confronted with emotional bimodal stimuli.