Influence of emotional expression on memory recognition bias: a functional magnetic resonance imaging study

Individual differences in the evaluation of ambiguous visual and auditory threat-related expressions

This study investigated the neural correlates of the judgement of auditory and visual ambiguous threat-related information, and the influence of state anxiety on this process. Healthy subjects were scanned using a fast, high-resolution functional magnetic resonance imaging (fMRI) multiband sequence while they performed a two-alternative forced-choice emotion judgement task on faces and vocal utterances conveying explicit anger or fear, as well as ambiguous ones. Critically, the latter was specific to each subject, obtained through a morphing procedure and selected prior to scanning following a perceptual decision-making task. Behavioural results confirmed a greater task-difficulty for subject-specific ambiguous stimuli and also revealed a judgement bias for visual fear, and, to a lesser extent, for auditory anger. Imaging results showed increased activity in regions of the salience and frontoparietal control networks (FPCNs) and deactivation in areas of the default mode network for ambiguous, relative to explicit, expressions. In contrast, the right amygdala (AMG) responded more strongly to explicit stimuli. Interestingly, its response to the same ambiguous stimulus depended on the subjective judgement of the expression. Finally, we found that behavioural and neural differences between ambiguous and explicit expressions decreased as a function of state anxiety scores. Taken together, our results show that behavioural and brain responses to emotional expressions are determined not only by emotional clarity but also modality and the subjects' subjective perception of the emotion expressed, and that some of these responses are modulated by state anxiety levels.

A matter of time: how does emotion influence temporal aspects of remembering?

Spatiotemporal context is an intrinsic aspect of episodic memory. Although a large literature has demonstrated that emotion enhances episodic memory, less research has considered whether and how emotion affects memory for the timing of an experience, despite theoretical and practical importance. In this review, we bridge three heavily researched cognitive domains - memory, emotion, and time - by discussing findings from a burgeoning literature on their intersection. We identify and review two broad ways in which memory for time has been conceptualised in the emotional memory literature, namely (1) memory for relative aspects of event timing ("when" an event detail occurred), which includes studies of temporal-order and source memory; and (2) memory for the time that elapsed during an event ("how long"), which includes studies of retrospective duration estimation. Emerging trends demonstrate that although temporal-order memory can be impaired or enhanced by emotion depending on study demands, temporal source memory, instead, is usually enhanced. Studies of duration memory show that the remembered duration of negative experiences is dilated, but it is less clear how duration memory is affected for positive events. These findings are considered under the lens of broader emotional memory literature theories, and directions for future research are proposed.

NEVER forget: negative emotional valence enhances recapitulation

A hallmark feature of episodic memory is that of "mental time travel," whereby an individual feels they have returned to a prior moment in time. Cognitive and behavioral neuroscience methods have revealed a neurobiological counterpart: Successful retrieval often is associated with reactivation of a prior brain state. We review the emerging literature on memory reactivation and recapitulation, and we describe evidence for the effects of emotion on these processes. Based on this review, we propose a new model: Negative Emotional Valence Enhances Recapitulation (NEVER). This model diverges from existing models of emotional memory in three key ways. First, it underscores the effects of emotion during retrieval. Second, it stresses the importance of sensory processing to emotional memory. Third, it emphasizes how emotional valence - whether an event is negative or positive - affects the way that information is remembered. The model specifically proposes that, as compared to positive events, negative events both trigger increased encoding of sensory detail and elicit a closer resemblance between the sensory encoding signature and the sensory retrieval signature. The model also proposes that negative valence enhances the reactivation and storage of sensory details over offline periods, leading to a greater divergence between the sensory recapitulation of negative and positive memories over time. Importantly, the model proposes that these valence-based differences occur even when events are equated for arousal, thus rendering an exclusively arousal-based theory of emotional memory insufficient. We conclude by discussing implications of the model and suggesting directions for future research to test the tenets of the model.

An Evaluation of the Specificity of Executive Function Impairment in Developmental Psychopathology

OBJECTIVE

Deficits in executive function (EF) are common in neuropsychiatric disorders, but the specificity of these deficits remains unclear. The aim of the present study was to elucidate the pattern of EF impairment across psychopathologies in children and adolescents. Associations among components of EF with dimensions of psychopathology, including an overall psychopathology factor, were assessed.

METHOD

Participants (8-21 years) were from the Philadelphia Neurodevelopmental Cohort (N = 9,498). Data from a structured clinical screening interview were reduced to 5 dimensional domains using factor analyses: overall psychopathology, anxious-misery, fear, externalizing, and psychosis. EF components of attentional vigilance, response inhibition, conceptual flexibility, and working memory were assessed. Associations of clinical dimensions with general EF ability and with specific EF components were examined.

RESULTS

EF ability showed common and domain-specific associations with clinical symptoms. General EF was directly associated with the general psychopathology, anxious-misery, and psychosis domains but not with the fear or externalizing domains. For the EF subcomponents, differences emerged in the magnitude and direction of the association between components and clinical domains. Poorer EF was typically associated with increased symptoms across clinical domains; however, in some instances, better EF ability was associated with greater symptom burden, particularly in the fear domain.

CONCLUSION

EF has widespread associations with psychopathology in youth. Findings showed some overlap in the type of EF impairment across clinical phenotypes, as indicated by similar patterns of associations between some clinical symptoms and EF. However, findings also showed domain-specific associations with EF that differed across EF components and clinical domains.

Memory for faces and voices varies as a function of sex and expressed emotion

We investigated how memory for faces and voices (presented separately and in combination) varies as a function of sex and emotional expression (anger, disgust, fear, happiness, sadness, and neutral). At encoding, participants judged the expressed emotion of items in forced-choice tasks, followed by incidental Remember/Know recognition tasks. Results from 600 participants showed that accuracy (hits minus false alarms) was consistently higher for neutral compared to emotional items, whereas accuracy for specific emotions varied across the presentation modalities (i.e., faces, voices, and face-voice combinations). For the subjective sense of recollection (“remember” hits), neutral items received the highest hit rates only for faces, whereas for voices and face-voice combinations anger and fear expressions instead received the highest recollection rates. We also observed better accuracy for items by female expressers, and own-sex bias where female participants displayed memory advantage for female faces and face-voice combinations. Results further suggest that own-sex bias can be explained by recollection, rather than familiarity, rates. Overall, results show that memory for faces and voices may be influenced by the expressions that they carry, as well as by the sex of both items and participants. Emotion expressions may also enhance the subjective sense of recollection without enhancing memory accuracy.

The role of attention in emotional memory enhancement in pathological and healthy aging

After short delays between encoding and retrieval, healthy young participants have better memory performance for emotional stimuli than for neutral stimuli. Divided-attention paradigms suggest that this emotional enhancement of memory (EEM) is due to different attention mechanisms involved during encoding: automatic processing for negative stimuli, and controlled processing for positive stimuli. As far as we know, no study on the influence of these factors on EEM in Alzheimer's disease (AD) and mild cognitive impairment (MCI) patients, as compared to healthy young and older controls, has been conducted. Thus, the goal of our study was to ascertain whether the EEM in these populations depends on the attention resources available at encoding. Participants completed two encoding phases: full attention (FA) and divided attention (DA), followed by two retrieval phases (recognition tasks). There was no EEM on the discrimination accuracy, independently of group and encoding condition. Nevertheless, all participants used a more liberal response criterion for the negative and positive stimuli than for neutral ones. In AD patients, larger numbers of false recognitions for negative and positive stimuli than for neutral ones were observed after both encoding conditions. In MCI patients and in healthy older and younger controls this effect was observed only for negative stimuli, and it depended on the encoding condition. Thus, this effect was observed in young controls after both encoding conditions, in older controls after the DA encoding, and in MCI patients after the FA encoding. In conclusion, our results suggest that emotional valence does not always enhance discrimination accuracy. Nevertheless, in certain conditions related to the attention resources available at encoding, emotional valence, especially the negative one, enhances the subjective feeling of familiarity and, consequently, engenders changes in response bias. This effect seems to be sensitive to the age and the pathology of participants.

Dissociable brain correlates for depression, anxiety, dissociation, and somatization in depersonalization-derealization disorder

OBJECTIVE

The cerebral mechanisms of traits associated with depersonalization-derealization disorder (DPRD) remain poorly understood.

METHOD

Happy and sad emotion expressions were presented to DPRD and non-referred control (NC) subjects in an implicit event-related functional magnetic resonance imaging (fMRI) design, and correlated with self report scales reflecting typical co-morbidities of DPRD: depression, dissociation, anxiety, somatization.

RESULTS

Significant differences between the slopes of the two groups were observed for somatization in the right temporal operculum (happy) and ventral striatum, bilaterally (sad). Discriminative regions for symptoms of depression were the right pulvinar (happy) and left amygdala (sad). For dissociation, discriminative regions were the left mesial inferior temporal gyrus (happy) and left supramarginal gyrus (sad). For state anxiety, discriminative regions were the left inferior frontal gyrus (happy) and parahippocampal gyrus (sad). For trait anxiety, discriminative regions were the right caudate head (happy) and left superior temporal gyrus (sad). Discussion The ascertained brain regions are in line with previous findings for the respective traits. The findings suggest separate brain systems for each trait.

CONCLUSION

Our results do not justify any bias for a certain nosological category in DPRD.

You are that smiling guy I met at the party! Socially positive signals foster memory for identities and contexts

The emotional influence of facial expressions on memory is well-known whereas the influence of emotional contextual information on memory for emotional faces is yet to be extensively explored. This study investigated the interplay between facial expression and the emotional surrounding context in affecting both memory for identities (item memory) and memory for associative backgrounds (source memory). At the encoding fearful and happy faces were presented embedded in fear or happy scenes (i.e.: fearful faces in fear-scenes, happy faces in happy-scenes, fearful faces in happy-scenes and happy faces in fear-scenes) and participants were asked to judge the emotional congruency of the face-scene compounds (i.e. fearful faces in fear-scenes and happy faces in happy-scenes were congruent compounds). In the recognition phase, the old faces were intermixed with the new ones: all the faces were presented isolated with a neutral expression. Participants were requested to indicate whether each face had been previously presented (item memory). Then, for each old face the memory for the scene originally compounded with the face was tested by a three alternative forced choice recognition task (source memory). The results evidenced that face identity memory is differently modulated by the valence in congruent face-context compounds with better identity recognition (item memory) for happy faces encoded in happy-scenarios. Moreover, also the memory for the surrounding context (source memory) benefits from the association with a smiling face. Our findings highlight that socially positive signals conveyed by smiling faces may prompt memory for identity and context.

Facial expression at retrieval affects recognition of facial identity

It is well known that memory can be modulated by emotional stimuli at the time of encoding and consolidation. For example, happy faces create better identity recognition than faces with certain other expressions. However, the influence of facial expression at the time of retrieval remains unknown in the literature. To separate the potential influence of expression at retrieval from its effects at earlier stages, we had participants learn neutral faces but manipulated facial expression at the time of memory retrieval in a standard old/new recognition task. The results showed a clear effect of facial expression, where happy test faces were identified more successfully than angry test faces. This effect is unlikely due to greater image similarity between the neural training face and the happy test face, because image analysis showed that the happy test faces are in fact less similar to the neutral training faces relative to the angry test faces. In the second experiment, we investigated whether this emotional effect is affected by the expression at the time of learning. We employed angry or happy faces as learning stimuli, and angry, happy, and neutral faces as test stimuli. The results showed that the emotional effect at retrieval is robust across different encoding conditions with happy or angry expressions. These findings indicate that emotional expressions do not only affect the stages of encoding and consolidation, but also the retrieval process in identity recognition.

Dissociating electrophysiological correlates of subjective, objective, and correct memory in investigating the emotion-induced recognition bias

Performance on tasks requiring discrimination of at least two stimuli can be viewed either from an objective perspective (referring to actual stimulus differences), or from a subjective perspective (corresponding to participant's responses). Using event-related potentials recorded during an old/new recognition memory test involving emotionally laden and neutral words studied either blockwise or randomly intermixed, we show here how the objective perspective (old versus new items) yields late effects of blockwise emotional item presentation at parietal sites that the subjective perspective fails to find, whereas the subjective perspective ("old" versus "new" responses) is more sensitive to early effects of emotion at anterior sites than the objective perspective. Our results demonstrate the potential advantage of dissociating the subjective and the objective perspective onto task performance (in addition to analyzing trials with correct responses), especially for investigations of illusions and information processing biases, in behavioral and cognitive neuroscience studies.

Fear across the senses: brain responses to music, vocalizations and facial expressions

Intrinsic emotional expressions such as those communicated by faces and vocalizations have been shown to engage specific brain regions, such as the amygdala. Although music constitutes another powerful means to express emotions, the neural substrates involved in its processing remain poorly understood. In particular, it is unknown whether brain regions typically associated with processing 'biologically relevant' emotional expressions are also recruited by emotional music. To address this question, we conducted an event-related functional magnetic resonance imaging study in 47 healthy volunteers in which we directly compared responses to basic emotions (fear, sadness and happiness, as well as neutral) expressed through faces, non-linguistic vocalizations and short novel musical excerpts. Our results confirmed the importance of fear in emotional communication, as revealed by significant blood oxygen level-dependent signal increased in a cluster within the posterior amygdala and anterior hippocampus, as well as in the posterior insula across all three domains. Moreover, subject-specific amygdala responses to fearful music and vocalizations were correlated, consistent with the proposal that the brain circuitry involved in the processing of musical emotions might be shared with the one that have evolved for vocalizations. Overall, our results show that processing of fear expressed through music, engages some of the same brain areas known to be crucial for detecting and evaluating threat-related information.

Emotional memory modulation in schizophrenia: an overview

OBJECTIVE

In healthy controls, the emotional charge of stimuli influences how well stimuli are remembered. Although patients with schizophrenia (SCZ) have deficits in memory and in emotional processing, studies on emotional memory modulation (EMM) in SCZ report contradictory results. The aim of this review was to investigate whether methodological differences could explain these contradictory results.

METHOD

We reviewed the literature to investigate whether task differences could explain these differences. Due to the methodological differences, a meta-analysis was not possible.

RESULTS

Fourteen studies were identified that used a total of 22 tasks to study EMM in patients with SCZ. Two-thirds of the tasks showed no differences in EMM between patients with SCZ and healthy controls. Differences in EMM were found more often when long-term compared to short-term memory was measured, when memory instructions were implicit instead of explicit and when stronger emotional stimuli were used. An overall memory deficit or the mode of retrieval was not related to EMM.

CONCLUSION

Deficits in EMM in long-term compared to short-term memory point toward impaired emotional modulation of memory consolidation. Reduced EMM on implicit, but not explicit, tasks suggests a deficit in unconsciously using emotional content to modulate memory.

Striatal intrinsic reinforcement signals during recognition memory: relationship to response bias and dysregulation in schizophrenia

Ventral striatum (VS) is a critical brain region for reinforcement learning and motivation, and VS hypofunction is implicated in psychiatric disorders including schizophrenia. Providing rewards or performance feedback has been shown to activate VS. Intrinsically motivated subjects performing challenging cognitive tasks are likely to engage reinforcement circuitry even in the absence of external feedback or incentives. However, such intrinsic reinforcement responses have received little attention, have not been examined in relation to behavioral performance, and have not been evaluated for impairment in neuropsychiatric disorders such as schizophrenia. Here we used fMRI to examine a challenging “old” vs. “new” visual recognition task in healthy subjects and patients with schizophrenia. Targets were unique fractal stimuli previously presented as salient distractors in a visual oddball task, producing incidental memory encoding. Based on the prediction error theory of reinforcement learning, we hypothesized that correct target recognition would activate VS in controls, and that this activation would be greater in subjects with lower expectation of responding correctly as indexed by a more conservative response bias. We also predicted these effects would be reduced in patients with schizophrenia. Consistent with these predictions, controls activated VS and other reinforcement processing regions during correct recognition, with greater VS activation in those with a more conservative response bias. Patients did not show either effect, with significant group differences suggesting hyporesponsivity in patients to internally generated feedback. These findings highlight the importance of accounting for intrinsic motivation and reward when studying cognitive tasks, and add to growing evidence of reward circuit dysfunction in schizophrenia that may impact cognition and function.

Emotional face processing and flat affect in schizophrenia: functional and structural neural correlates

BACKGROUND

There is a general consensus in the literature that schizophrenia causes difficulties with facial emotion perception and discrimination. Functional brain imaging studies have observed reduced limbic activity during facial emotion perception but few studies have examined the relation to flat affect severity.

METHOD

A total of 26 people with schizophrenia and 26 healthy controls took part in this event-related functional magnetic resonance imaging study. Sad, happy and neutral faces were presented in a pseudo-random order and participants indicated the gender of the face presented. Manual segmentation of the amygdala was performed on a structural T1 image.

RESULTS

Both the schizophrenia group and the healthy control group rated the emotional valence of facial expressions similarly. Both groups exhibited increased brain activity during the perception of emotional faces relative to neutral ones in multiple brain regions, including multiple prefrontal regions bilaterally, the right amygdala, right cingulate cortex and cuneus. Group comparisons, however, revealed increased activity in the healthy group in the anterior cingulate, right parahippocampal gyrus and multiple visual areas. In schizophrenia, the severity of flat affect correlated significantly with neural activity in several brain areas including the amygdala and parahippocampal region bilaterally.

CONCLUSIONS

These results suggest that many of the brain regions involved in emotional face perception, including the amygdala, are equally recruited in both schizophrenia and controls, but flat affect can also moderate activity in some other brain regions, notably in the left amygdala and parahippocampal gyrus bilaterally. There were no significant group differences in the volume of the amygdala.

A tale of two negatives: differential memory modulation by threat-related facial expressions

Facial expressions serve as cues that encourage viewers to learn about their immediate environment. In studies assessing the influence of emotional cues on behavior, fearful and angry faces are often combined into one category, such as "threat-related," because they share similar emotional valence and arousal properties. However, these expressions convey different information to the viewer. Fearful faces indicate the increased probability of a threat, whereas angry expressions embody a certain and direct threat. This conceptualization predicts that a fearful face should facilitate processing of the environment to gather information to disambiguate the threat. Here, we tested whether fearful faces facilitated processing of neutral information presented in close temporal proximity to the faces. In Experiment 1, we demonstrated that, compared with neutral faces, fearful faces enhanced memory for neutral words presented in the experimental context, whereas angry faces did not. In Experiment 2, we directly compared the effects of fearful and angry faces on subsequent memory for emotional faces versus neutral words. We replicated the findings of Experiment 1 and extended them by showing that participants remembered more faces from the angry face condition relative to the fear condition, consistent with the notion that anger differs from fear in that it directs attention toward the angry individual. Because these effects cannot be attributed to differences in arousal or valence processing, we suggest they are best understood in terms of differences in the predictive information conveyed by fearful and angry facial expressions.

Affective cognition and its disruption in mood disorders

In this review, we consider affective cognition, responses to emotional stimuli occurring in the context of cognitive evaluation. In particular, we discuss emotion categorization, biasing of memory and attention, as well as social/moral emotion. We discuss limited neuropsychological evidence suggesting that affective cognition depends critically on the amygdala, ventromedial frontal cortex, and the connections between them. We then consider neuroimaging studies of affective cognition in healthy volunteers, which have led to the development of more sophisticated neural models of these processes. Disturbances of affective cognition are a core and specific feature of mood disorders, and we discuss the evidence supporting this claim, both from behavioral and neuroimaging perspectives. Serotonin is considered to be a key neurotransmitter involved in depression, and there is a considerable body of research exploring whether serotonin may mediate disturbances of affective cognition. The final section presents an overview of this literature and considers implications for understanding the pathophysiology of mood disorder as well as developing and evaluating new treatment strategies.

Influence of emotional expression on memory recognition bias in schizophrenia as revealed by fMRI

We recently showed that, in healthy individuals, emotional expression influences memory for faces both in terms of accuracy and, critically, in memory response bias (tendency to classify stimuli as previously seen or not, regardless of whether this was the case). Although schizophrenia has been shown to be associated with deficit in episodic memory and emotional processing, the relation between these processes in this population remains unclear. Here, we used our previously validated paradigm to directly investigate the modulation of emotion on memory recognition. Twenty patients with schizophrenia and matched healthy controls completed functional magnetic resonance imaging (fMRI) study of recognition memory of happy, sad, and neutral faces. Brain activity associated with the response bias was obtained by correlating this measure with the contrast subjective old (ie, hits and false alarms) minus subjective new (misses and correct rejections) for sad and happy expressions. Although patients exhibited an overall lower memory performance than controls, they showed the same effects of emotion on memory, both in terms of accuracy and bias. For sad faces, the similar behavioral pattern between groups was mirrored by a largely overlapping neural network, mostly involved in familiarity-based judgments (eg, parahippocampal gyrus). In contrast, controls activated a much larger set of regions for happy faces, including areas thought to underlie recollection-based memory retrieval (eg, superior frontal gyrus and hippocampus) and in novelty detection (eg, amygdala). This study demonstrates that, despite an overall lower memory accuracy, emotional memory is intact in schizophrenia, although emotion-specific differences in brain activation exist, possibly reflecting different strategies.

Own-sex effects in emotional memory for faces

The amygdala is known to be critical for the enhancement of memory for emotional, especially negative, material. Importantly, some researchers have suggested a sex-specific hemispheric lateralization in this process. In the case of facial expressions, another important factor that could influence memory success is the sex of the face, which could interact with the emotion depicted as well as with the sex of the perceiver. Whether this is the case remains unknown, as all previous studies of sex difference in emotional memory have employed affective pictures. Here we directly explored this question using functional magnetic resonance imaging in a subsequent memory paradigm for facial expressions (fearful, happy and neutral). Consistent with our hypothesis, we found that the hemispheric laterality of the amygdala involvement in successful memory for emotional material was influenced not only by the sex of the subjects, as previously proposed, but also by the sex of the faces being remembered. Namely, the left amygdala was more active for successfully remembered female fearful faces in women, whereas in men the right amygdala was more involved in memory for male fearful faces. These results confirm the existence of sex differences in amygdala lateralization in emotional memory but also demonstrate a subtle relationship between the observer and the stimulus in this process.