Hemodynamic and electrophysiological relationship involved in human face processing: Evidence from a combined fMRI–ERP study

The effect of feedback timing on category learning and feedback processing in younger and older adults

Introduction

Corrective feedback can be received immediately after an action or with a temporal delay. Neuroimaging studies suggest that immediate and delayed feedback are processed by the striatum and medial temporal lobes (MTL), respectively. Age-related changes in the striatum and MTL may influence the efficiency of feedback-based learning in older adults. The current study leverages event-related potentials (ERPs) to evaluate age-related differences in immediate and delayed feedback processing and consequences for learning. The feedback-related negativity (FRN) captures activity in the frontostriatal circuit while the N170 is hypothesized to reflect MTL activation.

Methods

18 younger (Myears  = 24.4) and 20 older (Myears  = 65.5) adults completed learning tasks with immediate and delayed feedback. For each group, learning outcomes and ERP magnitudes were evaluated across timing conditions.

Results

Younger adults learned better than older adults in the immediate timing condition. This performance difference was associated with a typical FRN signature in younger but not older adults. For older adults, impaired processing of immediate feedback in the striatum may have negatively impacted learning. Conversely, learning was comparable across groups when feedback was delayed. For both groups, delayed feedback was associated with a larger magnitude N170 relative to immediate feedback, suggesting greater MTL activation.

Discussion and conclusion

Delaying feedback may increase MTL involvement and, for older adults, improve category learning. Age-related neural changes may differentially affect MTL- and striatal-dependent learning. Future research can evaluate the locus of age-related learning differences and how feedback can be manipulated to optimize learning across the lifespan.

Implications of depressive mood in OSAHS patients: insights from event-related potential

BACKGROUND

Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a chronic breathing disorder characterized by recurrent upper airway obstruction during sleep. Although previous studies have shown a link between OSAHS and depressive mood, the neurobiological mechanisms underlying mood disorders in OSAHS patients remain poorly understood. This study aims to investigate the emotion processing mechanism in OSAHS patients with depressive mood using event-related potentials (ERPs).

METHODS

Seventy-four OSAHS patients were divided into the depressive mood and non-depressive mood groups according to their Self-rating Depression Scale (SDS) scores. Patients underwent overnight polysomnography and completed various cognitive and emotional questionnaires. The patients were shown facial images displaying positive, neutral, and negative emotions and tasked to identify the emotion category, while their visual evoked potential was simultaneously recorded.

RESULTS

The two groups did not differ significantly in age, BMI, and years of education, but showed significant differences in their slow wave sleep ratio (P = 0.039), ESS (P = 0.006), MMSE (P < 0.001), and MOCA scores (P = 0.043). No significant difference was found in accuracy and response time on emotional face recognition between the two groups. N170 latency in the depressive group was significantly longer than the non-depressive group (P = 0.014 and 0.007) at the bilateral parieto-occipital lobe, while no significant difference in N170 amplitude was found. No significant difference in P300 amplitude or latency between the two groups. Furthermore, N170 amplitude at PO7 was positively correlated with the arousal index and negatively with MOCA scores (both P < 0.01).

CONCLUSION

OSAHS patients with depressive mood exhibit increased N170 latency and impaired facial emotion recognition ability. Special attention towards the depressive mood among OSAHS patients is warranted for its implications for patient care.

Does neural face processing explain effects of an attachment-based intervention on maternal sensitivity? A randomized controlled study including pre- and postintervention measures

BACKGROUND

Although there is a large body of literature highlighting the behavioral effects of parenting interventions, studies on the neurocognitive mechanisms involved in such intervention effects remain scarce.

PURPOSE

The aim of the current study was to test whether changes in neural face processing (as reflected in N170 amplitudes) would act as a mediator in the association between the Video-feedback Intervention to promote Positive Parenting and Sensitive Discipline (VIPP-SD) and maternal sensitivity.

METHODS

A total of 66 mothers of whom a random 33% received the VIPP-SD and the others a "dummy" intervention participated in pre- and postintervention assessments. We recorded mothers' electroencephalographic (EEG) activity in response to photographs of children's neutral, happy, and angry facial expressions. Maternal sensitivity was observed while mothers interacted with their offspring in a semi-structured play situation.

RESULTS

In contrast with our expectations, we did not find evidence for mediation of intervention effects on maternal sensitivity by the N170.

CONCLUSION

We discuss that parenting support programs may yield different effects on neurocognitive processes depending on the population and provide recommendations for future research. Our study underscores the importance of reporting null findings and preregistering studies in the field of neurocognitive research.

Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces

Decoding someone's facial expressions provides insights into his or her emotional experience. Recently, Automatic Facial Coding (AFC) software has been developed to provide measurements of emotional facial expressions. Previous studies provided first evidence for the sensitivity of such systems to detect facial responses in study participants. In the present experiment, we set out to generalise these results to affective responses as they can occur in variable social interactions. Thus, we presented facial expressions (happy, neutral, angry) and instructed participants (N = 64) to either actively mimic, to look at them passively (n = 21), or to inhibit their own facial reaction (n = 22). A video stream for AFC and an electromyogram (EMG) of the zygomaticus and corrugator muscles were registered continuously. In the mimicking condition, both AFC and EMG differentiated well between facial expressions in response to the different emotional pictures. In the passive viewing and in the inhibition condition AFC did not detect changes in facial expressions whereas EMG was still highly sensitive. Although only EMG is sensitive when participants intend to conceal their facial reactions, these data extend previous findings that Automatic Facial Coding is a promising tool for the detection of intense facial reaction.

Attractiveness and neural processing of infant faces: effects of a facial abnormality but not dopamine

Adults' caregiving responses toward infants may have important origins in the perception and processing of infant cues as well as the motivation to attend to these cues. Moreover, some biological processes, such as dopaminergic neurotransmission, may be crucially involved. Although infant stimuli are generally experienced as cute and rewarding, infants with a visible disability may be regarded much less favorably than others, perhaps dependent on differences in perception, motivation, and neural processing. The current study investigated effects of administered dopamine on the perceived attractiveness and neurophysiological indices of attention and processing (i.e., the P1, P2, and N170 components of the event-related potential) of infant faces with and without a cleft lip. No evidence for effects of dopamine was found, but we replicated the finding that the decreased attractiveness of infants with a cleft lip was mediated by decreased configural face processing (smaller N170 amplitudes), but not more general attentional and/or executive processing (P2). The current findings show once again the unfavorable consequences of a cleft lip, but also highlight the importance of combining and relating measures across various levels of analysis and underscore the importance of replication.

Contextual information resolves uncertainty about ambiguous facial emotions: Behavioral and magnetoencephalographic correlates

Environmental conditions bias our perception of other peoples' facial emotions. This becomes quite relevant in potentially threatening situations, when a fellow's facial expression might indicate potential danger. The present study tested the prediction that a threatening environment biases the recognition of facial emotions. To this end, low- and medium-expressive happy and fearful faces (morphed to 10%, 20%, 30%, or 40% emotional) were presented within a context of instructed threat-of-shock or safety. Self-reported data revealed that instructed threat led to a biased recognition of fearful, but not happy facial expressions. Magnetoencephalographic correlates revealed spatio-temporal clusters of neural network activity associated with emotion recognition and contextual threat/safety in early to mid-latency time intervals in the left parietal cortex, bilateral prefrontal cortex, and the left temporal pole regions. Early parietal activity revealed a double dissociation of face-context information as a function of the expressive level of facial emotions: When facial expressions were difficult to recognize (low-expressive), contextual threat enhanced fear processing and contextual safety enhanced processing of subtle happy faces. However, for rather easily recognizable faces (medium-expressive) the left hemisphere (parietal cortex, PFC, and temporal pole) showed enhanced activity to happy faces during contextual threat and fearful faces during safety. Thus, contextual settings reduce the salience threshold and boost early face processing of low-expressive congruent facial emotions, whereas face-context incongruity or mismatch effects drive neural activity of easier recognizable facial emotions. These results elucidate how environmental settings help recognize facial emotions, and the brain mechanisms underlying the recognition of subtle nuances of fear.

The neural sources of N170: Understanding timing of activation in face-selective areas

The N170 ERP component has been widely identified as a face-sensitive neural marker. Despite extensive investigations conducted to examine the neural sources of N170, there are two issues in prior literature: (a) few studies used individualized anatomy as head model for the cortical source analysis of the N170, and (b) the relationship between the N170 and face-selective regions from fMRI studies is unclear. Here, we addressed these questions by presenting pictures of faces and houses to the same group of healthy adults and recording structural MRI, fMRI, and high-density ERPs in separate sessions. Source analysis based on the participant's anatomy showed that the middle and posterior fusiform gyri were the primary neural sources for the face-sensitive aspects of the N170. Source analysis based on regions of interest from the fMRI revealed that the fMRI-defined fusiform face area was the major contributor to the N170. The current study suggests that the fusiform gyrus is a major neural contributor to the N170 ERP component and provides further insights about the spatiotemporal characteristics of face processing.

Neural responses to children's faces: Test-retest reliability of structural and functional MRI

INTRODUCTION

Functional MRI (fMRI) is commonly used to investigate the neural mechanisms underlying psychological processes and behavioral responses. However, to draw well-founded conclusions from fMRI studies, more research on the reliability of fMRI is needed.

METHODS

We invited a sample of 41 female students to participate in two identical fMRI sessions, separated by 5 weeks on average. To investigate the potential effect of left-handedness on the stability of neural activity, we oversampled left-handed participants (N = 20). Inside the scanner, we presented photographs of familiar and unfamiliar children's faces preceded by neutral and threatening primes to the participants. We calculated intraclass correlations (ICCs) to investigate the test-retest reliability of peak activity in areas that showed significant activity during the first session (primary visual cortex, fusiform face area, inferior frontal gyrus, and superior temporal gyrus). In addition, we examined how many trials were needed to reliably measure the effects.

RESULTS

Across all participants, only fusiform face area activity in response to faces showed good test-retest reliability (ICC = 0.71). All other test-retest reliabilities were low (0.01 ≤ ICC ≤ 0.35). Reliabilities varied only slightly with increasing numbers of trials, with no consistent increase in ICCs. Test-retest reliabilities for left-handed participants (0.28 ≤ ICC ≤0.66) were generally somewhat higher than for right-handed participants (-0.13 ≤ ICC ≤0.75), but not statistically significant.

CONCLUSION

Our study shows good test-retest reliability for fusiform facer area activity in response to faces, but low test-retest reliability for other contrasts and areas.

Changes in face-specific neural processing explain reduced cuteness and approachability of infants with cleft lip

The current study investigated whether changes in the neural processing of faces of infants with a facial abnormality - a cleft lip - mediate effects of the cleft lip on judgments of infant cuteness and approachability. Event-related potentials (ERPs) in response to pictures of faces of healthy infants and infants with a cleft lip, and ratings of cuteness and approachability of these infant faces, were obtained from 30 females. Infants with a cleft lip were rated as less attractive (less cute and approachable) than healthy infants, and both the N170 and P2 components of the ERP were of reduced amplitude in response to pictures of infants with a cleft lip. Importantly, decreased configural processing of infant faces with a cleft lip, as evidenced by reduced N170 amplitudes, mediated the reduced attractiveness ratings for infants with a cleft lip compared to healthy infants. Our findings help elucidate the mechanisms behind the less favorable responses to infants with a cleft lip, highlighting the role of face-specific rather than domain-general neural processes.

Decoding material-specific memory reprocessing during sleep in humans

Neuronal learning activity is reactivated during sleep but the dynamics of this reactivation in humans are still poorly understood. Here we use multivariate pattern classification to decode electrical brain activity during sleep and determine what type of images participants had viewed in a preceding learning session. We find significant patterns of learning-related processing during rapid eye movement (REM) and non-REM (NREM) sleep, which are generalizable across subjects. This processing occurs in a cyclic fashion during time windows congruous to critical periods of synaptic plasticity. Its spatial distribution over the scalp and relevant frequencies differ between NREM and REM sleep. Moreover, only the strength of reprocessing in slow-wave sleep influenced later memory performance, speaking for at least two distinct underlying mechanisms between these states. We thus show that memory reprocessing occurs in both NREM and REM sleep in humans and that it pertains to different aspects of the consolidation process.

Facial identity and emotional expression as predictors during economic decisions

Two sources of information most relevant to guide social decision making are the cooperative tendencies associated with different people and their facial emotional displays. This electrophysiological experiment aimed to study how the use of personal identity and emotional expressions as cues impacts different stages of face processing and their potential isolated or interactive processing. Participants played a modified trust game with 8 different alleged partners, and in separate blocks either the identity or the emotions carried information regarding potential trial outcomes (win or loss). Behaviorally, participants were faster to make decisions based on identity compared to emotional expressions. Also, ignored (nonpredictive) emotions interfered with decisions based on identity in trials where these sources of information conflicted. Electrophysiological results showed that expectations based on emotions modulated processing earlier in time than those based on identity. Whereas emotion modulated the central N1 and VPP potentials, identity judgments heightened the amplitude of the N2 and P3b. In addition, the conflict that ignored emotions generated was reflected on the N170 and P3b potentials. Overall, our results indicate that using identity or emotional cues to predict cooperation tendencies recruits dissociable neural circuits from an early point in time, and that both sources of information generate early and late interactive patterns.

The Cortical Development of Specialized Face Processing in Infancy

The aim of this study was to examine specialized face processing in forty-eight 4.5- to 7.5-month-old infants by recording event-related potentials (ERPs) in response to faces and toys, and to determine the cortical sources of these signals using realistic, age-appropriate head models. All ERP components (i.e., N290, P400, Nc) showed greater amplitude during periods of attention than inattention. Amplitude was greater to faces than toys during attention at the N290, and greater to toys at the P400. Cortical source analysis revealed activity in occipital-temporal brain areas as the source of the N290, particularly the middle fusiform gyrus. The Nc and P400 were the result of activation in midline frontal and parietal, anterior temporal, and posterior temporal and occipital brain areas.

Deficient cortical face-sensitive N170 responses and basic visual processing in schizophrenia

BACKGROUND

Face detection, an ability to identify a visual stimulus as a face, is impaired in patients with schizophrenia. It is unclear whether impaired face processing in this psychiatric disorder results from face-specific domains or stems from more basic visual domains. In this study, we examined cortical face-sensitive N170 response in schizophrenia, taking into account deficient basic visual contrast processing.

METHODS

We equalized visual contrast signals among patients (n=20) and controls (n=20) and between face and tree images, based on their individual perceptual capacities (determined using psychophysical methods). We measured N170, a putative temporal marker of face processing, during face detection and tree detection.

RESULTS

In controls, N170 amplitudes were significantly greater for faces than trees across all three visual contrast levels tested (perceptual threshold, two times perceptual threshold and 100%). In patients, however, N170 amplitudes did not differ between faces and trees, indicating diminished face selectivity (indexed by the differential responses to face vs. tree).

CONCLUSION

These results indicate a lack of face-selectivity in temporal responses of brain machinery putatively responsible for face processing in schizophrenia. This neuroimaging finding suggests that face-specific processing is compromised in this psychiatric disorder.

Anorexia nervosa and body dysmorphic disorder are associated with abnormalities in processing visual information

BACKGROUND

Anorexia nervosa (AN) and body dysmorphic disorder (BDD) are characterized by distorted body image and are frequently co-morbid with each other, although their relationship remains little studied. While there is evidence of abnormalities in visual and visuospatial processing in both disorders, no study has directly compared the two. We used two complementary modalities--event-related potentials (ERPs) and functional magnetic resonance imaging (fMRI)--to test for abnormal activity associated with early visual signaling.

METHOD

We acquired fMRI and ERP data in separate sessions from 15 unmedicated individuals in each of three groups (weight-restored AN, BDD, and healthy controls) while they viewed images of faces and houses of different spatial frequencies. We used joint independent component analyses to compare activity in visual systems.

RESULTS

AN and BDD groups demonstrated similar hypoactivity in early secondary visual processing regions and the dorsal visual stream when viewing low spatial frequency faces, linked to the N170 component, as well as in early secondary visual processing regions when viewing low spatial frequency houses, linked to the P100 component. Additionally, the BDD group exhibited hyperactivity in fusiform cortex when viewing high spatial frequency houses, linked to the N170 component. Greater activity in this component was associated with lower attractiveness ratings of faces.

CONCLUSIONS

Results provide preliminary evidence of similar abnormal spatiotemporal activation in AN and BDD for configural/holistic information for appearance- and non-appearance-related stimuli. This suggests a common phenotype of abnormal early visual system functioning, which may contribute to perceptual distortions.

Brain Signals of Face Processing as Revealed by Event-Related Potentials

We analyze the functional significance of different event-related potentials (ERPs) as electrophysiological indices of face perception and face recognition, according to cognitive and neurofunctional models of face processing. Initially, the processing of faces seems to be supported by early extrastriate occipital cortices and revealed by modulations of the occipital P1. This early response is thought to reflect the detection of certain primary structural aspects indicating the presence grosso modo of a face within the visual field. The posterior-temporal N170 is more sensitive to the detection of faces as complex-structured stimuli and, therefore, to the presence of its distinctive organizational characteristics prior to within-category identification. In turn, the relatively late and probably more rostrally generated N250r and N400-like responses might respectively indicate processes of access and retrieval of face-related information, which is stored in long-term memory (LTM). New methods of analysis of electrophysiological and neuroanatomical data, namely, dynamic causal modeling, single-trial and time-frequency analyses, are highly recommended to advance in the knowledge of those brain mechanisms concerning face processing.

How affective information from faces and scenes interacts in the brain

Facial expression perception can be influenced by the natural visual context in which the face is perceived. We performed an fMRI experiment presenting participants with fearful or neutral faces against threatening or neutral background scenes. Triangles and scrambled scenes served as control stimuli. The results showed that the valence of the background influences face selective activity in the right anterior parahippocampal place area (PPA) and subgenual anterior cingulate cortex (sgACC) with higher activation for neutral backgrounds compared to threatening backgrounds (controlled for isolated background effects) and that this effect correlated with trait empathy in the sgACC. In addition, the left fusiform gyrus (FG) responds to the affective congruence between face and background scene. The results show that valence of the background modulates face processing and support the hypothesis that empathic processing in sgACC is inhibited when affective information is present in the background. In addition, the findings reveal a pattern of complex scene perception showing a gradient of functional specialization along the posterior-anterior axis: from sensitivity to the affective content of scenes (extrastriate body area: EBA and posterior PPA), over scene emotion-face emotion interaction (left FG) via category-scene interaction (anterior PPA) to scene-category-personality interaction (sgACC).

The role of the eyes in processing an intact face and its scrambled image: a dense array ERP and low-resolution electromagnetic tomography (sLORETA) study

The aim of the present study was to test whether the eyes of an intact face produced a specific brain response compared to the mouth, nose, or hair and whether their specificity was also maintained in a scrambled face. Fifteen subjects were asked to focus visual attention on global and single elements in intact faces and in their scrambled image. EEG data were recorded from 256-Hydrocel Geodesic Sensor-Net200. Event-related potentials (ERPs) analyses showed a difference between the intact face and the scrambled face from N170 component until 600 ms on the occipito-temporal montage and at 400-600 ms on the frontal montage. Only the eyes showed a difference between conditions (intact/scrambled face) at 500 ms. The most activated source detected by sLORETA was the right middle temporal gyrus (BA21) for both conditions and for all elements. Left BA21 resulted in significantly more activation in response to eyes in the intact face compared to the eyes in the scrambled face at 500 ms. The left BA21 has a central role in high-level visual processing and in understanding others' intentions. These findings suggest a specificity of the eyes and indicate that the eyes play the social and communicative role of comprehending the nonverbal intentions of others only when inserted in an intact face.

Early electrophysiological basis of experience-associated holistic processing of Chinese characters

Recent studies have found holistic processing to be a marker of expertise for perception of words in alphabetic (e.g., English) and non-alphabetic (e.g., Chinese) writing systems, consistent with what has been found for faces and other objects of face-like expertise. It is unknown, however, whether holistic processing of words occurs in an early, perceptual stage as it does for faces. We examined how early holistic processing of Chinese characters emerges by recording the event-related potentials (ERPs) in an adaptation paradigm. Participants judged if the top parts of two sequentially presented characters were the same or different while ignoring the bottom part. An early potential (P1) at the posterior channels was smaller when the attended top parts were the same compared with when they are different, indicating an adaptation effect. Critically, for trials with identical top parts, P1 was larger when the irrelevant bottom parts were different, indicating a release of adaptation. This effect was present only when the two character parts were aligned but not misaligned, and only for characters but not for pseudocharacters. The finding of early sensitivity to all parts of a Chinese character suggests that Chinese characters are represented holistically at a perceptual level.

Neural substrates underlying intentional empathy

Although empathic responses to stimuli with emotional contents may occur automatically, humans are capable to intentionally empathize with other individuals. Intentional empathy for others is even possible when they do not show emotional expressions. However, little is known about the neuronal mechanisms of this intentionally controlled empathic process. To investigate the neuronal substrates underlying intentional empathy, we scanned 20 healthy Chinese subjects, using fMRI, when they tried to feel inside the emotional states of neutral or angry faces of familiar (Asian) and unfamiliar (Caucasian) models. Skin color evaluation of the same stimuli served as a control task. Compared to a baseline condition, the empathy task revealed a network of established empathy regions, including the anterior cingulate cortex, bilateral inferior frontal cortex and bilateral anterior insula. The contrast of intentional empathy vs skin color evaluation, however, revealed three regions: the bilateral inferior frontal cortex, whose hemodynamic responses were independent of perceived emotion and familiarity and the right-middle temporal gyrus, whose activity was modulated by emotion but not by familiarity. These findings extend our understanding of the role of the inferior frontal cortex and the middle temporal gyrus in empathy by demonstrating their involvement in intentional empathy.

Face-related ERPs are modulated by point of gaze

This study examined the influence of gaze fixation on face-sensitive ERPs. A fixation crosshair presented prior to face onset directed visual attention to upper, central, or lower face regions while ERPs were recorded. This manipulation modulated a face-sensitive component (N170) but not an early sensory component (P1). Upper and lower face fixations elicited enhanced N170 amplitude and longer N170 latency. Results expand upon extant hemodynamic research by demonstrating early effects at basic stages of face processing. These findings distinguish attention to facial features in context from attention to isolated features, and they inform electrophysiological studies of face processing in clinical populations.

Relationship of the BOLD signal with VEP for ultrashort duration visual stimuli (0.1 to 5 ms) in humans

There is currently a great interest to combine electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) to study brain function. Earlier studies have shown different EEG components to correlate well with the fMRI signal arguing for a complex relationship between both measurements. In this study, using separate EEG and fMRI measurements, we show that (1) 0.1 ms visual stimulation evokes detectable hemodynamic and visual-evoked potential (VEP) responses, (2) the negative VEP deflection at approximately 80 ms (N2) co-varies with stimulus duration/intensity such as with blood oxygenation level-dependent (BOLD) response; the positive deflection at approximately 120 ms (P2) does not, and (3) although the N2 VEP-BOLD relationship is approximately linear, deviation is evident at the limit of zero N2 VEP. The latter finding argues that, although EEG and fMRI measurements can co-vary, they reflect partially independent processes in the brain tissue. Finally, it is shown that the stimulus-induced impulse response function (IRF) at 0.1 ms and the intrinsic IRF during rest have different temporal dynamics, possibly due to predominance of neuromodulation during rest as compared with neurotransmission during stimulation. These results extend earlier findings regarding VEP-BOLD coupling and highlight the component- and context-dependency of the relationship between evoked potentials and hemodynamic responses.

On the functional significance of Novelty-P3: facilitation by unexpected novel sounds

The unexpected occurrence of a change in the auditory context has been shown to result in distraction due to a momentary attention shift. These unexpected sounds elicit the Novelty-P3 (NP3) response which has been proposed as an electrophysiological index of distractibility, and used as such in the evaluation of several clinical populations. However, unexpected sounds also result in facilitation under certain conditions. Here, we investigate the electrophysiological concomitants of novel sounds in a task in which these sounds facilitate visual task performance. Novel sounds elicited NP3 and resulted in an enhancement of the visual P300 response to subsequent visual targets. This result clearly argues against the use of NP3 as an index of distractibility and asks for a reformulation of the functional significance of this response. We suggest that the NP3 is a complex signal that comprises alerting, orienting and executive control processes triggered by the unexpected stimulus.

Hemodynamic and electrophysiological signals of conflict processing in the Chinese-character Stroop task: a simultaneous near-infrared spectroscopy and event-related potential study

A dual-modality method combining continuous-wave near-infrared spectroscopy (NIRS) and event-related potentials (ERPs) was developed for the Chinese-character color-word Stroop task, which included congruent, incongruent, and neutral stimuli. Sixteen native Chinese speakers participated in this study. Hemodynamic and electrophysiological signals in the prefrontal cortex (PFC) were monitored simultaneously by NIRS and ERP. The hemodynamic signals were represented by relative changes in oxy-, deoxy-, and total hemoglobin concentration, whereas the electrophysiological signals were characterized by the parameters P450, N500, and P600. Both types of signals measured at four regions of the PFC were analyzed and compared spatially and temporally among the three different stimuli. We found that P600 signals correlated significantly with the hemodynamic parameters, suggesting that the PFC executes conflict-solving function. Additionally, we observed that the change in deoxy-Hb concentration showed higher sensitivity in response to the Stroop task than other hemodynamic signals. Correlation between NIRS and ERP signals revealed that the vascular response reflects the cumulative effect of neural activities. Taken together, our findings demonstrate that this new dual-modality method is a useful approach to obtaining more information during cognitive and physiological studies.

Contextual blending of ingroup/outgroup face stimuli and word valence: LPP modulation and convergence of measures

Background

Several event related potential (ERP) studies have investigated the time course of different aspects of evaluative processing in social bias research. Various reports suggest that the late positive potential (LPP) is modulated by basic evaluative processes, and some reports suggest that in-/outgroup relative position affects ERP responses. In order to study possible LPP blending between facial race processing and semantic valence (positive or negative words), we recorded ERPs while indigenous and non-indigenous participants who were matched by age and gender performed an implicit association test (IAT). The task involved categorizing faces (ingroup and outgroup) and words (positive and negative). Since our paradigm implies an evaluative task with positive and negative valence association, a frontal distribution of LPPs similar to that found in previous reports was expected. At the same time, we predicted that LPP valence lateralization would be modulated not only by positive/negative associations but also by particular combinations of valence, face stimuli and participant relative position.

Results

Results showed that, during an IAT, indigenous participants with greater behavioral ingroup bias displayed a frontal LPP that was modulated in terms of complex contextual associations involving ethnic group and valence. The LPP was lateralized to the right for negative valence stimuli and to the left for positive valence stimuli. This valence lateralization was influenced by the combination of valence and membership type relevant to compatibility with prejudice toward a minority. Behavioral data from the IAT and an explicit attitudes questionnaire were used to clarify this finding and showed that ingroup bias plays an important role. Both ingroup favoritism and indigenous/non-indigenous differences were consistently present in the data.

Conclusion

Our results suggest that frontal LPP is elicited by contextual blending of evaluative judgments of in-/outgroup information and positive vs. negative valence association and confirm recent research relating in-/outgroup ERP modulation and frontal LPP. LPP modulation may cohere with implicit measures of attitudes. The convergence of measures that were observed supports the idea that racial and valence evaluations are strongly influenced by context. This result adds to a growing set of evidence concerning contextual sensitivity of different measures of prejudice.

Rapid influence of emotional scenes on encoding of facial expressions: an ERP study

In daily life, we perceive a person's facial reaction as part of the natural environment surrounding it. Because most studies have investigated how facial expressions are recognized by using isolated faces, it is unclear what role the context plays. Although it has been observed that the N170 for facial expressions is modulated by the emotional context, it was not clear whether individuals use context information on this stage of processing to discriminate between facial expressions. The aim of the present study was to investigate how the early stages of face processing are affected by emotional scenes when explicit categorizations of fearful and happy facial expressions are made. Emotion effects were found for the N170, with larger amplitudes for faces in fearful scenes as compared to faces in happy and neutral scenes. Critically, N170 amplitudes were significantly increased for fearful faces in fearful scenes as compared to fearful faces in happy scenes and expressed in left-occipito-temporal scalp topography differences. Our results show that the information provided by the facial expression is combined with the scene context during the early stages of face processing.

Regional differences in the coupling between resting cerebral blood flow and metabolism may indicate action preparedness as a default state

Although most functional neuroimaging studies examine task effects, interest intensifies in the "default" resting brain. Resting conditions show consistent regional activity, yet oxygen extraction fraction constancy across regions. We compared resting cerebral metabolic rates of glucose (CMRgl) measured with 18F-labeled 2-fluoro-2-deoxy-D-glucose to cerebral blood flow (CBF) 15O-H2O measures, using the same positron emission tomography scanner in 2 samples (n = 60 and 30) of healthy right-handed adults. Region to whole-brain ratios were calculated for 35 standard regions of interest, and compared between CBF and CMRgl to determine perfusion relative to metabolism. Primary visual and auditory areas showed coupling between CBF and CMRgl, limbic and subcortical regions--basal ganglia, thalamus and posterior fossa structures--were hyperperfused, whereas association cortices were hypoperfused. Hyperperfusion was higher in left than right hemisphere for most cortical and subcallosal limbic regions, but symmetric in cingulate, basal ganglia and somatomotor regions. Hyperperfused regions are perhaps those where activation is anticipated at short notice, whereas downstream cortical modulatory regions have longer "lead times" for deployment. The novel observation of systematic uncoupling of CBF and CMRgl may help elucidate the potential biological significance of the "default" resting state. Whether greater left hemispheric hyperperfusion reflects lateral dominance needs further examination.

Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze

In 2 human event-related brain potential (ERP) experiments, we examined the feedback error-related negativity (fERN), an ERP component associated with reward processing by the midbrain dopamine system, and the N170, an ERP component thought to be generated by the medial temporal lobe (MTL), to investigate the contributions of these neural systems toward learning to find rewards in a "virtual T-maze" environment. We found that feedback indicating the absence versus presence of a reward differentially modulated fERN amplitude, but only when the outcome was not predicted by an earlier stimulus. By contrast, when a cue predicted the reward outcome, then the predictive cue (and not the feedback) differentially modulated fERN amplitude. We further found that the spatial location of the feedback stimuli elicited a large N170 at electrode sites sensitive to right MTL activation and that the latency of this component was sensitive to the spatial location of the reward, occurring slightly earlier for rewards following a right versus left turn in the maze. Taken together, these results confirm a fundamental prediction of a dopamine theory of the fERN and suggest that the dopamine and MTL systems may interact in navigational learning tasks.

Early ERP components differentially extract facial features: evidence for spatial frequency-and-contrast detectors

It is generally accepted that the N170 component of an event-related potential (ERP) reflects the structural encoding of faces and is specialized for face processing. Recent neuroimaging and ERP studies have demonstrated that spatial frequency is a crucial factor for face recognition. To clarify which early ERP components reflect either coarse (low spatial frequency, LSF) or fine (high spatial frequency, HSF) processing of faces, we recorded ERPs induced by manipulated face stimuli. By filtering the original grayscale faces (broadband spatial frequency) spatially, we created LSF and HSF face stimuli. Next, we created physically equiluminant (PEL) face stimuli to eliminate the effects of lower order information, such as luminance and contrast. The P1 amplitude at the occipital region was augmented by LSF faces, while the N170 amplitude increased for HSF faces. The occipital P1 amplitude for PEL faces was relatively unaffected compared with that for PEL houses. In addition, the occipital N2 for PEL faces was spatiotemporally separable from N170 in a time-window between P1 and N170. These results indicate that P1 reflects coarse processing of faces, and that the face robustness further assures face-specific processing in the early component. Moreover, N2 reflects the early contrast processing of faces whereas N170 analyzes the fine facial features. Our findings suggest the presence of spatial frequency-and-contrast detectors for face processing.

A tale of two recognition systems: implications of the fusiform face area and the visual word form area for lateralized object recognition models

Two areas of current intense interest in the neuroimaging literature are that of the visual word form area (VWFA) and of the fusiform face area (FFA) and their roles in word and face perception, respectively. These two areas are of particular relevance to laterality research because visual word identification and face identification have long been shown to be especially lateralized to the left hemisphere and the right hemisphere, respectively. This review therefore seeks to evaluate their significance for the broader understanding of lateralization of object recognition. A multi-level model of lateralized object recognition is proposed based on a combination of behavioral and neuroimaging findings. Rather than seek to characterize hemispheric asymmetries according to a single principle (e.g., serial-parallel), it is suggested that current observations can be understood in terms of three asymmetric levels of processing, using the framework of the Janus model of hemispheric function. It is suggested that the left hemisphere represents features using an abstract-category code whereas the RH utilizes a specific-exemplar code. The relationships between these features are also coded asymmetrically, with the LH relying on associative co-occurrence values and the RH relying on spatial metrics. Finally, the LH controlled selection system focuses on isolating features and the RH focuses on conjoining features. It is suggested that each hemisphere utilizes efficient (apparently parallel) processing when stimuli are congruent with its preferred processing style and inefficient (apparently serial) processing when they are not, resulting in the typical left-lateralization for orthographic analysis and right-lateralization for face analysis.

Working memory load for faces modulates P300, N170, and N250r

We used event-related potential (ERP) methodology to examine neural activity associated with visual working memory (WM) for faces. There were two main goals. First, to extend previous findings of P300 load modulation to WM for faces. Second, to examine whether N170 and N250r are also influenced by WM load. Between one and four unfamiliar faces were simultaneously presented for memory encoding. After a 1-sec delay, a target face appeared, and participants had to judge whether this face was part of the previous face array. P300 amplitude decreased as WM load increased, and this P300 suppression was observed at both encoding and retrieval. WM load was also found to modulate other ERPs. The amplitude of the N170 elicited by the target face decreased with load, and this N170 decrease leveled off at load 2, reflecting the behavioral WM capacity of around two faces. In addition, the N250r, observed as an ERP difference for target faces that were present in the encoding array relative to target faces that were absent, was also reduced for higher WM loads. These findings extend previous work by showing that P300 modulation by WM load also occurs for faces. Furthermore, we show, for the first time, that WM load affects the N250r and the early visual N170 component. This suggests that higher visual areas play an important role in WM for faces.

Seeing sounds and hearing sights: the influence of prior learning on current perception

It is well known that previous perceptual experiences alter subsequent perception, but the details of the neural underpinnings of this general phenomenon are still sketchy. Here, we ask whether previous experiences with an item (such as seeing a person's face) leads to the alteration of the neural correlates related to processing of the item as such, or whether it creates additional associative connections between such substrates and those activated during prior experience. To address this question, we used magnetoencephalography (MEG) to identify neural changes accompanying subjects' viewing of unfamiliar versus famous faces and hearing the names of unfamiliar versus famous names. We were interested in the nature of the involvement of auditory brain regions in the viewing of faces, and in the involvement of visual regions in the hearing of names. Evoked responses from MEG recordings for the names and faces conditions were localized to auditory and visual cortices, respectively. Unsurprisingly, peak activation strength of evoked responses was larger for famous versus nonfamous names within the superior temporal gyrus (STG), and was similar for famous and nonfamous faces in the occipital cortex. More relevant to the issue of experience on perception, peak activation strength in the STG was larger for viewed famous versus nonfamous faces, and peak activation within the occipital cortex was larger for heard famous versus nonfamous names. Critically, these experience-related responses were present within 150-250 msec of stimulus onset. These findings support the hypothesis that prior experiences may influence processing of faces and names such that perception encompasses more than what is imparted on the senses.

Timing and sequence of brain activity in top-down control of visual-spatial attention

Recent brain imaging studies using functional magnetic resonance imaging (fMRI) have implicated a frontal-parietal network in the top-down control of attention. However, little is known about the timing and sequence of activations within this network. To investigate these timing questions, we used event-related electrical brain potentials (ERPs) and a specially designed visual-spatial attentional-cueing paradigm, which were applied as part of a multi-methodological approach that included a closely corresponding event-related fMRI study using an identical paradigm. In the first 400 ms post cue, attention-directing and control cues elicited similar general cue-processing activity, corresponding to the more lateral subregions of the frontal-parietal network identified with the fMRI. Following this, the attention-directing cues elicited a sustained negative-polarity brain wave that was absent for control cues. This activity could be linked to the more medial frontal-parietal subregions similarly identified in the fMRI as specifically involved in attentional orienting. Critically, both the scalp ERPs and the fMRI-seeded source modeling for this orienting-related activity indicated an earlier onset of frontal versus parietal contribution ( approximately 400 versus approximately 700 ms). This was then followed ( approximately 800-900 ms) by pretarget biasing activity in the region-specific visual-sensory occipital cortex. These results indicate an activation sequence of key components of the attentional-control brain network, providing insight into their functional roles. More specifically, these results suggest that voluntary attentional orienting is initiated by medial portions of frontal cortex, which then recruit medial parietal areas. Together, these areas then implement biasing of region-specific visual-sensory cortex to facilitate the processing of upcoming visual stimuli.

Facial emotion recognition in schizophrenia: when and why does it go awry?

OBJECTIVE

Schizophrenia patients demonstrate impaired emotional processing that may be due, in part, to impaired facial emotion recognition. This study examined event-related potential (ERP) responses to emotional faces in schizophrenia patients and controls to determine when, in the temporal processing stream, patient abnormalities occur.

METHOD

16 patients and 16 healthy control participants performed a facial emotion recognition task. Very sad, somewhat sad, neutral, somewhat happy, and very happy faces were each presented for 100 ms. Subjects indicated whether each face was "Happy", "Neutral", or "Sad". Evoked potential data were obtained using a 32-channel EEG system.

RESULTS

Controls performed better than patients in recognizing facial emotions. In patients, better recognition of happy faces correlated with less severe negative symptoms. Four ERP components corresponding to the P100, N170, N250, and P300 were identified. Group differences were noted for the N170 "face processing" component that underlies the structural encoding of facial features, but not for the subsequent N250 "affect modulation" component. Higher amplitude of the N170 response to sad faces was correlated with less severe delusional symptoms. Although P300 abnormalities were found, the variance of this component was explained by the earlier N170 response.

CONCLUSION

Patients with schizophrenia demonstrate abnormalities in early visual encoding of facial features that precedes the ERP response typically associated with facial affect recognition. This suggests that affect recognition deficits, at least for happy and sad discrimination, are secondary to faulty structural encoding of faces. The association of abnormal face encoding with delusions may denote the physiological basis for clinical misidentification syndromes.

Low spatial frequency filtering modulates early brain processing of affective complex pictures

Recent research on affective processing has suggested that low spatial frequency information of fearful faces provide rapid emotional cues to the amygdala, whereas high spatial frequencies convey fine-grained information to the fusiform gyrus, regardless of emotional expression. In the present experiment, we examined the effects of low (LSF, <15 cycles/image width) and high spatial frequency filtering (HSF, >25 cycles/image width) on brain processing of complex pictures depicting pleasant, unpleasant, and neutral scenes. Event-related potentials (ERP), percentage of recognized stimuli and response times were recorded in 19 healthy volunteers. Behavioral results indicated faster reaction times in response to unpleasant LSF than to unpleasant HSF pictures. Unpleasant LSF pictures and pleasant unfiltered pictures also elicited significant enhancements of P1 amplitudes at occipital electrodes as compared to neutral LSF and unfiltered pictures, respectively; whereas no significant effects of affective modulation were found for HSF pictures. Moreover, mean ERP amplitudes in the time between 200 and 500ms post-stimulus were significantly greater for affective (pleasant and unpleasant) than for neutral unfiltered pictures; whereas no significant affective modulation was found for HSF or LSF pictures at those latencies. The fact that affective LSF pictures elicited an enhancement of brain responses at early, but not at later latencies, suggests the existence of a rapid and preattentive neural mechanism for the processing of motivationally relevant stimuli, which could be driven by LSF cues. Our findings confirm thus previous results showing differences on brain processing of affective LSF and HSF faces, and extend these results to more complex and social affective pictures.