Facial expression recognition takes longer in the posterior superior temporal sulcus than in the occipital face area

Perceptual and semantic deficits in face recognition in semantic dementia

State of the art

Semantic dementia (SD) patients including semantic variant primary progressive aphasia (svPPA) and semantic behavioral variant frontotemporal dementia (sbvFTD) patients show semantic difficulties identifying faces and known people related to right anterior temporal lobe (ATL) atrophy. However, it remains unclear whether they also have perceptual deficits in face recognition.

Methodology

We selected 74 SD patients (54 with svPPA and predominant left ATL atrophy and 20 with sbvFTD and predominant right ATL atrophy) and 36 cognitively healthy controls (HC) from UCSF Memory and Aging Center. They underwent a perceptual face processing test (Benton facial recognition test-short version; BFRT-S), and semantic face processing tests (UCSF Famous people battery - Recognition, Naming, Semantic associations - pictures and words subtests), as well as structural magnetic resonance imaging (MRI). Neural correlates with the task's performance were conducted with a Voxel-based morphometry approach using CAT12.

Results

svPPA and sbvFTD patients were impaired on all semantic face processing tests, with sbvFTD patients performing significantly lower on the famous faces' recognition task in comparison to svPPA, and svPPA performing significantly lower on the naming task in comparison to sbvFTD. These tasks predominantly correlated with gray matter (GM) volumes in the right and left ATL, respectively. Compared to HC, both svPPA and sbvFTD patients showed preserved performance on the perceptual face processing test (BFRT-S), and performance on the BFRT-S negatively correlated with GM volume in the right posterior superior temporal sulcus (pSTS).

Conclusion

Our results suggest that early in the disease, with the atrophy mostly restricted to the anterior temporal regions, SD patients do not present with perceptual deficits. However, more severe SD cases with atrophy in right posterior temporal regions might show lower performance on face perception tests, in addition to the semantic face processing deficits. Early sparing of face perceptual deficits in SD patients, regardless of hemispheric lateralization, furthers our understanding of clinical phenomenology and therapeutical approaches of this complex disease.

A chronometric study of the posterior cerebellum's function in emotional processing

The posterior cerebellum is a recently discovered hub of the affective and social brain, with different subsectors contributing to different social functions. However, very little is known about when the posterior cerebellum plays a critical role in social processing. Due to its location and anatomy, it has been difficult to use traditional approaches to directly study the chronometry of the cerebellum. To address this gap in cerebellar knowledge, here we investigated the causal contribution of the posterior cerebellum to social processing using a chronometric transcranial magnetic stimulation (TMS) approach. We show that the posterior cerebellum is recruited at an early stage of emotional processing (starting from 100 ms after stimulus onset), simultaneously with the posterior superior temporal sulcus (pSTS), a key node of the social brain. Moreover, using a condition-and-perturb TMS approach, we found that the recruitment of the pSTS in emotional processing is dependent on cerebellar activation. Our results are the first to shed light on chronometric aspects of cerebellar function and its causal functional connectivity with other nodes of the social brain.

Acute transitory head mislocalization - a novel syndrome of pathological embodiment in a patient with traumatic brain injury - a case study

Feeling of body ownership is a complex process with different brain mechanisms involved in integrating the varied and multiple representations of the body . The ability to discriminate between one's own and others' body parts can be lost after brain damage. We report a unique case study of a patient with head injury who experienced a phenomenon where he felt that his head was positioned with another person standing next to him. We describe this as a form of pathological embodiment and call it the "head mislocalization" phenomenon. We report his clinical findings and using the methods of lesion mapping and lesion network mapping postulate the neural mechanisms for this symptom.

TMS disruption of the lateral prefrontal cortex increases neural activity in the default mode network when naming facial expressions

Recognizing facial expressions is dependent on multiple brain networks specialized for different cognitive functions. In the current study, participants (N = 20) were scanned using functional magnetic resonance imaging (fMRI), while they performed a covert facial expression naming task. Immediately prior to scanning thetaburst transcranial magnetic stimulation (TMS) was delivered over the right lateral prefrontal cortex (PFC), or the vertex control site. A group whole-brain analysis revealed that TMS induced opposite effects in the neural responses across different brain networks. Stimulation of the right PFC (compared to stimulation of the vertex) decreased neural activity in the left lateral PFC but increased neural activity in three nodes of the default mode network (DMN): the right superior frontal gyrus, right angular gyrus and the bilateral middle cingulate gyrus. A region of interest analysis showed that TMS delivered over the right PFC reduced neural activity across all functionally localised face areas (including in the PFC) compared to TMS delivered over the vertex. These results suggest that visually recognizing facial expressions is dependent on the dynamic interaction of the face-processing network and the DMN. Our study also demonstrates the utility of combined TMS/fMRI studies for revealing the dynamic interactions between different functional brain networks.

Structural Connectivity and Emotion Recognition Impairment in Children and Adolescents with Chromosome 22q11.2 Deletion Syndrome

Children with chromosome 22q11.2 deletion syndrome (22q11.2DS) exhibit impaired ability to process and understand emotions in others. We measured structural connectivity in children and adolescents with 22q11.2DS (n = 28) and healthy controls (n = 29). Compared to controls, those with 22q11.2DS had poorer social skills and more difficulty recognizing facial emotions. Children with 22q11.2DS also had higher fractional anisotropic diffusion in right amygdala to fusiform gyrus white matter pathways. Right amygdala to fusiform gyrus fractional anisotropy values partially mediated the relationship between 22q11.2DS and social skills, as well as the relationship between 22q11.2DS and emotion recognition accuracy. These findings provide insight into the neural origins of social skills deficits seen in 22q11.2DS and may serve as a biomarker for risk of future psychiatric problems.

Increasing associative plasticity in temporo-occipital back-projections improves visual perception of emotions

The posterior superior temporal sulcus (pSTS) is a critical node in a network specialized for perceiving emotional facial expressions that is reciprocally connected with early visual cortices (V1/V2). Current models of perceptual decision-making increasingly assign relevance to recursive processing for visual recognition. However, it is unknown whether inducing plasticity into reentrant connections from pSTS to V1/V2 impacts emotion perception. Using a combination of electrophysiological and neurostimulation methods, we demonstrate that strengthening the connectivity from pSTS to V1/V2 selectively increases the ability to perceive facial expressions associated with emotions. This behavior is associated with increased electrophysiological activity in both these brain regions, particularly in V1/V2, and depends on specific temporal parameters of stimulation that follow Hebbian principles. Therefore, we provide evidence that pSTS-to-V1/V2 back-projections are instrumental to perception of emotion from facial stimuli and functionally malleable via manipulation of associative plasticity.

Alcohol Use Predicts Face Perception Impairments and Difficulties in Face Recognition

Background: Risky alcohol use is related to a variety of cognitive impairments, including memory and visuo-perceptual difficulties. Remarkably, no prior work has assessed whether usage of alcohol can predict difficulties perceiving facial identity. Objectives: Therefore, this study aimed to investigate whether riskier alcohol consumption predicted impairments in face perception and self-reported difficulties in face recognition. Results: Participants (N = 239, male = 77) were over 18 years old and had normal or corrected-to-normal vision. Alcohol use was assessed using the Alcohol Use Disorder Identification Test (AUDIT), while face recognition difficulties were determined by the 20-item Prosopagnosia Index questionnaire (PI20). A subsample of participants (N = 126, male = 51) completed the Cambridge Face Perception task (CFPT) to assess their face perception ability. Multiple linear regressions showed significant models of prediction on both face perception and face recognition when considering AUDIT score and age as predictors. Conclusion: This study suggested, for the first time, that risky alcohol use predicts both poorer visuo-perceptual processing for faces and self-reported difficulties in face recognition.

Cognitive dysfunctions in high myopia: An overview of potential neural morpho-functional mechanisms

Dementia and cognitive impairment (CIM) carry high levels of mortality. Visual impairment (VI) is linked with CIM risk. High myopia (HM) is a chronic disease frequently leading to irreversible blindness. Current opinion has shifted from retinal injury as the cause of HM to the condition being considered an eye-brain disease. However, the pathogenesis of this disease and the manner in which neural structures are damaged are poorly understood. This review comprehensively discusses the relationship between HM, the central nervous system, and CIM, together with the novel concept of three visual pathways, and possible research perspectives.

Reconstruction of perceived face images from brain activities based on multi-attribute constraints

Reconstruction of perceived faces from brain signals is a hot topic in brain decoding and an important application in the field of brain-computer interfaces. Existing methods do not fully consider the multiple facial attributes represented in face images, and their different activity patterns at multiple brain regions are often ignored, which causes the reconstruction performance very poor. In the current study, we propose an algorithmic framework that efficiently combines multiple face-selective brain regions for precise multi-attribute perceived face reconstruction. Our framework consists of three modules: a multi-task deep learning network (MTDLN), which is developed to simultaneously extract the multi-dimensional face features attributed to facial expression, identity and gender from one single face image, a set of linear regressions (LR), which is built to map the relationship between the multi-dimensional face features and the brain signals from multiple brain regions, and a multi-conditional generative adversarial network (mcGAN), which is used to generate the perceived face images constrained by the predicted multi-dimensional face features. We conduct extensive fMRI experiments to evaluate the reconstruction performance of our framework both subjectively and objectively. The results show that, compared with the traditional methods, our proposed framework better characterizes the multi-attribute face features in a face image, better predicts the face features from brain signals, and achieves better reconstruction performance of both seen and unseen face images in both visual effects and quantitative assessment. Moreover, besides the state-of-the-art intra-subject reconstruction performance, our proposed framework can also realize inter-subject face reconstruction to a certain extent.

Representation of expression and identity by ventral prefrontal neurons

Evidence has suggested that the ventrolateral prefrontal cortex (VLPFC) processes social stimuli, including faces and vocalizations, which are essential for communication. Features embedded within audiovisual stimuli, including emotional expression and caller identity, provide abundant information about an individual's intention, emotional state, motivation, and social status, which are important to encode in a social exchange. However, it is unknown to what extent the VLPFC encodes such features. To investigate the role of VLPFC during social communication, we recorded single-unit activity while rhesus macaques (Macaca mulatta) performed a nonmatch-to-sample task using species-specific face-vocalization stimuli that differed in emotional expression or caller identity. 75% of recorded cells were task-related and of these >70% were responsive during the nonmatch period. A larger proportion of nonmatch cells encoded the stimulus rather than the context of the trial type. A subset of responsive neurons were most commonly modulated by the identity of the nonmatch stimulus and less by the emotional expression, or both features within the face-vocalization stimuli presented during the nonmatch period. Neurons encoding identity were found in VLPFC across a broader region than expression related cells which were confined to only the anterolateral portion of the recording chamber in VLPFC. These findings suggest that, within a working memory paradigm, VLPFC processes features of face and vocal stimuli, such as emotional expression and identity, in addition to task and contextual information. Thus, stimulus and contextual information may be integrated by VLPFC during social communication.

The Neurobiological Correlates of Gaze Perception in Healthy Individuals and Neurologic Patients

The ability to adaptively follow conspecific eye movements is crucial for establishing shared attention and survival. Indeed, in humans, interacting with the gaze direction of others causes the reflexive orienting of attention and the faster object detection of the signaled spatial location. The behavioral evidence of this phenomenon is called gaze-cueing. Although this effect can be conceived as automatic and reflexive, gaze-cueing is often susceptible to context. In fact, gaze-cueing was shown to interact with other factors that characterize facial stimulus, such as the kind of cue that induces attention orienting (i.e., gaze or non-symbolic cues) or the emotional expression conveyed by the gaze cues. Here, we address neuroimaging evidence, investigating the neural bases of gaze-cueing and the perception of gaze direction and how contextual factors interact with the gaze shift of attention. Evidence from neuroimaging, as well as the fields of non-invasive brain stimulation and neurologic patients, highlights the involvement of the amygdala and the superior temporal lobe (especially the superior temporal sulcus (STS)) in gaze perception. However, in this review, we also emphasized the discrepancies of the attempts to characterize the distinct functional roles of the regions in the processing of gaze. Finally, we conclude by presenting the notion of invariant representation and underline its value as a conceptual framework for the future characterization of the perceptual processing of gaze within the STS.

The structural neural correlates of atypical facial expression recognition in autism spectrum disorder

Previous studies have demonstrated that individuals with autism spectrum disorder (ASD) are worse at recognizing facial expressions than are typically developing (TD) individuals. The present study investigated the differences in structural neural correlates of emotion recognition between individuals with and without ASD using voxel-based morphometry (VBM). We acquired structural MRI data from 27 high-functioning adults with ASD and 27 age- and sex-matched TD individuals. The ability to recognize facial expressions was measured using a label-matching paradigm featuring six basic emotions (anger, disgust, fear, happiness, sadness, and surprise). The behavioural task did not find deficits of emotion recognition in ASD after controlling for intellectual ability. However, the VBM analysis for the region of interest showed a positive correlation between the averaged percent accuracy across six basic emotions and the grey matter volume of the right inferior frontal gyrus in TD individuals, but not in individuals with ASD. The VBM for the whole brain region under each emotion condition revealed a positive correlation between the percent accuracy for disgusted faces and the grey matter volume of the left dorsomedial prefrontal cortex in individuals with ASD, but not in TD individuals. The different pattern of correlations suggests that individuals with and without ASD use different processing mechanisms for recognizing others' facial expressions.

The Human Posterior Superior Temporal Sulcus Samples Visual Space Differently From Other Face-Selective Regions

Neuroimaging studies show that ventral face-selective regions, including the fusiform face area (FFA) and occipital face area (OFA), preferentially respond to faces presented in the contralateral visual field (VF). In the current study we measured the VF response of the face-selective posterior superior temporal sulcus (pSTS). Across 3 functional magnetic resonance imaging experiments, participants viewed face videos presented in different parts of the VF. Consistent with prior results, we observed a contralateral VF bias in bilateral FFA, right OFA (rOFA), and bilateral human motion-selective area MT+. Intriguingly, this contralateral VF bias was absent in the bilateral pSTS. We then delivered transcranial magnetic stimulation (TMS) over right pSTS (rpSTS) and rOFA, while participants matched facial expressions in both hemifields. TMS delivered over the rpSTS disrupted performance in both hemifields, but TMS delivered over the rOFA disrupted performance in the contralateral hemifield only. These converging results demonstrate that the contralateral bias for faces observed in ventral face-selective areas is absent in the pSTS. This difference in VF response is consistent with face processing models proposing 2 functionally distinct pathways. It further suggests that these models should account for differences in interhemispheric connections between the face-selective areas across these 2 pathways.

Analgesic Effect of Noninvasive Brain Stimulation for Neuropathic Pain Patients: A Systematic Review

Introduction

The objective of this review is to systematically summarize the consensus on best practices for different NP conditions of the two most commonly utilized noninvasive brain stimulation (NIBS) technologies, repetitive transcranial magnetic stimulation (rTMS), and transcranial direct current stimulation (tDCS).

Methods

PubMed was searched according to the predetermined keywords and criteria. Only English language studies and studies published up to January 31, 2020 were taken into consideration. Meta-analyses, reviews, and systematic reviews were excluded first, and those related to animal studies or involving healthy volunteers were also excluded. Finally, 29 studies covering 826 NP patients were reviewed.

Results

The results from the 24 enrolled studies and 736 NP patients indicate that rTMS successfully relieved the pain symptoms of 715 (97.1%) NP patients. Also, five studies involving 95 NP patients (81.4%) also showed that tDCS successfully relieved NP. In the included studied, the M1 region plays a key role in the analgesic treatment of NIBS. The motor evoked potentials (MEPs), the 10–20 electroencephalography system (EEG 10/20 system), and neuro-navigation methods are used in clinical practice to locate therapeutic targets. Based on the results of the review, the stimulation parameters of rTMS that best induce an analgesic effect are a stimulation frequency of 10–20 Hz, a stimulation intensity of 80–120% of RMT, 1000–2000 pulses, and 5–10 sessions, and the most effective parameters of tDCS are a current intensity of 2 mA, a session duration of 20–30 min, and 5–10 sessions.

Conclusions

Our systematically reviewed the evidence for positive and negative responses to rTMS and tDCS for NP patient care and underscores the analgesic efficacy of NIBS in patients with NP. The treatment of NP should allow the design of optimal treatments for individual patients.

Facemasks and face recognition: Potential impact on synaptic plasticity

Visual recognition of facial expression modulates our social interactions. Compelling experimental evidence indicates that face conveys plenty of information that are fundamental for humans to interact. These are encoded at neural level in specific cortical and subcortical brain regions through activity- and experience-dependent synaptic plasticity processes. The current pandemic, due to the spread of SARS-CoV-2 infection, is causing relevant social and psychological detrimental effects. The institutional recommendations on physical distancing, namely social distancing and wearing of facemasks are effective in reducing the rate of viral spread. However, by impacting social interaction, facemasks might impair the neural responses to recognition of facial cues that are overall critical to our behaviors. In this survey, we briefly review the current knowledge on the neurobiological substrate of facial recognition and discuss how the lack of salient stimuli might impact the ability to retain and consolidate learning and memory phenomena underlying face recognition. Such an "abnormal" visual experience raises the intriguing possibility of a "reset" mechanism, a renewed ability of adult brain to undergo synaptic plasticity adaptations.

Emotion recognition impairments and social well-being following right-hemisphere stroke

Accurately recognizing and responding to the emotions of others is essential for proper social communication and helps bind strong relationships that are particularly important for stroke survivors. Emotion recognition typically engages cortical areas that are predominantly right-lateralized including superior temporal and inferior frontal gyri - regions frequently impacted by right-hemisphere stroke. Since prior work already links right-hemisphere stroke to deficits in emotion recognition, this research aims to extend these findings to determine whether impaired emotion recognition after right-hemisphere stroke is associated with worse social well-being outcomes. Eighteen right-hemisphere stroke patients (≥6 months post-stroke) and 21 neurologically healthy controls completed a multimodal emotion recognition test (Geneva Emotion Recognition Test - Short) and reported engagement in social/non-social activities and levels of social support. Right-hemisphere stroke was associated with worse emotion recognition accuracy, though not all patients exhibited impairment. In line with hypotheses, emotion recognition impairments were associated with greater loss of social activities after stroke, an effect that could not be attributed to stroke severity or loss of non-social activities. Impairments were also linked to reduced patient-reported social support. Results implicate emotion recognition difficulties as a potential antecedent of social withdrawal after stroke and warrant future research to test emotion recognition training post-stroke.

Evidence for a Third Visual Pathway Specialized for Social Perception

Existing models propose that primate visual cortex is divided into two functionally distinct pathways. The ventral pathway computes the identity of an object; the dorsal pathway computes the location of an object, and the actions related to that object. Despite remaining influential, the two visual pathways model requires revision. Both human and non-human primate studies reveal the existence of a third visual pathway on the lateral brain surface. This third pathway projects from early visual cortex, via motion-selective areas, into the superior temporal sulcus (STS). Studies demonstrating that the STS computes the actions of moving faces and bodies (e.g., expressions, eye-gaze, audio-visual integration, intention, and mood) show that the third visual pathway is specialized for the dynamic aspects of social perception.

Cross-Subject Commonality of Emotion Representations in Dorsal Motion-Sensitive Areas

Emotion perception is a crucial question in cognitive neuroscience and the underlying neural substrates have been the subject of intense study. One of our previous studies demonstrated that motion-sensitive areas are involved in the perception of facial expressions. However, it remains unclear whether emotions perceived from whole-person stimuli can be decoded from the motion-sensitive areas. In addition, if emotions are represented in the motion-sensitive areas, we may further ask whether the representations of emotions in the motion-sensitive areas can be shared across individual subjects. To address these questions, this study collected neural images while participants viewed emotions (joy, anger, and fear) from videos of whole-person expressions (contained both face and body parts) in a block-design functional magnetic resonance imaging (fMRI) experiment. Multivariate pattern analysis (MVPA) was conducted to explore the emotion decoding performance in individual-defined dorsal motion-sensitive regions of interest (ROIs). Results revealed that emotions could be successfully decoded from motion-sensitive ROIs with statistically significant classification accuracies for three emotions as well as positive versus negative emotions. Moreover, results from the cross-subject classification analysis showed that a person’s emotion representation could be robustly predicted by others’ emotion representations in motion-sensitive areas. Together, these results reveal that emotions are represented in dorsal motion-sensitive areas and that the representation of emotions is consistent across subjects. Our findings provide new evidence of the involvement of motion-sensitive areas in the emotion decoding, and further suggest that there exists a common emotion code in the motion-sensitive areas across individual subjects.

Asymmetric Contributions of the Fronto-Parietal Network to Emotional Conflict in the Word–Face Interference Task

The fronto-parietal network is involved in top-down and bottom-up processes necessary to achieve cognitive control. We investigated the role of asymmetric enhancement of the left dorsolateral prefrontal cortex (lDLPFC) and right posterior parietal cortex (rPPC) in cognitive control under conditions of emotional conflict arising from emotional distractors. The effects of anodal tDCS over the lDLPFC/cathodal over the rPPC and the effects of anodal tDCS over the rPPC/cathodal over the lDLPFC were compared to sham tDCS in a double-blind design. The findings showed that anodal stimulation over the lDLPFC reduced interference from emotional distractors, but only when participants had already gained experience with the task. In contrast, having already performed the task only eliminated facilitation effects for positive stimuli. Importantly, anodal stimulation of the rPPC did not affect distractors’ interference. Therefore, the present findings indicate that the lDLPFC plays a crucial role in implementing top-down control to resolve emotional conflict, but that experience with the task is necessary to reveal this role.

Theta-burst TMS to the posterior superior temporal sulcus decreases resting-state fMRI connectivity across the face processing network

Humans process faces by using a network of face-selective regions distributed across the brain. Neuropsychological patient studies demonstrate that focal damage to nodes in this network can impair face recognition, but such patients are rare. We approximated the effects of damage to the face network in neurologically normal human participants by using theta burst transcranial magnetic stimulation (TBS). Multi-echo functional magnetic resonance imaging (fMRI) resting-state data were collected pre- and post-TBS delivery over the face-selective right superior temporal sulcus (rpSTS), or a control site in the right motor cortex. Results showed that TBS delivered over the rpSTS reduced resting-state connectivity across the extended face processing network. This connectivity reduction was observed not only between the rpSTS and other face-selective areas, but also between nonstimulated face-selective areas across the ventral, medial, and lateral brain surfaces (e.g., between the right amygdala and bilateral fusiform face areas and occipital face areas). TBS delivered over the motor cortex did not produce significant changes in resting-state connectivity across the face processing network. These results demonstrate that, even without task-induced fMRI signal changes, disrupting a single node in a brain network can decrease the functional connectivity between nodes in that network that have not been directly stimulated.

Human behavior is dependent on brain networks that perform different cognitive functions. We combined theta burst transcranial magnetic stimulation (TBS) with resting-state fMRI to study the face processing network. Disruption of the face-selective right posterior superior temporal sulcus (rpSTS) reduced fMRI connectivity across the face network. This impairment in connectivity was observed not only between the rpSTS and other face-selective areas, but also between nonstimulated face-selective areas on the ventral and medial brain surfaces (e.g., between the right amygdala and bilateral fusiform face areas and occipital face areas). Thus, combined TBS/fMRI can be used to approximate and measure the effects of focal brain damage on brain networks, and suggests such an approach may be useful for mapping intrinsic network organization.

Increased Accuracy of Emotion Recognition in Individuals with Autism-Like Traits after Five Days of Magnetic Stimulations

Individuals with autism-like traits (ALT) belong to a subclinical group with similar social deficits as autism spectrum disorders (ASD). Their main social deficits include atypical eye contact and difficulty in understanding facial expressions, both of which are associated with an abnormality of the right posterior superior temporal sulcus (rpSTS). It is still undetermined whether it is possible to improve the social function of ALT individuals through noninvasive neural modulation. To this end, we randomly assigned ALT individuals into the real (n = 16) and sham (n = 16) stimulation groups. All subjects received five consecutive days of intermittent theta burst stimulation (iTBS) on the rpSTS. Eye tracking data and functional magnetic resonance imaging (fMRI) data were acquired on the first and sixth days. The real group showed significant improvement in emotion recognition accuracy after iTBS, but the change was not significantly larger than that in the sham group. Resting-state functional connectivity (rsFC) between the rpSTS and the left cerebellum significantly decreased in the real group than the sham group after iTBS. At baseline, rsFC in the left cerebellum was negatively correlated with emotion recognition accuracy. Our findings indicated that iTBS of the rpSTS could improve emotion perception of ALT individuals by modulating associated neural networks. This stimulation protocol could be a vital therapeutic strategy for the treatment of ASD.

Dual-site TMS demonstrates causal functional connectivity between the left and right posterior temporal sulci during facial expression recognition

Background

Neuroimaging studies suggest that facial expression recognition is processed in the bilateral posterior superior temporal sulcus (pSTS). Our recent repetitive transcranial magnetic stimulation (rTMS) study demonstrates that the bilateral pSTS is causally involved in expression recognition, although involvement of the right pSTS is greater than involvement of the left pSTS.

Objective

/Hypothesis: In this study, we used a dual-site TMS to investigate whether the left pSTS is functionally connected to the right pSTS during expression recognition. We predicted that if this connection exists, simultaneous TMS disruption of the bilateral pSTS would impair expression recognition to a greater extent than unilateral stimulation of the right pSTS alone.

Methods

Participants attended two TMS sessions. In Session 1, participants performed an expression recognition task while rTMS was delivered to the face-sensitive right pSTS (experimental site), object-sensitive right lateral occipital complex (control site) or no rTMS was delivered (behavioural control). In Session 2, the same experimental design was used, except that continuous theta-burst stimulation (cTBS) was delivered to the left pSTS immediately before behavioural testing commenced. Session order was counter-balanced across participants.

Results

In Session 1, rTMS to the rpSTS impaired performance accuracy compared to the control conditions. Crucially in Session 2, the size of this impairment effect doubled after cTBS was delivered to the left pSTS.

Conclusions

Our results provide evidence for a causal functional connection between the left and right pSTS during expression recognition. In addition, this study further demonstrates the utility of the dual-site TMS for investigating causal functional links between brain regions.

•

Dual-site TMS was used to test causal functional connectivity between left and right pSTS during expression recognition.

•

rTMS impaired facial expression recognition when delivered to the right pSTS during a facial expression recognition task.

•

cTBS delivered to the left pSTS prior to the task doubled the impairment effect of rTMS to the right pSTS during the task.

•

The results demonstrate causal functional connectivity between the left and right pSTS during expression recognition.

•

The results also demonstrate the utility of dual-site TMS for investigating interregional causal functional connectivity.

The Causal Role of Left and Right Superior Temporal Gyri in Speech Perception in Noise: A Transcranial Magnetic Stimulation Study

Successful perception of speech in everyday listening conditions requires effective listening strategies to overcome common acoustic distortions, such as background noise. Convergent evidence from neuroimaging and clinical studies identify activation within the temporal lobes as key to successful speech perception. However, current neurobiological models disagree on whether the left temporal lobe is sufficient for successful speech perception or whether bilateral processing is required. We addressed this issue using TMS to selectively disrupt processing in either the left or right superior temporal gyrus (STG) of healthy participants to test whether the left temporal lobe is sufficient or whether both left and right STG are essential. Participants repeated keywords from sentences presented in background noise in a speech reception threshold task while receiving online repetitive TMS separately to the left STG, right STG, or vertex or while receiving no TMS. Results show an equal drop in performance following application of TMS to either left or right STG during the task. A separate group of participants performed a visual discrimination threshold task to control for the confounding side effects of TMS. Results show no effect of TMS on the control task, supporting the notion that the results of Experiment 1 can be attributed to modulation of cortical functioning in STG rather than to side effects associated with online TMS. These results indicate that successful speech perception in everyday listening conditions requires both left and right STG and thus have ramifications for our understanding of the neural organization of spoken language processing.

Effects of online repetitive transcranial magnetic stimulation (rTMS) on cognitive processing: A meta-analysis and recommendations for future studies

Online repetitive transcranial magnetic stimulation (rTMS), applied while subjects are performing a task, is widely used to disrupt brain regions underlying cognition. However, online rTMS has also induced "paradoxical enhancement". Given the rapid proliferation of this approach, it is crucial to develop a better understanding of how online stimulation influences cognition, and the optimal parameters to achieve desired effects. To accomplish this goal, a quantitative meta-analysis was performed with random-effects models fitted to reaction time (RT) and accuracy data. The final dataset included 126 studies published between 1998 and 2016, with 244 total effects for reaction times, and 202 for accuracy. Meta-analytically, rTMS at 10 Hz and 20 Hz disrupted accuracy for attention, executive, language, memory, motor, and perception domains, while no effects were found with 1 Hz or 5 Hz. Stimulation applied at and 10 and 20 Hz slowed down RTs in attention and perception tasks. No performance enhancement was found. Meta-regression analysis showed that fMRI-guided targeting and short inter-trial intervals are associated with increased disruptive effects with rTMS.

Neural correlates of visual aesthetic appreciation: insights from non-invasive brain stimulation

During the last decade, non-invasive brain stimulation techniques have been increasingly employed in the field of neuroaesthetics research to shed light on the possible causal role of different brain regions contributing to aesthetic appreciation. Here, I review studies that have employed transcranial magnetic stimulation (TMS) and transcranial direct current stimulation (tDCS) to investigate neurocognitive mechanisms mediating visual aesthetic appreciation for different stimuli categories (faces, bodies, paintings). The review first considers studies that have assessed the possible causal contribution of cortical regions in mediating aesthetic appreciation along the visual ventral and dorsal pathways (i.e., the extrastriate body area, the motion-sensitive region V5/MT+ , the lateral occipital complex and the posterior parietal cortex). It then considers TMS and tDCS studies that have targeted premotor and motor regions, as well as other areas involved in body and facial expression processing (such as the superior temporal sulcus and the somatosensory cortex) to assess their role in aesthetic evaluation. Finally, it discusses studies that have targeted medial and dorsolateral prefrontal regions leading to significant changes in aesthetic appreciation for both biological stimuli (faces and bodies) and artworks. Possible mechanisms mediating stimulation effects on aesthetic judgments are discussed. A final section considers both methodological limitations of the reviewed studies (including levels of statistical power and the need for further replication) and the future potential for non-invasive brain stimulation to significantly contribute to the understanding of the neural bases of visual aesthetic experiences.

Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks

The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception.

Deep convolutional neural networks (DCNNs) are able to identify faces on par with humans. Here, the authors record neuronal activity from higher visual areas in humans and show that face-selective responses in the brain show similarity to those in the intermediate layers of the DCNN.

Network Representations of Facial and Bodily Expressions: Evidence From Multivariate Connectivity Pattern Classification

Emotions can be perceived from both facial and bodily expressions. Our previous study has found the successful decoding of facial expressions based on the functional connectivity (FC) patterns. However, the role of the FC patterns in the recognition of bodily expressions remained unclear, and no neuroimaging studies have adequately addressed the question of whether emotions perceiving from facial and bodily expressions are processed rely upon common or different neural networks. To address this, the present study collected functional magnetic resonance imaging (fMRI) data from a block design experiment with facial and bodily expression videos as stimuli (three emotions: anger, fear, and joy), and conducted multivariate pattern classification analysis based on the estimated FC patterns. We found that in addition to the facial expressions, bodily expressions could also be successfully decoded based on the large-scale FC patterns. The emotion classification accuracies for the facial expressions were higher than that for the bodily expressions. Further contributive FC analysis showed that emotion-discriminative networks were widely distributed in both hemispheres, containing regions that ranged from primary visual areas to higher-level cognitive areas. Moreover, for a particular emotion, discriminative FCs for facial and bodily expressions were distinct. Together, our findings highlight the key role of the FC patterns in the emotion processing, indicating how large-scale FC patterns reconfigure in processing of facial and bodily expressions, and suggest the distributed neural representation for the emotion recognition. Furthermore, our results also suggest that the human brain employs separate network representations for facial and bodily expressions of the same emotions. This study provides new evidence for the network representations for emotion perception and may further our understanding of the potential mechanisms underlying body language emotion recognition.

A functional dissociation of face-, body- and scene-selective brain areas based on their response to moving and static stimuli

The human brain contains areas that respond selectively to faces, bodies and scenes. Neuroimaging studies have shown that a subset of these areas preferentially respond more to moving than static stimuli, but the reasons for this functional dissociation remain unclear. In the present study, we simultaneously mapped the responses to motion in face-, body- and scene-selective areas in the right hemisphere using moving and static stimuli. Participants (N = 22) were scanned using functional magnetic resonance imaging (fMRI) while viewing videos containing bodies, faces, objects, scenes or scrambled objects, and static pictures from the beginning, middle and end of each video. Results demonstrated that lateral areas, including face-selective areas in the posterior and anterior superior temporal sulcus (STS), the extrastriate body area (EBA) and the occipital place area (OPA) responded more to moving than static stimuli. By contrast, there was no difference between the response to moving and static stimuli in ventral and medial category-selective areas, including the fusiform face area (FFA), occipital face area (OFA), amygdala, fusiform body area (FBA), retrosplenial complex (RSC) and parahippocampal place area (PPA). This functional dissociation between lateral and ventral/medial brain areas that respond selectively to different visual categories suggests that face-, body- and scene-selective networks may be functionally organized along a common dimension.

Mapping the impairment in decoding static facial expressions of emotion in prosopagnosia

Acquired prosopagnosia is characterized by a deficit in face recognition due to diverse brain lesions, but interestingly most prosopagnosic patients suffering from posterior lesions use the mouth instead of the eyes for face identification. Whether this bias is present for the recognition of facial expressions of emotion has not yet been addressed. We tested PS, a pure case of acquired prosopagnosia with bilateral occipitotemporal lesions anatomically sparing the regions dedicated for facial expression recognition. PS used mostly the mouth to recognize facial expressions even when the eye area was the most diagnostic. Moreover, PS directed most of her fixations towards the mouth. Her impairment was still largely present when she was instructed to look at the eyes, or when she was forced to look at them. Control participants showed a performance comparable to PS when only the lower part of the face was available. These observations suggest that the deficits observed in PS with static images are not solely attentional, but are rooted at the level of facial information use. This study corroborates neuroimaging findings suggesting that the Occipital Face Area might play a critical role in extracting facial features that are integrated for both face identification and facial expression recognition in static images.

Face memory and face recognition in children and adolescents with attention deficit hyperactivity disorder: A systematic review

This review focuses on facial recognition abilities in children and adolescents with attention deficit hyperactivity disorder (ADHD). A systematic review, using PRISMA guidelines, was conducted to identify original articles published prior to May 2017 pertaining to memory, face recognition, affect recognition, facial expression recognition and recall of faces in children and adolescents with ADHD. The qualitative synthesis based on different studies shows a particular focus of the research on facial affect recognition without paying similar attention to the structural encoding of facial recognition. In this review, we further investigate facial recognition abilities in children and adolescents with ADHD, providing synthesis of the results observed in the literature, while detecting face recognition tasks used on face processing abilities in ADHD and identifying aspects not yet explored.

Face Processing Systems: From Neurons to Real-World Social Perception

Primate face processing depends on a distributed network of interlinked face-selective areas composed of face-selective neurons. In both humans and macaques, the network is divided into a ventral stream and a dorsal stream, and the functional similarities of the areas in humans and macaques indicate they are homologous. Neural correlates for face detection, holistic processing, face space, and other key properties of human face processing have been identified at the single neuron level, and studies providing causal evidence have established firmly that face-selective brain areas are central to face processing. These mechanisms give rise to our highly accurate familiar face recognition but also to our error-prone performance with unfamiliar faces. This limitation of the face system has important implications for consequential situations such as eyewitness identification and policing.

Faces, people and the brain: The 45th Sir Frederic Bartlett Lecture

The fact that the face is a source of diverse social signals allows us to use face and person perception as a model system for asking important psychological questions about how our brains are organised. A key issue concerns whether we rely primarily on some form of generic representation of the common physical source of these social signals (the face) to interpret them, or instead create multiple representations by assigning different aspects of the task to different specialist components. Variants of the specialist components hypothesis have formed the dominant theoretical perspective on face perception for more than three decades, but despite this dominance of formally and informally expressed theories, the underlying principles and extent of any division of labour remain uncertain. Here, I discuss three important sources of constraint: first, the evolved structure of the brain; second, the need to optimise responses to different everyday tasks; and third, the statistical structure of faces in the perceiver’s environment. I show how these constraints interact to determine the underlying functional organisation of face and person perception.

Fear boosts the early neural coding of faces

The rapid extraction of facial identity and emotional expressions is critical for adapted social interactions. These biologically relevant abilities have been associated with early neural responses on the face sensitive N170 component. However, whether all facial expressions uniformly modulate the N170, and whether this effect occurs only when emotion categorization is task-relevant, is still unclear. To clarify this issue, we recorded high-resolution electrophysiological signals while 22 observers perceived the six basic expressions plus neutral. We used a repetition suppression paradigm, with an adaptor followed by a target face displaying the same identity and expression (trials of interest). We also included catch trials to which participants had to react, by varying identity (identity-task), expression (expression-task) or both (dual-task) on the target face. We extracted single-trial Repetition Suppression (stRS) responses using a data-driven spatiotemporal approach with a robust hierarchical linear model to isolate adaptation effects on the trials of interest. Regardless of the task, fear was the only expression modulating the N170, eliciting the strongest stRS responses. This observation was corroborated by distinct behavioral performance during the catch trials for this facial expression. Altogether, our data reinforce the view that fear elicits distinct neural processes in the brain, enhancing attention and facilitating the early coding of faces.

Neural substrates of the ability to recognize facial expressions: a voxel-based morphometry study

The recognition of facial expressions of emotion is adaptive for human social interaction, but the ability to do this and the manner in which it is achieved differs among individuals. Previous functional neuroimaging studies have demonstrated that some brain regions, such as the inferior frontal gyrus (IFG), are active during the response to emotional facial expressions in healthy participants, and lesion studies have demonstrated that damage to these structures impairs the recognition of facial expressions. However, it remains to be established whether individual differences in the structure of these regions could be associated with differences in the ability to recognize facial expressions. We investigated this issue using acquired structural magnetic resonance imaging, and assessed the performance of healthy adults with respect to recognition of the facial expressions of six basic emotions. The gray matter volume of the right IFG positively correlated with the total accuracy of facial expression recognition. This suggests that individual differences in the ability to recognize facial expressions are associated with differences in the structure of the right IFG. Furthermore, the mirror neuron activity of the IFG may be important for establishing efficient facial mimicry to facilitate emotion recognition.

An Exploratory TMS Study on Prefrontal Lateralization in Valence Categorization of Facial Expressions

Abstract. Converging neuroimaging and patient data suggest that the dorsolateral prefrontal cortex (DLPFC) is involved in emotional processing. However, it is still not clear whether the DLPFC in the left and right hemisphere is differentially involved in emotion recognition depending on the emotion considered. Here we used transcranial magnetic stimulation (TMS) to shed light on the possible causal role of the left and right DLPFC in encoding valence of positive and negative emotional facial expressions. Participants were required to indicate whether a series of faces displayed a positive or negative expression, while TMS was delivered over the right DLPFC, the left DLPFC, and a control site (vertex). Interfering with activity in both the left and right DLPFC delayed valence categorization (compared to control stimulation) to a similar extent irrespective of emotion type. Overall, we failed to demonstrate any valence-related lateralization in the DLPFC by using TMS. Possible methodological limitations are discussed.

Concurrent development of facial identity and expression discrimination

Facial identity and facial expression processing both appear to follow a protracted developmental trajectory, yet these trajectories have been studied independently and have not been directly compared. Here we investigated whether these processes develop at the same or different rates using matched identity and expression discrimination tasks. The Identity task begins with a target face that is a morph between two identities (Identity A/Identity B). After a brief delay, the target face is replaced by two choice faces: 100% Identity A and 100% Identity B. Children 5-12-years-old were asked to pick the choice face that is most similar to the target identity. The Expression task is matched in format and difficulty to the Identity task, except the targets are morphs between two expressions (Angry/Happy, or Disgust/Surprise). The same children were asked to pick the choice face with the expression that is most similar to the target expression. There were significant effects of age, with performance improving (becoming more accurate and faster) on both tasks with increasing age. Accuracy and reaction times were not significantly different across tasks and there was no significant Age x Task interaction. Thus, facial identity and facial expression discrimination appear to develop at a similar rate, with comparable improvement on both tasks from age five to twelve. Because our tasks are so closely matched in format and difficulty, they may prove useful for testing face identity and face expression processing in special populations, such as autism or prosopagnosia, where one of these abilities might be impaired.

tDCS application over the STG improves the ability to recognize and appreciate elements involved in humor processing

The superior temporal gyrus (STG) has been found to play a crucial role in the recognition of actions and facial expressions and may, therefore, be critical for the processing of humorous information. Here we investigated whether tDCS application to the STG would modulate the ability to recognize and appreciate the comic element in serious and comedic situations of misfortune. To this aim, the effects of different types of tDCS stimulation on the STG were analyzed during a task in which the participants were instructed to categorize various misfortunate situations as "comic" or "not comic". Participants underwent three different tDCS conditions: Anodal-right/Cathodal-left; Cathodal-right/Anodal-left; Sham. Images depicting people involved in accidents were grouped into three categories based on the facial expression of the victim: angry or painful (Affective); bewildered and funny (Comic); and images that did not contain the victim's face (No Face). An improvement in mean reaction times in response to both the Comic and No Face stimuli was observed following Anodal-left/Cathodal-right stimulation when compared to sham stimulation. This suggests that this stimulation type reduced the reaction times to socio-emotional complex scenes, regardless of facial expression. The Anodal-right/Cathodal-left stimulation reduced the mean reaction times for Comic stimuli only, suggesting that specifically the right STG may be involved in facial expression recognition and in the appreciation of the comic element in misfortunate situations. These results suggest a functional hemispheric asymmetry in STG response to social stimuli: the left STG might have a role in a general comprehension of social complex situations, while the right STG may be involved in the ability to recognize and integrate specific emotional aspects in a complex scene.

The Superior Temporal Sulcus Is Causally Connected to the Amygdala: A Combined TBS-fMRI Study

Nonhuman primate neuroanatomical studies have identified a cortical pathway from the superior temporal sulcus (STS) projecting into dorsal subregions of the amygdala, but whether this same pathway exists in humans is unknown. Here, we addressed this question by combining theta burst transcranial magnetic stimulation (TBS) with fMRI to test the prediction that the STS and amygdala are functionally connected during face perception. Human participants (N = 17) were scanned, over two sessions, while viewing 3 s video clips of moving faces, bodies, and objects. During these sessions, TBS was delivered over the face-selective right posterior STS (rpSTS) or over the vertex control site. A region-of-interest analysis revealed results consistent with our hypothesis. Namely, TBS delivered over the rpSTS reduced the neural response to faces (but not to bodies or objects) in the rpSTS, right anterior STS (raSTS), and right amygdala, compared with TBS delivered over the vertex. By contrast, TBS delivered over the rpSTS did not significantly reduce the neural response to faces in the right fusiform face area or right occipital face area. This pattern of results is consistent with the existence of a cortico-amygdala pathway in humans for processing face information projecting from the rpSTS, via the raSTS, into the amygdala. This conclusion is consistent with nonhuman primate neuroanatomy and with existing face perception models.

SIGNIFICANCE STATEMENT

Neuroimaging studies have identified multiple face-selective regions in the brain, but the functional connections between these regions are unknown. In the present study, participants were scanned with fMRI while viewing movie clips of faces, bodies, and objects before and after transient disruption of the face-selective right posterior superior temporal sulcus (rpSTS). Results showed that TBS disruption reduced the neural response to faces, but not to bodies or objects, in the rpSTS, right anterior STS (raSTS), and right amygdala. These results are consistent with the existence of a cortico-amygdala pathway in humans for processing face information projecting from the rpSTS, via the raSTS, into the amygdala. This conclusion is consistent with nonhuman primate neuroanatomy and with existing face perception models.