Combined TMS and fMRI Reveal Dissociable Cortical Pathways for Dynamic and Static Face Perception

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Video recordings accurately capture facial expression movements; however, they are difficult for face perception researchers to standardise and manipulate. For this reason, dynamic morphs of photographs are often used, despite their lack of naturalistic facial motion. This study aimed to investigate how humans perceive emotions from faces using real videos and two different approaches to artificially generating dynamic expressions - dynamic morphs, and AI-synthesised deepfakes. Our participants perceived dynamic morphed expressions as less intense when compared with videos (all emotions) and deepfakes (fearful, happy, sad). Videos and deepfakes were perceived similarly. Additionally, they perceived morphed happiness and sadness, but not morphed anger or fear, as less genuine than other formats. Our findings support previous research indicating that social responses to morphed emotions are not representative of those to video recordings. The findings also suggest that deepfakes may offer a more suitable standardized stimulus type compared to morphs. Additionally, qualitative data were collected from participants and analysed using ChatGPT, a large language model. ChatGPT successfully identified themes in the data consistent with those identified by an independent human researcher. According to this analysis, our participants perceived dynamic morphs as less natural compared with videos and deepfakes. That participants perceived deepfakes and videos similarly suggests that deepfakes effectively replicate natural facial movements, making them a promising alternative for face perception research. The study contributes to the growing body of research exploring the usefulness of generative artificial intelligence for advancing the study of human perception.

Bidirectional and Cross-Hemispheric Modulations of Face-Selective Neural Activity Induced by Electrical Stimulation within the Human Cortical Face Network

A major scientific objective of cognitive neuroscience is to define cortico-cortical functional connections supporting cognitive functions. Here, we use an original approach combining frequency-tagging and direct electrical stimulation (DES) to test for bidirectional and cross-hemispheric category-specific modulations within the human cortical face network. A unique patient bilaterally implanted with depth electrodes in multiple face-selective cortical regions of the ventral occipito-temporal cortex (VOTC) was shown 70 s sequences of variable natural object images at a 6 Hz rate, objectively identifying deviant face-selective neural activity at 1.2 Hz (i.e., every five images). Concurrent electrical stimulation was separately applied for 10 seconds on four independently defined face-selective sites in the right and left VOTC. Upon stimulation, we observed reduced or even abolished face-selective neural activity locally and, most interestingly, at distant VOTC recording sites. Remote DES effects were found up to the anterior temporal lobe (ATL) in both forward and backward directions along the VOTC, as well as across the two hemispheres. This reduction was specific to face-selective neural activity, with the general 6 Hz visual response being mostly unaffected. Overall, these results shed light on the functional connectivity of the cortical face-selective network, supporting its non-hierarchical organization as well as bidirectional effective category-selective connections between posterior 'core' regions and the ATL. They also pave the way for widespread and systematic development of this approach to better understand the functional and effective connectivity of human brain networks.

EEG correlates of static and dynamic face perception: the role of naturalistic motion

Much of our understanding of how the brain processes dynamic faces comes from research that compares static photographs to dynamic morphs, which exhibit simplified, computer-generated motion. By comparing static, video recorded, and dynamic morphed expressions, we aim to identify the neural correlates of naturalistic facial dynamism, using time-domain and time-frequency analysis. Dynamic morphs were made from the neutral and peak frames of video recorded transitions of happy and fearful expressions, which retained expression change and removed asynchronous and non-linear features of naturalistic facial motion. We found that dynamic morphs elicited increased N400 amplitudes and lower LPP amplitudes compared to other stimulus types. Video recordings elicited higher LPP amplitudes and greater frontal delta activity compared to other stimuli. Thematic analysis of participant interviews using a large language model revealed that participants found it difficult to assess the genuineness of morphed expressions, and easier to analyse the genuineness of happy compared to fearful expressions. Our findings suggest that animating real faces with artificial motion may violate expectations (N400) and reduce the social salience (LPP) of dynamic morphs. Results also suggest that delta oscillations in the frontal region may be involved with the perception of naturalistic facial motion in happy and fearful expressions. Overall, our findings highlight the sensitivity of neural mechanisms required for face perception to subtle changes in facial motion characteristics, which has important implications for neuroimaging research using faces with simplified motion.

Working memory for gaze benefits from the face context

Retaining gaze in working memory (WM) is essential for successfully navigating through the social world. In the current study, we investigated how WM stores gaze direction by focusing on the role of face context in gaze WM. To address this question, we propose two competing hypotheses. The independence hypothesis predicts that eye gaze is stored independently and is not susceptible to the influence of the surrounding face context. Conversely, the embedding hypothesis claims that gaze WM involves face context and that disruption of holistic face processing would also impair memory for embedded gaze. In three experiments, we adopted different manipulations to disrupt holistic face processing and compared WM performance for gaze within and without face context. In Experiments 1 and 2, we tested WM for gaze direction with schematic upright or inverted faces. We found better performance for gaze within upright faces (vs. inverted faces) by increasing the probability of being remembered. In Experiment 3, we replaced schematic faces with photographic faces, and disrupted holistic processing by using scrambled faces. Results replicated our previous findings, showing that photographic gaze within intact faces was better remembered than gaze presented alone or gaze within scrambled faces. These findings indicate that gaze memory is face-dependent and support the embedding hypothesis.

Cortical Face-Selective Responses Emerge Early in Human Infancy

In human adults, multiple cortical regions respond robustly to faces, including the occipital face area (OFA) and fusiform face area (FFA), implicated in face perception, and the superior temporal sulcus (STS) and medial prefrontal cortex (MPFC), implicated in higher-level social functions. When in development, does face selectivity arise in each of these regions? Here, we combined two awake infant functional magnetic resonance imaging (fMRI) datasets to create a sample size twice the size of previous reports (n = 65 infants; 2.6-9.6 months). Infants watched movies of faces, bodies, objects, and scenes, while fMRI data were collected. Despite variable amounts of data from each infant, individual subject whole-brain activation maps revealed responses to faces compared to nonface visual categories in the approximate location of OFA, FFA, STS, and MPFC. To determine the strength and nature of face selectivity in these regions, we used cross-validated functional region of interest analyses. Across this larger sample size, face responses in OFA, FFA, STS, and MPFC were significantly greater than responses to bodies, objects, and scenes. Even the youngest infants (2-5 months) showed significantly face-selective responses in FFA, STS, and MPFC, but not OFA. These results demonstrate that face selectivity is present in multiple cortical regions within months of birth, providing powerful constraints on theories of cortical development.

The role of preSMA and STS in face recognition: A transcranial magnetic stimulation (TMS) study

Current models propose that facial recognition is mediated by two independent yet interacting anatomo-functional systems: one processing facial features mainly mediated by the Fusiform Face Area and the other involved in the extraction of dynamic information from faces, subserved by Superior Temporal Sulcus (STS). Also, the pre-Supplementary Motor Area (pre-SMA) is implicated in facial expression processing as it is involved in its motor mimicry. However, the literature only shows evidence of the implication of STS and preSMA for facial expression recognition, without relating it to face recognition. In addition, the literature shows a facilitatory role of facial motion in the recognition of unfamiliar faces, particularly for poor recognizers. The present study aimed at studying the role of STS and preSMA in unfamiliar face recognition in people with different face recognition skills. 34 healthy participants received repetitive transcranial magnetic stimulation over the right posterior STS, pre-SMA and as sham during a task of matching of faces encoded through: facial expression, rigid head movement or as static (i.e., absence of any facial or head motion). All faces were represented without emotional content. Results indicate that STS has a direct role in recognizing identities through rigid head movement and an indirect role in facial expression processing. This dissociation represents a step forward with respect to current face processing models suggesting that different types of motion involve separate brain and cognitive processes. PreSMA interacts with face recognition skills, increasing the performance of poor recognizers and decreasing that of good recognizers in all presentation conditions. Together, the results suggest the use of at least partially different mechanisms for face recognition in poor and good recognizers and a different role of STS and preSMA in face recognition.

Eye movement differences when recognising and learning moving and static faces

Seeing a face in motion can help subsequent face recognition. Several explanations have been proposed for this "motion advantage," but other factors that might play a role have received less attention. For example, facial movement might enhance recognition by attracting attention to the internal facial features, thereby facilitating identification. However, there is no direct evidence that motion increases attention to regions of the face that facilitate identification (i.e., internal features) compared with static faces. We tested this hypothesis by recording participants' eye movements while they completed the famous face recognition (Experiment 1, N = 32), and face-learning (Experiment 2, N = 60, Experiment 3, N = 68) tasks, with presentation style manipulated (moving or static). Across all three experiments, a motion advantage was found, and participants directed a higher proportion of fixations to the internal features (i.e., eyes, nose, and mouth) of moving faces versus static. Conversely, the proportion of fixations to the internal non-feature area (i.e., cheeks, forehead, chin) and external area (Experiment 3) was significantly reduced for moving compared with static faces (all ps < .05). Results suggest that during both familiar and unfamiliar face recognition, facial motion is associated with increased attention to internal facial features, but only during familiar face recognition is the magnitude of the motion advantage significantly related functionally to the proportion of fixations directed to the internal features.

Decoding the dynamic perception of risk and speed using naturalistic stimuli: A multivariate, whole-brain analysis

Time-resolved decoding of speed and risk perception in car driving is important for understanding the perceptual processes related to driving safety. In this study, we used an fMRI-compatible trackball with naturalistic stimuli to record dynamic ratings of perceived risk and speed and investigated the degree to which different brain regions were able to decode these. We presented participants with first-person perspective videos of cars racing on the same course. These videos varied in terms of subjectively perceived speed and risk profiles, as determined during a behavioral pilot. During the fMRI experiment, participants used the trackball to dynamically rate subjective risk in a first and speed in a second session and assessed overall risk and speed after watching each video. A standard multivariate correlation analysis based on these ratings revealed sparse decodability in visual areas only for the risk ratings. In contrast, the dynamic rating-based correlation analysis uncovered frontal, visual, and temporal region activation for subjective risk and dorsal visual stream and temporal region activation for subjectively perceived speed. Interestingly, further analyses showed that the brain regions for decoding risk changed over time, whereas those for decoding speed remained constant. Overall, our results demonstrate the advantages of time-resolved decoding to help our understanding of the dynamic networks associated with decoding risk and speed perception in realistic driving scenarios.

The association between childhood trauma and emotion recognition is reduced or eliminated when controlling for alexithymia and psychopathy traits

Emotion recognition shows large inter-individual variability, and is substantially affected by childhood trauma as well as modality, emotion portrayed, and intensity. While research suggests childhood trauma influences emotion recognition, it is unclear whether this effect is consistent when controlling for interrelated individual differences. Further, the universality of the effects has not been explored, most studies have not examined differing modalities or intensities. This study examined childhood trauma's association with accuracy, when controlling for alexithymia and psychopathy traits, and if this varied across modality, emotion portrayed, and intensity. An adult sample (N = 122) completed childhood trauma, alexithymia, and psychopathy questionnaires and three emotion tasks: faces, voices, audio-visual. When investigating childhood trauma alone, there was a significant association with poorer accuracy when exploring modality, emotion portrayed, and intensity. When controlling for alexithymia and psychopathy, childhood trauma remained significant when exploring emotion portrayed, however, it was no longer significant when exploring modality and intensity. In fact, alexithymia was significant when exploring intensity. The effect sizes overall were small. Our findings suggest the importance of controlling for interrelated individual differences. Future research should explore more sensitive measures of emotion recognition, such as intensity ratings and sensitivity to intensity, to see if these follow accuracy findings.

Looming Angry Faces: Preliminary Evidence of Differential Electrophysiological Dynamics for Filtered Stimuli via Low and High Spatial Frequencies

Looming motion interacts with threatening emotional cues in the initial stages of visual processing. However, the underlying neural networks are unclear. The current study investigated if the interactive effect of threat elicited by angry and looming faces is favoured by rapid, magnocellular neural pathways and if exogenous or endogenous attention influences such processing. Here, EEG/ERP techniques were used to explore the early ERP responses to moving emotional faces filtered for high spatial frequencies (HSF) and low spatial frequencies (LSF). Experiment 1 applied a passive-viewing paradigm, presenting filtered angry and neutral faces in static, approaching, or receding motions on a depth-cued background. In the second experiment, broadband faces (BSF) were included, and endogenous attention was directed to the expression of faces. Our main results showed that regardless of attentional control, P1 was enhanced by BSF angry faces, but neither HSF nor LSF faces drove the effect of facial expressions. Such findings indicate that looming motion and threatening expressions are integrated rapidly at the P1 level but that this processing relies neither on LSF nor on HSF information in isolation. The N170 was enhanced for BSF angry faces regardless of attention but was enhanced for LSF angry faces during passive viewing. These results suggest the involvement of a neural pathway reliant on LSF information at the N170 level. Taken together with previous reports from the literature, this may indicate the involvement of multiple parallel neural pathways during early visual processing of approaching emotional faces.

TMS disruption of the lateral prefrontal cortex increases neural activity in the default mode network when naming facial expressions

Recognizing facial expressions is dependent on multiple brain networks specialized for different cognitive functions. In the current study, participants (N = 20) were scanned using functional magnetic resonance imaging (fMRI), while they performed a covert facial expression naming task. Immediately prior to scanning thetaburst transcranial magnetic stimulation (TMS) was delivered over the right lateral prefrontal cortex (PFC), or the vertex control site. A group whole-brain analysis revealed that TMS induced opposite effects in the neural responses across different brain networks. Stimulation of the right PFC (compared to stimulation of the vertex) decreased neural activity in the left lateral PFC but increased neural activity in three nodes of the default mode network (DMN): the right superior frontal gyrus, right angular gyrus and the bilateral middle cingulate gyrus. A region of interest analysis showed that TMS delivered over the right PFC reduced neural activity across all functionally localised face areas (including in the PFC) compared to TMS delivered over the vertex. These results suggest that visually recognizing facial expressions is dependent on the dynamic interaction of the face-processing network and the DMN. Our study also demonstrates the utility of combined TMS/fMRI studies for revealing the dynamic interactions between different functional brain networks.

Embracing deepfakes and AI-generated images in neuroscience research

The rise of deepfakes and AI-generated images has raised concerns regarding their potential misuse. However, this commentary highlights the valuable opportunities these technologies offer for neuroscience research. Deepfakes deliver accessible, realistic and customisable dynamic face stimuli, while generative adversarial networks (GANs) can generate and modify diverse and high-quality static content. These advancements can enhance the variability and ecological validity of research methods and enable the creation of previously unattainable stimuli. When AI-generated images are informed by brain responses, they provide unique insights into the structure and function of visual systems. The authors argue that experimental psychologists and cognitive neuroscientists stay informed about these emerging tools and embrace their potential to advance the field of visual neuroscience.

Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n = 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested-even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.SIGNIFICANCE STATEMENT Previous work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.

A systematic review of the neurobiological effects of theta-burst stimulation (TBS) as measured using functional magnetic resonance imaging (fMRI)

Theta burst stimulation (TBS) is associated with the modulation of a range of clinical, cognitive, and behavioural outcomes, but specific neurobiological effects remain somewhat unclear. This systematic literature review investigated resting-state and task-based functional magnetic resonance imaging (fMRI) outcomes post-TBS in healthy human adults. Fifty studies that applied either continuous-or intermittent-(c/i) TBS, and adopted a pretest-posttest or sham-controlled design, were included. For resting-state outcomes following stimulation applied to motor, temporal, parietal, occipital, or cerebellar regions, functional connectivity generally decreased in response to cTBS and increased in response to iTBS, though there were some exceptions to this pattern of response. These findings are mostly consistent with the assumed long-term depression (LTD)/long-term potentiation (LTP)-like plasticity effects of cTBS and iTBS, respectively. Task-related outcomes following TBS were more variable. TBS applied to the prefrontal cortex, irrespective of task or state, also produced more variable responses, with no consistent patterns emerging. Individual participant and methodological factors are likely to contribute to the variability in responses to TBS. Future studies assessing the effects of TBS via fMRI must account for factors known to affect the TBS outcomes, both at the level of individual participants and of research methodology.

Global motion processing in infants' visual cortex and the emergence of autism

Autism is a heritable and common neurodevelopmental condition, with behavioural symptoms typically emerging around age 2 to 3 years. Differences in basic perceptual processes have been documented in autistic children and adults. Specifically, data from many experiments suggest links between autism and alterations in global visual motion processing (i.e., when individual motion information is integrated to perceive an overall coherent pattern). Yet, no study has investigated whether a distinctive organization of global motion processing precede the emergence of autistic symptoms in early childhood. Here, using a validated infant electroencephalography (EEG) experimental paradigm, we first establish the normative activation profiles for global form, global motion, local form, and local motion in the visual cortex based on data from two samples of 5-month-old infants (total n = 473). Further, in a sample of 5-month-olds at elevated likelihood of autism (n = 52), we show that a different topographical organization of global motion processing is associated with autistic symptoms in toddlerhood. These findings advance the understanding of neural organization of infants' basic visual processing, and its role in the development of autism.

Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience

In face-to-face communication, humans are faced with multiple layers of discontinuous multimodal signals, such as head, face, hand gestures, speech and non-speech sounds, which need to be interpreted as coherent and unified communicative actions. This implies a fundamental computational challenge: optimally binding only signals belonging to the same communicative action while segregating signals that are not connected by the communicative content. How do we achieve such an extraordinary feat, reliably, and efficiently? To address this question, we need to further move the study of human communication beyond speech-centred perspectives and promote a multimodal approach combined with interdisciplinary cooperation. Accordingly, we seek to reconcile two explanatory frameworks recently proposed in psycholinguistics and sensory neuroscience into a neurocognitive model of multimodal face-to-face communication. First, we introduce a psycholinguistic framework that characterises face-to-face communication at three parallel processing levels: multiplex signals, multimodal gestalts and multilevel predictions. Second, we consider the recent proposal of a lateral neural visual pathway specifically dedicated to the dynamic aspects of social perception and reconceive it from a multimodal perspective ("lateral processing pathway"). Third, we reconcile the two frameworks into a neurocognitive model that proposes how multiplex signals, multimodal gestalts, and multilevel predictions may be implemented along the lateral processing pathway. Finally, we advocate a multimodal and multidisciplinary research approach, combining state-of-the-art imaging techniques, computational modelling and artificial intelligence for future empirical testing of our model.

The medial temporal lobe structure and function support positive affect

Positive affect (PA) is not only associated with individuals' psychological and physical health, but also their cognitive processes. However, whether medial temporal lobe (MTL) and its subfields' volume/functional connectivity can explain individual variability in PA remains understudied. We investigated the morphological (i.e., grey matter volume; GMV) and functional characteristics (i.e., resting-state functional connectivity; rsFC) of PA with a combination of univariate and multivariate pattern analyses (MVPA) using a large sample of participants (n = 321). We simultaneously collected the T1-weighted (n = 321), high-resolution MTL T2-weighted, and resting-state functional imaging data (n = 209). The MTL and its subfields' volumes, including the CA1, CA2+3, DG, and subiculum (SUB), perirhinal cortex (PRC), and parahippocampus (PHC), were extracted using an automatic segmentation of hippocampal subfields (ASHS) software. The morphological results revealed that GMVs in the prefrontal-occipital and limbic (i.e., hippocampus, amygdala, and PHC) systems were associated with variability in PA at the whole-brain level using MVPA but not univariate analysis. Linear regression results further revealed a positive association between the MTL subfields' GMV, especially for the right PRC, and PA after controlling for several covariates. PRC-seed-based rsFC analyses further revealed that its couplings with the fronto-parietal-occipital system predicted PA in both univariate and MVPA. These findings provide novel insights into the neuroanatomical and functional substrates underlying human PA trait. Findings also suggest critical contributions of the MTL and its subfield of the perirhinal cortex, but not hippocampal subfields, as well as its functional coupling with the fronto-parietal control-system on the formation of PA.

Representational structure of fMRI/EEG responses to dynamic facial expressions

Face perception provides an excellent example of how the brain processes nuanced visual differences and transforms them into behaviourally useful representations of identities and emotional expressions. While a body of literature has looked into the spatial and temporal neural processing of facial expressions, few studies have used a dimensionally varying set of stimuli containing subtle perceptual changes. In the current study, we used 48 short videos varying dimensionally in their intensity and category (happy, angry, surprised) of expression. We measured both fMRI and EEG responses to these video clips and compared the neural response patterns to the predictions of models based on image features and models derived from behavioural ratings of the stimuli. In fMRI, the inferior frontal gyrus face area (IFG-FA) carried information related only to the intensity of the expression, independent of image-based models. The superior temporal sulcus (STS), inferior temporal (IT) and lateral occipital (LO) areas contained information about both expression category and intensity. In the EEG, the coding of expression category and low-level image features were most pronounced at around 400 ms. The expression intensity model did not, however, correlate significantly at any EEG timepoint. Our results show a specific role for IFG-FA in the coding of expressions and suggest that it contains image and category invariant representations of expression intensity.

Measuring the response to visually presented faces in the human lateral prefrontal cortex

Neuroimaging studies identify multiple face-selective areas in the human brain. In the current study, we compared the functional response of the face area in the lateral prefrontal cortex to that of other face-selective areas. In Experiment 1, participants (n = 32) were scanned viewing videos containing faces, bodies, scenes, objects, and scrambled objects. We identified a face-selective area in the right inferior frontal gyrus (rIFG). In Experiment 2, participants (n = 24) viewed the same videos or static images. Results showed that the rIFG, right posterior superior temporal sulcus (rpSTS), and right occipital face area (rOFA) exhibited a greater response to moving than static faces. In Experiment 3, participants (n = 18) viewed face videos in the contralateral and ipsilateral visual fields. Results showed that the rIFG and rpSTS showed no visual field bias, while the rOFA and right fusiform face area (rFFA) showed a contralateral bias. These experiments suggest two conclusions; firstly, in all three experiments, the face area in the IFG was not as reliably identified as face areas in the occipitotemporal cortex. Secondly, the similarity of the response profiles in the IFG and pSTS suggests the areas may perform similar cognitive functions, a conclusion consistent with prior neuroanatomical and functional connectivity evidence.

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Decoding functional brain networks through graph measures in infancy: The case of emotional faces

Graph measures represent an optimal way to investigate neural networks' organization, yet their application is still limited in developmental samples. To uncover the organization of 7-month-old infants' functional brain networks during an emotional perception task, we combined a decoding technique (i.e., Principal Component Regression) to graph metrics computation. Nodes' Within Module Degree Z Score (WMDZ) was computed as a measure of modular organization, and we decoded networks' functional organizations across EEG alpha and theta bands in response to static and dynamic facial expressions of emotions. We found that infants' brain topological activity differentiates between static and dynamic emotional faces due to the involvement of visual streams and sensorimotor areas, as often observed in adults. Moreover, network invariances point toward an already present rudimental network structure tuned to face processing already at 7-months of age. Overall, our results affirm the fruitfulness of the application of graph measures in developmental samples, due to their flexibility and the wealth of information they provide over infants' networks functional organization.

Anatomical and neurophysiological basis of face recognition

Face recognition is a highly developed and efficient human function that involves multiple neural networks. A main pathway links the occipital cortex, where an occipital face area (OFA) has been identified, to a fusiform face area (FFA) in the fusiform gyrus, which plays a critical role in face recognition. This core pathway deals with invariant aspects of the face. Another pathway, including the superior temporal sulcus, is involved in the perception of more changeable aspects of the face such as gaze orientation, face expression and lip movements. It has been defined by some authors as a "third pathway of visual recognition", i.e. a lateral pathway in addition to the "what" and "where" pathways. It deals with sociocognitive aspects of face perception. Many other accessory functional systems are connected to the core system of visual recognition to act in concert with it: the intraparietal sulcus (for the management of spatial attention), the primary auditory cortex (prelexical perception of speech), the amygdala, the insula and the limbic system (perception of emotions), the anterior temporal pole (access to the identity of the individual, his name, biographical information), etc. Functional brain imaging has made remarkable progress in the understanding of face perception, which in the early years was limited to the description of single cases of brain-damaged patients. This progress has made it possible to better analyse the many face recognition disorders, sometimes subtle, other times confusing, observed in human pathology.

Early social adversity modulates the relation between attention biases and socioemotional behaviour in juvenile macaques

Affect-biased attention may play a fundamental role in early socioemotional development, but factors influencing its emergence and associations with typical versus pathological outcomes remain unclear. Here, we adopted a nonhuman primate model of early social adversity (ESA) to: (1) establish whether juvenile, pre-adolescent macaques demonstrate attention biases to both threatening and reward-related dynamic facial gestures; (2) examine the effects of early social experience on such biases; and (3) investigate how this relation may be linked to socioemotional behaviour. Two groups of juvenile macaques (ESA exposed and non-ESA exposed) were presented with pairs of dynamic facial gestures comprising two conditions: neutral-threat and neutral-lipsmacking. Attention biases to threat and lipsmacking were calculated as the proportion of gaze to the affective versus neutral gesture. Measures of anxiety and social engagement were also acquired from videos of the subjects in their everyday social environment. Results revealed that while both groups demonstrated an attention bias towards threatening facial gestures, a greater bias linked to anxiety was demonstrated by the ESA group only. Only the non-ESA group demonstrated a significant attention bias towards lipsmacking, and the degree of this positive bias was related to duration and frequency of social engagement in this group. These findings offer important insights into the effects of early social experience on affect-biased attention and related socioemotional behaviour in nonhuman primates, and demonstrate the utility of this model for future investigations into the neural and learning mechanisms underlying this relationship across development.

Interleukin 6 predicts increased neural response during face processing in a sample of individuals with schizophrenia and healthy participants: A functional magnetic resonance imaging study

•

IL-6 has been associated with poorer facial emotion recognition.

•

fMRI was performed during a faces task and IL-6 measured from blood samples.

•

IL-6 predicted increased neural response during facial emotion recognition.

Background

Deficits in facial emotion recognition are a core feature of schizophrenia and predictive of functional outcome. Higher plasma levels of the cytokine interleukin 6 (IL-6) have recently been associated with poorer facial emotion recognition in individuals with schizophrenia and healthy participants, but the neural mechanisms affected remain poorly understood.

Methods

Forty-nine individuals with schizophrenia or schizoaffective disorder and 158 healthy participants were imaged using functional magnetic resonance imaging during a dynamic facial emotion recognition task. Plasma IL-6 was measured from blood samples taken outside the scanner. Multiple regression was used in statistical parametric mapping software to test whether higher plasma IL-6 predicted increased neural response during task performance.

Results

Higher plasma IL-6 predicted increased bilateral medial prefrontal response during neutral face processing compared to angry face processing in the total sample (N = 207, t_max = 5.67) and increased left insula response during angry face processing compared to neutral face processing (N = 207, t_max = 4.40) (p < 0.05, family-wise error corrected across the whole brain at the cluster level).

Conclusions

These findings suggest that higher peripheral IL-6 levels predict altered neural response within brain regions involved in social cognition and emotion during facial emotion recognition. This is consistent with recent neuroimaging research on IL-6 and suggesting a possible neural mechanism by which this cytokine might affect facial emotion recognition accuracy.

The relationship between early and recent life stress and emotional expression processing: A functional connectivity study

The aim of this study was to characterize neural activation during the processing of negative facial expressions in a non-clinical group of individuals characterized by two factors: the levels of stress experienced in early life and in adulthood. Two models of stress consequences were investigated: the match/mismatch and cumulative stress models. The match/mismatch model assumes that early adversities may promote optimal coping with similar events in the future through fostering the development of coping strategies. The cumulative stress model assumes that effects of stress are additive, regardless of the timing of the stressors. Previous studies suggested that stress can have both cumulative and match/mismatch effects on brain structure and functioning and, consequently, we hypothesized that effects on brain circuitry would be found for both models. We anticipated effects on the neural circuitry of structures engaged in face perception and emotional processing. Hence, the amygdala, fusiform face area, occipital face area, and posterior superior temporal sulcus were selected as seeds for seed-based functional connectivity analyses. The interaction between early and recent stress was related to alterations during the processing of emotional expressions mainly in to the cerebellum, middle temporal gyrus, and supramarginal gyrus. For cumulative stress levels, such alterations were observed in functional connectivity to the middle temporal gyrus, lateral occipital cortex, precuneus, precentral and postcentral gyri, anterior and posterior cingulate gyri, and Heschl's gyrus. This study adds to the growing body of literature suggesting that both the cumulative and the match/mismatch hypotheses are useful in explaining the effects of stress.

Visual Information Routes in the Posterior Dorsal and Ventral Face Network Studied with Intracranial Neurophysiology and White Matter Tract Endpoints

Occipitotemporal regions within the face network process perceptual and socioemotional information, but the dynamics and information flow between different nodes of this network are still debated. Here, we analyzed intracerebral EEG from 11 epileptic patients viewing a stimulus sequence beginning with a neutral face with direct gaze. The gaze could avert or remain direct, while the emotion changed to fearful or happy. N200 field potential peak latencies indicated that face processing begins in inferior occipital cortex and proceeds anteroventrally to fusiform and inferior temporal cortices, in parallel. The superior temporal sulcus responded preferentially to gaze changes with augmented field potential amplitudes for averted versus direct gaze, and large effect sizes relative to other network regions. An overlap analysis of posterior white matter tractography endpoints (from 1066 healthy brains) relative to active intracerebral electrodes in the 11 patients showed likely involvement of both dorsal and ventral posterior white matter pathways. Overall, our data provide new insight into the timing of face and social cue processing in the occipitotemporal brain and anchor the superior temporal cortex in dynamic gaze processing.

The Human Posterior Superior Temporal Sulcus Samples Visual Space Differently From Other Face-Selective Regions

Neuroimaging studies show that ventral face-selective regions, including the fusiform face area (FFA) and occipital face area (OFA), preferentially respond to faces presented in the contralateral visual field (VF). In the current study we measured the VF response of the face-selective posterior superior temporal sulcus (pSTS). Across 3 functional magnetic resonance imaging experiments, participants viewed face videos presented in different parts of the VF. Consistent with prior results, we observed a contralateral VF bias in bilateral FFA, right OFA (rOFA), and bilateral human motion-selective area MT+. Intriguingly, this contralateral VF bias was absent in the bilateral pSTS. We then delivered transcranial magnetic stimulation (TMS) over right pSTS (rpSTS) and rOFA, while participants matched facial expressions in both hemifields. TMS delivered over the rpSTS disrupted performance in both hemifields, but TMS delivered over the rOFA disrupted performance in the contralateral hemifield only. These converging results demonstrate that the contralateral bias for faces observed in ventral face-selective areas is absent in the pSTS. This difference in VF response is consistent with face processing models proposing 2 functionally distinct pathways. It further suggests that these models should account for differences in interhemispheric connections between the face-selective areas across these 2 pathways.

Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections

Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.

Sexual Dimorphism in Hemispheric Processing of Faces in Humans: A Meta-Analysis of 817 Cases

A well-established neuroimaging literature predicts a right-sided asymmetry in the activation of face-devoted areas such as the fusiform gyrus (FG) and its resulting M/N170 response during face processing. However, the face-related response sometimes appears to be bihemispheric. A few studies have argued that bilaterality depended on the sex composition of the sample. To shed light on this matter, two meta-analyses were conducted starting from a large initial database of 250 ERP (Event-related potentials)/MEG (Magnetoencephalography) peer-reviewed scientific articles. Paper coverage was from 1985 to 2020. Thirty-four articles met the inclusion criteria of a sufficiently large and balanced sample size with strictly right-handed and healthy participants aged 18–35 years and N170 measurements in response to neutral front view faces at left and right occipito/temporal sites. The data of 817 male (n = 414) and female (n = 403) healthy adults were subjected to repeated-measures analyses of variance. The results of statistical analyses from the data of 17 independent studies (from Asia, Europe and America) seem to robustly indicate the presence of a sex difference in the way the two cerebral hemispheres process facial information in humans, with a marked right-sided asymmetry of the bioelectrical activity in males and a bilateral or left-sided activity in females.

Synchronous facial action binds dynamic facial features

We asked how dynamic facial features are perceptually grouped. To address this question, we varied the timing of mouth movements relative to eyebrow movements, while measuring the detectability of a small temporal misalignment between a pair of oscillating eyebrows-an eyebrow wave. We found eyebrow wave detection performance was worse for synchronous movements of the eyebrows and mouth. Subsequently, we found this effect was specific to stimuli presented to the right visual field, implicating the involvement of left lateralised visual speech areas. Adaptation has been used as a tool in low-level vision to establish the presence of separable visual channels. Adaptation to moving eyebrows and mouths with various relative timings reduced eyebrow wave detection but only when the adapting mouth and eyebrows moved asynchronously. Inverting the face led to a greater reduction in detection after adaptation particularly for asynchronous facial motion at test. We conclude that synchronous motion binds dynamic facial features whereas asynchronous motion releases them, allowing adaptation to impair eyebrow wave detection.

Evidence for a Third Visual Pathway Specialized for Social Perception

Existing models propose that primate visual cortex is divided into two functionally distinct pathways. The ventral pathway computes the identity of an object; the dorsal pathway computes the location of an object, and the actions related to that object. Despite remaining influential, the two visual pathways model requires revision. Both human and non-human primate studies reveal the existence of a third visual pathway on the lateral brain surface. This third pathway projects from early visual cortex, via motion-selective areas, into the superior temporal sulcus (STS). Studies demonstrating that the STS computes the actions of moving faces and bodies (e.g., expressions, eye-gaze, audio-visual integration, intention, and mood) show that the third visual pathway is specialized for the dynamic aspects of social perception.

Theta-burst TMS of lateral occipital cortex reduces BOLD responses across category-selective areas in ventral temporal cortex

Human visual cortex contains three scene-selective regions in the lateral, medial and ventral cortex, termed the occipital place area (OPA), medial place area (MPA) and parahippocampal place area (PPA). Using functional magnetic resonance imaging (fMRI), all three regions respond more strongly when viewing visual scenes compared with isolated objects or faces. To determine how these regions are functionally and causally connected, we applied transcranial magnetic stimulation to OPA and measured fMRI responses before and after stimulation, using a theta-burst paradigm (TBS). To test for stimulus category-selectivity, we presented a range of visual categories (scenes, buildings, objects, faces). To test for specificity of any effects to TBS of OPA we employed two control conditions: Sham, with no TBS stimulation, and an active TBS-control with TBS to a proximal face-selective cortical region (occipital face area, or OFA). We predicted that TBS to OPA (but not OFA) would lead to decreased responses to scenes and buildings (but not other categories) in other scene-selective cortical regions. Across both ROI and whole-volume analyses, we observed decreased responses to scenes in PPA as a result of TBS. However, these effects were neither category specific, with decreased responses to all stimulus categories, nor limited to scene-selective regions, with decreases also observed in face-selective fusiform face area (FFA). Furthermore, similar effects were observed with TBS to OFA, thus effects were not specific to the stimulation site in the lateral occipital cortex. Whilst these data are suggestive of a causal, but non-specific relationship between lateral occipital and ventral temporal cortex, we discuss several factors that could have underpinned this result, such as the differences between TBS and online TMS, the role of anatomical distance between stimulated regions and how TMS effects are operationalised. Furthermore, our findings highlight the importance of active control conditions in brain stimulation experiments to accurately assess functional and causal connectivity between specific brain regions.

Face Recognition Brain Functional Connectivity in Patients With Major Depression: A Brain Source Localization Study by ERP

Objective: Patients with major depressive disorder (MDD) presents with face recognition defects. These defects negatively affect their social interactions. However, the cause of these defects is not clear. This study sought to explore whether MDD patients develop facial perceptual processing disorders with characteristics of brain functional connectivity (FC). Methods: Event-related potential (ERP) was used to explore differences between 20 MDD patients and 20 healthy participants with face and non-face recognition tasks based on 64 EEG parameters. After pre-processing of EEG data and source reconstruction using the minimum-norm estimate (MNE), data were converted to AAL90 template to obtain a time series of 90 brain regions. EEG power spectra were determined using Fieldtrip incorporating a Fast Fourier transform. FC was determined for all pairs of brain signals for theta band using debiased estimate of weighted phase-lag index (wPLI) in Fieldtrip. To explore group differences in wPLI, independent t-tests were performed with p < 0.05 to indicate statistical significance. False discovery rate (FDR) correction was used to adjust p-values. Results: The findings showed that amplitude induction by face pictures was higher compared with that of non-face pictures both in MDD and healthy control (HC) groups. Face recognition amplitude in MDD group was lower compared with that in the HC group. Two time periods with significant differences were then selected for further analysis. Analysis showed that FC was stronger in the MDD group compared with that in the HC group in most brain regions in both periods. However, only one FC between two brain regions in HC group was stronger compared with that in the MDD group. Conclusion: Dysfunction in brain FC among MDD patients is a relatively complex phenomenon, exhibiting stronger and multiple connectivity with several brain regions of emotions. The findings of the current study indicate that the brain FC of MDD patients is more complex and less efficient in the initial stage of face recognition.

Social Perception of Faces: Brain Imaging and Subjective Ratings

The aim of this study was to investigate how a female face is perceived in terms of its attractiveness, dominance, health, femininity-masculinity, and maturity in direct relation to the body fat percentage (BFP) conveyed by the face. To compare how young adults (ages 18 to 35) respond to different levels of body fat percentage both subjectively and objectively we collected survey ratings and electroencephalography (EEG) data across five different levels of BFP from 40 participants. We adapted the experimental design from a prior behavioral study and used calibrated and morphed female face images of five different BFP levels. The results of the survey are in consensus with the previous study and assessed to be a successful replication. From the EEG data, event-related potentials (ERPs) were extracted from one electrode location (right occipitotemporal brain region) known to be particularly sensitive to face-stimuli. We found statistically significant differences in the amplitudes of the P200 component (194 ms post stimulus onset) between the thickest face and all four other BFP conditions, and in the amplitudes of the N300 component (274 ms post stimulus onset) between the average face and three other BFP conditions. As expected, there were no significant differences among the N170 amplitudes of all five BFP conditions since this ERP component simply reflects the processing of faces in general. From these results, we can infer that holistic face encoding characterized by the N170 component in the right occipitotemporal area is followed by serial evaluative processes, whose categorical and qualitative matrix and spatiotemporal dynamics should be further explored in future studies, especially in relation to the social constructs that were focused on in this study.

Transcranial Magnetic Stimulation and the Understanding of Behavior

The development of the use of transcranial magnetic stimulation (TMS) in the study of psychological functions has entered a new phase of sophistication. This is largely due to an increasing physiological knowledge of its effects and to its being used in combination with other experimental techniques. This review presents the current state of our understanding of the mechanisms of TMS in the context of designing and interpreting psychological experiments. We discuss the major conceptual advances in behavioral studies using TMS. There are meaningful physiological and technical achievements to review, as well as a wealth of new perceptual and cognitive experiments. In doing so we summarize the different uses and challenges of TMS in mental chronometry, perception, awareness, learning, and memory.

Theta-burst TMS to the posterior superior temporal sulcus decreases resting-state fMRI connectivity across the face processing network

Humans process faces by using a network of face-selective regions distributed across the brain. Neuropsychological patient studies demonstrate that focal damage to nodes in this network can impair face recognition, but such patients are rare. We approximated the effects of damage to the face network in neurologically normal human participants by using theta burst transcranial magnetic stimulation (TBS). Multi-echo functional magnetic resonance imaging (fMRI) resting-state data were collected pre- and post-TBS delivery over the face-selective right superior temporal sulcus (rpSTS), or a control site in the right motor cortex. Results showed that TBS delivered over the rpSTS reduced resting-state connectivity across the extended face processing network. This connectivity reduction was observed not only between the rpSTS and other face-selective areas, but also between nonstimulated face-selective areas across the ventral, medial, and lateral brain surfaces (e.g., between the right amygdala and bilateral fusiform face areas and occipital face areas). TBS delivered over the motor cortex did not produce significant changes in resting-state connectivity across the face processing network. These results demonstrate that, even without task-induced fMRI signal changes, disrupting a single node in a brain network can decrease the functional connectivity between nodes in that network that have not been directly stimulated.

Human behavior is dependent on brain networks that perform different cognitive functions. We combined theta burst transcranial magnetic stimulation (TBS) with resting-state fMRI to study the face processing network. Disruption of the face-selective right posterior superior temporal sulcus (rpSTS) reduced fMRI connectivity across the face network. This impairment in connectivity was observed not only between the rpSTS and other face-selective areas, but also between nonstimulated face-selective areas on the ventral and medial brain surfaces (e.g., between the right amygdala and bilateral fusiform face areas and occipital face areas). Thus, combined TBS/fMRI can be used to approximate and measure the effects of focal brain damage on brain networks, and suggests such an approach may be useful for mapping intrinsic network organization.

Non-invasive stimulation in the social brain: the methodological challenges

Use of non-invasive brain stimulation methods (NIBS) has become a common approach to study social processing in addition to behavioural, imaging and lesion studies. However, research using NIBS to investigate social processing faces challenges. Overcoming these is important to allow valid and reliable interpretation of findings in neurotypical cohorts, but also to allow us to tailor NIBS protocols to atypical groups with social difficulties. In this review, we consider the utility of brain stimulation as a technique to study and modulate social processing. We also discuss challenges that face researchers using NIBS to study social processing in neurotypical adults with a view to highlighting potential solutions. Finally, we discuss additional challenges that face researchers using NIBS to study and modulate social processing in atypical groups. These are important to consider given that NIBS protocols are rarely tailored to atypical groups before use. Instead, many rely on protocols designed for neurotypical adults despite differences in brain function that are likely to impact response to NIBS.

Recognizing Genuine From Posed Facial Expressions: Exploring the Role of Dynamic Information and Face Familiarity

The accurate recognition of emotion is important for interpersonal interaction and when navigating our social world. However, not all facial displays reflect the emotional experience currently being felt by the expresser. Indeed, faces express both genuine and posed displays of emotion. In this article, we summarize the importance of motion for the recognition of face identity before critically outlining the role of dynamic information in determining facial expressions and distinguishing between genuine and posed expressions of emotion. We propose that both dynamic information and face familiarity may modulate our ability to determine whether an expression is genuine or not. Finally, we consider the shared role for dynamic information across different face recognition tasks and the wider impact of face familiarity on determining genuine from posed expressions during real-world interactions.

Brain Networks Processing Temporal Information in Dynamic Facial Expressions

This fMRI study examines the role of local and global motion information in facial movements during exposure to novel dynamic face stimuli. We found that synchronous expressions distinctively engaged medial prefrontal areas in the rostral and caudal sectors of anterior cingulate cortex (r/cACC) extending to inferior supplementary motor areas, as well as motor cortex and bilateral superior frontal gyrus (global temporal-spatial processing). Asynchronous expressions in which one part of the face unfolded before the other activated more the right superior temporal sulcus (STS) and inferior frontal gyrus (local temporal-spatial processing). These differences in temporal dynamics had no effect on visual face-responsive areas. Dynamic causal modeling analysis further showed that processing of asynchronous expression features was associated with a differential information flow, centered on STS, which received direct input from occipital cortex and projected to the amygdala. Moreover, STS and amygdala displayed selective interactions with cACC where the integration of both local and global motion cues could take place. These results provide new evidence for a role of local and global temporal dynamics in emotional expressions, extracted in partly separate brain pathways. Importantly, we show that dynamic expressions with synchronous movement cues may distinctively engage brain areas responsible for motor execution of expressions.

Connectivity at the origins of domain specificity in the cortical face and place networks

It is well established that the adult brain contains a mosaic of domain-specific networks. But how do these domain-specific networks develop? Here we tested the hypothesis that the brain comes prewired with connections that precede the development of domain-specific function. Using resting-state fMRI in the youngest sample of newborn humans tested to date, we indeed found that cortical networks that will later develop strong face selectivity (including the "proto" occipital face area and fusiform face area) and scene selectivity (including the "proto" parahippocampal place area and retrosplenial complex) by adulthood, already show domain-specific patterns of functional connectivity as early as 27 d of age (beginning as early as 6 d of age). Furthermore, we asked how these networks are functionally connected to early visual cortex and found that the proto face network shows biased functional connectivity with foveal V1, while the proto scene network shows biased functional connectivity with peripheral V1. Given that faces are almost always experienced at the fovea, while scenes always extend across the entire periphery, these differential inputs may serve to facilitate domain-specific processing in each network after that function develops, or even guide the development of domain-specific function in each network in the first place. Taken together, these findings reveal domain-specific and eccentricity-biased connectivity in the earliest days of life, placing new constraints on our understanding of the origins of domain-specific cortical networks.

Dorsal-movement and ventral-form regions are functionally connected during visual-speech recognition

Faces convey social information such as emotion and speech. Facial emotion processing is supported via interactions between dorsal-movement and ventral-form visual cortex regions. Here, we explored, for the first time, whether similar dorsal-ventral interactions (assessed via functional connectivity), might also exist for visual-speech processing. We then examined whether altered dorsal-ventral connectivity is observed in adults with high-functioning autism spectrum disorder (ASD), a disorder associated with impaired visual-speech recognition. We acquired functional magnetic resonance imaging (fMRI) data with concurrent eye tracking in pairwise matched control and ASD participants. In both groups, dorsal-movement regions in the visual motion area 5 (V5/MT) and the temporal visual speech area (TVSA) were functionally connected to ventral-form regions (i.e., the occipital face area [OFA] and the fusiform face area [FFA]) during the recognition of visual speech, in contrast to the recognition of face identity. Notably, parts of this functional connectivity were decreased in the ASD group compared to the controls (i.e., right V5/MT-right OFA, left TVSA-left FFA). The results confirmed our hypothesis that functional connectivity between dorsal-movement and ventral-form regions exists during visual-speech processing. Its partial dysfunction in ASD might contribute to difficulties in the recognition of dynamic face information relevant for successful face-to-face communication.

COMT-Polymorphisms Modulated Functional Profile of the Fusiform Face Area Contributes to Face-Specific Recognition Ability

Previous studies have shown that face-specific recognition ability (FRA) is heritable; however, the neural basis of this heritability is unclear. Candidate gene studies have suggested that the catechol-O-methyltransferase (COMT) rs4680 polymorphism is related to face perception. Here, using a partial least squares (PLS) method, we examined the multivariate association between 12 genotypes of 4 COMT polymorphisms (rs6269-rs4633-rs4818-rs4680) and multimodal MRI phenotypes in the human fusiform face area (FFA), which selectively responds to face stimuli, in 338 Han Chinese adults (mean age 20.45 years; 135 males). The MRI phenotypes included gray matter volume (GMV), resting-state fractional amplitude of low-frequency fluctuations (fALFF), and face-selective blood-oxygen-level-dependent (BOLD) responses (FS). We found that the first COMT-variant component (PLS1) was positively associated with the FS but negatively associated with the fALFF in the FFA. Moreover, participants with the COMT heterozygous-HEA-haplotype showed higher PLS1 FFA-MRI scores, which were positively associated with the FRA in an old/new face recognition task, than those with the COMT homozygous HEA haplotype and HEA non-carriers, suggesting that individuals with an appropriate (intermediate) level of dopamine activity in the FFA might have better FRA. In summary, our study provides empirical evidence for the genetic and neural basis for the heritability of face recognition and informs the formation of neural module functional specificity.

The bottom-up and top-down processing of faces in the human occipitotemporal cortex

Although face processing has been studied extensively, the dynamics of how face-selective cortical areas are engaged remains unclear. Here, we uncovered the timing of activation in core face-selective regions using functional Magnetic Resonance Imaging and Magnetoencephalography in humans. Processing of normal faces started in the posterior occipital areas and then proceeded to anterior regions. This bottom-up processing sequence was also observed even when internal facial features were misarranged. However, processing of two-tone Mooney faces lacking explicit prototypical facial features engaged top-down projection from the right posterior fusiform face area to right occipital face area. Further, face-specific responses elicited by contextual cues alone emerged simultaneously in the right ventral face-selective regions, suggesting parallel contextual facilitation. Together, our findings chronicle the precise timing of bottom-up, top-down, as well as context-facilitated processing sequences in the occipital-temporal face network, highlighting the importance of the top-down operations especially when faced with incomplete or ambiguous input.

MRI-based visualization of rTMS-induced cortical plasticity in the primary motor cortex

Repetitive transcranial magnetic stimulation (rTMS) induces changes in cortical excitability for minutes to hours after the end of intervention. However, it has not been precisely determined to what extent cortical plasticity prevails spatially in the cortex. Recent studies have shown that rTMS induces changes in “interhemispheric” functional connectivity, the resting-state functional connectivity between the stimulated region and the symmetrically corresponding region in the contralateral hemisphere. In the present study, quadripulse stimulation (QPS) was applied to the index finger representation in the left primary motor cortex (M1), while the position of the stimulation coil was constantly monitored by an online navigator. After QPS application, resting-state functional magnetic resonance imaging was performed, and the interhemispheric functional connectivity was compared with that before QPS. A cluster of connectivity changes was observed in the stimulated region in the central sulcus. The cluster was spatially extended approximately 10 mm from the center [half width at half maximum (HWHM): approximately 3 mm] and was extended approximately 20 mm long in depth (HWHM: approximately 7 mm). A localizer scan of the index finger motion confirmed that the cluster of interhemispheric connectivity changes overlapped spatially with the activation related to the index finger motion. These results indicate that cortical plasticity in M1 induced by rTMS was relatively restricted in space and suggest that rTMS can reveal functional dissociation associated with adjacent small areas by inducing neural plasticity in restricted cortical regions.

Action at a distance on object-related ventral temporal representations

The representation of objects in ventral temporal cortex is relatively resilient to transformations in the stimuli. There is emerging recognition that ventral temporal object representations are forged via interactions among a broader network of regions that receive independent inputs about a stimulus. Here we test whether ventral temporal representations are causally modulated by disrupting processing in distal associative areas. We used transcranial direct current stimulation (tDCS) to stimulate left parietal areas and functional Magnetic Resonance Imaging (fMRI) to measure object-related neural responses in the ventral stream. We find that representational geometries and category discriminability within ventral temporal cortex, as well as functional connectivity between ventral temporal and parietal areas, are enhanced by anodal compared to cathodal stimulation of left parietal associative cortex. These results demonstrate that ventral temporal representations can be causally modulated by processing distal to the ventral stream.

A functional dissociation of face-, body- and scene-selective brain areas based on their response to moving and static stimuli

The human brain contains areas that respond selectively to faces, bodies and scenes. Neuroimaging studies have shown that a subset of these areas preferentially respond more to moving than static stimuli, but the reasons for this functional dissociation remain unclear. In the present study, we simultaneously mapped the responses to motion in face-, body- and scene-selective areas in the right hemisphere using moving and static stimuli. Participants (N = 22) were scanned using functional magnetic resonance imaging (fMRI) while viewing videos containing bodies, faces, objects, scenes or scrambled objects, and static pictures from the beginning, middle and end of each video. Results demonstrated that lateral areas, including face-selective areas in the posterior and anterior superior temporal sulcus (STS), the extrastriate body area (EBA) and the occipital place area (OPA) responded more to moving than static stimuli. By contrast, there was no difference between the response to moving and static stimuli in ventral and medial category-selective areas, including the fusiform face area (FFA), occipital face area (OFA), amygdala, fusiform body area (FBA), retrosplenial complex (RSC) and parahippocampal place area (PPA). This functional dissociation between lateral and ventral/medial brain areas that respond selectively to different visual categories suggests that face-, body- and scene-selective networks may be functionally organized along a common dimension.

The neural sources of N170: Understanding timing of activation in face-selective areas

The N170 ERP component has been widely identified as a face-sensitive neural marker. Despite extensive investigations conducted to examine the neural sources of N170, there are two issues in prior literature: (a) few studies used individualized anatomy as head model for the cortical source analysis of the N170, and (b) the relationship between the N170 and face-selective regions from fMRI studies is unclear. Here, we addressed these questions by presenting pictures of faces and houses to the same group of healthy adults and recording structural MRI, fMRI, and high-density ERPs in separate sessions. Source analysis based on the participant's anatomy showed that the middle and posterior fusiform gyri were the primary neural sources for the face-sensitive aspects of the N170. Source analysis based on regions of interest from the fMRI revealed that the fMRI-defined fusiform face area was the major contributor to the N170. The current study suggests that the fusiform gyrus is a major neural contributor to the N170 ERP component and provides further insights about the spatiotemporal characteristics of face processing.