Anterior superior temporal sulcus is specialized for non-rigid facial motion in both monkeys and humans

The Role of Occipitotemporal Network for Speed-Reading: An fMRI Study

The activity of occipitotemporal regions involved in linguistic reading processes, such as the ventral occipitotemporal cortex (vOT), is believed to exhibit strong interactions during higher-order language processing, specifically in the connectivity between the occipital gyrus and the temporal gyrus. In this study, we utilized functional magnetic resonance imaging (fMRI) with psychophysiological interaction (PPI) and dynamic causal modeling (DCM) to investigate the functional and effective connectivity in the occipitotemporal network during speed reading. We conducted the experiment with native Japanese speakers who underwent and without speed-reading training and subsequently performed established reading tasks at different speeds (slow, medium, and fast) while undergoing 3-Tesla Siemens fMRI. Our activation analyses revealed significant changes in occipital and temporal regions as reading speed increased, indicating functional connectivity within the occipitotemporal network. DCM results further demonstrated more intricate effective connections and high involvement within the occipitotemporal pathway: (1) reading signals originated from the inferior occipital gyrus (iO), distributed to the vOT and the posterior superior temporal sulcus (pSTS), and then gathered in the anterior superior temporal sulcus (aSTS); (2) reading speed loads had modulation effects on the pathways from the aSTS to vOT and from the iO to vOT. These findings highlight the complex connectivity and dynamic interactions within the occipitotemporal network during speed-reading processes.

The role of preSMA and STS in face recognition: A transcranial magnetic stimulation (TMS) study

Current models propose that facial recognition is mediated by two independent yet interacting anatomo-functional systems: one processing facial features mainly mediated by the Fusiform Face Area and the other involved in the extraction of dynamic information from faces, subserved by Superior Temporal Sulcus (STS). Also, the pre-Supplementary Motor Area (pre-SMA) is implicated in facial expression processing as it is involved in its motor mimicry. However, the literature only shows evidence of the implication of STS and preSMA for facial expression recognition, without relating it to face recognition. In addition, the literature shows a facilitatory role of facial motion in the recognition of unfamiliar faces, particularly for poor recognizers. The present study aimed at studying the role of STS and preSMA in unfamiliar face recognition in people with different face recognition skills. 34 healthy participants received repetitive transcranial magnetic stimulation over the right posterior STS, pre-SMA and as sham during a task of matching of faces encoded through: facial expression, rigid head movement or as static (i.e., absence of any facial or head motion). All faces were represented without emotional content. Results indicate that STS has a direct role in recognizing identities through rigid head movement and an indirect role in facial expression processing. This dissociation represents a step forward with respect to current face processing models suggesting that different types of motion involve separate brain and cognitive processes. PreSMA interacts with face recognition skills, increasing the performance of poor recognizers and decreasing that of good recognizers in all presentation conditions. Together, the results suggest the use of at least partially different mechanisms for face recognition in poor and good recognizers and a different role of STS and preSMA in face recognition.

Beyond faces: the contribution of the amygdala to visual processing in the macaque brain

The amygdala is present in a diverse range of vertebrate species, such as lizards, rodents, and primates; however, its structure and connectivity differs across species. The increased connections to visual sensory areas in primate species suggests that understanding the visual selectivity of the amygdala in detail is critical to revealing the principles underlying its function in primate cognition. Therefore, we designed a high-resolution, contrast-agent enhanced, event-related fMRI experiment, and scanned 3 adult rhesus macaques, while they viewed 96 naturalistic stimuli. Half of these stimuli were social (defined by the presence of a conspecific), the other half were nonsocial. We also nested manipulations of emotional valence (positive, neutral, and negative) and visual category (faces, nonfaces, animate, and inanimate) within the stimulus set. The results reveal widespread effects of emotional valence, with the amygdala responding more on average to inanimate objects and animals than faces, bodies, or social agents in this experimental context. These findings suggest that the amygdala makes a contribution to primate vision that goes beyond an auxiliary role in face or social perception. Furthermore, the results highlight the importance of stimulus selection and experimental design when probing the function of the amygdala and other visually responsive brain regions.

Inability to move one's face dampens facial expression perception

Humans rely heavily on facial expressions for social communication to convey their thoughts and emotions and to understand them in others. One prominent but controversial view is that humans learn to recognize the significance of facial expressions by mimicking the expressions of others. This view predicts that an inability to make facial expressions (e.g., facial paralysis) would result in reduced perceptual sensitivity to others' facial expressions. To test this hypothesis, we developed a diverse battery of sensitive emotion recognition tasks to characterize expression perception in individuals with Moebius Syndrome (MBS), a congenital neurological disorder that causes facial palsy. Using computer-based detection tasks we systematically assessed expression perception thresholds for static and dynamic face and body expressions. We found that while MBS individuals were able to perform challenging perceptual control tasks and body expression tasks, they were less efficient at extracting emotion from facial expressions, compared to matched controls. Exploratory analyses of fMRI data from a small group of MBS participants suggested potentially reduced engagement of the amygdala in MBS participants during expression processing relative to matched controls. Collectively, these results suggest a role for facial mimicry and consequent facial feedback and motor experience in the perception of others' facial expressions.

Equivalent processing of facial expression and identity by macaque visual system and task-optimized neural network

Both the primate visual system and artificial deep neural network (DNN) models show an extraordinary ability to simultaneously classify facial expression and identity. However, the neural computations underlying the two systems are unclear. Here, we developed a multi-task DNN model that optimally classified both monkey facial expressions and identities. By comparing the fMRI neural representations of the macaque visual cortex with the best-performing DNN model, we found that both systems: (1) share initial stages for processing low-level face features which segregate into separate branches at later stages for processing facial expression and identity respectively, and (2) gain more specificity for the processing of either facial expression or identity as one progresses along each branch towards higher stages. Correspondence analysis between the DNN and monkey visual areas revealed that the amygdala and anterior fundus face patch (AF) matched well with later layers of the DNN's facial expression branch, while the anterior medial face patch (AM) matched well with later layers of the DNN's facial identity branch. Our results highlight the anatomical and functional similarities between macaque visual system and DNN model, suggesting a common mechanism between the two systems.

The contribution of dynamics to macaque body and face patch responses

Previous functional imaging studies demonstrated body-selective patches in the primate visual temporal cortex, comparing activations to static bodies and static images of other categories. However, the use of static instead of dynamic displays of moving bodies may have underestimated the extent of the body patch network. Indeed, body dynamics provide information about action and emotion and may be processed in patches not activated by static images. Thus, to map with fMRI the full extent of the macaque body patch system in the visual temporal cortex, we employed dynamic displays of natural-acting monkey bodies, dynamic monkey faces, objects, and scrambled versions of these videos, all presented during fixation. We found nine body patches in the visual temporal cortex, starting posteriorly in the superior temporal sulcus (STS) and ending anteriorly in the temporal pole. Unlike for static images, body patches were present consistently in both the lower and upper banks of the STS. Overall, body patches showed a higher activation by dynamic displays than by matched static images, which, for identical stimulus displays, was less the case for the neighboring face patches. These data provide the groundwork for future single-unit recording studies to reveal the spatiotemporal features the neurons of these body patches encode. These fMRI findings suggest that dynamics have a stronger contribution to population responses in body than face patches.

A broadly tuned network for affective body language in the macaque brain

Body language is a powerful tool that we use to communicate how we feel, but it is unclear whether other primates also communicate in this way. Here, we use functional magnetic resonance imaging to show that the body-selective patches in macaques are activated by affective body language. Unexpectedly, we found these regions to be tolerant of naturalistic variation in posture as well as species; the bodies of macaques, humans, and domestic cats all evoked a stronger response when they conveyed fear than when they conveyed no affect. Multivariate analyses confirmed that the neural representation of fear-related body expressions was species-invariant. Collectively, these findings demonstrate that, like humans, macaques have body-selective brain regions in the ventral visual pathway for processing affective body language. These data also indicate that representations of body stimuli in these regions are built on the basis of emergent properties, such as socio-affective meaning, and not just putative image properties.

Clutter Substantially Reduces Selectivity for Peripheral Faces in the Macaque Brain

According to a prominent view in neuroscience, visual stimuli are coded by discrete cortical networks that respond preferentially to specific categories, such as faces or objects. However, it remains unclear how these category-selective networks respond when viewing conditions are cluttered, i.e., when there is more than one stimulus in the visual field. Here, we asked three questions: (1) Does clutter reduce the response and selectivity for faces as a function of retinal location? (2) Is the preferential response to faces uniform across the visual field? And (3) Does the ventral visual pathway encode information about the location of cluttered faces? We used fMRI to measure the response of the face-selective network in awake, fixating macaques (two female, five male). Across a series of four experiments, we manipulated the presence and absence of clutter, as well as the location of the faces relative to the fovea. We found that clutter reduces the response to peripheral faces. When presented in isolation, without clutter, the selectivity for faces is fairly uniform across the visual field, but, when clutter is present, there is a marked decrease in the selectivity for peripheral faces. We also found no evidence of a contralateral visual field bias when faces were presented in clutter. Nonetheless, multivariate analyses revealed that the location of cluttered faces could be decoded from the multivoxel response of the face-selective network. Collectively, these findings demonstrate that clutter blunts the selectivity of the face-selective network to peripheral faces, although information about their retinal location is retained.SIGNIFICANCE STATEMENT Numerous studies that have measured brain activity in macaques have found visual regions that respond preferentially to faces. Although these regions are thought to be essential for social behavior, their responses have typically been measured while faces were presented in isolation, a situation atypical of the real world. How do these regions respond when faces are presented with other stimuli? We report that, when clutter is present, the preferential response to foveated faces is spared but preferential response to peripheral faces is reduced. Our results indicate that the presence of clutter changes the response of the face-selective network.

Representation of expression and identity by ventral prefrontal neurons

Evidence has suggested that the ventrolateral prefrontal cortex (VLPFC) processes social stimuli, including faces and vocalizations, which are essential for communication. Features embedded within audiovisual stimuli, including emotional expression and caller identity, provide abundant information about an individual's intention, emotional state, motivation, and social status, which are important to encode in a social exchange. However, it is unknown to what extent the VLPFC encodes such features. To investigate the role of VLPFC during social communication, we recorded single-unit activity while rhesus macaques (Macaca mulatta) performed a nonmatch-to-sample task using species-specific face-vocalization stimuli that differed in emotional expression or caller identity. 75% of recorded cells were task-related and of these >70% were responsive during the nonmatch period. A larger proportion of nonmatch cells encoded the stimulus rather than the context of the trial type. A subset of responsive neurons were most commonly modulated by the identity of the nonmatch stimulus and less by the emotional expression, or both features within the face-vocalization stimuli presented during the nonmatch period. Neurons encoding identity were found in VLPFC across a broader region than expression related cells which were confined to only the anterolateral portion of the recording chamber in VLPFC. These findings suggest that, within a working memory paradigm, VLPFC processes features of face and vocal stimuli, such as emotional expression and identity, in addition to task and contextual information. Thus, stimulus and contextual information may be integrated by VLPFC during social communication.

Faces and Voices Processing in Human and Primate Brains: Rhythmic and Multimodal Mechanisms Underlying the Evolution and Development of Speech

While influential works since the 1970s have widely assumed that imitation is an innate skill in both human and non-human primate neonates, recent empirical studies and meta-analyses have challenged this view, indicating other forms of reward-based learning as relevant factors in the development of social behavior. The visual input translation into matching motor output that underlies imitation abilities instead seems to develop along with social interactions and sensorimotor experience during infancy and childhood. Recently, a new visual stream has been identified in both human and non-human primate brains, updating the dual visual stream model. This third pathway is thought to be specialized for dynamics aspects of social perceptions such as eye-gaze, facial expression and crucially for audio-visual integration of speech. Here, we review empirical studies addressing an understudied but crucial aspect of speech and communication, namely the processing of visual orofacial cues (i.e., the perception of a speaker's lips and tongue movements) and its integration with vocal auditory cues. Along this review, we offer new insights from our understanding of speech as the product of evolution and development of a rhythmic and multimodal organization of sensorimotor brain networks, supporting volitional motor control of the upper vocal tract and audio-visual voices-faces integration.

Using Live and Video Stimuli to Localize Face and Object Processing Regions of the Canine Brain

Simple Summary

We showed dogs and humans live-action stimuli (actors and objects) and videos of the same stimuli during fMRI to measure the equivalency of live and two-dimensional stimuli in the dog’s brain. We found that video stimuli were effective in defining face and object regions. However, the human fusiform face area and posterior superior temporal sulcus, and the analogous area in the dog brain, appeared to respond preferentially to live stimuli. In object regions, there was not a significantly different response between live and video stimuli.

Abstract

Previous research to localize face areas in dogs’ brains has generally relied on static images or videos. However, most dogs do not naturally engage with two-dimensional images, raising the question of whether dogs perceive such images as representations of real faces and objects. To measure the equivalency of live and two-dimensional stimuli in the dog’s brain, during functional magnetic resonance imaging (fMRI) we presented dogs and humans with live-action stimuli (actors and objects) as well as videos of the same actors and objects. The dogs (n = 7) and humans (n = 5) were presented with 20 s blocks of faces and objects in random order. In dogs, we found significant areas of increased activation in the putative dog face area, and in humans, we found significant areas of increased activation in the fusiform face area to both live and video stimuli. In both dogs and humans, we found areas of significant activation in the posterior superior temporal sulcus (ectosylvian fissure in dogs) and the lateral occipital complex (entolateral gyrus in dogs) to both live and video stimuli. Of these regions of interest, only the area along the ectosylvian fissure in dogs showed significantly more activation to live faces than to video faces, whereas, in humans, both the fusiform face area and posterior superior temporal sulcus responded significantly more to live conditions than video conditions. However, using the video conditions alone, we were able to localize all regions of interest in both dogs and humans. Therefore, videos can be used to localize these regions of interest, though live conditions may be more salient.

Face processing in the temporal lobe

Face perception is a socially important but complex process with many stages and many facets. There is substantial evidence from many sources that it involves a large extent of the temporal lobe, from the ventral occipitotemporal cortex and superior temporal sulci to anterior temporal regions. While early human neuroimaging work suggested a core face network consisting of the occipital face area, fusiform face area, and posterior superior temporal sulcus, studies in both humans and monkeys show a system of face patches stretching from posterior to anterior in both the superior temporal sulcus and inferotemporal cortex. Sophisticated techniques such as fMRI adaptation have shown that these face-activated regions show responses that have many of the attributes of human face processing. Lesions of some of these regions in humans lead to variants of prosopagnosia, the inability to recognize the identity of a face. Lesion, imaging, and electrophysiologic data all suggest that there is a segregation between identity and expression processing, though some suggest this may be better characterized as a distinction between static and dynamic facial information.

Novel Cognitive Functions Arise at the Convergence of Macroscale Gradients

Functions in higher-order brain regions are the source of extensive debate. Although past trends have been to describe the brain-especially posterior cortical areas-in terms of a set of functional modules, a new emerging paradigm focuses on the integration of proximal functions. In this review, we synthesize emerging evidence that a variety of novel functions in the higher-order brain regions are due to convergence: convergence of macroscale gradients brings feature-rich representations into close proximity, presenting an opportunity for novel functions to arise. Using the TPJ as an example, we demonstrate that convergence is enabled via three properties of the brain: (1) hierarchical organization, (2) abstraction, and (3) equidistance. As gradients travel from primary sensory cortices to higher-order brain regions, information becomes abstracted and hierarchical, and eventually, gradients meet at a point maximally and equally distant from their sensory origins. This convergence, which produces multifaceted combinations, such as mentalizing another person's thought or projecting into a future space, parallels evolutionary and developmental characteristics in such regions, resulting in new cognitive and affective faculties.

One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system

Faces and bodies are often treated as distinct categories that are processed separately by face- and body-selective brain regions in the primate visual system. These regions occupy distinct regions of visual cortex and are often thought to constitute independent functional networks. Yet faces and bodies are part of the same object and their presence inevitably covary in naturalistic settings. Here, we re-evaluate both the evidence supporting the independent processing of faces and bodies and the organizational principles that have been invoked to explain this distinction. We outline four hypotheses ranging from completely separate networks to a single network supporting the perception of whole people or animals. The current evidence, especially in humans, is compatible with all of these hypotheses, making it presently unclear how the representation of faces and bodies is organized in the cortex.

Interspecies activation correlations reveal functional correspondences between marmoset and human brain areas

The common marmoset has enormous promise as a nonhuman primate model of human brain functions. While resting-state functional MRI (fMRI) has provided evidence for a similar organization of marmoset and human cortices, the technique cannot be used to map the functional correspondences of brain regions between species. This limitation can be overcome by movie-driven fMRI (md-fMRI), which has become a popular tool for noninvasively mapping the neural patterns generated by rich and naturalistic stimulation. Here, we used md-fMRI in marmosets and humans to identify whole-brain functional correspondences between the two primate species. In particular, we describe functional correlates for the well-known human face, body, and scene patches in marmosets. We find that these networks have a similar organization in both species, suggesting a largely conserved organization of higher-order visual areas between New World marmoset monkeys and humans. However, while face patches in humans and marmosets were activated by marmoset faces, only human face patches responded to the faces of other animals. Together, the results demonstrate that higher-order visual processing might be a conserved feature between humans and New World marmoset monkeys but that small, potentially important functional differences exist.

CA, Klingner CM. The Right Hemisphere Is Responsible for the Greatest Differences in Human Brain Response to High-Arousing Emotional versus Neutral Stimuli: A MEG Study

Studies investigating human brain response to emotional stimuli-particularly high-arousing versus neutral stimuli-have obtained inconsistent results. The present study was the first to combine magnetoencephalography (MEG) with the bootstrapping method to examine the whole brain and identify the cortical regions involved in this differential response. Seventeen healthy participants (11 females, aged 19 to 33 years; mean age, 26.9 years) were presented with high-arousing emotional (pleasant and unpleasant) and neutral pictures, and their brain responses were measured using MEG. When random resampling bootstrapping was performed for each participant, the greatest differences between high-arousing emotional and neutral stimuli during M300 (270-320 ms) were found to occur in the right temporo-parietal region. This finding was observed in response to both pleasant and unpleasant stimuli. The results, which may be more robust than previous studies because of bootstrapping and examination of the whole brain, reinforce the essential role of the right hemisphere in emotion processing.

Evidence for a Third Visual Pathway Specialized for Social Perception

Existing models propose that primate visual cortex is divided into two functionally distinct pathways. The ventral pathway computes the identity of an object; the dorsal pathway computes the location of an object, and the actions related to that object. Despite remaining influential, the two visual pathways model requires revision. Both human and non-human primate studies reveal the existence of a third visual pathway on the lateral brain surface. This third pathway projects from early visual cortex, via motion-selective areas, into the superior temporal sulcus (STS). Studies demonstrating that the STS computes the actions of moving faces and bodies (e.g., expressions, eye-gaze, audio-visual integration, intention, and mood) show that the third visual pathway is specialized for the dynamic aspects of social perception.

Using FACS to trace the neural specializations underlying the recognition of facial expressions: A commentary on Waller et al. (2020)

In the recent review by Waller et al. (2020) the authors discuss how the Facial Action Coding System (FACS) can be used to study the evolution of facial behaviors. This is a timely and thought-provoking review which highlights the numerous ways in which FACS could be used to compare the mechanisms responsible for the production of facial behaviors across species. We propose that FACS could also be used to study the recognition of facial behaviors in nonhuman subjects where one of the key challenges is finding suitable stimuli that convey different emotions. By using FACS-rated images in awake neuroimaging experiments, researchers could accurately identify the brain mechanisms responsible for recognizing expressions across mammalian species. This approach would reveal neural homologs and deepen our understanding of how nonverbal social communication has evolved.

Parallel Processing of Facial Expression and Head Orientation in the Macaque Brain

When we move the features of our face, or turn our head, we communicate changes in our internal state to the people around us. How this information is encoded and used by an observer's brain is poorly understood. We investigated this issue using a functional MRI adaptation paradigm in awake male macaques. Among face-selective patches of the superior temporal sulcus (STS), we found a double dissociation of areas processing facial expression and those processing head orientation. The face-selective patches in the STS fundus were most sensitive to facial expression, as was the amygdala, whereas those on the lower, lateral edge of the sulcus were most sensitive to head orientation. The results of this study reveal a new dimension of functional organization, with face-selective patches segregating within the STS. The findings thus force a rethinking of the role of the face-processing system in representing subject-directed actions and supporting social cognition.SIGNIFICANCE STATEMENT When we are interacting with another person, we make inferences about their emotional state based on visual signals. For example, when a person's facial expression changes, we are given information about their feelings. While primates are thought to have specialized cortical mechanisms for analyzing the identity of faces, less is known about how these mechanisms unpack transient signals, like expression, that can change from one moment to the next. Here, using an fMRI adaptation paradigm, we demonstrate that while the identity of a face is held constant, there are separate mechanisms in the macaque brain for processing transient changes in the face's expression and orientation. These findings shed new light on the function of the face-processing system during social exchanges.