Fine-grained stimulus representations in body selective areas of human occipito-temporal cortex

Neural Basis of Categorical Representations of Animal Body Silhouettes

Neural activities differentiating bodies versus non-body stimuli have been identified in the occipitotemporal cortex of both humans and nonhuman primates. However, the neural mechanisms of coding the similarity of different individuals' bodies of the same species to support their categorical representations remain unclear. Using electroencephalography (EEG) and magnetoencephalography (MEG), we investigated the temporal and spatial characteristics of neural processes shared by different individual body silhouettes of the same species by quantifying the repetition suppression of neural responses to human and animal (chimpanzee, dog, and bird) body silhouettes showing different postures. Our EEG results revealed significant repetition suppression of the amplitudes of early frontal/central activity at 180-220 ms (P2) and late occipitoparietal activity at 220-320 ms (P270) in response to animal (but not human) body silhouettes of the same species. Our MEG results further localized the repetition suppression effect related to animal body silhouettes in the left supramarginal gyrus and left frontal cortex at 200-440 ms after stimulus onset. Our findings suggest two neural processes that are involved in spontaneous categorical representations of animal body silhouettes as a cognitive basis of human-animal interactions.

Voxelwise encoding models of body stimuli reveal a representational gradient from low-level visual features to postural features in occipitotemporal cortex

Research on body representation in the brain has focused on category-specific representation, using fMRI to investigate the response pattern to body stimuli in occipitotemporal cortex without so far addressing the issue of the specific computations involved in body selective regions, only defined by higher order category selectivity. This study used ultra-high field fMRI and banded ridge regression to investigate the coding of body images, by comparing the performance of three encoding models in predicting brain activity in occipitotemporal cortex and specifically the extrastriate body area (EBA). Our results suggest that bodies are encoded in occipitotemporal cortex and in the EBA according to a combination of low-level visual features and postural features.

More Than the Face: Representations of Bodies in the Inferior Temporal Cortex

Visual representations of bodies, in addition to those of faces, contribute to the recognition of con- and heterospecifics, to action recognition, and to nonverbal communication. Despite its importance, the neural basis of the visual analysis of bodies has been less studied than that of faces. In this article, I review what is known about the neural processing of bodies, focusing on the macaque temporal visual cortex. Early single-unit recording work suggested that the temporal visual cortex contains representations of body parts and bodies, with the dorsal bank of the superior temporal sulcus representing bodily actions. Subsequent functional magnetic resonance imaging studies in both humans and monkeys showed several temporal cortical regions that are strongly activated by bodies. Single-unit recordings in the macaque body patches suggest that these represent mainly body shape features. More anterior patches show a greater viewpoint-tolerant selectivity for body features, which may reflect a processing principle shared with other object categories, including faces. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

It is not just the category: behavioral effects of fMRI-guided electrical microstimulation result from a complex interplay of factors

Functional imaging and electrophysiological studies in primates revealed the existence of patches selective for visual categories in the inferior temporal cortex. Understanding the contribution of these patches to perception requires causal techniques that assess the effect of neural activity manipulations on perception. We used electrical microstimulation (EM) to determine the role of body patch activity in visual categorization in macaques. We tested the hypothesis that EM in a body patch would affect the categorization of bodies versus objects but not of other visual categories. We employed low-current EM of an anterior body patch (ASB) in the superior temporal sulcus, which was defined by functional magnetic resonance imaging and verified with electrophysiological recordings in each session. EM of ASB affected body categorization, but the EM effects were more complex than the expected increase of body-related choices: EM affected the categorization of both body and inanimate images and showed interaction with the choice target location, but its effect was location-specific (tested in 1 subject) on a millimeter scale. Our findings suggest that the behavioral effects of EM in a category-selective patch are not merely a manifestation of the category selectivity of the underlying neuronal population but reflect a complex interplay of multiple factors.

How Visual Expertise Changes Representational Geometry: A Behavioral and Neural Perspective

Many people develop expertise in specific domains of interest, such as chess, microbiology, radiology, and, the case in point in our study: ornithology. It is poorly understood to what extent such expertise alters brain function. Previous neuroimaging studies of expertise have typically focused upon the category level, for example, selectivity for birds versus nonbird stimuli. We present a multivariate fMRI study focusing upon the representational similarity among objects of expertise at the subordinate level. We compare the neural representational spaces of experts and novices to behavioral judgments. At the behavioral level, ornithologists (n = 20) have more fine-grained and task-dependent representations of item similarity that are more consistent among experts compared to control participants. At the neural level, the neural patterns of item similarity are more distinct and consistent in experts than in novices, which is in line with the behavioral results. In addition, these neural patterns in experts show stronger correlations with behavior compared to novices. These findings were prominent in frontal regions, and some effects were also found in occipitotemporal regions. This study illustrates the potential of an analysis of representational geometry to understand to what extent expertise changes neural information processing.

Factors Determining Where Category-Selective Areas Emerge in Visual Cortex

A hallmark of functional localization in the human brain is the presence of areas in visual cortex specialized for representing particular categories such as faces and words. Why do these areas appear where they do during development? Recent findings highlight several general factors to consider when answering this question. Experience-driven category selectivity arises in regions that have: (i) pre-existing selectivity for properties of the stimulus, (ii) are appropriately placed in the computational hierarchy of the visual system, and (iii) exhibit domain-specific patterns of connectivity to nonvisual regions. In other words, cortical location of category selectivity is constrained by what category will be represented, how it will be represented, and why the representation will be used.

A behavioral face preference deficit in a monkey with an incomplete face patch system

Primates are experts in face perception and naturally show a preference for faces under free-viewing conditions. The primate ventral stream is characterized by a network of face patches that selectively responds to faces, but it remains uncertain how important such parcellation is for face perception. Here we investigated free-viewing behavior in a female monkey who naturally lacks fMRI-defined posterior and middle lateral face patches. We presented a series of content-rich images of scenes that included faces or other objects to that monkey during a free-viewing task and tested a group of 10 control monkeys on the same task for comparison. We found that, compared to controls, the monkey with missing face patches showed a marked reduction of face viewing preference that was most pronounced for the first few fixations. In addition, her gaze fixation patterns were substantially distinct from those of controls, especially for pictures with a face. These data demonstrate an association between the clustering of neurons in face selective patches and a behavioral bias for faces in natural images.

Face Repetition Probability Does Not Affect Repetition Suppression in Macaque Inferotemporal Cortex

Repetition suppression, which refers to reduced neural activity for repeated stimuli, is typically explained by bottom-up or local adaptation mechanisms. However, recent theories have emphasized the role of top-down processes, suggesting that this response reduction reflects the fulfillment of perceptual expectations. To support this, an influential human fMRI study showed that the magnitude of suppression is modulated by the probability of a repetition. No such repetition probability effect was found in macaque inferior temporal (IT) cortex for spiking activity despite the presence of repetition suppression. Contrary to the human fMRI studies that showed an effect of repetition probability, the macaque single-unit study used a large variety of unfamiliar stimuli and the monkeys were not required to attend the stimuli. Here, as in the human fMRI studies, we used faces as stimuli and made the monkeys attend to the stimulus content. We simultaneously recorded spiking activity and local field potentials (LFPs) in the middle lateral face patch (ML) of one monkey (male) and a face-responsive region of another (female). Although we observed significant repetition suppression of spiking activity and high gamma-band LFPs in both animals, there were no effects of repetition probability even when repetitions were task relevant and repetition probability affected behavioral decisions. In conclusion, despite the use of face stimuli and a stimulus-related task, no neural signature of repetition probability was present for faces in a face responsive patch of macaque IT. This further challenges a general perceptual expectation account of repetition suppression.SIGNIFICANCE STATEMENT Repetition suppression is a reduced brain activity for repeated stimuli commonly observed across species. In the predictive coding framework, such suppression is thought to reflect fulfilled perceptual expectations. Although this hypothesis is supported by several human fMRI studies reporting an effect of repetition probability on repetition suppression, this could not be replicated in single-cell recordings in monkey inferior temporal (IT) cortex. Subsequent studies narrowed down the conditions for the effect to requiring attention and being limited to particular stimulus categories such as faces. Here, we show that, even under these conditions, repetition suppression in monkey IT neurons is still unaffected by repetition probability, even in a task with a behavioral effect, challenging the perceptual expectation account of repetition suppression.

Investigating common coding of observed and executed actions in the monkey brain using cross-modal multi-variate fMRI classification

Mirror neurons are generally described as a neural substrate hosting shared representations of actions, by simulating or 'mirroring' the actions of others onto the observer's own motor system. Since single neuron recordings are rarely feasible in humans, it has been argued that cross-modal multi-variate pattern analysis (MVPA) of non-invasive fMRI data is a suitable technique to investigate common coding of observed and executed actions, allowing researchers to infer the presence of mirror neurons in the human brain. In an effort to close the gap between monkey electrophysiology and human fMRI data with respect to the mirror neuron system, here we tested this proposal for the first time in the monkey. Rhesus monkeys either performed reach-and-grasp or reach-and-touch motor acts with their right hand in the dark or observed videos of human actors performing similar motor acts. Unimodal decoding showed that both executed or observed motor acts could be decoded from numerous brain regions. Specific portions of rostral parietal, premotor and motor cortices, previously shown to house mirror neurons, in addition to somatosensory regions, yielded significant asymmetric action-specific cross-modal decoding. These results validate the use of cross-modal multi-variate fMRI analyses to probe the representations of own and others' actions in the primate brain and support the proposed mapping of others' actions onto the observer's own motor cortices.

Linear Representation of Emotions in Whole Persons by Combining Facial and Bodily Expressions in the Extrastriate Body Area

Our human brain can rapidly and effortlessly perceive a person’s emotional state by integrating the isolated emotional faces and bodies into a whole. Behavioral studies have suggested that the human brain encodes whole persons in a holistic rather than part-based manner. Neuroimaging studies have also shown that body-selective areas prefer whole persons to the sum of their parts. The body-selective areas played a crucial role in representing the relationships between emotions expressed by different parts. However, it remains unclear in which regions the perception of whole persons is represented by a combination of faces and bodies, and to what extent the combination can be influenced by the whole person’s emotions. In the present study, functional magnetic resonance imaging data were collected when participants performed an emotion distinction task. Multi-voxel pattern analysis was conducted to examine how the whole person-evoked responses were associated with the face- and body-evoked responses in several specific brain areas. We found that in the extrastriate body area (EBA), the whole person patterns were most closely correlated with weighted sums of face and body patterns, using different weights for happy expressions but equal weights for angry and fearful ones. These results were unique for the EBA. Our findings tentatively support the idea that the whole person patterns are represented in a part-based manner in the EBA, and modulated by emotions. These data will further our understanding of the neural mechanism underlying perceiving emotional persons.

Interpreting the dimensions of neural feature representations revealed by dimensionality reduction

Recent progress in understanding the structure of neural representations in the cerebral cortex has centred around the application of multivariate classification analyses to measurements of brain activity. These analyses have proved a sensitive test of whether given brain regions provide information about specific perceptual or cognitive processes. An exciting extension of this approach is to infer the structure of this information, thereby drawing conclusions about the underlying neural representational space. These approaches rely on exploratory data-driven dimensionality reduction to extract the natural dimensions of neural spaces, including natural visual object and scene representations, semantic and conceptual knowledge, and working memory. However, the efficacy of these exploratory methods is unknown, because they have only been applied to representations in brain areas for which we have little or no secondary knowledge. One of the best-understood areas of the cerebral cortex is area MT of primate visual cortex, which is known to be important in motion analysis. To assess the effectiveness of dimensionality reduction for recovering neural representational space we applied several dimensionality reduction methods to multielectrode measurements of spiking activity obtained from area MT of marmoset monkeys, made while systematically varying the motion direction and speed of moving stimuli. Despite robust tuning at individual electrodes, and high classifier performance, dimensionality reduction rarely revealed dimensions for direction and speed. We use this example to illustrate important limitations of these analyses, and suggest a framework for how to best apply such methods to data where the structure of the neural representation is unknown.

A data mining approach using cortical thickness for diagnosis and characterization of essential tremor

Essential tremor (ET) is one of the most prevalent movement disorders. Being that it is a common disorder, its diagnosis is considered routine. However, misdiagnoses may occur regularly. Over the past decade, several studies have identified brain morphometric changes in ET, but these changes remain poorly understood. Here, we tested the informativeness of measuring cortical thickness for the purposes of ET diagnosis, applying feature selection and machine learning methods to a study sample of 18 patients with ET and 18 age- and sex-matched healthy control subjects. We found that cortical thickness features alone distinguished the two, ET from controls, with 81% diagnostic accuracy. More specifically, roughness (i.e., the standard deviation of cortical thickness) of the right inferior parietal and right fusiform areas was shown to play a key role in ET characterization. Moreover, these features allowed us to identify subgroups of ET patients as well as healthy subjects at risk for ET. Since treatment of tremors is disease specific, accurate and early diagnosis plays an important role in tremor management. Supporting the clinical diagnosis with novel computer approaches based on the objective evaluation of neuroimage data, like the one presented here, may represent a significant step in this direction.

Functional dissociation between anterior temporal lobe and inferior frontal gyrus in the processing of dynamic body expressions: Insights from behavioral variant frontotemporal dementia

Several brain regions are involved in the processing of emotional stimuli, however, the contribution of specific regions to emotion perception is still under debate. To investigate this issue, we combined behavioral testing, structural and resting state imaging in patients diagnosed with behavioral variant frontotemporal dementia (bvFTD) and age matched controls, with task-based functional imaging in young, healthy volunteers. As expected, bvFTD patients were impaired in emotion detection as well as emotion categorization tasks, testing dynamic emotional body expressions as stimuli. Interestingly, their performance in the two tasks correlated with gray matter volume in two distinct brain regions, the left anterior temporal lobe for emotion detection and the left inferior frontal gyrus (IFG) for emotion categorization. Confirming this observation, multivoxel pattern analysis in healthy volunteers demonstrated that both ROIs contained information for emotion detection, but that emotion categorization was only possible from the pattern in the IFG. Furthermore, functional connectivity analysis showed reduced connectivity between the two regions in bvFTD patients. Our results illustrate that the mentalizing network and the action observation network perform distinct tasks during emotion processing. In bvFTD, communication between the networks is reduced, indicating one possible cause underlying the behavioral symptoms. Hum Brain Mapp 37:4472-4486, 2016. © 2016 Wiley Periodicals, Inc.

Steady-state visually evoked potential correlates of human body perception

In cognitive neuroscience, interest in the neuronal basis underlying the processing of human bodies is steadily increasing. Based on functional magnetic resonance imaging studies, it is assumed that the processing of pictures of human bodies is anchored in a network of specialized brain areas comprising the extrastriate and the fusiform body area (EBA, FBA). An alternative to examine the dynamics within these networks is electroencephalography, more specifically so-called steady-state visually evoked potentials (SSVEPs). In SSVEP tasks, a visual stimulus is presented repetitively at a predefined flickering rate and typically elicits a continuous oscillatory brain response at this frequency. This brain response is characterized by an excellent signal-to-noise ratio-a major advantage for source reconstructions. The main goal of present study was to demonstrate the feasibility of this method to study human body perception. To that end, we presented pictures of bodies and contrasted the resulting SSVEPs to two control conditions, i.e., non-objects and pictures of everyday objects (chairs). We found specific SSVEPs amplitude differences between bodies and both control conditions. Source reconstructions localized the SSVEP generators to a network of temporal, occipital and parietal areas. Interestingly, only body perception resulted in activity differences in middle temporal and lateral occipitotemporal areas, most likely reflecting the EBA/FBA.

Stimulus features coded by single neurons of a macaque body category selective patch

Body category-selective regions of the primate temporal cortex respond to images of bodies, but it is unclear which fragments of such images drive single neurons' responses in these regions. Here we applied the Bubbles technique to the responses of single macaque middle superior temporal sulcus (midSTS) body patch neurons to reveal the image fragments the neurons respond to. We found that local image fragments such as extremities (limbs), curved boundaries, and parts of the torso drove the large majority of neurons. Bubbles revealed the whole body in only a few neurons. Neurons coded the features in a manner that was tolerant to translation and scale changes. Most image fragments were excitatory but for a few neurons both inhibitory and excitatory fragments (opponent coding) were present in the same image. The fragments we reveal here in the body patch with Bubbles differ from those suggested in previous studies of face-selective neurons in face patches. Together, our data indicate that the majority of body patch neurons respond to local image fragments that occur frequently, but not exclusively, in bodies, with a coding that is tolerant to translation and scale. Overall, the data suggest that the body category selectivity of the midSTS body patch depends more on the feature statistics of bodies (e.g., extensions occur more frequently in bodies) than on semantics (bodies as an abstract category).

Repetition suppression for visual actions in the macaque superior temporal sulcus

In many brain areas, repetition of a stimulus usually weakens the neural response. This "adaptation" or repetition suppression effect has been observed with mass potential measures such as event-related potentials (ERPs), in fMRI BOLD responses, and locally with local field potentials (LFPs) and spiking activity. Recently, it has been reported that macaque F5 mirror neurons do not show repetition suppression of their spiking activity for single repetitions of hand actions, which disagrees with human fMRI adaptation studies. This finding also contrasts with numerous studies showing repetition suppression in macaque inferior temporal cortex, including the rostral superior temporal sulcus (STS). Since the latter studies employed static stimuli, we assessed here whether the use of dynamic action stimuli abolishes repetition suppression in the awake macaque STS. To assess adaptation effects in the STS, we employed the same hand action movies as used when examining adaptation in F5. The upper bank STS neurons showed repetition suppression during the approaching phase of the hand action, which corresponded to the phase of the action for which these neurons responded overall the strongest. The repetition suppression was present for the spiking activity measured in independent single-unit and multiunit recordings as well as for the LFP power at frequencies > 50 Hz. Together with previous data in F5, these findings suggest that adaptation effects differ between F5 mirror neurons and the STS neurons.

Seeing biological actions in 3D: An fMRI study

Precise kinematics or body configuration cannot be recovered from visual input without disparity information. Yet, no imaging study has investigated the role of disparity on action observation. Here, we investigated the interaction between disparity and the main cues of biological motion, kinematics and configuration, in two fMRI experiments. Stimuli were presented as point‐light figures, depicting complex action sequences lasting 21 s. We hypothesized that interactions could occur at any of the three levels of the action observation network, comprising occipitotemporal, parietal and premotor cortex, with premotor cortex being the most likely location. The main effects of kinematics and configuration confirmed that the biological motion sequences activated all three levels of the action observation network, validating our approach. The interaction between configuration and disparity activated only premotor cortex, whereas interactions between kinematics and disparity occurred at all levels of the action observation network but were strongest at the premotor level. Control experiments demonstrated that these interactions could not be accounted for by low level motion in depth, task effects, spatial attention, or eye movements, including vergence. These results underscore the role of premotor cortex in action observation, and in imitating others or responding to their actions. Hum Brain Mapp 37:203–219, 2016. © 2015 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

Common neural correlates of emotion perception in humans

Whether neuroimaging findings support discriminable neural correlates of emotion categories is a longstanding controversy. Two recent meta-analyses arrived at opposite conclusions, with one supporting (Vytal and Hamann []: J Cogn Neurosci 22:2864-2885) and the other opposing this proposition (Lindquist et al. []: Behav Brain Sci 35:121-143). To obtain direct evidence regarding this issue, we compared activations for four emotions within a single fMRI design. Angry, happy, fearful, sad and neutral stimuli were presented as dynamic body expressions. In addition, observers categorized motion morphs between neutral and emotional stimuli in a behavioral experiment to determine their relative sensitivities. Brain-behavior correlations revealed a large brain network that was identical for all four tested emotions. This network consisted predominantly of regions located within the default mode network and the salience network. Despite showing brain-behavior correlations for all emotions, muli-voxel pattern analyses indicated that several nodes of this emotion general network contained information capable of discriminating between individual emotions. However, significant discrimination was not limited to the emotional network, but was also observed in several regions within the action observation network. Taken together, our results favor the position that one common emotional brain network supports the visual processing and discrimination of emotional stimuli.