The overlap of the EBA and the MT/V5 cluster

Natural scenes reveal diverse representations of 2D and 3D body pose in the human brain

Human pose, defined as the spatial relationships between body parts, carries instrumental information supporting the understanding of motion and action of a person. A substantial body of previous work has identified cortical areas responsive to images of bodies and different body parts. However, the neural basis underlying the visual perception of body part relationships has received less attention. To broaden our understanding of body perception, we analyzed high-resolution fMRI responses to a wide range of poses from over 4,000 complex natural scenes. Using ground-truth annotations and an application of three-dimensional (3D) pose reconstruction algorithms, we compared similarity patterns of cortical activity with similarity patterns built from human pose models with different levels of depth availability and viewpoint dependency. Targeting the challenge of explaining variance in complex natural image responses with interpretable models, we achieved statistically significant correlations between pose models and cortical activity patterns (though performance levels are substantially lower than the noise ceiling). We found that the 3D view-independent pose model, compared with two-dimensional models, better captures the activation from distinct cortical areas, including the right posterior superior temporal sulcus (pSTS). These areas, together with other pose-selective regions in the LOTC, form a broader, distributed cortical network with greater view-tolerance in more anterior patches. We interpret these findings in light of the computational complexity of natural body images, the wide range of visual tasks supported by pose structures, and possible shared principles for view-invariant processing between articulated objects and ordinary, rigid objects.

Neural Representations of Observed Interpersonal Synchrony/Asynchrony in the Social Perception Network

The visual perception of individuals is thought to be mediated by a network of regions in the occipitotemporal cortex that supports specialized processing of faces, bodies, and actions. In comparison, we know relatively little about the neural mechanisms that support the perception of multiple individuals and the interactions between them. The present study sought to elucidate the visual processing of social interactions by identifying which regions of the social perception network represent interpersonal synchrony. In an fMRI study with 32 human participants (26 female, 6 male), we used multivoxel pattern analysis to investigate whether activity in face-selective, body-selective, and interaction-sensitive regions across the social perception network supports the decoding of synchronous versus asynchronous head-nodding and head-shaking. Several regions were found to support significant decoding of synchrony/asynchrony, including extrastriate body area (EBA), face-selective and interaction-sensitive mid/posterior right superior temporal sulcus, and occipital face area. We also saw robust cross-classification across actions in the EBA, suggestive of movement-invariant representations of synchrony/asynchrony. Exploratory whole-brain analyses also identified a region of the right fusiform cortex that responded more strongly to synchronous than to asynchronous motion. Critically, perceiving interpersonal synchrony/asynchrony requires the simultaneous extraction and integration of dynamic information from more than one person. Hence, the representation of synchrony/asynchrony cannot be attributed to augmented or additive processing of individual actors. Our findings therefore provide important new evidence that social interactions recruit dedicated visual processing within the social perception network that extends beyond that engaged by the faces and bodies of the constituent individuals.

Loss of action-related function and connectivity in the blind extrastriate body area

The Extrastriate Body Area (EBA) participates in the visual perception and motor actions of body parts. We recently showed that EBA’s perceptual function develops independently of visual experience, responding to stimuli with body-part information in a supramodal fashion. However, it is still unclear if the EBA similarly maintains its action-related function. Here, we used fMRI to study motor-evoked responses and connectivity patterns in the congenitally blind brain. We found that, unlike the case of perception, EBA does not develop an action-related response without visual experience. In addition, we show that congenital blindness alters EBA’s connectivity profile in a counterintuitive way—functional connectivity with sensorimotor cortices dramatically decreases, whereas connectivity with perception-related visual occipital cortices remains high. To the best of our knowledge, we show for the first time that action-related functions and connectivity in the visual cortex could be contingent on visuomotor experience. We further discuss the role of the EBA within the context of visuomotor control and predictive coding theory.

Functional relevance of the extrastriate body area for visual and haptic object recognition: a preregistered fMRI-guided TMS study

The extrastriate body area (EBA) is a region in the lateral occipito-temporal cortex (LOTC), which is sensitive to perceived body parts. Neuroimaging studies suggested that EBA is related to body and tool processing, regardless of the sensory modalities. However, how essential this region is for visual tool processing and nonvisual object processing remains a matter of controversy. In this preregistered fMRI-guided repetitive transcranial magnetic stimulation (rTMS) study, we examined the causal involvement of EBA in multisensory body and tool recognition. Participants used either vision or haptics to identify 3 object categories: hands, teapots (tools), and cars (control objects). Continuous theta-burst stimulation (cTBS) was applied over left EBA, right EBA, or vertex (control site). Performance for visually perceived hands and teapots (relative to cars) was more strongly disrupted by cTBS over left EBA than over the vertex, whereas no such object-specific effect was observed in haptics. The simulation of the induced electric fields confirmed that the cTBS affected regions including EBA. These results indicate that the LOTC is functionally relevant for visual hand and tool processing, whereas the rTMS over EBA may differently affect object recognition between the 2 sensory modalities.

A new model for freedom of movement using connectomic analysis

The problem of whether we can execute free acts or not is central in philosophical thought, and it has been studied by numerous scholars throughout the centuries. Recently, neurosciences have entered this topic contributing new data and insights into the neuroanatomical basis of cognitive processes. With the advent of connectomics, a more refined landscape of brain connectivity can be analysed at an unprecedented level of detail. Here, we identify the connectivity network involved in the movement process from a connectomics point of view, from its motivation through its execution until the sense of agency develops. We constructed a "volitional network" using data derived from the Brainnetome Atlas database considering areas involved in volitional processes as known in the literature. We divided this process into eight processes and used Graph Theory to measure several structural properties of the network. Our results show that the volitional network is small-world and that it contains four communities. Nodes of the right hemisphere are contained in three of these communities whereas nodes of the left hemisphere only in two. Centrality measures indicate the nucleus accumbens is one of the most connected nodes in the network. Extensive connectivity is observed in all processes except in Decision (to move) and modulation of Agency, which might correlate with a mismatch mechanism for perception of Agency.

More Than the Face: Representations of Bodies in the Inferior Temporal Cortex

Visual representations of bodies, in addition to those of faces, contribute to the recognition of con- and heterospecifics, to action recognition, and to nonverbal communication. Despite its importance, the neural basis of the visual analysis of bodies has been less studied than that of faces. In this article, I review what is known about the neural processing of bodies, focusing on the macaque temporal visual cortex. Early single-unit recording work suggested that the temporal visual cortex contains representations of body parts and bodies, with the dorsal bank of the superior temporal sulcus representing bodily actions. Subsequent functional magnetic resonance imaging studies in both humans and monkeys showed several temporal cortical regions that are strongly activated by bodies. Single-unit recordings in the macaque body patches suggest that these represent mainly body shape features. More anterior patches show a greater viewpoint-tolerant selectivity for body features, which may reflect a processing principle shared with other object categories, including faces. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

OUP accepted manuscript

Neuralgic amyotrophy is a common peripheral nerve disorder caused by autoimmune inflammation of the brachial plexus, clinically characterized by acute pain and weakness of the shoulder muscles, followed by motor impairment. Despite recovery of the peripheral nerves, patients often have residual motor dysfunction of the upper extremity, leading to persistent pain related to altered biomechanics of the shoulder region. Building on clinical signs that suggest a role for cerebral mechanisms in these residual complaints, here we show and characterize cerebral alterations following neuralgic amyotrophy. Neuralgic amyotrophy patients often develop alternative motor strategies, which suggests that (mal)adaptations may occur in somatomotor and/or visuomotor brain areas. Here, we tested where changes in cerebral sensorimotor representations occur in neuralgic amyotrophy, while controlling for altered motor execution due to peripheral neuropathy. We additionally explore the relation between potential cerebral alterations in neuralgic amyotrophy and clinical symptoms. During functional MRI scanning, 39 neuralgic amyotrophy patients with persistent, lateralized symptoms in the right upper extremity and 23 matched healthy participants solved a hand laterality judgement task that can activate sensorimotor representations of the upper extremity, across somatomotor and visuomotor brain areas. Behavioural and cerebral responses confirmed the involvement of embodied, sensorimotor processes across groups. Compared with healthy participants, neuralgic amyotrophy patients were slower in hand laterality judgement and had decreased cerebral activity specific to their affected limb in two higher-order visual brain regions: the right extrastriate cortex and the parieto-occipital sulcus. Exploratory analyses revealed that across patients, extrastriate activity specific to the affected limb decreased as persistent pain increased, and affected limb-related parieto-occipital activity decreased as imagery performance of the affected limb became slower. These findings suggest that maladaptive cerebral plasticity in visuomotor areas involved in sensorimotor integration plays a role in residual motor dysfunction and subsequent persistent pain in neuralgic amyotrophy. Rehabilitation interventions that apply visuomotor strategies to improve sensorimotor integration may help to treat neuralgic amyotrophy patients.

Neurophysiological mechanisms of perspective-taking: An MEG investigation of agency

According to the embodied cognition framework, sensory and motor areas are recruited during language understanding through simulation processes. Behavioral and imaging findings point to a dependence of the latter on perspective-taking (e.g., first person "I" versus third person "s/he"). The current study aims at identifying possible neurophysiological correlates of perspective in a linguistic context. Twenty healthy participants were measured with magnetoencephalography (MEG) while semantically processing visually presented inflected German verbs in the first- and third-person perspective, simple present tense. Results show that the first-person perspective induces stronger beta (15-25 Hz) desynchronization in the right-hemispheric posterior superior temporal sulcus, ventral posterior cingulate gyrus, and V5/MT+ area; no modulation of sensorimotor cortex emerged. Moreover, a stronger event-related field (ERF) was observed for the first-person perspective at about 150 ms after pronoun-verb onset, originating in occipital and moving to central and left temporal cortical sites. No effect of perspective on sensory gating was found when targeting the N1 component related to tones following the linguistic stimuli. Results indicate an effect of linguistic perspective-taking on brain activation patterns. The contribution of the single brain areas and their role in self-other distinction is further discussed.

Action Observation Responses Are Influenced by Movement Kinematics and Target Identity

In order to inform the debate whether cortical areas related to action observation provide a pragmatic or a semantic representation of goal-directed actions, we performed 2 functional magnetic resonance imaging (fMRI) experiments in humans. The first experiment, involving observation of aimless arm movements, resulted in activation of most of the components known to support action execution and action observation. Given the absence of a target/goal in this experiment and the activation of parieto-premotor cortical areas, which were associated in the past with direction, amplitude, and velocity of movement of biological effectors, our findings suggest that during action observation we could be monitoring movement kinematics. With the second, double dissociation fMRI experiment, we revealed the components of the observation-related cortical network affected by 1) actions that have the same target/goal but different reaching and grasping kinematics and 2) actions that have very similar kinematics but different targets/goals. We found that certain areas related to action observation, including the mirror neuron ones, are informed about movement kinematics and/or target identity, hence providing a pragmatic rather than a semantic representation of goal-directed actions. Overall, our findings support a process-driven simulation-like mechanism of action understanding, in agreement with the theory of motor cognition, and question motor theories of action concept processing.

The extrastriate body area is involved in reciprocal imitation of hand gestures, vocalizations, and facial expressions: A univariate and multivariate fMRI study

The extrastriate body area (EBA) in the lateral occipito-temporal cortex has an important role in reciprocal interaction, as it detects congruence between self and other's hand actions. However, it is unclear whether the EBA can detect congruence regardless of the type of action. In the present study, we examined the neural substrate underlying congruence detection of three types of actions: hand gestures, vocalizations, and facial expressions. A univariate analysis revealed a congruency effect, especially for imitating action, for all three types of actions in the EBA. A multi-voxel pattern analysis classifier in the EBA was able to distinguish between initiating interaction from responding to interaction in all experiments. Correspondingly, the congruency effect in the EBA revealed by univariate analysis was stronger for responding to than for initiating interaction. These findings suggest that the EBA might contribute to detect congruence regardless of the body part used (i.e. face or hand) and the type of action (i.e. gestural or vocal). Moreover, initiating and responding to interaction might be processed differently within the EBA. This study highlights the role of the EBA in comparing between self and other's actions beyond hand actions.Running head: Function of EBA in reciprocal imitation.

Visual Neuropsychology in Development: Anatomo-Functional Brain Mechanisms of Action/Perception Binding in Health and Disease

Vision is the main entrance for environmental input to the human brain. Even if vision is our most used sensory modality, its importance is not limited to environmental exploration. Rather it has strong links to motor competences, further extending to cognitive and social aspects of human life. These multifaceted relationships are particularly important in developmental age and become dramatically evident in presence of complex deficits originating from visual aberrancies. The present review summarizes the available neuropsychological evidence on the development of visual competences, with a particular focus on the associated visuo-motor integration skills in health and disease. With the aim of supporting future research and interventional settings, the goal of the present review is to constitute a solid base to help the translation of neuropsychological hypotheses into straightforward empirical investigations and rehabilitation/training protocols. This approach will further increase the impact, ameliorate the acceptance, and ease the use and implementation of lab-derived intervention protocols in real-life situations.

Neural interactions in occipitotemporal cortex during basic human movement perception by dynamic causal modeling

Action recognition is an essential component of our daily life. The occipitotemporal cortex (OTC) is an important area in human movement perception. The previous studies have revealed that three vital regions including the extrastriate body area (EBA), human middle temporal complex (hMT+), and posterior superior temporal sulcus (pSTS) in OTC play an important role in motion perception. The aim of the current study is to explore the neural interactions between these three regions during basic human movement perception. Functional magnetic resonance imaging data were acquired when participants viewed dynamic videos depicting basic human movements. By the dynamic causal modeling analysis, a model space consisting of 576 models was constructed and evaluated to select the optimal model given the data. The information of the visual movement was found to enter the system through hMT+. We speculated that hMT+ would be the region to show sensitivity to the presence of motion and it subsequently influence and be influenced by the other two regions. Our results also revealed the manner in which the three regions interact during action recognition. Furthermore, We found significantly enhanced modulated connectivity from hMT+ to both EBA and pSTS, as well as from EBA to both hMT+ and pSTS. We inferred that there may be multiple routes for human action perception. One responsible route for processing motion signals is through hMT+ to pSTS, and the other projects information to pSTS may be via the form-processing route. In addition, pSTS may integrate and mediate visual signals and possibly convey them to distributed areas to maintain high-order cognitive tasks.

Functional Segregation of the Middle Temporal Visual Motion Area Revealed With Coactivation-Based Parcellation

Traditionally, the visual motion area (MT) is considered as a brain region specialized for visual motion perception. However, accumulating evidence showed that MT is also related to various functions, suggesting that it is a complex functional area and different functional subregions might exist in this area. To delineate functional subregions of this area, left and right masks of MT were defined using meta-analysis in the BrainMap database, and coactivation-based parcellation was then performed on these two masks. Two dorsal subregions (Cl1 and Cl2) and one ventral subregion (Cl3) of left MT, as well as two dorsal-anterior subregions (Cl1 and Cl2), one ventral-anterior subregion (Cl3), and an additional posterior subregion (Cl4) of right MT were identified. In addition to vision motion, distinct and specific functions were identified in different subregions characterized by task-dependent functional connectivity mapping and forward/reverse inference on associated functions. These results not only were in accordance with the previous findings of a hemispheric asymmetry of MT, but also strongly confirmed the existence of subregions in this region with distinct and specific functions. Furthermore, our results extend the special role of visual motion perception on this area and might facilitate future cognitive study.

The representational space of observed actions

Categorizing and understanding other people’s actions is a key human capability. Whereas there exists a growing literature regarding the organization of objects, the representational space underlying the organization of observed actions remains largely unexplored. Here we examined the organizing principles of a large set of actions and the corresponding neural representations. Using multiple regression representational similarity analysis of fMRI data, in which we accounted for variability due to major action components (body parts, scenes, movements, objects, sociality, transitivity) and three control models (distance between observer and actor, number of people, HMAX-C1), we found that the semantic dissimilarity structure was best captured by patterns of activation in the lateral occipitotemporal cortex (LOTC). Together, our results demonstrate that the organization of observed actions in the LOTC resembles the organizing principles used by participants to classify actions behaviorally, in line with the view that this region is crucial for accessing the meaning of actions.

Cortical Regions Encoding Hardness Perception Modulated by Visual Information Identified by Functional Magnetic Resonance Imaging With Multivoxel Pattern Analysis

Recent studies have revealed that hardness perception is determined by visual information along with the haptic input. This study investigated the cortical regions involved in hardness perception modulated by visual information using functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA). Twenty-two healthy participants were enrolled. They were required to place their left and right hands at the front and back, respectively, of a mirror attached to a platform placed above them while lying in a magnetic resonance scanner. In conditions SFT, MED, and HRD, one of three polyurethane foam pads of varying hardness (soft, medium, and hard, respectively) was presented to the left hand in a given trial, while only the medium pad was presented to the right hand in all trials. MED was defined as the control condition, because the visual and haptic information was congruent. During the scan, the participants were required to push the pad with the both hands while observing the reflection of the left hand and estimate the hardness of the pad perceived by the right (hidden) hand based on magnitude estimation. Behavioral results showed that the perceived hardness was significantly biased toward softer or harder in >73% of the trials in conditions SFT and HRD; we designated these trials as visually modulated (SFTvm and HRDvm, respectively). The accuracy map was calculated individually for each of the pair-wise comparisons of (SFTvm vs. MED), (HRDvm vs. MED), and (SFTvm vs. HRDvm) by a searchlight MVPA, and the cortical regions encoding the perceived hardness with visual modulation were identified by conjunction of the three accuracy maps in group analysis. The cluster was observed in the right sensory motor cortex, left anterior intraparietal sulcus (aIPS), bilateral parietal operculum (PO), and occipito-temporal cortex (OTC). Together with previous findings on such cortical regions, we conclude that the visual information of finger movements processed in the OTC may be integrated with haptic input in the left aIPS, and the subjective hardness perceived by the right hand with visual modulation may be processed in the cortical network between the left PO and aIPS.

Rapid and specific processing of person-related information in human anterior temporal lobe

The anterior temporal lobe (ATL), located at the tip of the human temporal lobes, has been heavily implicated in semantic processing by neuropsychological and functional imaging studies. These techniques have revealed a hemispheric specialization of ATL, but little about the time scale on which it operates. Here we show that ATL is specifically activated in intracerebral recordings when subjects discriminate the gender of an actor presented in a static frame followed by a video. ATL recording sites respond briefly (100 ms duration) to the visual static presentation of an actor in a task-, but not in a stimulus-duration-dependent way. Their response latencies correlate with subjects' reaction times, as do their activity levels, but oppositely in the two hemispheres operating in a push-pull fashion. Comparison of ATL time courses with those of more posterior, less specific regions emphasizes the role of inhibitory operations sculpting the fast ATL responses underlying semantic processing.

Computational Feature Analysis of Body Movements Reveals Hierarchical Brain Organization

Social species spend considerable time observing the body movements of others to understand their actions, predict their emotions, watch their games, or enjoy their dance movements. Given the important information obtained from body movements, we still know surprisingly little about the details of brain mechanisms underlying movement perception. In this fMRI study, we investigated the relations between movement features obtained from automated computational analyses of video clips and the corresponding brain activity. Our results show that low-level computational features map to specific brain areas related to early visual- and motion-sensitive regions, while mid-level computational features are related to dynamic aspects of posture encoded in occipital–temporal cortex, posterior superior temporal sulcus and superior parietal lobe. Furthermore, behavioral features obtained from subjective ratings correlated with activity in higher action observation regions. Our computational feature-based analysis suggests that the neural mechanism of movement encoding is organized in the brain not so much by semantic categories than by feature statistics of the body movements.

fMRI-based Multivariate Pattern Analyses Reveal Imagery Modality and Imagery Content Specific Representations in Primary Somatosensory, Motor and Auditory Cortices

Previous studies have shown that the early visual cortex contains content-specific representations of stimuli during visual imagery, and that these representational patterns of imagery content have a perceptual basis. To date, there is little evidence for the presence of a similar organization in the auditory and tactile domains. Using fMRI-based multivariate pattern analyses we showed that primary somatosensory, auditory, motor, and visual cortices are discriminative for imagery of touch versus sound. In the somatosensory, motor and visual cortices the imagery modality discriminative patterns were similar to perception modality discriminative patterns, suggesting that top-down modulations in these regions rely on similar neural representations as bottom-up perceptual processes. Moreover, we found evidence for content-specific representations of the stimuli during auditory imagery in the primary somatosensory and primary motor cortices. Both the imagined emotions and the imagined identities of the auditory stimuli could be successfully classified in these regions.

Behavioural evidence for distinct mechanisms related to global and biological motion perception

The perception of human motion is a vital ability in our daily lives. Human movement recognition is often studied using point-light stimuli in which dots represent the joints of a moving person. Depending on task and stimulus, the local motion of the single dots, and the global form of the stimulus can be used to discriminate point-light stimuli. Previous studies often measured motion coherence for global motion perception and contrasted it with performance in biological motion perception to assess whether difficulties in biological motion processing are related to more general difficulties with motion processing. However, it is so far unknown as to how performance in global motion tasks relates to the ability to use local motion or global form to discriminate point-light stimuli. Here, we investigated this relationship in more detail. In Experiment 1, we measured participants' ability to discriminate the facing direction of point-light stimuli that contained primarily local motion, global form, or both. In Experiment 2, we embedded point-light stimuli in noise to assess whether previously found relationships in task performance are related to the ability to detect signal in noise. In both experiments, we also assessed motion coherence thresholds from random-dot kinematograms. We found relationships between performances for the different biological motion stimuli, but performance for global and biological motion perception was unrelated. These results are in accordance with previous neuroimaging studies that highlighted distinct areas for global and biological motion perception in the dorsal pathway, and indicate that results regarding the relationship between global motion perception and biological motion perception need to be interpreted with caution.

Following during physically-coupled joint action engages motion area MT+/V5

Interpersonal coordination during joint action depends on the perception of the partner's movements. In many such situations - for example, while moving furniture together or dancing a tango - there are kinesthetic interactions between the partners due to the forces shared between them that allow them to directly perceive one another's movements. Joint action of this type often involves a contrast between the roles of leader and follower, where the leader imparts forces onto the follower, and the follower has to be responsive to these force-cues during movement. We carried out a novel 2-person functional MRI study with trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, where the two alternated between being the leader and follower of joint improvised movements, all with the eyes closed. One brain area that was unexpectedly more active during following than leading was the region of MT+/V5. While classically described as an area for processing visual motion, it has more recently been shown to be responsive to tactile motion as well. We suggest that MT+/V5 responds to motion based on force-cues during joint haptic interaction, most especially when a follower responds to force-cues coming from a leader's movements.

Is the extrastriate body area part of the dorsal visuomotor stream?

The extrastriate body area (EBA) processes visual information about body parts, and it is considered one among a series of category-specific perceptual modules distributed across the occipito-temporal cortex. However, recent evidence raises the possibility that EBA might also provide an interface between perception and action, linking the ventral and dorsal streams of visual information processing. Here, we assess anatomical evidence supporting this possibility. We localise EBA in individual subjects using a perceptual task and compare the characteristics of its functional and structural connectivity to those of two perceptual areas, the lateral occipital complex (LOC) and the fusiform body area (FBA), separately for each hemisphere. We apply complementary analyses of resting-state fMRI and diffusion-weighted MRI data in a group of healthy right-handed human subjects (N = 31). Functional and structural connectivity profiles indicate that EBA interacts more strongly with dorsal-stream regions compared to other portions of the occipito-temporal cortex involved in processing body parts (FBA) and object identification (LOC). These findings provide anatomical ground for a revision of the functional role of EBA. Building on a number of recent observations, we suggest that EBA contributes to planning goal-directed actions, possibly by specifying a desired postural configuration to parieto-frontal areas involved in computing movement parameters.

The Lateral Occipito-temporal Cortex Is Involved in the Mental Manipulation of Body Part Imagery

The lateral occipito-temporal cortex (LOTC), including the extrastriate body area, is known to be involved in the perception of body parts. Although still controversial, recent studies have demonstrated the role of the LOTC in higher-level body-related cognition in humans. This study consisted of two experiments (E1 and E2). The first (E1) was an exploratory experiment to find the neural correlate of the mental manipulation of body part imagery, in which brain cerebral glucose metabolic rates and the performance of mental rotation of the hand were measured in 100 subjects who exhibited a range of symptoms of cognitive decline. In E1, we found that the level of glucose metabolism in the right LOTC was significantly correlated with performance in a task involving mental manipulation of the hand. Next, in E2, we performed a randomized, double-blind, controlled intervention study (clinical trial number: UMIN 000018310) in younger healthy adults to test whether right occipital (corresponding to the right LOTC) anodal stimulation using transcranial direct current stimulation (tDCS) could enhance the mental manipulation of the hand. In E2, we demonstrated a significant effect of tDCS on the accuracy rate in a task involving mental manipulation of the hand. Although further study is necessary to answer the question of whether these results are specific for the mental manipulation of body parts but not non-body parts, E1 demonstrated a possible role of the LOTC in carrying out the body mental manipulation task in patients with dementia, and E2 suggested the possible effect of tDCS on this task in healthy subjects.

Neural Representations of Observed Actions Generalize across Static and Dynamic Visual Input

People interact with entities in the environment in distinct and categorizable ways (e.g., kicking is making contact with foot). We can recognize these action categories across variations in actors, objects, and settings; moreover, we can recognize them from both dynamic and static visual input. However, the neural systems that support action recognition across these perceptual differences are unclear. Here, we used multivoxel pattern analysis of fMRI data to identify brain regions that support visual action categorization in a format-independent way. Human participants were scanned while viewing eight categories of interactions (e.g., pulling) depicted in two visual formats: (1) visually controlled videos of two interacting actors and (2) visually varied photographs selected from the internet involving different actors, objects, and settings. Action category was decodable across visual formats in bilateral inferior parietal, bilateral occipitotemporal, left premotor, and left middle frontal cortex. In most of these regions, the representational similarity of action categories was consistent across subjects and visual formats, a property that can contribute to a common understanding of actions among individuals. These results suggest that the identified brain regions support action category codes that are important for action recognition and action understanding.SIGNIFICANCE STATEMENT Humans tend to interpret the observed actions of others in terms of categories that are invariant to incidental features: whether a girl pushes a boy or a button and whether we see it in real-time or in a single snapshot, it is still pushing Here, we investigated the brain systems that facilitate the visual recognition of these action categories across such differences. Using fMRI, we identified several areas of parietal, occipitotemporal, and frontal cortex that exhibit action category codes that are similar across viewing of dynamic videos and still photographs. Our results provide strong evidence for the involvement of these brain regions in recognizing the way that people interact physically with objects and other people.

Stereoscopically Observing Manipulative Actions

The purpose of this study was to investigate the contribution of stereopsis to the processing of observed manipulative actions. To this end, we first combined the factors "stimulus type" (action, static control, and dynamic control), "stereopsis" (present, absent) and "viewpoint" (frontal, lateral) into a single design. Four sites in premotor, retro-insular (2) and parietal cortex operated specifically when actions were viewed stereoscopically and frontally. A second experiment clarified that the stereo-action-specific regions were driven by actions moving out of the frontoparallel plane, an effect amplified by frontal viewing in premotor cortex. Analysis of single voxels and their discriminatory power showed that the representation of action in the stereo-action-specific areas was more accurate when stereopsis was active. Further analyses showed that the 4 stereo-action-specific sites form a closed network converging onto the premotor node, which connects to parietal and occipitotemporal regions outside the network. Several of the specific sites are known to process vestibular signals, suggesting that the network combines observed actions in peripersonal space with gravitational signals. These findings have wider implications for the function of premotor cortex and the role of stereopsis in human behavior.

Lateral occipitotemporal cortex (LOTC) activity is greatest while viewing dance compared to visualization and movement: learning and expertise effects

The lateral occipitotemporal cortex (LOTC) is comprised of subregions selectively activated by images of human bodies (extrastriate body area, EBA), objects (lateral occipital complex, LO), and motion (MT+). However, their role in motor imagery and movement processing is unclear, as are the influences of learning and expertise on its recruitment. The purpose of our study was to examine putative changes in LOTC activation during action processing following motor learning of novel choreography in professional ballet dancers. Subjects were scanned with functional magnetic resonance imaging up to four times over 34 weeks and performed four tasks: viewing and visualizing a newly learned ballet dance, visualizing a dance that was not being learned, and movement of the foot. EBA, LO, and MT+ were activated most while viewing dance compared to visualization and movement. Significant increases in activation were observed over time in left LO only during visualization of the unlearned dance, and all subregions were activated bilaterally during the viewing task after 34 weeks of performance, suggesting learning-induced plasticity. Finally, we provide novel evidence for modulation of EBA with dance experience during the motor task, with significant activation elicited in a comparison group of novice dancers only. These results provide a composite of LOTC activation during action processing of newly learned ballet choreography and movement of the foot. The role of these areas is confirmed as primarily subserving observation of complex sequences of whole-body movement, with new evidence for modification by experience and over the course of real world ballet learning.

Decoding negative affect personality trait from patterns of brain activation to threat stimuli

INTRODUCTION

Pattern recognition analysis (PRA) applied to functional magnetic resonance imaging (fMRI) has been used to decode cognitive processes and identify possible biomarkers for mental illness. In the present study, we investigated whether the positive affect (PA) or negative affect (NA) personality traits could be decoded from patterns of brain activation in response to a human threat using a healthy sample.

METHODS

fMRI data from 34 volunteers (15 women) were acquired during a simple motor task while the volunteers viewed a set of threat stimuli that were directed either toward them or away from them and matched neutral pictures. For each participant, contrast images from a General Linear Model (GLM) between the threat versus neutral stimuli defined the spatial patterns used as input to the regression model. We applied a multiple kernel learning (MKL) regression combining information from different brain regions hierarchically in a whole brain model to decode the NA and PA from patterns of brain activation in response to threat stimuli.

RESULTS

The MKL model was able to decode NA but not PA from the contrast images between threat stimuli directed away versus neutral with a significance above chance. The correlation and the mean squared error (MSE) between predicted and actual NA were 0.52 (p-value=0.01) and 24.43 (p-value=0.01), respectively. The MKL pattern regression model identified a network with 37 regions that contributed to the predictions. Some of the regions were related to perception (e.g., occipital and temporal regions) while others were related to emotional evaluation (e.g., caudate and prefrontal regions).

CONCLUSION

These results suggest that there was an interaction between the individuals' NA and the brain response to the threat stimuli directed away, which enabled the MKL model to decode NA from the brain patterns. To our knowledge, this is the first evidence that PRA can be used to decode a personality trait from patterns of brain activation during emotional contexts.

Brain Regions Associated to a Kinesthetic Illusion Evoked by Watching a Video of One's Own Moving Hand

It is well known that kinesthetic illusions can be induced by stimulation of several sensory systems (proprioception, touch, vision…). In this study we investigated the cerebral network underlying a kinesthetic illusion induced by visual stimulation by using functional magnetic resonance imaging (fMRI) in humans. Participants were instructed to keep their hand still while watching the video of their own moving hand (Self Hand) or that of someone else's moving hand (Other Hand). In the Self Hand condition they experienced an illusory sensation that their hand was moving whereas the Other Hand condition did not induce any kinesthetic illusion. The contrast between the Self Hand and Other Hand conditions showed significant activation in the left dorsal and ventral premotor cortices, in the left Superior and Inferior Parietal lobules, at the right Occipito-Temporal junction as well as in bilateral Insula and Putamen. Most strikingly, there was no activation in the primary motor and somatosensory cortices, whilst previous studies have reported significant activation in these regions for vibration-induced kinesthetic illusions. To our knowledge, this is the first study that indicates that humans can experience kinesthetic perception without activation in the primary motor and somatosensory areas. We conclude that under some conditions watching a video of one's own moving hand could lead to activation of a network that is usually involved in processing copies of efference, thus leading to the illusory perception that the real hand is indeed moving.

Expressing our internal states and understanding those of others

Vitality form is a term that describes the style with which motor actions are performed (e.g., rude, gentle, etc.). They represent one characterizing element of conscious and unconscious bodily communication. Despite their importance in interpersonal behavior, vitality forms have been, until now, virtually neglected in neuroscience. Here, using the functional MRI (fMRI) technique, we investigated the neural correlates of vitality forms in three different tasks: action observation, imagination, and execution. Conjunction analysis showed that, in all three tasks, there is a common, consistent activation of the dorsocentral sector of the insula. In addition, a common activation of the parietofrontal network, typically active during arm movements production, planning, and observation, was also found. We conclude that the dorsocentral part of the insula is a key element of the system that modulates the cortical motor activity, allowing individuals to express their internal states through action vitality forms. Recent monkey anatomical data show that the dorsocentral sector of the insula is, indeed, connected with the cortical circuit involved in the control of arm movements.

The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study

The present fMRI study examined whether upper-limb action classes differing in their motor goal are encoded by different PPC sectors. Action observation was used as a proxy for action execution. Subjects viewed actors performing object-related (e.g., grasping), skin-displacing (e.g., rubbing the skin), and interpersonal upper limb actions (e.g., pushing someone). Observation of the three action classes activated a three-level network including occipito-temporal, parietal, and premotor cortex. The parietal region common to observing all three action classes was located dorsally to the left intraparietal sulcus (DIPSM/DIPSA border). Regions specific for observing an action class were obtained by combining the interaction between observing action classes and stimulus types with exclusive masking for observing the other classes, while for regions considered preferentially active for a class the interaction was exclusively masked with the regions common to all observed actions. Left putative human anterior intraparietal was specific for observing manipulative actions, and left parietal operculum including putative human SII region, specific for observing skin-displacing actions. Control experiments demonstrated that this latter activation depended on seeing the skin being moved and not simply on seeing touch. Psychophysiological interactions showed that the two specific parietal regions had similar connectivities. Finally, observing interpersonal actions preferentially activated a dorsal sector of left DIPSA, possibly the homologue of ventral intraparietal coding the impingement of the target person's body into the peripersonal space of the actor. These results support the importance of segregation according to the action class as principle of posterior parietal cortex organization for action observation and by implication for action execution.

The lateral occipitotemporal cortex in action

Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.

The neural basis of perceiving person interactions

This study examined whether the grouping of people into meaningful social scenes (e.g., two people having a chat) impacts the basic perceptual analysis of each partaking individual. To explore this issue, we measured neural activity using functional magnetic resonance imaging (fMRI) while participants sex-categorized congruent as well as incongruent person dyads (i.e., two people interacting in a plausible or implausible manner). Incongruent person dyads elicited enhanced neural processing in several high-level visual areas dedicated to face and body encoding and in the posterior middle temporal gyrus compared to congruent person dyads. Incongruent and congruent person scenes were also successfully differentiated by a linear multivariate pattern classifier in the right fusiform body area and the left extrastriate body area. Finally, increases in the person scenes' meaningfulness as judged by independent observers was accompanied by enhanced activity in the bilateral posterior insula. These findings demonstrate that the processing of person scenes goes beyond a mere stimulus-bound encoding of their partaking agents, suggesting that changes in relations between agents affect their representation in category-selective regions of the visual cortex and beyond.

Ventral aspect of the visual form pathway is not critical for the perception of biological motion

Identifying the movements of those around us is fundamental for many daily activities, such as recognizing actions, detecting predators, and interacting with others socially. A key question concerns the neurobiological substrates underlying biological motion perception. Although the ventral "form" visual cortex is standardly activated by biologically moving stimuli, whether these activations are functionally critical for biological motion perception or are epiphenomenal remains unknown. To address this question, we examined whether focal damage to regions of the ventral visual cortex, resulting in significant deficits in form perception, adversely affects biological motion perception. Six patients with damage to the ventral cortex were tested with sensitive point-light display paradigms. All patients were able to recognize unmasked point-light displays and their perceptual thresholds were not significantly different from those of three different control groups, one of which comprised brain-damaged patients with spared ventral cortex (n > 50). Importantly, these six patients performed significantly better than patients with damage to regions critical for biological motion perception. To assess the necessary contribution of different regions in the ventral pathway to biological motion perception, we complement the behavioral findings with a fine-grained comparison between the lesion location and extent, and the cortical regions standardly implicated in biological motion processing. This analysis revealed that the ventral aspects of the form pathway (e.g., fusiform regions, ventral extrastriate body area) are not critical for biological motion perception. We hypothesize that the role of these ventral regions is to provide enhanced multiview/posture representations of the moving person rather than to represent biological motion perception per se.

Can we simulate an action that we temporarily cannot perform?

AIMS OF THE STUDY

The scope of individuals' motor repertoire and expertise influences the way they perceive the actions of others. When observing skilled actions, experts recruit the cortical network subserving action perception (action observation network, AON) to a greater extent than non-experts. However, it remains unknown whether and how a temporary motor injury affects activation within the AON.

MATERIALS AND METHODS

To investigate this issue, brain hemodynamic activity was recorded twice in thirteen national female gymnasts suffering from a lower extremity injury at the onset of the experiment. The gymnasts were scanned one month after the injury and were shown gymnastics routines they were able and temporarily unable to perform. Six months later, after complete recovery, they were scanned again and shown the same routines they were now able to practice.

RESULTS

Results showed: first, that the level of activity within the inferior parietal lobule and MT/V5/EBA (extrastriate body area), areas constitutive of the AON, was independent of the gymnasts' physical condition. Second, when gymnasts were hurt (vs. when recovered), higher activity in the cerebellum was detected.

CONCLUSION

The equal contribution of MT/V5/EBA and inferior parietal lobule during the observation of movements the gymnasts were able or unable to practice suggests respectively that physical provisional incapacity does not interfere with the perceptual processing of body shape and motion information, and that motor expertise may prevent the decay of sensorimotor representations. Higher activations in the cerebellum may suggest that this structure plays a role in dissociating perceived physically feasible movements from those that are provisionally unfeasible.

Fine-grained stimulus representations in body selective areas of human occipito-temporal cortex

Neurophysiological and functional imaging studies have investigated the representation of animate and inanimate stimulus classes in monkey inferior temporal (IT) and human occipito-temporal cortex (OTC). These studies proposed a distributed representation of stimulus categories across IT and OTC and at the same time highlighted category specific modules for the processing of bodies, faces and objects. Here, we investigated whether the stimulus representation within the extrastriate (EBA) and the fusiform (FBA) body areas differed from the representation across OTC. To address this question, we performed an event-related fMRI experiment, evaluating the pattern of activation elicited by 200 individual stimuli that had already been extensively tested in our earlier monkey imaging and single cell studies (Popivanov et al., 2012, 2014). The set contained achromatic images of headless monkey and human bodies, two sets of man-made objects, monkey and human faces, four-legged mammals, birds, fruits, and sculptures. The fMRI response patterns within EBA and FBA primarily distinguished bodies from non-body stimuli, with subtle differences between the areas. However, despite responding on average stronger to bodies than to other categories, classification performance for preferred and non-preferred categories was comparable. OTC primarily distinguished animate from inanimate stimuli. However, cluster analysis revealed a much more fine-grained representation with several homogeneous clusters consisting entirely of stimuli of individual categories. Overall, our data suggest that category representation varies with location within OTC. Nevertheless, body modules contain information to discriminate also non-preferred stimuli and show an increasing specificity in a posterior to anterior gradient.

The transition in the ventral stream from feature to real-world entity representations

We propose that the ventral visual pathway of human and non-human primates is organized into three levels: (1) ventral retinotopic cortex including what is known as TEO in the monkey but corresponds to V4A and PITd/v, and the phPIT cluster in humans, (2) area TE in the monkey and its homolog LOC and neighboring fusiform regions, and more speculatively, (3) TGv in the monkey and its possible human equivalent, the temporal pole. We attribute to these levels the visual representations of features, partial real-world entities (RWEs), and known, complete RWEs, respectively. Furthermore, we propose that the middle level, TE and its homolog, is organized into three parallel substreams, lower bank STS, dorsal convexity of TE, and ventral convexity of TE, as are their corresponding human regions. These presumably process shape in depth, 2D shape and material properties, respectively, to construct RWE representations.

Correspondences between retinotopic areas and myelin maps in human visual cortex

We generated probabilistic area maps and maximum probability maps (MPMs) for a set of 18 retinotopic areas previously mapped in individual subjects (Georgieva et al., 2009 and Kolster et al., 2010) using four different inter-subject registration methods. The best results were obtained using a recently developed multimodal surface matching method. The best set of MPMs had relatively smooth borders between visual areas and group average area sizes that matched the typical size in individual subjects. Comparisons between retinotopic areas and maps of estimated cortical myelin content revealed the following correspondences: (i) areas V1, V2, and V3 are heavily myelinated; (ii) the MT cluster is heavily myelinated, with a peak near the MT/pMSTv border; (iii) a dorsal myelin density peak corresponds to area V3D; (iv) the phPIT cluster is lightly myelinated; and (v) myelin density differs across the four areas of the V3A complex. Comparison of the retinotopic MPM with cytoarchitectonic areas, including those previously mapped to the fs_LR cortical surface atlas, revealed a correspondence between areas V1–3 and hOc1–3, respectively, but little correspondence beyond V3. These results indicate that architectonic and retinotopic areal boundaries are in agreement in some regions, and that retinotopy provides a finer-grained parcellation in other regions. The atlas datasets from this analysis are freely available as a resource for other studies that will benefit from retinotopic and myelin density map landmarks in human visual cortex.

•

Maximum probability maps for 18 retinotopic areas were generated.

•

Multimodal surface matching was used to compare with myelin and cytoarchitectonic maps.

•

Early areas V1–3 areas are heavily myelinated, as are V3D and most of areas MT/pMSTv.

•

The phPIT cluster is lightly myelinated compared to other retinotopic areas.

•

Early areas V1–3 correspond to areas hOc1–3, with little correspondence beyond V3.

Hands in motion: an upper-limb-selective area in the occipitotemporal cortex shows sensitivity to viewed hand kinematics

Regions in the occipitotemporal cortex (OTC) show clear selectivity to static images of human body parts, and upper limbs in particular, with respect to other object categories. Such selectivity was previously attributed to shape aspects, which presumably vary across categories. Alternatively, it has been proposed that functional selectivity for upper limbs is driven by processing of their distinctive motion features. In the present study we show that selectivity to static upper-limb images and motion processing go hand in hand. Using resting-state and task-based functional MRI, we demonstrate that OTC voxels showing greater preference to static images of arms and hands also show stronger functional connectivity with motion coding regions within the human middle temporal complex (hMT+), but not with shape-selective midtier areas, such as hV4 or LO-1, suggesting a tight link between upper-limb selectivity and motion processing. To test this directly, we created a set of natural arm-movement videos where kinematic patterns were parametrically manipulated, while keeping shape information constant. Using multivariate pattern analysis, we show that the degree of (dis)similarity in arm-velocity profiles across the video set predicts, to a significant extent, the degree of (dis)similarity in multivoxel activation patterns in both upper-limb-selective OTC regions and the hMT+. Together, these results suggest that the functional specificity of upper-limb-selective regions may be partially determined by their involvement in the processing of upper-limb dynamics. We propose that the selectivity to static upper-limb images in the OTC may be a result of experience-dependent association between shape elements, which characterize upper limbs, and upper-limb-specific motion patterns.

Distinct neural mechanisms for body form and body motion discriminations

Actions can be understood based on form cues (e.g., static body posture) as well as motion cues (e.g., gait patterns). A fundamental debate centers on the question of whether the functional and neural mechanisms processing these two types of cues are dissociable. Here, using fMRI, psychophysics, and transcranial magnetic stimulation (TMS), all within the same human participants, we show that mechanisms underlying body form and body motion processing are functionally and neurally distinct. Multivoxel fMRI activity patterns in the extrastriate body area (EBA), but not in the posterior superior temporal sulcus (pSTS), carried cue invariant information about the body form of an acting human. Conversely, multivoxel patterns in pSTS, but not in EBA, carried information about the body motion of the same actor. In a psychophysical experiment, we selectively impaired body form and body motion discriminations by manipulating different visual cues: misaligning the ellipses that made up a dynamic walker stimulus selectively disrupted body form discriminations, while varying the presentation duration of the walker selectively affected body motion discriminations. Finally, a TMS experiment revealed causal evidence for a double-dissociation between neural mechanisms underlying body form and body motion discriminations: TMS over EBA selectively disrupted body form discrimination, whereas TMS over pSTS selectively disrupted body motion discrimination. Together, these findings reveal complementing but dissociable functions of EBA and pSTS during action perception. They provide constraints for theoretical and computational models of action perception by showing that action perception involves at least two parallel pathways that separately contribute to the understanding of others' behavior.

Electrophysiological markers of biological motion and human form recognition

Current models suggest that human form and motion information are initially processed through separate pathways, then integrated in action perception. Testing such a sequential model requires techniques with high temporal resolution. Prior work demonstrated sensitivity of a posterior temporal event-related potential (ERP) effect - the N2 - to biological motion, but did not test whether the N2 indexes biological motion perception specifically, or human form/action perception more generally. We recorded ERPs while participants viewed stimuli across 3 blocks: (1) static (non-moving) point-light displays of humans performing actions; (2) static stick figures with clear forms; and (3) point-light biological motion. A similar sequence of ERP components was elicited by human forms in all blocks (stationary and moving), and reliably discriminated between human and scrambled forms. The N2 showed similar scalp distribution and sensitivity to stimulus manipulations for both stick figures and biological motion, suggesting that it indexes integration of form and motion information, rather than biological motion perception exclusively - and that form and motion information are therefore integrated by approximately 200ms. We identified a component subsequent to the N2, which we label the medial parietal positivity/ventral-anterior negativity (MPP/VAN), that was also sensitive to both human form and motion information. We propose that the MPP/VAN reflects higher-order human action recognition that occurs subsequent to the integration of form and motion information reflected by the N2.

Interactions between motion and form processing in the human visual system

The predominant view of motion and form processing in the human visual system assumes that these two attributes are handled by separate and independent modules. Motion processing involves filtering by direction-selective sensors, followed by integration to solve the aperture problem. Form processing involves filtering by orientation-selective and size-selective receptive fields, followed by integration to encode object shape. It has long been known that motion signals can influence form processing in the well-known Gestalt principle of common fate; texture elements which share a common motion property are grouped into a single contour or texture region. However, recent research in psychophysics and neuroscience indicates that the influence of form signals on motion processing is more extensive than previously thought. First, the salience and apparent direction of moving lines depends on how the local orientation and direction of motion combine to match the receptive field properties of motion-selective neurons. Second, orientation signals generated by "motion-streaks" influence motion processing; motion sensitivity, apparent direction and adaptation are affected by simultaneously present orientation signals. Third, form signals generated by human body shape influence biological motion processing, as revealed by studies using point-light motion stimuli. Thus, form-motion integration seems to occur at several different levels of cortical processing, from V1 to STS.