The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study

The cognitive structure underlying the organization of observed actions

In daily life, we frequently encounter actions performed by other people. Here we aimed to examine the key categories and features underlying the organization of a wide range of actions in three behavioral experiments (N = 378 participants). In Experiment 1, we used a multi-arrangement task of 100 different actions. Inverse multidimensional scaling and hierarchical clustering revealed 11 action categories, including Locomotion, Communication, and Aggressive actions. In Experiment 2, we used a feature-listing paradigm to obtain a wide range of action features that were subsequently reduced to 59 key features and used in a rating study (Experiment 3). A direct comparison of the feature ratings obtained in Experiment 3 between actions belonging to the categories identified in Experiment 1 revealed a number of features that appear to be critical for the distinction between these categories, e.g., the features Harm and Noise for the category Aggressive actions, and the features Targeting a person and Contact with others for the category Interaction. Finally, we found that a part of the category-based organization is explained by a combination of weighted features, whereas a significant proportion of variability remained unexplained, suggesting that there are additional sources of information that contribute to the categorization of observed actions. The characterization of action categories and their associated features serves as an important extension of previous studies examining the cognitive structure of actions. Moreover, our results may serve as the basis for future behavioral, neuroimaging and computational modeling studies.

Two distinct networks containing position-tolerant representations of actions in the human brain

Humans can recognize others' actions in the social environment. This action recognition ability is rarely hindered by the movement of people in the environment. The neural basis of this position tolerance for observed actions is not fully understood. Here, we aimed to identify brain regions capable of generalizing representations of actions across different positions and investigate the representational content of these regions. In a functional magnetic resonance imaging experiment, participants viewed point-light displays of different human actions. Stimuli were presented in either the upper or the lower visual field. Multivariate pattern analysis and a surface-based searchlight approach were employed to identify brain regions that contain position-tolerant action representation: Classifiers were trained with patterns in response to stimuli presented in one position and were tested with stimuli presented in another position. Results showed above-chance classification in the left and right lateral occipitotemporal cortices, right intraparietal sulcus, and right postcentral gyrus. Further analyses exploring the representational content of these regions showed that responses in the lateral occipitotemporal regions were more related to subjective judgments, while those in the parietal regions were more related to objective measures. These results provide evidence for two networks that contain abstract representations of human actions with distinct representational content.

A Large Video Set of Natural Human Actions for Visual and Cognitive Neuroscience Studies and Its Validation with fMRI

The investigation of the perception of others' actions and underlying neural mechanisms has been hampered by the lack of a comprehensive stimulus set covering the human behavioral repertoire. To fill this void, we present a video set showing 100 human actions recorded in natural settings, covering the human repertoire except for emotion-driven (e.g., sexual) actions and those involving implements (e.g., tools). We validated the set using fMRI and showed that observation of the 100 actions activated the well-established action observation network. We also quantified the videos' low-level visual features (luminance, optic flow, and edges). Thus, this comprehensive video set is a valuable resource for perceptual and neuronal studies.

Task-Dependent Warping of Semantic Representations during Search for Visual Action Categories

Object and action perception in cluttered dynamic natural scenes relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. It has been suggested that during visual search for objects, distributed semantic representation of hundreds of object categories is warped to expand the representation of targets. Yet, little is known about whether and where in the brain visual search for action categories modulates semantic representations. To address this fundamental question, we studied brain activity recorded from five subjects (one female) via functional magnetic resonance imaging while they viewed natural movies and searched for either communication or locomotion actions. We find that attention directed to action categories elicits tuning shifts that warp semantic representations broadly across neocortex and that these shifts interact with intrinsic selectivity of cortical voxels for target actions. These results suggest that attention serves to facilitate task performance during social interactions by dynamically shifting semantic selectivity toward target actions and that tuning shifts are a general feature of conceptual representations in the brain.SIGNIFICANCE STATEMENT The ability to swiftly perceive the actions and intentions of others is a crucial skill for humans that relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. However, little is known about the nature of high-level semantic representations during natural visual search for action categories. Here, we provide the first evidence showing that attention significantly warps semantic representations by inducing tuning shifts in single cortical voxels, broadly spread across occipitotemporal, parietal, prefrontal, and cingulate cortices. This dynamic attentional mechanism can facilitate action perception by efficiently allocating neural resources to accentuate the representation of task-relevant action categories.

Vision of haptics tunes the somatosensory threshold

The interaction between different sensory modalities represents a crucial issue in the neuroscience of consciousness: when the processing of one modality is deficient, the concomitant presentation of stimuli of other spared modalities may sustain the restoring of the damaged sensory functions. In this regard, visual enhancement of touch may represent a viable tool in the rehabilitation from tactile disorders, yet the specific visual features mostly modulating the somatosensory experience remain unsettled. In this study, healthy subjects underwent a tactile detection task during the observation of videos displaying different contents, including static gratings, meaningless motions, natural or point-lights reach-to-grasp-and-manipulate actions. Concurrently, near-threshold stimuli were delivered to the median nerve at different time-points. Subjective report was collected after each trial; the sensory detection rate was computed and compared across video conditions. Our results indicate that the specific presence of haptic contents (i.e., vision of manipulation), either fully displayed or implied by point-lights, magnifies tactile sensitivity. The notion that such stimuli prompt an aware tactile experience opens to novel rehabilitation approaches for tactile consciousness disorders.

Moving in on human motor cortex. Characterizing the relationship between body parts with non-rigid population response fields

For cortical motor activity, the relationships between different body part representations is unknown. Through reciprocal body part relationships, functionality of cortical motor areas with respect to whole body motor control can be characterized. In the current study, we investigate the relationship between body part representations within individual neuronal populations in motor cortices, following a 7 Tesla fMRI 18-body-part motor experiment in combination with our newly developed non-rigid population Response Field (pRF) model and graph theory. The non-rigid pRF metrics reveal somatotopic structures in all included motor cortices covering frontal, parietal, medial and insular cortices and that neuronal populations in primary sensorimotor cortex respond to fewer body parts than secondary motor cortices. Reciprocal body part relationships are estimated in terms of uniqueness, clique-formation, and influence. We report unique response profiles for the knee, a clique of body parts surrounding the ring finger, and a central role for the shoulder and wrist. These results reveal associations among body parts from the perspective of the central nervous system, while being in agreement with intuitive notions of body part usage.

The macaque ventral intraparietal area has expanded into three homologue human parietal areas

The macaque ventral intraparietal area (VIP) in the fundus of the intraparietal sulcus has been implicated in a diverse range of sensorimotor and cognitive functions such as motion processing, multisensory integration, processing of head peripersonal space, defensive behavior, and numerosity coding. Here, we exhaustively review macaque VIP function, cytoarchitectonics, and anatomical connectivity and integrate it with human studies that have attempted to identify a potential human VIP homologue. We show that human VIP research has consistently identified three, rather than one, bilateral parietal areas that each appear to subsume some, but not all, of the macaque area's functionality. Available evidence suggests that this human "VIP complex" has evolved as an expansion of the macaque area, but that some precursory specialization within macaque VIP has been previously overlooked. The three human areas are dominated, roughly, by coding the head or self in the environment, visual heading direction, and the peripersonal environment around the head, respectively. A unifying functional principle may be best described as prediction in space and time, linking VIP to state estimation as a key parietal sensorimotor function. VIP's expansive differentiation of head and self-related processing may have been key in the emergence of human bodily self-consciousness.

Parietal maps of visual signals for bodily action planning

The posterior parietal cortex (PPC) has long been understood as a high-level integrative station for computing motor commands for the body based on sensory (i.e., mostly tactile and visual) input from the outside world. In the last decade, accumulating evidence has shown that the parietal areas not only extract the pragmatic features of manipulable objects, but also subserve sensorimotor processing of others’ actions. A paradigmatic case is that of the anterior intraparietal area (AIP), which encodes the identity of observed manipulative actions that afford potential motor actions the observer could perform in response to them. On these bases, we propose an AIP manipulative action-based template of the general planning functions of the PPC and review existing evidence supporting the extension of this model to other PPC regions and to a wider set of actions: defensive and locomotor actions. In our model, a hallmark of PPC functioning is the processing of information about the physical and social world to encode potential bodily actions appropriate for the current context. We further extend the model to actions performed with man-made objects (e.g., tools) and artifacts, because they become integral parts of the subject’s body schema and motor repertoire. Finally, we conclude that existing evidence supports a generally conserved neural circuitry that transforms integrated sensory signals into the variety of bodily actions that primates are capable of preparing and performing to interact with their physical and social world.

The unique role of parietal cortex in action observation: Functional organization for communicative and manipulative actions

Action observation is supported by a network of regions in occipito-temporal, parietal, and premotor cortex in primates. Recent research suggests that the parietal node has regions dedicated to different action classes including manipulation, interpersonal interactions, skin displacement, locomotion, and climbing. The goals of the current study consist of: 1) extending this work with new classes of actions that are communicative and specific to humans, 2) investigating how parietal cortex differs from the occipito-temporal and premotor cortex in representing action classes. Human subjects underwent fMRI scanning while observing three action classes: indirect communication, direct communication, and manipulation, plus two types of control stimuli, static controls which were static frames from the video clips, and dynamic controls consisting of temporally-scrambled optic flow information. Using univariate analysis, MVPA, and representational similarity analysis, our study presents several novel findings. First, we provide further evidence for the anatomical segregation in parietal cortex of different action classes: We have found a new site that is specific for representing human-specific indirect communicative actions in cytoarchitectonic parietal area PFt. Second, we found that the discriminability between action classes was higher in parietal cortex than the other two levels suggesting the coding of action identity information at this level. Finally, our results advocate the use of the control stimuli not just for univariate analysis of complex action videos but also when using multivariate techniques.

The role of mirror mechanism in the recovery, maintenance, and acquisition of motor abilities

While it is well documented that the motor system is more than a mere implementer of motor actions, the possible applications of its cognitive side are still under-exploited, often remaining as poorly organized evidence. Here, we will collect evidence showing the value of action observation treatment (AOT) in the recovery of impaired motor abilities for a vast number of clinical conditions, spanning from traumatological patients to brain injuries and neurodegenerative diseases. Alongside, we will discuss the use of AOT in the maintenance of appropriate motor behavior in subjects at risk for events with dramatic physical consequences, like fall prevention in elderly people or injury prevention in sports. Finally, we will report that AOT can help to tune existing motor competencies in fields requiring precise motor control. We will connect all these diverse dots into the neurophysiological scenario offered by decades of research on the human mirror mechanism, discussing the potentialities for individualization. Empowered by modern technologies, AOT can impact individuals' safety and quality of life across the whole lifespan.

From Observed Action Identity to Social Affordances

Others' observed actions cause continuously changing retinal images, making it challenging to build neural representations of action identity. The monkey anterior intraparietal area (AIP) and its putative human homologue (phAIP) host neurons selective for observed manipulative actions (OMAs). The neuronal activity of both AIP and phAIP allows a stable readout of OMA identity across visual formats, but human neurons exhibit greater invariance and generalize from observed actions to action verbs. These properties stem from the convergence in AIP of superior temporal signals concerning: (i) observed body movements; and (ii) the changes in the body-object relationship. We propose that evolutionarily preserved mechanisms underlie the specification of observed-actions identity and the selection of motor responses afforded by them, thereby promoting social behavior.

Size constancy affects the perception and parietal neural representation of object size

Humans and animals rely on accurate object size perception to guide behavior. Object size is judged from visual input, but the relationship between an object's retinal size and its real-world size varies with distance. Humans perceive object sizes to be relatively constant when retinal size changes. Such size constancy compensates for the variable relationship between retinal size and real-world size, using the context of recent retinal sizes of the same object to bias perception towards its likely real-world size. We therefore hypothesized that object size perception may be affected by the range of recently viewed object sizes, attracting perceived object sizes towards recently viewed sizes. We demonstrate two systematic biases: a central tendency attracting perceived size towards the average size across all trials, and a serial dependence attracting perceived size towards the size presented on the previous trial. We recently described topographic object size maps in the human parietal cortex. We therefore hypothesized that neural representations of object size here would be attracted towards recently viewed sizes. We used ultra-high-field (7T) functional MRI and population receptive field modeling to compare object size representations measured with small (0.05-1.4°diameter) and large objects sizes (0.1-2.8°). We found that parietal object size preferences and tuning widths follow this presented range, but change less than presented object sizes. Therefore, perception and neural representation of object size are attracted towards recently viewed sizes. This context-dependent object size representation reveals effects on neural response preferences that may underlie context dependence of object size perception.

A shared neural substrate for action verbs and observed actions in human posterior parietal cortex

High-level sensory and motor cortical areas are activated when processing the meaning of language, but it is unknown whether, and how, words share a neural substrate with corresponding sensorimotor representations. We recorded from single neurons in human posterior parietal cortex (PPC) while participants viewed action verbs and corresponding action videos from multiple views. We find that PPC neurons exhibit a common neural substrate for action verbs and observed actions. Further, videos were encoded with mixtures of invariant and idiosyncratic responses across views. Action verbs elicited selective responses from a fraction of these invariant and idiosyncratic neurons, without preference, thus associating with a statistical sampling of the diverse sensory representations related to the corresponding action concept. Controls indicated that the results are not the product of visual imagery or arbitrary learned associations. Our results suggest that language may activate the consolidated visual experience of the reader.

A parietal region processing numerosity of observed actions: An FMRI study

When observing others' behavior, it is important to perceive not only the identity of the observed actions (OAs), but also the number of times they were performed. Given the mounting evidence implicating posterior parietal cortex in action observation, and in particular that of manipulative actions, the aim of this study was to identify the parietal region, if any, that contributes to the processing of observed manipulative action (OMA) numerosity, using the functional magnetic resonance imaging technique. Twenty‐one right‐handed healthy volunteers performed two discrimination tasks while in the scanner, responding to video stimuli in which an actor performed manipulative actions on colored target balls that appeared four times consecutively. The subjects discriminated between two small numerosities of either OMAs (“Action” condition) or colors of balls (“Ball” condition). A significant difference between the “Action” and “Ball” conditions was observed in occipito‐temporal cortex and the putative human anterior intraparietal sulcus (phAIP) area as well as the third topographic map of numerosity‐selective neurons at the post‐central sulcus (NPC3) of the left parietal cortex. A further region of interest analysis of the group‐average data showed that at the single voxel level the latter area, more than any other parietal or occipito‐temporal numerosity map, favored numerosity of OAs. These results suggest that phAIP processes the identity of OMAs, while neighboring NPC3 likely processes the numerosity of the identified OAs.

Stable readout of observed actions from format-dependent activity of monkey's anterior intraparietal neurons

The anterior intraparietal area (AIP) is a crucial hub in the observed manipulative action (OMA) network of primates. While macaques observe manipulative action videos, their AIP neuronal activity robustly encodes first the viewpoint from which the action is observed, then the actor’s body posture, and finally the observed-action identity. Despite the lack of fully invariant OMA-selective single neurons, OMA exemplars could be decoded accurately from the activity of a set of units that maintain stable OMA selectivity despite rescaling their firing rate across formats. We propose that by integrating signals multiplicatively about others’ action and their visual format, the AIP can provide a stable readout of OMA identity at the population level.

Humans accurately identify observed actions despite large dynamic changes in their retinal images and a variety of visual presentation formats. A large network of brain regions in primates participates in the processing of others’ actions, with the anterior intraparietal area (AIP) playing a major role in routing information about observed manipulative actions (OMAs) to the other nodes of the network. This study investigated whether the AIP also contributes to invariant coding of OMAs across different visual formats. We recorded AIP neuronal activity from two macaques while they observed videos portraying seven manipulative actions (drag, drop, grasp, push, roll, rotate, squeeze) in four visual formats. Each format resulted from the combination of two actor’s body postures (standing, sitting) and two viewpoints (lateral, frontal). Out of 297 recorded units, 38% were OMA-selective in at least one format. Robust population code for viewpoint and actor’s body posture emerged shortly after stimulus presentation, followed by OMA selectivity. Although we found no fully invariant OMA-selective neuron, we discovered a population code that allowed us to classify action exemplars irrespective of the visual format. This code depends on a multiplicative mixing of signals about OMA identity and visual format, particularly evidenced by a set of units maintaining a relatively stable OMA selectivity across formats despite considerable rescaling of their firing rate depending on the visual specificities of each format. These findings suggest that the AIP integrates format-dependent information and the visual features of others’ actions, leading to a stable readout of observed manipulative action identity.

Anterior Intraparietal Area: A Hub in the Observed Manipulative Action Network

Current knowledge regarding the processing of observed manipulative actions (OMAs) (e.g., grasping, dragging, or dropping) is limited to grasping and underlying neural circuitry remains controversial. Here, we addressed these issues by combining chronic neuronal recordings along the anteroposterior extent of monkeys’ anterior intraparietal (AIP) area with tracer injections into the recorded sites. We found robust neural selectivity for 7 distinct OMAs, particularly in the posterior part of AIP (pAIP), where it was associated with motor coding of grip type and own-hand visual feedback. This cluster of functional properties appears to be specifically grounded in stronger direct connections of pAIP with the temporal regions of the ventral visual stream and the prefrontal cortex, as connections with skeletomotor related areas and regions of the dorsal visual stream exhibited opposite or no rostrocaudal gradients. Temporal and prefrontal areas may provide visual and contextual information relevant for manipulative action processing. These results revise existing models of the action observation network, suggesting that pAIP constitutes a parietal hub for routing information about OMA identity to the other nodes of the network.

Action execution and action observation elicit mirror responses with the same temporal profile in human SII

The properties of the secondary somatosensory area (SII) have been described by many studies in monkeys and humans. Recent studies on monkeys, however, showed that beyond somatosensory stimuli, SII responds to a wider number of stimuli, a finding requiring a revision that human SII is purely sensorimotor. By recording cortical activity with stereotactic electroencephalography (stereo-EEG), we examined the properties of SI and SII in response to a motor task requiring reaching, grasping and manipulation, as well as the observation of the same actions. Furthermore, we functionally characterized this area with a set of clinical tests, including tactile, acoustical, and visual stimuli. The results showed that only SII activates both during execution and observation with a common temporal profile, whereas SI response were limited to execution. Together with their peculiar response to tactile stimuli, we conclude that the role of SII is pivotal also in the observation of actions involving haptic control.

The role of putative human anterior intraparietal sulcus area in observed manipulative action discrimination

INTRODUCTION

Although it has become widely accepted that the action observation network (AON) includes three levels (occipito-temporal, parietal and premotor), little is known concerning the specific role of these levels within perceptual tasks probing action observation. Recent single cell studies suggest that the parietal level carries the information required to discriminate between two-alternative observed actions, but do not exclude possible contributions from the other two levels.

METHODS

Two functional magnetic resonance imaging experiments used a task-based attentional modulation paradigm in which subjects viewed videos of an actor performing a manipulative action on a coloured object, and discriminated between either two observed manipulative actions, two actors or two colours.

RESULTS

Both experiments demonstrated that relative to actor and colour discrimination, discrimination between observed manipulative actions involved the putative human anterior intraparietal sulcus (phAIP) area in parietal cortex. In one experiment, where the observed actions also differed with regard to effectors, premotor cortex was also specifically recruited.

CONCLUSIONS

Our results highlight the primary role of parietal cortex in discriminating between two-alternative observed manipulative actions, consistent with the view that this level plays a major role in representing the identity of an observed action.

Cytoarchitectonic segregation of human posterior intraparietal and adjacent parieto-occipital sulcus and its relation to visuomotor and cognitive functions

Human posterior intraparietal sulcus (pIPS) and adjacent posterior wall of parieto-occipital sulcus (POS) are functionally diverse, serving higher motor, visual and cognitive functions. Its microstructural basis, though, is still largely unknown. A similar or even more pronounced architectonical complexity, as described in monkeys, could be assumed. We cytoarchitectonically mapped the pIPS/POS in 10 human postmortem brains using an observer-independent, quantitative parcellation. 3D-probability maps were generated within MNI reference space and used for functional decoding and meta-analytic coactivation modeling based on the BrainMap database to decode the general structural-functional organization of the areas. Seven cytoarchitectonically distinct areas were identified: five within human pIPS, three on its lateral (hIP4-6) and two on its medial wall (hIP7-8); and two (hPO1, hOc6) in POS. Mediocaudal areas (hIP7, hPO1) were predominantly involved in visual processing, whereas laterorostral areas (hIP4-6, 8) were associated with higher cognitive functions, e.g. counting. This shift was mirrored by systematic changes in connectivity, from temporo-occipital to premotor and prefrontal cortex, and in cytoarchitecture, from prominent Layer IIIc pyramidal cells to homogeneous neuronal distribution. This architectonical mosaic within human pIPS/POS represents a structural basis of its functional and connectional heterogeneity. The new 3D-maps of the areas enable dedicated assessments of structure-function relationships.

Distinct representations in occipito-temporal, parietal, and premotor cortex during action perception revealed by fMRI and computational modeling

Visual processing of actions is supported by a network consisting of occipito-temporal, parietal, and premotor regions in the human brain, known as the Action Observation Network (AON). In the present study, we investigate what aspects of visually perceived actions are represented in this network using fMRI and computational modeling. Human subjects performed an action perception task during scanning. We characterized the different aspects of the stimuli starting from purely visual properties such as form and motion to higher-aspects such as intention using computer vision and categorical modeling. We then linked the models of the stimuli to the three nodes of the AON with representational similarity analysis. Our results show that different nodes of the network represent different aspects of actions. While occipito-temporal cortex performs visual analysis of actions by means of integrating form and motion information, parietal cortex builds on these visual representations and transforms them into more abstract and semantic representations coding target of the action, action type and intention. Taken together, these results shed light on the neuro-computational mechanisms that support visual perception of actions and provide support that AON is a hierarchical system in which increasing levels of the cortex code increasingly complex features.

Functional MRI Responses to Passive, Active, and Observed Touch in Somatosensory and Insular Cortices of the Macaque Monkey

Neurophysiological data obtained in primates suggests that merely observing others' actions can modulate activity in the observer's motor cortices. In humans, it has been suggested that these multimodal vicarious responses extend well beyond the motor cortices, including somatosensory and insular brain regions, which seem to yield vicarious responses when witnessing others' actions, sensations, or emotions (Gazzola and Keysers, 2009). Despite the wealth of data with respect to shared action responses in the monkey motor system, whether the somatosensory and insular cortices also yield vicarious responses during observation of touch remains largely unknown. Using independent tactile and motor fMRI localizers, we first mapped the hand representations of two male monkeys' primary (SI) and secondary (SII) somatosensory cortices. In two subsequent visual experiments, we examined fMRI brain responses to (1) observing a conspecific's hand being touched or (2) observing a human hand grasping or mere touching an object or another human hand. Whereas functionally defined "tactile SI" and "tactile SII" showed little involvement in representing observed touch, vicarious responses for touch were found in parietal area PFG, consistent with recent observations in humans (Chan and Baker, 2015). Interestingly, a more anterior portion of SII, and posterior insular cortex, both of which responded when monkeys performed active grasping movements, also yielded visual responses during different instances of touch observation.SIGNIFICANCE STATEMENT Common coding of one's own and others' actions, sensations, and emotions seems to be widespread in the brain. Although it is currently unclear to what extent human somatosensory cortices yield vicarious responses when observing touch, even less is known about the presence of similar vicarious responses in monkey somatosensory cortex. We therefore localized monkey somatosensory hand representations using fMRI and investigated whether these regions yield vicarious responses while observing various instances of touch. Whereas "tactile SI and SII" did not elicit responses during touch observation, a more anterior portion of SII, in addition to area PFG and posterior insular cortex, all of which responded during monkeys' own grasping movements, yielded vicarious responses during observed touch.

The mirror mechanism in the parietal lobe

The mirror mechanism is a basic mechanism that transforms sensory representations of others' actions into motor representations of the same actions in the brain of the observer. The mirror mechanism plays an important role in understanding actions of others. In the present chapter we discuss first the basic organization of the posterior parietal lobe in the monkey, stressing that it is best characterized as a motor scaffold, on the top of which sensory information is organized. We then describe the location of the mirror mechanism in the posterior parietal cortex of the monkey, and its functional role in areas PFG, and anterior, ventral, and lateral intraparietal areas. We will then present evidence that a similar functional organization is present in humans. We will conclude by discussing the role of the mirror mechanism in the recognition of action performed with tools.

Not all observed actions are perceived equally

Action observation is the visual process analyzing the actions of others to determine their goals and how the actor's body (part) movements permit attaining those goals. Our recent psychophysical study demonstrated that 1) observed action (OA) perception differs from shape perception in viewpoint and duration dependence, and 2) accuracy and reaction times of OA discrimination are fitted by the proportional-rate diffusion model whereby a sensory stage provides noisy evidence that is accumulated up to a criterion or bound by a decision stage. That study was devoted to observation of manipulative actions, following a general trend of the field. Recent functional imaging studies of action observation, however, have established various OA classes as separate entities with processing routes involving distinct posterior parietal cortex (PPC) regions. Here, we show that the diffusion model applies to multiple OA classes. Even more importantly, the observers' ability to discriminate exemplars of a given class differs considerably between OA classes and these performance differences correspond to differences in model parameters. In particular, OA classes differ in the bound parameter which we propose may reflect an urgency signal originating in the PPC regions corresponding to the sensory stages of different OA classes.

Mapping the complex topological organization of the human parietal face area

The macaque monkey ventral intraparietal area (VIP) contains neurons with aligned visual-tactile receptive fields anchored to the face and upper body. Our previous fMRI studies using standard head coils found a human parietal face area (VIP+ complex; putative macaque VIP homologue) containing superimposed topological maps of the face and near-face visual space. Here, we construct high signal-to-noise surface coils and used phase-encoded air puffs and looming stimuli to map topological organization of the parietal face area at higher resolution. This area is consistently identified as a region extending between the superior postcentral sulcus and the upper bank of the anterior intraparietal sulcus (IPS), avoiding the fundus of IPS. Using smaller voxel sizes, our surface coils picked up strong fMRI signals in response to tactile and visual stimuli. By analyzing tactile and visual maps in our current and previous studies, we constructed a set of topological models illustrating commonalities and differences in map organization across subjects. The most consistent topological feature of the VIP+ complex is a central-anterior upper face (and upper visual field) representation adjoined by lower face (and lower visual field) representations ventrally (laterally) and/or dorsally (medially), potentially forming two subdivisions VIPv (ventral) and VIPd (dorsal). The lower visual field representations typically extend laterally into the anterior IPS to adjoin human area AIP, and medially to overlap with the parietal body areas at the superior parietal ridge. Significant individual variations are then illustrated to provide an accurate and comprehensive view of the topological organization of the parietal face area.

Observing Others Speak or Sing Activates Spt and Neighboring Parietal Cortex

To obtain further evidence that action observation can serve as a proxy for action execution and planning in posterior parietal cortex, we scanned participants while they were (1) observing two classes of action: vocal communication and oral manipulation, which share the same effector but differ in nature, and (2) rehearsing and listening to nonsense sentences to localize area Spt, thought to be involved in audio-motor transformation during speech. Using this localizer, we found that Spt is specifically activated by vocal communication, indicating that Spt is not only involved in planning speech but also in observing vocal communication actions. In addition, we observed that Spt is distinct from the parietal region most specialized for observing vocal communication, revealed by an interaction contrast and located in PFm. The latter region, unlike Spt, processes the visual and auditory signals related to other's vocal communication independently. Our findings are consistent with the view that several small regions in the temporoparietal cortex near the ventral part of the supramarginal/angular gyrus border are involved in the planning of vocal communication actions and are also concerned with observation of these actions, though involvements in those two aspects are unequal.

Neural Representations of Observed Actions Generalize across Static and Dynamic Visual Input

People interact with entities in the environment in distinct and categorizable ways (e.g., kicking is making contact with foot). We can recognize these action categories across variations in actors, objects, and settings; moreover, we can recognize them from both dynamic and static visual input. However, the neural systems that support action recognition across these perceptual differences are unclear. Here, we used multivoxel pattern analysis of fMRI data to identify brain regions that support visual action categorization in a format-independent way. Human participants were scanned while viewing eight categories of interactions (e.g., pulling) depicted in two visual formats: (1) visually controlled videos of two interacting actors and (2) visually varied photographs selected from the internet involving different actors, objects, and settings. Action category was decodable across visual formats in bilateral inferior parietal, bilateral occipitotemporal, left premotor, and left middle frontal cortex. In most of these regions, the representational similarity of action categories was consistent across subjects and visual formats, a property that can contribute to a common understanding of actions among individuals. These results suggest that the identified brain regions support action category codes that are important for action recognition and action understanding.SIGNIFICANCE STATEMENT Humans tend to interpret the observed actions of others in terms of categories that are invariant to incidental features: whether a girl pushes a boy or a button and whether we see it in real-time or in a single snapshot, it is still pushing Here, we investigated the brain systems that facilitate the visual recognition of these action categories across such differences. Using fMRI, we identified several areas of parietal, occipitotemporal, and frontal cortex that exhibit action category codes that are similar across viewing of dynamic videos and still photographs. Our results provide strong evidence for the involvement of these brain regions in recognizing the way that people interact physically with objects and other people.

Action observation: the less-explored part of higher-order vision

Little is presently known about action observation, an important perceptual component of high-level vision. To investigate this aspect of perception, we introduce a two-alternative forced-choice task for observed manipulative actions while varying duration or signal strength by noise injection. We show that accuracy and reaction time in this task can be modeled by a diffusion process for different pairs of action exemplars. Furthermore, discrimination of observed actions is largely viewpoint-independent, cannot be reduced to judgments about the basic components of action: shape and local motion, and requires a minimum duration of about 150–200 ms. These results confirm that action observation is a distinct high-level aspect of visual perception based on temporal integration of visual input generated by moving body parts. This temporal integration distinguishes it from object or scene perception, which require only very brief presentations and are viewpoint-dependent. The applicability of a diffusion model suggests that these aspects of high-level vision differ mainly at the level of the sensory neurons feeding the decision processes.

Stereoscopically Observing Manipulative Actions

The purpose of this study was to investigate the contribution of stereopsis to the processing of observed manipulative actions. To this end, we first combined the factors "stimulus type" (action, static control, and dynamic control), "stereopsis" (present, absent) and "viewpoint" (frontal, lateral) into a single design. Four sites in premotor, retro-insular (2) and parietal cortex operated specifically when actions were viewed stereoscopically and frontally. A second experiment clarified that the stereo-action-specific regions were driven by actions moving out of the frontoparallel plane, an effect amplified by frontal viewing in premotor cortex. Analysis of single voxels and their discriminatory power showed that the representation of action in the stereo-action-specific areas was more accurate when stereopsis was active. Further analyses showed that the 4 stereo-action-specific sites form a closed network converging onto the premotor node, which connects to parietal and occipitotemporal regions outside the network. Several of the specific sites are known to process vestibular signals, suggesting that the network combines observed actions in peripersonal space with gravitational signals. These findings have wider implications for the function of premotor cortex and the role of stereopsis in human behavior.

Functional definitions of parietal areas in human and non-human primates

Establishing homologies between cortical areas in animal models and humans lies at the heart of translational neuroscience, as it demonstrates how knowledge obtained from these models can be applied to the human brain. Here, we review progress in using parallel functional imaging to ascertain homologies between parietal areas of human and non-human primates, species sharing similar behavioural repertoires. The human homologues of several areas along monkey IPS involved in action planning and observation, such as AIP, LIP and CIP, as well as those of opercular areas (SII complex), have been defined. In addition, uniquely human areas, such as the tool-use area in left anterior supramarginal gyrus, have also been identified.