Investigating common coding of observed and executed actions in the monkey brain using cross-modal multi-variate fMRI classification

Object-oriented hand dexterity and grasping abilities, from the animal quarters to the neurosurgical OR: a systematic review of the underlying neural correlates in non-human, human primate and recent findings in awake brain surgery

Introduction

The sensorimotor integrations subserving object-oriented manipulative actions have been extensively investigated in non-human primates via direct approaches, as intracortical micro-stimulation (ICMS), cytoarchitectonic analysis and anatomical tracers. However, the understanding of the mechanisms underlying complex motor behaviors is yet to be fully integrated in brain mapping paradigms and the consistency of these findings with intraoperative data obtained during awake neurosurgical procedures for brain tumor removal is still largely unexplored. Accordingly, there is a paucity of systematic studies reviewing the cross-species analogies in neural activities during object-oriented hand motor tasks in primates and investigating the concordance with intraoperative findings during brain mapping. The current systematic review was designed to summarize the cortical and subcortical neural correlates of object-oriented fine hand actions, as revealed by fMRI and PET studies, in non-human and human primates and how those were translated into neurosurgical studies testing dexterous hand-movements during intraoperative brain mapping.

Methods

A systematic literature review was conducted following the PRISMA guidelines. PubMed, EMBASE and Web of Science databases were searched. Original articles were included if they: (1) investigated cortical activation sites on fMRI and/or PET during grasping task; (2) included humans or non-human primates. A second query was designed on the databases above to collect studies reporting motor, hand manipulation and dexterity tasks for intraoperative brain mapping in patients undergoing awake brain surgery for any condition. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed more appropriate.

Results

We provided an updated overview of the current state of the art in translational neuroscience about the extended frontoparietal grasping-praxis network with a specific focus on the comparative functioning in non-human primates, healthy humans and how the latter knowledge has been implemented in the neurosurgical operating room during brain tumor resection.

Discussion

The anatomical and functional correlates we reviewed confirmed the evolutionary continuum from monkeys to humans, allowing a cautious but practical adoption of such evidence in intraoperative brain mapping protocols. Integrating the previous results in the surgical practice helps preserve complex motor abilities, prevent long-term disability and poor quality of life and allow the maximal safe resection of intrinsic brain tumors.

Motor actions are spatially organized in motor and dorsal premotor cortex

Frontal motor areas are central to controlling voluntary movements. In non-human primates, the motor areas contain independent, somatotopic, representations of the forelimb (i.e., motor maps). But are the neural codes for actions spatially organized within those forelimb representations? Addressing this question would provide insight into the poorly understood structure-function relationships of the cortical motor system. Here, we tackle the problem using high-resolution optical imaging and motor mapping in motor (M1) and dorsal premotor (PMd) cortex. Two macaque monkeys performed an instructed reach-to-grasp task while cortical activity was recorded with intrinsic signal optical imaging (ISOI). The spatial extent of activity in M1 and PMd was then quantified in relation to the forelimb motor maps, which we obtained from the same hemisphere with intracortical microstimulation. ISOI showed that task-related activity was concentrated in patches that collectively overlapped <40% of the M1 and PMd forelimb representations. The spatial organization of the patches was consistent across task conditions despite small variations in forelimb use. Nevertheless, the largest condition differences in forelimb use were reflected in the magnitude of cortical activity. Distinct time course profiles from patches in arm zones and patches in hand zones suggest functional differences within the forelimb representations. The results collectively support an organizational framework wherein the forelimb representations contain subzones enriched with neurons tuned for specific actions. Thus, the often-overlooked spatial dimension of neural activity appears to be an important organizing feature of the neural code in frontal motor areas.

No Evidence for Cross-Modal fMRI Adaptation in Macaque Parieto-Premotor Mirror Neuron Regions

To probe the presence of mirror neurons in the human brain, cross-modal fMRI adaptation has been suggested as a suitable technique. The rationale behind this suggestion is that this technique allows making more accurate inferences about neural response properties underlying fMRI voxel activations, beyond merely showing shared voxels that are active during both action observation and execution. However, the validity of using cross-modal fMRI adaptation to demonstrate the presence of mirror neurons in parietal and premotor brain regions has been questioned given the inconsistent and weak results obtained in human studies. A better understanding of cross-modal fMRI adaptation effects in the macaque brain is required as the rationale for using this approach is based on several assumptions related to macaque mirror neuron response properties that still need validation. Here, we conducted a cross-modal fMRI adaptation study in macaque monkeys, using the same action execution and action observation tasks that successfully yielded mirror neuron region cross-modal action decoding in a previous monkey MVPA study. We scanned two male rhesus monkeys while they first executed a sequence of either reach-and-grasp or reach-and-touch hand actions and then observed a video of a human actor performing these motor acts. Both whole-brain and region-of-interest analyses failed to demonstrate cross-modal fMRI adaptation effects in parietal and premotor mirror neuron regions. Our results, in line with previous findings in non-human primates, show that cross-modal motor-to-visual fMRI adaptation is not easily detected in monkey brain regions known to house mirror neurons. Thus, our results advocate caution in using cross-modal fMRI adaptation as a method to infer whether mirror neurons can be found in the primate brain.

Ultra-high field fMRI identifies an action-observation network in the common marmoset

The observation of others' actions activates a network of temporal, parietal and premotor/prefrontal areas in macaque monkeys and humans. This action-observation network (AON) has been shown to play important roles in social action monitoring, learning by imitation, and social cognition in both species. It is unclear whether a similar network exists in New-World primates, which separated from Old-Word primates ~35 million years ago. Here we used ultra-high field fMRI at 9.4 T in awake common marmosets (Callithrix jacchus) while they watched videos depicting goal-directed (grasping food) or non-goal-directed actions. The observation of goal-directed actions activates a temporo-parieto-frontal network, including areas 6 and 45 in premotor/prefrontal cortices, areas PGa-IPa, FST and TE in occipito-temporal region and areas V6A, MIP, LIP and PG in the occipito-parietal cortex. These results show overlap with the humans and macaques' AON, demonstrating the existence of an evolutionarily conserved network that likely predates the separation of Old and New-World primates.

The contribution of dynamics to macaque body and face patch responses

Previous functional imaging studies demonstrated body-selective patches in the primate visual temporal cortex, comparing activations to static bodies and static images of other categories. However, the use of static instead of dynamic displays of moving bodies may have underestimated the extent of the body patch network. Indeed, body dynamics provide information about action and emotion and may be processed in patches not activated by static images. Thus, to map with fMRI the full extent of the macaque body patch system in the visual temporal cortex, we employed dynamic displays of natural-acting monkey bodies, dynamic monkey faces, objects, and scrambled versions of these videos, all presented during fixation. We found nine body patches in the visual temporal cortex, starting posteriorly in the superior temporal sulcus (STS) and ending anteriorly in the temporal pole. Unlike for static images, body patches were present consistently in both the lower and upper banks of the STS. Overall, body patches showed a higher activation by dynamic displays than by matched static images, which, for identical stimulus displays, was less the case for the neighboring face patches. These data provide the groundwork for future single-unit recording studies to reveal the spatiotemporal features the neurons of these body patches encode. These fMRI findings suggest that dynamics have a stronger contribution to population responses in body than face patches.

Categorization learning induced changes in action representations in the macaque STS

Neuroimaging and single cell recordings have demonstrated the presence of STS body category-selective regions (body patches) containing neurons responding to presentation of static bodies and body parts. To date, it remains unclear if these body patches and additional STS regions respond during observation of different categories of dynamic actions and to what extent categorization learning influences representations of observed actions in the STS. In the present study, we trained monkeys to discriminate videos depicting three different actions categories (grasping, touching and reaching) with a forced-choice action categorization task. Before and after categorization training, we performed fMRI recordings while monkeys passively observed the same action videos. At the behavioral level, after categorization training, monkeys generalized to untrained action exemplars, in particular for grasping actions. Before training, uni- and/or multivariate fMRI analyses suggest a broad representation of dynamic action categories in particular in posterior and middle STS. Univariate analysis further suggested action category specific training effects in middle and anterior body patches, face patch ML and posterior STS region MT and FST. Overall, our fMRI experiments suggest a widespread representation of observed dynamic bodily actions in the STS that can be modulated by visual learning, supporting its proposed role in action recognition.

Mirror neurons 30 years later: implications and applications

Mirror neurons (MNs) were first described in a seminal paper in 1992 as a class of monkey premotor cells discharging during both action execution and observation. Despite their debated origin and function, recent studies in several species, from birds to humans, revealed that beyond MNs properly so called, a variety of cell types distributed among multiple motor, sensory, and emotional brain areas form a 'mirror mechanism' more complex and flexible than originally thought, which has an evolutionarily conserved role in social interaction. Here, we trace the current limits and envisage the future trends of this discovery, showing that it inspired translational research and the development of new neurorehabilitation approaches, and constitutes a point of no return in social and affective neuroscience.

Two-monkey fMRI setup for investigating multifaceted aspects of social cognition and behavior involving a real-live conspecific

While brain research over the past decades has shed light on the neural correlates of social cognition and behavior in human and non-human primates, most of this research has been performed in virtual settings requiring subjects to observe pictures or recorded videos instead of observing or interacting with another real-live individual. Here we present a two-monkey fMRI setup, allowing examining whole brain responses in macaque monkeys while they observe or interact face-to-face with another real-live conspecific. We tested this setup by comparing overall brain responses during observation of conspecific hand actions in a virtual (observation of recorded videos of actions) or live context (observation of a real-live conspecific performing actions). This dyadic monkey fMRI setup allows examining brain-wide responses in macaque monkeys during different aspects of social behavior, including observation of real-live actions and sensations, social facilitation, joint-attention and social interactions.

Decoding grip type and action goal during the observation of reaching-grasping actions: A multivariate fMRI study

During execution and observation of reaching-grasping actions, the brain must encode, at the same time, the final action goal and the type of grip necessary to achieve it. Recently, it has been proposed that the Mirror Neuron System (MNS) is involved not only in coding the final goal of the observed action, but also the type of grip used to grasp the object. However, the specific contribution of the different areas of the MNS, at both cortical and subcortical level, in disentangling action goal and grip type is still unclear. Here, twenty human volunteers participated in an fMRI study in which they performed two tasks: (a) observation of four different types of actions, consisting in reaching-to-grasp a box handle with two possible grips (precision, hook) and two possible goals (open, close); (b) action execution, in which participants performed grasping actions similar to those presented during the observation task. A conjunction analysis revealed the presence of shared activated voxels for both action observation and execution within several cortical areas including dorsal and ventral premotor cortex, inferior and superior parietal cortex, intraparietal sulcus, primary somatosensory cortex, and cerebellar lobules VI and VIII. ROI analyses showed a main effect for grip type in several premotor and parietal areas and cerebellar lobule VI, with higher BOLD activation during observation of precision vs hook actions. A grip x goal interaction was also present in the left inferior parietal cortex, with higher BOLD activity during precision-to-close actions. A multivariate pattern analysis (MVPA) revealed a significant accuracy for the grip model in all ROIs, while for the action goal model, significant accuracy was observed only for left inferior parietal cortex ROI. These findings indicate that a large network involving cortical and cerebellar areas is involved in the processing of type of grip, while final action goal appears to be mainly processed within the inferior parietal region, suggesting a differential contribution of the areas activated in this study.

Local and system mechanisms for action execution and observation in parietal and premotor cortices

The action observation network (AON) includes a system of brain areas largely shared with action execution in both human and nonhuman primates. Yet temporal and tuning specificities of distinct areas and of physiologically identified neuronal classes in the encoding of self and others’ action remain unknown. We recorded the activity of 355 single units from three crucial nodes of the AON, the anterior intraparietal area (AIP), and premotor areas F5 and F6, while monkeys performed a Go/No-Go grasping task and observed an experimenter performing it. At the system level, during task execution, F6 displays a prevalence of suppressed neurons and signals whether an action has to be performed, whereas AIP and F5 share a prevalence of facilitated neurons and remarkable target selectivity; during task observation, F5 stands out for its unique prevalence of facilitated neurons and its stronger and earlier modulation than AIP and F6. By applying unsupervised clustering of spike waveforms, we found distinct cell classes unevenly distributed across areas, with different firing properties and carrying specific visuomotor signals. Broadly spiking neurons exhibited a balanced amount of facilitated and suppressed activity during action execution and observation, whereas narrower spiking neurons showed more mutually facilitated responses during the execution of one’s own and others’ action, particularly in areas AIP and F5. Our findings elucidate the time course of activity and firing properties of neurons in the AON during one’s own and others’ action, from the system level of anatomically distinct areas to the local level of physiologically distinct cell classes.

•

F6 neurons show a prevalence of suppressed activity, encoding whether to act

•

Area F5 and AIP share a prevalence of facilitated neurons and target selectivity

•

Across-areas, waveform-based clustering distinguished three neuronal classes

•

Narrow-spiking neurons exhibit mutual modulation during self and others’ action

Examining cross-modal fMRI adaptation for observed and executed actions in the monkey brain

While mirror neurons have been found in several monkey brain regions, their existence in the human brain is still largely inferred from indirect non-invasive measurements like functional MRI. It has been proposed that, beyond showing overlapping brain responses during action observation and execution tasks, candidate mirror neuron regions should demonstrate cross-modal action specificity, in line with a defining physiological characteristic of these neurons in the monkey brain. Although cross-modal fMRI adaptation has been put forward as a suited method to test this key feature of cross-modal action specificity in the human brain, so far, the overall usefulness of this technique to demonstrate mirror neuron activity remains unclear. To date, it has never been tested to what extent monkey brain regions known to house mirror neurons, would yield uni- and/or cross-modal fMRI adaptation effects. We therefore performed an fMRI adaptation experiment while male rhesus macaques either performed or observed two different goal-directed hand actions. Executing grasp/lift or touch/press actions in the dark, as well as observing videos of these monkey hand actions, yielded robust responses throughout the brain, including overlapping fMRI responses in parietal and premotor mirror neuron regions. Uni-modal adaptation effects were mostly restricted to the visual modality and the early visual cortices. Both frequentist and Bayesian statistical analyses however suggested no evidence for cross-modal fMRI adaptation effects in monkey parietal and premotor mirror neuron regions. Overall, these findings suggest monkey mirror neuron activity does not readily translate into cross-modal repetition suppression effects that can be detected by fMRI.

Motor resonance in monkey parietal and premotor cortex during action observation: Influence of viewing perspective and effector identity

Observing others performing motor acts like grasping has been shown to elicit neural responses in the observer`s parieto-frontal motor network, which typically becomes active when the observer would perform these actions him/herself. While some human studies suggested strongest motor resonance during observation of first person or egocentric perspectives compared to third person or allocentric perspectives, other research either report the opposite or did not find any viewpoint-related preferences in parieto-premotor cortices. Furthermore, it has been suggested that these motor resonance effects are lateralized in the parietal cortex depending on the viewpoint and identity of the observed effector (left vs right hand). Other studies, however, do not find such straightforward hand identity dependent motor resonance effects. In addition to these conflicting findings in human studies, to date, little is known about the modulatory role of viewing perspective and effector identity (left or right hand) on motor resonance effects in monkey parieto-premotor cortices. Here, we investigated the extent to which different viewpoints of observed conspecific hand actions yield motor resonance in rhesus monkeys using fMRI. Observing first person, lateral and third person viewpoints of conspecific hand actions yielded significant activations throughout the so-called action observation network, including STS, parietal and frontal cortices. Although region-of-interest analysis of parietal and premotor motor/mirror neuron regions AIP, PFG and F5, showed robust responses in these regions during action observation in general, a clear preference for egocentric or allocentric perspectives was not evident. Moreover, except for lateralized effects due to visual field biases, motor resonance in the monkey brain during grasping observation did not reflect hand identity dependent coding.

Anterior Intraparietal Area: A Hub in the Observed Manipulative Action Network

Current knowledge regarding the processing of observed manipulative actions (OMAs) (e.g., grasping, dragging, or dropping) is limited to grasping and underlying neural circuitry remains controversial. Here, we addressed these issues by combining chronic neuronal recordings along the anteroposterior extent of monkeys’ anterior intraparietal (AIP) area with tracer injections into the recorded sites. We found robust neural selectivity for 7 distinct OMAs, particularly in the posterior part of AIP (pAIP), where it was associated with motor coding of grip type and own-hand visual feedback. This cluster of functional properties appears to be specifically grounded in stronger direct connections of pAIP with the temporal regions of the ventral visual stream and the prefrontal cortex, as connections with skeletomotor related areas and regions of the dorsal visual stream exhibited opposite or no rostrocaudal gradients. Temporal and prefrontal areas may provide visual and contextual information relevant for manipulative action processing. These results revise existing models of the action observation network, suggesting that pAIP constitutes a parietal hub for routing information about OMA identity to the other nodes of the network.

Action execution and action observation elicit mirror responses with the same temporal profile in human SII

The properties of the secondary somatosensory area (SII) have been described by many studies in monkeys and humans. Recent studies on monkeys, however, showed that beyond somatosensory stimuli, SII responds to a wider number of stimuli, a finding requiring a revision that human SII is purely sensorimotor. By recording cortical activity with stereotactic electroencephalography (stereo-EEG), we examined the properties of SI and SII in response to a motor task requiring reaching, grasping and manipulation, as well as the observation of the same actions. Furthermore, we functionally characterized this area with a set of clinical tests, including tactile, acoustical, and visual stimuli. The results showed that only SII activates both during execution and observation with a common temporal profile, whereas SI response were limited to execution. Together with their peculiar response to tactile stimuli, we conclude that the role of SII is pivotal also in the observation of actions involving haptic control.

Visual responsiveness in sensorimotor cortex is increased following amputation and reduced after mirror therapy

Phantom limb pain (PLP) following amputation, which is experienced by the vast majority of amputees, has been reported to be relieved with daily sessions of mirror therapy. During each session, a mirror is used to view the reflected image of the intact limb moving, providing visual feedback consistent with the movement of the missing/phantom limb. To investigate potential neural correlates of the treatment effect, we measured brain responses in volunteers with unilateral leg amputation using functional magnetic resonance imaging (fMRI) during a four-week course of mirror therapy. Mirror therapy commenced immediately following baseline scans, which were repeated after approximately two and four week intervals. We focused on responses in the region of sensorimotor cortex corresponding to primary somatosensory and motor representations of the missing leg. At baseline, prior to starting therapy, we found a strong and unexpected response in sensorimotor cortex of amputees to visually presented images of limbs. This response was stronger for images of feet compared to hands and there was no such response in matched controls. Further, this response to visually presented limbs was no longer present at the end of the four week mirror therapy treatment, when perceived phantom limb pain was also reduced. A similar pattern of results was also observed in extrastriate and parietal regions typically responsive to viewing hand actions, but not in regions corresponding to secondary somatosensory cortex. Finally, there was a significant correlation between initial visual responsiveness in sensorimotor cortex and reduction in PLP suggesting a potential marker for predicting efficacy of mirror therapy. Thus, enhanced visual responsiveness in sensorimotor cortex is associated with PLP and modulated over the course of mirror therapy.

•

Visual responsiveness to the sight of limbs in sensorimotor cortex of leg amputees but not matched controls

•

Consistent with prior studies, mirror therapy over 4 weeks reduced phantom limb pain

•

Visual responsiveness in sensorimotor cortex of amputees diminished following mirror therapy

•

Visual responsiveness in sensorimotor cortex might be useful in predicting the potential efficacy of mirror therapy

Functional specialization of macaque premotor F5 subfields with respect to hand and mouth movements: A comparison of task and resting-state fMRI

Based on architectonic, tract-tracing or functional criteria, the rostral portion of ventral premotor cortex in the macaque monkey, also termed area F5, has been divided into several subfields. Cytoarchitectonical investigations suggest the existence of three subfields, F5c (convexity), F5p (posterior) and F5a (anterior). Electrophysiological investigations have suggested a gradual dorso-ventral transition from hand- to mouth-dominated motor fields, with F5p and ventral F5c strictly related to hand movements and mouth movements, respectively. The involvement of F5a in this respect, however, has received much less attention. Recently, data-driven resting-state fMRI approaches have also been used to examine the presence of distinct functional fields in macaque ventral premotor cortex. Although these studies have suggested several functional clusters in/near macaque F5, so far the parcellation schemes derived from these clustering methods do not completely retrieve the same level of F5 specialization as suggested by aforementioned invasive techniques. Here, using seed-based resting-state fMRI analyses, we examined the functional connectivity of different F5 seeds with key regions of the hand and face/mouth parieto-frontal-insular motor networks. In addition, we trained monkeys to perform either hand grasping or ingestive mouth movements in the scanner in order to compare resting-state with task-derived functional hand and mouth motor networks. In line with previous single-cell investigations, task-fMRI suggests involvement of F5p, dorsal F5c and F5a in the execution of hand grasping movements, while non-communicative mouth movements yielded particularly pronounced responses in ventral F5c. Corroborating with anatomical tracing data of macaque F5 subfields, seed-based resting-state fMRI suggests a transition from predominant functional correlations with the hand-motor network in F5p to mostly mouth-motor network functional correlations in ventral F5c. Dorsal F5c yielded robust functional correlations with both hand- and mouth-motor networks. In addition, the deepest part of the fundus of the inferior arcuate, corresponding to area 44, displayed a strikingly different functional connectivity profile compared to neighboring F5a, suggesting a different functional specialization for these two neighboring regions.