FMRI reveals a dissociation between grasping and perceiving the size of real 3D objects

The importance of multisensory-motor learning on subsequent visual recognition

Speed of visual object recognition is facilitated after active manual exploration of objects relative to passive visual processing alone. Manual exploration allows viewers to select important information about object structure that may facilitate recognition. Viewpoints where the objects' axis of elongation is perpendicular or parallel to the line of sight are selected more during exploration, recognized faster than other viewpoints, and afford the most information about structure when object movement is controlled by the viewer. Prior work used virtual object exploration in active and passive viewing conditions, limiting multisensory structural object information. Adding multisensory information to encoding may change accuracy of overall recognition, viewpoint selection, and viewpoint recognition. We tested whether the known active advantage for object recognition would change when real objects were studied, affording visual and haptic information. Participants interacted with 3D novel objects during manual exploration or passive viewing of another's object interactions. Object recognition was tested using several viewpoints of rendered objects. We found that manually explored objects were recognized more accurately than objects studied through passive exploration and that recognition of viewpoints differed from previous work.

Object-oriented hand dexterity and grasping abilities, from the animal quarters to the neurosurgical OR: a systematic review of the underlying neural correlates in non-human, human primate and recent findings in awake brain surgery

Introduction

The sensorimotor integrations subserving object-oriented manipulative actions have been extensively investigated in non-human primates via direct approaches, as intracortical micro-stimulation (ICMS), cytoarchitectonic analysis and anatomical tracers. However, the understanding of the mechanisms underlying complex motor behaviors is yet to be fully integrated in brain mapping paradigms and the consistency of these findings with intraoperative data obtained during awake neurosurgical procedures for brain tumor removal is still largely unexplored. Accordingly, there is a paucity of systematic studies reviewing the cross-species analogies in neural activities during object-oriented hand motor tasks in primates and investigating the concordance with intraoperative findings during brain mapping. The current systematic review was designed to summarize the cortical and subcortical neural correlates of object-oriented fine hand actions, as revealed by fMRI and PET studies, in non-human and human primates and how those were translated into neurosurgical studies testing dexterous hand-movements during intraoperative brain mapping.

Methods

A systematic literature review was conducted following the PRISMA guidelines. PubMed, EMBASE and Web of Science databases were searched. Original articles were included if they: (1) investigated cortical activation sites on fMRI and/or PET during grasping task; (2) included humans or non-human primates. A second query was designed on the databases above to collect studies reporting motor, hand manipulation and dexterity tasks for intraoperative brain mapping in patients undergoing awake brain surgery for any condition. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed more appropriate.

Results

We provided an updated overview of the current state of the art in translational neuroscience about the extended frontoparietal grasping-praxis network with a specific focus on the comparative functioning in non-human primates, healthy humans and how the latter knowledge has been implemented in the neurosurgical operating room during brain tumor resection.

Discussion

The anatomical and functional correlates we reviewed confirmed the evolutionary continuum from monkeys to humans, allowing a cautious but practical adoption of such evidence in intraoperative brain mapping protocols. Integrating the previous results in the surgical practice helps preserve complex motor abilities, prevent long-term disability and poor quality of life and allow the maximal safe resection of intrinsic brain tumors.

Distinct Neural Components of Visually Guided Grasping during Planning and Execution

Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actor's body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here, we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants of either sex viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors, that is, grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral stream areas during grasp planning then in premotor regions during grasp execution. Object mass was encoded in ventral stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects.SIGNIFICANCE STATEMENT Grasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and, surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

Neural and behavioral signatures of the multidimensionality of manipulable object processing

Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives - e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant - i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge - i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.

Cortical areas involved in grasping and reaching actions with and without visual information: An ALE meta-analysis of neuroimaging studies

The functional specialization of the ventral stream in Perception and the dorsal stream in Action is the cornerstone of the leading model proposed by Goodale and Milner in 1992. This model is based on neuropsychological evidence and has been a matter of debate for almost three decades, during which the dual-visual stream hypothesis has received much attention, including support and criticism. The advent of functional magnetic resonance imaging (fMRI) has allowed investigating the brain areas involved in Perception and Action, and provided useful data on the functional specialization of the two streams. Research on this topic has been quite prolific, yet no meta-analysis so far has explored the spatial convergence in the involvement of the two streams in Action. The present meta-analysis (N = 53 fMRI and PET studies) was designed to reveal the specific neural activations associated with Action (i.e., grasping and reaching movements), and the extent to which visual information affects the involvement of the two streams during motor control. Our results provide a comprehensive view of the consistent and spatially convergent neural correlates of Action based on neuroimaging studies conducted over the past two decades. In particular, occipital-temporal areas showed higher activation likelihood in the Vision compared to the No vision condition, but no difference between reach and grasp actions. Frontal-parietal areas were consistently involved in both reach and grasp actions regardless of visual availability. We discuss our results in light of the well-established dual-visual stream model and frame these findings in the context of recent discoveries obtained with advanced fMRI methods, such as multivoxel pattern analysis.

The transition from invariant to action-dependent visual object representation in human dorsal pathway

The human brain can efficiently process action-related visual information, which supports our ability to quickly understand and learn others' actions. The visual information of goal-directed action is extensively represented in the parietal and frontal cortex, but how actions and goal-objects are represented within this neural network is not fully understood. Specifically, which part of this dorsal network represents the identity of goal-objects? Is such goal-object information encoded at an abstract level or highly interactive with action representations? Here, we used functional magnetic resonance imaging with a large number of participants (n = 94) to investigate the neural representation of goal-objects and actions when participants viewed goal-directed action videos. Our results showed that the goal-directed action information could be decoded across much of the dorsal pathway, but in contrast, the invariant goal-object information independent of action was mainly localized in the early stage of dorsal pathway in parietal cortex rather than the down-stream areas of the parieto-frontal cortex. These results help us to understand the relationship between action and goal-object representations in the dorsal pathway, and the evolution of interactive representation of goal-objects and actions along the dorsal pathway during goal-directed action observation.

Complementary contribution of the medial and lateral human parietal cortex to grasping: a repetitive TMS study

The dexterous control of our grasping actions relies on the cooperative activation of many brain areas. In the parietal lobe, 2 grasp-related areas collaborate to orchestrate an accurate grasping action: dorsolateral area AIP and dorsomedial area V6A. Single-cell recordings in monkeys and fMRI studies in humans have suggested that both these areas specify grip aperture and wrist orientation, but encode these grasping parameters differently, depending on the context. To elucidate the causal role of phAIP and hV6A, we stimulated these areas, while participants were performing grasping actions (unperturbed grasping). rTMS over phAIP impaired the wrist orientation process, whereas stimulation over hV6A impaired grip aperture encoding. In a small percentage of trials, an unexpected reprogramming of grip aperture or wrist orientation was required (perturbed grasping). In these cases, rTMS over hV6A or over phAIP impaired reprogramming of both grip aperture and wrist orientation. These results represent the first direct demonstration of a different encoding of grasping parameters by 2 grasp-related parietal areas.

Aging and the visual perception of object size

An experiment evaluated the ability of 30 younger and older adults to visually judge object size under three conditions: (1) full cue, (2) in the dark, with linear perspective, and (3) in complete darkness. Each observer made repeated judgments for the same square stimuli (the task was to adjust a separation until it matched the perceived size of the squares), enabling an evaluation of precision as well as accuracy. The judgments were just as accurate in the dark with linear perspective condition as in the full cue condition, indicating that linear perspective serves as an important source of optical information to support the perception of object size). In contrast, in complete darkness (where linear perspective information was unavailable), the accuracy of the observers' judgments was poor. Finally, there was no difference in either the accuracy or the precision of the observers' judgments between the two age groups, despite the fact that the older adults were more than 50 years older than the younger adults (mean age of the younger and older adults was 22.3 and 74.1 years, respectively). The ability to visually perceive object size is well maintained with increasing age, unlike a number of other important visual abilities.

The effects of TMS over the anterior intraparietal area on anticipatory fingertip force scaling and the size-weight illusion

When lifting an object skillfully, fingertip forces need to be carefully scaled to the object’s weight, which can be inferred from its apparent size and material. This anticipatory force scaling ensures smooth and efficient lifting movements. However, even with accurate motor plans, weight perception can still be biased. In the size-weight illusion, objects of different size but equal weight are perceived to differ in heaviness, with the small object perceived to be heavier than the large object. The neural underpinnings of anticipatory force scaling to object size and the size-weight illusion are largely unknown. In this study, we tested the role of anterior intraparietal cortex (aIPS) in predictive force scaling and the size-weight illusion, by applying continuous theta burst stimulation (cTBS) prior to participants lifting objects of different sizes. Participants received cTBS over aIPS, the primary motor cortex (control area), or Sham stimulation. We found no evidence that aIPS stimulation affected the size-weight illusion. Effects were, however, found on anticipatory force scaling, where grip force was less tuned to object size during initial lifts. These findings suggest that aIPS is not involved in the perception of object weight but plays a transient role in the sensorimotor predictions related to object size.

NEW & NOTEWORTHY Skilled object manipulation requires forming anticipatory motor plans according to the object’s properties. Here, we demonstrate the role of anterior intraparietal sulcus (aIPS) in anticipatory grip force scaling to object size, particularly during initial lifting experience. Interestingly, this role was not maintained after continued practice and was not related to perceptual judgments measured with the size-weight illusion.

The macaque ventral intraparietal area has expanded into three homologue human parietal areas

The macaque ventral intraparietal area (VIP) in the fundus of the intraparietal sulcus has been implicated in a diverse range of sensorimotor and cognitive functions such as motion processing, multisensory integration, processing of head peripersonal space, defensive behavior, and numerosity coding. Here, we exhaustively review macaque VIP function, cytoarchitectonics, and anatomical connectivity and integrate it with human studies that have attempted to identify a potential human VIP homologue. We show that human VIP research has consistently identified three, rather than one, bilateral parietal areas that each appear to subsume some, but not all, of the macaque area's functionality. Available evidence suggests that this human "VIP complex" has evolved as an expansion of the macaque area, but that some precursory specialization within macaque VIP has been previously overlooked. The three human areas are dominated, roughly, by coding the head or self in the environment, visual heading direction, and the peripersonal environment around the head, respectively. A unifying functional principle may be best described as prediction in space and time, linking VIP to state estimation as a key parietal sensorimotor function. VIP's expansive differentiation of head and self-related processing may have been key in the emergence of human bodily self-consciousness.

The Ties that Bind: Agnosia, Neglect and Selective Attention to Visual Scale

PURPOSE OF REVIEW

Historical and contemporary treatments of visual agnosia and neglect regard these disorders as largely unrelated. It is thought that damage to different neural processes leads directly to one or the other condition, yet apperceptive variants of agnosia and object-centered variants of neglect share remarkably similar deficits in the quality of conscious experience. Here we argue for a closer association between "apperceptive" variants of visual agnosia and "object-centered" variants of visual neglect. We introduce a theoretical framework for understanding these conditions based on "scale attention", which refers to selecting boundary and surface information at different levels of the structural hierarchy in the visual array.

RECENT FINDINGS

We review work on visual agnosia, the cortical structures and cortico-cortical pathways that underlie visual perception, visuospatial neglect and object-centered neglect, and attention to scale. We highlight direct and indirect pathways involved in these disorders and in attention to scale. The direct pathway involves the posterior vertical segments of the superior longitudinal fasciculus that are positioned to link the established dorsal and ventral attentional centers in the parietal cortex with structures in the inferior occipitotemporal cortex associated with visual apperceptive agnosia. The connections in the right hemisphere appear to be more important for visual conscious experience, whereas those in the left hemisphere appear to be more strongly associated with the planning and execution of visually guided grasps directed at multi-part objects such as tools. In the latter case, semantic and functional information must drive the selection of the appropriate hand posture and grasp points on the object. This view is supported by studies of grasping in patients with agnosia and in patients with neglect that show that the selection of grasp points when picking up a tool involves both scale attention and semantic contributions from inferotemporal cortex. The indirect pathways, which include the inferior fronto-occipital and horizontal components of the superior longitudinal fasciculi, involve the frontal lobe, working memory and the "multiple demands" network, which can shape the content of visual awareness through the maintenance of goal- and task-based abstractions and their influence on scale attention. Recent studies of human cortico-cortical pathways necessitate revisions to long-standing theoretical views on visual perception, visually guided action and their integrations. We highlight findings from a broad sample of seemingly disparate areas of research to support the proposal that attention to scale is necessary for typical conscious visual experience and for goal-directed actions that depend on functional and semantic information. Furthermore, we suggest that vertical pathways between the parietal and occipitotemporal cortex, along with indirect pathways that involve the premotor and prefrontal cortex, facilitate the operations of scale attention.

The Interrelation Between Peripersonal Action Space and Interpersonal Social Space: Psychophysiological Evidence and Clinical Implications

The peripersonal space is an adaptive and flexible interface between the body and the environment that fulfills a dual-motor function: preparing the body for voluntary object-oriented actions to interact with incentive stimuli and preparing the body for defensive responses when facing potentially harmful stimuli. In this position article, we provide arguments for the sensorimotor rooting of the peripersonal space representation and highlight the variables that contribute to its flexible and adaptive characteristics. We also demonstrate that peripersonal space represents a mediation zone between the body and the environment contributing to not only the control of goal-directed actions but also the organization of social life. The whole of the data presented and discussed led us to the proposal of a new theoretical framework linking the peripersonal action space and the interpersonal social space and we highlight how this theoretical framework can account for social behaviors in populations with socio-emotional deficits.

On the indicators for perceiving empty sets as zero

The question whether human beings process empty sets as zero has received little research attention. In this study, we used the distance and end effects as indicators for treating empty sets as a numerical entity that represents an absence of quantity. In a series of experiments, participants performed a magnitude comparison task. They were presented with empty sets and other numerosities from 1 to 9, presented as dot arrays. We manipulated task instructions relevant to the target (i.e., "choose the target that contains more/less dots" in Experiment 1) or the given numerical range mentioned in the instructions (i.e., 0-9 or 1-9 in Experiment 2) to create conditions in which an empty set would be perceived as the smallest value of the experimental numerical range. The results revealed distance effects for comparisons to empty sets, irrespective of task instructions. In Experiment 3, we manipulated the response mode. Two groups of participants responded to target location, one group with a key-press and the other vocally, while the third group responded vocally to target color. The results revealed distance effects for comparisons to empty sets only when responding to target location, regardless of the response mode, indicating that spatial features should be primed in order to perceive an empty set as a numerical entity. These findings show that perceiving an empty set as nothing or as zero depends on the context in which it is presented.

It's not all about looks: The role of object shape in parietal representations of manual tools

The ability to build and expertly manipulate manual tools sets humans apart from other animals. Watching images of manual tools has been shown to elicit a distinct pattern of neural activity in a network of parietal areas, assumingly because tools entail a potential for action-a unique feature related to their functional use and not shared with other manipulable objects. However, a question has been raised whether this selectivity reflects a processing of low-level visual properties-such as elongated shape that is idiosyncratic to most tool-objects-rather than action-specific features. To address this question, we created and behaviourally validated a stimulus set that dissociates objects that are manipulable and nonmanipulable, as well as objects with different degrees of body extension property (tools and non-tools), while controlling for object shape and low-level image properties. We tested the encoding of action-related features by investigating neural representations in two parietal regions of interest (intraparietal sulcus and superior parietal lobule) using functional MRI. Univariate differences between tools and non-tools were not observed when controlling for visual properties, but strong evidence for the action account was nevertheless revealed when using a multivariate approach. Overall, this study provides further evidence that the representational content in the dorsal visual stream reflects encoding of action-specific properties.

Object responses are highly malleable, rather than invariant, with changes in object appearance

Theoretical frameworks of human vision argue that object responses remain stable, or 'invariant', despite changes in viewing conditions that can alter object appearance but not identity. Here, in a major departure from previous approaches that have relied on two-dimensional (2-D) images to study object processing, we demonstrate that changes in an object's appearance, but not its identity, can lead to striking shifts in behavioral responses to objects. We used inverse multidimensional scaling (MDS) to measure the extent to which arrangements of objects in a sorting task were similar or different when the stimuli were displayed as scaled 2-D images, three-dimensional (3-D) augmented reality (AR) projections, or real-world solids. We were especially interested in whether sorting behavior in each display format was based on conceptual (e.g., typical location) versus physical object characteristics. We found that 2-D images of objects were arranged according to conceptual (typical location), but not physical, properties. AR projections, conversely, were arranged primarily according to physical properties such as real-world size, elongation and weight, but not conceptual properties. Real-world solid objects, unlike both 2-D and 3-D images, were arranged using multidimensional criteria that incorporated both conceptual and physical object characteristics. Our results suggest that object responses can be strikingly malleable, rather than invariant, with changes in the visual characteristics of the stimulus. The findings raise important questions about limits of invariance in object processing, and underscore the importance of studying responses to richer stimuli that more closely resemble those we encounter in real-world environments.

Disparity level identification using the voxel-wise Gabor model of fMRI data

Perceiving disparities is the intuitive basis for our understanding of the physical world. Although many electrophysiology studies have revealed the disparity-tuning characteristics of the neurons in the visual areas of the macaque brain, neuron population responses to disparity processing have seldom been investigated. Many disparity studies using functional magnetic resonance imaging (fMRI) have revealed the disparity-selective visual areas in the human brain. However, it is unclear how to characterize neuron population disparity-tuning responses using fMRI technique. In the present study, we constructed three voxel-wise encoding Gabor models to predict the voxel responses to novel disparity levels and used a decoding method to identify the new disparity levels from population responses in the cortex. Among the three encoding models, the fine-coarse model (FCM) that used fine/coarse disparities to fit the voxel responses to disparities outperformed the single model and uncrossed-crossed model. Moreover, the FCM demonstrated high accuracy in predicting voxel responses in V3A complex and high accuracy in identifying novel disparities from responses in V3A complex. Our results suggest that the FCM can better characterize the voxel responses to disparities than the other two models and V3A complex is a critical visual area for representing disparity information.

Category-selective processing in the two visual pathways as a function of stimulus degradation by noise

Understanding the organising principles and functional properties of the primate brain's numerous visually responsive cortical regions is one of the major goals in cognitive neuroscience. Functional magnetic resonance imaging (fMRI) studies have revealed that neural responses in higher-order visual cortex are shaped by object categories, task context, and spatiotemporal regularities. Beyond these properties, visual processing in the ventral pathway has been shown to be tightly linked to perceptual awareness, while the evidence regarding dorsal visual processing and awareness is mixed. Most previous studies targeting the dorsal pathway have used dichotomous "visible versus invisible" experimental designs and interocular suppression paradigms to modulate stimulus visibility. In this fMRI study, we sought to investigate category-selective processing of faces and tools in the ventral and dorsal visual streams as a function of parametric stimulus degradation by noise. Both frequentist and Bayesian statistics provide strong evidence for a linear relationship between category-selective processing and stimulus information in both visual pathways. Overall, multivariate category decoding accuracies turned out to be lower in the dorsal pathway. We discuss our results within the context of the emerging notion of highly interconnected visual streams, and provide an outlook on how future studies may help to further refine our understanding of the functional role of the dorsal pathway in visual object processing.

Decoding Brain States for Planning Functional Grasps of Tools: A Functional Magnetic Resonance Imaging Multivoxel Pattern Analysis Study

OBJECTIVES

We used multivoxel pattern analysis (MVPA) to investigate neural selectivity for grasp planning within the left-lateralized temporo-parieto-frontal network of areas (praxis representation network, PRN) typically associated with tool-related actions, as studied with traditional neuroimaging contrasts.

METHODS

We used data from 20 participants whose task was to plan functional grasps of tools, with either right or left hands. Region of interest and whole-brain searchlight analyses were performed to show task-related neural patterns.

RESULTS

MVPA revealed significant contributions to functional grasp planning from the anterior intraparietal sulcus (aIPS) and its immediate vicinities, supplemented by inputs from posterior subdivisions of IPS, and the ventral lateral occipital complex (vLOC). Moreover, greater local selectivity was demonstrated in areas near the superior parieto-occipital cortex and dorsal premotor cortex, putatively forming the dorso-dorsal stream.

CONCLUSIONS

A contribution from aIPS, consistent with its role in prospective grasp formation and/or encoding of relevant tool properties (e.g., potential graspable parts), is likely to accompany the retrieval of manipulation and/or mechanical knowledge subserved by the supramarginal gyrus for achieving action goals. An involvement of vLOC indicates that MVPA is particularly sensitive to coding of object properties, their identities and even functions, for a support of grip formation. Finally, the engagement of the superior parieto-frontal regions as revealed by MVPA is consistent with their selectivity for transient features of tools (i.e., variable affordances) for anticipatory hand postures. These outcomes support the notion that, compared to traditional approaches, MVPA can reveal more fine-grained patterns of neural activity. (JINS, 2018, 24, 1013-1025).

Task- and domain-specific modulation of functional connectivity in the ventral and dorsal object-processing pathways

A whole-brain network of regions collectively supports the ability to recognize and use objects-the Tool Processing Network. Little is known about how functional interactions within the Tool Processing Network are modulated in a task-dependent manner. We designed an fMRI experiment in which participants were required to either generate object pantomimes or to carry out a picture matching task over the same images of tools, while holding all aspects of stimulus presentation constant across the tasks. The Tool Processing Network was defined with an independent functional localizer, and functional connectivity within the network was measured during the pantomime and picture matching tasks. Relative to tool picture matching, tool pantomiming led to an increase in functional connectivity between ventral stream regions and left parietal and frontal-motor areas; in contrast, the matching task was associated with an increase in functional connectivity among regions in ventral temporo-occipital cortex, and between ventral temporal regions and the left inferior parietal lobule. Graph-theory analyses over the functional connectivity data indicated that the left premotor cortex and left lateral occipital complex were hub-like (exhibited high betweenness centrality) during tool pantomiming, while ventral stream regions (left medial fusiform gyrus and left posterior middle temporal gyrus) were hub-like during the picture matching task. These results demonstrate task-specific modulation of functional interactions among a common set of regions, and indicate dynamic coupling of anatomically remote regions in task-dependent manner.

The Neural Correlates of Grasping in Left-Handers: When Handedness Does Not Matter

Neurophysiological studies showed that in macaques, grasp-related visuomotor transformations are supported by a circuit involving the anterior part of the intraparietal sulcus, the ventral and the dorsal region of the premotor area. In humans, a similar grasp-related circuit has been revealed by means of neuroimaging techniques. However, the majority of "human" studies considered movements performed by right-handers only, leaving open the question of whether the dynamics underlying motor control during grasping is simply reversed in left-handers with respect to right-handers or not. To address this question, a group of left-handed participants has been scanned with functional magnetic resonance imaging while performing a precision grasping task with the left or the right hand. Dynamic causal modeling was used to assess how brain regions of the two hemispheres contribute to grasping execution and whether the intra- and inter-hemispheric connectivity is modulated by the choice of the performing hand. Results showed enhanced inter-hemispheric connectivity between anterior intraparietal and dorsal premotor cortices during grasping execution with the left dominant hand (LDH) (e.g., right hemisphere) compared to the right (e.g., left hemisphere). These findings suggest that that the left hand, although dominant and theoretically more skilled in left handers, might need additional resources in terms of the visuomotor control and on-line monitoring to accomplish a precision grasping movement. The results are discussed in light of theories on the modulation of parieto-frontal networks during the execution of prehensile movements, providing novel evidence supporting the hypothesis of a handedness-independent specialization of the left hemisphere in visuomotor control.

Neural substrates for allocentric-to-egocentric conversion of remembered reach targets in humans

Targets for goal-directed action can be encoded in allocentric coordinates (relative to another visual landmark), but it is not known how these are converted into egocentric commands for action. Here, we investigated this using a slow event-related fMRI paradigm, based on our previous behavioural finding that the allocentric-to-egocentric (Allo-Ego) conversion for reach is performed at the first possible opportunity. Participants were asked to remember (and eventually reach towards) the location of a briefly presented target relative to another visual landmark. After a first memory delay, participants were forewarned by a verbal instruction if the landmark would reappear at the same location (potentially allowing them to plan a reach following the auditory cue before the second delay), or at a different location where they had to wait for the final landmark to be presented before response, and then reach towards the remembered target location. As predicted, participants showed landmark-centred directional selectivity in occipital-temporal cortex during the first memory delay, and only developed egocentric directional selectivity in occipital-parietal cortex during the second delay for the 'Same cue' task, and during response for the 'Different cue' task. We then compared cortical activation between these two tasks at the times when the Allo-Ego conversion occurred, and found common activation in right precuneus, right presupplementary area and bilateral dorsal premotor cortex. These results confirm that the brain converts allocentric codes to egocentric plans at the first possible opportunity, and identify the four most likely candidate sites specific to the Allo-Ego transformation for reaches.

Disentangling Representations of Object and Grasp Properties in the Human Brain

The properties of objects, such as shape, influence the way we grasp them. To quantify the role of different brain regions during grasping, it is necessary to disentangle the processing of visual dimensions related to object properties from the motor aspects related to the specific hand configuration. We orthogonally varied object properties (shape, size, and elongation) and task (passive viewing, precision grip with two or five digits, or coarse grip with five digits) and used representational similarity analysis of functional magnetic resonance imaging data to infer the representation of object properties and hand configuration in the human brain. We found that object elongation is the most strongly represented object feature during grasping and is coded preferentially in the primary visual cortex as well as the anterior and posterior superior-parieto-occipital cortex. By contrast, primary somatosensory, motor, and ventral premotor cortices coded preferentially the number of digits while ventral-stream and dorsal-stream regions coded a mix of visual and motor dimensions. The representation of object features varied with task modality, as object elongation was less relevant during passive viewing than grasping. To summarize, this study shows that elongation is a particularly relevant property of the object to grasp, which along with the number of digits used, is represented within both ventral-stream and parietal regions, suggesting that communication between the two streams about these specific visual and motor dimensions might be relevant to the execution of efficient grasping actions.

SIGNIFICANCE STATEMENT

To grasp something, the visual properties of an object guide preshaping of the hand into the appropriate configuration. Different grips can be used, and different objects require different hand configurations. However, in natural actions, grip and object type are often confounded, and the few experiments that have attempted to separate them have produced conflicting results. As such, it is unclear how visual and motor properties are represented across brain regions during grasping. Here we orthogonally manipulated object properties and grip, and revealed the visual dimension (object elongation) and the motor dimension (number of digits) that are more strongly coded in ventral and dorsal streams. These results suggest that both streams play a role in the visuomotor coding essential for grasping.

Cortical Activation during Landmark-Centered vs. Gaze-Centered Memory of Saccade Targets in the Human: An FMRI Study

A remembered saccade target could be encoded in egocentric coordinates such as gaze-centered, or relative to some external allocentric landmark that is independent of the target or gaze (landmark-centered). In comparison to egocentric mechanisms, very little is known about such a landmark-centered representation. Here, we used an event-related fMRI design to identify brain areas supporting these two types of spatial coding (i.e., landmark-centered vs. gaze-centered) for target memory during the Delay phase where only target location, not saccade direction, was specified. The paradigm included three tasks with identical display of visual stimuli but different auditory instructions: Landmark Saccade (remember target location relative to a visual landmark, independent of gaze), Control Saccade (remember original target location relative to gaze fixation, independent of the landmark), and a non-spatial control, Color Report (report target color). During the Delay phase, the Control and Landmark Saccade tasks activated overlapping areas in posterior parietal cortex (PPC) and frontal cortex as compared to the color control, but with higher activation in PPC for target coding in the Control Saccade task and higher activation in temporal and occipital cortex for target coding in Landmark Saccade task. Gaze-centered directional selectivity was observed in superior occipital gyrus and inferior occipital gyrus, whereas landmark-centered directional selectivity was observed in precuneus and midposterior intraparietal sulcus. During the Response phase after saccade direction was specified, the parietofrontal network in the left hemisphere showed higher activation for rightward than leftward saccades. Our results suggest that cortical activation for coding saccade target direction relative to a visual landmark differs from gaze-centered directional selectivity for target memory, from the mechanisms for other types of allocentric tasks, and from the directionally selective mechanisms for saccade planning and execution.

The destination defines the journey: an examination of the kinematics of hand-to-mouth movements

Long-train electrical stimulation of the motor and premotor cortices of nonhuman primates can produce either hand-to-mouth or grasp-to-inspect movements, depending on the precise location of stimulation. Furthermore, single-neuron recording studies identify discrete neuronal populations in the inferior parietal and ventral premotor cortices that respond uniquely to either grasp-to-eat or grasp-to-place movements, despite their identical mechanistic requirements. These studies demonstrate that the macaque motor cortex is organized around producing functional, goal-oriented movements, rather than simply fulfilling muscular prerequisites of action. In humans, right-handed hand-to-mouth movements have a unique kinematic signature; smaller maximum grip apertures are produced when grasping to eat than when grasping to place identical targets. This is evidence that the motor cortex in humans is also organized around producing functional movements. However, in both macaques and humans, grasp-to-eat/hand-to-mouth movements have always been elicited using edible targets and have (necessarily) been paired with mouth movement. It is therefore unknown whether the kinematic distinction is a natural result of grasping food and/or is simply attributable to concurrent opening of the mouth while grasping. In experiment 1, we used goal-differentiated grasping tasks, directed toward edible and inedible targets, to show that the unique kinematic signature is present even with inedible targets. In experiment 2, we used the same goal-differentiated grasping tasks, either coupled with or divorced from an open-mouth movement, to show that the signature is not attributable merely to a planned opening of the mouth during the grasp. These results are discussed in relation to the role of hand-to-mouth movements in human development, independently of grasp-to-eat behavior.

Visual information about object size and object position are retained differently in the visual brain: Evidence from grasping studies

Many experiments have examined how the visual information used for action control is represented in our brain, and whether or not visually-guided and memory-guided hand movements rely on dissociable visual representations that are processed in different brain areas (dorsal vs. ventral). However, little is known about how these representations decay over longer time periods and whether or not different visual properties are retained in a similar fashion. In three experiments we investigated how information about object size and object position affect grasping as visual memory demands increase. We found that position information decayed rapidly with increasing delays between viewing the object and initiating subsequent actions - impacting both the accuracy of the transport component (lower end-point accuracy) and the grasp component (larger grip apertures) of the movement. In contrast, grip apertures and fingertip forces remained well-adjusted to target size in conditions in which positional information was either irrelevant or provided, regardless of delay, indicating that object size is encoded in a more stable manner than object position. The findings provide evidence that different grasp-relevant properties are encoded differently by the visual system. Furthermore, we argue that caution is required when making inferences about object size representations based on alterations in the grip component as these variations are confounded with the accuracy with which object position is represented. Instead fingertip forces seem to provide a reliable and confound-free measure to assess internal size estimations in conditions of increased visual uncertainty.

Gender differences in non-standard mapping tasks: A kinematic study using pantomimed reach-to-grasp actions

Comparison between real and pantomimed actions is used in neuroscience to dissociate stimulus-driven (real) as compared to internally driven (pantomimed) visuomotor transformations, with the goal of testing models of vision (Milner & Goodale, 1995) and diagnosing neuropsychological deficits (apraxia syndrome). Real actions refer to an overt movement directed toward a visible target whereas pantomimed actions refer to an overt movement directed either toward an object that is no longer available. Although similar, real and pantomimed actions differ in their kinematic parameters and in their neural substrates. Pantomimed-reach-to-grasp-actions show reduced reaching velocities, higher wrist movements, and reduced grip apertures. In addition, seminal neuropsychological studies and recent neuroimaging findings confirmed that real and pantomimed actions are underpinned by separate brain networks. Although previous literature suggests differences in the praxis system between males and females, no research to date has investigated whether or not gender differences exist in the context of real versus pantomimed reach-to-grasp actions. We asked ten male and ten female participants to perform real and pantomimed reach-to-grasp actions toward objects of different sizes, either with or without visual feedback. During pantomimed actions participants were required to pick up an imaginary object slightly offset relative to the location of the real one (which was in turn the target of the real reach-to-grasp actions). Results demonstrate a significant difference between the kinematic parameters recorded in male and female participants performing pantomimed, but not real reach-to-grasp tasks, depending on the availability of visual feedback. With no feedback both males and females showed smaller grip aperture, slower movement velocity and lower reach height. Crucially, these same differences were abolished when visual feedback was available in male, but not in female participants. Our results suggest that male and female participants should be evaluated separately in the clinical environment and in future research in the field.

Resilience to the contralateral visual field bias as a window into object representations

Viewing images of manipulable objects elicits differential blood oxygen level-dependent (BOLD) contrast across parietal and dorsal occipital areas of the human brain that support object-directed reaching, grasping, and complex object manipulation. However, it is unknown which object-selective regions of parietal cortex receive their principal inputs from the ventral object-processing pathway and which receive their inputs from the dorsal object-processing pathway. Parietal areas that receive their inputs from the ventral visual pathway, rather than from the dorsal stream, will have inputs that are already filtered through object categorization and identification processes. This predicts that parietal regions that receive inputs from the ventral visual pathway should exhibit object-selective responses that are resilient to contralateral visual field biases. To test this hypothesis, adult participants viewed images of tools and animals that were presented to the left or right visual fields during functional magnetic resonance imaging (fMRI). We found that the left inferior parietal lobule showed robust tool preferences independently of the visual field in which tool stimuli were presented. In contrast, a region in posterior parietal/dorsal occipital cortex in the right hemisphere exhibited an interaction between visual field and category: tool-preferences were strongest contralateral to the stimulus. These findings suggest that action knowledge accessed in the left inferior parietal lobule operates over inputs that are abstracted from the visual input and is contingent on analysis by the ventral visual pathway, consistent with its putative role in supporting object manipulation knowledge.

Functional definitions of parietal areas in human and non-human primates

Establishing homologies between cortical areas in animal models and humans lies at the heart of translational neuroscience, as it demonstrates how knowledge obtained from these models can be applied to the human brain. Here, we review progress in using parallel functional imaging to ascertain homologies between parietal areas of human and non-human primates, species sharing similar behavioural repertoires. The human homologues of several areas along monkey IPS involved in action planning and observation, such as AIP, LIP and CIP, as well as those of opercular areas (SII complex), have been defined. In addition, uniquely human areas, such as the tool-use area in left anterior supramarginal gyrus, have also been identified.

Memory-guided reaching in a patient with visual hemiagnosia

The two-visual-systems hypothesis (TVSH) postulates that memory-guided movements rely on intact functions of the ventral stream. Its particular importance for memory-guided actions was initially inferred from behavioral dissociations in the well-known patient DF. Despite of rather accurate reaching and grasping movements to visible targets, she demonstrated grossly impaired memory-guided grasping as much as impaired memory-guided reaching. These dissociations were later complemented by apparently reversed dissociations in patients with dorsal damage and optic ataxia. However, grasping studies in DF and optic ataxia patients differed with respect to the retinotopic position of target objects, questioning the interpretation of the respective findings as a double dissociation. In contrast, the findings for reaching errors in both types of patients came from similar peripheral target presentations. However, new data on brain structural changes and visuomotor deficits in DF also questioned the validity of a double dissociation in reaching. A severe visuospatial short-term memory deficit in DF further questioned the specificity of her memory-guided reaching deficit. Therefore, we compared movement accuracy in visually-guided and memory-guided reaching in a new patient who suffered a confined unilateral damage to the ventral visual system due to stroke. Our results indeed support previous descriptions of memory-guided movements' inaccuracies in DF. Furthermore, our data suggest that recently discovered optic-ataxia like misreaching in DF is most likely caused by her parieto-occipital and not by her ventral stream damage. Finally, multiple visuospatial memory measurements in HWS suggest that inaccuracies in memory-guided reaching tasks in patients with ventral damage cannot be explained by visuospatial short-term memory or perceptual deficits, but by a specific deficit in visuomotor processing.

Haptically Guided Grasping. fMRI Shows Right-Hemisphere Parietal Stimulus Encoding, and Bilateral Dorso-Ventral Parietal Gradients of Object- and Action-Related Processing during Grasp Execution

The neural bases of haptically-guided grasp planning and execution are largely unknown, especially for stimuli having no visual representations. Therefore, we used functional magnetic resonance imaging (fMRI) to monitor brain activity during haptic exploration of novel 3D complex objects, subsequent grasp planning, and the execution of the pre-planned grasps. Haptic object exploration, involving extraction of shape, orientation, and length of the to-be-grasped targets, was associated with the fronto-parietal, temporo-occipital, and insular cortex activity. Yet, only the anterior divisions of the posterior parietal cortex (PPC) of the right hemisphere were significantly more engaged in exploration of complex objects (vs. simple control disks). None of these regions were re-recruited during the planning phase. Even more surprisingly, the left-hemisphere intraparietal, temporal, and occipital areas that were significantly invoked for grasp planning did not show sensitivity to object features. Finally, grasp execution, involving the re-recruitment of the critical right-hemisphere PPC clusters, was also significantly associated with two kinds of bilateral parieto-frontal processes. The first represents transformations of grasp-relevant target features and is linked to the dorso-dorsal (lateral and medial) parieto-frontal networks. The second monitors grasp kinematics and belongs to the ventro-dorsal networks. Indeed, signal modulations associated with these distinct functions follow dorso-ventral gradients, with left aIPS showing significant sensitivity to both target features and the characteristics of the required grasp. Thus, our results from the haptic domain are consistent with the notion that the parietal processing for action guidance reflects primarily transformations from object-related to effector-related coding, and these mechanisms are rather independent of sensory input modality.

Probing the reaching-grasping network in humans through multivoxel pattern decoding

INTRODUCTION

The quest for a putative human homolog of the reaching-grasping network identified in monkeys has been the focus of many neuropsychological and neuroimaging studies in recent years. These studies have shown that the network underlying reaching-only and reach-to-grasp movements includes the superior parieto-occipital cortex (SPOC), the anterior part of the human intraparietal sulcus (hAIP), the ventral and the dorsal portion of the premotor cortex, and the primary motor cortex (M1). Recent evidence for a wider frontoparietal network coding for different aspects of reaching-only and reach-to-grasp actions calls for a more fine-grained assessment of the reaching-grasping network in humans by exploiting pattern decoding methods (multivoxel pattern analysis--MVPA).

METHODS

Here, we used MPVA on functional magnetic resonance imaging (fMRI) data to assess whether regions of the frontoparietal network discriminate between reaching-only and reach-to-grasp actions, natural and constrained grasping, different grasp types, and object sizes. Participants were required to perform either reaching-only movements or two reach-to-grasp types (precision or whole hand grasp) upon spherical objects of different sizes.

RESULTS

Multivoxel pattern analysis highlighted that, independently from the object size, all the selected regions of both hemispheres contribute in coding for grasp type, with the exception of SPOC and the right hAIP. Consistent with recent neurophysiological findings on monkeys, there was no evidence for a clear-cut distinction between a dorsomedial and a dorsolateral pathway that would be specialized for reaching-only and reach-to-grasp actions, respectively. Nevertheless, the comparison of decoding accuracy across brain areas highlighted their different contributions to reaching-only and grasping actions.

CONCLUSIONS

Altogether, our findings enrich the current knowledge regarding the functional role of key brain areas involved in the cortical control of reaching-only and reach-to-grasp actions in humans, by revealing novel fine-grained distinctions among action types within a wide frontoparietal network.

Action preparation shapes processing in early visual cortex

Preparation for an action, such as grasping an object, is accompanied by an enhanced perception of the object's action-relevant features, such as orientation and size. Cortical feedback from motor planning areas to early visual areas may drive this enhanced perception. To examine whether action preparation modulates activity in early human visual cortex, subjects grasped or pointed to oriented objects while high-resolution fMRI data were acquired. Using multivoxel pattern analysis techniques, we could decode with >70% accuracy whether a grasping or pointing action was prepared from signals in visual cortex as early as V1. These signals in early visual cortex were observed even when actions were only prepared but not executed. Anterior parietal cortex, on the other hand, showed clearest modulation for actual movements. This demonstrates that preparation of actions, even without execution, modulates relevant neuronal populations in early visual areas.

Automatic representation of a visual stimulus relative to a background in the right precuneus

Our brains represent the position of a visual stimulus egocentrically, in either retinal or craniotopic coordinates. In addition, recent behavioral studies have shown that the stimulus position is automatically represented allocentrically relative to a large frame in the background. Here, we investigated neural correlates of the ‘background coordinate’ using an fMRI adaptation technique. A red dot was presented at different locations on a screen, in combination with a rectangular frame that was also presented at different locations, while the participants looked at a fixation cross. When the red dot was presented repeatedly at the same location relative to the rectangular frame, the fMRI signals significantly decreased in the right precuneus. No adaptation was observed after repeated presentations relative to a small, but salient, landmark. These results suggest that the background coordinate is implemented in the right precuneus.

Exploring manual asymmetries during grasping: a dynamic causal modeling approach

Recording of neural activity during grasping actions in macaques showed that grasp-related sensorimotor transformations are accomplished in a circuit constituted by the anterior part of the intraparietal sulcus (AIP), the ventral (F5) and the dorsal (F2) region of the premotor area. In humans, neuroimaging studies have revealed the existence of a similar circuit, involving the putative homolog of macaque areas AIP, F5, and F2. These studies have mainly considered grasping movements performed with the right dominant hand and only a few studies have measured brain activity associated with a movement performed with the left non-dominant hand. As a consequence of this gap, how the brain controls for grasping movement performed with the dominant and the non-dominant hand still represents an open question. A functional magnetic resonance imaging (fMRI) experiment has been conducted, and effective connectivity (dynamic causal modeling, DCM) was used to assess how connectivity among grasping-related areas is modulated by hand (i.e., left and right) during the execution of grasping movements toward a small object requiring precision grasping. Results underlined boosted inter-hemispheric couplings between dorsal premotor cortices during the execution of movements performed with the left rather than the right dominant hand. More specifically, they suggest that the dorsal premotor cortices may play a fundamental role in monitoring the configuration of fingers when grasping movements are performed by either the right and the left hand. This role becomes particularly evident when the hand less-skilled (i.e., the left hand) to perform such action is utilized. The results are discussed in light of recent theories put forward to explain how parieto-frontal connectivity is modulated by the execution of prehensile movements.

Contribution of the motor system to the perception of reachable space: an fMRI study

The present functional magnetic resonance imaging (fMRI) study investigates the neural correlates of reachability judgements. In a block design experiment, 14 healthy participants judged whether a visual target presented at different distances in a virtual environment display was reachable or not with the right hand. In two control tasks, they judged the colour or the relative position of the visual target according to flankers. Contrasting the activations registered in the reachability judgement task and in the control tasks, we found activations in the frontal structures, and in the bilateral inferior and superior parietal lobe, including the precuneus, and the bilateral cerebellum. This fronto-parietal network including the cerebellum overlaps with the brain network usually activated during actual motor production and motor imagery. In a following event-related design experiment, we contrasted brain activations when targets were rated as 'reachable' with those when they were rated as 'unreachable'. We found activations in the left premotor cortex, the bilateral frontal structures, and the left middle temporal gyrus. At a lower threshold, we also found activations in the left motor cortex, and in the bilateral cerebellum. Given that reaction time increased with target distance in reachable space, we performed a subsequent parametric analysis that revealed a related increase of activity in the fronto-parietal network including the cerebellum. Unreachable targets did not show similar activation, and particularly in regions associated to motor production and motor imagery. Taken together, these results suggest that dynamical motor representations used to determine what is reachable are also part of the perceptual process leading to the distinct representation of peripersonal and extrapersonal spaces.

Eating interrupted: the effect of intent on hand-to-mouth actions

Evidence from recent neurophysiological studies on nonhuman primates as well as from human behavioral studies suggests that actions with similar kinematic requirements but different end-state goals are supported by separate neural networks. It is unknown whether these different networks supporting seemingly similar reach-to-grasp actions are lateralized, or if they are equally represented in both hemispheres. Recently published behavioral evidence suggests certain networks are lateralized to the left hemisphere. Specifically, when participants used their right hand, their maximum grip aperture (MGA) was smaller when grasping to eat food items than when grasping to place the same items. Left-handed movements showed no difference between tasks. The present study investigates whether the differences between grasp-to-eat and grasp-to-place actions are driven by an intent to eat, or if placing an item into the mouth (sans ingestion) is sufficient to produce asymmetries. Twelve right-handed adults were asked to reach to grasp food items to 1) eat them, 2) place them in a bib, or 3) place them between their lips and then toss them into a nearby receptacle. Participants performed each task with large and small food items, using both their dominant and nondominant hands. The current study replicated the previous finding of smaller MGAs for the eat condition during right-handed but not left-handed grasps. MGAs in the eat and spit conditions did not significantly differ from each other, suggesting that eating and bringing a food item to the mouth both utilize similar motor plans, likely originating within the same neural network. Results are discussed in relation to neurophysiology and development.

Parcellation of left parietal tool representations by functional connectivity

Manipulating a tool according to its function requires the integration of visual, conceptual, and motor information, a process subserved in part by left parietal cortex. How these different types of information are integrated and how their integration is reflected in neural responses in the parietal lobule remains an open question. Here, participants viewed images of tools and animals during functional magnetic resonance imaging (fMRI). k-Means clustering over time series data was used to parcellate left parietal cortex into subregions based on functional connectivity to a whole brain network of regions involved in tool processing. One cluster, in the inferior parietal cortex, expressed privileged functional connectivity to the left ventral premotor cortex. A second cluster, in the vicinity of the anterior intraparietal sulcus, expressed privileged functional connectivity with the left medial fusiform gyrus. A third cluster in the superior parietal lobe expressed privileged functional connectivity with dorsal occipital cortex. Control analyses using Monte Carlo style permutation tests demonstrated that the clustering solutions were outside the range of what would be observed based on chance 'lumpiness' in random data, or mere anatomical proximity. Finally, hierarchical clustering analyses were used to formally relate the resulting parcellation scheme of left parietal tool representations to previous work that has parcellated the left parietal lobule on purely anatomical grounds. These findings demonstrate significant heterogeneity in the functional organization of manipulable object representations in left parietal cortex, and outline a framework that generates novel predictions about the causes of some forms of upper limb apraxia.

A common representation of spatial features drives action and perception: grasping and judging object features within trials

Spatial features of an object can be specified using two different response types: either by use of symbols or motorically by directly acting upon the object. Is this response dichotomy reflected in a dual representation of the visual world: one for perception and one for action? Previously, symbolic and motoric responses, specifying location, has been shown to rely on a common representation. What about more elaborate features such as length and orientation? Here we show that when motoric and symbolic responses are made within the same trial, the probability of making the same symbolic and motoric response is well above chance for both length and orientation. This suggests that motoric and symbolic responses to length and orientation are driven by a common representation. We also show that, for both response types, the spatial features of an object are processed independently. This finding of matching object-processing characteristics is also in agreement with the idea of a common representation driving both response types.

How (and why) the visual control of action differs from visual perception

Vision not only provides us with detailed knowledge of the world beyond our bodies, but it also guides our actions with respect to objects and events in that world. The computations required for vision-for-perception are quite different from those required for vision-for-action. The former uses relational metrics and scene-based frames of reference while the latter uses absolute metrics and effector-based frames of reference. These competing demands on vision have shaped the organization of the visual pathways in the primate brain, particularly within the visual areas of the cerebral cortex. The ventral 'perceptual' stream, projecting from early visual areas to inferior temporal cortex, helps to construct the rich and detailed visual representations of the world that allow us to identify objects and events, attach meaning and significance to them and establish their causal relations. By contrast, the dorsal 'action' stream, projecting from early visual areas to the posterior parietal cortex, plays a critical role in the real-time control of action, transforming information about the location and disposition of goal objects into the coordinate frames of the effectors being used to perform the action. The idea of two visual systems in a single brain might seem initially counterintuitive. Our visual experience of the world is so compelling that it is hard to believe that some other quite independent visual signal-one that we are unaware of-is guiding our movements. But evidence from a broad range of studies from neuropsychology to neuroimaging has shown that the visual signals that give us our experience of objects and events in the world are not the same ones that control our actions.

Dorsal and ventral streams across sensory modalities

In this review, we describe the current models of dorsal and ventral streams in vision, audition and touch. Available theories take their first steps from the model of Milner and Goodale, which was developed to explain how human actions can be efficiently carried out using visual information. Since then, similar concepts have also been applied to other sensory modalities. We propose that advances in the knowledge of brain functioning can be achieved through models explaining action and perception patterns independently from sensory modalities.

Posterior parietal cortex activity reflects the significance of others' actions during natural viewing

The posterior parietal cortex (PPC) has been associated with multiple stimulus-driven (e.g., processing stimulus movements, providing visual signals for the motor system), goal-directed (e.g., directing visual attention to a target, processing behavioral priority of intentions), and action-related functions in previous studies with non-naturalistic paradigms. Here, we examined how these functions reflect PPC activity during natural viewing. Fourteen healthy volunteers watched a re-edited movie during functional magnetic resonance imaging (fMRI). Participants separately annotated behavioral priority (accounting for percepts, thoughts, and emotions) they had experienced during movie episodes. Movements in the movie were quantified with computer vision and eye movements were recorded from a separate group of subjects. Our results show that while overlapping dorsomedial PPC areas respond to episodes with multiple types of stimulus content, ventrolateral PPC areas exhibit enhanced activity when viewing goal-directed human hand actions. Furthermore, PPC activity related to viewing goal-directed human hand actions was more accurately explained by behavioral priority than by movements of the stimulus or eye movements. Taken together, our results suggest that PPC participates in perception of goal-directed human hand actions, supporting the view that PPC has a special role in providing visual signals for the motor system ("how"), in addition to processing visual spatial movements ("where").