Disentangling Representations of Object and Grasp Properties in the Human Brain

Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions

Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.

Vision matters for shape representation: Evidence from sculpturing and drawing in the blind

Shape is a property that could be perceived by vision and touch, and is classically considered to be supramodal. While there is mounting evidence for the shared cognitive and neural representation space between visual and tactile shape, previous research tended to rely on dissimilarity structures between objects and had not examined the detailed properties of shape representation in the absence of vision. To address this gap, we conducted three explicit object shape knowledge production experiments with congenitally blind and sighted participants, who were asked to produce verbal features, 3D clay models, and 2D drawings of familiar objects with varying levels of tactile exposure, including tools, large nonmanipulable objects, and animals. We found that the absence of visual experience (i.e., in the blind group) led to stronger differences in animals than in tools and large objects, suggesting that direct tactile experience of objects is essential for shape representation when vision is unavailable. For tools with rich tactile/manipulation experiences, the blind produced overall good shapes comparable to the sighted, yet also showed intriguing differences. The blind group had more variations and a systematic bias in the geometric property of tools (making them stubbier than the sighted), indicating that visual experience contributes to aligning internal representations and calibrating overall object configurations, at least for tools. Taken together, the object shape representation reflects the intricate orchestration of vision, touch and language.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Object-oriented hand dexterity and grasping abilities, from the animal quarters to the neurosurgical OR: a systematic review of the underlying neural correlates in non-human, human primate and recent findings in awake brain surgery

Introduction

The sensorimotor integrations subserving object-oriented manipulative actions have been extensively investigated in non-human primates via direct approaches, as intracortical micro-stimulation (ICMS), cytoarchitectonic analysis and anatomical tracers. However, the understanding of the mechanisms underlying complex motor behaviors is yet to be fully integrated in brain mapping paradigms and the consistency of these findings with intraoperative data obtained during awake neurosurgical procedures for brain tumor removal is still largely unexplored. Accordingly, there is a paucity of systematic studies reviewing the cross-species analogies in neural activities during object-oriented hand motor tasks in primates and investigating the concordance with intraoperative findings during brain mapping. The current systematic review was designed to summarize the cortical and subcortical neural correlates of object-oriented fine hand actions, as revealed by fMRI and PET studies, in non-human and human primates and how those were translated into neurosurgical studies testing dexterous hand-movements during intraoperative brain mapping.

Methods

A systematic literature review was conducted following the PRISMA guidelines. PubMed, EMBASE and Web of Science databases were searched. Original articles were included if they: (1) investigated cortical activation sites on fMRI and/or PET during grasping task; (2) included humans or non-human primates. A second query was designed on the databases above to collect studies reporting motor, hand manipulation and dexterity tasks for intraoperative brain mapping in patients undergoing awake brain surgery for any condition. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed more appropriate.

Results

We provided an updated overview of the current state of the art in translational neuroscience about the extended frontoparietal grasping-praxis network with a specific focus on the comparative functioning in non-human primates, healthy humans and how the latter knowledge has been implemented in the neurosurgical operating room during brain tumor resection.

Discussion

The anatomical and functional correlates we reviewed confirmed the evolutionary continuum from monkeys to humans, allowing a cautious but practical adoption of such evidence in intraoperative brain mapping protocols. Integrating the previous results in the surgical practice helps preserve complex motor abilities, prevent long-term disability and poor quality of life and allow the maximal safe resection of intrinsic brain tumors.

Action-specific feature processing in the human cortex: An fMRI study

Sensorimotor integration involves feedforward and reentrant processing of sensory input. Grasp-related motor activity precedes and is thought to influence visual object processing. Yet, while the importance of reentrant feedback is well established in perception, the top-down modulations for action and the neural circuits involved in this process have received less attention. Do action-specific intentions influence the processing of visual information in the human cortex? Using a cue-separation fMRI paradigm, we found that action-specific instruction processing (manual alignment vs. grasp) became apparent only after the visual presentation of oriented stimuli, and occurred as early as in the primary visual cortex and extended to the dorsal visual stream, motor and premotor areas. Further, dorsal stream area aIPS, known to be involved in object manipulation, and the primary visual cortex showed task-related functional connectivity with frontal, parietal and temporal areas, consistent with the idea that reentrant feedback from dorsal and ventral visual stream areas modifies visual inputs to prepare for action. Importantly, both the task-dependent modulations and connections were linked specifically to the object presentation phase of the task, suggesting a role in processing the action goal. Our results show that intended manual actions have an early, pervasive, and differential influence on the cortical processing of vision.

How to construct liquid-crystal spectacles to control vision of real-world objects and environments

A major challenge in studying naturalistic vision lies in controlling stimulus and scene viewing time. This is especially the case for studies using real-world objects as stimuli (rather than computerized images) because real objects cannot be "onset" and "offset" in the same way that images can be. Since the late 1980s, one solution to this problem has been to have the observer wear electro-optic spectacles with computer-controlled liquid-crystal lenses that switch between transparent ("open") and translucent ("closed") states. Unfortunately, the commercially available glasses (PLATO Visual Occlusion Spectacles) command a high price tag, the hardware is fragile, and the glasses cannot be customized. This led us to explore how to manufacture liquid-crystal occlusion glasses in our own laboratory. Here, we share the products of our work by providing step-by-step instructions for researchers to design, build, operate, and test liquid-crystal glasses for use in experimental contexts. The glasses can be assembled with minimal technical knowledge using readily available components, and they can be customized for different populations and applications. The glasses are robust, and they can be produced at a fraction of the cost of commercial alternatives. Tests of reliability and temporal accuracy show that the performance of our laboratory prototype was comparable to that of the PLATO glasses. We discuss the results of our work with respect to implications for promoting rigor and reproducibility, potential use cases, comparisons with other liquid-crystal shutter glasses, and how users can find information regarding future updates and developments.

A focus on the multiple interfaces between action and perception and their neural correlates

Successful behaviour relies on the appropriate interplay between action and perception. The well-established dorsal and ventral stream theories depicted two distinct functional pathways for the processes of action and perception, respectively. In physiological conditions, the two pathways closely cooperate in order to produce successful adaptive behaviour. As the coupling between perception and action exists, this requires an interface that is responsible for a common reading of the two functions. Several studies have proposed different types of perception and action interfaces, suggesting their role in the creation of the shared interaction channel. In the present review, we describe three possible perception and action interfaces: i) the motor code, including common coding approaches, ii) attention, and iii) object affordance; we highlight their potential neural correlates. From this overview, a recurrent neural substrate that underlies all these interface functions appears to be crucial: the parieto-frontal circuit. This network is involved in the mirror mechanism which underlies the perception and action interfaces identified as common coding and motor code theories. The same network is also involved in the spotlight of attention and in the encoding of potential action towards objects; these are manifested in the perception and action interfaces for common attention and object affordance, respectively. Within this framework, most studies were dedicated to the description of the role of the inferior parietal lobule; growing evidence, however, suggests that the superior parietal lobule also plays a crucial role in the interplay between action and perception. The present review proposes a novel model that is inclusive of the superior parietal regions and their relative contribution to the different action and perception interfaces.

The functional organization of skilled actions in the adextral and atypical brain

When planning functional grasps of tools, right-handed individuals (dextrals) show mostly left-lateralized neural activity in the praxis representation network (PRN), regardless of the used hand. Here we studied whether or not similar cerebral asymmetries are evident in non-righthanded individuals (adextrals). Sixty two participants, 28 righthanders and 34 non-righthanders (21 lefthanders, 13 mixedhanders), planned functional grasps of tools vs. grasps of control objects, and subsequently performed their pantomimed executions, in an event-related functional magnetic resonance imaging (fMRI) project. Both hands were tested, separately in two different sessions, counterbalanced across participants. After accounting for non-functional components of the prospective grasp, planning functional grasps of tools was associated with greater engagement of the same, left-hemisphere occipito-temporal, parietal and frontal areas of PRN, regardless of hand and handedness. Only when the analyses involved signal changes referenced to resting baseline intervals, differences between adextrals and dextrals emerged. Whereas in the left hemisphere the neural activity was equivalent in both groups (except for the occipito-temporo-parietal junction), its increases in the right occipito-temporal cortex, medial intraparietal sulcus (area MIP), the supramarginal gyrus (area PFt/PF), and middle frontal gyrus (area p9-46v) were significantly greater in adextrals. The inverse contrast was empty. Notably, when individuals with atypical and typical hemispheric phenotypes were directly compared, planning functional (vs. control) grasps invoked, instead, significant clusters located nearly exclusively in the left hemisphere of the typical phenotype. Previous studies interpret similar right-sided vs. left-sided increases in neural activity for skilled actions as handedness dependent, i.e., located in the hemisphere dominant for manual skills. Yet, none of the effects observed here can be purely handedness dependent because there were mixed-handed individuals among adextrals, and numerous mixed-handed and left-handed individuals possess the typical phenotype. Thus, our results clearly show that hand dominance has limited power in driving the cerebral organization of motor cognitive functions.

Distinct Neural Components of Visually Guided Grasping during Planning and Execution

Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actor's body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here, we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants of either sex viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors, that is, grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral stream areas during grasp planning then in premotor regions during grasp execution. Object mass was encoded in ventral stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects.SIGNIFICANCE STATEMENT Grasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and, surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

Orientation representation in human visual cortices: contributions of non-visual information and action-related process

Orientation processing in the human brain plays a crucial role in guiding grasping actions toward an object. Remarkably, despite the absence of visual input, the human visual cortex can still process orientation information. Instead of visual input, non-visual information, including tactile and proprioceptive sensory input from the hand and arm, as well as feedback from action-related processes, may contribute to orientation processing. However, the precise mechanisms by which the visual cortices process orientation information in the context of non-visual sensory input and action-related processes remain to be elucidated. Thus, our study examined the orientation representation within the visual cortices by analyzing the blood-oxygenation-level-dependent (BOLD) signals under four action conditions: direct grasp (DG), air grasp (AG), non-grasp (NG), and uninformed grasp (UG). The images of the cylindrical object were shown at +45° or - 45° orientations, corresponding to those of the real object to be grasped with the whole-hand gesture. Participants judged their orientation under all conditions. Grasping was performed without online visual feedback of the hand and object. The purpose of this design was to investigate the visual areas under conditions involving tactile feedback, proprioception, and action-related processes. To address this, a multivariate pattern analysis was used to examine the differences among the cortical patterns of the four action conditions in orientation representation by classification. Overall, significant decoding accuracy over chance level was discovered for the DG; however, during AG, only the early visual areas showed significant accuracy, suggesting that the object's tactile feedback influences the orientation process in higher visual areas. The NG showed no statistical significance in any area, indicating that without the grasping action, visual input does not contribute to cortical pattern representation. Interestingly, only the dorsal and ventral divisions of the third visual area (V3d and V3v) showed significant decoding accuracy during the UG despite the absence of visual instructions, suggesting that the orientation representation was derived from action-related processes in V3d and visual recognition of object visualization in V3v. The processing of orientation information during non-visually guided grasping of objects relies on other non-visual sources and is specifically divided by the purpose of action or recognition.

Neural and behavioral signatures of the multidimensionality of manipulable object processing

Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives - e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant - i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge - i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.

Grasping with a Twist: Dissociating Action Goals from Motor Actions in Human Frontoparietal Circuits

In daily life, prehension is typically not the end goal of hand-object interactions but a precursor for manipulation. Nevertheless, functional MRI (fMRI) studies investigating manual manipulation have primarily relied on prehension as the end goal of an action. Here, we used slow event-related fMRI to investigate differences in neural activation patterns between prehension in isolation and prehension for object manipulation. Sixteen (seven males and nine females) participants were instructed either to simply grasp the handle of a rotatable dial (isolated prehension) or to grasp and turn it (prehension for object manipulation). We used representational similarity analysis (RSA) to investigate whether the experimental conditions could be discriminated from each other based on differences in task-related brain activation patterns. We also used temporal multivoxel pattern analysis (tMVPA) to examine the evolution of regional activation patterns over time. Importantly, we were able to differentiate isolated prehension and prehension for manipulation from activation patterns in the early visual cortex, the caudal intraparietal sulcus (cIPS), and the superior parietal lobule (SPL). Our findings indicate that object manipulation extends beyond the putative cortical grasping network (anterior intraparietal sulcus, premotor and motor cortices) to include the superior parietal lobule and early visual cortex.SIGNIFICANCE STATEMENT A simple act such as turning an oven dial requires not only that the CNS encode the initial state (starting dial orientation) of the object but also the appropriate posture to grasp it to achieve the desired end state (final dial orientation) and the motor commands to achieve that state. Using advanced temporal neuroimaging analysis techniques, we reveal how such actions unfold over time and how they differ between object manipulation (turning a dial) versus grasping alone. We find that a combination of brain areas implicated in visual processing and sensorimotor integration can distinguish between the complex and simple tasks during planning, with neural patterns that approximate those during the actual execution of the action.

A large-scale fMRI dataset for human action recognition

Human action recognition is a critical capability for our survival, allowing us to interact easily with the environment and others in everyday life. Although the neural basis of action recognition has been widely studied using a few action categories from simple contexts as stimuli, how the human brain recognizes diverse human actions in real-world environments still needs to be explored. Here, we present the Human Action Dataset (HAD), a large-scale functional magnetic resonance imaging (fMRI) dataset for human action recognition. HAD contains fMRI responses to 21,600 video clips from 30 participants. The video clips encompass 180 human action categories and offer a comprehensive coverage of complex activities in daily life. We demonstrate that the data are reliable within and across participants and, notably, capture rich representation information of the observed human actions. This extensive dataset, with its vast number of action categories and exemplars, has the potential to deepen our understanding of human action recognition in natural environments.

Pulvinar Response Profiles and Connectivity Patterns to Object Domains

Distributed cortical regions show differential responses to visual objects belonging to different domains varying by animacy (e.g., animals vs tools), yet it remains unclear whether this is an organization principle also applying to the subcortical structures. Combining multiple fMRI activation experiments (two main experiments and six validation datasets; 12 females and 9 males in the main Experiment 1; 10 females and 10 males in the main Experiment 2), resting-state functional connectivity, and task-based dynamic causal modeling analysis in human subjects, we found that visual processing of images of animals and tools elicited different patterns of response in the pulvinar, with robust left lateralization for tools, and distinct, bilateral (with rightward tendency) clusters for animals. Such domain-preferring activity distribution in the pulvinar was associated with the magnitude with which the voxels were intrinsically connected with the corresponding domain-preferring regions in the cortex. The pulvinar-to-right-amygdala path showed a one-way shortcut supporting the perception of animals, and the modulation connection from pulvinar to parietal showed an advantage to the perception of tools. These results incorporate the subcortical regions into the object processing network and highlight that domain organization appears to be an overarching principle across various processing stages in the brain.SIGNIFICANCE STATEMENT Viewing objects belonging to different domains elicited different cortical regions, but whether the domain organization applied to the subcortical structures (e.g., pulvinar) was unknown. Multiple fMRI activation experiments revealed that object pictures belonging to different domains elicited differential patterns of response in the pulvinar, with robust left lateralization for tool pictures, and distinct, bilateral (with rightward tendency) clusters for animals. Combining the resting-state functional connectivity and dynamic causal modeling analysis on task-based fMRI data, we found domain-preferring activity distribution in the pulvinar aligned with that in cortical regions. These results highlight the need for coherent visual theories that explain the mechanisms underlying the domain organization across various processing stages.

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Multifingered robotic hands (usually referred to as dexterous hands) are designed to achieve human-level or human-like manipulations for robots or as prostheses for the disabled. The research dates back 30 years ago, yet, there remain great challenges to effectively design and control them due to their high dimensionality of configuration, frequently switched interaction modes, and various task generalization requirements. This article aims to give a brief overview of multifingered robotic manipulation from three aspects: a) the biological results, b) the structural evolvements, and c) the learning methods, and discuss potential future directions. First, we investigate the structure and principle of hand-centered visual sensing, tactile sensing, and motor control and related behavioral results. Then, we review several typical multifingered dexterous hands from task scenarios, actuation mechanisms, and in-hand sensors points. Third, we report the recent progress of various learning-based multifingered manipulation methods, including but not limited to reinforcement learning, imitation learning, and other sub-class methods. The article concludes with open issues and our thoughts on future directions.

The Interactions Between Primary Somatosensory and Motor Cortex during Human Grasping Behaviors

Afferent inputs to the primary somatosensory cortex (S1) are differentially processed during precision and power grip in humans. However, it remains unclear how S1 interacts with the primary motor cortex (M1) during these two grasping behaviors. To address this question, we measured short-latency afferent inhibition (SAI), reflecting S1-M1 interactions via thalamo-cortical pathways, using paired-pulse transcranial magnetic stimulation (TMS) during precision and power grip. The TMS coil over the hand representation of M1 was oriented in the posterior-anterior (PA) and anterior-posterior (AP) direction to activate distinct sets of corticospinal neurons. We found that SAI increased during precision compared with power grip when AP, but not PA, currents were applied. Notably, SAI tested in the AP direction were similar during two-digit than five-digit precision grip. The M1 receives movement information from S1 through direct cortico-cortical pathways, so intra-hemispheric S1-M1 interactions using dual-site TMS were also evaluated. Stimulation of S1 attenuated M1 excitability (S1-M1 inhibition) during precision and power grip, while the S1-M1 inhibition ratio remained similar across tasks. Taken together,our findings suggest that distinct neural mechanisms for S1-M1 interactions mediate precision and power grip, presumably by modulating neural activity along thalamo-cortical pathways.

Graspability Modulates the Stronger Neural Signature of Motor Preparation for Real Objects vs. Pictures

The cognitive and neural bases of visual perception are typically studied using pictures rather than real-world stimuli. Unlike pictures, real objects are actionable solids that can be manipulated with the hands. Recent evidence from human brain imaging suggests that neural responses to real objects differ from responses to pictures; however, little is known about the neural mechanisms that drive these differences. Here, we tested whether brain responses to real objects versus pictures are differentially modulated by the "in-the-moment" graspability of the stimulus. In human dorsal cortex, electroencephalographic responses show a "real object advantage" in the strength and duration of mu (μ) and low beta (β) rhythm desynchronization-well-known neural signatures of visuomotor action planning. We compared desynchronization for real tools versus closely matched pictures of the same objects, when the stimuli were positioned unoccluded versus behind a large transparent barrier that prevented immediate access to the stimuli. We found that, without the barrier in place, real objects elicited stronger μ and β desynchronization compared to pictures, both during stimulus presentation and after stimulus offset, replicating previous findings. Critically, however, with the barrier in place, this real object advantage was attenuated during the period of stimulus presentation, whereas the amplification in later periods remained. These results suggest that the "real object advantage" is driven initially by immediate actionability, whereas later differences perhaps reflect other, more inherent properties of real objects. The findings showcase how the use of richer multidimensional stimuli can provide a more complete and ecologically valid understanding of object vision.

Multivariate Analysis of Electrophysiological Signals Reveals the Time Course of Precision Grasps Programs: Evidence for Nonhierarchical Evolution of Grasp Control

Current understanding of the neural processes underlying human grasping suggests that grasp computations involve gradients of higher to lower level representations and, relatedly, visual to motor processes. However, it is unclear whether these processes evolve in a strictly canonical manner from higher to intermediate and to lower levels given that this knowledge importantly relies on functional imaging, which lacks temporal resolution. To examine grasping in fine temporal detail here we used multivariate EEG analysis. We asked participants to grasp objects while controlling the time at which crucial elements of grasp programs were specified. We first specified the orientation with which participants should grasp objects, and only after a delay we instructed participants about which effector to use to grasp, either the right or the left hand. We also asked participants to grasp with both hands because bimanual and left-hand grasping share intermediate-level grasp representations. We observed that grasp programs evolved in a canonical manner from visual representations, which were independent of effectors to motor representations that distinguished between effectors. However, we found that intermediate representations of effectors that partially distinguished between effectors arose after representations that distinguished among all effector types. Our results show that grasp computations do not proceed in a strictly hierarchically canonical fashion, highlighting the importance of the fine temporal resolution of EEG for a comprehensive understanding of human grasp control.SIGNIFICANCE STATEMENT A long-standing assumption of the grasp computations is that grasp representations progress from higher to lower level control in a regular, or canonical, fashion. Here, we combined EEG and multivariate pattern analysis to characterize the temporal dynamics of grasp representations while participants viewed objects and were subsequently cued to execute an unimanual or bimanual grasp. Interrogation of the temporal dynamics revealed that lower level effector representations emerged before intermediate levels of grasp representations, thereby suggesting a partially noncanonical progression from higher to lower and then to intermediate level grasp control.

Individual differences in representational similarity of first and second languages in the bilingual brain

Current theories of bilingualism disagree on the extent to which separate brain regions are used to maintain or process one's first and second language. The present study took a novel multivariate approach to address this question. We examined whether bilinguals maintain distinct neural representations of two languages; specifically, we tested whether brain areas that are involved in processing word meaning in either language are reliably representing each language differently, and whether language representation is influenced by individual differences in proficiency level and age of acquisition (AoA) of L2. Thirty‐one English–Mandarin bilingual adults performed a picture–word matching task in both languages. We then used representational similarity analysis to examine which brain regions reliably showed different patterns of activity for each language. We found that both proficiency and AoA predicted dissimilarity between language representations in several brain areas within the language network as well as several regions of the ventral visual pathway, demonstrating that top‐down language knowledge and individual language experience shapes concept representation in this processing stream. The results support the model of an integrated language system in bilinguals, along with a novel description of how representations for each language change with proficiency level and L2 AoA.

Hand-Selective Visual Regions Represent How to Grasp 3D Tools: Brain Decoding during Real Actions

Most neuroimaging experiments that investigate how tools and their actions are represented in the brain use visual paradigms where tools or hands are displayed as 2D images and no real movements are performed. These studies discovered selective visual responses in occipitotemporal and parietal cortices for viewing pictures of hands or tools, which are assumed to reflect action processing, but this has rarely been directly investigated. Here, we examined the responses of independently visually defined category-selective brain areas when participants grasped 3D tools (N = 20; 9 females). Using real-action fMRI and multivoxel pattern analysis, we found that grasp typicality representations (i.e., whether a tool is grasped appropriately for use) were decodable from hand-selective areas in occipitotemporal and parietal cortices, but not from tool-, object-, or body-selective areas, even if partially overlapping. Importantly, these effects were exclusive for actions with tools, but not for biomechanically matched actions with control nontools. In addition, grasp typicality decoding was significantly higher in hand than tool-selective parietal regions. Notably, grasp typicality representations were automatically evoked even when there was no requirement for tool use and participants were naive to object category (tool vs nontools). Finding a specificity for typical tool grasping in hand-selective, rather than tool-selective, regions challenges the long-standing assumption that activation for viewing tool images reflects sensorimotor processing linked to tool manipulation. Instead, our results show that typicality representations for tool grasping are automatically evoked in visual regions specialized for representing the human hand, the primary tool of the brain for interacting with the world.

Tool heads prime saccades

Tools are wielded by their handles, but a lot of information about their function comes from their heads (the action-ends). Here we investigated whether eye saccadic movements are primed by tool handles, or whether they are primed by tool heads. We measured human saccadic reaction times while subjects were performing an attentional task. We found that saccades were executed quicker when performed to the side congruent with the tool head, even though "toolness" was irrelevant for the task. Our results show that heads are automatically processed by the visual system to orient eye movements, indicating that eyes are attracted by functional parts of manipulable objects and by the characteristic information these parts convey.

The unique role of parietal cortex in action observation: Functional organization for communicative and manipulative actions

Action observation is supported by a network of regions in occipito-temporal, parietal, and premotor cortex in primates. Recent research suggests that the parietal node has regions dedicated to different action classes including manipulation, interpersonal interactions, skin displacement, locomotion, and climbing. The goals of the current study consist of: 1) extending this work with new classes of actions that are communicative and specific to humans, 2) investigating how parietal cortex differs from the occipito-temporal and premotor cortex in representing action classes. Human subjects underwent fMRI scanning while observing three action classes: indirect communication, direct communication, and manipulation, plus two types of control stimuli, static controls which were static frames from the video clips, and dynamic controls consisting of temporally-scrambled optic flow information. Using univariate analysis, MVPA, and representational similarity analysis, our study presents several novel findings. First, we provide further evidence for the anatomical segregation in parietal cortex of different action classes: We have found a new site that is specific for representing human-specific indirect communicative actions in cytoarchitectonic parietal area PFt. Second, we found that the discriminability between action classes was higher in parietal cortex than the other two levels suggesting the coding of action identity information at this level. Finally, our results advocate the use of the control stimuli not just for univariate analysis of complex action videos but also when using multivariate techniques.

Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types

We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.

Scene context shapes category representational geometry during processing of tools

Tools are ubiquitous in human environments and to think about them we use concepts. Increasingly, conceptual representation is thought to be dynamic and sensitive to the goals of the observer. Indeed, observer goals can reshape representational geometry within cortical networks supporting concepts. In the present study, we investigated the novel hypothesis that task-irrelevant scene context may implicitly alter the representational geometry of regions within the tool network. Participants performed conceptual judgments on images of tools embedded in scenes that either suggested their use (i.e., a kitchen timer sitting on a kitchen counter with vegetables in a frying pan) or that they would simply be moved (i.e., a kitchen timer sitting in an open drawer with other miscellaneous kitchen items around). We investigated whether representations in the tool network reflect category, grip, and shape information using a representational similarity analysis (RSA). We show that a) a number of regions of the tool network reflect category information about tools and b) category information predicts patterns in supramarginal gyrus more strongly in use contexts than in move contexts. Together, these results show that information about tool category is distributed across different regions of the tool network and that scene context helps shape the representational geometry of the tool network.

Overlapping but distinct: Distal connectivity dissociates hand and tool processing networks

The processes and organizational principles of information involved in object recognition have been a subject of intense debate. These research efforts led to the understanding that local computations and feedforward/feedback connections are essential to our representations and their organization. Recent data, however, has demonstrated that distal computations also play a role in how information is locally processed. Here we focus on how long-range connectivity and local functional organization of information are related, by exploring regions that show overlapping category-preferences for two categories and testing whether their connections are related with distal representations in a category-specific way. We used an approach that relates functional connectivity with distal areas to local voxel-wise category-preferences. Specifically, we focused on two areas that show an overlap in category-preferences for tools and hands-the inferior parietal lobule/anterior intraparietal sulcus (IPL/aIPS) and the posterior middle temporal gyrus/lateral occipital temporal cortex (pMTG/LOTC) - and how connectivity from these two areas relate to voxel-wise category-preferences in two ventral temporal regions dedicated to the processing of tools and hands separately-the left medial fusiform gyrus and the fusiform body area respectively-as well as across the brain. We show that the functional connections of the two overlap areas correlate with categorical preferences for each category independently. These results show that regions that process both tools and hands maintain object topography in a category-specific way. This potentially allows for a category-specific flow of information that is pertinent to computing object representations.

The mouth matters most: A functional magnetic resonance imaging study of how dogs perceive inanimate objects

The perception and representation of objects in the world are foundational to all animals. The relative importance of objects' physical properties versus how the objects are interacted with continues to be debated. Neural evidence in humans and nonhuman primates suggests animate-inanimate and face-body dimensions of objects are represented in the temporal cortex. However, because primates have opposable thumbs and interact with objects in similar ways, the question remains as to whether this similarity represents the evolution of a common cognitive process or whether it reflects a similarity of physical interaction. Here, we used functional magnetic resonance imaging (fMRI) in dogs to test whether the type of interaction affects object processing in an animal that interacts primarily with its mouth. In Study 1, we identified object-processing regions of cortex by having dogs passively view movies of faces and objects. In Study 2, dogs were trained to interact with two new objects with either the mouth or the paw. Then, we measured responsivity in the object regions to the presentation of these objects. Mouth-objects elicited significantly greater activity in object regions than paw-objects. Mouth-objects were also associated with activity in somatosensory cortex, suggesting dogs were anticipating mouthing interactions. These findings suggest that object perception in dogs is affected by how dogs expect to interact with familiar objects.

Decoding stimuli (tool-hand) and viewpoint invariant grasp-type information

When we see a manipulable object (henceforth tool) or a hand performing a grasping movement, our brain is automatically tuned to how that tool can be grasped (i.e., its affordance) or what kind of grasp that hand is performing (e.g., a power or precision grasp). However, it remains unclear where visual information related to tools or hands are transformed into abstract grasp representations. We therefore investigated where different levels of abstractness in grasp information are processed: grasp information that is invariant to the kind of stimuli that elicits it (tool-hand invariance); and grasp information that is hand-specific but viewpoint-invariant (viewpoint invariance). We focused on brain areas activated when viewing both tools and hands, i.e., the posterior parietal cortices (PPC), ventral premotor cortices (PMv), and lateral occipitotemporal cortex/posterior middle temporal cortex (LOTC/pMTG). To test for invariant grasp representations, we presented participants with tool images and grasp videos (from first or third person perspective; 1pp or 3pp) inside an MRI scanner, and cross-decoded power versus precision grasps across (i) grasp perspectives (viewpoint invariance), (ii) tool images and grasp 1pp videos (tool-hand 1pp invariance), and (iii) tool images and grasp 3pp videos (tool-hand 3pp invariance). Tool-hand 1pp, but not tool-hand 3pp, invariant grasp information was found in left PPC, whereas viewpoint-invariant information was found bilaterally in PPC, left PMv, and left LOTC/pMTG. These findings suggest different levels of abstractness-where visual information is transformed into stimuli-invariant grasp representations/tool affordances in left PPC, and viewpoint invariant but hand-specific grasp representations in the hand network.

Assessing the effective connectivity of premotor areas during real vs imagined grasping: a DCM-PEB approach

The parieto-frontal circuit underlying grasping, which requires the serial involvement of the anterior intraparietal area (aIPs) and the ventral premotor cortex (PMv), has been recently extended enlightening the role of the dorsal premotor cortex (PMd). The supplementary motor area (SMA) has been also suggested to encode grip force for grasping actions; furthermore, both PMd and SMA are known to play a crucial role in motor imagery. Here, we aimed at assessing the dynamic couplings between left aIPs, PMv, PMd, SMA and primary motor cortex (M1) by comparing executed and imagined right-hand grasping, using Dynamic Causal Modelling (DCM) and Parametrical Empirical Bayes (PEB) analyses. 24 subjects underwent an fMRI exam (3T) during which they were asked to perform or imagine a grasping movement visually cued by photographs of commonly used objects. We tested whether the two conditions a) exert a modulatory effect on both forward and feedback couplings among our areas of interest, and b) differ in terms of strength and sign of these parameters. Results of the real condition confirmed the serial involvement of aIPs, PMv and M1. PMv also exerted a positive influence on PMd and SMA, but received an inhibitory feedback only from PMd. Our results suggest that a general motor program for grasping is planned by the aIPs-PMv circuit; then, PMd and SMA encode high-level features of the movement. During imagery, the connection strength from aIPs to PMv was weaker and the information flow stopped in PMv; thus, a less complex motor program was planned. Moreover, results suggest that SMA and PMd cooperate to prevent motor execution. In conclusion, the comparison between execution and imagery reveals that during grasping premotor areas dynamically interplay in different ways, depending on task demands.

Grasping and Manipulation: Neural Bases and Anatomical Circuitry in Humans

Neurophysiological and neuroimaging evidence suggests a significant contribution of several brain areas, including subdivisions of the parietal and the premotor cortex, during the processing of different components of hand and arm movements. Many investigations improved our knowledge about the neural processes underlying the execution of reaching and grasping actions, while few studies have directly investigated object manipulation. Most studies on the latter topic concern the use of tools to achieve specific goals. Yet, there are very few studies on pure manipulation performed in order to explore and recognize objects, as well as on manipulation performed with a high level of manual dexterity. Another dimension that is quite neglected by the available studies on grasping and manipulation is, on the one hand, the contribution of the subcortical nodes, first of all the basal ganglia and cerebellum, to these functions, and, on the other hand, recurrent connections of these structures with cortical areas. In the first part, we have reviewed the parieto-premotor and subcortical circuits underlying reaching and grasping in humans, with a focus on functional neuroimaging data. Then, we have described the main structures recruited during object manipulation. We have also reported the contribution of recent structural connectivity techniques whereby the cortico-cortical and cortico-subcortical connections of grasping-related and manipulation-related areas in the human brain can be determined. Based on our review, we have concluded that studies on cortical and subcortical circuits involved in grasping and manipulation might be promising to provide new insights about motor learning and brain plasticity in patients with motor disorders.

It's not all about looks: The role of object shape in parietal representations of manual tools

The ability to build and expertly manipulate manual tools sets humans apart from other animals. Watching images of manual tools has been shown to elicit a distinct pattern of neural activity in a network of parietal areas, assumingly because tools entail a potential for action-a unique feature related to their functional use and not shared with other manipulable objects. However, a question has been raised whether this selectivity reflects a processing of low-level visual properties-such as elongated shape that is idiosyncratic to most tool-objects-rather than action-specific features. To address this question, we created and behaviourally validated a stimulus set that dissociates objects that are manipulable and nonmanipulable, as well as objects with different degrees of body extension property (tools and non-tools), while controlling for object shape and low-level image properties. We tested the encoding of action-related features by investigating neural representations in two parietal regions of interest (intraparietal sulcus and superior parietal lobule) using functional MRI. Univariate differences between tools and non-tools were not observed when controlling for visual properties, but strong evidence for the action account was nevertheless revealed when using a multivariate approach. Overall, this study provides further evidence that the representational content in the dorsal visual stream reflects encoding of action-specific properties.

Overt and Covert Object Features Mediate Timing of Patterned Brain Activity during Motor Planning

Humans are seamless in their ability to efficiently and reliably generate fingertip forces to gracefully interact with objects. Such interactions rarely end in awkward outcomes like spilling, crushing, or tilting given advanced motor planning. Here we combine multiband imaging with deconvolution- and Bayesian pattern component modeling of functional magnetic resonance imaging data and in-scanner kinematics, revealing compelling evidence that the human brain differentially represents preparatory information for skillful object interactions depending on the saliency of visual cues. Earlier patterned activity was particularly evident in ventral visual processing stream-, but also selectively in dorsal visual processing stream and cerebellum in conditions of heightened uncertainty when an object’s superficial shape was incompatible rather than compatible with a key underlying object feature.

Primary motor cortical activity during unimanual movements with increasing demand on precision

In functional magnetic resonance imaging (fMRI) studies, performance of unilateral hand movements is associated with primary motor cortex activity ipsilateral to the moving hand (M1_ipsi), in addition to contralateral activity (M1_contra). The magnitude of M1_ipsi activity increases with the demand on precision of the task. However, it is unclear how demand-dependent increases in M1_ipsi recruitment relate to the control of hand movements. To address this question, we used fMRI to measure blood oxygenation level-dependent (BOLD) activity during performance of a task that varied in demand on precision. Participants (n = 23) manipulated an MRI-compatible joystick with their right or left hand to move a cursor into targets of different sizes (small, medium, large, extra large). Performance accuracy, movement time, and number of velocity peaks scaled with target size, whereas reaction time, maximum velocity, and initial direction error did not. In the univariate analysis, BOLD activation in M1_contra and M1_ipsi was higher for movements to smaller targets. Representational similarity analysis, corrected for mean activity differences, revealed multivoxel BOLD activity patterns during movements to small targets were most similar to those for medium targets and least similar to those for extra-large targets. Only models that varied with demand (target size, performance accuracy, and number of velocity peaks) correlated with the BOLD dissimilarity patterns, though differently for right and left hands. Across individuals, M1_contra and M1_ipsi similarity patterns correlated with each other. Together, these results suggest that increasing demand on precision in a unimanual motor task increases M1 activity and modulates M1 activity patterns.NEW & NOTEWORTHY Contralateral primary motor cortex (M1) predominantly controls unilateral hand movements, but the role of ipsilateral M1 is unclear. We used functional magnetic resonance imaging (fMRI) to investigate how M1 activity is modulated by unimanual movements at different levels of demand on precision. Our results show that task characteristics related to demand on precision influence bilateral M1 activity, suggesting that in addition to contralateral M1, ipsilateral M1 plays a key role in controlling hand movements to meet performance precision requirements.

A parietal region processing numerosity of observed actions: An FMRI study

When observing others' behavior, it is important to perceive not only the identity of the observed actions (OAs), but also the number of times they were performed. Given the mounting evidence implicating posterior parietal cortex in action observation, and in particular that of manipulative actions, the aim of this study was to identify the parietal region, if any, that contributes to the processing of observed manipulative action (OMA) numerosity, using the functional magnetic resonance imaging technique. Twenty‐one right‐handed healthy volunteers performed two discrimination tasks while in the scanner, responding to video stimuli in which an actor performed manipulative actions on colored target balls that appeared four times consecutively. The subjects discriminated between two small numerosities of either OMAs (“Action” condition) or colors of balls (“Ball” condition). A significant difference between the “Action” and “Ball” conditions was observed in occipito‐temporal cortex and the putative human anterior intraparietal sulcus (phAIP) area as well as the third topographic map of numerosity‐selective neurons at the post‐central sulcus (NPC3) of the left parietal cortex. A further region of interest analysis of the group‐average data showed that at the single voxel level the latter area, more than any other parietal or occipito‐temporal numerosity map, favored numerosity of OAs. These results suggest that phAIP processes the identity of OMAs, while neighboring NPC3 likely processes the numerosity of the identified OAs.

What Does Dorsal Cortex Contribute to Perception?

According to the influential "Two Visual Pathways" hypothesis, the cortical visual system is segregated into two pathways, with the ventral, occipitotemporal pathway subserving object perception, and the dorsal, occipitoparietal pathway subserving the visuomotor control of action. However, growing evidence suggests that the dorsal pathway also plays a functional role in object perception. In the current article, we present evidence that the dorsal pathway contributes uniquely to the perception of a range of visuospatial attributes that are not redundant with representations in ventral cortex. We describe how dorsal cortex is recruited automatically during perception, even when no explicit visuomotor response is required. Importantly, we propose that dorsal cortex may selectively process visual attributes that can inform the perception of potential actions on objects and environments, and we consider plausible developmental and cognitive mechanisms that might give rise to these representations. As such, we consider whether naturalistic stimuli, such as real-world solid objects, might engage dorsal cortex more so than simplified or artificial stimuli such as images that do not afford action, and how the use of suboptimal stimuli might limit our understanding of the functional contribution of dorsal cortex to visual perception.

Activation of cerebellum and basal ganglia during the observation and execution of manipulative actions

Studies on action observation mostly described the activation of a network of cortical areas, while less investigation focused specifically on the activation and role of subcortical nodes. In the present fMRI study, we investigated the recruitment of cerebellum and basal ganglia during the execution and observation of object manipulation performed with the right hand. The observation conditions consisted in: (a) observation of manipulative actions; (b) observation of sequences of random finger movements. In the execution conditions, participants had to perform the same actions or movements as in (a) and (b), respectively. The results of conjunction analysis showed significant shared activations during both observation and execution of manipulation in several subcortical structures, including: (1) cerebellar lobules V, VI, crus I, VIIIa and VIIIb (bilaterally); (2) globus pallidus, bilaterally, and left subthalamic nucleus; (3) red nucleus (bilaterally) and left thalamus. These findings support the hypothesis that the action observation/execution network also involves subcortical structures, such as cerebellum and basal ganglia, forming an integrated network. This suggests possible mechanisms, involving these subcortical structures, underlying learning of new motor skills, through action observation and imitation.

Neural representations of haptic object size in the human brain revealed by multivoxel fMRI patterns

The brain must interpret sensory input from diverse receptor systems to estimate object properties. Much has been learned about the brain mechanisms behind these processes in vision, but our understanding of haptic perception remains less clear. Here we examined haptic judgments of object size, which require integrating multiple cutaneous and proprioceptive afferent signals, as a model problem. To identify candidate human brain regions that support this process, participants (n = 16) in an event-related functional MRI experiment grasped objects to categorize them as one of four sizes. Object sizes were calibrated psychophysically to be equally distinct for each participant. We applied representational similarity logic to whole brain, multivoxel searchlight analyses to identify brain regions that exhibit size-relevant voxelwise activity patterns. Of particular interest was to identify regions for which more similar sizes produce more similar patterns of activity, which constitutes evidence of a metric size code. Regions of the intraparietal sulcus and the lateral prefrontal cortex met this criterion, both within hands and across hands. We suggest that these regions compute representations of haptic size that abstract over the specific peripheral afferent signals generated in a grasp. Results of a matched visual size task, performed by the same participants and analyzed in the same fashion, identified similar regions, indicating that these representations may be partly modality general. We consider these results with respect to perspectives on magnitude estimation in general and to computational views on perceptual signal integration.NEW & NOTEWORTHY Our understanding of the neural basis of haptics (perceiving the world through touch) remains incomplete. We used functional MRI to study human haptic judgments of object size, which require integrating multiple afferent signals. Multivoxel pattern analyses identified intraparietal and prefrontal regions that encode size haptically in a metric and hand-invariant fashion. Effector-independent haptic size estimates are useful on their own and in combination with other sensory estimates for a variety of perceptual and motor tasks.

Sociality and interaction envelope organize visual action representations

Humans observe a wide range of actions in their surroundings. How is the visual cortex organized to process this diverse input? Using functional neuroimaging, we measured brain responses while participants viewed short videos of everyday actions, then probed the structure in these responses using voxel-wise encoding modeling. Responses are well fit by feature spaces that capture the body parts involved in an action and the action’s targets (i.e. whether the action was directed at an object, another person, the actor, and space). Clustering analyses reveal five large-scale networks that summarize the voxel tuning: one related to social aspects of an action, and four related to the scale of the interaction envelope, ranging from fine-scale manipulations directed at objects, to large-scale whole-body movements directed at distant locations. We propose that these networks reveal the major representational joints in how actions are processed by visual regions of the brain.

How is action perception organized in the brain? Here, the authors report evidence for five networks tuned to actions’ social content and the scale of their effect on the world and propose that sociality and interaction envelope are organizing dimensions of visual action representation.

Roles of Category, Shape, and Spatial Frequency in Shaping Animal and Tool Selectivity in the Occipitotemporal Cortex

Does the nature of representation in the category-selective regions in the occipitotemporal cortex reflect visual or conceptual properties? Previous research showed that natural variability in visual features across categories, quantified by image gist statistics, is highly correlated with the different neural responses observed in the occipitotemporal cortex. Using fMRI, we examined whether category selectivity for animals and tools would remain, when image gist statistics were comparable across categories. Critically, we investigated how category, shape, and spatial frequency may contribute to the category selectivity in the animal- and tool-selective regions. Female and male human observers viewed low- or high-passed images of round or elongated animals and tools that shared comparable gist statistics in the main experiment, and animal and tool images of naturally varied gist statistics in a separate localizer. Univariate analysis revealed robust category-selective responses for images with comparable gist statistics across categories. Successful classification for category (animals/tools), shape (round/elongated), and spatial frequency (low/high) was also observed, with highest classification accuracy for category. Representational similarity analyses further revealed that the activation patterns in the animal-selective regions were most correlated with a model that represents only animal information, whereas the activation patterns in the tool-selective regions were most correlated with a model that represents only tool information, suggesting that these regions selectively represent information of only animals or tools. Together, in addition to visual features, the distinction between animal and tool representations in the occipitotemporal cortex is likely shaped by higher-level conceptual influences such as categorization or interpretation of visual inputs.

SIGNIFICANCE STATEMENT Since different categories often vary systematically in both visual and conceptual features, it remains unclear what kinds of information determine category-selective responses in the occipitotemporal cortex. To minimize the influences of low- and mid-level visual features, here we used a diverse image set of animals and tools that shared comparable gist statistics. We manipulated category (animals/tools), shape (round/elongated), and spatial frequency (low/high), and found that the representational content of the animal- and tool-selective regions is primarily determined by their preferred categories only, regardless of shape or spatial frequency. Our results show that category-selective responses in the occipitotemporal cortex are influenced by higher-level processing such as categorization or interpretation of visual inputs, and highlight the specificity in these category-selective regions.

Parietal Cortex Integrates Saccade and Object Orientation Signals to Update Grasp Plans

Coordinated reach-to-grasp movements are often accompanied by rapid eye movements (saccades) that displace the desired object image relative to the retina. Parietal cortex compensates for this by updating reach goals relative to current gaze direction, but its role in the integration of oculomotor and visual orientation signals for updating grasp plans is unknown. Based on a recent perceptual experiment, we hypothesized that inferior parietal cortex (specifically supramarginal gyrus [SMG]) integrates saccade and visual signals to update grasp plans in additional intraparietal/superior parietal regions. To test this hypothesis in humans (7 females, 6 males), we used a functional magnetic resonance paradigm, where saccades sometimes interrupted grasp preparation toward a briefly presented object that later reappeared (with the same/different orientation) just before movement. Right SMG and several parietal grasp regions, namely, left anterior intraparietal sulcus and bilateral superior parietal lobule, met our criteria for transsaccadic orientation integration: they showed task-dependent saccade modulations and, during grasp execution, they were specifically sensitive to changes in object orientation that followed saccades. Finally, SMG showed enhanced functional connectivity with both prefrontal saccade regions (consistent with oculomotor input) and anterior intraparietal sulcus/superior parietal lobule (consistent with sensorimotor output). These results support the general role of parietal cortex for the integration of visuospatial perturbations, and provide specific cortical modules for the integration of oculomotor and visual signals for grasp updating.SIGNIFICANCE STATEMENT How does the brain simultaneously compensate for both external and internally driven changes in visual input? For example, how do we grasp an unstable object while eye movements are simultaneously changing its retinal location? Here, we used fMRI to identify a group of inferior parietal (supramarginal gyrus) and superior parietal (intraparietal and superior parietal) regions that show saccade-specific modulations during unexpected changes in object/grasp orientation, and functional connectivity with frontal cortex saccade centers. This provides a network, complementary to the reach goal updater, that integrates visuospatial updating into grasp plans, and may help to explain some of the more complex symptoms associated with parietal damage, such as constructional ataxia.

Compound grips in tufted capuchin monkeys (Sapajus spp and Sapajus libidinosus)

An experimental study with captive individuals and study of video recordings of wild monkeys explored whether and how tufted capuchin monkeys use onehand to hold one or more objects with multiple grips (compound grips). A task designed to elicit compound grip was presented to five captive tufted capuchin monkeys (Sapajus spp). The monkeys held one to four balls in onehand and dropped the balls individually into a vertical tube. Multiple simple grips and independent digit movements enabled separate control of multiple objects in one hand. Monkeys always supported the wrist on the horizontal edge of the tube before releasing the ball. Increasing the number of balls decreased the likelihood that the monkeys managed the task. Wild bearded capuchins (Sapajus libidinosus) used compound grips spontaneously to store multiple food items. Compound grips have been described in macaques, gorillas, chimpanzees, and humans, and now in a New World primate. We predict that any primate species that exhibits precision grips and independent digit movement can perform compound grips. Our findings suggest many aspects of compound grip that await investigation.

Object responses are highly malleable, rather than invariant, with changes in object appearance

Theoretical frameworks of human vision argue that object responses remain stable, or 'invariant', despite changes in viewing conditions that can alter object appearance but not identity. Here, in a major departure from previous approaches that have relied on two-dimensional (2-D) images to study object processing, we demonstrate that changes in an object's appearance, but not its identity, can lead to striking shifts in behavioral responses to objects. We used inverse multidimensional scaling (MDS) to measure the extent to which arrangements of objects in a sorting task were similar or different when the stimuli were displayed as scaled 2-D images, three-dimensional (3-D) augmented reality (AR) projections, or real-world solids. We were especially interested in whether sorting behavior in each display format was based on conceptual (e.g., typical location) versus physical object characteristics. We found that 2-D images of objects were arranged according to conceptual (typical location), but not physical, properties. AR projections, conversely, were arranged primarily according to physical properties such as real-world size, elongation and weight, but not conceptual properties. Real-world solid objects, unlike both 2-D and 3-D images, were arranged using multidimensional criteria that incorporated both conceptual and physical object characteristics. Our results suggest that object responses can be strikingly malleable, rather than invariant, with changes in the visual characteristics of the stimulus. The findings raise important questions about limits of invariance in object processing, and underscore the importance of studying responses to richer stimuli that more closely resemble those we encounter in real-world environments.

Transcranial Direct Current Stimulation Alters Functional Network Structure in Humans: A Graph Theoretical Analysis

Transcranial direct current stimulation (tDCS) is routinely used in basic and clinical research, but its efficacy has been challenged on a methodological, statistical and technical basis recently. The arguments against tDCS derive from an insufficient understanding of how this technique interacts with brain processes physiologically. Because of its potential as a central tool in neuroscience, it is important to clarify whether tDCS affects neuronal activity. Here, we investigate influences of offline tDCS on network architecture measured by functional magnetic resonance imaging. Applied to one network node only, offline tDCS affects the architecture of the entire functional network. Furthermore, offline tDCS exerts polarity-specific effects on the topology of the functional network attached. Our results confirm in a functioning brain and in a bias free and independent fashion that offline tDCS influences neuronal activity. Moreover, our results suggest that network-specific connectivity has an important role in improving our understanding of the effects of tDCS.

Multivariate Analysis of Electrophysiological Signals Reveals the Temporal Properties of Visuomotor Computations for Precision Grips

The frontoparietal networks underlying grasping movements have been extensively studied, especially using fMRI. Accordingly, whereas much is known about their cortical locus much less is known about the temporal dynamics of visuomotor transformations. Here, we show that multivariate EEG analysis allows for detailed insights into the time course of visual and visuomotor computations of precision grasps. Male and female human participants first previewed one of several objects and, upon its reappearance, reached to grasp it with the thumb and index finger along one of its two symmetry axes. Object shape classifiers reached transient accuracies of 70% at ∼105 ms, especially based on scalp sites over visual cortex, dropping to lower levels thereafter. Grasp orientation classifiers relied on a system of occipital-to-frontal electrodes. Their accuracy rose concurrently with shape classification but ramped up more gradually, and the slope of the classification curve predicted individual reaction times. Further, cross-temporal generalization revealed that dynamic shape representation involved early and late neural generators that reactivated one another. In contrast, grasp computations involved a chain of generators attaining a sustained state about 100 ms before movement onset. Our results reveal the progression of visual and visuomotor representations over the course of planning and executing grasp movements.SIGNIFICANCE STATEMENT Grasping an object requires the brain to perform visual-to-motor transformations of the object's properties. Although much of the neuroanatomic basis of visuomotor transformations has been uncovered, little is known about its time course. Here, we orthogonally manipulated object visual characteristics and grasp orientation, and used multivariate EEG analysis to reveal that visual and visuomotor computations follow similar time courses but display different properties and dynamics.

Decision-making in sensorimotor control

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

What Role Does "Elongation" Play in "Tool-Specific" Activation and Connectivity in the Dorsal and Ventral Visual Streams?

Images of tools induce stronger activation than images of nontools in a left-lateralized network that includes ventral-stream areas implicated in tool identification and dorsal-stream areas implicated in tool manipulation. Importantly, however, graspable tools tend to be elongated rather than stubby, and so the tool-selective responses in some of these areas may, to some extent, reflect sensitivity to elongation rather than "toolness" per se. Using functional magnetic resonance imaging, we investigated the role of elongation in driving tool-specific activation in the 2 streams and their interconnections. We showed that in some "tool-selective" areas, the coding of toolness and elongation coexisted, but in others, elongation and toolness were coded independently. Psychophysiological interaction analysis revealed that toolness, but not elongation, had a strong modulation of the connectivity between the ventral and dorsal streams. Dynamic causal modeling revealed that viewing tools (either elongated or stubby) increased the connectivity from the ventral- to the dorsal-stream tool-selective areas, but only viewing elongated tools increased the reciprocal connectivity between these areas. Overall, these data disentangle how toolness and elongation affect the activation and connectivity of the tool network and help to resolve recent controversies regarding the relative contribution of "toolness" versus elongation in driving dorsal-stream "tool-selective" areas.

An Information-Driven 2-Pathway Characterization of Occipitotemporal and Posterior Parietal Visual Object Representations

Recent studies have demonstrated the existence of rich visual representations in both occipitotemporal cortex (OTC) and posterior parietal cortex (PPC). Using fMRI decoding and a bottom-up data-driven approach, we showed that although robust object category representations exist in both OTC and PPC, there is an information-driven 2-pathway separation among these regions in the representational space, with occipitotemporal regions arranging hierarchically along 1 pathway and posterior parietal regions along another pathway. We obtained 10 independent replications of this 2-pathway distinction, accounting for 58-81% of the total variance of the region-wise differences in visual representation. The separation of the PPC regions from higher occipitotemporal regions was not driven by a difference in tolerance to changes in low-level visual features, did not rely on the presence of special object categories, and was present whether or not object category was task relevant. Our information-driven 2-pathway structure differs from the well-known ventral-what and dorsal-where/how characterization of posterior brain regions. Here both pathways contain rich nonspatial visual representations. The separation we see likely reflects a difference in neural coding scheme used by PPC to represent visual information compared with that of OTC.

Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions

Patients with visual agnosia show severe deficits in recognizing two-dimensional (2-D) images of objects, despite the fact that early visual processes such as figure-ground segmentation, and stereopsis, are largely intact. Strikingly, however, these patients can nevertheless show a preservation in their ability to recognize real-world objects -a phenomenon known as the 'real-object advantage' (ROA) in agnosia. To uncover the mechanisms that support the ROA, patients were asked to identify objects whose size was congruent or incongruent with typical real-world size, presented in different display formats (real objects, 2-D and 3-D images). While recognition of images was extremely poor, real object recognition was surprisingly preserved, but only when physical size matched real-world size. Analogous display format and size manipulations did not influence the recognition of common geometric shapes that lacked real-world size associations. These neuropsychological data provide evidence for a surprising preservation of size-coding of real-world-sized tangible objects in patients for whom ventral contributions to image processing are severely disrupted. We propose that object size information is largely mediated by dorsal visual cortex and that this information, together with detailed representation of object shape which is also subserved by dorsal cortex, serve as the basis of the ROA.