Grasp Posture Variability Leads to Greater Ipsilateral Sensorimotor Beta Activation During Simulated Prosthesis Use

Motor behaviour using upper-extremity prostheses of different levels is greatly variable, leading to challenges interpreting ideal rehabilitation strategies. Elucidating the underlying neural control mechanisms driving variability benefits our understanding of adaptation after limb loss. In this follow-up study, non-amputated participants completed simple and complex reach-to-grasp motor tasks using a body-powered transradial or partial-hand prosthesis simulator. We hypothesised that under complex task constraints, individuals employing variable grasp postures will show greater sensorimotor beta activation compared to individuals relying on uniform grasping, and activation will occur later in variable compared to uniform graspers. In the simple task, partial-hand variable and transradial users showed increased neural activation from the early to late phase of the reach, predominantly in the hemisphere ipsilateral to device use. In the complex task, only partial-hand variable graspers showed a significant increase in neural activation of the sensorimotor cortex from the early to the late phase of the reach. These results suggest that grasp variability may be a crucial component in the mechanism of neural adaptation to prosthesis use, and may be mediated by device level and task complexity, with implications for rehabilitation after amputation.

Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions

Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.

Neural and behavioral signatures of the multidimensionality of manipulable object processing

Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives - e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant - i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge - i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.

Left anterior supramarginal gyrus activity during tool use action observation after extensive tool use training

The advanced use of complex tools is considered a primary characteristic of human evolution and technological advancement. However, questions remain regarding whether humans possess unique underlying brain networks that support advanced tool-using abilities. Specifically, previous studies have demonstrated the presence of a structurally and functionally unique region in the left anterior supramarginal gyrus (aSMG), that is consistently active during tool use action observation. This region has been proposed as a primary hub for integrating semantic and technical information to form action plans with tools. However, it is still largely unknown how tool use motor learning affects left aSMG activation or connectivity with other brain regions. To address this, participants with little experience using chopsticks observed an experimenter using chopsticks to perform a novel task while undergoing two functional magnetic resonance imaging (fMRI) scans. Between the scans, participants underwent four weeks of behavioral training where they learned to use chopsticks and achieve proficiency in the observed task. Results demonstrated a significant change in effective connectivity between the left aSMG and the left anterior intraparietal sulcus (aIPS), a region involved in object affordances and planning grasping actions. These findings suggest that during unfamiliar tool use, the left aSMG integrates semantic and technical information to communicate with regions involved with grasp selection, such as the aIPS. This communication then allows appropriate grasps to be planned based on the physical properties of the objects involved and their potential interactions.

Domain-specific connectivity drives the organization of object knowledge in the brain

The goal of this chapter is to review neuropsychological and functional MRI findings that inform a theory of the causes of functional specialization for semantic categories within occipito-temporal cortex-the ventral visual processing pathway. The occipito-temporal pathway supports visual object processing and recognition. The theoretical framework that drives this review considers visual object recognition through the lens of how "downstream" systems interact with the outputs of visual recognition processes. Those downstream processes include conceptual interpretation, grasping and object use, navigating and orienting in an environment, physical reasoning about the world, and inferring future actions and the inner mental states of agents. The core argument of this chapter is that innately constrained connectivity between occipito-temporal areas and other regions of the brain is the basis for the emergence of neural specificity for a limited number of semantic domains in the brain.

Neural encoding and functional interactions underlying pantomimed movements

Pantomimes are a unique movement category which can convey complex information about our intentions in the absence of any interaction with real objects. Indeed, we can pretend to use the same tool to perform different actions or to achieve the same goal adopting different tools. Nevertheless, how our brain implements pantomimed movements is still poorly understood. In our study, we explored the neural encoding and functional interactions underlying pantomimes adopting multivariate pattern analysis (MVPA) and connectivity analysis of fMRI data. Participants performed pantomimed movements, either grasp-to-move or grasp-to-use, as if they were interacting with two different tools (scissors or axe). These tools share the possibility to achieve the same goal. We adopted MVPA to investigate two levels of representation during the planning and execution of pantomimes: (1) distinguishing different actions performed with the same tool, (2) representing the same final goal irrespective of the adopted tool. We described widespread encoding of action information within regions of the so-called “tool” network. Several nodes of the network—comprising regions within the ventral and the dorsal stream—also represented goal information. The spatial distribution of goal information changed from planning—comprising posterior regions (i.e. parietal and temporal)—to execution—including also anterior regions (i.e. premotor cortex). Moreover, connectivity analysis provided evidence for task-specific bidirectional coupling between the ventral stream and parieto-frontal motor networks. Overall, we showed that pantomimes were characterized by specific patterns of action and goal encoding and by task-dependent cortical interactions.

Hand-Selective Visual Regions Represent How to Grasp 3D Tools: Brain Decoding during Real Actions

Most neuroimaging experiments that investigate how tools and their actions are represented in the brain use visual paradigms where tools or hands are displayed as 2D images and no real movements are performed. These studies discovered selective visual responses in occipitotemporal and parietal cortices for viewing pictures of hands or tools, which are assumed to reflect action processing, but this has rarely been directly investigated. Here, we examined the responses of independently visually defined category-selective brain areas when participants grasped 3D tools (N = 20; 9 females). Using real-action fMRI and multivoxel pattern analysis, we found that grasp typicality representations (i.e., whether a tool is grasped appropriately for use) were decodable from hand-selective areas in occipitotemporal and parietal cortices, but not from tool-, object-, or body-selective areas, even if partially overlapping. Importantly, these effects were exclusive for actions with tools, but not for biomechanically matched actions with control nontools. In addition, grasp typicality decoding was significantly higher in hand than tool-selective parietal regions. Notably, grasp typicality representations were automatically evoked even when there was no requirement for tool use and participants were naive to object category (tool vs nontools). Finding a specificity for typical tool grasping in hand-selective, rather than tool-selective, regions challenges the long-standing assumption that activation for viewing tool images reflects sensorimotor processing linked to tool manipulation. Instead, our results show that typicality representations for tool grasping are automatically evoked in visual regions specialized for representing the human hand, the primary tool of the brain for interacting with the world.

Overlapping but distinct: Distal connectivity dissociates hand and tool processing networks

The processes and organizational principles of information involved in object recognition have been a subject of intense debate. These research efforts led to the understanding that local computations and feedforward/feedback connections are essential to our representations and their organization. Recent data, however, has demonstrated that distal computations also play a role in how information is locally processed. Here we focus on how long-range connectivity and local functional organization of information are related, by exploring regions that show overlapping category-preferences for two categories and testing whether their connections are related with distal representations in a category-specific way. We used an approach that relates functional connectivity with distal areas to local voxel-wise category-preferences. Specifically, we focused on two areas that show an overlap in category-preferences for tools and hands-the inferior parietal lobule/anterior intraparietal sulcus (IPL/aIPS) and the posterior middle temporal gyrus/lateral occipital temporal cortex (pMTG/LOTC) - and how connectivity from these two areas relate to voxel-wise category-preferences in two ventral temporal regions dedicated to the processing of tools and hands separately-the left medial fusiform gyrus and the fusiform body area respectively-as well as across the brain. We show that the functional connections of the two overlap areas correlate with categorical preferences for each category independently. These results show that regions that process both tools and hands maintain object topography in a category-specific way. This potentially allows for a category-specific flow of information that is pertinent to computing object representations.

Decoding stimuli (tool-hand) and viewpoint invariant grasp-type information

When we see a manipulable object (henceforth tool) or a hand performing a grasping movement, our brain is automatically tuned to how that tool can be grasped (i.e., its affordance) or what kind of grasp that hand is performing (e.g., a power or precision grasp). However, it remains unclear where visual information related to tools or hands are transformed into abstract grasp representations. We therefore investigated where different levels of abstractness in grasp information are processed: grasp information that is invariant to the kind of stimuli that elicits it (tool-hand invariance); and grasp information that is hand-specific but viewpoint-invariant (viewpoint invariance). We focused on brain areas activated when viewing both tools and hands, i.e., the posterior parietal cortices (PPC), ventral premotor cortices (PMv), and lateral occipitotemporal cortex/posterior middle temporal cortex (LOTC/pMTG). To test for invariant grasp representations, we presented participants with tool images and grasp videos (from first or third person perspective; 1pp or 3pp) inside an MRI scanner, and cross-decoded power versus precision grasps across (i) grasp perspectives (viewpoint invariance), (ii) tool images and grasp 1pp videos (tool-hand 1pp invariance), and (iii) tool images and grasp 3pp videos (tool-hand 3pp invariance). Tool-hand 1pp, but not tool-hand 3pp, invariant grasp information was found in left PPC, whereas viewpoint-invariant information was found bilaterally in PPC, left PMv, and left LOTC/pMTG. These findings suggest different levels of abstractness-where visual information is transformed into stimuli-invariant grasp representations/tool affordances in left PPC, and viewpoint invariant but hand-specific grasp representations in the hand network.

Partial repetition between action plans delays responses to ideomotor compatible stimuli

Often one must depart from an intended course of events to react to sudden situational demands before resuming his or her original action retained in working memory. Retaining an action plan in working memory (WM) can delay or facilitate the execution of an intervening action when the action features of the two action plans partly overlap (partial repetition) compared to when they do not overlap. We investigated whether partial repetition costs (PRCs) or benefits (PRBs) occur when the intervening event is an ideomotor-compatible stimulus that is a biological representation of the response required by the participant. Participants viewed two visual events and retained an action plan to the first event (A) while executing a speeded response to the second, intervening event (B). In Experiment 1A, the two visual events were ideomotor compatible, non-ideomotor compatible (abstract), or one was ideomotor compatible, and the other abstract. Results showed PRCs for all event A-B stimulus combinations with reduced PRCs for intervening, ideomotor compatible events. In contrast to previous research, there was no evidence that ideomotor-compatible actions were automatic and bypassed the selection bottleneck. Experiment 1B confirmed PRCs for ideomotor compatible stimuli that more accurately mimicked the required response. Findings suggest that mechanisms for activating, selecting, and retaining action plans are similar between ideomotor compatible and abstract visual events. We conclude that PRCs occur in response to intervening events when action plans are generated offline and rely on WM, including those for ideomotor-compatible stimuli; but PRBs may be restricted to actions generated online. This conclusion is consistent with the perceptual-motor framework by Goodale and Milner (Trends in Neuroscience 15:22-25, 1992).

Structural Disconnection of the Tool Use Network after Left Hemisphere Stroke Predicts Limb Apraxia Severity

Producing a tool use gesture is a complex process drawing upon the integration of stored knowledge of tools and their associated actions with sensory-motor mechanisms supporting the planning and control of hand and arm actions. Understanding how sensory-motor systems in parietal cortex interface with semantic representations of actions and objects in the temporal lobe remains a critical issue and is hypothesized to be a key determinant of the severity of limb apraxia, a deficit in producing skilled action after left hemisphere stroke. We used voxel-based and connectome-based lesion-symptom mapping with data from 57 left hemisphere stroke participants to assess the lesion sites and structural disconnection patterns associated with poor tool use gesturing. We found that structural disconnection among the left inferior parietal lobule, lateral and ventral temporal cortices, and middle and superior frontal gyri predicted the severity of tool use gesturing performance. Control analyses demonstrated that reductions in right-hand grip strength were associated with motor system disconnection, largely bypassing regions supporting tool use gesturing. Our findings provide evidence that limb apraxia may arise, in part, from a disconnection between conceptual representations in the temporal lobe and mechanisms enabling skilled action production in the inferior parietal lobule.

Training Skills in Minimally Invasive, Robotic and Open Surgery: Brain Activation as an Opportunity for Learning

INTRODUCTION

The advantages of the robotic approach in surgery are undisputed. However, during surgical training, how this technique influences the learning curve has not been described. We provide a tentative model for analyzing the learning curves associated with observation and active participation in learning different surgical techniques, using functional imaging.

METHODS

Forty medical students were enrolled and assigned to 4 groups who underwent training in robotic (ROB), laparoscopic (LAP), or open (OPEN) surgery, and a control group that performed motor training without surgical instruments. Surgical/motor training included six 1-h sessions completed over 6 days of the same week. All subjects underwent functional magnetic resonance imaging (fMRI) scanning sessions, before and after surgical training during.

RESULTS

Twenty-three participants completed the study. The 3 surgical groups exhibited different learning curves during training. The main effects of the day of training (p < 0.01) and the group (p < 0.01) as well as a significant interaction of day of training group (p < 0.01) were observed. The performance increased in the first 4 days, reaching a peak at day 4, when all groups were considered together. The OPEN group showed the best performance compared to all other groups (p < 0.04). The OPEN group showed a rapid improvement in performance, which peaked at day 4 and decreased on the last day. Similarly, the LAP group showed a steady increase in the number of exercises they completed, which continued for the entire training period and reached a peak on the last day. However, the participants training in ROB surgery, after a performance initially indistinguishable from that of the LAP group, had a dip in their performance, quickly followed by an improvement and reaching a plateau on day 4. fMRI analysis documented the different involvement of the cortical and subcortical areas based on the type of training. Surgical training modified the activation of some brain regions during both observation and the execution of tasks.

CONCLUSIONS

Differences in the learning curves of the 3 surgical groups were noted. Functional brain activity represents an interesting starting point to guide training programs.

Delineating functional segregations of the human middle temporal gyrus with resting-state functional connectivity and coactivation patterns

Although the middle temporal gyrus (MTG) has been parcellated into subregions with distinguished anatomical connectivity patterns, whether the structural topography of MTG can inform functional segregations of this area remains largely unknown. Accumulating evidence suggests that the brain's underlying organization and function can be directly and effectively delineated with resting-state functional connectivity (RSFC) by identifying putative functional boundaries between cortical areas. Here, RSFC profiles were used to explore functional segregations of the MTG and defined four subregions from anterior to posterior in two independent datasets, which showed a similar pattern with MTG parcellation scheme obtained using anatomical connectivity. The functional segregations of MTG were further supported by whole brain RSFC, coactivation, and specific RFSC, and coactivation mapping. Furthermore, the fingerprint with predefined 10 networks and functional characterizations of each subregion using meta-analysis also identified functional distinction between subregions. The specific connectivity analysis and functional characterization indicated that the bilateral most anterior subregions mainly participated in social cognition and semantic processing; the ventral middle subregions were involved in social cognition in left hemisphere and auditory processing in right hemisphere; the bilateral ventro-posterior subregions participated in action observation, whereas the left subregion was also involved in semantic processing; both of the dorsal subregions in superior temporal sulcus were involved in language, social cognition, and auditory processing. Taken together, our findings demonstrated MTG sharing similar structural and functional topographies and provide more detailed information about the functional organization of the MTG, which may facilitate future clinical and cognitive research on this area.

Domain-Specific Diaschisis: Lesions to Parietal Action Areas Modulate Neural Responses to Tools in the Ventral Stream

Neural responses to small manipulable objects ("tools") in high-level visual areas in ventral temporal cortex (VTC) provide an opportunity to test how anatomically remote regions modulate ventral stream processing in a domain-specific manner. Prior patient studies indicate that grasp-relevant information can be computed about objects by dorsal stream structures independently of processing in VTC. Prior functional neuroimaging studies indicate privileged functional connectivity between regions of VTC exhibiting tool preferences and regions of parietal cortex supporting object-directed action. Here we test whether lesions to parietal cortex modulate tool preferences within ventral and lateral temporal cortex. We found that lesions to the left anterior intraparietal sulcus, a region that supports hand-shaping during object grasping and manipulation, modulate tool preferences in left VTC and in the left posterior middle temporal gyrus. Control analyses demonstrated that neural responses to "place" stimuli in left VTC were unaffected by lesions to parietal cortex, indicating domain-specific consequences for ventral stream neural responses in the setting of parietal lesions. These findings provide causal evidence that neural specificity for "tools" in ventral and lateral temporal lobe areas may arise, in part, from online inputs to VTC from parietal areas that receive inputs via the dorsal visual pathway.

What Role Does "Elongation" Play in "Tool-Specific" Activation and Connectivity in the Dorsal and Ventral Visual Streams?

Images of tools induce stronger activation than images of nontools in a left-lateralized network that includes ventral-stream areas implicated in tool identification and dorsal-stream areas implicated in tool manipulation. Importantly, however, graspable tools tend to be elongated rather than stubby, and so the tool-selective responses in some of these areas may, to some extent, reflect sensitivity to elongation rather than "toolness" per se. Using functional magnetic resonance imaging, we investigated the role of elongation in driving tool-specific activation in the 2 streams and their interconnections. We showed that in some "tool-selective" areas, the coding of toolness and elongation coexisted, but in others, elongation and toolness were coded independently. Psychophysiological interaction analysis revealed that toolness, but not elongation, had a strong modulation of the connectivity between the ventral and dorsal streams. Dynamic causal modeling revealed that viewing tools (either elongated or stubby) increased the connectivity from the ventral- to the dorsal-stream tool-selective areas, but only viewing elongated tools increased the reciprocal connectivity between these areas. Overall, these data disentangle how toolness and elongation affect the activation and connectivity of the tool network and help to resolve recent controversies regarding the relative contribution of "toolness" versus elongation in driving dorsal-stream "tool-selective" areas.

Gesturing tool use and tool transport actions modulates inferior parietal functional connectivity with the dorsal and ventral object processing pathways

Interacting with manipulable objects (tools) requires the integration of diverse computations supported by anatomically remote regions. Previous functional neuroimaging research has demonstrated the left supramarginal (SMG) exhibits functional connectivity to both ventral and dorsal pathways, supporting the integration of ventrally-mediated tool properties and conceptual knowledge with dorsally-computed volumetric and structural representations of tools. This architecture affords us the opportunity to test whether interactions between the left SMG, ventral visual pathway, and dorsal visual pathway are differentially modulated when participants plan and generate tool-directed gestures emphasizing functional manipulation (tool use gesturing) or structure-based grasping (tool transport gesturing). We found that functional connectivity between the left SMG, ventral temporal cortex (bilateral fusiform gyri), and dorsal visual pathway (left superior parietal lobule/posterior intraparietal sulcus) was maximal for tool transport planning and gesturing, whereas functional connectivity between the left SMG, left ventral anterior temporal lobe, and left frontal operculum was maximal for tool use planning and gesturing. These results demonstrate that functional connectivity to the left SMG is differentially modulated by tool use and tool transport gesturing, suggesting that distinct tool features computed by the two object processing pathways are integrated in the parietal lobe in the service of tool-directed action.

The role of putative human anterior intraparietal sulcus area in observed manipulative action discrimination

INTRODUCTION

Although it has become widely accepted that the action observation network (AON) includes three levels (occipito-temporal, parietal and premotor), little is known concerning the specific role of these levels within perceptual tasks probing action observation. Recent single cell studies suggest that the parietal level carries the information required to discriminate between two-alternative observed actions, but do not exclude possible contributions from the other two levels.

METHODS

Two functional magnetic resonance imaging experiments used a task-based attentional modulation paradigm in which subjects viewed videos of an actor performing a manipulative action on a coloured object, and discriminated between either two observed manipulative actions, two actors or two colours.

RESULTS

Both experiments demonstrated that relative to actor and colour discrimination, discrimination between observed manipulative actions involved the putative human anterior intraparietal sulcus (phAIP) area in parietal cortex. In one experiment, where the observed actions also differed with regard to effectors, premotor cortex was also specifically recruited.

CONCLUSIONS

Our results highlight the primary role of parietal cortex in discriminating between two-alternative observed manipulative actions, consistent with the view that this level plays a major role in representing the identity of an observed action.

Bilateral engagement of the occipito-temporal cortex in response to dance kinematics in experts

Previous evidence has shown neuroplastic changes in brain anatomy and connectivity associated with the acquisition of professional visuomotor skills. Reduced hemispherical asymmetry was found in the sensorimotor and visual areas in expert musicians and athletes compared with non-experts. Moreover, increased expertise with faces, body, and objects resulted in an enhanced engagement of the occipito-temporal cortex (OTC) during stimulus observation. The present study aimed at investigating whether intense and extended practice with dance would result in an enhanced symmetric response of OTC at an early stage of action processing. Expert ballet dancers and non-dancer controls were presented with videos depicting ballet steps during EEG recording. The observation of the moving dancer elicited a posterior N2 component, being larger over the left hemisphere in dancers than controls. The source reconstruction (swLORETA) of the negativity showed the engagement of the bilateral inferior and middle temporal regions in experts, while right-lateralized activity was found in controls. The dancers also showed an early P2 and enhanced P300 responses, indicating faster stimulus processing and subsequent recognition. This evidence seemed to suggest expertise-related increased sensitivity of the OTC in encoding body kinematics. Thus, we speculated that long-term whole-body practice would result in enriched and refined action processing.

Getting a handle on virtual tools: An examination of the neuronal activity associated with virtual tool use

Tool use is associated with three visual streams-dorso-dorsal, ventro-dorsal, and ventral visual streams. These streams are involved in processing online motor planning, action semantics, and tool semantics features, respectively. Little is known about the way in which the brain represents virtual tools. To directly assess this question, a virtual tool paradigm was created that provided the ability to manipulate tool components in isolation of one another. During functional magnetic resonance imaging (fMRI), adult participants performed a series of virtual tool manipulation tasks in which vision and movement kinematics of the tool were manipulated. Reaction time and hand movement direction were monitored while the tasks were performed. Functional imaging revealed that activity within all three visual streams was present, in a similar pattern to what would be expected with physical tool use. However, a previously unreported network of right-hemisphere activity was found including right inferior parietal lobule, middle and superior temporal gyri and supramarginal gyrus - regions well known to be associated with tool processing within the left hemisphere. These results provide evidence that both virtual and physical tools are processed within the same brain regions, though virtual tools recruit bilateral tool processing regions to a greater extent than physical tools.

The Neural Bases of Event Monitoring across Domains: a Simultaneous ERP-fMRI Study

The ability to check and evaluate the environment over time with the aim to detect the occurrence of target stimuli is supported by sustained/tonic as well as transient/phasic control processes, which overall might be referred to as event monitoring. The neural underpinning of sustained attentional control processes involves a fronto-parietal network. However, it has not been well-defined yet whether this cortical circuit acts irrespective of the specific material to be monitored and whether this mediates sustained as well as transient monitoring processes. In the current study, the functional activity of brain during an event monitoring task was investigated and compared between two cognitive domains, whose processing is mediated by differently lateralized areas. Namely, participants were asked to monitor sequences of either faces (supported by right-hemisphere regions) or tools (left-hemisphere). In order to disentangle sustained from transient components of monitoring, a simultaneous EEG-fMRI technique was adopted within a block design. When contrasting monitoring versus control blocks, the conventional fMRI analysis revealed the sustained involvement of bilateral fronto-parietal regions, in both task domains. Event-related potentials (ERPs) showed a more positive amplitude over frontal sites in monitoring compared to control blocks, providing evidence of a transient monitoring component. The joint ERP-fMRI analysis showed that, in the case of face monitoring, this transient component relies on right-lateralized areas, including the inferior parietal lobule and the middle frontal gyrus. In the case of tools, no fronto-parietal areas correlated with the transient ERP activity, suggesting that in this domain phasic monitoring processes were masked by tonic ones. Overall, the present findings highlight the role of bilateral fronto-parietal regions in sustained monitoring, independently of the specific task requirements, and suggest that right-lateralized areas subtend transient monitoring processes, at least in some task contexts.

Functional coupling between frontoparietal and occipitotemporal pathways during action and perception

Several lines of evidence point to areas in the occipitotemporal pathway as being critical in the processes of visual perception and object recognition. Much less appreciated, however, is the role that this pathway plays in object-related processing for the purposes of visually guided action. Here, using functional MRI (fMRI) and functional connectivity (FC) measures, we examined interactions between areas in frontoparietal cortex (FPC) involved in grasping, reaching, eye movements, and tool use and areas in occipitotemporal cortex (OTC) involved in object-, face-, scene-, body-, tool-, and motion-related processing, both during the performance of sensorimotor and visual-perceptual tasks, as well as during passive fixation (resting-state). Cluster analysis of regional time course data identified correspondence in the patterns of FPC and OTC connectivity during the visual-perceptual tasks and rest that both tended to segregate regions along traditional dorsal/ventral pathway boundaries. During the sensorimotor tasks, however, we observed a notable separation in functional coupling between ventral-medial and ventral-lateral regions of OTC, with several of the latter areas often being clustered together with sensorimotor-defined areas in parietal cortex. These findings indicate that the functional coupling of ventral-lateral OTC areas to dorsal parietal and ventral-medial structures is flexible and task-dependent, and suggests that regions in lateral occipital cortex, in particular, may play an important role in mediating interactions between the dorsal and ventral pathways during tasks involving sensorimotor control.

Visual information about object size and object position are retained differently in the visual brain: Evidence from grasping studies

Many experiments have examined how the visual information used for action control is represented in our brain, and whether or not visually-guided and memory-guided hand movements rely on dissociable visual representations that are processed in different brain areas (dorsal vs. ventral). However, little is known about how these representations decay over longer time periods and whether or not different visual properties are retained in a similar fashion. In three experiments we investigated how information about object size and object position affect grasping as visual memory demands increase. We found that position information decayed rapidly with increasing delays between viewing the object and initiating subsequent actions - impacting both the accuracy of the transport component (lower end-point accuracy) and the grasp component (larger grip apertures) of the movement. In contrast, grip apertures and fingertip forces remained well-adjusted to target size in conditions in which positional information was either irrelevant or provided, regardless of delay, indicating that object size is encoded in a more stable manner than object position. The findings provide evidence that different grasp-relevant properties are encoded differently by the visual system. Furthermore, we argue that caution is required when making inferences about object size representations based on alterations in the grip component as these variations are confounded with the accuracy with which object position is represented. Instead fingertip forces seem to provide a reliable and confound-free measure to assess internal size estimations in conditions of increased visual uncertainty.

Beta band modulations underlie action representations for movement planning

To be able to interact with our environment, we need to transform incoming sensory information into goal-directed motor outputs. Whereas our ability to plan an appropriate movement based on sensory information appears effortless and simple, the underlying brain dynamics are still largely unknown. Here we used magnetoencephalography (MEG) to investigate this issue by recording brain activity during the planning of non-visually guided reaching and grasping actions, performed with either the left or right hand. Adopting a combination of univariate and multivariate analyses, we revealed specific patterns of beta power modulations underlying varying levels of neural representations during movement planning. (1) Effector-specific modulations were evident as a decrease in power in the beta band. Within both hemispheres, this decrease was stronger while planning a movement with the contralateral hand. (2) The comparison of planned grasping and reaching led to a relative increase in power in the beta band. These power changes were localized within temporal, premotor and posterior parietal cortices. Action-related modulations overlapped with effector-related beta power changes within widespread frontal and parietal regions, suggesting the possible integration of these two types of neural representations. (3) Multivariate analyses of action-specific power changes revealed that part of this broadband beta modulation also contributed to the encoding of an effector-independent neural representation of a planned action within fronto-parietal and temporal regions. Our results suggest that beta band power modulations play a central role in movement planning, within both the dorsal and ventral stream, by coding and integrating different levels of neural representations, ranging from the simple representation of the to-be-moved effector up to an abstract, effector-independent representation of the upcoming action.

MEG Multivariate Analysis Reveals Early Abstract Action Representations in the Lateral Occipitotemporal Cortex

Understanding other people's actions is a fundamental prerequisite for social interactions. Whether action understanding relies on simulating the actions of others in the observers' motor system or on the access to conceptual knowledge stored in nonmotor areas is strongly debated. It has been argued previously that areas that play a crucial role in action understanding should (1) distinguish between different actions, (2) generalize across the ways in which actions are performed (Dinstein et al., 2008; Oosterhof et al., 2013; Caramazza et al., 2014), and (3) have access to action information around the time of action recognition (Hauk et al., 2008). Whereas previous studies focused on the first two criteria, little is known about the dynamics underlying action understanding. We examined which human brain regions are able to distinguish between pointing and grasping, regardless of reach direction (left or right) and effector (left or right hand), using multivariate pattern analysis of magnetoencephalography data. We show that the lateral occipitotemporal cortex (LOTC) has the earliest access to abstract action representations, which coincides with the time point from which there was enough information to allow discriminating between the two actions. By contrast, precentral regions, though recruited early, have access to such abstract representations substantially later. Our results demonstrate that in contrast to the LOTC, the early recruitment of precentral regions does not contain the detailed information that is required to recognize an action. We discuss previous theoretical claims of motor theories and how they are incompatible with our data.

SIGNIFICANCE STATEMENT

It is debated whether our ability to understand other people's actions relies on the simulation of actions in the observers' motor system, or is based on access to conceptual knowledge stored in nonmotor areas. Here, using magnetoencephalography in combination with machine learning, we examined where in the brain and at which point in time it is possible to distinguish between pointing and grasping actions regardless of the way in which they are performed (effector, reach direction). We show that, in contrast to the predictions of motor theories of action understanding, the lateral occipitotemporal cortex has access to abstract action representations substantially earlier than precentral regions.

Temporal Frequency Tuning Reveals Interactions between the Dorsal and Ventral Visual Streams

Visual processing of complex objects is supported by the ventral visual pathway in the service of object identification and by the dorsal visual pathway in the service of object-directed reaching and grasping. Here, we address how these two streams interact during tool processing, by exploiting the known asymmetry in projections of subcortical magnocellular and parvocellular inputs to the dorsal and ventral streams. The ventral visual pathway receives both parvocellular and magnocellular input, whereas the dorsal visual pathway receives largely magnocellular input. We used fMRI to measure tool preferences in parietal cortex when the images were presented at either high or low temporal frequencies, exploiting the fact that parvocellular channels project principally to the ventral but not dorsal visual pathway. We reason that regions of parietal cortex that exhibit tool preferences for stimuli presented at frequencies characteristic of the parvocellular pathway receive their inputs from the ventral stream. We found that the left inferior parietal lobule, in the vicinity of the supramarginal gyrus, exhibited tool preferences for images presented at low temporal frequencies, whereas superior and posterior parietal regions exhibited tool preferences for images present at high temporal frequencies. These data indicate that object identity, processed within the ventral stream, is communicated to the left inferior parietal lobule and may there combine with inputs from the dorsal visual pathway to allow for functionally appropriate object manipulation.

Resilience to the contralateral visual field bias as a window into object representations

Viewing images of manipulable objects elicits differential blood oxygen level-dependent (BOLD) contrast across parietal and dorsal occipital areas of the human brain that support object-directed reaching, grasping, and complex object manipulation. However, it is unknown which object-selective regions of parietal cortex receive their principal inputs from the ventral object-processing pathway and which receive their inputs from the dorsal object-processing pathway. Parietal areas that receive their inputs from the ventral visual pathway, rather than from the dorsal stream, will have inputs that are already filtered through object categorization and identification processes. This predicts that parietal regions that receive inputs from the ventral visual pathway should exhibit object-selective responses that are resilient to contralateral visual field biases. To test this hypothesis, adult participants viewed images of tools and animals that were presented to the left or right visual fields during functional magnetic resonance imaging (fMRI). We found that the left inferior parietal lobule showed robust tool preferences independently of the visual field in which tool stimuli were presented. In contrast, a region in posterior parietal/dorsal occipital cortex in the right hemisphere exhibited an interaction between visual field and category: tool-preferences were strongest contralateral to the stimulus. These findings suggest that action knowledge accessed in the left inferior parietal lobule operates over inputs that are abstracted from the visual input and is contingent on analysis by the ventral visual pathway, consistent with its putative role in supporting object manipulation knowledge.

Preserved Haptic Shape Processing after Bilateral LOC Lesions

The visual and haptic perceptual systems are understood to share a common neural representation of object shape. A region thought to be critical for recognizing visual and haptic shape information is the lateral occipital complex (LOC). We investigated whether LOC is essential for haptic shape recognition in humans by studying behavioral responses and brain activation for haptically explored objects in a patient (M.C.) with bilateral lesions of the occipitotemporal cortex, including LOC. Despite severe deficits in recognizing objects using vision, M.C. was able to accurately recognize objects via touch. M.C.'s psychophysical response profile to haptically explored shapes was also indistinguishable from controls. Using fMRI, M.C. showed no object-selective visual or haptic responses in LOC, but her pattern of haptic activation in other brain regions was remarkably similar to healthy controls. Although LOC is routinely active during visual and haptic shape recognition tasks, it is not essential for haptic recognition of object shape.

SIGNIFICANCE STATEMENT

The lateral occipital complex (LOC) is a brain region regarded to be critical for recognizing object shape, both in vision and in touch. However, causal evidence linking LOC with haptic shape processing is lacking. We studied recognition performance, psychophysical sensitivity, and brain response to touched objects, in a patient (M.C.) with extensive lesions involving LOC bilaterally. Despite being severely impaired in visual shape recognition, M.C. was able to identify objects via touch and she showed normal sensitivity to a haptic shape illusion. M.C.'s brain response to touched objects in areas of undamaged cortex was also very similar to that observed in neurologically healthy controls. These results demonstrate that LOC is not necessary for recognizing objects via touch.

The lateral occipitotemporal cortex in action

Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.

Planning Ahead: Object-Directed Sequential Actions Decoded from Human Frontoparietal and Occipitotemporal Networks

Object-manipulation tasks (e.g., drinking from a cup) typically involve sequencing together a series of distinct motor acts (e.g., reaching toward, grasping, lifting, and transporting the cup) in order to accomplish some overarching goal (e.g., quenching thirst). Although several studies in humans have investigated the neural mechanisms supporting the planning of visually guided movements directed toward objects (such as reaching or pointing), only a handful have examined how manipulatory sequences of actions-those that occur after an object has been grasped-are planned and represented in the brain. Here, using event-related functional MRI and pattern decoding methods, we investigated the neural basis of real-object manipulation using a delayed-movement task in which participants first prepared and then executed different object-directed action sequences that varied either in their complexity or final spatial goals. Consistent with previous reports of preparatory brain activity in non-human primates, we found that activity patterns in several frontoparietal areas reliably predicted entire action sequences in advance of movement. Notably, we found that similar sequence-related information could also be decoded from pre-movement signals in object- and body-selective occipitotemporal cortex (OTC). These findings suggest that both frontoparietal and occipitotemporal circuits are engaged in transforming object-related information into complex, goal-directed movements.

The neural correlates of planning and executing actual tool use

Human tool use is complex, and underlying neural mechanisms seem to be widely distributed across several brain systems; however, neuroimaging studies of actual tool use are rare because of experimental challenges hindering detailed analysis within one acting subject. We developed a "Tool-Carousel" that enabled us to test actual manipulation of different objects during fMRI and investigate the planning and execution of goal-directed actions. Particularly, we focused on the effects of three factors on object manipulations: the type of object manipulated, the type of manipulation, and the hand to be used. The main focus lay on the question of how complex object use compared with unspecific actions are processed and especially how such representations interact with the knowledge about the object in the action-related dorsal stream. We found that object manipulations with both right and left hand recruit a common network strongly lateralized to the left hemisphere especially during planning but also action execution. Specifically, while activity in the ventral stream was involved in processing semantic information and object properties, a dorso-dorsal pathway (i.e., superior occipital gyrus, superior parietal lobule, and dorsal premotor area) was relevant for monitoring the online control of objects and also a ventro-dorsal pathway (i.e., middle occipital gyrus, inferior parietal lobule, and ventral premotor area) was specifically involved in processing known object manipulations, such as tool use. Data further indicate an interaction of ventral stream areas, such as middle temporal gyrus and lateral occipital complex, with both dorsal pathways. These results provide evidence for left-lateralized occipito-temporo-parieto-frontal network of everyday tool use, which may help to characterize specific deficits in patients suffering from apraxia.

Left occipitotemporal cortex contributes to the discrimination of tool-associated hand actions: fMRI and TMS evidence

Functional neuroimaging studies have implicated the left lateral occipitotemporal cortex (LOTC) in both tool and hand perception but the functional role of this region is not fully known. Here, by using a task manipulation, we tested whether tool-/hand-selective LOTC contributes to the discrimination of tool-associated hand actions. Participants viewed briefly presented pictures of kitchen and garage tools while they performed one of two tasks: in the action task, they judged whether the tool is associated with a hand rotation action (e.g., screwdriver) or a hand squeeze action (e.g., garlic press), while in the location task they judged whether the tool is typically found in the kitchen (e.g., garlic press) or in the garage (e.g., screwdriver). Both tasks were performed on the same stimulus set and were matched for difficulty. Contrasting fMRI responses between these tasks showed stronger activity during the action task than the location task in both tool- and hand-selective LOTC regions, which closely overlapped. No differences were found in nearby object- and motion-selective control regions. Importantly, these findings were confirmed by a TMS study, which showed that effective TMS over the tool-/hand-selective LOTC region significantly slowed responses for tool action discriminations relative to tool location discriminations, with no such difference during sham TMS. We conclude that left LOTC contributes to the discrimination of tool-associated hand actions.

The tool in the brain: apraxia in ADL. Behavioral and neurological correlates of apraxia in daily living

Humans differ from other animals in the way they can skilfully and precisely operate or invent tools to facilitate their everyday life. Tools have dominated our home, travel and work environment, becoming an integral step for our motor skills development. What happens when the part of the brain responsible for tool use is damaged in our adult life due to a cerebrovascular accident? How does daily life change when we lose the previously mastered ability to make use of the objects around us? How do patients suffering from compromised tool use cope with food preparation, personal hygiene, grooming, housework, or use of home appliances? In this literature review we present a state of the art for single and multiple tool use research, with a focus on the impact that apraxia (impaired ability to perform tool-based actions) and action disorganization syndrome (ADS; impaired ability to carry out multi-step actions) have on activities of daily living (ADL). Firstly, we summarize the behavioral studies investigating the impact of apraxia and other comorbidity syndromes, such as neglect or visual extinction, on ADL. We discuss the hallmarks of the compromised tool use in terms of the sequencing of action steps, conceptual errors committed, spatial motor control, and temporal organization of the movement. In addition, we present an up-to-date overview of the neuroimaging and lesion analyses studies that provide an insight into neural correlates of tool use in the human brain and functional changes in the neural organization following a stroke, in the context of ADL. Finally we discuss the current practice in neurorehabilitation of ADL in apraxia and ADS aiming at increasing patients' independence.

Body and object effectors: the organization of object representations in high-level visual cortex reflects body-object interactions

The principles driving the functional organization of object representations in high-level visual cortex are not yet fully understood. In four human fMRI experiments, we provide evidence that the organization of high-level visual cortex partly reflects the degree to which objects are typically controlled by the body to interact with the world, thereby extending the body's boundaries. Univariate whole-brain analysis showed an overlap between responses to body effectors (e.g., hands, feet, and limbs) and object effectors (e.g., hammers, combs, and tennis rackets) in lateral occipitotemporal cortex (LOTC) and parietal cortex. Region of interest analyses showed that a hand-selective region in left LOTC responded selectively to object effectors relative to a range of noneffector object control conditions (e.g., graspable objects, "act-on" objects, musical instruments). Object ratings showed that the strong response to object effectors in hand-selective LOTC was not due to general action-related object properties shared with these control conditions, such as hand priming, hand grasping, and hand-action centrality. Finally, whole-brain representational similarity analysis revealed that the similarity of multivoxel object response patterns in left lateral occipitotemporal cortex selectively predicted the degree to which objects were rated as being controlled by and extending the body. Together, these results reveal a clustering of body and object effector representations, indicating that the organization of object representations in high-level visual cortex partly reflects how objects relate to the body.

Seeking a bridge between language and motor cortices: a PPI study

The relation between the sensorimotor cortex and the language network has been widely discussed but still remains controversial. Two independent theories compete to explain how this area is involved during action-related verbs processing. The embodied view assumes that action word representations activate sensorimotor representations which are accessed when an action word is processed or when an action is observed. The abstract hypothesis states that the mental representations of words are abstract and independent of the objects' sensorimotor properties they refer to. We combined neuropsychological and fMRI-PPI connectivity data, to address action-related verbs processing in neurosurgical patients with lesions involving (N = 5) or sparing (N = 5) the primary motor cortex and healthy controls (N = 12). A lack of significant changes in the functional coupling between the left M1 cortex and functional nodes of the linguistic network during the verb generation task was found for all the groups. In addition, we found that the ability to perform an action verb naming task was not related to a damaged M1. These data showed that there was not a task-specific functional interaction active between M1 and the inferior frontal gyrus. We will discuss how these findings indicate that action words do not automatically activate the M1 cortex; we suggest rather that its enrolment could be related to other not strictly linguistic processing.

Decoding the neural mechanisms of human tool use

Sophisticated tool use is a defining characteristic of the primate species but how is it supported by the brain, particularly the human brain? Here we show, using functional MRI and pattern classification methods, that tool use is subserved by multiple distributed action-centred neural representations that are both shared with and distinct from those of the hand. In areas of frontoparietal cortex we found a common representation for planned hand- and tool-related actions. In contrast, in parietal and occipitotemporal regions implicated in hand actions and body perception we found that coding remained selectively linked to upcoming actions of the hand whereas in parietal and occipitotemporal regions implicated in tool-related processing the coding remained selectively linked to upcoming actions of the tool. The highly specialized and hierarchical nature of this coding suggests that hand- and tool-related actions are represented separately at earlier levels of sensorimotor processing before becoming integrated in frontoparietal cortex. DOI:http://dx.doi.org/10.7554/eLife.00425.001.

Tool selectivity in left occipitotemporal cortex develops without vision

Previous studies have provided evidence for a tool-selective region in left lateral occipitotemporal cortex (LOTC). This region responds selectively to pictures of tools and to characteristic visual tool motion. The present human fMRI study tested whether visual experience is required for the development of tool-selective responses in left LOTC. Words referring to tools, animals, and nonmanipulable objects were presented auditorily to 14 congenitally blind and 16 sighted participants. Sighted participants additionally viewed pictures of these objects. In whole-brain group analyses, sighted participants showed tool-selective activity in left LOTC in both visual and auditory tasks. Importantly, virtually identical tool-selective LOTC activity was found in the congenitally blind group performing the auditory task. Furthermore, both groups showed equally strong tool-selective activity for auditory stimuli in a tool-selective LOTC region defined by the picture-viewing task in the sighted group. Detailed analyses in individual participants showed significant tool-selective LOTC activity in 13 of 14 blind participants and 14 of 16 sighted participants. The strength and anatomical location of this activity were indistinguishable across groups. Finally, both blind and sighted groups showed significant resting state functional connectivity between left LOTC and a bilateral frontoparietal network. Together, these results indicate that tool-selective activity in left LOTC develops without ever having seen a tool or its motion. This finding puts constraints on the possible role that this region could have in tool processing and, more generally, provides new insights into the principles shaping the functional organization of OTC.