fMRI repetition suppression for familiar but not arbitrary actions with tools

The motifs of radical embodied neuroscience

In this paper, I analyse how the emerging scientific framework of radical embodied neuroscience is different from contemporary mainstream cognitive neuroscience. To do so, I propose the notion of motif to enrich the philosophical toolkit of cognitive neuroscience. This notion can be used to characterize the guiding ideas of any given scientific framework in psychology and neuroscience. Motifs are highly unconstrained, open-ended concepts that support equally open-ended families of explanations. Different scientific frameworks-e.g., psychophysics or cognitive neuroscience-provide these motifs to answer the overarching themes of these disciplines, such as the relationship between stimuli and sensations or the proper methods of the sciences of the mind. Some motifs of mainstream cognitive neuroscience are the motif of encoding, the motif of input-output systems, and the motif of algorithms. The two first ones answer the question about the relationship between stimuli, sensations and experience (e.g., stimuli are input and are encoded by brain structures). The latter one answers the question regarding the mechanism of cognition and experience. The three of them are equally unconstrained and open-ended, and they serve as an umbrella for different kinds of explanation-i.e., different positions regarding what counts as a code or as an input. Along with the articulation of the notion of motif, the main aim of this article is to present three motifs for radical embodied neuroscience: the motif of complex stimulation, the motif of organic behaviour and the motif of resonance.

Internal representations of the canonical real-world distance of objects

In the real world, every object has its canonical distance from observers. For example, airplanes are usually far away from us, whereas eyeglasses are close to us. Do we have an internal representation of the canonical real-world distance of objects in our cognitive system? If we do, does the canonical distance influence the perceived size of an object? Here, we conducted two experiments to address these questions. In Experiment 1, we first asked participants to rate the canonical distance of objects. Participants gave consistent ratings to each object. Then, pairs of object images were presented one by one in a trial, and participants were asked to rate the distance of the second object (i.e., a priming paradigm). We found that the rating of the perceived distance of the target object was modulated by the canonical real-world distance of the prime. In Experiment 2, participants were asked to judge the perceived size of canonically near or far objects that were presented at the converging end (i.e., far location) or the opening end (i.e., near location) of a background image with converging lines. We found that regardless of the presentation location, participants perceived the canonically near object as smaller than the canonically far object even though their retinal and real-world sizes were matched. In all, our results suggest that we have an internal representation of the canonical real-world distance of objects, which affects the perceived distance of subsequent objects and the perceived size of the objects themselves.

The functional organization of skilled actions in the adextral and atypical brain

When planning functional grasps of tools, right-handed individuals (dextrals) show mostly left-lateralized neural activity in the praxis representation network (PRN), regardless of the used hand. Here we studied whether or not similar cerebral asymmetries are evident in non-righthanded individuals (adextrals). Sixty two participants, 28 righthanders and 34 non-righthanders (21 lefthanders, 13 mixedhanders), planned functional grasps of tools vs. grasps of control objects, and subsequently performed their pantomimed executions, in an event-related functional magnetic resonance imaging (fMRI) project. Both hands were tested, separately in two different sessions, counterbalanced across participants. After accounting for non-functional components of the prospective grasp, planning functional grasps of tools was associated with greater engagement of the same, left-hemisphere occipito-temporal, parietal and frontal areas of PRN, regardless of hand and handedness. Only when the analyses involved signal changes referenced to resting baseline intervals, differences between adextrals and dextrals emerged. Whereas in the left hemisphere the neural activity was equivalent in both groups (except for the occipito-temporo-parietal junction), its increases in the right occipito-temporal cortex, medial intraparietal sulcus (area MIP), the supramarginal gyrus (area PFt/PF), and middle frontal gyrus (area p9-46v) were significantly greater in adextrals. The inverse contrast was empty. Notably, when individuals with atypical and typical hemispheric phenotypes were directly compared, planning functional (vs. control) grasps invoked, instead, significant clusters located nearly exclusively in the left hemisphere of the typical phenotype. Previous studies interpret similar right-sided vs. left-sided increases in neural activity for skilled actions as handedness dependent, i.e., located in the hemisphere dominant for manual skills. Yet, none of the effects observed here can be purely handedness dependent because there were mixed-handed individuals among adextrals, and numerous mixed-handed and left-handed individuals possess the typical phenotype. Thus, our results clearly show that hand dominance has limited power in driving the cerebral organization of motor cognitive functions.

Human Neuroimaging Reveals Differences in Activation and Connectivity between Real and Pantomimed Tool Use

Because the sophistication of tool use is vastly enhanced in humans compared with other species, a rich understanding of its neural substrates requires neuroscientific experiments in humans. Although functional magnetic resonance imaging (fMRI) has enabled many studies of tool-related neural processing, surprisingly few studies have examined real tool use. Rather, because of the many constraints of fMRI, past research has typically used proxies such as pantomiming despite neuropsychological dissociations between pantomimed and real tool use. We compared univariate activation levels, multivariate activation patterns, and functional connectivity when participants used real tools (a plastic knife or fork) to act on a target object (scoring or poking a piece of putty) or pantomimed the same actions with similar movements and timing. During the Execute phase, we found higher activation for real versus pantomimed tool use in sensorimotor regions and the anterior supramarginal gyrus, and higher activation for pantomimed than real tool use in classic tool-selective areas. Although no regions showed significant differences in activation magnitude during the Plan phase, activation patterns differed between real versus pantomimed tool use and motor cortex showed differential functional connectivity. These results reflect important differences between real tool use, a closed-loop process constrained by real consequences, and pantomimed tool use, a symbolic gesture that requires conceptual knowledge of tools but with limited consequences. These results highlight the feasibility and added value of employing natural tool use tasks in functional imaging, inform neuropsychological dissociations, and advance our theoretical understanding of the neural substrates of natural tool use.SIGNIFICANCE STATEMENT The study of tool use offers unique insights into how the human brain synthesizes perceptual, cognitive, and sensorimotor functions to accomplish a goal. We suggest that the reliance on proxies, such as pantomiming, for real tool use has (1) overestimated the contribution of cognitive networks, because of the indirect, symbolic nature of pantomiming; and (2) underestimated the contribution of sensorimotor networks necessary for predicting and monitoring the consequences of real interactions between hand, tool, and the target object. These results enhance our theoretical understanding of the full range of human tool functions and inform our understanding of neuropsychological dissociations between real and pantomimed tool use.

Investigating the effects of the aging brain on real tool use performance-an fMRI study

Introduction

Healthy aging affects several domains of cognitive and motor performance and is further associated with multiple structural and functional neural reorganization patterns. However, gap of knowledge exists, referring to the impact of these age-related alterations on the neural basis of tool use-an important, complex action involved in everyday life throughout the entire lifespan. The current fMRI study aims to investigate age-related changes of neural correlates involved in planning and executing a complex object manipulation task, further providing a better understanding of impaired tool use performance in apraxia patients.

Methods

A balanced number of sixteen older and younger healthy adults repeatedly manipulated everyday tools in an event-related Go-No-Go fMRI paradigm.

Results

Our data indicates that the left-lateralized network, including widely distributed frontal, temporal, parietal and occipital regions, involved in tool use performance is not subjected to age-related functional reorganization processes. However, age-related changes regarding the applied strategical procedure can be detected, indicating stronger investment into the planning, preparatory phase of such an action in older participants.

Variational relevance evaluation of individual fMRI data enables deconstruction of task-dependent neural dynamics

In neuroimaging research, univariate analysis has always been used to localize "representations" at the microscale, whereas network approaches have been applied to characterize transregional "operations". How are representations and operations linked through dynamic interactions? We developed the variational relevance evaluation (VRE) method to analyze individual task fMRI data, which selects informative voxels during model training to localize the "representation", and quantifies the dynamic contributions of single voxels across the whole-brain to different cognitive functions to characterize the "operation". Using 15 individual fMRI data files for higher visual area localizers, we evaluated the characterization of selected voxel positions of VRE and revealed different object-selective regions functioning in similar dynamics. Using another 15 individual fMRI data files for memory retrieval after offline learning, we found similar task-related regions working in different neural dynamics for tasks with diverse familiarities. VRE demonstrates a promising horizon in individual fMRI research.

The parietal lobe evolution and the emergence of material culture in the human genus

Traditional and new disciplines converge in suggesting that the parietal lobe underwent a considerable expansion during human evolution. Through the study of endocasts and shape analysis, paleoneurology has shown an increased globularity of the braincase and bulging of the parietal region in modern humans, as compared to other human species, including Neandertals. Cortical complexity increased in both the superior and inferior parietal lobules. Emerging fields bridging archaeology and neuroscience supply further evidence of the involvement of the parietal cortex in human-specific behaviors related to visuospatial capacity, technological integration, self-awareness, numerosity, mathematical reasoning and language. Here, we complement these inferences on the parietal lobe evolution, with results from more classical neuroscience disciplines, such as behavioral neurophysiology, functional neuroimaging, and brain lesions; and apply these to define the neural substrates and the role of the parietal lobes in the emergence of functions at the core of material culture, such as tool-making, tool use and constructional abilities.

Action goals and the praxis network: an fMRI study

The praxis representation network (PRN) of the left cerebral hemisphere is typically linked to the control of functional interactions with familiar tools. Surprisingly, little is known about the PRN engagement in planning and execution of tool-directed actions motivated by non-functional but purposeful action goals. Here we used functional neuroimaging to perform both univariate and multi-voxel pattern analyses (MVPA) in 20 right-handed participants who planned and later executed, with their dominant and non-dominant hands, disparate grasps of tools for different goals, including: (1) planning simple vs. demanding functional grasps of conveniently vs. inconveniently oriented tools with an intention to immediately use them, (2) planning simple—but non-functional—grasps of inconveniently oriented tools with a goal to pass them to a different person, (3) planning reaching movements directed at such tools with an intention to move/push them with the back of the hand, and (4) pantomimed execution of the earlier planned tasks. While PRN contributed to the studied interactions with tools, the engagement of its critical nodes, and/or complementary right hemisphere processing, was differently modulated by task type. E.g., planning non-functional/structural grasp-to-pass movements of inconveniently oriented tools, regardless of the hand, invoked the left parietal and prefrontal nodes significantly more than simple, non-demanding functional grasps. MVPA corroborated decoding capabilities of critical PRN areas and some of their right hemisphere counterparts. Our findings shed new lights on how performance of disparate action goals influences the extraction of object affordances, and how or to what extent it modulates the neural activity within the parieto-frontal brain networks.

The role of the anterior temporal cortex in action: evidence from fMRI multivariate searchlight analysis during real object grasping

Intelligent manipulation of handheld tools marks a major discontinuity between humans and our closest ancestors. Here we identified neural representations about how tools are typically manipulated within left anterior temporal cortex, by shifting a searchlight classifier through whole-brain real action fMRI data when participants grasped 3D-printed tools in ways considered typical for use (i.e., by their handle). These neural representations were automatically evocated as task performance did not require semantic processing. In fact, findings from a behavioural motion-capture experiment confirmed that actions with tools (relative to non-tool) incurred additional processing costs, as would be suspected if semantic areas are being automatically engaged. These results substantiate theories of semantic cognition that claim the anterior temporal cortex combines sensorimotor and semantic content for advanced behaviours like tool manipulation.

Domain-specific connectivity drives the organization of object knowledge in the brain

The goal of this chapter is to review neuropsychological and functional MRI findings that inform a theory of the causes of functional specialization for semantic categories within occipito-temporal cortex-the ventral visual processing pathway. The occipito-temporal pathway supports visual object processing and recognition. The theoretical framework that drives this review considers visual object recognition through the lens of how "downstream" systems interact with the outputs of visual recognition processes. Those downstream processes include conceptual interpretation, grasping and object use, navigating and orienting in an environment, physical reasoning about the world, and inferring future actions and the inner mental states of agents. The core argument of this chapter is that innately constrained connectivity between occipito-temporal areas and other regions of the brain is the basis for the emergence of neural specificity for a limited number of semantic domains in the brain.

Neural encoding and functional interactions underlying pantomimed movements

Pantomimes are a unique movement category which can convey complex information about our intentions in the absence of any interaction with real objects. Indeed, we can pretend to use the same tool to perform different actions or to achieve the same goal adopting different tools. Nevertheless, how our brain implements pantomimed movements is still poorly understood. In our study, we explored the neural encoding and functional interactions underlying pantomimes adopting multivariate pattern analysis (MVPA) and connectivity analysis of fMRI data. Participants performed pantomimed movements, either grasp-to-move or grasp-to-use, as if they were interacting with two different tools (scissors or axe). These tools share the possibility to achieve the same goal. We adopted MVPA to investigate two levels of representation during the planning and execution of pantomimes: (1) distinguishing different actions performed with the same tool, (2) representing the same final goal irrespective of the adopted tool. We described widespread encoding of action information within regions of the so-called “tool” network. Several nodes of the network—comprising regions within the ventral and the dorsal stream—also represented goal information. The spatial distribution of goal information changed from planning—comprising posterior regions (i.e. parietal and temporal)—to execution—including also anterior regions (i.e. premotor cortex). Moreover, connectivity analysis provided evidence for task-specific bidirectional coupling between the ventral stream and parieto-frontal motor networks. Overall, we showed that pantomimes were characterized by specific patterns of action and goal encoding and by task-dependent cortical interactions.

Hand-Selective Visual Regions Represent How to Grasp 3D Tools: Brain Decoding during Real Actions

Most neuroimaging experiments that investigate how tools and their actions are represented in the brain use visual paradigms where tools or hands are displayed as 2D images and no real movements are performed. These studies discovered selective visual responses in occipitotemporal and parietal cortices for viewing pictures of hands or tools, which are assumed to reflect action processing, but this has rarely been directly investigated. Here, we examined the responses of independently visually defined category-selective brain areas when participants grasped 3D tools (N = 20; 9 females). Using real-action fMRI and multivoxel pattern analysis, we found that grasp typicality representations (i.e., whether a tool is grasped appropriately for use) were decodable from hand-selective areas in occipitotemporal and parietal cortices, but not from tool-, object-, or body-selective areas, even if partially overlapping. Importantly, these effects were exclusive for actions with tools, but not for biomechanically matched actions with control nontools. In addition, grasp typicality decoding was significantly higher in hand than tool-selective parietal regions. Notably, grasp typicality representations were automatically evoked even when there was no requirement for tool use and participants were naive to object category (tool vs nontools). Finding a specificity for typical tool grasping in hand-selective, rather than tool-selective, regions challenges the long-standing assumption that activation for viewing tool images reflects sensorimotor processing linked to tool manipulation. Instead, our results show that typicality representations for tool grasping are automatically evoked in visual regions specialized for representing the human hand, the primary tool of the brain for interacting with the world.

Decoding stimuli (tool-hand) and viewpoint invariant grasp-type information

When we see a manipulable object (henceforth tool) or a hand performing a grasping movement, our brain is automatically tuned to how that tool can be grasped (i.e., its affordance) or what kind of grasp that hand is performing (e.g., a power or precision grasp). However, it remains unclear where visual information related to tools or hands are transformed into abstract grasp representations. We therefore investigated where different levels of abstractness in grasp information are processed: grasp information that is invariant to the kind of stimuli that elicits it (tool-hand invariance); and grasp information that is hand-specific but viewpoint-invariant (viewpoint invariance). We focused on brain areas activated when viewing both tools and hands, i.e., the posterior parietal cortices (PPC), ventral premotor cortices (PMv), and lateral occipitotemporal cortex/posterior middle temporal cortex (LOTC/pMTG). To test for invariant grasp representations, we presented participants with tool images and grasp videos (from first or third person perspective; 1pp or 3pp) inside an MRI scanner, and cross-decoded power versus precision grasps across (i) grasp perspectives (viewpoint invariance), (ii) tool images and grasp 1pp videos (tool-hand 1pp invariance), and (iii) tool images and grasp 3pp videos (tool-hand 3pp invariance). Tool-hand 1pp, but not tool-hand 3pp, invariant grasp information was found in left PPC, whereas viewpoint-invariant information was found bilaterally in PPC, left PMv, and left LOTC/pMTG. These findings suggest different levels of abstractness-where visual information is transformed into stimuli-invariant grasp representations/tool affordances in left PPC, and viewpoint invariant but hand-specific grasp representations in the hand network.

"Neural differences in expert guitarists during over-learned non-standard visuomotor mapping of abstract versus concrete information"

Using visual information to perform actions is a fundamental aspect of human behavior. Musicians commonly translate visual information into action using both concrete and abstract visual information. We exposed expert guitarists to four types of familiar visual depictions of action instruction including musical notation (very abstract), tablature (abstract), chord diagrams (more concrete), and actual pictures of guitars chords being formed (very concrete). These were shown during fMRI scanning as the guitarists formed the appropriate chords (as visually depicted) on a magnet safe guitar fret board with strings, or where they simply viewed the visual stimuli without an action. Whole brain contrasts revealed that the right precuneus was more active for abstract instruction while an occipito-insular circuit was more active for concrete instruction. The current findings highlight that the degree of over-learned visual abstraction is an important factor modulating visual-motor processing.

Distinct auditory and visual tool regions with multisensory response properties in human parietal cortex

Left parietal cortex has been associated with the human-specific ability of sophisticated tool use. Yet, it is unclear how tool information is represented across senses. Here, we compared auditory and visual tool-specific activations within healthy human subjects to probe the relation of tool-specific networks, uni- and multisensory response properties, and functional and structural connectivity using functional and diffusion-weighted MRI. In each subject, we identified an auditory tool network with regions in left anterior inferior parietal cortex (aud-aIPL), bilateral posterior lateral sulcus, and left inferior precentral sulcus, and a visual tool network with regions in left aIPL (vis-aIPL) and bilateral inferior temporal gyrus. Aud-aIPL was largely separate and anterior/inferior from vis-aIPL, with varying degrees of overlap across subjects. Both regions displayed a strong preference for tools versus other stimuli presented within the same modality. Despite their modality preference, aud-aIPL and vis-aIPL and a region in left inferior precentral sulcus displayed multisensory response properties, as revealed in multivariate analyses. Thus, two largely separate tool networks are engaged by the visual and auditory modalities with nodes in parietal and prefrontal cortex potentially integrating information across senses. The diversification of tool processing in human parietal cortex underpins its critical role in complex object processing.

Invariant representations of mass in the human brain

An intuitive understanding of physical objects and events is critical for successfully interacting with the world. Does the brain achieve this understanding by running simulations in a mental physics engine, which represents variables such as force and mass, or by analyzing patterns of motion without encoding underlying physical quantities? To investigate, we scanned participants with fMRI while they viewed videos of objects interacting in scenarios indicating their mass. Decoding analyses in brain regions previously implicated in intuitive physical inference revealed mass representations that generalized across variations in scenario, material, friction, and motion energy. These invariant representations were found during tasks without action planning, and tasks focusing on an orthogonal dimension (object color). Our results support an account of physical reasoning where abstract physical variables serve as inputs to a forward model of dynamics, akin to a physics engine, in parietal and frontal cortex.

Language as a Tool: Motor Proficiency Using a Tool Predicts Individual Linguistic Abilities

Different disciplines converge to trace language evolution from motor skills. The human ability to use tools has been advocated as a fundamental step toward the emergence of linguistic processes in the brain. Neuropsychological and neuroimaging research has established that linguistic functions and tool-use are mediated by partially overlapping brain networks. Yet, scholars still theoretically debate whether the relationship between tool-use and language is contingent or functionally relevant, since empirical evidence is critically missing. Here, we measured both linguistic production and tool-use abilities in the same participants, as well as manual and linguistic motor skills. A path analysis ruling out unspecific contributions from manual or linguistic motor skills, showed that motor proficiency using a tool lawfully predicts differences in individual linguistic production. In addition, more complex tool-use reveals stronger association between linguistic production and tool mastery. These findings establish the existence of shared cognitive processes between tool-use and language.

Grasping with a new hand: Improved performance and normalized grasp-selective brain responses despite persistent functional changes in primary motor cortex and low-level sensory and motor impairments

Hand loss can now be reversed through surgical transplantation years or decades after amputation. Remarkably, these patients come to use their new hand to skilfully grasp and manipulate objects. The brain mechanisms that make this possible are unknown. Here we test the hypothesis that the anterior intraparietal cortex (aIPC) - a multimodal region implicated in hand preshaping and error correction during grasping - plays a key role in this compensatory grasp control. Motion capture and fMRI are used to characterize hand kinematics and brain responses during visually guided grasping with a transplanted hand at 26 and 41 months post-transplant in patient DR, a former hand amputee of 13 years. Compared with matched controls, DR shows increasingly normal grasp kinematics paralleled by increasingly robust grasp-selective fMRI responses within the very same brain areas that show grasp-selectivity in controls, including the aIPC, premotor and cerebellar cortices. Paradoxically, over this same time DR exhibits significant limitations in basic sensory and motor functions, and persistent amputation-related functional reorganization of primary motor cortex. Movements of the non-transplanted hand positively activate the ipsilateral primary motor hand area - a functional marker of persistent interhemispheric amputation-related reorganization. Our data demonstrate for the first time that even after more than a decade of living as an amputee the normative functional brain organization governing the control of grasping can be restored. We propose that the aIPC and interconnected premotor and cerebellar cortices enable grasp normalization by compensating for the functional impact of reorganizational changes in primary sensorimotor cortex and targeting errors in regenerating peripheral nerves.

Decoding Brain States for Planning Functional Grasps of Tools: A Functional Magnetic Resonance Imaging Multivoxel Pattern Analysis Study

OBJECTIVES

We used multivoxel pattern analysis (MVPA) to investigate neural selectivity for grasp planning within the left-lateralized temporo-parieto-frontal network of areas (praxis representation network, PRN) typically associated with tool-related actions, as studied with traditional neuroimaging contrasts.

METHODS

We used data from 20 participants whose task was to plan functional grasps of tools, with either right or left hands. Region of interest and whole-brain searchlight analyses were performed to show task-related neural patterns.

RESULTS

MVPA revealed significant contributions to functional grasp planning from the anterior intraparietal sulcus (aIPS) and its immediate vicinities, supplemented by inputs from posterior subdivisions of IPS, and the ventral lateral occipital complex (vLOC). Moreover, greater local selectivity was demonstrated in areas near the superior parieto-occipital cortex and dorsal premotor cortex, putatively forming the dorso-dorsal stream.

CONCLUSIONS

A contribution from aIPS, consistent with its role in prospective grasp formation and/or encoding of relevant tool properties (e.g., potential graspable parts), is likely to accompany the retrieval of manipulation and/or mechanical knowledge subserved by the supramarginal gyrus for achieving action goals. An involvement of vLOC indicates that MVPA is particularly sensitive to coding of object properties, their identities and even functions, for a support of grip formation. Finally, the engagement of the superior parieto-frontal regions as revealed by MVPA is consistent with their selectivity for transient features of tools (i.e., variable affordances) for anticipatory hand postures. These outcomes support the notion that, compared to traditional approaches, MVPA can reveal more fine-grained patterns of neural activity. (JINS, 2018, 24, 1013-1025).

Concurrent Cortical Representations of Function- and Size-Related Object Affordances: An fMRI Study

Previous work has shown that the perception of a graspable object may automatically potentiate actions that are tailored to specific action-related features of the object (e.g., its size) and may be related to its immediate grasping as well as to its long-term, functional use. We investigated the neural correlates of function- and size-related object affordances that may be concurrently potentiated by a graspable object. Participants were lying in a MR scanner holding a large switch in one hand and a small switch in the other hand. They passively attended a large or a small object with clearly separated functional and graspable end that was displayed centrally at an average angle of 45 degrees. Participants responded to the direction of an arrow that was overlaid on the object after a mean period of 1,000 ms after object onset and was pointing to the left or to the right with equal probability. Response times were shorter when the arrow pointed to the functional end of the object and when the responses were made with the switch that was congruent to the size of the perceived object. A clear distinction was found in the representation of function- and size-related affordances; the former was represented in the posterior parietal cortex and the latter in prefrontal, premotor, and primary sensorimotor cortices. We conclude that different aspects of object-directed actions may be automatically potentiated by individual object features and are represented in distinct brain areas.

Fronto-parietal coding of goal-directed actions performed by artificial agents

With advances in technology, artificial agents such as humanoid robots will soon become a part of our daily lives. For safe and intuitive collaboration, it is important to understand the goals behind their motor actions. In humans, this process is mediated by changes in activity in fronto-parietal brain areas. The extent to which these areas are activated when observing artificial agents indicates the naturalness and easiness of interaction. Previous studies indicated that fronto-parietal activity does not depend on whether the agent is human or artificial. However, it is unknown whether this activity is modulated by observing grasping (self-related action) and pointing actions (other-related action) performed by an artificial agent depending on the action goal. Therefore, we designed an experiment in which subjects observed human and artificial agents perform pointing and grasping actions aimed at two different object categories suggesting different goals. We found a signal increase in the bilateral inferior parietal lobule and the premotor cortex when tool versus food items were pointed to or grasped by both agents, probably reflecting the association of hand actions with the functional use of tools. Our results show that goal attribution engages the fronto-parietal network not only for observing a human but also a robotic agent for both self-related and social actions. The debriefing after the experiment has shown that actions of human-like artificial agents can be perceived as being goal-directed. Therefore, humans will be able to interact with service robots intuitively in various domains such as education, healthcare, public service, and entertainment.

Does it fit? - Impaired affordance perception after stroke

Affordance perception comprises the evaluation of whether our given bodily capabilities and properties of the environment allow particular actions. Typical impairments after left brain damage in motor cognition as well as after right brain damage in visuo-spatial abilities may affect the evaluation of whether interactions with objects are possible. Further it is unclear whether deficient motor function is accounted for when deciding upon action opportunities. For these purposes we developed a paradigm with two tasks that differ in their type of demands on affordance perception and tested it in healthy young adults (Randerath and Frey, 2016). Here, we applied one of these two tasks in stroke patients and age matched healthy participants. A sample of 34 stroke patients with either left (LBD) or right brain damage (RBD) and 29 healthy controls made decisions about whether their hands would fit through a defined horizontal aperture presented in various sizes, while they remained still. Data was analyzed using a detection theory approach and included criterion, perceptual sensitivity and diagnostic accuracy as dependent variables. In addition we applied modern voxel based lesion analyses to explore neural correlates. Compared to controls, both patient groups demonstrated lower perceptual sensitivity. As predicted, increased motor cognitive deficiencies after left brain damage and visuo-spatial deficits after right brain damage were associated with worse performance. Preliminary lesion analyses demonstrated that next to lesions in ventro-dorsal regions, damage in the cortex-claustrum-cingulate pathway may affect perceptual sensitivity. Results were similar for left and right brain damage suggesting a bilateral network. Accordingly, we propose that perceptual sensitivity for affordance based judgments is a capability depending on motor-cognitive and visuo-spatial processing, which frequently is deficient after left or right brain damage, respectively. Further research on diagnostics and training in affordance perception after brain damage is needed.

When the affordances disappear: Dynamical and computational explanations of optic ataxia

Two options fuel the debate on the cognitive processes underlying the perception of affordances. On the one hand, the ecological theory of affordance fits with the methodological assumptions of the dynamical systems theory of cognition. On the other hand, it is nowadays common to conceive the perception of affordances within a computational framework. This article defends the explanatory power of a computational approach and aims to extend the concept of affordance beyond the boundaries of the dynamical systems theory of cognition. For that purpose, I consider the case of patients suffering from optic ataxia, a condition in which some aspects of visual guidance over reaching with the hand are lost following a lesion in the left parietal cortex. Etiological considerations, indeed, reveal that a computational approach to the perception of affordances allows for an explanation of ataxic behavior that is not available to the dynamical systems theory.

Planning Functional Grasps of Simple Tools Invokes the Hand-independent Praxis Representation Network: An fMRI Study

OBJECTIVES

Neuropsychological and neuroimaging evidence indicates that tool use knowledge and abilities are represented in the praxis representation network (PRN) of the left cerebral hemisphere. We investigated whether PRN would also underlie the planning of function-appropriate grasps of tools, even though such an assumption is inconsistent with some neuropsychological evidence for independent representations of tool grasping and skilled tool use.

METHODS

Twenty right-handed participants were tested in an event-related functional magnetic resonance imaging (fMRI) study wherein they planned functionally appropriate grasps of tools versus grasps of non-tools matched for size and/or complexity, and later executed the pantomimed grasps of these objects. The dominant right, and non-dominant left hands were used in two different sessions counterbalanced across participants. The tool and non-tool stimuli were presented at three different orientations, some requiring uncomfortable hand rotations for effective grips, with the difficulty matched for both hands.

RESULTS

Planning functional grasps of tools (vs. non-tools) was associated with significant asymmetrical increases of activity in the temporo/occipital-parieto-frontal networks. The greater involvement of the left hemisphere PRN was particularly evident when hand movement kinematics (including wrist rotations) for grasping tools and non-tools were matched. The networks engaged in the task for the dominant and non-dominant hand were virtually identical. The differences in neural activity for the two object categories disappeared during grasp execution.

CONCLUSIONS

The greater hand-independent engagement of the left-hemisphere praxis representation network for planning functional grasps reveals a genuine effect of an early affordance/function-based visual processing of tools. (JINS, 2017, 23, 108-120).

Distinct Neural Suppression and Encoding Effects for Conceptual Novelty and Familiarity

Like yin and yang, novelty and familiarity are typically described as separate-yet-complementary aspects of an experience, two ends of a single continuum. However, novelty and familiarity are also multifaceted. For instance, novelty can sometimes result in enhanced mnemonic performance, whereas at other times familiarity is better remembered. As previous investigations focused primarily on the experimental aspect of novelty, the mechanisms supporting conceptual novelty (the novel combination of two previously unrelated existing concepts) remain unclear. Importantly, conceptual novelty can be recognized as such only when compared with preexperimental familiar knowledge, regardless of experimental status. Here we applied a combined repetition suppression/subsequent memory fMRI paradigm, focusing on the conceptual aspect of novelty and familiarity as the subject matter. Conceptual novelty was characterized by sustained neural activity; familiarity, on the other hand, exhibited repetition effects in multiple cortical regions, a subset of which was modulated by successful encoding. Subsequent memory of novelty was associated only with activation differences in a distinct set of regions, including the hippocampus and medial cortical regions. These results suggest that conceptual novelty (a) does not (easily) trigger the repetition suppression phenomenon but requires sustained neural recruitment and (b) activates dedicated encoding mechanisms. Conceptual familiarity, in contrast, allows rapid neural processing that depends upon existing neural representations. Overall, these findings challenge the definition of novelty as a unitary concept. Furthermore, they bear important implications for research into the neural bases of knowledge representation and recognition memory.

'What' Is Happening in the Dorsal Visual Pathway

The cortical visual system is almost universally thought to be segregated into two anatomically and functionally distinct pathways: a ventral occipitotemporal pathway that subserves object perception, and a dorsal occipitoparietal pathway that subserves object localization and visually guided action. Accumulating evidence from both human and non-human primate studies, however, challenges this binary distinction and suggests that regions in the dorsal pathway contain object representations that are independent of those in ventral cortex and that play a functional role in object perception. We review here the evidence implicating dorsal object representations, and we propose an account of the anatomical organization, functional contributions, and origins of these representations in the service of perception.

Functional neuroanatomy of intuitive physical inference

To engage with the world-to understand the scene in front of us, plan actions, and predict what will happen next-we must have an intuitive grasp of the world's physical structure and dynamics. How do the objects in front of us rest on and support each other, how much force would be required to move them, and how will they behave when they fall, roll, or collide? Despite the centrality of physical inferences in daily life, little is known about the brain mechanisms recruited to interpret the physical structure of a scene and predict how physical events will unfold. Here, in a series of fMRI experiments, we identified a set of cortical regions that are selectively engaged when people watch and predict the unfolding of physical events-a "physics engine" in the brain. These brain regions are selective to physical inferences relative to nonphysical but otherwise highly similar scenes and tasks. However, these regions are not exclusively engaged in physical inferences per se or, indeed, even in scene understanding; they overlap with the domain-general "multiple demand" system, especially the parts of that system involved in action planning and tool use, pointing to a close relationship between the cognitive and neural mechanisms involved in parsing the physical content of a scene and preparing an appropriate action.

Functional versus effector-specific organization of the human posterior parietal cortex: revisited

In the present study, we show that regions in posterior parietal regions process information independent of the currently used effector (hand, foot, or eye) during goal-directed actions. Functional MRI repetition suppression analysis suggests that generality across effectors holds also on the neuronal level and not just at the level of entire regions. More anterior parietal regions process information only for a specific effector or a subset of effectors.

It has been proposed that the posterior parietal cortex (PPC) is characterized by an effector-specific organization. However, strikingly similar functional MRI (fMRI) activation patterns have been found in the PPC for hand and foot movements. Because the fMRI signal is related to average neuronal activity, similar activation levels may result either from effector-unspecific neurons or from intermingled subsets of effector-specific neurons within a voxel. We distinguished between these possibilities using fMRI repetition suppression (RS). Participants made delayed, goal-directed eye, hand, and foot movements to visual targets. In each trial, the instructed effector was identical or different to that of the previous trial. RS effects indicated an attenuation of the fMRI signal in repeat trials. The caudal PPC was active during the delay but did not show RS, suggesting that its planning activity was effector independent. Hand and foot-specific RS effects were evident in the anterior superior parietal lobule (SPL), extending to the premotor cortex, with limb overlap in the anterior SPL. Connectivity analysis suggested information flow between the caudal PPC to limb-specific anterior SPL regions and between the limb-unspecific anterior SPL toward limb-specific motor regions. These results underline that both function and effector specificity should be integrated into a concept of PPC action representation not only on a regional but also on a fine-grained, subvoxel level.

Organization and evolution of parieto-frontal processing streams in macaque monkeys and humans

The functional organization of the parieto-frontal system is crucial for understanding cognitive-motor behavior and provides the basis for interpreting the consequences of parietal lesions in humans from a neurobiological perspective. The parieto-frontal connectivity defines some main information streams that, rather than being devoted to restricted functions, underlie a rich behavioral repertoire. Surprisingly, from macaque to humans, evolution has added only a few, new functional streams, increasing however their complexity and encoding power. In fact, the characterization of the conduction times of parietal and frontal areas to different target structures has recently opened a new window on cortical dynamics, suggesting that evolution has amplified the probability of dynamic interactions between the nodes of the network, thanks to communication patterns based on temporally-dispersed conduction delays. This might allow the representation of sensory-motor signals within multiple neural assemblies and reference frames, as to optimize sensory-motor remapping within an action space characterized by different and more complex demands across evolution.

Human posterior parietal cortex mediates hand-specific planning

The processes underlying action planning are fundamental to adaptive behavior and can be influenced by recent motor experience. Here, we used a novel fMRI Repetition Suppression (RS) design to test the hypotheses that action planning unfolds more efficiently for successive actions made with the same hand. More efficient processing was predicted to correspond with both faster response times (RTs) to initiate actions and reduced fMRI activity levels - RS. Consistent with these predictions, we detected faster RTs for actions made with the same hand and accompanying fMRI-RS within bilateral posterior parietal cortex and right-lateralized parietal operculum. Within posterior parietal cortex, these RS effects were localized to intraparietal and superior parietal cortices. These same areas were more strongly activated for actions involving the contralateral hand. The findings provide compelling new evidence for the specification of action plans in hand-specific terms, and indicate that these processes are sensitive to recent motor history. Consistent with computational efficiency accounts of motor history effects, the findings are interpreted as evidence for comparatively more efficient processing underlying action planning when successive actions involve the same versus opposite hand.

Action-Based Learning of Multistate Objects in the Medial Temporal Lobe

Actions constrain perception by changing the appearance of objects in the environment. As such, they provide an interactive basis for learning the structure of visual input. If an action systematically transforms one stimulus into another, then these stimuli are more likely to reflect different states of the same persisting object over time. Here we show that such multistate objects are represented in the human medial temporal lobe--the result of a mechanism in which actions influence associative learning of how objects transition between states. We further demonstrate that greater recruitment of these action-based representations during object perception is accompanied by attenuated activity in stimulus-selective visual cortex. In this way, our interactions with the environment help build visual knowledge that predictively facilitates perceptual processing.

The neural correlates of planning and executing actual tool use

Human tool use is complex, and underlying neural mechanisms seem to be widely distributed across several brain systems; however, neuroimaging studies of actual tool use are rare because of experimental challenges hindering detailed analysis within one acting subject. We developed a "Tool-Carousel" that enabled us to test actual manipulation of different objects during fMRI and investigate the planning and execution of goal-directed actions. Particularly, we focused on the effects of three factors on object manipulations: the type of object manipulated, the type of manipulation, and the hand to be used. The main focus lay on the question of how complex object use compared with unspecific actions are processed and especially how such representations interact with the knowledge about the object in the action-related dorsal stream. We found that object manipulations with both right and left hand recruit a common network strongly lateralized to the left hemisphere especially during planning but also action execution. Specifically, while activity in the ventral stream was involved in processing semantic information and object properties, a dorso-dorsal pathway (i.e., superior occipital gyrus, superior parietal lobule, and dorsal premotor area) was relevant for monitoring the online control of objects and also a ventro-dorsal pathway (i.e., middle occipital gyrus, inferior parietal lobule, and ventral premotor area) was specifically involved in processing known object manipulations, such as tool use. Data further indicate an interaction of ventral stream areas, such as middle temporal gyrus and lateral occipital complex, with both dorsal pathways. These results provide evidence for left-lateralized occipito-temporo-parieto-frontal network of everyday tool use, which may help to characterize specific deficits in patients suffering from apraxia.

Done that: short-term repetition related modulations of motor cortex activity as a stable signature for overnight motor memory consolidation

An almost universally accepted tacit expectation is that learning and memory consolidation processes must be reflected in the average brain activity in brain areas relevant to task performance. Motor cortex (M1) plasticity has been implicated in motor skill acquisition and its consolidation. Nevertheless, no consistent pattern of changes in the average signal, related to motor learning or motor memory consolidation following a single session of training, has emerged from imaging studies. Here we show that the pattern and magnitude of short-term brain activity modulations in response to task repetition, in M1, may provide a robust signature for effective motor memory consolidation processes. We studied participants during the paced performance of a finger-to-thumb opposition sequence (FOS), intensively trained a day earlier, and a similarly constructed untrained FOS. In addition to within-session "on-line" gains, most participants expressed delayed, consolidation-phase gains in the performance of the trained FOS. The execution of the trained FOS induced repetition enhancements in the contralateral M1 and bilaterally in the medial-temporal lobes, offsetting novelty-related repetition suppression effects. Moreover, the M1 modulations were positively correlated with the magnitude of each participant's overnight delayed gains but not with absolute performance levels. Our results suggest that short-term enhancements of brain signals upon task repetition reflect the effectiveness of overnight motor memory consolidation. We propose that procedural memory consolidation processes may affect the excitation-inhibition balance within cortical representations of the trained movements; this new balance is better reflected in repetition effects than in the average level of evoked neural activity.