From pantomime to actual use: How affordances can facilitate actual tool-use

Trainability of affordance judgments in right and left hemisphere stroke patients

Whenever we are confronted with action opportunities in everyday life, e.g., when passing an opening, we rely on our ability to precisely estimate our own bodily capabilities in relation to the environmental conditions. So-called affordance judgments can be affected after brain damage. Previous studies with healthy adults showed that such judgments appeared to be trainable within one session. In the current study, we examined whether stroke patients with either right brain damage (n = 30) or left brain damage (n = 30) may similarly profit from training in an aperture task. Further, the role of neuropsychological deficits in trainability was investigated. In the administered task, stroke patients decided whether their hand would fit into a presented opening with varying horizontal width (Aperture Task). During one training session, patients were asked to try to fit their hand into the opening and received feedback on their decisions. We analyzed accuracy and the detection theory parameters perceptual sensitivity and judgment tendency. Both patients with right brain damage and patients with left brain damage showed improved performance during training as well as post training. High variability with differential profiles of trainability was revealed in these patients. Patients with impaired performance in a visuo-spatial or motor-cognitive task appeared to profit considerably from the target-driven action phase with feedback, but the performance increase in judgments did not last when the action was withdrawn. Future studies applying lesion analysis with a larger sample may shed further light on the dissociation in the trainability of affordance judgments observed in patients with versus without visuo-spatial or motor-cognitive deficits.

Human Neuroimaging Reveals Differences in Activation and Connectivity between Real and Pantomimed Tool Use

Because the sophistication of tool use is vastly enhanced in humans compared with other species, a rich understanding of its neural substrates requires neuroscientific experiments in humans. Although functional magnetic resonance imaging (fMRI) has enabled many studies of tool-related neural processing, surprisingly few studies have examined real tool use. Rather, because of the many constraints of fMRI, past research has typically used proxies such as pantomiming despite neuropsychological dissociations between pantomimed and real tool use. We compared univariate activation levels, multivariate activation patterns, and functional connectivity when participants used real tools (a plastic knife or fork) to act on a target object (scoring or poking a piece of putty) or pantomimed the same actions with similar movements and timing. During the Execute phase, we found higher activation for real versus pantomimed tool use in sensorimotor regions and the anterior supramarginal gyrus, and higher activation for pantomimed than real tool use in classic tool-selective areas. Although no regions showed significant differences in activation magnitude during the Plan phase, activation patterns differed between real versus pantomimed tool use and motor cortex showed differential functional connectivity. These results reflect important differences between real tool use, a closed-loop process constrained by real consequences, and pantomimed tool use, a symbolic gesture that requires conceptual knowledge of tools but with limited consequences. These results highlight the feasibility and added value of employing natural tool use tasks in functional imaging, inform neuropsychological dissociations, and advance our theoretical understanding of the neural substrates of natural tool use.SIGNIFICANCE STATEMENT The study of tool use offers unique insights into how the human brain synthesizes perceptual, cognitive, and sensorimotor functions to accomplish a goal. We suggest that the reliance on proxies, such as pantomiming, for real tool use has (1) overestimated the contribution of cognitive networks, because of the indirect, symbolic nature of pantomiming; and (2) underestimated the contribution of sensorimotor networks necessary for predicting and monitoring the consequences of real interactions between hand, tool, and the target object. These results enhance our theoretical understanding of the full range of human tool functions and inform our understanding of neuropsychological dissociations between real and pantomimed tool use.

The low dimensionality of post-stroke cognitive deficits: it's the lesion anatomy!

For years, dissociation studies on neurological single-case patients with brain lesions were the dominant method to infer fundamental cognitive functions in neuropsychology. In contrast, the association between deficits was considered to be of less epistemological value. Still, associational computational methods for dimensionality reduction-such as principal component analysis or factor analysis-became popular for the identification of fundamental cognitive functions and to understand human cognitive brain architecture from post-stroke neuropsychological profiles. In the present in silico study with lesion imaging of 300 stroke patients, we investigated the dimensionality of artificial simulated neuropsychological profiles that exclusively contained independent fundamental cognitive functions without any underlying low-dimensional cognitive architecture. Still, the anatomy of stroke lesions alone was sufficient to create a dependence between variables that allowed a low-dimensional description of the data with principal component analysis. All criteria that we used to estimate the dimensionality of data, including the Kaiser criterion, were strongly affected by lesion anatomy, while the Joliffe criterion provided the least affected estimates. The dimensionality of profiles was reduced by 62-70% for the Kaiser criterion, up to the degree that is commonly found in neuropsychological studies on actual cognitive measures. The interpretability of such low-dimensional factors as deficits of fundamental cognitive functions and their provided insights into human cognitive architecture thus seem to be severely limited, and the heavy focus of current cognitive neuroscience on group studies and associations calls for improvements. We suggest that qualitative criteria and dissociation patterns could be used to refine estimates for the dimensionality of the cognitive architecture behind post-stroke deficits. Further, given the strong impact of lesion anatomy on the associational structure of data, we see the need for further optimization of interpretation strategies of computational factors in post-stroke lesion studies of cognitive deficits.

The social cognitive dimension of pantomime

Pantomime production is commonly interpreted as reflecting tool-use-related cognitive processes. Yet, in everyday life, pantomime deserves a communication function and the exaggeration of amplitude found during pantomime compared to real tool use may reflect the individual's attempt to communicate the intended gesture. Therefore, the question arises about whether pantomime is a communicative behavior that is nevertheless supported only by non-social cognitive processes. We contribute to answering this question by using kinematic analyses. Participants performed the pantomime of using a saw or a hammer from visual presentation in three conditions: Free (no specific instructions), Self (focus on the real tool-use action), and Others (focus on the communicative dimension). Finally, they used the tool with the corresponding object (Actual condition). Participants' social cognition were assessed using gold standard questionnaires. Our results indicated that the manipulation of instructions had a minor effect on the exaggeration of amplitude during pantomime. We reported a link between the social cognition score and the amplitude in the Others condition for the hammer, which suggests that social cognitive processes could take part in pantomime production in some conditions. Nevertheless, this result does not alter our conclusion that social cognitive processes might be far from necessary for pantomime production.

Production of co-speech gestures in the right hemisphere: Evidence from individuals with complete or anterior callosotomy

INTRODUCTION

A right-hand preference for co-speech gestures in right-handed neurotypical individuals as well as the co-occurrence of speech and gesture has induced neuropsychological research to primarily target the left hemisphere when investigating co-speech gesture production. However, the substantial number of spontaneous left-hand gestures in right-handed individuals has, thus far, been unexplained. Recent studies in individuals with complete callosotomy and exclusive left hemisphere speech production show a reliable left-hand preference for co-speech gestures, indicating a right hemispheric generation. However, the findings raise the issue if the separate right hemisphere is able to also generate representational gestures. The present study challenges the proposition of a specific right hemispheric contribution to gesture production by differentiating gesture types including representational ones in individuals with complete callosotomy and by including individuals with anterior callosotomy in whom neural reorganization is less extensive.

METHODS

Three right-handed individuals with complete commissurotomy (A.A., N.G., G.C.) and three right-handed individuals with anterior callosotomy (C.E., S.R., L. D), all with left hemisphere language dominance, and a matched right-handed neurotypical control group (n = 10) were examined in an experimental setting, including re-narration of a nonverbal animated cartoon and responding to intelligence questions. The participants' video-taped hand movement behavior was analyzed by two independent certified raters with the NEUROGES-ELAN system for nonverbal behavior and gesture. Unimanual right-hand and left-hand gestures were classified into eight gesture types.

RESULTS

The individuals with complete and anterior callosotomy performed unimanual co-speech gestures with the left as well as the right hand, with no significant preference of one hand for gestures overall. Concerning the specific gesture types, the group with complete callosotomy showed a significant right-hand preference for pantomime gestures, which also applied to the callosotomy total group. The group with anterior callosotomy displayed a significant left-hand preference for form presentation gestures. As a trend, the callosotomy total group differed from the neurotypical group as they performed more left-hand egocentric deictic and left-hand form presentation gestures.

DISCUSSION

The present study replicates the finding of a substantial left-hand use for unimanual co-speech gestures in individuals with complete callosotomy. The proposition of a right hemispheric contribution to gesture production independent from left hemispheric language production is corroborated by the finding that individuals with anterior callosotomy show a similar pattern of hand use for gestures. Representational gestures were displayed with either hand, suggesting that in particular right hemispheric spatial cognition can be directly expressed in gesture. The significant right-hand preference for pantomime gesture was outstanding and compatible with the established left hemispheric specialization for tool use praxis. The findings shed a new light on the left-hand gestures in neurotypical individuals, suggesting that these can be generated in the right hemisphere.

Representational Processes of Actions Toward and Away from the Body

The aim of the present study was to investigate the role of mental representation processes during the planning, reaching, and use phases of actions with tools commonly used toward the body (TB, e.g., toothbrush) or away from the body (AB, e.g., pencil). In the first session, healthy participants were asked to perform TB (i.e., making circular movements with the toothbrush near the mouth) and AB (i.e., making circular movements with the pencil near the desk) actions both with (i.e., actual use) and without the tool in hand (i.e., the pantomime of tool use). In the second session, the same participants performed a series of mental rotation tasks involving body- (i.e., face and hands) and object-related (i.e., abstract lines) stimuli. The temporal and kinematic analysis of the motor actions showed that the time required to start the pantomimes (i.e., the planning phase) was shorter for the AB action than for the TB action. In contrast, the reaching phase lasted longer for the AB action than for the TB action. Furthermore, the TB pantomime was associated with the performance in the mental rotation of body-related stimuli, especially during the planning and reaching phases, whereas the AB pantomime was more related to the performance in the mental rotation of object-related stimuli, especially during the tool use phase. Thus, the results revealed that the direction of a goal-directed motor action influences the dynamics of the different phases of the motor action and can determine the type of mental images involved in the planning and execution of the action.

Pantomime of tool use: looking beyond apraxia

Pantomime has a long tradition in clinical neuropsychology of apraxia. It has been much more used by researchers and clinicians to assess tool-use disorders than real tool use. Nevertheless, it remains incompletely understood and has given rise to controversies, such as the involvement of the left inferior parietal lobe or the nature of the underlying cognitive processes. The present article offers a comprehensive framework, with the aim of specifying the neural and cognitive bases of pantomime. To do so, we conducted a series of meta-analyses of brain-lesion, neuroimaging and behavioural studies about pantomime and other related tasks (i.e. real tool use, imitation of meaningless postures and semantic knowledge). The first key finding is that the area PF (Area PF complex) within the left inferior parietal lobe is crucially involved in both pantomime and real tool use as well as in the kinematics component of pantomime. The second key finding is the absence of a well-defined neural substrate for the posture component of pantomime (both grip errors and body-part-as-tool responses). The third key finding is the role played by the intraparietal sulcus in both pantomime and imitation of meaningless postures. The fourth key finding is that the left angular gyrus seems to be critical in the production of motor actions directed towards the body. The fifth key finding is that performance on pantomime is strongly correlated with the severity of semantic deficits. Taken together, these findings invite us to offer a neurocognitive model of pantomime, which provides an integrated alternative to the two hypotheses that dominate the field: The gesture-engram hypothesis and the communicative hypothesis. More specifically, this model assumes that technical reasoning (notably the left area PF), the motor-control system (notably the intraparietal sulcus), body structural description (notably the left angular gyrus), semantic knowledge (notably the polar temporal lobes) and potentially theory of mind (notably the middle prefrontal cortex) work in concert to produce pantomime. The original features of this model open new avenues for understanding the neurocognitive bases of pantomime, emphasizing that pantomime is a communicative task that nevertheless originates in specific tool-use (not motor-related) cognitive processes.

<Please insert Graphical abstract here>

Distinct cognitive components and their neural substrates underlying praxis and language deficits following left hemisphere stroke

Apraxia is characterised by multiple deficits of higher motor functions, primarily caused by left hemisphere (LH) lesions to parietal-frontal praxis networks. While previous neuropsychological and lesion studies tried to relate the various apraxic deficits to specific lesion sites, a comprehensive analysis of the different apraxia profiles and the related (impaired) motor-cognitive processes as well as their differential neural substrates in LH stroke is lacking. To reveal the cognitive mechanisms that underlie the different patterns of praxis and (related) language deficits, we applied principal component analysis (PCA) to the scores of sub-acute LH stroke patients (n = 91) in several tests of apraxia and aphasia. Voxel-based lesion-symptom mapping (VLSM) analyses were then used to investigate the neural substrates of the identified components. The PCA yielded a first component related to language functions and three components related to praxis functions, with each component associated with specific lesion patterns. Regarding praxis functions, performance in imitating arm/hand gestures was accounted for by a second component related to the left precentral gyrus and the inferior parietal lobule. Imitating finger configurations, pantomiming the use of objects related to the face, and actually using objects loaded on component 3, related to the left anterior intraparietal sulcus and angular gyrus. The last component represented the imitation of bucco-facial gestures and was linked to the basal ganglia and LH white matter tracts. The results further revealed that pantomime of (limb-related) object use depended on both the component 2 and 3, which were shared with gesture imitation and actual object use. Data support and extend the notion that apraxia represents a multi-componential syndrome comprising different (impaired) motor-cognitive processes, which dissociate - at least partially - from language processes. The distinct components might be disturbed to a varying degree following LH stroke since they are associated with specific lesion patterns within the LH.

Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

Background: Defective pantomime of tool use is a hall mark of limb apraxia. Contextual information has been demonstrated to improve tool use performance. Further, knowledge about the potential impact of technological aids such as augmented reality for patients with limb apraxia is still scarce. Objective: Since augmented reality offers a new way to provide contextual information, we applied it to pantomime of tool use. We hypothesize that the disturbed movement execution can be mitigated by holographic stimulation. If visual stimuli facilitate the access to the appropriate motor program in patients with apraxia, their performance should improve with increased saliency, i.e., should be better when supported by dynamic and holographic cues vs. static and screen-based cues. Methods: Twenty one stroke patients and 23 healthy control subjects were randomized to mime the use of five objects, presented in two Environments (Screen vs. Head Mounted Display, HMD) and two Modes (Static vs. Dynamic) resulting in four conditions (ScreenStat, ScreenDyn, HMDStat, HMDDyn), followed by a real tool demonstration. Pantomiming was analyzed by a scoring system using video recordings. Additionally, the sense of presence was assessed using a questionnaire. Results: Healthy control participants performed close to ceiling and significantly better than patients. Patients achieved significantly higher scores with holographic or dynamic cues. Remarkably, when their performance was supported by animated holographic cues (e.g., striking hammer), it did not differ significantly from real tool demonstration. As the sense of presence increases with animated holograms, so does the pantomiming. Conclusion: Patients' performance improved with visual stimuli of increasing saliency. Future assistive technology could be implemented upon this knowledge and thus, positively impact the rehabilitation process and a patient's autonomy.

Knowing how to do it or doing it? A double dissociation between tool-gesture production and tool-gesture knowledge

Deciding how to manipulate an object to fulfill a goal requires accessing different types of object-related information. How these different types of information are integrated and represented in the brain is still an open question. Here, we focus on examining two types of object-related information-tool-gesture knowledge (i.e., how to manipulate an object), and tool-gesture production (i.e., the actual manipulation of an object). We show a double dissociation between tool-gesture knowledge and tool-gesture production: Patient FP presents problems in pantomiming tool use in the context of a spared ability to perform judgments about an object's manipulation, whereas Patient LS can pantomime tool use, but is impaired at performing manipulation judgments. Moreover, we compared the location of the lesions in FP and LS with those sustained by two classic ideomotor apraxic patients (IMA), using a cortical thickness approach. Patient FP presented lesions in common with our classic IMA that included the left inferior parietal lobule (IPL), and specifically the supramarginal gyrus, the left parietal operculum, the left premotor cortex and the left inferior frontal gyrus, whereas Patient LS and our classic IMA patients presented common lesions in regions of the superior parietal lobule (SPL), motor areas (as primary somatosensory cortex, premotor cortex and primary motor cortex), and frontal areas. Our results show that tool-gesture production and tool-gesture knowledge can be behaviorally and neurally doubly dissociated and put strong constraints on extant theories of action and object recognition and use.

Post-stroke object affordances: An EEG investigation

Rehabilitating upper limb function after stroke is a key therapeutic goal. In healthy brains, objects, especially tools, are said to cause automatic motoric 'affordances'; affecting our preparation to handle objects. For example, the N2 event-related potential has been shown to correlate with the functional properties of objects in healthy adults during passive viewing. We posited that such an affordance effect might also be observed in chronic-stage stroke survivors. With either dominant or non-dominant hand forward, we presented three kinds of stimuli in stereoscopic depth; grasp objects affording a power-grip, pinch objects affording a thumb and forefinger precision-grip and an empty desk, affording no action. EEG data from 10 stroke survivors and 15 neurologically healthy subjects were analysed for the N1 and N2 ERP components. Both components revealed differences between the two object stimuli categories and the empty desk for both groups, suggesting the presence of affordance-related motor priming from around 100 to 370 ms after stimulus onset. Hence, we speculate that stroke survivors with loss of upper limb function may benefit from object presentation regimes designed to maximise motor priming when attempting movements with manipulable objects. However, further investigation would be necessary with acute stage patients, especially those diagnosed with apraxia.

Multimodal semantic battery to monitor progressive loss of concepts in the semantic variant of primary progressive aphasia (svPPA): an innovative proposal

Semantic variant primary progressive aphasia (svPPA) is a rare neurodegenerative disease characterized by a progressive loss of semantic knowledge. Patients with svPPA show anomia, impaired word comprehension, poor object recognition, and difficulties in retrieving semantic information. svPPA is also a unique "natural" model that allows clinicians and cognitive neuroscientists to study the organization of semantic memory because only semantic knowledge is affected in the initial period of the disease, with relative sparing of other cognitive domains. In the clinical practice, semantic memory is commonly tested only with verbal tests. The aim of the present study was to preliminary test a new Multimodal Semantic Battery developed in our laboratory, which comprised 11 subtests designed to assess the semantic knowledge of multiple items via all input modalities. The battery was administered twice, over four years, to a patient diagnosed with svPPA. We found that when extensively tested with multiple tests, in some cases, he was still able to recall semantic features of the items that otherwise would not have emerged with standard semantic tests. These results are discussed for the clinical practice: monitoring semantic memory through all modalities in a practical and reliable way could be useful for both clinicians and experimental researchers to better investigate the breakdown of semantic knowledge.

Looking to recognise: the pre-eminence of semantic over sensorimotor processing in human tool use

Alongside language and bipedal locomotion, tool use is a characterizing activity of human beings. Current theories in the field embrace two contrasting approaches: "manipulation-based" theories, which are anchored in the embodied-cognition view, explain tool use as deriving from past sensorimotor experiences, whereas "reasoning-based" theories suggest that people reason about object properties to solve everyday-life problems. Here, we present results from two eye-tracking experiments in which we manipulated the visuo-perceptual context (thematically consistent vs. inconsistent object-tool pairs) and the goal of the task (free observation or looking to recognise). We found that participants exhibited reversed tools' visual-exploration patterns, focusing on the tool's manipulation area under thematically consistent conditions and on its functional area under thematically inconsistent conditions. Crucially, looking at the tools with the aim of recognising them produced longer fixations on the tools' functional areas irrespective of thematic consistency. In addition, tools (but not objects) were recognised faster in the thematically consistent conditions. These results strongly support reasoning-based theories of tool use, as they indicate that people primarily process semantic rather than sensorimotor information to interact with the environment in an agent's consistent-with-goal way. Such a pre-eminence of semantic processing challenges the mainstream embodied-cognition view of human tool use.

Effect of test instructions: The example of the pantomime production task

The production of pantomime is a sensible task to detect praxis deficits. It is usually assessed by presenting objects visually or by verbal command. Verbal instructions are given either by providing the name of the object (e.g., "Show me how to use a pen") or by requiring the object function (e.g., "Show me how to write"). These modes of testing are used interchangeably. The aim of this study is to investigate whether the different instructions generate different performances. Fifty-one healthy participants (17-89 years old) were assessed on three pantomime production tasks differing for the instruction given: two with verbal instructions (Pantomime by Name and Pantomime by Function) and one with the object visually presented (Pantomime by Object). Results showed that Pantomime by Function produced the poorest performance and the highest frequency of Body Parts as Tool (BPT) errors, suggesting that the way the instructions are given may determine the performance in a task. Nuances in test instructions could result in misleading outcome.

Tool and object affordances: An ecological eye-tracking study

In the present eye-tracking study, we analysed the visuo-spatial attentional patterns of participants looking at 3D images depicting single tools and object-tool pairs. The object-tool pairs could be thematically consistent, thematically inconsistent or spatially inconsistent. During the first 500 ms of visual exploration, tools were fixated longer on their functional area in all experimental conditions. However, extending the time-window of analysis to 1750 ms, the visual scene was encoded in a faster and more suited-for-action way in the thematically consistent condition (e.g., hammer-nail). Most important, the visual exploration of the thematically consistent pairs focused on the manipulation area of the tool (e.g., the handle of the hammer) more than on its functional area (e.g., the head of the hammer). Finally, when single tools were shown and the entire time-window of analysis was considered (1750 ms), fixation focused on the tool's manipulation area. These results are discussed within the reasoning-based framework of tool use. They highlight the relative role of the visuo-perceptual context in affordance perception and suggest a novel interpretation of the cognitive mechanisms underlying the processing of tools and object-tool pairs in terms of action reappraisal (i.e., a re-functionalization process when the action possibility is mined by the visuo-perceptual context).

The role of conflict, feedback, and action comprehension in monitoring of action errors: Evidence for internal and external routes

The mechanisms and brain regions underlying error monitoring in complex action are poorly understood, yet errors and impaired error correction in these tasks are hallmarks of apraxia, a common disorder associated with left hemisphere stroke. Accounts of monitoring of language posit an internal route by which production planning or competition between candidate representations provide predictive signals that monitoring is required to prevent error, and an external route in which output is monitored using the comprehension system. Abnormal reliance on the external route has been associated with damage to brain regions critical for sensory-motor transformation and a pattern of gradual error 'clean-up' called conduite d'approche (CD). Action pantomime data from 67 participants with left hemisphere stroke were consistent with versions of internal route theories positing that competition signals monitoring requirements. Support Vector Regression Lesion Symptom Mapping (SVR-LSM) showed that lesions in the inferior parietal, posterior temporal, and arcuate fasciculus/superior longitudinal fasciculus predicted action conduite d'approche, overlapping the regions previously observed in the language domain. A second experiment with 12 patients who produced substantial action CD assessed whether factors impacting the internal route (action production ability, competition) versus external route (vision of produced actions, action comprehension) influenced correction attempts. In these 'high CD' patients, vision of produced actions and integrity of gesture comprehension interacted to determine successful error correction, supporting external route theories. Viewed together, these and other data suggest that skilled actions are monitored both by an internal route in which conflict aids in detection and correction of errors during production planning, and an external route that detects mismatches between produced actions and stored knowledge of action appearance. The parallels between language and action monitoring mechanisms and neuroanatomical networks pave the way for further exploration of common and distinct processes across these domains.

The effect of aging and contextual information on manual asymmetry in tool use

Healthy aging affects manual asymmetries in simple motor tasks, such as unilateral reaching and aiming. The effects of aging on manual asymmetries in the performance of a complex, naturalistic task are unknown, but are relevant for investigating the praxis system. This study examined how aging influences manual asymmetry in different contexts in a tool manipulation task. Fifty healthy, right-hand-dominant young (N = 29; 21.41 ± 2.87 years), and elderly (N = 21; mean: 74.14 ± 6.64 years) participants performed a 'slicing' gesture in response to a verbal command in two contexts: with (tool) and without the tool (pantomime). For interjoint relationships between shoulder plane of elevation and elbow flexion, a HAND × AGE × CONTEXT interaction existed (F_1,43 = 4.746, p = 0.035). In pantomime, interjoint control deviated more in the left (non-dominant) than the right (dominant) limb in the elderly adult group (Wilcoxon, p = 0.010). No such differences existed in the young adult group (Wilcoxon, p = 0.471). Furthermore, contextual information reduced interjoint deviation in young adults when the task was performed with the right (dominant) hand (Wilcoxon, p = 0.001) and in the elderly adults when the task was performed with the left (non-dominant) hand (Wilcoxon, p = 0.012). The presence of the tool did not reduce interjoint deviation for the right hand in the elderly group (Wilcoxon, p = 0.064) or the left hand in the young group (Wilcoxon, p = 0.044). Deviation within trials (i.e., intrasubject deviation) in elbow flexion was higher in the elderly relative to the young adult group (p = 0.003). Finally, resultant peak velocities were smaller (p = 0.002) and cycle duration longer (p < 0.0001) in the elderly adult group. This study provides novel evidence that aging affects manual asymmetries and sensorimotor control in a naturalistic task and warrants that aging research considers the context in which the task is performed.

Come together: human-avatar on-line interactions boost joint-action performance in apraxic patients

Limb apraxia (LA) is a high-order motor disorder linked to left-hemisphere damage. It is characterized by defective execution of purposeful actions upon delayed imitation, or verbal command when the actions are performed in isolated, non-naturalistic, conditions. Whether interpersonal interactions provide social affordances that activate neural resources different from those requested by individual action execution, which may improve LA performance, is unknown. To fill this gap, we measured interaction performance, behavioral and kinematic indexes of left-brain damaged patients with/without LA in a social reach-to-grasp task involving two different degrees of spatio-temporal interactivity with an avatar. We found that LA patients' impairment in coordinating with the virtual partner was abolished in highly interactive conditions (where patients selected their actions on-line based on the behavior of the virtual partner) with respect to low interactive conditions (where actions were selected beforehand based on abstract instructions). Voxel-based-Lesion-Symptom-Mapping indicated that impairments in low-interactive conditions were underpinned by lesions of premotor, motor and insular areas, and of the basal ganglia. Our approach expands current understanding of the behavioral and neural correlates of interactive motor performance by highlighting the important role of social affordances, and provides novel, potentially important, views on rehabilitation of higher-order motor cognition disorders.

Evolution of Cognitive Rehabilitation After Stroke From Traditional Techniques to Smart and Personalized Home-Based Information and Communication Technology Systems: Literature Review

Background

Neurological patients after stroke usually present cognitive deficits that cause dependencies in their daily living. These deficits mainly affect the performance of some of their daily activities. For that reason, stroke patients need long-term processes for their cognitive rehabilitation. Considering that classical techniques are focused on acting as guides and are dependent on help from therapists, significant efforts are being made to improve current methodologies and to use eHealth and Web-based architectures to implement information and communication technology (ICT) systems that achieve reliable, personalized, and home-based platforms to increase efficiency and level of attractiveness for patients and carers.

Objective

The goal of this work was to provide an overview of the practices implemented for the assessment of stroke patients and cognitive rehabilitation. This study puts together traditional methods and the most recent personalized platforms based on ICT technologies and Internet of Things.

Methods

A literature review has been distributed to a multidisciplinary team of researchers from engineering, psychology, and sport science fields. The systematic review has been focused on published scientific research, other European projects, and the most current innovative large-scale initiatives in the area. A total of 3469 results were retrieved from Web of Science, 284 studies from Journal of Medical Internet Research, and 15 European research projects from Community Research and Development Information Service from the last 15 years were reviewed for classification and selection regarding their relevance.

Results

A total of 7 relevant studies on the screening of stroke patients have been presented with 6 additional methods for the analysis of kinematics and 9 studies on the execution of goal-oriented activities. Meanwhile, the classical methods to provide cognitive rehabilitation have been classified in the 5 main techniques implemented. Finally, the review has been finalized with the selection of 8 different ICT–based approaches found in scientific-technical studies, 9 European projects funded by the European Commission that offer eHealth architectures, and other large-scale activities such as smart houses and the initiative City4Age.

Conclusions

Stroke is one of the main causes that most negatively affect countries in the socioeconomic aspect. The design of new ICT-based systems should provide 4 main features for an efficient and personalized cognitive rehabilitation: support in the execution of complex daily tasks, automatic error detection, home-based performance, and accessibility. Only 33% of the European projects presented fulfilled those requirements at the same time. For this reason, current and future large-scale initiatives focused on eHealth and smart environments should try to solve this situation by providing more complete and sophisticated platforms.

Efficiency in Rule- vs. Plan-Based Movements Is Modulated by Action-Mode

The rule/plan motor cognition (RPMC) paradigm elicits visually indistinguishable motor outputs, resulting from either plan- or rule-based action-selection, using a combination of essentially interchangeable stimuli. Previous implementations of the RPMC paradigm have used pantomimed movements to compare plan- vs. rule-based action-selection. In the present work we attempt to determine the generalizability of previous RPMC findings to real object interaction by use of a grasp-to-rotate task. In the plan task, participants had to use prospective planning to achieve a comfortable post-handle rotation hand posture. The rule task used implementation intentions (if-then rules) leading to the same comfortable end-state. In Experiment A, we compare RPMC performance of 16 healthy participants in pantomime and real object conditions of the experiment, within-subjects. Higher processing efficiency of rule- vs. plan-based action-selection was supported by diffusion model analysis. Results show a significant response-time increase in the pantomime condition compared to the real object condition and a greater response-time advantage of rule-based vs. plan-based actions in the pantomime compared to the real object condition. In Experiment B, 24 healthy participants performed the real object RPMC task in a task switching vs. a blocked condition. Results indicate that plan-based action-selection leads to longer response-times and less efficient information processing than rule-based action-selection in line with previous RPMC findings derived from the pantomime action-mode. Particularly in the task switching mode, responses were faster in the rule compared to the plan task suggesting a modulating influence of cognitive load. Overall, results suggest an advantage of rule-based action-selection over plan-based action-selection; whereby differential mechanisms appear to be involved depending on the action-mode. We propose that cognitive load is a factor that modulates the advantageous effect of implementation intentions in motor cognition on different levels as illustrated by the varying speed advantages and the variation in diffusion parameters per action-mode or condition, respectively.

Limb apraxia and the left parietal lobe

Limb apraxia is a heterogeneous disorder of skilled action and tool use that has long perplexed clinicians and researchers. It occurs after damage to various loci in a densely interconnected network of regions in the left temporal, parietal, and frontal lobes. Historically, a highly classificatory approach to the study of apraxia documented numerous patterns of performance related to two major apraxia subtypes: ideational and ideomotor apraxia. More recently, there have been advances in our understanding of the functional neuroanatomy and connectivity of the left-hemisphere "tool use network," and the patterns of performance that emerge from lesions to different loci within this network. This chapter focuses on the left inferior parietal lobe, and its role in tool and body representation, action prediction, and action selection, and how these functions relate to the deficits seen in patients with apraxia subsequent to parietal lesions. Finally, suggestions are offered for several future directions that will benefit the study of apraxia, including increased attention to research on rehabilitation of this disabling disorder.

Does it fit? - Impaired affordance perception after stroke

Affordance perception comprises the evaluation of whether our given bodily capabilities and properties of the environment allow particular actions. Typical impairments after left brain damage in motor cognition as well as after right brain damage in visuo-spatial abilities may affect the evaluation of whether interactions with objects are possible. Further it is unclear whether deficient motor function is accounted for when deciding upon action opportunities. For these purposes we developed a paradigm with two tasks that differ in their type of demands on affordance perception and tested it in healthy young adults (Randerath and Frey, 2016). Here, we applied one of these two tasks in stroke patients and age matched healthy participants. A sample of 34 stroke patients with either left (LBD) or right brain damage (RBD) and 29 healthy controls made decisions about whether their hands would fit through a defined horizontal aperture presented in various sizes, while they remained still. Data was analyzed using a detection theory approach and included criterion, perceptual sensitivity and diagnostic accuracy as dependent variables. In addition we applied modern voxel based lesion analyses to explore neural correlates. Compared to controls, both patient groups demonstrated lower perceptual sensitivity. As predicted, increased motor cognitive deficiencies after left brain damage and visuo-spatial deficits after right brain damage were associated with worse performance. Preliminary lesion analyses demonstrated that next to lesions in ventro-dorsal regions, damage in the cortex-claustrum-cingulate pathway may affect perceptual sensitivity. Results were similar for left and right brain damage suggesting a bilateral network. Accordingly, we propose that perceptual sensitivity for affordance based judgments is a capability depending on motor-cognitive and visuo-spatial processing, which frequently is deficient after left or right brain damage, respectively. Further research on diagnostics and training in affordance perception after brain damage is needed.

Object-Tool-Actor Interaction: Object Information Drives Intended Action

Tool use is typically explored via actor-tool interactions. However, the target-object (that which is being acted on) may influence perceived action possibilities and thereby guide action. Three different tool-target-object pairings were tested (Experiment 1). The hammering action demonstrated the greatest sensitivity and therefore subsequently used to further investigate target-object pairings. The hammer was removed as an option and instructions were provided using pictorial (Experiment 2), written (Experiment 3), and both pictorial and written formats (Experiment 4). The designed tool is chosen when available (Experiment 1) and when removed as a choice (i.e., the hammer), participants perform the same action associated with the designed tool (i.e., hammering) regardless of instruction method (Experiments 2, 3, and 4).

Planning Functional Grasps of Simple Tools Invokes the Hand-independent Praxis Representation Network: An fMRI Study

OBJECTIVES

Neuropsychological and neuroimaging evidence indicates that tool use knowledge and abilities are represented in the praxis representation network (PRN) of the left cerebral hemisphere. We investigated whether PRN would also underlie the planning of function-appropriate grasps of tools, even though such an assumption is inconsistent with some neuropsychological evidence for independent representations of tool grasping and skilled tool use.

METHODS

Twenty right-handed participants were tested in an event-related functional magnetic resonance imaging (fMRI) study wherein they planned functionally appropriate grasps of tools versus grasps of non-tools matched for size and/or complexity, and later executed the pantomimed grasps of these objects. The dominant right, and non-dominant left hands were used in two different sessions counterbalanced across participants. The tool and non-tool stimuli were presented at three different orientations, some requiring uncomfortable hand rotations for effective grips, with the difficulty matched for both hands.

RESULTS

Planning functional grasps of tools (vs. non-tools) was associated with significant asymmetrical increases of activity in the temporo/occipital-parieto-frontal networks. The greater involvement of the left hemisphere PRN was particularly evident when hand movement kinematics (including wrist rotations) for grasping tools and non-tools were matched. The networks engaged in the task for the dominant and non-dominant hand were virtually identical. The differences in neural activity for the two object categories disappeared during grasp execution.

CONCLUSIONS

The greater hand-independent engagement of the left-hemisphere praxis representation network for planning functional grasps reveals a genuine effect of an early affordance/function-based visual processing of tools. (JINS, 2017, 23, 108-120).

Facets of Pantomime

OBJECTIVES

Exploring the nature of defective pantomime in apraxia.

METHODS

Critical review of behavioral associations and dissociations between defective pantomime, imitation of gestures, and real tool use. Analysis of congruencies between crucial lesions for pantomime, imitation, and tool use.

RESULTS

There are behavioral double dissociations between pantomime and imitation, and their cerebral substrates show very little overlap. Whereas defective pantomime is bound to temporal and inferior frontal lesions, imitation is mainly affected by parietal lesions. Pantomime usually replicates the motor actions of real use but on scrutiny there are important differences between the movements of real use and of pantomime that cast doubt on the assumption that pantomime is produced by the same motor programs as actual use. A more plausible proposal posits that pantomime is a communicative gesture that uses manual actions for conveying information about objects and their use. The manual actions are constructed by selection and combination of distinctive features of tools and actions. They frequently include replications of characteristic motor actions of real use, but the main criterion for selection and modification of features is the comprehensibility of the gestures rather than the accurate replication of the motor actions of real use.

CONCLUSIONS

Pantomime of tool use is a communicative gesture rather than a replication of the motor actions of real use. (JINS, 2017, 23, 121-127).

Specialization of the left supramarginal gyrus for hand-independent praxis representation is not related to hand dominance

Data from focal brain injury and functional neuroimaging studies implicate a distributed network of parieto-fronto-temporal areas in the human left cerebral hemisphere as playing distinct roles in the representation of meaningful actions (praxis). Because these data come primarily from right-handed individuals, the relationship between left cerebral specialization for praxis representation and hand dominance remains unclear. We used functional magnetic resonance imaging (fMRI) to evaluate the hypothesis that strongly left-handed (right hemisphere motor dominant) adults also exhibit this left cerebral specialization. Participants planned familiar actions for subsequent performance with the left or right hand in response to transitive (e.g., "pounding") or intransitive (e.g. "waving") action words. In linguistic control trials, cues denoted non-physical actions (e.g., "believing"). Action planning was associated with significant, exclusively left-lateralized and extensive increases of activity in the supramarginal gyrus (SMg), and more focal modulations in the left caudal middle temporal gyrus (cMTg). This activity was hand- and gesture-independent, i.e., unaffected by the hand involved in subsequent action performance, and the type of gesture (i.e., transitive or intransitive). Compared directly with right-handers, left-handers exhibited greater involvement of the right angular gyrus (ANg) and dorsal premotor cortex (dPMC), which is indicative of a less asymmetric functional architecture for praxis representation. We therefore conclude that the organization of mechanisms involved in planning familiar actions is influenced by one's motor dominance. However, independent of hand dominance, the left SMg and cMTg are specialized for ideomotor transformations-the integration of conceptual knowledge and motor representations into meaningful actions. These findings support the view that higher-order praxis representation and lower-level motor dominance rely on dissociable mechanisms.

A cognitive-based model of tool use in normal aging

While several cognitive domains have been widely investigated in the field of aging, the age-related effects on tool use are still an open issue and hardly any studies on tool use and aging is available. A significant body of literature has indicated that tool use skills might be supported by at least two different types of knowledge, namely, mechanical knowledge and semantic knowledge. However, neither the contribution of these kinds of knowledge to familiar tool use, nor the effects of aging on mechanical and semantic knowledge have been explored in normal aging. The aim of the present study was to fill this gap. To do so, 98 healthy elderly adults were presented with three tasks: a classical, familiar tool use task, a novel tool use task assessing mechanical knowledge, and a picture matching task assessing semantic knowledge. The results showed that aging has a negative impact on tool use tasks and on knowledge supporting tool use skills. We also found that aging did not impact mechanical and semantic knowledge in the same way, confirming the distinct nature of those forms of knowledge. Finally, our results stressed that mechanical and semantic knowledge are both involved in the ability to use familiar tools.

Gestural praxis in young adults with mild to moderate intellectual disabilities

BACKGROUND

Praxis functioning in the population with intellectual disabilities (ID) has been poorly studied. The goal of this research was to look for a starting point to study the praxic functioning in young adults with mild to moderate ID.

METHOD

Thirty young adults with ID and 30 young adults without ID, between the ages of 18 and 35 years, participated in this study. All participants completed tests that assessed gestural praxis.

RESULTS

It was possible to observe similar praxis behaviour in the group with ID in almost all domains studied, albeit showing statistical values lower than those of the group without ID.

DISCUSSIONS

Despite the high number of errors committed, the sample of participants with ID was able to reach the goal of praxic tasks performed; such errors may be associated with a deficit in the development of various brain functions and not only with praxis functioning, mainly related to a lower yield in terms of planning, monitoring and correcting intentional movement.

Impaired Communication Between the Dorsal and Ventral Stream: Indications from Apraxia

Patients with apraxia perform poorly when demonstrating how an object is used, particularly when pantomiming the action. However, these patients are able to accurately identify, and to pick up and move objects, demonstrating intact ventral and dorsal stream visuomotor processing. Appropriate object manipulation for skilled use is thought to rely on integration of known and visible object properties associated with “ventro-dorsal” stream neural processes. In apraxia, it has been suggested that stored object knowledge from the ventral stream may be less readily available to incorporate into the action plan, leading to an over-reliance on the objects’ visual affordances in object-directed motor behavior. The current study examined grasping performance in left hemisphere stroke patients with (N = 3) and without (N = 9) apraxia, and in age-matched healthy control participants (N = 14), where participants repeatedly grasped novel cylindrical objects of varying weight distribution. Across two conditions, object weight distribution was indicated by either a memory-associated cue (object color) or visual-spatial cue (visible dot over the weighted end). Participants were required to incorporate object-weight associations to effectively grasp and balance each object. Control groups appropriately adjusted their grasp according to each object’s weight distribution across each condition, whereas throughout the task two of the three apraxic patients performed poorly on both the memory-associated and visual-spatial cue conditions. A third apraxic patient seemed to compensate for these difficulties but still performed differently to control groups. Patients with apraxia performed normally on the neutral control condition when grasping the evenly weighted version. The pattern of behavior in apraxic patients suggests impaired integration of visible and known object properties attributed to the ventro-dorsal stream: in learning to grasp the weighted object accurately, apraxic patients applied neither pure knowledge-based information (the memory-associated condition) nor higher-level information given in the visual-spatial cue condition. Disruption to ventro-dorsal stream predicts that apraxic patients will have difficulty learning to manipulate new objects on the basis of information other than low-level visual cues such as shape and size.

The influence of action execution on end-state comfort and underlying movement kinematics: An examination of right and left handed participants

People typically move in an anticipatory manner, planning the intended action in advance to minimize the energy costs associated with producing the action (e.g., Rosenbaum et al., 2009). This is exemplified behaviorally in the end-state comfort effect, which is characterized by the selection of an uncomfortable initial posture to enable a comfortable posture upon completion of the movement (Rosenbaum et al., 1990). The main objective of this study was to further investigate the end-state comfort effect in left- and right-handers (N=20). More specifically, to: (a) understand the influence of mode of action execution; and (b) delineate the role of handedness. The overturned glass task (Fischman, 1997) was used as means of assessment, where participants were asked to demonstrate picking up a glass to pour water in four modes of execution: (1) pantomime without a stimulus; (2) pantomime with image of the glass as a guide; (3) pantomime with glass as a guide; and (4) grasping the glass. End-state comfort was displayed regardless of mode of execution, hand used to complete the task or handedness group. However, kinematic analysis revealed distinct differences, highlighting how movement parameters are altered as a result the mode of action execution.

Diagnostics and Training of Affordance Perception in Healthy Young Adults-Implications for Post-Stroke Neurorehabilitation

Affordance perception is critical to adaptive behavior. It comprises the ability to evaluate whether the environment and the actor's capabilities enable particular actions. It remains unclear how brain damage and its behavioral sequela impact this ability. Two affordance based judgment tasks were applied in healthy young adults that were adapted for prospective diagnostic purposes in patients. In addition to the commonly analyzed error-rate we included response times and accuracy measures based on a detection theory approach. Moreover, a manipulation was added intended to determine the effectiveness of feedback-based learning. We further applied control tasks that consider whether errors in affordance perception can be explained by errors in perception. Participants responded yes or no to decide prospectively if a given setting would afford a particular action. In study1, 27 participants judged whether their hand would fit through a given aperture (adapted from Ishak et al., 2008). In study2, 19 participants judged whether objects are reachable [adapted from Gabbard et al. (2005)]. For both studies two sessions were administered. In the first session all participants solved the judgment-task without executing the action. In the second session (feedback manipulation), half of the participants were allowed to first judge and then perform the task for each trial (reach forward and touch the object, or fitting the hand into the aperture). Judgments were slowest and errors most frequent for openings or distances close to the individual's actual physical limits. With more extreme settings accuracy increased and responses became faster. Importantly, we found an advantageous effect of feedback on performance in both tasks suggesting that affordance perception is rapidly trainable. Further, the aperture task demonstrated that feedback experienced with one hand can transfer to the other. This may have important implications for rehabilitation.

Nonverbal social communication and gesture control in schizophrenia

Schizophrenia patients are severely impaired in nonverbal communication, including social perception and gesture production. However, the impact of nonverbal social perception on gestural behavior remains unknown, as is the contribution of negative symptoms, working memory, and abnormal motor behavior. Thus, the study tested whether poor nonverbal social perception was related to impaired gesture performance, gestural knowledge, or motor abnormalities. Forty-six patients with schizophrenia (80%), schizophreniform (15%), or schizoaffective disorder (5%) and 44 healthy controls matched for age, gender, and education were included. Participants completed 4 tasks on nonverbal communication including nonverbal social perception, gesture performance, gesture recognition, and tool use. In addition, they underwent comprehensive clinical and motor assessments. Patients presented impaired nonverbal communication in all tasks compared with controls. Furthermore, in contrast to controls, performance in patients was highly correlated between tasks, not explained by supramodal cognitive deficits such as working memory. Schizophrenia patients with impaired gesture performance also demonstrated poor nonverbal social perception, gestural knowledge, and tool use. Importantly, motor/frontal abnormalities negatively mediated the strong association between nonverbal social perception and gesture performance. The factors negative symptoms and antipsychotic dosage were unrelated to the nonverbal tasks. The study confirmed a generalized nonverbal communication deficit in schizophrenia. Specifically, the findings suggested that nonverbal social perception in schizophrenia has a relevant impact on gestural impairment beyond the negative influence of motor/frontal abnormalities.

Tool use disorders after left brain damage

In this paper we review studies that investigated tool use disorders in left-brain damaged (LBD) patients over the last 30 years. Four tasks are classically used in the field of apraxia: Pantomime of tool use, single tool use, real tool use and mechanical problem solving. Our aim was to address two issues, namely, (1) the role of mechanical knowledge in real tool use and (2) the cognitive mechanisms underlying pantomime of tool use, a task widely employed by clinicians and researchers. To do so, we extracted data from 36 papers and computed the difference between healthy subjects and LBD patients. On the whole, pantomime of tool use is the most difficult task and real tool use is the easiest one. Moreover, associations seem to appear between pantomime of tool use, real tool use and mechanical problem solving. These results suggest that the loss of mechanical knowledge is critical in LBD patients, even if all of those tasks (and particularly pantomime of tool use) might put differential demands on semantic memory and working memory.

The role of action representations in thematic object relations

A number of studies have explored the role of associative/event-based (thematic) and categorical (taxonomic) relations in the organization of object representations. Recent evidence suggests that thematic information may be particularly important in determining relationships between manipulable artifacts. However, although sensorimotor information is on many accounts an important component of manipulable artifact representations, little is known about the role that action may play during the processing of semantic relationships (particularly thematic relationships) between multiple objects. In this study, we assessed healthy and left hemisphere stroke participants to explore three questions relevant to object relationship processing. First, we assessed whether participants tended to favor thematic relations including action (Th+A, e.g., wine bottle-corkscrew), thematic relationships without action (Th-A, e.g., wine bottle-cheese), or taxonomic relationships (Tax, e.g., wine bottle-water bottle) when choosing between them in an association judgment task with manipulable artifacts. Second, we assessed whether the underlying constructs of event relatedness, action relatedness, and categorical relatedness determined the choices that participants made. Third, we assessed the hypothesis that degraded action knowledge and/or damage to temporo-parietal cortex, a region of the brain associated with the representation of action knowledge, would reduce the influence of action on the choice task. Experiment 1 showed that explicit ratings of event, action, and categorical relatedness were differentially predictive of healthy participants' choices, with action relatedness determining choices between Th+A and Th-A associations above and beyond event and categorical ratings. Experiment 2 focused more specifically on these Th+A vs. Th-A choices and demonstrated that participants with left temporo-parietal lesions, a brain region known to be involved in sensorimotor processing, were less likely than controls and tended to be less likely than patients with lesions sparing that region to use action relatedness in determining their choices. These data indicate that action knowledge plays a critical role in processing of thematic relations for manipulable artifacts.