Tool and object affordances: An ecological eye-tracking study

Effect of Object on Kinesthetic Motor Imagery in Autism Spectrum Disorder: A Pilot Study Based on Eye-Tracking Methodology

Introduction

Social disturbance is a significant autism spectrum disorder (ASD) symptom. Action representation, which is a fundamental component of social interaction, can be investigated through kinesthetic motor imagery (KMI). KMI has been commonly studied with the well-developed laterality judgment paradigm, wherein participants are required to discriminate the laterality of a hand rotated by different angles along one or more axes. Here, we investigated the KMI processing in individuals with ASD by hand laterality judgment paradigm with eye-tracking methodology.

Methods

The current study included 22 participants with ASD and 22 typical developing (TD) peers matched for age, gender, and intelligence. Participants were asked to judge the laterality of hand-with-tooth brush images.

Results

Compared to the TD controls, individuals with ASD performed KMI with lower accuracy and longer response time in both correct and incorrect action conditions. The incorrect action representation had greater effect on KMI for individuals with ASD. Differences in eye-movement patterns were also observed, characterized by individuals with ASD were more focused on the object area while TD peers were more focused on the hand area.

Conclusion

Results suggest that while altered KMI performance was observed, the incorrect action representation elicited more engagement of KMI in both groups. The object-centered eye-movement pattern may contribute to the refine of motor simulation intervention for individuals with ASD.

On the functional brain networks involved in tool-related action understanding

Tool-use skills represent a significant cognitive leap in human evolution, playing a crucial role in the emergence of complex technologies. Yet, the neural mechanisms underlying such capabilities are still debated. Here we explore with fMRI the functional brain networks involved in tool-related action understanding. Participants viewed images depicting action-consistent (e.g., nail-hammer) and action-inconsistent (e.g., scarf-hammer) object-tool pairs, under three conditions: semantic (recognizing the tools previously seen in the pairs), mechanical (assessing the usability of the pairs), and control (looking at the pairs without explicit tasks). During the observation of the pairs, task-based left-brain functional connectivity differed within conditions. Compared to the control, both the semantic and mechanical conditions exhibited co-activations in dorsal (precuneus) and ventro-dorsal (inferior frontal gyrus) regions. However, the semantic condition recruited medial and posterior temporal areas, whereas the mechanical condition engaged inferior parietal and posterior temporal regions. Also, when distinguishing action-consistent from action-inconsistent pairs, an extensive frontotemporal neural circuit was activated. These findings support recent accounts that view tool-related action understanding as the combined product of semantic and mechanical knowledge. Furthermore, they emphasize how the left inferior parietal and anterior temporal lobes might be considered as hubs for the cross-modal integration of physical and conceptual knowledge, respectively.

Too much to handle? Interference from distractors with similar affordances on target selection for handled objects

The existence of handle affordances has been classically demonstrated using the Stimulus-Response Compatibility paradigm, with shorter response times when the orientation of the object handle and the response hand are compatible in comparison to incompatible. Yet the activation of handle affordances from visual objects has been investigated in very simple situations involving single stimulus and motor response. As natural perceptual scenes are usually composed of multiple objects that could activate multiple affordances, the consequence of multiple affordance activation on the perception and processing of a given object of the scene requires more investigation. The aim of this study was to determine the impact of distractor affordances on the processing of a target object in situations involving several familiar graspable objects. In two online experiments, 229 participants had to select a target object (the kitchen utensil or the tool) in a visual scene displaying a pair of objects. They performed left key presses when the target was on the left and right key presses when the target was on the right. Target handle orientation and response side could be compatible or incompatible. Critically, target and distractor objects had similar or dissimilar handle affordances, with handles oriented for left- or right-hand grasps. Results from the two experiments showed slower response times when target and distractor objects had similar handle affordances in comparison to dissimilar affordances, when participants performed right hand responses and when target orientation and response were compatible. Thus, affordance similarity between objects may interfere rather than facilitate object processing and slow down target selection. These findings are in line with models of affordance and object selection assuming automatic inhibition of distractors' affordances for appropriate object interaction.

Cultural evolution in the science of culture and cultural evolution

My critical review [1] elicited a welcome diversity of perspectives across the 12 commentaries now published [2-13]. In total 28 co-authors were inspired to contribute. In addition to engaging with the critical perspectives of my review, several of the commentaries take the debates and discussions into insightful and potentially important supplementary domains that I highlight in what follows. I have extracted a number of major themes in which I detected overlaps in the foci of different commentaries, and I use these to organise my replies. I hope that our shared efforts will constitute some degree of 'cultural evolution' in our science, as suggested in the title of this reply to commentaries.

The visual encoding of graspable unfamiliar objects

We explored by eye-tracking the visual encoding modalities of participants (N = 20) involved in a free-observation task in which three repetitions of ten unfamiliar graspable objects were administered. Then, we analysed the temporal allocation (t = 1500 ms) of visual-spatial attention to objects' manipulation (i.e., the part aimed at grasping the object) and functional (i.e., the part aimed at recognizing the function and identity of the object) areas. Within the first 750 ms, participants tended to shift their gaze on the functional areas while decreasing their attention on the manipulation areas. Then, participants reversed this trend, decreasing their visual-spatial attention to the functional areas while fixing the manipulation areas relatively more. Crucially, the global amount of visual-spatial attention for objects' functional areas significantly decreased as an effect of stimuli repetition while remaining stable for the manipulation areas, thus indicating stimulus familiarity effects. These findings support the action reappraisal theoretical approach, which considers object/tool processing as abilities emerging from semantic, technical/mechanical, and sensorimotor knowledge integration.

Bringing cumulative technological culture beyond copying versus reasoning

The dominant view of cumulative technological culture suggests that high-fidelity transmission rests upon a high-fidelity copying ability, which allows individuals to reproduce the tool-use actions performed by others without needing to understand them (i.e., without causal reasoning). The opposition between copying versus reasoning is well accepted but with little supporting evidence. In this article, we investigate this distinction by examining the cognitive science literature on tool use. Evidence indicates that the ability to reproduce others' tool-use actions requires causal understanding, which questions the copying versus reasoning distinction and the cognitive reality of the so-called copying ability. We conclude that new insights might be gained by considering causal understanding as a key driver of cumulative technological culture.

The cortical thickness of the area PF of the left inferior parietal cortex mediates technical-reasoning skills

Most recent research highlights how a specific form of causal understanding, namely technical reasoning, may support the increasing complexity of tools and techniques developed by humans over generations, i.e., the cumulative technological culture (CTC). Thus, investigating the neurocognitive foundations of technical reasoning is essential to comprehend the emergence of CTC in our lineage. Whereas functional neuroimaging evidence started to highlight the critical role of the area PF of the left inferior parietal cortex (IPC) in technical reasoning, no studies explored the links between the structural characteristics of such a brain region and technical reasoning skills. Therefore, in this study, we assessed participants’ technical-reasoning performance by using two ad-hoc psycho-technical tests; then, we extracted from participants’ 3 T T1-weighted magnetic-resonance brain images the cortical thickness (i.e., a volume-related measure which is associated with cognitive performance as reflecting the size, density, and arrangement of cells in a brain region) of all the IPC regions for both hemispheres. We found that the cortical thickness of the left area PF predicts participants’ technical-reasoning performance. Crucially, we reported no correlations between technical reasoning and the other IPC regions, possibly suggesting the specificity of the left area PF in generating technical knowledge. We discuss these findings from an evolutionary perspective, by speculating about how the evolution of parietal lobes may have supported the emergence of technical reasoning in our lineage.

Visual Attention and Cognitive Archaeology: An Eye-Tracking Study of Palaeolithic Stone Tools

The study of lithic technology can provide information on human cultural evolution. This article aims to analyse visual behaviour associated with the exploration of ancient stone artefacts and how this relates to perceptual mechanisms in humans. In Experiment 1, we used eye tracking to record patterns of eye fixations while participants viewed images of stone tools, including examples of worked pebbles and handaxes. The results showed that the focus of gaze was directed more towards the upper regions of worked pebbles and on the basal areas for handaxes. Knapped surfaces also attracted more fixation than natural cortex for both tool types. Fixation distribution was different to that predicted by models that calculate visual salience. Experiment 2 was an online study using a mouse-click attention tracking technique and included images of unworked pebbles and 'mixed' images combining the handaxe's outline with the pebble's unworked texture. The pattern of clicks corresponded to that revealed using eye tracking and there were differences between tools and other images. Overall, the findings suggest that visual exploration is directed towards functional aspects of tools. Studies of visual attention and exploration can supply useful information to inform understanding of human cognitive evolution and tool use.

Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

Background: Defective pantomime of tool use is a hall mark of limb apraxia. Contextual information has been demonstrated to improve tool use performance. Further, knowledge about the potential impact of technological aids such as augmented reality for patients with limb apraxia is still scarce. Objective: Since augmented reality offers a new way to provide contextual information, we applied it to pantomime of tool use. We hypothesize that the disturbed movement execution can be mitigated by holographic stimulation. If visual stimuli facilitate the access to the appropriate motor program in patients with apraxia, their performance should improve with increased saliency, i.e., should be better when supported by dynamic and holographic cues vs. static and screen-based cues. Methods: Twenty one stroke patients and 23 healthy control subjects were randomized to mime the use of five objects, presented in two Environments (Screen vs. Head Mounted Display, HMD) and two Modes (Static vs. Dynamic) resulting in four conditions (ScreenStat, ScreenDyn, HMDStat, HMDDyn), followed by a real tool demonstration. Pantomiming was analyzed by a scoring system using video recordings. Additionally, the sense of presence was assessed using a questionnaire. Results: Healthy control participants performed close to ceiling and significantly better than patients. Patients achieved significantly higher scores with holographic or dynamic cues. Remarkably, when their performance was supported by animated holographic cues (e.g., striking hammer), it did not differ significantly from real tool demonstration. As the sense of presence increases with animated holograms, so does the pantomiming. Conclusion: Patients' performance improved with visual stimuli of increasing saliency. Future assistive technology could be implemented upon this knowledge and thus, positively impact the rehabilitation process and a patient's autonomy.

Modulation of Working Memory and Resting-State fMRI by tDCS of the Right Frontoparietal Network

Many cognitive functions, including working memory, are processed within large-scale brain networks. We targeted the right frontoparietal network (FPN) with one session of transcranial direct current stimulation (tDCS) in an attempt to modulate the cognitive speed of a visual working memory task (WMT) in 27 young healthy subjects using a double-blind crossover design. We further explored the neural underpinnings of induced changes by performing resting-state fMRI prior to and immediately after each stimulation session with the main focus on the interaction between a task-positive FPN and a task-negative default mode network (DMN). Twenty minutes of 2 mA anodal tDCS was superior to sham stimulation in terms of cognitive speed manipulation of a subtask with processing of objects and tools in unconventional views (i.e., the higher cognitive load subtask of the offline WMT). This result was linked to the magnitude of resting-state functional connectivity decreases between the stimulated FPN seed and DMN seeds. We provide the first evidence for the action reappraisal mechanism of object and tool processing. Modulation of cognitive speed of the task by tDCS was reflected by FPN-DMN cross-talk changes.

When the Action to Be Performed at the Stage of Retrieval Enacts Memory of Action Verbs

Abstract. According to the embodied approach of language, concepts are grounded in sensorimotor mental states, and when we process language, the brain simulates some of the perceptions and actions that are involved when interacting with real objects. Moreover, several studies have highlighted that cognitive performances are dependent on the overlap between the motor action simulated and the motor action required by the task. On the other hand, in the field of memory, the role of action is under debate. The aim of this work was to show that performing an action at the stage of retrieval influences memory performance in a recognition task (experiment 1) and a cued recall task (experiment 2), even if the participants were never instructed to consider the implied action. The results highlighted an action-based memory effect at the retrieval stage. These findings contribute to the debate about the implication of motor system in action verb processing and its role for memory.

Tool heads prime saccades

Tools are wielded by their handles, but a lot of information about their function comes from their heads (the action-ends). Here we investigated whether eye saccadic movements are primed by tool handles, or whether they are primed by tool heads. We measured human saccadic reaction times while subjects were performing an attentional task. We found that saccades were executed quicker when performed to the side congruent with the tool head, even though "toolness" was irrelevant for the task. Our results show that heads are automatically processed by the visual system to orient eye movements, indicating that eyes are attracted by functional parts of manipulable objects and by the characteristic information these parts convey.

Semantic congruency effects of prime words on tool visual exploration

Most recent research on human tool use highlighted how people might integrate multiple sources of information through different neurocognitive systems to exploit the environment for action. This mechanism of integration is known as "action reappraisal". In the present eye-tracking study, we further tested the action reappraisal idea by devising a word-priming paradigm to investigate how semantically congruent (e.g., "nail") vs. semantically incongruent words (e.g., "jacket") that preceded the vision of tools (e.g., a hammer) may affect participants' visual exploration of them. We found an implicit modulation of participants' temporal allocation of visuospatial attention as a function of the object-word consistency. Indeed, participants tended to increase over time their fixations on tools' manipulation areas under semantically congruent conditions. Conversely, participants tended to concentrate their visual-spatial attention on tools' functional areas when inconsistent object-word pairs were presented. These results support and extend the information-integrated perspective of the action reappraisal approach. Also, these findings provide further evidence about how higher-level semantic information may influence tools' visual exploration.

How the fear of COVID-19 changed the way we look at human faces

Do we look at persons currently or previously affected by COVID-19 the same way as we do with healthy ones? In this eye-tracking study, we investigated how participants (N = 54) looked at faces of individuals presented as "COVID-19 Free", "Sick with COVID-19", or "Recovered from COVID-19". Results showed that participants tend to look at the eyes of COVID-19-free faces longer than at those of both COVID-19-related faces. Crucially, we also found an increase of visual attention for the mouth of the COVID-19-related faces, possibly due to the threatening characterisation of such area as a transmission vehicle for SARS-CoV-2. Thus, by detailing how people dynamically changed the way of looking at faces as a function of the perceived risk of contagion, we provide the first evidence in the literature about the impact of the pandemic on the most basic level of social interaction.

Hazardous tools: the emergence of reasoning in human tool use

Humans are unique in the way they understand the causal relationships between the use of tools and achieving a goal. The idea at the core of the present research is that tool use can be considered as an instance of problem-solving situations supported by technical reasoning. In an eye-tracking study, we investigated the fixation patterns of participants (N = 32) looking at 3D images of thematically consistent (e.g., nail-steel hammer) and thematically inconsistent (e.g., scarf-steel hammer) object-tool pairs that could be either "hazardous" (accidentally electrified) or not. Results showed that under thematically consistent conditions, participants focused on the tool's manipulation area (e.g., the handle of a steel hammer). However, when electrified tools were present or when the visual scene was not action-prompting, regardless of the presence of electricity, the tools' functional/identity areas (e.g., the head of a steel hammer) were fixated longer than the tools' manipulation areas. These results support an integrated and reasoning-based approach to human tool use and document, for the first time, the crucial role of mechanical/semantic knowledge in tool visual exploration.

Effects of Tool Novelty and Action Demands on Gaze Searching During Tool Observation

Technical reasoning refers to making inferences about how to use tools. The degree of technical reasoning is indicated by the bias of the gaze (fixation) on the functional part of the tool when in use. Few studies have examined whether technical reasoning differs between familiar and unfamiliar novel tools. In addition, what effect the intention to use the tool has on technical reasoning has not been determined. This study examined gaze shifts in relation to familiar or unfamiliar tools, under three conditions (free viewing, lift, and use), among 14 healthy adults (mean age ± standard deviation, 29.4 ± 3.9 years). The cumulative fixation time on the functional part of the tool served as a quantitative indicator of the degree of technical reasoning. The two-way analysis of variance for tools (familiar and unfamiliar) and conditions (free viewing, lift, and use) revealed that the cumulative fixation time significantly increased under free viewing and use conditions, compared to lift conditions. Relative to the free viewing condition, cumulative fixation time for unfamiliar tools significantly decreased in the lift condition and significantly increased in the use condition. Importantly, the results showed that technical reasoning was performed in both the use and the free viewing conditions. However, technical reasoning in the free viewing condition was not as strong as in the use condition. The difference between technical reasoning in free viewing and use conditions may indicate the difference between automatic and intentional technical reasoning.

Looking to recognise: the pre-eminence of semantic over sensorimotor processing in human tool use

Alongside language and bipedal locomotion, tool use is a characterizing activity of human beings. Current theories in the field embrace two contrasting approaches: "manipulation-based" theories, which are anchored in the embodied-cognition view, explain tool use as deriving from past sensorimotor experiences, whereas "reasoning-based" theories suggest that people reason about object properties to solve everyday-life problems. Here, we present results from two eye-tracking experiments in which we manipulated the visuo-perceptual context (thematically consistent vs. inconsistent object-tool pairs) and the goal of the task (free observation or looking to recognise). We found that participants exhibited reversed tools' visual-exploration patterns, focusing on the tool's manipulation area under thematically consistent conditions and on its functional area under thematically inconsistent conditions. Crucially, looking at the tools with the aim of recognising them produced longer fixations on the tools' functional areas irrespective of thematic consistency. In addition, tools (but not objects) were recognised faster in the thematically consistent conditions. These results strongly support reasoning-based theories of tool use, as they indicate that people primarily process semantic rather than sensorimotor information to interact with the environment in an agent's consistent-with-goal way. Such a pre-eminence of semantic processing challenges the mainstream embodied-cognition view of human tool use.