Incidental and context-responsive activation of structure- and function-based action features during object identification

What we mean when we say semantic: Toward a multidisciplinary semantic glossary

Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).

No need to integrate action information during coarse semantic processing of man-made tools

Action representation of man-made tools consists of two subtypes: structural action representation concerning how to grasp an object, and functional action representation concerning the skilled use of an object. Compared to structural action representation, functional action representation plays the dominant role in fine-grained (i.e., basic level) object recognition. However, it remains unclear whether the two types of action representation are involved differently in the coarse semantic processing in which the object is recognized at a superordinate level (i.e., living/non-living). Here we conducted three experiments using the priming paradigm, in which video clips displaying structural and functional action hand gestures were used as prime stimuli and grayscale photos of man-made tools were used as target stimuli. Participants recognized the target objects at the basic level in Experiment 1 (i.e., naming task) and at the superordinate level in Experiments 2 and 3 (i.e., categorization task). We observed a significant priming effect for functional action prime-target pairs only in the naming task. In contrast, no priming effect was found in either the naming or the categorization task for the structural action prime-target pairs (Experiment 2), even when the categorization task was preceded by a preliminary action imitation of the prime gestures (Experiment 3). Our results suggest that only functional action information is retrieved during fine-grained object processing. In contrast, coarse semantic processing does not require the integration of either structural or functional action information.

Mechanisms and neuroanatomy of response selection in tool and non-tool action tasks: Evidence from left-hemisphere stroke

The ability to select between potential actions is central to the complex process of tool use. After left hemisphere stroke, individuals with limb apraxia make more hand action errors when gesturing the use of tools with conflicting hand actions for grasping-to-move and use (e.g., screwdriver) relative to tools that are grasped-to-move and used with the same hand action (e.g., hammer). Prior research indicates that this grasp-use interference effect is driven by abnormalities in the competitive action selection process. The goal of this project was to determine whether common mechanisms and neural substrates support the competitive selection of task-appropriate responses in both tool and non-tool domains. If so, the grasp-use interference effect in a tool use gesturing task should be correlated with response interference effects in the classic Eriksen flanker and Simon tasks, and at least partly overlapping neural regions should subserve the 3 tasks. Sixty-four left hemisphere stroke survivors (33 with apraxia) participated in the tool- and non-tool interference tasks and underwent T1 anatomical MRI. There were robust grasp-use interference effects (grasp-use conflict test) and response interference effects (Eriksen flanker and Simon tasks), but these effects were not correlated. Lesion-symptom mapping analyses showed that lesions to the left inferior parietal lobule, ventral premotor cortex, and insula were associated with grasp-use interference. Lesions to the left inferior parietal lobule, postcentral gyrus, insula, caudate, and putamen were associated with response interference in the Eriksen flanker task. Lesions to the left caudate and putamen were also associated with response interference in the Simon task. Our results suggest that the selection of hand posture for tool use is mediated by distinct cognitive mechanisms and partly distinct neuroanatomic substrates from those mapping a stimulus to an appropriate motor response in non-tool domains.

Too much to handle? Interference from distractors with similar affordances on target selection for handled objects

The existence of handle affordances has been classically demonstrated using the Stimulus-Response Compatibility paradigm, with shorter response times when the orientation of the object handle and the response hand are compatible in comparison to incompatible. Yet the activation of handle affordances from visual objects has been investigated in very simple situations involving single stimulus and motor response. As natural perceptual scenes are usually composed of multiple objects that could activate multiple affordances, the consequence of multiple affordance activation on the perception and processing of a given object of the scene requires more investigation. The aim of this study was to determine the impact of distractor affordances on the processing of a target object in situations involving several familiar graspable objects. In two online experiments, 229 participants had to select a target object (the kitchen utensil or the tool) in a visual scene displaying a pair of objects. They performed left key presses when the target was on the left and right key presses when the target was on the right. Target handle orientation and response side could be compatible or incompatible. Critically, target and distractor objects had similar or dissimilar handle affordances, with handles oriented for left- or right-hand grasps. Results from the two experiments showed slower response times when target and distractor objects had similar handle affordances in comparison to dissimilar affordances, when participants performed right hand responses and when target orientation and response were compatible. Thus, affordance similarity between objects may interfere rather than facilitate object processing and slow down target selection. These findings are in line with models of affordance and object selection assuming automatic inhibition of distractors' affordances for appropriate object interaction.

Nonspecific Effects of Normal Aging on Taxonomic and Thematic Semantic Processing

OBJECTIVE

This study aimed to assess the effect of normal aging on the processing of taxonomic and thematic semantic relations.

METHOD

We used the Visual-World-Paradigm coupled with eye-movement recording. We compared performance of healthy younger and older adults on a word-to-picture matching task in which participants had to identify each target among semantically related (taxonomic or thematic) and unrelated distractors.

RESULTS

Younger and older participants exhibited similar patterns of gaze fixations in the two semantic conditions. The effect of aging took the form of an overall reduction in sensitivity to semantic competitors, with no difference between the taxonomic and thematic conditions. Moreover, comparison of the proportions of fixations between the younger and older participants indicated that targets were identified equally quickly in both age groups. This was not the case when mouse-click reaction times were analyzed.

CONCLUSIONS

Findings argue in favor of nonspecific effects of normal aging on semantic processing that similarly affect taxonomic and thematic processing. There are important clinical implications, as pathological aging has been repeatedly shown to selectively affect either taxonomic or thematic relations. Measuring eye-movements in a semantic task is also an interesting approach in the elderly, as these seem to be less impacted by aging than other motor responses.

What first drives visual attention during the recognition of object-directed actions? The role of kinematics and goal information

The recognition of others' object-directed actions is known to involve the decoding of both the visual kinematics of the action and the action goal. Yet whether action recognition is first guided by the processing of visual kinematics or by a prediction about the goal of the actor remains debated. In order to provide experimental evidence to this issue, the present study aimed at investigating whether visual attention would be preferentially captured by visual kinematics or by action goal information when processing others' actions. In a visual search task, participants were asked to find correct actions (e.g., drinking from glass) among distractor actions. Distractors actions contained grip and/or goal violations and could therefore share the correct goal and/or the correct grip with the target. The time course of fixation proportion on each distractor action has been taken as an indicator of visual attention allocation. Results show that visual attention is first captured by the distractor action with similar goal. Then the withdrawal of visual attention from the action distractor with similar goal suggests a later attentional capture by the action distractor with similar grip. Overall, results are in line with predictive approaches of action understanding, which assume that observers first make a prediction about the actor's goal before verifying this prediction using the visual kinematics of the action.

Reduced competition between tool action neighbors in left hemisphere stroke

When pantomiming the use of tools, patients with limb apraxia after left hemisphere stroke (LCVA) produce more spatiotemporal hand action errors with tools associated with conflicting actions for use versus grasp-to-pick-up (e.g., corkscrew) than tools having a single action for both use and grasp (e.g., hammer). There are two possible accounts for this pattern of results. Reduced performance with 'conflict' tools may simply reflect weakened automaticity of use action activation, which is evident only when the use and grasp actions are not redundant. Alternatively, poor use performance may reflect a reduced ability of appropriate tool use actions to compete with task-inappropriate action representations. To address this issue, we developed a Stroop-like experiment in which 21 LCVA and 8 neurotypical participants performed pantomime actions in blocks containing two tools that were similar ("neighbors") in terms of hand action or function, or were unrelated on either dimension. In a congruent condition, they pantomimed the use action associated with the visually presented tool, whereas in an incongruent condition, they pantomimed the use action for the other tool in the block. Relative to controls and other task conditions, LCVA participants showed reductions in hand action errors in incongruent relative to congruent action trials; furthermore, the degree of reduction in this incongruence effect was related to the participants' susceptibility to grasp-on-use conflict in a separate test of pantomime to the sight of tools. Support vector regression lesion-symptom mapping analyses identified the left inferior frontal gyrus, supramarginal gyrus, and superior longitudinal fasciculus as core neuroanatomical sites associated with abnormal performance. Collectively, the results indicate that weakened activation of tool use actions in limb apraxia gives rise to reduced ability of these actions to compete for task-appropriate selection when competition arises within single tools (grasp-on-use conflict) as well as between two tools (reduced neighborhood effects).

Do manufactured and natural objects evoke similar motor information? The case of action priming

There is considerable evidence that visually presented manipulable objects evoke motor information, supporting the existence of affordance effects during object perception. However, most arguments come from stimulus-response compatibility paradigms, raising the issue of the automaticity of affordance effects. Action priming paradigms overcome this issue but show less reliable results, possibly because affordance effects are moderated by additional factors. The present study aimed to assess whether affordance effects highlighted in action priming paradigms could be affected by object category (manufactured or natural). A total of 24 young adults performed a semantic categorisation task on natural and manufactured target objects presented after neutral (non-grasping hand postures) or action (congruent power or precision grips) primes. Results revealed a modulation of action priming effects as a function of object category. Object semantic categorisation was faster after action than neutral primes, but only for manufactured objects. Results suggest that natural and manufactured objects evoke distinct types of affordances and that action priming paradigms favour the evocation of functional affordances during object semantic categorisation. This finding fuels the debate on the nature of the motor information evoked by visual objects.

The role of conflict, feedback, and action comprehension in monitoring of action errors: Evidence for internal and external routes

The mechanisms and brain regions underlying error monitoring in complex action are poorly understood, yet errors and impaired error correction in these tasks are hallmarks of apraxia, a common disorder associated with left hemisphere stroke. Accounts of monitoring of language posit an internal route by which production planning or competition between candidate representations provide predictive signals that monitoring is required to prevent error, and an external route in which output is monitored using the comprehension system. Abnormal reliance on the external route has been associated with damage to brain regions critical for sensory-motor transformation and a pattern of gradual error 'clean-up' called conduite d'approche (CD). Action pantomime data from 67 participants with left hemisphere stroke were consistent with versions of internal route theories positing that competition signals monitoring requirements. Support Vector Regression Lesion Symptom Mapping (SVR-LSM) showed that lesions in the inferior parietal, posterior temporal, and arcuate fasciculus/superior longitudinal fasciculus predicted action conduite d'approche, overlapping the regions previously observed in the language domain. A second experiment with 12 patients who produced substantial action CD assessed whether factors impacting the internal route (action production ability, competition) versus external route (vision of produced actions, action comprehension) influenced correction attempts. In these 'high CD' patients, vision of produced actions and integrity of gesture comprehension interacted to determine successful error correction, supporting external route theories. Viewed together, these and other data suggest that skilled actions are monitored both by an internal route in which conflict aids in detection and correction of errors during production planning, and an external route that detects mismatches between produced actions and stored knowledge of action appearance. The parallels between language and action monitoring mechanisms and neuroanatomical networks pave the way for further exploration of common and distinct processes across these domains.

The cost of switching between taxonomic and thematic semantics

Current models and theories of semantic knowledge primarily capture taxonomic relationships (DOG and WOLF) and largely do not address the role of thematic relationships in semantic knowledge (DOG and LEASH). Recent evidence suggests that processing or representation of thematic relationships may be distinct from taxonomic relationships. If taxonomic and thematic relations are distinct, then there should be a cost associated with switching between them even when the task remains constant. This hypothesis was tested using two different semantic-relatedness judgment tasks: Experiment 1 used a triads task and Experiment 2 used an oddball task. In both experiments, participants were faster to respond when the same relationship appeared on consecutive trials than when the relationship types were different, even though the task remained the same and the specific relations were different on each trial. These results are consistent with the theory that taxonomic and thematic relations rely on distinct processes or representations.

The dominant role of functional action representation in object recognition

Action representation of manipulable objects has been found to be involved in object recognition. Recently, studies have indicated the existence of two distinct action systems: functional action specifying how to use an object and structural action concerning how to grasp an object. Despite evidence revealing the systems' anatomical and functional differences, few preceding studies have dissociated their respective roles in object recognition. The present study aimed to tease apart their roles in the recognition of manipulable objects with a priming paradigm. Specifically, we used static stimuli (photos, Experiments 1 and 2) and dynamic stimuli (video clips, Experiments 3 and 4) depicting functional and structural action hand gestures as primes and measured the magnitude of functional and structural action priming effect in object recognition. We found that static and dynamic priming stimuli induced a robust action priming effect only for functional action prime-target pairs. Naming latencies of the target objects were shorter when functional action representations of the prime and target were congruent than when they were incongruent. Moreover, as compared to static priming photos, dynamic priming stimuli induced a larger functional action priming effect. By contrast, neither static nor dynamic priming stimuli elicited a structural action priming effect. Behavioral data from our four experiments provide consistent evidence of the dominant role of functional action representation in the recognition of manipulable objects, suggesting that action knowledge regarding how to use rather than grasp an object is more likely an intrinsic component of objects' conceptual representation.

Sensory and semantic activations evoked by action attributes of manipulable objects: Evidence from ERPs

"Two route" theories of object-related action processing posit different temporal activation profiles of grasp-to-move actions (rapidly evoked based on object structure) versus skilled use actions (more slowly activated based on semantic knowledge). We capitalized on the exquisite temporal resolution and multidimensionality of Event-Related Potentials (ERPs) to directly test this hypothesis. Participants viewed manipulable objects (e.g., calculator) preceded by objects sharing either "grasp", "use", or no action attributes (e.g., bar of soap, keyboard, earring, respectively), as well as by action-unrelated but taxonomically-related objects (e.g., abacus); participants judged whether the two objects were related. The results showed more positive responses to "grasp-to-move" primed objects than "skilled use" primed objects or unprimed objects starting in the P1 (0-150 ms) time window and continuing onto the subsequent N1 and P2 components (150-300 ms), suggesting that only "grasp-to-move", but not "skilled use", actions may facilitate visual attention to object attributes. Furthermore, reliably reduced N400s (300-500 ms), an index of semantic processing, were observed to taxonomically primed and "skilled use" primed objects relative to unprimed objects, suggesting that "skilled use" action attributes are a component of distributed, multimodal semantic representations of objects. Together, our findings provide evidence supporting two-route theories by demonstrating that "grasp-to-move" and "skilled use" actions impact different aspects of object processing and highlight the relationship of "skilled use" information to other aspects of semantic memory.

Differential Tuning of Ventral and Dorsal Streams during the Generation of Common and Uncommon Tool Uses

Our use of tools is situated in different contexts. Prior evidence suggests that diverse regions within the ventral and dorsal streams represent information supporting common tool use. However, given the flexibility of object concepts, these regions may be tuned to different types of information when generating novel or uncommon uses of tools. To investigate this, we collected fMRI data from participants who reported common or uncommon tool uses in response to visually presented familiar objects. We performed a pattern dissimilarity analysis in which we correlated cortical patterns with behavioral measures of visual, action, and category information. The results showed that evoked cortical patterns within the dorsal tool use network reflected action and visual information to a greater extent in the uncommon use group, whereas evoked neural patterns within the ventral tool use network reflected categorical information more strongly in the common use group. These results reveal the flexibility of cortical representations of tool use and the situated nature of cortical representations more generally.

Learning, remembering, and predicting how to use tools: Distributed neurocognitive mechanisms: Comment on Osiurak and Badets (2016)

The reasoning-based approach championed by Francois Osiurak and Arnaud Badets (Osiurak & Badets, 2016) denies the existence of sensory-motor memories of tool use except in limited circumstances, and suggests instead that most tool use is subserved solely by online technical reasoning about tool properties. In this commentary, I highlight the strengths and limitations of the reasoning-based approach and review a number of lines of evidence that manipulation knowledge is in fact used in tool action tasks. In addition, I present a "two route" neurocognitive model of tool use called the "Two Action Systems Plus (2AS+)" framework that posits a complementary role for online and stored information and specifies the neurocognitive substrates of task-relevant action selection. This framework, unlike the reasoning based approach, has the potential to integrate the existing psychological and functional neuroanatomic data in the tool use domain. (PsycINFO Database Record

Is there a Competition between Functional and Situational Affordances during Action Initiation with Everyday Tools?

Most studies of human-tool interactions focus on the typical use of a tool (e.g., cutting in the case of a knife). However, little is known about situations requiring atypical tool use (e.g., using a knife to tighten a screw). The present study focused on a selection of atypical uses of everyday tools which might be in conflict with their typical use. Our objective was to study how tool function influences the selection of the relevant action. In Experiment 1, which involved visuomotor priming, two everyday tools (a knife and a screwdriver) and two neutral tools (two bars, with no strong functional affordance) were used as primes and targets. Participants had to use the target with the appropriate box (indicated by the color) that allowed to make an action. Longer initiation times were observed when the prime was an everyday tool, irrespective of the nature of the target. We therefore observed a conflict between functional and situational affordances. To investigate whether the priming effect is caused by the task-irrelevance of the prime, we asked the participants in Experiment 2 to perform an action associated with the prime. The results showed longer initiation times only when the prime and target were everyday tools, irrespective of their precise nature. This suggests that activation of the typical use of a tool might not be fully automatic but flexible depending on the situation.

The modulation of the motor resonance triggered by reach-to-grasp movements: No role of human physical similarity as conveyed by age

The activation of the mirror-neuron circuit during the observation of motor acts is thought to be the basis of human capacity to read the intentions behind the behavior of others. Growing empirical evidence shows a different activation of the mirror-neuron resonance mechanism depending on how much the observer and the observed agent share their motor repertoires. Here, the possible modulatory effect of physical similarity between the observer and the agent was investigated in three studies. We used a visuo-motor priming task in which participants were asked to categorize manipulable and non-manipulable objects into natural or man-made kinds after having watched precision and power reach-to-grasp movements. Physical similarity was manipulated by presenting reach-to-grasp movements performed by the hands of actors of three different age ranges that are adults of the same age as the participants, children, and elderly. Faster responses were observed in trials where power grip movements were performed by the adults and precision grip movements were performed by the elderly (Main Study). This finding is not in keeping with the idea that physical similarity shapes the mirror-neuron resonance. Instead, it suggests an effect of the kinematic organization of the reach-to-grasp movements, which systematically changed with the actor age as revealed by a kinematic analysis. The differential effect played by adult and elderly actor primes was lost when static grasping hands (Control Study 1) and reach-to-grasp movements with uniform kinematic profiles (Control Study 2) were used. Therefore, we found preliminary evidence that mirror-neuron resonance is not shaped by physical similarity but by the kinematics of the observed action. This finding is novel as it suggests that human ability to read the intentions behind the behavior of others may benefit from a mere visual processing of spatiotemporal patterns.

Effects of motion speed in action representations

Grounded cognition accounts of semantic representation posit that brain regions traditionally linked to perception and action play a role in grounding the semantic content of words and sentences. Sensory-motor systems are thought to support partially abstract simulations through which conceptual content is grounded. However, which details of sensory-motor experience are included in, or excluded from these simulations, is not well understood. We investigated whether sensory-motor brain regions are differentially involved depending on the speed of actions described in a sentence. We addressed this issue by examining the neural signature of relatively fast (The old lady scurried across the road) and slow (The old lady strolled across the road) action sentences. The results showed that sentences that implied fast motion modulated activity within the right posterior superior temporal sulcus and the angular and middle occipital gyri, areas associated with biological motion and action perception. Sentences that implied slow motion resulted in greater signal within the right primary motor cortex and anterior inferior parietal lobule, areas associated with action execution and planning. These results suggest that the speed of described motion influences representational content and modulates the nature of conceptual grounding. Fast motion events are represented more visually whereas motor regions play a greater role in representing conceptual content associated with slow motion.

Motor experience influences object knowledge

An object's perceived readiness-for-action (e.g., its size, the degree of rotation from its canonical position, the user's viewpoint) can influence semantic knowledge retrieval. Yet, the organization of object knowledge may also be affected by body-specific sensorimotor experiences. Here, we investigated whether people's history of performing motor actions with their hands influences the knowledge they store and retrieve about graspable objects. We compared object representations between healthy right- and left-handers (Experiment 1), and between unilateral stroke patients, whose motor experience was changed by impairment of either their right or left hand (Experiment 2). Participants saw pictures of graspable everyday items with the handles oriented toward either the left or right hand, and they generated the type of grasp they would employ (i.e., clench or pinch) when using each object, responding orally. In both experiments, hand dominance and object orientation interacted to predict response times. In Experiment 1, judgments were fastest when objects were oriented toward the right hand in right-handers, but not in left-handers. In Experiment 2, judgments were fastest when objects were oriented toward the left hand in patients who had lost the use of their right hand, even though these patients were right-handed prior to brain injury. Results suggest that at least some aspects of object knowledge are determined by motor experience, and can be changed by new patterns of motor experience. People with different bodily characteristics, who interact with objects in systematically different ways, form correspondingly different neurocognitive representations of the same common objects. (PsycINFO Database Record

Chained Activation of the Motor System during Language Understanding

Two experiments were carried out to investigate whether and how one important characteristic of the motor system, that is its goal-directed organization in motor chains, is reflected in language processing. This possibility stems from the embodied theory of language, according to which the linguistic system re-uses the structures of the motor system. The participants were presented with nouns of common tools preceded by a pair of verbs expressing grasping or observational motor chains (i.e., grasp-to-move, grasp-to-use, look-at-to-grasp, and look-at-to-stare). They decided whether the tool mentioned in the sentence was the same as that displayed in a picture presented shortly after. A primacy of the grasp-to-use motor chain over the other motor chains in priming the participants' performance was observed in both the experiments. More interestingly, we found that the motor information evoked by the noun was modulated by the specific motor-chain expressed by the preceding verbs. Specifically, with the grasping chain aimed at using the tool, the functional motor information prevailed over the volumetric information, and vice versa with the grasping chain aimed at moving the tool (Experiment 2). Instead, the functional and volumetric information were balanced for those motor chains that comprise at least an observational act (Experiment 1). Overall our results are in keeping with the embodied theory of language and suggest that understanding sentences expressing an action directed toward a tool drives a chained activation of the motor system.

Putting concepts into context

At first glance, conceptual representations (e.g., our internal notion of the object "lemon") seem static; we have the impression that there is something that the concept lemon "means" (a sour, yellow, football-shaped citrus fruit) and that this meaning does not vary. Research in semantic memory has traditionally taken this "static" perspective. Consequently, only effects demonstrated across a variety of contexts have typically been considered informative regarding the architecture of the semantic system. In this review, we take the opposite approach: We review instances of context-dependent conceptual activation at many different timescales-from long-term experience, to recent experience, to the current task goals, to the unfolding process of conceptual activation itself-and suggest that the pervasive effects of context across all of these timescales indicate that rather than being static, conceptual representations are constantly changing and are inextricably linked to their contexts.

Manipulation gesture effect in visual and auditory presentations: the link between tools in perceptual and motor tasks

There is much behavioral and neurophysiological evidence in support of the idea that seeing a tool activates motor components of action related to the perceived object (e.g., grasping, use manipulation). However, the question remains as to whether the processing of the motor components associated with the tool is automatic or depends on the situation, including the task and the modality of tool presentation. The present study investigated whether the activation of motor components involved in tool use in response to the simple perception of a tool is influenced by the link between prime and target tools, as well as by the modality of presentation, in perceptual or motor tasks. To explore this issue, we manipulated the similarity of gesture involved in the use of the prime and target (identical, similar, different) with two tool presentation modalities of the presentation tool (visual or auditory) in perceptual and motor tasks. Across the experiments, we also manipulated the relevance of the prime (i.e., associated or not with the current task). The participants saw a first tool (or heard the sound it makes), which was immediately followed by a second tool on which they had to perform a perceptual task (i.e., indicate whether the second tool was identical to or different from the first tool) or a motor task (i.e., manipulate the second tool as if it were the first tool). In both tasks, the similarity between the gestures employed for the first and the second tool was manipulated (Identical, Similar or Different gestures). The results showed that responses were faster when the manipulation gestures for the two tools were identical or similar, but only in the motor task. This effect was observed irrespective of the modality of presentation of the first tool, i.e., visual or auditory. We suggest that the influence of manipulation gesture on response time depends on the relevance of the first tool in motor tasks. We discuss these motor activation results in terms of the relevance and demands of the tasks.

Stable and variable affordances are both automatic and flexible

The mere observation of pictures or words referring to manipulable objects is sufficient to evoke their affordances since objects and their nouns elicit components of appropriate motor programs associated with object interaction. While nobody doubts that objects actually evoke motor information, the degree of automaticity of this activation has been recently disputed. Recent evidence has indeed revealed that affordances activation is flexibly modulated by the task and by the physical and social context. It is therefore crucial to understand whether these results challenge previous evidence showing that motor information is activated independently from the task. The context and the task can indeed act as an early or late filter. We will review recent data consistent with the notion that objects automatically elicit multiple affordances and that top-down processes select among them probably inhibiting motor information that is not consistent with behavior goals. We will therefore argue that automaticity and flexibility of affordances are not in conflict. We will also discuss how language can incorporate affordances showing similarities, but also differences, between the motor information elicited by vision and language. Finally we will show how the distinction between stable and variable affordances can accommodate all these effects.

Impact of action primes on implicit processing of thematic and functional similarity relations: evidence from eye-tracking

The aim of this study was to specify the role of action representations in thematic and functional similarity relations between manipulable artifact objects. Recent behavioral and neurophysiological evidence indicates that while they are all relevant for manipulable artifact concepts, semantic relations based on thematic (e.g., saw-wood), specific function similarity (e.g., saw-axe), and general function similarity (e.g., saw-knife) are differently processed, and may relate to different levels of action representation. Point-light displays of object-related actions previously encoded at the gesture level (e.g., "sawing") or at the higher level of action representation (e.g., "cutting") were used as primes before participants identified target objects (e.g., saw) among semantically related and unrelated distractors (e.g., wood, feather, piano). Analysis of eye movements on the different objects during target identification informed about the amplitude and the timing of implicit activation of the different semantic relations. Results showed that action prime encoding impacted the processing of thematic relations, but not that of functional similarity relations. Semantic competition with thematic distractors was greater and earlier following action primes encoded at the gesture level compared to action primes encoded at higher level. As a whole, these findings highlight the direct influence of action representations on thematic relation processing, and suggest that thematic relations involve gesture-level representations rather than intention-level representations.

A saw is first identified as an object used on wood: ERP evidence for temporal differences between Thematic and Functional similarity relations

The role of functional and motor information in manipulable artifact object semantic organization is still poorly understood. In particular, several types of semantic relations involving object functional knowledge may be distinguished. Functional similarity relations group objects with similar functions at relatively specific (e.g. saw-axe, both used to cut wood) or general (saw-knife, both used to cut) levels. Thematic relations group objects based on their complementarity in events (saw used upon/with wood). Recent eye-tracking data showed distinct temporal time courses for the different semantic relations, with fastest processing of thematic relations and slowest processing of general function similarity relations. Behavioral data suggest the involvement of distinct cognitive mechanisms in manipulable artifact object semantic processing. The aim of the present study was to assess the neural correlates of thematic, and specific and general function similarity relation processing. Specifically, we investigated whether time course differences between semantic relations could be highlighted at the neurophysiological level. We used a protocol combining semantic priming with electroencephalography, and manipulated the type of semantic relation and the duration of the interval between prime and target objects. Two consistent and complementary results were shown. On N1 and P3 components, semantic priming was observed for thematic relations only. On N400 component, the type of semantic relation interacted with interval duration, and semantic priming was visible for all 3 relations after the longest interval only. Results revealed graded processing time courses for thematic, specific function similarity, and general function similarity relations at the neural level, and further indicate that thematic relations impact object processing during the early stages of object recognition. Findings suggest a hierarchical organization of three types of semantic relations based on functional knowledge. The parallel between semantic relations involving manipulable artifact objects and levels of action representations is discussed.

Visual context modulates potentiation of grasp types during semantic object categorization

Substantial evidence suggests that conceptual processing of manipulable objects is associated with potentiation of action. Such data have been viewed as evidence that objects are recognized via access to action features. Many objects, however, are associated with multiple actions. For example, a kitchen timer may be clenched with a power grip to move it but pinched with a precision grip to use it. The present study tested the hypothesis that action evocation during conceptual object processing is responsive to the visual scene in which objects are presented. Twenty-five healthy adults were asked to categorize object pictures presented in different naturalistic visual contexts that evoke either move- or use-related actions. Categorization judgments (natural vs. artifact) were performed by executing a move- or use-related action (clench vs. pinch) on a response device, and response times were assessed as a function of contextual congruence. Although the actions performed were irrelevant to the categorization judgment, responses were significantly faster when actions were compatible with the visual context. This compatibility effect was largely driven by faster pinch responses when objects were presented in use-compatible, as compared with move-compatible, contexts. The present study is the first to highlight the influence of visual scene on stimulus-response compatibility effects during semantic object processing. These data support the hypothesis that action evocation during conceptual object processing is biased toward context-relevant actions.

A distributed network critical for selecting among tool-directed actions

Tools pose a challenge to the need to select actions appropriate for task goals and environmental constraints. For many tools (e.g., calculator), actions for "using" and "grasping-to-move" conflict with each other and may compete during selection. To date, little is known about the mechanisms that enable selection between possible tool actions or their neural substrates. The study of patients with chronic left hemisphere stroke, many of whom are deficient in tool-use action (apraxic), provides an opportunity to elucidate these issues. Here, 31 such patients pantomimed or recognized tool use actions for "conflict" and "non-conflict" tools. Voxel-based lesion-symptom mapping (VLSM), lesion subtraction, and tractographic overlap analyses were used to determine brain regions necessary for selecting among tool-directed actions. Lesions to posterior middle temporal gyrus (pMTG) and anterior intraparietal sulcus (aIPS) tended to impair production of use actions similarly for both conflict and non-conflict tools. By contrast, lesions to the supramarginal gyrus (SMG), inferior frontal gyrus (IFG)/anterior insula, and superior longitudinal fasciculus (SLF) specifically impaired production of use actions for conflict tools. Patients' errors on conflict tools suggested inappropriate selection of grasping actions and difficulty selecting single actions. Use/grasp conflict had no effect on action recognition. We suggest that the SMG/SLF/IFG pathway implements biased competition between possible tool actions, while aIPS and pMTG compute the structure-based and skilled use actions, respectively, that constitute input to this competitive process. This is the first study to demonstrate a reliable link between a characteristic of single tools (i.e., their association with different use and grasp actions) and action selection difficulties. Additionally, the data allow us to posit an SMG-involved subtype of apraxia characterized by an inability to resolve action competition.

How do you hold your mouse? Tracking the compatibility effect between hand posture and stimulus size

In keeping with the idea that observing objects activates possible motor responses, several experiments revealed compatibility effects between the hand postures used to report a choice and some characteristics of the stimuli. The real-time dynamics of such compatibility effects are currently unknown. We tracked the time course of a categorization experiment requiring subjects to categorize as natural or artifact figures of big and small objects. Participants reported their choice using either a big mouse (requiring a power grip: a hand posture compatible with the grasping of big objects) or a small mouse (requiring a precision grip: a hand posture compatible with the grasping of small objects). We found a compatibility effect between the grip required by the mouse and the grip elicited by objects, even if it was irrelevant to the task. In a following experiment with the same paradigm, lexical stimuli failed to reproduce the same effect. Nevertheless, a compatibility effect mediated by the target-word category (artificial vs. natural) was observed. We discuss the results in the context of affordance effects literature and grounded theories of cognition.

Uncovering the architecture of action semantics

Despite research suggesting that stored sensorimotor information about tool use is a component of the semantic representations of tools, little is known about the action features or organizing principles that underlie this knowledge. We used methods similar to those applied in other semantic domains to examine the "architecture" of action semantic knowledge. In Experiment 1, participants sorted photographs of tools into groups according to the similarity of their associated "use" actions and rated tools on dimensions related to action. The results suggest that the magnitude of arm movement, configuration of the hand, and manner of motion during tool use play a role in determining how tools cluster in action "semantic space." In Experiment 2, we validated the architecture uncovered in Experiment 1 using an implicit semantic task for which tool use knowledge was not ostensibly relevant (blocked cyclic word-picture matching). Using stimuli from Experiment 1, we found that participants performed more poorly during blocks of trials containing tools used with similar versus unrelated actions, and the amount of semantic interference depended on the magnitude of action similarity among tools. Thus, the degree of featural overlap between tool use actions plays a role in determining the overall semantic similarity of tools.

Critical brain regions for tool-related and imitative actions: a componential analysis

Numerous functional neuroimaging studies suggest that widespread bilateral parietal, temporal, and frontal regions are involved in tool-related and pantomimed gesture performance, but the role of these regions in specific aspects of gestural tasks remains unclear. In the largest prospective study of apraxia-related lesions to date, we performed voxel-based lesion-symptom mapping with data from 71 left hemisphere stroke participants to assess the critical neural substrates of three types of actions: gestures produced in response to viewed tools, imitation of tool-specific gestures demonstrated by the examiner, and imitation of meaningless gestures. Thus, two of the three gesture types were tool-related, and two of the three were imitative, enabling pairwise comparisons designed to highlight commonalities and differences. Gestures were scored separately for postural (hand/arm positioning) and kinematic (amplitude/timing) accuracy. Lesioned voxels in the left posterior temporal gyrus were significantly associated with lower scores on the posture component for both of the tool-related gesture tasks. Poor performance on the kinematic component of all three gesture tasks was significantly associated with lesions in left inferior parietal and frontal regions. These data enable us to propose a componential neuroanatomic model of action that delineates the specific components required for different gestural action tasks. Thus, visual posture information and kinematic capacities are differentially critical to the three types of actions studied here: the kinematic aspect is particularly critical for imitation of meaningless movement, capacity for tool-action posture representations are particularly necessary for pantomimed gestures to the sight of tools, and both capacities inform imitation of tool-related movements. These distinctions enable us to advance traditional accounts of apraxia.

Abnormal dynamics of activation of object use information in apraxia: evidence from eyetracking

Action representations associated with object use may be incidentally activated during visual object processing, and the time course of such activations may be influenced by lexical-semantic context (e.g., Lee, Middleton, Mirman, Kalénine, & Buxbaum (2012). Journal of Experimental Psychology: Human Perception and Performance, 39(1), 257-270). In this study we used the "visual world" eye-tracking paradigm to examine whether a deficit in producing skilled object-use actions (apraxia) is associated with abnormalities in incidental activation of action information, and assessed the neuroanatomical substrates of any such deficits. Twenty left hemisphere stroke patients, ten of whom were apraxic, performed a task requiring identification of a named object in a visual display containing manipulation-related and unrelated distractor objects. Manipulation relationships among objects were not relevant to the identification task. Objects were cued with neutral ("S/he saw the…."), or action-relevant ("S/he used the….") sentences. Non-apraxic participants looked at use-related non-target objects significantly more than at unrelated non-target objects when cued both by neutral and action-relevant sentences, indicating that action information is incidentally activated. In contrast, apraxic participants showed delayed activation of manipulation-based action information during object identification when cued by neutral sentences. The magnitude of delayed activation in the neutral sentence condition was reliably predicted by lower scores on a test of gesture production to viewed objects, as well as by lesion loci in the inferior parietal and posterior temporal lobes. However, when cued by a sentence containing an action verb, apraxic participants showed fixation patterns that were statistically indistinguishable from non-apraxic controls. In support of grounded theories of cognition, these results suggest that apraxia and temporal-parietal lesions may be associated with abnormalities in incidental activation of action information from objects. Further, they suggest that the previously-observed facilitative role of action verbs in the retrieval of object-related action information extends to participants with apraxia.

Predicting and interpreting identification errors in military vehicle training using multidimensional scaling

We compared methods for predicting and understanding the source of confusion errors during military vehicle identification training. Participants completed training to identify main battle tanks. They also completed card-sorting and similarity-rating tasks to express their mental representation of resemblance across the set of training items. We expected participants to selectively attend to a subset of vehicle features during these tasks, and we hypothesised that we could predict identification confusion errors based on the outcomes of the card-sort and similarity-rating tasks. Based on card-sorting results, we were able to predict about 45% of observed identification confusions. Based on multidimensional scaling of the similarity-rating data, we could predict more than 80% of identification confusions. These methods also enabled us to infer the dimensions receiving significant attention from each participant. This understanding of mental representation may be crucial in creating personalised training that directs attention to features that are critical for accurate identification.

PRACTITIONER SUMMARY

Participants completed military vehicle identification training and testing, along with card-sorting and similarity-rating tasks. The data enabled us to predict up to 84% of identification confusion errors and to understand the mental representation underlying these errors. These methods have potential to improve training and reduce identification errors leading to fratricide.

The role of action representations in thematic object relations

A number of studies have explored the role of associative/event-based (thematic) and categorical (taxonomic) relations in the organization of object representations. Recent evidence suggests that thematic information may be particularly important in determining relationships between manipulable artifacts. However, although sensorimotor information is on many accounts an important component of manipulable artifact representations, little is known about the role that action may play during the processing of semantic relationships (particularly thematic relationships) between multiple objects. In this study, we assessed healthy and left hemisphere stroke participants to explore three questions relevant to object relationship processing. First, we assessed whether participants tended to favor thematic relations including action (Th+A, e.g., wine bottle-corkscrew), thematic relationships without action (Th-A, e.g., wine bottle-cheese), or taxonomic relationships (Tax, e.g., wine bottle-water bottle) when choosing between them in an association judgment task with manipulable artifacts. Second, we assessed whether the underlying constructs of event relatedness, action relatedness, and categorical relatedness determined the choices that participants made. Third, we assessed the hypothesis that degraded action knowledge and/or damage to temporo-parietal cortex, a region of the brain associated with the representation of action knowledge, would reduce the influence of action on the choice task. Experiment 1 showed that explicit ratings of event, action, and categorical relatedness were differentially predictive of healthy participants' choices, with action relatedness determining choices between Th+A and Th-A associations above and beyond event and categorical ratings. Experiment 2 focused more specifically on these Th+A vs. Th-A choices and demonstrated that participants with left temporo-parietal lesions, a brain region known to be involved in sensorimotor processing, were less likely than controls and tended to be less likely than patients with lesions sparing that region to use action relatedness in determining their choices. These data indicate that action knowledge plays a critical role in processing of thematic relations for manipulable artifacts.

Two action systems in the human brain

The distinction between dorsal and ventral visual processing streams, first proposed by Ungerleider and Mishkin (1982) and later refined by Milner and Goodale (1995) has been elaborated substantially in recent years, spurred by two developments. The first was proposed in large part by Rizzolatti and Matelli (2003) and is a more detailed description of the multiple neural circuits connecting the frontal, temporal, and parietal cortices. Secondly, there are a number of behavioral observations that the classic "two visual systems" hypothesis is unable to accommodate without additional assumptions. The notion that the Dorsal stream is specialized for "where" or "how" actions and the Ventral stream for "What" knowledge cannot account for two prominent disorders of action, limb apraxia and optic ataxia, that represent a double dissociation in terms of the types of actions that are preserved and impaired. A growing body of evidence, instead, suggests that there are at least two distinct Dorsal routes in the human brain, referred to as the "Grasp" and "Use" systems. Both of these may be differentiated from the Ventral route in terms of neuroanatomic localization, representational specificity, and time course of information processing.

Temporal dynamics of activation of thematic and functional knowledge during conceptual processing of manipulable artifacts

The current research aimed at specifying the activation time course of different types of semantic information during object conceptual processing and the effect of context on this time course. We distinguished between thematic and functional knowledge and the specificity of functional similarity. Two experiments were conducted with healthy older adults using eye tracking in a word-to-picture matching task. The time course of gaze fixations was used to assess activation of distractor objects during the identification of manipulable artifact targets (e.g., broom). Distractors were (a) thematically related (e.g., dustpan), (b) related by a specific function (e.g., vacuum cleaner), or (c) related by a general function (e.g., sponge). Growth curve analyses were used to assess competition effects when target words were presented in isolation (Experiment 1) and embedded in contextual sentences of different generality levels (Experiment 2). In the absence of context, there was earlier and shorter lasting activation of thematically related as compared to functionally related objects. The time course difference was more pronounced for general functions than specific functions. When contexts were provided, functional similarities that were congruent with context generality level increased in salience with earlier activation of those objects. Context had little impact on thematic activation time course. These data demonstrate that processing a single manipulable artifact concept implicitly activates thematic and functional knowledge with different time courses and that context speeds activation of context-congruent functional similarity.