Visual context modulates potentiation of grasp types during semantic object categorization

A fist bump in a political meeting? The influence of social context on affordance selection

Historically, understanding human cognition such as action processing has been a challenging issue in cognitive neuropsychology and the more we know about cognition, the more we shape it as a complex, multi-determined phenomenon that is embedded in a social context. The present study aimed at understanding how the social context could influence affordance selection. We hypothesized that affordance selection would be modulated by social context and that a given hand configuration would be considered appropriate or not, as a function of the presence or absence of social context. Twenty-six healthy participants were asked to judge the appropriateness of three variants of 10 hand-object interactions based on photographs presented with or without a visual, social context. In our results, hand configurations were intrinsically acceptable or not, but this effect was modulated by the social context. A three-step model of the influence of social context on affordance selection was proposed, according to which selection depends on social norms, in the form of social knowledge and social context analysis.

Manipulable objects can potentiate pointing and unusual grasping besides habitual grasping behaviors

People are faster to perform a precision grip when they see a cherry (i.e., a small graspable object) than to perform a power grip, and the reverse holds true when they see an apple (i.e., a large graspable object). This potentiation effect supports that object representations could include motor components that would be simulated when a graspable object is seen. However, the nature of these motor components remains unclear. The embodied account posits that seeing an object only potentiates the most frequent actions associated with it (i.e., usual actions). In contrast, the size-coding account posits that seeing an object potentiates any actions associated to spatial codes compatible with those associated to the objects. We conducted three experiments to disentangle these two alternative accounts. We especially varied the nature of the responses while participants saw either large or small graspable objects. Our results showed a potentiation effect when participants performed the usual grasping actions (Experiment 1: power and precision grip) but also when they performed unusual grasping actions (Experiment 2: grasping between the thumb and little finger) and even when they had to perform non-grasping actions (Experiment 3: pointing actions). By supporting the size-coding account, our contribution underlines the need for a better understanding of the nature of the motor components of object representations and for using a proper control condition (i.e., pointing action) before arguing that the embodied account convincingly explains the potentiation effect of grasping behaviors.

Too much to handle? Interference from distractors with similar affordances on target selection for handled objects

The existence of handle affordances has been classically demonstrated using the Stimulus-Response Compatibility paradigm, with shorter response times when the orientation of the object handle and the response hand are compatible in comparison to incompatible. Yet the activation of handle affordances from visual objects has been investigated in very simple situations involving single stimulus and motor response. As natural perceptual scenes are usually composed of multiple objects that could activate multiple affordances, the consequence of multiple affordance activation on the perception and processing of a given object of the scene requires more investigation. The aim of this study was to determine the impact of distractor affordances on the processing of a target object in situations involving several familiar graspable objects. In two online experiments, 229 participants had to select a target object (the kitchen utensil or the tool) in a visual scene displaying a pair of objects. They performed left key presses when the target was on the left and right key presses when the target was on the right. Target handle orientation and response side could be compatible or incompatible. Critically, target and distractor objects had similar or dissimilar handle affordances, with handles oriented for left- or right-hand grasps. Results from the two experiments showed slower response times when target and distractor objects had similar handle affordances in comparison to dissimilar affordances, when participants performed right hand responses and when target orientation and response were compatible. Thus, affordance similarity between objects may interfere rather than facilitate object processing and slow down target selection. These findings are in line with models of affordance and object selection assuming automatic inhibition of distractors' affordances for appropriate object interaction.

Congruent action context releases Mu rhythm desynchronization when visual objects activate competing action representations

Recent findings demonstrated that object perception is affected by the competition between action representations. Simultaneous activation of distinct structural ("grasp-to-move") and functional ("grasp-to-use") action representations slows down perceptual judgements on objects. At the brain level, competition reduces motor resonance effects during manipulable object perception, reflected by an extinction of μ rhythm desynchronization. However, how this competition is solved in the absence of object-directed action remains unclear. The present study investigates the role of context in the resolution of the competition between conflicting action representations during mere object perception. To this aim, thirty-eight volunteers were instructed to perform a reachability judgment task on 3D objects presented at different distances in a virtual environment. Objects were conflictual objects associated with distinct structural and functional action representations. Verbs were used to provide a neutral or congruent action context prior or after object presentation. Neurophysiological correlates of the competition between action representation were recorded using EEG. The main result showed a release of μ rhythm desynchronization when reachable conflictual objects were presented with a congruent action context. Context influenced μ rhythm desynchronization when the action context was provided prior or after object presentation in a time-window compatible with object-context integration (around 1000 ms after the presentation of the first stimulus). These findings revealed that action context biases competition between co-activated action representations during mere object perception and demonstrated that μ rhythm desynchronization may be an index of activation but also competition between action representations in perception.

From Affordances to Abstract Words: The Flexibility of Sensorimotor Grounding

The sensorimotor system plays a critical role in several cognitive processes. Here, we review recent studies documenting this interplay at different levels. First, we concentrate on studies that have shown how the sensorimotor system is flexibly involved in interactions with objects. We report evidence demonstrating how social context and situations influence affordance activation, and then focus on tactile and kinesthetic components in body-object interactions. Then, we turn to word use, and review studies that have shown that not only concrete words, but also abstract words are grounded in the sensorimotor system. We report evidence that abstract concepts activate the mouth effector more than concrete concepts, and discuss this effect in light of studies on adults, children, and infants. Finally, we pinpoint possible sensorimotor mechanisms at play in the acquisition and use of abstract concepts. Overall, we show that the involvement of the sensorimotor system is flexibly modulated by context, and that its role can be integrated and flanked by that of other systems such as the linguistic system. We suggest that to unravel the role of the sensorimotor system in cognition, future research should fully explore the complexity of this intricate, and sometimes slippery, relation.

Treating limb apraxia via action semantics: a preliminary study

Limb apraxia is evident in approximately 50% of patients after left hemisphere cerebral vascular accident (LCVA) and increases disability and caregiver dependence. Individuals with apraxia exhibit abnormalities in spatio-temporal aspects of gesture production and/or in knowledge of tool-related actions (action semantics). This preliminary study of three LCVA participants aimed to (i) explore the efficacy of a novel Action Network Treatment (ANT) that focused on improving the semantic association between tool actions and other types of tool knowledge, an intervention inspired by successful semantic network treatments in aphasia (e.g., Edmonds et al., 2009), and (ii) explore whether there are individuals with apraxia who benefit from ANT relative to a version of a comparatively well-studied existing apraxia treatment (Smania et al., 2006; Smania et al., 2000) that shapes gesture via focus on practicing the spatio-temporal aspects of gesture production (Tool Use Treatment or TUT). One participant demonstrated treatment benefits from both ANT and TUT, while another only benefited from TUT. These findings indicate that our novel semantic network strengthening approach to gesture training may be efficacious in at least some individuals with apraxia, and provide a foundation for future study of the characteristics of people with apraxia who benefit from each approach.

Priming of grasp affordance in an ambiguous object: evidence from ERPs, source localization, and motion tracking

Object affordance refers to possibilities to interact with the objects in our environment, such as grasping. Previous research shows that objects that afford an action activate the motor system and attract attention, for example they elicit an enhanced frontal negativity and posterior P1 in the event-related potential. An effect on posterior N1 is discussed. However, previous findings might have resulted from physical differences between affording and non-affording stimuli, rather than affordance per se. Here we replicated the frontal negativity and posterior P1 effects and further explored the posterior N1 in affordance processing under constant visual input. An ambiguous target was primed either with an affording (pencils) or non-affording (trees) context. Although physically always identical, the target elicited an enhanced frontal negativity and posterior P1 in the pencil prime condition. Posterior N1 was reduced and grip aperture in a grasping task was smaller in the affording context. Source localization revealed stronger activation in occipital and parietal regions for targets in pencil versus tree prime trials. Thus, we successfully show that an ambiguous object primed with an affording context is processed differently than when primed with a non-affording context. This could be related to the ambiguous object acquiring a potential for action through priming.

Scene context shapes category representational geometry during processing of tools

Tools are ubiquitous in human environments and to think about them we use concepts. Increasingly, conceptual representation is thought to be dynamic and sensitive to the goals of the observer. Indeed, observer goals can reshape representational geometry within cortical networks supporting concepts. In the present study, we investigated the novel hypothesis that task-irrelevant scene context may implicitly alter the representational geometry of regions within the tool network. Participants performed conceptual judgments on images of tools embedded in scenes that either suggested their use (i.e., a kitchen timer sitting on a kitchen counter with vegetables in a frying pan) or that they would simply be moved (i.e., a kitchen timer sitting in an open drawer with other miscellaneous kitchen items around). We investigated whether representations in the tool network reflect category, grip, and shape information using a representational similarity analysis (RSA). We show that a) a number of regions of the tool network reflect category information about tools and b) category information predicts patterns in supramarginal gyrus more strongly in use contexts than in move contexts. Together, these results show that information about tool category is distributed across different regions of the tool network and that scene context helps shape the representational geometry of the tool network.

Curvilinear features are important for animate/inanimate categorization in macaques

The current experiment investigated the extent to which perceptual categorization of animacy (i.e., the ability to discriminate animate and inanimate objects) is facilitated by image-based features that distinguish the two object categories. We show that, with nominal training, naïve macaques could classify a trial-unique set of 1000 novel images with high accuracy. To test whether image-based features that naturally differ between animate and inanimate objects, such as curvilinear and rectilinear information, contribute to the monkeys’ accuracy, we created synthetic images using an algorithm that distorted the global shape of the original animate/inanimate images while maintaining their intermediate features (Portilla & Simoncelli, 2000). Performance on the synthesized images was significantly above chance and was predicted by the amount of curvilinear information in the images. Our results demonstrate that, without training, macaques can use an intermediate image feature, curvilinearity, to facilitate their categorization of animate and inanimate objects.

Motor Inhibition to Dangerous Objects: Electrophysiological Evidence for Task-dependent Aversive Affordances

Previous work suggests that perception of an object automatically facilitates actions related to object grasping and manipulation. Recently, the notion of automaticity has been challenged by behavioral studies suggesting that dangerous objects elicit aversive affordances that interfere with encoding of an object's motor properties; however, related EEG studies have provided little support for these claims. We sought EEG evidence that would support the operation of an inhibitory mechanism that interferes with the motor encoding of dangerous objects, and we investigated whether such mechanism would be modulated by the perceived distance of an object and the goal of a given task. EEGs were recorded by 24 participants who passively perceived dangerous and neutral objects in their peripersonal, boundary, or extrapersonal space and performed either a reachability judgment task or a categorization task. Our results showed that greater attention, reflected in the visual P1 potential, was drawn by dangerous and reachable objects. Crucially, a frontal N2 potential, associated with motor inhibition, was larger for dangerous objects only when participants performed a reachability judgment task. Furthermore, a larger parietal P3b potential for dangerous objects indicated the greater difficulty in linking a dangerous object to the appropriate response, especially when it was located in the participants' extrapersonal space. Taken together, our results show that perception of dangerous objects elicits aversive affordances in a task-dependent way and provides evidence for the operation of a neural mechanism that does not code affordances of dangerous objects automatically, but rather on the basis of contextual information.

Knowing what you need to know in advance: The neural processes underpinning flexible semantic retrieval of thematic and taxonomic relations

Semantic retrieval is flexible, allowing us to focus on subsets of features and associations that are relevant to the current task or context: for example, we use taxonomic relations to locate items in the supermarket (carrots are a vegetable), but thematic associations to decide which tools we need when cooking (carrot goes with peeler). We used fMRI to investigate the neural basis of this form of semantic flexibility; in particular, we asked how retrieval unfolds differently when participants have advanced knowledge of the type of link to retrieve between concepts (taxonomic or thematic). Participants performed a semantic relatedness judgement task: on half the trials, they were cued to search for a taxonomic or thematic link, while on the remaining trials, they judged relatedness without knowing which type of semantic relationship would be relevant. Left inferior frontal gyrus showed greater activation when participants knew the trial type in advance. An overlapping region showed a stronger response when the semantic relationship between the items was weaker, suggesting this structure supports both top-down and bottom-up forms of semantic control. Multivariate pattern analysis further revealed that the neural response in left inferior frontal gyrus reflects goal information related to different conceptual relationships. Top-down control specifically modulated the response in visual cortex: when the goal was unknown, there was greater deactivation to the first word, and greater activation to the second word. We conclude that top-down control of semantic retrieval is primarily achieved through the gating of task-relevant 'spoke' regions.

Size coding of alternative responses is sufficient to induce a potentiation effect with manipulable objects

The mere perception of manipulable objects usually grasped with a power-grip (e.g., an apple) or a precision-grip (e.g., a cherry) potentiate power-grip- and precision-grip-responses, respectively. This effect is seen as to be driven by automatic access of the representation of manipulable objects that includes a motor representation of usually performed grasping behaviors (i.e., the embodied view). Nevertheless, a competing account argues that this effect could be due to an overlapping of size codes used to represent both manipulable objects and response options. Indeed, objects usually grasped with a power- and a precision-grip (e.g., an apple vs. a cherry) could be coded as large- and small-objects, respectively; and power- and precision-grip responses as large- and small-responses, respectively. We conducted 4 experiments to test this hypothesis. In Experiment 1, the response device usually used in studies reporting a potentiation effect is fixed horizontally (the grasping component of responses was removed). We instructed participants to press the small-switch with their index-digit and the large-switch with their palm-hand. In line with the size-coding-hypothesis, responses on the small-switch performed with the index-digit led to shorter RTs when objects usually associated with a precision-grip (e.g., a cherry) were presented compared to objects usually associated with a power-grip (e.g., an apple). A reverse pattern was obtained for responses on the large-switch performed with the palm-hand. In Experiments 2, 3 and 4, we went further by investigating which factors of Experiment 1 allow the size coding of responses: the size of switch and/or the size of the effector part used. Data confirmed the critical involvement of the size of switches and the possible involvement of the size of the effector part used. Thus, data support the possibility that the potentiation of grasping is due to a compatibility/incompatibility between size codes rather than involving motor representations of usually performed grasping behaviors as advocated in several embodied views. Moreover, data support the possibility that responses are coded thanks to a size code that extends the Theory of Event Coding.

Affordance matching predictively shapes the perceptual representation of others' ongoing actions

Predictive processing accounts of social perception argue that action observation is a predictive process, in which inferences about others' goals are tested against the perceptual input, inducing a subtle perceptual confirmation bias that distorts observed action kinematics toward the inferred goals. Here we test whether such biases are induced even when goals are not explicitly given but have to be derived from the unfolding action kinematics. In 2 experiments, participants briefly saw an actor reach ambiguously toward a large object and a small object, with either a whole-hand power grip or an index-finger and thumb precision grip. During its course, the hand suddenly disappeared, and participants reported its last seen position on a touch-screen. As predicted, judgments were consistently biased toward apparent action targets, such that power grips were perceived closer to large objects and precision grips closer to small objects, even if the reach kinematics were identical. Strikingly, these biases were independent of participants' explicit goal judgments. They were of equal size when action goals had to be explicitly derived in each trial (Experiment 1) or not (Experiment 2) and, across trials and across participants, explicit judgments and perceptual biases were uncorrelated. This provides evidence, for the first time, that people make online adjustments of observed actions based on the match between hand grip and object goals, distorting their perceptual representation toward implied goals. These distortions may not reflect high-level goal assumptions, but emerge from relatively low-level processing of kinematic features within the perceptual system. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Is a letterbox always a letterbox? The role of affordances in guiding perceptual categorization

Classically investigated in the context of judgment tasks about achievable actions, affordances have also been investigated in the context of the stimulus-response compatibility paradigm. Earlier work showed that perceptual categorization performance is significantly faster and more accurate when the orientation of the graspable part of a presented object, and the orientation of the participant's response are compatible, suggesting that the main function of affordances is restricted to action preparation. Here, we investigate the potential role of affordances in the categorization of ambiguous stimuli through a stimulus-response compatibility paradigm. In other words, we investigate if in ambiguous situations, such as ones in which a stimulus may give rise to two percepts, affordances would stabilize perception on one of these two and, therefore, helps in the subsequent categorizations. Two experiments were run, based on the forced-choice stimulus-response compatibility (SRC) paradigm, with a progressive series of ambiguous (bistable) lateral-graspable objects. In Experiment 1, subjects responded by pressing horizontally opposite keyboard keys, while in Experiment 2, the keyboard keys were vertically separated. Experiment 1 found that subjects perceived the initial object in a bistable series for longer, and exhibited greater response stability in compatible than incompatible situations. In Experiment 2, none of these modulations were significant. Overall, our results show that affordances operationalized through a SRC paradigm modulated how subjects categorized ambiguous stimuli. We argue that affordances may play a substantial role in ambiguous contexts by reducing the uncertainty of such situations.

Effects of grasp compatibility on long-term memory for objects

Previous studies have shown action potentiation during conceptual processing of manipulable objects. In four experiments, we investigated whether these motor actions also play a role in long-term memory. Participants categorized objects that afforded either a power grasp or a precision grasp as natural or artifact by grasping cylinders with either a power grasp or a precision grasp. In all experiments, responses were faster when the affordance of the object was compatible with the type of grasp response. However, subsequent free recall and recognition memory tasks revealed no better memory for object pictures and object names for which the grasp affordance was compatible with the grasp response. The present results therefore do not support the hypothesis that motor actions play a role in long-term memory.

Making sense of objects lying around: How contextual objects shape brain activity during action observation

Action recognition involves not only the readout of body movements and involved objects but also the integration of contextual information, e.g. the environment in which an action takes place. Notably, inferring superordinate goals and generating predictions about forthcoming action steps should benefit from screening the actor's immediate environment, in particular objects located in the actor's peripersonal space and thus potentially used in following action steps. Critically, if such contextual objects (COs) afford actions that are semantically related to the observed action, they may trigger or facilitate the inference of goals and the prediction of following actions. This fMRI study investigated the neural mechanisms underlying the integration of COs in semantic and spatial relation to observed actions. Specifically, we tested the hypothesis that the inferior frontal gyrus (IFG) subserves this integration. Participants observed action videos in which COs and observed actions had common overarching goals or not (goal affinity) and varied in their location relative to the actor. High goal affinity increased bilateral activity in action observation network nodes, i.e. the occipitotemporal cortex and the intraparietal sulcus, but also in the precuneus and middle frontal gyri. This finding suggests that the semantic relation between COs and actions is considered during action observation and triggers (rather than facilitates) processes beyond those usually involved in action observation. Moreover, COs with high goal affinity located close to the actor's dominant hand additionally engaged bilateral IFG, corroborating the view that IFG is critically involved in the integration of action steps under a common overarching goal.

Differential Tuning of Ventral and Dorsal Streams during the Generation of Common and Uncommon Tool Uses

Our use of tools is situated in different contexts. Prior evidence suggests that diverse regions within the ventral and dorsal streams represent information supporting common tool use. However, given the flexibility of object concepts, these regions may be tuned to different types of information when generating novel or uncommon uses of tools. To investigate this, we collected fMRI data from participants who reported common or uncommon tool uses in response to visually presented familiar objects. We performed a pattern dissimilarity analysis in which we correlated cortical patterns with behavioral measures of visual, action, and category information. The results showed that evoked cortical patterns within the dorsal tool use network reflected action and visual information to a greater extent in the uncommon use group, whereas evoked neural patterns within the ventral tool use network reflected categorical information more strongly in the common use group. These results reveal the flexibility of cortical representations of tool use and the situated nature of cortical representations more generally.

Learning, remembering, and predicting how to use tools: Distributed neurocognitive mechanisms: Comment on Osiurak and Badets (2016)

The reasoning-based approach championed by Francois Osiurak and Arnaud Badets (Osiurak & Badets, 2016) denies the existence of sensory-motor memories of tool use except in limited circumstances, and suggests instead that most tool use is subserved solely by online technical reasoning about tool properties. In this commentary, I highlight the strengths and limitations of the reasoning-based approach and review a number of lines of evidence that manipulation knowledge is in fact used in tool action tasks. In addition, I present a "two route" neurocognitive model of tool use called the "Two Action Systems Plus (2AS+)" framework that posits a complementary role for online and stored information and specifies the neurocognitive substrates of task-relevant action selection. This framework, unlike the reasoning based approach, has the potential to integrate the existing psychological and functional neuroanatomic data in the tool use domain. (PsycINFO Database Record

Is there a Competition between Functional and Situational Affordances during Action Initiation with Everyday Tools?

Most studies of human-tool interactions focus on the typical use of a tool (e.g., cutting in the case of a knife). However, little is known about situations requiring atypical tool use (e.g., using a knife to tighten a screw). The present study focused on a selection of atypical uses of everyday tools which might be in conflict with their typical use. Our objective was to study how tool function influences the selection of the relevant action. In Experiment 1, which involved visuomotor priming, two everyday tools (a knife and a screwdriver) and two neutral tools (two bars, with no strong functional affordance) were used as primes and targets. Participants had to use the target with the appropriate box (indicated by the color) that allowed to make an action. Longer initiation times were observed when the prime was an everyday tool, irrespective of the nature of the target. We therefore observed a conflict between functional and situational affordances. To investigate whether the priming effect is caused by the task-irrelevance of the prime, we asked the participants in Experiment 2 to perform an action associated with the prime. The results showed longer initiation times only when the prime and target were everyday tools, irrespective of their precise nature. This suggests that activation of the typical use of a tool might not be fully automatic but flexible depending on the situation.

Chained Activation of the Motor System during Language Understanding

Two experiments were carried out to investigate whether and how one important characteristic of the motor system, that is its goal-directed organization in motor chains, is reflected in language processing. This possibility stems from the embodied theory of language, according to which the linguistic system re-uses the structures of the motor system. The participants were presented with nouns of common tools preceded by a pair of verbs expressing grasping or observational motor chains (i.e., grasp-to-move, grasp-to-use, look-at-to-grasp, and look-at-to-stare). They decided whether the tool mentioned in the sentence was the same as that displayed in a picture presented shortly after. A primacy of the grasp-to-use motor chain over the other motor chains in priming the participants' performance was observed in both the experiments. More interestingly, we found that the motor information evoked by the noun was modulated by the specific motor-chain expressed by the preceding verbs. Specifically, with the grasping chain aimed at using the tool, the functional motor information prevailed over the volumetric information, and vice versa with the grasping chain aimed at moving the tool (Experiment 2). Instead, the functional and volumetric information were balanced for those motor chains that comprise at least an observational act (Experiment 1). Overall our results are in keeping with the embodied theory of language and suggest that understanding sentences expressing an action directed toward a tool drives a chained activation of the motor system.

Is there any Influence of Variations in Context on Object-Affordance Effects in Schizophrenia? Perception of Property and Goals of Action

The simple perception of an object can potentiate an associated action. This affordance effect depends heavily on the action context in which the object is presented. In recent years, psychologists, psychiatrists, and phenomenologists have agreed that subjects with schizophrenia may not perceive the affordances of people or objects that could lead to a loss of ease in their actions. We examined whether the addition of contextually congruent elements, during the perception of everyday objects, could promote the emergence of object-affordance effects in subjects with schizophrenia and controls. Participants performed two Stimulus–Response-Compatibility tasks in which they were presented with semantic primes related to sense of property (Experiment 1) or goal of action (Experiment 2) prior to viewing each graspable object. Controls responded faster when their response hand and the graspable part of the object were compatibly oriented, but only when the context was congruent with the individual’s needs and goals. When the context operated as a constraint, the affordance-effect was disrupted. These results support the understanding that object-affordance is flexible and not just intrinsic to an object. However, the absence of this object-affordance effect in subjects with schizophrenia suggests the possible impairment of their ability to experience the internal simulation of motor action potentialities. In such case, all activities of daily life would require the involvement of higher cognitive processes rather than lower level sensorimotor processes. The study of schizophrenia requires the consideration of concepts and methods that arise from the theories of embodied and situated cognition.

Putting concepts into context

At first glance, conceptual representations (e.g., our internal notion of the object "lemon") seem static; we have the impression that there is something that the concept lemon "means" (a sour, yellow, football-shaped citrus fruit) and that this meaning does not vary. Research in semantic memory has traditionally taken this "static" perspective. Consequently, only effects demonstrated across a variety of contexts have typically been considered informative regarding the architecture of the semantic system. In this review, we take the opposite approach: We review instances of context-dependent conceptual activation at many different timescales-from long-term experience, to recent experience, to the current task goals, to the unfolding process of conceptual activation itself-and suggest that the pervasive effects of context across all of these timescales indicate that rather than being static, conceptual representations are constantly changing and are inextricably linked to their contexts.

Through the forest of motor representations

Following neuroscience, and using different labels, several philosophers have addressed the idea of the presence of a single representational mechanism lying in between (visual) perceptual processes and motor processes involved in different functions and useful for shaping suitable action performances: a motor representation (MR). MRs are the naturalized mental antecedents of action. This paper presents a new, non-monolithic view of MRs, according to which, contrarily to the received view, when looking at in between (visual) perceptual processes and motor processes, we find not only a single representational mechanism with different functions, but an ensemble of different sub-representational phenomena, each of which with a different function. This new view is able to avoid several issues emerging from the literature and to address something the literature is silent about, which however turns out to be crucial for a theory of MRs.

Square bananas, blue horses: the relative weight of shape and color in concept recognition and representation

The present study investigates the role that shape and color play in the representation of animate (i.e., animals) and inanimate manipulable entities (i.e., fruits), and how the importance of these features is modulated by different tasks. Across three experiments participants were shown either images of entities (e.g., a sheep or a pineapple) or images of the same entities modified in color (e.g., a blue pineapple) or in shape (e.g., an elongated pineapple). In Experiment 1 we asked participants to categorize the entities as fruit or animal. Results showed that with animals color does not matter, while shape modifications determined a deterioration of the performance - stronger for fruit than for animals. To better understand our findings, in Experiments 2 we asked participants to judge if entities were graspable (manipulation evaluation task). Participants were faster with manipulable entities (fruit) than with animals; moreover alterations in shape affected the response latencies more for animals than for fruit. In Experiment 3 (motion evaluation task), we replicated the disadvantage for shape-altered animals, while with fruits shape and color modifications produced no effect. By contrasting shape- and color- alterations the present findings provide information on shape/color relative weight, suggesting that the action based property of shape is more crucial than color for fruit categorization, while with animals it is critical for both manipulation and motion tasks. This contextual dependency is further revealed by explicit judgments on similarity - between the altered entities and the prototypical ones - provided after the different tasks. These results extend current literature on affordances and biofunctionally embodied understanding, revealing the relative robustness of biofunctional activity compared to intellectual one.

Stable and variable affordances are both automatic and flexible

The mere observation of pictures or words referring to manipulable objects is sufficient to evoke their affordances since objects and their nouns elicit components of appropriate motor programs associated with object interaction. While nobody doubts that objects actually evoke motor information, the degree of automaticity of this activation has been recently disputed. Recent evidence has indeed revealed that affordances activation is flexibly modulated by the task and by the physical and social context. It is therefore crucial to understand whether these results challenge previous evidence showing that motor information is activated independently from the task. The context and the task can indeed act as an early or late filter. We will review recent data consistent with the notion that objects automatically elicit multiple affordances and that top-down processes select among them probably inhibiting motor information that is not consistent with behavior goals. We will therefore argue that automaticity and flexibility of affordances are not in conflict. We will also discuss how language can incorporate affordances showing similarities, but also differences, between the motor information elicited by vision and language. Finally we will show how the distinction between stable and variable affordances can accommodate all these effects.

A saw is first identified as an object used on wood: ERP evidence for temporal differences between Thematic and Functional similarity relations

The role of functional and motor information in manipulable artifact object semantic organization is still poorly understood. In particular, several types of semantic relations involving object functional knowledge may be distinguished. Functional similarity relations group objects with similar functions at relatively specific (e.g. saw-axe, both used to cut wood) or general (saw-knife, both used to cut) levels. Thematic relations group objects based on their complementarity in events (saw used upon/with wood). Recent eye-tracking data showed distinct temporal time courses for the different semantic relations, with fastest processing of thematic relations and slowest processing of general function similarity relations. Behavioral data suggest the involvement of distinct cognitive mechanisms in manipulable artifact object semantic processing. The aim of the present study was to assess the neural correlates of thematic, and specific and general function similarity relation processing. Specifically, we investigated whether time course differences between semantic relations could be highlighted at the neurophysiological level. We used a protocol combining semantic priming with electroencephalography, and manipulated the type of semantic relation and the duration of the interval between prime and target objects. Two consistent and complementary results were shown. On N1 and P3 components, semantic priming was observed for thematic relations only. On N400 component, the type of semantic relation interacted with interval duration, and semantic priming was visible for all 3 relations after the longest interval only. Results revealed graded processing time courses for thematic, specific function similarity, and general function similarity relations at the neural level, and further indicate that thematic relations impact object processing during the early stages of object recognition. Findings suggest a hierarchical organization of three types of semantic relations based on functional knowledge. The parallel between semantic relations involving manipulable artifact objects and levels of action representations is discussed.

How do you hold your mouse? Tracking the compatibility effect between hand posture and stimulus size

In keeping with the idea that observing objects activates possible motor responses, several experiments revealed compatibility effects between the hand postures used to report a choice and some characteristics of the stimuli. The real-time dynamics of such compatibility effects are currently unknown. We tracked the time course of a categorization experiment requiring subjects to categorize as natural or artifact figures of big and small objects. Participants reported their choice using either a big mouse (requiring a power grip: a hand posture compatible with the grasping of big objects) or a small mouse (requiring a precision grip: a hand posture compatible with the grasping of small objects). We found a compatibility effect between the grip required by the mouse and the grip elicited by objects, even if it was irrelevant to the task. In a following experiment with the same paradigm, lexical stimuli failed to reproduce the same effect. Nevertheless, a compatibility effect mediated by the target-word category (artificial vs. natural) was observed. We discuss the results in the context of affordance effects literature and grounded theories of cognition.

Cortical processing of object affordances for self and others' action

The perception of objects does not rely only on visual brain areas, but also involves cortical motor regions. In particular, different parietal and premotor areas host neurons discharging during both object observation and grasping. Most of these cells often show similar visual and motor selectivity for a specific object (or set of objects), suggesting that they might play a crucial role in representing the “potential motor act” afforded by the object. The existence of such a mechanism for the visuomotor transformation of object physical properties in the most appropriate motor plan for interacting with them has been convincingly demonstrated in humans as well. Interestingly, human studies have shown that visually presented objects can automatically trigger the representation of an action provided that they are located within the observer's reaching space (peripersonal space). The “affordance effect” also occurs when the presented object is outside the observer's peripersonal space, but inside the peripersonal space of an observed agent. These findings recently received direct support by single neuron studies in monkey, indicating that space-constrained processing of objects in the ventral premotor cortex might be relevant to represent objects as potential targets for one's own or others' action.

The affordance-matching hypothesis: how objects guide action understanding and prediction

Action understanding lies at the heart of social interaction. Prior research has often conceptualized this capacity in terms of a motoric matching of observed actions to an action in one's motor repertoire, but has ignored the role of object information. In this manuscript, we set out an alternative conception of intention understanding, which places the role of objects as central to our observation and comprehension of the actions of others. We outline the current understanding of the interconnectedness of action and object knowledge, demonstrating how both rely heavily on the other. We then propose a novel framework, the affordance-matching hypothesis, which incorporates these findings into a simple model of action understanding, in which object knowledge-what an object is for and how it is used-can inform and constrain both action interpretation and prediction. We will review recent empirical evidence that supports such an object-based view of action understanding and we relate the affordance matching hypothesis to recent proposals that have re-conceptualized the role of mirror neurons in action understanding.

I give you a cup, I get a cup: a kinematic study on social intention

While affordances have been intensively studied, the mechanisms according to how their activation is modulated by context are poorly understood. We investigated how the Agent׳s reach-to-grasp movement towards a target-object (e.g. a can) is influenced by the other׳s interaction with a second object (manipulative/functional) and by his/her eye-gaze communication. To manipulate physical context we showed participants two objects that could be linked by a spatial relation (e.g. can-knife, typically found in the same context), or by different functional relations. The functional relations could imply an action to perform with another person (functional-cooperative: e.g. can-glass), or on our own (functional-individual: e.g. can-straw). When objects were not related (e.g. can-toothbrush) participants had to refrain from responding. In order to respond, in the giving condition participants had to move the target object towards the other person, in the getting condition towards their own body. When participants (Agents) performed a reach-to-grasp movement to give the target object, in presence of eye-gaze communication they reached the wrist׳s acceleration peak faster if the Other previously interacted with the second object in accordance with its conventional use. Consistently participants reached faster the MFA when the objects were related by a functional-individual than a functional-cooperative relation. The Agent׳s getting response strongly affected the grasping component of the movement: in case of eye-gaze sharing, MFA was greater when the other previously performed a manipulative than a functional grip. Results reveal that humans have developed a sophisticated capability in detecting information from hand posture and eye-gaze, which are informative as to the Agent׳s intention.