Action relations facilitate the identification of briefly-presented objects

Learned associations serve as target proxies during difficult but not easy visual search

The target template contains information in memory that is used to guide attention during visual search and is typically thought of as containing features of the actual target object. However, when targets are hard to find, it is advantageous to use other information in the visual environment that is predictive of the target's location to help guide attention. The purpose of these studies was to test if newly learned associations between face and scene category images lead observers to use scene information as a proxy for the face target. Our results showed that scene information was used as a proxy for the target to guide attention but only when the target face was difficult to discriminate from the distractor face; when the faces were easy to distinguish, attention was no longer guided by the scene unless the scene was presented earlier. The results suggest that attention is flexibly guided by both target features as well as features of objects that are predictive of the target location. The degree to which each contributes to guiding attention depends on the efficiency with which that information can be used to decode the location of the target in the current moment. The results contribute to the view that attentional guidance is highly flexible in its use of information to rapidly locate the target.

Visual Attention and Cognitive Archaeology: An Eye-Tracking Study of Palaeolithic Stone Tools

The study of lithic technology can provide information on human cultural evolution. This article aims to analyse visual behaviour associated with the exploration of ancient stone artefacts and how this relates to perceptual mechanisms in humans. In Experiment 1, we used eye tracking to record patterns of eye fixations while participants viewed images of stone tools, including examples of worked pebbles and handaxes. The results showed that the focus of gaze was directed more towards the upper regions of worked pebbles and on the basal areas for handaxes. Knapped surfaces also attracted more fixation than natural cortex for both tool types. Fixation distribution was different to that predicted by models that calculate visual salience. Experiment 2 was an online study using a mouse-click attention tracking technique and included images of unworked pebbles and 'mixed' images combining the handaxe's outline with the pebble's unworked texture. The pattern of clicks corresponded to that revealed using eye tracking and there were differences between tools and other images. Overall, the findings suggest that visual exploration is directed towards functional aspects of tools. Studies of visual attention and exploration can supply useful information to inform understanding of human cognitive evolution and tool use.

To Grasp the World at a Glance: The Role of Attention in Visual and Semantic Associative Processing

Associative relations among words, concepts and percepts are the core building blocks of high-level cognition. When viewing the world ‘at a glance’, the associative relations between objects in a scene, or between an object and its visual background, are extracted rapidly. The extent to which such relational processing requires attentional capacity, however, has been heavily disputed over the years. In the present manuscript, I review studies investigating scene–object and object–object associative processing. I then present a series of studies in which I assessed the necessity of spatial attention to various types of visual–semantic relations within a scene. Importantly, in all studies, the spatial and temporal aspects of visual attention were tightly controlled in an attempt to minimize unintentional attention shifts from ‘attended’ to ‘unattended’ regions. Pairs of stimuli—either objects, scenes or a scene and an object—were briefly presented on each trial, while participants were asked to detect a pre-defined target category (e.g., an animal, a nonsense shape). Response times (RTs) to the target detection task were registered when visual attention spanned both stimuli in a pair vs. when attention was focused on only one of two stimuli. Among non-prioritized stimuli that were not defined as to-be-detected targets, findings consistently demonstrated rapid associative processing when stimuli were fully attended, i.e., shorter RTs to associated than unassociated pairs. Focusing attention on a single stimulus only, however, largely impaired this relational processing. Notably, prioritized targets continued to affect performance even when positioned at an unattended location, and their associative relations with the attended items were well processed and analyzed. Our findings portray an important dissociation between unattended task-irrelevant and task-relevant items: while the former require spatial attentional resources in order to be linked to stimuli positioned inside the attentional focus, the latter may influence high-level recognition and associative processes via feature-based attentional mechanisms that are largely independent of spatial attention.

Task-Irrelevant Semantic Properties of Objects Impinge on Sensory Representations within the Early Visual Cortex

Objects can be described in terms of low-level (e.g., boundaries) and high-level properties (e.g., object semantics). While recent behavioral findings suggest that the influence of semantic relatedness between objects on attentional allocation can be independent of task-relevance, the underlying neural substrate of semantic influences on attention remains ill-defined. Here, we employ behavioral and functional magnetic resonance imaging measures to uncover the mechanism by which semantic information increases visual processing efficiency. We demonstrate that the strength of the semantic relatedness signal decoded from the left inferior frontal gyrus: 1) influences attention, producing behavioral semantic benefits; 2) biases spatial attention maps in the intraparietal sulcus, subsequently modulating early visual cortex activity; and 3) directly predicts the magnitude of behavioral semantic benefit. Altogether, these results identify a specific mechanism driving task-independent semantic influences on attention.

Semantic congruency effects of prime words on tool visual exploration

Most recent research on human tool use highlighted how people might integrate multiple sources of information through different neurocognitive systems to exploit the environment for action. This mechanism of integration is known as "action reappraisal". In the present eye-tracking study, we further tested the action reappraisal idea by devising a word-priming paradigm to investigate how semantically congruent (e.g., "nail") vs. semantically incongruent words (e.g., "jacket") that preceded the vision of tools (e.g., a hammer) may affect participants' visual exploration of them. We found an implicit modulation of participants' temporal allocation of visuospatial attention as a function of the object-word consistency. Indeed, participants tended to increase over time their fixations on tools' manipulation areas under semantically congruent conditions. Conversely, participants tended to concentrate their visual-spatial attention on tools' functional areas when inconsistent object-word pairs were presented. These results support and extend the information-integrated perspective of the action reappraisal approach. Also, these findings provide further evidence about how higher-level semantic information may influence tools' visual exploration.

Hazardous tools: the emergence of reasoning in human tool use

Humans are unique in the way they understand the causal relationships between the use of tools and achieving a goal. The idea at the core of the present research is that tool use can be considered as an instance of problem-solving situations supported by technical reasoning. In an eye-tracking study, we investigated the fixation patterns of participants (N = 32) looking at 3D images of thematically consistent (e.g., nail-steel hammer) and thematically inconsistent (e.g., scarf-steel hammer) object-tool pairs that could be either "hazardous" (accidentally electrified) or not. Results showed that under thematically consistent conditions, participants focused on the tool's manipulation area (e.g., the handle of a steel hammer). However, when electrified tools were present or when the visual scene was not action-prompting, regardless of the presence of electricity, the tools' functional/identity areas (e.g., the head of a steel hammer) were fixated longer than the tools' manipulation areas. These results support an integrated and reasoning-based approach to human tool use and document, for the first time, the crucial role of mechanical/semantic knowledge in tool visual exploration.

Vision at a glance: The role of attention in processing object-to-object categorical relations

When viewing a scene at a glance, the visual and categorical relations between objects in the scene are extracted rapidly. In the present study, the involvement of spatial attention in the processing of such relations was investigated. Participants performed a category detection task (e.g., "is there an animal") on briefly flashed object pairs. In one condition, visual attention spanned both stimuli, and in another, attention was focused on a single object while its counterpart object served as a task-irrelevant distractor. The results showed that when participants attended to both objects, a categorical relation effect was obtained (Exp. 1). Namely, latencies were shorter to objects from the same category than to those from different superordinate categories (e.g., clothes, vehicles), even if categories were not prioritized by the task demands. Focusing attention on only one of two stimuli, however, largely eliminated this effect (Exp. 2). Some relational processing was seen when categories were narrowed to the basic level and were highly distinct from each other (Exp. 3), implying that categorical relational processing necessitates attention, unless the unattended input is highly predictable. Critically, when a prioritized (to-be-detected) object category, positioned in a distractor's location, differed from an attended object, a robust distraction effect was consistently observed, regardless of category homogeneity and/or of response conflict factors (Exp. 4). This finding suggests that object relations that involve stimuli that are highly relevant to the task settings may survive attentional deprivation at the distractor location. The involvement of spatial attention in object-to-object categorical processing is most critical in situations that include wide categories that are irrelevant to one's current goals.

The functional arrangement of objects biases gaze direction

A growing number of studies suggest that semantic knowledge can influence the control of gaze in scenes. For example, observers are more likely to look toward objects that are semantically related to the currently fixated object. Recent evidence also suggests that an object's functional orientation can bias gaze direction. However, it is unknown whether these semantic and functional relationships can interact to determine gaze control. To address this issue, the present study assessed whether the functional arrangement of multiple objects can influence gaze control. Participants fixated a central object (e.g., a key) flanked by two peripheral objects. After a brief delay, participants were free to shift their gaze toward the peripheral object of their choice. One of the peripheral objects was semantically related to the central object (e.g., a lock), and the objects were arranged to depict a functional or non-functional interaction (e.g., a key pointing toward or away from a lock). When the orientation of the central object was manipulated, participants were more likely to look in the direction this object was pointing. Moreover, the functional arrangement of objects modulated this central orienting bias. However, when the orientation of the peripheral objects was manipulated, only the peripheral objects' semantic relationships influenced gaze control. Together, these findings suggest that functional relationships play an important role in the allocation of gaze, and can interact with semantic relationships to determine gaze control.

On the Evolution of a Radical Concept: Affordances According to Gibson and Their Subsequent Use and Development

James J. Gibson, the founder of ecological psychology, introduced a radical empiricist approach to perception and action centered on direct perception in naturalistic environments that was counter to popular representational views of his time. This direct perception approach and the associated introduction of the affordance concept have been extremely influential in several fields of study. However, since its inception, the affordance concept has evolved in a manner such that it now deviates significantly from Gibson's original intention. This review follows use of the affordance concept by four sets of influential experimental psychologists: Gibson, Donald Norman, Mike Tucker and Rob Ellis, and Daniel Bub and Michael Masson. Particular attention is paid to the manner in which they applied the concept and the contributions provided by each set of researchers. The primary goal of this review is to determine what cognitive psychologists can take away from developments within the field and what considerations should be taken into account when using the term affordance. Having a more thorough understanding of the factors that led to the concept of affordance and its recent reformulations will better equip cognitive psychologists and, by extension, human factors researchers to further advance the study of perception-action relations.

Tool and object affordances: An ecological eye-tracking study

In the present eye-tracking study, we analysed the visuo-spatial attentional patterns of participants looking at 3D images depicting single tools and object-tool pairs. The object-tool pairs could be thematically consistent, thematically inconsistent or spatially inconsistent. During the first 500 ms of visual exploration, tools were fixated longer on their functional area in all experimental conditions. However, extending the time-window of analysis to 1750 ms, the visual scene was encoded in a faster and more suited-for-action way in the thematically consistent condition (e.g., hammer-nail). Most important, the visual exploration of the thematically consistent pairs focused on the manipulation area of the tool (e.g., the handle of the hammer) more than on its functional area (e.g., the head of the hammer). Finally, when single tools were shown and the entire time-window of analysis was considered (1750 ms), fixation focused on the tool's manipulation area. These results are discussed within the reasoning-based framework of tool use. They highlight the relative role of the visuo-perceptual context in affordance perception and suggest a novel interpretation of the cognitive mechanisms underlying the processing of tools and object-tool pairs in terms of action reappraisal (i.e., a re-functionalization process when the action possibility is mined by the visuo-perceptual context).

Object Properties Influence Visual Guidance of Motor Actions

The dynamic nature of the real world poses challenges for predicting where best to allocate gaze during object interactions. The same object may require different visual guidance depending on its current or upcoming state. Here, we explore how object properties (the material and shape of objects) and object state (whether it is full of liquid, or to be set down in a crowded location) influence visual supervision while setting objects down, which is an element of object interaction that has been relatively neglected in the literature. In a liquid pouring task, we asked participants to move empty glasses to a filling station; to leave them empty, half fill, or completely fill them with water; and then move them again to a tray. During the first putdown (when the glasses were all empty), visual guidance was determined only by the type of glass being set down—with more unwieldy champagne flutes being more likely to be guided than other types of glasses. However, when the glasses were then filled, glass type no longer mattered, with the material and fill level predicting whether the glasses were set down with visual supervision: full, glass material containers were more likely to be guided than empty, plastic ones. The key finding from this research is that the visual system responds flexibly to dynamic changes in object properties, likely based on predictions of risk associated with setting-down the object unsupervised by vision. The factors that govern these mechanisms can vary within the same object as it changes state.

Object Vision in a Structured World

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world.

A network view on brain regions involved in experts' object and pattern recognition: Implications for the neural mechanisms of skilled visual perception

Skilled visual object and pattern recognition form the basis of many everyday behaviours. The game of chess has often been used as a model case for studying how long-term experience aides in perceiving objects and their spatio-functional interrelations. Earlier research revealed two brain regions, posterior middle temporal gyrus (pMTG) and collateral sulcus (CoS), to be linked to chess experts' superior object and pattern recognition, respectively. Here we elucidated the brain networks these two expertise-related regions are embedded in, employing resting-state functional connectivity analysis and meta-analytic connectivity modelling with the BrainMap database. pMTG was preferentially connected with dorsal visual stream areas and a parieto-prefrontal network for action planning, while CoS was preferentially connected with posterior medial cortex and hippocampus, linked to scene perception, perspective-taking and navigation. Functional profiling using BrainMap meta-data revealed that pMTG was linked to semantic processing as well as inhibition and attention, while CoS was linked to face and shape perception as well as passive viewing. Our findings suggest that pMTG subserves skilled object recognition by mediating the link between object identity and object affordances, while CoS subserves skilled pattern recognition by linking the position of individual objects with typical spatio-functional layouts of their environment stored in memory.

Dorsal and ventral stream contribution to the paired-object affordance effect

Visual extinction, a parietal syndrome in which patients exhibit perceptual impairments when two objects are simultaneously presented in the visual field, is reduced when objects are correctly positioned for action, indicating that action helps patients' visual attention. Similarly, healthy individuals make faster action decisions on object pairs that appear in left/right standard co-location for actions in comparison to object pairs that appear in a mirror location, a phenomenon called the paired-object affordance effect. However, the neural locus of such effect remains debated and may be related to the activity of ventral or dorsal brain regions. The present fMRI study aims at determining the neural substrates of the paired-object affordance effect. Fourteen right-handed participants made decisions about semantically related (i.e. thematically related and co-manipulated) and unrelated object pairs. Pairs were either positioned in a standard location for a right-handed action (with the active object - lid - in the right visual hemifield, and the passive object - pan - in the left visual hemifield), or in the reverse location. Behavioral results showed a suppression of the observed cost of correctly positioning related pairs for action when performing action decisions (deciding if the two objects are usually used together), but not when performing contextual decisions (deciding if the two objects are typically found in the kitchen). Anterior regions of the dorsal stream (e.g. supplementary motor area) responded to inadequate object co-positioning for action, but only when the perceptual task required action decisions. In the ventral cortex, the left lateral occipital complex showed increased activation for objects correctly positioned for action in all conditions except when neither task demands nor object relatedness was relevant for action. Thus, fMRI results demonstrated a joint contribution of ventral and dorsal cortical streams to the paired-affordance effect. They further suggest that this contribution may depend on contextual situations and task demands, in line with flexible views of affordance evocation.

Sensory and semantic activations evoked by action attributes of manipulable objects: Evidence from ERPs

"Two route" theories of object-related action processing posit different temporal activation profiles of grasp-to-move actions (rapidly evoked based on object structure) versus skilled use actions (more slowly activated based on semantic knowledge). We capitalized on the exquisite temporal resolution and multidimensionality of Event-Related Potentials (ERPs) to directly test this hypothesis. Participants viewed manipulable objects (e.g., calculator) preceded by objects sharing either "grasp", "use", or no action attributes (e.g., bar of soap, keyboard, earring, respectively), as well as by action-unrelated but taxonomically-related objects (e.g., abacus); participants judged whether the two objects were related. The results showed more positive responses to "grasp-to-move" primed objects than "skilled use" primed objects or unprimed objects starting in the P1 (0-150 ms) time window and continuing onto the subsequent N1 and P2 components (150-300 ms), suggesting that only "grasp-to-move", but not "skilled use", actions may facilitate visual attention to object attributes. Furthermore, reliably reduced N400s (300-500 ms), an index of semantic processing, were observed to taxonomically primed and "skilled use" primed objects relative to unprimed objects, suggesting that "skilled use" action attributes are a component of distributed, multimodal semantic representations of objects. Together, our findings provide evidence supporting two-route theories by demonstrating that "grasp-to-move" and "skilled use" actions impact different aspects of object processing and highlight the relationship of "skilled use" information to other aspects of semantic memory.

Human-Object Interactions Are More than the Sum of Their Parts

Understanding human-object interactions is critical for extracting meaning from everyday visual scenes and requires integrating complex relationships between human pose and object identity into a new percept. To understand how the brain builds these representations, we conducted 2 fMRI experiments in which subjects viewed humans interacting with objects, noninteracting human-object pairs, and isolated humans and objects. A number of visual regions process features of human-object interactions, including object identity information in the lateral occipital complex (LOC) and parahippocampal place area (PPA), and human pose information in the extrastriate body area (EBA) and posterior superior temporal sulcus (pSTS). Representations of human-object interactions in some regions, such as the posterior PPA (retinotopic maps PHC1 and PHC2) are well predicted by a simple linear combination of the response to object and pose information. Other regions, however, especially pSTS, exhibit representations for human-object interaction categories that are not predicted by their individual components, indicating that they encode human-object interactions as more than the sum of their parts. These results reveal the distributed networks underlying the emergent representation of human-object interactions necessary for social perception.

Action properties of object images facilitate visual search

There is mounting evidence that constraints from action can influence the early stages of object selection, even in the absence of any explicit preparation for action. Here, we examined whether action properties of images can influence visual search, and whether such effects were modulated by hand preference. Observers searched for an oddball target among 3 distractors. The search arrays consisted either of images of graspable "handles" ("action-related" stimuli), or images that were otherwise identical to the handles but in which the semicircular fulcrum element was reoriented so that the stimuli no longer looked like graspable objects ("non-action-related" stimuli). In Experiment 1, right-handed observers, who have been shown previously to prefer to use the right hand over the left for manual tasks, were faster to detect targets in action-related versus non-action-related arrays, and showed a response time (reaction time [RT]) advantage for rightward- versus leftward-oriented action-related handles. In Experiment 2, left-handed observers, who have been shown to use the left and right hands relatively equally in manual tasks, were also faster to detect targets in the action-related versus non-action-related arrays, but RTs were equally fast for rightward- and leftward-oriented handle targets. Together, or results suggest that action properties in images, and constraints for action imposed by preferences for manual interaction with objects, can influence attentional selection in the context of visual search. (PsycINFO Database Record

The involvement of the dorsal stream in processing implied actions between paired objects: A TMS study

Perceiving and selecting the action possibilities (affordances) provided by objects is an important challenge to human vision, and is not limited to single-object scenarios. Xu et al. (2015) identified two effects of implied actions between paired objects on response selection: an inhibitory effect on responses aligned with the passive object in the pair (e.g. a bowl) and an advantage associated with responses aligned with the active objects (e.g. a spoon). The present study investigated the neurocognitive mechanisms behind these effects by examining the involvement of the ventral (vision for perception) and the dorsal (vision for action) visual streams, as defined in Goodale and Milner's (1992) two visual stream theory. Online repetitive transcranial magnetic stimulation (rTMS) applied to the left anterior intraparietal sulcus (aIPS) reduced both the inhibitory effect of implied actions on responses aligned with the passive objects and the advantage of those aligned with the active objects, but only when the active objects were contralateral to the stimulation. rTMS to the left lateral occipital areas (LO) did not significantly alter the influence of implied actions. The results reveal that the dorsal visual stream is crucial not only in single-object affordance processing, but also in responding to implied actions between objects.

Early Visual Perception Potentiated by Object Affordances: Evidence From a Temporal Order Judgment Task

Perceived objects automatically potentiate afforded action. Object affordances also facilitate perception of such objects, and this occurrence is known as the affordance effect. This study examined whether object affordances facilitate the initial visual processing stage, or perceptual entry processes, using the temporal order judgment task. The onset of the graspable (right-handled) coffee cup was perceived earlier than that of the less graspable (left-handled) cup for right-handed participants. The affordance effect was eliminated when the coffee cups were inverted, which presumably conveyed less affordance information. These results suggest that objects preattentively potentiate the perceptual entry processes in response to their affordances.

Feature Confirmation in Object Perception: Feature Integration Theory 26 Years on from the Treisman Bartlett Lecture

The Treisman Bartlett lecture, reported in the Quarterly Journal of Experimental Psychology in 1988, provided a major overview of the feature integration theory of attention. This has continued to be a dominant account of human visual attention to this day. The current paper provides a summary of the work reported in the lecture and an update on critical aspects of the theory as applied to visual object perception. The paper highlights the emergence of findings that pose significant challenges to the theory and which suggest that revisions are required that allow for (a) several rather than a single form of feature integration, (b) some forms of feature integration to operate preattentively, (c) stored knowledge about single objects and interactions between objects to modulate perceptual integration, (d) the application of feature-based inhibition to object files where visual features are specified, which generates feature-based spreading suppression and scene segmentation, and (e) a role for attention in feature confirmation rather than feature integration in visual selection. A feature confirmation account of attention in object perception is outlined.

The role of motor affordances in immediate and long-term retention of objects

In line with the embodied cognition perspective stating that cognitive processing results from the activation of the sensorimotor systems involved in perception and action (e.g., Glenberg, 1997), recent studies provided evidence that motor affordances played a role in serial memory for objects (e.g., see Downing-Doucet & Guérard, 2014). In the present study, we extended this line of research by investigating whether objects' motor affordances played a role in item memory, in immediate and long-term retention. Participants had to retain pairs of objects that were positioned in a way that was congruent for action or not. The results showed that motor suppression disrupted the retention of congruent pairs, but not that of incongruent pairs when short lists of six objects had to be retained over a short period of time (Experiment 1). However, when participants had to retain lists of 60 pairs, motor suppression had no effect on retention (Experiment 2). These results suggest that the motor system was recruited for the immediate retention of objects, but not for their long-term retention.

Effects of broken affordance on visual extinction

Previous studies have shown that visual extinction can be reduced if two objects are positioned to “afford” an action. Here we tested if this affordance effect was disrupted by “breaking” the affordance, i.e., if one of the objects actively used in the action had a broken handle. We assessed the effects of broken affordance on recovery from extinction in eight patients with right hemisphere lesions and left-sided extinction. Patients viewed object pairs that were or were not commonly used together and that were positioned for left- or right-hand actions. In the unrelated pair conditions, either two tools or two objects were presented. In line with previous research (e.g., Riddoch et al., 2006), extinction was reduced when action-related object pairs and when unrelated tool pairs were presented compared to unrelated object pairs. There was no significant difference in recovery rate between action-related (object-tool) and unrelated tool pairs. In addition, performance with action-related objects decreased when the tool appeared on the ipsilesional side compared to when it was on the contralesional side, but only when the tool handle was intact. There were minimal effects of breaking the handle of an object rather than a tool, and there was no effect of breaking the handle on either tools or objects on single item trials. The data suggest that breaking the handle of a tool lessens the degree to which it captures attention, with this attentional capture being strongest when the tool appears on the ipsilesional side. The capture of attention by the ipsilesional item then reduces the chance of detecting the contralesional stimulus. This attentional capture effect is mediated by the affordance to the intact tool.

Attention is required for the perceptual integration of action object pairs

Previous studies have demonstrated that functionally related objects are perceptually grouped during visual identification if they are depicted as if interacting with each other (Green and Hummel in J Exp Psychol Hum Percept Perform 32(5):1107-1119, 2006). However, it is unclear whether this integration requires attention or occurs pre-attentively. Here, we used a divided-attention task with variable attentional load to address this question. Participants matched a word label to a target object that was immediately preceded by a briefly presented, task-irrelevant tool that was either functionally related or unrelated to the word label (e.g., axe | "log" or hammer | "log"). The tool was either positioned to interact with the target object or faced away from it. The amount of attention available to process the tool was manipulated by asking participants to make a concurrent perceptual discrimination of varying difficulty on a surrounding frame stimulus. The previously demonstrated advantage for the related-and-interacting condition was replicated under conditions of no or low attentional load. This benefit disappeared under high competing attentional load, indicating that attention is required to integrate functionally related objects together into a single perceptual unit.

Mechanisms underlying selecting objects for action

We assessed the factors which affect the selection of objects for action, focusing on the role of action knowledge and its modulation by distracters. Fourteen neuropsychological patients and 10 healthy aged-matched controls selected pairs of objects commonly used together among distracters in two contexts: with real objects and with pictures of the same objects presented sequentially on a computer screen. Across both tasks, semantically related distracters led to slower responses and more errors than unrelated distracters and the object actively used for action was selected prior to the object that would be passively held during the action. We identified a sub-group of patients (N = 6) whose accuracy was 2SDs below the controls performances in the real object task. Interestingly, these impaired patients were more affected by the presence of unrelated distracters during both tasks than intact patients and healthy controls. Note that the impaired patients had lesions to left parietal, right anterior temporal and bilateral pre-motor regions. We conclude that: (1) motor procedures guide object selection for action, (2) semantic knowledge affects action-based selection, (3) impaired action decision making is associated with the inability to ignore distracting information and (4) lesions to either the dorsal or ventral visual stream can lead to deficits in making action decisions. Overall, the data indicate that impairments in everyday tasks can be evaluated using a simulated computer task. The implications for rehabilitation are discussed.

Selecting object pairs for action: Is the active object always first?

Perception is linked to action via two routes: a direct route based on affordance information in the environment and an indirect route based on semantic knowledge about objects. The present study explored the factors modulating the recruitment of the two routes, in particular which factors affecting the selection of paired objects. In Experiment 1, we presented real objects among semantically related or unrelated distracters. Participants had to select two objects that can interact. The presence of distracters affected selection times, but not the semantic relations of the objects with the distracters. Furthermore, participants first selected the active object (e.g. teaspoon) with their right hand, followed by the passive object (e.g. mug), often with their left hand. In Experiment 2, we presented pictures of the same objects with no hand grip, congruent or incongruent hand grip. Participants had to decide whether the two objects can interact. Action decisions were faster when the presentation of the active object preceded the presentation of the passive object, and when the grip was congruent. Interestingly, participants were slower when the objects were semantically but not functionally related; this effect increased with congruently gripped objects. Our data showed that action decisions in the presence of strong affordance cues (real objects, pictures of congruently gripped objects) relied on sensory-motor representation, supporting the direct route from perception-to-action that bypasses semantic knowledge. However, in the case of weak affordance cues (pictures), semantic information interfered with action decisions, indicating that semantic knowledge impacts action decisions. The data support the dual-route account from perception-to-action.

When motor congruency modulates immediate memory for objects

In line with the embodied cognition view, some researchers have suggested that our capacity to retain information relies on the perceptual and motor systems used to interact with our environment (Barsalou, 1999; Glenberg, 1997). For instance, the language production architecture would be responsible for the retention of verbal materials such as a list of words (Acheson & MacDonald, 2009). However, evidence for the role of the motor system in object memory is still limited. In the present experiments, participants were asked to retain lists of objects in memory. During encoding, participants had to pantomime an action to grasp (Experiments 1A & 1B) or to use the objects (Experiment 2) that was either congruent or incongruent with the objects to be retained. The results showed that performing an incongruent action impaired memory performance compared to a congruent action. This suggests that motor affordances play a role during object retention. The results are discussed in light of the embodied cognition view.

The neural basis of perceiving person interactions

This study examined whether the grouping of people into meaningful social scenes (e.g., two people having a chat) impacts the basic perceptual analysis of each partaking individual. To explore this issue, we measured neural activity using functional magnetic resonance imaging (fMRI) while participants sex-categorized congruent as well as incongruent person dyads (i.e., two people interacting in a plausible or implausible manner). Incongruent person dyads elicited enhanced neural processing in several high-level visual areas dedicated to face and body encoding and in the posterior middle temporal gyrus compared to congruent person dyads. Incongruent and congruent person scenes were also successfully differentiated by a linear multivariate pattern classifier in the right fusiform body area and the left extrastriate body area. Finally, increases in the person scenes' meaningfulness as judged by independent observers was accompanied by enhanced activity in the bilateral posterior insula. These findings demonstrate that the processing of person scenes goes beyond a mere stimulus-bound encoding of their partaking agents, suggesting that changes in relations between agents affect their representation in category-selective regions of the visual cortex and beyond.

Action representations activated by task-irrelevant information: is it really irrelevant?

Accessing action knowledge is believed to rely on the activation of action representations through the retrieval of functional, manipulative, and spatial information associated with objects. However, it remains unclear whether action representations can be activated in this way when the object information is irrelevant to the current judgment. The present study investigated this question by independently manipulating the correctness of three types of action-related information: the functional relation between the two objects, the grip applied to the objects, and the orientation of the objects. In each of three tasks in Experiment 1, participants evaluated the correctness of only one of the three information types (function, grip or orientation). Similar results were achieved with all three tasks: "correct" judgments were facilitated when the other dimensions were correct; however, "incorrect" judgments were facilitated when the other two dimensions were both correct and also when they were both incorrect. In Experiment 2, when participants attended to an action-irrelevant feature (object color), there was no interaction between function, grip, and orientation. These results clearly indicate that action representations can be activated by retrieval of functional, manipulative, and spatial knowledge about objects, even though this is task-irrelevant information.

Associative recognition processes are modulated by the semantic unitizability of memoranda

Although memory of episodic associations is generally considered to be recollective in nature, it has been suggested that when stimuli are experienced as a unit, familiarity processes might contribute to their subsequent associative recognition. To investigate the effect of semantic relatedness during episodic encoding on the processes of retrieval of associative information, we had participants interactively encode pairs of object pictures, vertically arranged so as to suggest a functional or configural relationship between them. Half the pairs were independently judged to be of related objects (e.g., a lamp over a table) and half of unrelated objects (e.g., a key-ring over an apple). At test, participants discriminated between intact, recombined, and new pairs while event related potentials (ERPs) were recorded. In an early ERP marker of retrieval success generally associated with familiarity processes, differences related to associative memory only emerged for related pairs, while differences associated with item memory emerged for both related and unrelated pairs. In contrast, in a later ERP effect associated with recollection, differences related to associative memory emerged for both related and unrelated pairs. These findings may indicate that retrieval of episodic associations formed between two semantically related visual stimuli can be supported by familiarity-related processes.

Abnormal dynamics of activation of object use information in apraxia: evidence from eyetracking

Action representations associated with object use may be incidentally activated during visual object processing, and the time course of such activations may be influenced by lexical-semantic context (e.g., Lee, Middleton, Mirman, Kalénine, & Buxbaum (2012). Journal of Experimental Psychology: Human Perception and Performance, 39(1), 257-270). In this study we used the "visual world" eye-tracking paradigm to examine whether a deficit in producing skilled object-use actions (apraxia) is associated with abnormalities in incidental activation of action information, and assessed the neuroanatomical substrates of any such deficits. Twenty left hemisphere stroke patients, ten of whom were apraxic, performed a task requiring identification of a named object in a visual display containing manipulation-related and unrelated distractor objects. Manipulation relationships among objects were not relevant to the identification task. Objects were cued with neutral ("S/he saw the…."), or action-relevant ("S/he used the….") sentences. Non-apraxic participants looked at use-related non-target objects significantly more than at unrelated non-target objects when cued both by neutral and action-relevant sentences, indicating that action information is incidentally activated. In contrast, apraxic participants showed delayed activation of manipulation-based action information during object identification when cued by neutral sentences. The magnitude of delayed activation in the neutral sentence condition was reliably predicted by lower scores on a test of gesture production to viewed objects, as well as by lesion loci in the inferior parietal and posterior temporal lobes. However, when cued by a sentence containing an action verb, apraxic participants showed fixation patterns that were statistically indistinguishable from non-apraxic controls. In support of grounded theories of cognition, these results suggest that apraxia and temporal-parietal lesions may be associated with abnormalities in incidental activation of action information from objects. Further, they suggest that the previously-observed facilitative role of action verbs in the retrieval of object-related action information extends to participants with apraxia.