Visual responses to action between unfamiliar object pairs modulateextinction

Left hemispatial neglect and overt orienting in naturalistic conditions: Role of high-level and stimulus-driven signals

Deficits of visuospatial orienting in brain-damaged patients affected by hemispatial neglect have been extensively investigated. Nonetheless, spontaneous spatial orienting in naturalistic conditions is still poorly understood. Here, we investigated the role played by top-down and stimulus-driven signals in overt spatial orienting of neglect patients during free-viewing of short videos portraying everyday life situations. In Experiment 1, we assessed orienting when meaningful visual events competed on the left and right side of space, and tested whether sensory salience on the two sides biased orienting. In Experiment 2, we examined whether the spatial alignment of visual and auditory signals modulates orienting. The results of Experiment 1 showed that in neglect patients severe deficits in contralesional orienting were restricted to viewing conditions with bilateral visual events competing for attentional capture. In contrast, orienting towards the contralesional side was largely spared when the videos contained a single event on the left side. In neglect patients the processing of stimulus-driven salience was relatively spared and helped orienting towards the left side when multiple events were present. Experiment 2 showed that sounds spatially aligned with visual events on the left side improved orienting towards the otherwise neglected hemispace. Anatomical scans indicated that neglect patients suffered grey and white matter damages primarily in the ventral frontoparietal cortex. This suggests that the improvement of contralesional orienting associated with visual salience and audiovisual spatial alignment may be due to processing in the relatively intact dorsal frontoparietal areas. Our data show that in naturalistic environments, the presence of multiple meaningful events is a major determinant of spatial orienting deficits in neglect patients, whereas the salience of visual signals and the spatial alignment between auditory and visual signals can counteract spatial orienting deficits. These results open new perspectives to develop novel rehabilitation strategies based on the use of naturalistic stimuli.

Feature Confirmation in Object Perception: Feature Integration Theory 26 Years on from the Treisman Bartlett Lecture

The Treisman Bartlett lecture, reported in the Quarterly Journal of Experimental Psychology in 1988, provided a major overview of the feature integration theory of attention. This has continued to be a dominant account of human visual attention to this day. The current paper provides a summary of the work reported in the lecture and an update on critical aspects of the theory as applied to visual object perception. The paper highlights the emergence of findings that pose significant challenges to the theory and which suggest that revisions are required that allow for (a) several rather than a single form of feature integration, (b) some forms of feature integration to operate preattentively, (c) stored knowledge about single objects and interactions between objects to modulate perceptual integration, (d) the application of feature-based inhibition to object files where visual features are specified, which generates feature-based spreading suppression and scene segmentation, and (e) a role for attention in feature confirmation rather than feature integration in visual selection. A feature confirmation account of attention in object perception is outlined.

Effects of broken affordance on visual extinction

Previous studies have shown that visual extinction can be reduced if two objects are positioned to “afford” an action. Here we tested if this affordance effect was disrupted by “breaking” the affordance, i.e., if one of the objects actively used in the action had a broken handle. We assessed the effects of broken affordance on recovery from extinction in eight patients with right hemisphere lesions and left-sided extinction. Patients viewed object pairs that were or were not commonly used together and that were positioned for left- or right-hand actions. In the unrelated pair conditions, either two tools or two objects were presented. In line with previous research (e.g., Riddoch et al., 2006), extinction was reduced when action-related object pairs and when unrelated tool pairs were presented compared to unrelated object pairs. There was no significant difference in recovery rate between action-related (object-tool) and unrelated tool pairs. In addition, performance with action-related objects decreased when the tool appeared on the ipsilesional side compared to when it was on the contralesional side, but only when the tool handle was intact. There were minimal effects of breaking the handle of an object rather than a tool, and there was no effect of breaking the handle on either tools or objects on single item trials. The data suggest that breaking the handle of a tool lessens the degree to which it captures attention, with this attentional capture being strongest when the tool appears on the ipsilesional side. The capture of attention by the ipsilesional item then reduces the chance of detecting the contralesional stimulus. This attentional capture effect is mediated by the affordance to the intact tool.

Mechanisms underlying selecting objects for action

We assessed the factors which affect the selection of objects for action, focusing on the role of action knowledge and its modulation by distracters. Fourteen neuropsychological patients and 10 healthy aged-matched controls selected pairs of objects commonly used together among distracters in two contexts: with real objects and with pictures of the same objects presented sequentially on a computer screen. Across both tasks, semantically related distracters led to slower responses and more errors than unrelated distracters and the object actively used for action was selected prior to the object that would be passively held during the action. We identified a sub-group of patients (N = 6) whose accuracy was 2SDs below the controls performances in the real object task. Interestingly, these impaired patients were more affected by the presence of unrelated distracters during both tasks than intact patients and healthy controls. Note that the impaired patients had lesions to left parietal, right anterior temporal and bilateral pre-motor regions. We conclude that: (1) motor procedures guide object selection for action, (2) semantic knowledge affects action-based selection, (3) impaired action decision making is associated with the inability to ignore distracting information and (4) lesions to either the dorsal or ventral visual stream can lead to deficits in making action decisions. Overall, the data indicate that impairments in everyday tasks can be evaluated using a simulated computer task. The implications for rehabilitation are discussed.

Selecting object pairs for action: Is the active object always first?

Perception is linked to action via two routes: a direct route based on affordance information in the environment and an indirect route based on semantic knowledge about objects. The present study explored the factors modulating the recruitment of the two routes, in particular which factors affecting the selection of paired objects. In Experiment 1, we presented real objects among semantically related or unrelated distracters. Participants had to select two objects that can interact. The presence of distracters affected selection times, but not the semantic relations of the objects with the distracters. Furthermore, participants first selected the active object (e.g. teaspoon) with their right hand, followed by the passive object (e.g. mug), often with their left hand. In Experiment 2, we presented pictures of the same objects with no hand grip, congruent or incongruent hand grip. Participants had to decide whether the two objects can interact. Action decisions were faster when the presentation of the active object preceded the presentation of the passive object, and when the grip was congruent. Interestingly, participants were slower when the objects were semantically but not functionally related; this effect increased with congruently gripped objects. Our data showed that action decisions in the presence of strong affordance cues (real objects, pictures of congruently gripped objects) relied on sensory-motor representation, supporting the direct route from perception-to-action that bypasses semantic knowledge. However, in the case of weak affordance cues (pictures), semantic information interfered with action decisions, indicating that semantic knowledge impacts action decisions. The data support the dual-route account from perception-to-action.

The neural basis of perceiving person interactions

This study examined whether the grouping of people into meaningful social scenes (e.g., two people having a chat) impacts the basic perceptual analysis of each partaking individual. To explore this issue, we measured neural activity using functional magnetic resonance imaging (fMRI) while participants sex-categorized congruent as well as incongruent person dyads (i.e., two people interacting in a plausible or implausible manner). Incongruent person dyads elicited enhanced neural processing in several high-level visual areas dedicated to face and body encoding and in the posterior middle temporal gyrus compared to congruent person dyads. Incongruent and congruent person scenes were also successfully differentiated by a linear multivariate pattern classifier in the right fusiform body area and the left extrastriate body area. Finally, increases in the person scenes' meaningfulness as judged by independent observers was accompanied by enhanced activity in the bilateral posterior insula. These findings demonstrate that the processing of person scenes goes beyond a mere stimulus-bound encoding of their partaking agents, suggesting that changes in relations between agents affect their representation in category-selective regions of the visual cortex and beyond.

Attending to the possibilities of action

Actions taking place in the environment are critical for our survival. We review evidence on attention to action, drawing on sets of converging evidence from neuropsychological patients through to studies of the time course and neural locus of action-based cueing of attention in normal observers. We show that the presence of action relations between stimuli helps reduce visual extinction in patients with limited attention to the contralesional side of space, while the first saccades made by normal observers and early perceptual and attentional responses measured using electroencephalography/event-related potentials are modulated by preparation of action and by seeing objects being grasped correctly or incorrectly for action. With both normal observers and patients, there is evidence for two components to these effects based on both visual perceptual and motor-based responses. While the perceptual responses reflect factors such as the visual familiarity of the action-related information, the motor response component is determined by factors such as the alignment of the objects with the observer's effectors and not by the visual familiarity of the stimuli. In addition to this, we suggest that action relations between stimuli can be coded pre-attentively, in the absence of attention to the stimulus, and action relations cue perceptual and motor responses rapidly and automatically. At present, formal theories of visual attention are not set up to account for these action-related effects; we suggest ways that theories could be expected to enable action effects to be incorporated.