Attention, spatial representation, and visual neglect: simulating emergent attention and spatial memory in the selective attention for identification model (SAIM)

Continuous action with a neurobiologically inspired computational approach reveals the dynamics of selection history

Everyday perception-action interaction often requires selection of a single goal from multiple possibilities. According to a recent framework of attentional control, object selection is guided not only by the well-established factors of perceptual salience and current goals but also by selection history. Yet, underlying mechanisms linking selection history and visually-guided actions are poorly understood. To examine such interplay and disentangle the impact of target and distractor history on action selection, we employed a priming-of-popout (PoP) paradigm combined with continuous tracking of reaching movements and computational modeling. Participants reached an odd-colored target among homogeneous distractors while we systematically manipulated the sequence of target and distractor colors from one trial to the next. We observed that current reach movements were significantly influenced by the interaction between attraction by the prior target feature and repulsion by the prior distractor feature. With principal component regression, we found that inhibition led by prior distractors influenced reach target selection earlier than facilitation led by the prior target. In parallel, our newly developed computational model validated that current reach target selection can be explained best by the mechanism postulating the preceded impact of previous distractors followed by a previous target. Such converging empirical and computational evidence suggests that the prior selection history triggers a dynamic interplay between target facilitation and distractor inhibition to guide goal-directed action successfully. This, in turn, highlights the necessity of an explicitly integrated approach to determine how visual attentional selection links with adaptive actions in a complex environment.

A failure to learn object shape geometry: Implications for convolutional neural networks as plausible models of biological vision

Here we examine the plausibility of deep convolutional neural networks (CNNs) as a theoretical framework for understanding biological vision in the context of image classification. Recent work on object recognition in human vision has shown that both global, and local, shape information is computed, and integrated, early during perceptual processing. Our goal was to compare the similarity in how object shape information is processed by CNNs and human observers. We tested the hypothesis that, unlike the human system, CNNs do not compute representations of global and local object geometry during image classification. To do so, we trained and tested six CNNs (AlexNet, VGG-11, VGG-16, ResNet-18, ResNet-50, GoogLeNet), and human observers, to discriminate geometrically possible and impossible objects. The ability to complete this task requires computation of a representational structure of shape that encodes both global and local object geometry because the detection of impossibility derives from an incongruity between well-formed local feature conjunctions and their integration into a geometrically well-formed 3D global shape. Unlike human observers, none of the tested CNNs could reliably discriminate between possible and impossible objects. Detailed analyses using gradient-weighted class activation mapping (GradCam) of CNN image feature processing showed that network classification performance was not constrained by object geometry. In contrast, if classification could be made based solely on local feature information in line drawings the CNNs were highly accurate. We argue that these findings reflect fundamental differences between CNNs and human vision in terms of underlying image processing structure. Notably, unlike human vision, CNNs do not compute representations of object geometry. The results challenge the plausibility of CNNs as a framework for understanding image classification in biological vision systems.

Studying the dynamics of visual search behavior using RT hazard and micro-level speed-accuracy tradeoff functions: A role for recurrent object recognition and cognitive control processes

Thanks to the work of Anne Treisman and many others, the visual search paradigm has become one of the most popular paradigms in the study of visual attention. However, statistics like mean correct response time (RT) and percent error do not usually suffice to decide between the different search models that have been developed. Recently, to move beyond mean performance measures in visual search, RT histograms have been plotted, theoretical waiting time distributions have been fitted, and whole RT and error distributions have been simulated. Here we promote and illustrate the general application of discrete-time hazard analysis to response times, and of micro-level speed-accuracy tradeoff analysis to timed response accuracies. An exploratory analysis of published benchmark search data from feature, conjunction, and spatial configuration search tasks reveals new features of visual search behavior, such as a relatively flat hazard function in the right tail of the RT distributions for all tasks, a clear effect of set size on the shape of the RT distribution for the feature search task, and individual differences in the presence of a systematic pattern of early errors. Our findings suggest that the temporal dynamics of visual search behavior results from a decision process that is temporally modulated by concurrently active recurrent object recognition, learning, and cognitive control processes, next to attentional selection processes.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Attention in Psychology, Neuroscience, and Machine Learning

Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine learning. The relationship between the study of biological attention and its use as a tool to enhance artificial neural networks is not always clear. This review starts by providing an overview of how attention is conceptualized in the neuroscience and psychology literature. It then covers several use cases of attention in machine learning, indicating their biological counterparts where they exist. Finally, the ways in which artificial attention can be further inspired by biology for the production of complex and integrative systems is explored.

Scene memory and spatial inhibition in visual search : A neural dynamic process model and new experimental evidence

Any object-oriented action requires that the object be first brought into the attentional foreground, often through visual search. Outside the laboratory, this would always take place in the presence of a scene representation acquired from ongoing visual exploration. The interaction of scene memory with visual search is still not completely understood. Feature integration theory (FIT) has shaped both research on visual search, emphasizing the scaling of search times with set size when searches entail feature conjunctions, and research on visual working memory through the change detection paradigm. Despite its neural motivation, there is no consistently neural process account of FIT in both its dimensions. We propose such an account that integrates (1) visual exploration and the building of scene memory, (2) the attentional detection of visual transients and the extraction of search cues, and (3) visual search itself. The model uses dynamic field theory in which networks of neural dynamic populations supporting stable activation states are coupled to generate sequences of processing steps. The neural architecture accounts for basic findings in visual search and proposes a concrete mechanism for the integration of working memory into the search process. In a behavioral experiment, we address the long-standing question of whether both the overall speed and the efficiency of visual search can be improved by scene memory. We find both effects and provide model fits of the behavioral results. In a second experiment, we show that the increase in efficiency is fragile, and trace that fragility to the resetting of spatial working memory.

Distinguishing between parallel and serial processing in visual attention from neurobiological data

Serial and parallel processing in visual search have been long debated in psychology, but the processing mechanism remains an open issue. Serial processing allows only one object at a time to be processed, whereas parallel processing assumes that various objects are processed simultaneously. Here, we present novel neural models for the two types of processing mechanisms based on analysis of simultaneously recorded spike trains using electrophysiological data from prefrontal cortex of rhesus monkeys while processing task-relevant visual displays. We combine mathematical models describing neuronal attention and point process models for spike trains. The same model can explain both serial and parallel processing by adopting different parameter regimes. We present statistical methods to distinguish between serial and parallel processing based on both maximum likelihood estimates and decoding the momentary focus of attention when two stimuli are presented simultaneously. Results show that both processing mechanisms are in play for the simultaneously recorded neurons, but neurons tend to follow parallel processing in the beginning after the onset of the stimulus pair, whereas they tend to serial processing later on.

The Anatomy of Inference: Generative Models and Brain Structure

To infer the causes of its sensations, the brain must call on a generative (predictive) model. This necessitates passing local messages between populations of neurons to update beliefs about hidden variables in the world beyond its sensory samples. It also entails inferences about how we will act. Active inference is a principled framework that frames perception and action as approximate Bayesian inference. This has been successful in accounting for a wide range of physiological and behavioral phenomena. Recently, a process theory has emerged that attempts to relate inferences to their neurobiological substrates. In this paper, we review and develop the anatomical aspects of this process theory. We argue that the form of the generative models required for inference constrains the way in which brain regions connect to one another. Specifically, neuronal populations representing beliefs about a variable must receive input from populations representing the Markov blanket of that variable. We illustrate this idea in four different domains: perception, planning, attention, and movement. In doing so, we attempt to show how appealing to generative models enables us to account for anatomical brain architectures. Ultimately, committing to an anatomical theory of inference ensures we can form empirical hypotheses that can be tested using neuroimaging, neuropsychological, and electrophysiological experiments.

The Computational Anatomy of Visual Neglect

Visual neglect is a debilitating neuropsychological phenomenon that has many clinical implications and-in cognitive neuroscience-offers an important lesion deficit model. In this article, we describe a computational model of visual neglect based upon active inference. Our objective is to establish a computational and neurophysiological process theory that can be used to disambiguate among the various causes of this important syndrome; namely, a computational neuropsychology of visual neglect. We introduce a Bayes optimal model based upon Markov decision processes that reproduces the visual searches induced by the line cancellation task (used to characterize visual neglect at the bedside). We then consider 3 distinct ways in which the model could be lesioned to reproduce neuropsychological (visual search) deficits. Crucially, these 3 levels of pathology map nicely onto the neuroanatomy of saccadic eye movements and the systems implicated in visual neglect.

Multiple Diffusion Models to Compare Saccadic and Manual Responses for Inhibition of Return

Cuing a location in space produces a short-lived advantage in reaction time to targets at that location. This early advantage, however, switches to a reaction time cost and has been termed inhibition of return (IOR). IOR behaves differently for different response modalities, suggesting that it may not be a unified effect. This letter presents new data from two experiments testing the gradient of IOR with random, continuous cue-target Euclidean distance and cue-target onset asynchrony. These data were then used to train multiple diffusion models of saccadic and manual reaction time for these cuing experiments. Diffusion models can generate accurate distributions of reaction time data by modeling a response as a buildup of evidence toward a response threshold. If saccadic and attentional IOR are based on similar processes, then differences in distribution will be best explained by adjusting parameter values such as signal and noise within the same model structure. Although experimental data show differences in the timing of IOR across modality, best-fit models are shown to have similar model parameters for the gradient of IOR, suggesting similar underlying mechanisms for saccadic and manual IOR.

Choice reaching with a LEGO arm robot (CoRLEGO): The motor system guides visual attention to movement-relevant information

We present an extension of a neurobiologically inspired robotics model, termed CoRLEGO (Choice reaching with a LEGO arm robot). CoRLEGO models experimental evidence from choice reaching tasks (CRT). In a CRT participants are asked to rapidly reach and touch an item presented on the screen. These experiments show that non-target items can divert the reaching movement away from the ideal trajectory to the target item. This is seen as evidence attentional selection of reaching targets can leak into the motor system. Using competitive target selection and topological representations of motor parameters (dynamic neural fields) CoRLEGO is able to mimic this leakage effect. Furthermore if the reaching target is determined by its colour oddity (i.e. a green square among red squares or vice versa), the reaching trajectories become straighter with repetitions of the target colour (colour streaks). This colour priming effect can also be modelled with CoRLEGO. The paper also presents an extension of CoRLEGO. This extension mimics findings that transcranial direct current stimulation (tDCS) over the motor cortex modulates the colour priming effect (Woodgate et al., 2015). The results with the new CoRLEGO suggest that feedback connections from the motor system to the brain’s attentional system (parietal cortex) guide visual attention to extract movement-relevant information (i.e. colour) from visual stimuli. This paper adds to growing evidence that there is a close interaction between the motor system and the attention system. This evidence contradicts the traditional conceptualization of the motor system as the endpoint of a serial chain of processing stages. At the end of the paper we discuss CoRLEGO’s predictions and also lessons for neurobiologically inspired robotics emerging from this work.

A unifying mechanistic model of selective attention in spiking neurons

Visuospatial attention produces myriad effects on the activity and selectivity of cortical neurons. Spiking neuron models capable of reproducing a wide variety of these effects remain elusive. We present a model called the Attentional Routing Circuit (ARC) that provides a mechanistic description of selective attentional processing in cortex. The model is described mathematically and implemented at the level of individual spiking neurons, with the computations for performing selective attentional processing being mapped to specific neuron types and laminar circuitry. The model is used to simulate three studies of attention in macaque, and is shown to quantitatively match several observed forms of attentional modulation. Specifically, ARC demonstrates that with shifts of spatial attention, neurons may exhibit shifting and shrinking of receptive fields; increases in responses without changes in selectivity for non-spatial features (i.e. response gain), and; that the effect on contrast-response functions is better explained as a response-gain effect than as contrast-gain. Unlike past models, ARC embodies a single mechanism that unifies the above forms of attentional modulation, is consistent with a wide array of available data, and makes several specific and quantifiable predictions.

At a given moment, a tremendous amount of visual information falls on the retinae, far more than the brain is capable of processing. By directing attention to a spatial location, stimuli at that position can be selectively processed, while irrelevant information from non-attended locations can be largely ignored. We present a detailed model that describes the mechanisms by which visual spatial attention may be implemented in the brain. Using this model, we simulated three previous studies of spatial attention in primates, and analysed the simulation data using the same methods as in the original experiments. Across these simulations, and without altering model parameters, our model produces results that are statistically indistinguishable from those recorded in primates. Unlike previous work, our model provides greater biological detail of how the brain performs selective visual processing, while also accurately demonstrating numerous forms of selective attention. Our results suggest that these seemingly different forms of attentional effects may result from a single mechanism for selectively processing attended stimuli.

Attending to the possibilities of action

Actions taking place in the environment are critical for our survival. We review evidence on attention to action, drawing on sets of converging evidence from neuropsychological patients through to studies of the time course and neural locus of action-based cueing of attention in normal observers. We show that the presence of action relations between stimuli helps reduce visual extinction in patients with limited attention to the contralesional side of space, while the first saccades made by normal observers and early perceptual and attentional responses measured using electroencephalography/event-related potentials are modulated by preparation of action and by seeing objects being grasped correctly or incorrectly for action. With both normal observers and patients, there is evidence for two components to these effects based on both visual perceptual and motor-based responses. While the perceptual responses reflect factors such as the visual familiarity of the action-related information, the motor response component is determined by factors such as the alignment of the objects with the observer's effectors and not by the visual familiarity of the stimuli. In addition to this, we suggest that action relations between stimuli can be coded pre-attentively, in the absence of attention to the stimulus, and action relations cue perceptual and motor responses rapidly and automatically. At present, formal theories of visual attention are not set up to account for these action-related effects; we suggest ways that theories could be expected to enable action effects to be incorporated.

The prognosis of allocentric and egocentric neglect: evidence from clinical scans

We contrasted the neuroanatomical substrates of sub-acute and chronic visuospatial deficits associated with different aspects of unilateral neglect using computed tomography scans acquired as part of routine clinical diagnosis. Voxel-wise statistical analyses were conducted on a group of 160 stroke patients scanned at a sub-acute stage. Lesion-deficit relationships were assessed across the whole brain, separately for grey and white matter. We assessed lesions that were associated with behavioural performance (i) at a sub-acute stage (within 3 months of the stroke) and (ii) at a chronic stage (after 9 months post stroke). Allocentric and egocentric neglect symptoms at the sub-acute stage were associated with lesions to dissociated regions within the frontal lobe, amongst other regions. However the frontal lesions were not associated with neglect at the chronic stage. On the other hand, lesions in the angular gyrus were associated with persistent allocentric neglect. In contrast, lesions within the superior temporal gyrus extending into the supramarginal gyrus, as well as lesions within the basal ganglia and insula, were associated with persistent egocentric neglect. Damage within the temporo-parietal junction was associated with both types of neglect at the sub-acute stage and 9 months later. Furthermore, white matter disconnections resulting from damage along the superior longitudinal fasciculus were associated with both types of neglect and critically related to both sub-acute and chronic deficits. Finally, there was a significant difference in the lesion volume between patients who recovered from neglect and patients with chronic deficits. The findings presented provide evidence that (i) the lesion location and lesion size can be used to successfully predict the outcome of neglect based on clinical CT scans, (ii) lesion location alone can serve as a critical predictor for persistent neglect symptoms, (iii) wide spread lesions are associated with neglect symptoms at the sub-acute stage but only some of these are critical for predicting whether neglect will become a chronic disorder and (iv) the severity of behavioural symptoms can be a useful predictor of recovery in the absence of neuroimaging findings on clinical scans. We discuss the implications for understanding the symptoms of the neglect syndrome, the recovery of function and the use of clinical scans to predict outcome.

I can see what you are doing: Action familiarity and affordance promote recovery from extinction

We assessed the effects of three factors on recovery from extinction in patients with lesions including the posterior parietal lobe: (a) whether objects were frequently used together in action versus whether they could be used in action together; (b) whether there was an effect of positioning objects for action; and (c) whether the surface properties of objects mediated performance. There was greater recovery from extinction for objects used frequently together, along with effects produced by objects that could be used together. There were also positive effects of correctly positioning objects for action. There were no effects of surface information on the benefit of reporting both members of a pair, though there were some effects on error trials. The results provide positive evidence for an effect on attention of affordances based on objects falling in plausible co-locations for action. Such affordances may also be influenced by the frequency with which objects are used together and can be generated from edge-based representations of objects (surface colour and depth are not necessary). However, surface-based representations may influence the speed with which action-related properties of objects engage attention.

Long-term effects of prism adaptation in chronic visual neglect: A single case study

We investigated the effects of long-term training using prism adaptation on a patient with chronic neglect. A positive effect of prism adaptation was apparent on tests of visuo-spatial processing (cancellation, bisection, and grasping), but there was no benefit for neglect on the detection of errors on the contralesional side of words or on the detection the left side of chimeric faces. Across training sessions, the benefit of adaptation on immediate performance decreased, but it increased across sessions and within sessions when performance was tested up to 90 minutes after adaptation. The beneficial effect was maintained up to 1 year posttraining. In later sessions there were also increased negative after-effects following prism adaptation, as training progressed. There was no improvement on tests of mathematical cognition, which comprised an independent deficit in this patient. The data suggest that prolonged prism training can induce long-term adaptive spatial realignment of visuo-motor mappings, ameliorating some aspects of neglect. We discuss the implications for the rehabilitation of neglect and for understanding the neglect syndrome more generally.

Neuroanatomical Dissections of Unilateral Visual Neglect Symptoms: ALE Meta-Analysis of Lesion-Symptom Mapping

Unilateral visual neglect is commonly defined as impaired ability to attend to stimuli presented on the side of visual space contralateral to the brain lesion. However, behavioral analyses indicate that different neglect symptoms can dissociate. The neuroanatomy of the syndrome has been hotly debated. Some groups have argued that the syndrome is linked to posterior parietal cortex lesions, while others report damage within regions including the superior temporal gyrus, insula, and basal ganglia. Several recent neuroimaging studies provide evidence that heterogeneity in the behavioral symptoms of neglect can be matched by variations in the brain lesions, and that some of the discrepancies across earlier findings might have resulted from the use of different neuropsychological tests and/or varied measures within the same task for diagnosing neglect. In this paper, we review the evidence for dissociations between both the symptoms and the neural substrates of unilateral visual neglect, drawing on ALE (anatomic likelihood estimation) meta-analyses of lesion-symptom mapping studies. Specifically, we examine dissociations between neglect symptoms associated with impaired control of attention across space (in an egocentric frame of reference) and within objects (in an allocentric frame of reference). Results of ALE meta-analyses indicated that, while egocentric symptoms are associated with damage within perisylvian network (pre- and postcentral, supramarginal, and superior temporal gyri) and damage within sub-cortical structures, more posterior lesions including the angular, middle temporal, and middle occipital gyri are associated with allocentric symptoms. Furthermore, there was high concurrence in deficits associated with white matter lesions within long association (superior longitudinal, inferior fronto-occipital, and inferior longitudinal fasciculi) and projection (corona radiata and thalamic radiation) pathways, supporting a disconnection account of the syndrome. Using this evidence we argue that different forms of neglect link to both distinct and common patterns of gray and white matter lesions. The findings are discussed in terms of functional accounts of neglect and theoretical models based on computational studies of both normal and impaired attention functions.

When connectedness increases hemispatial neglect

Patients with left neglect were tested with “chimeric” figures composed of the right and left halves of two different objects. The connectivity relation was modulated between the two half figures. For some displays, the two chimeric halves were separated by a small gap, while in others, the separate halves were connected by a line segment. In line with previous reports, performance on reporting the left half improved when the chimera were separated; but when a line connected the two separated halves the advantage was lost. If the connecting line was broken, the performance was again enhanced. The results suggest an important role for connectedness in the representation of perceptual objects and in the distribution of attention in neglect.

Eliminating inhibition of return by changing salient nonspatial attributes in a complex environment

Inhibition of return (IOR) occurs when a target is preceded by an irrelevant stimulus (cue) at the same location: Target detection is slowed, relative to uncued locations. In the present study, we used relatively complex displays to examine the effect of repetition of nonspatial attributes. For both color and shape, attribute repetition produced a robust inhibitory effect that followed a time course similar to that for location-based IOR. However, the effect only occurred when the target shared both the feature (i.e., color or shape) and location with the cue; this constraint implicates a primary role for location. The data are consistent with the idea that the system integrates consecutive stimuli into a single object file when attributes repeat, hindering detection of the second stimulus. The results are also consistent with an interpretation of IOR as a form of habituation, with greater habituation occurring with increasing featural overlap of a repeated stimulus. Critically, both of these interpretations bring the IOR effect within more general approaches to attention and perception, rather than requiring a specialized process with a limited function. In this view, there is no process specifically designed to inhibit return, suggesting that IOR may be the wrong framing of inhibitory repetition effects. Instead, we suggest that repetition of stimulus properties can interfere with the ability to focus attention on the aspects of a complex display that are needed to detect the occurrence of the target stimulus; this is a failure of activation, not an inhibition of processing.

Distinguishing non-spatial from spatial biases in visual selection: neuropsychological evidence

Recently, it has been shown that patients with visual extinction can show enhanced awareness of contralesional stimuli that match the contents of working memory [Soto, D., & Humphreys, G.W. (2006). Proceedings of the National Academy of Sciences, USA, 103, 4789-4792]. Here we investigate whether these effects extend to cases where the working memory cue is verbal rather than visual (Study 1), and we show that non-spatial cues can overcome problems in spatial disengagement in patients, and this can affect the first eye movement made in search (Study 2). We discuss the implications of the data for understanding the relations between the cueing of attention from visual and conceptual information held in working memory and the cueing of spatial attention, and for understanding the relations between non-spatial and spatial biases in selection.

Working memory and target-related distractor effects on visual search

We examined two forms of top-down effects on visual selection: (1) information held in working memory (WM) and (2) the semantic relations between targets and distractors. We found that items held in WM affected search for a different target. This WM-based interference effect generalized across different exemplars, even though participants could remember the specific exemplar on the trial. This argues against a memory top-up account of performance. In addition, there was interference from distractors that were not held in WM but were semantically related to the target. The effects of WM capture and the effects of capture by a distractor related to the target combined additively. The data suggest that task-irrelevant information in WM and task-relevant templates for targets compete separately for selection. The implications for understanding top-down processes in search are discussed.

Separating neural correlates of allocentric and egocentric neglect: distinct cortical sites and common white matter disconnections

Insights into the functional nature and neuroanatomy of spatial attention have come from research in neglect patients but to date many conflicting results have been reported. The novelty of the current study is that we used voxel-wise analyses based on information from segmented grey and white matter tissue combined with diffusion tensor imaging to decompose neural substrates of different neglect symptoms. Allocentric neglect was associated with damage to posterior cortical regions (posterior superior temporal sulcus, angular, middle temporal and middle occipital gyri). In contrast, egocentric neglect was associated with more anterior cortical damage (middle frontal, postcentral, supramarginal, and superior temporal gyri) and damage within subcortical structures. Damage to intraparietal sulcus (IPS) and the temporo-parietal junction (TPJ) was associated with both forms of neglect. Importantly, we showed that both disorders were associated with white matter lesions suggesting damage within long association and projection pathways such as the superior longitudinal, superior fronto-occipital, inferior longitudinal, and inferior fronto-occipital fascicule, thalamic radiation, and corona radiata. We conclude that distinct cortical regions control attention (a) across space (using an egocentric frame of reference) and (b) within objects (using an allocentric frame of reference), while common cortical regions (TPJ, IPS) and common white matter pathways support interactions across the different cortical regions.

Modelling Visual Search with the Selective Attention for Identification Model (VS-SAIM): A Novel Explanation for Visual Search Asymmetries

In earlier work, we developed the Selective Attention for Identification Model (SAIM [16]). SAIM models the human ability to perform translation-invariant object identification in multiple object scenes. SAIM suggests that central for this ability is an interaction between parallel competitive processes in a selection stage and a object identification stage. In this paper, we applied the model to visual search experiments involving simple lines and letters. We presented successful simulation results for asymmetric and symmetric searches and for the influence of background line orientations. Search asymmetry refers to changes in search performance when the roles of target item and non-target item (distractor) are swapped. In line with other models of visual search, the results suggest that a large part of the empirical evidence can be explained by competitive processes in the brain, which are modulated by the similarity between target and distractor. The simulations also suggest that another important factor is the feature properties of distractors. Finally, the simulations indicate that search asymmetries can be the outcome of interactions between top-down (knowledge about search items) and bottom-up (feature of search items) processing. This interaction in VS-SAIM is dominated by a novel mechanism, the knowledge-based on-centre-off-surround receptive field. This receptive field is reminiscent of the classical receptive fields but the exact shape is modulated by both, top-down and bottom-up processes. The paper discusses supporting evidence for the existence of this novel concept.

Modelling visual neglect: computational insights into conscious perception

Background

Visual neglect is an attentional deficit typically resulting from parietal cortex lesion and sometimes frontal lesion. Patients fail to attend to objects and events in the visual hemifield contralateral to their lesion during visual search.

Methodology/Principal Finding

The aim of this work was to examine the effects of parietal and frontal lesion in an existing computational model of visual attention and search and simulate visual search behaviour under lesion conditions. We find that unilateral parietal lesion in this model leads to symptoms of visual neglect in simulated search scan paths, including an inhibition of return (IOR) deficit, while frontal lesion leads to milder neglect and to more severe deficits in IOR and perseveration in the scan path. During simulations of search under unilateral parietal lesion, the model's extrastriate ventral stream area exhibits lower activity for stimuli in the neglected hemifield compared to that for stimuli in the normally perceived hemifield. This could represent a computational correlate of differences observed in neuroimaging for unconscious versus conscious perception following parietal lesion.

Conclusions/Significance

Our results lead to the prediction, supported by effective connectivity evidence, that connections between the dorsal and ventral visual streams may be an important factor in the explanation of perceptual deficits in parietal lesion patients and of conscious perception in general.

The brain's router: a cortical network model of serial processing in the primate brain

The human brain efficiently solves certain operations such as object recognition and categorization through a massively parallel network of dedicated processors. However, human cognition also relies on the ability to perform an arbitrarily large set of tasks by flexibly recombining different processors into a novel chain. This flexibility comes at the cost of a severe slowing down and a seriality of operations (100-500 ms per step). A limit on parallel processing is demonstrated in experimental setups such as the psychological refractory period (PRP) and the attentional blink (AB) in which the processing of an element either significantly delays (PRP) or impedes conscious access (AB) of a second, rapidly presented element. Here we present a spiking-neuron implementation of a cognitive architecture where a large number of local parallel processors assemble together to produce goal-driven behavior. The precise mapping of incoming sensory stimuli onto motor representations relies on a "router" network capable of flexibly interconnecting processors and rapidly changing its configuration from one task to another. Simulations show that, when presented with dual-task stimuli, the network exhibits parallel processing at peripheral sensory levels, a memory buffer capable of keeping the result of sensory processing on hold, and a slow serial performance at the router stage, resulting in a performance bottleneck. The network captures the detailed dynamics of human behavior during dual-task-performance, including both mean RTs and RT distributions, and establishes concrete predictions on neuronal dynamics during dual-task experiments in humans and non-human primates.

The neural mechanisms of visual selection: the view from neuropsychology

In this review, we discuss how neuropsychological impairments in visual selection can inform us about how selection normally operates. Using neuroanatomical and behavioral evidence on the disorders of neglect, extinction, and simultanagnosia, we propose functional and anatomical links between different aspects of visual selection and distinct sites in the posterior parietal cortex (PPC). This includes linking: (i) bottom-up attentional capture and the right temporo-parietal junction (TPJ); (ii) top-down segmentation of displays and the medial PPC; (iii) grouping, individuation and identification, and the inferior intra-parietal sulcus (IPS) bilaterally; and (iv) the suppression of saliency and the left IPS. In addition, when neuropsychological studies are combined with fMRI, there is evidence that these regions of the PPC interact with striate and extra-striate cortical areas, which respond to specific properties of stimuli. Selection should be viewed as an emergent property of a network of areas involving both ventral and dorsal cortex.

Using biologically plausible neural models to specify the functional and neural mechanisms of visual search

We review research from our laboratory that attempts to pull apart the functional and neural mechanisms of visual search using converging, inter-disciplinary evidence from experimental studies with normal participants, neuropsychological studies with brain lesioned patients, functional brain imaging and computational modelling. The work suggests that search is determined by excitatory mechanisms that support the selection of target stimuli, and inhibitory mechanisms that suppress irrelevant distractors. These mechanisms operate through separable though overlapping neural circuits which can be functionally decomposed by imposing model-based analyses on brain imaging data. The chapter highlights the need for inter-disciplinary research for understanding complex cognitive processes at several levels.

Reflexive and preparatory selection and suppression of salient information in the right and left posterior parietal cortex

Attentional cues can trigger activity in the parietal cortex in anticipation of visual displays, and this activity may, in turn, induce changes in other areas of the visual cortex, hence, implementing attentional selection. In a recent TMS study [Mevorach, C., Humphreys, G. W., & Shalev, L. Opposite biases in salience-based selection for the left and right posterior parietal cortex. Nature Neuroscience, 9, 740-742, 2006b], it was shown that the posterior parietal cortex (PPC) can utilize the relative saliency (a nonspatial property) of a target and a distractor to bias visual selection. Furthermore, selection was lateralized so that the right PPC is engaged when salient information must be selected and the left PPC when the salient information must be ignored. However, it is not clear how the PPC implements these complementary forms of selection. Here we used on-line triple-pulse TMS over the right or left PPC prior to or after the onset of global/local displays. When delivered after the onset of the display, TMS to the right PPC disrupted the selection of the more salient aspect of the hierarchical letter. In contrast, left PPC TMS delivered prior to the onset of the stimulus disrupted responses to the lower saliency stimulus. These findings suggest that selection and suppression of saliency, rather than being "two sides of the same coin," are fundamentally different processes. Selection of saliency seems to operate reflexively, whereas suppression of saliency relies on a preparatory phase that "sets up" the system in order to effectively ignore saliency.

Increased effect of target eccentricity on covert shifts of visual attention in patients with neglect

Debate continues regarding the mechanisms underlying covert shifts of visual attention. We examined the relationship between target eccentricity and the speed of covert shifts of attention in normal subjects and patients with brain lesions using a cued-response task in which cues and targets were presented at 2 degrees or 8 degrees lateral to the fixation point. Normal subjects were slower on invalid trials in the 8 degrees as compared to 2 degrees condition. Patients with right-hemisphere stroke with neglect were slower in their responses to left-sided invalid targets compared to valid targets, and demonstrated a significant increase in the effect of target validity as a function of target eccentricity. Additional data from one neglect patient (JM) demonstrated an exaggerated validity x eccentricity x side interaction for contralesional targets on a cued reaction time task with a central (arrow) cue. We frame these results in the context of a continuous 'moving spotlight' model of attention, and also consider the potential role of spatial saliency maps. By either account, we argue that neglect is characterized by an eccentricity-dependent deficit in the allocation of attention.