Effects of instantaneous object input and past experience on object-based attention

Attention can operate on object representations in visual sensory memory

Numerous studies have shown that attention can be allocated to various types of objects, such as low-level objects developed by perceptual organization and high-level objects developed by semantic associations. However, little is known about whether attention can also be affected solely by object representations in the brain, after the disappearance of physical objects. Here, we used a modified double-rectangle paradigm to investigate how attention is affected by object representation in visual sensory memory when the physical objects disappear for a short period of time before the target onset. By manipulating the interstimulus interval (ISI) between the offset of the objects and the onset of the target, an object-based attention effect, with shorter reaction times (RTs) for within-object relative to between-object conditions, was observed in the short-ISI (within 500 ms in Experiments 1a, 1b, 2, and 3) conditions while disappearing in the long-ISI (800 ms in Experiment 4) conditions. This result demonstrated that the mere presence of object representation in visual sensory memory, or the sensory memory-maintained object, can serve as an object unit that attention can operate on. This provides evidence for the relationship between object-based attention and visual sensory memory: object representation in visual sensory memory could affect attentional allocation, or attention can operate on a sensory memory-maintained object.

Figure-Ground Organization in Natural Scenes: Performance of a Recurrent Neural Model Compared with Neurons of Area V2

A crucial step in understanding visual input is its organization into meaningful components, in particular object contours and partially occluded background structures. This requires that all contours are assigned to either the foreground or the background (border ownership assignment). While earlier studies showed that neurons in primate extrastriate cortex signal border ownership for simple geometric shapes, recent studies show consistent border ownership coding also for complex natural scenes. In order to understand how the brain performs this task, we developed a biologically plausible recurrent neural network that is fully image computable. Our model uses local edge detector ( B ) cells and grouping ( G ) cells whose activity represents proto-objects based on the integration of local feature information. G cells send modulatory feedback connections to those B cells that caused their activation, making the B cells border ownership selective. We found close agreement between our model and neurophysiological results in terms of the timing of border ownership signals (BOSs) as well as the consistency of BOSs across scenes. We also benchmarked our model on the Berkeley Segmentation Dataset and achieved performance comparable to recent state-of-the-art computer vision approaches. Our proposed model provides insight into the cortical mechanisms of figure-ground organization.

A recurrent neural model for proto-object based contour integration and figure-ground segregation

Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects ("proto-objects") based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594-6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492-1499 2007; Chen et al. Neuron, 82(3), 682-694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.

Strength of object representation: its key role in object-based attention for determining the competition result between Gestalt and top-down objects

It was found in previous studies that two types of objects (rectangles formed according to the Gestalt principle and Chinese words formed in a top-down fashion) can both induce an object-based effect. The aim of the present study was to investigate how the strength of an object representation affects the result of the competition between these two types of objects based on research carried out by Liu, Wang and Zhou [(2011) Acta Psychologica, 138(3), 397-404]. In Experiment 1, the rectangles were filled with two different colors to increase the strength of Gestalt object representation, and we found that the object effect changed significantly for the different stimulus types. Experiment 2 used Chinese words with various familiarities to manipulate the strength of the top-down object representation. As a result, the object-based effect induced by rectangles was observed only when the Chinese word familiarity was low. These results suggest that the strength of object representation determines the result of competition between different types of objects.

The effects of attention pre-allocation and target-background integration on object-based attention

Object-based attention has been documented as an important mechanism with which to control attention in several studies. To date, two main hypotheses have been proposed to interpret object-based attention: attention spreading and prioritization of search. There is evidence that supports these hypotheses in the literature. In the present study, we sought to compare these two hypotheses systematically by manipulating two factors: the integration of the target and background and the presence of attention pre-allocation. For this purpose, we used a flanker task in which the location of the task-relevant target was fixed, but the relationship between the target and the background varied. In addition, attention pre-allocation was presented in only half of the conditions. The results revealed that the attention spreading hypothesis was supported only when attention was not pre-allocated and target-background integration was high; however, the prioritization hypothesis was supported in all other conditions. Our findings provide insight into the comparisons of the attention spreading and prioritization hypotheses. Furthermore, our findings suggest that attention resources may be the underlying factor determining appropriate strategy in the control of attention.

Forced to remember: when memory is biased by salient information

The last decades have seen a rapid growing in the attempt to understand the key factors involved in the internal memory representation of the external world. Visual salience have been found to provide a major contribution in predicting the probability for an item/object embedded in a complex setting (i.e., a natural scene) to be encoded and then remembered later on. Here I review the existing literature highlighting the impact of perceptual- (based on low-level sensory features) and semantics-related salience (based on high-level knowledge) on short-term memory representation, along with the neural mechanisms underpinning the interplay between these factors. The available evidence reveal that both perceptual- and semantics-related factors affect attention selection mechanisms during the encoding of natural scenes. Biasing internal memory representation, both perceptual and semantics factors increase the probability to remember high- to the detriment of low-saliency items. The available evidence also highlight an interplay between these factors, with a reduced impact of perceptual-related salience in biasing memory representation as a function of the increasing availability of semantics-related salient information. The neural mechanisms underpinning this interplay involve the activation of different portions of the frontoparietal attention control network. Ventral regions support the assignment of selection/encoding priorities based on high-level semantics, while the involvement of dorsal regions reflects priorities assignment based on low-level sensory features.

A model of proto-object based saliency

Organisms use the process of selective attention to optimally allocate their computational resources to the instantaneously most relevant subsets of a visual scene, ensuring that they can parse the scene in real time. Many models of bottom-up attentional selection assume that elementary image features, like intensity, color and orientation, attract attention. Gestalt psychologists, however, argue that humans perceive whole objects before they analyze individual features. This is supported by recent psychophysical studies that show that objects predict eye-fixations better than features. In this report we present a neurally inspired algorithm of object based, bottom-up attention. The model rivals the performance of state of the art non-biologically plausible feature based algorithms (and outperforms biologically plausible feature based algorithms) in its ability to predict perceptual saliency (eye fixations and subjective interest points) in natural scenes. The model achieves this by computing saliency as a function of proto-objects that establish the perceptual organization of the scene. All computational mechanisms of the algorithm have direct neural correlates, and our results provide evidence for the interface theory of attention.

Object-based attention guided by an invisible object

Evidence for object-based attention typically comes from studies using displays with visible objects, and little is known about whether object-based attention can occur with invisible objects. We investigated this issue with a modified double-rectangle cuing paradigm, which was originally developed by Egly et al. (J Exp Psychol Gen 123:161-177, 1994). In this study, low-contrast rectangles were presented very briefly, which rendered them invisible to subjects. With the invisible rectangles, we found a classical object-based attentional effect as indexed by the same-object effect. We also found the instantaneous object effect-object-based attention was dependent on the orientation of the rectangles presented with the target, providing evidence for the dynamic updating hypothesis (Ho and Yeh in Acta Psychol 132:31-39, 2009). These results suggest that object-based attention can be guided by an invisible object in an automatic way, with a minimal influence from high-level top-down control.

Object-based attention: sensory enhancement or scanning prioritization

The three experiments reported here test whether object-modulated attentional spreading can be obtained when the target location is 100% certain. Experiment 1 uses the reaction time (RT)-based flanker task similar to Shomstein and Yantis (2002), and replicates the null result of the object-modulated attentional spreading. RT and accuracy (ACC) have been shown to reflect different processes: postperceptual decision vs. perceptual process (Santee & Egeth, 1982). Experiment 2 adopts the data-limited ACC-based measure and reports that attention could spread within the attended object. To avoid ceiling effects, Experiment 3 adjusts the presentation time based on the trials where the target and flankers were compatible and on the same objects, and provides the convergent evidence supporting the object-modulated attentional spreading. These results suggest that because the RT-based measure is less sensitive in reflecting the quality of perceptual representations, it is not sufficiently a strong evidence to distinguish between sensory enhancement and scanning prioritization accounts.

Pattern of BOLD signal in auditory cortex relates acoustic response to perceptual streaming

BACKGROUND

Segregating auditory scenes into distinct objects or streams is one of our brain's greatest perceptual challenges. Streaming has classically been studied with bistable sound stimuli, perceived alternately as a single group or two separate groups. Throughout the last decade different methodologies have yielded inconsistent evidence about the role of auditory cortex in the maintenance of streams. In particular, studies using functional magnetic resonance imaging (fMRI) have been unable to show persistent activity within auditory cortex (AC) that distinguishes between perceptual states.

RESULTS

We use bistable stimuli, an explicit perceptual categorization task, and a focused region of interest (ROI) analysis to demonstrate an effect of perceptual state within AC. We find that AC has more activity when listeners perceive the split percept rather than the grouped percept. In addition, within this ROI the pattern of acoustic response across voxels is significantly correlated with the pattern of perceptual modulation. In a whole-brain exploratory test, we corroborate previous work showing an effect of perceptual state in the intraparietal sulcus.

CONCLUSIONS

Our results show that the maintenance of auditory streams is reflected in AC activity, directly relating sound responses to perception, and that perceptual state is further represented in multiple, higher level cortical regions.

First come, first served? Influence of changed object configuration on object-based attention

Evidence for object-based attention is based mainly on studies using object displays that remain unchanged throughout, with the assumption that object representation should be completed and stabilized before it is selected for further processing. We used the modified double-rectangle cuing paradigm of Egly, Driver, and Rafal (Journal of Experimental Psychology: General, 123, 161-177, 1994) but introduced a configuration change to the cued-object display to test whether object-based attention is determined by the cued- or, alternatively, the changed-object display. Four small rectangles were presented in the initial display; critically, after one was cued, an occluder was presented to make the four small rectangles amodally completed into the double-rectangle configuration. Results show that object-based attention is determined by the changed display, but not by the cued display. This suggests that object-based attention is an interactively evolving process between object representation and attention, rather than a serial process in which attention operates after object representation is completed.