Orienting attention to objects in visual short-term memory

Looking, language, and memory: bridging research from the visual world and visual search paradigms

In the visual world paradigm as used in psycholinguistics, eye gaze (i.e. visual orienting) is measured in order to draw conclusions about linguistic processing. However, current theories are underspecified with respect to how visual attention is guided on the basis of linguistic representations. In the visual search paradigm as used within the area of visual attention research, investigators have become more and more interested in how visual orienting is affected by higher order representations, such as those involved in memory and language. Within this area more specific models of orienting on the basis of visual information exist, but they need to be extended with mechanisms that allow for language-mediated orienting. In the present paper we review the evidence from these two different - but highly related - research areas. We arrive at a model in which working memory serves as the nexus in which long-term visual as well as linguistic representations (i.e. types) are bound to specific locations (i.e. tokens or indices). The model predicts that the interaction between language and visual attention is subject to a number of conditions, such as the presence of the guiding representation in working memory, capacity limitations, and cognitive control mechanisms.

Selection within visual memory representations activates the oculomotor system

Humans tend to create and maintain internal representations of the environment that help guiding actions during the everyday activities. Previous studies have shown that the oculomotor system is involved in coding and maintenance of locations in visual-spatial working memory. In these studies selection of the relevant location for maintenance in working memory took place on the screen (selecting the location of a dot presented on the screen). The present study extended these findings by showing that the oculomotor system also codes selection of location from an internal memory representation. Participants first memorized two locations and after a retention interval selected one location for further maintenance. The results show that saccade trajectories deviated away from the ultimately remembered location. Furthermore, selection of the location from the memorized representation produced sustained oculomotor preparation to it. The results show that oculomotor system is very flexible and plays an active role for coding and maintaining information selected within internal memory representations.

Bayesian filtering of human brain hemodynamic activity elicited by visual short-term maintenance recorded through functional near-infrared spectroscopy (fNIRS)

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique that measures changes in oxy-hemoglobin (ΔHbO) and deoxy-hemoglobin (ΔHbR) concentration associated with brain activity. The signal acquired with fNIRS is naturally affected by disturbances engendering from ongoing physiological activity (e.g., cardiac, respiratory, Mayer wave) and random measurement noise. Despite its several drawbacks, the so-called conventional averaging (CA) is still widely used to estimate the hemodynamic response function (HRF) from noisy signal. One such drawback is related to the number of trials necessary to derive stable HRF functions adopting the CA approach, which must be substantial (N >> 50). In this work, a pre-processing procedure to remove artifacts followed by the application of a non-parametric Bayesian approach is proposed that capitalizes on a priori available knowledge about HRF and noise. Results with the proposed Bayesian approach were compared with CA and with a straightforward band-pass filtering approach. On simulated data, a five times lower estimation error on HRF was obtained with respect to that obtained by CA, and 2.5 times lower than that obtained by band pass filtering. On real data, the improvement achieved by the present method was attested by an increase in the contrast to noise ratio (CNR) and by a reduced variability in single trial estimation. An application of the present Bayesian approach is illustrated that was optimized to monitor changes in hemodynamic activity reflecting variations in visual short-term memory load in humans, which are notoriously hard to detect using functional magnetic resonance imaging (fMRI). In particular, statistical analyses of HRFs recorded during a memory task established with high reliability the crucial role of the intraparietal sulcus and the intra-occipital sulcus in posterior areas of the human brain in visual short-term memory maintenance.

Object-substitution masking modulates spatial attention deployment and the encoding of information in visual short-term memory: insights from occipito-parietal ERP components

If object-substitution masking (OSM) arises from mask representations replacing target representations, OSM should impede the formation of representations in visual short-term memory (VSTM). We utilized event-related potentials to examine the effect of OSM on target processing. An N2pc was observed on trials with delayed-offset masks, indicating that focused attention was directed to the target. The sustained posterior contralateral negativity (SPCN), an index of VSTM storage, was observed in delayed-offset trials only on trials with correct responses. This supports the hypothesis that inaccurate performance on delayed-offset trials arises from a failure to encode the target in VSTM. On co-termination trials, accuracy was high and neither the N2pc nor SPCN was observed. This indicates that, in the absence of masking, the task was accomplished by maintaining a diffuse attentional state that enabled the joint encoding of the potential target items.

Subliminally presented and stored objects capture spatial attention

When objects disappear from view, we can still bring them to mind, at least for brief periods of time, because we can represent those objects in visual short-term memory (VSTM) (Sperling, 1960; Cowan, 2001). A defining characteristic of this representation is that it is topographic, that is, it preserves a spatial organization based on the original visual percept (Vogel and Machizawa, 2004; Astle et al., 2009; Kuo et al., 2009). Recent research has also shown that features or locations of visual items that match those being maintained in conscious VSTM automatically capture our attention (Awh and Jonides, 2001; Olivers et al., 2006; Soto et al., 2008). But do objects leave some trace that can guide spatial attention, even without participants intentionally remembering them? Furthermore, could subliminally presented objects leave a topographically arranged representation that can capture attention? We presented objects either supraliminally or subliminally and then 1 s later re-presented one of those objects in a new location, as a "probe" shape. As participants made an arbitrary perceptual judgment on the probe shape, their covert spatial attention was drawn to the original location of that shape, regardless of whether its initial presentation had been supraliminal or subliminal. We demonstrate this with neural and behavioral measures of memory-driven attentional capture. These findings reveal the existence of a topographically arranged store of "visual" objects, the content of which is beyond our explicit awareness but which nonetheless guides spatial attention.