Visual Working Memory Enhances the Neural Response to Matching Visual Input

Working memory expedites the processing of visual signals within the extrastriate cortex

Working memory is the ability to maintain information in the absence of sensory input. In this study, we investigated how working memory benefits processing in visual areas. Using a measure of phase consistency to detect the arrival time of visual signals to the middle temporal (MT) area, we assessed the impact of working memory on the speed of sensory processing. We recorded from MT neurons in two monkeys during a spatial working memory task with visual probes. When the memorized location closely matches the receptive field center of the recording site, visual input arrives sooner, but if the memorized location does not match the receptive field center then the arrival of visual information is delayed. Thus, working memory expedites the arrival of visual input in MT. These results reveal that even in the absence of firing rate changes, working memory can still benefit the processing of information within sensory areas.

The effects of search-irrelevant working memory content on visual search

Previous experiments investigating visual search have shown that distractors that are semantically related to a search target can capture attention and slow the search process. In two experiments, we examine if distractors exactly matching, or semantically related to, search-irrelevant information held in working memory (WM) can also influence visual search while ruling out potential effects of color similarity. Participants first viewed and memorized an image of an everyday object, then they determined if a target item was present or absent in a two-object search array. On exact-match trials, the memorized object appeared as a distractor; on semantic-match trials, an object semantically related to the memorized object appeared as a distractor. Both exact-match and semantic-match distractors slowed search when the target was present in the search array. Our findings extend previous findings by demonstrating WM-driven attentional guidance by complex objects rather than simple features. The results also suggest that visual search can be influenced by distractors sharing only semantic features with a search-irrelevant, but active, WM representation.

Visual-cortical enhancement by acoustic distractors: The effects of endogenous spatial attention and visual working memory load

Past work has shown that when a peripheral sound captures our attention, it activates the contralateral visual cortex as revealed by an event-related potential component labelled the auditory-evoked contralateral occipital positivity (ACOP). This cross-modal activation of the visual cortex has been observed even when the sounds were not relevant to the ongoing task (visual or auditory), suggesting that peripheral sounds automatically activate the visual cortex. However, it is unclear whether top-down factors such as visual working memory (VWM) load and endogenous attention, which modulate the impact of task-irrelevant information, may modulate this spatially-specific component. Here, we asked participants to perform a lateralized VWM task (change detection), whose performance is supported by both endogenous spatial attention and VWM storage. A peripheral sound that was unrelated to the ongoing task was delivered during the retention interval. The amplitude of sound-elicited ACOP was analyzed as a function of the spatial correspondence with the cued hemifield, and of the memory array set-size. The typical ACOP modulation was observed over parieto-occipital sites in the 280-500 ms time window after sound onset. Its amplitude was not affected by VWM load but was modulated when the location of the sound did not correspond to the hemifield (right or left) that was cued for the change detection task. Our results suggest that sound-elicited activation of visual cortices, as reflected in the ACOP modulation, is unaffected by visual working memory load. However, endogenous spatial attention affects the ACOP, challenging the hypothesis that it reflects an automatic process.

Considering Readout to Understand Working Memory

While working memory (WM) allows us to store past information, its function is to guide future behavior. Given this role, the tight link between how WMs are maintained and how they are read out to be transformed into context-appropriate actions remains relatively unexplored. Beyond helping us understand memory-guided behavior, focusing on WM readout may also help us better understand the neural basis of memory maintenance.

Quiet Trajectories as Neural Building Blocks

Our concept of the neural mechanisms of working memory has recently undergone an upheaval, because of two transformative concepts: multivariate neural state trajectories and the activity-silent hypothesis. I will argue that putting these concepts together raises the difficult problem of "quiet trajectories," where future neural activity is not fully determined by current activity. However, this also promises new building blocks for neural computation.

The Rejection Template of Working Memory Operates after Attention Capture

Although scientists know that information stored in working memory guides visual attention, how this is accomplished is still under debate. Specifically, there is a dispute between the Biased Competition Model and Visual Attention Theory. The current study used two experiments to resolve this controversy based on previous research. Experiment 1 found that although inverse efficiency scores for High and Low numbers of memory distractors were both longer than the Baseline (no memory distractors), they did not significantly differ from each other. This indicated that memory might guide attention via a capture-then-global-inhibition process. Experiment 2 addressed the possibility that the findings resulted from the time needed to reject the interfering objects by requiring both memory-matching and memory-mismatching conditions to be rejected under a highlighted target. This result showed that the memory-matching condition resulted in longer search times than the memory-mismatching condition, indicating an attention-capture effect based on working memory. Together, the two experiments support the idea that when multiple memory-matching distractors in a search array first capture an individual’s attention, it then acts as a template that allows the individual to suppress all interfering items that containing memory information holds. This study supports the Biased Competition Model early on in visual search. However, the late stage of visual search supports the Visual Attention Theory. These advance our knowledge regarding the relationship between working memory content and attention.

Neural Reinstatement Tracks Spread of Attention between Object Features in Working Memory

Attention can be allocated in working memory (WM) to select and privilege relevant content. It is unclear whether attention selects individual features or whole objects in WM. Here, we used behavioral measures, eye-tracking, and EEG to test the hypothesis that attention spreads between an object's features in WM. Twenty-six participants completed a WM task that asked them to recall the angle of one of two oriented, colored bars after a delay while EEG and eye-tracking data were collected. During the delay, an orthogonal "incidental task" cued the color of one item for a match/mismatch judgment. On congruent trials (50%), the cued item was probed for subsequent orientation recall; on incongruent trials (50%), the other memory item was probed. As predicted, selecting the color of an object in WM brought other features of the cued object into an attended state as revealed by EEG decoding, oscillatory α-power, gaze bias, and improved orientation recall performance. Together, the results show that attentional selection spreads between an object's features in WM, consistent with object-based attentional selection. Analyses of neural processing at recall revealed that the selected object was automatically compared with the probe, whether it was the target for recall or not. This provides a potential mechanism for the observed benefits of nonpredictive cueing in WM, where a selected item is prioritized for subsequent decision-making.

Attentional Prioritization of Complex, Naturalistic Stimuli Maintained in Working-Memory–A Dot-Probe Event-Related Potentials Study

Recent studies suggest that a stimulus actively maintained in working memory (WM) automatically captures visual attention when subsequently perceived. Such a WM-based guidance effect has been consistently observed for stimuli defined by simple features, such as color or orientation, but studies using more complex stimuli provided inconclusive results. Therefore, we investigated whether the WM-based guidance effect occurs also for naturalistic stimuli, whose identity is defined by multiple features and relations among them, specifically for faces and houses. The experiment consisted of multiple blocks in which participants (N = 28) either memorized or merely saw (WM or exposure condition, respectively) a template stimulus and then performed several dot-probe trials, with pairs of stimuli (template and control) presented laterally as distractors and followed by a target-asterisk. Analysis of reaction-times (RT) in the dot-probe task shows that memorized stimuli were prioritized by attention and points toward attention-hold, rather than capture, as a mechanism of attentional prioritization. Consistent with this interpretation, memorized items did not evoke a lateralized N2pc ERP component, thought to indicate attention shifts. However, in an exploratory ERP analysis we found evidence for a very early (100–200 ms post-stimulus) prioritization specific to the memorized faces, which is in line with the sensory recruitment theory of WM. In conclusion, our data provide evidence that complex stimuli are prioritized by attention when maintained in WM, and that the mechanism of such prioritization is based on a prolonged hold of spatial attention.

Neural basis of distractor resistance during visual working memory maintenance

Visual working memory representations must be protected from the intervening irrelevant visual input. While it is well known that interference resistance is most challenging when distractors match the prioritised mnemonic information, its neural mechanisms remain poorly understood. Here, we identify two top-down attentional control processes that have opposing effects on distractor resistance. We reveal an early selection negativity in the EEG responses to matching as compared to non-matching distractors, the magnitude of which is negatively associated with behavioural distractor resistance. Additionally, matching distractors lead to reduced post-stimulus alpha power as well as increased fMRI responses in the object-selective visual cortical areas and the inferior frontal gyrus. However, the congruency effect found on the post-stimulus periodic alpha power and the inferior frontal gyrus fMRI responses show a positive association with distractor resistance. These findings suggest that distractor interference is enhanced by proactive memory content-guided selection processes and diminished by reactive allocation of top-down attentional resources to protect memorandum representations within visual cortical areas retaining the most selective mnemonic code.

EEG cross-frequency phase synchronization as an index of memory matching in visual search

Visual perception is influenced by our expectancies about incoming sensory information. It is assumed that mental templates of expected sensory input are created and compared to actual input, which can be matching or not. When such mental templates are held in working memory, cross-frequency phase synchronization (CFS) between theta and gamma band activity has been proposed to serve matching processes between prediction and sensation. We investigated how this is affected by the number of activated templates that could be matched by comparing conditions where participants had to keep either one or multiple templates in mind for successful visual search. We found a transient CFS between EEG theta and gamma activity in an early time window around 150 ms after search display presentation, in right hemispheric parietal cortex. Our results suggest that for single template conditions, stronger transient theta-gamma CFS at posterior sites contralateral to target presentation can be observed than for multiple templates. This can be interpreted as evidence to the idea of sequential attentional templates. But mainly, it is understood in line with previous theoretical accounts strongly arguing for transient synchronization between posterior theta and gamma phase as a neural correlate of matching incoming sensory information with contents from working memory and as evidence for limitations in memory matching during multiple template search.

Dynamics of Heading and Choice-Related Signals in the Parieto-Insular Vestibular Cortex of Macaque Monkeys

Perceptual decision-making is increasingly being understood to involve an interaction between bottom-up sensory-driven signals and top-down choice-driven signals, but how these signals interact to mediate perception is not well understood. The parieto-insular vestibular cortex (PIVC) is an area with prominent vestibular responsiveness, and previous work has shown that inactivating PIVC impairs vestibular heading judgments. To investigate the nature of PIVC's contribution to heading perception, we recorded extracellularly from PIVC neurons in two male rhesus macaques during a heading discrimination task, and compared findings with data from previous studies of dorsal medial superior temporal (MSTd) and ventral intraparietal (VIP) areas using identical stimuli. By computing partial correlations between neural responses, heading, and choice, we find that PIVC activity reflects a dynamically changing combination of sensory and choice signals. In addition, the sensory and choice signals are more balanced in PIVC, in contrast to the sensory dominance in MSTd and choice dominance in VIP. Interestingly, heading and choice signals in PIVC are negatively correlated during the middle portion of the stimulus epoch, reflecting a mismatch in the polarity of heading and choice signals. We anticipate that these results will help unravel the mechanisms of interaction between bottom-up sensory signals and top-down choice signals in perceptual decision-making, leading to more comprehensive models of self-motion perception.SIGNIFICANCE STATEMENT Vestibular information is important for our perception of self-motion, and various cortical regions in primates show vestibular heading selectivity. Inactivation of the macaque vestibular cortex substantially impairs the precision of vestibular heading discrimination, more so than inactivation of other multisensory areas. Here, we record for the first time from the vestibular cortex while monkeys perform a forced-choice heading discrimination task, and we compare results with data collected previously from other multisensory cortical areas. We find that vestibular cortex activity reflects a dynamically changing combination of sensory and choice signals, with both similarities and notable differences with other multisensory areas.

Distraction in Visual Working Memory: Resistance is Not Futile

Over half a century of research focused on understanding how working memory is capacity constrained has overshadowed the fact that it is also remarkably resistant to interference. Protecting goal-relevant information from distraction is a cornerstone of cognitive function that involves a multifaceted collection of control processes and storage mechanisms. Here, we discuss recent advances in cognitive psychology and neuroscience that have produced new insights into the nature of visual working memory and its ability to resist distraction. We propose that distraction resistance should be an explicit component in any model of working memory and that understanding its behavioral and neural correlates is essential for building a comprehensive understanding of real-world memory function.

Visual working memory modulates the temporal resolution of visual perception

The present study examined how visual working memory affects temporal resolution of visual stimuli that match the memorised nontemporal information about stimulus features (e.g., shapes). The results show that active maintenance of a sample shape in visual working memory impaired the concurrent detection of a sufficiently brief temporal gap within a matching visual target stimulus. The impairment of temporal gap detection was not likely due to the mechanism of visual priming from the presentation of the sample shape, because there was no evidence for priming effects when the sample was only passively viewed without working memory demands or when the sample was initially encoded into memory but did not need to be actively maintained in mind by the time the target stimulus appeared. These findings demonstrate that visual working memory can lower the temporal resolution of visual perception and thereby impair performance in the temporal gap detection task that requires fine temporal segregation of visual information, and they suggest the existence of a previously unknown downside to the interplay between working memory and visual perception.

Theoretical distinction between functional states in working memory and their corresponding neural states

Working memory (WM) is important for guiding behaviour, but not always for the next possible action. Here we define a WM item that is currently relevant for guiding behaviour as the functionally "active" item; whereas items maintained in WM, but not immediately relevant to behaviour, are defined as functionally "latent". Traditional neurophysiological theories of WM proposed that content is maintained via persistent neural activity (e.g., stable attractors); however, more recent theories have highlighted the potential role for "activity-silent" mechanisms (e.g., short-term synaptic plasticity). Given these somewhat parallel dichotomies, functionally active and latent cognitive states of WM have been associated with storage based on persistent-activity and activity-silent neural mechanisms, respectively. However, in this article we caution against a one-to-one correspondence between functional and activity states. We argue that the principal theoretical requirement for active and latent WM is that the corresponding neural states play qualitatively different functional roles. We consider a number of candidate solutions, and conclude that the neurophysiological mechanisms for functionally active and latent WM items are theoretically independent of the distinction between persistent activity-based and activity-silent forms of WM storage.

Structural Neural Correlates of Double Decision Performance in Older Adults

Speed of processing is a cognitive domain that encompasses the speed at which an individual can perceive a given stimulus, interpret the information, and produce a correct response. Speed of processing has been shown to decline more rapidly than other cognitive domains in an aging population, suggesting that this domain is particularly vulnerable to cognitive aging (Chee et al., 2009). However, given the heterogeneity of neuropsychological measures used to assess the domains underpinning speed of processing, a diffuse pattern of brain regions has been implicated. The current study aims to investigate the structural neural correlates of speed of processing by assessing cortical volume and speed of processing scores on the POSIT Double Decision task within a healthy older adult population (N = 186; mean age = 71.70 ± 5.32 years). T1-weighted structural images were collected via a 3T Siemens scanner. The current study shows that less cortical thickness in right temporal, posterior frontal, parietal and occipital lobe structures were significantly associated with poorer Double Decision scores. Notably, these include the lateral orbitofrontal gyrus, precentral gyrus, superior, transverse, and inferior temporal gyrus, temporal pole, insula, parahippocampal gyrus, fusiform gyrus, lingual gyrus, superior and inferior parietal gyrus and lateral occipital gyrus. Such findings suggest that speed of processing performance is associated with a wide array of cortical regions that provide unique contributions to performance on the Double Decision task.

Visual working memory enhances target discrimination accuracy with single-item displays

The maintenance of information in visual working memory has been shown to bias the concurrent processing in favor of matching visual input. The present study aimed to examine whether this bias can act at an early stage of processing to enhance target feature perception in single-item displays. Participants were sequentially presented with two distinct colored stimuli as memory samples and a retro-cue indicating which of the two samples should be maintained for subsequent memory test. During the retention interval, they had to discriminate the gap orientation of a Landolt target presented through a single visual stimulus that could match one or neither of the two samples. Across two experiments, we consistently found that discrimination performance was more accurate when the Landolt target was situated within a stimulus that matched the sample being retained in visual working memory, as compared with when the target was not. This effect cannot be attributed to the mechanism of passive priming, because we failed to observe priming effects when the stimulus containing the target matched the sample that was retro-cued to be irrelevant to the working memory task, as compared to when the stimulus matched neither sample. Given the fact that target stimuli were presented in single-item displays wherein external noise was precluded, the present findings demonstrate that the working memory bias of visual attention operating in the absence of stimulus competition facilitates early perceptual processing at the attended location via signal enhancement.

No evidence for mnemonic modulation of interocularly suppressed visual input

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Oscillatory Control over Representational States in Working Memory

In the visual world, attention is guided by perceptual goals activated in visual working memory (VWM). However, planning multiple-task sequences also requires VWM to store representations for future goals. These future goals need to be prevented from interfering with the current perceptual task. Recent findings have implicated neural oscillations as a control mechanism serving the implementation and switching of different states of prioritization of VWM representations. We review recent evidence that posterior alpha-band oscillations underlie the flexible activation and deactivation of VWM representations and that frontal delta-to-theta-band oscillations play a role in the executive control of this process. That is, frontal delta-to-theta appears to orchestrate posterior alpha through long-range oscillatory networks to flexibly set up and change VWM states during multitask sequences.

The contents of visual working memory delay the perceived offset of matching visual stimuli

Previous research suggests that the perception of stimulus onset can be accelerated by a match between the contents of visual working memory and the stimulus presented alone in the peripheral visual field. This onset acceleration effect might contribute to previously reported effects of working memory on perceived stimulus duration. However, it remains possible that the contents of visual working memory may also modulate the offset perception of matching visual stimuli, thereby contributing to the modulation of duration perception by working memory. The present study directly tested this possibility by using a simple reaction time task to assess the effect of visual working memory on perceived stimulus offset. Participants were asked to maintain a sample stimulus in working memory and subsequently had to respond to the offset of a single visual target. Across three experiments, we showed that the offset response was reliably slower when the target matched the sample held in visual working memory, as compared with when the target did not. This effect was not likely attributed to the mechanism of repetition priming from the presentation of the sample, because we failed to observe a priming effect either when the sample was only passively viewed without working memory demands or when the sample was initially encoded into memory but did not need to be actively maintained in mind by the time the offset target appeared. The findings provide direct evidence indicating that active maintenance of information in visual working memory delays the perceived offset of matching visual stimuli.

Scenes Modulate Object Processing Before Interacting With Memory Templates

When searching for relevant objects in our environment (say, an apple), we create a memory template (a red sphere), which causes our visual system to favor template-matching visual input (applelike objects) at the expense of template-mismatching visual input (e.g., leaves). Although this principle seems straightforward in a lab setting, it poses a problem in naturalistic viewing: Two objects that have the same size on the retina will differ in real-world size if one is nearby and the other is far away. Using the Ponzo illusion to manipulate perceived size while keeping retinal size constant, we demonstrated across 71 participants that visual objects attract attention when their perceived size matches a memory template, compared with mismatching objects that have the same size on the retina. This shows that memory templates affect visual selection after object representations are modulated by scene context, thus providing a working mechanism for template-based search in naturalistic vision.

Oscillatory Mechanisms of Preparing for Visual Distraction

Evidence shows that observers preactivate a target representation in preparation of a visual selection task. In this study, we addressed the question if and how preparing to ignore an anticipated distractor differs from preparing for an anticipated target. We measured EEG while participants memorized a laterally presented color, which was cued to be either a target or a distractor in two subsequent visual search tasks. Decoding the location of items in the search display from EOG channels revealed that, initially, the anticipated distractor attracted attention and could only be ignored later during the trial. This suggests that distractors could not be suppressed in advance but were represented in an active, attention-guiding format. Consistent with this, lateralized posterior alpha power did not dissociate between target and distractor templates during the delay periods, suggesting similar encoding and maintenance. However, distractor preparation did lead to relatively enhanced nonlateralized posterior alpha power, which appeared to gate sensory processing at search display onset to prevent attentional capture in general. Finally, anticipating distractors also led to enhanced midfrontal theta power during the delay period, a signal that was predictive of how strongly both target and distractor were represented in the search display. Together, our results speak against a distractor-specific advance inhibitory template, thus contrary to the preactivation of specific target templates. Rather, we demonstrate a general selection suppression mechanism, which serves to prevent initial involuntary capture by anticipated distracting input.

Selection of Visual Objects in Perception and Working Memory One at a Time

How does the content of visual working memory influence the way we process the visual environment? We addressed this question using the steady-state visual evoked potential (SSVEP), which provides a discernible measure of visuocortical activation to multiple stimuli simultaneously. Fifty-six adults were asked to remember a set of two oriented gratings. During the retention interval, two frequency-tagged oriented gratings were presented to probe the visuocortical processing of matching versus mismatching orientations relative to the memory set. Matching probes prompted an increased visuocortical response, whereas mismatching stimuli were suppressed. This suggests that the visual cortex prioritizes attentional selection of memory-relevant features at the expense of non-memory-relevant features. When two memory items were probed simultaneously, visuocortical amplification alternated between the two stimuli at a rate of 3 Hz to 4 Hz, consistent with the rate of attentional sampling of sensory events from the external world. These results suggest a serial, single-item attentional sampling of remembered features.

Coexisting representations of sensory and mnemonic information in human visual cortex

Traversing sensory environments requires keeping relevant information in mind while simultaneously processing new inputs. Visual information is kept in working memory via feature selective responses in early visual cortex, but recent work had suggested that new sensory inputs obligatorily wipe out this information. Here we show region-wide multiplexing abilities in classic sensory areas, with population-level response patterns in early visual cortex representing the contents of working memory alongside new sensory inputs. In a second experiment, we show that when people get distracted, this leads to both disruptions of mnemonic information in early visual cortex and decrements in behavioral recall. Representations in the intraparietal sulcus reflect actively remembered information encoded in a transformed format, but not task-irrelevant sensory inputs. Together these results suggest that early visual areas play a key role in supporting high resolution working memory representations that can serve as a template for comparing incoming sensory information.

Prospectively reinstated memory drives conscious access of matching visual input

Maintaining information in visual working memory (VWM) biases attentional selection of concurrent visual input, by favoring VWM-matching over VWM-mismatching visual input. Recently, it was shown that this bias disappears when the same item is memorized on consecutive occasions (as memoranda presumably transit from VWM to long-term memory), but reemerges when observers anticipate to memorize a novel item on a subsequent trial. Here, we aimed to conceptually replicate and extend this intriguing finding, by investigating whether prospectively reinstated memory drives conscious access of memory-matching visual input. We measured the time it took for participants to detect interocularly suppressed target stimuli, which were either from the same color category as a concurrently memorized color or not. Our results showed that the advantage of memory-matching targets in overcoming suppression progresses non-monotonically across consecutive memorizations of the same color ('repetitions'): the advantage for memory-matching visual input initially declined to asymptote, before being fully revived on the last repetition. This revival was not observed in a control experiment in which targets were not interocularly suppressed. The results suggest that, as observers anticipate to memorize a novel item imminently, VWM usage is prospectively reinstated, causing memory-matching visual input to gain accelerated access to consciousness again.

Priority Switches in Visual Working Memory are Supported by Frontal Delta and Posterior Alpha Interactions

Visual working memory (VWM) distinguishes between representations relevant for imminent versus future perceptual goals. We investigated how the brain sequentially prioritizes visual working memory representations that serve consecutive tasks. Observers remembered two targets for a sequence of two visual search tasks, thus making one target currently relevant, and the other prospectively relevant. We show that during the retention interval prior to the first search, lateralized parieto-occipital EEG alpha (8-14 Hz) suppression is stronger for current compared with prospective search targets. Crucially, between the first and second search task, this difference in posterior alpha lateralization reverses, reflecting the change in priority states of the two target representations. Connectivity analyses indicate that this switch in posterior alpha lateralization is driven by frontal delta/low-theta (2-6 Hz) activity. Moreover, this frontal low-frequency signal also predicts task performance after the switch. We thus obtained evidence for large-scale network interactions underlying the flexible shifting between the priority states of multiple memory representations in VWM.

Eye Movement-Related Confounds in Neural Decoding of Visual Working Memory Representations

A relatively new analysis technique, known as neural decoding or multivariate pattern analysis (MVPA), has become increasingly popular for cognitive neuroimaging studies over recent years. These techniques promise to uncover the representational contents of neural signals, as well as the underlying code and the dynamic profile thereof. A field in which these techniques have led to novel insights in particular is that of visual working memory (VWM). In the present study, we subjected human volunteers to a combined VWM/imagery task while recording their neural signals using magnetoencephalography (MEG). We applied multivariate decoding analyses to uncover the temporal profile underlying the neural representations of the memorized item. Analysis of gaze position however revealed that our results were contaminated by systematic eye movements, suggesting that the MEG decoding results from our originally planned analyses were confounded. In addition to the eye movement analyses, we also present the original analyses to highlight how these might have readily led to invalid conclusions. Finally, we demonstrate a potential remedy, whereby we train the decoders on a functional localizer that was specifically designed to target bottom-up sensory signals and as such avoids eye movements. We conclude by arguing for more awareness of the potentially pervasive and ubiquitous effects of eye movement-related confounds.

Hide and seek: Directing top-down attention is not sufficient for accelerating conscious access

At any moment in time, we have a single conscious visual experience representing a minute part of our visual world. As such, the visual input stimulating our retinae is in continuous competition for reaching conscious access. Many complex cognitive operations can only be applied to consciously accessible visual information, thereby raising the question whether humans have the ability to select which parts of their visual input reaches consciousness. Top-down attention allows humans to flexibly assign more processing resources to certain parts of our visual input, making it a likely mechanism to volitionally bias conscious access. Here, we investigated whether directing top-down attention to a particular location or feature accelerates conscious access of an initially suppressed visual stimulus at the attended location, or of the attended feature. We instructed participants to attend a spatial location (Experiment 1) or color (Experiment 2) for a speeded discrimination task, using a highly predictive cue. The predictive cues were highly effective in prompting sustained attention towards the cued location or color, as evidenced by faster discrimination of cued relative to uncued targets. We simultaneously measured detection times to interocularly suppressed probes that were either of the cued (i.e., attended) color/location or not, and were visually distinct from the targets used for the discrimination task. Despite our successful manipulation of top-down attention, suppressed probes were not released from suppression faster when they were presented at the attended location, or in the attended color. In contrast, when observers were cued to attend a color for locating targets of an ill-defined shape (inciting a broader attentional template), we did observe faster conscious access of probes in the attended color (Experiment 3). We discuss our findings in light of the specificity of attentional templates, and the inherent limitations that this poses for top-down attentional biases on conscious access.

Evaluating the Role of the Dorsolateral Prefrontal Cortex and Posterior Parietal Cortex in Memory-Guided Attention With Repetitive Transcranial Magnetic Stimulation

The contents of working memory (WM) can affect the subsequent visual search performance, resulting in either beneficial or cost effects, when the visual search target is included in or spatially dissociated from the memorized contents, respectively. The right dorsolateral prefrontal cortex (rDLPFC) and the right posterior parietal cortex (rPPC) have been suggested to be associated with the congruence/incongruence effects of the WM content and the visual search target. Thus, in the present study, we investigated the role of the dorsolateral prefrontal cortex and the PPC in controlling the interaction between WM and attention during a visual search, using repetitive transcranial magnetic stimulation (rTMS). Subjects maintained a color in WM while performing a search task. The color cue contained the target (valid), the distractor (invalid) or did not reappear in the search display (neutral). Concurrent stimulation with the search onset showed that relative to rTMS over the vertex, rTMS over rPPC and rDLPFC further decreased the search reaction time, when the memory cue contained the search target. The results suggest that the rDLPFC and the rPPC are critical for controlling WM biases in human visual attention.

Sharpening of Hierarchical Visual Feature Representations of Blurred Images

The robustness of the visual system lies in its ability to perceive degraded images. This is achieved through interacting bottom-up, recurrent, and top-down pathways that process the visual input in concordance with stored prior information. The interaction mechanism by which they integrate visual input and prior information is still enigmatic. We present a new approach using deep neural network (DNN) representation to reveal the effects of such integration on degraded visual inputs. We transformed measured human brain activity resulting from viewing blurred images to the hierarchical representation space derived from a feedforward DNN. Transformed representations were found to veer toward the original nonblurred image and away from the blurred stimulus image. This indicated deblurring or sharpening in the neural representation, and possibly in our perception. We anticipate these results will help unravel the interplay mechanism between bottom-up, recurrent, and top-down pathways, leading to more comprehensive models of vision.

Visuospatial Working Memory as a Fundamental Component of the Eye Movement System

Humans make frequent movements of the eyes (saccades) to explore the visual environment. Here we argue that visuo-spatial working memory (VSWM) is a fundamental component of the eye movement system. Memory representations in VSWM are functionally integrated at all stages of orienting, from selection of the target, to maintenance of visual features across the saccade, to processes supporting the experience of perceptual continuity after the saccade, to the correction of gaze when the eyes fail to land on the intended object. VSWM is finely tuned to meet the challenges of active vision.

Pupil Dilation Reflects Task Relevance Prior to Search

When observers search for a specific target, it is assumed that they activate a representation of the task relevant object in visual working memory (VWM). This representation – often referred to as the template – guides attention towards matching visual input. In two experiments we tested whether the pupil response can be used to differentiate stimuli that match the task-relevant template from irrelevant input. Observers memorized a target color to be searched for in a multi-color visual search display, presented after a delay period. In Experiment 1, one color appeared at the start of the trial, which was then automatically the search template. In Experiments 2, two colors were presented, and a retro-cue indicated which of these was relevant for the upcoming search task. Crucially, before the search display appeared, we briefly presented one colored probe stimulus. The probe could match either the relevant-template color, the non-cued color (irrelevant), or be a new color not presented in the trial. We measured the pupil response to the probe as a signature of task relevance. Experiment 1 showed significantly smaller pupil size in response to probes matching the search template than for irrelevant colors. Experiment 2 replicated the template matching effect and allowed us to rule out that it was solely due to repetition priming. Taken together, we show that the pupil responds selectively to participants’ target template prior to search.

Neural Representation of Working Memory Content Is Modulated by Visual Attentional Demand

Recent theories assert that visual working memory (WM) relies on the same attentional resources and sensory substrates as visual attention to external stimuli. Behavioral studies have observed competitive tradeoffs between internal (i.e., WM) and external (i.e., visual) attentional demands, and neuroimaging studies have revealed representations of WM content as distributed patterns of activity within the same cortical regions engaged by perception of that content. Although a key function of WM is to protect memoranda from competing input, it remains unknown how neural representations of WM content are impacted by incoming sensory stimuli and concurrent attentional demands. Here, we investigated how neural evidence for WM information is affected when attention is occupied by visual search-at varying levels of difficulty-during the delay interval of a WM match-to-sample task. Behavioral and fMRI analyses suggested that WM maintenance was impacted by the difficulty of a concurrent visual task. Critically, multivariate classification analyses of category-specific ventral visual areas revealed a reduction in decodable WM-related information when attention was diverted to a visual search task, especially when the search was more difficult. This study suggests that the amount of available attention during WM maintenance influences the detection of sensory WM representations.