Neural Representation of Working Memory Content Is Modulated by Visual Attentional Demand

Attention influences the effects of the previous form orientation on the current motion direction estimation

Recent studies have found that the estimates of motion directions are biased toward the previous form orientations, showing serial dependence, and the serial dependence does not involve cognitive abilities. In the current study, we conducted two experiments to investigate whether and how attention-a cognitive ability-affected the serial dependence. The results showed that serial dependence was present in the current study, reproducing the previous findings. Importantly, when the attentional load reduced the reliability (i.e., estimation accuracy and precision) of previous form orientations (Experiment 1), the serial dependence decreased, meaning that the biases of motion direction estimates toward previous form orientations were reduced; in contrast, when the attentional load reduced the reliability of current motion directions (Experiment 2), the serial dependence increased, meaning that the biases of motion direction estimates toward previous form orientations were increased. These trends were well consistent with the prediction of the Bayesian inference theory. Therefore, the current study revealed the involvement of attention in the serial dependence of current motion direction estimation on the previous form orientation, demonstrating that the serial dependence was cognitive and the attentional effect can be a Bayesian inference process, initially revealing its computational mechanism.

The role of middle frontal gyrus in working memory retrieval by the effect of target detection tasks: a simultaneous EEG-fMRI study

Maintained working memory (WM) representations have been shown to influence visual target detection selection, while the effect of the visual target detection process on WM retrieval remains largely unknown. In the current research, we used the dual-paradigm of the visual target detection task and the delayed matching task (DMT), which contained the following four conditions: the match condition: the DMT target contained the detection target; the mismatch condition: the DMT target contained the detection distractor; the neutral condition: only the detection target was presented; the catch condition: only the DMT target was presented. Twenty-six subjects were recruited in the experiment with simultaneous EEG-fMRI data. Behaviorally, faster responses were found in the mismatch condition than in the match and neutral conditions. The EEG data found a greater parieto-occipital N1 component in the mismatch condition compared to the neutral condition, and a greater frontal N2 component in the match condition than in the mismatch condition. Moreover, compared to the match and neutral conditions, weaker activations of the bilateral middle frontal gyrus (MFG) were observed in the mismatch condition. And the representational similarity analysis (RSA) revealed significant differences in the representational patterns of the bilateral MFG between mismatch and match conditions, as well as in the representational patterns of the left MFG between mismatch and neutral conditions. Additionally, the left MFG may be the brain source of the N1 component in the mismatch condition. These findings suggest that the mismatch between the DMT target and detection target affects early attention allocation and attentional control in WM retrieval, and the MFG may play an important role in WM retrieval by the effect of the target detection task. In conclusion, our work deepens the understanding of the neural mechanisms by which visual target detection affects WM retrieval.

Decoding of Working Memory Contents in Auditory Cortex Is Not Distractor-Resistant

Working memory enables the temporary storage of relevant information in the service of behavior. Neuroimaging studies have suggested that sensory cortex is involved in maintaining contents in working memory. This raised the question of how sensory regions maintain memory representations during the exposure to distracting stimuli. Multivariate pattern analysis of fMRI signals in visual cortex has shown that the contents of visual working memory could be decoded concurrently with passively viewed distractors. The present fMRI study tested whether this finding extends to auditory working memory and to active distractor processing. We asked participants to memorize the pitch of a target sound and to compare it with a probe sound presented after a 13 s delay period. In separate conditions, we compared a blank delay phase (no distraction) with either passive listening to, or active processing of, an auditory distractor presented throughout the memory delay. Consistent with previous reports, pitch-specific memory information could be decoded in auditory cortex during the delay in trials without distraction. In contrast, decoding of target sounds in early auditory cortex dropped to chance level during both passive and active distraction. This was paralleled by memory performance decrements under distraction. Extending the analyses beyond sensory cortex yielded some evidence for memory content-specific activity in inferior frontal and superior parietal cortex during active distraction. In summary, while our findings question the involvement of early auditory cortex in the maintenance of distractor-resistant working memory contents, further research should elucidate the role of hierarchically higher regions.SIGNIFICANCE STATEMENT Information about sensory features held in working memory can be read out from hemodynamic activity recorded in human sensory cortices. Moreover, visual cortex can in parallel store visual content and process newly incoming, task-irrelevant visual input. The present study investigated the role of auditory cortex for working memory maintenance under distraction. While memorized sound frequencies could be decoded in auditory cortex in the absence of distraction, auditory distraction during the delay phase impaired memory performance and prevented decoding of information stored in working memory. Apparently, early auditory cortex is not sufficient to represent working memory contents under distraction that impairs performance. However, exploratory analyses indicated that, under distraction, higher-order frontal and parietal regions might contribute to content-specific working memory storage.

The ERP correlates of color-based center-surround inhibition in working memory

The color-based center-surround inhibition (CSI) in working memory (WM) refers to that remembering a color inhibits the memory of similar colors but not of distinct colors. This study aimed to investigate the neural activity of color-based CSI in WM. Two WM items (distance 0°, 10°, 20°, 30°, 40°, 50°, or 60° in color space) were displayed sequentially, then one of them was retrieved to compare with a later probe. Behavioral results revealed that participants showed longer RTs for distances 20° and 30° than distances 0° and 40°, suggesting a CSI between similar items. ERP results revealed that: 1) WM item-induced late positive component (LPC) was more positive for distance 30° than the other distances, suggesting an enhanced resource allocation process for encoding similar items; 2) Cue-induced LPC was more positive for distances 20° and 30° than distances 0° and 60°, suggesting a greater difficulty for retrieving similar items; Cue-induced contingent negative variation was less negative for distance 20° than distances 40°, 50°, and 60°, suggesting a reduced response preparation process during retrieving similar items; 3) Probe-induced LPC was more positive for distances 20° and 30° than distances 50° and 60°, suggesting a greater effort for comparing probe with one item retrieved from two similar items. These results revealed a colored-based CSI during WM encoding and retrieval processes. An enhanced top-down control might be required to resolve the greater interference between similar items than identical or distinct items conditions.

Distinct monitoring strategies underlie costs and performance in prospective memory

Prospective memory (PM) describes the ability to remember to perform goal-relevant actions at an appropriate time in the future amid concurrent demands. A key contributor to PM performance is thought to be the effortful monitoring of the environment for PM-related cues, a process whose existence is typically inferred from a behavioral interference measure of reaction times. This measure, referred to as "PM costs," is an informative but indirect proxy for monitoring, and it may not be sufficient to understand PM behaviors in all situations. In this study, we asked participants to perform a visual search task with arrows that varied in difficulty while concurrently performing a delayed-recognition PM task with pictures of faces and scenes. To gain a precise measurement of monitoring behavior, we used eye-tracking to record fixations to all task-relevant stimuli and related these fixation measures to both PM costs and PM accuracy. We found that PM costs reflected dissociable monitoring strategies: higher costs were associated with early and frequent monitoring while lower costs were associated with delayed and infrequent monitoring. Moreover, the link between fixations and PM costs varied with cognitive load, and the inclusion of fixation data yielded better predictions of PM accuracy than using PM costs alone. This study demonstrates the benefit of eye-tracking to disentangle the nature of PM costs and more precisely describe strategies involved in prospective remembering.

Irrelevant features of distractors in intervening visual search tasks cause active visual working memory interference - the more difficult the search task, the more interference it causes

Visual working memory (VWM) content disrupts visual search performance when there is a singleton in the search array that is similar to the content in VWM, even when this singleton is task irrelevant. Typically, the memory-similar singleton captures attention, which results in slower search performance for memory-similar conditions compared to conditions where memory-similar content is absent. Recently, it has also been shown that VWM content may be affected when memory-similar stimuli are processed. Specifically, it appears that VWM representations bias toward memory-similar information that is processed but not memory-dissimilar information. Here, we test whether the bias caused by processing memory-similar information is an active interference process (growing with engagement with the memory-similar stimuli) or a passive interference process (indifferent to the engagement with memory-similar stimuli). To test this, observers were tasked with memorizing a single color followed by a search task. The search task was either easy or difficult, and the search items could either be memory-similar or memory-dissimilar. Critically, the target in the search task was defined by its shape, so the color of the search items was irrelevant to the search task. At the end of each trial, participants reported the color in memory using a continuous report color wheel. The results showed that VWM representations drifted towards the irrelevant color of the search items in the memory-similar conditions, and this effect was larger in the difficult search condition. The results provide evidence that VWM representations receive active interference from processing memory-similar stimuli.

A method to adjust a prior distribution in Bayesian second-level fMRI analysis

Previous research has shown the potential value of Bayesian methods in fMRI (functional magnetic resonance imaging) analysis. For instance, the results from Bayes factor-applied second-level fMRI analysis showed a higher hit rate compared with frequentist second-level fMRI analysis, suggesting greater sensitivity. Although the method reported more positives as a result of the higher sensitivity, it was able to maintain a reasonable level of selectivity in term of the false positive rate. Moreover, employment of the multiple comparison correction method to update the default prior distribution significantly improved the performance of Bayesian second-level fMRI analysis. However, previous studies have utilized the default prior distribution and did not consider the nature of each individual study. Thus, in the present study, a method to adjust the Cauchy prior distribution based on a priori information, which can be acquired from the results of relevant previous studies, was proposed and tested. A Cauchy prior distribution was adjusted based on the contrast, noise strength, and proportion of true positives that were estimated from a meta-analysis of relevant previous studies. In the present study, both the simulated images and real contrast images from two previous studies were used to evaluate the performance of the proposed method. The results showed that the employment of the prior adjustment method resulted in improved performance of Bayesian second-level fMRI analysis.

Selective attention to stimulus representations in perception and memory: commonalities and differences

It has been proposed that the deployment of selective attention to perceptual and memory representations might be governed by similar cognitive processes and neural resources. However, evidence for this simple and appealing proposal remains inconclusive, which might be due to a considerable divergence in tasks and cognitive demands when comparing attentional selection in memory versus perception. To examine whether selection in both domains share common attentional processes and only differ in the stimuli they act upon (external vs. internal), we compared behavioral costs or benefits between selection domains. In both domains, participants had to attend a target stimulus from a set of simultaneously presented stimuli or simultaneously active memory representations, respectively, with set, target, or both, being repeated or changed across trials. The results of two experiments delineated principal similarities and differences of selection processes in both domains: While positive priming from stimulus repetition was found in both selection domains, we found no consistent effects of negative priming when shifting the focus of attention to a previously to-be-ignored stimulus. However, priming in the perception task was mainly due to repetitions of the target feature (here: color), whereas for the memory task, repetition of the same set of stimulus representations was most important. We propose that the differences can be attributed to a reduced cognitive effort when the now relevant memory representation had already been pre-activated (even as a distractor) in the previous trial. Additionally, our experiments both underscore the importance of taking stimulus-response associations into account, which may be a hidden factor behind differences between domains. We conclude that any attempt of comparing internal versus external attentional selection has to consider inherent differences in selection dynamics across representational domains.

Is the Free-Energy Principle a Formal Theory of Semantics? From Variational Density Dynamics to Neural and Phenotypic Representations

The aim of this paper is twofold: (1) to assess whether the construct of neural representations plays an explanatory role under the variational free-energy principle and its corollary process theory, active inference; and (2) if so, to assess which philosophical stance—in relation to the ontological and epistemological status of representations—is most appropriate. We focus on non-realist (deflationary and fictionalist-instrumentalist) approaches. We consider a deflationary account of mental representation, according to which the explanatorily relevant contents of neural representations are mathematical, rather than cognitive, and a fictionalist or instrumentalist account, according to which representations are scientifically useful fictions that serve explanatory (and other) aims. After reviewing the free-energy principle and active inference, we argue that the model of adaptive phenotypes under the free-energy principle can be used to furnish a formal semantics, enabling us to assign semantic content to specific phenotypic states (the internal states of a Markovian system that exists far from equilibrium). We propose a modified fictionalist account—an organism-centered fictionalism or instrumentalism. We argue that, under the free-energy principle, pursuing even a deflationary account of the content of neural representations licenses the appeal to the kind of semantic content involved in the ‘aboutness’ or intentionality of cognitive systems; our position is thus coherent with, but rests on distinct assumptions from, the realist position. We argue that the free-energy principle thereby explains the aboutness or intentionality in living systems and hence their capacity to parse their sensory stream using an ontology or set of semantic factors.

Cognitive Flexibility Improves Memory for Delayed Intentions

The ability to delay the execution of a goal until the appropriate time, prospective memory (PM), can be supported by the following two different cognitive control strategies: proactive control involving working memory maintenance of the goal and active monitoring of the environment; or reactive control relying on timely retrieval of goal information from episodic memory. Certain situations tend to favor each strategy, but the manner in which individuals adjust their strategy in response to changes in the environment is unknown. Across two experiments, human participants performed a delayed-recognition PM task embedded in an ongoing visual search task that fluctuated in difficulty. A control strategy was identified from moment to moment using reaction time costs and fMRI measures of goal maintenance. We found that people fluidly modified control strategies in accordance with changes in task demands (e.g., shifting toward proactive control when task difficulty decreased). This cognitive flexibility proved adaptive as it was associated with improved PM performance.

Electrophysiological evidence supports the role of sustained visuospatial attention in maintaining visual WM contents

Recent empirical and theoretical work suggests that there is a close relationship between visual working memory (WM) and visuospatial attention. Here, we investigated whether visuospatial attention was involved in maintaining object representations in visual WM. To this end, the alpha lateralization and contralateral delay activity (CDA) were analyzed as neural markers for visuospatial attention and visual WM storage, respectively. In the single-task condition, participants performed a grating change-detection task. To probe the role of visuospatial attention in maintaining WM contents, two color squares were presented above and below the fixation point during the retention interval, which remained visible until the detection display was present. In the dual-task condition, participants were required to maintain lateralized gratings while staring at the center-presented color squares, to detect possible subsequent color change. With this task, sustained visuospatial attention that guided to individual memory representations was disrupted. The behavioral data showed that, the insertion of secondary task significantly deteriorated WM performance. For electrophysiological data, we divided the retention interval into two stages, the early stage and late stage, bounded by the onset of the secondary task. We found that CDA amplitude was lower under the dual-task condition than the single-task condition during the late stage, but not the early stage, and the extent to which CDA reduced tracked the impaired memory performance at the individual level. Also, alpha lateralization only could be observed in the single-task condition of the late stage, and completely disappeared in the dual-task condition, indicating the disruption of visuospatial attention directed to memory representations. Individuals who experienced greater visuospatial attention disruption, as indicated by the alpha lateralization, had lower maintenance-associated neural activity (CDA), and suffered greater impairment of memory performance. These findings confirm that sustained visuospatial attention continues improving visual WM processing after the initial encoding phase, and most likely participates in this process by supporting the maintenance of representations in an active state.

EEG dynamics reveal a dissociation between storage and selective attention within working memory

Selective attention plays a prominent role in prioritizing information in working memory (WM), improving performance for attended representations. However, it remains unclear whether unattended WM representations suffer from information loss. Here we tested the hypothesis that within WM, selectively attending to an item and stopping storing other items are independent mechanisms. We recorded EEG while participants performed a WM recall task in which the item most likely to be tested was cued retrospectively during retention. By manipulating retro-cue reliability (i.e., the ratio of valid to invalid cue trials), we varied the incentive to retain non-cued items. Storage and selective attention in WM were measured during the retention interval by contralateral delay activity (CDA) and contralateral alpha power suppression, respectively. Soon after highly reliable cues, the cued item was attended, and non-cued items suffered information loss. However, for less reliable cues, initially the cued item was attended, but unattended items were kept in WM. Later during the delay, previously unattended items suffered information loss despite now attention being reallocated to their locations, presumably to strengthen their weakening traces. These results show that storage and attention in WM are distinct processes that can behave differently depending on the relative importance of representations.

We Need a Taxonomy of Working Memory

A vast and diverse literature describes the relationship between working memory and attention. That literature encompasses tests of the interactions between the two functions, comparisons of the processes underlying them, and theoretical formulations of the cognitive constructs. In a recent review, Oberauer (2019) reins in this varied work to create a roadmap for future research. Here, I delineate several additional considerations to guide the evaluation and development of research into working memory and attention. Namely, working memory is a complex construct that can entail many processes and take on several meanings. Research and theory about working memory-and its relation to attention-must consider what particular demands are being tapped, what type of information is being operated on, and what goal is ultimately at stake.

Working Memory and Attention - Response to Commentaries

This is a brief reply to the commentaries by Adam and deBettencourt (2019); Allen (2019); Kiyonaga (2019); Schneider (2019); and Van der Stigchel and Olivers (2019), focusing on four topics: (1) I defend the idea that attention need not be characterized as a limited resource. (2) I explain how I conceptualize the role of WM in cognitive control, and how recruitment of sensory processing networks contributes to control but not maintenance. (3) I discuss different ways in which information can be selectively prioritized during or after being encoded into WM, and the different consequences of these processes. (4) I argue that sustained attention to a task can be understood as the mind's ability to prioritize that task over task-unrelated representations.

Holistic Reinforcement Learning: The Role of Structure and Attention

Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.

Behavioral decoding of working memory items inside and outside the focus of attention

How we attend to our thoughts affects how we attend to our environment. Holding information in working memory can automatically bias visual attention toward matching information. By observing attentional biases on reaction times to visual search during a memory delay, it is possible to reconstruct the source of that bias using machine learning techniques and thereby behaviorally decode the content of working memory. Can this be done when more than one item is held in working memory? There is some evidence that multiple items can simultaneously bias attention, but the effects have been inconsistent. One explanation may be that items are stored in different states depending on the current task demands. Recent models propose functionally distinct states of representation for items inside versus outside the focus of attention. Here, we use behavioral decoding to evaluate whether multiple memory items-including temporarily irrelevant items outside the focus of attention-exert biases on visual attention. Only the single item in the focus of attention was decodable. The other item showed a brief attentional bias that dissipated until it returned to the focus of attention. These results support the idea of dynamic, flexible states of working memory across time and priority.