Shared mechanisms underlie the control of working memory and attention

Comparing Neural Correlates of Memory Encoding and Maintenance for Foveal and Peripheral Stimuli

Visual working memory is believed to rely on top-down attentional mechanisms that sustain active sensory representations in early visual cortex, a mechanism referred to as sensory recruitment. However, both bottom-up sensory input and top-down attentional modulations thereof appear to prioritize the fovea over the periphery, such that initially peripheral percepts may even be assimilated by foveal processes. This raises the question whether and how visual working memory differs for central and peripheral input. To address this, we conducted a delayed orientation recall task in which an orientation was presented either at the center of the screen or at 15° eccentricity to the left or right. Response accuracy, EEG activity, and gaze position were recorded from 30 participants. Accuracy was slightly but significantly higher for foveal versus peripheral memories. Decoding of EEG recordings revealed a clear dissociation between early sensory and later maintenance signals. Although sensory signals were clearly decodable for foveal stimuli, they were not for peripheral input. In contrast, maintenance signals were equally decodable for both foveal and peripheral memories, suggesting comparable top-down components regardless of eccentricity. Moreover, although memory representations were initially spatially specific and reflected in voltage fluctuations, later during the maintenance period, they generalized across locations, as emerged in alpha oscillations, thus revealing a dynamic transformation within memory from separate sensory traces to what we propose are common output-related codes. Furthermore, the combined absence of reliable decoding of sensory signals and robust presence of maintenance decoding indicates that storage activity patterns as measured by EEG reflect signals beyond primary visual cortex. We discuss the implications for the sensory recruitment hypothesis.

Internal Coupling: Eye Behavior Coupled to Visual Imagery

Our eyes do not only respond to visual perception but also to internal cognition involving visual imagery, which can be referred to as internal coupling. This review synthesizes evidence on internal coupling across diverse domains including episodic memory and simulation, visuospatial memory, numerical cognition, object movement, body movement, and brightness imagery. In each domain, eye movements consistently reflect distinct aspects of mental imagery typically akin to those seen in corresponding visual experiences. Several findings further suggest that internal coupling may not only coincide with but also supports internal cognition as evidenced by improved cognitive performance. Available theoretical accounts suggest that internal coupling may serve at least two functional roles in visual imagery: facilitating memory reconstruction and indicating shifts in internal attention. Moreover, recent insights into the neurobiology of internal coupling highlight substantially shared neural pathways in externally and internally directed cognition. The review concludes by identifying open questions and promising avenues for future research such as exploring moderating roles of context and individual differences in internal coupling.

The neural basis of swap errors in working memory

When making decisions in a cluttered world, humans and other animals often have to hold multiple items in memory at once-such as the different items on a shopping list. Psychophysical experiments in humans and other animals have shown remembered stimuli can sometimes become confused, with participants reporting chimeric stimuli composed of features from different stimuli. In particular, subjects will often make "swap errors" where they misattribute a feature from one object as belonging to another object. While swap errors have been described behaviorally and theoretical explanations have been proposed, their neural mechanisms are unknown. Here, we elucidate these neural mechanisms by analyzing neural population recordings from monkeys performing two multistimulus working memory tasks. In these tasks, monkeys were cued to report the color of an item that either was previously shown at a corresponding location or will be shown at the corresponding location. Animals made swap errors in both tasks. In the neural data, we find evidence that the neural correlates of swap errors emerged when correctly remembered information is selected from working memory. This led to a representation of the distractor color as if it were the target color, underlying the eventual swap error. We did not find consistent evidence that swap errors arose from misinterpretation of the cue or errors during encoding or storage in working memory. These results provide evidence that swap errors emerge during selection of correctly remembered information from working memory, and highlight this selection as a crucial-yet surprisingly brittle-neural process.

Prefrontal neuronal dynamics in the absence of task execution

Prefrontal cortical activity represents stimuli in working memory tasks in a low-dimensional manifold that transforms over the course of a trial. Such transformations reflect specific cognitive operations, so that, for example, the rotation of stimulus representations is thought to reduce interference by distractor stimuli. Here we show that rotations occur in the low-dimensional activity space of prefrontal neurons in naïve male monkeys (Macaca mulatta), while passively viewing familiar stimuli. Moreover, some aspects of these rotations remain remarkably unchanged after training to perform working memory tasks. Significant training effects are still present in population dynamics, which further distinguish correct and error trials during task execution. Our results reveal automatic functions of prefrontal neural circuits allow transformations that may aid cognitive flexibility.

Multi-timescale neural dynamics for multisensory integration

Carrying out any everyday task, be it driving in traffic, conversing with friends or playing basketball, requires rapid selection, integration and segregation of stimuli from different sensory modalities. At present, even the most advanced artificial intelligence-based systems are unable to replicate the multisensory processes that the human brain routinely performs, but how neural circuits in the brain carry out these processes is still not well understood. In this Perspective, we discuss recent findings that shed fresh light on the oscillatory neural mechanisms that mediate multisensory integration (MI), including power modulations, phase resetting, phase-amplitude coupling and dynamic functional connectivity. We then consider studies that also suggest multi-timescale dynamics in intrinsic ongoing neural activity and during stimulus-driven bottom-up and cognitive top-down neural network processing in the context of MI. We propose a new concept of MI that emphasizes the critical role of neural dynamics at multiple timescales within and across brain networks, enabling the simultaneous integration, segregation, hierarchical structuring and selection of information in different time windows. To highlight predictions from our multi-timescale concept of MI, real-world scenarios in which multi-timescale processes may coordinate MI in a flexible and adaptive manner are considered.

Neural representational geometries reflect behavioral differences in monkeys and recurrent neural networks

Animals likely use a variety of strategies to solve laboratory tasks. Traditionally, combined analysis of behavioral and neural recording data across subjects employing different strategies may obscure important signals and give confusing results. Hence, it is essential to develop techniques that can infer strategy at the single-subject level. We analyzed an experiment in which two male monkeys performed a visually cued rule-based task. The analysis of their performance shows no indication that they used a different strategy. However, when we examined the geometry of stimulus representations in the state space of the neural activities recorded in dorsolateral prefrontal cortex, we found striking differences between the two monkeys. Our purely neural results induced us to reanalyze the behavior. The new analysis showed that the differences in representational geometry are associated with differences in the reaction times, revealing behavioral differences we were unaware of. All these analyses suggest that the monkeys are using different strategies. Finally, using recurrent neural network models trained to perform the same task, we show that these strategies correlate with the amount of training, suggesting a possible explanation for the observed neural and behavioral differences.

Mixed selectivity: Cellular computations for complexity

The property of mixed selectivity has been discussed at a computational level and offers a strategy to maximize computational power by adding versatility to the functional role of each neuron. Here, we offer a biologically grounded implementational-level mechanistic explanation for mixed selectivity in neural circuits. We define pure, linear, and nonlinear mixed selectivity and discuss how these response properties can be obtained in simple neural circuits. Neurons that respond to multiple, statistically independent variables display mixed selectivity. If their activity can be expressed as a weighted sum, then they exhibit linear mixed selectivity; otherwise, they exhibit nonlinear mixed selectivity. Neural representations based on diverse nonlinear mixed selectivity are high dimensional; hence, they confer enormous flexibility to a simple downstream readout neural circuit. However, a simple neural circuit cannot possibly encode all possible mixtures of variables simultaneously, as this would require a combinatorially large number of mixed selectivity neurons. Gating mechanisms like oscillations and neuromodulation can solve this problem by dynamically selecting which variables are mixed and transmitted to the readout.

Is working memory domain-general or domain-specific?

Given the fundamental role of working memory (WM) in all domains of cognition, a central question has been whether WM is domain-general. However, the term 'domain-general' has been used in different, and sometimes misleading, ways. By reviewing recent evidence and biologically plausible models of WM, we show that the level of domain-generality varies substantially between three facets of WM: in terms of computations, WM is largely domain-general. In terms of neural correlates, it contains both domain-general and domain-specific elements. Finally, in terms of application, it is mostly domain-specific. This variance encourages a shift of focus towards uncovering domain-general computational principles and away from domain-general approaches to the analysis of individual differences and WM training, favoring newer perspectives, such as training-as-skill-learning.

Effects of CACNA1C and ANK3 on cognitive function in patients with bipolar disorder

Bipolar disorder (BD) is a complex, severe mental illness with cognitive impairment. Impairments in attention and memory are particularly evident. A large number of previous studies have identified CACNA1C and ANK3 gene variants as risk factors for BD and both affect cognitive function in people with BD. However, it is unclear whether there is an interaction effects between the two genes on cognitive impairment in patients. We used 153 Chinese Han Chinese patients with BD to explore the association of CACNA1C and ANK3 variants with attention and immediate memory using Plink software and and performed a epistatic interaction effects analysis. We found that CACNA1C and ANK3 gene variants respectively affected patients' scores on attention and memory tests. The significant SNP in the CACNA1C and ANK3 genes are rs73042126(P = 3.16 × 10-5,FDR = 0.0253) and rs2393640(P = 1.50 × 10-4,FDR = 0.0353) respectively. And they also interacted to affect cognitive functioning in BD patients (attention: P = 0.0289; immediate memory: P = 0.0398). Follow-up studies should increase the sample size, improve the assessment methods and experimental design, and further explore the pathogenic mechanisms of BD.

Goal-directed attention transforms both working and long-term memory representations in the human parietal cortex

The abundance of distractors in the world poses a major challenge to our brain's limited processing capacity, but little is known about how selective attention modulates stimulus representations in the brain to reduce interference and support durable target memory. Here, we collected functional magnetic resonance imaging (fMRI) data in a selective attention task in which target and distractor pictures of different visual categories were simultaneously presented. Participants were asked to selectively process the target according to the effective cue, either before the encoding period (i.e., perceptual attention) or the maintenance period (i.e., reflective attention). On the next day, participants were asked to perform a memory recognition task in the scanner in which the targets, distractors, and novel items were presented in a pseudorandom order. Behavioral results showed that perceptual attention was better at enhancing target memory and reducing distractor memory than reflective attention, although the overall memory capacity (memory for both target and distractor) was comparable. Using multiple-voxel pattern analysis of the neural data, we found more robust target representation and weaker distractor representation in working memory for perceptual attention than for reflective attention. Interestingly, perceptual attention partially shifted the regions involved in maintaining the target representation from the visual cortex to the parietal cortex. Furthermore, the targets and distractors simultaneously presented in the perceptual attention condition showed reduced pattern similarity in the parietal cortex during retrieval compared to items not presented together. This neural pattern repulsion positively correlated with individuals' recognition of both targets and distractors. These results emphasize the critical role of selective attention in transforming memory representations to reduce interference and improve long-term memory performance.

The computational foundations of dynamic coding in working memory

Working memory (WM) is a fundamental aspect of cognition. WM maintenance is classically thought to rely on stable patterns of neural activities. However, recent evidence shows that neural population activities during WM maintenance undergo dynamic variations before settling into a stable pattern. Although this has been difficult to explain theoretically, neural network models optimized for WM typically also exhibit such dynamics. Here, we examine stable versus dynamic coding in neural data, classical models, and task-optimized networks. We review principled mathematical reasons for why classical models do not, while task-optimized models naturally do exhibit dynamic coding. We suggest an update to our understanding of WM maintenance, in which dynamic coding is a fundamental computational feature rather than an epiphenomenon.

Timescales of learning in prefrontal cortex

The lateral prefrontal cortex (PFC) in humans and other primates is critical for immediate, goal-directed behaviour and working memory, which are classically considered distinct from the cognitive and neural circuits that support long-term learning and memory. Over the past few years, a reconsideration of this textbook perspective has emerged, in that different timescales of memory-guided behaviour are in constant interaction during the pursuit of immediate goals. Here, we will first detail how neural activity related to the shortest timescales of goal-directed behaviour (which requires maintenance of current states and goals in working memory) is sculpted by long-term knowledge and learning - that is, how the past informs present behaviour. Then, we will outline how learning across different timescales (from seconds to years) drives plasticity in the primate lateral PFC, from single neuron firing rates to mesoscale neuroimaging activity patterns. Finally, we will review how, over days and months of learning, dense local and long-range connectivity patterns in PFC facilitate longer-lasting changes in population activity by changing synaptic weights and recruiting additional neural resources to inform future behaviour. Our Review sheds light on how the machinery of plasticity in PFC circuits facilitates the integration of learned experiences across time to best guide adaptive behaviour.

Never-Ending Learning for Explainable Brain Computing

Exploring the nature of human intelligence and behavior is a longstanding pursuit in cognitive neuroscience, driven by the accumulation of knowledge, information, and data across various studies. However, achieving a unified and transparent interpretation of findings presents formidable challenges. In response, an explainable brain computing framework is proposed that employs the never-ending learning paradigm, integrating evidence combination and fusion computing within a Knowledge-Information-Data (KID) architecture. The framework supports continuous brain cognition investigation, utilizing joint knowledge-driven forward inference and data-driven reverse inference, bolstered by the pre-trained language modeling techniques and the human-in-the-loop mechanisms. In particular, it incorporates internal evidence learning through multi-task functional neuroimaging analyses and external evidence learning via topic modeling of published neuroimaging studies, all of which involve human interactions at different stages. Based on two case studies, the intricate uncertainty surrounding brain localization in human reasoning is revealed. The present study also highlights the potential of systematization to advance explainable brain computing, offering a finer-grained understanding of brain activity patterns related to human intelligence.

Representing Context and Priority in Working Memory

The ability to prioritize among contents in working memory (WM) is critical for successful control of thought and behavior. Recent work has demonstrated that prioritization in WM can be implemented by representing different states of priority in different representational formats. Here, we explored the mechanisms underlying WM prioritization by simulating the double serial retrocuing task with recurrent neural networks. Visualization of stimulus representational dynamics using principal component analysis revealed that the network represented trial context (order of presentation) and priority via different mechanisms. Ordinal context, a stable property lasting the duration of the trial, was accomplished by segregating representations into orthogonal subspaces. Priority, which changed multiple times during a trial, was accomplished by separating representations into different strata within each subspace. We assessed the generality of these mechanisms by applying dimensionality reduction and multiclass decoding to fMRI and EEG data sets and found that priority and context are represented differently along the dorsal visual stream and that behavioral performance is sensitive to trial-by-trial variability of priority coding, but not context coding.

Nonfrontal Control of Working Memory

Items held in visual working memory can be quickly updated, replaced, removed, and even manipulated in accordance with current behavioral goals. Here, we use multivariate pattern analyses to identify the patterns of neuronal activity that realize the executive control processes supervising these flexible stores. We find that portions of the middle temporal gyrus and the intraparietal sulcus represent what item is cued for continued memorization independently of representations of the item itself. Importantly, this selection-specific activity could not be explained by sensory representations of the cue and is only present when control is exerted. Our results suggest that the selection of memorized items might be controlled in a distributed and decentralized fashion. This evidence provides an alternative perspective to the notion of "domain general" central executive control over memory function.

Same, Same but Different? A Multi-Method Review of the Processes Underlying Executive Control

Attention, working memory, and executive control are commonly considered distinct cognitive functions with important reciprocal interactions. Yet, longstanding evidence from lesion studies has demonstrated both overlap and dissociation in their behavioural expression and anatomical underpinnings, suggesting that a lower dimensional framework could be employed to further identify processes supporting goal-directed behaviour. Here, we describe the anatomical and functional correspondence between attention, working memory, and executive control by providing an overview of cognitive models, as well as recent data from lesion studies, invasive and non-invasive multimodal neuroimaging and brain stimulation. We emphasize the benefits of considering converging evidence from multiple methodologies centred on the identification of brain mechanisms supporting goal-driven behaviour. We propose that expanding on this approach should enable the construction of a comprehensive anatomo-functional framework with testable new hypotheses, and aid clinical neuroscience to intervene on impairments of executive functions.

Abstract deliberation by visuomotor neurons in prefrontal cortex

During visually guided behavior, the prefrontal cortex plays a pivotal role in mapping sensory inputs onto appropriate motor plans. When the sensory input is ambiguous, this involves deliberation. It is not known whether the deliberation is implemented as a competition between possible stimulus interpretations or between possible motor plans. Here we study neural population activity in the prefrontal cortex of macaque monkeys trained to flexibly report perceptual judgments of ambiguous visual stimuli. We find that the population activity initially represents the formation of a perceptual choice before transitioning into the representation of the motor plan. Stimulus strength and prior expectations both bear on the formation of the perceptual choice, but not on the formation of the action plan. These results suggest that prefrontal circuits involved in action selection are also used for the deliberation of abstract propositions divorced from a specific motor plan, thus providing a crucial mechanism for abstract reasoning.

Jointly looking to the past and the future in visual working memory

Working memory enables us to bridge past sensory information to upcoming future behaviour. Accordingly, by its very nature, working memory is concerned with two components: the past and the future. Yet, in conventional laboratory tasks, these two components are often conflated, such as when sensory information in working memory is encoded and tested at the same location. We developed a task in which we dissociated the past (encoded location) and future (to-be-tested location) attributes of visual contents in working memory. This enabled us to independently track the utilisation of past and future memory attributes through gaze, as observed during mnemonic selection. Our results reveal the joint consideration of past and future locations. This was prevalent even at the single-trial level of individual saccades that were jointly biased to the past and future. This uncovers the rich nature of working memory representations, whereby both past and future memory attributes are retained and can be accessed together when memory contents become relevant for behaviour.

The meso-connectomes of mouse, marmoset, and macaque: network organization and the emergence of higher cognition

The recent publications of the inter-areal connectomes for mouse, marmoset, and macaque cortex have allowed deeper comparisons across rodent vs. primate cortical organization. In general, these show that the mouse has very widespread, "all-to-all" inter-areal connectivity (i.e. a "highly dense" connectome in a graph theoretical framework), while primates have a more modular organization. In this review, we highlight the relevance of these differences to function, including the example of primary visual cortex (V1) which, in the mouse, is interconnected with all other areas, therefore including other primary sensory and frontal areas. We argue that this dense inter-areal connectivity benefits multimodal associations, at the cost of reduced functional segregation. Conversely, primates have expanded cortices with a modular connectivity structure, where V1 is almost exclusively interconnected with other visual cortices, themselves organized in relatively segregated streams, and hierarchically higher cortical areas such as prefrontal cortex provide top-down regulation for specifying precise information for working memory storage and manipulation. Increased complexity in cytoarchitecture, connectivity, dendritic spine density, and receptor expression additionally reveal a sharper hierarchical organization in primate cortex. Together, we argue that these primate specializations permit separable deconstruction and selective reconstruction of representations, which is essential to higher cognition.

Natural language instructions induce compositional generalization in networks of neurons

A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.

Orthogonal neural encoding of targets and distractors supports multivariate cognitive control

The complex challenges of our mental life require us to coordinate multiple forms of neural information processing. Recent behavioural studies have found that people can coordinate multiple forms of attention, but the underlying neural control process remains obscure. We hypothesized that the brain implements multivariate control by independently monitoring feature-specific difficulty and independently prioritizing feature-specific processing. During functional MRI, participants performed a parametric conflict task that separately tags target and distractor processing. Consistent with feature-specific monitoring, univariate analyses revealed spatially segregated encoding of target and distractor difficulty in the dorsal anterior cingulate cortex. Consistent with feature-specific attentional priority, our encoding geometry analysis revealed overlapping but orthogonal representations of target and distractor coherence in the intraparietal sulcus. Coherence representations were mediated by control demands and aligned with both performance and frontoparietal activity, consistent with top-down attention. Together, these findings provide evidence for the neural geometry necessary to coordinate multivariate cognitive control.

Causal functional maps of brain rhythms in working memory

Human working memory is a key cognitive process that engages multiple functional anatomical nodes across the brain. Despite a plethora of correlative neuroimaging evidence regarding the working memory architecture, our understanding of critical hubs causally controlling overall performance is incomplete. Causal interpretation requires cognitive testing following safe, temporal, and controllable neuromodulation of specific functional anatomical nodes. Such experiments became available in healthy humans with the advance of transcranial alternating current stimulation (tACS). Here, we synthesize findings of 28 placebo-controlled studies (in total, 1,057 participants) that applied frequency-specific noninvasive stimulation of neural oscillations and examined working memory performance in neurotypical adults. We use a computational meta-modeling method to simulate each intervention in realistic virtual brains and test reported behavioral outcomes against the stimulation-induced electric fields in different brain nodes. Our results show that stimulating anterior frontal and medial temporal theta oscillations and occipitoparietal gamma rhythms leads to significant dose-dependent improvement in working memory task performance. Conversely, prefrontal gamma modulation is detrimental to performance. Moreover, we found distinct spatial expression of theta subbands, where working memory changes followed orbitofrontal high-theta modulation and medial temporal low-theta modulation. Finally, all these results are driven by changes in working memory accuracy rather than processing time measures. These findings provide a fresh view of the working memory mechanisms, complementary to neuroimaging research, and propose hypothesis-driven targets for the clinical treatment of working memory deficits.

Internal attention modulates the functional state of novel stimulus-response associations in working memory

Information in working memory (WM) is crucial for guiding behavior. However, not all WM representations are equally relevant simultaneously. Current theoretical frameworks propose a functional dissociation between 'latent' and 'active' states, in which relevant representations are prioritized into an optimal (active) state to face current demands, while relevant information that is not immediately needed is maintained in a dormant (latent) state. In this context, task demands can induce rapid and flexible prioritization of information from latent to active state. Critically, these functional states have been primarily studied using simple visual memories, with attention selecting and prioritizing relevant representations to serve as templates to guide subsequent behavior. It remains unclear whether more complex WM representations, such as novel stimulus-response associations, can also be prioritized into different functional states depending on their task relevance, and if so how these different formats relate to each other. In the present study, we investigated whether novel WM-guided actions can be brought into different functional states depending on current task demands. Our results reveal that planned actions can be flexibly prioritized when needed and show how their functional state modulates their influence on ongoing behavior. Moreover, they suggest the representations of novel actions of different functional states are maintained in WM via a non-orthogonal coding scheme, thus are prone to interference.

Shifting attention between perception and working memory

Most everyday tasks require shifting the focus of attention between sensory signals in the external environment and internal contents in working memory. To date, shifts of attention have been investigated within each domain, but shifts between the external and internal domain remain poorly understood. We developed a combined perception and working-memory task to investigate and compare the consequences of shifting spatial attention within and between domains in the service of a common orientation-reproduction task. Participants were sequentially cued to attend to items either in working memory or to an upcoming sensory stimulation. Stay trials provided a baseline condition, while shift trials required participants to shift their attention to another item within the same or different domain. Validating our experimental approach, we found evidence that participants shifted attention effectively in either domain (Experiment 1). In addition, we observed greater costs when transitioning attention between as compared to within domains (Experiments 1, 2). Strikingly, these costs persisted even when participants were given more time to complete the attentional shift (Experiment 2). Biases in fixational gaze behaviour tracked attentional orienting in both domains, but revealed no latency or magnitude difference for within- versus between-domain shifts (Experiment 1). Collectively, the results from Experiments 1 and 2 suggest that shifting between attentional domains might be regulated by a unique control function. Our results break new ground for exploring the ubiquitous act of shifting attention between perception and working memory to guide adaptive behaviour in everyday cognition.

We know what attention is!

Attention is one of the most thoroughly investigated psychological phenomena, yet skepticism about attention is widespread: we do not know what it is, it is too many things, there is no such thing. The deficiencies highlighted are not about experimental work but the adequacy of the scientific theory of attention. Combining common scientific claims about attention into a single theory leads to internal inconsistency. This paper demonstrates that a specific functional conception of attention is incorporated into the tasks used in standard experimental paradigms. In accepting these paradigms as valid probes of attention, we commit to this common conception. The conception unifies work at multiple levels of analysis into a coherent scientific explanation of attention. Thus, we all know what attention is.

Causal phase-dependent control of non-spatial attention in human prefrontal cortex

Non-spatial attention is a fundamental cognitive mechanism that allows organisms to orient the focus of conscious awareness towards sensory information that is relevant to a behavioural goal while shifting it away from irrelevant stimuli. It has been suggested that attention is regulated by the ongoing phase of slow excitability fluctuations of neural activity in the prefrontal cortex, a hypothesis that has been challenged with no consensus. Here we developed a behavioural and non-invasive stimulation paradigm aiming at modulating slow excitability fluctuations of the inferior frontal junction. Using this approach, we show that non-spatial attention can be selectively modulated as a function of the ongoing phase of exogenously modulated excitability states of this brain structure. These results demonstrate that non-spatial attention relies on ongoing prefrontal excitability states, which are probably regulated by slow oscillatory dynamics, that orchestrate goal-oriented behaviour.

Perisaccadic and attentional remapping of receptive fields in lateral intraparietal area and frontal eye fields

The nature and function of perisaccadic receptive field (RF) remapping have been controversial. We use a delayed saccade task to reduce previous confounds and examine the remapping time course in the lateral intraparietal area and frontal eye fields. In the delay period, the RF shift direction turns from the initial fixation to the saccade target. In the perisaccadic period, RFs first shift toward the target (convergent remapping), but around the time of saccade onset/offset, the shifts become predominantly toward the post-saccadic RF locations (forward remapping). Thus, unlike forward remapping that depends on the corollary discharge (CD) of the saccade command, convergent remapping appears to follow attention from the initial fixation to the target. We model the data with attention-modulated and CD-gated connections and show that both sets of connections emerge automatically in neural networks trained to update stimulus retinal locations across saccades. Our work thus unifies previous findings into a mechanism for transsaccadic visual stability.

Building compositional tasks with shared neural subspaces

Cognition is remarkably flexible; we are able to rapidly learn and perform many different tasks1. Theoretical modeling has shown artificial neural networks trained to perform multiple tasks will re-use representations2 and computational components3 across tasks. By composing tasks from these sub-components, an agent can flexibly switch between tasks and rapidly learn new tasks4. Yet, whether such compositionality is found in the brain is unknown. Here, we show the same subspaces of neural activity represent task-relevant information across multiple tasks, with each task compositionally combining these subspaces in a task-specific manner. We trained monkeys to switch between three compositionally related tasks. Neural recordings found task-relevant information about stimulus features and motor actions were represented in subspaces of neural activity that were shared across tasks. When monkeys performed a task, neural representations in the relevant shared sensory subspace were transformed to the relevant shared motor subspace. Subspaces were flexibly engaged as monkeys discovered the task in effect; their internal belief about the current task predicted the strength of representations in task-relevant subspaces. In sum, our findings suggest that the brain can flexibly perform multiple tasks by compositionally combining task-relevant neural representations across tasks.

The influence of depth on object selection and manipulation in visual working memory within a 3D context

Recent studies have examined whether the internal selection mechanism functions similarly for perception and visual working memory (VWM). However, the process of how we access and manipulate object representations distributed in a 3D space remains unclear. In this study, we utilized a memory search task to investigate the effect of depth on object selection and manipulation within VWM. The memory display consisted of colored items half positioned at the near depth plane and the other half at the far plane. During memory maintenance, the participants were instructed to search for a target representation and update its color. The results showed that under object-based attention (Experiments 1, 3, and 5), the update time was faster for targets at the near plane than for those at the far plane. This effect was absent in VWM when deploying spatial attention (Experiment 2) and in visual search regardless of the type of attention deployed (Experiment 4). The differential effects of depth on spatial and object-based attention in VWM suggest that spatial attention primarily relied on 2D location information irrespective of depth, whereas object-based attention seemed to prioritize memory representations at the front plane before shifting to the back. Our findings shed light on the interaction between depth perception and the selection mechanisms within VWM in a 3D context, emphasizing the importance of ordinal, rather than metric, spatial information in guiding object-based attention in VWM.

Learning attentional templates for value-based decision-making

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed across tasks.

Dissociable neuronal substrates of visual feature attention and working memory

Attention and working memory (WM) are distinct cognitive functions, yet given their close interactions, it is often assumed that they share the same neuronal mechanisms. We show that in macaques performing a WM-guided feature attention task, the activity of most neurons in areas middle temporal (MT), medial superior temporal (MST), lateral intraparietal (LIP), and posterior lateral prefrontal cortex (LPFC-p) displays attentional modulation or WM coding and not both. One area thought to play a role in both functions is LPFC-p. To test this, we optogenetically inactivated LPFC-p bilaterally during different task periods. Attention period inactivation reduced attentional modulation in LPFC-p, MST, and LIP neurons and impaired task performance. In contrast, WM period inactivation did not affect attentional modulation or performance and minimally affected WM coding. Our results suggest that feature attention and WM have dissociable neuronal substrates and that LPFC-p plays a critical role in feature attention, but not in WM.

Rationality, preferences, and emotions with biological constraints: it all starts from our senses

Is the role of our sensory systems to represent the physical world as accurately as possible? If so, are our preferences and emotions, often deemed irrational, decoupled from these 'ground-truth' sensory experiences? We show why the answer to both questions is 'no'. Brain function is metabolically costly, and the brain loses some fraction of the information that it encodes and transmits. Therefore, if brains maximize objective functions that increase the fitness of their species, they should adapt to the objective-maximizing rules of the environment at the earliest stages of sensory processing. Consequently, observed 'irrationalities', preferences, and emotions stem from the necessity for our early sensory systems to adapt and process information while considering the metabolic costs and internal states of the organism.

Identifying Interpretable Latent Factors with Sparse Component Analysis

In many neural populations, the computationally relevant signals are posited to be a set of 'latent factors' - signals shared across many individual neurons. Understanding the relationship between neural activity and behavior requires the identification of factors that reflect distinct computational roles. Methods for identifying such factors typically require supervision, which can be suboptimal if one is unsure how (or whether) factors can be grouped into distinct, meaningful sets. Here, we introduce Sparse Component Analysis (SCA), an unsupervised method that identifies interpretable latent factors. SCA seeks factors that are sparse in time and occupy orthogonal dimensions. With these simple constraints, SCA facilitates surprisingly clear parcellations of neural activity across a range of behaviors. We applied SCA to motor cortex activity from reaching and cycling monkeys, single-trial imaging data from C. elegans, and activity from a multitask artificial network. SCA consistently identified sets of factors that were useful in describing network computations.

New Evidence for Retrospectively Cued Perception

Past research suggests a continuity between perception and memory, as reflected in influences of orienting of spatial attention by cues presented after a visual target offset (post-target cues) on target perception. Conducting two experiments, we tested and confirmed this claim. Our study revealed an elevated reliance on post-target cues for target detection with diminishing target visibility, leading to better performance in validly versus invalidly cued trials, indicative of contrast gain. We demonstrated this post-target cueing impact on target perception without a postcue response prompt, meaning that our results truly reflected a continuity between perception and memory rather than a task-specific impact of having to memorize the target due to a response prompt. While previous studies found an improvement in accuracy through valid compared to invalid cues using liminal targets, in Experiment 1, we further showed an influence of attention on participants' response time by the post-target cues with cues presented away from a clearly visible target. This suggests that visual interactions at the target location provided no better explanation of post-target cueing effects. Our results generalize prior research with liminal targets and confirm the view of a perception-memory continuum so that visual target processing is not shielded against visuospatial orienting of attention elicited by events following the offset of the visual target.

Altered functional connectivity in default mode network maintains attention task performance in school-age children with frontal lobe tumor

This study examines functional brain network changes in children with frontal lobe tumors (FLT). Ten pediatric FLT patients from Beijing Tiantan Hospital and 20 healthy children were compared in terms of cognitive performance and resting-state functional magnetic resonance imaging (rs-fMRI) connectivity. The FLT group showed lower cognitive performance, particularly in visual and working memory domains, but had comparable attention abilities to the healthy controls. There were notable differences in connectivity between the default mode network (DMN) and sensorimotor network (SMN) in both groups. The FLT group also displayed a significant reduction in local efficiency in the left lateral parietal area within the DMN. Importantly, reduced DMN-SMN connections and increased DMN-lateral prefrontal cortex connectivity may facilitate maintaining attention and memory tasks in FLT children. This study sheds light on how the brains of children with FLT adapt, preserving "normal" attention functions despite frontal lobe damage.

Sequential neuronal processing of number values, abstract decision, and action in the primate prefrontal cortex

Decision-making requires processing of sensory information, comparing the gathered evidence to make a judgment, and performing the action to communicate it. How neuronal representations transform during this cascade of representations remains a matter of debate. Here, we studied the succession of neuronal representations in the primate prefrontal cortex (PFC). We trained monkeys to judge whether a pair of sequentially presented displays had the same number of items. We used a combination of single neuron and population-level analyses and discovered a sequential transformation of represented information with trial progression. While numerical values were initially represented with high precision and in conjunction with detailed information such as order, the decision was encoded in a low-dimensional subspace of neural activity. This decision encoding was invariant to both retrospective numerical values and prospective motor plans, representing only the binary judgment of "same number" versus "different number," thus facilitating the generalization of decisions to novel number pairs. We conclude that this transformation of neuronal codes within the prefrontal cortex supports cognitive flexibility and generalizability of decisions to new conditions.

Strengthening of alpha synchronization is a neural correlate of cognitive transfer

Cognitive training can lead to improvements in both task-specific strategies and general capacities, such as visuo-spatial working memory (VSWM). The latter emerge slowly and linearly throughout training, in contrast to strategy where changes typically occur within the first days of training. Changes in strategy and capacity have not been separated in prior neuroimaging studies. Here, we used a within-participants design with dense temporal sampling to capture the time dynamics of neural mechanisms associated with change in capacity. In four participants, neural activity was recorded with magnetoencephalography on seven occasions over two months of visuo-spatial working memory training. During scanning, the participants performed a trained visuo-spatial working memory task, a transfer task, and a control task. First, we extracted an individual visuo-spatial working memory-load-dependent synchronization network for each participant. Next, we identified linear changes over time in the network, congruent with the temporal dynamics of capacity change. Three out of four participants showed a gradual strengthening of alpha synchronization. Strengthening of the same connections was also found in the transfer task but not in the control task. This suggests that cognitive transfer occurs through slow, gradual strengthening of alpha synchronization between cortical regions that are vital for both the trained task and the transfer task.

Basis of executive functions in fine-grained architecture of cortical and subcortical human brain networks

Theoretical models suggest that executive functions rely on both domain-general and domain-specific processes. Supporting this view, prior brain imaging studies have revealed that executive activations converge and diverge within broadly characterized brain networks. However, the lack of precise anatomical mappings has impeded our understanding of the interplay between domain-general and domain-specific processes. To address this challenge, we used the high-resolution multimodal magnetic resonance imaging approach of the Human Connectome Project to scan participants performing 3 canonical executive tasks: n-back, rule switching, and stop signal. The results reveal that, at the individual level, different executive activations converge within 9 domain-general territories distributed in frontal, parietal, and temporal cortices. Each task exhibits a unique topography characterized by finely detailed activation gradients within domain-general territory shifted toward adjacent resting-state networks; n-back activations shift toward the default mode, rule switching toward dorsal attention, and stop signal toward cingulo-opercular networks. Importantly, the strongest activations arise at multimodal neurobiological definitions of network borders. Matching results are seen in circumscribed regions of the caudate nucleus, thalamus, and cerebellum. The shifting peaks of local gradients at the intersection of task-specific networks provide a novel mechanistic insight into how partially-specialized networks interact with neighboring domain-general territories to generate distinct executive functions.

Inferring neural dynamics of memory during naturalistic social communication

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present 'Natural Continuation' (NC)-a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

The Relation Between Attention and Memory

The relation between attention and memory has long been deemed important for understanding cognition, and it was heavily researched even in the first experimental psychology laboratory by Wilhelm Wundt and his colleagues. Since then, the importance of the relation between attention and memory has been explored in myriad subdisciplines of psychology, and we incorporate a wide range of these diverse fields. Here, we examine some of the practical consequences of this relation and summarize work with various methodologies relating attention to memory in the fields of working memory, long-term memory, individual differences, life-span development, typical brain function, and neuropsychological conditions. We point out strengths and unanswered questions for our own embedded processes view of information processing, which is used to organize a large body of evidence. Last, we briefly consider the relation of the evidence to a range of other theoretical views before drawing conclusions about the state of the field.

A neural substrate for short-term taste memories

Real-time decisions on what foods to select for consumption, particularly in the wild, require a sensitive sense of taste and an effective system to maintain short-term taste memories, also defined as working memory in the scale of seconds. Here, we used a behavioral memory assay, combined with recordings of neural activity, to identify the brain substrate for short-term taste memories. We demonstrate that persistent activity in taste cortex functions as an essential memory trace of a recent taste experience. Next, we manipulated the decay of this persistent activity and showed that early termination of the memory trace abolished the memory. Notably, extending the memory trace by transiently disinhibiting taste cortical activity dramatically extended the retention of a short-term taste memory. Together, our results uncover taste cortex as a neural substrate for working memory and substantiate the role of sensory cortex in memory-guided actions while imposing meaning to a sensory stimulus.

Attention influences the effects of the previous form orientation on the current motion direction estimation

Recent studies have found that the estimates of motion directions are biased toward the previous form orientations, showing serial dependence, and the serial dependence does not involve cognitive abilities. In the current study, we conducted two experiments to investigate whether and how attention-a cognitive ability-affected the serial dependence. The results showed that serial dependence was present in the current study, reproducing the previous findings. Importantly, when the attentional load reduced the reliability (i.e., estimation accuracy and precision) of previous form orientations (Experiment 1), the serial dependence decreased, meaning that the biases of motion direction estimates toward previous form orientations were reduced; in contrast, when the attentional load reduced the reliability of current motion directions (Experiment 2), the serial dependence increased, meaning that the biases of motion direction estimates toward previous form orientations were increased. These trends were well consistent with the prediction of the Bayesian inference theory. Therefore, the current study revealed the involvement of attention in the serial dependence of current motion direction estimation on the previous form orientation, demonstrating that the serial dependence was cognitive and the attentional effect can be a Bayesian inference process, initially revealing its computational mechanism.

Gamma Power as an Index of Sustained Attention in Simulated Vigilance Tasks

Performance on the psychomotor vigilance test (PVT; Dinges & Powell, 1985)-a common index of sustained attention-is affected by the opposing forces of fatigue and sustained effort, where reaction times and error rates typically increase across trials and are sometimes offset by additional efforts deployed toward the end of the task (i.e., an "end-spurt"; cf. Bergum & Klein, 1961). In ACT-R (Adaptive Control of Thought-Rational; Anderson et al., 2004), these influences on task performance have been modeled as latent variables that are inferred from performance (e.g., Jongman, 1998; Veksler & Gunzelmann, 2018) without connections to directly observable variables. We propose the use of frontal gamma (γ) spectral power as a direct measure of vigilant effort and demonstrate its efficacy in modeling performance on the PVT in both the aggregate and in individuals.

The Moderating Role of Cortisol and Negative Emotionality in the Effects of Classroom Size and Window View on Young Children's Executive Functions

This study probed how baseline cortisol (BC), negative emotionality (NE), and environmental facets-classroom size and window view-interact to affect executive function (EF) in preschoolers using virtual reality (VR). In a cohort of 144 children aged 61-85 months, BC levels were measured by saliva assays and NE by parental surveys. Participants completed computerized EF assessments both pre- and post-exposure to one of four VR conditions, which varied by classroom size (large vs. small) and window view (natural vs. built). Due to missing data and outlier responses, three children were removed from the analyses. Regression analyses, accounting for initial EFs, revealed that higher BC was significantly associated with better Digit-span task scores in the nature view, while lower BC correlated with improved performance in the built view. With regard to classroom size, children with varying levels of NE benefitted from the large classroom environment, as evidenced by marginally significant improvements on the Corsi block task. However, higher NE children outperformed their lower NE peers in the large classroom, while a trend inverted in the small classroom context. The findings illuminate how the physical components of preschool environments may interact with children's physiological reactivity, potentially influencing the development of working memory.

No obligatory trade-off between the use of space and time for working memory

Space and time can each act as scaffolds for the individuation and selection of visual objects in working memory. Here we ask whether there is a trade-off between the use of space and time for visual working memory: whether observers will rely less on space, when memoranda can additionally be individuated through time. We tracked the use of space through directional biases in microsaccades after attention was directed to memory contents that had been encoded simultaneously or sequentially to the left and right of fixation. We found that spatial gaze biases were preserved when participants could (Experiment 1) and even when they had to (Experiment 2) additionally rely on time for object individuation. Thus, space remains a profound organizing medium for working memory even when other organizing sources are available and utilized, with no evidence for an obligatory trade-off between the use of space and time.

Prefrontal Cortex Codes Representations of Target Identity and Feature Uncertainty

Many objects in the real world have features that vary over time, creating uncertainty in how they will look in the future. This uncertainty makes statistical knowledge about the likelihood of features critical to attention demanding processes such as visual search. However, little is known about how the uncertainty of visual features is integrated into predictions about search targets in the brain. In the current study, we test the idea that regions prefrontal cortex code statistical knowledge about search targets before the onset of search. Across 20 human participants (13 female; 7 male), we observe target identity in the multivariate pattern and uncertainty in the overall activation of dorsolateral prefrontal cortex (DLPFC) and inferior frontal junction (IFJ) in advance of the search display. This indicates that the target identity (mean) and uncertainty (variance) of the target distribution are coded independently within the same regions. Furthermore, once the search display appears the univariate IFJ signal scaled with the distance of the actual target from the expected mean, but more so when expected variability was low. These results inform neural theories of attention by showing how the prefrontal cortex represents both the identity and expected variability of features in service of top-down attentional control.SIGNIFICANCE STATEMENT Theories of attention and working memory posit that when we engage in complex cognitive tasks our performance is determined by how precisely we remember task-relevant information. However, in the real world the properties of objects change over time, creating uncertainty about many aspects of the task. There is currently a gap in our understanding of how neural systems represent this uncertainty and combine it with target identity information in anticipation of attention demanding cognitive tasks. In this study, we show that the prefrontal cortex represents identity and uncertainty as unique codes before task onset. These results advance theories of attention by showing that the prefrontal cortex codes both target identity and uncertainty to implement top-down attentional control.

Attention with or without working memory: mnemonic reselection of attended information

Attention has been regarded as the 'gatekeeper' controlling what information gets selected into working memory. However, a new perspective has emerged with the discovery of attribute amnesia, a phenomenon revealing that people are frequently unable to report information they have just attended to moments ago. This report failure is thought to stem from a lack of consolidating the attended information into working memory, indicating a dissociation between attention and working memory. Building on these findings, a new concept called memory reselection is proposed to describe a secondary round of selection among the attended information. These discoveries challenge the conventional view of how attention and working memory are related and shed new light onto modeling attention and memory as dissociable processes.

What Is the Relationship between Metacognition and Mental Effort in Executive Functions? The Contribution of Neurophysiology

Prolonged cognitive effort can be considered one of the core determinants of mental fatigue and may negatively affect the efficacy and efficiency of cognitive performance. Metacognition-understood as a multi-componential set of skills concerning awareness and control of one's own cognition-might reduce such negative outcomes. This study aimed to explore the relation between metacognitive skills, neurocognitive performance, and the level of mental effort as mirrored by electrophysiological (EEG) markers of cognitive load and task demand. A challenging cognitive task was used to prompt and collect metacognition reports, performance data (accuracy and response times-RTs), and physiological markers of mental effort (task-related changes of spectral power for standard EEG frequency bands) via wearable EEG. Data analysis highlighted that different aspects of metacognitive skills are associated with performance as measured by, respectively, accuracy and RTs. Furthermore, specific aspects of metacognitive skills were found to be consistently correlated with EEG markers of cognitive effort, regardless of increasing task demands. Finally, behavioral metrics mirroring the efficiency of information processing were found to be associated with different EEG markers of cognitive effort depending on the low or high demand imposed by the task.

Functional alterations of the prefrontal circuit underlying cognitive aging in mice

Executive function is susceptible to aging. How aging impacts the circuit-level computations underlying executive function remains unclear. Using calcium imaging and optogenetic manipulation during memory-guided behavior, we show that working-memory coding and the relevant recurrent connectivity in the mouse medial prefrontal cortex (mPFC) are altered as early as middle age. Population activity in the young adult mPFC exhibits dissociable yet overlapping patterns between tactile and auditory modalities, enabling crossmodal memory coding concurrent with modality-dependent coding. In middle age, however, crossmodal coding remarkably diminishes while modality-dependent coding persists, and both types of coding decay in advanced age. Resting-state functional connectivity, especially among memory-coding neurons, decreases already in middle age, suggesting deteriorated recurrent circuits for memory maintenance. Optogenetic inactivation reveals that the middle-aged mPFC exhibits heightened vulnerability to perturbations. These findings elucidate functional alterations of the prefrontal circuit that unfold in middle age and deteriorate further as a hallmark of cognitive aging.

Working memory performance is tied to stimulus complexity

Working memory is the cognitive capability to maintain and process information over short periods. Behavioral and computational studies have shown that visual information is associated with working memory performance. However, the underlying neural correlates remain unknown. To identify how visual information affects working memory performance, we conducted behavioral experiments in pigeons (Columba livia) and single unit recordings in the avian prefrontal analog, the nidopallium caudolaterale (NCL). Complex pictures featuring luminance, spatial and color information, were associated with higher working memory performance compared to uniform gray pictures in conjunction with distinct neural coding patterns. For complex pictures, we found a multiplexed neuronal code displaying visual and value-related features that switched to a representation of the upcoming choice during a delay period. When processing gray stimuli, NCL neurons did not multiplex and exclusively represented the choice already during stimulus presentation and throughout the delay period. The prolonged representation possibly resulted in a decay of the memory trace ultimately leading to a decrease in performance. In conclusion, we found that high stimulus complexity is associated with neuronal multiplexing of the working memory representation possibly allowing a facilitated read-out of the neural code resulting in enhancement of working memory performance.