Neural substrates of cognitive capacity limitations

Working Memory: From Neural Activity to the Sentient Mind

Working memory (WM) is the ability to maintain and manipulate information in the conscious mind over a timescale of seconds. This ability is thought to be maintained through the persistent discharges of neurons in a network of brain areas centered on the prefrontal cortex, as evidenced by neurophysiological recordings in nonhuman primates, though both the localization and the neural basis of WM has been a matter of debate in recent years. Neural correlates of WM are evident in species other than primates, including rodents and corvids. A specialized network of excitatory and inhibitory neurons, aided by neuromodulatory influences of dopamine, is critical for the maintenance of neuronal activity. Limitations in WM capacity and duration, as well as its enhancement during development, can be attributed to properties of neural activity and circuits. Changes in these factors can be observed through training-induced improvements and in pathological impairments. WM thus provides a prototypical cognitive function whose properties can be tied to the spiking activity of brain neurons. © 2021 American Physiological Society. Compr Physiol 11:1-41, 2021.

Sensory Recruitment Revisited: Ipsilateral V1 Involved in Visual Working Memory

The "sensory recruitment hypothesis" posits an essential role of sensory cortices in working memory, beyond the well-accepted frontoparietal areas. Yet, this hypothesis has recently been challenged. In the present study, participants performed a delayed orientation recall task while high-spatial-resolution 3 T functional magnetic resonance imaging (fMRI) signals were measured in posterior cortices. A multivariate inverted encoding model approach was used to decode remembered orientations based on blood oxygen level-dependent fMRI signals from visual cortices during the delay period. We found that not only did activity in the contralateral primary visual cortex (V1) retain high-fidelity representations of the visual stimuli, but activity in the ipsilateral V1 also contained such orientation tuning. Moreover, although the encoded tuning was faded in the contralateral V1 during the late delay period, tuning information in the ipsilateral V1 remained sustained. Furthermore, the ipsilateral representation was presented in secondary visual cortex (V2) as well, but not in other higher-level visual areas. These results thus supported the sensory recruitment hypothesis and extended it to the ipsilateral sensory areas, which indicated the distributed involvement of visual areas in visual working memory.

Emotion-Attention Interaction in the Right Hemisphere

Hemispheric asymmetries in affective and cognitive functions have been extensively studied. While both cerebral hemispheres contribute to most affective and cognitive processes, neuroscientific literature and neuropsychological evidence support an overall right hemispheric dominance for emotion, attention and arousal. Emotional stimuli, especially those with survival value such as threat, tend to be prioritized in attentional resource competition. Arousing unpleasant emotional stimuli have prioritized access, especially to right-lateralized attention networks. Interference of task performance may be observed when limited resources are exhausted by task- and emotion-related processing. Tasks that rely on right hemisphere-dependent processing, like attending to the left visual hemifield or global-level visual features, are especially vulnerable to interference due to attention capture by unpleasant emotional stimuli. The aim of this review is to present literature regarding the special role of the right hemisphere in affective and attentional brain processes and their interaction. Furthermore, clinical and technological implications of this interaction will be presented. Initially, the effects of focal right hemisphere lesion or atrophy on emotional functions will be introduced. Neurological right hemisphere syndromes including aprosodia, anosognosia and neglect, which further point to the predominance of the intact right hemisphere in emotion, attention and arousal will be presented. Then there will be a brief review of electrophysiological evidence, as well as evidence from patients with neglect that support attention capture by emotional stimuli in the right hemisphere. Subsequently, experimental work on the interaction of emotion, attention and cognition in the right hemispheres of healthy subjects will be presented. Finally, clinical implications for better understanding and assessment of alterations in emotion-attention interaction due to brain disorder or treatment, such as neuromodulation, that impact affective brain functions will be discussed. It will be suggested that measuring right hemispheric emotion-attention interactions may provide basis for novel biomarkers of brain health. Such biomarkers allow for improved diagnostics in brain damage and disorders and optimized treatments. To conclude, future technological applications will be outlined regarding brain physiology-based measures that reflect engagement of the right hemisphere in affective and attentional processes.

Brain Lateralization and Cognitive Capacity

One way to increase cognitive capacity is to avoid duplication of functions on the left and right sides of the brain. There is a convincing body of evidence showing that such asymmetry, or lateralization, occurs in a wide range of both vertebrate and invertebrate species. Each hemisphere of the brain can attend to different types of stimuli or to different aspects of the same stimulus and each hemisphere analyses information using different neural processes. A brain can engage in more than one task at the same time, as in monitoring for predators (right hemisphere) while searching for food (left hemisphere). Increased cognitive capacity is achieved if individuals are lateralized in one direction or the other. The advantages and disadvantages of individual lateralization are discussed. This paper argues that directional, or population-level, lateralization, which occurs when most individuals in a species have the same direction of lateralization, provides no additional increase in cognitive capacity compared to individual lateralization although directional lateralization is advantageous in social interactions. Strength of lateralization is considered, including the disadvantage of being very strongly lateralized. The role of brain commissures is also discussed with consideration of cognitive capacity.

Focused Representation of Successive Task Episodes in Frontal and Parietal Cortex

Complex cognition is dynamic, with each stage of a task requiring new cognitive processes appropriately linked to stimulus or other content. To investigate control over successive task stages, we recorded neural activity in lateral frontal and parietal cortex as monkeys carried out a complex object selection task, with each trial separated into phases of visual selection and learning from feedback. To study capacity limitation, complexity was manipulated by varying the number of object targets to be learned in each problem. Different task phases were associated with quasi-independent patterns of activity and information coding, with no suggestion of sustained activity linked to a current target. Object and location coding were largely parallel in frontal and inferior parietal cortex, though frontal cortex showed somewhat stronger object representation at feedback, and more sustained location coding at choice. At both feedback and choice, coding precision diminished as task complexity increased, matching a decline in performance. We suggest that, across successive task steps, there is radical but capacity-limited reorganization of frontoparietal activity, selecting different cognitive operations linked to their current targets.

Balancing Flexibility and Interference in Working Memory

Working memory is central to cognition, flexibly holding the variety of thoughts needed for complex behavior. Yet, despite its importance, working memory has a severely limited capacity, holding only three to four items at once. In this article, I review experimental and computational evidence that the flexibility and limited capacity of working memory reflect the same underlying neural mechanism. I argue that working memory relies on interactions between high-dimensional, integrative representations in the prefrontal cortex and structured representations in the sensory cortex. Together, these interactions allow working memory to flexibly maintain arbitrary representations. However, the distributed nature of working memory comes at the cost of causing interference between items in memory, resulting in a limited capacity. Finally, I discuss several mechanisms used by the brain to reduce interference and maximize the effective capacity of working memory. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Rotational dynamics reduce interference between sensory and memory representations

Cognition depends on integrating sensory percepts with the memory of recent stimuli. However, the distributed nature of neural coding can lead to interference between sensory and memory representations. Here, we show that the brain mitigates such interference by rotating sensory representations into orthogonal memory representations over time. To study how sensory inputs and memories are represented, we recorded neurons from the auditory cortex of mice as they implicitly learned sequences of sounds. We found that the neural population represented sensory inputs and the memory of recent stimuli in two orthogonal dimensions. The transformation of sensory information into a memory was facilitated by a combination of 'stable' neurons, which maintained their selectivity over time, and 'switching' neurons, which inverted their selectivity over time. Together, these neural responses rotated the population representation, transforming sensory inputs into memory. Theoretical modeling showed that this rotational dynamic is an efficient mechanism for generating orthogonal representations, thereby protecting memories from sensory interference.

Up to the magical number seven: An evolutionary perspective on the capacity of short term memory

Working memory and its components are among the most determinant factors in human cognition. However, in spite of their critical importance, many aspects of their evolution remain underinvestigated. The present study is devoted to reviewing the literature of memory studies from an evolutionary, comparative perspective, focusing particularly on short term memory capacity. The findings suggest the limited capacity to be the common attribute of different species of birds and mammals. Moreover, the results imply an increasing trend of capacity from our non-human ancestors to modern humans. The present evidence shows that non-human mammals and birds, regardless of their limitations, are capable of performing memory strategies, although there seem to be some differences between their ability and that of humans in terms of flexibility and efficiency. These findings have several implications relevant to the psychology of memory and cognition, and are likely to explain differences between higher cognitive abilities of humans and non-humans. The adaptive benefits of the limited capacity and the reasons for the growing trend found in the present study are broadly discussed.

The presence of semantic content in a visual recognition memory task reduces the severity of neglect

Patients with right hemisphere damage often show a lateral bias when asked to report the left side of mental images held in visual working memory (i.e. representational neglect). The neural basis of representational neglect is not well understood. One hypothesis suggests that it reflects a deficit in attentional-exploratory mechanisms, i.e. an inability to direct attention to the left side of the image. Another proposition states that intact visual working memory (VWM) is necessary for correctly creating a mental image. Here we examined two components of VWM in patients with unilateral spatial neglect (USN): memory for identity, and memory for spatial position. We manipulated the strength of memory representations by presenting two distinct categories of objects, in separate blocks. These were familiar namable objects (fruits, etc.), and unfamiliar abstract objects. The former category elicits stronger working-memory traces, thanks to preexisting visual and semantic representations in long-term memory. We hypothesized that if USN patients show a lateralized deficit in VWM, it should be more pronounced for abstract objects, due to their weaker working-memory traces. Importantly, to isolate a spatially lateralized deficit in memory from a failure to fully perceive the object-arrays, we ensured that all included patients perceived every item during the encoding phase. We used a working-memory task: participants viewed object arrays and had to memorize items' identities and spatial positions. Then, single objects were presented requiring 'old/new' recognition, and retrieval of 'old' items' original positions. Our results show a lateral bias in patients' recognition-memory performance. Remarkably, it was threefold milder for namable objects compared to abstract objects. We conclude that VWM lateralized deficit is substantial in USN patients and could play a role in representational neglect.

Ketamine-Induced Alteration of Working Memory Utility during Oculomotor Foraging Task in Monkeys

Impairments of working memory (WM) are commonly observed in a variety of neurodegenerative disorders but they are difficult to quantitatively assess in clinical cases. Recent studies in experimental animals have used low-dose ketamine (an NMDA receptor antagonist) to disrupt WM, partly mimicking the pathophysiology of schizophrenia. Here, we developed a novel behavioral paradigm to assess multiple components of WM and applied it to monkeys with and without ketamine administration. In an oculomotor foraging task, the animals were presented with 15 identical objects on the screen. One of the objects was associated with a liquid reward, and monkeys were trained to search for the target by generating sequential saccades under a time constraint. We assumed that the occurrence of recursive movements to the same object might reflect WM dysfunction. We constructed a "foraging model" that incorporated (1) memory capacity, (2) memory decay, and (3) utility rate; this model was able to explain more than 92% of the variations in behavioral data obtained from three monkeys. Following systemic administration of low dosages of ketamine, the memory capacity and utility rate were dramatically reduced by 15% and 57%, respectively, while memory decay remained largely unchanged. These results suggested that the behavioral deficits during the blockade of NMDA receptors were mostly due to the decreased usage of short-term memory. Our oculomotor paradigm and foraging model appear to be useful for quantifying multiple components of WM and could be applicable to clinical cases in future studies.

Shared mechanisms underlie the control of working memory and attention

Cognitive control guides behaviour by controlling what, when, and how information is represented in the brain¹. For example, attention controls sensory processing; top-down signals from prefrontal and parietal cortex strengthen the representation of task-relevant stimuli^2-4. A similar 'selection' mechanism is thought to control the representations held 'in mind'-in working memory^5-10. Here we show that shared neural mechanisms underlie the selection of items from working memory and attention to sensory stimuli. We trained rhesus monkeys to switch between two tasks, either selecting one item from a set of items held in working memory or attending to one stimulus from a set of visual stimuli. Neural recordings showed that similar representations in prefrontal cortex encoded the control of both selection and attention, suggesting that prefrontal cortex acts as a domain-general controller. By contrast, both attention and selection were represented independently in parietal and visual cortex. Both selection and attention facilitated behaviour by enhancing and transforming the representation of the selected memory or attended stimulus. Specifically, during the selection task, memory items were initially represented in independent subspaces of neural activity in prefrontal cortex. Selecting an item caused its representation to transform from its own subspace to a new subspace used to guide behaviour. A similar transformation occurred for attention. Our results suggest that prefrontal cortex controls cognition by dynamically transforming representations to control what and when cognitive computations are engaged.

Greater Visual Working Memory Capacity for Visually Matched Stimuli When They Are Perceived as Meaningful

Almost all models of visual working memory-the cognitive system that holds visual information in an active state-assume it has a fixed capacity: Some models propose a limit of three to four objects, where others propose there is a fixed pool of resources for each basic visual feature. Recent findings, however, suggest that memory performance is improved for real-world objects. What supports these increases in capacity? Here, we test whether the meaningfulness of a stimulus alone influences working memory capacity while controlling for visual complexity and directly assessing the active component of working memory using EEG. Participants remembered ambiguous stimuli that could either be perceived as a face or as meaningless shapes. Participants had higher performance and increased neural delay activity when the memory display consisted of more meaningful stimuli. Critically, by asking participants whether they perceived the stimuli as a face or not, we also show that these increases in visual working memory capacity and recruitment of additional neural resources are because of the subjective perception of the stimulus and thus cannot be driven by physical properties of the stimulus. Broadly, this suggests that the capacity for active storage in visual working memory is not fixed but that more meaningful stimuli recruit additional working memory resources, allowing them to be better remembered.

Insights into visual working memory precision at the feature- and object-level from a hemispheric encoding manipulation

Mnemonic precision is an important aspect of visual working memory (WM). Here, we probed mechanisms that affect precision for spatial (size) and non-spatial (colour) features of an object, and whether these features are encoded and/or stored separately in WM. We probed precision at the feature-level—that is, whether different features of a single object are represented separately or together in WM—and the object-level—that is, whether different features across a set of sequentially presented objects are represented in the same or different WM stores. By manipulating whether stimuli were encoded by the left and/or right hemisphere, we gained further insights into how objects are represented in WM. At the feature-level, we tested whether recall fidelity for the two features of an object fluctuated in tandem from trial to trial. We observed no significant coupling under either central or lateralized encoding, supporting the claim of parallel feature channels at encoding. At the level of WM storage of a set of objects, we found asymmetric feature interference under central encoding, whereby an increase in colour load led to a decrease in size precision. When objects were encoded by a single hemisphere, however, we found largely independent feature stores. Precision for size was more resistant to interference from the size of another object under right-hemisphere encoding; by contrast, precision for colour did not differ across hemispheres, suggesting a more distributed WM store. These findings suggest that distinct features of a single object are represented separately but are then partially integrated during maintenance of a set of sequentially presented objects.

Integrated Intelligence from Distributed Brain Activity

How does organized cognition arise from distributed brain activity? Recent analyses of fluid intelligence suggest a core process of cognitive focus and integration, organizing the components of a cognitive operation into the required computational structure. A cortical 'multiple-demand' (MD) system is closely linked to fluid intelligence, and recent imaging data define nine specific MD patches distributed across frontal, parietal, and occipitotemporal cortex. Wide cortical distribution, relative functional specialization, and strong connectivity suggest a basis for cognitive integration, matching electrophysiological evidence for binding of cognitive operations to their contents. Though still only in broad outline, these data suggest how distributed brain activity can build complex, organized cognition.

Working Memory as an Indicator for Comparative Cognition - Detecting Qualitative and Quantitative Differences

Working memory (WM), the representation of information held accessible for manipulation over time, is an essential component of all higher cognitive abilities. It allows for complex behaviors that go beyond simple stimulus-response associations and inflexible behavioral patterns. WM capacity determines how many different pieces of information (items) can be used for these cognitive processes, and in humans, it correlates with fluid intelligence. As such, WM might be a useful tool for comparison of cognition across species. WM can be tested using comparatively simple behavioral protocols, based on operant conditioning, in a multitude of different species. Species-specific contextual variables that influence an animal’s performance on a non-cognitive level are controlled by adapting the WM paradigm. The neuronal mechanisms by which WM emerges in the brain, as sustained neuronal activity, are comparable between the different species studied (mammals and birds), as are the areas of the brain in which WM activity can be measured. Thus WM is comparable between vastly different species within their respective niches, accounting for specific contextual variables and unique adaptations. By approaching the question of “general cognitive abilities” or “intelligence” within the animal kingdom from the perspective of WM, the complexity of the core question at hand is reduced to a fundamental memory system required to allow for complex cognitive abilities. This article argues that measuring WM can be a suitable addition to the toolkit of comparative cognition. By measuring WM on a behavioral level and going beyond behavior to the underlying physiological processes, qualitative and quantitative differences in cognition between different animal species can be identified, free of contextual restraints.

The potential interruptive effect of tinnitus-related distress on attention

The mechanism through which tinnitus affects attention is unclear. This study examines whether distress mediates the relationship(s) between tinnitus and sustained, selective and executive attentions as well as response inhibition. Eighteen participants with tinnitus and fifteen controls completed the Counting Stroop, Vigilance and Stop Signal tasks. Tinnitus distress was assessed using the Tinnitus Questionnaire (TQ), severity of depressive mood states examined using the Beck Depression Inventory-II, and general distress assessed using the Hospital Anxiety and Depression Scale. Tinnitus participants had significantly slower reactions during the Vigilance task (F = 4.86, p = .035), and incongruent trials of the Cognitive Counting task (F = 3.45, p = .045) compared to controls. Tinnitus-related distress significantly mediated the effect of tinnitus in incongruent trials (TQ: Sobel test t = 1.73, p = .042) of the Cognitive Counting Task. Complaints of distress and concentration difficulties are common amongst tinnitus patients in clinical settings and these afflictions have been shown to negatively impact an individual’s quality of life. If confirmed in future studies, results suggest that distress may be an important factor in the causal mechanism between tinnitus and attention.

Safety Science, a Systems Thinking Perspective: From Events to Mental Models and Sustainable Safety

In the past one hundred years, concepts such as risk, safety and security have become ever more important and they represent a growing concern in our society. These concepts are also important subjects of study to enhance sustainability. During the past fifty years, safety science has gradually developed as an independent field of science. In this period, different concepts, theories, models and research traditions have emerged, each with its specific perspective. Safety science is now focused on finding ways to proactively achieve safety versus reaching safety in a reactive way. We think this increasing awareness and search for proactiveness can be found and presented when viewed in light of the systems thinking iceberg model, where increasing awareness and proactiveness can be seen as digging deeper into this systems thinking iceberg, discovering the levels of systems, structures and ultimately the mental models that are “below the waterline”. It offers a way forward in understanding, and proactively managing, risk, safety, security and sustainable performance, in organizations and ultimately in society as a whole.

Working Memory Load Modulates Neuronal Coupling

There is a severe limitation in the number of items that can be held in working memory. However, the neurophysiological limits remain unknown. We asked whether the capacity limit might be explained by differences in neuronal coupling. We developed a theoretical model based on Predictive Coding and used it to analyze Cross Spectral Density data from the prefrontal cortex (PFC), frontal eye fields (FEF), and lateral intraparietal area (LIP). Monkeys performed a change detection task. The number of objects that had to be remembered (memory load) was varied (1-3 objects in the same visual hemifield). Changes in memory load changed the connectivity in the PFC-FEF-LIP network. Feedback (top-down) coupling broke down when the number of objects exceeded cognitive capacity. Thus, impaired behavioral performance coincided with a break-down of Prediction signals. This provides new insights into the neuronal underpinnings of cognitive capacity and how coupling in a distributed working memory network is affected by memory load.

The Case of the Cognitive (Opti)miser: Electrophysiological Correlates of Working Memory Maintenance Predict Demand Avoidance

People are often considered cognitive misers. When given a free choice between two tasks, people tend to choose tasks requiring less cognitive effort. Such demand avoidance (DA) is associated with cognitive control, but it is still not clear to what extent individual differences in cognitive control can account for variations in DA. We sought to elucidate the relation between cognitive control and cognitive effort preferences by investigating the extent to which sustained neural activity in a task requiring cognitive control is correlated with DA. We hypothesized that neural measures of efficient filtering will predict individual variations in demand preferences. To test this hypothesis, we had participants perform a delayed-match-to-sample paradigm with their ERPs recorded, as well as a separate behavioral demand-selection task. We focused on the ERP correlates of cognitive filtering efficiency (CFE)-the ability to ignore task-irrelevant distractors during working memory maintenance-as it manifests in a modulation of the contralateral delay activity, an ERP correlate of cognitive control. As predicted, we found a significant positive correlation between CFE and DA. Individuals with high CFE tended to be significantly more demand avoidant than their low-CFE counterparts. Low-CFE individuals, in comparison, did not form distinct cognitive effort preferences. Overall, our results suggest that cognitive control over the contents of visual working memory contribute to individual differences in the expression of cognitive effort preferences. This further implies that these observed preferences are the product of sensitivity to cognitive task demands.

Gray Matter Volume in Different Cortical Structures Dissociably Relates to Individual Differences in Capacity and Precision of Visual Working Memory

Visual working memory (VWM) refers to our ability to selectively maintain visual information in a mental representation. While cognitive limits of VWM greatly influence a variety of mental operations, it remains controversial whether the quantity or quality of representations in mind constrains VWM. Here, we examined behavior-to-brain anatomical relations as well as brain activity to brain anatomy associations with a "neural" marker specific to the retention interval of VWM. Our results consistently indicated that individuals who maintained a larger number of items in VWM tended to have a larger gray matter (GM) volume in their left lateral occipital region. In contrast, individuals with a superior ability to retain with high precision tended to have a larger GM volume in their right parietal lobe. These results indicate that individual differences in quantity and quality of VWM may be associated with regional GM volumes in a dissociable manner, indicating willful integration of information in VWM may recruit separable cortical subsystems.

Limitations of concurrently representing objects within view and in visual working memory

Representing visibly present stimuli is as limited in capacity as representing invisible stimuli in visual working memory (WM). In this study, we explored whether concurrently representing stimuli within view affects representing objects in visual WM, and if so, whether this effect is modulated by the storage states (active and silent state) of memory contents? In experiment 1, participants were asked to perform the change-detect task in a simultaneous-representing condition in which WM content and the continuously-visible stimuli in view were simultaneously represented, as well as a baseline condition in which only the representations of visual WM content were maintained. The results showed that the representations in visual WM would be impaired when the continuously-visible stimuli in view were concurrently represented, revealed by the reduced CDA amplitude and the lower behavior performance. In experiment 2, a dual-serial retro-cue paradigm was adopted to guide participants to maintain memory items in two different storage states, and the results revealed that simultaneously representing the continuously-visible stimuli and the WM content would only impair the WM representations in the active state. These evidences demonstrated that only the visual WM representations that were maintained in the active state would definitely share the limited resources with the representations of continuously-visible information, and further supported the dissociation between the active state and silent state of visual WM storage.

Theory of neural coding predicts an upper bound on estimates of memory variability

Observers reproducing elementary visual features from memory after a short delay produce errors consistent with the encoding-decoding properties of neural populations. While inspired by electrophysiological observations of sensory neurons in cortex, the population coding account of these errors is based on a mathematical idealization of neural response functions that abstracts away most of the heterogeneity and complexity of real neuronal populations. Here we examine a more physiologically grounded model based on the tuning of a large set of neurons recorded in macaque V1 and show that key predictions of the idealized model are preserved. Both models predict long-tailed distributions of error when memory resources are taxed, as observed empirically in behavioral experiments and commonly approximated with a mixture of normal and uniform error components. Specifically, for an idealized homogeneous neural population, the width of the fitted normal distribution cannot exceed the average tuning width of the component neurons, and this also holds to a good approximation for more biologically realistic populations. Examining eight published studies of orientation recall, we find a consistent pattern of results suggestive of a median tuning width of approximately 20°, which compares well with neurophysiological observations. The finding that estimates of variability obtained by the normal-plus-uniform mixture method are bounded from above leads us to reevaluate previous studies that interpreted a saturation in width of the normal component as evidence for fundamental limits on the precision of perception, working memory, and long-term memory.

Crows control working memory before and after stimulus encoding

The capacity of working memory is limited and this limit is comparable in crows and primates. To maximize this resource, humans use attention to select only relevant information for maintenance. Interestingly, attention-cues are effective not only before but also after the presentation of to-be-remembered stimuli, highlighting control mechanisms beyond sensory selection. Here we explore if crows are also capable of these forms of control over working memory. Two crows (Corvus corone) were trained to memorize two, four or six visual stimuli. Comparable to our previous results, the crows showed a decrease in performance with increasing working memory load. Using attention cues, we indicated the critical stimulus on a given trial. These cues were either presented before (pre-cue) or after sample-presentation (retro-cue). On other trials no cue was given as to which stimulus was critical. We found that both pre- and retro-cues enhance the performance of the birds. These results show that crows, like humans, can utilize attention to select relevant stimuli for maintenance in working memory. Importantly, crows can also utilize cues to make the most of their working memory capacity even after the stimuli are already held in working memory. This strongly implies that crows can engage in efficient control over working memory.

Differential Contribution of Cortical Thickness, Surface Area, and Gyrification to Fluid and Crystallized Intelligence

Human intelligence can be broadly subdivided into fluid (gf) and crystallized (gc) intelligence, each tapping into distinct cognitive abilities. Although neuroanatomical correlates of intelligence have been previously studied, differential contribution of cortical morphologies to gf and gc has not been fully delineated. Here, we tried to disentangle the contribution of cortical thickness, cortical surface area, and cortical gyrification to gf and gc in a large sample of healthy young subjects (n = 740, Human Connectome Project) with high-resolution MRIs, followed by replication in a separate data set with distinct cognitive measures indexing gf and gc. We found that while gyrification in distributed cortical regions had positive association with both gf and gc, surface area and thickness showed more regional associations. Specifically, higher performance in gf was associated with cortical expansion in regions related to working memory, attention, and visuo-spatial processing, while gc was associated with thinner cortex as well as higher cortical surface area in language-related networks. We discuss the results in a framework where "horizontal" cortical expansion enables higher resource allocation, computational capacity, and functional specificity relevant to gf and gc, while lower cortical thickness possibly reflects cortical pruning facilitating "vertical" intracolumnar efficiency in knowledge-based tasks relevant mostly to gc.

Activity in LIP, But not V4, Matches Performance When Attention is Spread

The enhancement of neuronal responses in many visual areas while animals perform spatial attention tasks has widely been thought to be the neural correlate of visual attention, but it is unclear whether the presence or absence of this modulation contributes to our striking inability to notice changes in change blindness examples. We asked whether neuronal responses in visual area V4 and the lateral intraparietal area (LIP) in posterior parietal cortex could explain the limited ability of subjects to attend multiple items in a display. We trained animals to perform a change detection task in which they had to compare 2 arrays of stimuli separated briefly in time and found that each animal's performance decreased as function of set-size. Neuronal discriminability in V4 was consistent across set-sizes, but decreased for higher set-sizes in LIP. The introduction of a reward bias produced attentional enhancement in V4, but this could not explain the vast improvement in performance, whereas the enhancement in LIP responses could. We suggest that behavioral set-size effects and the marked improvement in performance with focused attention may not be related to response enhancement in V4 but, instead, may occur in or on the way to LIP.

Establishing an infrastructure for collaboration in primate cognition research

Inferring the evolutionary history of cognitive abilities requires large and diverse samples. However, such samples are often beyond the reach of individual researchers or institutions, and studies are often limited to small numbers of species. Consequently, methodological and site-specific-differences across studies can limit comparisons between species. Here we introduce the ManyPrimates project, which addresses these challenges by providing a large-scale collaborative framework for comparative studies in primate cognition. To demonstrate the viability of the project we conducted a case study of short-term memory. In this initial study, we were able to include 176 individuals from 12 primate species housed at 11 sites across Africa, Asia, North America and Europe. All subjects were tested in a delayed-response task using consistent methodology across sites. Individuals could access food rewards by remembering the position of the hidden reward after a 0, 15, or 30-second delay. Overall, individuals performed better with shorter delays, as predicted by previous studies. Phylogenetic analysis revealed a strong phylogenetic signal for short-term memory. Although, with only 12 species, the validity of this analysis is limited, our initial results demonstrate the feasibility of a large, collaborative open-science project. We present the ManyPrimates project as an exciting opportunity to address open questions in primate cognition and behaviour with large, diverse datasets.

Error-correcting dynamics in visual working memory

Working memory is critical to cognition, decoupling behavior from the immediate world. Yet, it is imperfect; internal noise introduces errors into memory representations. Such errors have been shown to accumulate over time and increase with the number of items simultaneously held in working memory. Here, we show that discrete attractor dynamics mitigate the impact of noise on working memory. These dynamics pull memories towards a few stable representations in mnemonic space, inducing a bias in memory representations but reducing the effect of random diffusion. Model-based and model-free analyses of human and monkey behavior show that discrete attractor dynamics account for the distribution, bias, and precision of working memory reports. Furthermore, attractor dynamics are adaptive. They increase in strength as noise increases with memory load and experiments in humans show these dynamics adapt to the statistics of the environment, such that memories drift towards contextually-predicted values. Together, our results suggest attractor dynamics mitigate errors in working memory by counteracting noise and integrating contextual information into memories.

Neural representations in working memory are susceptible to internal noise, which scales with memory load. Here, the authors show that attractor dynamics mitigate the influence of internal noise by pulling memories towards a few stable representations.

Anterior insular cortex is a bottleneck of cognitive control

Cognitive control, with a limited capacity, is a core process in human cognition for the coordination of thoughts and actions. Although the regions involved in cognitive control have been identified as the cognitive control network (CCN), it is still unclear whether a specific region of the CCN serves as a bottleneck limiting the capacity of cognitive control (CCC). Here, we used a perceptual decision-making task with conditions of high cognitive load to challenge the CCN and to assess the CCC in a functional magnetic resonance imaging study. We found that the activation of the right anterior insular cortex (AIC) of the CCN increased monotonically as a function of cognitive load, reached its plateau early, and showed a significant correlation to the CCC. In a subsequent study of patients with unilateral lesions of the AIC, we found that lesions of the AIC were associated with a significant impairment of the CCC. Simulated lesions of the AIC resulted in a reduction of the global efficiency of the CCN in a network analysis. These findings suggest that the AIC, as a critical hub in the CCN, is a bottleneck of cognitive control.

Circuit mechanisms for the maintenance and manipulation of information in working memory

Recently it has been proposed that information in working memory (WM) may not always be stored in persistent neuronal activity but can be maintained in 'activity-silent' hidden states, such as synaptic efficacies endowed with short-term synaptic plasticity. To test this idea computationally, we investigated recurrent neural network models trained to perform several WM-dependent tasks, in which WM representation emerges from learning and is not a priori assumed to depend on self-sustained persistent activity. We found that short-term synaptic plasticity can support the short-term maintenance of information, provided that the memory delay period is sufficiently short. However, in tasks that require actively manipulating information, persistent activity naturally emerges from learning, and the amount of persistent activity scales with the degree of manipulation required. These results shed insight into the current debate on WM encoding and suggest that persistent activity can vary markedly between short-term memory tasks with different cognitive demands.

Different features are stored independently in visual working memory but mediated by object-based representations

The question of whether visual working memory (VWM) stores individual features or bound objects as basic units is actively debated. Evidence exists for both feature-based and object-based storages, as well as hierarchically organized representations maintaining both types of information at different levels. One argument for feature-based storage is that features belonging to different dimensions (e.g., color and orientations) can be stored without interference suggesting independent capacities for every dimension. Here, we studied whether the lack of cross-dimensional interference reflects genuinely independent feature storages or mediated by common objects. In three experiments, participants remembered and recalled the colors and orientations of sets of objects. We independently manipulated set sizes within each feature dimension (making colors and orientations either identical or differing across objects). Critically, we assigned to-be-remembered colors and orientations either to same spatially integrated or to different spatially separated objects. We found that the precision and recall probability within each dimension was not affected by set size manipulations in a different dimension when the features belonged to integrated objects. However, manipulations with color set sizes did affect orientation memory when the features were separated. We conclude therefore that different feature dimensions can be encoded and stored independently but the advantage of the independent storages are mediated at the object-based level. This conclusion is consistent with the idea of hierarchically organized VWM.

A Flexible Model of Working Memory

Working memory is fundamental to cognition, allowing one to hold information "in mind." A defining characteristic of working memory is its flexibility: we can hold anything in mind. However, typical models of working memory rely on finely tuned, content-specific attractors to persistently maintain neural activity and therefore do not allow for the flexibility observed in behavior. Here, we present a flexible model of working memory that maintains representations through random recurrent connections between two layers of neurons: a structured "sensory" layer and a randomly connected, unstructured layer. As the interactions are untuned with respect to the content being stored, the network maintains any arbitrary input. However, in our model, this flexibility comes at a cost: the random connections overlap, leading to interference between representations and limiting the memory capacity of the network. Additionally, our model captures several other key behavioral and neurophysiological characteristics of working memory.

Multi-target attention and visual short-term memory capacity are closely linked in the intraparietal sulcus

The existing literature suggests a critical role for both the right intraparietal sulcus (IPS) and the right temporo‐parietal junction (TPJ) in our ability to attend to multiple simultaneously‐presented lateralized targets (multi‐target attention), and the failure of this ability in extinction patients. Currently, however, the precise role of each of these areas in multi‐target attention is unclear. In this study, we combined the theory of visual attention (TVA) with functional magnetic resonance imaging (fMRI) guided continuous theta burst stimulation (cTBS) in neurologically healthy subjects to directly investigate the role of the right IPS and TPJ in multi‐target attention. Our results show that cTBS at an area of the right IPS associated with multi‐target attention elicits a reduction of visual short‐term memory capacity. This suggests that the right IPS is associated with a general capacity‐limited encoding mechanism that is engaged regardless of whether targets have to be attended or remembered. Curiously, however, cTBS to the right IPS failed to elicit extinction‐like behavior in our study, supporting previous suggestions that different areas of the right IPS may provide different contributions to multi‐target attention. CTBS to the right TPJ failed to induce a change in either TVA parameters or extinction‐like behavior.

Item-specific delay activity demonstrates concurrent storage of multiple active neural representations in working memory

Persistent neural activity that encodes online mental representations plays a central role in working memory (WM). However, there has been debate regarding the number of items that can be concurrently represented in this active neural state, which is often called the “focus of attention.” Some models propose a strict single-item limit, such that just 1 item can be neurally active at once while other items are relegated to an activity-silent state. Although past studies have decoded multiple items stored in WM, these studies cannot rule out a switching account in which only a single item is actively represented at a time. Here, we directly tested whether multiple representations can be held concurrently in an active state. We tracked spatial representations in WM using alpha-band (8–12 Hz) activity, which encodes spatial positions held in WM. Human observers remembered 1 or 2 positions over a short delay while we recorded electroencephalography (EEG) data. Using a spatial encoding model, we reconstructed active stimulus-specific representations (channel-tuning functions [CTFs]) from the scalp distribution of alpha-band power. Consistent with past work, we found that the selectivity of spatial CTFs was lower when 2 items were stored than when 1 item was stored. Critically, data-driven simulations revealed that the selectivity of spatial representations in the two-item condition could not be explained by models that propose that only a single item can exist in an active state at once. Thus, our findings demonstrate that multiple items can be concurrently represented in an active neural state.

An electroencephalography-based study shows that working memory delay activity concurrently represents two items, posing a challenge for models which propose that only one item can be actively represented at a time.

Working memory capacity is enhanced by distributed prefrontal activation and invariant temporal dynamics

The amount of information that can be stored in working memory is limited but may be improved with practice. The basis of improved efficiency at the level of neural activity is unknown. To investigate this question, we trained monkeys to perform a working memory task that required memory for multiple stimuli. Performance decreased as a function of number of stimuli to be remembered, but improved as the animals practiced the task. Neuronal recordings acquired during this training revealed two hitherto unknown mechanisms of working memory capacity improvement. First, more prefrontal neurons became active as working memory improved, but their baseline activity decreased. Second, improved working memory capacity was characterized by less variable temporal dynamics, resulting in a more consistent firing rate at each time point during the course of a trial. Our results reveal that improved performance of working memory tasks is achieved through more distributed activation and invariant neuronal dynamics.

Visual stimulus-driven functional organization of macaque prefrontal cortex

The extent to which the major subdivisions of prefrontal cortex (PFC) can be functionally partitioned is unclear. In approaching the question, it is often assumed that the organization is task dependent. Here we use fMRI to show that PFC can respond in a task-independent way, and we leverage these responses to uncover a stimulus-driven functional organization. The results were generated by mapping the relative location of responses to faces, bodies, scenes, disparity, color, and eccentricity in four passively fixating macaques. The results control for individual differences in functional architecture and provide the first account of a systematic visual stimulus-driven functional organization across PFC. Responses were focused in dorsolateral PFC (DLPFC), in the ventral prearcuate region; and in ventrolateral PFC (VLPFC), extending into orbital PFC. Face patches were in the VLPFC focus and were characterized by a striking lack of response to non-face stimuli rather than an especially strong response to faces. Color-biased regions were near but distinct from face patches. One scene-biased region was consistently localized with different contrasts and overlapped the disparity-biased region to define the DLPFC focus. All visually responsive regions showed a peripheral visual-field bias. These results uncover an organizational scheme that presumably constrains the flow of information about different visual modalities into PFC.

Dissecting the Neural Focus of Attention Reveals Distinct Processes for Spatial Attention and Object-Based Storage in Visual Working Memory

Complex cognition relies on both on-line representations in working memory (WM), said to reside in the focus of attention, and passive off-line representations of related information. Here, we dissected the focus of attention by showing that distinct neural signals index the on-line storage of objects and sustained spatial attention. We recorded electroencephalogram (EEG) activity during two tasks that employed identical stimulus displays but varied the relative demands for object storage and spatial attention. We found distinct delay-period signatures for an attention task (which required only spatial attention) and a WM task (which invoked both spatial attention and object storage). Although both tasks required active maintenance of spatial information, only the WM task elicited robust contralateral delay activity that was sensitive to mnemonic load. Thus, we argue that the focus of attention is maintained via a collaboration between distinct processes for covert spatial orienting and object-based storage.

The reliability and stability of visual working memory capacity

Because of the central role of working memory capacity in cognition, many studies have used short measures of working memory capacity to examine its relationship to other domains. Here, we measured the reliability and stability of visual working memory capacity, measured using a single-probe change detection task. In Experiment 1, the participants (N = 135) completed a large number of trials of a change detection task (540 in total, 180 each of set sizes 4, 6, and 8). With large numbers of both trials and participants, reliability estimates were high (α > .9). We then used an iterative down-sampling procedure to create a look-up table for expected reliability in experiments with small sample sizes. In Experiment 2, the participants (N = 79) completed 31 sessions of single-probe change detection. The first 30 sessions took place over 30 consecutive days, and the last session took place 30 days later. This unprecedented number of sessions allowed us to examine the effects of practice on stability and internal reliability. Even after much practice, individual differences were stable over time (average between-session r = .76).

EEG microstates distinguish between cognitive components of fluid reasoning

Fluid reasoning is considered central to general intelligence. How its psychometric structure relates to brain function remains poorly understood. For instance, what is the dynamic composition of ability-specific processes underlying fluid reasoning? We investigated whether distinct fluid reasoning abilities could be differentiated by electroencephalography (EEG) microstate profiles. EEG microstates specifically capture rapidly altering activity of distributed cortical networks with a high temporal resolution as scalp potential topographies that dynamically vary over time in an organized manner. EEG was recorded simultaneously with functional magnetic resonance imaging (fMRI) in twenty healthy adult participants during cognitively distinct fluid reasoning tasks: induction, spatial relationships and visualization. Microstate parameters successfully discriminated between fluid reasoning and visuomotor control tasks as well as between the fluid reasoning tasks. Mainly, microstate B coverage was significantly higher during spatial relationships and visualization, compared to induction, while microstate C coverage was significantly decreased during spatial relationships and visualization, compared to induction. Additionally, microstate D coverage was highest during spatial relationships and microstate A coverage was most strongly reduced during the same condition. Consistently, multivariate analysis with a leave-one-out cross-validation procedure accurately classified the fluid reasoning tasks based on the coverage parameter. These EEG data and their correlation with fMRI data suggest that especially the tasks most strongly relying on visuospatial processing modulated visual and default mode network activity. We propose that EEG microstates can provide valuable information about neural activity patterns with a dynamic and complex temporal structure during fluid reasoning, suggesting cognitive ability-specific interplays between multiple brain networks.

Working Memory 2.0

Working memory is the fundamental function by which we break free from reflexive input-output reactions to gain control over our own thoughts. It has two types of mechanisms: online maintenance of information and its volitional or executive control. Classic models proposed persistent spiking for maintenance but have not explicitly addressed executive control. We review recent theoretical and empirical studies that suggest updates and additions to the classic model. Synaptic weight changes between sparse bursts of spiking strengthen working memory maintenance. Executive control acts via interplay between network oscillations in gamma (30-100 Hz) in superficial cortical layers (layers 2 and 3) and alpha and beta (10-30 Hz) in deep cortical layers (layers 5 and 6). Deep-layer alpha and beta are associated with top-down information and inhibition. It regulates the flow of bottom-up sensory information associated with superficial layer gamma. We propose that interactions between different rhythms in distinct cortical layers underlie working memory maintenance and its volitional control.

Fast-backward replay of sequentially memorized items in humans

Storing temporal sequences of events (i.e., sequence memory) is fundamental to many cognitive functions. However, it is unknown how the sequence order information is maintained and represented in working memory and its behavioral significance, particularly in human subjects. We recorded electroencephalography (EEG) in combination with a temporal response function (TRF) method to dissociate item-specific neuronal reactivations. We demonstrate that serially remembered items are successively reactivated during memory retention. The sequential replay displays two interesting properties compared to the actual sequence. First, the item-by-item reactivation is compressed within a 200 – 400 ms window, suggesting that external events are associated within a plasticity-relevant window to facilitate memory consolidation. Second, the replay is in a temporally reversed order and is strongly related to the recency effect in behavior. This fast-backward replay, previously revealed in rat hippocampus and demonstrated here in human cortical activities, might constitute a general neural mechanism for sequence memory and learning.

Have you ever played the ‘Memory Maze Challenge’ game, or its predecessor from the 1980s, ‘Simon’? Players must memorize a sequence of colored lights, and then reproduce the sequence by tapping the colors on a pad. The sequence becomes longer with each trial, making the task more and more difficult. One wrong response and the game is over.

Storing and retrieving sequences is key to many cognitive processes, from following speech to hitting a tennis ball to recalling what you did last week. Such tasks require memorizing the order in which items occur as well as the items themselves. But how do we hold this information in memory? Huang et al. reveal the answer by using scalp electrodes to record the brain activity of healthy volunteers as they memorize and then recall a sequence.

Memorizing, or encoding, each of the items in the sequence triggered a distinct pattern of brain activity. As the volunteers held the sequence in memory, their brains replayed these activity patterns one after the other. But this replay showed two non-intuitive features. First, it was speeded up relative to the original encoding. In fact, the brain compressed the entire sequence into about 200 to 400 milliseconds. Second, the brain replayed the sequence backwards. The activity pattern corresponding to the last item was replayed first, while that corresponding to the first item was replayed last. This ‘fast-backward’ replay may explain why we tend to recall items at the end of a list better than those in the middle, a phenomenon known as the recency effect.

The results of Huang et al. suggest that when we hold a list of items in memory, the brain does not replay the list in its original form, like an echo. Instead, the brain restructures and reorganizes the list, compressing and reversing it. This process, which is also seen in rodents, helps the brain to incorporate the list of items into existing neuronal networks for memory storage.

Internal but not external noise frees working memory resources

The precision with which visual information can be recalled from working memory declines as the number of items in memory increases. This finding has been explained in terms of the distribution of a limited representational resource between items. Here we investigated how the sensory strength of memoranda affects resource allocation. We manipulated signal strength of an orientation stimulus in two ways: we varied the internal (sensory) noise by adjusting stimulus contrast, and varied the external (stimulus) noise by altering the within-stimulus variability. Both manipulations had similar effects on the precision with which the orientation could be recalled, but differed in their impact on memory for other stimuli. These results indicate that increasing internal noise released resources that could be used to store other stimuli more precisely; increasing external noise had no such effect. We show that these observations can be captured by a simple neural model of working memory encoding, in which spiking activity takes on the role of the limited resource.

Investigations of visual short-term memory typically involve memorising clearly visible objects with elementary features, such as monochromatic disks or oriented bars. Results of such studies indicate that memory is allocated like a limited resource, i.e. shared out between objects. However, in daily life we are often confronted with visual features that are difficult to make out, like when an object is in shadow, or poorly-defined, like the color of a variegated leaf. Here we asked whether these kinds of features occupy as much memory resource as simple highly-visible objects. Our results demonstrate that reducing the sensory strength of a stimulus makes the quality of recall worse, but also takes up less resource so other objects can be remembered more precisely. Increasing the variability within a stimulus worsens recall, but has no effect on how other objects are remembered. These findings can be explained by considering how visual information is stored in populations of neurons: only the manipulation of sensory strength changes the amount of spiking activity dedicated to a stimulus.

Intrinsic neuronal dynamics predict distinct functional roles during working memory

Working memory (WM) is characterized by the ability to maintain stable representations over time; however, neural activity associated with WM maintenance can be highly dynamic. We explore whether complex population coding dynamics during WM relate to the intrinsic temporal properties of single neurons in lateral prefrontal cortex (lPFC), the frontal eye fields (FEF), and lateral intraparietal cortex (LIP) of two monkeys (Macaca mulatta). We find that cells with short timescales carry memory information relatively early during memory encoding in lPFC; whereas long-timescale cells play a greater role later during processing, dominating coding in the delay period. We also observe a link between functional connectivity at rest and the intrinsic timescale in FEF and LIP. Our results indicate that individual differences in the temporal processing capacity predict complex neuronal dynamics during WM, ranging from rapid dynamic encoding of stimuli to slower, but stable, maintenance of mnemonic information.

Prefrontal neurons exhibit both transient and persistent firing in working memory tasks. Here the authors report that the intrinsic timescale of neuronal firing outside the task is predictive of the temporal dynamics of coding during working memory in three frontoparietal brain areas.

Distinct roles of prefrontal and parietal areas in the encoding of attentional priority

When searching for an object in a crowded scene, information about the similarity of stimuli to the target object is thought to be encoded in spatial priority maps, which are subsequently used to guide shifts of attention and gaze to likely targets. Two key cortical areas that have been described as holding priority maps are the frontal eye field (FEF) and the lateral intraparietal area (LIP). However, little is known about their distinct contributions in priority encoding. Here, we compared neuronal responses in FEF and LIP during free-viewing visual search. Although saccade selection signals emerged earlier in FEF, information about the target emerged at similar latencies in distinct populations within the two areas. Notably, however, effects in FEF were more pronounced. Moreover, LIP neurons encoded the similarity of stimuli to the target independent of saccade selection, whereas in FEF, encoding of target similarity was strongly modulated by saccade selection. Taken together, our findings suggest hierarchical processing of saccade selection signals and parallel processing of feature-based attention signals within the parietofrontal network with FEF having a more prominent role in priority encoding. Furthermore, they suggest discrete roles of FEF and LIP in the construction of priority maps.

The Posterior Parietal Cortex in Adaptive Visual Processing

Although the primate posterior parietal cortex (PPC) has been largely associated with space, attention, and action-related processing, a growing number of studies have reported the direct representation of a diverse array of action-independent nonspatial visual information in the PPC during both perception and visual working memory. By describing the distinctions and the close interactions of visual representation with space, attention, and action-related processing in the PPC, here I propose that we may understand these diverse PPC functions together through the unique contribution of the PPC to adaptive visual processing and form a more integrated and structured view of the role of the PPC in vision, cognition, and action.