Neural substrates of cognitive capacity limitations

Representation of multiple objects in macaque category-selective areas

Object recognition in the natural world usually occurs in the presence of multiple surrounding objects, but responses of neurons in inferotemporal (IT) cortex, the large brain area responsible for object recognition, have mostly been studied only to isolated objects. We study rules governing responses to multiple objects by cells in two category-selective regions of macaque IT cortex, the middle lateral face patch (ML) and the middle body patch (MB). We find that responses of single ML and MB cells to pairs of objects can be explained by the widely accepted framework of normalization, with one added ingredient: homogeneous category selectivity of neighboring neurons forming the normalization pool. This rule leads to winner-take-all, contralateral-take-all, or weighted averaging behavior in single cells, depending on the category, spatial configuration, and relative contrast of the two objects. The winner-take-all behavior suggests a potential mechanism for clutter-invariant representation of face and bodies under certain conditions.

Inferotemporal cortex (IT) neurons respond to specific objects but the precise neural mechanisms for clutter-invariant representation is not known. Here the authors show that face and body patch IT neurons respond to multiple objects with winner-take-all, contralateral-take-all or weighted averaging depending on the stimulus properties.

Synaptic efficacy shapes resource limitations in working memory

Working memory (WM) is limited in its temporal length and capacity. Classic conceptions of WM capacity assume the system possesses a finite number of slots, but recent evidence suggests WM may be a continuous resource. Resource models typically assume there is no hard upper bound on the number of items that can be stored, but WM fidelity decreases with the number of items. We analyze a neural field model of multi-item WM that associates each item with the location of a bump in a finite spatial domain, considering items that span a one-dimensional continuous feature space. Our analysis relates the neural architecture of the network to accumulated errors and capacity limitations arising during the delay period of a multi-item WM task. Networks with stronger synapses support wider bumps that interact more, whereas networks with weaker synapses support narrower bumps that are more susceptible to noise perturbations. There is an optimal synaptic strength that both limits bump interaction events and the effects of noise perturbations. This optimum shifts to weaker synapses as the number of items stored in the network is increased. Our model not only provides a circuit-based explanation for WM capacity, but also speaks to how capacity relates to the arrangement of stored items in a feature space.

The speed of parietal theta frequency drives visuospatial working memory capacity

The speed of theta brain oscillatory activity is thought to play a key role in determining working memory (WM) capacity. Individual differences in the length of a theta cycle (ranging between 4 and 7 Hz) might determine how many gamma cycles (>30 Hz) can be nested into a theta wave. Gamma cycles are thought to represent single memory items; therefore, this interplay could determine individual memory capacity. We directly tested this hypothesis by means of parietal transcranial alternating current stimulation (tACS) set at slower (4 Hz) and faster (7 Hz) theta frequencies during a visuospatial WM paradigm. Accordingly, we found that 4-Hz tACS enhanced WM capacity, while 7-Hz tACS reduced WM capacity. Notably, these effects were found only for items presented to the hemifield contralateral to the stimulation site. This provides causal evidence for a frequency-dependent and spatially specific organization of WM storage, supporting the theta-gamma phase coupling theory of WM capacity.

Mixed selectivity morphs population codes in prefrontal cortex

The prefrontal cortex maintains working memory information in the presence of distracting stimuli. It has long been thought that sustained activity in individual neurons or groups of neurons was responsible for maintaining information in the form of a persistent, stable code. Here we show that, upon the presentation of a distractor, information in the lateral prefrontal cortex was reorganized into a different pattern of activity to create a morphed stable code without losing information. In contrast, the code in the frontal eye fields persisted across different delay periods but exhibited substantial instability and information loss after the presentation of a distractor. We found that neurons with mixed-selective responses were necessary and sufficient for the morphing of code and that these neurons were more abundant in the lateral prefrontal cortex than the frontal eye fields. This suggests that mixed selectivity provides populations with code-morphing capability, a property that may underlie cognitive flexibility.

Lateralization of Executive Function: Working Memory Advantage for Same Hemifield Stimuli in the Monkey

Working memory capacity, the amount of information that may be maintained in mind over a period of seconds, is extremely limited, to a handful of items. Some evidence exists that the number of visual items that may be maintained in working memory is independent for the two hemifields. To test this idea, we trained monkeys to perform visual working memory tasks that required maintenance in memory of the locations and/or shapes of 3–5 visual stimuli. We then tested whether systematic performance differences were present for stimuli concentrated in the same hemifield, vs. distributed across hemifields. We found little evidence to support the expectation that working memory capacity is independent in the two hemifields. Instead, when an advantage of stimulus arrangement was present, it involved multiple stimuli presented in the same hemifield. This conclusion was consistent across variations of the task, performance levels, and apparent strategies adopted by individual subjects. This result suggests that factors such as grouping that favor processing of stimuli in relative proximity may counteract the benefits of independent processing in the two hemispheres. Our results reveal an important property of working memory and place constraints on models of working memory capacity.

Cognitive-motivational interactions: beyond boxes-and-arrows models of the mind-brain

How do motivation and cognitive control interact in brain and behavior? The past decade has witnessed a steady growth in studies investigating both the behavioral and the brain basis of these interactions. In this paper, I describe such interactions in the context of the dual completion model, which proposes that motivational significance influences both perceptual and executive competition. Embracing a research agenda that attempts to understand cognition-motivation interactions highlights considerable challenges faced by investigators. For example, even the standard language utilized, with terms such as "perception," "attention," "cognition," and "motivation," encourages a modular-like conceptualization of the underlying processes and mechanisms. I propose that large-scale interactions involving both task-related and valuation-related networks help understand how motivation shapes executive function. I argue that, ultimately, the mind and brain sciences need to move beyond "boxes and arrows" and fully embrace the richness and complexity of the interactions between motivation and cognition. In the last 10 years, the study in humans of the interactions of motivation with perception and cognition has grown at a fast pace. The growth has included behavioral studies characterizing the processes involved, and neuroimaging studies investigating the regions and circuits underlying the behaviors in question. This literature acknowledges the fact that perception and cognition do not happen in a vacuum but are, instead, situated in contexts that feature value. Although this assertion is uncontroversial, the mind and brain sciences have studied perception and cognition for many decades by largely extricating value from them. Fortunately, this state of affairs has now changed and the field has a newfound vigor in attempting to understand the impact of motivation on these mental functions.

Working memory accuracy for multiple targets is driven by reward expectation and stimulus contrast with different time-courses

The richness of sensory input dictates that the brain must prioritize and select information for further processing and storage in working memory. Stimulus salience and reward expectations influence this prioritization but their relative contributions and underlying mechanisms are poorly understood. Here we investigate how the quality of working memory for multiple stimuli is determined by priority during encoding and later memory phases. Selective attention could, for instance, act as the primary gating mechanism when stimuli are still visible. Alternatively, observers might still be able to shift priorities across memories during maintenance or retrieval. To distinguish between these possibilities, we investigated how and when reward cues determine working memory accuracy and found that they were only effective during memory encoding. Previously learned, but currently non-predictive, color-reward associations had a similar influence, which gradually weakened without reinforcement. Finally, we show that bottom-up salience, manipulated through varying stimulus contrast, influences memory accuracy during encoding with a fundamentally different time-course than top-down reward cues. While reward-based effects required long stimulus presentation, the influence of contrast was strongest with brief presentations. Our results demonstrate how memory resources are distributed over memory targets and implicates selective attention as a main gating mechanism between sensory and memory systems.

Crows Rival Monkeys in Cognitive Capacity

The present study compares the ‘bandwidth of cognition’ between crows and primates. Working memory is the ability to maintain and manipulate information over short periods of time – a core component of cognition. The capacity of working memory is tightly limited, in humans correlated with individual intelligence and commonly used synonymously with cognitive capacity. Crows have remarkable cognitive skills and while birds and mammals share neural principles of working memory, its capacity has not been tested in crows. Here we report the performance of two carrion crows on a working memory paradigm adapted from a recent experiment in rhesus monkeys. Capacity of crows is remarkably similar to monkeys and estimated at about four items. In both species, the visual hemifields show largely independent capacity. These results show that crows, like primates evolved a high-capacity working memory that reflects the result of convergent evolution of higher cognitive abilities in both species.

Feature-Selective Attentional Modulations in Human Frontoparietal Cortex

Control over visual selection has long been framed in terms of a dichotomy between "source" and "site," where top-down feedback signals originating in frontoparietal cortical areas modulate or bias sensory processing in posterior visual areas. This distinction is motivated in part by observations that frontoparietal cortical areas encode task-level variables (e.g., what stimulus is currently relevant or what motor outputs are appropriate), while posterior sensory areas encode continuous or analog feature representations. Here, we present evidence that challenges this distinction. We used fMRI, a roving searchlight analysis, and an inverted encoding model to examine representations of an elementary feature property (orientation) across the entire human cortical sheet while participants attended either the orientation or luminance of a peripheral grating. Orientation-selective representations were present in a multitude of visual, parietal, and prefrontal cortical areas, including portions of the medial occipital cortex, the lateral parietal cortex, and the superior precentral sulcus (thought to contain the human homolog of the macaque frontal eye fields). Additionally, representations in many-but not all-of these regions were stronger when participants were instructed to attend orientation relative to luminance. Collectively, these findings challenge models that posit a strict segregation between sources and sites of attentional control on the basis of representational properties by demonstrating that simple feature values are encoded by cortical regions throughout the visual processing hierarchy, and that representations in many of these areas are modulated by attention.

SIGNIFICANCE STATEMENT

Influential models of visual attention posit a distinction between top-down control and bottom-up sensory processing networks. These models are motivated in part by demonstrations showing that frontoparietal cortical areas associated with top-down control represent abstract or categorical stimulus information, while visual areas encode parametric feature information. Here, we show that multivariate activity in human visual, parietal, and frontal cortical areas encode representations of a simple feature property (orientation). Moreover, representations in several (though not all) of these areas were modulated by feature-based attention in a similar fashion. These results provide an important challenge to models that posit dissociable top-down control and sensory processing networks on the basis of representational properties.

Restoring Latent Visual Working Memory Representations in Human Cortex

Working memory (WM) enables the storage and manipulation of limited amounts of information over short periods. Prominent models posit that increasing the number of remembered items decreases the spiking activity dedicated to each item via mutual inhibition, which irreparably degrades the fidelity of each item's representation. We tested these models by determining if degraded memory representations could be recovered following a post-cue indicating which of several items in spatial WM would be recalled. Using an fMRI-based image reconstruction technique, we identified impaired behavioral performance and degraded mnemonic representations with elevated memory load. However, in several cortical regions, degraded mnemonic representations recovered substantially following a post-cue, and this recovery tracked behavioral performance. These results challenge pure spike-based models of WM and suggest that remembered items are additionally encoded within latent or hidden neural codes that can help reinvigorate active WM representations.

Superior Intraparietal Sulcus Controls the Variability of Visual Working Memory Precision

Limitations of working memory (WM) capacity depend strongly on the cognitive resources that are available for maintaining WM contents in an activated state. Increasing the number of items to be maintained in WM was shown to reduce the precision of WM and to increase the variability of WM precision over time. Although WM precision was recently associated with neural codes particularly in early sensory cortex, we have so far no understanding of the neural bases underlying the variability of WM precision, and how WM precision is preserved under high load. To fill this gap, we combined human fMRI with computational modeling of behavioral performance in a delayed color-estimation WM task. Behavioral results replicate a reduction of WM precision and an increase of precision variability under high loads (5 > 3 > 1 colors). Load-dependent BOLD signals in primary visual cortex (V1) and superior intraparietal sulcus (IPS), measured during the WM task at 2-4 s after sample onset, were modulated by individual differences in load-related changes in the variability of WM precision. Although stronger load-related BOLD increase in superior IPS was related to lower increases in precision variability, thus stabilizing WM performance, the reverse was observed for V1. Finally, the detrimental effect of load on behavioral precision and precision variability was accompanied by a load-related decline in the accuracy of decoding the memory stimuli (colors) from left superior IPS. We suggest that the superior IPS may contribute to stabilizing visual WM performance by reducing the variability of memory precision in the face of higher load.

SIGNIFICANCE STATEMENT

This study investigates the neural bases of capacity limitations in visual working memory by combining fMRI with cognitive modeling of behavioral performance, in human participants. It provides evidence that the superior intraparietal sulcus (IPS) is a critical brain region that influences the variability of visual working memory precision between and within individuals (Fougnie et al., 2012; van den Berg et al., 2012) under increased memory load, possibly in cooperation with perceptual systems of the occipital cortex. These findings substantially extend our understanding of the nature of capacity limitations in visual working memory and their neural bases. Our work underlines the importance of integrating cognitive modeling with univariate and multivariate methods in fMRI research, thus improving our knowledge of brain-behavior relationships.

Functional network connectivity alterations in schizophrenia and depression

There is a high degree of overlap between the symptoms of major depressive disorder (MDD) and schizophrenia, but it remains unclear whether the similar symptoms are derived from convergent alterations in functional network connectivity. In this study, we performed a group independent component analysis on resting-state functional MRI data from 20 MDD patients, 24 schizophrenia patients, and 43 matched healthy controls. The functional network connectivity analysis revealed that, compared to healthy controls, the MDD and schizophrenia patients exhibited convergent decreased positive connectivity between the left and right fronto-parietal control network and decreased negative connectivity between the left control and medial visual networks. Furthermore, the MDD patients showed decreased negative connectivity between the left control and auditory networks, and the schizophrenia patients showed decreased positive connectivity between the bilateral control and language networks and decreased negative connectivity between the right control and dorsal attention networks. The convergent network connectivity alterations may underlie the common primary control and regulation disorders, and the divergent connectivity alterations may enable the distinction between the two disorders. All of the convergent and divergent network connectivity alterations were relevant to the control network, suggesting an important role of the network in the pathophysiology of MDD and schizophrenia.

Mnemonic Encoding and Cortical Organization in Parietal and Prefrontal Cortices

Persistent activity within the frontoparietal network is consistently observed during tasks that require working memory. However, the neural circuit mechanisms underlying persistent neuronal encoding within this network remain unresolved. Here, we ask how neural circuits support persistent activity by examining population recordings from posterior parietal (PPC) and prefrontal (PFC) cortices in two male monkeys that performed spatial and motion direction-based tasks that required working memory. While spatially selective persistent activity was observed in both areas, robust selective persistent activity for motion direction was only observed in PFC. Crucially, we find that this difference between mnemonic encoding in PPC and PFC is associated with the presence of functional clustering: PPC and PFC neurons up to ∼700 μm apart preferred similar spatial locations, and PFC neurons up to ∼700 μm apart preferred similar motion directions. In contrast, motion-direction tuning similarity between nearby PPC neurons was much weaker and decayed rapidly beyond ∼200 μm. We also observed a similar association between persistent activity and functional clustering in trained recurrent neural network models embedded with a columnar topology. These results suggest that functional clustering facilitates mnemonic encoding of sensory information.SIGNIFICANCE STATEMENT Working memory refers to our ability to temporarily store and manipulate information. Numerous studies have observed that, during working memory, neurons in higher cortical areas, such as the parietal and prefrontal cortices, mnemonically encode the remembered stimulus. However, several recent studies have failed to observe mnemonic encoding during working memory, raising the question as to why mnemonic encoding is observed during some, but not all, conditions. In this study, we show that mnemonic encoding occurs when a cortical area is organized such that nearby neurons preferentially respond to the same stimulus. This result provides plausible neuronal conditions that allow for mnemonic encoding, and gives us further understanding of the brain's mechanisms that support working memory.

Working Memory in the Prefrontal Cortex

The prefrontal cortex participates in a variety of higher cognitive functions. The concept of working memory is now widely used to understand prefrontal functions. Neurophysiological studies have revealed that stimulus-selective delay-period activity is a neural correlate of the mechanism for temporarily maintaining information in working memory processes. The central executive, which is the master component of Baddeley's working memory model and is thought to be a function of the prefrontal cortex, controls the performance of other components by allocating a limited capacity of memory resource to each component based on its demand. Recent neurophysiological studies have attempted to reveal how prefrontal neurons achieve the functions of the central executive. For example, the neural mechanisms of memory control have been examined using the interference effect in a dual-task paradigm. It has been shown that this interference effect is caused by the competitive and overloaded recruitment of overlapping neural populations in the prefrontal cortex by two concurrent tasks and that the information-processing capacity of a single neuron is limited to a fixed level, can be flexibly allocated or reallocated between two concurrent tasks based on their needs, and enhances behavioral performance when its allocation to one task is increased. Further, a metamemory task requiring spatial information has been used to understand the neural mechanism for monitoring its own operations, and it has been shown that monitoring the quality of spatial information represented by prefrontal activity is an important factor in the subject's choice and that the strength of spatially selective delay-period activity reflects confidence in decision-making. Although further studies are needed to elucidate how the prefrontal cortex controls memory resource and supervises other systems, some important mechanisms related to the central executive have been identified.

Monkey prefrontal neurons during Sternberg task performance: full contents of working memory or most recent item?

To explore the brain mechanisms underlying multi-item working memory, we monitored the activity of neurons in the dorsolateral prefrontal cortex while macaque monkeys performed spatial and chromatic versions of a Sternberg working-memory task. Each trial required holding three sequentially presented samples in working memory so as to identify a subsequent probe matching one of them. The monkeys were able to recall all three samples at levels well above chance, exhibiting modest load and recency effects. Prefrontal neurons signaled the identity of each sample during the delay period immediately following its presentation. However, as each new sample was presented, the representation of antecedent samples became weak and shifted to an anomalous code. A linear classifier operating on the basis of population activity during the final delay period was able to perform at approximately the level of the monkeys on trials requiring recall of the third sample but showed a falloff in performance on trials requiring recall of the first or second sample much steeper than observed in the monkeys. We conclude that delay-period activity in the prefrontal cortex robustly represented only the most recent item. The monkeys apparently based performance of this classic working-memory task on some storage mechanism in addition to the prefrontal delay-period firing rate. Possibilities include delay-period activity in areas outside the prefrontal cortex and changes within the prefrontal cortex not manifest at the level of the firing rate.NEW & NOTEWORTHY It has long been thought that items held in working memory are encoded by delay-period activity in the dorsolateral prefrontal cortex. Here we describe evidence contrary to that view. In monkeys performing a serial multi-item working memory task, dorsolateral prefrontal neurons encode almost exclusively the identity of the sample presented most recently. Information about earlier samples must be encoded outside the prefrontal cortex or represented within the prefrontal cortex in a cryptic code.

Peripheral target identification performance modulates eye movements

We often shift our eyes to an interesting stimulus, but it is important to inhibit that eye movement in some environments (e.g., a no-look pass in basketball). Here, we investigated participants' ability to inhibit eye movements when they had to process a peripheral target with a requirement to maintain strict fixation. An array of eight letters composed of four characters was briefly presented and a directional cue was centrally presented to indicate the target location. The stimulus onset asynchrony (SOA) between the cue and the stimulus array was chosen from six values, consisting of pre-cue conditions (-400 and -200ms), a simultaneous cue condition (0ms), and post-cue conditions (200, 400, and 800ms). We found the following: 1) participants shifted their eyes toward the cued location even though the stimulus array was absent at the onset of eye movements, but the eye movement amplitude was smaller than the actual location of the target; 2) eye movements occurred approximately 150ms after the onset of stimulus array in the pre-cue conditions and 250ms after cue onset in the simultaneous and post-cue conditions; and 3) eye movement onsets were delayed and their amplitudes were smaller in correct trials than incorrect trials. These results indicate that the inhibitory process controlling eye movements also compete for cognitive resources like other cognitive processes.

The Distributed Nature of Working Memory

Studies in humans and non-human primates have provided evidence for storage of working memory contents in multiple regions ranging from sensory to parietal and prefrontal cortex. We discuss potential explanations for these distributed representations: (i) features in sensory regions versus prefrontal cortex differ in the level of abstractness and generalizability; and (ii) features in prefrontal cortex reflect representations that are transformed for guidance of upcoming behavioral actions. We propose that the propensity to produce persistent activity is a general feature of cortical networks. Future studies may have to shift focus from asking where working memory can be observed in the brain to how a range of specialized brain areas together transform sensory information into a delayed behavioral response.

Induced and Evoked Human Electrophysiological Correlates of Visual Working Memory Set-Size Effects at Encoding

The ability to encode, store, and retrieve visually presented objects is referred to as visual working memory (VWM). Although crucial for many cognitive processes, previous research reveals that VWM strictly capacity limited. This capacity limitation is behaviorally observable in the set size effect: the ability to successfully report items in VWM asymptotes at a small number of items. Research into the neural correlates of set size effects and VWM capacity limits in general largely focus on the maintenance period of VWM. However, we previously reported that neural resources allocated to individual items during VWM encoding correspond to successful VWM performance. Here we expand on those findings by investigating neural correlates of set size during VWM encoding. We hypothesized that neural signatures of encoding-related VWM capacity limitations should be differentiable as a function of set size. We tested our hypothesis using High Density Electroencephalography (HD-EEG) to analyze frequency components evoked by flickering target items in VWM displays of set size 2 or 4. We found that set size modulated the amplitude of the 1st and 2nd harmonic frequencies evoked during successful VWM encoding across frontal and occipital-parietal electrodes. Frontal sites exhibited the most robust effects for the 2nd harmonic (set size 2 > set size 4). Additionally, we found a set-size effect on the induced power of delta-band (1-4 Hz) activity (set size 2 > set size 4). These results are consistent with a capacity limited VWM resource at encoding that is distributed across to-be-remembered items in a VWM display. This resource may work in conjunction with a task-specific selection process that determines which items are to be encoded and which are to be ignored. These neural set size effects support the view that VWM capacity limitations begin with encoding related processes.

Synchronous beta rhythms of frontoparietal networks support only behaviorally relevant representations

Categorization has been associated with distributed networks of the primate brain, including the prefrontal cortex (PFC) and posterior parietal cortex (PPC). Although category-selective spiking in PFC and PPC has been established, the frequency-dependent dynamic interactions of frontoparietal networks are largely unexplored. We trained monkeys to perform a delayed-match-to-spatial-category task while recording spikes and local field potentials from the PFC and PPC with multiple electrodes. We found category-selective beta- and delta-band synchrony between and within the areas. However, in addition to the categories, delta synchrony and spiking activity also reflected irrelevant stimulus dimensions. By contrast, beta synchrony only conveyed information about the task-relevant categories. Further, category-selective PFC neurons were synchronized with PPC beta oscillations, while neurons that carried irrelevant information were not. These results suggest that long-range beta-band synchrony could act as a filter that only supports neural representations of the variables relevant to the task at hand.

DOI:http://dx.doi.org/10.7554/eLife.17822.001

A brain that could store only exact experiences would bog us down with details. We have instead evolved to be able to detect the common elements in different experiences and group them into meaningful categories. This imbues the world with meaning. We can recognize and respond appropriately to objects, situations and expressions even if we have never encountered those exact examples before. Without this ability, experiences would be fragmented and unrelated. Things would seem strange and unfamiliar if they differed even trivially from previous examples. This situation describes many of the characteristics of neuropsychiatric disorders such as autism and schizophrenia.

Most studies have focused on how single brain areas or single neurons categorize experiences. The brain, however, is composed of many interacting networks of neurons that extend across several different areas of the brain. Repetitive rhythms, or waves, of electrical activity generated by the neurons seem to play a major role in network interactions. These rhythms are given different names depending on how rapidly they oscillate. In order for our brain to work successfully, these rhythms synchronize across the relevant brain areas.

Antzoulatos and Miller trained monkeys to categorize stimuli as “Above” or “Below” depending on where dots and lines appeared on a screen. The activity of the neurons in two regions of the monkey’s brain – called the prefrontal cortex and the parietal cortex – was recorded as each monkey performed the task. The recordings revealed that synchronized rhythms between the prefrontal and parietal cortices supported the monkey’s ability to categorize the stimuli. Changes in how strongly the rhythmic electrical activity of the neurons was synchronized – particularly for a type of wave called a beta wave – conveyed information about the category of stimuli (i.e., whether they counted as Above or Below). Single neurons also conveyed this categorization, but unlike rhythms, they also carried irrelevant information. Therefore the synchronized beta waves could act as a filter for the features of an object or experience that are relevant to the task at hand.

The prefrontal cortex and the parietal cortex are only two of many brain areas involved in categorization. Much more territory remains to explore.

DOI:http://dx.doi.org/10.7554/eLife.17822.002

Oscillatory Dynamics of Prefrontal Cognitive Control

The prefrontal cortex (PFC) provides the structural basis for numerous higher cognitive functions. However, it is still largely unknown which mechanisms provide the functional basis for flexible cognitive control of goal-directed behavior. Here, we review recent findings that suggest that the functional architecture of cognition is profoundly rhythmic and propose that the PFC serves as a conductor to orchestrate task-relevant large-scale networks. We highlight several studies that demonstrated that oscillatory dynamics, such as phase resetting, cross-frequency coupling (CFC), and entrainment, support PFC-dependent recruitment of task-relevant regions into coherent functional networks. Importantly, these findings support the notion that distinct spectral signatures reflect different cortical computations supporting effective multiplexing on different temporal channels along the same anatomical pathways.

Cross-frequency synchronization connects networks of fast and slow oscillations during visual working memory maintenance

Neuronal activity in sensory and fronto-parietal (FP) areas underlies the representation and attentional control, respectively, of sensory information maintained in visual working memory (VWM). Within these regions, beta/gamma phase-synchronization supports the integration of sensory functions, while synchronization in theta/alpha bands supports the regulation of attentional functions. A key challenge is to understand which mechanisms integrate neuronal processing across these distinct frequencies and thereby the sensory and attentional functions. We investigated whether such integration could be achieved by cross-frequency phase synchrony (CFS). Using concurrent magneto- and electroencephalography, we found that CFS was load-dependently enhanced between theta and alpha–gamma and between alpha and beta-gamma oscillations during VWM maintenance among visual, FP, and dorsal attention (DA) systems. CFS also connected the hubs of within-frequency-synchronized networks and its strength predicted individual VWM capacity. We propose that CFS integrates processing among synchronized neuronal networks from theta to gamma frequencies to link sensory and attentional functions.

DOI:http://dx.doi.org/10.7554/eLife.13451.001

The Capacity of Cognitive Control Estimated from a Perceptual Decision Making Task

Cognitive control refers to the processes that permit selection and prioritization of information processing in different cognitive domains to reach the capacity-limited conscious mind. Although previous studies have suggested that the capacity of cognitive control itself is limited, a direct quantification of this capacity has not been attempted. In this behavioral study, we manipulated the information rate of cognitive control by parametrically varying both the uncertainty of stimul measured as information entropy and the exposure time of the stimuli. We used the relationship between the participants' response accuracy and the information rate of cognitive control (in bits per second, bps) in the model fitting to estimate the capacity of cognitive control. We found that the capacity of cognitive control was approximately 3 to 4 bps, demonstrating that cognitive control as a higher-level function has a remarkably low capacity. This quantification of the capacity of cognitive control may have significant theoretical and clinical implications.

Brain structural changes following adaptive cognitive training assessed by Tensor-Based Morphometry (TBM)

Tensor-Based Morphometry (TBM) allows the automatic mapping of brain changes across time building 3D deformation maps. This technique has been applied for tracking brain degeneration in Alzheimer's and other neurodegenerative diseases with high sensitivity and reliability. Here we applied TBM to quantify changes in brain structure after completing a challenging adaptive cognitive training program based on the n-back task. Twenty-six young women completed twenty-four training sessions across twelve weeks and they showed, on average, large cognitive improvements. High-resolution MRI scans were obtained before and after training. The computed longitudinal deformation maps were analyzed for answering three questions: (a) Are there differential brain structural changes in the training group as compared with a matched control group? (b) Are these changes related to performance differences in the training program? (c) Are standardized changes in a set of psychological factors (fluid and crystallized intelligence, working memory, and attention control) measured before and after training, related to structural changes in the brain? Results showed (a) greater structural changes for the training group in the temporal lobe, (b) a negative correlation between these changes and performance across training sessions (the greater the structural change, the lower the cognitive performance improvements), and (c) negligible effects regarding the psychological factors measured before and after training.

Competition between movement plans increases motor variability: evidence of a shared resource for movement planning

Various lines of evidence indicate that multiple movements can be prepared in parallel. Here, we show that preparing more than one movement comes with a cost: a movement plan is more variable if it is prepared simultaneously with another plan. This suggests that the representations of movement plans share a common neural resource and implies that the number of alternative plans is constrained by noise.

Do movement plans, like representations in working memory, share a limited pool of resources? If so, the precision with which each individual movement plan is specified should decrease as the total number of movement plans increases. To explore this, human participants made speeded reaching movements toward visual targets. We examined if preparing one movement resulted in less variability than preparing two movements. The number of planned movements was manipulated in a delayed response cueing procedure that limited planning to a single target (experiment 1) or hand (experiment 2) or required planning of movements toward two targets (or with two hands). For both experiments, initial movement direction variability was higher in the two-plan condition than in the one-plan condition, demonstrating a cost associated with planning multiple movements, consistent with the limited resource hypothesis. In experiment 3, we showed that the advantage in initial variability of preparing a single movement was present only when the trajectory could be fully specified. This indicates that the difference in variability between one and two plans reflects the specification of full motor plans, not a general preparedness to move. The precision cost related to concurrent plans represents a novel constraint on motor preparation, indicating that multiple movements cannot be planned independently, even if they involve different limbs.

Working memory is not fixed-capacity: More active storage capacity for real-world objects than for simple stimuli

Visual working memory is the cognitive system that holds visual information active to make it resistant to interference from new perceptual input. Information about simple stimuli-colors and orientations-is encoded into working memory rapidly: In under 100 ms, working memory ‟fills up," revealing a stark capacity limit. However, for real-world objects, the same behavioral limits do not hold: With increasing encoding time, people store more real-world objects and do so with more detail. This boost in performance for real-world objects is generally assumed to reflect the use of a separate episodic long-term memory system, rather than working memory. Here we show that this behavioral increase in capacity with real-world objects is not solely due to the use of separate episodic long-term memory systems. In particular, we show that this increase is a result of active storage in working memory, as shown by directly measuring neural activity during the delay period of a working memory task using EEG. These data challenge fixed-capacity working memory models and demonstrate that working memory and its capacity limitations are dependent upon our existing knowledge.

The neuroscience of working memory capacity and training

Working memory - the ability to maintain and manipulate information over a period of seconds - is a core component of higher cognitive functions. The storage capacity of working memory is limited but can be expanded by training, and evidence of the neural mechanisms underlying this effect is accumulating. Human imaging studies and neurophysiological recordings in non-human primates, together with computational modelling studies, reveal that training increases the activity of prefrontal neurons and the strength of connectivity in the prefrontal cortex and between the prefrontal and parietal cortex. Dopaminergic transmission could have a facilitatory role. These changes more generally inform us of the plasticity of higher cognitive functions.

The same type of visual working memory limitations in humans and monkeys

Rhesus monkeys are widely used as an animal model for human memory, including visual working memory (VWM). It is, however, unknown whether the same principles govern VWM in humans and rhesus monkeys. Here, we tested both species in nearly identical change-localization paradigms and formally compared the same set of models of VWM limitations. These models include the classic item-limit model and recent noise-based (resource) models, as well as hybrid models that combine a noise-based representation with an item limit. By varying the magnitude of the change in addition to the typical set size manipulation, we were able to show large differences in goodness of fit among the five models tested. In spite of quantitative performance differences between the species, we find that the variable-precision model--a noise-based model--best describes the behavior of both species. Adding an item limit to this model does not help to account for the data. Our results suggest evolutionary continuity of VWM across primates and help establish the rhesus monkey as a model system for studying the neural substrates of multiple-item VWM.

Gamma and Beta Bursts Underlie Working Memory

Working memory is thought to result from sustained neuron spiking. However, computational models suggest complex dynamics with discrete oscillatory bursts. We analyzed local field potential (LFP) and spiking from the prefrontal cortex (PFC) of monkeys performing a working memory task. There were brief bursts of narrow-band gamma oscillations (45-100 Hz), varied in time and frequency, accompanying encoding and re-activation of sensory information. They appeared at a minority of recording sites associated with spiking reflecting the to-be-remembered items. Beta oscillations (20-35 Hz) also occurred in brief, variable bursts but reflected a default state interrupted by encoding and decoding. Only activity of neurons reflecting encoding/decoding correlated with changes in gamma burst rate. Thus, gamma bursts could gate access to, and prevent sensory interference with, working memory. This supports the hypothesis that working memory is manifested by discrete oscillatory dynamics and spiking, not sustained activity.

From Behavior to Neural Dynamics: An Integrated Theory of Attention

The brain has a limited capacity and therefore needs mechanisms to selectively enhance the information most relevant to one's current behavior. We refer to these mechanisms as "attention." Attention acts by increasing the strength of selected neural representations and preferentially routing them through the brain's large-scale network. This is a critical component of cognition and therefore has been a central topic in cognitive neuroscience. Here we review a diverse literature that has studied attention at the level of behavior, networks, circuits, and neurons. We then integrate these disparate results into a unified theory of attention.

Role of Prefrontal Persistent Activity in Working Memory

The prefrontal cortex is activated during working memory, as evidenced by fMRI results in human studies and neurophysiological recordings in animal models. Persistent activity during the delay period of working memory tasks, after the offset of stimuli that subjects are required to remember, has traditionally been thought of as the neural correlate of working memory. In the last few years several findings have cast doubt on the role of this activity. By some accounts, activity in other brain areas, such as the primary visual and posterior parietal cortex, is a better predictor of information maintained in visual working memory and working memory performance; dynamic patterns of activity may convey information without requiring persistent activity at all; and prefrontal neurons may be ill-suited to represent non-spatial information about the features and identity of remembered stimuli. Alternative interpretations about the role of the prefrontal cortex have thus been suggested, such as that it provides a top-down control of information represented in other brain areas, rather than maintaining a working memory trace itself. Here we review evidence for and against the role of prefrontal persistent activity, with a focus on visual neurophysiology. We show that persistent activity predicts behavioral parameters precisely in working memory tasks. We illustrate that prefrontal cortex represents features of stimuli other than their spatial location, and that this information is largely absent from early cortical areas during working memory. We examine memory models not dependent on persistent activity, and conclude that each of those models could mediate only a limited range of memory-dependent behaviors. We review activity decoded from brain areas other than the prefrontal cortex during working memory and demonstrate that these areas alone cannot mediate working memory maintenance, particularly in the presence of distractors. We finally discuss the discrepancy between BOLD activation and spiking activity findings, and point out that fMRI methods do not currently have the spatial resolution necessary to decode information within the prefrontal cortex, which is likely organized at the micrometer scale. Therefore, we make the case that prefrontal persistent activity is both necessary and sufficient for the maintenance of information in working memory.

Gray matter responsiveness to adaptive working memory training: a surface-based morphometry study

Here we analyze gray matter indices before and after completing a challenging adaptive cognitive training program based on the n-back task. The considered gray matter indices were cortical thickness (CT) and cortical surface area (CSA). Twenty-eight young women (age range 17-22 years) completed 24 training sessions over the course of 3 months (12 weeks, 24 sessions), showing expected performance improvements. CT and CSA values for the training group were compared with those of a matched control group. Statistical analyses were computed using a ROI framework defined by brain areas distinguished by their genetic underpinning. The interaction between group and time was analyzed. Middle temporal, ventral frontal, inferior parietal cortices, and pars opercularis were the regions where the training group showed conservation of gray matter with respect to the control group. These regions support working memory, resistance to interference, and inhibition. Furthermore, an interaction with baseline intelligence differences showed that the expected decreasing trend at the biological level for individuals showing relatively low intelligence levels at baseline was attenuated by the completed training.

Stimulus Load and Oscillatory Activity in Higher Cortex

Exploring and exploiting a rich visual environment requires perceiving, attending, and remembering multiple objects simultaneously. Recent studies have suggested that this mental "juggling" of multiple objects may depend on oscillatory neural dynamics. We recorded local field potentials from the lateral intraparietal area, frontal eye fields, and lateral prefrontal cortex while monkeys maintained variable numbers of visual stimuli in working memory. Behavior suggested independent processing of stimuli in each hemifield. During stimulus presentation, higher-frequency power (50-100 Hz) increased with the number of stimuli (load) in the contralateral hemifield, whereas lower-frequency power (8-50 Hz) decreased with the total number of stimuli in both hemifields. During the memory delay, lower-frequency power increased with contralateral load. Load effects on higher frequencies during stimulus encoding and lower frequencies during the memory delay were stronger when neural activity also signaled the location of the stimuli. Like power, higher-frequency synchrony increased with load, but beta synchrony (16-30 Hz) showed the opposite effect, increasing when power decreased (stimulus presentation) and decreasing when power increased (memory delay). Our results suggest roles for lower-frequency oscillations in top-down processing and higher-frequency oscillations in bottom-up processing.

Context-Dependent Duration Signals in the Primate Prefrontal Cortex

The activity of some prefrontal (PF) cortex neurons distinguishes short from long time intervals. Here, we examined whether this property reflected a general timing mechanism or one dependent on behavioral context. In one task, monkeys discriminated the relative duration of 2 stimuli; in the other, they discriminated the relative distance of 2 stimuli from a fixed reference point. Both tasks had a pre-cue period (interval 1) and a delay period (interval 2) with no discriminant stimulus. Interval 1 elapsed before the presentation of the first discriminant stimulus, and interval 2 began after that stimulus. Both intervals had durations of either 400 or 800 ms. Most PF neurons distinguished short from long durations in one task or interval, but not in the others. When neurons did signal something about duration for both intervals, they did so in an uncorrelated or weakly correlated manner. These results demonstrate a high degree of context dependency in PF time processing. The PF, therefore, does not appear to signal durations abstractedly, as would be expected of a general temporal encoder, but instead does so in a highly context-dependent manner, both within and between tasks.

Neural circuit basis of visuo-spatial working memory precision: a computational and behavioral study

The amount of information that can be retained in working memory (WM) is limited. Limitations of WM capacity have been the subject of intense research, especially in trying to specify algorithmic models for WM. Comparatively, neural circuit perspectives have barely been used to test WM limitations in behavioral experiments. Here we used a neuronal microcircuit model for visuo-spatial WM (vsWM) to investigate memory of several items. The model assumes that there is a topographic organization of the circuit responsible for spatial memory retention. This assumption leads to specific predictions, which we tested in behavioral experiments. According to the model, nearby locations should be recalled with a bias, as if the two memory traces showed attraction or repulsion during the delay period depending on distance. Another prediction is that the previously reported loss of memory precision for an increasing number of memory items (memory load) should vanish when the distances between items are controlled for. Both predictions were confirmed experimentally. Taken together, our findings provide support for a topographic neural circuit organization of vsWM, they suggest that interference between similar memories underlies some WM limitations, and they put forward a circuit-based explanation that reconciles previous conflicting results on the dependence of WM precision with load.

Spikes not slots: noise in neural populations limits working memory

This opinion article argues that noise (randomness) in neural activity is the limiting factor in visual working memory (WM), determining how accurately we can maintain stable internal representations of external stimuli. Sharing of a fixed amount of neural activity between items in memory explains why WM can be successfully described as a continuous resource. This contrasts with the popular conception of WM as comprising a limited number of memory slots, each holding a representation of one stimulus - I argue that this view is challenged by computational theory and the latest neurophysiological evidence.

Goal-direction and top-down control

We review the neural mechanisms that support top-down control of behaviour and suggest that goal-directed behaviour uses two systems that work in concert. A basal ganglia-centred system quickly learns simple, fixed goal-directed behaviours while a prefrontal cortex-centred system gradually learns more complex (abstract or long-term) goal-directed behaviours. Interactions between these two systems allow top-down control mechanisms to learn how to direct behaviour towards a goal but also how to guide behaviour when faced with a novel situation.

'Activity-silent' working memory in prefrontal cortex: a dynamic coding framework

•

WM is thought to depend on persistent maintenance of stationary activity states.

•

However, population-level analyses reveal that brain activity is highly dynamic.

•

Accumulating evidence implicates activity-silent neural states for WM.

•

Dynamic coding suggests that WM is encoded in patterns of functional connectivity.

Working memory (WM) provides the functional backbone to high-level cognition. Maintenance in WM is often assumed to depend on the stationary persistence of neural activity patterns that represent memory content. However, accumulating evidence suggests that persistent delay activity does not always accompany WM maintenance but instead seems to wax and wane as a function of the current task relevance of memoranda. Furthermore, new methods for measuring and analysing population-level patterns show that activity states are highly dynamic. At first glance, these dynamics seem at odds with the very nature of WM. How can we keep a stable thought in mind while brain activity is constantly changing? This review considers how neural dynamics might be functionally important for WM maintenance.

Different Roles of COMT and HTR2A Genotypes in Working Memory Subprocesses

Working memory is linked to the functions of the frontal areas, in which neural activity is mediated by dopaminergic and serotonergic tones. However, there is no consensus regarding how the dopaminergic and serotonergic systems influence working memory subprocesses. The present study used an imaging genetics approach to examine the interaction between neurochemical functions and working memory performance. We focused on functional polymorphisms of the catechol-O-methyltransferase (COMT) Val¹⁵⁸Met and serotonin 2A receptor (HTR2A) -1438G/A genes, and devised a delayed recognition task to isolate the encoding, retention, and retrieval processes for visual information. The COMT genotypes affected recognition accuracy, whereas the HTR2A genotypes were associated with recognition response times. Activations specifically related to working memory were found in the right frontal and parietal areas, such as the middle frontal gyrus (MFG), inferior frontal gyrus (IFG), anterior cingulate cortex (ACC), and inferior parietal lobule (IPL). MFG and ACC/IPL activations were sensitive to differences between the COMT genotypes and between the HTR2A genotypes, respectively. Structural equation modeling demonstrated that stronger connectivity in the ACC-MFG and ACC-IFG networks is related to better task performance. The behavioral and fMRI results suggest that the dopaminergic and serotonergic systems play different roles in the working memory subprocesses and modulate closer cooperation between lateral and medial frontal activations.

Prefrontal cortex and sensory cortices during working memory: quantity and quality

The activity in sensory cortices and the prefrontal cortex (PFC) throughout the delay interval of working memory (WM) tasks reflect two aspects of WM-quality and quantity, respectively. The delay activity in sensory cortices is fine-tuned to sensory information and forms the neural basis of the precision of WM storage, while the delay activity in the PFC appears to represent behavioral goals and filters out irrelevant distractions, forming the neural basis of the quantity of task-relevant information in WM. The PFC and sensory cortices interact through different frequency bands of neuronal oscillation (theta, alpha, and gamma) to fulfill goal-directed behaviors.

Hemispheric differences in relational reasoning: novel insights based on an old technique

Relational reasoning, or the ability to integrate multiple mental relations to arrive at a logical conclusion, is a critical component of higher cognition. A bilateral brain network involving lateral prefrontal and parietal cortices has been consistently implicated in relational reasoning. Some data suggest a preferential role for the left hemisphere in this form of reasoning, whereas others suggest that the two hemispheres make important contributions. To test for a hemispheric asymmetry in relational reasoning, we made use of an old technique known as visual half-field stimulus presentation to manipulate whether stimuli were presented briefly to one hemisphere or the other. Across two experiments, 54 neurologically healthy young adults performed a visuospatial transitive inference task. Pairs of colored shapes were presented rapidly in either the left or right visual hemifield as participants maintained central fixation, thereby isolating initial encoding to the contralateral hemisphere. We observed a left-hemisphere advantage for encoding a series of ordered visuospatial relations, but both hemispheres contributed equally to task performance when the relations were presented out of order. To our knowledge, this is the first study to reveal hemispheric differences in relational encoding in the intact brain. We discuss these findings in the context of a rich literature on hemispheric asymmetries in cognition.

A probabilistic palimpsest model of visual short-term memory

Working memory plays a key role in cognition, and yet its mechanisms remain much debated. Human performance on memory tasks is severely limited; however, the two major classes of theory explaining the limits leave open questions about key issues such as how multiple simultaneously-represented items can be distinguished. We propose a palimpsest model, with the occurrent activity of a single population of neurons coding for several multi-featured items. Using a probabilistic approach to storage and recall, we show how this model can account for many qualitative aspects of existing experimental data. In our account, the underlying nature of a memory item depends entirely on the characteristics of the population representation, and we provide analytical and numerical insights into critical issues such as multiplicity and binding. We consider representations in which information about individual feature values is partially separate from the information about binding that creates single items out of multiple features. An appropriate balance between these two types of information is required to capture fully the different types of error seen in human experimental data. Our model provides the first principled account of misbinding errors. We also suggest a specific set of stimuli designed to elucidate the representations that subjects actually employ.

Humans can remember several visual items for a few seconds and recall them; however, performance deteriorates surprisingly quickly with the number of items that must be stored. Along with increasingly inaccurate recollection, subjects make association errors, sometimes apparently recalling the wrong item altogether. No current model accounts for these data fully. We discuss a simple model that focuses attention on the population representations that are putatively involved, and thereby on limits to the amount of information that can be stored and recalled. We use theoretical and numerical methods to examine the characteristics and performance of our model.