Multivariate analysis reveals a generalizable human electrophysiological signature of working memory load

Cortically Disparate Visual Features Evoke Content-Independent Load Signals during Storage in Working Memory

It is well established that holding information in working memory (WM) elicits sustained stimulus-specific patterns of neural activity. Nevertheless, here we provide evidence for a distinct class of neural activity that tracks the number of individuated items in working memory, independent of the type of visual features stored. We present two EEG studies of young adults of both sexes that provide robust evidence for a signal tracking the number of individuated representations in working memory, regardless of the specific feature values stored. In Study 1, subjects maintained either colors or orientations across separate blocks in a single session. We found near-perfect generalization of the load signal between these two conditions, despite being able to simultaneously decode which feature had been voluntarily stored. In Study 2, participants attended to two features with very distinct cortical representations: color and motion coherence. We again found evidence for a neural load signal that robustly generalized across these distinct visual features, even though cortically disparate regions process color and motion coherence. Moreover, representational similarity analysis provided converging evidence for a content-independent load signal, while simultaneously showing that unique variance in EEG activity tracked the specific features that were stored. We posit that this load signal reflects a content-independent "pointer" operation that binds objects to the current context while parallel but distinct neural signals represent the features that are stored for each item in memory.

Electroencephalogram Decoding Reveals Distinct Processes for Directing Spatial Attention and Encoding Into Working Memory

Past work reveals a tight relationship between spatial attention and storage in visual working memory. But is spatially attending an item tantamount to working memory encoding? Here, we tracked electroencephalography (EEG) signatures of spatial attention and working memory encoding while independently manipulating the number of memory items and the spatial extent of attention in two studies of adults (N = 39; N = 33). Neural measures of spatial attention tracked the position and size of the attended area independent of the number of individuated items encoded into working memory. At the same time, multivariate decoding of the number of items stored in working memory was insensitive to variations in the breadth and position of spatial attention. Finally, representational similarity analyses provided converging evidence for a pure load signal that is insensitive to the spatial extent of the stored items. Thus, although spatial attention is a persistent partner of visual working memory, it is functionally dissociable from the selection and maintenance of individuated representations in working memory.

Dynamic layer-specific processing in the prefrontal cortex during working memory

The dorsolateral prefrontal cortex (dlPFC) is reliably engaged in working memory (WM) and comprises different cytoarchitectonic layers, yet their functional role in human WM is unclear. Here, participants completed a delayed-match-to-sample task while undergoing functional magnetic resonance imaging (fMRI) at ultra-high resolution. We examine layer-specific activity to manipulations in WM load and motor response. Superficial layers exhibit a preferential response to WM load during the delay and retrieval periods of a WM task, indicating a lamina-specific activation of the frontoparietal network. Multivariate patterns encoding WM load in the superficial layer dynamically change across the three periods of the task. Last, superficial and deep layers are non-differentially involved in the motor response, challenging earlier findings of a preferential deep layer activation. Taken together, our results provide new insights into the functional laminar circuitry of the dlPFC during WM and support a dynamic account of dlPFC coding.

Decoding cognition in neurodevelopmental, psychiatric and neurological conditions with multivariate pattern analysis of EEG data

Multivariate pattern analysis (MVPA) of electroencephalographic (EEG) data represents a revolutionary approach to investigate how the brain encodes information. By considering complex interactions among spatio-temporal features at the individual level, MVPA overcomes the limitations of univariate techniques, which often fail to account for the significant inter- and intra-individual neural variability. This is particularly relevant when studying clinical populations, and therefore MVPA of EEG data has recently started to be employed as a tool to study cognition in brain disorders. Here, we review the insights offered by this methodology in the study of anomalous patterns of neural activity in conditions such as autism, ADHD, schizophrenia, dyslexia, neurological and neurodegenerative disorders, within different cognitive domains (perception, attention, memory, consciousness). Despite potential drawbacks that should be attentively addressed, these studies reveal a peculiar sensitivity of MVPA in unveiling dysfunctional and compensatory neurocognitive dynamics of information processing, which often remain blind to traditional univariate approaches. Such higher sensitivity in characterizing individual neurocognitive profiles can provide unique opportunities to optimise assessment and promote personalised interventions.

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

Decoding the content of working memory in school-aged children

Developmental improvements in working memory (WM) maintenance predict many real-world outcomes, including educational attainment. It is thus critical to understand which WM mechanisms support these behavioral improvements, and how WM maintenance strategies might change through development. One challenge is that specific WM neural mechanisms cannot easily be measured behaviorally, especially in a child population. However, new multivariate decoding techniques have been designed, primarily in adult populations, that can sensitively decode the contents of WM. The goal of this study was to deploy multivariate decoding techniques known to decode memory representations in adults to decode the contents of WM in children. We created a simple computerized WM game for children, in which children maintained different categories of information (visual, spatial or verbal). We collected electroencephalography (EEG) data from 20 children (7-12-year-olds) while they played the game. Using Multivariate Pattern Analysis (MVPA) on children's EEG signals, we reliably decoded the category of the maintained information during the sensory and maintenance period. Across exploratory reliability and validity analyses, we examined the robustness of these results when trained on less data, and how these patterns generalized within individuals throughout the testing session. Furthermore, these results matched theory-based predictions of WM across individuals and across ages. Our proof-of-concept study proposes a direct and age-appropriate potential alternative to exclusively behavioral WM maintenance measures in children. Our study demonstrates the utility of MVPA to measure and track the uninstructed representational content of children's WM. Future research could use our technique to investigate children's WM maintenance and strategies.

Visual short-term memory-related EEG components in a virtual reality setup

Virtual reality (VR) offers a powerful tool for investigating cognitive processes, as it allows researchers to gauge behaviors and mental states in complex, yet highly controlled, scenarios. The use of VR head-mounted displays in combination with physiological measures such as EEG presents new challenges and raises the question whether established findings also generalize to a VR setup. Here, we used a VR headset to assess the spatial constraints underlying two well-established EEG correlates of visual short-term memory: the amplitude of the contralateral delay activity (CDA) and the lateralization of induced alpha power during memory retention. We tested observers' visual memory in a change detection task with bilateral stimulus arrays of either two or four items while varying the horizontal eccentricity of the memory arrays (4, 9, or 14 degrees of visual angle). The CDA amplitude differed between high and low memory load at the two smaller eccentricities, but not at the largest eccentricity. Neither memory load nor eccentricity significantly influenced the observed alpha lateralization. We further fitted time-resolved spatial filters to decode memory load from the event-related potential as well as from its time-frequency decomposition. Classification performance during the retention interval was above-chance level for both approaches and did not vary significantly across eccentricities. We conclude that commercial VR hardware can be utilized to study the CDA and lateralized alpha power, and we provide caveats for future studies targeting these EEG markers of visual memory in a VR setup.

Neural mechanisms of domain-general inhibitory control

Inhibitory control is a fundamental mechanism underlying flexible behavior and features in theories across many areas of cognitive and psychological science. However, whereas many theories implicitly or explicitly assume that inhibitory control is a domain-general process, the vast majority of neuroscientific work has hitherto focused on individual domains, such as motor, mnemonic, or attentional inhibition. Here, we attempt to close this gap by highlighting recent work that demonstrates shared neuroanatomical and neurophysiological signatures of inhibitory control across domains. We propose that the regulation of thalamocortical drive by a fronto-subthalamic mechanism operating in the β band might be a domain-general mechanism for inhibitory control in the human brain.

Decoding the content of working memory in school-aged children

Developmental improvements in working memory (WM) maintenance predict many real-world outcomes, including educational attainment. It is thus critical to understand which WM mechanisms support these behavioral improvements, and how WM maintenance strategies might change through development. One challenge is that specific WM neural mechanisms cannot easily be measured behaviorally, especially in a child population. However, new multivariate decoding techniques have been designed, primarily in adult populations, that can sensitively decode the contents of WM. The goal of this study was to deploy multivariate decoding techniques known to decode memory representations in adults to decode the contents of WM in children. We created a simple computerized WM game for children, in which children maintained different categories of information (visual, spatial or verbal). We collected electroencephalography (EEG) data from 20 children (7-12-year-olds) while they played the game. Using Multivariate Pattern Analysis (MVPA) on children's EEG signals, we reliably decoded the category of the maintained information during the sensory and maintenance period. Across exploratory reliability and validity analyses, we examined the robustness of these results when trained on less data, and how these patterns generalized within individuals throughout the testing session. Furthermore, these results matched theory-based predictions of WM across individuals and across ages. Our proof-of-concept study proposes a direct and age-appropriate potential alternative to exclusively behavioral WM maintenance measures in children. Our study demonstrates the utility of MVPA to measure and track the uninstructed representational content of children's WM. Future research could use our technique to investigate children's WM maintenance and strategies.

Prioritizing flexible working memory representations through retrospective attentional strengthening

Previous work has proposed two potential benefits of retrospective attention on working memory (WM): target strengthening and non-target inhibition. It remains unknown which hypothesis contributes to the improved WM performance, yet the neural mechanisms responsible for this attentional benefit are unclear. Here, we recorded electroencephalography (EEG) signals while 33 participants performed a retrospective-cue WM task. Multivariate pattern classification analysis revealed that only representations of target features were enhanced by valid retrospective attention during retention, supporting the target strengthening hypothesis. Further univariate analysis found that mid-frontal theta inter-trial phase coherence (ITPC) and ERP components were modulated by valid retrospective attention and correlated with individual differences and moment-to-moment fluctuations on behavioral outcomes, suggesting that both trait- and state-level variability in attentional preparatory processes influence goal-directed behavior. Furthermore, task-irrelevant target spatial location could be decoded from EEG signals, indicating that enhanced spatial binding of target representation is vital to high WM precision. Importantly, frontoparietal theta-alpha phase-amplitude coupling was increased by valid retrospective attention and predicted the reduced random guessing rates. This long-range connection supported top-down information flow in the engagement of frontoparietal networks, which might organize attentional states to integrate target features. Altogether, these results provide neurophysiological bases that retrospective attention improves WM precision by enhancing flexible target representation and emphasize the critical role of the frontoparietal attentional network in the control of WM representations.

Is the Contralateral Delay Activity (CDA) a robust neural correlate for Visual Working Memory (VWM) tasks? A reproducibility study

Visual working memory (VWM) allows us to actively store, update, and manipulate visual information surrounding us. While the underlying neural mechanisms of VWM remain unclear, contralateral delay activity (CDA), a sustained negativity over the hemisphere contralateral to the positions of visual items to be remembered, is often used to study VWM. To investigate if the CDA is a robust neural correlate for VWM tasks, we reproduced eight CDA-related studies with a publicly accessible EEG data set. We used the raw EEG data from these eight studies and analyzed all of them with the same basic pipeline to extract CDA. We were able to reproduce the results from all the studies and show that with a basic automated EEG pipeline we can extract a clear CDA signal. We share insights from the trends observed across the studies and raise some questions about the CDA decay and the CDA during the recall phase, which surprisingly, none of the eight studies did address. Finally, we also provide reproducibility recommendations based on our experience and challenges in reproducing these studies.

Storage in Visual Working Memory Recruits a Content-Independent Pointer System

Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.

Alpha Oscillations Track Content-Specific Working Memory Capacity

Although the neural basis of working memory (WM) capacity is often studied by exploiting interindividual differences, capacity may also differ across memory materials within a given individual. Here, we exploit the content dependence of WM capacity as a novel approach to investigate the oscillatory correlates of WM capacity, focusing on posterior 9-12 Hz alpha activity during retention. We recorded scalp electroencephalography (EEG) while male and female human participants performed WM tasks with varying memory loads (two vs. four items) and materials (English letters vs. regular shapes vs. abstract shapes). First, behavioral data confirmed that memory capacity was fundamentally content dependent; capacity for abstract shapes plateaued at around two, whereas the participants could remember more letters and regular shapes. Critically, content-specific capacity was paralleled in the degree of attenuation of EEG-alpha activity that plateaued in a similar content-specific manner. Although we observed greater alpha attenuation for higher loads for all materials, we found larger load effects for letters and regular shapes than for abstract shapes, which is consistent with our behavioral data showing a lower capacity plateau for abstract shapes. Moreover, when only considering two-item trials, alpha attenuation was greater for abstract shapes where two items were close to the capacity plateau than for other materials. Multivariate decoding of alpha activity patterns reinforced these findings. Finally, for each material, load effects on capacity (K) and alpha attenuation were correlated across individuals. Our results demonstrate that alpha oscillations track memory capacity in a content-specific manner and track not just the number of items but also their complexity.SIGNIFICANCE STATEMENT WM is limited in its capacity. We show that capacity is not fixed for an individual but is rather memory-content dependent. Moreover, we used this as a novel approach to investigate the neural basis of WM capacity with EEG. We found that both behavioral capacity estimates and neural oscillations in the alpha band varied with memory loads and materials. The critical finding is a capacity plateau of approximately two items only for the more complex materials, accompanied by a similar plateau in the EEG alpha attenuation. The load effects on capacity and alpha attenuation were furthermore correlated across individuals for each of the materials. Our results demonstrate that alpha oscillations track the content-specific nature of WM capacity.

When the Whole Is Less Than the Sum of Its Parts: Maximum Object Category Information and Behavioral Prediction in Multiscale Activation Patterns

Neural codes are reflected in complex neural activation patterns. Conventional electroencephalography (EEG) decoding analyses summarize activations by averaging/down-sampling signals within the analysis window. This diminishes informative fine-grained patterns. While previous studies have proposed distinct statistical features capable of capturing variability-dependent neural codes, it has been suggested that the brain could use a combination of encoding protocols not reflected in any one mathematical feature alone. To check, we combined 30 features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across three datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed across most of the time points by the multiscale feature of Wavelet coefficients. Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the EEG neural codes better than any combination of protocols. Our findings put new constraints on the models of neural information encoding in EEG.

Inter-electrode correlations measured with EEG predict individual differences in cognitive ability

Human brains share a broadly similar functional organization with consequential individual variation. This duality in brain function has primarily been observed when using techniques that consider the spatial organization of the brain, such as MRI. Here, we ask whether these common and unique signals of cognition are also present in temporally sensitive but spatially insensitive neural signals. To address this question, we compiled electroencephalogram (EEG) data from individuals of both sexes while they performed multiple working memory tasks at two different data-collection sites (n = 171 and 165). Results revealed that trial-averaged EEG activity exhibited inter-electrode correlations that were stable within individuals and unique across individuals. Furthermore, models based on these inter-electrode correlations generalized across datasets to predict participants' working memory capacity and general fluid intelligence. Thus, inter-electrode correlation patterns measured with EEG provide a signature of working memory and fluid intelligence in humans and a new framework for characterizing individual differences in cognitive abilities.