Single-Trial EEG Analysis Predicts Memory Retrieval and Reveals Source-Dependent Differences

Decoding EEG for optimizing naturalistic memory

BACKGROUND

Spectral features of human electroencephalographic (EEG) recordings during learning predict subsequent recall variability.

NEW METHOD

Capitalizing on these fluctuating neural features, we develop a non-invasive closed-loop (NICL) system for real-time optimization of human learning. Participants play a virtual navigation-and-memory game; recording multi-session data across days allowed us to build participant-specific classification models of recall success. In subsequent closed-loop sessions, our platform manipulated the timing of memory encoding, selectively presenting items during periods of predicted good or poor memory function based on EEG features decoded in real time.

RESULTS

The induced memory effect (the difference between recall rates when presenting items during predicted good vs. poor learning periods) increased with the accuracy of neural decoding.

COMPARISON WITH EXISTING METHODS

This study demonstrates greater-than-chance memory decoding from EEG recordings in a naturalistic virtual navigation task with greater real-world validity than basic word-list recall paradigms. Here we modulate memory by timing stimulus presentation based on noninvasive scalp EEG recordings, whereas prior closed-loop studies for memory improvement involved intracranial recordings and direct electrical stimulation. Other noninvasive studies have investigated the use of neurofeedback or remedial study for memory improvement.

CONCLUSIONS

These findings present a proof-of-concept for using non-invasive closed-loop technology to optimize human learning and memory through principled stimulus timing, but only in those participants for whom classifiers reliably predict out-of-sample memory function.

EEG decoders track memory dynamics

Encoding- and retrieval-related neural activity jointly determine mnemonic success. We ask whether electroencephalographic activity can reliably predict encoding and retrieval success on individual trials. Each of 98 participants performed a delayed recall task on 576 lists across 24 experimental sessions. Logistic regression classifiers trained on spectral features measured immediately preceding spoken recall of individual words successfully predict whether or not those words belonged to the target list. Classifiers trained on features measured during word encoding also reliably predict whether those words will be subsequently recalled and further predict the temporal and semantic organization of the recalled items. These findings link neural variability predictive of successful memory with item-to-context binding, a key cognitive process thought to underlie episodic memory function.

Category-based attention facilitates memory search

We often need to decide whether the object we look at is also the object we look for. When we look for one specific object, this process can be facilitated by feature-based attention. However, when we look for many objects at the same time (e.g., the products on our shopping list) such a strategy may no longer be possible, as research has shown that we can actively prepare to detect only one or two objects at a time. Therefore, looking for multiple objects additionally requires long-term memory search, slowing down decision making. Interestingly, however, previous research has shown that distractor objects can be efficiently rejected during memory search when they are from a different category than the items in the memory set. Here, using EEG, we show that this efficiency is supported by top-down attention at the category level. In Experiment 1, human participants (both sexes) performed a memory search task on individually presented objects from different categories, most of which were distractors. We observed category-level attentional modulation of distractor processing from ∼150 ms after stimulus onset, expressed both as an evoked response modulation and as an increase in decoding accuracy of same-category distractors. In Experiment 2, memory search was performed on two concurrently presented objects. When both objects were distractors, spatial attention (indexed by the N2pc component) was directed to the object that was of the same category as the objects in the memory set. Together, these results demonstrate how top-down attention can facilitate memory search.Significance statement When we are in the supermarket, we repeatedly decide whether a product we look at (e.g., a banana) is on our memorized shopping list (e.g., apples, oranges, kiwis). This requires searching our memory, which takes time. However, when the product is of an entirely different category (e.g., dairy instead of fruit), the decision can be made quickly. Here, we used EEG to show that this between-category advantage in memory search tasks is supported by top-down attentional modulation of visual processing: The visual response evoked by distractor objects was modulated by category membership, and spatial attention was quickly directed to the location of within-category (vs. between-category) distractors. These results demonstrate a close link between attention and memory.

Evaluation of classification approaches for distinguishing brain states predictive of episodic memory performance from electroencephalography: Abbreviated Title: Evaluating methods of classifying memory states from EEG

Previous studies have attempted to separate single trial neural responses for events a person is likely to remember from those they are likely to forget using machine learning classification methods. Successful single trial classification holds potential for translation into the clinical realm for real-time detection of memory and other cognitive states to provide real-time interventions (i.e., brain-computer interfaces). However, most of these studies-and classification analyses in general- do not make clear if the chosen methodology is optimally suited for the classification of memory-related brain states. To address this problem, we systematically compared different methods for every step of classification (i.e., feature extraction, feature selection, classifier selection) to investigate which methods work best for decoding episodic memory brain states-the first analysis of its kind. Using an adult lifespan sample EEG dataset collected during performance of an episodic context encoding and retrieval task, we found that no specific feature type (including Common Spatial Pattern (CSP)-based features, mean, variance, correlation, features based on AR model, entropy, phase, and phase synchronization) outperformed others consistently in distinguishing different memory classes. However, extracting all of these feature types consistently outperformed extracting only one type of feature. Additionally, the combination of filtering and sequential forward selection was the optimal method to select the effective features compared to filtering alone or performing no feature selection at all. Moreover, although all classifiers performed at a fairly similar level, LASSO was consistently the highest performing classifier compared to other commonly used options (i.e., naïve Bayes, SVM, and logistic regression) while naïve Bayes was the fastest classifier. Lastly, for multiclass classification (i.e., levels of context memory confidence and context feature perception), generalizing the binary classification using the binary decision tree performed better than the voting or one versus rest method. These methods were shown to outperform alternative approaches for three orthogonal datasets (i.e., EEG working memory, EEG motor imagery, and MEG working memory), supporting their generalizability. Our results provide an optimized methodological process for classifying single-trial neural data and provide important insight and recommendations for a cognitive neuroscientist's ability to make informed choices at all stages of the classification process for predicting memory and other cognitive states.

EEG biomarkers of free recall

Brain activity in the moments leading up to spontaneous verbal recall provide a window into the cognitive processes underlying memory retrieval. But these same recordings also subsume neural signals unrelated to mnemonic retrieval, such as response-related motor activity. Here we examined spectral EEG biomarkers of memory retrieval under an extreme manipulation of mnemonic demands: subjects either recalled items after a few seconds or after several days. This manipulation helped to isolate EEG components specifically related to long-term memory retrieval. In the moments immediately preceding recall we observed increased theta (4-8 Hz) power (+T), decreased alpha (8-20 Hz) power (-A), and increased gamma (40-128 Hz) power (+G), with this spectral pattern (+T-A + G) distinguishing the long-delay and immediate recall conditions. As subjects vocalized the same set of studied words in both conditions, we interpret the spectral +T-A + G as a biomarker of episodic memory retrieval.

Gauging Working Memory Capacity From Differential Resting Brain Oscillations in Older Individuals With A Wearable Device

Working memory is a core cognitive function and its deficits is one of the most common cognitive impairments. Reduced working memory capacity manifests as reduced accuracy in memory recall and prolonged speed of memory retrieval in older adults. Currently, the relationship between healthy older individuals’ age-related changes in resting brain oscillations and their working memory capacity is not clear. Eyes-closed resting electroencephalogram (rEEG) is gaining momentum as a potential neuromarker of mild cognitive impairments. Wearable and wireless EEG headset measuring key electrophysiological brain signals during rest and a working memory task was utilized. This research’s central hypothesis is that rEEG (e.g., eyes closed for 90 s) frequency and network features are surrogate markers for working memory capacity in healthy older adults. Forty-three older adults’ memory performance (accuracy and reaction times), brain oscillations during rest, and inter-channel magnitude-squared coherence during rest were analyzed. We report that individuals with a lower memory retrieval accuracy showed significantly increased alpha and beta oscillations over the right parietal site. Yet, faster working memory retrieval was significantly correlated with increased delta and theta band powers over the left parietal sites. In addition, significantly increased coherence between the left parietal site and the right frontal area is correlated with the faster speed in memory retrieval. The frontal and parietal dynamics of resting EEG is associated with the “accuracy and speed trade-off” during working memory in healthy older adults. Our results suggest that rEEG brain oscillations at local and distant neural circuits are surrogates of working memory retrieval’s accuracy and processing speed. Our current findings further indicate that rEEG frequency and coherence features recorded by wearable headsets and a brief resting and task protocol are potential biomarkers for working memory capacity. Additionally, wearable headsets are useful for fast screening of cognitive impairment risk.

Predicting memory from study-related brain activity

For both basic and applied reasons, an important goal is to identify brain activity present while people study materials that enable us to predict whether they will remember those materials. We show that this is possible with the conventional event-related potential “subsequent-memory-effect” signals as well as with machine learning classifiers, but only to a small degree. This is in line with behavioral research, which supports many determinants of memory apart from the cognitive processes during study.

Usefulness of familiarity signals during recognition depends on test format: Neurocognitive evidence for a core assumption of the CLS framework

Familiarity-based discrimination between studied items and similar foils in yes/no recognition memory tests is relatively poor. The complementary learning systems (CLS) framework explains this with the small difference in familiarity strength between targets and foils. The framework, however, also predicts that familiarity values of targets and corresponding similar foils are directly comparable - as long as they are presented side by side in a forced-choice corresponding (FCC) test. This is because in each trial, targets tend to be more familiar than their corresponding foils. In contrast, when forced-choice displays contain non-corresponding foils (FCNC) which are similar to other studied items, familiarity values are not directly comparable (as in yes/no-tasks). In a recognition memory task with pictures of objects, we found that the putative ERP correlate of familiarity, the mid-frontal old/new effect for targets vs. foils, was significantly larger in FCC compared to FCNC displays. Moreover, single-trial target-foil amplitude differences predicted the accuracy of the recognition judgment. This study supports the assumption of the CLS framework that the test format can influence the diagnostic reliability of familiarity. Moreover, it implies that the mid-frontal old/new effect does not reflect the difference in the familiarity signal between studied and non-studied items but the task-adequate assessment of this signal.

EEG-based Quantitative Analysis of Aesthetic Emotion in Clothing Design

This paper combines the perceptual aesthetic sentiments with the objective physiological responses of the human brain. By analysing the characteristics of the brain electrical activities in processing of different types of sentiments, this paper uses the specific physiological indicators of the brain to quantify the aesthetic sentiments about clothing. The purpose is to introduce the EEG quantitative analysis method for sentiment perception into the aesthetic field of clothing, and establish a theory of quantifying aesthetic sentiments towards clothing design through the brain electrical physiological responses.