Brain computer interface to enhance episodic memory in human participants

Enhancing mathematical learning outcomes through a low-cost single-channel BCI system

This study investigates the effectiveness of a Low-Cost Single-Channel BCI system in improving mathematical learning outcomes, self-efficacy, and alpha power in university students. Eighty participants were randomly assigned to either a BCI group receiving real-time neurofeedback based on alpha rhythms or a sham feedback group. Results showed that the BCI group had significantly higher mathematical performance, self-efficacy, and alpha power compared to the sham feedback group. Mathematics performance, alpha wave intensity, and self-efficacy showed significant positive correlations after training, indicating that neurofeedback training may have promoted their interaction and integration. These findings demonstrate the potential of BCI technology in enhancing mathematical learning outcomes and highlight the importance of considering pre-test performance and self-efficacy in predicting learning outcomes, with implications for personalized learning interventions and the integration of BCI technology in educational settings.

Decoding EEG for optimizing naturalistic memory

BACKGROUND

Spectral features of human electroencephalographic (EEG) recordings during learning predict subsequent recall variability.

NEW METHOD

Capitalizing on these fluctuating neural features, we develop a non-invasive closed-loop (NICL) system for real-time optimization of human learning. Participants play a virtual navigation-and-memory game; recording multi-session data across days allowed us to build participant-specific classification models of recall success. In subsequent closed-loop sessions, our platform manipulated the timing of memory encoding, selectively presenting items during periods of predicted good or poor memory function based on EEG features decoded in real time.

RESULTS

The induced memory effect (the difference between recall rates when presenting items during predicted good vs. poor learning periods) increased with the accuracy of neural decoding.

COMPARISON WITH EXISTING METHODS

This study demonstrates greater-than-chance memory decoding from EEG recordings in a naturalistic virtual navigation task with greater real-world validity than basic word-list recall paradigms. Here we modulate memory by timing stimulus presentation based on noninvasive scalp EEG recordings, whereas prior closed-loop studies for memory improvement involved intracranial recordings and direct electrical stimulation. Other noninvasive studies have investigated the use of neurofeedback or remedial study for memory improvement.

CONCLUSIONS

These findings present a proof-of-concept for using non-invasive closed-loop technology to optimize human learning and memory through principled stimulus timing, but only in those participants for whom classifiers reliably predict out-of-sample memory function.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.

Intracranial Neurofeedback Modulating Neural Activity in the Mesial Temporal Lobe During Memory Encoding: A Pilot Study

Removal of the mesial temporal lobe (MTL) is an established surgical procedure that leads to seizure freedom in patients with intractable MTL epilepsy; however, it carries the potential risk of memory damage. Neurofeedback (NF), which regulates brain function by converting brain activity into perceptible information and providing feedback, has attracted considerable attention in recent years for its potential as a novel complementary treatment for many neurological disorders. However, no research has attempted to artificially reorganize memory functions by applying NF before resective surgery to preserve memory functions. Thus, this study aimed (1) to construct a memory NF system that used intracranial electrodes to feedback neural activity on the language-dominant side of the MTL during memory encoding and (2) to verify whether neural activity and memory function in the MTL change with NF training. Two intractable epilepsy patients with implanted intracranial electrodes underwent at least five sessions of memory NF training to increase the theta power in the MTL. There was an increase in theta power and a decrease in fast beta and gamma powers in one of the patients in the late stage of memory NF sessions. NF signals were not correlated with memory function. Despite its limitations as a pilot study, to our best knowledge, this study is the first to report that intracranial NF may modulate neural activity in the MTL, which is involved in memory encoding. The findings provide important insights into the future development of NF systems for the artificial reorganization of memory functions.

Evaluation of classification approaches for distinguishing brain states predictive of episodic memory performance from electroencephalography: Abbreviated Title: Evaluating methods of classifying memory states from EEG

Previous studies have attempted to separate single trial neural responses for events a person is likely to remember from those they are likely to forget using machine learning classification methods. Successful single trial classification holds potential for translation into the clinical realm for real-time detection of memory and other cognitive states to provide real-time interventions (i.e., brain-computer interfaces). However, most of these studies-and classification analyses in general- do not make clear if the chosen methodology is optimally suited for the classification of memory-related brain states. To address this problem, we systematically compared different methods for every step of classification (i.e., feature extraction, feature selection, classifier selection) to investigate which methods work best for decoding episodic memory brain states-the first analysis of its kind. Using an adult lifespan sample EEG dataset collected during performance of an episodic context encoding and retrieval task, we found that no specific feature type (including Common Spatial Pattern (CSP)-based features, mean, variance, correlation, features based on AR model, entropy, phase, and phase synchronization) outperformed others consistently in distinguishing different memory classes. However, extracting all of these feature types consistently outperformed extracting only one type of feature. Additionally, the combination of filtering and sequential forward selection was the optimal method to select the effective features compared to filtering alone or performing no feature selection at all. Moreover, although all classifiers performed at a fairly similar level, LASSO was consistently the highest performing classifier compared to other commonly used options (i.e., naïve Bayes, SVM, and logistic regression) while naïve Bayes was the fastest classifier. Lastly, for multiclass classification (i.e., levels of context memory confidence and context feature perception), generalizing the binary classification using the binary decision tree performed better than the voting or one versus rest method. These methods were shown to outperform alternative approaches for three orthogonal datasets (i.e., EEG working memory, EEG motor imagery, and MEG working memory), supporting their generalizability. Our results provide an optimized methodological process for classifying single-trial neural data and provide important insight and recommendations for a cognitive neuroscientist's ability to make informed choices at all stages of the classification process for predicting memory and other cognitive states.

Poststroke Cognitive Impairment Research Progress on Application of Brain-Computer Interface

Brain-computer interfaces (BCIs), a new type of rehabilitation technology, pick up nerve cell signals, identify and classify their activities, and convert them into computer-recognized instructions. This technique has been widely used in the rehabilitation of stroke patients in recent years and appears to promote motor function recovery after stroke. At present, the application of BCI in poststroke cognitive impairment is increasing, which is a common complication that also affects the rehabilitation process. This paper reviews the promise and potential drawbacks of using BCI to treat poststroke cognitive impairment, providing a solid theoretical basis for the application of BCI in this area.

Brain-Computer Interfaces, Open-Source, and Democratizing the Future of Augmented Consciousness

Accessibility, adaptability, and transparency of Brain-Computer Interface (BCI) tools and the data they collect will likely impact how we collectively navigate a new digital age. This discussion reviews some of the diverse and transdisciplinary applications of BCI technology and draws speculative inferences about the ways in which BCI tools, combined with machine learning (ML) algorithms may shape the future. BCIs come with substantial ethical and risk considerations, and it is argued that open source principles may help us navigate complex dilemmas by encouraging experimentation and making developments public as we build safeguards into this new paradigm. Bringing open-source principles of adaptability and transparency to BCI tools can help democratize the technology, permitting more voices to contribute to the conversation of what a BCI-driven future should look like. Open-source BCI tools and access to raw data, in contrast to black-box algorithms and limited access to summary data, are critical facets enabling artists, DIYers, researchers and other domain experts to participate in the conversation about how to study and augment human consciousness. Looking forward to a future in which augmented and virtual reality become integral parts of daily life, BCIs will likely play an increasingly important role in creating closed-loop feedback for generative content. Brain-computer interfaces are uniquely situated to provide artificial intelligence (AI) algorithms the necessary data for determining the decoding and timing of content delivery. The extent to which these algorithms are open-source may be critical to examine them for integrity, implicit bias, and conflicts of interest.

Single-Trial EEG Connectivity of Default Mode Network Before and During Encoding Predicts Subsequent Memory Outcome

The successful memory process produces specific activity in the brain network. As the brain activity of the prestimulus and encoding phases has a crucial effect on subsequent memory outcomes (e.g., remembered or forgotten), previous studies have tried to predict the memory performance in this period. Conventional studies have used the spectral power or event-related potential of specific regions as the classification feature. However, as multiple brain regions work collaboratively to process memory, it could be a better option to use functional connectivity within the memory-related brain network to predict subsequent memory performance. In this study, we acquired the EEG signals while performing an associative memory task that remembers scene-word pairs. For the connectivity analysis, we estimated the cross-mutual information within the default mode network with the time-frequency spectra at the prestimulus and encoding phases. Then, we predicted the success or failure of subsequent memory outcome with the connectivity features. We found that the classifier with support vector machine achieved the highest classification accuracy of 80.83% ± 12.65% (mean ± standard deviation) using the beta (13-30 Hz) connectivity at encoding phase among the multiple frequency bands and task phases. Using the prestimulus beta connectivity, the classification accuracy of 72.45% ± 12.52% is also achieved. Among the features, the connectivity related to the dorsomedial prefrontal cortex was found to contribute to successful memory encoding. The connectivity related to the posterior cingulate cortex was found to contribute to the failure of memory encoding. The present study showed for the first time the successful prediction with high accuracy of subsequent memory outcome using single-trial functional connectivity.

Insights into human cognition from intracranial EEG: A review of audition, memory, internal cognition, and causality

By recording neural activity directly from the human brain, researchers gain unprecedented insight into how neurocognitive processes unfold in real time. We first briefly discuss how intracranial electroencephalography (iEEG) recordings, performed for clinical practice, are used to study human cognition with the spatiotemporal and single-trial precision traditionally limited to non-human animal research. We then delineate how studies using iEEG have informed our understanding of issues fundamental to human cognition: auditory prediction, working and episodic memory, and internal cognition. We also discuss the potential of iEEG to infer causality through the manipulation or 'engineering' of neurocognitive processes via spatiotemporally precise electrical stimulation. We close by highlighting limitations of iEEG, potential of burgeoning techniques to further increase spatiotemporal precision, and implications for future research using intracranial approaches to understand, restore, and enhance human cognition.

Brain Computer Interfaces for Improving the Quality of Life of Older Adults and Elderly Patients

All people experience aging, and the related physical and health changes, including changes in memory and brain function. These changes may become debilitating leading to an increase in dependence as people get older. Many external aids and tools have been developed to allow older adults and elderly patients to continue to live normal and comfortable lives. This mini-review describes some of the recent studies on cognitive decline and motor control impairment with the goal of advancing non-invasive brain computer interface (BCI) technologies to improve health and wellness of older adults and elderly patients. First, we describe the state of the art in cognitive prosthetics for psychiatric diseases. Then, we describe the state of the art of possible assistive BCI applications for controlling an exoskeleton, a wheelchair and smart home for elderly people with motor control impairments. The basic age-related brain and body changes, the effects of age on cognitive and motor abilities, and several BCI paradigms with typical tasks and outcomes are thoroughly described. We also discuss likely future trends and technologies to assist healthy older adults and elderly patients using innovative BCI applications with minimal technical oversight.

SimBSI: An open-source Simulink library for developing closed-loop brain signal interfaces in animals and humans

OBJECTIVE

A promising application of BCI technology is in the development of personalized therapies that can target neural circuits linked to mental or physical disabilities. Typical BCIs, however, offer limited value due to simplistic designs and poor understanding of the conditions being treated. Building BCIs on more solid grounds may require the characterization of the brain dynamics supporting cognition and behavior at multiple scales, from single-cell and local field potential (LFP) recordings in animals to non-invasive electroencephalography (EEG) in humans. Despite recent efforts, a unifying software framework to support closed-loop studies in both animals and humans is still lacking. The objective of this paper is to develop such a unifying neurotechnological software framework.

APPROACH

Here we develop the Simulink for Brain Signal Interfaces library (SimBSI). Simulink is a mature graphical programming environment within MATLAB that has gained traction for processing electrophysiological data. SimBSI adds to this ecosystem: 1) advanced human EEG source imaging, 2) cross-species multimodal data acquisition based on the Lab Streaming Layer library, and 3) a graphical experimental design platform.

MAIN RESULTS

We use several examples to demonstrate the capabilities of the library, ranging from simple signal processing, to online EEG source imaging, cognitive task design, and closed-loop neuromodulation. We further demonstrate the simplicity of developing a sophisticated experimental environment for rodents within this environment.

SIGNIFICANCE

With the SimBSI library we hope to aid BCI practitioners of dissimilar backgrounds in the development of, much needed, single and cross-species closed-loop neuroscientific experiments. These experiments may provide the necessary mechanistic data for BCIs to become effective therapeutic tools.

Unsupervised machine-learning classification of electrophysiologically active electrodes during human cognitive task performance

Identification of active electrodes that record task-relevant neurophysiological activity is needed for clinical and industrial applications as well as for investigating brain functions. We developed an unsupervised, fully automated approach to classify active electrodes showing event-related intracranial EEG (iEEG) responses from 115 patients performing a free recall verbal memory task. Our approach employed new interpretable metrics that quantify spectral characteristics of the normalized iEEG signal based on power-in-band and synchrony measures. Unsupervised clustering of the metrics identified distinct sets of active electrodes across different subjects. In the total population of 11,869 electrodes, our method achieved 97% sensitivity and 92.9% specificity with the most efficient metric. We validated our results with anatomical localization revealing significantly greater distribution of active electrodes in brain regions that support verbal memory processing. We propose our machine-learning framework for objective and efficient classification and interpretation of electrophysiological signals of brain activities supporting memory and cognition.

Hacking the Brain: Dimensions of Cognitive Enhancement

In an increasingly complex information society, demands for cognitive functioning are growing steadily. In recent years, numerous strategies to augment brain function have been proposed. Evidence for their efficacy (or lack thereof) and side effects has prompted discussions about ethical, societal, and medical implications. In the public debate, cognitive enhancement is often seen as a monolithic phenomenon. On a closer look, however, cognitive enhancement turns out to be a multifaceted concept: There is not one cognitive enhancer that augments brain function per se, but a great variety of interventions that can be clustered into biochemical, physical, and behavioral enhancement strategies. These cognitive enhancers differ in their mode of action, the cognitive domain they target, the time scale they work on, their availability and side effects, and how they differentially affect different groups of subjects. Here we disentangle the dimensions of cognitive enhancement, review prominent examples of cognitive enhancers that differ across these dimensions, and thereby provide a framework for both theoretical discussions and empirical research.

Network Brain-Computer Interface (nBCI): An Alternative Approach for Cognitive Prosthetics

Brain computer interfaces (BCIs) have been applied to sensorimotor systems for many years. However, BCI technology has broad potential beyond sensorimotor systems. The emerging field of cognitive prosthetics, for example, promises to improve learning and memory for patients with cognitive impairment. Unfortunately, our understanding of the neural mechanisms underlying these cognitive processes remains limited in part due to the extensive individual variability in neural coding and circuit function. As a consequence, the development of methods to ascertain optimal control signals for cognitive decoding and restoration remains an active area of inquiry. To advance the field, robust tools are required to quantify time-varying and task-dependent brain states predictive of cognitive performance. Here, we suggest that network science is a natural language in which to formulate and apply such tools. In support of our argument, we offer a simple demonstration of the feasibility of a network approach to BCI control signals, which we refer to as network BCI (nBCI). Finally, in a single subject example, we show that nBCI can reliably predict online cognitive performance and is superior to certain common spectral approaches currently used in BCIs. Our review of the literature and preliminary findings support the notion that nBCI could provide a powerful approach for future applications in cognitive prosthetics.

Brain-Computer Interface for Clinical Purposes: Cognitive Assessment and Rehabilitation

Alongside the best-known applications of brain-computer interface (BCI) technology for restoring communication abilities and controlling external devices, we present the state of the art of BCI use for cognitive assessment and training purposes. We first describe some preliminary attempts to develop verbal-motor free BCI-based tests for evaluating specific or multiple cognitive domains in patients with Amyotrophic Lateral Sclerosis, disorders of consciousness, and other neurological diseases. Then we present the more heterogeneous and advanced field of BCI-based cognitive training, which has its roots in the context of neurofeedback therapy and addresses patients with neurological developmental disorders (autism spectrum disorder and attention-deficit/hyperactivity disorder), stroke patients, and elderly subjects. We discuss some advantages of BCI for both assessment and training purposes, the former concerning the possibility of longitudinally and reliably evaluating cognitive functions in patients with severe motor disabilities, the latter regarding the possibility of enhancing patients' motivation and engagement for improving neural plasticity. Finally, we discuss some present and future challenges in the BCI use for the described purposes.

EEG Error Prediction as a Solution for Combining the Advantages of Retrieval Practice and Errorless Learning

Given the frequency of naming errors in aphasia, a common aim of speech and language rehabilitation is the improvement of naming. Based on evidence of significant word recall improvements in patients with memory impairments, errorless learning methods have been successfully applied to naming therapy in aphasia; however, other evidence suggests that although errorless learning can lead to better performance during treatment sessions, retrieval practice may be the key to lasting improvements. Task performance may vary with brain state (e.g., level of arousal, degree of task focus), and changes in brain state can be detected using EEG. With the ultimate goal of designing a system that monitors patient brain state in real time during therapy, we sought to determine whether errors could be predicted using spectral features obtained from an analysis of EEG. Thus, this study aimed to investigate the use of individual EEG responses to predict error production in aphasia. Eight participants with aphasia each completed 900 object-naming trials across three sessions while EEG was recorded and response accuracy scored for each trial. Analysis of the EEG response for seven of the eight participants showed significant correlations between EEG features and response accuracy (correct vs. incorrect) and error correction (correct, self-corrected, incorrect). Furthermore, upon combining the training data for the first two sessions, the model generalized to predict accuracy for performance in the third session for seven participants when accuracy was used as a predictor, and for five participants when error correction category was used as a predictor. With such ability to predict errors during therapy, it may be possible to use this information to intervene with errorless learning strategies only when necessary, thereby allowing patients to benefit from both the high within-session success of errorless learning as well as the longer-term improvements associated with retrieval practice.

Therapeutic Applications of BCI Technologies

Brain-computer interface (BCI) technology can restore communication and control to people who are severely paralyzed. There has been speculation that this technology might also be useful for a variety of diverse therapeutic applications. This survey considers possible ways that BCI technology can be applied to motor rehabilitation following stroke, Parkinson's disease, and psychiatric disorders. We consider potential neural signals as well as the design and goals of BCI-based therapeutic applications. These diverse applications all share a reliance on neuroimaging and signal processing technologies. At the same time, each of these potential applications presents a series of unique challenges.

Differential effects of ongoing EEG beta and theta power on memory formation

Recently, elevated ongoing pre-stimulus beta power (13–17 Hz) at encoding has been associated with subsequent memory formation for visual stimulus material. It is unclear whether this activity is merely specific to visual processing or whether it reflects a state facilitating general memory formation, independent of stimulus modality. To answer that question, the present study investigated the relationship between neural pre-stimulus oscillations and verbal memory formation in different sensory modalities. For that purpose, a within-subject design was employed to explore differences between successful and failed memory formation in the visual and auditory modality. Furthermore, associative memory was addressed by presenting the stimuli in combination with background images. Results revealed that similar EEG activity in the low beta frequency range (13–17 Hz) is associated with subsequent memory success, independent of stimulus modality. Elevated power prior to stimulus onset differentiated successful from failed memory formation. In contrast, differential effects between modalities were found in the theta band (3–7 Hz), with an increased oscillatory activity before the onset of later remembered visually presented words. In addition, pre-stimulus theta power dissociated between successful and failed encoding of associated context, independent of the stimulus modality of the item itself. We therefore suggest that increased ongoing low beta activity reflects a memory promoting state, which is likely to be moderated by modality-independent attentional or inhibitory processes, whereas high ongoing theta power is suggested as an indicator of the enhanced binding of incoming interlinked information.

Intention to encode boosts memory-related pre-stimulus EEG beta power

Pre-stimulus oscillatory brain activity can predict the degree to which an upcoming stimulus will be remembered at a later point in time. Recently, increased pre-stimulus power in ongoing theta (5-8Hz) and low beta (13-17Hz) bands during encoding has been associated with enhanced memory performance. When a cue is presented before stimulus onset, encoding-related brain activations may be regarded as a sign of preparation for the upcoming stimulus. Here, we investigated whether the intention to encode the following stimulus into long-term memory affects these preparatory pre-stimulus activations during encoding. Two groups of 18 participants took part in a subsequent memory paradigm. Electroencephalogram (EEG) was recorded while participants were presented with a series of pictures, each one preceded by a cue, which were supposed to be classified according to animacy. One group was informed about the upcoming recognition task and therefore was enabled to develop the intention to encode the stimuli (intentional encoding), whereas the other group did not receive this information (incidental encoding). Afterwards, recognition of the pictures was tested. During intentional encoding only, power in theta and low beta bands was found to be significantly increased before the onset of pictures that were later remembered compared to later forgotten ones. Group comparisons confirmed greater memory-related power increases in the low beta band for intentional than incidental encoding. These findings indicate that oscillatory states that are associated with successful encoding can be initiated voluntarily if the intention to encode the stimuli is given. We therefore suggest low beta band activation before stimulus onset to be an indicator of memory-specific preparation for an upcoming stimulus.

Expectation modulates neural representations of valence throughout the human brain

The brain's sensitivity to unexpected gains or losses plays an important role in our ability to learn new behaviors (Rescorla and Wagner, 1972; Sutton and Barto, 1990). Recent work suggests that gains and losses are ubiquitously encoded throughout the human brain (Vickery et al., 2011), however, the extent to which reward expectation modulates these valence representations is not known. To address this question, we analyzed recordings from 4306 intracranially implanted electrodes in 39 neurosurgical patients as they performed a two-alternative probability learning task. Using high-frequency activity (HFA, 70-200 Hz) as an indicator of local firing rates, we found that expectation modulated reward-related neural activity in widespread brain regions, including regions that receive sparse inputs from midbrain dopaminergic neurons. The strength of unexpected gain signals predicted subjects' abilities to encode stimulus-reward associations. Thus, neural signals that are functionally related to learning are widely distributed throughout the human brain.