Plug-and-play control of a brain-computer interface through neural map stabilization

Stability of ECoG high gamma signals during speech and implications for a speech BCI system in an individual with ALS: a year-long longitudinal study

Objective.Speech brain-computer interfaces (BCIs) have the potential to augment communication in individuals with impaired speech due to muscle weakness, for example in amyotrophic lateral sclerosis (ALS) and other neurological disorders. However, to achieve long-term, reliable use of a speech BCI, it is essential for speech-related neural signal changes to be stable over long periods of time. Here we study, for the first time, the stability of speech-related electrocorticographic (ECoG) signals recorded from a chronically implanted ECoG BCI over a 12 month period.Approach.ECoG signals were recorded by an ECoG array implanted over the ventral sensorimotor cortex in a clinical trial participant with ALS. Because ECoG-based speech decoding has most often relied on broadband high gamma (HG) signal changes relative to baseline (non-speech) conditions, we studied longitudinal changes of HG band power at baseline and during speech, and we compared these with residual high frequency noise levels at baseline. Stability was further assessed by longitudinal measurements of signal-to-noise ratio, activation ratio, and peak speech-related HG response magnitude (HG response peaks). Lastly, we analyzed the stability of the event-related HG power changes (HG responses) for individual syllables at each electrode.Main Results.We found that speech-related ECoG signal responses were stable over a range of syllables activating different articulators for the first year after implantation.Significance.Together, our results indicate that ECoG can be a stable recording modality for long-term speech BCI systems for those living with severe paralysis.Clinical Trial Information.ClinicalTrials.gov, registration number NCT03567213.

The speech neuroprosthesis

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages

Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish-English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS

Brain-computer interfaces (BCIs) that reconstruct and synthesize speech using brain activity recorded with intracranial electrodes may pave the way toward novel communication interfaces for people who have lost their ability to speak, or who are at high risk of losing this ability, due to neurological disorders. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a man with impaired articulation due to ALS, participating in a clinical trial (ClinicalTrials.gov, NCT03567213) exploring different strategies for BCI communication. The 3-stage approach reported here relies on recurrent neural networks to identify, decode and synthesize speech from electrocorticographic (ECoG) signals acquired across motor, premotor and somatosensory cortices. We demonstrate a reliable BCI that synthesizes commands freely chosen and spoken by the participant from a vocabulary of 6 keywords previously used for decoding commands to control a communication board. Evaluation of the intelligibility of the synthesized speech indicates that 80% of the words can be correctly recognized by human listeners. Our results show that a speech-impaired individual with ALS can use a chronically implanted BCI to reliably produce synthesized words while preserving the participant's voice profile, and provide further evidence for the stability of ECoG for speech-based BCIs.

The digital motor output: a conceptual framework for a meaningful clinical performance metric for a motor neuroprosthesis

In recent years, the majority of the population has become increasingly reliant on continuous and independent control of smart devices to conduct activities of daily living. Upper extremity movement is typically required to generate the motor outputs that control these interfaces, such as rapidly and accurately navigating and clicking a mouse, or activating a touch screen. For people living with tetraplegia, these abilities are lost, significantly compromising their ability to interact with their environment. Implantable brain computer interfaces (BCIs) hold promise for restoring lost neurologic function, including motor neuroprostheses (MNPs). An implantable MNP can directly infer motor intent by detecting brain signals and transmitting the motor signal out of the brain to generate a motor output and subsequently control computer actions. This physiological function is typically performed by the motor neurons in the human body. To evaluate the use of these implanted technologies, there is a need for an objective measurement of the effectiveness of MNPs in restoring motor outputs. Here, we propose the concept of digital motor outputs (DMOs) to address this: a motor output decoded directly from a neural recording during an attempted limb or orofacial movement is transformed into a command that controls an electronic device. Digital motor outputs are diverse and can be categorized as discrete or continuous representations of motor control, and the clinical utility of the control of a single, discrete DMO has been reported in multiple studies. This sets the stage for the DMO to emerge as a quantitative measure of MNP performance.

Visual tracking brain-computer interface

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's information transfer rate (ITR) of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI based-control method to go beyond discrete commands, allowing natural continuous control based on neural activity.

Cortico-cerebellar coordination facilitates neuroprosthetic control

Temporally coordinated neural activity is central to nervous system function and purposeful behavior. Still, there is a paucity of evidence demonstrating how this coordinated activity within cortical and subcortical regions governs behavior. We investigated this between the primary motor (M1) and contralateral cerebellar cortex as rats learned a neuroprosthetic/brain-machine interface (BMI) task. In neuroprosthetic task, actuator movements are causally linked to M1 "direct" neurons that drive the decoder for successful task execution. However, it is unknown how task-related M1 activity interacts with the cerebellum. We observed a notable 3 to 6 hertz coherence that emerged between these regions' local field potentials (LFPs) with learning that also modulated task-related spiking. We identified robust task-related indirect modulation in the cerebellum, which developed a preferential relationship with M1 task-related activity. Inhibiting cerebellar cortical and deep nuclei activity through optogenetics led to performance impairments in M1-driven neuroprosthetic control. Together, these results demonstrate that cerebellar influence is necessary for M1-driven neuroprosthetic control.

Assistive sensory-motor perturbations influence learned neural representations

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Population dimensionality remained constant or increased with learning, counter to trends with non-adaptive BCIs. Yet, over time, task information was contained in a smaller subset of neurons or population modes. Moreover, task information was ultimately stored in neural modes that occupied a small fraction of the population variance. An artificial neural network model suggests the adaptive decoders contribute to forming these compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations, like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing

Brain-computer interfaces (BCIs) can translate brain signals directly into commands for external devices. Electroencephalography (EEG)-based BCIs mostly rely on the classification of discrete mental states, leading to unintuitive control. The ERC-funded project "Feel Your Reach" aimed to establish a novel framework based on continuous decoding of hand/arm movement intention, for a more natural and intuitive control. Over the years, we investigated various aspects of natural control, however, the individual components had not yet been integrated. Here, we present a first implementation of the framework in a comprehensive online study, combining (i) goal-directed movement intention, (ii) trajectory decoding, and (iii) error processing in a unique closed-loop control paradigm. Testing involved twelve able-bodied volunteers, performing attempted movements, and one spinal cord injured (SCI) participant. Similar movement-related cortical potentials and error potentials to previous studies were revealed, and the attempted movement trajectories were overall reconstructed. Source analysis confirmed the involvement of sensorimotor and posterior parietal areas for goal-directed movement intention and trajectory decoding. The increased experiment complexity and duration led to a decreased performance than each single BCI. Nevertheless, the study contributes to understanding natural motor control, providing insights for more intuitive strategies for individuals with motor impairments.

Transfer learning promotes acquisition of individual BCI skills

Subject training is crucial for acquiring brain-computer interface (BCI) control. Typically, this requires collecting user-specific calibration data due to high inter-subject neural variability that limits the usability of generic decoders. However, calibration is cumbersome and may produce inadequate data for building decoders, especially with naïve subjects. Here, we show that a decoder trained on the data of a single expert is readily transferrable to inexperienced users via domain adaptation techniques allowing calibration-free BCI training. We introduce two real-time frameworks, (i) Generic Recentering (GR) through unsupervised adaptation and (ii) Personally Assisted Recentering (PAR) that extends GR by employing supervised recalibration of the decoder parameters. We evaluated our frameworks on 18 healthy naïve subjects over five online sessions, who operated a customary synchronous bar task with continuous feedback and a more challenging car racing game with asynchronous control and discrete feedback. We show that along with improved task-oriented BCI performance in both tasks, our frameworks promoted subjects' ability to acquire individual BCI skills, as the initial neurophysiological control features of an expert subject evolved and became subject specific. Furthermore, those features were task-specific and were learned in parallel as participants practiced the two tasks in every session. Contrary to previous findings implying that supervised methods lead to improved online BCI control, we observed that longitudinal training coupled with unsupervised domain matching (GR) achieved similar performance to supervised recalibration (PAR). Therefore, our presented frameworks facilitate calibration-free BCIs and have immediate implications for broader populations-such as patients with neurological pathologies-who might struggle to provide suitable initial calibration data.

Decoding motor plans using a closed-loop ultrasonic brain-machine interface

Brain-machine interfaces (BMIs) enable people living with chronic paralysis to control computers, robots and more with nothing but thought. Existing BMIs have trade-offs across invasiveness, performance, spatial coverage and spatiotemporal resolution. Functional ultrasound (fUS) neuroimaging is an emerging technology that balances these attributes and may complement existing BMI recording technologies. In this study, we use fUS to demonstrate a successful implementation of a closed-loop ultrasonic BMI. We streamed fUS data from the posterior parietal cortex of two rhesus macaque monkeys while they performed eye and hand movements. After training, the monkeys controlled up to eight movement directions using the BMI. We also developed a method for pretraining the BMI using data from previous sessions. This enabled immediate control on subsequent days, even those that occurred months apart, without requiring extensive recalibration. These findings establish the feasibility of ultrasonic BMIs, paving the way for a new class of less-invasive (epidural) interfaces that generalize across extended time periods and promise to restore function to people with neurological impairments.

Stable Decoding from a Speech BCI Enables Control for an Individual with ALS without Recalibration for 3 Months

Brain-computer interfaces (BCIs) can be used to control assistive devices by patients with neurological disorders like amyotrophic lateral sclerosis (ALS) that limit speech and movement. For assistive control, it is desirable for BCI systems to be accurate and reliable, preferably with minimal setup time. In this study, a participant with severe dysarthria due to ALS operates computer applications with six intuitive speech commands via a chronic electrocorticographic (ECoG) implant over the ventral sensorimotor cortex. Speech commands are accurately detected and decoded (median accuracy: 90.59%) throughout a 3-month study period without model retraining or recalibration. Use of the BCI does not require exogenous timing cues, enabling the participant to issue self-paced commands at will. These results demonstrate that a chronically implanted ECoG-based speech BCI can reliably control assistive devices over long time periods with only initial model training and calibration, supporting the feasibility of unassisted home use.

Neuronal functional connectivity is impaired in a layer dependent manner near the chronically implanted microelectrodes

Objective

This study aims to reveal longitudinal changes in functional network connectivity within and across different brain structures near the chronically implanted microelectrode. While it is well established that the foreign-body response (FBR) contributes to the gradual decline of the signals recorded from brain implants over time, how does the FBR impact affect the functional stability of neural circuits near implanted Brain-Computer Interfaces (BCIs) remains unknown. This research aims to illuminate how the chronic FBR can alter local neural circuit function and the implications for BCI decoders.

Approach

This study utilized multisite Michigan-style microelectrodes that span all cortical layers and the hippocampal CA1 region to collect spontaneous and visually-evoked electrophysiological activity. Alterations in neuronal activity near the microelectrode were tested assessing cross-frequency synchronization of LFP and spike entrainment to LFP oscillatory activity throughout 16 weeks after microelectrode implantation.

Main Results

The study found that cortical layer 4, the input-receiving layer, maintained activity over the implantation time. However, layers 2/3 rapidly experienced severe impairment, leading to a loss of proper intralaminar connectivity in the downstream output layers 5/6. Furthermore, the impairment of interlaminar connectivity near the microelectrode was unidirectional, showing decreased connectivity from Layers 2/3 to Layers 5/6 but not the reverse direction. In the hippocampus, CA1 neurons gradually became unable to properly entrain to the surrounding LFP oscillations.

Significance

This study provides a detailed characterization of network connectivity dysfunction over long-term microelectrode implantation periods. This new knowledge could contribute to the development of targeted therapeutic strategies aimed at improving the health of the tissue surrounding brain implants and potentially inform engineering of adaptive decoders as the FBR progresses. Our study's understanding of the dynamic changes in the functional network over time opens the door to developing interventions for improving the long-term stability and performance of intracortical microelectrodes.

Flexible Electrodes for Brain-Computer Interface System

Brain-computer interface (BCI) has been the subject of extensive research recently. Governments and companies have substantially invested in relevant research and applications. The restoration of communication and motor function, the treatment of psychological disorders, gaming, and other daily and therapeutic applications all benefit from BCI. The electrodes hold the key to the essential, fundamental BCI precondition of electrical brain activity detection and delivery. However, the traditional rigid electrodes are limited due to their mismatch in Young's modulus, potential damages to the human body, and a decline in signal quality with time. These factors make the development of flexible electrodes vital and urgent. Flexible electrodes made of soft materials have grown in popularity in recent years as an alternative to conventional rigid electrodes because they offer greater conformance, the potential for higher signal-to-noise ratio (SNR) signals, and a wider range of applications. Therefore, the latest classifications and future developmental directions of fabricating these flexible electrodes are explored in this paper to further encourage the speedy advent of flexible electrodes for BCI. In summary, the perspectives and future outlook for this developing discipline are provided.

Brainmask: an ultrasoft and moist micro-electrocorticography electrode for accurate positioning and long-lasting recordings

Bacterial cellulose (BC), a natural biomaterial synthesized by bacteria, has a unique structure of a cellulose nanofiber-weaved three-dimensional reticulated network. BC films can be ultrasoft with sufficient mechanical strength, strong water absorption and moisture retention and have been widely used in facial masks. These films have the potential to be applied to implantable neural interfaces due to their conformality and moisture, which are two critical issues for traditional polymer or silicone electrodes. In this work, we propose a micro-electrocorticography (micro-ECoG) electrode named "Brainmask", which comprises a BC film as the substrate and separated multichannel parylene-C microelectrodes bonded on the top surface. Brainmask can not only guarantee the precise position of microelectrode sites attached to any nonplanar epidural surface but also improve the long-lasting signal quality during acute implantation with an exposed cranial window for at least one hour, as well as the in vivo recording validated for one week. This novel ultrasoft and moist device stands as a next-generation neural interface regardless of complex surface or time of duration.

A click-based electrocorticographic brain-computer interface enables long-term high-performance switch-scan spelling

Background

Brain-computer interfaces (BCIs) can restore communication in movement- and/or speech-impaired individuals by enabling neural control of computer typing applications. Single command "click" decoders provide a basic yet highly functional capability.

Methods

We sought to test the performance and long-term stability of click-decoding using a chronically implanted high density electrocorticographic (ECoG) BCI with coverage of the sensorimotor cortex in a human clinical trial participant (ClinicalTrials.gov, NCT03567213) with amyotrophic lateral sclerosis (ALS). We trained the participant's click decoder using a small amount of training data (< 44 minutes across four days) collected up to 21 days prior to BCI use, and then tested it over a period of 90 days without any retraining or updating.

Results

Using this click decoder to navigate a switch-scanning spelling interface, the study participant was able to maintain a median spelling rate of 10.2 characters per min. Though a transient reduction in signal power modulation interrupted testing with this fixed model, a new click decoder achieved comparable performance despite being trained with even less data (< 15 min, within one day).

Conclusion

These results demonstrate that a click decoder can be trained with a small ECoG dataset while retaining robust performance for extended periods, providing functional text-based communication to BCI users.

Neural Data Transformer 2: Multi-context Pretraining for Neural Spiking Activity

The neural population spiking activity recorded by intracortical brain-computer interfaces (iBCIs) contain rich structure. Current models of such spiking activity are largely prepared for individual experimental contexts, restricting data volume to that collectable within a single session and limiting the effectiveness of deep neural networks (DNNs). The purported challenge in aggregating neural spiking data is the pervasiveness of context-dependent shifts in the neural data distributions. However, large scale unsupervised pretraining by nature spans heterogeneous data, and has proven to be a fundamental recipe for successful representation learning across deep learning. We thus develop Neural Data Transformer 2 (NDT2), a spatiotemporal Transformer for neural spiking activity, and demonstrate that pretraining can leverage motor BCI datasets that span sessions, subjects, and experimental tasks. NDT2 enables rapid adaptation to novel contexts in downstream decoding tasks and opens the path to deployment of pretrained DNNs for iBCI control. Code: https://github.com/joel99/context_general_bci.

EEG error-related potentials encode magnitude of errors and individual perceptual thresholds

Error-related potentials (ErrPs) are a prominent electroencephalogram (EEG) correlate of performance monitoring, and so crucial for learning and adapting our behavior. It is poorly understood whether ErrPs encode further information beyond error awareness. We report an experiment with sixteen participants over three sessions in which occasional visual rotations of varying magnitude occurred during a cursor reaching task. We designed a brain-computer interface (BCI) to detect ErrPs that provided real-time feedback. The individual ErrP-BCI decoders exhibited good transfer across sessions and scalability over the magnitude of errors. A non-linear relationship between the ErrP-BCI output and the magnitude of errors predicts individual perceptual thresholds to detect errors. We also reveal theta-gamma oscillatory coupling that co-varied with the magnitude of the required adjustment. Our findings open new avenues to probe and extend current theories of performance monitoring by incorporating continuous human interaction tasks and analysis of the ErrP complex rather than individual peaks.

I know what you're thinking; can neuroimaging truly reveal our innermost thoughts?

[Formula: see text] Advances in neuroimaging, combined with developments in artificial intelligence software, have allowed researchers to noninvasively decode the brain and 'read the mind'.

Flexible regulation of representations on a drifting manifold enables long-term stable complex neuroprosthetic control

The nervous system needs to balance the stability of neural representations with plasticity. It is unclear what is the representational stability of simple actions, particularly those that are well-rehearsed in humans, and how it changes in new contexts. Using an electrocorticography brain-computer interface (BCI), we found that the mesoscale manifold and relative representational distances for a repertoire of simple imagined movements were remarkably stable. Interestingly, however, the manifold's absolute location demonstrated day-to-day drift. Strikingly, representational statistics, especially variance, could be flexibly regulated to increase discernability during BCI control without somatotopic changes. Discernability strengthened with practice and was specific to the BCI, demonstrating remarkable contextual specificity. Accounting for drift, and leveraging the flexibility of representations, allowed neuroprosthetic control of a robotic arm and hand for over 7 months without recalibration. Our study offers insight into how electrocorticography can both track representational statistics across long periods and allow long-term complex neuroprosthetic control.

Brain-computer interface: trend, challenges, and threats

Brain-computer interface (BCI), an emerging technology that facilitates communication between brain and computer, has attracted a great deal of research in recent years. Researchers provide experimental results demonstrating that BCI can restore the capabilities of physically challenged people, hence improving the quality of their lives. BCI has revolutionized and positively impacted several industries, including entertainment and gaming, automation and control, education, neuromarketing, and neuroergonomics. Notwithstanding its broad range of applications, the global trend of BCI remains lightly discussed in the literature. Understanding the trend may inform researchers and practitioners on the direction of the field, and on where they should invest their efforts more. Noting this significance, we have analyzed 25,336 metadata of BCI publications from Scopus to determine advancement of the field. The analysis shows an exponential growth of BCI publications in China from 2019 onwards, exceeding those from the United States that started to decline during the same period. Implications and reasons for this trend are discussed. Furthermore, we have extensively discussed challenges and threats limiting exploitation of BCI capabilities. A typical BCI architecture is hypothesized to address two prominent BCI threats, privacy and security, as an attempt to make the technology commercially viable to the society.

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS

Recent studies have shown that speech can be reconstructed and synthesized using only brain activity recorded with intracranial electrodes, but until now this has only been done using retrospective analyses of recordings from able-bodied patients temporarily implanted with electrodes for epilepsy surgery. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a clinical trial participant (ClinicalTrials.gov, NCT03567213) with dysarthria due to amyotrophic lateral sclerosis (ALS). We demonstrate a reliable BCI that synthesizes commands freely chosen and spoken by the user from a vocabulary of 6 keywords originally designed to allow intuitive selection of items on a communication board. Our results show for the first time that a speech-impaired individual with ALS can use a chronically implanted BCI to reliably produce synthesized words that are intelligible to human listeners while preserving the participants voice profile.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Nine decades of electrocorticography: A comparison between epidural and subdural recordings

In recent years, electrocorticography (ECoG) has arisen as a neural signal recording tool in the development of clinically viable neural interfaces. ECoG electrodes are generally placed below the dura mater (subdural) but can also be placed on top of the dura (epidural). In deciding which of these modalities best suits long-term implants, complications and signal quality are important considerations. Conceptually, epidural placement may present a lower risk of complications as the dura is left intact but also a lower signal quality due to the dura acting as a signal attenuator. The extent to which complications and signal quality are affected by the dura, however, has been a matter of debate. To improve our understanding of the effects of the dura on complications and signal quality, we conducted a literature review. We inventorized the effect of the dura on signal quality, decodability and longevity of acute and chronic ECoG recordings in humans and non-human primates. Also, we compared the incidence and nature of serious complications in studies that employed epidural and subdural ECoG. Overall, we found that, even though epidural recordings exhibit attenuated signal amplitude over subdural recordings, particularly for high-density grids, the decodability of epidural recorded signals does not seem to be markedly affected. Additionally, we found that the nature of serious complications was comparable between epidural and subdural recordings. These results indicate that both epidural and subdural ECoG may be suited for long-term neural signal recordings, at least for current generations of clinical and high-density ECoG grids.

Recent Progress in Long-Term Sleep Monitoring Technology

Sleep is an essential physiological activity, accounting for about one-third of our lives, which significantly impacts our memory, mood, health, and children's growth. Especially after the COVID-19 epidemic, sleep health issues have attracted more attention. In recent years, with the development of wearable electronic devices, there have been more and more studies, products, or solutions related to sleep monitoring. Many mature technologies, such as polysomnography, have been applied to clinical practice. However, it is urgent to develop wearable or non-contacting electronic devices suitable for household continuous sleep monitoring. This paper first introduces the basic knowledge of sleep and the significance of sleep monitoring. Then, according to the types of physiological signals monitored, this paper describes the research progress of bioelectrical signals, biomechanical signals, and biochemical signals used for sleep monitoring. However, it is not ideal to monitor the sleep quality for the whole night based on only one signal. Therefore, this paper reviews the research on multi-signal monitoring and introduces systematic sleep monitoring schemes. Finally, a conclusion and discussion of sleep monitoring are presented to propose potential future directions and prospects for sleep monitoring.

Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performance

Introduction

In brain-computer interfaces (BCI) research, recording data is time-consuming and expensive, which limits access to big datasets. This may influence the BCI system performance as machine learning methods depend strongly on the training dataset size. Important questions arise: taking into account neuronal signal characteristics (e.g., non-stationarity), can we achieve higher decoding performance with more data to train decoders? What is the perspective for further improvement with time in the case of long-term BCI studies? In this study, we investigated the impact of long-term recordings on motor imagery decoding from two main perspectives: model requirements regarding dataset size and potential for patient adaptation.

Methods

We evaluated the multilinear model and two deep learning (DL) models on a long-term BCI &amp; Tetraplegia (ClinicalTrials.gov identifier: NCT02550522) clinical trial dataset containing 43 sessions of ECoG recordings performed with a tetraplegic patient. In the experiment, a participant executed 3D virtual hand translation using motor imagery patterns. We designed multiple computational experiments in which training datasets were increased or translated to investigate the relationship between models' performance and different factors influencing recordings.

Results

Our results showed that DL decoders showed similar requirements regarding the dataset size compared to the multilinear model while demonstrating higher decoding performance. Moreover, high decoding performance was obtained with relatively small datasets recorded later in the experiment, suggesting motor imagery patterns improvement and patient adaptation during the long-term experiment. Finally, we proposed UMAP embeddings and local intrinsic dimensionality as a way to visualize the data and potentially evaluate data quality.

Discussion

DL-based decoding is a prospective approach in BCI which may be efficiently applied with real-life dataset size. Patient-decoder co-adaptation is an important factor to consider in long-term clinical BCI.

Use of Invasive Brain-Computer Interfaces in Pediatric Neurosurgery: Technical and Ethical Considerations

Invasive brain-computer interfaces hold promise to alleviate disabilities in individuals with neurologic injury, with fully implantable brain-computer interface systems expected to reach the clinic in the upcoming decade. Children with severe neurologic disabilities, like quadriplegic cerebral palsy or cervical spine trauma, could benefit from this technology. However, they have been excluded from clinical trials of intracortical brain-computer interface to date. In this manuscript, we discuss the ethical considerations related to the use of invasive brain-computer interface in children with severe neurologic disabilities. We first review the technical hardware and software considerations for the application of intracortical brain-computer interface in children. We then discuss ethical issues related to motor brain-computer interface use in pediatric neurosurgery. Finally, based on the input of a multidisciplinary panel of experts in fields related to brain-computer interface (functional and restorative neurosurgery, pediatric neurosurgery, mathematics and artificial intelligence research, neuroengineering, pediatric ethics, and pragmatic ethics), we then formulate initial recommendations regarding the clinical use of invasive brain-computer interfaces in children.

Towards clinical application of implantable brain-computer interfaces for people with late-stage ALS: medical and ethical considerations

Individuals with amyotrophic lateral sclerosis (ALS) frequently develop speech and communication problems in the course of their disease. Currently available augmentative and alternative communication technologies do not present a solution for many people with advanced ALS, because these devices depend on residual and reliable motor activity. Brain-computer interfaces (BCIs) use neural signals for computer control and may allow people with late-stage ALS to communicate even when conventional technology falls short. Recent years have witnessed fast progression in the development and validation of implanted BCIs, which place neural signal recording electrodes in or on the cortex. Eventual widespread clinical application of implanted BCIs as an assistive communication technology for people with ALS will have significant consequences for their daily life, as well as for the clinical management of the disease, among others because of the potential interaction between the BCI and other procedures people with ALS undergo, such as tracheostomy. This article aims to facilitate responsible real-world implementation of implanted BCIs. We review the state of the art of research on implanted BCIs for communication, as well as the medical and ethical implications of the clinical application of this technology. We conclude that the contribution of all BCI stakeholders, including clinicians of the various ALS-related disciplines, will be needed to develop procedures for, and shape the process of, the responsible clinical application of implanted BCIs.

Learning to control a BMI-driven wheelchair for people with severe tetraplegia

Mind-controlled wheelchairs are an intriguing assistive mobility solution applicable in complete paralysis. Despite progress in brain-machine interface (BMI) technology, its translation remains elusive. The primary objective of this study is to probe the hypothesis that BMI skill acquisition by end-users is fundamental to control a non-invasive brain-actuated intelligent wheelchair in real-world settings. We demonstrate that three tetraplegic spinal-cord injury users could be trained to operate a non-invasive, self-paced thought-controlled wheelchair and execute complex navigation tasks. However, only the two users exhibiting increasing decoding performance and feature discriminancy, significant neuroplasticity changes and improved BMI command latency, achieved high navigation performance. In addition, we show that dexterous, continuous control of robots is possible through low-degree of freedom, discrete and uncertain control channels like a motor imagery BMI, by blending human and artificial intelligence through shared-control methodologies. We posit that subject learning and shared-control are the key components paving the way for translational non-invasive BMI.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.

Design-development of an at-home modular brain-computer interface (BCI) platform in a case study of cervical spinal cord injury

OBJECTIVE

The objective of this study was to develop a portable and modular brain-computer interface (BCI) software platform independent of input and output devices. We implemented this platform in a case study of a subject with cervical spinal cord injury (C5 ASIA A).

BACKGROUND

BCIs can restore independence for individuals with paralysis by using brain signals to control prosthetics or trigger functional electrical stimulation. Though several studies have successfully implemented this technology in the laboratory and the home, portability, device configuration, and caregiver setup remain challenges that limit deployment to the home environment. Portability is essential for transitioning BCI from the laboratory to the home.

METHODS

The BCI platform implementation consisted of an Activa PC + S generator with two subdural four-contact electrodes implanted over the dominant left hand-arm region of the sensorimotor cortex, a minicomputer fixed to the back of the subject's wheelchair, a custom mobile phone application, and a mechanical glove as the end effector. To quantify the performance for this at-home implementation of the BCI, we quantified system setup time at home, chronic (14-month) decoding accuracy, hardware and software profiling, and Bluetooth communication latency between the App and the minicomputer. We created a dataset of motor-imagery labeled signals to train a binary motor imagery classifier on a remote computer for online, at-home use.

RESULTS

Average bluetooth data transmission delay between the minicomputer and mobile App was 23 ± 0.014 ms. The average setup time for the subject's caregiver was 5.6 ± 0.83 min. The average times to acquire and decode neural signals and to send those decoded signals to the end-effector were respectively 404.1 ms and 1.02 ms. The 14-month median accuracy of the trained motor imagery classifier was 87.5 ± 4.71% without retraining.

CONCLUSIONS

The study presents the feasibility of an at-home BCI system that subjects can seamlessly operate using a friendly mobile user interface, which does not require daily calibration nor the presence of a technical person for at-home setup. The study also describes the portability of the BCI system and the ability to plug-and-play multiple end effectors, providing the end-user the flexibility to choose the end effector to accomplish specific motor tasks for daily needs. Trial registration ClinicalTrials.gov: NCT02564419. First posted on 9/30/2015.

Noninvasively recorded high-gamma signals improve synchrony of force feedback in a novel neurorehabilitation brain-machine interface for brain injury

OBJECTIVE

Brain injury is the leading cause of long-term disability worldwide, often resulting in impaired hand function. Brain-machine interfaces (BMIs) offer a potential way to improve hand function. BMIs often target replacing lost function, but may also be employed in neurorehabilitation (nrBMI) by facilitating neural plasticity and functional recovery. Here, we report a novel nrBMI capable of acquiring high-γ (70-115 Hz) information through a unique post-TBI hemicraniectomy window model, and delivering sensory feedback that is synchronized with, and proportional to, intended grasp force.

APPROACH

We developed the nrBMI to use electroencephalogram recorded over a hemicraniectomy (hEEG) in individuals with traumatic brain injury (TBI). The nrBMI empowered users to exert continuous, proportional control of applied force, and provided continuous force feedback. We report the results of an initial testing group of three human participants with TBI, along with a control group of three skull- and motor-intact volunteers.

MAIN RESULTS

All participants controlled the nrBMI successfully, with high initial success rates (2 of 6 participants) or performance that improved over time (4 of 6 participants). We observed high-γ modulation with force intent in hEEG but not skull-intact EEG. Most significantly, we found that high-γ control significantly improved the timing synchronization between neural modulation onset and nrBMI output/haptic feedback (compared to low-frequency nrBMI control).

SIGNIFICANCE

These proof-of-concept results show that high-γ nrBMIs can be used by individuals with impaired ability to control force (without immediately resorting to invasive signals like ECoG). Of note, the nrBMI includes a parameter to change the fraction of control shared between decoded intent and volitional force, to adjust for recovery progress. The improved synchrony between neural modulations and force control for high-γ signals is potentially important for maximizing the ability of nrBMIs to induce plasticity in neural circuits. Inducing plasticity is critical to functional recovery after brain injury.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Decoding ECoG signal into 3D hand translation using deep learning

Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.

Feel Your Reach: An EEG-Based Framework to Continuously Detect Goal-Directed Movements and Error Processing to Gate Kinesthetic Feedback Informed Artificial Arm Control

Establishing the basic knowledge, methodology, and technology for a framework for the continuous decoding of hand/arm movement intention was the aim of the ERC-funded project “Feel Your Reach”. In this work, we review the studies and methods we performed and implemented in the last 6 years, which build the basis for enabling severely paralyzed people to non-invasively control a robotic arm in real-time from electroencephalogram (EEG). In detail, we investigated goal-directed movement detection, decoding of executed and attempted movement trajectories, grasping correlates, error processing, and kinesthetic feedback. Although we have tested some of our approaches already with the target populations, we still need to transfer the “Feel Your Reach” framework to people with cervical spinal cord injury and evaluate the decoders’ performance while participants attempt to perform upper-limb movements. While on the one hand, we made major progress towards this ambitious goal, we also critically discuss current limitations.

Compartmentalized dynamics within a common multi-area mesoscale manifold represent a repertoire of human hand movements

The human hand is unique in the animal kingdom for unparalleled dexterity, ranging from complex prehension to fine finger individuation. How does the brain represent such a diverse repertoire of movements? We evaluated mesoscale neural dynamics across the human "grasp network," using electrocorticography and dimensionality reduction methods, for a repertoire of hand movements. Strikingly, we found that the grasp network represented both finger and grasping movements alike. Specifically, the manifold characterizing the multi-areal neural covariance structure was preserved during all movements across this distributed network. In contrast, latent neural dynamics within this manifold were surprisingly specific to movement type. Aligning latent activity to kinematics further uncovered distinct submanifolds despite similarities in synergistic coupling of joints between movements. We thus find that despite preserved neural covariance at the distributed network level, mesoscale dynamics are compartmentalized into movement-specific submanifolds; this mesoscale organization may allow flexible switching between a repertoire of hand movements.

Brain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication

Damage or degeneration of motor pathways necessary for speech and other movements, as in brainstem strokes or amyotrophic lateral sclerosis (ALS), can interfere with efficient communication without affecting brain structures responsible for language or cognition. In the worst-case scenario, this can result in the locked in syndrome (LIS), a condition in which individuals cannot initiate communication and can only express themselves by answering yes/no questions with eye blinks or other rudimentary movements. Existing augmentative and alternative communication (AAC) devices that rely on eye tracking can improve the quality of life for people with this condition, but brain-computer interfaces (BCIs) are also increasingly being investigated as AAC devices, particularly when eye tracking is too slow or unreliable. Moreover, with recent and ongoing advances in machine learning and neural recording technologies, BCIs may offer the only means to go beyond cursor control and text generation on a computer, to allow real-time synthesis of speech, which would arguably offer the most efficient and expressive channel for communication. The potential for BCI speech synthesis has only recently been realized because of seminal studies of the neuroanatomical and neurophysiological underpinnings of speech production using intracranial electrocorticographic (ECoG) recordings in patients undergoing epilepsy surgery. These studies have shown that cortical areas responsible for vocalization and articulation are distributed over a large area of ventral sensorimotor cortex, and that it is possible to decode speech and reconstruct its acoustics from ECoG if these areas are recorded with sufficiently dense and comprehensive electrode arrays. In this article, we review these advances, including the latest neural decoding strategies that range from deep learning models to the direct concatenation of speech units. We also discuss state-of-the-art vocoders that are integral in constructing natural-sounding audio waveforms for speech BCIs. Finally, this review outlines some of the challenges ahead in directly synthesizing speech for patients with LIS.

Timescales of Local and Cross-Area Interactions during Neuroprosthetic Learning

How does the brain integrate signals with different timescales to drive purposeful actions? Brain-machine interfaces (BMIs) offer a powerful tool to causally test how distributed neural networks achieve specific neural patterns. During neuroprosthetic learning, actuator movements are causally linked to primary motor cortex (M1) neurons, i.e., "direct" neurons that project to the decoder and whose firing is required to successfully perform the task. However, it is unknown how such direct M1 neurons interact with both "indirect" local (in M1 but not part of the decoder) and across area neural populations (e.g., in premotor cortex/M2), all of which are embedded in complex biological recurrent networks. Here, we trained male rats to perform a M1-BMI task and simultaneously recorded the activity of indirect neurons in both M2 and M1. We found that both M2 and M1 indirect neuron populations could be used to predict the activity of the direct neurons (i.e., "BMI-potent activity"). Interestingly, compared with M1 indirect activity, M2 neural activity was correlated with BMI-potent activity across a longer set of time lags, and the timescale of population activity patterns evolved more slowly. M2 units also predicted the activity of both M1 direct and indirect neural populations, suggesting that M2 population dynamics provide a continuous modulatory influence on M1 activity as a whole, rather than a moment-by-moment influence solely on neurons most relevant to a task. Together, our results indicate that longer timescale M2 activity provides modulatory influence over extended time lags on shorter-timescale control signals in M1.SIGNIFICANCE STATEMENT A central question in the study of motor control is whether primary motor cortex (M1) and premotor cortex (M2) interact through task-specific subpopulations of neurons, or whether tasks engage broader correlated networks. Brain-machine interfaces (BMIs) are powerful tools to study cross-area interactions. Here, we performed simultaneous recordings of M1 and M2 in a BMI task using a subpopulation of M1 neurons (direct neurons). We found that activity outside of direct neurons in M1 and M2 was predictive of M1-BMI task activity, and that M2 activity evolved at slower timescales than M1. These findings suggest that M2 provides a continuous modulatory influence on M1 as a whole, supporting a model of interactions through broad correlated networks rather than task-specific neural subpopulations.

Graphene and graphene-related materials as brain electrodes

Neural electrodes are used for acquiring neuron signals in brain-machine interfaces, and they are crucial for next-generation neuron engineering and related medical applications. Thus, developing flexible, stable and high-resolution neural electrodes will play an important role in stimulation, acquisition, recording and analysis of signals. Compared with traditional metallic electrodes, electrodes based on graphene and other two-dimensional materials have attracted wide attention in electrophysiological recording and stimulation due to their excellent physical properties such as unique flexibility, low resistance, and high optical transparency. In this review, we have reviewed the recent progress of electrodes based on graphene, graphene/polymer compounds and graphene-related materials for neuron signal recording, stimulation, and related optical signal coupling technology, which provides an outlook on the role of electrodes in the nanotechnology-neuron interface as well as medical diagnosis.

Long-term stability of the chronic epidural wireless recorder WIMAGINE in tetraplegic patients

Objective.The evaluation of the long-term stability of ElectroCorticoGram (ECoG) signals is an important scientific question as new implantable recording devices can be used for medical purposes such as Brain-Computer Interface (BCI) or brain monitoring.Approach.The long-term clinical validation of wireless implantable multi-channel acquisition system for generic interface with neurons (WIMAGINE), a wireless 64-channel epidural ECoG recorder was investigated. The WIMAGINE device was implanted in two quadriplegic patients within the context of a BCI protocol. This study focused on the ECoG signal stability in two patients bilaterally implanted in June 2017 (P1) and in November 2019 (P2).Methods. The ECoG signal was recorded at rest prior to each BCI session resulting in a 32 month and in a 14 month follow-up for P1 and P2 respectively. State-of-the-art signal evaluation metrics such as root mean square (RMS), the band power (BP), the signal to noise ratio (SNR), the effective bandwidth (EBW) and the spectral edge frequency (SEF) were used to evaluate stability of signal over the implantation time course. The time-frequency maps obtained from task-related motor activations were also studied to investigate the long-term selectivity of the electrodes.Mainresults.Based on temporal linear regressions, we report a limited decrease of the signal average level (RMS), spectral distribution (BP) and SNR, and a remarkable steadiness of the EBW and SEF. Time-frequency maps obtained during motor imagery, showed a high level of discrimination 1 month after surgery and also after 2 years.Conclusions.The WIMAGINE epidural device showed high stability over time. The signal evaluation metrics of two quadriplegic patients during 32 months and 14 months respectively provide strong evidence that this wireless implant is well-suited for long-term ECoG recording.Significance.These findings are relevant for the future of implantable BCIs, and could benefit other patients with spinal cord injury, amyotrophic lateral sclerosis, neuromuscular diseases or drug-resistant epilepsy.

Multi-scale neural decoding and analysis

Objective.

Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.

Approach.

We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.

Main results.

We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.

Significance.

This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria

BACKGROUND

Technology to restore the ability to communicate in paralyzed persons who cannot speak has the potential to improve autonomy and quality of life. An approach that decodes words and sentences directly from the cerebral cortical activity of such patients may represent an advancement over existing methods for assisted communication.

METHODS

We implanted a subdural, high-density, multielectrode array over the area of the sensorimotor cortex that controls speech in a person with anarthria (the loss of the ability to articulate speech) and spastic quadriparesis caused by a brain-stem stroke. Over the course of 48 sessions, we recorded 22 hours of cortical activity while the participant attempted to say individual words from a vocabulary set of 50 words. We used deep-learning algorithms to create computational models for the detection and classification of words from patterns in the recorded cortical activity. We applied these computational models, as well as a natural-language model that yielded next-word probabilities given the preceding words in a sequence, to decode full sentences as the participant attempted to say them.

RESULTS

We decoded sentences from the participant's cortical activity in real time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%. In post hoc analyses, we detected 98% of the attempts by the participant to produce individual words, and we classified words with 47.1% accuracy using cortical signals that were stable throughout the 81-week study period.

CONCLUSIONS

In a person with anarthria and spastic quadriparesis caused by a brain-stem stroke, words and sentences were decoded directly from cortical activity during attempted speech with the use of deep-learning models and a natural-language model. (Funded by Facebook and others; ClinicalTrials.gov number, NCT03698149.).

Home Use of a Percutaneous Wireless Intracortical Brain-Computer Interface by Individuals With Tetraplegia

OBJECTIVE

Individuals with neurological disease or injury such as amyotrophic lateral sclerosis, spinal cord injury or stroke may become tetraplegic, unable to speak or even locked-in. For people with these conditions, current assistive technologies are often ineffective. Brain-computer interfaces are being developed to enhance independence and restore communication in the absence of physical movement. Over the past decade, individuals with tetraplegia have achieved rapid on-screen typing and point-and-click control of tablet apps using intracortical brain-computer interfaces (iBCIs) that decode intended arm and hand movements from neural signals recorded by implanted microelectrode arrays. However, cables used to convey neural signals from the brain tether participants to amplifiers and decoding computers and require expert oversight, severely limiting when and where iBCIs could be available for use. Here, we demonstrate the first human use of a wireless broadband iBCI.

METHODS

Based on a prototype system previously used in pre-clinical research, we replaced the external cables of a 192-electrode iBCI with wireless transmitters and achieved high-resolution recording and decoding of broadband field potentials and spiking activity from people with paralysis. Two participants in an ongoing pilot clinical trial completed on-screen item selection tasks to assess iBCI-enabled cursor control.

RESULTS

Communication bitrates were equivalent between cabled and wireless configurations. Participants also used the wireless iBCI to control a standard commercial tablet computer to browse the web and use several mobile applications. Within-day comparison of cabled and wireless interfaces evaluated bit error rate, packet loss, and the recovery of spike rates and spike waveforms from the recorded neural signals. In a representative use case, the wireless system recorded intracortical signals from two arrays in one participant continuously through a 24-hour period at home.

SIGNIFICANCE

Wireless multi-electrode recording of broadband neural signals over extended periods introduces a valuable tool for human neuroscience research and is an important step toward practical deployment of iBCI technology for independent use by individuals with paralysis. On-demand access to high-performance iBCI technology in the home promises to enhance independence and restore communication and mobility for individuals with severe motor impairment.

Reinvigorating electrochemistry education

Electrochemistry is an established discipline with modern frontiers spanning energy conversion and storage, neuroscience, and organic synthesis. In spite of the expanding opportunities for academic and industrial electrochemists, particularly in the growing energy-storage sector, rigorous training of electrochemists is generally lacking at academic institutions in the United States. In this perspective, we highlight the core concepts of electrochemistry and discuss ways in which it has been historically taught. We identify challenges faced when teaching inherently interdisciplinary electrochemical concepts and discuss how technology provides new tools for teaching, such as inexpensive electronics and open-source software, to help address these challenges. Finally, we outline example programs and discuss how new tools and approaches can be brought together to prepare scientists and engineers for careers in electrochemical technology where they can accelerate the research, development, and deployment of the clean energy technology essential to combat climate change in the coming decades.

An artificial intelligence that increases simulated brain-computer interface performance

Objective.Brain-computer interfaces (BCIs) translate neural activity into control signals for assistive devices in order to help people with motor disabilities communicate effectively. In this work, we introduce a new BCI architecture that improves control of a BCI computer cursor to type on a virtual keyboard.Approach.Our BCI architecture incorporates an external artificial intelligence (AI) that beneficially augments the movement trajectories of the BCI. This AI-BCI leverages past user actions, at both long (100 s of seconds ago) and short (100 s of milliseconds ago) timescales, to modify the BCI's trajectories.Main results.We tested our AI-BCI in a closed-loop BCI simulator with nine human subjects performing a typing task. We demonstrate that our AI-BCI achieves: (1) categorically higher information communication rates, (2) quicker ballistic movements between targets, (3) improved precision control to 'dial in' on targets, and (4) more efficient movement trajectories. We further show that our AI-BCI increases performance across a wide control quality spectrum from poor to proficient control.Significance.This AI-BCI architecture, by increasing BCI performance across all key metrics evaluated, may increase the clinical viability of BCI systems.

A Framework for Optimizing Co-adaptation in Body-Machine Interfaces

The operation of a human-machine interface is increasingly often referred to as a two-learners problem, where both the human and the interface independently adapt their behavior based on shared information to improve joint performance over a specific task. Drawing inspiration from the field of body-machine interfaces, we take a different perspective and propose a framework for studying co-adaptation in scenarios where the evolution of the interface is dependent on the users' behavior and that do not require task goals to be explicitly defined. Our mathematical description of co-adaptation is built upon the assumption that the interface and the user agents co-adapt toward maximizing the interaction efficiency rather than optimizing task performance. This work describes a mathematical framework for body-machine interfaces where a naïve user interacts with an adaptive interface. The interface, modeled as a linear map from a space with high dimension (the user input) to a lower dimensional feedback, acts as an adaptive “tool” whose goal is to minimize transmission loss following an unsupervised learning procedure and has no knowledge of the task being performed by the user. The user is modeled as a non-stationary multivariate Gaussian generative process that produces a sequence of actions that is either statistically independent or correlated. Dependent data is used to model the output of an action selection module concerned with achieving some unknown goal dictated by the task. The framework assumes that in parallel to this explicit objective, the user is implicitly learning a suitable but not necessarily optimal way to interact with the interface. Implicit learning is modeled as use-dependent learning modulated by a reward-based mechanism acting on the generative distribution. Through simulation, the work quantifies how the system evolves as a function of the learning time scales when a user learns to operate a static vs. an adaptive interface. We show that this novel framework can be directly exploited to readily simulate a variety of interaction scenarios, to facilitate the exploration of the parameters that lead to optimal learning dynamics of the joint system, and to provide an empirical proof for the superiority of human-machine co-adaptation over user adaptation.