High-performance brain-to-text communication via handwriting

A high-performance speech neuroprosthesis

Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speaking movements into text or sound. Early demonstrations, while promising, have not yet achieved accuracies high enough for communication of unconstrainted sentences from a large vocabulary. Here, we demonstrate the first speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant, who can no longer speak intelligibly due amyotrophic lateral sclerosis (ALS), achieved a 9.1% word error rate on a 50 word vocabulary (2.7 times fewer errors than the prior state of the art speech BCI2) and a 23.8% word error rate on a 125,000 word vocabulary (the first successful demonstration of large-vocabulary decoding). Our BCI decoded speech at 62 words per minute, which is 3.4 times faster than the prior record for any kind of BCI and begins to approach the speed of natural conversation (160 words per minute). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for using intracortical speech BCIs to restore rapid communication to people with paralysis who can no longer speak.

Translating deep learning to neuroprosthetic control

Advances in deep learning have given rise to neural network models of the relationship between movement and brain activity that appear to far outperform prior approaches. Brain-computer interfaces (BCIs) that enable people with paralysis to control external devices, such as robotic arms or computer cursors, might stand to benefit greatly from these advances. We tested recurrent neural networks (RNNs) on a challenging nonlinear BCI problem: decoding continuous bimanual movement of two computer cursors. Surprisingly, we found that although RNNs appeared to perform well in offline settings, they did so by overfitting to the temporal structure of the training data and failed to generalize to real-time neuroprosthetic control. In response, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously, far outperforming standard linear methods. Our results provide evidence that preventing models from overfitting to temporal structure in training data may, in principle, aid in translating deep learning advances to the BCI setting, unlocking improved performance for challenging applications.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

The nested hierarchy of overt, mouthed, and imagined speech activity evident in intracranial recordings

Recent studies have demonstrated that it is possible to decode and synthesize various aspects of acoustic speech directly from intracranial measurements of electrophysiological brain activity. In order to continue progressing toward the development of a practical speech neuroprosthesis for the individuals with speech impairments, better understanding and modeling of imagined speech processes are required. The present study uses intracranial brain recordings from participants that performed a speaking task with trials consisting of overt, mouthed, and imagined speech modes, representing various degrees of decreasing behavioral output. Speech activity detection models are constructed using spatial, spectral, and temporal brain activity features, and the features and model performances are characterized and compared across the three degrees of behavioral output. The results indicate the existence of a hierarchy in which the relevant channels for the lower behavioral output modes form nested subsets of the relevant channels from the higher behavioral output modes. This provides important insights for the elusive goal of developing more effective imagined speech decoding models with respect to the better-established overt speech decoding counterparts.

A brain-computer typing interface using finger movements

Intracortical brain computer interfaces (iBCIs) decode neural activity from the cortex and enable motor and communication prostheses, such as cursor control, handwriting and speech, for people with paralysis. This paper introduces a new iBCI communication prosthesis using a 3D keyboard interface for typing using continuous, closed loop movement of multiple fingers. A participant-specific BCI keyboard prototype was developed for a BrainGate2 clinical trial participant (T5) using neural recordings from the hand-knob area of the left premotor cortex. We assessed the relative decoding accuracy of flexion/extension movements of individual single fingers (5 degrees of freedom (DOF)) vs. three groups of fingers (thumb, index-middle, and ring-small fingers, 3 DOF). Neural decoding using 3 independent DOF was more accurate (95%) than that using 5 DOF (76%). A virtual keyboard was then developed where each finger group moved along a flexion-extension arc to acquire targets that corresponded to English letters and symbols. The locations of these letter/symbols were optimized using natural language statistics, resulting in an approximately a 2× reduction in distance traveled by fingers on average compared to a random keyboard layout. This keyboard was tested using a simple real-time closed loop decoder enabling T5 to type with 31 symbols at 90% accuracy and approximately 2.3 sec/symbol (excluding a 2 second hold time) on average.

Neurocognitive and motor-control challenges for the realization of bionic augmentation

Robotic fingers and arms that augment the motor abilities of non-disabled individuals are increasingly feasible yet face neurocognitive barriers and hurdles in efferent motor control.

Using Automatic Speech Recognition to Measure the Intelligibility of Speech Synthesized from Brain Signals

Brain-computer interfaces (BCIs) can potentially restore lost function in patients with neurological injury. A promising new application of BCI technology has focused on speech restoration. One approach is to synthesize speech from the neural correlates of a person who cannot speak, as they attempt to do so. However, there is no established gold-standard for quantifying the quality of BCI-synthesized speech. Quantitative metrics, such as applying correlation coefficients between true and decoded speech, are not applicable to anarthric users and fail to capture intelligibility by actual human listeners; by contrast, methods involving people completing forced-choice multiple-choice questionnaires are imprecise, not practical at scale, and cannot be used as cost functions for improving speech decoding algorithms. Here, we present a deep learning-based "AI Listener" that can be used to evaluate BCI speech intelligibility objectively, rapidly, and automatically. We begin by adapting several leading Automatic Speech Recognition (ASR) deep learning models - Deepspeech, Wav2vec 2.0, and Kaldi - to suit our application. We then evaluate the performance of these ASRs on multiple speech datasets with varying levels of intelligibility, including: healthy speech, speech from people with dysarthria, and synthesized BCI speech. Our results demonstrate that the multiple-language ASR model XLSR-Wav2vec 2.0, trained to output phonemes, yields superior performance in terms of speech transcription accuracy. Notably, the AI Listener reports that several previously published BCI output datasets are not intelligible, which is consistent with human listeners.

Dynamical flexible inference of nonlinear latent structures in neural population activity

Inferring complex spatiotemporal dynamics in neural population activity is critical for investigating neural mechanisms and developing neurotechnology. These activity patterns are noisy observations of lower-dimensional latent factors and their nonlinear dynamical structure. A major unaddressed challenge is to model this nonlinear structure, but in a manner that allows for flexible inference, whether causally, non-causally, or in the presence of missing neural observations. We address this challenge by developing DFINE, a new neural network that separates the model into dynamic and manifold latent factors, such that the dynamics can be modeled in tractable form. We show that DFINE achieves flexible nonlinear inference across diverse behaviors and brain regions. Further, despite enabling flexible inference unlike prior neural network models of population activity, DFINE also better predicts the behavior and neural activity, and better captures the latent neural manifold structure. DFINE can both enhance future neurotechnology and facilitate investigations across diverse domains of neuroscience.

Interim Safety Profile From the Feasibility Study of the BrainGate Neural Interface System

BACKGROUND AND OBJECTIVES

Brain-computer interfaces (BCIs) are being developed to restore mobility, communication, and functional independence to people with paralysis. Though supported by decades of preclinical data, the safety of chronically implanted microelectrode array BCIs in humans is unknown. We report safety results from the prospective, open-label, nonrandomized BrainGate feasibility study (NCT00912041), the largest and longest-running clinical trial of an implanted BCI.

METHODS

Adults aged 18-75 years with quadriparesis from spinal cord injury, brainstem stroke, or motor neuron disease were enrolled through 7 clinical sites in the United States. Participants underwent surgical implantation of 1 or 2 microelectrode arrays in the motor cortex of the dominant cerebral hemisphere. The primary safety outcome was device-related serious adverse events (SAEs) requiring device explantation or resulting in death or permanently increased disability during the 1-year postimplant evaluation period. The secondary outcomes included the type and frequency of other adverse events and the feasibility of the BrainGate system for controlling a computer or other assistive technologies.

RESULTS

From 2004 to 2021, 14 adults enrolled in the BrainGate trial had devices surgically implanted. The average duration of device implantation was 872 days, yielding 12,203 days of safety experience. There were 68 device-related adverse events, including 6 device-related SAEs. The most common device-related adverse event was skin irritation around the percutaneous pedestal. There were no safety events that required device explantation, no unanticipated adverse device events, no intracranial infections, and no participant deaths or adverse events resulting in permanently increased disability related to the investigational device.

DISCUSSION

The BrainGate Neural Interface system has a safety record comparable with other chronically implanted medical devices. Given rapid recent advances in this technology and continued performance gains, these data suggest a favorable risk/benefit ratio in appropriately selected individuals to support ongoing research and development.

TRIAL REGISTRATION INFORMATION

ClinicalTrials.gov Identifier: NCT00912041.

CLASSIFICATION OF EVIDENCE

This study provides Class IV evidence that the neurosurgically placed BrainGate Neural Interface system is associated with a low rate of SAEs defined as those requiring device explantation, resulting in death, or resulting in permanently increased disability during the 1-year postimplant period.

Assessment of Safety of a Fully Implanted Endovascular Brain-Computer Interface for Severe Paralysis in 4 Patients: The Stentrode With Thought-Controlled Digital Switch (SWITCH) Study

Importance

Brain-computer interface (BCI) implants have previously required craniotomy to deliver penetrating or surface electrodes to the brain. Whether a minimally invasive endovascular technique to deliver recording electrodes through the jugular vein to superior sagittal sinus is safe and feasible is unknown.

Objective

To assess the safety of an endovascular BCI and feasibility of using the system to control a computer by thought.

Design, Setting, and Participants

The Stentrode With Thought-Controlled Digital Switch (SWITCH) study, a single-center, prospective, first in-human study, evaluated 5 patients with severe bilateral upper-limb paralysis, with a follow-up of 12 months. From a referred sample, 4 patients with amyotrophic lateral sclerosis and 1 with primary lateral sclerosis met inclusion criteria and were enrolled in the study. Surgical procedures and follow-up visits were performed at the Royal Melbourne Hospital, Parkville, Australia. Training sessions were performed at patients' homes and at a university clinic. The study start date was May 27, 2019, and final follow-up was completed January 9, 2022.

Interventions

Recording devices were delivered via catheter and connected to subcutaneous electronic units. Devices communicated wirelessly to an external device for personal computer control.

Main Outcomes and Measures

The primary safety end point was device-related serious adverse events resulting in death or permanent increased disability. Secondary end points were blood vessel occlusion and device migration. Exploratory end points were signal fidelity and stability over 12 months, number of distinct commands created by neuronal activity, and use of system for digital device control.

Results

Of 4 patients included in analyses, all were male, and the mean (SD) age was 61 (17) years. Patients with preserved motor cortex activity and suitable venous anatomy were implanted. Each completed 12-month follow-up with no serious adverse events and no vessel occlusion or device migration. Mean (SD) signal bandwidth was 233 (16) Hz and was stable throughout study in all 4 patients (SD range across all sessions, 7-32 Hz). At least 5 attempted movement types were decoded offline, and each patient successfully controlled a computer with the BCI.

Conclusions and Relevance

Endovascular access to the sensorimotor cortex is an alternative to placing BCI electrodes in or on the dura by open-brain surgery. These final safety and feasibility data from the first in-human SWITCH study indicate that it is possible to record neural signals from a blood vessel. The favorable safety profile could promote wider and more rapid translation of BCI to people with paralysis.

Trial Registration

ClinicalTrials.gov Identifier: NCT03834857.

Use of Invasive Brain-Computer Interfaces in Pediatric Neurosurgery: Technical and Ethical Considerations

Invasive brain-computer interfaces hold promise to alleviate disabilities in individuals with neurologic injury, with fully implantable brain-computer interface systems expected to reach the clinic in the upcoming decade. Children with severe neurologic disabilities, like quadriplegic cerebral palsy or cervical spine trauma, could benefit from this technology. However, they have been excluded from clinical trials of intracortical brain-computer interface to date. In this manuscript, we discuss the ethical considerations related to the use of invasive brain-computer interface in children with severe neurologic disabilities. We first review the technical hardware and software considerations for the application of intracortical brain-computer interface in children. We then discuss ethical issues related to motor brain-computer interface use in pediatric neurosurgery. Finally, based on the input of a multidisciplinary panel of experts in fields related to brain-computer interface (functional and restorative neurosurgery, pediatric neurosurgery, mathematics and artificial intelligence research, neuroengineering, pediatric ethics, and pragmatic ethics), we then formulate initial recommendations regarding the clinical use of invasive brain-computer interfaces in children.

Restoring arm function with a soft robotic wearable for individuals with amyotrophic lateral sclerosis

Despite promising results in the rehabilitation field, it remains unclear whether upper limb robotic wearables, e.g., for people with physical impairments resulting from neurodegenerative disease, can be made portable and suitable for everyday use. We present a lightweight, fully portable, textile-based, soft inflatable wearable robot for shoulder elevation assistance that provides dynamic active support to the upper limbs. The technology is mechanically transparent when unpowered, can quantitatively assess free movement of the user, and adds only 150 grams of weight to each upper limb. In 10 individuals with amyotrophic lateral sclerosis (ALS) with different degrees of neuromuscular impairment, we demonstrated immediate improvement in the active range of motion and compensation for continuing physical deterioration in two individuals with ALS over 6 months. Along with improvements in movement, we show that this robotic wearable can improve functional activity without any training, restoring performance of basic activities of daily living. In addition, a reduction in shoulder muscle activity and perceived muscular exertion, coupled with increased endurance for holding objects, highlight the potential of this device to mitigate the impact of muscular fatigue for patients with ALS. These results represent a further step toward everyday use of assistive, soft, robotic wearables for the upper limbs.

Conformable neural interface based on off-stoichiometry thiol-ene-epoxy thermosets

Off-stoichiometry thiol-ene-epoxy (OSTE+) thermosets show low permeability to gases and little absorption of dissolved molecules, allow direct low-temperature dry bonding without surface treatments, have a low Young's modulus, and can be manufactured via UV polymerisation. For these reasons, OSTE+ thermosets have recently gained attention for the rapid prototyping of microfluidic chips. Moreover, their compatibility with standard clean-room processes and outstanding mechanical properties make OSTE+ an excellent candidate as a novel material for neural implants. Here we exploit OSTE+ to manufacture a conformable multilayer micro-electrocorticography array with 16 platinum electrodes coated with platinum black. The mechanical properties allow conformability to curved surfaces such as the brain. The low permeability and strong adhesion between layers improve the stability of the device. Acute experiments in mice show the multimodal capacity of the array to record and stimulate the neural tissue by smoothly conforming to the mouse cortex. Devices are not cytotoxic, and immunohistochemistry stainings reveal only modest foreign body reaction after two and six weeks of chronic implantation. This work introduces OSTE+ as a promising material for implantable neural interfaces.

Power-efficientin vivobrain-machine interfaces via brain-state estimation

Objective.Advances in brain-machine interfaces (BMIs) can potentially improve the quality of life of millions of users with spinal cord injury or other neurological disorders by allowing them to interact with the physical environment at their will.Approach.To reduce the power consumption of the brain-implanted interface, this article presents the first hardware realization of anin vivointention-aware interface via brain-state estimation.Main Results.It is shown that incorporating brain-state estimation reduces thein vivopower consumption and reduces total energy dissipation by over 1.8× compared to those of the current systems, enabling longer better life for implanted circuits. The synthesized application-specific integrated circuit (ASIC) of the designed intention-aware multi-unit spike detection system in a standard 180 nm CMOS process occupies 0.03 mm²of silicon area and consumes 0.63 µW of power per channel, which is the least power consumption among the currentin vivoASIC realizations.Significance.The proposed interface is the first practical approach towards realizing asynchronous BMIs while reducing the power consumption of the BMI interface and enhancing neural decoding performance compared to those of the conventional synchronous BMIs.

Proceedings of the 10th annual deep brain stimulation think tank: Advances in cutting edge technologies, artificial intelligence, neuromodulation, neuroethics, interventional psychiatry, and women in neuromodulation

The deep brain stimulation (DBS) Think Tank X was held on August 17-19, 2022 in Orlando FL. The session organizers and moderators were all women with the theme women in neuromodulation. Dr. Helen Mayberg from Mt. Sinai, NY was the keynote speaker. She discussed milestones and her experiences in developing depression DBS. The DBS Think Tank was founded in 2012 and provides an open platform where clinicians, engineers and researchers (from industry and academia) can freely discuss current and emerging DBS technologies as well as the logistical and ethical issues facing the field. The consensus among the DBS Think Tank X speakers was that DBS has continued to expand in scope however several indications have reached the "trough of disillusionment." DBS for depression was considered as "re-emerging" and approaching a slope of enlightenment. DBS for depression will soon re-enter clinical trials. The group estimated that globally more than 244,000 DBS devices have been implanted for neurological and neuropsychiatric disorders. This year's meeting was focused on advances in the following areas: neuromodulation in Europe, Asia, and Australia; cutting-edge technologies, closed loop DBS, DBS tele-health, neuroethics, lesion therapy, interventional psychiatry, and adaptive DBS.

Modulating Brain Activity with Invasive Brain-Computer Interface: A Narrative Review

Brain-computer interface (BCI) can be used as a real-time bidirectional information gateway between the brain and machines. In particular, rapid progress in invasive BCI, propelled by recent developments in electrode materials, miniature and power-efficient electronics, and neural signal decoding technologies has attracted wide attention. In this review, we first introduce the concepts of neuronal signal decoding and encoding that are fundamental for information exchanges in BCI. Then, we review the history and recent advances in invasive BCI, particularly through studies using neural signals for controlling external devices on one hand, and modulating brain activity on the other hand. Specifically, regarding modulating brain activity, we focus on two types of techniques, applying electrical stimulation to cortical and deep brain tissues, respectively. Finally, we discuss the related ethical issues concerning the clinical application of this emerging technology.

Brain-Computer interface control of stepping from invasive electrocorticography upper-limb motor imagery in a patient with quadriplegia

Introduction: Most spinal cord injuries (SCI) result in lower extremities paralysis, thus diminishing ambulation. Using brain-computer interfaces (BCI), patients may regain leg control using neural signals that actuate assistive devices. Here, we present a case of a subject with cervical SCI with an implanted electrocorticography (ECoG) device and determined whether the system is capable of motor-imagery-initiated walking in an assistive ambulator. Methods: A 24-year-old male subject with cervical SCI (C5 ASIA A) was implanted before the study with an ECoG sensing device over the sensorimotor hand region of the brain. The subject used motor-imagery (MI) to train decoders to classify sensorimotor rhythms. Fifteen sessions of closed-loop trials followed in which the subject ambulated for one hour on a robotic-assisted weight-supported treadmill one to three times per week. We evaluated the stability of the best-performing decoder over time to initiate walking on the treadmill by decoding upper-limb (UL) MI. Results: An online bagged trees classifier performed best with an accuracy of 84.15% averaged across 9 weeks. Decoder accuracy remained stable following throughout closed-loop data collection. Discussion: These results demonstrate that decoding UL MI is a feasible control signal for use in lower-limb motor control. Invasive BCI systems designed for upper-extremity motor control can be extended for controlling systems beyond upper extremity control alone. Importantly, the decoders used were able to use the invasive signal over several weeks to accurately classify MI from the invasive signal. More work is needed to determine the long-term consequence between UL MI and the resulting lower-limb control.

Neuroprosthetics: from sensorimotor to cognitive disorders

Neuroprosthetics is a multidisciplinary field at the interface between neurosciences and biomedical engineering, which aims at replacing or modulating parts of the nervous system that get disrupted in neurological disorders or after injury. Although neuroprostheses have steadily evolved over the past 60 years in the field of sensory and motor disorders, their application to higher-order cognitive functions is still at a relatively preliminary stage. Nevertheless, a recent series of proof-of-concept studies suggest that electrical neuromodulation strategies might also be useful in alleviating some cognitive and memory deficits, in particular in the context of dementia. Here, we review the evolution of neuroprosthetics from sensorimotor to cognitive disorders, highlighting important common principles such as the need for neuroprosthetic systems that enable multisite bidirectional interactions with the nervous system.

Non-human primate models and systems for gait and neurophysiological analysis

Brain-computer interfaces (BCIs) have garnered extensive interest and become a groundbreaking technology to restore movement, tactile sense, and communication in patients. Prior to their use in human subjects, clinical BCIs require rigorous validation and verification (V&V). Non-human primates (NHPs) are often considered the ultimate and widely used animal model for neuroscience studies, including BCIs V&V, due to their proximity to humans. This literature review summarizes 94 NHP gait analysis studies until 1 June, 2022, including seven BCI-oriented studies. Due to technological limitations, most of these studies used wired neural recordings to access electrophysiological data. However, wireless neural recording systems for NHPs enabled neuroscience research in humans, and many on NHP locomotion, while posing numerous technical challenges, such as signal quality, data throughout, working distance, size, and power constraint, that have yet to be overcome. Besides neurological data, motion capture (MoCap) systems are usually required in BCI and gait studies to capture locomotion kinematics. However, current studies have exclusively relied on image processing-based MoCap systems, which have insufficient accuracy (error: ≥4° and 9 mm). While the role of the motor cortex during locomotion is still unclear and worth further exploration, future BCI and gait studies require simultaneous, high-speed, accurate neurophysiological, and movement measures. Therefore, the infrared MoCap system which has high accuracy and speed, together with a high spatiotemporal resolution neural recording system, may expand the scope and improve the quality of the motor and neurophysiological analysis in NHPs.

TC-Net: A Transformer Capsule Network for EEG-based emotion recognition

Deep learning has recently achieved remarkable success in emotion recognition based on Electroencephalogram (EEG), in which convolutional neural networks (CNNs) are the mostly used models. However, due to the local feature learning mechanism, CNNs have difficulty in capturing the global contextual information involving temporal domain, frequency domain, intra-channel and inter-channel. In this paper, we propose a Transformer Capsule Network (TC-Net), which mainly contains an EEG Transformer module to extract EEG features and an Emotion Capsule module to refine the features and classify the emotion states. In the EEG Transformer module, EEG signals are partitioned into non-overlapping windows. A Transformer block is adopted to capture global features among different windows, and we propose a novel patch merging strategy named EEG-PatchMerging (EEG-PM) to better extract local features. In the Emotion Capsule module, each channel of the EEG feature maps is encoded into a capsule to better characterize the spatial relationships among multiple features. Experimental results on two popular datasets (i.e., DEAP and DREAMER) demonstrate that the proposed method achieves the state-of-the-art performance in the subject-dependent scenario. Specifically, on DEAP (DREAMER), our TC-Net achieves the average accuracies of 98.76% (98.59%), 98.81% (98.61%) and 98.82% (98.67%) at valence, arousal and dominance dimensions, respectively. Moreover, the proposed TC-Net also shows high effectiveness in multi-state emotion recognition tasks using the popular VA and VAD models. The main limitation of the proposed model is that it tends to obtain relatively low performance in the cross-subject recognition task, which is worthy of further study in the future.

Design of virtual BCI channels based on informer

The precision and reliability of electroencephalogram (EEG) data are essential for the effective functioning of a brain-computer interface (BCI). As the number of BCI acquisition channels increases, more EEG information can be gathered. However, having too many channels will reduce the practicability of the BCI system, raise the likelihood of poor-quality channels, and lead to information misinterpretation. These issues pose challenges to the advancement of BCI systems. Determining the optimal configuration of BCI acquisition channels can minimize the number of channels utilized, but it is challenging to maintain the original operating system and accommodate individual variations in channel layout. To address these concerns, this study introduces the EEG-completion-informer (EC-informer), which is based on the Informer architecture known for its effectiveness in time-series problems. By providing input from four BCI acquisition channels, the EC-informer can generate several virtual acquisition channels to extract additional EEG information for analysis. This approach allows for the direct inheritance of the original model, significantly reducing researchers' workload. Moreover, EC-informers demonstrate strong performance in damaged channel repair and poor channel identification. Using the Informer as a foundation, the study proposes the EC-informer, tailored to BCI requirements and demanding only a small number of training samples. This approach eliminates the need for extensive computing units to train an efficient, lightweight model while preserving comprehensive information about target channels. The study also confirms that the proposed model can be transferred to other operators with minimal loss, exhibiting robust applicability. The EC-informer's features enable original BCI devices to adapt to a broader range of classification algorithms and relax the operational requirements of BCI devices, which could facilitate the promotion of the use of BCI devices in daily life.

Towards clinical application of implantable brain-computer interfaces for people with late-stage ALS: medical and ethical considerations

Individuals with amyotrophic lateral sclerosis (ALS) frequently develop speech and communication problems in the course of their disease. Currently available augmentative and alternative communication technologies do not present a solution for many people with advanced ALS, because these devices depend on residual and reliable motor activity. Brain-computer interfaces (BCIs) use neural signals for computer control and may allow people with late-stage ALS to communicate even when conventional technology falls short. Recent years have witnessed fast progression in the development and validation of implanted BCIs, which place neural signal recording electrodes in or on the cortex. Eventual widespread clinical application of implanted BCIs as an assistive communication technology for people with ALS will have significant consequences for their daily life, as well as for the clinical management of the disease, among others because of the potential interaction between the BCI and other procedures people with ALS undergo, such as tracheostomy. This article aims to facilitate responsible real-world implementation of implanted BCIs. We review the state of the art of research on implanted BCIs for communication, as well as the medical and ethical implications of the clinical application of this technology. We conclude that the contribution of all BCI stakeholders, including clinicians of the various ALS-related disciplines, will be needed to develop procedures for, and shape the process of, the responsible clinical application of implanted BCIs.

Neural Dielet: A 0.4mm 3 Battery-Less Crystal-Less Neural-Recording System on Die Achieving 1.6cm Backscatter Range with 2mm×2mm On-Chip Antenna

Miniaturization is essential in the design of wireless neural-recording systems. In recent years, the battery in neural-recording systems can be eliminated by wireless power transfer (WPT), while antenna and crystal become two main bottlenecks to minimize a battery-less neural implant. In conventional battery-less designs, the miniaturization of antenna led to a short communication range, and a crystal-less clock suffered from noise issue or power-hungry circuits. In this work, we demonstrate a 0.4mm ³ neural dielet, which is a battery-less crystal-less neural-recording system on die (SoD) within a 2mm×2mm on-chip coil antenna. The communication range through the ultra-small antenna is extended by a proposed dither-based 3 ^rd-order intermodulation (IM3) technique, which prevents the backscatter communication from WPT blocker. Meanwhile, a dither-based 2 ^nd-order intermodulation (IM2) wireless-lock technique is proposed to remove the crystal. Measured results show that the SoD consumes 53.2 μ W power and achieves a wireless communication range of 1.6cm at a bit-error rate (BER) of 8 ×10^-6, accompanied by simultaneous WPT for battery-less operation. In the animal experiment, the neural signal wirelessly recorded by our SoD in a battery-less way matches favorably with the wire-test results obtained by a commercial chip.

Microscale multicircuit brain stimulation: Achieving real-time brain state control for novel applications

Neurological and psychiatric disorders typically result from dysfunction across multiple neural circuits. Most of these disorders lack a satisfactory neuromodulation treatment. However, deep brain stimulation (DBS) has been successful in a limited number of disorders; DBS typically targets one or two brain areas with single contacts on relatively large electrodes, allowing for only coarse modulation of circuit function. Because of the dysfunction in distributed neural circuits - each requiring fine, tailored modulation - that characterizes most neuropsychiatric disorders, this approach holds limited promise. To develop the next generation of neuromodulation therapies, we will have to achieve fine-grained, closed-loop control over multiple neural circuits. Recent work has demonstrated spatial and frequency selectivity using microstimulation with many small, closely-spaced contacts, mimicking endogenous neural dynamics. Using custom electrode design and stimulation parameters, it should be possible to achieve bidirectional control over behavioral outcomes, such as increasing or decreasing arousal during central thalamic stimulation. Here, we discuss one possible approach, which we term microscale multicircuit brain stimulation (MMBS). We discuss how machine learning leverages behavioral and neural data to find optimal stimulation parameters across multiple contacts, to drive the brain towards desired states associated with behavioral goals. We expound a mathematical framework for MMBS, where behavioral and neural responses adjust the model in real-time, allowing us to adjust stimulation in real-time. These technologies will be critical to the development of the next generation of neurostimulation therapies, which will allow us to treat problems like disorders of consciousness and cognition.

Noninvasive neuroimaging and spatial filter transform enable ultra low delay motor imagery EEG decoding

Objective.The brain-computer interface (BCI) system based on sensorimotor rhythm can convert the human spirit into instructions for machine control, and it is a new human-computer interaction system with broad applications. However, the spatial resolution of scalp electroencephalogram (EEG) is limited due to the presence of volume conduction effects. Therefore, it is very meaningful to explore intracranial activities in a noninvasive way and improve the spatial resolution of EEG. Meanwhile, low-delay decoding is an essential factor for the development of a real-time BCI system.Approach.In this paper, EEG conduction is modeled by using public head anatomical templates, and cortical EEG is obtained using dynamic parameter statistical mapping. To solve the problem of a large amount of computation caused by the increase in the number of channels, the filter bank common spatial pattern method is used to obtain a spatial filter kernel, which reduces the computational cost of feature extraction to a linear level. And the feature classification and selection of important features are completed using a neural network containing band-spatial-time domain self-attention mechanisms.Main results.The results show that the method proposed in this paper achieves high accuracy for the four types of motor imagery EEG classification tasks, with fairly low latency and high physiological interpretability.Significance.The proposed decoding framework facilitates the realization of low-latency human-computer interaction systems.

Partially binarized neural networks for efficient spike sorting

While brain-implantable neural spike sorting can be realized using efficient algorithms, the presence of noise may make it difficult to maintain high-peformance sorting using conventional techniques. In this article, we explore the use of partially binarized neural networks (PBNNs), to the best of our knowledge for the first time, for sorting of neural spike feature vectors. It is shown that compared to the waveform template-based methods, PBNNs offer robust spike sorting over various datasets and noise levels. The ASIC implementation of the PBNN-based spike sorting system in a standard 180-nm CMOS process is presented. The post place and route simulations results show that the synthesized PBNN consumes only 0.59 μ W of power from a 1.8 V supply while operating at 24 kHz and occupies 0.15 mm 2 of silicon area. It is shown that the designed PBNN-based spike sorting system not only offers comparable accuracy to the state-of-the-art spike sorting systems over various noise levels and datasets, it also occupies a smaller silicon area and consumes less power and energy. This makes PBNNs a viable alternative towards the implementation of brain-implantable spike sorting systems.

Neuron devices: emerging prospects in neural interfaces and recognition

Neuron interface devices can be used to explore the relationships between neuron firing and synaptic transmission, as well as to diagnose and treat neurological disorders, such as epilepsy and Alzheimer's disease. It is crucial to exploit neuron devices with high sensitivity, high biocompatibility, multifunctional integration and high-speed data processing. During the past decades, researchers have made significant progress in neural electrodes, artificial sensory neuron devices, and neuromorphic optic neuron devices. The main part of the review is divided into two sections, providing an overview of recently developed neuron interface devices for recording electrophysiological signals, as well as applications in neuromodulation, simulating the human sensory system, and achieving memory and recognition. We mainly discussed the development, characteristics, functional mechanisms, and applications of neuron devices and elucidated several key points for clinical translation. The present review highlights the advances in neuron devices on brain-computer interfaces and neuroscience research.

Recent advances in regenerative biomaterials

Nowadays, biomaterials have evolved from the inert supports or functional substitutes to the bioactive materials able to trigger or promote the regenerative potential of tissues. The interdisciplinary progress has broadened the definition of 'biomaterials', and a typical new insight is the concept of tissue induction biomaterials. The term 'regenerative biomaterials' and thus the contents of this article are relevant to yet beyond tissue induction biomaterials. This review summarizes the recent progress of medical materials including metals, ceramics, hydrogels, other polymers and bio-derived materials. As the application aspects are concerned, this article introduces regenerative biomaterials for bone and cartilage regeneration, cardiovascular repair, 3D bioprinting, wound healing and medical cosmetology. Cell-biomaterial interactions are highlighted. Since the global pandemic of coronavirus disease 2019, the review particularly mentions biomaterials for public health emergency. In the last section, perspectives are suggested: (i) creation of new materials is the source of innovation; (ii) modification of existing materials is an effective strategy for performance improvement; (iii) biomaterial degradation and tissue regeneration are required to be harmonious with each other; (iv) host responses can significantly influence the clinical outcomes; (v) the long-term outcomes should be paid more attention to; (vi) the noninvasive approaches for monitoring in vivo dynamic evolution are required to be developed; (vii) public health emergencies call for more research and development of biomaterials; and (viii) clinical translation needs to be pushed forward in a full-chain way. In the future, more new insights are expected to be shed into the brilliant field-regenerative biomaterials.

Decoding cognition in real-time

How can we study unobservable cognitive processes that cannot be measured directly? This has been an enduring challenge for cognitive scientists. In this essay we discuss advances in neurotechnology that could allow cognitive processes to be decoded in real-time and the implications that this may have for cognitive science and the treatment of neuropsychiatric disease.

Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network

Objective. Speech decoding, one of the most intriguing brain-computer interface applications, opens up plentiful opportunities from rehabilitation of patients to direct and seamless communication between human species. Typical solutions rely on invasive recordings with a large number of distributed electrodes implanted through craniotomy. Here we explored the possibility of creating speech prosthesis in a minimally invasive setting with a small number of spatially segregated intracranial electrodes.Approach. We collected one hour of data (from two sessions) in two patients implanted with invasive electrodes. We then used only the contacts that pertained to a single stereotactic electroencephalographic (sEEG) shaft or an electrocorticographic (ECoG) stripe to decode neural activity into 26 words and one silence class. We employed a compact convolutional network-based architecture whose spatial and temporal filter weights allow for a physiologically plausible interpretation.Mainresults. We achieved on average 55% accuracy using only six channels of data recorded with a single minimally invasive sEEG electrode in the first patient and 70% accuracy using only eight channels of data recorded for a single ECoG strip in the second patient in classifying 26+1 overtly pronounced words. Our compact architecture did not require the use of pre-engineered features, learned fast and resulted in a stable, interpretable and physiologically meaningful decision rule successfully operating over a contiguous dataset collected during a different time interval than that used for training. Spatial characteristics of the pivotal neuronal populations corroborate with active and passive speech mapping results and exhibit the inverse space-frequency relationship characteristic of neural activity. Compared to other architectures our compact solution performed on par or better than those recently featured in neural speech decoding literature.Significance. We showcase the possibility of building a speech prosthesis with a small number of electrodes and based on a compact feature engineering free decoder derived from a small amount of training data.

Real-time brain-machine interface in non-human primates achieves high-velocity prosthetic finger movements using a shallow feedforward neural network decoder

Despite the rapid progress and interest in brain-machine interfaces that restore motor function, the performance of prosthetic fingers and limbs has yet to mimic native function. The algorithm that converts brain signals to a control signal for the prosthetic device is one of the limitations in achieving rapid and realistic finger movements. To achieve more realistic finger movements, we developed a shallow feed-forward neural network to decode real-time two-degree-of-freedom finger movements in two adult male rhesus macaques. Using a two-step training method, a recalibrated feedback intention-trained (ReFIT) neural network is introduced to further improve performance. In 7 days of testing across two animals, neural network decoders, with higher-velocity and more natural appearing finger movements, achieved a 36% increase in throughput over the ReFIT Kalman filter, which represents the current standard. The neural network decoders introduced herein demonstrate real-time decoding of continuous movements at a level superior to the current state-of-the-art and could provide a starting point to using neural networks for the development of more naturalistic brain-controlled prostheses.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.

Bioelectronic medicine: Preclinical insights and clinical advances

The nervous system maintains homeostasis and health. Homeostatic disruptions underlying the pathobiology of many diseases can be controlled by bioelectronic devices targeting CNS and peripheral neural circuits. New insights into the regulatory functions of the nervous system and technological developments in bioelectronics drive progress in the emerging field of bioelectronic medicine. Here, we provide an overview of key aspects of preclinical research, translation, and clinical advances in bioelectronic medicine.

Methodological Recommendations for Studies on the Daily Life Implementation of Implantable Communication-Brain-Computer Interfaces for Individuals With Locked-in Syndrome

Implantable brain-computer interfaces (BCIs) promise to be a viable means to restore communication in individuals with locked-in syndrome (LIS). In 2016, we presented the world-first fully implantable BCI system that uses subdural electrocorticography electrodes to record brain signals and a subcutaneous amplifier to transmit the signals to the outside world, and that enabled an individual with LIS to communicate via a tablet computer by selecting icons in spelling software. For future clinical implementation of implantable communication-BCIs, however, much work is still needed, for example, to validate these systems in daily life settings with more participants, and to improve the speed of communication. We believe the design and execution of future studies on these and other topics may benefit from the experience we have gained. Therefore, based on relevant literature and our own experiences, we here provide an overview of procedures, as well as recommendations, for recruitment, screening, inclusion, imaging, hospital admission, implantation, training, and support of participants with LIS, for studies on daily life implementation of implantable communication-BCIs. With this article, we not only aim to inform the BCI community about important topics of concern, but also hope to contribute to improved methodological standardization of implantable BCI research.

Deep brain-machine interfaces: sensing and modulating the human deep brain

Different from conventional brain-machine interfaces that focus more on decoding the cerebral cortex, deep brain-machine interfaces enable interactions between external machines and deep brain structures. They sense and modulate deep brain neural activities, aiming at function restoration, device control and therapeutic improvements. In this article, we provide an overview of multiple deep brain recording and stimulation techniques that can serve as deep brain-machine interfaces. We highlight two widely used interface technologies, namely deep brain stimulation and stereotactic electroencephalography, for technical trends, clinical applications and brain connectivity research. We discuss the potential to develop closed-loop deep brain-machine interfaces and achieve more effective and applicable systems for the treatment of neurological and psychiatric disorders.

Validation of a non-invasive, real-time, human-in-the-loop model of intracortical brain-computer interfaces

Objective. Despite the tremendous promise of invasive brain-computer interfaces (iBCIs), the associated study costs, risks, and ethical considerations limit the opportunity to develop and test the algorithms that decode neural activity into a user's intentions. Our goal was to address this challenge by designing an iBCI model capable of testing many human subjects in closed-loop.Approach. We developed an iBCI model that uses artificial neural networks (ANNs) to translate human finger movements into realistic motor cortex firing patterns, which can then be decoded in real time. We call the model the joint angle BCI, or jaBCI. jaBCI allows readily recruited, healthy subjects to perform closed-loop iBCI tasks using any neural decoder, preserving subjects' control-relevant short-latency error correction and learning dynamics.Main results. We validated jaBCI offline through emulated neuron firing statistics, confirming that emulated neural signals have firing rates, low-dimensional PCA geometry, and rotational jPCA dynamics that are quite similar to the actual neurons (recorded in monkey M1) on which we trained the ANN. We also tested jaBCI in closed-loop experiments, our single study examining roughly as many subjects as have been tested world-wide with iBCIs (n= 25). Performance was consistent with that of the paralyzed, human iBCI users with implanted intracortical electrodes. jaBCI allowed us to imitate the experimental protocols (e.g. the same velocity Kalman filter decoder and center-out task) and compute the same seven behavioral measures used in three critical studies.Significance. These encouraging results suggest the jaBCI's real-time firing rate emulation is a useful means to provide statistically robust sample sizes for rapid prototyping and optimization of decoding algorithms, the study of bi-directional learning in iBCIs, and improving iBCI control.

Soft integration of a neural cells network and bionic interfaces

Both glial cells and neurons can be considered basic computational units in neural networks, and the brain–computer interface (BCI) can play a role in awakening the latency portion and being sensitive to positive feedback through learning. However, high-quality information gained from BCI requires invasive approaches such as microelectrodes implanted under the endocranium. As a hard foreign object in the aqueous microenvironment, the soft cerebral cortex’s chronic inflammation state and scar tissue appear subsequently. To avoid the obvious defects caused by hard electrodes, this review focuses on the bioinspired neural interface, guiding and optimizing the implant system for better biocompatibility and accuracy. At the same time, the bionic techniques of signal reception and transmission interfaces are summarized and the structural units with functions similar to nerve cells are introduced. Multiple electrical and electromagnetic transmissions, regulating the secretion of neuromodulators or neurotransmitters via nanofluidic channels, have been flexibly applied. The accurate regulation of neural networks from the nanoscale to the cellular reconstruction of protein pathways will make BCI the extension of the brain.

Brain-Computer Interfaces for Treatment of Focal Dystonia

Task-specificity in isolated focal dystonias is a powerful feature that may successfully be targeted with therapeutic brain-computer interfaces. While performing a symptomatic task, the patient actively modulates momentary brain activity (disorder signature) to match activity during an asymptomatic task (target signature), which is expected to translate into symptom reduction.

Cost function for low-dimensional manifold topology assessment

In reduced-order modeling, complex systems that exhibit high state-space dimensionality are described and evolved using a small number of parameters. These parameters can be obtained in a data-driven way, where a high-dimensional dataset is projected onto a lower-dimensional basis. A complex system is then restricted to states on a low-dimensional manifold where it can be efficiently modeled. While this approach brings computational benefits, obtaining a good quality of the manifold topology becomes a crucial aspect when models, such as nonlinear regression, are built on top of the manifold. Here, we present a quantitative metric for characterizing manifold topologies. Our metric pays attention to non-uniqueness and spatial gradients in physical quantities of interest, and can be applied to manifolds of arbitrary dimensionality. Using the metric as a cost function in optimization algorithms, we show that optimized low-dimensional projections can be found. We delineate a few applications of the cost function to datasets representing argon plasma, reacting flows and atmospheric pollutant dispersion. We demonstrate how the cost function can assess various dimensionality reduction and manifold learning techniques as well as data preprocessing strategies in their capacity to yield quality low-dimensional projections. We show that improved manifold topologies can facilitate building nonlinear regression models.

Discriminating Free Hand Movements Using Support Vector Machine and Recurrent Neural Network Algorithms

Decoding natural hand movements is of interest for human-computer interaction and may constitute a helpful tool in the diagnosis of motor diseases and rehabilitation monitoring. However, the accurate measurement of complex hand movements and the decoding of dynamic movement data remains challenging. Here, we introduce two algorithms, one based on support vector machine (SVM) classification combined with dynamic time warping, and the other based on a long short-term memory (LSTM) neural network, which were designed to discriminate small differences in defined sequences of hand movements. We recorded hand movement data from 17 younger and 17 older adults using an exoskeletal data glove while they were performing six different movement tasks. Accuracy rates in decoding the different movement types were similarly high for SVM and LSTM in across-subject classification, but, for within-subject classification, SVM outperformed LSTM. The SVM-based approach, therefore, appears particularly promising for the development of movement decoding tools, in particular if the goal is to generalize across age groups, for example for detecting specific motor disorders or tracking their progress over time.

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

The spiking activity of populations of cortical neurons is well described by the dynamics of a small number of population-wide covariance patterns, the 'latent dynamics'. These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFP). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable throughout the behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.