High-performance brain-to-text communication via handwriting

Large-scale Foundation Models and Generative AI for BigData Neuroscience

Recent advances in machine learning have led to revolutionary breakthroughs in computer games, image and natural language understanding, and scientific discovery. Foundation models and large-scale language models (LLMs) have recently achieved human-like intelligence thanks to BigData. With the help of self-supervised learning (SSL) and transfer learning, these models may potentially reshape the landscapes of neuroscience research and make a significant impact on the future. Here we present a mini-review on recent advances in foundation models and generative AI models as well as their applications in neuroscience, including natural language and speech, semantic memory, brain-machine interfaces (BMIs), and data augmentation. We argue that this paradigm-shift framework will open new avenues for many neuroscience research directions and discuss the accompanying challenges and opportunities.

Right to mental integrity and neurotechnologies: implications of the extended mind thesis

The possibility of neurotechnological interference with our brain and mind raises questions about the moral rights that would protect against the (mis)use of these technologies. One such moral right that has received recent attention is the right to mental integrity. Though the metaphysical boundaries of the mind are a matter of live debate, most defences of this moral right seem to assume an internalist (brain-based) view of the mind. In this article, we will examine what an extended account of the mind might imply for the right to mental integrity and the protection it provides against neurotechnologies. We argue that, on an extended account of the mind, the scope of the right to mental integrity would expand significantly, implying that neurotechnologies would no longer pose a uniquely serious threat to the right. In addition, some neurotechnologies may even be protected by the right to mental integrity, as the technologies would become part of the mind. We conclude that adopting an extended account of the mind has significant implications for the right to mental integrity in terms of its protective scope and capacity to protect against neurotechnologies, demonstrating that metaphysical assumptions about the mind play an important role in determining the moral protection provided by the right.

An optimized EEGNet decoder for decoding motor image of four class fingers flexion

As a cutting-edge technology of connecting biological brain and external devices, brain-computer interface (BCI) exhibits promising applications on extensive fields such as medical and military. As for the disable individuals with four limbs losing the motor functions, it is a potential treatment way to drive mechanical equipments by the means of non-invasive BCI, which is badly depended on the accuracy of the decoded electroencephalogram (EEG) singles. In this study, an explanatory convolutional neural network namely EEGNet based on SimAM attention module was proposed to enhance the accuracy of decoding the EEG singles of index and thumb fingers for both left and right hand using sensory motor rhythm (SMR). An average classification accuracy of 72.91% the data of eight healthy subjects was obtained, which were captured from the one second before finger movement to two seconds after action. Furthermore, the character of event-related desynchronization (ERD) and event related synchronization (ERS) of index and thumb fingers was also studied in this study. These findings have significant importance for controlling external devices or other rehabilitation equipment using BCI in a fine way.

Transcranial focused ultrasound to V5 enhances human visual motion brain-computer interface by modulating feature-based attention

A brain-computer interface (BCI) enables users to control devices with their minds. Despite advancements, non-invasive BCIs still exhibit high error rates, prompting investigation into the potential reduction through concurrent targeted neuromodulation. Transcranial focused ultrasound (tFUS) is an emerging non-invasive neuromodulation technology with high spatiotemporal precision. This study examines whether tFUS neuromodulation can improve BCI outcomes, and explores the underlying mechanism of action using high-density electroencephalography (EEG) source imaging (ESI). As a result, V5-targeted tFUS significantly reduced the error in a BCI speller task. Source analyses revealed a significantly increase in theta and alpha activities in the tFUS condition at both V5 and downstream in the dorsal visual processing pathway. Correlation analysis indicated that the connection within the dorsal processing pathway was preserved during tFUS stimulation, while the ventral connection was weakened. These findings suggest that V5-targeted tFUS enhances feature-based attention to visual motion.

Comprehensive evaluation methods for translating BCI into practical applications: usability, user satisfaction and usage of online BCI systems

Although brain-computer interface (BCI) is considered a revolutionary advancement in human-computer interaction and has achieved significant progress, a considerable gap remains between the current technological capabilities and their practical applications. To promote the translation of BCI into practical applications, the gold standard for online evaluation for classification algorithms of BCI has been proposed in some studies. However, few studies have proposed a more comprehensive evaluation method for the entire online BCI system, and it has not yet received sufficient attention from the BCI research and development community. Therefore, the qualitative leap from analyzing and modeling for offline BCI data to the construction of online BCI systems and optimizing their performance is elaborated, and then user-centred is emphasized, and then the comprehensive evaluation methods for translating BCI into practical applications are detailed and reviewed in the article, including the evaluation of the usability (including effectiveness and efficiency of systems), the evaluation of the user satisfaction (including BCI-related aspects, etc.), and the evaluation of the usage (including the match between the system and user, etc.) of online BCI systems. Finally, the challenges faced in the evaluation of the usability and user satisfaction of online BCI systems, the efficacy of online BCI systems, and the integration of BCI and artificial intelligence (AI) and/or virtual reality (VR) and other technologies to enhance the intelligence and user experience of the system are discussed. It is expected that the evaluation methods for online BCI systems elaborated in this review will promote the translation of BCI into practical applications.

Less is more: selection from a small set of options improves BCI velocity control

We designed the discrete direction selection (DDS) decoder for intracortical brain computer interface (iBCI) cursor control and showed that it outperformed currently used decoders in a human-operated real-time iBCI simulator and in monkey iBCI use. Unlike virtually all existing decoders that map between neural activity and continuous velocity commands, DDS uses neural activity to select among a small menu of preset cursor velocities. We compared closed-loop cursor control across four visits by each of 48 naïve, able-bodied human subjects using either DDS or one of three common continuous velocity decoders: direct regression with assist (an affine map from neural activity to cursor velocity), ReFIT, and the velocity Kalman Filter. DDS outperformed all three by a substantial margin. Subsequently, a monkey using an iBCI also had substantially better performance with DDS than with the Wiener filter decoder (direct regression decoder that includes time history). Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of simplifying online iBCI control.

The Promise of Endovascular Neurotechnology: A Brain-Computer Interface to Restore Autonomy to People With Motor Impairment

ABSTRACT

This Joel A. DeLisa Lecture on endovascular brain-computer interfaces was presented by Dr Thomas Oxley on February 23, 2023, at the Association of Academic Physiatrists Annual Scientific Meeting. The lecture described how brain-computer interfaces replace lost physiological function to enable direct communication between the brain and external digital devices, such as computers, smartphones, and robotic limbs. Specifically, the potential of a novel endovascular brain-computer interface technology was discussed. The brain-computer interface uses a stent-electrode array delivered via the jugular vein and is permanently implanted in a vein adjacent to the motor cortex. In a first-in-human clinical trial, participants with upper limb paralysis who received the endovascular brain-computer interface could use the system independently and at home to operate laptop computers for various instrumental activities of daily living. A Food and Drug Administration-approved trial of the endovascular brain-computer interface in the United States is in progress. Future development of the system will provide recipients with continuous autonomy through digital access with minimal caregiver assistance. Physiatrists and occupational therapists will have a vital role in helping people with paralysis achieve the potential of implantable brain-computer interfaces.

Efficient in Vivo Neural Signal Compression Using an Autoencoder-Based Neural Network

Conventional in vivo neural signal processing involves extracting spiking activity within the recorded signals from an ensemble of neurons and transmitting only spike counts over an adequate interval. However, for brain-computer interface (BCI) applications utilizing continuous local field potentials (LFPs) for cognitive decoding, the volume of neural data to be transmitted to a computer imposes relatively high data rate requirements. This is particularly true for BCIs employing high-density intracortical recordings with hundreds or thousands of electrodes. This article introduces the first autoencoder-based compression digital circuit for the efficient transmission of LFP neural signals. Various algorithmic and architectural-level optimizations are implemented to significantly reduce the computational complexity and memory requirements of the designed in vivo compression circuit. This circuit employs an autoencoder-based neural network, providing a robust signal reconstruction. The application-specific integrated circuit (ASIC) of the in vivo compression logic occupies the smallest silicon area and consumes the lowest power among the reported state-of-the-art compression ASICs. Additionally, it offers a higher compression rate and a superior signal-to-noise and distortion ratio.

A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages

Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish-English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.

Text and image generation from intracranial electroencephalography using an embedding space for text and images

Objective.Invasive brain-computer interfaces (BCIs) are promising communication devices for severely paralyzed patients. Recent advances in intracranial electroencephalography (iEEG) coupled with natural language processing have enhanced communication speed and accuracy. It should be noted that such a speech BCI uses signals from the motor cortex. However, BCIs based on motor cortical activities may experience signal deterioration in users with motor cortical degenerative diseases such as amyotrophic lateral sclerosis. An alternative approach to using iEEG of the motor cortex is necessary to support patients with such conditions.Approach. In this study, a multimodal embedding of text and images was used to decode visual semantic information from iEEG signals of the visual cortex to generate text and images. We used contrastive language-image pretraining (CLIP) embedding to represent images presented to 17 patients implanted with electrodes in the occipital and temporal cortices. A CLIP image vector was inferred from the high-γpower of the iEEG signals recorded while viewing the images.Main results.Text was generated by CLIPCAP from the inferred CLIP vector with better-than-chance accuracy. Then, an image was created from the generated text using StableDiffusion with significant accuracy.Significance.The text and images generated from iEEG through the CLIP embedding vector can be used for improved communication.

The speech neuroprosthesis

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

Machine learning decoding of single neurons in the thalamus for speech brain-machine interfaces

Objective. Our goal is to decode firing patterns of single neurons in the left ventralis intermediate nucleus (Vim) of the thalamus, related to speech production, perception, and imagery. For realistic speech brain-machine interfaces (BMIs), we aim to characterize the amount of thalamic neurons necessary for high accuracy decoding.Approach. We intraoperatively recorded single neuron activity in the left Vim of eight neurosurgical patients undergoing implantation of deep brain stimulator or RF lesioning during production, perception and imagery of the five monophthongal vowel sounds. We utilized the Spade decoder, a machine learning algorithm that dynamically learns specific features of firing patterns and is based on sparse decomposition of the high dimensional feature space.Main results. Spade outperformed all algorithms compared with, for all three aspects of speech: production, perception and imagery, and obtained accuracies of 100%, 96%, and 92%, respectively (chance level: 20%) based on pooling together neurons across all patients. The accuracy was logarithmic in the amount of neurons for all three aspects of speech. Regardless of the amount of units employed, production gained highest accuracies, whereas perception and imagery equated with each other.Significance. Our research renders single neuron activity in the left Vim a promising source of inputs to BMIs for restoration of speech faculties for locked-in patients or patients with anarthria or dysarthria to allow them to communicate again. Our characterization of how many neurons are necessary to achieve a certain decoding accuracy is of utmost importance for planning BMI implantation.

Closed-loop modulation of remote hippocampal representations with neurofeedback

Humans can remember specific events without acting on them and can influence which memories are retrieved based on internal goals. However, current animal models of memory typically present sensory cues to trigger retrieval and assess retrieval based on action 1-5 . As a result, it is difficult to determine whether measured patterns of neural activity relate to the cue(s), the retrieved memory, or the behavior. We therefore asked whether we could develop a paradigm to isolate retrieval-related neural activity in animals without retrieval cues or the requirement of a behavioral report. To do this, we focused on hippocampal "place cells." These cells primarily emit spiking patterns that represent the animal's current location (local representations), but they can also generate representations of previously visited locations distant from the animal's current location (remote representations) 6-13 . It is not known whether animals can deliberately engage specific remote representations, and if so, whether this engagement would occur during specific brain states. So, we used a closed-loop neurofeedback system to reward expression of remote representations that corresponded to uncued, experimenter-selected locations, and found that rats could increase the prevalence of these specific remote representations over time; thus, demonstrating memory retrieval modulated by internal goals in an animal model. These representations occurred predominately during periods of immobility but outside of hippocampal sharp-wave ripple (SWR) 13-15 events. This paradigm enables future direct studies of memory retrieval mechanisms in the healthy brain and in models of neurological disorders.

Depletion of complement factor 3 delays the neuroinflammatory response to intracortical microelectrodes

The neuroinflammatory response to intracortical microelectrodes (IMEs) used with brain-machine interfacing (BMI) applications is regarded as the primary contributor to poor chronic performance. Recent developments in high-plex gene expression technologies have allowed for an evolution in the investigation of individual proteins or genes to be able to identify specific pathways of upregulated genes that may contribute to the neuroinflammatory response. Several key pathways that are upregulated following IME implantation are involved with the complement system. The complement system is part of the innate immune system involved in recognizing and eliminating pathogens - a significant contributor to the foreign body response against biomaterials. Specifically, we have identified Complement 3 (C3) as a gene of interest because it is the intersection of several key complement pathways. In this study, we investigated the role of C3 in the IME inflammatory response by comparing the neuroinflammatory gene expression at the microelectrode implant site between C3 knockout (C3-/-) and wild-type (WT) mice. We have found that, like in WT mice, implantation of intracortical microelectrodes in C3-/- mice yields a dramatic increase in the neuroinflammatory gene expression at all post-surgery time points investigated. However, compared to WT mice, C3 depletion showed reduced expression of many neuroinflammatory genes pre-surgery and 4 weeks post-surgery. Conversely, depletion of C3 increased the expression of many neuroinflammatory genes at 8 weeks and 16 weeks post-surgery, compared to WT mice. Our results suggest that C3 depletion may be a promising therapeutic target for acute, but not chronic, relief of the neuroinflammatory response to IME implantation. Additional compensatory targets may also be required for comprehensive long-term reduction of the neuroinflammatory response for improved intracortical microelectrode performance.

Artificial intelligence in neurology: opportunities, challenges, and policy implications

Neurological conditions are the leading cause of disability and mortality combined, demanding innovative, scalable, and sustainable solutions. Brain health has become a global priority with adoption of the World Health Organization's Intersectoral Global Action Plan in 2022. Simultaneously, rapid advancements in artificial intelligence (AI) are revolutionizing neurological research and practice. This scoping review of 66 original articles explores the value of AI in neurology and brain health, systematizing the landscape for emergent clinical opportunities and future trends across the care trajectory: prevention, risk stratification, early detection, diagnosis, management, and rehabilitation. AI's potential to advance personalized precision neurology and global brain health directives hinges on resolving core challenges across four pillars-models, data, feasibility/equity, and regulation/innovation-through concerted pursuit of targeted recommendations. Paramount actions include swift, ethical, equity-focused integration of novel technologies into clinical workflows, mitigating data-related issues, counteracting digital inequity gaps, and establishing robust governance frameworks balancing safety and innovation.

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS

Brain-computer interfaces (BCIs) that reconstruct and synthesize speech using brain activity recorded with intracranial electrodes may pave the way toward novel communication interfaces for people who have lost their ability to speak, or who are at high risk of losing this ability, due to neurological disorders. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a man with impaired articulation due to ALS, participating in a clinical trial (ClinicalTrials.gov, NCT03567213) exploring different strategies for BCI communication. The 3-stage approach reported here relies on recurrent neural networks to identify, decode and synthesize speech from electrocorticographic (ECoG) signals acquired across motor, premotor and somatosensory cortices. We demonstrate a reliable BCI that synthesizes commands freely chosen and spoken by the participant from a vocabulary of 6 keywords previously used for decoding commands to control a communication board. Evaluation of the intelligibility of the synthesized speech indicates that 80% of the words can be correctly recognized by human listeners. Our results show that a speech-impaired individual with ALS can use a chronically implanted BCI to reliably produce synthesized words while preserving the participant's voice profile, and provide further evidence for the stability of ECoG for speech-based BCIs.

A flexible intracortical brain-computer interface for typing using finger movements

Keyboard typing with finger movements is a versatile digital interface for users with diverse skills, needs, and preferences. Currently, such an interface does not exist for people with paralysis. We developed an intracortical brain-computer interface (BCI) for typing with attempted flexion/extension movements of three finger groups on the right hand, or both hands, and demonstrated its flexibility in two dominant typing paradigms. The first paradigm is "point-and-click" typing, where a BCI user selects one key at a time using continuous real-time control, allowing selection of arbitrary sequences of symbols. During cued character selection with this paradigm, a human research participant with paralysis achieved 30-40 selections per minute with nearly 90% accuracy. The second paradigm is "keystroke" typing, where the BCI user selects each character by a discrete movement without real-time feedback, often giving a faster speed for natural language sentences. With 90 cued characters per minute, decoding attempted finger movements and correcting errors using a language model resulted in more than 90% accuracy. Notably, both paradigms matched the state-of-the-art for BCI performance and enabled further flexibility by the simultaneous selection of multiple characters as well as efficient decoder estimation across paradigms. Overall, the high-performance interface is a step towards the wider accessibility of BCI technology by addressing unmet user needs for flexibility.

Nonlinear manifolds underlie neural population activity during behaviour

There is rich variety in the activity of single neurons recorded during behaviour. Yet, these diverse single neuron responses can be well described by relatively few patterns of neural co-modulation. The study of such low-dimensional structure of neural population activity has provided important insights into how the brain generates behaviour. Virtually all of these studies have used linear dimensionality reduction techniques to estimate these population-wide co-modulation patterns, constraining them to a flat "neural manifold". Here, we hypothesised that since neurons have nonlinear responses and make thousands of distributed and recurrent connections that likely amplify such nonlinearities, neural manifolds should be intrinsically nonlinear. Combining neural population recordings from monkey, mouse, and human motor cortex, and mouse striatum, we show that: 1) neural manifolds are intrinsically nonlinear; 2) their nonlinearity becomes more evident during complex tasks that require more varied activity patterns; and 3) manifold nonlinearity varies across architecturally distinct brain regions. Simulations using recurrent neural network models confirmed the proposed relationship between circuit connectivity and manifold nonlinearity, including the differences across architecturally distinct regions. Thus, neural manifolds underlying the generation of behaviour are inherently nonlinear, and properly accounting for such nonlinearities will be critical as neuroscientists move towards studying numerous brain regions involved in increasingly complex and naturalistic behaviours.

The digital motor output: a conceptual framework for a meaningful clinical performance metric for a motor neuroprosthesis

In recent years, the majority of the population has become increasingly reliant on continuous and independent control of smart devices to conduct activities of daily living. Upper extremity movement is typically required to generate the motor outputs that control these interfaces, such as rapidly and accurately navigating and clicking a mouse, or activating a touch screen. For people living with tetraplegia, these abilities are lost, significantly compromising their ability to interact with their environment. Implantable brain computer interfaces (BCIs) hold promise for restoring lost neurologic function, including motor neuroprostheses (MNPs). An implantable MNP can directly infer motor intent by detecting brain signals and transmitting the motor signal out of the brain to generate a motor output and subsequently control computer actions. This physiological function is typically performed by the motor neurons in the human body. To evaluate the use of these implanted technologies, there is a need for an objective measurement of the effectiveness of MNPs in restoring motor outputs. Here, we propose the concept of digital motor outputs (DMOs) to address this: a motor output decoded directly from a neural recording during an attempted limb or orofacial movement is transformed into a command that controls an electronic device. Digital motor outputs are diverse and can be categorized as discrete or continuous representations of motor control, and the clinical utility of the control of a single, discrete DMO has been reported in multiple studies. This sets the stage for the DMO to emerge as a quantitative measure of MNP performance.

Visual tracking brain-computer interface

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's information transfer rate (ITR) of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI based-control method to go beyond discrete commands, allowing natural continuous control based on neural activity.

Recruiting neural field theory for data augmentation in a motor imagery brain-computer interface

We introduce a novel approach to training data augmentation in brain-computer interfaces (BCIs) using neural field theory (NFT) applied to EEG data from motor imagery tasks. BCIs often suffer from limited accuracy due to a limited amount of training data. To address this, we leveraged a corticothalamic NFT model to generate artificial EEG time series as supplemental training data. We employed the BCI competition IV '2a' dataset to evaluate this augmentation technique. For each individual, we fitted the model to common spatial patterns of each motor imagery class, jittered the fitted parameters, and generated time series for data augmentation. Our method led to significant accuracy improvements of over 2% in classifying the "total power" feature, but not in the case of the "Higuchi fractal dimension" feature. This suggests that the fit NFT model may more favorably represent one feature than the other. These findings pave the way for further exploration of NFT-based data augmentation, highlighting the benefits of biophysically accurate artificial data.

BRAND: a platform for closed-loop experiments with deep network models

Objective.Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g. Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g. C and C++).Approach.To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termednodes, which communicate with each other in agraphvia streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis, an in-memory database, to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes.Main results.In our tests, BRAND achieved <600 microsecond latency between processes when sending large quantities of data (1024 channels of 30 kHz neural data in 1 ms chunks). BRAND runs a brain-computer interface with a recurrent neural network (RNN) decoder with less than 8 ms of latency from neural data input to decoder prediction. In a real-world demonstration of the system, participant T11 in the BrainGate2 clinical trial (ClinicalTrials.gov Identifier: NCT00912041) performed a standard cursor control task, in which 30 kHz signal processing, RNN decoding, task control, and graphics were all executed in BRAND. This system also supports real-time inference with complex latent variable models like Latent Factor Analysis via Dynamical Systems.Significance.By providing a framework that is fast, modular, and language-agnostic, BRAND lowers the barriers to integrating the latest tools in neuroscience and machine learning into closed-loop experiments.

An accurate and rapidly calibrating speech neuroprosthesis

Brain-computer interfaces can enable rapid, intuitive communication for people with paralysis by transforming the cortical activity associated with attempted speech into text on a computer screen. Despite recent advances, communication with brain-computer interfaces has been restricted by extensive training data requirements and inaccurate word output. A man in his 40's with ALS with tetraparesis and severe dysarthria (ALSFRS-R = 23) was enrolled into the BrainGate2 clinical trial. He underwent surgical implantation of four microelectrode arrays into his left precentral gyrus, which recorded neural activity from 256 intracortical electrodes. We report a speech neuroprosthesis that decoded his neural activity as he attempted to speak in both prompted and unstructured conversational settings. Decoded words were displayed on a screen, then vocalized using text-to-speech software designed to sound like his pre-ALS voice. On the first day of system use, following 30 minutes of attempted speech training data, the neuroprosthesis achieved 99.6% accuracy with a 50-word vocabulary. On the second day, the size of the possible output vocabulary increased to 125,000 words, and, after 1.4 additional hours of training data, the neuroprosthesis achieved 90.2% accuracy. With further training data, the neuroprosthesis sustained 97.5% accuracy beyond eight months after surgical implantation. The participant has used the neuroprosthesis to communicate in self-paced conversations for over 248 hours. In an individual with ALS and severe dysarthria, an intracortical speech neuroprosthesis reached a level of performance suitable to restore naturalistic communication after a brief training period.

Applying the IEEE BRAIN neuroethics framework to intra-cortical brain-computer interfaces

Objective. Brain-computer interfaces (BCIs) are neuroprosthetic devices that allow for direct interaction between brains and machines. These types of neurotechnologies have recently experienced a strong drive in research and development, given, in part, that they promise to restore motor and communication abilities in individuals experiencing severe paralysis. While a rich literature analyzes the ethical, legal, and sociocultural implications (ELSCI) of these novel neurotechnologies, engineers, clinicians and BCI practitioners often do not have enough exposure to these topics.Approach. Here, we present the IEEE Neuroethics Framework, an international, multiyear, iterative initiative aimed at developing a robust, accessible set of considerations for diverse stakeholders.Main results. Using the framework, we provide practical examples of ELSCI considerations for BCI neurotechnologies. We focus on invasive technologies, and in particular, devices that are implanted intra-cortically for medical research applications.Significance. We demonstrate the utility of our framework in exposing a wide range of implications across different intra-cortical BCI technology modalities and conclude with recommendations on how to utilize this knowledge in the development and application of ethical guidelines for BCI neurotechnologies.

Estimating and approaching the maximum information rate of noninvasive visual brain-computer interface

An essential priority of visual brain-computer interfaces (BCIs) is to enhance the information transfer rate (ITR) to achieve high-speed communication. Despite notable progress, noninvasive visual BCIs have encountered a plateau in ITRs, leaving it uncertain whether higher ITRs are achievable. In this study, we used information theory to study the characteristics and capacity of the visual-evoked channel, which leads us to investigate whether and how we can decode higher information rates in a visual BCI system. Using information theory, we estimate the upper and lower bounds of the information rate with the white noise (WN) stimulus. Consequently, we found out that the information rate is determined by the signal-to-noise ratio (SNR) in the frequency domain, which reflects the spectrum resources of the channel. Based on this discovery, we propose a broadband WN BCI by implementing stimuli on a broader frequency band than the steady-state visual evoked potentials (SSVEPs)-based BCI. Through validation, the broadband BCI outperforms the SSVEP BCI by an impressive 7 bps, setting a record of 50 bps. The integration of information theory and the decoding analysis presented in this study offers valuable insights applicable to general sensory-evoked BCIs, providing a potential direction of next-generation human-machine interaction systems.

Preparatory activity and the expansive null-space

The study of the cortical control of movement experienced a conceptual shift over recent decades, as the basic currency of understanding shifted from single-neuron tuning towards population-level factors and their dynamics. This transition was informed by a maturing understanding of recurrent networks, where mechanism is often characterized in terms of population-level factors. By estimating factors from data, experimenters could test network-inspired hypotheses. Central to such hypotheses are 'output-null' factors that do not directly drive motor outputs yet are essential to the overall computation. In this Review, we highlight how the hypothesis of output-null factors was motivated by the venerable observation that motor-cortex neurons are active during movement preparation, well before movement begins. We discuss how output-null factors then became similarly central to understanding neural activity during movement. We discuss how this conceptual framework provided key analysis tools, making it possible for experimenters to address long-standing questions regarding motor control. We highlight an intriguing trend: as experimental and theoretical discoveries accumulate, the range of computational roles hypothesized to be subserved by output-null factors continues to expand.

The evolution of neuromodulation for chronic stroke: From neuroplasticity mechanisms to brain-computer interfaces

Stroke is one of the most common and debilitating neurological conditions worldwide. Those who survive experience motor, sensory, speech, vision, and/or cognitive deficits that severely limit remaining quality of life. While rehabilitation programs can help improve patients' symptoms, recovery is often limited, and patients frequently continue to experience impairments in functional status. In this review, invasive neuromodulation techniques to augment the effects of conventional rehabilitation methods are described, including vagus nerve stimulation (VNS), deep brain stimulation (DBS) and brain-computer interfaces (BCIs). In addition, the evidence base for each of these techniques, pivotal trials, and future directions are explored. Finally, emerging technologies such as functional near-infrared spectroscopy (fNIRS) and the shift to artificial intelligence-enabled implants and wearables are examined. While the field of implantable devices for chronic stroke recovery is still in a nascent stage, the data reviewed are suggestive of immense potential for reducing the impact and impairment from this globally prevalent disorder.

Several inaccurate or erroneous conceptions and misleading propaganda about brain-computer interfaces

Brain-computer interface (BCI) is a revolutionizing human-computer interaction, which has potential applications for specific individuals or groups in specific scenarios. Extensive research has been conducted on the principles and implementation methods of BCI, and efforts are currently being made to bridge the gap from research to real-world applications. However, there are inaccurate or erroneous conceptions about BCI among some members of the public, and certain media outlets, as well as some BCI researchers, developers, manufacturers, and regulators, propagate misleading or overhyped claims about BCI technology. Therefore, this article summarizes the several misconceptions and misleading propaganda about BCI, including BCI being capable of "mind-controlled," "controlling brain," "mind reading," and the ability to "download" or "upload" information from or to the brain using BCI, among others. Finally, the limitations (shortcomings) and limits (boundaries) of BCI, as well as the necessity of conducting research aimed at countering BCI systems are discussed, and several suggestions are offered to reduce misconceptions and misleading claims about BCI.

PMF-CNN: parallel multi-band fusion convolutional neural network for SSVEP-EEG decoding

Steady-state visual evoked potential (SSVEP) is a key technique of electroencephalography (EEG)-based brain-computer interfaces (BCI), which has been widely applied to neurological function assessment and postoperative rehabilitation. However, accurate decoding of the user's intended based on the SSVEP-EEG signals is challenging due to the low signal-to-noise ratio and large individual variability of the signals. To address these issues, we proposed a parallel multi-band fusion convolutional neural network (PMF-CNN). Multi frequency band signals were served as the input of PMF-CNN to fully utilize the time-frequency information of EEG. Three parallel modules, spatial self-attention (SAM), temporal self-attention (TAM), and squeeze-excitation (SEM), were proposed to automatically extract multi-dimensional features from spatial, temporal, and frequency domains, respectively. A novel spatial-temporal-frequency representation were designed to capture the correlation of electrode channels, time intervals, and different sub-harmonics by using SAM, TAM, and SEM, respectively. The three parallel modules operate independently and simultaneously. A four layers CNN classification module was designed to fuse parallel multi-dimensional features and achieve the accurate classification of SSVEP-EEG signals. The PMF-CNN was further interpreted by using brain functional connectivity analysis. The proposed method was validated using two large publicly available datasets. After trained using our proposed dual-stage training pattern, the classification accuracies were 99.37% and 93.96%, respectively, which are superior to the current state-of-the-art SSVEP-EEG classification algorithms. The algorithm exhibits high classification accuracy and good robustness, which has the potential to be applied to postoperative rehabilitation.

Artificial neural network for brain-machine interface consistently produces more naturalistic finger movements than linear methods

Brain-machine interfaces (BMI) aim to restore function to persons living with spinal cord injuries by 'decoding' neural signals into behavior. Recently, nonlinear BMI decoders have outperformed previous state-of-the-art linear decoders, but few studies have investigated what specific improvements these nonlinear approaches provide. In this study, we compare how temporally convolved feedforward neural networks (tcFNNs) and linear approaches predict individuated finger movements in open and closed-loop settings. We show that nonlinear decoders generate more naturalistic movements, producing distributions of velocities 85.3% closer to true hand control than linear decoders. Addressing concerns that neural networks may come to inconsistent solutions, we find that regularization techniques improve the consistency of tcFNN convergence by 194.6%, along with improving average performance, and training speed. Finally, we show that tcFNN can leverage training data from multiple task variations to improve generalization. The results of this study show that nonlinear methods produce more naturalistic movements and show potential for generalizing over less constrained tasks.

Teaser

A neural network decoder produces consistent naturalistic movements and shows potential for real-world generalization through task variations.

End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging

Deep learning is a powerful tool for neural decoding, broadly applied to systems neuroscience and clinical studies. Interpretable and transparent models that can explain neural decoding for intended behaviors are crucial to identifying essential features of deep learning decoders in brain activity. In this study, we examine the performance of deep learning to classify mouse behavioral states from mesoscopic cortex-wide calcium imaging data. Our convolutional neural network (CNN)-based end-to-end decoder combined with recurrent neural network (RNN) classifies the behavioral states with high accuracy and robustness to individual differences on temporal scales of sub-seconds. Using the CNN-RNN decoder, we identify that the forelimb and hindlimb areas in the somatosensory cortex significantly contribute to behavioral classification. Our findings imply that the end-to-end approach has the potential to be an interpretable deep learning method with unbiased visualization of critical brain regions.

Intracortical brain-computer interfaces for improved motor function: a systematic review

In this systematic review, we address the status of intracortical brain-computer interfaces (iBCIs) applied to the motor cortex to improve function in patients with impaired motor ability. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 Guidelines for Systematic Reviews. Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) and the Effective Public Health Practice Project (EPHPP) were used to assess bias and quality. Advances in iBCIs in the last two decades demonstrated the use of iBCI to activate limbs for functional tasks, achieve neural typing for communication, and other applications. However, the inconsistency of performance metrics employed by these studies suggests the need for standardization. Each study was a pilot clinical trial consisting of 1-4, majority male (64.28 %) participants, with most trials featuring participants treated for more than 12 months (55.55 %). The systems treated patients with various conditions: amyotrophic lateral sclerosis, stroke, spinocerebellar degeneration without cerebellar involvement, and spinal cord injury. All participants presented with tetraplegia at implantation and were implanted with microelectrode arrays via pneumatic insertion, with nearly all electrode locations solely at the precentral gyrus of the motor cortex (88.88 %). The development of iBCI devices using neural signals from the motor cortex to improve motor-impaired patients has enhanced the ability of these systems to return ability to their users. However, many milestones remain before these devices can prove their feasibility for recovery. This review summarizes the achievements and shortfalls of these systems and their respective trials.

[Ethical considerations for medical applications of implantable brain-computer interfaces]

Implantable brain-computer interfaces (BCIs) have potentially important clinical applications due to the high spatial resolution and signal-to-noise ratio of electrodes that are closer to or implanted in the cerebral cortex. However, the surgery and electrodes of implantable BCIs carry safety risks of brain tissue damage, and their medical applications face ethical challenges, with little literature to date systematically considering ethical norms for the medical applications of implantable BCIs. In order to promote the clinical translation of this type of BCI, we considered the ethics of practice for the medical application of implantable BCIs, including: reducing the risk of brain tissue damage from implantable BCI surgery and electrodes, providing patients with customized and personalized implantable BCI treatments, ensuring multidisciplinary collaboration in the clinical application of implantable BCIs, and the responsible use of implantable BCIs, among others. It is expected that this article will provide thoughts and references for the research and development of ethics of the medical application of implantable BCI.

In Vivo Characterization of Intracortical Probes with Focused Ion Beam-Etched Nanopatterned Topographies

(1) Background: Intracortical microelectrodes (IMEs) are an important part of interfacing with the central nervous system (CNS) and recording neural signals. However, recording electrodes have shown a characteristic steady decline in recording performance owing to chronic neuroinflammation. The topography of implanted devices has been explored to mimic the nanoscale three-dimensional architecture of the extracellular matrix. Our previous work used histology to study the implant sites of non-recording probes and showed that a nanoscale topography at the probe surface mitigated the neuroinflammatory response compared to probes with smooth surfaces. Here, we hypothesized that the improvement in the neuroinflammatory response for probes with nanoscale surface topography would extend to improved recording performance. (2) Methods: A novel design modification was implemented on planar silicon-based neural probes by etching nanopatterned grooves (with a 500 nm pitch) into the probe shank. To assess the hypothesis, two groups of rats were implanted with either nanopatterned (n = 6) or smooth control (n = 6) probes, and their recording performance was evaluated over 4 weeks. Postmortem gene expression analysis was performed to compare the neuroinflammatory response from the two groups. (3) Results: Nanopatterned probes demonstrated an increased impedance and noise floor compared to controls. However, the recording performances of the nanopatterned and smooth probes were similar, with active electrode yields for control probes and nanopatterned probes being approximately 50% and 45%, respectively, by 4 weeks post-implantation. Gene expression analysis showed one gene, Sirt1, differentially expressed out of 152 in the panel. (4) Conclusions: this study provides a foundation for investigating novel nanoscale topographies on neural probes.

Ethical considerations for integrating multimodal computer perception and neurotechnology

Background

Artificial intelligence (AI)-based computer perception technologies (e.g., digital phenotyping and affective computing) promise to transform clinical approaches to personalized care in psychiatry and beyond by offering more objective measures of emotional states and behavior, enabling precision treatment, diagnosis, and symptom monitoring. At the same time, passive and continuous nature by which they often collect data from patients in non-clinical settings raises ethical issues related to privacy and self-determination. Little is known about how such concerns may be exacerbated by the integration of neural data, as parallel advances in computer perception, AI, and neurotechnology enable new insights into subjective states. Here, we present findings from a multi-site NCATS-funded study of ethical considerations for translating computer perception into clinical care and contextualize them within the neuroethics and neurorights literatures.

Methods

We conducted qualitative interviews with patients (n = 20), caregivers (n = 20), clinicians (n = 12), developers (n = 12), and clinician developers (n = 2) regarding their perspective toward using PC in clinical care. Transcripts were analyzed in MAXQDA using Thematic Content Analysis.

Results

Stakeholder groups voiced concerns related to (1) perceived invasiveness of passive and continuous data collection in private settings; (2) data protection and security and the potential for negative downstream/future impacts on patients of unintended disclosure; and (3) ethical issues related to patients' limited versus hyper awareness of passive and continuous data collection and monitoring. Clinicians and developers highlighted that these concerns may be exacerbated by the integration of neural data with other computer perception data.

Discussion

Our findings suggest that the integration of neurotechnologies with existing computer perception technologies raises novel concerns around dignity-related and other harms (e.g., stigma, discrimination) that stem from data security threats and the growing potential for reidentification of sensitive data. Further, our findings suggest that patients' awareness and preoccupation with feeling monitored via computer sensors ranges from hypo- to hyper-awareness, with either extreme accompanied by ethical concerns (consent vs. anxiety and preoccupation). These results highlight the need for systematic research into how best to implement these technologies into clinical care in ways that reduce disruption, maximize patient benefits, and mitigate long-term risks associated with the passive collection of sensitive emotional, behavioral and neural data.

NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

A real-time, high-performance brain-computer interface for finger decoding and quadcopter control

People with paralysis express unmet needs for peer support, leisure activities, and sporting activities. Many within the general population rely on social media and massively multiplayer video games to address these needs. We developed a high-performance finger brain-computer-interface system allowing continuous control of 3 independent finger groups with 2D thumb movements. The system was tested in a human research participant over sequential trials requiring fingers to reach and hold on targets, with an average acquisition rate of 76 targets/minute and completion time of 1.58 ± 0.06 seconds. Performance compared favorably to previous animal studies, despite a 2-fold increase in the decoded degrees-of-freedom (DOF). Finger positions were then used for 4-DOF velocity control of a virtual quadcopter, demonstrating functionality over both fixed and random obstacle courses. This approach shows promise for controlling multiple-DOF end-effectors, such as robotic fingers or digital interfaces for work, entertainment, and socialization.

A design and implementation of multi-character classification scheme based on motor imagery EEG signals

In the field of brain-to-text communication, it is difficult to finish highly dexterous behaviors of writing multi-character by motor-imagery-based brain-computer interface (MI-BCI), setting a barrier to restore communication in people who have lost the ability to move and speak. In this paper, we design and implement a multi-character classification scheme based on 29 characters of motor imagery (MI) electroencephalogram (EEG) signals, which contains 26 English letters and 3 punctuation marks. Firstly, we design a novel experimental paradigm to increase the variety of BCI inputs by asking subjects to imagine the movement of writing 29 characters instead of gross motor skills such as reaching or grasping. Secondly, because of the high dimension of EEG signals, we adopt power spectral density (PSD), principal components analysis (PCA), kernel principal components analysis (KPCA) respectively to decompose EEG signals and extract feature, and then test the results with pearson product-moment correlation coefficient (PCCs). Thirdly, we respectively employ k-nearest neighbor (kNN), support vector machine (SVM), extreme learning machine (ELM) and light gradient boosting machine (LightGBM) to classify 29 characters and compare the results. We have implemented a complete scheme, including paradigm design, signal acquisition, feature extraction and classification, which can effectively classify 29 characters. The experimental results show that the KPCA has the best feature extraction effect and the kNN has the highest classification accuracy, with the final classification accuracy reaching 96.2%, which is better than other studies.

Transforming modeling in neurorehabilitation: clinical insights for personalized rehabilitation

Practicing clinicians in neurorehabilitation continue to lack a systematic evidence base to personalize rehabilitation therapies to individual patients and thereby maximize outcomes. Computational modeling- collecting, analyzing, and modeling neurorehabilitation data- holds great promise. A key question is how can computational modeling contribute to the evidence base for personalized rehabilitation? As representatives of the clinicians and clinician-scientists who attended the 2023 NSF DARE conference at USC, here we offer our perspectives and discussion on this topic. Our overarching thesis is that clinical insight should inform all steps of modeling, from construction to output, in neurorehabilitation and that this process requires close collaboration between researchers and the clinical community. We start with two clinical case examples focused on motor rehabilitation after stroke which provide context to the heterogeneity of neurologic injury, the complexity of post-acute neurologic care, the neuroscience of recovery, and the current state of outcome assessment in rehabilitation clinical care. Do we provide different therapies to these two different patients to maximize outcomes? Asking this question leads to a corollary: how do we build the evidence base to support the use of different therapies for individual patients? We discuss seven points critical to clinical translation of computational modeling research in neurorehabilitation- (i) clinical endpoints, (ii) hypothesis- versus data-driven models, (iii) biological processes, (iv) contextualizing outcome measures, (v) clinical collaboration for device translation, (vi) modeling in the real world and (vii) clinical touchpoints across all stages of research. We conclude with our views on key avenues for future investment (clinical-research collaboration, new educational pathways, interdisciplinary engagement) to enable maximal translational value of computational modeling research in neurorehabilitation.

Endovascular Brain-Computer Interfaces in Poststroke Paralysis

Stroke is a leading cause of paralysis, most frequently affecting the upper limbs and vocal folds. Despite recent advances in care, stroke recovery invariably reaches a plateau, after which there are permanent neurological impairments. Implantable brain-computer interface devices offer the potential to bypass permanent neurological lesions. They function by (1) recording neural activity, (2) decoding the neural signal occurring in response to volitional motor intentions, and (3) generating digital control signals that may be used to control external devices. While brain-computer interface technology has the potential to revolutionize neurological care, clinical translation has been limited. Endovascular arrays present a novel form of minimally invasive brain-computer interface devices that have been deployed in human subjects during early feasibility studies. This article provides an overview of endovascular brain-computer interface devices and critically evaluates the patient with stroke as an implant candidate. Future opportunities are mapped, along with the challenges arising when decoding neural activity following infarction. Limitations arise when considering intracerebral hemorrhage and motor cortex lesions; however, future directions are outlined that aim to address these challenges.

Noise-Factorized Disentangled Representation Learning for Generalizable Motor Imagery EEG Classification

Motor Imagery (MI) Electroencephalography (EEG) is one of the most common Brain-Computer Interface (BCI) paradigms that has been widely used in neural rehabilitation and gaming. Although considerable research efforts have been dedicated to developing MI EEG classification algorithms, they are mostly limited in handling scenarios where the training and testing data are not from the same subject or session. Such poor generalization capability significantly limits the realization of BCI in real-world applications. In this paper, we proposed a novel framework to disentangle the representation of raw EEG data into three components, subject/session-specific, MI-task-specific, and random noises, so that the subject/session-specific feature extends the generalization capability of the system. This is realized by a joint discriminative and generative framework, supported by a series of fundamental training losses and training strategies. We evaluated our framework on three public MI EEG datasets, and detailed experimental results show that our method can achieve superior performance by a large margin compared to current state-of-the-art benchmark algorithms.

Brain-computer interface digital prescription for neurological disorders

Neurological and psychiatric diseases can lead to motor, language, emotional disorder, and cognitive, hearing or visual impairment By decoding the intention of the brain in real time, the Brain-computer interface (BCI) can first assist in the diagnosis of diseases, and can also compensate for its damaged function by directly interacting with the environment; In addition, provide output signals in various forms, such as actual motion, tactile or visual feedback, to assist in rehabilitation training; Further intervention in brain disorders is achieved by close-looped neural modulation. In this article, we envision the future BCI digital prescription system for patients with different functional disorders and discuss the key contents in the prescription the brain signals, coding and decoding protocols and interaction paradigms, and assistive technology. Then, we discuss the details that need to be specially included in the digital prescription for different intervention technologies. The third part summarizes previous examples of intervention, focusing on how to select appropriate interaction paradigms for patients with different functional impairments. For the last part, we discussed the indicators and influencing factors in evaluating the therapeutic effect of BCI as intervention.

Time-varying generalized linear models: characterizing and decoding neuronal dynamics in higher visual areas

To create a behaviorally relevant representation of the visual world, neurons in higher visual areas exhibit dynamic response changes to account for the time-varying interactions between external (e.g., visual input) and internal (e.g., reward value) factors. The resulting high-dimensional representational space poses challenges for precisely quantifying individual factors' contributions to the representation and readout of sensory information during a behavior. The widely used point process generalized linear model (GLM) approach provides a powerful framework for a quantitative description of neuronal processing as a function of various sensory and non-sensory inputs (encoding) as well as linking particular response components to particular behaviors (decoding), at the level of single trials and individual neurons. However, most existing variations of GLMs assume the neural systems to be time-invariant, making them inadequate for modeling nonstationary characteristics of neuronal sensitivity in higher visual areas. In this review, we summarize some of the existing GLM variations, with a focus on time-varying extensions. We highlight their applications to understanding neural representations in higher visual areas and decoding transient neuronal sensitivity as well as linking physiology to behavior through manipulation of model components. This time-varying class of statistical models provide valuable insights into the neural basis of various visual behaviors in higher visual areas and hold significant potential for uncovering the fundamental computational principles that govern neuronal processing underlying various behaviors in different regions of the brain.

Free-Standing Carbon Nanotube Embroidered Graphene Film Electrode Array for Stable Neural Interfacing

Implantable neural probes that are mechanically flexible yet robust are attractive candidates for achieving stable neural interfacing in the brain. Current flexible neural probes consist mainly of metal thin-film electrodes integrated on micrometer-thick polymer substrates, making it challenging to achieve electrode-tissue interfacing on the cellular scale. Here, we describe implantable neural probes that consist of robust carbon nanotube network embroidered graphene (CeG) films as free-standing recording microelectrodes. Our CeG film microelectrode arrays (CeG_MEAs) are ultraflexible yet mechanically robust, thus enabling cellular-scale electrode-tissue interfacing. Chronically implanted CeG_MEAs can stably track the activities of the same population of neurons over two months. Our results highlight the potential of ultraflexible and free-standing carbon nanofilms for stable neural interfacing in the brain.

Decoding Single and Paired Phonemes Using 7T Functional MRI

Several studies have shown that mouth movements related to the pronunciation of individual phonemes are represented in the sensorimotor cortex. This would theoretically allow for brain computer interfaces that are capable of decoding continuous speech by training classifiers based on the activity in the sensorimotor cortex related to the production of individual phonemes. To address this, we investigated the decodability of trials with individual and paired phonemes (pronounced consecutively with one second interval) using activity in the sensorimotor cortex. Fifteen participants pronounced 3 different phonemes and 3 combinations of two of the same phonemes in a 7T functional MRI experiment. We confirmed that support vector machine (SVM) classification of single and paired phonemes was possible. Importantly, by combining classifiers trained on single phonemes, we were able to classify paired phonemes with an accuracy of 53% (33% chance level), demonstrating that activity of isolated phonemes is present and distinguishable in combined phonemes. A SVM searchlight analysis showed that the phoneme representations are widely distributed in the ventral sensorimotor cortex. These findings provide insights about the neural representations of single and paired phonemes. Furthermore, it supports the notion that speech BCI may be feasible based on machine learning algorithms trained on individual phonemes using intracranial electrode grids.

Brain control of bimanual movement enabled by recurrent neural networks

Brain-computer interfaces have so far focused largely on enabling the control of a single effector, for example a single computer cursor or robotic arm. Restoring multi-effector motion could unlock greater functionality for people with paralysis (e.g., bimanual movement). However, it may prove challenging to decode the simultaneous motion of multiple effectors, as we recently found that a compositional neural code links movements across all limbs and that neural tuning changes nonlinearly during dual-effector motion. Here, we demonstrate the feasibility of high-quality bimanual control of two cursors via neural network (NN) decoders. Through simulations, we show that NNs leverage a neural 'laterality' dimension to distinguish between left and right-hand movements as neural tuning to both hands become increasingly correlated. In training recurrent neural networks (RNNs) for two-cursor control, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously. Our results suggest that neural network decoders may be advantageous for multi-effector decoding, provided they are designed to transfer to the online setting.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.