The Liquid Metal Age: A Transition From Hg to Ga

This review offers an illuminating journey through the historical evolution and modern-day applications of liquid metals, presenting a comprehensive view of their significance in diverse fields. Tracing the trajectory from mercury applications to contemporary innovations, the paper explores their pivotal role in industry and research. The analysis spans electrical switches, mechanical applications, electrodes, chemical synthesis, energy storage, thermal transport, electronics, and biomedicine. Each section examines the intricacies of liquid metal integration, elucidating their contributions to technological advancements and societal progress. Moreover, the review critically appraises the challenges and prospects inherent in liquid metal applications, addressing issues of recycling, corrosion management, device stability, economic feasibility, translational hurdles, and market dynamics. By delving into these complexities, the paper advances scholarly understanding and offers actionable insights for researchers, engineers, and policymakers. It aims to catalyze innovation, foster interdisciplinary collaboration, and promote liquid metal-enabled solutions for societal needs. Through its comprehensive analysis and forward-looking perspective, this review serves as a guide for navigating the landscape of liquid metal applications, bridging historical legacies with contemporary challenges, and highlighting the transformative potential of liquid metals in shaping future technologies.

Development and evaluation of a non-invasive brain-spine interface using transcutaneous spinal cord stimulation

Motor rehabilitation is a therapeutic process to facilitate functional recovery in people with spinal cord injury (SCI). However, its efficacy is limited to areas with remaining sensorimotor function. Spinal cord stimulation (SCS) creates a temporary prosthetic effect that may allow further rehabilitation-induced recovery in individuals without remaining sensorimotor function, thereby extending the therapeutic reach of motor rehabilitation to individuals with more severe injuries. In this work, we report our first steps in developing a non-invasive brain-spine interface (BSI) based on electroencephalography (EEG) and transcutaneous spinal cord stimulation (tSCS). The objective of this study was to identify EEG-based neural correlates of lower limb movement in the sensorimotor cortex of unimpaired individuals and to quantify the performance of a linear discriminant analysis (LDA) decoder in detecting movement onset from these neural correlates. Our results show that initiation of knee extension was associated with event-related desynchronization in the central-medial cortical regions at frequency bands between 4-44 Hz. Our neural decoder using µ (8-12 Hz), low β (16-20 Hz), and high β (24-28 Hz) frequency bands achieved an average area under the curve (AUC) of 0.83 ± 0.06 s.d. (n = 7) during a cued movement task offline. Generalization to imagery and uncued movement tasks served as positive controls to verify robustness against movement artifacts and cue-related confounds, respectively. With the addition of real-time decoder-modulated tSCS, the neural decoder performed with an average AUC of 0.81 ± 0.05 s.d. (n = 9) on cued movement and 0.68 ± 0.12 s.d. (n = 9) on uncued movement. Our results suggest that the decrease in decoder performance in uncued movement may be due to differences in underlying cortical strategies between conditions. Furthermore, we explore alternative applications of the BSI system by testing neural decoders trained on uncued movement and imagery tasks. By developing a non-invasive BSI, tSCS can be timed to be delivered only during voluntary effort, which may have implications for improving rehabilitation.

Sustainability inspired fabrication of next generation neurostimulation and cardiac rhythm management electrodes via reactive hierarchical surface restructuring

Over the last two decades, platinum group metals (PGMs) and their alloys have dominated as the materials of choice for electrodes in long-term implantable neurostimulation and cardiac rhythm management devices due to their superior conductivity, mechanical and chemical stability, biocompatibility, corrosion resistance, radiopacity, and electrochemical performance. Despite these benefits, PGM manufacturing processes are extremely costly, complex, and challenging with potential health hazards. Additionally, the volatility in PGM prices and their high supply risk, combined with their scarce concentration of approximately 0.01 ppm in the earth's upper crust and limited mining geographical areas, underscores their classification as critical raw materials, thus, their effective recovery or substitution worldwide is of paramount importance. Since postmortem recovery from deceased patients and/or refining of PGMs that are used in the manufacturing of the electrodes and microelectrode arrays is extremely rare, challenging, and highly costly, therefore, substitution of PGM-based electrodes with other biocompatible materials that can yield electrochemical performance values equal or greater than PGMs is the only viable and sustainable solution to reduce and ultimately substitute the use of PGMs in long-term implantable neurostimulation and cardiac rhythm management devices. In this article, we demonstrate for the first time how the novel technique of "reactive hierarchical surface restructuring" can be utilized on titanium-that is widely used in many non-stimulation medical device and implant applications-to manufacture biocompatible, low-cost, sustainable, and high-performing neurostimulation and cardiac rhythm management electrodes. We have shown how the surface of titanium electrodes with extremely poor electrochemical performance undergoes compositional and topographical transformations that result in electrodes with outstanding electrochemical performance.

A Polymer Thick Film on an Organic Substrate Grid Electrode and an Open-Source Recording System for UHF MRI: An Imaging Study

Electrocorticography (ECoG) is a critical tool in preclinical neuroscience research for studying global network activity. However, integrating ECoG with functional magnetic resonance imaging (fMRI) has posed challenges, due to metal electrode interference with imaging quality and heating around the metallic electrodes. Here, we introduce recent advancements in ECoG grid development that utilize a polymer-thick film on an organic substrate (PTFOS). PTFOS offers notable advantages over traditional ECoG grids. Firstly, it significantly reduces imaging artifacts, ensuring minimal interference with MR image quality when overlaying brain tissue with PTFOS grids. Secondly, during a 30-min fMRI acquisition, the temperature increase associated with PTFOS grids is remarkably low, measuring only 0.4 °C. These findings suggest that utilizing ECoG with PTFOS grids has the potential to enhance the safety and efficacy of neurosurgical procedures. By providing clearer imaging results and mitigating risk factors such as excessive heating during MRI scans, PTFOS-based ECoG grids represent a promising advancement in neurosurgical technology. Furthermore, we describe a cutting-edge open-source system designed for simultaneous electrophysiology and fMRI. This system stands out due to its exceptionally low input noise levels (<0.6 V peak-to-peak), robust electromagnetic compatibility (it is suitable for use in MRI environments up to 9.4 teslas), and the inclusion of user-programmable real-time signal-processing capabilities. The open-platform software is a key feature, enabling researchers to swiftly implement and customize real-time signal-processing algorithms to meet specific experimental needs. This innovative system has been successfully utilized in several rodent EEG/fMRI studies, particularly at magnetic field strengths of 4.7 and 9.4 teslas, focusing on the somatosensory system. These studies have allowed for detailed observation of neural activity and responses within this sensory system, providing insights that are critical for advancing our understanding of neurophysiological processes. The versatility and high performance of our system make it an invaluable tool for researchers aiming to integrate and analyze complex datasets from advanced imaging and electrophysiological recordings, ultimately enhancing the depth and scope of neuroscience research.

Recording of single-unit activities with flexible micro-electrocorticographic array in rats for decoding of whole-body navigation

Objective.Micro-electrocorticographic (μECoG) arrays are able to record neural activities from the cortical surface, without the need to penetrate the brain parenchyma. Owing in part to small electrode sizes, previous studies have demonstrated that single-unit spikes could be detected from the cortical surface, and likely from Layer I neurons of the neocortex. Here we tested the ability to useμECoG arrays to decode, in rats, body position during open field navigation, through isolated single-unit activities.Approach. μECoG arrays were chronically implanted onto primary motor cortex (M1) of Wistar rats, and neural recording was performed in awake, behaving rats in an open-field enclosure. The signals were band-pass filtered between 300-3000 Hz. Threshold-crossing spikes were identified and sorted into distinct units based on defined criteria including waveform morphology and refractory period. Body positions were derived from video recordings. We used gradient-boosting machine to predict body position based on previous 100 ms of spike data, and correlation analyses to elucidate the relationship between position and spike patterns.Main results.Single-unit spikes could be extracted during chronic recording fromμECoG, and spatial position could be decoded from these spikes with a mean absolute error of prediction of 0.135 and 0.090 in the x- and y- dimensions (of a normalized range from 0 to 1), and Pearson's r of 0.607 and 0.571, respectively.Significance. μECoG can detect single-unit activities that likely arise from superficial neurons in the cortex and is a promising alternative to intracortical arrays, with the added benefit of scalability to cover large cortical surface with minimal incremental risks. More studies should be performed in human related to its use as brain-machine interface.

Stability of ECoG high gamma signals during speech and implications for a speech BCI system in an individual with ALS: a year-long longitudinal study

Objective.Speech brain-computer interfaces (BCIs) have the potential to augment communication in individuals with impaired speech due to muscle weakness, for example in amyotrophic lateral sclerosis (ALS) and other neurological disorders. However, to achieve long-term, reliable use of a speech BCI, it is essential for speech-related neural signal changes to be stable over long periods of time. Here we study, for the first time, the stability of speech-related electrocorticographic (ECoG) signals recorded from a chronically implanted ECoG BCI over a 12 month period.Approach.ECoG signals were recorded by an ECoG array implanted over the ventral sensorimotor cortex in a clinical trial participant with ALS. Because ECoG-based speech decoding has most often relied on broadband high gamma (HG) signal changes relative to baseline (non-speech) conditions, we studied longitudinal changes of HG band power at baseline and during speech, and we compared these with residual high frequency noise levels at baseline. Stability was further assessed by longitudinal measurements of signal-to-noise ratio, activation ratio, and peak speech-related HG response magnitude (HG response peaks). Lastly, we analyzed the stability of the event-related HG power changes (HG responses) for individual syllables at each electrode.Main Results.We found that speech-related ECoG signal responses were stable over a range of syllables activating different articulators for the first year after implantation.Significance.Together, our results indicate that ECoG can be a stable recording modality for long-term speech BCI systems for those living with severe paralysis.Clinical Trial Information.ClinicalTrials.gov, registration number NCT03567213.

Non-Invasive Brain Sensing Technologies for Modulation of Neurological Disorders

The non-invasive brain sensing modulation technology field is experiencing rapid development, with new techniques constantly emerging. This study delves into the field of non-invasive brain neuromodulation, a safer and potentially effective approach for treating a spectrum of neurological and psychiatric disorders. Unlike traditional deep brain stimulation (DBS) surgery, non-invasive techniques employ ultrasound, electrical currents, and electromagnetic field stimulation to stimulate the brain from outside the skull, thereby eliminating surgery risks and enhancing patient comfort. This study explores the mechanisms of various modalities, including transcranial direct current stimulation (tDCS) and transcranial magnetic stimulation (TMS), highlighting their potential to address chronic pain, anxiety, Parkinson's disease, and depression. We also probe into the concept of closed-loop neuromodulation, which personalizes stimulation based on real-time brain activity. While we acknowledge the limitations of current technologies, our study concludes by proposing future research avenues to advance this rapidly evolving field with its immense potential to revolutionize neurological and psychiatric care and lay the foundation for the continuing advancement of innovative non-invasive brain sensing technologies.

Injectable Fluorescent Neural Interfaces for Cell-Specific Stimulating and Imaging

Building on current explorations in chronic optical neural interfaces, it is essential to address the risk of photothermal damage in traditional optogenetics. By focusing on calcium fluorescence for imaging rather than stimulation, injectable fluorescent neural interfaces significantly minimize photothermal damage and improve the accuracy of neuronal imaging. Key advancements including the use of injectable microelectronics for targeted electrical stimulation and their integration with cell-specific genetically encoded calcium indicators have been discussed. These injectable electronics that allow for post-treatment retrieval offer a minimally invasive solution, enhancing both usability and reliability. Furthermore, the integration of genetically encoded fluorescent calcium indicators with injectable bioelectronics enables precise neuronal recording and imaging of individual neurons. This shift not only minimizes risks such as photothermal conversion but also boosts safety, specificity, and effectiveness of neural imaging. Embracing these advancements represents a significant leap forward in biomedical engineering and neuroscience, paving the way for advanced brain-machine interfaces.

The role of distinct ECoG frequency features in decoding finger movement

Objective.To identify the electrocorticography (ECoG) frequency features that encode distinct finger movement states during repeated finger flexions.Approach.We used the publicly available Stanford ECoG dataset of cue-based, repeated single finger flexions. Using linear regression, we identified the spectral features that contributed most to the encoding of movement dynamics and discriminating movement events from rest, and combined them to predict finger movement trajectories. Furthermore, we also looked into the effect of the used frequency range and the spatial distribution of the identified features.Main results.Two frequency features generate superior performance, each one for a different movement aspect: high gamma band activity distinguishes movement events from rest, whereas the local motor potential (LMP) codes for movement dynamics. Combining these two features in a finger movement decoder outperformed comparable prior work where the entire spectrum was used as the average correlation coefficient with the true trajectories increased from 0.45 to 0.5, both applied to the Stanford dataset, and erroneous predictions during rest were demoted. In addition, for the first time, our results show the influence of the upper cut-off frequency used to extract LMP, yielding a higher performance when this range is adjusted to the finger movement rate.Significance.This study shows the benefit of a detailed feature analysis prior to designing the finger movement decoder.

Advancing the interfacing performances of chronically implantable neural probes in the era of CMOS neuroelectronics

Tissue penetrating microelectrode neural probes can record electrophysiological brain signals at resolutions down to single neurons, making them invaluable tools for neuroscience research and Brain-Computer-Interfaces (BCIs). The known gradual decrease of their electrical interfacing performances in chronic settings, however, remains a major challenge. A key factor leading to such decay is Foreign Body Reaction (FBR), which is the cascade of biological responses that occurs in the brain in the presence of a tissue damaging artificial device. Interestingly, the recent adoption of Complementary Metal Oxide Semiconductor (CMOS) technology to realize implantable neural probes capable of monitoring hundreds to thousands of neurons simultaneously, may open new opportunities to face the FBR challenge. Indeed, this shift from passive Micro Electro-Mechanical Systems (MEMS) to active CMOS neural probe technologies creates important, yet unexplored, opportunities to tune probe features such as the mechanical properties of the probe, its layout, size, and surface physicochemical properties, to minimize tissue damage and consequently FBR. Here, we will first review relevant literature on FBR to provide a better understanding of the processes and sources underlying this tissue response. Methods to assess FBR will be described, including conventional approaches based on the imaging of biomarkers, and more recent transcriptomics technologies. Then, we will consider emerging opportunities offered by the features of CMOS probes. Finally, we will describe a prototypical neural probe that may meet the needs for advancing clinical BCIs, and we propose axial insertion force as a potential metric to assess the influence of probe features on acute tissue damage and to control the implantation procedure to minimize iatrogenic injury and subsequent FBR.

Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation

Emotion recognition is crucial in understanding human affective states with various applications. Electroencephalography (EEG)-a non-invasive neuroimaging technique that captures brain activity-has gained attention in emotion recognition. However, existing EEG-based emotion recognition systems are limited to specific sensory modalities, hindering their applicability. Our study innovates EEG emotion recognition, offering a comprehensive framework for overcoming sensory-focused limits and cross-sensory challenges. We collected cross-sensory emotion EEG data using multimodal emotion simulations (three sensory modalities: audio/visual/audio-visual with two emotion states: pleasure or unpleasure). The proposed framework-filter bank adversarial domain adaptation Riemann method (FBADR)-leverages filter bank techniques and Riemannian tangent space methods for feature extraction from cross-sensory EEG data. Compared with Riemannian methods, filter bank and adversarial domain adaptation could improve average accuracy by 13.68% and 8.36%, respectively. Comparative analysis of classification results proved that the proposed FBADR framework achieved a state-of-the-art cross-sensory emotion recognition performance and reached an average accuracy of 89.01% ± 5.06%. Moreover, the robustness of the proposed methods could ensure high cross-sensory recognition performance under a signal-to-noise ratio (SNR) ≥ 1 dB. Overall, our study contributes to the EEG-based emotion recognition field by providing a comprehensive framework that overcomes limitations of sensory-oriented approaches and successfully tackles the difficulties of cross-sensory situations.

Using principles of motor control to analyze performance of human machine interfaces

There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of an end-effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.

Human-Centric Digital Twins in Industry: A Comprehensive Review of Enabling Technologies and Implementation Strategies

The last decade saw the emergence of highly autonomous, flexible, re-configurable Cyber-Physical Systems. Research in this domain has been enhanced by the use of high-fidelity simulations, including Digital Twins, which are virtual representations connected to real assets. Digital Twins have been used for process supervision, prediction, or interaction with physical assets. Interaction with Digital Twins is enhanced by Virtual Reality and Augmented Reality, and Industry 5.0-focused research is evolving with the involvement of the human aspect in Digital Twins. This paper aims to review recent research on Human-Centric Digital Twins (HCDTs) and their enabling technologies. A systematic literature review is performed using the VOSviewer keyword mapping technique. Current technologies such as motion sensors, biological sensors, computational intelligence, simulation, and visualization tools are studied for the development of HCDTs in promising application areas. Domain-specific frameworks and guidelines are formed for different HCDT applications that highlight the workflow and desired outcomes, such as the training of AI models, the optimization of ergonomics, the security policy, task allocation, etc. A guideline and comparative analysis for the effective development of HCDTs are created based on the criteria of Machine Learning requirements, sensors, interfaces, and Human Digital Twin inputs.

The nested hierarchy of overt, mouthed, and imagined speech activity evident in intracranial recordings

Recent studies have demonstrated that it is possible to decode and synthesize various aspects of acoustic speech directly from intracranial measurements of electrophysiological brain activity. In order to continue progressing toward the development of a practical speech neuroprosthesis for the individuals with speech impairments, better understanding and modeling of imagined speech processes are required. The present study uses intracranial brain recordings from participants that performed a speaking task with trials consisting of overt, mouthed, and imagined speech modes, representing various degrees of decreasing behavioral output. Speech activity detection models are constructed using spatial, spectral, and temporal brain activity features, and the features and model performances are characterized and compared across the three degrees of behavioral output. The results indicate the existence of a hierarchy in which the relevant channels for the lower behavioral output modes form nested subsets of the relevant channels from the higher behavioral output modes. This provides important insights for the elusive goal of developing more effective imagined speech decoding models with respect to the better-established overt speech decoding counterparts.

An integrated perspective for the diagnosis and therapy of neurodevelopmental disorders - From an engineering point of view

Neurodevelopmental disorders (NDDs) are complex conditions with largely unknown pathophysiology. While many NDD symptoms are familiar, the cause of these disorders remains unclear and may involve a combination of genetic, biological, psychosocial, and environmental risk factors. Current diagnosis relies heavily on behaviorally defined criteria, which may be biased by the clinical team's professional and cultural expectations, thus a push for new biological-based biomarkers for NDDs diagnosis is underway. Emerging new research technologies offer an unprecedented view into the electrical, chemical, and physiological activity in the brain and with further development in humans may provide clinically relevant diagnoses. These could also be extended to new treatment options, which can start to address the underlying physiological issues. When combined with current speech, language, occupational therapy, and pharmacological treatment these could greatly improve patient outcomes. The current review will discuss the latest technologies that are being used or may be used for NDDs diagnosis and treatment. The aim is to provide an inspiring and forward-looking view for future research in the field.

Identifying Thematics in a Brain-Computer Interface Research

This umbrella review is motivated to understand the shift in research themes on brain-computer interfacing (BCI) and it determined that a shift away from themes that focus on medical advancement and system development to applications that included education, marketing, gaming, safety, and security has occurred. The background of this review examined aspects of BCI categorisation, neuroimaging methods, brain control signal classification, applications, and ethics. The specific area of BCI software and hardware development was not examined. A search using One Search was undertaken and 92 BCI reviews were selected for inclusion. Publication demographics indicate the average number of authors on review papers considered was 4.2 ± 1.8. The results also indicate a rapid increase in the number of BCI reviews from 2003, with only three reviews before that period, two in 1972, and one in 1996. While BCI authors were predominantly Euro-American in early reviews, this shifted to a more global authorship, which China dominated by 2020-2022. The review revealed six disciplines associated with BCI systems: life sciences and biomedicine (n = 42), neurosciences and neurology (n = 35), and rehabilitation (n = 20); (2) the second domain centred on the theme of functionality: computer science (n = 20), engineering (n = 28) and technology (n = 38). There was a thematic shift from understanding brain function and modes of interfacing BCI systems to more applied research novel areas of research-identified surround artificial intelligence, including machine learning, pre-processing, and deep learning. As BCI systems become more invasive in the lives of "normal" individuals, it is expected that there will be a refocus and thematic shift towards increased research into ethical issues and the need for legal oversight in BCI application.

Quo vadis artificial intelligence?

The study of artificial intelligence (AI) has been a continuous endeavor of scientists and engineers for over 65 years. The simple contention is that human-created machines can do more than just labor-intensive work; they can develop human-like intelligence. Being aware or not, AI has penetrated into our daily lives, playing novel roles in industry, healthcare, transportation, education, and many more areas that are close to the general public. AI is believed to be one of the major drives to change socio-economical lives. In another aspect, AI contributes to the advancement of state-of-the-art technologies in many fields of study, as helpful tools for groundbreaking research. However, the prosperity of AI as we witness today was not established smoothly. During the past decades, AI has struggled through historical stages with several winters. Therefore, at this juncture, to enlighten future development, it is time to discuss the past, present, and have an outlook on AI. In this article, we will discuss from a historical perspective how challenges were faced on the path of revolution of both the AI tools and the AI systems. Especially, in addition to the technical development of AI in the short to mid-term, thoughts and insights are also presented regarding the symbiotic relationship of AI and humans in the long run.

Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network

Objective. Speech decoding, one of the most intriguing brain-computer interface applications, opens up plentiful opportunities from rehabilitation of patients to direct and seamless communication between human species. Typical solutions rely on invasive recordings with a large number of distributed electrodes implanted through craniotomy. Here we explored the possibility of creating speech prosthesis in a minimally invasive setting with a small number of spatially segregated intracranial electrodes.Approach. We collected one hour of data (from two sessions) in two patients implanted with invasive electrodes. We then used only the contacts that pertained to a single stereotactic electroencephalographic (sEEG) shaft or an electrocorticographic (ECoG) stripe to decode neural activity into 26 words and one silence class. We employed a compact convolutional network-based architecture whose spatial and temporal filter weights allow for a physiologically plausible interpretation.Mainresults. We achieved on average 55% accuracy using only six channels of data recorded with a single minimally invasive sEEG electrode in the first patient and 70% accuracy using only eight channels of data recorded for a single ECoG strip in the second patient in classifying 26+1 overtly pronounced words. Our compact architecture did not require the use of pre-engineered features, learned fast and resulted in a stable, interpretable and physiologically meaningful decision rule successfully operating over a contiguous dataset collected during a different time interval than that used for training. Spatial characteristics of the pivotal neuronal populations corroborate with active and passive speech mapping results and exhibit the inverse space-frequency relationship characteristic of neural activity. Compared to other architectures our compact solution performed on par or better than those recently featured in neural speech decoding literature.Significance. We showcase the possibility of building a speech prosthesis with a small number of electrodes and based on a compact feature engineering free decoder derived from a small amount of training data.

Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery

Objective.Among the existing active brain-computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy.Approach.This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition.Main results.The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects' scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm.Significance.The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.

Artifact propagation in subdural cortical electrostimulation: Characterization and modeling

Cortical stimulation via electrocorticography (ECoG) may be an effective method for inducing artificial sensation in bi-directional brain-computer interfaces (BD-BCIs). However, strong electrical artifacts caused by electrostimulation may significantly degrade or obscure neural information. A detailed understanding of stimulation artifact propagation through relevant tissues may improve existing artifact suppression techniques or inspire the development of novel artifact mitigation strategies. Our work thus seeks to comprehensively characterize and model the propagation of artifacts in subdural ECoG stimulation. To this end, we collected and analyzed data from eloquent cortex mapping procedures of four subjects with epilepsy who were implanted with subdural ECoG electrodes. From this data, we observed that artifacts exhibited phase-locking and ratcheting characteristics in the time domain across all subjects. In the frequency domain, stimulation caused broadband power increases, as well as power bursts at the fundamental stimulation frequency and its super-harmonics. The spatial distribution of artifacts followed the potential distribution of an electric dipole with a median goodness-of-fit of R² = 0.80 across all subjects and stimulation channels. Artifacts as large as ±1,100 μV appeared anywhere from 4.43 to 38.34 mm from the stimulation channel. These temporal, spectral and spatial characteristics can be utilized to improve existing artifact suppression techniques, inspire new strategies for artifact mitigation, and aid in the development of novel cortical stimulation protocols. Taken together, these findings deepen our understanding of cortical electrostimulation and provide critical design specifications for future BD-BCI systems.

Deep brain-machine interfaces: sensing and modulating the human deep brain

Different from conventional brain-machine interfaces that focus more on decoding the cerebral cortex, deep brain-machine interfaces enable interactions between external machines and deep brain structures. They sense and modulate deep brain neural activities, aiming at function restoration, device control and therapeutic improvements. In this article, we provide an overview of multiple deep brain recording and stimulation techniques that can serve as deep brain-machine interfaces. We highlight two widely used interface technologies, namely deep brain stimulation and stereotactic electroencephalography, for technical trends, clinical applications and brain connectivity research. We discuss the potential to develop closed-loop deep brain-machine interfaces and achieve more effective and applicable systems for the treatment of neurological and psychiatric disorders.

ABOT: an open-source online benchmarking tool for machine learning-based artefact detection and removal methods from neuronal signals

Brain signals are recorded using different techniques to aid an accurate understanding of brain function and to treat its disorders. Untargeted internal and external sources contaminate the acquired signals during the recording process. Often termed as artefacts, these contaminations cause serious hindrances in decoding the recorded signals; hence, they must be removed to facilitate unbiased decision-making for a given investigation. Due to the complex and elusive manifestation of artefacts in neuronal signals, computational techniques serve as powerful tools for their detection and removal. Machine learning (ML) based methods have been successfully applied in this task. Due to ML's popularity, many articles are published every year, making it challenging to find, compare and select the most appropriate method for a given experiment. To this end, this paper presents ABOT (Artefact removal Benchmarking Online Tool) as an online benchmarking tool which allows users to compare existing ML-driven artefact detection and removal methods from the literature. The characteristics and related information about the existing methods have been compiled as a knowledgebase (KB) and presented through a user-friendly interface with interactive plots and tables for users to search it using several criteria. Key characteristics extracted from over 120 articles from the literature have been used in the KB to help compare the specific ML models. To comply with the FAIR (Findable, Accessible, Interoperable and Reusable) principle, the source code and documentation of the toolbox have been made available via an open-access repository.

Flexural bending to approximate cortical forces exerted by electrocorticography (ECoG) arrays

Objective.The force that an electrocorticography (ECoG) array exerts on the brain manifests when it bends to match the curvature of the skull and cerebral cortex. This force can negatively impact both short-term and long-term patient outcomes. Here we provide a mechanical characterization of a novel liquid crystal polymer (LCP) ECoG array prototype to demonstrate that its thinner geometry reduces the force potentially applied to the cortex of the brain.Approach.We built a low-force flexural testing machine to measure ECoG array bending forces, calculate their effective flexural moduli, and approximate the maximum force they could exerted on the human brain.Main results.The LCP ECoG prototype was found to have a maximal force less than 20% that of any commercially available ECoG arrays that were tested. However, as a material, LCP was measured to be as much as 24× more rigid than silicone, which is traditionally used in ECoG arrays. This suggests that the lower maximal force resulted from the prototype's thinner profile (2.9×-3.25×).Significance.While decreasing material stiffness can lower the force an ECoG array exhibits, our LCP ECoG array prototype demonstrated that flexible circuit manufacturing techniques can also lower these forces by decreasing ECoG array thickness. Flexural tests of ECoG arrays are necessary to accurately assess these forces, as material properties for polymers and laminates are often scale dependent. As the polymers used are anisotropic, elastic modulus cannot be used to predict ECoG flexural behavior. Accounting for these factors, we used our four-point flexure testing procedure to quantify the forces exerted on the brain by ECoG array bending. With this experimental method, ECoG arrays can be designed to minimize force exerted on the brain, potentially improving both acute and chronic clinical utility.

Femtosecond laser hierarchical surface restructuring for next generation neural interfacing electrodes and microelectrode arrays

Long-term implantable neural interfacing devices are able to diagnose, monitor, and treat many cardiac, neurological, retinal and hearing disorders through nerve stimulation, as well as sensing and recording electrical signals to and from neural tissue. To improve specificity, functionality, and performance of these devices, the electrodes and microelectrode arrays-that are the basis of most emerging devices-must be further miniaturized and must possess exceptional electrochemical performance and charge exchange characteristics with neural tissue. In this report, we show for the first time that the electrochemical performance of femtosecond-laser hierarchically-restructured electrodes can be tuned to yield unprecedented performance values that significantly exceed those reported in the literature, e.g. charge storage capacity and specific capacitance were shown to have improved by two orders of magnitude and over 700-fold, respectively, compared to un-restructured electrodes. Additionally, correlation amongst laser parameters, electrochemical performance and surface parameters of the electrodes was established, and while performance metrics exhibit a relatively consistent increasing behavior with laser parameters, surface parameters tend to follow a less predictable trend negating a direct relationship between these surface parameters and performance. To answer the question of what drives such performance and tunability, and whether the widely adopted reasoning of increased surface area and roughening of the electrodes are the key contributors to the observed increase in performance, cross-sectional analysis of the electrodes using focused ion beam shows, for the first time, the existence of subsurface features that may have contributed to the observed electrochemical performance enhancements. This report is the first time that such performance enhancement and tunability are reported for femtosecond-laser hierarchically-restructured electrodes for neural interfacing applications.

Channel selection from source localization: A review of four EEG-based brain-computer interfaces paradigms

Channel selection is a critical part of the classification procedure for multichannel electroencephalogram (EEG)-based brain-computer interfaces (BCI). An optimized subset of electrodes reduces computational complexity and optimizes accuracy. Different tasks activate different sources in the brain and are characterized by distinctive channels. The goal of the current review is to define a subset of electrodes for each of four popular BCI paradigms: motor imagery, motor execution, steady-state visual evoked potentials and P300. Twenty-one studies have been reviewed to identify the most significant activations of cortical sources. The relevant EEG sensors are determined from the reported 3D Talairach coordinates. They are scored by their weighted mean Cohen's d and its confidence interval, providing the magnitude of the corresponding effect size and its statistical significance. Our goal is to create a knowledge-based channel selection framework with a sufficient statistical power. The core channel selection (CCS) could be used as a reference by EEG researchers and would have the advantages of practicality and rapidity, allowing for an easy implementation of semiparametric algorithms.

Contributions of Stereotactic EEG Electrodes in Grey and White Matter to Speech Activity Detection

Recent studies have shown it is possible to decode and synthesize speech directly using brain activity recorded from implanted electrodes. While this activity has been extensively examined using electrocorticographic (ECoG) recordings from cortical surface grey matter, stereotactic electroen-cephalography (sEEG) provides comparatively broader coverage and access to deeper brain structures including both grey and white matter. The present study examines the relative and joint contributions of grey and white matter electrodes for speech activity detection in a brain-computer interface.

Brain Implantable End-Fire Antenna with Enhanced Gain and Bandwidth

An end-fire radiating implantable antenna with a small footprint and broadband operation at the frequency range of 3–5 GHz is proposed for high-data-rate wireless communication in a brain–machine interface. The proposed Vivaldi antenna was implanted vertically along the height of the skull to avoid deformation in the radiation pattern and to compensate for a gain–loss caused by surrounding lossy brain tissues. It was shown that the vertically implanted end-fire antenna had a 3 dB higher antenna gain than a horizontally implanted broadside radiating antenna discussed in recent literature. Additionally, comb-shaped slot arrays imprinted on the Vivaldi antenna lowered the resonant frequency by approximately 2 GHz and improved the antenna gain by more than 2 dB compared to an ordinary Vivaldi antenna. An antenna prototype was fabricated and then tested for verification inside a seven-layered semi-solid brain phantom where each layer had similar electromagnetic material properties as actual brain tissues. The measured data showed that the antenna radiated toward the end-fire direction with an average gain of −15.7 dBi under the frequency of interest, 3–5 GHz. A link budget analysis shows that reliable wireless communication can be achieved over a distance of 10.8 cm despite the electromagnetically harsh environment.

Decoding Continuous Kinetic Information of Grasp from Stereo-electroencephalographic (SEEG) Recordings

OBJECTIVE

Brain-computer interfaces (BCIs) have the potential to bypass damaged neural pathways and restore functionality lost due to injury or disease. Approaches to decoding kinematic information are well documented; however, the decoding of kinetic information has received less attention. Additionally, the possibility of using stereo-electroencephalography (SEEG) for kinetic decoding during hand grasping tasks is still largely unknown. Thus, the objective of this paper is to demonstrate kinetic parameter decoding using SEEG in patients performing a grasping task with two different force levels under two different ascending rates.

APPROACH

Temporal-spectral representations were studied to investigate frequency modulation under different force tasks. Then, force amplitude was decoded from SEEG recordings using multiple decoders, including a linear model, a partial least squares (PLS) model, an unscented Kalman filter, and three deep learning models (shallow convolutional neural network, deep convolutional neural network and the proposed CNN+RNN neural network).

MAIN RESULTS

The current study showed that: 1) for some channel, both low-frequency modulation (event-related desynchronization (ERD)) and high-frequency modulation (event-related synchronization (ERS)) were sustained during prolonged force holding periods; 2) continuously changing grasp force can be decoded from the SEEG signals; 3) the novel CNN+RNN deep learning model achieved the best decoding performance, with the predicted force magnitude closely aligned to the ground truth under different force amplitudes and changing rates.

SIGNIFICANCE

This work verified the possibility of decoding continuously changing grasp force using SEEG recordings. The result presented in this study demonstrated the potential of SEEG recordings for future BCI application.

Spontaneous State Detection Using Time-Frequency and Time-Domain Features Extracted From Stereo-Electroencephalography Traces

As a minimally invasive recording technique, stereo-electroencephalography (SEEG) measures intracranial signals directly by inserting depth electrodes shafts into the human brain, and thus can capture neural activities in both cortical layers and subcortical structures. Despite gradually increasing SEEG-based brain-computer interface (BCI) studies, the features utilized were usually confined to the amplitude of the event-related potential (ERP) or band power, and the decoding capabilities of other time-frequency and time-domain features have not been demonstrated for SEEG recordings yet. In this study, we aimed to verify the validity of time-domain and time-frequency features of SEEG, where classification performances served as evaluating indicators. To do this, using SEEG signals under intermittent auditory stimuli, we extracted features including the average amplitude, root mean square, slope of linear regression, and line-length from the ERP trace and three traces of band power activities (high-gamma, beta, and alpha). These features were used to detect the active state (including activations to two types of names) against the idle state. Results suggested that valid time-domain and time-frequency features distributed across multiple regions, including the temporal lobe, parietal lobe, and deeper structures such as the insula. Among all feature types, the average amplitude, root mean square, and line-length extracted from high-gamma (60–140 Hz) power and the line-length extracted from ERP were the most informative. Using a hidden Markov model (HMM), we could precisely detect the onset and the end of the active state with a sensitivity of 95.7 ± 1.3% and a precision of 91.7 ± 1.6%. The valid features derived from high-gamma power and ERP in this work provided new insights into the feature selection procedure for further SEEG-based BCI applications.

Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming

The usability of EEG-based visual brain–computer interfaces (BCIs) based on event-related potentials (ERPs) benefits from reducing the calibration time before BCI operation. Linear decoding models, such as the spatiotemporal beamformer model, yield state-of-the-art accuracy. Although the training time of this model is generally low, it can require a substantial amount of training data to reach functional performance. Hence, BCI calibration sessions should be sufficiently long to provide enough training data. This work introduces two regularized estimators for the beamformer weights. The first estimator uses cross-validated L2-regularization. The second estimator exploits prior information about the structure of the EEG by assuming Kronecker–Toeplitz-structured covariance. The performances of these estimators are validated and compared with the original spatiotemporal beamformer and a Riemannian-geometry-based decoder using a BCI dataset with P300-paradigm recordings for 21 subjects. Our results show that the introduced estimators are well-conditioned in the presence of limited training data and improve ERP classification accuracy for unseen data. Additionally, we show that structured regularization results in lower training times and memory usage, and a more interpretable classification model.

An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic

OBJECTIVE

The article aims at addressing 2 challenges to step motor BCI out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration.

APPROACH

Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based Recursive Exponentially Weighted Markov-Switching multi-Linear Model (REW-MSLM) decoder is proposed. REW-MSLM uses a Mixture of Expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a "gating" model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action.

MAIN RESULTS

Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of 6 months (without decoder recalibration) 8-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated.

SIGNIFICANCE

Based on the long-term (>36 months) chronic bilateral epidural ECoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behaviour (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.

Investigating Data Cleaning Methods to Improve Performance of Brain-Computer Interfaces Based on Stereo-Electroencephalography

Stereo-electroencephalography (SEEG) utilizes localized and penetrating depth electrodes to directly measure electrophysiological brain activity. The implanted electrodes generally provide a sparse sampling of multiple brain regions, including both cortical and subcortical structures, making the SEEG neural recordings a potential source for the brain–computer interface (BCI) purpose in recent years. For SEEG signals, data cleaning is an essential preprocessing step in removing excessive noises for further analysis. However, little is known about what kinds of effect that different data cleaning methods may exert on BCI decoding performance and, moreover, what are the reasons causing the differentiated effects. To address these questions, we adopted five different data cleaning methods, including common average reference, gray–white matter reference, electrode shaft reference, bipolar reference, and Laplacian reference, to process the SEEG data and evaluated the effect of these methods on improving BCI decoding performance. Additionally, we also comparatively investigated the changes of SEEG signals induced by these different methods from multiple-domain (e.g., spatial, spectral, and temporal domain). The results showed that data cleaning methods could improve the accuracy of gesture decoding, where the Laplacian reference produced the best performance. Further analysis revealed that the superiority of the data cleaning method with excellent performance might be attributed to the increased distinguishability in the low-frequency band. The findings of this work highlighted the importance of applying proper data clean methods for SEEG signals and proposed the application of Laplacian reference for SEEG-based BCI.

Multiple states in ongoing neural activity in the rat visual cortex

The brain continuously produces internal activity in the absence of afferently salient sensory input. Spontaneous neural activity is intrinsically defined by circuit structures and associated with the mode of information processing and behavioral responses. However, the spatiotemporal dynamics of spontaneous activity in the visual cortices of behaving animals remain almost elusive. Using a custom-made electrode array, we recorded 32-site electrocorticograms in the primary and secondary visual cortex of freely behaving rats and determined the propagation patterns of spontaneous neural activity. Nonlinear dimensionality reduction and unsupervised clustering revealed multiple discrete states of the activity patterns. The activity remained stable in one state and suddenly jumped to another state. The diversity and dynamics of the internally switching cortical states would imply flexibility of neural responses to various external inputs.

Thin-film microfabrication and intraoperative testing ofµECoG and iEEG depth arrays for sense and stimulation

Objective.Intracranial neural recordings and electrical stimulation are tools used in an increasing range of applications, including intraoperative clinical mapping and monitoring, therapeutic neuromodulation, and brain computer interface control and feedback. However, many of these applications suffer from a lack of spatial specificity and localization, both in terms of sensed neural signal and applied stimulation. This stems from limited manufacturing processes of commercial-off-the-shelf (COTS) arrays unable to accommodate increased channel density, higher channel count, and smaller contact size.Approach.Here, we describe a manufacturing and assembly approach using thin-film microfabrication for 32-channel high density subdural micro-electrocorticography (µECoG) surface arrays (contacts 1.2 mm diameter, 2 mm pitch) and intracranial electroencephalography (iEEG) depth arrays (contacts 0.5 mm × 1.5 mm, pitch 0.8 mm × 2.5 mm). Crucially, we tackle the translational hurdle and test these arrays during intraoperative studies conducted in four humans under regulatory approval.Main results.We demonstrate that the higher-density contacts provide additional unique information across the recording span compared to the density of COTS arrays which typically have electrode pitch of 8 mm or greater; 4 mm in case of specially ordered arrays. Our intracranial stimulation study results reveal that refined spatial targeting of stimulation elicits evoked potentials with differing spatial spread.Significance.Thin-film,μECoG and iEEG depth arrays offer a promising substrate for advancing a number of clinical and research applications reliant on high-resolution neural sensing and intracranial stimulation.

An integrated deep learning model for motor intention recognition of multi-class EEG Signals in upper limb amputees

BACKGROUND AND OBJECTIVE

Recognition of motor intention based on electroencephalogram (EEG) signals has attracted considerable research interest in the field of pattern recognition due to its notable application of non-muscular communication and control for those with severe motor disabilities. In analysis of EEG data, achieving a higher classification performance is dependent on the appropriate representation of EEG features which is mostly characterized by one unique frequency before applying a learning model. Neglecting other frequencies of EEG signals could deteriorate the recognition performance of the model because each frequency has its unique advantages. Motivated by this idea, we propose to obtain distinguishable features with different frequencies by introducing an integrated deep learning model to accurately classify multiple classes of upper limb movement intentions.

METHODS

The proposed model is a combination of long short-term memory (LSTM) and stacked autoencoder (SAE). To validate the method, four high-level amputees were recruited to perform five motor intention tasks. The acquired EEG signals were first preprocessed before exploring the consequence of input representation on the performance of LSTM-SAE by feeding four frequency bands related to the tasks into the model. The learning model was further improved by t-distributed stochastic neighbor embedding (t-SNE) to eliminate feature redundancy, and to enhance the motor intention recognition.

RESULTS

The experimental results of the classification performance showed that the proposed model achieves an average performance of 99.01% for accuracy, 99.10% for precision, 99.09% for recall, 99.09% for f1_score, 99.77% for specificity, and 99.0% for Cohen's kappa, across multi-subject and multi-class scenarios. Further evaluation with 2-dimensional t-SNE revealed that the signal decomposition has a distinct multi-class separability in the feature space.

CONCLUSION

This study demonstrated the predominance of the proposed model in its ability to accurately classify upper limb movements from multiple classes of EEG signals, and its potential application in the development of a more intuitive and naturalistic prosthetic control.

Developments in the human machine interface technologies and their applications: a review

Human-machine interface (HMI) techniques use bioelectrical signals to gain real-time synchronised communication between the human body and machine functioning. HMI technology not only provides a real-time control access but also has the ability to control multiple functions at a single instance of time with modest human inputs and increased efficiency. The HMI technologies yield advanced control access on numerous applications such as health monitoring, medical diagnostics, development of prosthetic and assistive devices, automotive and aerospace industry, robotic controls and many more fields. In this paper, various physiological signals, their acquisition and processing techniques along with their respective applications in different HMI technologies have been discussed.

Chronic embedded cortico-thalamic closed-loop deep brain stimulation for the treatment of essential tremor

Deep brain stimulation (DBS) is an approved therapy for the treatment of medically refractory and severe movement disorders. However, most existing neurostimulators can only apply continuous stimulation [open-loop DBS (OL-DBS)], ignoring patient behavior and environmental factors, which consequently leads to an inefficient therapy, thus limiting the therapeutic window. Here, we established the feasibility of a self-adjusting therapeutic DBS [closed-loop DBS (CL-DBS)], fully embedded in a chronic investigational neurostimulator (Activa PC + S), for three patients affected by essential tremor (ET) enrolled in a longitudinal (6 months) within-subject crossover protocol (DBS OFF, OL-DBS, and CL-DBS). Most patients with ET experience involuntary limb tremor during goal-directed movements, but not during rest. Hence, the proposed CL-DBS paradigm explored the efficacy of modulating the stimulation amplitude based on patient-specific motor behavior, suppressing the pathological tremor on-demand based on a cortical electrode detecting upper limb motor activity. Here, we demonstrated how the proposed stimulation paradigm was able to achieve clinical efficacy and tremor suppression comparable with OL-DBS in a range of movements (cup reaching, proximal and distal posture, water pouring, and writing) while having a consistent reduction in energy delivery. The proposed paradigm is an important step toward a behaviorally modulated fully embedded DBS system, capable of delivering stimulation only when needed, and potentially mitigating pitfalls of OL-DBS, such as DBS-induced side effects and premature device replacement.

A minimally invasive flexible electrode array for simultaneous recording of ECoG signals from multiple brain regions

The minimal invasiveness of electrocorticography (ECoG) enabled its widespread use in clinical areas as well as in neuroscience research. However, most existing ECoG arrays require that the entire surface area of the brain that is to be recorded be exposed through a large craniotomy. We propose a device that overcomes this limitation, i.e., a minimally invasive, polyimide-based flexible array of electrodes that can enable the recording of ECoG signals in multiple regions of the brain with minimal exposure of the surface of the brain. Magnetic force-assisted positioning of a flexible electrode array enables recording from distant brain regions with a small cranial window. Also, a biodegradable organic compound used for attaching a magnet on the electrodes allows simple retrieval of the magnet. We demonstrate with an in vivo chronic recording that an implanted ECoG electrode array can record ECoG signals from the visual cortex and the motor cortex during a rat's free behavior. Our results indicate that the proposed device induced minimal damage to the animal. We expect the proposed device to be utilized for experiments for large-scale brain circuit analyses as well as clinical applications for intra-operative monitoring of epileptic activity.

Interface, interaction, and intelligence in generalized brain-computer interfaces

A brain-computer interface (BCI) establishes a direct communication channel between a brain and an external device. With recent advances in neurotechnology and artificial intelligence (AI), the brain signals in BCI communication have been advanced from sensation and perception to higher-level cognition activities. While the field of BCI has grown rapidly in the past decades, the core technologies and innovative ideas behind seemingly unrelated BCI systems have never been summarized from an evolutionary point of view. Here, we review various BCI paradigms and present an evolutionary model of generalized BCI technology which comprises three stages: interface, interaction, and intelligence (I3). We also highlight challenges, opportunities, and future perspectives in the development of new BCI technology.

Behavioral and Neural Variability of Naturalistic Arm Movements

Motor behaviors are central to many functions and dysfunctions of the brain, and understanding their neural basis has consequently been a major focus in neuroscience. However, most studies of motor behaviors have been restricted to artificial, repetitive paradigms, far removed from natural movements performed "in the wild." Here, we leveraged recent advances in machine learning and computer vision to analyze intracranial recordings from 12 human subjects during thousands of spontaneous, unstructured arm reach movements, observed over several days for each subject. These naturalistic movements elicited cortical spectral power patterns consistent with findings from controlled paradigms, but with considerable neural variability across subjects and events. We modeled interevent variability using 10 behavioral and environmental features; the most important features explaining this variability were reach angle and day of recording. Our work is among the first studies connecting behavioral and neural variability across cortex in humans during unstructured movements and contributes to our understanding of long-term naturalistic behavior.

Defining Surgical Terminology and Risk for Brain Computer Interface Technologies

With the emergence of numerous brain computer interfaces (BCI), their form factors, and clinical applications the terminology to describe their clinical deployment and the associated risk has been vague. The terms “minimally invasive” or “non-invasive” have been commonly used, but the risk can vary widely based on the form factor and anatomic location. Thus, taken together, there needs to be a terminology that best accommodates the surgical footprint of a BCI and their attendant risks. This work presents a semantic framework that describes the BCI from a procedural standpoint and its attendant clinical risk profile. We propose extending the common invasive/non-invasive distinction for BCI systems to accommodate three categories in which the BCI anatomically interfaces with the patient and whether or not a surgical procedure is required for deployment: (1) Non-invasive—BCI components do not penetrate the body, (2) Embedded—components are penetrative, but not deeper than the inner table of the skull, and (3) Intracranial –components are located within the inner table of the skull and possibly within the brain volume. Each class has a separate risk profile that should be considered when being applied to a given clinical population. Optimally, balancing this risk profile with clinical need provides the most ethical deployment of these emerging classes of devices. As BCIs gain larger adoption, and terminology becomes standardized, having an improved, more precise language will better serve clinicians, patients, and consumers in discussing these technologies, particularly within the context of surgical procedures.

EEG-based diagnostics of the auditory system using cochlear implant electrodes as sensors

The cochlear implant is one of the most successful medical prostheses, allowing deaf and severely hearing-impaired persons to hear again by electrically stimulating the auditory nerve. A trained audiologist adjusts the stimulation settings for good speech understanding, known as "fitting" the implant. This process is based on subjective feedback from the user, making it time-consuming and challenging, especially in paediatric or communication-impaired populations. Furthermore, fittings only happen during infrequent sessions at a clinic, and therefore cannot take into account variable factors that affect the user's hearing, such as physiological changes and different listening environments. Objective audiometry, in which brain responses evoked by auditory stimulation are collected and analysed, removes the need for active patient participation. However, recording of brain responses still requires expensive equipment that is cumbersome to use. An elegant solution is to record the neural signals using the implant itself. We demonstrate for the first time the recording of continuous electroencephalographic (EEG) signals from the implanted intracochlear electrode array in human subjects, using auditory evoked potentials originating from different brain regions. This was done using a temporary recording set-up with a percutaneous connector used for research purposes. Furthermore, we show that the response morphologies and amplitudes depend crucially on the recording electrode configuration. The integration of an EEG system into cochlear implants paves the way towards chronic neuro-monitoring of hearing-impaired patients in their everyday environment, and neuro-steered hearing prostheses, which can autonomously adjust their output based on neural feedback.

An Intracortical Implantable Brain-Computer Interface for Telemetric Real-Time Recording and Manipulation of Neuronal Circuits for Closed-Loop Intervention

Recording and manipulating neuronal ensemble activity is a key requirement in advanced neuromodulatory and behavior studies. Devices capable of both recording and manipulating neuronal activity brain-computer interfaces (BCIs) should ideally operate un-tethered and allow chronic longitudinal manipulations in the freely moving animal. In this study, we designed a new intracortical BCI feasible of telemetric recording and stimulating local gray and white matter of visual neural circuit after irradiation exposure. To increase the translational reliance, we put forward a Göttingen minipig model. The animal was stereotactically irradiated at the level of the visual cortex upon defining the target by a fused cerebral MRI and CT scan. A fully implantable neural telemetry system consisting of a 64 channel intracortical multielectrode array, a telemetry capsule, and an inductive rechargeable battery was then implanted into the visual cortex to record and manipulate local field potentials, and multi-unit activity. We achieved a 3-month stability of the functionality of the un-tethered BCI in terms of telemetric radio-communication, inductive battery charging, and device biocompatibility for 3 months. Finally, we could reliably record the local signature of sub- and suprathreshold neuronal activity in the visual cortex with high bandwidth without complications. The ability to wireless induction charging combined with the entirely implantable design, the rather high recording bandwidth, and the ability to record and stimulate simultaneously put forward a wireless BCI capable of long-term un-tethered real-time communication for causal preclinical circuit-based closed-loop interventions.

Decoding and interpreting cortical signals with a compact convolutional neural network

OBJECTIVE

Brain-computer interfaces (BCIs) decode information from neural activity and send it to external devices. The use of Deep Learning approaches for decoding allows for automatic feature engineering within the specific decoding task. Physiologically plausible interpretation of the network parameters ensures the robustness of the learned decision rules and opens the exciting opportunity for automatic knowledge discovery.

APPROACH

We describe a compact convolutional network-based architecture for adaptive decoding of electrocorticographic (ECoG) data into finger kinematics. We also propose a novel theoretically justified approach to interpreting the spatial and temporal weights in the architectures that combine adaptation in both space and time. The obtained spatial and frequency patterns characterizing the neuronal populations pivotal to the specific decoding task can then be interpreted by fitting appropriate spatial and dynamical models.

MAIN RESULTS

We first tested our solution using realistic Monte-Carlo simulations. Then, when applied to the ECoG data from Berlin BCI competition IV dataset, our architecture performed comparably to the competition winners without requiring explicit feature engineering. Using the proposed approach to the network weights interpretation we could unravel the spatial and the spectral patterns of the neuronal processes underlying the successful decoding of finger kinematics from an ECoG dataset. Finally we have also applied the entire pipeline to the analysis of a 32-channel EEG motor-imagery dataset and observed physiologically plausible patterns specific to the task.

SIGNIFICANCE

We described a compact and interpretable CNN architecture derived from the basic principles and encompassing the knowledge in the field of neural electrophysiology. For the first time in the context of such multibranch architectures with factorized spatial and temporal processing we presented theoretically justified weights interpretation rules. We verified our recipes using simulations and real data and demonstrated that the proposed solution offers a good decoder and a tool for investigating motor control neural mechanisms.

Generalized neural decoders for transfer learning across participants and recording modalities

OBJECTIVE

Advances in neural decoding have enabled brain-computer interfaces to perform increasingly complex and clinically-relevant tasks. However, such decoders are often tailored to specific participants, days, and recording sites, limiting their practical long-term usage. Therefore, a fundamental challenge is to develop neural decoders that can robustly train on pooled, multi-participant data and generalize to new participants.

APPROACH

We introduce a new decoder, HTNet, which uses a convolutional neural network with two innovations: (1) a Hilbert transform that computes spectral power at data-driven frequencies and (2) a layer that projects electrode-level data onto predefined brain regions. The projection layer critically enables applications with intracranial electrocorticography (ECoG), where electrode locations are not standardized and vary widely across participants. We trained HTNet to decode arm movements using pooled ECoG data from 11 of 12 participants and tested performance on unseen ECoG or electroencephalography (EEG) participants; these pretrained models were also subsequently fine-tuned to each test participant.

MAIN RESULTS

HTNet outperformed state-of-the-art decoders when tested on unseen participants, even when a different recording modality was used. By fine-tuning these generalized HTNet decoders, we achieved performance approaching the best tailored decoders with as few as 50 ECoG or 20 EEG events. We were also able to interpret HTNet's trained weights and demonstrate its ability to extract physiologically-relevant features.

SIGNIFICANCE

By generalizing to new participants and recording modalities, robustly handling variations in electrode placement, and allowing participant-specific fine-tuning with minimal data, HTNet is applicable across a broader range of neural decoding applications compared to current state-of-the-art decoders.

Wireless, miniaturized, semi-implantable electrocorticography microsystem validated in vivo

This paper reports on the design, development, and test of a multi-channel wireless micro-electrocorticography (µECoG) system. The system consists of a semi-implantable, ultra-compact recording unit and an external unit, interfaced through a 2.4 GHz radio frequency data telemetry link with 2 Mbps (partially used) data transfer rate. Encased in a 3D-printed 2.9 cm × 2.9 cm × 2.5 cm cubic package, the semi-implantable recording unit consists of a microelectrode array, a vertically-stacked PCB platform containing off-the-shelf components, and commercially-available small-size 3.7-V, 50 mAh lithium-ion batteries. Two versions of microelectrode array were developed for the recording unit: a rigid 4 × 2 microelectrode array, and a flexible 12 × 6 microelectrode array, 36 of which routed to bonding pads for actual recording. The external unit comprises a transceiver board, a data acquisition board, and a host computer, on which reconstruction of the received signals is performed. After development, assembly, and integration, the system was tested and validated in vivo on anesthetized rats. The system successfully recorded both spontaneous and evoked activities from the brain of the subject.

A Trimodal Wireless Implantable Neural Interface System-on-Chip

A wireless and battery-less trimodal neural interface system-on-chip (SoC), capable of 16-ch neural recording, 8-ch electrical stimulation, and 16-ch optical stimulation, all integrated on a 5 ×  3 mm2 chip fabricated in 0.35-μm standard CMOS process. The trimodal SoC is designed to be inductively powered and communicated. The downlink data telemetry utilizes on-off keying pulse-position modulation (OOK-PPM) of the power carrier to deliver configuration and control commands at 50 kbps. The analog front-end (AFE) provides adjustable mid-band gain of 55-70 dB, low/high cut-off frequencies of 1-100 Hz/10 kHz, and input-referred noise of 3.46 μVrms within 1 Hz-50 kHz band. AFE outputs of every two-channel are digitized by a 50 kS/s 10-bit SAR-ADC, and multiplexed together to form a 6.78 Mbps data stream to be sent out by OOK modulating a 434 MHz RF carrier through a power amplifier (PA) and 6 cm monopole antenna, which form the uplink data telemetry. Optical stimulation has a switched-capacitor based stimulation (SCS) architecture, which can sequentially charge four storage capacitor banks up to 4 V and discharge them in selected μLEDs at instantaneous current levels of up to 24.8 mA on demand. Electrical stimulation is supported by four independently driven stimulating sites at 5-bit controllable current levels in ±(25-775) μA range, while active/passive charge balancing circuits ensure safety. In vivo testing was conducted on four anesthetized rats to verify the functionality of the trimodal SoC.