High-performance brain-to-text communication via handwriting

Getting a grasp on BMIs: Decoding prehension and speech signals

Wandelt et al. (2022) show that different grasps can be decoded from neural activity in the human supramarginal gyrus (SMG), ventral premotor cortex, and somatosensory cortex during motor imagery and speech, highlighting the attractiveness of higher-level areas such as the SMG for brain-machine interface applications.

A Power-Efficient Brain-Machine Interface System With a Sub-mw Feature Extraction and Decoding ASIC Demonstrated in Nonhuman Primates

Intracortical brain-machine interfaces have shown promise for restoring function to people with paralysis, but their translation to portable and implantable devices is hindered by their high power consumption. Recent devices have drastically reduced power consumption compared to standard experimental brain-machine interfaces, but still require wired or wireless connections to computing hardware for feature extraction and inference. Here, we introduce a Neural Recording And Decoding (NeuRAD) application specific integrated circuit (ASIC) in 180 nm CMOS that can extract neural spiking features and predict two-dimensional behaviors in real-time. To reduce amplifier and feature extraction power consumption, the NeuRAD has a hardware accelerator for extracting spiking band power (SBP) from intracortical spiking signals and includes an M0 processor with a fixed-point Matrix Acceleration Unit (MAU) for efficient and flexible decoding. We validated device functionality by recording SBP from a nonhuman primate implanted with a Utah microelectrode array and predicting the one- and two-dimensional finger movements the monkey was attempting to execute in closed-loop using a steady-state Kalman filter (SSKF). Using the NeuRAD's real-time predictions, the monkey achieved 100% success rate and 0.82 s mean target acquisition time to control one-dimensional finger movements using just 581 μW. To predict two-dimensional finger movements, the NeuRAD consumed 588 μW to enable the monkey to achieve a 96% success rate and 2.4 s mean acquisition time. By employing SBP, ASIC brain-machine interfaces can close the gap to enable fully implantable therapies for people with paralysis.

Harnessing the Power of Artificial Intelligence in Otolaryngology and the Communication Sciences

Use of artificial intelligence (AI) is a burgeoning field in otolaryngology and the communication sciences. A virtual symposium on the topic was convened from Duke University on October 26, 2020, and was attended by more than 170 participants worldwide. This review presents summaries of all but one of the talks presented during the symposium; recordings of all the talks, along with the discussions for the talks, are available at https://www.youtube.com/watch?v=ktfewrXvEFg and https://www.youtube.com/watch?v=-gQ5qX2v3rg . Each of the summaries is about 2500 words in length and each summary includes two figures. This level of detail far exceeds the brief summaries presented in traditional reviews and thus provides a more-informed glimpse into the power and diversity of current AI applications in otolaryngology and the communication sciences and how to harness that power for future applications.

A low-power communication scheme for wireless, 1000 channel brain-machine interfaces

OBJECTIVE

Brain-machine interfaces (BMIs) have the potential to restore motor function but are currently limited by electrode count and long-term recording stability. These challenges may be solved through the use of free-floating "motes" which wirelessly transmit recorded neural signals, if power consumption can be kept within safe levels when scaling to thousands of motes. Here, we evaluated a pulse-interval modulation (PIM) communication scheme for infrared (IR)-based motes that aims to reduce the wireless data rate and system power consumption.

APPROACH

To test PIM's ability to efficiently communicate neural information, we simulated the communication scheme in a real-time closed-loop BMI with non-human primates. Additionally, we performed circuit simulations of an IR-based 1000-mote system to calculate communication accuracy and total power consumption.

MAIN RESULTS

We found that PIM at 1kb/s per channel maintained strong correlations with true firing rate and matched online BMI performance of a traditional wired system. Closed-loop BMI tests suggest that lags as small as 30 ms can have significant performance effects. Finally, unlike other IR communication schemes, PIM is feasible in terms of power, and neural data can accurately be recovered on a receiver using 3mW for 1000 channels.

SIGNIFICANCE

These results suggest that PIM-based communication could significantly reduce power usage of wireless motes to enable higher channel-counts for high-performance BMIs.

SRI-EEG: State-Based Recurrent Imputation for EEG Artifact Correction

Electroencephalogram (EEG) signals are often used as an input modality for Brain Computer Interfaces (BCIs). While EEG signals can be beneficial for numerous types of interaction scenarios in the real world, high levels of noise limits their usage to strictly noise-controlled environments such as a research laboratory. Even in a controlled environment, EEG is susceptible to noise, particularly from user motion, making it highly challenging to use EEG, and consequently BCI, as a ubiquitous user interaction modality. In this work, we address the EEG noise/artifact correction problem. Our goal is to detect physiological artifacts in EEG signal and automatically replace the detected artifacts with imputed values to enable robust EEG sensing overall requiring significantly reduced manual effort than is usual. We present a novel EEG state-based imputation model built upon a recurrent neural network, which we call SRI-EEG, and evaluate the proposed method on three publicly available EEG datasets. From quantitative and qualitative comparisons with six conventional and neural network based approaches, we demonstrate that our method achieves comparable performance to the state-of-the-art methods on the EEG artifact correction task.

Emerging 4D Printing Strategies for Next-Generation Tissue Regeneration and Medical Devices

The rapid development of 3D printing has led to considerable progress in the field of biomedical engineering. Notably, 4D printing provides a potential strategy to achieve a time-dependent physical change within tissue scaffolds or replicate the dynamic biological behaviors of native tissues for smart tissue regeneration and the fabrication of medical devices. The fabricated stimulus-responsive structures can offer dynamic, reprogrammable deformation or actuation to mimic complex physical, biochemical, and mechanical processes of native tissues. Although there is notable progress made in the development of the 4D printing approach for various biomedical applications, its more broad-scale adoption for clinical use and tissue engineering purposes is complicated by a notable limitation of printable smart materials and the simplistic nature of achievable responses possible with current sources of stimulation. In this review, the recent progress made in the field of 4D printing by discussing the various printing mechanisms that are achieved with great emphasis on smart ink mechanisms of 4D actuation, construct structural design, and printing technologies, is highlighted. Recent 4D printing studies which focus on the applications of tissue/organ regeneration and medical devices are then summarized. Finally, the current challenges and future perspectives of 4D printing are also discussed.

Analytic beamformer transformation for transfer learning in motion-onset visual evoked potential decoding

OBJECTIVE

While decoders of EEG-based event-related potentials (ERPs) are routinely tailored to the individual user to maximize performance, developing them on populations for individual usage has proven much more challenging. We propose the analytic beamformer transformation (ABT) to extract phase and/or magnitude information from spatiotemporal ERPs in response to motion-onset stimulation.

APPROACH

We have tested ABT on 52 motion-onset visual evoked potential (mVEP) datasets from 26 healthy subjects and compared the classification accuracy of support vector machine (SVM), spatiotemporal beamformer (stBF) and stepwise linear discriminant analysis (SWLDA) when trained on individual subjects and on a population thereof.

MAIN RESULTS

When using phase- and combined phase/magnitude information extracted by ABT, we show significant improvements in accuracy of population-trained classifiers applied to individual users (p<0.001). We also show that 450 epochs are needed for a correct functioning of ABT, which corresponds to 2 minutes of paradigm stimulation.

SIGNIFICANCE

We have shown that ABT can be used to create population-trained mVEP classifiers using a limited number of epochs. We expect this to pertain to other ERPs or synchronous stimulation paradigms, allowing for a more effective, population-based training of visual BCIs. Finally, as ABT renders recordings across subjects more structurally invariant, it could be used for transfer learning purposes in view of plug-and-play BCI applications.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

A Multimodal Neural-Recording IC With Reconfigurable Analog Front-Ends for Improved Availability and Usability for Recording Channels

In this work, we present an 8-channel reconfigurable multimodal neural-recording IC, which provides improved availability and usability of recording channels in various experiment scenarios. Each recording channel changes its configuration depending on whether the channel is assigned to record voltage or current signal. As a result, although the total number of channels is fixed by design, the channels utilized for voltage and current recording can be set freely and optimally for given experiment targets, scenarios, and circumstances, maximizing the availability and usability of recording channels.The proposed concept was demonstrated by fabricating the IC using a standard 180-nm CMOS process.Using the IC, we successfully performed an in vivo experiment from the hippocampal area of a mouse brain. The measured input noise of the reconfigurable front-end is 4.75 μV_rms at voltage-recording mode and 7.4 pA_rms at current-recording mode while consuming 5.72 μW/channel.

Decoding ECoG signal into 3D hand translation using deep learning

Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.

A Survey of Multi-Agent Cross Domain Cooperative Perception

Intelligent unmanned systems for ground, sea, aviation, and aerospace application are important research directions for the new generation of artificial intelligence in China. Intelligent unmanned systems are also important carriers of interactive mapping between physical space and cyberspace in the process of the digitization of human society. Based on the current domestic and overseas development status of unmanned systems for ground, sea, aviation, and aerospace application, this paper reviewed the theoretical problems and research trends of multi-agent cross-domain cooperative perception. The scenarios of multi-agent cooperative perception tasks in different areas were deeply investigated and analyzed, the scientific problems of cooperative perception were analyzed, and the development direction of multi-agent cooperative perception theory research for solving the challenges of the complex environment, interactive communication, and cross-domain tasks was expounded.

Spinal cord bioelectronic interfaces: opportunities in neural recording and clinical challenges

Bioelectronic stimulation of the spinal cord has demonstrated significant progress in restoration of motor function in spinal cord injury (SCI). The proximal, uninjured spinal cord presents a viable target for the recording and generation of control signals to drive targeted stimulation. Signals have been directly recorded from the spinal cord in behaving animals and correlated with limb kinematics. Advances in flexible materials, electrode impedance and signal analysis will allow SCR to be used in next-generation neuroprosthetics. In this review, we summarize the technological advances enabling progress in SCR and describe systematically the clinical challenges facing spinal cord bioelectronic interfaces and potential solutions, from device manufacture, surgical implantation to chronic effects of foreign body reaction and stress-strain mismatches between electrodes and neural tissue. Finally, we establish our vision of bi-directional closed-loop spinal cord bioelectronic bypass interfaces that enable the communication of disrupted sensory signals and restoration of motor function in SCI.

Spelling interface using intracortical signals in a completely locked-in patient enabled via auditory neurofeedback training

Patients with amyotrophic lateral sclerosis (ALS) can lose all muscle-based routes of communication as motor neuron degeneration progresses, and ultimately, they may be left without any means of communication. While others have evaluated communication in people with remaining muscle control, to the best of our knowledge, it is not known whether neural-based communication remains possible in a completely locked-in state. Here, we implanted two 64 microelectrode arrays in the supplementary and primary motor cortex of a patient in a completely locked-in state with ALS. The patient modulated neural firing rates based on auditory feedback and he used this strategy to select letters one at a time to form words and phrases to communicate his needs and experiences. This case study provides evidence that brain-based volitional communication is possible even in a completely locked-in state.

Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network

Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. The explanation of the system's decision can be explained by directly comparing the real and synthetic data. Recently, by taking advantage of advances in DNN-based image-to-image translation, several studies successfully applied counterfactual explanation to image domains. In principle, the same approach could be used in functional magnetic resonance imaging (fMRI) data. Because fMRI datasets often contain multiple classes (e.g., multiple behavioral tasks), the image-to-image transformation applicable to counterfactual explanation needs to learn mapping among multiple classes simultaneously. Recently, a new generative neural network (StarGAN) that enables image-to-image transformation among multiple classes has been developed. By adapting StarGAN with some modifications, here, we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among all the seven classes in a publicly available fMRI dataset. Thus, CAG could provide a counterfactual explanation of DNN-based multiclass classifiers of brain activations. Furthermore, iterative applications of CAG were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses.

Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming

The usability of EEG-based visual brain–computer interfaces (BCIs) based on event-related potentials (ERPs) benefits from reducing the calibration time before BCI operation. Linear decoding models, such as the spatiotemporal beamformer model, yield state-of-the-art accuracy. Although the training time of this model is generally low, it can require a substantial amount of training data to reach functional performance. Hence, BCI calibration sessions should be sufficiently long to provide enough training data. This work introduces two regularized estimators for the beamformer weights. The first estimator uses cross-validated L2-regularization. The second estimator exploits prior information about the structure of the EEG by assuming Kronecker–Toeplitz-structured covariance. The performances of these estimators are validated and compared with the original spatiotemporal beamformer and a Riemannian-geometry-based decoder using a BCI dataset with P300-paradigm recordings for 21 subjects. Our results show that the introduced estimators are well-conditioned in the presence of limited training data and improve ERP classification accuracy for unseen data. Additionally, we show that structured regularization results in lower training times and memory usage, and a more interpretable classification model.

Ten-Hour Stable Noninvasive Brain-Computer Interface Realized by Semidry Hydrogel-Based Electrodes

Noninvasive brain-computer interface (BCI) has been extensively studied from many aspects in the past decade. In order to broaden the practical applications of BCI technique, it is essential to develop electrodes for electroencephalogram (EEG) collection with advanced characteristics such as high conductivity, long-term effectiveness, and biocompatibility. In this study, we developed a silver-nanowire/PVA hydrogel/melamine sponge (AgPHMS) semidry EEG electrode for long-lasting monitoring of EEG signal. Benefiting from the water storage capacity of PVA hydrogel, the electrolyte solution can be continuously released to the scalp-electrode interface during used. The electrolyte solution can infiltrate the stratum corneum and reduce the scalp-electrode impedance to 10 kΩ-15 kΩ. The flexible structure enables the electrode with mechanical stability, increases the wearing comfort, and reduces the scalp-electrode gap to reduce contact impedance. As a result, a long-term BCI application based on measurements of motion-onset visual evoked potentials (mVEPs) shows that the 3-hour BCI accuracy of the new electrode (77% to 100%) is approximately the same as that of conventional electrodes supported by a conductive gel during the first hour. Furthermore, the BCI system based on the new electrode can retain low contact impedance for 10 hours on scalp, which greatly improved the ability of BCI technique.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Algorithm and hardware considerations for real-time neural signal on-implant processing

Objective Various on-workstation neural-spike-based brain machine interface(BMI) systems have reached the point of in-human trials, but on-node and on-implant BMI systems are still under exploration. Such systems are constrained by the area and battery. Researchers should consider the algorithm complexity, available resources, power budgets, CMOS technologies, and the choice of platforms when designing BMI systems. However, the effect of these factors is currently still unclear. Approaches. Here we have proposed a novel real-time 128 channel spike detection algorithm and optimised it on Microcontroller(MCU) and Field Programmable Gate Array(FPGA) platforms towards consuming minimal power and memory/resources. It is presented as a use case to explore the different considerations in system design. Main results. The proposed spike detection algorithm achieved over 97% sensitivity and a smaller than 3% false detection rate. The MCU implementation occupies less than 3KB RAM and consumes 31.5μW/ch. The FPGA platform only occupies 299 logic cells and 3KB RAM for 128 channels and consumes 0.04μW/ch. Significance. On the spike detection algorithm front, we have eliminated the processing bottleneck by reducing the dynamic power consumption to lower than the hardware static power, without sacrificing detection performance. More importantly, we have explored the considerations in algorithm and hardware design with respect to scalability, portability, and costs. These findings can facilitate and guide the future development of real-time on-implant neural signal processing platforms.

Ten ignored questions for stress psychology research

Stress psychology is an interesting and important interdisciplinary research field. In this perspective article, we briefly discuss 10 challenges related to the conceptual definition, research methodology, and translation in the field of stress that do not receive sufficient attention or are ignored entirely. Future research should attempt to integrate a comprehensive stress conceptual framework into a multidimensional comprehensive stress model, incorporating subjective and objective indicators as comprehensive measures. The popularity of machine learning, cognitive neuroscience, and gene epigenetics is a promising approach that brings innovation to the field of stress psychology. The development of wearable devices that precisely record physiological signals to assess stress responses in naturalistic situations, standardize real-life stressors, and measure baselines presents challenges to address in the future. Conducting large individualized and digital intervention studies could be crucial steps in enhancing the translation of research.

Large-scale neural recordings with single neuron resolution using Neuropixels probes in human cortex

Recent advances in multi-electrode array technology have made it possible to monitor large neuronal ensembles at cellular resolution in animal models. In humans, however, current approaches restrict recordings to a few neurons per penetrating electrode or combine the signals of thousands of neurons in local field potential (LFP) recordings. Here we describe a new probe variant and set of techniques that enable simultaneous recording from over 200 well-isolated cortical single units in human participants during intraoperative neurosurgical procedures using silicon Neuropixels probes. We characterized a diversity of extracellular waveforms with eight separable single-unit classes, with differing firing rates, locations along the length of the electrode array, waveform spatial spread and modulation by LFP events such as inter-ictal discharges and burst suppression. Although some challenges remain in creating a turnkey recording system, high-density silicon arrays provide a path for studying human-specific cognitive processes and their dysfunction at unprecedented spatiotemporal resolution.

Decoding motor imagery tasks using ESI and hybrid feature CNN

Objective. BCI based on motor imaging electroencephalogram (MI-EEG) can be useful in a natural interaction system. In this paper, a new framework is proposed to solve the MI-EEG binary classification problem. Approach. Electrophysiological source imaging (ESI) technology is used to solve the influence of volume conduction effect and improve spatial resolution. Continuous wavelet transform (CWT) and best time of interest (TOI) are combined to extract the optimal discriminant spatial-frequency features. Finally, a CNN network with seven convolution layers is used to classify the features. In addition, we also validated several new data augment methods to solve the problem of small data sets and reduce network over-fitting. Main results. The model achieved an average classification accuracy of 93.2% and 95.4% on the BCI Competition III IVa and high-gamma data sets, which is better than most of the published advanced algorithms. By selecting the best TOI for each subject, the classification accuracy rate increased by about 2%. The effects of 4 data augment methods on the classification results were also verified. Among them, the noise addition and overlap methods are better than the other two, and the classification accuracy is improved by at least 4%. On the contrary, the rotation and flip data augment methods reduced the classification accuracy. Significance. Decoding motor imagery tasks can benefited from combing the ESI technology and the data augment technology, which is used to solve the problem of low spatial resolution and small samples of EEG signals, respectively. Based on the results, the model proposed has higher accuracy and application potential in the task of MI-EEG binary classification.

Fast and accurate decoding of finger movements from ECoG through Riemannian features and modern machine learning techniques

Objective: Accurate decoding of individual finger movements is crucial for advanced prosthetic control. In this work, we introduce the use of Riemannian-space features and temporal dynamics of electrocorticography (ECoG) signal combined with modern machine learning tools to improve the motor decoding accuracy at the level of individual fingers. Approach: We selected a set of informative biomarkers that correlated with finger movements and evaluated the performance of state-of-the-art machine learning algorithms on the BCI competition IV dataset (ECoG, three subjects) and a second ECoG dataset with a similar recording paradigm (Stanford, 9 subjects). We further explored the temporal concatenation of features to effectively capture the history of ECoG signal, which led to a significant improvement over single-epoch decoding in both classification (p<0.01) and regression tasks (p<0.01). Main results: Using feature concatenation and gradient boosted trees (the top-performing model), we achieved a classification accuracy of 77.0% in detecting individual finger movements (6-class task, including rest state), improving over the state-of-the-art conditional random fields (CRF) by 11.7% on the 3 BCI competition subjects. In continuous decoding of movement trajectory, our approach resulted in an average Pearson's correlation coefficient (r) of 0.537 across subjects and fingers, outperforming both the BCI competition winner and the state-of-the-art approach reported on the same dataset (CNN+LSTM). Furthermore, our proposed method features a low time complexity, with only <17.2s required for training and <50ms for inference. This enables about 250× speed-up in training compared to previously reported deep learning method with state-of-the-art performance. Significance: The proposed techniques enable fast, reliable, and high-performance prosthetic control through minimally-invasive cortical signals.

Effects of Gaze Fixation on the Performance of a Motor Imagery-Based Brain-Computer Interface

Motor imagery-based brain-computer interfaces (BCIs) have been studied without controlling subjects’ gaze fixation position previously. The effect of gaze fixation and covert attention on the behavioral performance of BCI is still unknown. This study designed a gaze fixation controlled experiment. Subjects were required to conduct a secondary task of gaze fixation when performing the primary task of motor imagination. Subjects’ performance was analyzed according to the relationship between motor imagery target and the gaze fixation position, resulting in three BCI control conditions, i.e., congruent, incongruent, and center cross trials. A group of fourteen subjects was recruited. The average group performances of three different conditions did not show statistically significant differences in terms of BCI control accuracy, feedback duration, and trajectory length. Further analysis of gaze shift response time revealed a significantly shorter response time for congruent trials compared to incongruent trials. Meanwhile, the parietal occipital cortex also showed active neural activities for congruent and incongruent trials, and this was revealed by a contrast analysis of R-square values and lateralization index. However, the lateralization index computed from the parietal and occipital areas was not correlated with the BCI behavioral performance. Subjects’ BCI behavioral performance was not affected by the position of gaze fixation and covert attention. This indicated that motor imagery-based BCI could be used freely in robotic arm control without sacrificing performance.

Locoregional Recurrence Prediction Using a Deep Neural Network of Radiological and Radiotherapy Images

Radiation therapy (RT) is an important and potentially curative modality for head and neck squamous cell carcinoma (HNSCC). Locoregional recurrence (LR) of HNSCC after RT is ranging from 15% to 50% depending on the primary site and stage. In addition, the 5-year survival rate of patients with LR is low. To classify high-risk patients who might develop LR, a deep learning model for predicting LR needs to be established. In this work, 157 patients with HNSCC who underwent RT were analyzed. Based on the National Cancer Institute’s multi-institutional TCIA data set containing FDG-PET/CT/dose, a 3D deep learning model was proposed to predict LR without time-consuming segmentation or feature extraction. Our model achieved an averaged area under the curve (AUC) of 0.856. Adding clinical factors into the model improved the AUC to an average of 0.892 with the highest AUC of up to 0.974. The 3D deep learning model could perform individualized risk quantification of LR in patients with HNSCC without time-consuming tumor segmentation.

AI in health and medicine

Artificial intelligence (AI) is poised to broadly reshape medicine, potentially improving the experiences of both clinicians and patients. We discuss key findings from a 2-year weekly effort to track and share key developments in medical AI. We cover prospective studies and advances in medical image analysis, which have reduced the gap between research and deployment. We also address several promising avenues for novel medical AI research, including non-image data sources, unconventional problem formulations and human-AI collaboration. Finally, we consider serious technical and ethical challenges in issues spanning from data scarcity to racial bias. As these challenges are addressed, AI's potential may be realized, making healthcare more accurate, efficient and accessible for patients worldwide.

Cortical Control of Virtual Self-Motion Using Task-Specific Subspaces

Brain-machine interfaces (BMIs) for reaching have enjoyed continued performance improvements, yet there remains significant need for BMIs that control other movement classes. Recent scientific findings suggest that the intrinsic covariance structure of neural activity depends strongly on movement class, potentially necessitating different decode algorithms across classes. To address this possibility, we developed a self-motion BMI based on cortical activity as monkeys cycled a hand-held pedal to progress along a virtual track. Unlike during reaching, we found no high-variance dimensions that directly correlated with to-be-decoded variables. This was due to no neurons having consistent correlations between their responses and kinematic variables. Yet we could decode a single variable-self-motion-by nonlinearly leveraging structure that spanned multiple high-variance neural dimensions. Resulting online BMI-control success rates approached those during manual control. These findings make two broad points regarding how to build decode algorithms that harmonize with the empirical structure of neural activity in motor cortex. First, even when decoding from the same cortical region (e.g., arm-related motor cortex), different movement classes may need to employ very different strategies. Although correlations between neural activity and hand velocity are prominent during reaching tasks, they are not a fundamental property of motor cortex and cannot be counted on to be present in general. Second, although one generally desires a low-dimensional readout, it can be beneficial to leverage a multidimensional high-variance subspace. Fully embracing this approach requires highly nonlinear approaches tailored to the task at hand, but can produce near-native levels of performance.SIGNIFICANCE STATEMENT Many brain-machine interface decoders have been constructed for controlling movements normally performed with the arm. Yet it is unclear how these will function beyond the reach-like scenarios where they were developed. Existing decoders implicitly assume that neural covariance structure, and correlations with to-be-decoded kinematic variables, will be largely preserved across tasks. We find that the correlation between neural activity and hand kinematics, a feature typically exploited when decoding reach-like movements, is essentially absent during another task performed with the arm: cycling through a virtual environment. Nevertheless, the use of a different strategy, one focused on leveraging the highest-variance neural signals, supported high performance real-time brain-machine interface control.

Exploring Cognition with Brain-Machine Interfaces

Traditional brain-machine interfaces decode cortical motor commands to control external devices. These commands are the product of higher-level cognitive processes, occurring across a network of brain areas, that integrate sensory information, plan upcoming motor actions, and monitor ongoing movements. We review cognitive signals recently discovered in the human posterior parietal cortex during neuroprosthetic clinical trials. These signals are consistent with small regions of cortex having a diverse role in cognitive aspects of movement control and body monitoring, including sensorimotor integration, planning, trajectory representation, somatosensation, action semantics, learning, and decision making. These variables are encoded within the same population of cells using structured representations that bind related sensory and motor variables, an architecture termed partially mixed selectivity. Diverse cognitive signals provide complementary information to traditional motor commands to enable more natural and intuitive control of external devices.

Brain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication

Damage or degeneration of motor pathways necessary for speech and other movements, as in brainstem strokes or amyotrophic lateral sclerosis (ALS), can interfere with efficient communication without affecting brain structures responsible for language or cognition. In the worst-case scenario, this can result in the locked in syndrome (LIS), a condition in which individuals cannot initiate communication and can only express themselves by answering yes/no questions with eye blinks or other rudimentary movements. Existing augmentative and alternative communication (AAC) devices that rely on eye tracking can improve the quality of life for people with this condition, but brain-computer interfaces (BCIs) are also increasingly being investigated as AAC devices, particularly when eye tracking is too slow or unreliable. Moreover, with recent and ongoing advances in machine learning and neural recording technologies, BCIs may offer the only means to go beyond cursor control and text generation on a computer, to allow real-time synthesis of speech, which would arguably offer the most efficient and expressive channel for communication. The potential for BCI speech synthesis has only recently been realized because of seminal studies of the neuroanatomical and neurophysiological underpinnings of speech production using intracranial electrocorticographic (ECoG) recordings in patients undergoing epilepsy surgery. These studies have shown that cortical areas responsible for vocalization and articulation are distributed over a large area of ventral sensorimotor cortex, and that it is possible to decode speech and reconstruct its acoustics from ECoG if these areas are recorded with sufficiently dense and comprehensive electrode arrays. In this article, we review these advances, including the latest neural decoding strategies that range from deep learning models to the direct concatenation of speech units. We also discuss state-of-the-art vocoders that are integral in constructing natural-sounding audio waveforms for speech BCIs. Finally, this review outlines some of the challenges ahead in directly synthesizing speech for patients with LIS.

Volitional control of individual neurons in the human brain

Brain-machine interfaces allow neuroscientists to causally link specific neural activity patterns to a particular behaviour. Thus, in addition to their current clinical applications, brain-machine interfaces can also be used as a tool to investigate neural mechanisms of learning and plasticity in the brain. Decades of research using such brain-machine interfaces have shown that animals (non-human primates and rodents) can be operantly conditioned to self-regulate neural activity in various motor-related structures of the brain. Here, we ask whether the human brain, a complex interconnected structure of over 80 billion neurons, can learn to control itself at the most elemental scale-a single neuron. We used the unique opportunity to record single units in 11 individuals with epilepsy to explore whether the firing rate of a single (direct) neuron in limbic and other memory-related brain structures can be brought under volitional control. To do this, we developed a visual neurofeedback task in which participants were trained to move a block on a screen by modulating the activity of an arbitrarily selected neuron from their brain. Remarkably, participants were able to volitionally modulate the firing rate of the direct neuron in these previously uninvestigated structures. We found that a subset of participants (learners), were able to improve their performance within a single training session. Successful learning was characterized by (i) highly specific modulation of the direct neuron (demonstrated by significantly increased firing rates and burst frequency); (ii) a simultaneous decorrelation of the activity of the direct neuron from the neighbouring neurons; and (iii) robust phase-locking of the direct neuron to local alpha/beta-frequency oscillations, which may provide some insights in to the potential neural mechanisms that facilitate this type of learning. Volitional control of neuronal activity in mnemonic structures may provide new ways of probing the function and plasticity of human memory without exogenous stimulation. Furthermore, self-regulation of neural activity in these brain regions may provide an avenue for the development of novel neuroprosthetics for the treatment of neurological conditions that are commonly associated with pathological activity in these brain structures, such as medically refractory epilepsy.

Functional Characterization of Human Pluripotent Stem Cell-Derived Models of the Brain with Microelectrode Arrays

Human pluripotent stem cell (hPSC)-derived neuron cultures have emerged as models of electrical activity in the human brain. Microelectrode arrays (MEAs) measure changes in the extracellular electric potential of cell cultures or tissues and enable the recording of neuronal network activity. MEAs have been applied to both human subjects and hPSC-derived brain models. Here, we review the literature on the functional characterization of hPSC-derived two- and three-dimensional brain models with MEAs and examine their network function in physiological and pathological contexts. We also summarize MEA results from the human brain and compare them to the literature on MEA recordings of hPSC-derived brain models. MEA recordings have shown network activity in two-dimensional hPSC-derived brain models that is comparable to the human brain and revealed pathology-associated changes in disease models. Three-dimensional hPSC-derived models such as brain organoids possess a more relevant microenvironment, tissue architecture and potential for modeling the network activity with more complexity than two-dimensional models. hPSC-derived brain models recapitulate many aspects of network function in the human brain and provide valid disease models, but certain advancements in differentiation methods, bioengineering and available MEA technology are needed for these approaches to reach their full potential.

Neural Decoding of Chinese Sign Language With Machine Learning for Brain-Computer Interfaces

Limb motion decoding is an important part of brain-computer interface (BCI) research. Among the limb motion, sign language not only contains rich semantic information and abundant maneuverable actions but also provides different executable commands. However, many researchers focus on decoding the gross motor skills, such as the decoding of ordinary motor imagery or simple upper limb movements. Here we explored the neural features and decoding of Chinese sign language from electroencephalograph (EEG) signal with motor imagery and motor execution. Sign language not only contains rich semantic information, but also has abundant maneuverable actions, and provides us with more different executable commands. In this paper, twenty subjects were instructed to perform movement execution and movement imagery based on Chinese sign language. Seven classifiers are employed to classify the selected features of sign language EEG. L1 regularization is used to learn and select features that contain more information from the mean, power spectral density, sample entropy, and brain network connectivity. The best average classification accuracy of the classifier is 89.90% (imagery sign language is 83.40%). These results have shown the feasibility of decoding between different sign languages. The source location reveals that the neural circuits involved in sign language are related to the visual contact area and the pre-movement area. Experimental evaluation shows that the proposed decoding strategy based on sign language can obtain outstanding classification results, which provides a certain reference value for the subsequent research of limb decoding based on sign language.

Skilled independent control of individual motor units via a non-invasive neuromuscular-machine interface

Objective.Brain-machine interfaces (BMIs) have the potential to augment human functions and restore independence in people with disabilities, yet a compromise between non-invasiveness and performance limits their relevance.Approach.Here, we hypothesized that a non-invasive neuromuscular-machine interface providing real-time neurofeedback of individual motor units within a muscle could enable independent motor unit control to an extent suitable for high-performance BMI applications.Main results.Over 6 days of training, eight participants progressively learned to skillfully and independently control three biceps brachii motor units to complete a 2D center-out task. We show that neurofeedback enabled motor unit activity that largely violated recruitment constraints observed during ramp-and-hold isometric contractions thought to limit individual motor unit controllability. Finally, participants demonstrated the suitability of individual motor units for powering general applications through a spelling task.Significance.These results illustrate the flexibility of the sensorimotor system and highlight individual motor units as a promising source of control for BMI applications.

CyberEye: New Eye-Tracking Interfaces for Assessment and Modulation of Cognitive Functions beyond the Brain

The emergence of innovative neurotechnologies in global brain projects has accelerated research and clinical applications of BCIs beyond sensory and motor functions. Both invasive and noninvasive sensors are developed to interface with cognitive functions engaged in thinking, communication, or remembering. The detection of eye movements by a camera offers a particularly attractive external sensor for computer interfaces to monitor, assess, and control these higher brain functions without acquiring signals from the brain. Features of gaze position and pupil dilation can be effectively used to track our attention in healthy mental processes, to enable interaction in disorders of consciousness, or to even predict memory performance in various brain diseases. In this perspective article, we propose the term ‘CyberEye’ to encompass emerging cognitive applications of eye-tracking interfaces for neuroscience research, clinical practice, and the biomedical industry. As CyberEye technologies continue to develop, we expect BCIs to become less dependent on brain activities, to be less invasive, and to thus be more applicable.

Prospects for cardiovascular medicine using artificial intelligence

As the importance of artificial intelligence (AI) in the clinical setting increases, the need for clinicians to understand AI is also increasing. This review focuses on the fundamental principles of AI and the current state of cardiovascular AI. Various types of cardiovascular AI have been developed for evaluating examinations such as X-rays, electrocardiogram, echocardiography, computed tomography, and magnetic resonance imaging. Cardiovascular AI achieves high accuracy in diagnostic support and prognosis prediction. Furthermore, it can even detect abnormalities that were previously difficult for cardiologists to detect. Randomized controlled trials begin to be reported to verify the usefulness of cardiovascular AI. The day is approaching when cardiovascular AI will be commonly used in clinical practice. Various types of medical AI will be used for cardiovascular care; however, it will not replace medical doctors. We need to understand the strengths and weaknesses of medical AI so that cardiologists can effectively use AI to improve the medical care of patients.

Implantable brain machine interfaces: first-in-human studies, technology challenges and trends

Implantable brain machine interfaces (BMIs) are now on a trajectory to go mainstream, wherein what was once considered last resort will progressively become elective at earlier stages in disease treatment. First-in-human successes have demonstrated the ability to decode highly dexterous motor skills such as handwriting, and speech from human cortical activity. These have been used for cursor and prosthesis control, direct-to-text communication and speech synthesis. Along with these breakthrough studies, technology advancements have enabled the observation of more channels of neural activity through new concepts for centralised/distributed implant architectures. This is complemented by research in flexible substrates, packaging, surgical workflows and data processing. New regulatory guidance and funding has galvanised the field. This culmination of resource, efforts and capability is now attracting significant investment for BMI commercialisation. This paper reviews recent developments and describes the paradigm shift in BMI development that is leading to new innovations, insights and BMI translation.

Decoding four hand gestures with a single bipolar pair of electrocorticography electrodes

Objective.Electrocorticography (ECoG) based brain-computer interfaces (BCIs) can be used to restore communication in individuals with locked-in syndrome. In motor-based BCIs, the number of degrees-of-freedom, and thus the speed of the BCI, directly depends on the number of classes that can be discriminated from the neural activity in the sensorimotor cortex. When considering minimally invasive BCI implants, the size of the subdural ECoG implant must be minimized without compromising the number of degrees-of-freedom.Approach.Here we investigated if four hand gestures could be decoded using a single ECoG strip of four consecutive electrodes spaced 1 cm apart and compared the performance between a unipolar and a bipolar montage. For that we collected data of seven individuals with intractable epilepsy implanted with ECoG grids, covering the hand region of the sensorimotor cortex. Based on the implanted grids, we generated virtual ECoG strips and compared the decoding accuracy between (a) a single unipolar electrode (Unipolar Electrode), (b) a combination of four unipolar electrodes (Unipolar Strip), (c) a single bipolar pair (Bipolar Pair) and (d) a combination of six bipolar pairs (Bipolar Strip).Main results.We show that four hand gestures can be equally well decoded using 'Unipolar Strips' (mean 67.4 ± 11.7%), 'Bipolar Strips' (mean 66.6 ± 12.1%) and 'Bipolar Pairs' (mean 67.6 ± 9.4%), while 'Unipolar Electrodes' (61.6 ± 5.9%) performed significantly worse compared to 'Unipolar Strips' and 'Bipolar Pairs'.Significance.We conclude that a single bipolar pair is a potential candidate for minimally invasive motor-based BCIs and encourage the use of ECoG as a robust and reliable BCI platform for multi-class movement decoding.

Visual percepts evoked with an Intracortical 96-channel microelectrode array inserted in human occipital cortex

BACKGROUND

A long-held dream of scientists is to transfer information directly to the visual cortex of blind individuals, thereby restoring a rudimentary form of sight. However, no clinically available cortical visual prosthesis yet exists.

METHODS

We implanted an intracortical microelectrode array consisting of 96 electrodes in the visual cortex of a 57-year-old person with complete blindness for a six- month period. We measured thresholds and the characteristics of the visual percepts elicited by intracortical microstimulation.

RESULTS

Implantation and subsequent explantation of intracortical microelectrodes were carried out without complications. The mean stimulation threshold for single electrodes was 66.8 ± 36.5 μA. We consistently obtained high-quality recordings from visually deprived neurons and the stimulation parameters remained stable over time. Simultaneous stimulation via multiple electrodes were associated with a significant reduction in thresholds (p<0.001, ANOVA test) and evoked discriminable phosphene percepts, allowing the blind participant to identify some letters and recognize object boundaries. Furthermore, we observed a learning process that helped the subject to recognize complex patterns over time.

CONCLUSIONS

Our results demonstrate the safety and efficacy of chronic intracortical microstimulation via a large number of electrodes in human visual cortex, showing its high potential for restoring functional vision in the blind.

TRIAL REGISTRATION

ClinicalTrials.gov identifier NCT02983370.

FUNDING

Funding was provided by grant RTI2018-098969-B-100 from the Spanish Ministerio de Ciencia Innovación y Universidades, by grant PROMETEO/2019/119 from the Generalitat Valenciana (Spain), by the Bidons Egara Research Chair of the University Miguel Hernández (Spain) and by the John Moran Eye Center of the University of Utah (US).

Silent EEG-Speech Recognition Using Convolutional and Recurrent Neural Network with 85% Accuracy of 9 Words Classification

In this work, we focus on silent speech recognition in electroencephalography (EEG) data of healthy individuals to advance brain–computer interface (BCI) development to include people with neurodegeneration and movement and communication difficulties in society. Our dataset was recorded from 270 healthy subjects during silent speech of eight different Russia words (commands): ‘forward’, ‘backward’, ‘up’, ‘down’, ‘help’, ‘take’, ‘stop’, and ‘release’, and one pseudoword. We began by demonstrating that silent word distributions can be very close statistically and that there are words describing directed movements that share similar patterns of brain activity. However, after training one individual, we achieved 85% accuracy performing 9 words (including pseudoword) classification and 88% accuracy on binary classification on average. We show that a smaller dataset collected on one participant allows for building a more accurate classifier for a given subject than a larger dataset collected on a group of people. At the same time, we show that the learning outcomes on a limited sample of EEG-data are transferable to the general population. Thus, we demonstrate the possibility of using selected command-words to create an EEG-based input device for people on whom the neural network classifier has not been trained, which is particularly important for people with disabilities.

Design a novel BCI for neurorehabilitation using concurrent LFP and EEG features: a case study

Brain-computer interfaces (BCI) that enables people with severe motor disabilities to use their brain signals for direct control of objects have attracted increased interest in rehabilitation. To date, no study has investigated feasibility of the BCI framework incorporating both intracortical and scalp signals. Methods: Concurrent local field potential (LFP) from the hand-knob area and scalp EEG were recorded in a paraplegic patient undergoing a spike-based close-loop neurorehabilitation training. Based upon multimodal spatio-spectral feature extraction and Naive Bayes classification, we developed, for the first time, a novel LFP-EEG-BCI for motor intention decoding. A transfer learning (TL) approach was employed to further improve the feasibility. The performance of the proposed LFP-EEG-BCI for four-class upper-limb motor intention decoding was assessed. Results: Using a decision fusion strategy, we showed that the LFP-EEG-BCI significantly (p <0.05) outperformed single modal BCI (LFP-BCI and EEG-BCI) in terms of decoding accuracy with the best performance achieved using regularized common spatial pattern features. Interrogation of feature characteristics revealed discriminative spatial and spectral patterns, which may lead to new insights for better understanding of brain dynamics during different motor imagery tasks and promote development of efficient decoding algorithms. Moreover, we showed that similar classification performance could be obtained with few training trials, therefore highlighting the efficacy of TL. Conclusion: The present findings demonstrated the superiority of the novel LFP-EEG-BCI in motor intention decoding. Significance: This work introduced a novel LFP-EEG-BCI that may lead to new directions for developing practical neurorehabilitation systems with high detection accuracy and multi-paradigm feasibility in clinical applications.

An end-to-end CNN with attentional mechanism applied to raw EEG in a BCI classification task

Objective.Motor-imagery (MI) classification base on electroencephalography (EEG) has been long studied in neuroscience and more recently widely used in healthcare applications such as mobile assistive robots and neurorehabilitation. In particular, EEG-based MI classification methods that rely on convolutional neural networks (CNNs) have achieved relatively high classification accuracy. However, naively training CNNs to classify raw EEG data from all channels, especially for high-density EEG, is computationally demanding and requires huge training sets. It often also introduces many irrelevant input features, making it difficult for the CNN to extract the informative ones. This problem is compounded by a dearth of training data, which is particularly acute for MI tasks, because these are cognitively demanding and thus fatigue inducing.Approach.To address these issues, we proposed an end-to-end CNN-based neural network with attentional mechanism together with different data augmentation (DA) techniques. We tested it on two benchmark MI datasets, brain-computer interface (BCI) competition IV 2a and 2b. In addition, we collected a new dataset, recorded using high-density EEG, and containing both MI and motor execution (ME) tasks, which we share with the community.Main results.Our proposed neural-network architecture outperformed all state-of-the-art methods that we found in the literature, with and without DA, reaching an average classification accuracy of 93.6% and 87.83% on BCI 2a and 2b, respectively. We also directly compare decoding of MI and ME tasks. Focusing on MI classification, we find optimal channel configurations and the best DA techniques as well as investigate combining data across participants and the role of transfer learning.Significance.Our proposed approach improves the classification accuracy for MI in the benchmark datasets. In addition, collecting our own dataset enables us to compare MI and ME and investigate various aspects of EEG decoding critical for neuroscience and BCI.

Multi-scale neural decoding and analysis

Objective.

Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.

Approach.

We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.

Main results.

We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.

Significance.

This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

The neuroethics of disorders of consciousness: a brief history of evolving ideas

Neuroethical questions raised by recent advances in the diagnosis and treatment of disorders of consciousness are rapidly expanding, increasingly relevant, and yet underexplored. The aim of this thematic review is to provide a clinically applicable framework for understanding the current taxonomy of disorders of consciousness and to propose an approach to identifying and critically evaluating actionable neuroethical issues that are frequently encountered in research and clinical care for this vulnerable population. Increased awareness of these issues and clarity about opportunities for optimizing ethically-responsible care in this domain are especially timely given recent surges in critically ill patients with unusually prolonged disorders of consciousness associated with coronavirus disease 2019 (COVID-19) around the world. We begin with an overview of the field of neuroethics: what it is, its history and evolution in the context of biomedical ethics at large. We then explore nomenclature used in disorders of consciousness, covering categories proposed by the American Academy of Neurology, the American Congress of Rehabilitation Medicine, and the National Institute on Disability, Independent Living, and Rehabilitation Research, including definitions of terms such as coma, the vegetative state, unresponsive wakefulness syndrome, minimally conscious state, covert consciousness, and the confusional state. We discuss why these definitions matter, and why there has been such evolution in this nosology over the years, from Jennett and Plum in 1972 to the Multi-Society Task Force in 1994, the Aspen Working Group in 2002 and up until the 2018 American and 2020 European Disorders of Consciousness guidelines. We then move to a discussion of clinical aspects of disorders of consciousness, the natural history of recovery, and ethical issues that arise within the context of caring for persons with disorders of consciousness. We conclude with a discussion of key challenges associated with assessing residual consciousness in disorders of consciousness, potential solutions and future directions, including integration of crucial disability rights perspectives.

Brain-Machine Interfaces: Lessons for Prosthetic Hand Control

Brain-machine interfaces (BMI) are being developed to restore upper limb function for persons with spinal cord injury or other motor degenerative conditions. BMI and implantable sensors for myoelectric prostheses directly extract information from the central or peripheral nervous system to provide users with high fidelity control of their prosthetic device. Control algorithms have been highly transferable between the 2 technologies but also face common issues. In this review of the current state of the art in each field, the authors point out similarities and differences between the 2 technologies that may guide the implementation of common solutions to these challenges.

Gated Recurrent Units Viewed Through the Lens of Continuous Time Dynamical Systems

Gated recurrent units (GRUs) are specialized memory elements for building recurrent neural networks. Despite their incredible success on various tasks, including extracting dynamics underlying neural data, little is understood about the specific dynamics representable in a GRU network. As a result, it is both difficult to know a priori how successful a GRU network will perform on a given task, and also their capacity to mimic the underlying behavior of their biological counterparts. Using a continuous time analysis, we gain intuition on the inner workings of GRU networks. We restrict our presentation to low dimensions, allowing for a comprehensive visualization. We found a surprisingly rich repertoire of dynamical features that includes stable limit cycles (nonlinear oscillations), multi-stable dynamics with various topologies, and homoclinic bifurcations. At the same time we were unable to train GRU networks to produce continuous attractors, which are hypothesized to exist in biological neural networks. We contextualize the usefulness of different kinds of observed dynamics and support our claims experimentally.