A click-based electrocorticographic brain-computer interface enables long-term high-performance switch-scan spelling

Background

Brain-computer interfaces (BCIs) can restore communication in movement- and/or speech-impaired individuals by enabling neural control of computer typing applications. Single command "click" decoders provide a basic yet highly functional capability.

Methods

We sought to test the performance and long-term stability of click-decoding using a chronically implanted high density electrocorticographic (ECoG) BCI with coverage of the sensorimotor cortex in a human clinical trial participant (ClinicalTrials.gov, NCT03567213) with amyotrophic lateral sclerosis (ALS). We trained the participant's click decoder using a small amount of training data (< 44 minutes across four days) collected up to 21 days prior to BCI use, and then tested it over a period of 90 days without any retraining or updating.

Results

Using this click decoder to navigate a switch-scanning spelling interface, the study participant was able to maintain a median spelling rate of 10.2 characters per min. Though a transient reduction in signal power modulation interrupted testing with this fixed model, a new click decoder achieved comparable performance despite being trained with even less data (< 15 min, within one day).

Conclusion

These results demonstrate that a click decoder can be trained with a small ECoG dataset while retaining robust performance for extended periods, providing functional text-based communication to BCI users.

Neuromorphic applications in medicine

In recent years, there has been a growing demand for miniaturization, low power consumption, quick treatments, and non-invasive clinical strategies in the healthcare industry. To meet these demands, healthcare professionals are seeking new technological paradigms that can improve diagnostic accuracy while ensuring patient compliance. Neuromorphic engineering, which uses neural models in hardware and software to replicate brain-like behaviors, can help usher in a new era of medicine by delivering low power, low latency, small footprint, and high bandwidth solutions. This paper provides an overview of recent neuromorphic advancements in medicine, including medical imaging and cancer diagnosis, processing of biosignals for diagnosis, and biomedical interfaces, such as motor, cognitive, and perception prostheses. For each section, we provide examples of how brain-inspired models can successfully compete with conventional artificial intelligence algorithms, demonstrating the potential of neuromorphic engineering to meet demands and improve patient outcomes. Lastly, we discuss current struggles in fitting neuromorphic hardware with non-neuromorphic technologies and propose potential solutions for future bottlenecks in hardware compatibility.

Cortical decoding of individual finger and wrist kinematics for an upper-limb neuroprosthesis

Previous research has shown that neuronal activity can be used to continuously decode the kinematics of gross movements involving arm and hand trajectory. However, decoding the kinematics of fine motor movements, such as the manipulation of individual fingers, has not been demonstrated. In this study, single unit activities were recorded from task-related neurons in M1 of two trained rhesus monkey as they performed individuated movements of the fingers and wrist. The primates' hand was placed in a manipulandum, and strain gauges at the tips of each finger were used to track the digit's position. Both linear and non-linear filters were designed to simultaneously predict kinematics of each digit and the wrist, and their performance compared using mean squared error and correlation coefficients. All models had high decoding accuracy, but the feedforward ANN (R = 0.76-0.86, MSE = 0.04-0.05) and Kalman filter (R = 0.68-0.86, MSE = 0.04-0.07) performed better than a simple linear regression filter (0.58-0.81, 0.05-0.07). These results suggest that individual finger and wrist kinematics can be decoded with high accuracy, and be used to control a multi-fingered prosthetic hand in real-time.

Continuous decoding of finger position from surface EMG signals for the control of powered prostheses

As development toward multi-fingered dexterous prosthetic hands continues, there is a growing need for more flexible and intuitive control schemes. Through the use of generalized electrode placement and well-established methods of pattern recognition, we have developed a basis for asynchronous decoding of finger positions. With the present method, correlations as large as 0.91 and mean overall decoding errors of approximately 11% have been achieved with average decoding errors of between decoded and actual conformation of the metacarpophalangeal joints of individual fingers. It is hoped that these results will serve as a foundation from which to encourage further investigation into more intuitive methods of myoelectric control of powered upper limb prostheses.

An embedded controller for a 7-degree of freedom prosthetic arm

We present results from an embedded real-time hardware system capable of decoding surface myoelectric signals (sMES) to control a seven degree of freedom upper limb prosthesis. This is one of the first hardware implementations of sMES decoding algorithms and the most advanced controller to-date. We compare decoding results from the device to simulation results from a real-time PC-based operating system. Performance of both systems is shown to be similar, with decoding accuracy greater than 90% for the floating point software simulation and 80% for fixed point hardware and software implementations.

A silicon central pattern generator controls locomotion in vivo

We present a neuromorphic silicon chip that emulates the activity of the biological spinal central pattern generator (CPG) and creates locomotor patterns to support walking. The chip implements ten integrate-and-fire silicon neurons and 190 programmable digital-to-analog converters that act as synapses. This architecture allows for each neuron to make synaptic connections to any of the other neurons as well as to any of eight external input signals and one tonic bias input. The chip's functionality is confirmed by a series of experiments in which it controls the motor output of a paralyzed animal in real-time and enables it to walk along a three-meter platform. The walking is controlled under closed-loop conditions with the aide of sensory feedback that is recorded from the animal's legs and fed into the silicon CPG. Although we and others have previously described biomimetic silicon locomotor control systems for robots, this is the first demonstration of a neuromorphic device that can replace some functions of the central nervous system in vivo.

Asynchronous decoding of dexterous finger movements using M1 neurons

Previous efforts in brain-machine interfaces (BMI) have looked at decoding movement intent or hand and arm trajectory, but current cortical control strategies have not focused on the decoding of dexterous [corrected] actions such as finger movements. The present work demonstrates the asynchronous decoding (i.e., where cues indicating the onset of movement are not known) of individual and combined finger movements. Single-unit activities were recorded sequentially from a population of neurons in the M1 hand area of trained rhesus monkeys during flexion and extension movements of each finger and the wrist. Nonlinear filters were designed to detect the onset of movement and decode the movement type from randomly selected neuronal ensembles (assembled from individually recorded single-unit activities). Average asynchronous decoding accuracies as high as 99.8%, 96.2%, and 90.5%, were achieved for individuated finger and wrist movements with three monkeys. Average decoding accuracy was still 92.5% when combined movements of two fingers were included. These results demonstrate that it is possible to asynchronously decode dexterous finger movements from a neuronal ensemble with high accuracy. This work takes an important step towards the development of a BMI for direct neural control of a state-of-the-art, multifingered hand prosthesis.

Decoding individuated finger movements using volume-constrained neuronal ensembles in the M1 hand area

Individuated finger and wrist movements can be decoded using random subpopulations of neurons that are widely distributed in the primary motor (M1) hand area. This work investigates 1) whether it is possible to decode dexterous finger movements using spatially-constrained volumes of neurons as typically recorded from a microelectrode array; and 2) whether decoding accuracy differs due to the configuration or location of the array within the M1 hand area. Single-unit activities were sequentially recorded from task-related neurons in two rhesus monkeys as they performed individuated movements of the fingers and the wrist. Simultaneous neuronal ensembles were simulated by constraining these activities to the recording field dimensions of conventional microelectrode array architectures. Artificial neural network (ANN) based filters were able to decode individuated finger movements with greater than 90% accuracy for the majority of movement types, using as few as 20 neurons from these ensemble activities. Furthermore, for the large majority of cases there were no significant differences (p < 0.01) in decoding accuracy as a function of the location of the recording volume. The results suggest that a brain-machine interface (BMI) for dexterous control of individuated fingers and the wrist can be implemented using microelectrode arrays placed broadly in the M1 hand area.

Towards control of dexterous hand manipulations using a silicon Pattern Generator

This work demonstrates how an in silico Pattern Generator (PG) can be used as a low power control system for rhythmic hand movements in an upper-limb prosthesis. Neural spike patterns, which encode rotation of a cylindrical object, were implemented in a custom Very Large Scale Integration chip. PG control was tested by using the decoded control signals to actuate the fingers of a virtual prosthetic arm. This system provides a framework for prototyping and controlling dexterous hand manipulation tasks in a compact and efficient solution.

Dynamic control of the central pattern generator for locomotion

We show that an ongoing locomotor pattern can be dynamically controlled by applying discrete pulses of electrical stimulation to the central pattern generator (CPG) for locomotion. Data are presented from a pair of experiments on biological (wetware) and electrical (hardware) models of the CPG demonstrating that stimulation causes brief deviations from the CPG's limit cycle activity. The exact characteristics of the deviation depend strongly on the phase of stimulation. Applications of this work are illustrated by examples showing how locomotion can be controlled by using a feedback loop to monitor CPG activity and applying stimuli at the appropriate times to modulate motor output. Eventually, this approach could lead to development of a neuroprosthetic device for restoring locomotion after paralysis.