What limits the performance of current invasive brain machine interfaces?

Calibration-Free and Hardware-Efficient Neural Spike Detection for Brain Machine Interfaces

Recent translational efforts in brain-machine interfaces (BMI) are demonstrating the potential to help people with neurological disorders. The current trend in BMI technology is to increase the number of recording channels to the thousands, resulting in the generation of vast amounts of raw data. This in turn places high bandwidth requirements for data transmission, which increases power consumption and thermal dissipation of implanted systems. On-implant compression and/or feature extraction are therefore becoming essential to limiting this increase in bandwidth, but add further power constraints - the power required for data reduction must remain less than the power saved through bandwidth reduction. Spike detection is a common feature extraction technique used for intracortical BMIs. In this article, we develop a novel firing-rate-based spike detection algorithm that requires no external training and is hardware efficient and therefore ideally suited for real-time applications. Key performance and implementation metrics such as detection accuracy, adaptability in chronic deployment, power consumption, area utilization, and channel scalability are benchmarked against existing methods using various datasets. The algorithm is first validated using a reconfigurable hardware (FPGA) platform and then ported to a digital ASIC implementation in both 65 nm and 0.18 μm CMOS technologies. The 128-channel ASIC design implemented in a 65 nm CMOS technology occupies 0.096 mm2 silicon area and consumes 4.86 μW from a 1.2 V power supply. The adaptive algorithm achieves a 96% spike detection accuracy on a commonly used synthetic dataset, without the need for any prior training.

Brain augmentation and neuroscience technologies: current applications, challenges, ethics and future prospects

Ever since the dawn of antiquity, people have strived to improve their cognitive abilities. From the advent of the wheel to the development of artificial intelligence, technology has had a profound leverage on civilization. Cognitive enhancement or augmentation of brain functions has become a trending topic both in academic and public debates in improving physical and mental abilities. The last years have seen a plethora of suggestions for boosting cognitive functions and biochemical, physical, and behavioral strategies are being explored in the field of cognitive enhancement. Despite expansion of behavioral and biochemical approaches, various physical strategies are known to boost mental abilities in diseased and healthy individuals. Clinical applications of neuroscience technologies offer alternatives to pharmaceutical approaches and devices for diseases that have been fatal, so far. Importantly, the distinctive aspect of these technologies, which shapes their existing and anticipated participation in brain augmentations, is used to compare and contrast them. As a preview of the next two decades of progress in brain augmentation, this article presents a plausible estimation of the many neuroscience technologies, their virtues, demerits, and applications. The review also focuses on the ethical implications and challenges linked to modern neuroscientific technology. There are times when it looks as if ethics discussions are more concerned with the hypothetical than with the factual. We conclude by providing recommendations for potential future studies and development areas, taking into account future advancements in neuroscience innovation for brain enhancement, analyzing historical patterns, considering neuroethics and looking at other related forecasts.

Implantable brain machine interfaces: first-in-human studies, technology challenges and trends

Implantable brain machine interfaces (BMIs) are now on a trajectory to go mainstream, wherein what was once considered last resort will progressively become elective at earlier stages in disease treatment. First-in-human successes have demonstrated the ability to decode highly dexterous motor skills such as handwriting, and speech from human cortical activity. These have been used for cursor and prosthesis control, direct-to-text communication and speech synthesis. Along with these breakthrough studies, technology advancements have enabled the observation of more channels of neural activity through new concepts for centralised/distributed implant architectures. This is complemented by research in flexible substrates, packaging, surgical workflows and data processing. New regulatory guidance and funding has galvanised the field. This culmination of resource, efforts and capability is now attracting significant investment for BMI commercialisation. This paper reviews recent developments and describes the paradigm shift in BMI development that is leading to new innovations, insights and BMI translation.

Signal Generation, Acquisition, and Processing in Brain Machine Interfaces: A Unified Review

Brain machine interfaces (BMIs), or brain computer interfaces (BCIs), are devices that act as a medium for communications between the brain and the computer. It is an emerging field with numerous applications in domains of prosthetic devices, robotics, communication technology, gaming, education, and security. It is noted in such a multidisciplinary field, many reviews have surveyed on various focused subfields of interest, such as neural signaling, microelectrode fabrication, and signal classification algorithms. A unified review is lacking to cover and link all the relevant areas in this field. Herein, this review intends to connect on the relevant areas that circumscribe BMIs to present a unified script that may help enhance our understanding of BMIs. Specifically, this article discusses signal generation within the cortex, signal acquisition using invasive, non-invasive, or hybrid techniques, and the signal processing domain. The latest development is surveyed in this field, particularly in the last decade, with discussions regarding the challenges and possible solutions to allow swift disruption of BMI products in the commercial market.

Long-term intracortical microelectrode array performance in a human: a 5 year retrospective analysis

Objective. Brain-computer interfaces (BCIs) that record neural activity using intracortical microelectrode arrays (MEAs) have shown promise for mitigating disability associated with neurological injuries and disorders. While the chronic performance and failure modes of MEAs have been well studied and systematically described in non-human primates, there is far less reported about long-term MEA performance in humans. Our group has collected one of the largest neural recording datasets from a Utah MEA in a human subject, spanning over 5 years (2014-2019). Here we present both long-term signal quality and BCI performance as well as highlight several acute signal disruption events observed during the clinical study.Approach. Long-term Utah array performance was evaluated by analyzing neural signal metric trends and decoding accuracy for tasks regularly performed across 448 clinical recording sessions. For acute signal disruptions, we identify or hypothesize the root cause of the disruption, show how the disruption manifests in the collected data, and discuss potential identification and mitigation strategies for the disruption.Main results. Neural signal quality metrics deteriorated rapidly within the first year, followed by a slower decline through the remainder of the study. Nevertheless, BCI performance remained high 5 years after implantation, which is encouraging for the translational potential of this technology as an assistive device. We also present examples of unanticipated signal disruptions during chronic MEA use, which are critical to detect as BCI technology progresses toward home usage.Significance. Our work fills a gap in knowledge around long-term MEA performance in humans, providing longevity and efficacy data points to help characterize the performance of implantable neural sensors in a human population. The trial was registered on ClinicalTrials.gov (Identifier NCT01997125) and conformed to institutional requirements for the conduct of human subjects research.

Auditory cues reveal intended movement information in middle frontal gyrus neuronal ensemble activity of a person with tetraplegia

Intracortical brain-computer interfaces (iBCIs) allow people with paralysis to directly control assistive devices using neural activity associated with the intent to move. Realizing the full potential of iBCIs critically depends on continued progress in understanding how different cortical areas contribute to movement control. Here we present the first comparison between neuronal ensemble recordings from the left middle frontal gyrus (MFG) and precentral gyrus (PCG) of a person with tetraplegia using an iBCI. As expected, PCG was more engaged in selecting and generating intended movements than in earlier perceptual stages of action planning. By contrast, MFG displayed movement-related information during the sensorimotor processing steps preceding the appearance of the action plan in PCG, but only when the actions were instructed using auditory cues. These results describe a previously unreported function for neurons in the human left MFG in auditory processing contributing to motor control.

The combination of brain-computer interfaces and artificial intelligence: applications and challenges

Brain-computer interfaces (BCIs) have shown great prospects as real-time bidirectional links between living brains and actuators. Artificial intelligence (AI), which can advance the analysis and decoding of neural activity, has turbocharged the field of BCIs. Over the past decade, a wide range of BCI applications with AI assistance have emerged. These "smart" BCIs including motor and sensory BCIs have shown notable clinical success, improved the quality of paralyzed patients' lives, expanded the athletic ability of common people and accelerated the evolution of robots and neurophysiological discoveries. However, despite technological improvements, challenges remain with regard to the long training periods, real-time feedback, and monitoring of BCIs. In this article, the authors review the current state of AI as applied to BCIs and describe advances in BCI applications, their challenges and where they could be headed in the future.

Reinforcement schedules differentially affect learning in neuronal operant conditioning in rats

Operant conditioning of neuronal activity is a core process for better operation of brain-machine interfaces. However, few studies have investigated the role of reinforcement schedules in neuronal operant conditioning, although they are very effective in behavioral operant conditioning. To test the effect of different reinforcement schedules, the authors trained single-neuron activity in the motor cortex using fixed ratio (FR) and variable ratio (VR) schedules in rats. Neuronal firing rates were enhanced in the FR but not in the VR schedule during conditioning, suggesting that the principles of operant conditioning of neuronal activity are different from those of behavioral responses.

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.

Implantable Neural Probes for Brain-Machine Interfaces - Current Developments and Future Prospects

A Brain-Machine interface (BMI) allows for direct communication between the brain and machines. Neural probes for recording neural signals are among the essential components of a BMI system. In this report, we review research regarding implantable neural probes and their applications to BMIs. We first discuss conventional neural probes such as the tetrode, Utah array, Michigan probe, and electroencephalography (ECoG), following which we cover advancements in next-generation neural probes. These next-generation probes are associated with improvements in electrical properties, mechanical durability, biocompatibility, and offer a high degree of freedom in practical settings. Specifically, we focus on three key topics: (1) novel implantable neural probes that decrease the level of invasiveness without sacrificing performance, (2) multi-modal neural probes that measure both electrical and optical signals, (3) and neural probes developed using advanced materials. Because safety and precision are critical for practical applications of BMI systems, future studies should aim to enhance these properties when developing next-generation neural probes.

Recovering Motor Activation with Chronic Peripheral Nerve Computer Interface

Interfaces with the peripheral nerve provide the ability to extract motor activation and restore sensation to amputee patients. The ability to chronically extract motor activations from the peripheral nervous system remains an unsolved problem. In this study, chronic recordings with the Flat Interface Nerve Electrode (FINE) are employed to recover the activation levels of innervated muscles. The FINEs were implanted on the sciatic nerves of canines, and neural recordings were obtained as the animal walked on a treadmill. During these trials, electromyograms (EMG) from the surrounding hamstring muscles were simultaneously recorded and the neural recordings are shown to be free of interference or crosstalk from these muscles. Using a novel Bayesian algorithm, the signals from individual fascicles were recovered and then compared to the corresponding target EMG of the lower limb. High correlation coefficients (0.84 ± 0.07 and 0.61 ± 0.12) between the extracted tibial fascicle/medial gastrocnemius and peroneal fascicle/tibialis anterior muscle were obtained. Analysis calculating the information transfer rate (ITR) from the muscle to the motor predictions yielded approximately 5 and 1 bit per second (bps) for the two sources. This method can predict motor signals from neural recordings and could be used to drive a prosthesis by interfacing with residual nerves.

Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review

Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed.

Intracortical Microstimulation Modulates Cortical Induced Responses

Recent advances in cortical prosthetics relied on intracortical microstimulation (ICMS) to activate the cortical neural network and convey information to the brain. Here we show that activity elicited by low-current ICMS modulates induced cortical responses to a sensory stimulus in the primary auditory cortex (A1). A1 processes sensory stimuli in a stereotyped manner, encompassing two types of activity: evoked activity (phase-locked to the stimulus) and induced activity (non-phase-locked to the stimulus). Time-frequency analyses of extracellular potentials recorded from all layers and the surface of the auditory cortex of anesthetized guinea pigs of both sexes showed that ICMS during the processing of a transient acoustic stimulus differentially affected the evoked and induced response. Specifically, ICMS enhanced the long-latency-induced component, mimicking physiological gain increasing top-down feedback processes. Furthermore, the phase of the local field potential at the time of stimulation was predictive of the response amplitude for acoustic stimulation, ICMS, as well as combined acoustic and electric stimulation. Together, this was interpreted as a sign that the response to electrical stimulation was integrated into the ongoing cortical processes in contrast to substituting them. Consequently, ICMS modulated the cortical response to a sensory stimulus. We propose such targeted modulation of cortical activity (as opposed to a stimulation that substitutes the ongoing processes) as an alternative approach for cortical prostheses.SIGNIFICANCE STATEMENT Intracortical microstimulation (ICMS) is commonly used to activate a specific subset of cortical neurons, without taking into account the ongoing activity at the time of stimulation. Here, we found that a low-current ICMS pulse modulated the way the auditory cortex processed a peripheral stimulus, by supra-additively combining the response to the ICMS with the cortical processing of the peripheral stimulus. This artificial modulation mimicked natural modulations of response magnitude such as attention or expectation. In contrast to what was implied in earlier studies, this shows that the response to electrical stimulation is not substituting ongoing cortical activity but is integrated into the natural processes.

A rodent brain-machine interface paradigm to study the impact of paraplegia on BMI performance

BACKGROUND

Most brain machine interfaces (BMI) focus on upper body function in non-injured animals, not addressing the lower limb functional needs of those with paraplegia. A need exists for a novel BMI task that engages the lower body and takes advantage of well-established rodent spinal cord injury (SCI) models to study methods to improve BMI performance.

NEW METHOD

A tilt BMI task was designed that randomly applies different types of tilts to a platform, decodes the tilt type applied and rights the platform if the decoder correctly classifies the tilt type. The task was tested on female rats and is relatively natural such that it does not require the animal to learn a new skill. It is self-rewarding such that there is no need for additional rewards, eliminating food or water restriction, which can be especially hard on spinalized rats. Finally, task difficulty can be adjusted by making the tilt parameters.

RESULTS

This novel BMI task bilaterally engages the cortex without visual feedback regarding limb position in space and animals learn to improve their performance both pre and post-SCI.Comparison with Existing Methods: Most BMI tasks primarily engage one hemisphere, are upper-body, rely heavily on visual feedback, do not perform investigations in animal models of SCI, and require nonnaturalistic extrinsic motivation such as water rewarding for performance improvement. Our task addresses these gaps.

CONCLUSIONS

The BMI paradigm presented here will enable researchers to investigate the interaction of plasticity after SCI and plasticity during BMI training on performance.

Progress in Neuroengineering for brain repair: New challenges and open issues

Background

In recent years, biomedical devices have proven to be able to target also different neurological disorders. Given the rapid ageing of the population and the increase of invalidating diseases affecting the central nervous system, there is a growing demand for biomedical devices of immediate clinical use. However, to reach useful therapeutic results, these tools need a multidisciplinary approach and a continuous dialogue between neuroscience and engineering, a field that is named neuroengineering. This is because it is fundamental to understand how to read and perturb the neural code in order to produce a significant clinical outcome.

Results

In this review, we first highlight the importance of developing novel neurotechnological devices for brain repair and the major challenges expected in the next years. We describe the different types of brain repair strategies being developed in basic and clinical research and provide a brief overview of recent advances in artificial intelligence that have the potential to improve the devices themselves. We conclude by providing our perspective on their implementation to humans and the ethical issues that can arise.

Conclusions

Neuroengineering approaches promise to be at the core of future developments for clinical applications in brain repair, where the boundary between biology and artificial intelligence will become increasingly less pronounced.

Optimizing the learning rate for adaptive estimation of neural encoding models

Closed-loop neurotechnologies often need to adaptively learn an encoding model that relates the neural activity to the brain state, and is used for brain state decoding. The speed and accuracy of adaptive learning algorithms are critically affected by the learning rate, which dictates how fast model parameters are updated based on new observations. Despite the importance of the learning rate, currently an analytical approach for its selection is largely lacking and existing signal processing methods vastly tune it empirically or heuristically. Here, we develop a novel analytical calibration algorithm for optimal selection of the learning rate in adaptive Bayesian filters. We formulate the problem through a fundamental trade-off that learning rate introduces between the steady-state error and the convergence time of the estimated model parameters. We derive explicit functions that predict the effect of learning rate on error and convergence time. Using these functions, our calibration algorithm can keep the steady-state parameter error covariance smaller than a desired upper-bound while minimizing the convergence time, or keep the convergence time faster than a desired value while minimizing the error. We derive the algorithm both for discrete-valued spikes modeled as point processes nonlinearly dependent on the brain state, and for continuous-valued neural recordings modeled as Gaussian processes linearly dependent on the brain state. Using extensive closed-loop simulations, we show that the analytical solution of the calibration algorithm accurately predicts the effect of learning rate on parameter error and convergence time. Moreover, the calibration algorithm allows for fast and accurate learning of the encoding model and for fast convergence of decoding to accurate performance. Finally, larger learning rates result in inaccurate encoding models and decoders, and smaller learning rates delay their convergence. The calibration algorithm provides a novel analytical approach to predictably achieve a desired level of error and convergence time in adaptive learning, with application to closed-loop neurotechnologies and other signal processing domains.

Dexterous Control of Seven Functional Hand Movements Using Cortically-Controlled Transcutaneous Muscle Stimulation in a Person With Tetraplegia

Individuals with tetraplegia identify restoration of hand function as a critical, unmet need to regain their independence and improve quality of life. Brain-Computer Interface (BCI)-controlled Functional Electrical Stimulation (FES) technology addresses this need by reconnecting the brain with paralyzed limbs to restore function. In this study, we quantified performance of an intuitive, cortically-controlled, transcutaneous FES system on standardized object manipulation tasks from the Grasp and Release Test (GRT). We found that a tetraplegic individual could use the system to control up to seven functional hand movements, each with >95% individual accuracy. He was able to select one movement from the possible seven movements available to him and use it to appropriately manipulate all GRT objects in real-time using naturalistic grasps. With the use of the system, the participant not only improved his GRT performance over his baseline, demonstrating an increase in number of transfers for all objects except the Block, but also significantly improved transfer times for the heaviest objects (videocassette (VHS), Can). Analysis of underlying motor cortex neural representations associated with the hand grasp states revealed an overlap or non-separability in neural activation patterns for similarly shaped objects that affected BCI-FES performance. These results suggest that motor cortex neural representations for functional grips are likely more related to hand shape and force required to hold objects, rather than to the objects themselves. These results, demonstrating multiple, naturalistic functional hand movements with the BCI-FES, constitute a further step toward translating BCI-FES technologies from research devices to clinical neuroprosthetics.

Sub 100 nW Volatile Nano-Metal-Oxide Memristor as Synaptic-Like Encoder of Neuronal Spikes

Advanced neural interfaces mediate a bioelectronic link between the nervous system and microelectronic devices, bearing great potential as innovative therapy for various diseases. Spikes from a large number of neurons are recorded leading to creation of big data that require online processing under most stringent conditions, such as minimal power dissipation and on-chip space occupancy. Here, we present a new concept where the inherent volatile properties of a nano-scale memristive device are used to detect and compress information on neural spikes as recorded by a multielectrode array. Simultaneously, and similarly to a biological synapse, information on spike amplitude and frequency is transduced in metastable resistive state transitions of the device, which is inherently capable of self-resetting and of continuous encoding of spiking activity. Furthermore, operating the memristor in a very high resistive state range reduces its average in-operando power dissipation to less than 100 nW, demonstrating the potential to build highly scalable, yet energy-efficient on-node processors for advanced neural interfaces.

A SSVEP Stimuli Encoding Method Using Trinary Frequency-Shift Keying Encoded SSVEP (TFSK-SSVEP)

SSVEP is a kind of BCI technology with advantage of high information transfer rate. However, due to its nature, frequencies could be used as stimuli are scarce. To solve such problem, a stimuli encoding method which encodes SSVEP signal using Frequency Shift–Keying (FSK) method is developed. In this method, each stimulus is controlled by a FSK signal which contains three different frequencies that represent “Bit 0,” “Bit 1” and “Bit 2” respectively. Different to common BFSK in digital communication, “Bit 0” and “Bit 1” composited the unique identifier of stimuli in binary bit stream form, while “Bit 2” indicates the ending of a stimuli encoding. EEG signal is acquired on channel Oz, O1, O2, Pz, P3, and P4, using ADS1299 at the sample rate of 250 SPS. Before original EEG signal is quadrature demodulated, it is detrended and then band-pass filtered using FFT-based FIR filtering to remove interference. Valid peak of the processed signal is acquired by calculating its derivative and converted into bit stream using window method. Theoretically, this coding method could implement at least 2ⁿ⁻¹ (n is the length of bit command) stimulus while keeping the ITR the same. This method is suitable to implement stimuli on a monitor and where the frequency and phase could be used to code stimuli is limited as well as implementing portable BCI devices which is not capable of performing complex calculations.

State-Dependent Decoding Algorithms Improve the Performance of a Bidirectional BMI in Anesthetized Rats

Brain-machine interfaces (BMIs) promise to improve the quality of life of patients suffering from sensory and motor disabilities by creating a direct communication channel between the brain and the external world. Yet, their performance is currently limited by the relatively small amount of information that can be decoded from neural activity recorded form the brain. We have recently proposed that such decoding performance may be improved when using state-dependent decoding algorithms that predict and discount the large component of the trial-to-trial variability of neural activity which is due to the dependence of neural responses on the network's current internal state. Here we tested this idea by using a bidirectional BMI to investigate the gain in performance arising from using a state-dependent decoding algorithm. This BMI, implemented in anesthetized rats, controlled the movement of a dynamical system using neural activity decoded from motor cortex and fed back to the brain the dynamical system's position by electrically microstimulating somatosensory cortex. We found that using state-dependent algorithms that tracked the dynamics of ongoing activity led to an increase in the amount of information extracted form neural activity by 22%, with a consequently increase in all of the indices measuring the BMI's performance in controlling the dynamical system. This suggests that state-dependent decoding algorithms may be used to enhance BMIs at moderate computational cost.

Improving Detection Accuracy of Memristor-Based Bio-Signal Sensing Platform

Recently a novel neuronal activity sensor exploiting the intrinsic thresholded integrator capabilities of memristor devices has been proposed. Extracellular potentials captured by a standard bio-signal acquisition platform are fed into a memristive device which reacts to the input by changing its resistive state (RS) only when the signal ampitude exceeds a threshold. Thus, significant peaks in the neural signal can be stored as non-volatile changes in memristor resistive state whilst noise is effectively suppressed. However, as a memristor is subjected to increasing numbers of supra-threshold stimuli during practical operation, it accumulates (RS) changes and eventually saturates. This leads to severely redsignaluced neural activity detection capabilities. In this work we explore different signal processing and memristor operating procedure strategies in order to improve the detection rate of significant neuronal activity events. We analyse the data obtained from a single-memristive device biased with a reference neural recording and observe that performance can be improved markedly by a) increasing the frequency at which the memristor is reset to an initial resistive state where it is known to be highly responsive, b) appropriately preconditioning the input waveform through application of gain and offset in order to optimally exploit the intrinsic device behaviour. All results are validated by benchmarking obtained spike detection performance against a state-of-the-art template matching system utilising computationally-heavy, multi-dimensional, principal component analysis.

An Improved Unscented Kalman Filter Based Decoder for Cortical Brain-Machine Interfaces

Brain-machine interfaces (BMIs) seek to connect brains with machines or computers directly, for application in areas such as prosthesis control. For this application, the accuracy of the decoding of movement intentions is crucial. We aim to improve accuracy by designing a better encoding model of primary motor cortical activity during hand movements and combining this with decoder engineering refinements, resulting in a new unscented Kalman filter based decoder, UKF2, which improves upon our previous unscented Kalman filter decoder, UKF1. The new encoding model includes novel acceleration magnitude, position-velocity interaction, and target-cursor-distance features (the decoder does not require target position as input, it is decoded). We add a novel probabilistic velocity threshold to better determine the user's intent to move. We combine these improvements with several other refinements suggested by others in the field. Data from two Rhesus monkeys indicate that the UKF2 generates offline reconstructions of hand movements (mean CC 0.851) significantly more accurately than the UKF1 (0.833) and the popular position-velocity Kalman filter (0.812). The encoding model of the UKF2 could predict the instantaneous firing rate of neurons (mean CC 0.210), given kinematic variables and past spiking, better than the encoding models of these two decoders (UKF1: 0.138, p-v Kalman: 0.098). In closed-loop experiments where each monkey controlled a computer cursor with each decoder in turn, the UKF2 facilitated faster task completion (mean 1.56 s vs. 2.05 s) and higher Fitts's Law bit rate (mean 0.738 bit/s vs. 0.584 bit/s) than the UKF1. These results suggest that the modeling and decoder engineering refinements of the UKF2 improve decoding performance. We believe they can be used to enhance other decoders as well.

A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder

Bidirectional brain-machine interfaces (BMIs) establish a two-way direct communication link between the brain and the external world. A decoder translates recorded neural activity into motor commands and an encoder delivers sensory information collected from the environment directly to the brain creating a closed-loop system. These two modules are typically integrated in bulky external devices. However, the clinical support of patients with severe motor and sensory deficits requires compact, low-power, and fully implantable systems that can decode neural signals to control external devices. As a first step toward this goal, we developed a modular bidirectional BMI setup that uses a compact neuromorphic processor as a decoder. On this chip we implemented a network of spiking neurons built using its ultra-low-power mixed-signal analog/digital circuits. On-chip on-line spike-timing-dependent plasticity synapse circuits enabled the network to learn to decode neural signals recorded from the brain into motor outputs controlling the movements of an external device. The modularity of the BMI allowed us to tune the individual components of the setup without modifying the whole system. In this paper, we present the features of this modular BMI and describe how we configured the network of spiking neuron circuits to implement the decoder and to coordinate it with the encoder in an experimental BMI paradigm that connects bidirectionally the brain of an anesthetized rat with an external object. We show that the chip learned the decoding task correctly, allowing the interfaced brain to control the object's trajectories robustly. Based on our demonstration, we propose that neuromorphic technology is mature enough for the development of BMI modules that are sufficiently low-power and compact, while being highly computationally powerful and adaptive.

Real-time encoding and compression of neuronal spikes by metal-oxide memristors

Advanced brain-chip interfaces with numerous recording sites bear great potential for investigation of neuroprosthetic applications. The bottleneck towards achieving an efficient bio-electronic link is the real-time processing of neuronal signals, which imposes excessive requirements on bandwidth, energy and computation capacity. Here we present a unique concept where the intrinsic properties of memristive devices are exploited to compress information on neural spikes in real-time. We demonstrate that the inherent voltage thresholds of metal-oxide memristors can be used for discriminating recorded spiking events from background activity and without resorting to computationally heavy off-line processing. We prove that information on spike amplitude and frequency can be transduced and stored in single devices as non-volatile resistive state transitions. Finally, we show that a memristive device array allows for efficient data compression of signals recorded by a multi-electrode array, demonstrating the technology's potential for building scalable, yet energy-efficient on-node processors for brain-chip interfaces.

The need for intelligent compression of big data, for example in neuroscience, has sparked interest in neuromorphic data processing. Here, Gupta et al. use memristors as event integrators to encode and compress neuronal spiking activity recorded by multi-electrode arrays.

The Effects of Closed-Loop Medical Devices on the Autonomy and Accountability of Persons and Systems

Closed-loop medical devices such as brain-computer interfaces are an emerging and rapidly advancing neurotechnology. The target patients for brain-computer interfaces (BCIs) are often severely paralyzed, and thus particularly vulnerable in terms of personal autonomy, decisionmaking capacity, and agency. Here we analyze the effects of closed-loop medical devices on the autonomy and accountability of both persons (as patients or research participants) and neurotechnological closed-loop medical systems. We show that although BCIs can strengthen patient autonomy by preserving or restoring communicative abilities and/or motor control, closed-loop devices may also create challenges for moral and legal accountability. We advocate the development of a comprehensive ethical and legal framework to address the challenges of emerging closed-loop neurotechnologies like BCIs and stress the centrality of informed consent and refusal as a means to foster accountability. We propose the creation of an international neuroethics task force with members from medical neuroscience, neuroengineering, computer science, medical law, and medical ethics, as well as representatives of patient advocacy groups and the public.

Implications of the Dependence of Neuronal Activity on Neural Network States for the Design of Brain-Machine Interfaces

Brain-machine interfaces (BMIs) can improve the quality of life of patients with sensory and motor disabilities by both decoding motor intentions expressed by neural activity, and by encoding artificially sensed information into patterns of neural activity elicited by causal interventions on the neural tissue. Yet, current BMIs can exchange relatively small amounts of information with the brain. This problem has proved difficult to overcome by simply increasing the number of recording or stimulating electrodes, because trial-to-trial variability of neural activity partly arises from intrinsic factors (collectively known as the network state) that include ongoing spontaneous activity and neuromodulation, and so is shared among neurons. Here we review recent progress in characterizing the state dependence of neural responses, and in particular of how neural responses depend on endogenous slow fluctuations of network excitability. We then elaborate on how this knowledge may be used to increase the amount of information that BMIs exchange with brain. Knowledge of network state can be used to fine-tune the stimulation pattern that should reliably elicit a target neural response used to encode information in the brain, and to discount part of the trial-by-trial variability of neural responses, so that they can be decoded more accurately.

Robust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering

Much progress has been made in brain-machine interfaces (BMI) using decoders such as Kalman filters and finding their parameters with closed-loop decoder adaptation (CLDA). However, current decoders do not model the spikes directly, and hence may limit the processing time-scale of BMI control and adaptation. Moreover, while specialized CLDA techniques for intention estimation and assisted training exist, a unified and systematic CLDA framework that generalizes across different setups is lacking. Here we develop a novel closed-loop BMI training architecture that allows for processing, control, and adaptation using spike events, enables robust control and extends to various tasks. Moreover, we develop a unified control-theoretic CLDA framework within which intention estimation, assisted training, and adaptation are performed. The architecture incorporates an infinite-horizon optimal feedback-control (OFC) model of the brain's behavior in closed-loop BMI control, and a point process model of spikes. The OFC model infers the user's motor intention during CLDA-a process termed intention estimation. OFC is also used to design an autonomous and dynamic assisted training technique. The point process model allows for neural processing, control and decoder adaptation with every spike event and at a faster time-scale than current decoders; it also enables dynamic spike-event-based parameter adaptation unlike current CLDA methods that use batch-based adaptation on much slower adaptation time-scales. We conducted closed-loop experiments in a non-human primate over tens of days to dissociate the effects of these novel CLDA components. The OFC intention estimation improved BMI performance compared with current intention estimation techniques. OFC assisted training allowed the subject to consistently achieve proficient control. Spike-event-based adaptation resulted in faster and more consistent performance convergence compared with batch-based methods, and was robust to parameter initialization. Finally, the architecture extended control to tasks beyond those used for CLDA training. These results have significant implications towards the development of clinically-viable neuroprosthetics.

Wireless Cortical Brain-Machine Interface for Whole-Body Navigation in Primates

Several groups have developed brain-machine-interfaces (BMIs) that allow primates to use cortical activity to control artificial limbs. Yet, it remains unknown whether cortical ensembles could represent the kinematics of whole-body navigation and be used to operate a BMI that moves a wheelchair continuously in space. Here we show that rhesus monkeys can learn to navigate a robotic wheelchair, using their cortical activity as the main control signal. Two monkeys were chronically implanted with multichannel microelectrode arrays that allowed wireless recordings from ensembles of premotor and sensorimotor cortical neurons. Initially, while monkeys remained seated in the robotic wheelchair, passive navigation was employed to train a linear decoder to extract 2D wheelchair kinematics from cortical activity. Next, monkeys employed the wireless BMI to translate their cortical activity into the robotic wheelchair’s translational and rotational velocities. Over time, monkeys improved their ability to navigate the wheelchair toward the location of a grape reward. The navigation was enacted by populations of cortical neurons tuned to whole-body displacement. During practice with the apparatus, we also noticed the presence of a cortical representation of the distance to reward location. These results demonstrate that intracranial BMIs could restore whole-body mobility to severely paralyzed patients in the future.

Characterization of Artifacts Produced by Gel Displacement on Non-invasive Brain-Machine Interfaces during Ambulation

So far, Brain-Machine Interfaces (BMIs) have been mainly used to study brain potentials during movement-free conditions. Recently, due to the emerging concern of improving rehabilitation therapies, these systems are also being used during gait experiments. Under this new condition, the evaluation of motion artifacts has become a critical point to assure the validity of the results obtained. Due to the high signal to noise ratio provided, the use of wet electrodes is a widely accepted technic to acquire electroencephalographic (EEG signals). To perform these recordings it is necessary to apply a conductive gel between the scalp and the electrodes. This work is focused on the study of gel displacements produced during ambulation and how they affect the amplitude of EEG signals. Data recorded during three ambulation conditions (gait training) and one movement-free condition (BMI motor imagery task) are compared to perform this study. Two phenomenons, manifested as unusual increases of the signals' amplitude, have been identified and characterized during this work. Results suggest that they are caused by abrupt changes on the conductivity between the electrode and the scalp due to gel displacement produced during ambulation and head movements. These artifacts significantly increase the Power Spectral Density (PSD) of EEG recordings at all frequencies from 5 to 90 Hz, corresponding to the main bandwidth of electrocortical potentials. They should be taken into consideration before performing EEG recordings in order to asses the correct gel allocation and to avoid the use of electrodes on certain scalp areas depending on the experimental conditions.

A bidirectional brain-machine interface connecting alert rodents to a dynamical system

We present a novel experimental framework that implements a bidirectional brain-machine interface inspired by the operation of the spinal cord in vertebrates that generates a control policy in the form of a force field. The proposed experimental set-up allows connecting the brain of freely moving rats to an external device. We tested this apparatus in a preliminary experiment with an alert rat that used the interface for acquiring a food reward. The goal of this approach to bidirectional interfaces is to explore the role of voluntary neural commands in controlling a dynamical system represented by a small cart moving on vertical plane and connected to a water/pellet dispenser.

A Generalizable Brain-Computer Interface (BCI) Using Machine Learning for Feature Discovery

This work describes a generalized method for classifying motor-related neural signals for a brain-computer interface (BCI), based on a stochastic machine learning method. The method differs from the various feature extraction and selection techniques employed in many other BCI systems. The classifier does not use extensive a-priori information, resulting in reduced reliance on highly specific domain knowledge. Instead of pre-defining features, the time-domain signal is input to a population of multi-layer perceptrons (MLPs) in order to perform a stochastic search for the best structure. The results showed that the average performance of the new algorithm outperformed other published methods using the Berlin BCI IV (2008) competition dataset and was comparable to the best results in the Berlin BCI II (2002–3) competition dataset. The new method was also applied to electroencephalography (EEG) data recorded from five subjects undertaking a hand squeeze task and demonstrated high levels of accuracy with a mean classification accuracy of 78.9% after five-fold cross-validation. Our new approach has been shown to give accurate results across different motor tasks and signal types as well as between subjects.

A novel wireless recording and stimulating multichannel epicortical grid for supplementing or enhancing the sensory-motor functions in monkey (Macaca fascicularis)

Artificial brain-machine interfaces (BMIs) represent a prospective step forward supporting or replacing faulty brain functions. So far, several obstacles, such as the energy supply, the portability and the biocompatibility, have been limiting their effective translation in advanced experimental or clinical applications. In this work, a novel 16 channel chronically implantable epicortical grid has been proposed. It provides wireless transmission of cortical recordings and stimulations, with induction current recharge. The grid has been chronically implanted in a non-human primate (Macaca fascicularis) and placed over the somato-motor cortex such that 13 electrodes recorded or stimulated the primary motor cortex and three the primary somatosensory cortex, in the deeply anaesthetized animal. Cortical sensory and motor recordings and stimulations have been performed within 3 months from the implant. In detail, by delivering motor cortex epicortical single spot stimulations (1-8 V, 1-10 Hz, 500 ms, biphasic waves), we analyzed the motor topographic precision, evidenced by tunable finger or arm movements of the anesthetized animal. The responses to light mechanical peripheral sensory stimuli (blocks of 100 stimuli, each single stimulus being <1 ms and interblock intervals of 1.5-4 s) have been analyzed. We found 150-250 ms delayed cortical responses from fast finger touches, often spread to nearby motor stations. We also evaluated the grid electrical stimulus interference with somatotopic natural tactile sensory processing showing no suppressing interference with sensory stimulus detection. In conclusion, we propose a chronically implantable epicortical grid which can accommodate most of current technological restrictions, representing an acceptable candidate for BMI experimental and clinical uses.

Bottlenecks to clinical translation of direct brain-computer interfaces

Despite several decades of research into novel brain-implantable devices to treat a range of diseases, only two—cochlear implants for sensorineural hearing loss and deep brain stimulation for movement disorders—have yielded any appreciable clinical benefit. Obstacles to translation include technical factors (e.g., signal loss due to gliosis or micromotion), lack of awareness of current clinical options for patients that the new therapy must outperform, traversing between federal and corporate funding needed to support clinical trials, and insufficient management expertise. This commentary reviews these obstacles preventing the translation of promising new neurotechnologies into clinical application and suggests some principles that interdisciplinary teams in academia and industry could adopt to enhance their chances of success.

Combining decoder design and neural adaptation in brain-machine interfaces

Brain-machine interfaces (BMIs) aim to help people with paralysis by decoding movement-related neural signals into control signals for guiding computer cursors, prosthetic arms, and other assistive devices. Despite compelling laboratory experiments and ongoing FDA pilot clinical trials, system performance, robustness, and generalization remain challenges. We provide a perspective on how two complementary lines of investigation, that have focused on decoder design and neural adaptation largely separately, could be brought together to advance BMIs. This BMI paradigm should also yield new scientific insights into the function and dysfunction of the nervous system.

Should I stay or should I go? Conceptual underpinnings of goal-directed actions

All actions, even the simplest like moving an arm to grasp a pen, are associated with energy costs. Thus all mobile organisms possess the ability to evaluate resources and select those behaviors that are most likely to lead to the greatest accrual of valuable items (reward) in the near or, especially in the case of humans, distant future. The evaluation process is performed at all possible stages of the series of decisions that lead to the building of a goal-directed action or to its suppression. This is because all animals have a limited amount of energy and resources; to survive and be able to reproduce they have to minimize the costs and maximize the outcomes of their actions. These computations are at the root of behavioral flexibility. Two executive functions play a major role in generating flexible behaviors: (i) the ability to predict future outcomes of goal-directed actions; and (ii) the ability to cancel them when they are unlikely to accomplish valuable results. These two processes operate continuously during the entire course of a movement: during its genesis, its planning and even its execution, so that the motor output can be modulated or suppressed at any time before its execution. In this review, functional interactions of the extended neural network subserving generation and inhibition of goal-directed movements will be outlined, leading to the intriguing hypothesis that the performance of actions and their suppression are not specified by independent sets of brain regions. Rather, it will be proposed that acting and stopping are functions emerging from specific interactions between largely overlapping brain regions, whose activity is intimately linked (directly or indirectly) to the evaluations of pros and cons of an action. Such mechanism would allow the brain to perform as a highly efficient and flexible system, as different functions could be computed exploiting the same components operating in different configurations.

Decoding methods for neural prostheses: where have we reached?

This article reviews advances in decoding methods for brain-machine interfaces (BMIs). Recent work has focused on practical considerations for future clinical deployment of prosthetics. This review is organized by open questions in the field such as what variables to decode, how to design neural tuning models, which neurons to select, how to design models of desired actions, how to learn decoder parameters during prosthetic operation, and how to adapt to changes in neural signals and neural tuning. The concluding discussion highlights the need to design and test decoders within the context of their expected use and the need to answer the question of how much control accuracy is good enough for a prosthetic.

Brain-machine interfaces can accelerate clarification of the principal mysteries and real plasticity of the brain

This perspective emphasizes that the brain-machine interface (BMI) research has the potential to clarify major mysteries of the brain and that such clarification of the mysteries by neuroscience is needed to develop BMIs. I enumerate five principal mysteries. The first is “how is information encoded in the brain?” This is the fundamental question for understanding what our minds are and is related to the verification of Hebb’s cell assembly theory. The second is “how is information distributed in the brain?” This is also a reconsideration of the functional localization of the brain. The third is “what is the function of the ongoing activity of the brain?” This is the problem of how the brain is active during no-task periods and what meaning such spontaneous activity has. The fourth is “how does the bodily behavior affect the brain function?” This is the problem of brain-body interaction, and obtaining a new “body” by a BMI leads to a possibility of changes in the owner’s brain. The last is “to what extent can the brain induce plasticity?” Most BMIs require changes in the brain’s neuronal activity to realize higher performance, and the neuronal operant conditioning inherent in the BMIs further enhances changes in the activity.