A Hardware-Efficient Scalable Spike Sorting Neural Signal Processor Module for Implantable High-Channel-Count Brain Machine Interfaces

Machine learning decoding of single neurons in the thalamus for speech brain-machine interfaces

Objective. Our goal is to decode firing patterns of single neurons in the left ventralis intermediate nucleus (Vim) of the thalamus, related to speech production, perception, and imagery. For realistic speech brain-machine interfaces (BMIs), we aim to characterize the amount of thalamic neurons necessary for high accuracy decoding.Approach. We intraoperatively recorded single neuron activity in the left Vim of eight neurosurgical patients undergoing implantation of deep brain stimulator or RF lesioning during production, perception and imagery of the five monophthongal vowel sounds. We utilized the Spade decoder, a machine learning algorithm that dynamically learns specific features of firing patterns and is based on sparse decomposition of the high dimensional feature space.Main results. Spade outperformed all algorithms compared with, for all three aspects of speech: production, perception and imagery, and obtained accuracies of 100%, 96%, and 92%, respectively (chance level: 20%) based on pooling together neurons across all patients. The accuracy was logarithmic in the amount of neurons for all three aspects of speech. Regardless of the amount of units employed, production gained highest accuracies, whereas perception and imagery equated with each other.Significance. Our research renders single neuron activity in the left Vim a promising source of inputs to BMIs for restoration of speech faculties for locked-in patients or patients with anarthria or dysarthria to allow them to communicate again. Our characterization of how many neurons are necessary to achieve a certain decoding accuracy is of utmost importance for planning BMI implantation.

Calibration-Free and Hardware-Efficient Neural Spike Detection for Brain Machine Interfaces

Recent translational efforts in brain-machine interfaces (BMI) are demonstrating the potential to help people with neurological disorders. The current trend in BMI technology is to increase the number of recording channels to the thousands, resulting in the generation of vast amounts of raw data. This in turn places high bandwidth requirements for data transmission, which increases power consumption and thermal dissipation of implanted systems. On-implant compression and/or feature extraction are therefore becoming essential to limiting this increase in bandwidth, but add further power constraints - the power required for data reduction must remain less than the power saved through bandwidth reduction. Spike detection is a common feature extraction technique used for intracortical BMIs. In this article, we develop a novel firing-rate-based spike detection algorithm that requires no external training and is hardware efficient and therefore ideally suited for real-time applications. Key performance and implementation metrics such as detection accuracy, adaptability in chronic deployment, power consumption, area utilization, and channel scalability are benchmarked against existing methods using various datasets. The algorithm is first validated using a reconfigurable hardware (FPGA) platform and then ported to a digital ASIC implementation in both 65 nm and 0.18 μm CMOS technologies. The 128-channel ASIC design implemented in a 65 nm CMOS technology occupies 0.096 mm2 silicon area and consumes 4.86 μW from a 1.2 V power supply. The adaptive algorithm achieves a 96% spike detection accuracy on a commonly used synthetic dataset, without the need for any prior training.

Partially binarized neural networks for efficient spike sorting

While brain-implantable neural spike sorting can be realized using efficient algorithms, the presence of noise may make it difficult to maintain high-peformance sorting using conventional techniques. In this article, we explore the use of partially binarized neural networks (PBNNs), to the best of our knowledge for the first time, for sorting of neural spike feature vectors. It is shown that compared to the waveform template-based methods, PBNNs offer robust spike sorting over various datasets and noise levels. The ASIC implementation of the PBNN-based spike sorting system in a standard 180-nm CMOS process is presented. The post place and route simulations results show that the synthesized PBNN consumes only 0.59 μ W of power from a 1.8 V supply while operating at 24 kHz and occupies 0.15 mm 2 of silicon area. It is shown that the designed PBNN-based spike sorting system not only offers comparable accuracy to the state-of-the-art spike sorting systems over various noise levels and datasets, it also occupies a smaller silicon area and consumes less power and energy. This makes PBNNs a viable alternative towards the implementation of brain-implantable spike sorting systems.

From End to End: Gaining, Sorting, and Employing High-Density Neural Single Unit Recordings

The meaning behind neural single unit activity has constantly been a challenge, so it will persist in the foreseeable future. As one of the most sourced strategies, detecting neural activity in high-resolution neural sensor recordings and then attributing them to their corresponding source neurons correctly, namely the process of spike sorting, has been prevailing so far. Support from ever-improving recording techniques and sophisticated algorithms for extracting worthwhile information and abundance in clustering procedures turned spike sorting into an indispensable tool in electrophysiological analysis. This review attempts to illustrate that in all stages of spike sorting algorithms, the past 5 years innovations' brought about concepts, results, and questions worth sharing with even the non-expert user community. By thoroughly inspecting latest innovations in the field of neural sensors, recording procedures, and various spike sorting strategies, a skeletonization of relevant knowledge lays here, with an initiative to get one step closer to the original objective: deciphering and building in the sense of neural transcript.

A 16-Channel Fully Configurable Neural SoC With 1.52 μW/Ch Signal Acquisition, 2.79 μW/Ch Real-Time Spike Classifier, and 1.79 TOPS/W Deep Neural Network Accelerator in 22 nm FDSOI

With the advent of high-density micro-electrodes arrays, developing neural probes satisfying the real-time and stringent power-efficiency requirements becomes more challenging. A smart neural probe is an essential device in future neuroscientific research and medical applications. To realize such devices, we present a 22 nm FDSOI SoC with complex on-chip real-time data processing and training for neural signal analysis. It consists of a digitally-assisted 16-channel analog front-end with 1.52 μW/Ch, dedicated bio-processing accelerators for spike detection and classification with 2.79 μW/Ch, and a 125 MHz RISC-V CPU, utilizing adaptive body biasing at 0.5 V with a supporting 1.79 TOPS/W MAC array. The proposed SoC shows a proof-of-concept of how to realize a high-level integration of various on-chip accelerators to satisfy the neural probe requirements for modern applications.

A Vector Quantization-Based Spike Compression Approach Dedicated to Multichannel Neural Recording Microsystems

Implantable high-density multichannel neural recording microsystems provide simultaneous recording of brain activities. Wireless transmission of the entire recorded data causes high bandwidth usage, which is not tolerable for implantable applications. As a result, a hardware-friendly compression module is required to reduce the amount of data before it is transmitted. This paper presents a novel compression approach that utilizes a spike extractor and a vector quantization (VQ)-based spike compressor. In this approach, extracted spikes are vector quantized using an unsupervised learning process providing a high spike compression ratio (CR) of 10-80. A combination of extracting and compressing neural spikes results in a significant data reduction as well as preserving the spike waveshapes. The compression performance of the proposed approach was evaluated under variant conditions. We also developed new architectures such that the hardware blocks of our approach can be implemented more efficiently. The compression module was implemented in a 180-nm standard CMOS process achieving a SNDR of 14.49[Formula: see text]dB and a classification accuracy (CA) of 99.62% at a CR of 20, while consuming 4[Formula: see text][Formula: see text]W power and 0.16[Formula: see text]mm² chip area per channel.

A Low Power 1024-Channels Spike Detector Using Latch-Based RAM for Real-Time Brain Silicon Interfaces

High-density microelectrode arrays allow the neuroscientist to study a wider neurons population, however, this causes an increase of communication bandwidth. Given the limited resources available for an implantable silicon interface, an on-fly data reduction is mandatory to stay within the power/area constraints. This can be accomplished by implementing a spike detector aiming at sending only the useful information about spikes. We show that the novel non-linear energy operator called ASO in combination with a simple but robust noise estimate, achieves a good trade-off between performance and consumption. The features of the investigated technique make it a good candidate for implantable BMIs. Our proposal is tested both on synthetic and real datasets providing a good sensibility at low SNR. We also provide a 1024-channels VLSI implementation using a Random-Access Memory composed by latches to reduce as much as possible the power consumptions. The final architecture occupies an area of 2.3 mm2, dissipating 3.6 µW per channels. The comparison with the state of art shows that our proposal finds a place among other methods presented in literature, certifying its suitability for BMIs.

A Multi-Channel Spike Sorting Processor With Accurate Clustering Algorithm Using Convolutional Autoencoder

This paper presents a spike sorting processor based on an accurate spike clustering algorithm. The proposed spike sorting algorithm employs an L2-normalized convolutional autoencoder to extract features from the input, where the autoencoder is trained using the proposed spike sorting-aware loss. In addition, we propose a similarity-based K-means clustering algorithm that conditionally updates the means by observing the cosine similarity. The modified K-means algorithm exhibits better convergence and enables online clustering with higher classification accuracy. We implement a spike sorting processor based on the proposed algorithm using an efficient time-multiplexed hardware architecture in a 40-nm CMOS process. Experimental results show that the processor consumes 224.75μW/mm² when processing 16 input channels at 7.68 MHz and 0.55 V. Our design achieves 95.54% clustering accuracy, outperforming prior spike sorting processor designs.

CluSem: Accurate clustering-based ensemble method to predict motor imagery tasks from multi-channel EEG data

BACKGROUND

The classification of motor imagery electroencephalogram (MI-EEG) is a pivotal task in the biosignal classification process in the brain-computer interface (BCI) applications. Currently, this bio-engineering-based technology is being employed by researchers in various fields to develop cutting-edge applications. The classification of real-time MI-EEG signals is the most challenging task in these applications. The prediction performance of the existing classification methods is still limited due to the high dimensionality and dynamic behaviors of the real-time EEG data.

PROPOSED METHOD

To enhance the classification performance of real-time BCI applications, this paper presents a new clustering-based ensemble technique called CluSem to mitigate this problem. We also develop a new brain game called CluGame using this method to evaluate the classification performance of real-time motor imagery movements. In this game, real-time EEG signal classification and prediction tabulation through animated balls are controlled via threads. By playing this game, users can control the movements of the balls via the brain signals of motor imagery movements without using any traditional input devices.

RESULTS

Our results demonstrate that CluSem is able to improve the classification accuracy between 5% and 15% compared to the existing methods on our collected as well as the publicly available EEG datasets. The source codes used to implement CluSem and CluGame are publicly available at https://github.com/MdOchiuddinMiah/MI-BCI_ML.

Closed-Loop Neural Prostheses With On-Chip Intelligence: A Review and a Low-Latency Machine Learning Model for Brain State Detection

The application of closed-loop approaches in systems neuroscience and therapeutic stimulation holds great promise for revolutionizing our understanding of the brain and for developing novel neuromodulation therapies to restore lost functions. Neural prostheses capable of multi-channel neural recording, on-site signal processing, rapid symptom detection, and closed-loop stimulation are critical to enabling such novel treatments. However, the existing closed-loop neuromodulation devices are too simplistic and lack sufficient on-chip processing and intelligence. In this paper, we first discuss both commercial and investigational closed-loop neuromodulation devices for brain disorders. Next, we review state-of-the-art neural prostheses with on-chip machine learning, focusing on application-specific integrated circuits (ASIC). System requirements, performance and hardware comparisons, design trade-offs, and hardware optimization techniques are discussed. To facilitate a fair comparison and guide design choices among various on-chip classifiers, we propose a new energy-area (E-A) efficiency figure of merit that evaluates hardware efficiency and multi-channel scalability. Finally, we present several techniques to improve the key design metrics of tree-based on-chip classifiers, both in the context of ensemble methods and oblique structures. A novel Depth-Variant Tree Ensemble (DVTE) is proposed to reduce processing latency (e.g., by 2.5× on seizure detection task). We further develop a cost-aware learning approach to jointly optimize the power and latency metrics. We show that algorithm-hardware co-design enables the energy- and memory-optimized design of tree-based models, while preserving a high accuracy and low latency. Furthermore, we show that our proposed tree-based models feature a highly interpretable decision process that is essential for safety-critical applications such as closed-loop stimulation.

A 10.8 µW Neural Signal Recorder and Processor With Unsupervised Analog Classifier for Spike Sorting

Implantable brain machine interfaces for treatment of neurological disorders require on-chip, real-time signal processing of action potentials (spikes). In this work, we present the first spike sorting SoC with integrated neural recording front-end and analog unsupervised classifier. The event-driven, low power spike sorter features a novel hardware-optimized, K-means based algorithm that effectively eliminates duplicate clusters and is implemented using a novel clockless and ADC-less analog architecture. The 1.4 mm² chip is fabricated in a 180-nm CMOS SOI process. The analog front-end achieves a 3.3 μV_rms noise floor over the spike bandwidth (400 - 5000 Hz) and consumes 6.42 μW from a 1.5 V supply. The analog spike sorter consumes 4.35 μW and achieves 93.2% classification accuracy on a widely used synthetic test dataset. In addition, higher than 93% agreement between the chip classification result and that of a standard spike sorting software is observed using pre-recorded real neural signals. Simulations of the implemented spike sorter show robust performance under process-voltage-temperature variations.

Neural Spike Sorting Using Binarized Neural Networks

This article presents the design and efficient hardware implementation of binarized neural networks (BNNs) for brain-implantable neural spike sorting. In contrast to the conventional artificial neural networks (ANNs), in which the weights and activation functions of neurons are represented using real values, the BNNs utilize binarized weights and activation functions to dramatically reduce the memory requirement and computational complexity of the ANNs. The designed BNN is trained using several realistic neural datasets to verify its accuracy for neural spike sorting. The application-specific integrated circuit (ASIC) implementation of the designed BNN in a standard 0.18- [Formula: see text] CMOS process occupies 0.33 mm ² of silicon area. Power consumption estimation of the ASIC layout shows that the BNN dissipates [Formula: see text] of power from a 1.8 V supply while operating at 24 kHz. The designed BNN-based spike sorting system is also implemented on a field-programmable gate array and is shown to reduce the required on-chip memory by 89% compared to those of the alternative state-of-the-art spike sorting systems. To the best of our knowledge, this is the first work employing BNNs for real-time in vivo neural spike sorting.

A framework for on-implant spike sorting based on salient feature selection

On-implant spike sorting methods employ static feature extraction/selection techniques to minimize the hardware cost. Here we propose a novel framework for real-time spike sorting based on dynamic selection of features. We select salient features that maximize the geometric-mean of between-class distances as well as the associated homogeneity index effectively to best discriminate spikes for classification. Wave-shape classification is performed based on a multi-label window discrimination approach. An external module calculates the salient features and discrimination windows through optimizing a replica of the on-implant operation, and then configures the on-implant spike sorter for real-time online operation. Hardware implementation of the on-implant online spike sorter for 512 channels of concurrent extra-cellular neural signals is reported, with an average classification accuracy of ~88%. Compared with other similar methods, our method shows reduction in classification error by a factor of ~2, and also reduction in the required memory space by a factor of ~5.

Charge-Redistribution Based Quadratic Operators for Neural Feature Extraction

This paper presents a SAR converter based mixed-signal multiplier for the feature extraction of neural signals using quadratic operators. After a thorough analysis of design principles and circuit-level aspects, the proposed architecture is explored for the implementation of two quadratic operators often used for the characterization of neural activity, the moving average energy (MAE) operator and the nonlinear energy operator (NEO). Programmable chips for both operators have been implemented in a HV-180 nm CMOS process. Experimental results confirm their suitability for energy computation and action potential detection and the accomplished area×power performance is compared to prior art. The MAE and NEO prototypes, at a sampling rate of 30kS/s, consume 116 nW and 178 nW, respectively, and digitize both the input neural signal and the operator outcome, with no need for digital multipliers.

A Real-Time Spike Sorting System Using Parallel OSort Clustering

This article presents an efficient design and implementation of a real-time spike sorting system using unsupervised clustering. We utilize the online sorting (OSort) algorithm and model it first in both floating-point and fixed-point numerical representations to accurately assess the feasibility of our hardware architecture and also reliably analyze the sorting accuracy. For efficient hardware realization of OSort, we propose a modified parallel OSort algorithm. By reducing the number of required memory accesses, the number of computations performed for the management and upkeep of cluster averages and cluster merging is substantially reduced. By limiting the number of supported clusters per channel, the classification/clustering latency is significantly reduced compared to the previously published work, making the designed OSort system applicable for in-vivo spike sorting. The proposed OSort hardware architecture utilizes a novel memory configuration scheme to parallelize the OSort algorithm, which allows us to avoid using relatively large memory queues for storing detected spike waveforms and process them concurrently to the spike cluster management. The characteristics and implementation results of the designed OSort-based spike sorting system on a Xilinx Artix-7 field-programmable gate array (FPGA) are presented. The ASIC implementation of the designed system is estimated to occupy 2.57 mm² in a standard 32-nm CMOS process. Post-layout power estimation shows that the ASIC will dissipate 2.78 mW, while operating at 24 kHz.

Frameworks for Efficient Brain-Computer Interfacing

One challenge present in brain-computer interface (BCI) circuits is finding a balance between real-time on-chip processing in-vivo and wireless transmission of neural signals for off-chip in-silico processing. This article presents three potential frameworks for investigating an area- and energy-efficient realization of BCI circuits. The first framework performs spike detection on the filtered neural signal on a brain-implantable chip and only transmits detected spikes wirelessly for offline classification and decoding. The second framework performs in-vivo compression of the on-chip detected spikes prior to wireless transmission for substantially reducing wireless transmission overhead. The third framework performs spike sorting in-vivo on the brain-implantable chip to classify detected spikes on-chip and hence, even further reducing wireless data transmission rate at the expense of more signal processing. To alleviate the on-chip computation of spike sorting and also utilizing a more area- and energy-effective design, this work employs, for the first time, to the best of our knowledge, an artificial neural network (ANN) instead of using relatively computationally-intensive conventional spike sorting algorithms. The ASIC implementation results of the designed frameworks are presented and their feasibility for efficient in-vivo processing of neural signals is discussed. Compared to the previously-published BCI systems, the presented frameworks reduce the area and power consumption of implantable circuits.

BioWolf: A Sub-10-mW 8-Channel Advanced Brain-Computer Interface Platform With a Nine-Core Processor and BLE Connectivity

Advancements in digital signal processing (DSP) and machine learning techniques have boosted the popularity of brain-computer interfaces (BCIs), where electroencephalography is a widely accepted method to enable intuitive human-machine interaction. Nevertheless, the evolution of such interfaces is currently hampered by the unavailability of embedded platforms capable of delivering the required computational power at high energy efficiency and allowing for a small and unobtrusive form factor. To fill this gap, we developed BioWolf, a highly wearable (40 mm × 20 mm × 2 mm) BCI platform based on Mr. Wolf, a parallel ultra low power system-on-chip featuring nine RISC-V cores with DSP-oriented instruction set extensions. BioWolf also integrates a commercial 8-channel medical-grade analog-to-digital converter, and an ARM-Cortex M4 microcontroller unit (MCU) with bluetooth low-energy connectivity. To demonstrate the capabilities of the system, we implemented and tested a BCI featuring canonical correlation analysis (CCA) of steady-state visual evoked potentials. The system achieves an average information transfer rate of 1.46 b/s (aligned with the state-of-the-art of bench-top systems). Thanks to the reduced power envelope of the digital computational platform, which consumes less than the analog front-end, the total power budget is just 6.31 mW, providing up to 38 h operation (65 mAh battery). To our knowledge, our design is the first to explore the significant energy boost of a parallel MCU with respect to single-core MCUs for CCA-based BCI.

A data-driven regularization approach for template matching in spike sorting with high-density neural probes

Spike sorting is the process of assigning neural spikes in an extracellular brain recording to their putative neurons. Optimal pre-whitened template matching filters that are used in spike sorting typically suffer from ill-conditioning. In this paper, we investigate the origin of this ill-conditioning and the way in which it influences the resulting filters. Two data-driven subspace regularization approaches are proposed, and those are shown to outperform a regularization approach used in recent literature. The comparison of the methods is based on ground truth data that are recorded in-vivo.

An Efficient Hardware Architecture for Template Matching-Based Spike Sorting

This paper presents an efficient hardware architecture for the design and implementation of a spike sorting system using online template matching. Over the past decade, various spike sorting algorithms have been proposed; however, due to their computational complexity, they may not be suitable for implantable devices that have stringent area and power consumption requirements. We first developed a software-based spike sorting system in both floating-point and fixed-point representations. Then, we used our developed software-based spike sorting system for: 1) studying various neural signal processing algorithms and assessing their feasibility for efficient hardware implementations; and 2) offline processing of previously recorded neural data and extracting the threshold data and spike templates for configuring our spike sorting hardware architecture. The characteristics and implementation results of the designed spike sorting system on a Xilinx Artix-7 A200TFBG676-2 field-programmable gate array are presented. The application-specific integrated circuit (ASIC) implementation of the designed spike sorting system is estimated to occupy 0.3 mm². Postlayout synthesis and simulation shows that the ASIC implementation will dissipate 64 nW from a 0.25-V supply, while operating at a 24-kHz frequency in a standard 45-nm CMOS technology. Compared to the previously published work, our ASIC implementation consumes 96.8% less power, while maintaining a comparable sorting accuracy. Moreover, our design can run at a higher clock frequency and uses fewer hardware resources, while achieving a 168% reduction in output data rate when comparing the raw data sampling rate and the sorted spike output rate and, yet, offers comparable spike sorting accuracy.

An Adaptive Neural Spike Processor With Embedded Active Learning for Improved Unsupervised Sorting Accuracy

There is a need for integrated spike sorting processors in implantable devices with low power consumption that have improved accuracy. Learning the characteristics of the variable input neural signals and adapting the functionality of the sorting process can improve the accuracy. An adaptive spike sorting processor is presented accounting for the variation in the input signal noise characteristics and the variable difficulty in the selection of the spike characteristics, which significantly improves the accuracy. The adaptive spike processor was fabricated in 180-nm CMOS technology for proof of concept. It performs conditional detection, alignment, adaptive feature extraction, and online clustering with sorting threshold self-tuning capability. The chip was tested under different input signal conditions to demonstrate its adaptation capability providing a median classification accuracy of 84.5% and consuming 148 μW from a 1.8 V supply voltage.