Towards online spike sorting for high-density neural probes using discriminative template matching with suppression of interfering spikes

An unsupervised real-time spike sorting system based on optimized OSort

Objective. The OSort algorithm, a pivotal unsupervised spike sorting method, has been implemented in dedicated hardware devices for real-time spike sorting. However, due to the inherent complexity of neural recording environments, OSort still grapples with numerous transient cluster occurrences during the practical sorting process. This leads to substantial memory usage, heavy computational load, and complex hardware architectures, especially in noisy recordings and multi-channel systems.Approach. This study introduces an optimized OSort algorithm (opt-OSort) which utilizes correlation coefficient (CC), instead of Euclidean distance as classification criterion. TheCCmethod not only bolsters the robustness of spike classification amidst the diverse and ever-changing conditions of physiological and recording noise environments, but also can finish the entire sorting procedure within a fixed number of cluster slots, thus preventing a large number of transient clusters. Moreover, the opt-OSort incorporates two configurable validation loops to efficiently reject cluster outliers and track recording variations caused by electrode drifting in real-time.Main results. The opt-OSort significantly reduces transient cluster occurrences by two orders of magnitude and decreases memory usage by 2.5-80 times in the number of pre-allocated transient clusters compared with other hardware implementations of OSort. The opt-OSort maintains an accuracy comparable to offline OSort and other commonly-used algorithms, with a sorting time of 0.68µs as measured by the hardware-implemented system in both simulated datasets and experimental data. The opt-OSort's ability to handle variations in neural activity caused by electrode drifting is also demonstrated.Significance. These results present a rapid, precise, and robust spike sorting solution suitable for integration into low-power, portable, closed-loop neural control systems and brain-computer interfaces.

A Neuromorphic Brain Interface based on RRAM Crossbar Arrays for High Throughput Real-time Spike Sorting

Real-time spike sorting and processing are crucial for closed-loop brain-machine interfaces and neural prosthetics. Recent developments in high-density multi-electrode arrays with hundreds of electrodes have enabled simultaneous recordings of spikes from a large number of neurons. However, the high channel count imposes stringent demands on real-time spike sorting hardware regarding data transmission bandwidth and computation complexity. Thus, it is necessary to develop a specialized real-time hardware that can sort neural spikes on the fly with high throughputs while consuming minimal power. Here, we present a real-time, low latency spike sorting processor that utilizes high-density CuO_x resistive crossbars to implement in-memory spike sorting in a massively parallel manner. We developed a fabrication process which is compatible with CMOS BEOL integration. We extensively characterized switching characteristics and statistical variations of the CuO_x memory devices. In order to implement spike sorting with crossbar arrays, we developed a template matching-based spike sorting algorithm that can be directly mapped onto RRAM crossbars. By using synthetic and in vivo recordings of extracellular spikes, we experimentally demonstrated energy efficient spike sorting with high accuracy. Our neuromorphic interface offers substantial improvements in area (~1000× less area), power (~200× less power), and latency (4.8μs latency for sorting 100 channels) for real-time spike sorting compared to other hardware implementations based on FPGAs and microcontrollers.

Objective evaluation of stimulation artefact removal techniques in the context of neural spike sorting

Objective - We present a framework to objectively test and compare stimulation artefact removal techniques in the context of neural spike sorting. Approach - To this end, we used realistic hybrid ground-truth spiking data, with superimposed artefacts from in vivo recordings. We used the framework to evaluate and compare several techniques: blanking, template subtraction by averaging, linear regression, and a multi-channel Wiener filter (MWF). Main results - Our study demonstrates that blanking and template subtraction result in a poorer spike sorting performance than linear regression and MWF, while the latter two perform similarly. Finally, to validate the conclusions found from the hybrid evaluation framework, we also performed a qualitative analysis on in vivo recordings without artificial manipulations. Significance - Our framework allows direct quantification of the impact of the residual artefact on the spike sorting accuracy, thereby allowing for a more objective and more relevant comparison compared to indirect signal quality metrics that are estimated from the signal statistics. Furthermore, the availability of a ground truth in the form of single-unit spiking activity also facilitates a better estimation of such signal quality metrics.

A tutorial on generalized eigendecomposition for denoising, contrast enhancement, and dimension reduction in multichannel electrophysiology

The goal of this paper is to present a theoretical and practical introduction to generalized eigendecomposition (GED), which is a robust and flexible framework used for dimension reduction and source separation in multichannel signal processing. In cognitive electrophysiology, GED is used to create spatial filters that maximize a researcher-specified contrast. For example, one may wish to exploit an assumption that different sources have different frequency content, or that sources vary in magnitude across experimental conditions. GED is fast and easy to compute, performs well in simulated and real data, and is easily adaptable to a variety of specific research goals. This paper introduces GED in a way that ties together myriad individual publications and applications of GED in electrophysiology, and provides sample MATLAB and Python code that can be tested and adapted. Practical considerations and issues that often arise in applications are discussed.

A data-driven spike sorting feature map for resolving spike overlap in the feature space

Objective.Spike sorting is the process of extracting neuronal action potentials, or spikes, from an extracellular brain recording, and assigning each spike to its putative source neuron. Spike sorting is usually treated as a clustering problem. However, this clustering process is known to be affected by overlapping spikes. Existing methods for resolving spike overlap typically require an expensive post-processing of the clustering results. In this paper, we propose the design of a domain-specific feature map, which enables the resolution of spike overlap directly in the feature space.Approach.The proposed domain-specific feature map is based on a neural network architecture that is trained to simultaneously perform spike sorting and spike overlap resolution. Overlapping spikes clusters can be identified in the feature space through a linear relation with the single-neuron clusters for which the neurons contribute to the overlapping spikes. To aid the feature map training, a data augmentation procedure is presented that is based on biophysical simulations.Main results.We demonstrate the potential of our method on independent and realistic test data. We show that our novel approach for resolving spike overlap generalizes to unseen and realistic test data. Furthermore, the sorting performance of our method is shown to be similar to the state-of-the-art, but our method does not assume the availability of spike templates for resolving spike overlap.Significance.Resolving spike overlap directly in the feature space, results in an overall simplified spike sorting pipeline compared to the state-of-the-art. For the state-of-the-art, the overlapping spike snippets exhibit a large spread in the feature space and do not appear as concentrated clusters. This can lead to biased spike template estimates which affect the sorting performance of the state-of-the-art. In our proposed approach, overlapping spikes form concentrated clusters and spike overlap resolution does not depend on the availability of spike templates.

SHYBRID: A Graphical Tool for Generating Hybrid Ground-Truth Spiking Data for Evaluating Spike Sorting Performance

Spike sorting is the process of retrieving the spike times of individual neurons that are present in an extracellular neural recording. Over the last decades, many spike sorting algorithms have been published. In an effort to guide a user towards a specific spike sorting algorithm, given a specific recording setting (i.e., brain region and recording device), we provide an open-source graphical tool for the generation of hybrid ground-truth data in Python. Hybrid ground-truth data is a data-driven modelling paradigm in which spikes from a single unit are moved to a different location on the recording probe, thereby generating a virtual unit of which the spike times are known. The tool enables a user to efficiently generate hybrid ground-truth datasets and make informed decisions between spike sorting algorithms, fine-tune the algorithm parameters towards the used recording setting, or get a deeper understanding of those algorithms.

Efficient neural spike sorting using data subdivision and unification

Neural spike sorting is prerequisite to deciphering useful information from electrophysiological data recorded from the brain, in vitro and/or in vivo. Significant advancements in nanotechnology and nanofabrication has enabled neuroscientists and engineers to capture the electrophysiological activities of the brain at very high resolution, data rate and fidelity. However, the evolution in spike sorting algorithms to deal with the aforementioned technological advancement and capability to quantify higher density data sets is somewhat limited. Both supervised and unsupervised clustering algorithms do perform well when the data to quantify is small, however, their efficiency degrades with the increase in the data size in terms of processing time and quality of spike clusters being formed. This makes neural spike sorting an inefficient process to deal with large and dense electrophysiological data recorded from brain. The presented work aims to address this challenge by providing a novel data pre-processing framework, which can enhance the efficiency of the conventional spike sorting algorithms significantly. The proposed framework is validated by applying on ten widely used algorithms and six large feature sets. Feature sets are calculated by employing PCA and Haar wavelet features on three widely adopted large electrophysiological datasets for consistency during the clustering process. A MATLAB software of the proposed mechanism is also developed and provided to assist the researchers, active in this domain.

Adaptive virtual referencing for the extraction of extracellularly recorded action potentials in noisy environments

OBJECTIVE

Removal of common mode noise and artifacts from extracellularly measured action potentials, herein referred to as spikes, recorded with multi-electrode arrays (MEAs) which included severe noise and artifacts generated by an ultrahigh field (UHF) 16.4 Tesla magnetic resonance imaging (MRI) scanner.

APPROACH

An adaptive virtual referencing (AVR) algorithm is used to remove artifacts and thus enable extraction of neural spike signals from extracellular recordings in anesthetized rat brains. A 16-channel MEA with 150-micron inter-site spacing is used, and a virtual reference is created by spatially averaging the 16 signal channels which results in a reference signal without extracellular spiking activity while preserving common mode noise and artifacts. This virtual reference signal is then used as the input to an adaptive FIR filter which optimally scales and time-shifts the reference to each specific electrode site to remove the artifacts and noise.

MAIN RESULTS

By removing artifacts and reducing noise, the neural spikes at each electrode site can be well extracted, even from data originally recorded with a high noise floor due to electromagnetic interference and artifacts generated by a 16.4T MRI scanner. The AVR method enables many more spikes to be detected than would otherwise be possible. Further, the filtered spike waveforms can be well separated from each other using PCA feature extraction and semi-supervised k-means clustering. While data in a 16.4T MRI scanner contains significantly more noise and artifacts, the developed AVR method enables similar data quality to be extracted as recorded on benchtop experiments outside the MRI scanner.

SIGNIFICANCE

AVR of extracellular spike signals recorded with MEAs has not been previously reported and fills a technical need by enabling low-noise extracellular spike extraction in noisy and challenging environments such as UHF MRI that will enable further study of neuro-vascular coupling at UHF.

Computationally-Efficient Algorithm for Real-Time Absence Seizure Detection in Wearable Electroencephalography

Advances in electroencephalography (EEG) equipment now allow monitoring of people with epilepsy in their daily-life environment. The large volumes of data that can be collected from long-term out-of-clinic monitoring require novel algorithms to process the recordings on board of the device to identify and log or transmit only relevant data epochs. Existing seizure-detection algorithms are generally designed for post-processing purposes, so that memory and computing power are rarely considered as constraints. We propose a novel multi-channel EEG signal processing method for automated absence seizure detection which is specifically designed to run on a microcontroller with minimal memory and processing power. It is based on a linear multi-channel filter that is precomputed offline in a data-driven fashion based on the spatial-temporal signature of the seizure and peak interference statistics. At run-time, the algorithm requires only standard linear filtering operations, which are cheap and efficient to compute, in particular on microcontrollers with a multiply-accumulate unit (MAC). For validation, a dataset of eight patients with juvenile absence epilepsy was collected. Patients were equipped with a 20-channel mobile EEG unit and discharged for a day-long recording. The algorithm achieves a median of 0.5 false detections per day at 95% sensitivity. We compare our algorithm with state-of-the-art absence seizure detection algorithms and conclude it performs on par with these at a much lower computational cost.

Stimulus-aware spatial filtering for single-trial neural response and temporal response function estimation in high-density EEG with applications in auditory research

A common problem in neural recordings is the low signal-to-noise ratio (SNR), particularly when using non-invasive techniques like magneto- or electroencephalography (M/EEG). To address this problem, experimental designs often include repeated trials, which are then averaged to improve the SNR or to infer statistics that can be used in the design of a denoising spatial filter. However, collecting enough repeated trials is often impractical and even impossible in some paradigms, while analyses on existing data sets may be hampered when these do not contain such repeated trials. Therefore, we present a data-driven method that takes advantage of the knowledge of the presented stimulus, to achieve a joint noise reduction and dimensionality reduction without the need for repeated trials. The method first estimates the stimulus-driven neural response using the given stimulus, which is then used to find a set of spatial filters that maximize the SNR based on a generalized eigenvalue decomposition. As the method is fully data-driven, the dimensionality reduction enables researchers to perform their analyses without having to rely on their knowledge of brain regions of interest, which increases accuracy and reduces the human factor in the results. In the context of neural tracking of a speech stimulus using EEG, our method resulted in more accurate short-term temporal response function (TRF) estimates, higher correlations between predicted and actual neural responses, and higher attention decoding accuracies compared to existing TRF-based decoding methods. We also provide an extensive discussion on the central role played by the generalized eigenvalue decomposition in various denoising methods in the literature, and address the conceptual similarities and differences with our proposed method.

Low-latency single channel real-time neural spike sorting system based on template matching

Recent technical advancements in neural engineering allow for precise recording and control of neural circuits simultaneously, opening up new opportunities for closed-loop neural control. In this work, a rapid spike sorting system was developed based on template matching to rapidly calculate instantaneous firing rates for each neuron in a multi-unit extracellular recording setting. Cluster templates were first generated by a desktop computer using a non-parameter spike sorting algorithm (Super-paramagnetic clustering) and then transferred to a field-programmable gate array digital circuit for rapid sorting through template matching. Two different matching techniques–Euclidean distance (ED) and correlational matching (CM)–were compared for the accuracy of sorting and the performance of calculating firing rates. The performance of the system was first verified using publicly available artificial data and was further confirmed with pre-recorded neural spikes from an anesthetized Mongolian gerbil. Real-time recording and sorting from an awake mouse were also conducted to confirm the system performance in a typical behavioral neuroscience experimental setting. Experimental results indicated that high sorting accuracies were achieved for both template-matching methods, but CM can better handle spikes with non-Gaussian spike distributions, making it more robust for in vivo recording. The technique was also compared to several other off-line spike sorting algorithms and the results indicated that the sorting accuracy is comparable but sorting time is significantly shorter than these other techniques. A low sorting latency of under 2 ms and a maximum spike sorting rate of 941 spikes/second have been achieved with our hybrid hardware/software system. The low sorting latency and fast sorting rate allow future system developments of neural circuit modulation through analyzing neural activities in real-time.

Cluster tendency assessment in neuronal spike data

Sorting spikes from extracellular recording into clusters associated with distinct single units (putative neurons) is a fundamental step in analyzing neuronal populations. Such spike sorting is intrinsically unsupervised, as the number of neurons are not known a priori. Therefor, any spike sorting is an unsupervised learning problem that requires either of the two approaches: specification of a fixed value k for the number of clusters to seek, or generation of candidate partitions for several possible values of c, followed by selection of a best candidate based on various post-clustering validation criteria. In this paper, we investigate the first approach and evaluate the utility of several methods for providing lower dimensional visualization of the cluster structure and on subsequent spike clustering. We also introduce a visualization technique called improved visual assessment of cluster tendency (iVAT) to estimate possible cluster structures in data without the need for dimensionality reduction. Experimental results are conducted on two datasets with ground truth labels. In data with a relatively small number of clusters, iVAT is beneficial in estimating the number of clusters to inform the initialization of clustering algorithms. With larger numbers of clusters, iVAT gives a useful estimate of the coarse cluster structure but sometimes fails to indicate the presumptive number of clusters. We show that noise associated with recording extracellular neuronal potentials can disrupt computational clustering schemes, highlighting the benefit of probabilistic clustering models. Our results show that t-Distributed Stochastic Neighbor Embedding (t-SNE) provides representations of the data that yield more accurate visualization of potential cluster structure to inform the clustering stage. Moreover, The clusters obtained using t-SNE features were more reliable than the clusters obtained using the other methods, which indicates that t-SNE can potentially be used for both visualization and to extract features to be used by any clustering algorithm.

A data-driven regularization approach for template matching in spike sorting with high-density neural probes

Spike sorting is the process of assigning neural spikes in an extracellular brain recording to their putative neurons. Optimal pre-whitened template matching filters that are used in spike sorting typically suffer from ill-conditioning. In this paper, we investigate the origin of this ill-conditioning and the way in which it influences the resulting filters. Two data-driven subspace regularization approaches are proposed, and those are shown to outperform a regularization approach used in recent literature. The comparison of the methods is based on ground truth data that are recorded in-vivo.

Accurate Estimation of Neural Population Dynamics without Spike Sorting

A central goal of systems neuroscience is to relate an organism's neural activity to behavior. Neural population analyses often reduce the data dimensionality to focus on relevant activity patterns. A major hurdle to data analysis is spike sorting, and this problem is growing as the number of recorded neurons increases. Here, we investigate whether spike sorting is necessary to estimate neural population dynamics. The theory of random projections suggests that we can accurately estimate the geometry of low-dimensional manifolds from a small number of linear projections of the data. We recorded data using Neuropixels probes in motor cortex of nonhuman primates and reanalyzed data from three previous studies and found that neural dynamics and scientific conclusions are quite similar using multiunit threshold crossings rather than sorted neurons. This finding unlocks existing data for new analyses and informs the design and use of new electrode arrays for laboratory and clinical use.

Reliability of motor and sensory neural decoding by threshold crossings for intracortical brain-machine interface

OBJECTIVE

For intracortical neurophysiological studies, spike sorting is an important procedure to isolate single units for analyzing specific functions. However, whether spike sorting is necessary or not for neural decoding applications is controversial. Several studies showed that using threshold crossings (TC) instead of spike sorting could also achieve a similar satisfactory performance. However, such studies were limited in similar behavioral tasks, and the neural signal source mainly focused on the motor-related cortical regions. It is not certain if this conclusion is applicable to other situations. Therefore, we compared the performance of TC and spike sorting in neural decoding with more comprehensive paradigms and parameters.

APPROACH

Two rhesus macaques implanted with Utah or floating microelectrode arrays (FMAs) in motor or sensory-related cortical regions were trained to perform a motor or a sensory task. Data from each monkey were preprocessed with three different schemes: TC, automatic sorting (AS), and manual sorting (MS). A support vector machine was used as the decoder, and the decoding accuracy was used for evaluating the performance of three preprocessing methods. Different neural signal sources, different decoders, and related parameters and decoding stability were further tested to systematically compare three preprocessing methods.

MAIN RESULTS

TC could achieve a similar (-4.5 RMS threshold) or better (-3.0 RMS threshold) decoding performance compared to the other two sorting methods in the motor or sensory tasks even if the neural signal sources or decoder-related parameters were changed. Moreover, TC was much more stable in neural decoding across sessions and robust to changes of threshold.

SIGNIFICANCE

Our results indicated that spike-firing patterns could be stably extracted through TC from multiple cortices in both motor and sensory neural decoding applications. Considering the stability of TC, it might be more suitable for neural decoding compared to sorting methods.

Spike detection and spike sorting with a hidden Markov model improves offline decoding of motor cortical recordings

OBJECTIVE

Detection and sorting (classification) of action potentials from extracellular recordings are two important pre-processing steps for brain-computer interfaces (BCIs) and some neuroscientific studies. Traditional approaches perform these two steps serially, but using shapes of action potential waveforms during detection, i.e. combining the two steps, may lead to better performance, especially during high noise. We propose a hidden Markov model (HMM) based method for combined detecting and sorting of spikes, with the aim of improving the final decoding accuracy of BCIs.

APPROACH

The states of the HMM indicate whether there is a spike, what unit a spike belongs to, and the time course within a waveform. The HMM outputs probabilities of spike detection, and from this we can calculate expectations of spike counts in time bins, which can replace integer spike counts as input to BCI decoders. We evaluate the HMM method on simulated spiking data. We then examine the impact of using this method on decoding real neural data recorded from primary motor cortex of two Rhesus monkeys.

MAIN RESULTS

Our comparisons on simulated data to detection-then-sorting approaches and combined detection-and-sorting algorithms indicate that the HMM method performs more accurately at detection and sorting (0.93 versus 0.73 spike count correlation, 0.73 versus 0.49 adjusted mutual information). On real neural data, the HMM method led to higher adjusted mutual information between spike counts and kinematics (monkey K: 0.034 versus 0.027; monkey M: 0.033 versus 0.022) and better neuron encoding model predictions (K: 0.016 dB improvement; M: 0.056 dB improvement). Lastly, the HMM method facilitated higher offline decoding accuracy (Kalman filter, K: 8.5% mean squared error reduction, M: 18.6% reduction).

SIGNIFICANCE

The HMM spike detection and sorting method offers a new approach to spike pre-processing for BCIs and neuroscientific studies.

Unsupervised and real-time spike sorting chip for neural signal processing in hippocampal prosthesis

BACKGROUND

Damage to the hippocampus will result in the loss of ability to form new long-term memories and cognitive disorders. At present, there is no effective medical treatment for this issue. Hippocampal cognitive prosthesis is proposed to replace damaged regions of the hippocampus to mimic the function of original biological tissue. This prosthesis requires a spike sorter to detect and classify spikes in the recorded neural signal.

NEW METHOD

A 16-channel spike sorting processor is presented in this paper, where all channels are considered as independent. An automatic threshold estimation method suitable for hardware implementation is proposed for the Osort clustering algorithm. A new distance metric is also introduced to facilitate clustering. Bayes optimal template matching classification algorithm is optimized to reduce computational complexity by introducing a preselection mechanism.

RESULTS

The chip was fabricated in 40-nm CMOS process with a core area of 0.0175 mm²/ch and power consumption of 19.0 μW/ch. Synthetic and realistic test data are used to evaluate the chip. The test result shows that it has high performance on both data.

COMPARISON WITH EXISTING METHOD(S)

Compared with the other three spike sorting processors, the proposed chip achieves the highest detection and classification accuracy. It also has the ability to deal with partially overlapping spikes, which is not reported in the other work.

CONCLUSIONS

We have developed a 16-channel spike sorting chip used in hippocampal prosthesis, which provides unsupervised clustering and real-time detection and classification. It also has the ability to deal with partially overlapping spikes.