1
|
Hong N, Kim B, Lee J, Choe HK, Jin KH, Kang H. Machine learning-based high-frequency neuronal spike reconstruction from low-frequency and low-sampling-rate recordings. Nat Commun 2024; 15:635. [PMID: 38245509 PMCID: PMC10799928 DOI: 10.1038/s41467-024-44794-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 01/03/2024] [Indexed: 01/22/2024] Open
Abstract
Recording neuronal activity using multiple electrodes has been widely used to understand the functional mechanisms of the brain. Increasing the number of electrodes allows us to decode more variety of functionalities. However, handling massive amounts of multichannel electrophysiological data is still challenging due to the limited hardware resources and unavoidable thermal tissue damage. Here, we present machine learning (ML)-based reconstruction of high-frequency neuronal spikes from subsampled low-frequency band signals. Inspired by the equivalence between high-frequency restoration and super-resolution in image processing, we applied a transformer ML model to neuronal data recorded from both in vitro cultures and in vivo male mouse brains. Even with the x8 downsampled datasets, our trained model reasonably estimated high-frequency information of spiking activity, including spike timing, waveform, and network connectivity. With our ML-based data reduction applicable to existing multichannel recording hardware while achieving neuronal signals of broad bandwidths, we expect to enable more comprehensive analysis and control of brain functions.
Collapse
Affiliation(s)
- Nari Hong
- Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea
- Information and Communication Engineering Research Center, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea
| | - Boil Kim
- Department of Brain Sciences, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea
| | - Jaewon Lee
- Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea
- Information and Communication Engineering Research Center, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea
| | - Han Kyoung Choe
- Department of Brain Sciences, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea
| | - Kyong Hwan Jin
- Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea.
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea.
| | - Hongki Kang
- Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea.
- Information and Communication Engineering Research Center, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, 42988, Republic of Korea.
| |
Collapse
|
2
|
Yan P, Akhoundi A, Shah NP, Tandon P, Muratore DG, Chichilnisky EJ, Murmann B. Data Compression Versus Signal Fidelity Tradeoff in Wired-OR Analog-to-Digital Compressive Arrays for Neural Recording. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2023; 17:754-767. [PMID: 37402181 DOI: 10.1109/tbcas.2023.3292058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/06/2023]
Abstract
Future high-density and high channel count neural interfaces that enable simultaneous recording of tens of thousands of neurons will provide a gateway to study, restore and augment neural functions. However, building such technology within the bit-rate limit and power budget of a fully implantable device is challenging. The wired-OR compressive readout architecture addresses the data deluge challenge of a high channel count neural interface using lossy compression at the analog-to-digital interface. In this article, we assess the suitability of wired-OR for several steps that are important for neuroengineering, including spike detection, spike assignment and waveform estimation. For various wiring configurations of wired-OR and assumptions about the quality of the underlying signal, we characterize the trade-off between compression ratio and task-specific signal fidelity metrics. Using data from 18 large-scale microelectrode array recordings in macaque retina ex vivo, we find that for an event SNR of 7-10, wired-OR correctly detects and assigns at least 80% of the spikes with at least 50× compression. The wired-OR approach also robustly encodes action potential waveform information, enabling downstream processing such as cell-type classification. Finally, we show that by applying an LZ77-based lossless compressor (gzip) to the output of the wired-OR architecture, 1000× compression can be achieved over the baseline recordings.
Collapse
|
3
|
Wang M, Zhang L, Yu H, Chen S, Zhang X, Zhang Y, Gao D. A deep learning network based on CNN and sliding window LSTM for spike sorting. Comput Biol Med 2023; 159:106879. [PMID: 37080004 DOI: 10.1016/j.compbiomed.2023.106879] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 02/08/2023] [Accepted: 03/30/2023] [Indexed: 04/05/2023]
Abstract
Spike sorting plays an essential role to obtain electrophysiological activity of single neuron in the fields of neural signal decoding. With the development of electrode array, large numbers of spikes are recorded simultaneously, which rises the need for accurate automatic and generalization algorithms. Hence, this paper proposes a spike sorting model with convolutional neural network (CNN) and a spike classification model with combination of CNN and Long-Short Term Memory (LSTM). The recall rate of our detector could reach 94.40% in low noise level dataset. Although the recall declined with the increasing noise level, our model still presented higher feasibility and better robustness than other models. In addition, the results of our classification model presented an accuracy of greater than 99% in simulated data and an average accuracy of about 95% in experimental data, suggesting our classifier outperforms the current "WMsorting" and other deep learning models. Moreover, the performance of our whole algorithm was evaluated through simulated data and the results shows that the accuracy of spike sorting reached about 97%. It is noteworthy to say that, this proposed algorithm could be used to achieve accurate and robust automated spike detection and spike classification.
Collapse
Affiliation(s)
- Manqing Wang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China; School of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, 611731, China.
| | - Liangyu Zhang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China
| | - Haixiang Yu
- School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China
| | - Siyu Chen
- School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China
| | - Xiaomeng Zhang
- Gingko College of Hospitality Management, Chengdu, 611730, China
| | - Yongqing Zhang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China
| | - Dongrui Gao
- School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China; School of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, 611731, China.
| |
Collapse
|
4
|
Sun B, Mu C, Wu Z, Zhu X. Training-Free Deep Generative Networks for Compressed Sensing of Neural Action Potentials. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:5190-5199. [PMID: 33830927 DOI: 10.1109/tnnls.2021.3069436] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Energy consumption is an important issue for resource-constrained wireless neural recording applications with limited data bandwidth. Compressed sensing (CS) is a promising framework for addressing this challenge because it can compress data in an energy-efficient way. Recent work has shown that deep neural networks (DNNs) can serve as valuable models for CS of neural action potentials (APs). However, these models typically require impractically large datasets and computational resources for training, and they do not easily generalize to novel circumstances. Here, we propose a new CS framework, termed APGen, for the reconstruction of APs in a training-free manner. It consists of a deep generative network and an analysis sparse regularizer. We validate our method on two in vivo datasets. Even without any training, APGen outperformed model-based and data-driven methods in terms of reconstruction accuracy, computational efficiency, and robustness to AP overlap and misalignment. The computational efficiency of APGen and its ability to perform without training make it an ideal candidate for long-term, resource-constrained, and large-scale wireless neural recording. It may also promote the development of real-time, naturalistic brain-computer interfaces.
Collapse
|
5
|
Radmanesh M, Rezaei AA, Jalili M, Hashemi A, Goudarzi MM. Online spike sorting via deep contractive autoencoder. Neural Netw 2022; 155:39-49. [DOI: 10.1016/j.neunet.2022.08.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Revised: 07/03/2022] [Accepted: 08/01/2022] [Indexed: 11/16/2022]
|
6
|
Classification of overlapping spikes using convolutional neural networks and long short term memory. Comput Biol Med 2022; 148:105888. [PMID: 35872414 DOI: 10.1016/j.compbiomed.2022.105888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 06/29/2022] [Accepted: 07/16/2022] [Indexed: 11/21/2022]
Abstract
Spike sorting is one of the key techniques to understand brain activity. In this paper, we propose a novel deep learning approach based on convolutional neural networks (CNN) and long short term memory (LSTM) to implement overlapping spike sorting. The results of the simulated data demonstrated that the clustering accuracy was greater than 99.9% and 99.0% for non-overlapping spikes and overlapping spikes, respectively. Moreover, the proposed method performed better than our previous deep learning approach named "1D-CNN". In addition, the experimental data recorded from the primary visual cortex of a macaque monkey were used to evaluate the proposed method in a practical application. It was shown that the method could successfully isolate most overlapping spikes of different neurons (ranging from two to five). In summary, the CNN + LSTM method proposed in this paper is of great advantage for overlapping spike sorting with high accuracy. It lays the foundation for application in more challenging works, such as distinguishing the simultaneous recordings of multichannel neuronal activities.
Collapse
|
7
|
Zamani M, Okreghe C, Demosthenous A. Efficient Approximation of Action Potentials with High-Order Shape Preservation in Unsupervised Spike Sorting. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:4884-4887. [PMID: 36086429 DOI: 10.1109/embc48229.2022.9871487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
This paper presents a novel approximation unit added to the conventional spike processing chain which provides an appreciable reduction of complexity of the high-hardware cost feature extractors. The use of the Taylor polynomial is proposed and modelled employing its cascaded derivatives to non-uniformly capture the essential samples in each spike for reliable feature extraction and sorting. Inclusion of the approximation unit can provide 3X compression (i.e. from 66 to 22 samples) to the spike waveforms while preserving their shapes. Detailed spike waveform sequences based on in-vivo measurements have been generated using a customized neural simulator for performance assessment of the approximation unit tested on six published feature extractors. For noise levels σN between 0.05 and 0.3 and groups of 3 spikes in each channel, all the feature extractors provide almost same sorting performance before and after approximation. The overall implementation cost when including the approximation unit and feature extraction shows a large reduction (i.e. up to 8.7X) in the hardware costly and more accurate feature extractors, offering a substantial improvement in feature extraction design.
Collapse
|
8
|
Bod RB, Rokai J, Meszéna D, Fiáth R, Ulbert I, Márton G. From End to End: Gaining, Sorting, and Employing High-Density Neural Single Unit Recordings. Front Neuroinform 2022; 16:851024. [PMID: 35769832 PMCID: PMC9236662 DOI: 10.3389/fninf.2022.851024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2022] [Accepted: 05/06/2022] [Indexed: 11/15/2022] Open
Abstract
The meaning behind neural single unit activity has constantly been a challenge, so it will persist in the foreseeable future. As one of the most sourced strategies, detecting neural activity in high-resolution neural sensor recordings and then attributing them to their corresponding source neurons correctly, namely the process of spike sorting, has been prevailing so far. Support from ever-improving recording techniques and sophisticated algorithms for extracting worthwhile information and abundance in clustering procedures turned spike sorting into an indispensable tool in electrophysiological analysis. This review attempts to illustrate that in all stages of spike sorting algorithms, the past 5 years innovations' brought about concepts, results, and questions worth sharing with even the non-expert user community. By thoroughly inspecting latest innovations in the field of neural sensors, recording procedures, and various spike sorting strategies, a skeletonization of relevant knowledge lays here, with an initiative to get one step closer to the original objective: deciphering and building in the sense of neural transcript.
Collapse
Affiliation(s)
- Réka Barbara Bod
- Laboratory of Experimental Neurophysiology, Department of Physiology, Faculty of Medicine, George Emil Palade University of Medicine, Pharmacy, Science and Technology of Târgu Mureş, Târgu Mureş, Romania
| | - János Rokai
- Integrative Neuroscience Group, Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary
- School of PhD Studies, Semmelweis University, Budapest, Hungary
| | - Domokos Meszéna
- Integrative Neuroscience Group, Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary
| | - Richárd Fiáth
- Integrative Neuroscience Group, Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary
| | - István Ulbert
- Integrative Neuroscience Group, Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary
| | - Gergely Márton
- Integrative Neuroscience Group, Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary
| |
Collapse
|
9
|
Garg R, Roman DS, Wang Y, Cohen-Karni D, Cohen-Karni T. Graphene nanostructures for input-output bioelectronics. BIOPHYSICS REVIEWS 2021; 2:041304. [PMID: 35005709 PMCID: PMC8717360 DOI: 10.1063/5.0073870] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 12/03/2021] [Indexed: 01/01/2023]
Abstract
The ability to manipulate the electrophysiology of electrically active cells and tissues has enabled a deeper understanding of healthy and diseased tissue states. This has primarily been achieved via input/output (I/O) bioelectronics that interface engineered materials with biological entities. Stable long-term application of conventional I/O bioelectronics advances as materials and processing techniques develop. Recent advancements have facilitated the development of graphene-based I/O bioelectronics with a wide variety of functional characteristics. Engineering the structural, physical, and chemical properties of graphene nanostructures and integration with modern microelectronics have enabled breakthrough high-density electrophysiological investigations. Here, we review recent advancements in 2D and 3D graphene-based I/O bioelectronics and highlight electrophysiological studies facilitated by these emerging platforms. Challenges and present potential breakthroughs that can be addressed via graphene bioelectronics are discussed. We emphasize the need for a multidisciplinary approach across materials science, micro-fabrication, and bioengineering to develop the next generation of I/O bioelectronics.
Collapse
Affiliation(s)
- Raghav Garg
- Department of Materials Science and Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA
| | - Daniel San Roman
- Department of Materials Science and Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA
| | - Yingqiao Wang
- Department of Materials Science and Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA
| | - Devora Cohen-Karni
- Preclinical education biochemistry, Lake Erie College of Osteopathic Medicine at Seton Hill, Greensburg, Pennsylvania 15601, USA
| | | |
Collapse
|
10
|
Rapeaux AB, Constandinou TG. Implantable brain machine interfaces: first-in-human studies, technology challenges and trends. Curr Opin Biotechnol 2021; 72:102-111. [PMID: 34749248 DOI: 10.1016/j.copbio.2021.10.001] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Revised: 09/29/2021] [Accepted: 10/02/2021] [Indexed: 11/29/2022]
Abstract
Implantable brain machine interfaces (BMIs) are now on a trajectory to go mainstream, wherein what was once considered last resort will progressively become elective at earlier stages in disease treatment. First-in-human successes have demonstrated the ability to decode highly dexterous motor skills such as handwriting, and speech from human cortical activity. These have been used for cursor and prosthesis control, direct-to-text communication and speech synthesis. Along with these breakthrough studies, technology advancements have enabled the observation of more channels of neural activity through new concepts for centralised/distributed implant architectures. This is complemented by research in flexible substrates, packaging, surgical workflows and data processing. New regulatory guidance and funding has galvanised the field. This culmination of resource, efforts and capability is now attracting significant investment for BMI commercialisation. This paper reviews recent developments and describes the paradigm shift in BMI development that is leading to new innovations, insights and BMI translation.
Collapse
Affiliation(s)
- Adrien B Rapeaux
- Department of Electrical and Electronic Engineering, Imperial College London, UK; Centre for Bio-Inspired Technology, Imperial College London, UK; Care Research and Technology (CR&T) based at Imperial College London and the University of Surrey, UK Dementia Research Institute (UK DRI), UK
| | - Timothy G Constandinou
- Department of Electrical and Electronic Engineering, Imperial College London, UK; Centre for Bio-Inspired Technology, Imperial College London, UK; Care Research and Technology (CR&T) based at Imperial College London and the University of Surrey, UK Dementia Research Institute (UK DRI), UK.
| |
Collapse
|
11
|
Sun B, Zhao W. Compressed Sensing of Extracellular Neurophysiology Signals: A Review. Front Neurosci 2021; 15:682063. [PMID: 34512238 PMCID: PMC8427310 DOI: 10.3389/fnins.2021.682063] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Accepted: 07/08/2021] [Indexed: 11/13/2022] Open
Abstract
This article presents a comprehensive survey of literature on the compressed sensing (CS) of neurophysiology signals. CS is a promising technique to achieve high-fidelity, low-rate, and hardware-efficient neural signal compression tasks for wireless streaming of massively parallel neural recording channels in next-generation neural interface technologies. The main objective is to provide a timely retrospective on applying the CS theory to the extracellular brain signals in the past decade. We will present a comprehensive review on the CS-based neural recording system architecture, the CS encoder hardware exploration and implementation, the sparse representation of neural signals, and the signal reconstruction algorithms. Deep learning-based CS methods are also discussed and compared with the traditional CS-based approaches. We will also extend our discussion to cover the technical challenges and prospects in this emerging field.
Collapse
Affiliation(s)
- Biao Sun
- School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Wenfeng Zhao
- Department of Electrical and Computer Engineering, Binghamton University, State University of New York, Binghamton, NY, United States
| |
Collapse
|
12
|
Cheng J, Yang Z, Overstreet CK, Keefer E. Fascicle-Specific Targeting of Longitudinal Intrafascicular Electrodes for Motor and Sensory Restoration in Upper-Limb Amputees. Hand Clin 2021; 37:401-414. [PMID: 34253313 DOI: 10.1016/j.hcl.2021.04.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Multichannel longitudinal intrafascicular electrode (LIFE) interfaces provide optimized balance of invasiveness and stability for chronic sensory stimulation and motor recording/decoding of peripheral nerve signals. Using a fascicle-specific targeting (FAST)-LIFE approach, where electrodes are individually placed within discrete sensory- and motor-related fascicular subdivisions of the residual ulnar and/or median nerves in an amputated upper limb, FAST-LIFE interfacing can provide discernment of motor intent for individual digit control of a robotic hand, and restoration of touch- and movement-related sensory feedback. The authors describe their findings from clinical studies performed with 6 human amputee trials using FAST-LIFE interfacing of the residual upper limb.
Collapse
Affiliation(s)
- Jonathan Cheng
- Department of Plastic Surgery, University of Texas Southwestern Medical Center, 1801 Inwood Road, Dallas, TX 75390, USA.
| | - Zhi Yang
- Department of Biomedical Engineering, University of Minnesota, Nils Hasselmo Hall, Room 6-120, 312 Church Street Southeast, Minneapolis, MN 55455, USA
| | | | - Edward Keefer
- Nerves Incorporated, P.O. Box 141295, Dallas, TX 75214, USA
| |
Collapse
|
13
|
Rokai J, Rácz M, Fiáth R, Ulbert I, Márton G. ELVISort: encoding latent variables for instant sorting, an artificial intelligence-based end-to-end solution. J Neural Eng 2021; 18. [PMID: 33823497 DOI: 10.1088/1741-2552/abf521] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Accepted: 04/06/2021] [Indexed: 11/12/2022]
Abstract
Objective.The growing number of recording sites of silicon-based probes means that an increasing amount of neural cell activities can be recorded simultaneously, facilitating the investigation of underlying complex neural dynamics. In order to overcome the challenges generated by the increasing number of channels, highly automated signal processing tools are needed. Our goal was to build a spike sorting model that can perform as well as offline solutions while maintaining high efficiency, enabling high-performance online sorting.Approach.In this paper we present ELVISort, a deep learning method that combines the detection and clustering of different action potentials in an end-to-end fashion.Main results.The performance of ELVISort is comparable with other spike sorting methods that use manual or semi-manual techniques, while exceeding the methods which use an automatic approach: ELVISort has been tested on three independent datasets and yielded average F1scores of 0.96, 0.82 and 0.81, which comparable with the results of state-of-the-art algorithms on the same data. We show that despite the good performance, ELVISort is capable to process data in real-time: the time it needs to execute the necessary computations for a sample of given length is only 1/15.71 of its actual duration (i.e. the sampling time multiplied by the number of the sampling points).Significance.ELVISort, because of its end-to-end nature, can exploit the massively parallel processing capabilities of GPUs via deep learning frameworks by processing multiple batches in parallel, with the potential to be used on other cutting-edge AI-specific hardware such as TPUs, enabling the development of integrated, portable and real-time spike sorting systems with similar performance to offline sorters.
Collapse
Affiliation(s)
- János Rokai
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Magyar tudósok körútja 2, Budapest H-1117, Hungary.,School of PhD Studies, Semmelweis University, Üllői út 26, H-1085 Budapest, Hungary
| | - Melinda Rácz
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Magyar tudósok körútja 2, Budapest H-1117, Hungary.,School of PhD Studies, Semmelweis University, Üllői út 26, H-1085 Budapest, Hungary
| | - Richárd Fiáth
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Magyar tudósok körútja 2, Budapest H-1117, Hungary.,Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter utca 50/a, H-1083 Budapest, Hungary
| | - István Ulbert
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Magyar tudósok körútja 2, Budapest H-1117, Hungary.,Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter utca 50/a, H-1083 Budapest, Hungary
| | - Gergely Márton
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Magyar tudósok körútja 2, Budapest H-1117, Hungary.,Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter utca 50/a, H-1083 Budapest, Hungary
| |
Collapse
|
14
|
Sun B, Zhang H, Zhang Y, Wu Z, Bao B, Hu Y, Li T. Compressed sensing of large-scale local field potentials using adaptive sparsity analysis and Non-convex Optimization. J Neural Eng 2020; 18. [PMID: 33348334 DOI: 10.1088/1741-2552/abd578] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Accepted: 12/21/2020] [Indexed: 01/19/2023]
Abstract
OBJECTIVE Energy consumption is a critical issue in resource-constrained wireless neural recording applications with limited data bandwidth. Compressed sensing (CS) has emerged as a powerful framework in addressing this issue owing to its highly efficient data compression procedure. In this paper, a CS-based approach termed Simultaneous Analysis Non-Convex Optimization (SANCO) is proposed for large-scale, multi-channel local field potentials (LFPs) recording. APPROACH The SANCO method consists of three parts: (1) the analysis model is adopted to reinforce sparsity of the multi-channel LFPs, therefore overcoming the drawbacks of conventional synthesis models. (2) An optimal continuous order difference matrix is constructed as the analysis operator, enhancing the recovery performance while saving both computational resources and data storage space. (3) A non-convex optimizer that can by efficiently solved with alternating direction method of multipliers (ADMM) is developed for multi-channel LFPs reconstruction. MAIN RESULTS Experimental results on real datasets reveal that the proposed approach outperforms state-of-the-art CS methods in terms of both recovery quality and computational efficiency. SIGNIFICANCE Energy efficiency of the SANCO make it an ideal candidate for resource-constrained, large scale wireless neural recording. Particularly, the proposed method ensures that the key features of LFPs had little degradation even when data are compressed by 16x, making it very suitable for long term wireless neural recording applications.
Collapse
Affiliation(s)
- Biao Sun
- School of Electrical and Information Engineering, Tianjin University, No92, Weijin Road, Nankai District, Tianjin, Tianjin, 300072, CHINA
| | - Han Zhang
- School of Electrical and Information Engineering, Tianjin University, No92, Weijin Road, Nankai District, Tianjin, 300072, CHINA
| | - Yunyan Zhang
- Department of Physics, Paderborn University, Warburger Strase 100, 33098 Paderborn, Paderborn, Nordrhein-Westfalen, 33098, GERMANY
| | - Zexu Wu
- School of Electrical and Information Engineering, Tianjin University, No92, Weijin Road, Nankai District, Tianjin, 300072, CHINA
| | - Botao Bao
- Chinese Academy of Medical Sciences & Peking Union Medical College Institute of Biomedical Engineering, No 236, Baidi Road, Nankai District, Tianjin, Tianjin, 300192, CHINA
| | - Yong Hu
- Department of Orthopaedics and Traumatology, Hong Kong University, Professorial Block, Queen Mary Hospital, Pok Fu Lam, Hong Kong, Hong Kong, 999077, HONG KONG
| | - Ting Li
- Chinese Academy of Medical Sciences & Peking Union Medical College Institute of Biomedical Engineering, No 236, Baidi Road, Nankai District, Tianjin, 300192, CHINA
| |
Collapse
|
15
|
An Accurate and Robust Method for Spike Sorting Based on Convolutional Neural Networks. Brain Sci 2020; 10:brainsci10110835. [PMID: 33187098 PMCID: PMC7696441 DOI: 10.3390/brainsci10110835] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2020] [Revised: 10/29/2020] [Accepted: 11/09/2020] [Indexed: 12/17/2022] Open
Abstract
In the fields of neuroscience and biomedical signal processing, spike sorting is a crucial step to extract the information of single neurons from extracellular recordings. In this paper, we propose a novel deep learning approach based on one-dimensional convolutional neural networks (1D-CNNs) to implement accurate and robust spike sorting. The results of the simulated data demonstrated that the clustering accuracy in most datasets was greater than 99%, despite the multiple levels of noise and various degrees of overlapped spikes. Moreover, the proposed method performed significantly better than the state-of-the-art method named “WMsorting” and a deep-learning-based multilayer perceptron (MLP) model. In addition, the experimental data recorded from the primary visual cortex of a macaque monkey were used to evaluate the proposed method in a practical application. It was shown that the method could successfully isolate most spikes of different neurons (ranging from two to five) by training the 1D-CNN model with a small number of manually labeled spikes. Considering the above, the deep learning method proposed in this paper is of great advantage for spike sorting with high accuracy and strong robustness. It lays the foundation for application in more challenging works, such as distinguishing overlapped spikes and the simultaneous sorting of multichannel recordings.
Collapse
|
16
|
Even-Chen N, Muratore DG, Stavisky SD, Hochberg LR, Henderson JM, Murmann B, Shenoy KV. Power-saving design opportunities for wireless intracortical brain-computer interfaces. Nat Biomed Eng 2020; 4:984-996. [PMID: 32747834 PMCID: PMC8286886 DOI: 10.1038/s41551-020-0595-9] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2018] [Accepted: 06/30/2020] [Indexed: 12/17/2022]
Abstract
The efficacy of wireless intracortical brain-computer interfaces (iBCIs) is limited in part by the number of recording channels, which is constrained by the power budget of the implantable system. Designing wireless iBCIs that provide the high-quality recordings of today's wired neural interfaces may lead to inadvertent over-design at the expense of power consumption and scalability. Here, we report analyses of neural signals collected from experimental iBCI measurements in rhesus macaques and from a clinical-trial participant with implanted 96-channel Utah multielectrode arrays to understand the trade-offs between signal quality and decoder performance. Moreover, we propose an efficient hardware design for clinically viable iBCIs, and suggest that the circuit design parameters of current recording iBCIs can be relaxed considerably without loss of performance. The proposed design may allow for an order-of-magnitude power savings and lead to clinically viable iBCIs with a higher channel count.
Collapse
Affiliation(s)
- Nir Even-Chen
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Dante G Muratore
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Sergey D Stavisky
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- Department of Veterans Affairs Medical Center, Center for Neurorestoration and Neurotechnology, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jaimie M Henderson
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Boris Murmann
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- The Bio-X Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|
17
|
Muratore DG, Tandon P, Wootters M, Chichilnisky EJ, Mitra S, Murmann B. A Data-Compressive Wired-OR Readout for Massively Parallel Neural Recording. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:1128-1140. [PMID: 31425051 DOI: 10.1109/tbcas.2019.2935468] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Neural interfaces of the future will be used to help restore lost sensory, motor, and other capabilities. However, realizing this futuristic promise requires a major leap forward in how electronic devices interface with the nervous system. Next generation neural interfaces must support parallel recording from tens of thousands of electrodes within the form factor and power budget of a fully implanted device, posing a number of significant engineering challenges. In this paper, we exploit sparsity and diversity of neural signals to achieve simultaneous data compression and channel multiplexing for neural recordings. The architecture uses wired-OR interactions within an array of single-slope A/D converters to obtain massively parallel digitization of neural action potentials. The achieved compression is lossy but effective at retaining the critical samples belonging to action potentials, enabling efficient spike sorting and cell type identification. Simulation results of the architecture using data obtained from primate retina ex-vivo with a 512-channel electrode array show average compression rates up to ∼ 40× while missing less than 5% of cells. In principle, the techniques presented here could be used to design interfaces to other parts of the nervous system.
Collapse
|