1
|
Erbslöh A, Buron L, Ur-Rehman Z, Musall S, Hrycak C, Löhler P, Klaes C, Seidl K, Schiele G. Technical survey of end-to-end signal processing in BCIs using invasive MEAs. J Neural Eng 2024; 21:051003. [PMID: 39326451 DOI: 10.1088/1741-2552/ad8031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 09/26/2024] [Indexed: 09/28/2024]
Abstract
Modern brain-computer interfaces and neural implants allow interaction between the tissue, the user and the environment, where people suffer from neurodegenerative diseases or injuries.This interaction can be achieved by using penetrating/invasive microelectrodes for extracellular recordings and stimulation, such as Utah or Michigan arrays. The application-specific signal processing of the extracellular recording enables the detection of interactions and enables user interaction. For example, it allows to read out movement intentions from recordings of brain signals for controlling a prosthesis or an exoskeleton. To enable this, computationally complex algorithms are used in research that cannot be executed on-chip or on embedded systems. Therefore, an optimization of the end-to-end processing pipeline, from the signal condition on the electrode array over the analog pre-processing to spike-sorting and finally the neural decoding process, is necessary for hardware inference in order to enable a local signal processing in real-time and to enable a compact system for achieving a high comfort level. This paper presents a survey of system architectures and algorithms for end-to-end signal processing pipelines of neural activity on the hardware of such neural devices, including (i) on-chip signal pre-processing, (ii) spike-sorting on-chip or on embedded hardware and (iii) neural decoding on workstations. A particular focus for the hardware implementation is on low-power electronic design and artifact-robust algorithms with low computational effort and very short latency. For this, current challenges and possible solutions with support of novel machine learning techniques are presented in brief. In addition, we describe our future vision for next-generation BCIs.
Collapse
Affiliation(s)
| | - Leo Buron
- University of Duisburg-Essen, Duisburg, Germany
| | | | | | | | | | | | - Karsten Seidl
- University of Duisburg-Essen, Duisburg, Germany
- Fraunhofer Institute for Microelectronic Circuits and Systems, Duisburg, Germany
| | | |
Collapse
|
2
|
Sbandati C, Stathopoulos S, Foster P, Peer ND, Sestito C, Serb A, Vassanelli S, Cohen D, Prodromakis T. Single-trial detection of auditory cues from the rat brain using memristors. SCIENCE ADVANCES 2024; 10:eadp7613. [PMID: 39231225 PMCID: PMC11373585 DOI: 10.1126/sciadv.adp7613] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2024] [Accepted: 07/29/2024] [Indexed: 09/06/2024]
Abstract
Implantable devices hold the potential to address conditions currently lacking effective treatments, such as drug-resistant neural impairments and prosthetic control. Medical devices need to be biologically compatible while providing enhanced performance metrics of low-power consumption, high accuracy, small size, and minimal latency to enable ongoing intervention in brain function. Here, we demonstrate a memristor-based processing system for single-trial detection of behaviorally meaningful brain signals within a timeframe that supports real-time closed-loop intervention. We record neural activity from the reward center of the brain, the ventral tegmental area, in rats trained to associate a musical tone with a reward, and we use the memristors built-in thresholding properties to detect nontrivial biomarkers in local field potentials. This approach yields consistent and accurate detection of biomarkers >98% while maintaining power consumption as low as 4.14 nanowatt per channel. The efficacy of our system's capabilities to process real-time in vivo neural data paves the way for low-power chronic neural activity monitoring and biomedical implants.
Collapse
Affiliation(s)
- Caterina Sbandati
- Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, Edinburgh, UK
| | - Spyros Stathopoulos
- Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, Edinburgh, UK
| | - Patrick Foster
- Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, Edinburgh, UK
| | - Noam D Peer
- The Gonda Brain Research Center, Bar-Ilan University, Ramat-Gan 52900, Israel
| | - Cristian Sestito
- Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, Edinburgh, UK
| | - Alex Serb
- Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, Edinburgh, UK
| | - Stefano Vassanelli
- Padua Neuroscience Center, University of Padua, via Orus 2/B, 35131 Padua, Italy
| | - Dana Cohen
- The Gonda Brain Research Center, Bar-Ilan University, Ramat-Gan 52900, Israel
| | - Themis Prodromakis
- Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, Edinburgh, UK
| |
Collapse
|
3
|
Zhang Z, Constandinou TG. Selecting an effective amplitude threshold for neural spike detection. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:2328-2331. [PMID: 36085877 DOI: 10.1109/embc48229.2022.9871955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
This paper assesses and challenges whether commonly used methods for defining amplitude thresholds for spike detection are optimal. This is achieved through empirical testing of single amplitude thresholds across multiple recordings of varying SNR levels. Our results suggest that the most widely used noise-statistics-driven threshold can suffer from parameter deviation in different noise levels. The spike-noise-driven threshold can be an ideal approach to set the threshold for spike detection, which suffers less from the parameter deviation and is robust to sub-optimal settings.
Collapse
|
4
|
Zhang Z, Savolainen OW, Constandinou T. Algorithm and hardware considerations for real-time neural signal on-implant processing. J Neural Eng 2022; 19. [PMID: 35130536 DOI: 10.1088/1741-2552/ac5268] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2021] [Accepted: 02/07/2022] [Indexed: 11/12/2022]
Abstract
Objective Various on-workstation neural-spike-based brain machine interface(BMI) systems have reached the point of in-human trials, but on-node and on-implant BMI systems are still under exploration. Such systems are constrained by the area and battery. Researchers should consider the algorithm complexity, available resources, power budgets, CMOS technologies, and the choice of platforms when designing BMI systems. However, the effect of these factors is currently still unclear. Approaches. Here we have proposed a novel real-time 128 channel spike detection algorithm and optimised it on Microcontroller(MCU) and Field Programmable Gate Array(FPGA) platforms towards consuming minimal power and memory/resources. It is presented as a use case to explore the different considerations in system design. Main results. The proposed spike detection algorithm achieved over 97% sensitivity and a smaller than 3% false detection rate. The MCU implementation occupies less than 3KB RAM and consumes 31.5μW/ch. The FPGA platform only occupies 299 logic cells and 3KB RAM for 128 channels and consumes 0.04μW/ch. Significance. On the spike detection algorithm front, we have eliminated the processing bottleneck by reducing the dynamic power consumption to lower than the hardware static power, without sacrificing detection performance. More importantly, we have explored the considerations in algorithm and hardware design with respect to scalability, portability, and costs. These findings can facilitate and guide the future development of real-time on-implant neural signal processing platforms.
Collapse
Affiliation(s)
- Zheng Zhang
- Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London, SW7 2AZ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Oscar W Savolainen
- Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London, London, SW7 2AZ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Timothy Constandinou
- Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London, London, SW7 2AZ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| |
Collapse
|
5
|
Zhang Z, Constandinou TG. Adaptive spike detection and hardware optimization towards autonomous, high-channel-count BMIs. J Neurosci Methods 2021; 354:109103. [PMID: 33617917 DOI: 10.1016/j.jneumeth.2021.109103] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 01/23/2021] [Accepted: 02/15/2021] [Indexed: 01/20/2023]
Abstract
BACKGROUND The progress in microtechnology has enabled an exponential trend in the number of neurons that can be simultaneously recorded. The data bandwidth requirement is however increasing with channel count. The vast majority of experimental work involving electrophysiology stores the raw data and then processes this offline; to detect the underlying spike events. Emerging applications however require new methods for local, real-time processing. NEW METHODS We have developed an adaptive, low complexity spike detection algorithm that combines three novel components for: (1) removing the local field potentials; (2) enhancing the signal-to-noise ratio; and (3) computing an adaptive threshold. The proposed algorithm has been optimised for hardware implementation (i.e. minimising computations, translating to a fixed-point implementation), and demonstrated on low-power embedded targets. MAIN RESULTS The algorithm has been validated on both synthetic datasets and real recordings yielding a detection sensitivity of up to 90%. The initial hardware implementation using an off-the-shelf embedded platform demonstrated a memory requirement of less than 0.1 kb ROM and 3 kb program flash, consuming an average power of 130 μW. COMPARISON WITH EXISTING METHODS The method presented has the advantages over other approaches, that it allows spike events to be robustly detected in real-time from neural activity in a completely autonomous way, without the need for any calibration, and can be implemented with low hardware resources. CONCLUSION The proposed method can detect spikes effectively and adaptively. It alleviates the need for re-calibration, which is critical towards achieving a viable BMI, and more so with future 'high bandwidth' systems' targeting 1000s of channels.
Collapse
Affiliation(s)
- Zheng Zhang
- Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London SW7 2AZ, UK.
| | - Timothy G Constandinou
- Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London SW7 2AZ, UK; UK Dementia Research Institute (UKDRI) Care Research & Technology Centre, based at Imperial College London and the University of Surrey, UK.
| |
Collapse
|
6
|
Fiorelli R, Delgado-Restituto M, Rodriguez-Vazquez A. Charge-Redistribution Based Quadratic Operators for Neural Feature Extraction. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2020; 14:606-619. [PMID: 32305936 DOI: 10.1109/tbcas.2020.2987389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
This paper presents a SAR converter based mixed-signal multiplier for the feature extraction of neural signals using quadratic operators. After a thorough analysis of design principles and circuit-level aspects, the proposed architecture is explored for the implementation of two quadratic operators often used for the characterization of neural activity, the moving average energy (MAE) operator and the nonlinear energy operator (NEO). Programmable chips for both operators have been implemented in a HV-180 nm CMOS process. Experimental results confirm their suitability for energy computation and action potential detection and the accomplished area×power performance is compared to prior art. The MAE and NEO prototypes, at a sampling rate of 30kS/s, consume 116 nW and 178 nW, respectively, and digitize both the input neural signal and the operator outcome, with no need for digital multipliers.
Collapse
|
7
|
Shaikh S, So R, Sibindi T, Libedinsky C, Basu A. Towards Intelligent Intracortical BMI (i 2BMI): Low-Power Neuromorphic Decoders That Outperform Kalman Filters. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:1615-1624. [PMID: 31581098 DOI: 10.1109/tbcas.2019.2944486] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Fully-implantable wireless intracortical Brain Machine Interfaces (iBMI) is one of the most promising next frontiers in the nascent field of neurotechnology. However, scaling the number of channels in such systems by another 10× is difficult due to power and bandwidth requirements of the wireless transmitter. One promising solution for that is to include more processing, up to the decoder, in the implant so that transmission data-rate is reduced drastically. Earlier work on neuromorphic decoder chips only showed classification of discrete states. We present results for continuous state decoding using a low-power neuromorphic decoder chip termed Spike-input Extreme Learning Machine (SELMA) that implements a nonlinear decoder without memory and its memory-based version with time-delayed bins, SELMA-bins. We have compared SELMA, SELMA-bins against state-of-the-art Steady-State Kalman Filter (SSKF), a linear decoder with memory, across two different datasets involving a total of 4 non-human primates (NHPs). Results show at least a 10% (20%) or more increase in the fraction of variance accounted for (FVAF) by SELMA (SELMA-bins) over SSKF across the datasets. Estimated energy consumption comparison shows SELMA (SELMA-bins) consuming ≈ 9 nJ/update (23 nJ/update) against SSKF's ≈ 7.4 nJ/update for an iBMI with a 10 degree of freedom control. Thus, SELMA yields better performance against SSKF while consuming energy in the same range as SSKF whereas SELMA-bins performs the best with moderately increased energy consumption, albeit far less than energy required for raw data transmission. This paves the way for reducing transmission data rates in future scaled iBMI systems.
Collapse
|
8
|
Kim SJ, Han SH, Cha JH, Liu L, Yao L, Gao Y, Je M. A Sub- μW/Ch Analog Front-End for ∆-Neural Recording With Spike-Driven Data Compression. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:1-14. [PMID: 30418918 DOI: 10.1109/tbcas.2018.2880257] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
We present a fully implantable neural recording IC with a spike-driven data compression scheme to improve the power efficiency and preserve crucial data for monitoring brain activities. A difference between two consecutive neural signals, ∆-neural signal, is sampled in each channel to reduce the full dynamic range and the required resolution of an analog-to-digital converter (ADC), enabling the whole analog chain to be operated at a 0.5-V supply. A set of multiple ∆-signals are stored in analog memory to extract the magnitude and frequency features of the incoming neural signals, which are utilized to discriminate spikes in these signals instantaneously after the acquisition in the analog domain. The energy- and area-efficient successive approximation ADC is implemented and only converts detected spikes, decreasing the power dissipation and the amount of neural data. A prototype 16-channel neural interface IC was fabricated using a 0.18-μm CMOS process, and each component in the analog front-end was fully characterized. We successfully demonstrated precise spike detection through both in vitro and in vivo acquisition of the neural signal. The prototype chip consumed 0.88 μW/channel at a 0.5-V supply for the recording and compressed about 89% of neural data, saving the power consumption and bandwidth in the system.
Collapse
|
9
|
Hong KS, Aziz N, Ghafoor U. Motor-commands decoding using peripheral nerve signals: a review. J Neural Eng 2018; 15:031004. [PMID: 29498358 DOI: 10.1088/1741-2552/aab383] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
During the last few decades, substantial scientific and technological efforts have been focused on the development of neuroprostheses. The major emphasis has been on techniques for connecting the human nervous system with a robotic prosthesis via natural-feeling interfaces. The peripheral nerves provide access to highly processed and segregated neural command signals from the brain that can in principle be used to determine user intent and control muscles. If these signals could be used, they might allow near-natural and intuitive control of prosthetic limbs with multiple degrees of freedom. This review summarizes the history of neuroprosthetic interfaces and their ability to record from and stimulate peripheral nerves. We also discuss the types of interfaces available and their applications, the kinds of peripheral nerve signals that are used, and the algorithms used to decode them. Finally, we explore the prospects for future development in this area.
Collapse
|
10
|
Wu T, Zhao W, Guo H, Lim HH, Yang Z. A Streaming PCA VLSI Chip for Neural Data Compression. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2017; 11:1290-1302. [PMID: 28809707 DOI: 10.1109/tbcas.2017.2717281] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Neural recording system miniaturization and integration with low-power wireless technologies require compressing neural data before transmission. Feature extraction is a procedure to represent data in a low-dimensional space; its integration into a recording chip can be an efficient approach to compress neural data. In this paper, we propose a streaming principal component analysis algorithm and its microchip implementation to compress multichannel local field potential (LFP) and spike data. The circuits have been designed in a 65-nm CMOS technology and occupy a silicon area of 0.06 mm. Throughout the experiments, the chip compresses LFPs by 10 at the expense of as low as 1% reconstruction errors and 144-nW/channel power consumption; for spikes, the achieved compression ratio is 25 with 8% reconstruction errors and 3.05-W/channel power consumption. In addition, the algorithm and its hardware architecture can swiftly adapt to nonstationary spiking activities, which enables efficient hardware sharing among multiple channels to support a high-channel count recorder.
Collapse
|
11
|
Yang Y, Boling S, Mason AJ. A Hardware-Efficient Scalable Spike Sorting Neural Signal Processor Module for Implantable High-Channel-Count Brain Machine Interfaces. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2017; 11:743-754. [PMID: 28541908 DOI: 10.1109/tbcas.2017.2679032] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Next-generation brain machine interfaces demand a high-channel-count neural recording system to wirelessly monitor activities of thousands of neurons. A hardware efficient neural signal processor (NSP) is greatly desirable to ease the data bandwidth bottleneck for a fully implantable wireless neural recording system. This paper demonstrates a complete multichannel spike sorting NSP module that incorporates all of the necessary spike detector, feature extractor, and spike classifier blocks. To meet high-channel-count and implantability demands, each block was designed to be highly hardware efficient and scalable while sharing resources efficiently among multiple channels. To process multiple channels in parallel, scalability analysis was performed, and the utilization of each block was optimized according to its input data statistics and the power, area and/or speed of each block. Based on this analysis, a prototype 32-channel spike sorting NSP scalable module was designed and tested on an FPGA using synthesized datasets over a wide range of signal to noise ratios. The design was mapped to 130 nm CMOS to achieve 0.75 μW power and 0.023 mm2 area consumptions per channel based on post synthesis simulation results, which permits scalability of digital processing to 690 channels on a 4×4 mm2 electrode array.
Collapse
|
12
|
Yang Y, Mason AJ. Hardware Efficient Automatic Thresholding for NEO-Based Neural Spike Detection. IEEE Trans Biomed Eng 2017; 64:826-833. [DOI: 10.1109/tbme.2016.2580319] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|