A Survey of Encoding Techniques for Signal Processing in Spiking Neural Networks

Low-power and lightweight spiking transformer for EEG-based auditory attention detection

EEG signal analysis can be used to study brain activity and the function and structure of neural networks, helping to understand neural mechanisms such as cognition, emotion, and behavior. EEG-based auditory attention detection is using EEG signals to determine an individual's level of attention to specific auditory stimuli. In this technique, researchers record and analyze a subject's electrical activity to infer whether an individual is paying attention to a specific auditory stimulus. The model deployed in edge devices will be greatly convenient for subjects to use. However, most of the existing EEG-based auditory attention detection models use traditional neural network models, and their high computing load makes deployment on edge devices challenging. We present a pioneering approach in the form of a binarized spiking Transformer for EEG-based auditory attention detection, which is characterized by high accuracy, low power consumption, and lightweight design, making it highly suitable for deployment on edge devices. In terms of low power consumption, the network is constructed using spiking neurons, which emit sparse and binary spike sequences, which can effectively reduce computing power consumption. In terms of lightweight, we use a post-training quantization strategy to quantize the full-precision network weights into binary weights, which greatly reduces the model size. In addition, the structure of the Transformer ensures that the model can learn effective information and ensure its high performance. We verify the model through mainstream datasets, and experimental results show that our model performance can exceed the existing state-of-the-art models, and the model size can be reduced by more than 21 times compared with the original full-precision network counterpart.

Towards scalable memristive hardware for spiking neural networks

Spiking neural networks (SNNs) represent a promising frontier in artificial intelligence (AI), offering event-driven, energy-efficient computation that mimics rich neural dynamics in the brain. However, running large-scale SNNs on mainstream computing hardware faces significant challenges to efficiently emulate these dynamical processes using synchronized and logical chips. Memristor based systems have recently demonstrated great potential for AI acceleration, sparking speculations and explorations of using these emerging devices for SNN tasks. This paper reviews the promises and challenges of memristive devices in SNN implementations, and our discussions are focused on the scaling and integration of neuronal and synaptic devices. We survey recent progress in device and circuit development, discuss possible pathways for chip-level integration, and finally probe into hardware-oriented algorithm designs. This review offers a system-level perspective on implementing scalable memristor based SNN platforms.

Recent Advances in Artificial Sensory Neurons: Biological Fundamentals, Devices, Applications, and Challenges

Spike-based neural networks, which use spikes or action potentials to represent information, have gained a lot of attention because of their high energy efficiency and low power consumption. To fully leverage its advantages, converting the external analog signals to spikes is an essential prerequisite. Conventional approaches including analog-to-digital converters or ring oscillators, and sensors suffer from high power and area costs. Recent efforts are devoted to constructing artificial sensory neurons based on emerging devices inspired by the biological sensory system. They can simultaneously perform sensing and spike conversion, overcoming the deficiencies of traditional sensory systems. This review summarizes and benchmarks the recent progress of artificial sensory neurons. It starts with the presentation of various mechanisms of biological signal transduction, followed by the systematic introduction of the emerging devices employed for artificial sensory neurons. Furthermore, the implementations with different perceptual capabilities are briefly outlined and the key metrics and potential applications are also provided. Finally, we highlight the challenges and perspectives for the future development of artificial sensory neurons.

Stress-induced artificial neuron spiking in diffusive memristors

Diffusive memristors owing to their ability to produce current spiking when a constant or slowly changing voltage is applied are competitive candidates for development of artificial electronic neurons. These artificial neurons can be integrated into various prospective autonomous and robotic systems as sensors, e.g. ones implementing object grasping and classification. We report here Ag nanoparticle-based diffusive memristor prepared on a flexible polyethylene terephthalate substrate in which the electric spiking behaviour was induced by the electric voltage under an additional stimulus of external mechanical impact. By changing the magnitude and frequency of the mechanical impact, we are able to manipulate the spiking response of our artificial neuron. This functionality to control the spiking characteristics paves a pathway for the development of touch-perception sensors that can convert local pressure into electrical spikes for further processing in neural networks. We have proposed a mathematical model which captures the operation principle of the fabricated memristive sensors and qualitatively describes the measured spiking behaviour. Employing such flexible diffusive memristors that can directly translate tactile information into spikes, similar to force and pressure sensors, could offer substantial benefits for various applications in robotics.

Spiking Neural Network Pressure Sensor

Von Neumann architecture requires information to be encoded as numerical values. For that reason, artificial neural networks running on computers require the data coming from sensors to be discretized. Other network architectures that more closely mimic biological neural networks (e.g., spiking neural networks) can be simulated on von Neumann architecture, but more important, they can also be executed on dedicated electrical circuits having orders of magnitude less power consumption. Unfortunately, input signal conditioning and encoding are usually not supported by such circuits, so a separate module consisting of an analog-to-digital converter, encoder, and transmitter is required. The aim of this article is to propose a sensor architecture, the output signal of which can be directly connected to the input of a spiking neural network. We demonstrate that the output signal is a valid spike source for the Izhikevich model neurons, ensuring the proper operation of a number of neurocomputational features. The advantages are clear: much lower power consumption, smaller area, and a less complex electronic circuit. The main disadvantage is that sensor characteristics somehow limit the parameters of applicable spiking neurons. The proposed architecture is illustrated by a case study involving a capacitive pressure sensor circuit, which is compatible with most of the neurocomputational properties of the Izhikevich neuron model. The sensor itself is characterized by very low power consumption: it draws only 3.49 μA at 3.3 V.

Spiking generative adversarial network with attention scoring decoding

Generative models based on neural networks present a substantial challenge within deep learning. As it stands, such models are primarily limited to the domain of artificial neural networks. Spiking neural networks, as the third generation of neural networks, offer a closer approximation to brain-like processing due to their rich spatiotemporal dynamics. However, generative models based on spiking neural networks are not well studied. Particularly, previous works on generative adversarial networks based on spiking neural networks are conducted on simple datasets and do not perform well. In this work, we pioneer constructing a spiking generative adversarial network capable of handling complex images and having higher performance. Our first task is to identify the problems of out-of-domain inconsistency and temporal inconsistency inherent in spiking generative adversarial networks. We address these issues by incorporating the Earth-Mover distance and an attention-based weighted decoding method, significantly enhancing the performance of our algorithm across several datasets. Experimental results reveal that our approach outperforms existing methods on the MNIST, FashionMNIST, CIFAR10, and CelebA. In addition to our examination of static datasets, this study marks our inaugural investigation into event-based data, through which we achieved noteworthy results. Moreover, compared with hybrid spiking generative adversarial networks, where the discriminator is an artificial analog neural network, our methodology demonstrates closer alignment with the information processing patterns observed in the mouse. Our code can be found at https://github.com/Brain-Cog-Lab/sgad.

Snn and sound: a comprehensive review of spiking neural networks in sound

The rapid advancement of AI and machine learning has significantly enhanced sound and acoustic recognition technologies, moving beyond traditional models to more sophisticated neural network-based methods. Among these, Spiking Neural Networks (SNNs) are particularly noteworthy. SNNs mimic biological neurons and operate on principles similar to the human brain, using analog computing mechanisms. This capability allows for efficient sound processing with low power consumption and minimal latency, ideal for real-time applications in embedded systems. This paper reviews recent developments in SNNs for sound recognition, underscoring their potential to overcome the limitations of digital computing and suggesting directions for future research. The unique attributes of SNNs could lead to breakthroughs in mimicking human auditory processing more closely.

Backpropagation-Based Learning Techniques for Deep Spiking Neural Networks: A Survey

With the adoption of smart systems, artificial neural networks (ANNs) have become ubiquitous. Conventional ANN implementations have high energy consumption, limiting their use in embedded and mobile applications. Spiking neural networks (SNNs) mimic the dynamics of biological neural networks by distributing information over time through binary spikes. Neuromorphic hardware has emerged to leverage the characteristics of SNNs, such as asynchronous processing and high activation sparsity. Therefore, SNNs have recently gained interest in the machine learning community as a brain-inspired alternative to ANNs for low-power applications. However, the discrete representation of the information makes the training of SNNs by backpropagation-based techniques challenging. In this survey, we review training strategies for deep SNNs targeting deep learning applications such as image processing. We start with methods based on the conversion from an ANN to an SNN and compare these with backpropagation-based techniques. We propose a new taxonomy of spiking backpropagation algorithms into three categories, namely, spatial, spatiotemporal, and single-spike approaches. In addition, we analyze different strategies to improve accuracy, latency, and sparsity, such as regularization methods, training hybridization, and tuning of the parameters specific to the SNN neuron model. We highlight the impact of input encoding, network architecture, and training strategy on the accuracy-latency tradeoff. Finally, in light of the remaining challenges for accurate and efficient SNN solutions, we emphasize the importance of joint hardware-software codevelopment.

Baseline Drift Tolerant Signal Encoding for ECG Classification with Deep Learning

Common artefacts such as baseline drift, rescaling, and noise critically limit the performance of machine learning-based automated ECG analysis and interpretation. This study proposes Derived Peak (DP) encoding, a non-parametric method that generates signed spikes corresponding to zero crossings of the signal's first and second-order time derivatives. Notably, DP encoding is invariant to shift and scaling artefacts, and its implementation is further simplified by the absence of user-defined parameters. DP encoding was used to encode the 12-lead ECG data from the PTB-XL dataset (n=18,869 participants) and was fed to 1D-ResNet-18 models trained to identify myocardial infarction, conductive deficits and ST-segment abnormalities. Robustness to artefacts was assessed by corrupting ECG data with sinusoidal baseline drift, shift, rescaling and noise, before encoding. The addition of these artefacts resulted in a significant drop in accuracy for seven other methods from prior art, while DP encoding maintained a baseline AUC of 0.88 under drift, shift and rescaling. DP achieved superior performance to unencoded inputs in the presence of shift (AUC under 1 mV shift: 0.91 vs 0.62), and rescaling artefacts (AUC 0.91 vs 0.79). Thus, DP encoding is a simple method by which robustness to common ECG artefacts may be improved for automated ECG analysis and interpretation.

Energy-Efficient PPG-Based Respiratory Rate Estimation Using Spiking Neural Networks

Respiratory rate (RR) is a vital indicator for assessing the bodily functions and health status of patients. RR is a prominent parameter in the field of biomedical signal processing and is strongly associated with other vital signs such as blood pressure, heart rate, and heart rate variability. Various physiological signals, such as photoplethysmogram (PPG) signals, are used to extract respiratory information. RR is also estimated by detecting peak patterns and cycles in the signals through signal processing and deep-learning approaches. In this study, we propose an end-to-end RR estimation approach based on a third-generation artificial neural network model-spiking neural network. The proposed model employs PPG segments as inputs, and directly converts them into sequential spike events. This design aims to reduce information loss during the conversion of the input data into spike events. In addition, we use feedback-based integrate-and-fire neurons as the activation functions, which effectively transmit temporal information. The network is evaluated using the BIDMC respiratory dataset with three different window sizes (16, 32, and 64 s). The proposed model achieves mean absolute errors of 1.37 ± 0.04, 1.23 ± 0.03, and 1.15 ± 0.07 for the 16, 32, and 64 s window sizes, respectively. Furthermore, it demonstrates superior energy efficiency compared with other deep learning models. This study demonstrates the potential of the spiking neural networks for RR monitoring, offering a novel approach for RR estimation from the PPG signal.

Machine unlearning in brain-inspired neural network paradigms

Machine unlearning, which is crucial for data privacy and regulatory compliance, involves the selective removal of specific information from a machine learning model. This study focuses on implementing machine unlearning in Spiking Neuron Models (SNMs) that closely mimic biological neural network behaviors, aiming to enhance both flexibility and ethical compliance of AI models. We introduce a novel hybrid approach for machine unlearning in SNMs, which combines selective synaptic retraining, synaptic pruning, and adaptive neuron thresholding. This methodology is designed to effectively eliminate targeted information while preserving the overall integrity and performance of the neural network. Extensive experiments were conducted on various computer vision datasets to assess the impact of machine unlearning on critical performance metrics such as accuracy, precision, recall, and ROC AUC. Our findings indicate that the hybrid approach not only maintains but in some cases enhances the neural network's performance post-unlearning. The results confirm the practicality and efficiency of our approach, underscoring its applicability in real-world AI systems.

Vector Symbolic Finite State Machines in Attractor Neural Networks

Hopfield attractor networks are robust distributed models of human memory, but they lack a general mechanism for effecting state-dependent attractor transitions in response to input. We propose construction rules such that an attractor network may implement an arbitrary finite state machine (FSM), where states and stimuli are represented by high-dimensional random vectors and all state transitions are enacted by the attractor network's dynamics. Numerical simulations show the capacity of the model, in terms of the maximum size of implementable FSM, to be linear in the size of the attractor network for dense bipolar state vectors and approximately quadratic for sparse binary state vectors. We show that the model is robust to imprecise and noisy weights, and so a prime candidate for implementation with high-density but unreliable devices. By endowing attractor networks with the ability to emulate arbitrary FSMs, we propose a plausible path by which FSMs could exist as a distributed computational primitive in biological neural networks.

Topological features of spike trains in recurrent spiking neural networks that are trained to generate spatiotemporal patterns

In this study, we focus on training recurrent spiking neural networks to generate spatiotemporal patterns in the form of closed two-dimensional trajectories. Spike trains in the trained networks are examined in terms of their dissimilarity using the Victor-Purpura distance. We apply algebraic topology methods to the matrices obtained by rank-ordering the entries of the distance matrices, specifically calculating the persistence barcodes and Betti curves. By comparing the features of different types of output patterns, we uncover the complex relations between low-dimensional target signals and the underlying multidimensional spike trains.

Sparse-firing regularization methods for spiking neural networks with time-to-first-spike coding

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first-spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike-timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.

Unsupervised Spiking Neural Network with Dynamic Learning of Inhibitory Neurons

A spiking neural network (SNN) is a type of artificial neural network that operates based on discrete spikes to process timing information, similar to the manner in which the human brain processes real-world problems. In this paper, we propose a new spiking neural network (SNN) based on conventional, biologically plausible paradigms, such as the leaky integrate-and-fire model, spike timing-dependent plasticity, and the adaptive spiking threshold, by suggesting new biological models; that is, dynamic inhibition weight change, a synaptic wiring method, and Bayesian inference. The proposed network is designed for image recognition tasks, which are frequently used to evaluate the performance of conventional deep neural networks. To manifest the bio-realistic neural architecture, the learning is unsupervised, and the inhibition weight is dynamically changed; this, in turn, affects the synaptic wiring method based on Hebbian learning and the neuronal population. In the inference phase, Bayesian inference successfully classifies the input digits by counting the spikes from the responding neurons. The experimental results demonstrate that the proposed biological model ensures a performance improvement compared with other biologically plausible SNN models.

Patient-Specific Preictal Pattern-Aware Epileptic Seizure Prediction with Federated Learning

Electroencephalography (EEG) signals are the primary source for discriminating the preictal from the interictal stage, enabling early warnings before the seizure onset. Epileptic siezure prediction systems face significant challenges due to data scarcity, diversity, and privacy. This paper proposes a three-tier architecture for epileptic seizure prediction associated with the Federated Learning (FL) model, which is able to achieve enhanced capability by utilizing a significant number of seizure patterns from globally distributed patients while maintaining data privacy. The determination of the preictal state is influenced by global and local model-assisted decision making by modeling the two-level edge layer. The Spiking Encoder (SE), integrated with the Graph Convolutional Neural Network (Spiking-GCNN), works as the local model trained using a bi-timescale approach. Each local model utilizes the aggregated seizure knowledge obtained from the different medical centers through FL and determines the preictal probability in the coarse-grained personalization. The Adaptive Neuro-Fuzzy Inference System (ANFIS) is utilized in fine-grained personalization to recognize epileptic seizure patients by examining the outcomes of the FL model, heart rate variability features, and patient-specific clinical features. Thus, the proposed approach achieved 96.33% sensitivity and 96.14% specificity when tested on the CHB-MIT EEG dataset when modeling was performed using the bi-timescale approach and Spiking-GCNN-based epileptic pattern learning. Moreover, the adoption of federated learning greatly assists the proposed system, yielding a 96.28% higher accuracy as a result of addressing data scarcity.

Toward robust and scalable deep spiking reinforcement learning

Deep reinforcement learning (DRL) combines reinforcement learning algorithms with deep neural networks (DNNs). Spiking neural networks (SNNs) have been shown to be a biologically plausible and energy efficient alternative to DNNs. Since the introduction of surrogate gradient approaches that allowed to overcome the discontinuity in the spike function, SNNs can now be trained with the backpropagation through time (BPTT) algorithm. While largely explored on supervised learning problems, little work has been done on investigating the use of SNNs as function approximators in DRL. Here we show how SNNs can be applied to different DRL algorithms like Deep Q-Network (DQN) and Twin-Delayed Deep Deteministic Policy Gradient (TD3) for discrete and continuous action space environments, respectively. We found that SNNs are sensitive to the additional hyperparameters introduced by spiking neuron models like current and voltage decay factors, firing thresholds, and that extensive hyperparameter tuning is inevitable. However, we show that increasing the simulation time of SNNs, as well as applying a two-neuron encoding to the input observations helps reduce the sensitivity to the membrane parameters. Furthermore, we show that randomizing the membrane parameters, instead of selecting uniform values for all neurons, has stabilizing effects on the training. We conclude that SNNs can be utilized for learning complex continuous control problems with state-of-the-art DRL algorithms. While the training complexity increases, the resulting SNNs can be directly executed on neuromorphic processors and potentially benefit from their high energy efficiency.

Comparison and Selection of Spike Encoding Algorithms for SNN on FPGA

The information in Spiking Neural Networks (SNNs) is carried by discrete spikes. Therefore, the conversion between the spiking signals and real-value signals has an important impact on the encoding efficiency and performance of SNNs, which is usually completed by spike encoding algorithms. In order to select suitable spike encoding algorithms for different SNNs, this work evaluates four commonly used spike encoding algorithms. The evaluation is based on the FPGA implementation results of the algorithms, including calculation speed, resource consumption, accuracy, and anti-noiseability, so as to better adapt to the neuromorphic implementation of SNN. Two real-world applicaitons are also used to verify the evaluation results. By analyzing and comparing the evaluation results, this work summarizes the characteristics and application range of different algorithms. In general, the sliding window algorithm has relatively low accuracy and is suitable for observing signal trends. Pulsewidth modulated-Based algorithm and step-forward algorithm are suitable for accurate reconstruction of various signals except for square wave signals, while Ben's Spiker algorithm can remedy this. Finally, a scoring method that can be used for spiking coding algorithm selection is proposed, which can help to improve the encoding efficiency of neuromorphic SNNs.

Spike encoding techniques for IoT time-varying signals benchmarked on a neuromorphic classification task

Spiking Neural Networks (SNNs), known for their potential to enable low energy consumption and computational cost, can bring significant advantages to the realm of embedded machine learning for edge applications. However, input coming from standard digital sensors must be encoded into spike trains before it can be elaborated with neuromorphic computing technologies. We present here a detailed comparison of available spike encoding techniques for the translation of time-varying signals into the event-based signal domain, tested on two different datasets both acquired through commercially available digital devices: the Free Spoken Digit dataset (FSD), consisting of 8-kHz audio files, and the WISDM dataset, composed of 20-Hz recordings of human activity through mobile and wearable inertial sensors. We propose a complete pipeline to benchmark these encoding techniques by performing time-dependent signal classification through a Spiking Convolutional Neural Network (sCNN), including a signal preprocessing step consisting of a bank of filters inspired by the human cochlea, feature extraction by production of a sonogram, transfer learning via an equivalent ANN, and model compression schemes aimed at resource optimization. The resulting performance comparison and analysis provides a powerful practical tool, empowering developers to select the most suitable coding method based on the type of data and the desired processing algorithms, and further expands the applicability of neuromorphic computational paradigms to embedded sensor systems widely employed in the IoT and industrial domains.

Machine learning with a snapshot of data: Spiking neural network 'predicts' reinforcement histories of pigeons' choice behavior

An accumulated body of choice research has demonstrated that choice behavior can be understood within the context of its history of reinforcement by measuring response patterns. Traditionally, work on predicting choice behaviors has been based on the relationship between the history of reinforcement—the reinforcer arrangement used in training conditions—and choice behavior. We suggest an alternative method that treats the reinforcement history as unknown and focuses only on operant choices to accurately predict (more precisely, retrodict) reinforcement histories. We trained machine learning models known as artificial spiking neural networks (SNNs) on previously published pigeon datasets to detect patterns in choices with specific reinforcement histories—seven arranged concurrent variable‐interval schedules in effect for nine reinforcers. Notably, SNN extracted information from a small ‘window’ of observational data to predict reinforcer arrangements. The models' generalization ability was then tested with new choices of the same pigeons to predict the type of schedule used in training. We examined whether the amount of the data provided affected the prediction accuracy and our results demonstrated that choices made by the pigeons immediately after the delivery of reinforcers provided sufficient information for the model to determine the reinforcement history. These results support the idea that SNNs can process small sets of behavioral data for pattern detection, when the reinforcement history is unknown. This novel approach can influence our decisions to determine appropriate interventions; it can be a valuable addition to our toolbox, for both therapy design and research.