A Time Series Forecasting Approach Based on Nonlinear Spiking Neural Systems

Nonlinear spiking neural P (NSNP) systems are a recently developed theoretical model, which is abstracted by nonlinear spiking mechanism of biological neurons. NSNP systems have a nonlinear structure and the potential to describe nonlinear dynamic systems. Based on NSNP systems, a novel time series forecasting approach is developed in this paper. During the training phase, a time series is first converted to frequency domain by using a redundant wavelet transform, and then according to the frequency data, an NSNP system is automatically constructed and adaptively trained in frequency domain. Then, the well-trained NSNP system can automatically generate sequence data for future time as the prediction results. Eight benchmark time series data sets and two real-life time series data sets are utilized to compare the proposed approach with several state-of-the-art forecasting approaches. The comparison results demonstrate availability and effectiveness of the proposed forecasting approach.

Parallel Binary Image Cryptosystem Via Spiking Neural Networks Variants

Due to the inefficiency of multiple binary images encryption, a parallel binary image encryption framework based on the typical variants of spiking neural networks, spiking neural P (SNP) systems is proposed in this paper. More specifically, the two basic units in the proposed image cryptosystem, the permutation unit and the diffusion unit, are designed through SNP systems with multiple channels and polarizations (SNP-MCP systems), and SNP systems with astrocyte-like control (SNP-ALC systems), respectively. Different from the serial computing of the traditional image permutation/diffusion unit, SNP-MCP-based permutation/SNP-ALC-based diffusion unit can realize parallel computing through the parallel use of rules inside the neurons. Theoretical analysis results confirm the high efficiency of the binary image proposed cryptosystem. Security analysis experiments demonstrate the security of the proposed cryptosystem.

Design and Implementation of a Spiking Neural Network with Integrate-and-Fire Neuron Model for Pattern Recognition

In contrast to the previous artificial neural networks (ANNs), spiking neural networks (SNNs) work based on temporal coding approaches. In the proposed SNN, the number of neurons, neuron models, encoding method, and learning algorithm design are described in a correct and pellucid fashion. It is also discussed that optimizing the SNN parameters based on physiology, and maximizing the information they pass leads to a more robust network. In this paper, inspired by the "center-surround" structure of the receptive fields in the retina, and the amount of overlap that they have, a robust SNN is implemented. It is based on the Integrate-and-Fire (IF) neuron model and uses the time-to-first-spike coding to train the network by a newly proposed method. The Iris and MNIST datasets were employed to evaluate the performance of the proposed network whose accuracy, with 60 input neurons, was 96.33% on the Iris dataset. The network was trained in only 45 iterations indicating its reasonable convergence rate. For the MNIST dataset, when the gray level of each pixel was considered as input to the network, 600 input neurons were required, and the accuracy of the network was 90.5%. Next, 14 structural features were used as input. Therefore, the number of input neurons decreased to 210, and accuracy increased up to 95%, meaning that an SNN with fewer input neurons and good skill was implemented. Also, the ABIDE1 dataset is applied to the proposed SNN. Of the 184 data, 79 are used for healthy people and 105 for people with autism. One of the characteristics that can differentiate between these two classes is the entropy of the existing data. Therefore, Shannon entropy is used for feature extraction. Applying these values to the proposed SNN, an accuracy of 84.42% was achieved by only 120 iterations, which is a good result compared to the recent results.

On the Modeling of the Three Types of Non-spiking Neurons of the Caenorhabditis elegans

The nematode Caenorhabditis elegans (C. elegans) is a well-known model organism in neuroscience. The relative simplicity of its nervous system, made up of few hundred neurons, shares some essential features with more sophisticated nervous systems, including the human one. If we are able to fully characterize the nervous system of this organism, we will be one step closer to understanding the mechanisms underlying the behavior of living things. Following a recently conducted electrophysiological survey on different C. elegans neurons, this paper aims at modeling the three non-spiking RIM, AIY and AFD neurons (arbitrarily named with three upper case letters by convention). To date, they represent the three possible forms of non-spiking neuronal responses of the C. elegans. To achieve this objective, we propose a conductance-based neuron model adapted to the electrophysiological features of each neuron. These features are based on current biological research and a series of in-silico experiments which use differential evolution to fit the model to experimental data. From the obtained results, we formulate a series of biological hypotheses regarding currents involved in the neuron dynamics. These models reproduce experimental data with a high degree of accuracy while being biologically consistent with state-of-the-art research.

Dendrite P Systems Toolbox: Representation, Algorithms and Simulators

Dendrite P systems (DeP systems) are a recently introduced neural-like model of computation. They provide an alternative to the more classical spiking neural (SN) P systems. In this paper, we present the first software simulator for DeP systems, and we investigate the key features of the representation of the syntax and semantics of such systems. First, the conceptual design of a simulation algorithm is discussed. This is helpful in order to shade a light on the differences with simulators for SN P systems, and also to identify potential parallelizable parts. Second, a novel simulator implemented within the P-Lingua simulation framework is presented. Moreover, MeCoSim, a GUI tool for abstract representation of problems based on P system models has been extended to support this model. An experimental validation of this simulator is also covered.

Quaternion Spiking and Quaternion Quantum Neural Networks: Theory and Applications

Biological evidence shows that there are neural networks specialized for recognition of signals and patterns acting as associative memories. The spiking neural networks are another kind which receive input from a broad range of other brain areas to produce output that selects particular cognitive or motor actions to perform. An important contribution of this work is to consider the geometric processing in the modeling of feed-forward neural networks. Since quaternions are well suited to represent 3D rotations, it is then well justified to extend real-valued neural networks to quaternion-valued neural networks for task of perception and control of robot manipulators. This work presents the quaternion spiking neural networks which are able to control robots, where the examples confirm that these artificial neurons have the capacity to adapt on-line the robot to reach the desired position. Also, we present the quaternionic quantum neural networks for pattern recognition using just one quaternion neuron. In the experimental analysis, we show the excellent performance of both quaternion neural networks.

An Adaptive Optimization Spiking Neural P System for Binary Problems

Optimization Spiking Neural P System (OSNPS) is the first membrane computing model to directly derive an approximate solution of combinatorial problems with a specific reference to the 0/1 knapsack problem. OSNPS is composed of a family of parallel Spiking Neural P Systems (SNPS) that generate candidate solutions of the binary combinatorial problem and a Guider algorithm that adjusts the spiking probabilities of the neurons of the P systems. Although OSNPS is a pioneering structure in membrane computing optimization, its performance is competitive with that of modern and sophisticated metaheuristics for the knapsack problem only in low dimensional cases. In order to overcome the limitations of OSNPS, this paper proposes a novel Dynamic Guider algorithm which employs an adaptive learning and a diversity-based adaptation to control its moving operators. The resulting novel membrane computing model for optimization is here named Adaptive Optimization Spiking Neural P System (AOSNPS). Numerical result shows that the proposed approach is effective to solve the 0/1 knapsack problems and outperforms multiple various algorithms proposed in the literature to solve the same class of problems even for a large number of items (high dimensionality). Furthermore, case studies show that a AOSNPS is effective in fault sections estimation of power systems in different types of fault cases: including a single fault, multiple faults and multiple faults with incomplete and uncertain information in the IEEE 39 bus system and IEEE 118 bus system.

A Complete Arithmetic Calculator Constructed from Spiking Neural P Systems and its Application to Information Fusion

Several variants of spiking neural P systems (SNPS) have been presented in the literature to perform arithmetic operations. However, each of these variants was designed only for one specific arithmetic operation. In this paper, a complete arithmetic calculator implemented by SNPS is proposed. An application of the proposed calculator to information fusion is also proposed. The information fusion is implemented by integrating the following three elements: (1) an addition and subtraction SNPS already reported in the literature; (2) a modified multiplication and division SNPS; (3) a novel storage SNPS, i.e. a method based on SNPS is introduced to calculate basic probability assignment of an event. This is the first attempt to apply arithmetic operation SNPS to fuse multiple information. The effectiveness of the presented general arithmetic SNPS calculator is verified by means of several examples.

Medical Image Fusion Method Based on Coupled Neural P Systems in Nonsubsampled Shearlet Transform Domain

Coupled neural P (CNP) systems are a recently developed Turing-universal, distributed and parallel computing model, combining the spiking and coupled mechanisms of neurons. This paper focuses on how to apply CNP systems to handle the fusion of multi-modality medical images and proposes a novel image fusion method. Based on two CNP systems with local topology, an image fusion framework in nonsubsampled shearlet transform (NSST) domain is designed, where the two CNP systems are used to control the fusion of low-frequency NSST coefficients. The proposed fusion method is evaluated on 20 pairs of multi-modality medical images and compared with seven previous fusion methods and two deep-learning-based fusion methods. Quantitative and qualitative experimental results demonstrate the advantage of the proposed fusion method in terms of visual quality and fusion performance.

Spiking Neural P Systems with Delay on Synapses

Based on the feature and communication of neurons in animal neural systems, spiking neural P systems (SN P systems) were proposed as a kind of powerful computing model. Considering the length of axons and the information transmission speed on synapses, SN P systems with delay on synapses (SNP-DS systems) are proposed in this work. Unlike the traditional SN P systems, where all the postsynaptic neurons receive spikes at the same instant from their presynaptic neuron, the postsynaptic neurons in SNP-DS systems would receive spikes at different instants, depending on the delay time on the synapses connecting them. It is proved that the SNP-DS systems are universal as number generators. Two small universal SNP-DS systems, with standard or extended rules, are constructed to compute functions, using 56 and 36 neurons, respectively. Moreover, a simulator has been provided, in order to check the correctness of these two SNP-DS systems, thus providing an experimental validation of the universality of the systems designed.

How the Cerebellum and Prefrontal Cortex Cooperate During Trace Eyeblinking Conditioning

Several data have demonstrated that during the widely used experimental paradigm for studying associative learning, trace eye blinking conditioning (TEBC), there is a strong interaction between cerebellum and medial prefrontal cortex (mPFC). Despite this evidence, the neural mechanisms underlying this interaction are still not clear. Here, we propose a neurophysiologically plausible computational model to address this issue. The model is constrained on the basis of two critical anatomo-physiological features: (i) the cerebello-cortical organization through two circuits, respectively, targeting M1 and mPFC; (ii) the different timing in the plasticity mechanisms of these parallel circuits produced by the granule cells time sensitivity according to which different subpopulations are active at different moments during conditioned stimuli. The computer simulations run with the model suggest that these features are critical to understand how the cooperation between cerebellum and mPFC supports motor areas during TEBC. In particular, a greater trace interval produces greater plasticity changes at the slow path synapses involving mPFC with respect to plasticity changes at the fast path involving M1. As a consequence, the greater is the trace interval, the stronger is the mPFC involvement. The model has been validated by reproducing data collected through recent real mice experiments.

Temporal Backpropagation for Spiking Neural Networks with One Spike per Neuron

We propose a new supervised learning rule for multilayer spiking neural networks (SNNs) that use a form of temporal coding known as rank-order-coding. With this coding scheme, all neurons fire exactly one spike per stimulus, but the firing order carries information. In particular, in the readout layer, the first neuron to fire determines the class of the stimulus. We derive a new learning rule for this sort of network, named S4NN, akin to traditional error backpropagation, yet based on latencies. We show how approximated error gradients can be computed backward in a feedforward network with any number of layers. This approach reaches state-of-the-art performance with supervised multi-fully connected layer SNNs: test accuracy of 97.4% for the MNIST dataset, and 99.2% for the Caltech Face/Motorbike dataset. Yet, the neuron model that we use, nonleaky integrate-and-fire, is much simpler than the one used in all previous works. The source codes of the proposed S4NN are publicly available at https://github.com/SRKH/S4NN .

Nonlinear Spiking Neural P Systems

This paper proposes a new variant of spiking neural P systems (in short, SNP systems), nonlinear spiking neural P systems (in short, NSNP systems). In NSNP systems, the state of each neuron is denoted by a real number, and a real configuration vector is used to characterize the state of the whole system. A new type of spiking rules, nonlinear spiking rules, is introduced to handle the neuron’s firing, where the consumed and generated amounts of spikes are often expressed by the nonlinear functions of the state of the neuron. NSNP systems are a class of distributed parallel and nondeterministic computing systems. The computational power of NSNP systems is discussed. Specifically, it is proved that NSNP systems as number-generating/accepting devices are Turing-universal. Moreover, we establish two small universal NSNP systems for function computing and number generator, containing 117 neurons and 164 neurons, respectively.

Descriptor Selection Improvements for Quantitative Structure-Activity Relationships

Molecular descriptor selection is an essential procedure to improve a predictive quantitative structure–activity relationship (QSAR) model. However, within the QSAR model, there are a number of redundant, noisy and irrelevant descriptors. In this study, we propose a novel descriptor selection framework using self-paced learning (SPL) via sparse logistic regression (LR) with Logsum penalty (SPL-Logsum), which can simultaneously adaptively identify the simple and complex samples and avoid over-fitting. SPL is inspired by the learning process of humans or animals gradually learned from simple and complex samples to train models, and the Logsum penalized LR helps to select a small subset of significant molecular descriptors for improving the QSAR models. Experimental results on some simulations and three public QSAR datasets show that our proposed SPL-Logsum framework outperforms other existing sparse methods regarding the area under the curve, sensitivity, specificity, accuracy, and [Formula: see text]-values.

Neurons with Multiplicative Interactions of Nonlinear Synapses

Neurons are the fundamental units of the brain and nervous system. Developing a good modeling of human neurons is very important not only to neurobiology but also to computer science and many other fields. The McCulloch and Pitts neuron model is the most widely used neuron model, but has long been criticized as being oversimplified in view of properties of real neuron and the computations they perform. On the other hand, it has become widely accepted that dendrites play a key role in the overall computation performed by a neuron. However, the modeling of the dendritic computations and the assignment of the right synapses to the right dendrite remain open problems in the field. Here, we propose a novel dendritic neural model (DNM) that mimics the essence of known nonlinear interaction among inputs to the dendrites. In the model, each input is connected to branches through a distance-dependent nonlinear synapse, and each branch performs a simple multiplication on the inputs. The soma then sums the weighted products from all branches and produces the neuron’s output signal. We show that the rich nonlinear dendritic response and the powerful nonlinear neural computational capability, as well as many known neurobiological phenomena of neurons and dendrites, may be understood and explained by the DNM. Furthermore, we show that the model is capable of learning and developing an internal structure, such as the location of synapses in the dendritic branch and the type of synapses, that is appropriate for a particular task — for example, the linearly nonseparable problem, a real-world benchmark problem — Glass classification and the directional selectivity problem.

An Attention-Based Spiking Neural Network for Unsupervised Spike-Sorting

Bio-inspired computing using artificial spiking neural networks promises performances outperforming currently available computational approaches. Yet, the number of applications of such networks remains limited due to the absence of generic training procedures for complex pattern recognition, which require the design of dedicated architectures for each situation. We developed a spike-timing-dependent plasticity (STDP) spiking neural network (SSN) to address spike-sorting, a central pattern recognition problem in neuroscience. This network is designed to process an extracellular neural signal in an online and unsupervised fashion. The signal stream is continuously fed to the network and processed through several layers to output spike trains matching the truth after a short learning period requiring only few data. The network features an attention mechanism to handle the scarcity of action potential occurrences in the signal, and a threshold adaptation mechanism to handle patterns with different sizes. This method outperforms two existing spike-sorting algorithms at low signal-to-noise ratio (SNR) and can be adapted to process several channels simultaneously in the case of tetrode recordings. Such attention-based STDP network applied to spike-sorting opens perspectives to embed neuromorphic processing of neural data in future brain implants.

Compact Hardware Synthesis of Stochastic Spiking Neural Networks

Spiking neural networks (SNN) are able to emulate real neural behavior with high confidence due to their bio-inspired nature. Many designs have been proposed for the implementation of SNN in hardware, although the realization of high-density and biologically-inspired SNN is currently a complex challenge of high scientific and technical interest. In this work, we propose a compact digital design for the implementation of high-volume SNN that considers the intrinsic stochastic processes present in biological neurons and enables high-density hardware implementation. The proposed stochastic SNN model (SSNN) is compared with previous SSNN models, achieving a higher processing speed. We also show how the proposed model can be scaled to high-volume neural networks trained by using back propagation and applied to a pattern classification task. The proposed model achieves better results compared with other recently-published SNN models configured with unsupervised STDP learning.

Fluctuation Scaling of Neuronal Firing and Bursting in Spontaneously Active Brain Circuits

We employed high-density microelectrode arrays to investigate spontaneous firing patterns of neurons in brain circuits of the primary somatosensory cortex (S1) in mice. We recorded from over 150 neurons for 10min in each of eight different experiments, identified their location in S1, sorted their action potentials (spikes), and computed their power spectra and inter-spike interval (ISI) statistics. Of all persistently active neurons, 92% fired with a single dominant frequency - regularly firing neurons (RNs) - from 1 to 8Hz while 8% fired in burst with two dominant frequencies - bursting neurons (BNs) - corresponding to the inter-burst (2-6Hz) and intra-burst intervals (20-160Hz). RNs were predominantly located in layers 2/3 and 5/6 while BNs localized to layers 4 and 5. Across neurons, the standard deviation of ISI was a power law of its mean, a property known as fluctuation scaling, with a power law exponent of 1 for RNs and 1.25 for BNs. The power law implies that firing and bursting patterns are scale invariant: the firing pattern of a given RN or BN resembles that of another RN or BN, respectively, after a time contraction or dilation. An explanation for this scale invariance is discussed in the context of previous computational studies as well as its potential role in information processing.

Proprioceptive Recognition with Artificial Neural Networks Based on Organizations of Spinocerebellar Tract and Cerebellum

Muscle kinematics and kinetics are nonlinearly encoded by proprioceptors, and the changes in muscle length and velocity are integrated into Ia afferent. Besides, proprioceptive signals from multiple muscles are probably mixed in afferent pathways, which all lead to difficulties in proprioceptive recognition for the cerebellum. In this study, the artificial neural networks, whose organizations are biologically based on the spinocerebellar tract and cerebellum, are utilized to decode the proprioceptive signals. Consistent with the controversy of the proprioceptive division in the dorsal spinocerebellar tract, the spinocerebellar tract networks incorporated two distinct inferences, (1) the centralized networks, which mixed Ia, II, and Ib and processed them together; (2) the decentralized networks, which processed Ia, II, and Ib afferents separately. The cerebellar networks were based on the Marr-Albus model to recognize the kinematic states. The networks were trained by a specific movement, and the trained networks were subsequently required to predict kinematic states of six movements. The results demonstrated that the centralized networks, which were more consistent with the physiological findings in recent years, had better recognition accuracy than the decentralized networks, and the networks were still effective even when proprioceptive afferents from multiple muscles were integrated.

How Far can Neural Correlations Reduce Uncertainty? Comparison of Information Transmission Rates for Markov and Bernoulli Processes

The nature of neural codes is central to neuroscience. Do neurons encode information through relatively slow changes in the firing rates of individual spikes (rate code) or by the precise timing of every spike (temporal code)? Here we compare the loss of information due to correlations for these two possible neural codes. The essence of Shannon's definition of information is to combine information with uncertainty: the higher the uncertainty of a given event, the more information is conveyed by that event. Correlations can reduce uncertainty or the amount of information, but by how much? In this paper we address this question by a direct comparison of the information per symbol conveyed by the words coming from a binary Markov source (temporal code) with the information per symbol coming from the corresponding Bernoulli source (uncorrelated, rate code). In a previous paper we found that a crucial role in the relation between information transmission rates (ITRs) and firing rates is played by a parameter s, which is the sum of transition probabilities from the no-spike state to the spike state and vice versa. We found that in this case too a crucial role is played by the same parameter s. We calculated the maximal and minimal bounds of the quotient of ITRs for these sources. Next, making use of the entropy grouping axiom, we determined the loss of information in a Markov source compared with the information in the corresponding Bernoulli source for a given word length. Our results show that in the case of correlated signals the loss of information is relatively small, and thus temporal codes, which are more energetically efficient, can replace rate codes effectively. These results were confirmed by experiments.