Investigation of Deep Spiking Neural Networks Utilizing Gated Schottky Diode as Synaptic Devices

Deep learning produces a remarkable performance in various applications such as image classification and speech recognition. However, state-of-the-art deep neural networks require a large number of weights and enormous computation power, which results in a bottleneck of efficiency for edge-device applications. To resolve these problems, deep spiking neural networks (DSNNs) have been proposed, given the specialized synapse and neuron hardware. In this work, the hardware neuromorphic system of DSNNs with gated Schottky diodes was investigated. Gated Schottky diodes have a near-linear conductance response, which can easily implement quantized weights in synaptic devices. Based on modeling of synaptic devices, two-layer fully connected neural networks are trained by off-chip learning. The adaptation of a neuron's threshold is proposed to reduce the accuracy degradation caused by the conversion from analog neural networks (ANNs) to event-driven DSNNs. Using left-justified rate coding as an input encoding method enables low-latency classification. The effect of device variation and noisy images to the classification accuracy is investigated. The time-to-first-spike (TTFS) scheme can significantly reduce power consumption by reducing the number of firing spikes compared to a max-firing scheme.

Implementation of an efficient magnetic tunnel junction-based stochastic neural network with application to iris data classification

Stochastic neuromorphic computation (SNC) has the potential to enable a low power, error tolerant and scalable computing platform in comparison to its deterministic counterparts. However, the hardware implementation of complementary metal oxide semiconductor (CMOS)-based stochastic circuits involves conversion blocks that cost more than the actual processing circuits. The realization of the activation function for SNCs also requires a complicated circuit that results in a significant amount of power dissipation and area overhead. The inherent probabilistic switching behavior of nanomagnets provides an advantage to overcome these complexity issues for the realization of low power and area efficient SNC systems. This paper presents magnetic tunnel junction (MTJ)-based stochastic computing methodology for the implementation of a neural network. The stochastic switching behavior of the MTJ has been exploited to design a binary to stochastic converter to mitigate the complexity of the CMOS-based design. The paper also presents the technique for realizing stochastic sigmoid activation function using an MTJ. Such circuits are simpler than existing ones and use considerably less power. An image classification system employing the proposed circuits has been implemented to verify the effectiveness of the technique. The MTJ-based SNC system shows area and energy reduction by a factor of 13.5 and 2.5, respectively, while the prediction accuracy is 86.66%. Furthermore, this paper investigates how crucial parameters, such as stochastic bitstream length, number of hidden layers and number of nodes in a hidden layer, need to be set precisely to realize an efficient MTJ-based stochastic neural network (SNN). The proposed methodology can prove a promising alternative for highly efficient digital stochastic computing applications.

An Adaptive Optimization Spiking Neural P System for Binary Problems

Optimization Spiking Neural P System (OSNPS) is the first membrane computing model to directly derive an approximate solution of combinatorial problems with a specific reference to the 0/1 knapsack problem. OSNPS is composed of a family of parallel Spiking Neural P Systems (SNPS) that generate candidate solutions of the binary combinatorial problem and a Guider algorithm that adjusts the spiking probabilities of the neurons of the P systems. Although OSNPS is a pioneering structure in membrane computing optimization, its performance is competitive with that of modern and sophisticated metaheuristics for the knapsack problem only in low dimensional cases. In order to overcome the limitations of OSNPS, this paper proposes a novel Dynamic Guider algorithm which employs an adaptive learning and a diversity-based adaptation to control its moving operators. The resulting novel membrane computing model for optimization is here named Adaptive Optimization Spiking Neural P System (AOSNPS). Numerical result shows that the proposed approach is effective to solve the 0/1 knapsack problems and outperforms multiple various algorithms proposed in the literature to solve the same class of problems even for a large number of items (high dimensionality). Furthermore, case studies show that a AOSNPS is effective in fault sections estimation of power systems in different types of fault cases: including a single fault, multiple faults and multiple faults with incomplete and uncertain information in the IEEE 39 bus system and IEEE 118 bus system.

Molecular Docking: Challenges, Advances and its Use in Drug Discovery Perspective

Molecular docking is a process through which small molecules are docked into the macromolecular structures for scoring its complementary values at the binding sites. It is a vibrant research area with dynamic utility in structure-based drug-designing, lead optimization, biochemical pathway and for drug designing being the most attractive tools. Two pillars for a successful docking experiment are correct pose and affinity prediction. Each program has its own advantages and drawbacks with respect to their docking accuracy, ranking accuracy and time consumption so a general conclusion cannot be drawn. Moreover, users don't always consider sufficient diversity in their test sets which results in certain programs to outperform others. In this review, the prime focus has been laid on the challenges of docking and troubleshooters in existing programs, underlying algorithmic background of docking, preferences regarding the use of docking programs for best results illustrated with examples, comparison of performance for existing tools and algorithms, state of art in docking, recent trends of diseases and current drug industries, evidence from clinical trials and post-marketing surveillance are discussed. These aspects of the molecular drug designing paradigm are quite controversial and challenging and this review would be an asset to the bioinformatics and drug designing communities.

An Attention-Based Spiking Neural Network for Unsupervised Spike-Sorting

Bio-inspired computing using artificial spiking neural networks promises performances outperforming currently available computational approaches. Yet, the number of applications of such networks remains limited due to the absence of generic training procedures for complex pattern recognition, which require the design of dedicated architectures for each situation. We developed a spike-timing-dependent plasticity (STDP) spiking neural network (SSN) to address spike-sorting, a central pattern recognition problem in neuroscience. This network is designed to process an extracellular neural signal in an online and unsupervised fashion. The signal stream is continuously fed to the network and processed through several layers to output spike trains matching the truth after a short learning period requiring only few data. The network features an attention mechanism to handle the scarcity of action potential occurrences in the signal, and a threshold adaptation mechanism to handle patterns with different sizes. This method outperforms two existing spike-sorting algorithms at low signal-to-noise ratio (SNR) and can be adapted to process several channels simultaneously in the case of tetrode recordings. Such attention-based STDP network applied to spike-sorting opens perspectives to embed neuromorphic processing of neural data in future brain implants.

A Stochastic Spiking Neural Network for Virtual Screening

Virtual screening (VS) has become a key computational tool in early drug design and screening performance is of high relevance due to the large volume of data that must be processed to identify molecules with the sought activity-related pattern. At the same time, the hardware implementations of spiking neural networks (SNNs) arise as an emerging computing technique that can be applied to parallelize processes that normally present a high cost in terms of computing time and power. Consequently, SNN represents an attractive alternative to perform time-consuming processing tasks, such as VS. In this brief, we present a smart stochastic spiking neural architecture that implements the ultrafast shape recognition (USR) algorithm achieving two order of magnitude of speed improvement with respect to USR software implementations. The neural system is implemented in hardware using field-programmable gate arrays allowing a highly parallelized USR implementation. The results show that, due to the high parallelization of the system, millions of compounds can be checked in reasonable times. From these results, we can state that the proposed architecture arises as a feasible methodology to efficiently enhance time-consuming data-mining processes such as 3-D molecular similarity search.

Optimal path-finding through mental exploration based on neural energy field gradients

Rodent animal can accomplish self-locating and path-finding task by forming a cognitive map in the hippocampus representing the environment. In the classical model of the cognitive map, the system (artificial animal) needs large amounts of physical exploration to study spatial environment to solve path-finding problems, which costs too much time and energy. Although Hopfield's mental exploration model makes up for the deficiency mentioned above, the path is still not efficient enough. Moreover, his model mainly focused on the artificial neural network, and clear physiological meanings has not been addressed. In this work, based on the concept of mental exploration, neural energy coding theory has been applied to the novel calculation model to solve the path-finding problem. Energy field is constructed on the basis of the firing power of place cell clusters, and the energy field gradient can be used in mental exploration to solve path-finding problems. The study shows that the new mental exploration model can efficiently find the optimal path, and present the learning process with biophysical meaning as well. We also analyzed the parameters of the model which affect the path efficiency. This new idea verifies the importance of place cell and synapse in spatial memory and proves that energy coding is effective to study cognitive activities. This may provide the theoretical basis for the neural dynamics mechanism of spatial memory.

Defense Against Chip Cloning Attacks Based on Fractional Hopfield Neural Networks

This paper presents a state-of-the-art application of fractional hopfield neural networks (FHNNs) to defend against chip cloning attacks, and provides insight into the reason that the proposed method is superior to physically unclonable functions (PUFs). In the past decade, PUFs have been evolving as one of the best types of hardware security. However, the development of the PUFs has been somewhat limited by its implementation cost, its temperature variation effect, its electromagnetic interference effect, the amount of entropy in it, etc. Therefore, it is imperative to discover, through promising mathematical methods and physical modules, some novel mechanisms to overcome the aforementioned weaknesses of the PUFs. Motivated by this need, in this paper, we propose applying the FHNNs to defend against chip cloning attacks. At first, we implement the arbitrary-order fractor of a FHNN. Secondly, we describe the implementation cost of the FHNNs. Thirdly, we propose the achievement of the constant-order performance of a FHNN when ambient temperature varies. Fourthly, we analyze the electrical performance stability of the FHNNs under electromagnetic disturbance conditions. Fifthly, we study the amount of entropy of the FHNNs. Lastly, we perform experiments to analyze the pass-band width of the fractor of an arbitrary-order FHNN and the defense against chip cloning attacks capability of the FHNNs. In particular, the capabilities of defense against chip cloning attacks, anti-electromagnetic interference, and anti-temperature variation of a FHNN are illustrated experimentally in detail. Some significant advantages of the FHNNs are that their implementation cost is considerably lower than that of the PUFs, their electrical performance is much more stable than that of the PUFs under different temperature conditions, their electrical performance stability of the FHNNs under electromagnetic disturbance conditions is much more robust than that of the PUFs, and their amount of entropy is significantly higher than that of the PUFs with the same rank circuit scale.

Asynchronous Detection of Trials Onset from Raw EEG Signals

Clinical processing of event-related potentials (ERPs) requires a precise synchrony between the stimulation and the acquisition units that are guaranteed by means of a physical link between them. This precise synchrony is needed since temporal misalignments during trial averaging can lead to high deviations of peak times, thus causing error in diagnosis or inefficiency in classification in brain-computer interfaces (BCIs). Out of the laboratory, mobile EEG systems and BCI headsets are not provided with the physical link, thus being inadequate for acquisition of ERPs. In this study, we propose a method for the asynchronous detection of trials onset from raw EEG without physical links. We validate it with a BCI application based on the dichotic listening task. The user goal was to attend the cued auditory message and to report three keywords contained in it while ignoring the other message. The BCI goal was to detect the attended message from the analysis of auditory ERPs. The rate of successful onset detection in both synchronous (using the real onset) and asynchronous (blind detection of trial onset from raw EEG) was 73% with a synchronization error of less than 1[Formula: see text]ms. The level of synchronization provided by this proposal would allow home-based acquisition of ERPs with low cost BCI headsets and any media player unit without physical links between them.

High-Density Liquid-State Machine Circuitry for Time-Series Forecasting

Spiking neural networks (SNN) are the last neural network generation that try to mimic the real behavior of biological neurons. Although most research in this area is done through software applications, it is in hardware implementations in which the intrinsic parallelism of these computing systems are more efficiently exploited. Liquid state machines (LSM) have arisen as a strategic technique to implement recurrent designs of SNN with a simple learning methodology. In this work, we show a new low-cost methodology to implement high-density LSM by using Boolean gates. The proposed method is based on the use of probabilistic computing concepts to reduce hardware requirements, thus considerably increasing the neuron count per chip. The result is a highly functional system that is applied to high-speed time series forecasting.

A New Stochastic Computing Methodology for Efficient Neural Network Implementation

This paper presents a new methodology for the hardware implementation of neural networks (NNs) based on probabilistic laws. The proposed encoding scheme circumvents the limitations of classical stochastic computing (based on unipolar or bipolar encoding) extending the representation range to any real number using the ratio of two bipolar-encoded pulsed signals. Furthermore, the novel approach presents practically a total noise-immunity capability due to its specific codification. We introduce different designs for building the fundamental blocks needed to implement NNs. The validity of the present approach is demonstrated through a regression and a pattern recognition task. The low cost of the methodology in terms of hardware, along with its capacity to implement complex mathematical functions (such as the hyperbolic tangent), allows its use for building highly reliable systems and parallel computing.

Real-Time EEG-Based Detection of Fatigue Driving Danger for Accident Prediction

This paper proposes a real-time electroencephalogram (EEG)-based detection method of the potential danger during fatigue driving. To determine driver fatigue in real time, wavelet entropy with a sliding window and pulse coupled neural network (PCNN) were used to process the EEG signals in the visual area (the main information input route). To detect the fatigue danger, the neural mechanism of driver fatigue was analyzed. The functional brain networks were employed to track the fatigue impact on processing capacity of brain. The results show the overall functional connectivity of the subjects is weakened after long time driving tasks. The regularity is summarized as the fatigue convergence phenomenon. Based on the fatigue convergence phenomenon, we combined both the input and global synchronizations of brain together to calculate the residual amount of the information processing capacity of brain to obtain the dangerous points in real time. Finally, the danger detection system of the driver fatigue based on the neural mechanism was validated using accident EEG. The time distributions of the output danger points of the system have a good agreement with those of the real accident points.

Exploring a Type of Central Pattern Generator Based on Hindmarsh–Rose Model: From Theory to Application

This paper proposes the idea that Hindmarsh–Rose (HR) neuronal model can be used to develop a new type of central pattern generator (CPG). Some key properties of HR model are studied and proved to meet the requirements of CPG. Pros and cons of HR model are provided. A CPG network based on HR model is developed and the related properties are investigated. We explore the bipedal primary gaits generated by the CPG network. The preliminary applications of HR model are tested on humanoid locomotion model and functional electrical stimulation (FES) walking system. The positive results of stimulation and experiment show the feasibility of HR model as a valid CPG.

THE SUPER-TURING COMPUTATIONAL POWER OF PLASTIC RECURRENT NEURAL NETWORKS

We study the computational capabilities of a biologically inspired neural model where the synaptic weights, the connectivity pattern, and the number of neurons can evolve over time rather than stay static. Our study focuses on the mere concept of plasticity of the model so that the nature of the updates is assumed to be not constrained. In this context, we show that the so-called plastic recurrent neural networks (RNNs) are capable of the precise super-Turing computational power — as the static analog neural networks — irrespective of whether their synaptic weights are modeled by rational or real numbers, and moreover, irrespective of whether their patterns of plasticity are restricted to bi-valued updates or expressed by any other more general form of updating. Consequently, the incorporation of only bi-valued plastic capabilities in a basic model of RNNs suffices to break the Turing barrier and achieve the super-Turing level of computation. The consideration of more general mechanisms of architectural plasticity or of real synaptic weights does not further increase the capabilities of the networks. These results support the claim that the general mechanism of plasticity is crucially involved in the computational and dynamical capabilities of biological neural networks. They further show that the super-Turing level of computation reflects in a suitable way the capabilities of brain-like models of computation.

ADAPTATION-DEPENDENT SYNCHRONIZATION TRANSITIONS AND BURST GENERATIONS IN ELECTRICALLY COUPLED NEURAL NETWORKS

A typical feature of neurons is their ability to encode neural information dynamically through spike frequency adaptation (SFA). Previous studies of SFA on neuronal synchronization were mainly concentrated on the correlated firing between neuron pairs, while the synchronization of neuron populations in the presence of SFA is still unclear. In this study, the influence of SFA on the population synchronization of neurons was numerically explored in electrically coupled networks, with regular, small-world, and random connectivity, respectively. The simulation results indicate that cross-correlation indices decrease significantly when the neurons have adaptation compared with those of nonadapting neurons, similar to previous experimental observations. However, the synchronous activity of population neurons exhibits a rather complex adaptation-dependent manner. Specifically, synchronization strength of neuron populations changes nonmonotonically, depending on the degree of adaptation. In addition, single neurons in the networks can switch from regular spiking to bursting with the increase of adaptation degree. Furthermore, the connection probability among neurons exhibits significant influence on the population synchronous activity, but has little effect on the burst generation of single neurons. Accordingly, the results may suggest that synchronous activity and burst firing of population neurons are both adaptation-dependent.

A new work mechanism on neuronal activity

By re-examining the neuronal activity energy model, we show the inadequacies in the current understanding of the energy consumption associated with neuron activity. Specifically, we show computationally that a neuron first absorbs and then consumes energy during firing action potential, and this result cannot be produced from any current neuron models or biological neural networks. Based on this finding, we provide an explanation for the observation that when neurons are excited in the brain, blood flow increases significantly while the incremental oxygen consumption is very small. We can also explain why external stimulation and perception emergence are synchronized. We also show that negative energy presence in neurons at the sub-threshold state is an essential reason that leads to blood flow incremental response time in the brain rather than neural excitation to delay.

AN OPTIMIZATION SPIKING NEURAL P SYSTEM FOR APPROXIMATELY SOLVING COMBINATORIAL OPTIMIZATION PROBLEMS

Membrane systems (also called P systems) refer to the computing models abstracted from the structure and the functioning of the living cell as well as from the cooperation of cells in tissues, organs, and other populations of cells. Spiking neural P systems (SNPS) are a class of distributed and parallel computing models that incorporate the idea of spiking neurons into P systems. To attain the solution of optimization problems, P systems are used to properly organize evolutionary operators of heuristic approaches, which are named as membrane-inspired evolutionary algorithms (MIEAs). This paper proposes a novel way to design a P system for directly obtaining the approximate solutions of combinatorial optimization problems without the aid of evolutionary operators like in the case of MIEAs. To this aim, an extended spiking neural P system (ESNPS) has been proposed by introducing the probabilistic selection of evolution rules and multi-neurons output and a family of ESNPS, called optimization spiking neural P system (OSNPS), are further designed through introducing a guider to adaptively adjust rule probabilities to approximately solve combinatorial optimization problems. Extensive experiments on knapsack problems have been reported to experimentally prove the viability and effectiveness of the proposed neural system.

OPTIMAL SPARSE APPROXIMATION WITH INTEGRATE AND FIRE NEURONS

Sparse approximation is a hypothesized coding strategy where a population of sensory neurons (e.g. V1) encodes a stimulus using as few active neurons as possible. We present the Spiking LCA (locally competitive algorithm), a rate encoded Spiking Neural Network (SNN) of integrate and fire neurons that calculate sparse approximations. The Spiking LCA is designed to be equivalent to the nonspiking LCA, an analog dynamical system that converges on a ℓ1-norm sparse approximations exponentially. We show that the firing rate of the Spiking LCA converges on the same solution as the analog LCA, with an error inversely proportional to the sampling time. We simulate in NEURON a network of 128 neuron pairs that encode 8 × 8 pixel image patches, demonstrating that the network converges to nearly optimal encodings within 20 ms of biological time. We also show that when using more biophysically realistic parameters in the neurons, the gain function encourages additional ℓ0-norm sparsity in the encoding, relative both to ideal neurons and digital solvers.

BEHIND THE MAGICAL NUMBERS: HIERARCHICAL CHUNKING AND THE HUMAN WORKING MEMORY CAPACITY

To explore the influence of chunking on the capacity limits of working memory, a model for chunking in sequential working memory is proposed, using hierarchical bidirectional inhibition-connected neural networks with winnerless competition. With the assumption of the existence of an upper bound to the inhibitory weights in neurobiological networks, it is shown that chunking increases the number of memorized items in working memory from the "magical number 7" to 16 items. The optimal number of chunks and the number of the memorized items in each chunk are the "magical number 4".