Competitive Decomposition-Based Multiobjective Architecture Search for the Dendritic Neural Model

The dendritic neural model (DNM) is computationally faster than other machine-learning techniques, because its architecture can be implemented by using logic circuits and its calculations can be performed entirely in binary form. To further improve the computational speed, a straightforward approach is to generate a more concise architecture for the DNM. Actually, the architecture search is a large-scale multiobjective optimization problem (LSMOP), where a large number of parameters need to be set with the aim of optimizing accuracy and structural complexity simultaneously. However, the issues of irregular Pareto front, objective discontinuity, and population degeneration strongly limit the performances of conventional multiobjective evolutionary algorithms (MOEAs) on the specific problem. Therefore, a novel competitive decomposition-based MOEA is proposed in this study, which decomposes the original problem into several constrained subproblems, with neighboring subproblems sharing overlapping regions in the objective space. The solutions in the overlapping regions participate in environmental selection for the neighboring subproblems and then propagate the selection pressure throughout the entire population. Experimental results demonstrate that the proposed algorithm can possess a more powerful optimization ability than the state-of-the-art MOEAs. Furthermore, both the DNM itself and its hardware implementation can achieve very competitive classification performances when trained by the proposed algorithm, compared with numerous widely used machine-learning approaches.

Grid cells, border cells, and discrete complex analysis

We propose a mechanism enabling the appearance of border cells-neurons firing at the boundaries of the navigated enclosures. The approach is based on the recent discovery of discrete complex analysis on a triangular lattice, which allows constructing discrete epitomes of complex-analytic functions and making use of their inherent ability to attain maximal values at the boundaries of generic lattice domains. As it turns out, certain elements of the discrete-complex framework readily appear in the oscillatory models of grid cells. We demonstrate that these models can extend further, producing cells that increase their activity toward the frontiers of the navigated environments. We also construct a network model of neurons with border-bound firing that conforms with the oscillatory models.

Single-layer perceptron artificial visual system for orientation detection

Orientation detection is an essential function of the visual system. In our previous works, we have proposed a new orientation detection mechanism based on local orientation-selective neurons. We assume that there are neurons solely responsible for orientation detection, with each neuron dedicated to detecting a specific local orientation. The global orientation is inferred from the local orientation information. Based on this mechanism, we propose an artificial visual system (AVS) by utilizing a single-layer of McCulloch-Pitts neurons to realize these local orientation-sensitive neurons and a layer of sum pooling to realize global orientation detection neurons. We demonstrate that such a single-layer perceptron artificial visual system (AVS) is capable of detecting global orientation by identifying the orientation with the largest number of activated orientation-selective neurons as the global orientation. To evaluate the effectiveness of this single-layer perceptron AVS, we perform computer simulations. The results show that the AVS works perfectly for global orientation detection, aligning with the majority of physiological experiments and models. Moreover, we compare the performance of the single-layer perceptron AVS with that of a traditional convolutional neural network (CNN) on orientation detection tasks. We find that the single-layer perceptron AVS outperforms CNN in various aspects, including identification accuracy, noise resistance, computational and learning cost, hardware implementation feasibility, and biological plausibility.

Grid Cells, Border Cells and Discrete Complex Analysis

We propose a mechanism enabling the appearance of border cells-neurons firing at the boundaries of the navigated enclosures. The approach is based on the recent discovery of discrete complex analysis on a triangular lattice, which allows constructing discrete epitomes of complex-analytic functions and making use of their inherent ability to attain maximal values at the boundaries of generic lattice domains. As it turns out, certain elements of the discrete-complex framework readily appear in the oscillatory models of grid cells. We demonstrate that these models can extend further, producing cells that increase their activity towards the frontiers of the navigated environments. We also construct a network model of neurons with border-bound firing that conforms with the oscillatory models.

Fully Complex-Valued Dendritic Neuron Model

A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.

Single dendritic neural classification with an effective spherical search-based whale learning algorithm

McCulloch-Pitts neuron-based neural networks have been the mainstream deep learning methods, achieving breakthrough in various real-world applications. However, McCulloch-Pitts neuron is also under longtime criticism of being overly simplistic. To alleviate this issue, the dendritic neuron model (DNM), which employs non-linear information processing capabilities of dendrites, has been widely used for prediction and classification tasks. In this study, we innovatively propose a hybrid approach to co-evolve DNM in contrast to back propagation (BP) techniques, which are sensitive to initial circumstances and readily fall into local minima. The whale optimization algorithm is improved by spherical search learning to perform co-evolution through dynamic hybridizing. Eleven classification datasets were selected from the well-known UCI Machine Learning Repository. Its efficiency in our model was verified by statistical analysis of convergence speed and Wilcoxon sign-rank tests, with receiver operating characteristic curves and the calculation of area under the curve. In terms of classification accuracy, the proposed co-evolution method beats 10 existing cutting-edge non-BP methods and BP, suggesting that well-learned DNMs are computationally significantly more potent than conventional McCulloch-Pitts types and can be employed as the building blocks for the next-generation deep learning methods.

An Extension Network of Dendritic Neurons

Deep learning (DL) has achieved breakthrough successes in various tasks, owing to its layer-by-layer information processing and sufficient model complexity. However, DL suffers from the issues of both redundant model complexity and low interpretability, which are mainly because of its oversimplified basic McCulloch-Pitts neuron unit. A widely recognized biologically plausible dendritic neuron model (DNM) has demonstrated its effectiveness in alleviating the aforementioned issues, but it can only solve binary classification tasks, which significantly limits its applicability. In this study, a novel extended network based on the dendritic structure is innovatively proposed, thereby enabling it to solve multiple-class classification problems. Also, for the first time, an efficient error-back-propagation learning algorithm is derived. In the extensive experimental results, the effectiveness and superiority of the proposed method in comparison with other nine state-of-the-art classifiers on ten datasets are demonstrated, including a real-world quality of web service application. The experimental results suggest that the proposed learning algorithm is competent and reliable in terms of classification performance and stability and has a notable advantage in small-scale disequilibrium data. Additionally, aspects of network structure constrained by scale are examined.

Dendrite Net: A White-Box Module for Classification, Regression, and System Identification

The simulation of biological dendrite computations is vital for the development of artificial intelligence (AI). This article presents a basic machine-learning (ML) algorithm, called Dendrite Net or DD, just like the support vector machine (SVM) or multilayer perceptron (MLP). DD's main concept is that the algorithm can recognize this class after learning, if the output's logical expression contains the corresponding class's logical relationship among inputs (and \ or \ not). Experiments and main results: DD, a white-box ML algorithm, showed excellent system identification performance for the black-box system. Second, it was verified by nine real-world applications that DD brought better generalization capability relative to the MLP architecture that imitated neurons' cell body (Cell body Net) for regression. Third, by MNIST and FASHION-MNIST datasets, it was verified that DD showed higher testing accuracy under greater training loss than the cell body net for classification. The number of modules can effectively adjust DD's logical expression capacity, which avoids overfitting and makes it easy to get a model with outstanding generalization capability. Finally, repeated experiments in MATLAB and PyTorch (Python) demonstrated that DD was faster than Cell body Net both in epoch and forwardpropagation. The main contribution of this article is the basic ML algorithm (DD) with a white-box attribute, controllable precision for better generalization capability, and lower computational complexity. Not only can DD be used for generalized engineering, but DD has vast development potential as a module for deep learning. DD code is available at https://github.com/liugang1234567/Gang-neuron.

A novel non-linear neuron model based on multiplicative aggregation in quaternionic domain

The learning algorithm for a three-layered neural structure with novel non-linear quaternionic-valued multiplicative (QVM) neurons is proposed in this paper. The computing capability of non-linear aggregation in the cell body of biological neurons inspired the development of a non-linear neuron model. However, unlike linear neuron models, most non-linear neuron models are built on higher order aggregation, which is more mathematically complex and difficult to train. As a result, building non-linear neuron models with a simple structure is a difficult and time-consuming endeavor in the neurocomputing field. The concept of a QVM neuron model was influenced by the non-linear neuron model, which has a simple structure and the great computational ability. The suggested neuron’s linearity is determined by the weight and bias associated with each quaternionic-valued input. Non-commutative multiplication of all linearly connected quaternionic input-weight terms accommodates the non-linearity. To train three-layered networks with QVM neurons, the standard quaternionic-gradient-based backpropagation (QBP) algorithm is utilized. The computational and generalization capabilities of the QVM neuron are assessed through training and testing in the quaternionic domain utilizing benchmark problems, such as 3D and 4D chaotic time-series predictions, 3D geometrical transformations, and 3D face recognition. The training and testing outcomes are compared to conventional and root-power mean (RPM) neurons in quaternionic domain using training–testing MSEs, network topology (parameters), variance, and AIC as statistical measures. According to these findings, networks with QVM neurons have greater computational and generalization capabilities than networks with conventional and RPM neurons in quaternionic domain.

Decision-Tree-Initialized Dendritic Neuron Model for Fast and Accurate Data Classification

This work proposes a decision tree (DT)-based method for initializing a dendritic neuron model (DNM). Neural networks become larger and larger, thus consuming more and more computing resources. This calls for a strong need to prune neurons that do not contribute much to their network's output. Pruning those with low contribution may lead to a loss of accuracy of DNM. Our proposed method is novel because 1) it can reduce the number of dendrites in DNM while improving training efficiency without affecting accuracy and 2) it can select proper initialization weight and threshold of neurons. The Adam algorithm is used to train DNM after its initialization with our proposed DT-based method. To verify its effectiveness, we apply it to seven benchmark datasets. The results show that decision-tree-initialized DNM is significantly better than the original DNM, k-nearest neighbor, support vector machine, back-propagation neural network, and DT classification methods. It exhibits the lowest model complexity and highest training speed without losing any accuracy. The interactions among attributes can also be observed in its dendritic neurons.

Artificial Visual System for Orientation Detection Based on Hubel–Wiesel Model

The Hubel–Wiesel (HW) model is a classical neurobiological model for explaining the orientation selectivity of cortical cells. However, the HW model still has not been fully proved physiologically, and there are few concise but efficient systems to quantify and simulate the HW model and can be used for object orientation detection applications. To realize a straightforward and efficient quantitive method and validate the HW model’s reasonability and practicality, we use McCulloch-Pitts (MP) neuron model to simulate simple cells and complex cells and implement an artificial visual system (AVS) for two-dimensional object orientation detection. First, we realize four types of simple cells that are only responsible for detecting a specific orientation angle locally. Complex cells are realized with the sum function. Every local orientation information of an object is collected by simple cells and subsequently converged to the corresponding same type complex cells for computing global activation degree. Finally, the global orientation is obtained according to the activation degree of each type of complex cell. Based on this scheme, an AVS for global orientation detection is constructed. We conducted computer simulations to prove the feasibility and effectiveness of our scheme and the AVS. Computer simulations show that the mechanism-based AVS can make accurate orientation discrimination and shows striking biological similarities with the natural visual system, which indirectly proves the rationality of the Hubel–Wiesel model. Furthermore, compared with traditional CNN, we find that our AVS beats CNN on orientation detection tasks in identification accuracy, noise resistance, computation and learning cost, hardware implementation, and reasonability.

Artificial Visual System for Orientation Detection

The human visual system is one of the most important components of the nervous system, responsible for visual perception. The research on orientation detection, in which neurons of the visual cortex respond only to a line stimulus in a particular orientation, is an important driving force of computer vision and biological vision. However, the principle underlying orientation detection remains a mystery. In order to solve this mystery, we first propose a completely new mechanism that explains planar orientation detection in a quantitative manner. First, we assume that there are planar orientation-detective neurons which respond only to a particular planar orientation locally and that these neurons detect local planar orientation information based on nonlinear interactions that take place on the dendrites. Then, we propose an implementation of these local planar orientation-detective neurons based on their dendritic computations, use them to extract the local planar orientation information, and infer the global planar orientation information from the local planar orientation information. Furthermore, based on this mechanism, we propose an artificial visual system (AVS) for planar orientation detection and other visual information processing. In order to prove the effectiveness of our mechanism and the AVS, we conducted a series of experiments on rectangular images which included rectangles of various sizes, shapes and positions. Computer simulations show that the mechanism can perfectly perform planar orientation detection regardless of their sizes, shapes and positions in all experiments. Furthermore, we compared the performance of both AVS and a traditional convolution neural network (CNN) on planar orientation detection and found that AVS completely outperformed CNN in planar orientation detection in terms of identification accuracy, noise resistance, computation and learning cost, hardware implementation and reasonability.

IC neuron: An efficient unit to construct neural networks

As a popular machine learning method, neural networks can be used to solve many complex tasks. Their strong generalization ability comes from the representation ability of the basic neuron models. The most popular neuron model is the McCulloch-Pitts (MP) neuron, which uses a simple transformation to process the input signal. A common trend is to use the MP neuron to design various neural networks. However, the optimization of the neuron structure is rarely considered. Inspired by the elastic collision model in physics, we propose a new neuron model that can represent more complex distributions. We term it the Inter-layer Collision (IC) neuron which divides the input space into multiple subspaces to represent different linear transformations. Through this operation, the IC neuron enhances the non-linear representation ability and emphasizes useful input features for a given task. We build the IC networks by integrating the IC neurons into the fully-connected, the convolutional, and the recurrent structures. The IC networks outperform the traditional neural networks in a wide range of tasks. Besides, we combine the IC neuron with deep learning models and show the superiority of the IC neuron. Our research proves that the IC neuron can be an effective basic unit to build network structures and make the network performance better.

Optically Modulated HfS2-Based Synapses for Artificial Vision Systems

The simulation of human brain neurons by synaptic devices could be an effective strategy to break through the notorious "von Neumann Bottleneck" and "Memory Wall". Herein, opto-electronic synapses based on layered hafnium disulfide (HfS₂) transistors have been investigated. The basic functions of biological synapses are realized and optimized by modifying pulsed light conditions. Furthermore, 2 × 2 pixel imaging chips have also been developed. Two-pixel visual information is illuminated on diagonal pixels of the imaging array by applying light pulses (λ = 405 nm) with different pulse frequencies, mimicking short-term memory and long-term memory characteristics of the human vision system. In addition, an optically/electrically driven neuromorphic computation is demonstrated by machine learning to classify hand-written numbers with an accuracy of about 88.5%. This work will be an important step toward an artificial neural network comprising neuromorphic vision sensing and training functions.

A Neurorobotic Embodiment for Exploring the Dynamical Interactions of a Spiking Cerebellar Model and a Robot Arm During Vision-Based Manipulation Tasks

While the original goal for developing robots is replacing humans in dangerous and tedious tasks, the final target shall be completely mimicking the human cognitive and motor behavior. Hence, building detailed computational models for the human brain is one of the reasonable ways to attain this. The cerebellum is one of the key players in our neural system to guarantee dexterous manipulation and coordinated movements as concluded from lesions in that region. Studies suggest that it acts as a forward model providing anticipatory corrections for the sensory signals based on observed discrepancies from the reference values. While most studies consider providing the teaching signal as error in joint-space, few studies consider the error in task-space and even fewer consider the spiking nature of the cerebellum on the cellular-level. In this study, a detailed cellular-level forward cerebellar model is developed, including modeling of Golgi and Basket cells which are usually neglected in previous studies. To preserve the biological features of the cerebellum in the developed model, a hyperparameter optimization method tunes the network accordingly. The efficiency and biological plausibility of the proposed cerebellar-based controller is then demonstrated under different robotic manipulation tasks reproducing motor behavior observed in human reaching experiments.

Design and Implementation of a Spiking Neural Network with Integrate-and-Fire Neuron Model for Pattern Recognition

In contrast to the previous artificial neural networks (ANNs), spiking neural networks (SNNs) work based on temporal coding approaches. In the proposed SNN, the number of neurons, neuron models, encoding method, and learning algorithm design are described in a correct and pellucid fashion. It is also discussed that optimizing the SNN parameters based on physiology, and maximizing the information they pass leads to a more robust network. In this paper, inspired by the "center-surround" structure of the receptive fields in the retina, and the amount of overlap that they have, a robust SNN is implemented. It is based on the Integrate-and-Fire (IF) neuron model and uses the time-to-first-spike coding to train the network by a newly proposed method. The Iris and MNIST datasets were employed to evaluate the performance of the proposed network whose accuracy, with 60 input neurons, was 96.33% on the Iris dataset. The network was trained in only 45 iterations indicating its reasonable convergence rate. For the MNIST dataset, when the gray level of each pixel was considered as input to the network, 600 input neurons were required, and the accuracy of the network was 90.5%. Next, 14 structural features were used as input. Therefore, the number of input neurons decreased to 210, and accuracy increased up to 95%, meaning that an SNN with fewer input neurons and good skill was implemented. Also, the ABIDE1 dataset is applied to the proposed SNN. Of the 184 data, 79 are used for healthy people and 105 for people with autism. One of the characteristics that can differentiate between these two classes is the entropy of the existing data. Therefore, Shannon entropy is used for feature extraction. Applying these values to the proposed SNN, an accuracy of 84.42% was achieved by only 120 iterations, which is a good result compared to the recent results.

On the Modeling of the Three Types of Non-spiking Neurons of the Caenorhabditis elegans

The nematode Caenorhabditis elegans (C. elegans) is a well-known model organism in neuroscience. The relative simplicity of its nervous system, made up of few hundred neurons, shares some essential features with more sophisticated nervous systems, including the human one. If we are able to fully characterize the nervous system of this organism, we will be one step closer to understanding the mechanisms underlying the behavior of living things. Following a recently conducted electrophysiological survey on different C. elegans neurons, this paper aims at modeling the three non-spiking RIM, AIY and AFD neurons (arbitrarily named with three upper case letters by convention). To date, they represent the three possible forms of non-spiking neuronal responses of the C. elegans. To achieve this objective, we propose a conductance-based neuron model adapted to the electrophysiological features of each neuron. These features are based on current biological research and a series of in-silico experiments which use differential evolution to fit the model to experimental data. From the obtained results, we formulate a series of biological hypotheses regarding currents involved in the neuron dynamics. These models reproduce experimental data with a high degree of accuracy while being biologically consistent with state-of-the-art research.

Spiking Neural P Systems with Astrocytes Producing Calcium

The astrocytes are cells which play an essential role in the functioning and interaction of neurons by feeding the respective neurons with calcium ions. Drawing inspiration from this two-way relationship in which the astrocytes influence and are influenced by the neurons by means of calcium ions, in this paper, we define and study spiking neural P systems with astrocytes producing calcium. Distinct from the usual firing rules in spiking neural P systems, the firing condition not only depends on the spikes collected in a neuron but also on the calcium units received from astrocytes. From the perspective of topological structure, the new variant is shown as a directed graph in which synapses link either astrocytes or neurons, as well as astrocytes to neurons and conversely. The computational power of spiking neural P systems with astrocytes producing calcium is investigated; it is proved that these systems using a limited number of rules are Turing universal as both number generating and number accepting devices. It is also presented how to obtain normal forms by removing forgetting rules and delays while preserving the computational power.

A Complete Arithmetic Calculator Constructed from Spiking Neural P Systems and its Application to Information Fusion

Several variants of spiking neural P systems (SNPS) have been presented in the literature to perform arithmetic operations. However, each of these variants was designed only for one specific arithmetic operation. In this paper, a complete arithmetic calculator implemented by SNPS is proposed. An application of the proposed calculator to information fusion is also proposed. The information fusion is implemented by integrating the following three elements: (1) an addition and subtraction SNPS already reported in the literature; (2) a modified multiplication and division SNPS; (3) a novel storage SNPS, i.e. a method based on SNPS is introduced to calculate basic probability assignment of an event. This is the first attempt to apply arithmetic operation SNPS to fuse multiple information. The effectiveness of the presented general arithmetic SNPS calculator is verified by means of several examples.

Medical Image Fusion Method Based on Coupled Neural P Systems in Nonsubsampled Shearlet Transform Domain

Coupled neural P (CNP) systems are a recently developed Turing-universal, distributed and parallel computing model, combining the spiking and coupled mechanisms of neurons. This paper focuses on how to apply CNP systems to handle the fusion of multi-modality medical images and proposes a novel image fusion method. Based on two CNP systems with local topology, an image fusion framework in nonsubsampled shearlet transform (NSST) domain is designed, where the two CNP systems are used to control the fusion of low-frequency NSST coefficients. The proposed fusion method is evaluated on 20 pairs of multi-modality medical images and compared with seven previous fusion methods and two deep-learning-based fusion methods. Quantitative and qualitative experimental results demonstrate the advantage of the proposed fusion method in terms of visual quality and fusion performance.

Spiking Neural P Systems with Delay on Synapses

Based on the feature and communication of neurons in animal neural systems, spiking neural P systems (SN P systems) were proposed as a kind of powerful computing model. Considering the length of axons and the information transmission speed on synapses, SN P systems with delay on synapses (SNP-DS systems) are proposed in this work. Unlike the traditional SN P systems, where all the postsynaptic neurons receive spikes at the same instant from their presynaptic neuron, the postsynaptic neurons in SNP-DS systems would receive spikes at different instants, depending on the delay time on the synapses connecting them. It is proved that the SNP-DS systems are universal as number generators. Two small universal SNP-DS systems, with standard or extended rules, are constructed to compute functions, using 56 and 36 neurons, respectively. Moreover, a simulator has been provided, in order to check the correctness of these two SNP-DS systems, thus providing an experimental validation of the universality of the systems designed.

How the Cerebellum and Prefrontal Cortex Cooperate During Trace Eyeblinking Conditioning

Several data have demonstrated that during the widely used experimental paradigm for studying associative learning, trace eye blinking conditioning (TEBC), there is a strong interaction between cerebellum and medial prefrontal cortex (mPFC). Despite this evidence, the neural mechanisms underlying this interaction are still not clear. Here, we propose a neurophysiologically plausible computational model to address this issue. The model is constrained on the basis of two critical anatomo-physiological features: (i) the cerebello-cortical organization through two circuits, respectively, targeting M1 and mPFC; (ii) the different timing in the plasticity mechanisms of these parallel circuits produced by the granule cells time sensitivity according to which different subpopulations are active at different moments during conditioned stimuli. The computer simulations run with the model suggest that these features are critical to understand how the cooperation between cerebellum and mPFC supports motor areas during TEBC. In particular, a greater trace interval produces greater plasticity changes at the slow path synapses involving mPFC with respect to plasticity changes at the fast path involving M1. As a consequence, the greater is the trace interval, the stronger is the mPFC involvement. The model has been validated by reproducing data collected through recent real mice experiments.

Nonlinear Spiking Neural P Systems

This paper proposes a new variant of spiking neural P systems (in short, SNP systems), nonlinear spiking neural P systems (in short, NSNP systems). In NSNP systems, the state of each neuron is denoted by a real number, and a real configuration vector is used to characterize the state of the whole system. A new type of spiking rules, nonlinear spiking rules, is introduced to handle the neuron’s firing, where the consumed and generated amounts of spikes are often expressed by the nonlinear functions of the state of the neuron. NSNP systems are a class of distributed parallel and nondeterministic computing systems. The computational power of NSNP systems is discussed. Specifically, it is proved that NSNP systems as number-generating/accepting devices are Turing-universal. Moreover, we establish two small universal NSNP systems for function computing and number generator, containing 117 neurons and 164 neurons, respectively.