Dendritic Neuron Model With Effective Learning Algorithms for Classification, Approximation, and Prediction

Population interaction network in representative gravitational search algorithms: Logistic distribution leads to worse performance

In this paper, a novel study on the way inter-individual information interacts in meta-heuristic algorithms (MHAs) is carried out using a scheme known as population interaction networks (PIN). Specifically, three representative MHAs, including the differential evolutionary algorithm (DE), the particle swarm optimization algorithm (PSO), the gravitational search algorithm (GSA), and four classical variations of the gravitational search algorithm, are analyzed in terms of inter-individual information interactions and the differences in the performance of each of the algorithms on IEEE Congress on Evolutionary Computation 2017 benchmark functions. The cumulative distribution function (CDF) of the node degree obtained by the algorithm on the benchmark function is fitted to the seven distribution models by using PIN. The results show that among the seven compared algorithms, the more powerful DE is more skewed towards the Poisson distribution, and the weaker PSO, GSA, and GSA variants are more skewed towards the Logistic distribution. The more deviation from Logistic distribution GSA variants conform, the stronger their performance. From the point of view of the CDF, deviating from the Logistic distribution facilitates the improvement of the GSA. Our findings suggest that the population interaction network is a powerful tool for characterizing and comparing the performance of different MHAs in a more comprehensive and meaningful way.

ADGAN: Attribute-Driven Generative Adversarial Network for Synthesis and Multiclass Classification of Pulmonary Nodules

Lung cancer is the leading cause of cancer-related deaths worldwide. According to the American Cancer Society, early diagnosis of pulmonary nodules in computed tomography (CT) scans can improve the five-year survival rate up to 70% with proper treatment planning. In this article, we propose an attribute-driven Generative Adversarial Network (ADGAN) for synthesis and multiclass classification of Pulmonary Nodules. A self-attention U-Net (SaUN) architecture is proposed to improve the generation mechanism of the network. The generator is designed with two modules, namely, self-attention attribute module (SaAM) and a self-attention spatial module (SaSM). SaAM generates a nodule image based on given attributes whereas SaSM specifies the nodule region of the input image to be altered. A reconstruction loss along with an attention localization loss (AL) is used to produce an attention map prioritizing the nodule regions. To avoid resemblance between a generated image and a real image, we further introduce an adversarial loss containing a regularization term based on KL divergence. The discriminator part of the proposed model is designed to achieve the multiclass nodule classification task. Our proposed approach is validated over two challenging publicly available datasets, namely LIDC-IDRI and LUNGX. Exhaustive experimentation on these two datasets clearly indicate that we have achieved promising classification accuracy as compared to other state-of-the-art methods.

Knowledge Transfer-Based Sparse Deep Belief Network

Deep learning has made remarkable achievements in various applications in recent years. With the increasing computing power and the "black box" problem of neural networks, however, the development of deep neural networks (DNNs) has entered a bottleneck period. This article proposes a novel deep belief network (DBN) based on knowledge transfer and optimization of the network structure. First, a neural-symbolic model is proposed to extract rules to describe the dynamic operation mechanism of the deep network. Second, knowledge fusion is proposed based on the merge and deletion of the extracted rules from the DBN model. Finally, a new DNN, knowledge transfer-based sparse DBN (KT-SDBN) is constructed to generate a sparse network without excessive information loss. In comparison with DBN, KT-SDBN has a more sparse network structure and better learning performance on the existing knowledge and data. The experimental results in the benchmark data indicate that KT-SDBN not only has effective feature learning performance with 30% of the original network parameters but also shows a large compression rate that is far larger than other structure optimization algorithms.

Competitive Decomposition-Based Multiobjective Architecture Search for the Dendritic Neural Model

The dendritic neural model (DNM) is computationally faster than other machine-learning techniques, because its architecture can be implemented by using logic circuits and its calculations can be performed entirely in binary form. To further improve the computational speed, a straightforward approach is to generate a more concise architecture for the DNM. Actually, the architecture search is a large-scale multiobjective optimization problem (LSMOP), where a large number of parameters need to be set with the aim of optimizing accuracy and structural complexity simultaneously. However, the issues of irregular Pareto front, objective discontinuity, and population degeneration strongly limit the performances of conventional multiobjective evolutionary algorithms (MOEAs) on the specific problem. Therefore, a novel competitive decomposition-based MOEA is proposed in this study, which decomposes the original problem into several constrained subproblems, with neighboring subproblems sharing overlapping regions in the objective space. The solutions in the overlapping regions participate in environmental selection for the neighboring subproblems and then propagate the selection pressure throughout the entire population. Experimental results demonstrate that the proposed algorithm can possess a more powerful optimization ability than the state-of-the-art MOEAs. Furthermore, both the DNM itself and its hardware implementation can achieve very competitive classification performances when trained by the proposed algorithm, compared with numerous widely used machine-learning approaches.

Diversified Regularization Enhanced Training for Effective Manipulator Calibration

Recently, robot arms have become an irreplaceable production tool, which play an important role in the industrial production. It is necessary to ensure the absolute positioning accuracy of the robot to realize automatic production. Due to the influence of machining tolerance, assembly tolerance, the robot positioning accuracy is poor. Therefore, in order to enable the precise operation of the robot, it is necessary to calibrate the robotic kinematic parameters. The least square method and Levenberg-Marquardt (LM) algorithm are commonly used to identify the positioning error of robot. However, it generally has the overfitting caused by improper regularization schemes. To solve this problem, this article discusses six regularization schemes based on its error models, i.e., L1 , L2 , dropout, elastic, log, and swish. Moreover, this article proposes a scheme with six regularization to obtain a reliable ensemble, which can effectively avoid overfitting. The positioning accuracy of the robot is improved significantly after calibration by enough experiments, which verifies the feasibility of the proposed method.

An Improved Non-Negative Latent Factor Model for Missing Data Estimation via Extragradient-Based Alternating Direction Method

In this article, an improved double factorization-based symmetric and non-negative latent factor (Im-DF-SNLF) model is proposed to make the estimation for missing data in symmetric, high-dimensional, and sparse (SHiDS) matrices. The main idea of the Im-DF-SNLF model is fourfold: 1) considering the data variety in the practical engineering, non-negative latent factors (NLFs) in different cases are considered to better reflect the latent relationships between entries; 2) the l2 -norm regularization and the Lagrangian multiplier technique are simultaneously adopted to handle the overfitting and satisfy the non-negative constraint for latent factors (LFs); 3) the extragradient-based alternating direction (EGAD) method is utilized to accelerate the model training and rigidly guarantee the non-negativity of LFS; and 4) a rigorous proof is provided to show that, under the given assumption that the objective function is smooth and has a Lipschitz continuous gradient, the designed algorithm can find an ϵ -optimal solution within O(1/ϵ) , and the upper bound of the learning rate is given by 1/2. Finally, experimental results on public datasets are given to demonstrate the effectiveness of our proposed Im-DF-SNLF model with EGAD.

Maximum Lyapunov exponent-based multiple chaotic slime mold algorithm for real-world optimization

Slime mold algorithm (SMA) is a nature-inspired algorithm that simulates the biological optimization mechanisms and has achieved great results in various complex stochastic optimization problems. Owing to the simulated biological search principle of slime mold, SMA has a unique advantage in global optimization problem. However, it still suffers from issues of missing the optimal solution or collapsing to local optimum when facing complicated problems. To conquer these drawbacks, we consider adding a novel multi-chaotic local operator to the bio-shock feedback mechanism of SMA to compensate for the lack of exploration of the local solution space with the help of the perturbation nature of the chaotic operator. Based on this, we propose an improved algorithm, namely MCSMA, by investigating how to improve the probabilistic selection of chaotic operators based on the maximum Lyapunov exponent (MLE), an inherent property of chaotic maps. We implement the comparison between MCSMA with other state-of-the-art methods on IEEE Congress on Evolution Computation (CEC) i.e., CEC2017 benchmark test suits and CEC2011 practical problems to demonstrate its potency and perform dendritic neuron model training to test the robustness of MCSMA on classification problems. Finally, the parameters' sensitivities of MCSMA, the utilization of the solution space, and the effectiveness of the MLE are adequately discussed.

A Segmented Variable-Parameter ZNN for Dynamic Quadratic Minimization With Improved Convergence and Robustness

As a category of the recurrent neural network (RNN), zeroing neural network (ZNN) can effectively handle time-variant optimization issues. Compared with the fixed-parameter ZNN that needs to be adjusted frequently to achieve good performance, the conventional variable-parameter ZNN (VPZNN) does not require frequent adjustment, but its variable parameter will tend to infinity as time grows. Besides, the existing noise-tolerant ZNN model is not good enough to deal with time-varying noise. Therefore, a new-type segmented VPZNN (SVPZNN) for handling the dynamic quadratic minimization issue (DQMI) is presented in this work. Unlike the previous ZNNs, the SVPZNN includes an integral term and a nonlinear activation function, in addition to two specially constructed time-varying piecewise parameters. This structure keeps the time-varying parameters stable and makes the model have strong noise tolerance capability. Besides, theoretical analysis on SVPZNN is proposed to determine the upper bound of convergence time in the absence or presence of noise interference. Numerical simulations verify that SVPZNN has shorter convergence time and better robustness than existing ZNN models when handling DQMI.

Evolutionary Weighted Broad Learning and Its Application to Fault Diagnosis in Self-Organizing Cellular Networks

As a novel neural network-based learning framework, a broad learning system (BLS) has attracted much attention due to its excellent performance on regression and balanced classification problems. However, it is found to be unsuitable for imbalanced data classification problems because it treats each class in an imbalanced dataset equally. To address this issue, this work proposes a weighted BLS (WBLS) in which the weight assigned to each class depends on the number of samples in it. In order to further boost its classification performance, an improved differential evolution algorithm is proposed to automatically optimize its parameters, including the ones in BLS and newly generated weights. We first optimize the parameters with a training dataset, and then apply them to WBLS on a test dataset. The experiments on 20 imbalanced classification problems have shown that our proposed method can achieve higher classification accuracy than the other methods in terms of several widely used performance metrics. Finally, it is applied to fault diagnosis in self-organizing cellular networks to further show its applicability to industrial application problems.

Prediction of PM2.5 time series by seasonal trend decomposition-based dendritic neuron model

The rapid industrial development in the human society has brought about the air pollution, which seriously affects human health. PM2.5 concentration is one of the main factors causing the air pollution. To accurately predict PM2.5 microns, we propose a dendritic neuron model (DNM) trained by an improved state-of-matter heuristic algorithm (DSMS) based on STL-LOESS, namely DS-DNM. Firstly, DS-DNM adopts STL-LOESS for the data preprocessing to obtain three characteristic quantities from original data: seasonal, trend, and residual components. Then, DNM trained by DSMS predicts the residual values. Finally, three sets of feature quantities are summed to obtain the predicted values. In the performance test experiments, five real-world PM2.5 concentration data are used to test DS-DNM. On the other hand, four training algorithms and seven prediction models were selected for comparison to verify the rationality of the training algorithms and the accuracy of the prediction models, respectively. The experimental results show that DS-DNM has the more competitive performance in PM2.5 concentration prediction problem.

Fully Complex-Valued Dendritic Neuron Model

A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.

Solving Dynamic Traveling Salesman Problems With Deep Reinforcement Learning

A traveling salesman problem (TSP) is a well-known NP-complete problem. Traditional TSP presumes that the locations of customers and the traveling time among customers are fixed and constant. In real-life cases, however, the traffic conditions and customer requests may change over time. To find the most economic route, the decisions can be made constantly upon the time-point when the salesman completes his service of each customer. This brings in a dynamic version of the traveling salesman problem (DTSP), which takes into account the information of real-time traffic and customer requests. DTSP can be extended to a dynamic pickup and delivery problem (DPDP). In this article, we ameliorate the attention model to make it possible to perceive environmental changes. A deep reinforcement learning algorithm is proposed to solve DTSP and DPDP instances with a size of up to 40 customers in 100 locations. Experiments show that our method can capture the dynamic changes and produce a highly satisfactory solution within a very short time. Compared with other baseline approaches, more than 5% improvements can be observed in many cases.

Many-Objective Job-Shop Scheduling: A Multiple Populations for Multiple Objectives-Based Genetic Algorithm Approach

The job-shop scheduling problem (JSSP) is a challenging scheduling and optimization problem in the industry and engineering, which relates to the work efficiency and operational costs of factories. The completion time of all jobs is the most commonly considered optimization objective in the existing work. However, factories focus on both time and cost objectives, including completion time, total tardiness, advance time, production cost, and machine loss. Therefore, this article first time proposes a many-objective JSSP that considers all these five objectives to make the model more practical to reflect the various demands of factories. To optimize these five objectives simultaneously, a novel multiple populations for multiple objectives (MPMO) framework-based genetic algorithm (GA) approach, called MPMOGA, is proposed. First, MPMOGA employs five populations to optimize the five objectives, respectively. Second, to avoid each population only focusing on its corresponding single objective, an archive sharing technique (AST) is proposed to store the elite solutions collected from the five populations so that the populations can obtain optimization information about the other objectives from the archive. This way, MPMOGA can approximate different parts of the entire Pareto front (PF). Third, an archive update strategy (AUS) is proposed to further improve the quality of the solutions in the archive. The test instances in the widely used test sets are adopted to evaluate the performance of MPMOGA. The experimental results show that MPMOGA outperforms the compared state-of-the-art algorithms on most of the test instances.

Single dendritic neural classification with an effective spherical search-based whale learning algorithm

McCulloch-Pitts neuron-based neural networks have been the mainstream deep learning methods, achieving breakthrough in various real-world applications. However, McCulloch-Pitts neuron is also under longtime criticism of being overly simplistic. To alleviate this issue, the dendritic neuron model (DNM), which employs non-linear information processing capabilities of dendrites, has been widely used for prediction and classification tasks. In this study, we innovatively propose a hybrid approach to co-evolve DNM in contrast to back propagation (BP) techniques, which are sensitive to initial circumstances and readily fall into local minima. The whale optimization algorithm is improved by spherical search learning to perform co-evolution through dynamic hybridizing. Eleven classification datasets were selected from the well-known UCI Machine Learning Repository. Its efficiency in our model was verified by statistical analysis of convergence speed and Wilcoxon sign-rank tests, with receiver operating characteristic curves and the calculation of area under the curve. In terms of classification accuracy, the proposed co-evolution method beats 10 existing cutting-edge non-BP methods and BP, suggesting that well-learned DNMs are computationally significantly more potent than conventional McCulloch-Pitts types and can be employed as the building blocks for the next-generation deep learning methods.

Quantification of interfacial interaction related with adhesive membrane fouling by genetic algorithm back propagation (GABP) neural network

Since adhesive membrane fouling is critically determined by the interfacial interaction between a foulant and a rough membrane surface, efficient quantification of the interfacial interaction is critically important for adhesive membrane fouling mitigation. As a current available method, the advanced extended Derjaguin-Landau-Verwey-Overbeek (XDLVO) theory involves complicated rigorous thermodynamic equations and massive amounts of computation, restricting its application. To solve this problem, artificial intelligence (AI) visualization technology was used to analyze the existing literature, and the genetic algorithm back propagation (GABP) artificial neural network (ANN) was employed to simplify thermodynamic calculation. The results showed that GABP ANN with 5 neurons could obtain reliable prediction performance in seconds, versus several hours or even days time-consuming by the advanced XDLVO theory. Moreover, the regression coefficient (R) of GABP reached 0.9999, and the error between the prediction results and the simulation results was less than 0.01%, indicating feasibility of the GABP ANN technique for quantification of interfacial interaction related with adhesive membrane fouling. This work provided a novel strategy to efficiently optimize the thermodynamic prediction of adhesive membrane fouling, beneficial for better understanding and control of adhesive membrane fouling.

Dendritic Deep Residual Learning for COVID‐19 Prediction

Deep residual network (ResNet), one of the mainstream deep learning models, has achieved groundbreaking results in various fields. However, all neurons used in ResNet are based on the McCulloch‐Pitts model which has long been criticized for its oversimplified structure. Accordingly, this paper for the first time proposes a novel dendritic residual network by considering the powerful information processing capacity of dendrites in neurons. Experimental results based on the challenging COVID‐19 prediction problem show the superiority of the proposed method in comparison with other state‐of‐the‐art ones. © 2022 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

An Extension Network of Dendritic Neurons

Deep learning (DL) has achieved breakthrough successes in various tasks, owing to its layer-by-layer information processing and sufficient model complexity. However, DL suffers from the issues of both redundant model complexity and low interpretability, which are mainly because of its oversimplified basic McCulloch-Pitts neuron unit. A widely recognized biologically plausible dendritic neuron model (DNM) has demonstrated its effectiveness in alleviating the aforementioned issues, but it can only solve binary classification tasks, which significantly limits its applicability. In this study, a novel extended network based on the dendritic structure is innovatively proposed, thereby enabling it to solve multiple-class classification problems. Also, for the first time, an efficient error-back-propagation learning algorithm is derived. In the extensive experimental results, the effectiveness and superiority of the proposed method in comparison with other nine state-of-the-art classifiers on ten datasets are demonstrated, including a real-world quality of web service application. The experimental results suggest that the proposed learning algorithm is competent and reliable in terms of classification performance and stability and has a notable advantage in small-scale disequilibrium data. Additionally, aspects of network structure constrained by scale are examined.

An approach to robust condition monitoring in industrial processes using pythagorean membership grades

In this paper, a robust approach to improve the performance of a condition monitoring process in industrial plants by using Pythagorean membership grades is presented. The FCM algorithm is modified by using Pythagorean fuzzy sets, to obtain a new variant of it called Pythagorean Fuzzy C-Means (PyFCM). In addition, a kernel version of PyFCM (KPyFCM) is obtained in order to achieve greater separability among classes, and reduce classification errors. The approach proposed is validated using experimental datasets and the Tennessee Eastman (TE) process benchmark. The results are compared with the results obtained with other algorithms that use standard and non-standard membership grades. The highest performance obtained by the approach proposed indicate its feasibility.

Dendrite Net: A White-Box Module for Classification, Regression, and System Identification

The simulation of biological dendrite computations is vital for the development of artificial intelligence (AI). This article presents a basic machine-learning (ML) algorithm, called Dendrite Net or DD, just like the support vector machine (SVM) or multilayer perceptron (MLP). DD's main concept is that the algorithm can recognize this class after learning, if the output's logical expression contains the corresponding class's logical relationship among inputs (and \ or \ not). Experiments and main results: DD, a white-box ML algorithm, showed excellent system identification performance for the black-box system. Second, it was verified by nine real-world applications that DD brought better generalization capability relative to the MLP architecture that imitated neurons' cell body (Cell body Net) for regression. Third, by MNIST and FASHION-MNIST datasets, it was verified that DD showed higher testing accuracy under greater training loss than the cell body net for classification. The number of modules can effectively adjust DD's logical expression capacity, which avoids overfitting and makes it easy to get a model with outstanding generalization capability. Finally, repeated experiments in MATLAB and PyTorch (Python) demonstrated that DD was faster than Cell body Net both in epoch and forwardpropagation. The main contribution of this article is the basic ML algorithm (DD) with a white-box attribute, controllable precision for better generalization capability, and lower computational complexity. Not only can DD be used for generalized engineering, but DD has vast development potential as a module for deep learning. DD code is available at https://github.com/liugang1234567/Gang-neuron.

Virtual Intelligence: A Systematic Review of the Development of Neural Networks in Brain Simulation Units

The functioning of the brain has been a complex and enigmatic phenomenon. From the first approaches made by Descartes about this organism as the vehicle of the mind to contemporary studies that consider the brain as an organism with emergent activities of primary and higher order, this organism has been the object of continuous exploration. It has been possible to develop a more profound study of brain functions through imaging techniques, the implementation of digital platforms or simulators through different programming languages and the use of multiple processors to emulate the speed at which synaptic processes are executed in the brain. The use of various computational architectures raises innumerable questions about the possible scope of disciplines such as computational neurosciences in the study of the brain and the possibility of deep knowledge into different devices with the support that information technology (IT) brings. One of the main interests of cognitive science is the opportunity to develop human intelligence in a system or mechanism. This paper takes the principal articles of three databases oriented to computational sciences (EbscoHost Web, IEEE Xplore and Compendex Engineering Village) to understand the current objectives of neural networks in studying the brain. The possible use of this kind of technology is to develop artificial intelligence (AI) systems that can replicate more complex human brain tasks (such as those involving consciousness). The results show the principal findings in research and topics in developing studies about neural networks in computational neurosciences. One of the principal developments is the use of neural networks as the basis of much computational architecture using multiple techniques such as computational neuromorphic chips, MRI images and brain–computer interfaces (BCI) to enhance the capacity to simulate brain activities. This article aims to review and analyze those studies carried out on the development of different computational architectures that focus on affecting various brain activities through neural networks. The aim is to determine the orientation and the main lines of research on this topic and work in routes that allow interdisciplinary collaboration.

Prediction of the Nitrogen Content of Rice Leaf Using Multi-Spectral Images Based on Hybrid Radial Basis Function Neural Network and Partial Least-Squares Regression

This paper's novel focus is predicting the leaf nitrogen content of rice during growing and maturing. A multispectral image processing-based prediction model of the Radial Basis Function Neural Network (RBFNN) model was proposed. Moreover, this paper depicted three primary points as the following: First, collect images of rice leaves (RL) from a controlled condition experimental laboratory and new shoot leaves in different stages in the visible light spectrum, and apply digital image processing technology to extract the color characteristics of RL and the morphological characteristics of the new shoot leaves. Secondly, the RBFNN model, the General Regression Model (GRL), and the General Regression Method (GRM) model were constructed based on the extracted image feature parameters and the nitrogen content of rice leaves. Third, the RBFNN is optimized by and Partial Least-Squares Regression (RBFNN-PLSR) model. Finally, the validation results show that the nitrogen content prediction models at growing and mature stages that the mean absolute error (MAE), the Mean Absolute Percentage Error (MAPE), and the Root Mean Square Error (RMSE) of the RFBNN model during the rice-growing stage and the mature stage are 0.6418 (%), 0.5399 (%), 0.0652 (%), and 0.3540 (%), 0.1566 (%), 0.0214 (%) respectively, the predicted value of the model fits well with the actual value. Finally, the model may be used to give the best foundation for achieving exact fertilization control by continuously monitoring the nitrogen nutrition status of rice. In addition, at the growing stage, the RBFNN model shows better results compared to both GRL and GRM, in which MAE is reduced by 0.2233% and 0.2785%, respectively.

Resource-Aware Distributed Differential Evolution for Training Expensive Neural-Network-Based Controller in Power Electronic Circuit

The neural-network (NN)-based control method is a new emerging promising technique for controller design in a power electronic circuit (PEC). However, the optimization of NN-based controllers (NNCs) has significant challenges in two aspects. The first challenge is that the search space of the NNC optimization problem is such complex that the global optimization ability of the existing algorithms still needs to be improved. The second challenge is that the training process of the NNC parameters is very computationally expensive and requires a long execution time. Thus, in this article, we develop a powerful evolutionary computation-based algorithm to find a high-quality solution and reduce computational time. First, the differential evolution (DE) algorithm is adopted because it is a powerful global optimizer in solving a complex optimization problem. This can help to overcome the premature convergence in local optima to train the NNC parameters well. Second, to reduce the computational time, the DE is extended to distribute DE (DDE) by dispatching all the individuals to different distributed computing resources for parallel computing. Moreover, a resource-aware strategy (RAS) is designed to further efficiently utilize the resources by adaptively dispatching individuals to resources according to the real-time performance of the resources, which can simultaneously concern the computing ability and load state of each resource. Experimental results show that, compared with some other typical evolutionary algorithms, the proposed algorithm can get significantly better solutions within a shorter computational time.

Scalable Gamma-Driven Multilayer Network for Brain Workload Detection Through Functional Near-Infrared Spectroscopy

This work proposes a scalable gamma non-negative matrix network (SGNMN), which uses a Poisson randomized Gamma factor analysis to obtain the neurons of the first layer of a network. These neurons obey Gamma distribution whose shape parameter infers the neurons of the next layer of the network and their related weights. Upsampling the connection weights follows a Dirichlet distribution. Downsampling hidden units obey Gamma distribution. This work performs up-down sampling on each layer to learn the parameters of SGNMN. Experimental results indicate that the width and depth of SGNMN are closely related, and a reasonable network structure for accurately detecting brain fatigue through functional near-infrared spectroscopy can be obtained by considering network width, depth, and parameters.

Hybrid model of a cement rotary kiln using an improved attention-based recurrent neural network

A rotary kiln is core equipment in cement calcination. Significant time delay, time-varying, and nonlinear characteristics cause challenges in the advance process control and operational optimization of the rotary kiln. However, the traditional mechanism model with many assumptions cannot accurately represent the dynamic kiln process because kinetic parameters are difficult to obtain. This paper proposes a novel hybrid strategy to develop a dynamic model of a rotary kiln by combining a process mechanism and a recurrent neural network to address this issue. A time delay mechanism is used to estimate the kiln's residence time to compensate for the time delay. A long short-term memory model that combines an attention mechanism and an ordinary differential equation solver is proposed to capture the time-varying and nonlinear behaviors of the kiln process. Case studies from two real-world cement plants with different processing loads are used to verify the effectiveness of the proposed hybrid modeling strategy. The results show that the proposed method has better accuracy and robustness than the traditional methods. The sensitivity analysis of the proposed model also makes it practical for t control system design and real-time optimization.

Direction Control and Adaptive Path Following of 3-D Snake-Like Robot Motion

This work investigates direction control and path following of a 3-D snake-like robot. In order to control such robots accurately, this work researches the relationships between its phase offsets of pitch joints and directions. A new direction control method is proposed for the robot based on these relationships. An adaptive path-following algorithm based on the line-of-sight guidance law is proposed and combined with the direction control method to steer the robot to move forward and along desired paths. Simulation and experimental results are presented to demonstrate the performances of the proposed 3-D model and control methods. They well outperform the classical and commonly used path-following method.

Attribute analysis and modeling of color harmony based on multi-color feature extraction in real-life scenes

Color harmony is the focus of many researchers in the field of art and design, and its research results have been widely used in artistic creation and design activities. With the development of signal processing and artificial intelligence technology, new ideas and methods are provided for color harmony theory and color harmony calculation. In this article, psychological experimental methods and information technology are combined to design and quantify the 16-dimensional physical features of multiple colors, including multi-color statistical features and multi-color contrast features. Eighty-four subjects are invited to give a 5-level score on the degree of color harmony for 164 multi-color materials selected from the screenshots of film and television scenes. Based on the multi-color physical features and the subjective evaluation experiment, the correlation analysis is firstly carried out, which shows that the overall lightness, difference of the color tones, number of multiple colors, lightness contrast, color tone contrast, and cool/warm contrast are significantly correlated with color harmony. On the other hand, the regression prediction model and classification prediction model of color harmony are constructed based on machine learning algorithms. In terms of regression prediction model, the prediction accuracy of linear models is higher than that of nonlinear models, with 63.9% as the highest, indicating that the multi-color physical features can explain color harmony well. In terms of classification prediction model, the Random Forest (RF) has the best prediction performance, with an accuracy of 80.2%.

Decision-Tree-Initialized Dendritic Neuron Model for Fast and Accurate Data Classification

This work proposes a decision tree (DT)-based method for initializing a dendritic neuron model (DNM). Neural networks become larger and larger, thus consuming more and more computing resources. This calls for a strong need to prune neurons that do not contribute much to their network's output. Pruning those with low contribution may lead to a loss of accuracy of DNM. Our proposed method is novel because 1) it can reduce the number of dendrites in DNM while improving training efficiency without affecting accuracy and 2) it can select proper initialization weight and threshold of neurons. The Adam algorithm is used to train DNM after its initialization with our proposed DT-based method. To verify its effectiveness, we apply it to seven benchmark datasets. The results show that decision-tree-initialized DNM is significantly better than the original DNM, k-nearest neighbor, support vector machine, back-propagation neural network, and DT classification methods. It exhibits the lowest model complexity and highest training speed without losing any accuracy. The interactions among attributes can also be observed in its dendritic neurons.

DLIN: Deep Ladder Imputation Network

Many efforts have been dedicated to addressing data loss in various domains. While task-specific solutions may eliminate the respective issue in certain applications, finding a generic method for missing data estimation is rather complex. In this regard, this article proposes a novel missing data imputation algorithm, which has supreme generalization ability for a vast variety of applications. Making use of both complete and incomplete parts of data, the proposed algorithm reduces the effect of missing ratio, which makes it suitable for situations with very high missing ratios. In addition, this feature enables model construction on incomplete training sets, which is rarely addressed in the literature. Moreover, the nonparametric nature of this new algorithm brings about supreme flexibility against all variations of missing values and data distribution. We incorporate the advantages of denoising autoencoders and ladder architecture into a novel formulation based on deep neural networks. To evaluate the proposed algorithm, a comparative study is performed using a number of reputable imputation techniques. In this process, real-world benchmark datasets from different domains are selected. On top of that, a real cyber-physical system is also evaluated to study the generalization ability of the proposed algorithm for distinct applications. To do so, we conduct studies based on three missing data mechanisms, namely: 1) missing completely at random; 2) missing at random; and 3) missing not at random. The attained results indicate the superiority of the proposed method in these experiments.

Moving-Distance-Minimized PSO for Mobile Robot Swarm

Particle swarm optimizer (PSO) and mobile robot swarm are two typical swarm techniques. Many applications emerge separately along both of them while the similarity between them is rarely considered. When a solution space is a certain region in reality, a robot swarm can replace a particle swarm to explore the optimal solution by performing PSO. In this way, a mobile robot swarm should be able to efficiently explore an area just like the particle swarm and uninterruptedly work even under the shortage of robots or in the case of unexpected failure of robots. Furthermore, the moving distances of robots are highly constrained because energy and time can be costly. Inspired by such requirements, this article proposes a moving-distance-minimized PSO (MPSO) for a mobile robot swarm to minimize the total moving distance of its robots while performing optimization. The distances between the current robot positions and the particle ones in the next generation are utilized to derive paths for robots such that the total distance that robots move is minimized, hence minimizing the energy and time for a robot swarm to locate the optima. Experiments on 28 CEC2013 benchmark functions show the advantage of the proposed method over the standard PSO. By adopting the given algorithm, the moving distance can be reduced by more than 66% and the makespan can be reduced by nearly 70% while offering the same optimization effects.

An adaptive particle swarm optimization-based hybrid long short-term memory model for stock price time series forecasting

In this paper, we presented a long short-term memory (LSTM) network and adaptive particle swarm optimization (PSO)-based hybrid deep learning model for forecasting the stock price of three major stock indices such as Sensex, S&P 500, and Nifty 50 for short term and long term. Although the LSTM can handle uncertain, sequential, and nonlinear data, the biggest challenge in it is optimizing its weights and bias. The back-propagation through time algorithm has a drawback to overfit the data and being stuck in local minima. Thus, we proposed PSO-based hybrid deep learning model for evolving the initial weights of LSTM and fully connected layer (FCL). Furthermore, we introduced an adaptive approach for improving the inertia coefficient of PSO using the velocity of particles. The proposed method is an aggregation of adaptive PSO and Adam optimizer for training the LSTM. The adaptive PSO attempts to evolve the initial weights in different layers of the LSTM network and FCL. This research also compares the forecasting efficacy of the proposed method to the genetic algorithm (GA)-based hybrid LSTM model, the Elman neural network (ENN), and standard LSTM. Experimental findings demonstrate that the suggested model is successful in achieving the optimum initial weights and bias of the LSTM and FC layers, as well as superior forecasting accuracy.

Solving Complex-Valued Time-Varying Linear Matrix Equations via QR Decomposition With Applications to Robotic Motion Tracking and on Angle-of-Arrival Localization

The problem of solving linear equations is considered as one of the fundamental problems commonly encountered in science and engineering. In this article, the complex-valued time-varying linear matrix equation (CVTV-LME) problem is investigated. Then, by employing a complex-valued, time-varying QR (CVTVQR) decomposition, the zeroing neural network (ZNN) method, equivalent transformations, Kronecker product, and vectorization techniques, we propose and study a CVTVQR decomposition-based linear matrix equation (CVTVQR-LME) model. In addition to the usage of the QR decomposition, the further advantage of the CVTVQR-LME model is reflected in the fact that it can handle a linear system with square or rectangular coefficient matrix in both the matrix and vector cases. Its efficacy in solving the CVTV-LME problems have been tested in a variety of numerical simulations as well as in two applications, one in robotic motion tracking and the other in angle-of-arrival localization.

An α-β-Divergence-Generalized Recommender for Highly Accurate Predictions of Missing User Preferences

To quantify user-item preferences, a recommender system (RS) commonly adopts a high-dimensional and sparse (HiDS) matrix. Such a matrix can be represented by a non-negative latent factor analysis model relying on a single latent factor (LF)-dependent, non-negative, and multiplicative update algorithm. However, existing models' representative abilities are limited due to their specialized learning objective. To address this issue, this study proposes an α- β -divergence-generalized model that enjoys fast convergence. Its ideas are three-fold: 1) generalizing its learning objective with α- β -divergence to achieve highly accurate representation of HiDS data; 2) incorporating a generalized momentum method into parameter learning for fast convergence; and 3) implementing self-adaptation of controllable hyperparameters for excellent practicability. Empirical studies on six HiDS matrices from real RSs demonstrate that compared with state-of-the-art LF models, the proposed one achieves significant accuracy and efficiency gain to estimate huge missing data in an HiDS matrix.

Weak Estimator-Based Stochastic Searching on the Line in Dynamic Dual Environments

Stochastic point location deals with the problem of finding a target point on a real line through a learning mechanism (LM) with the stochastic environment (SE) offering directional information. The SE can be further categorized into an informative or deceptive one, according to whether p is above 0.5 or not, where p is the probability of providing a correct suggestion of a direction to LM. Several attempts have been made for LM to work in both types of environments, but none of them considers a dynamically changing environment where p varies with time. A dynamic dual environment involves fierce changes that frequently cause its environment to switch from an informative one to a deceptive one, or vice versa. This article presents a novel weak estimator-based adaptive step search solution, to enable LM to track the target in a dynamic dual environment, with the help of a weak estimator. The experimental results show that the proposed solution is feasible and efficient.