Prototype-Incorporated Emotional Neural Network

Prototype-Based Interpretation of the Functionality of Neurons in Winner-Take-All Neural Networks

Prototype-based learning (PbL) using a winner-take-all (WTA) network based on minimum Euclidean distance (ED-WTA) is an intuitive approach to multiclass classification. By constructing meaningful class centers, PbL provides higher interpretability and generalization than hyperplane-based learning (HbL) methods based on maximum inner product (IP-WTA) and can efficiently detect and reject samples that do not belong to any classes. In this article, we first prove the equivalence of IP-WTA and ED-WTA from a representational power perspective. Then, we show that naively using this equivalence leads to unintuitive ED-WTA networks in which the centers have high distances to data that they represent. We propose ±ED-WTA that models each neuron with two prototypes: one positive prototype, representing samples modeled by that neuron, and a negative prototype, representing the samples erroneously won by that neuron during training. We propose a novel training algorithm for the ±ED-WTA network, which cleverly switches between updating the positive and negative prototypes and is essential to the emergence of interpretable prototypes. Unexpectedly, we observed that the negative prototype of each neuron is indistinguishably similar to the positive one. The rationale behind this observation is that the training data that are mistaken for a prototype are indeed similar to it. The main finding of this article is this interpretation of the functionality of neurons as computing the difference between the distances to a positive and a negative prototype, which is in agreement with the BCM theory. Our experiments show that the proposed ±ED-WTA method constructs highly interpretable prototypes that can be successfully used for explaining the functionality of deep neural networks (DNNs), and detecting outlier and adversarial examples.

Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis

Long short-term memory (LSTM) neural networks and attention mechanism have been widely used in sentiment representation learning and detection of texts. However, most of the existing deep learning models for text sentiment analysis ignore emotion's modulation effect on sentiment feature extraction, and the attention mechanisms of these deep neural network architectures are based on word- or sentence-level abstractions. Ignoring higher level abstractions may pose a negative effect on learning text sentiment features and further degrade sentiment classification performance. To address this issue, in this article, a novel model named AEC-LSTM is proposed for text sentiment detection, which aims to improve the LSTM network by integrating emotional intelligence (EI) and attention mechanism. Specifically, an emotion-enhanced LSTM, named ELSTM, is first devised by utilizing EI to improve the feature learning ability of LSTM networks, which accomplishes its emotion modulation of learning system via the proposed emotion modulator and emotion estimator. In order to better capture various structure patterns in text sequence, ELSTM is further integrated with other operations, including convolution, pooling, and concatenation. Then, topic-level attention mechanism is proposed to adaptively adjust the weight of text hidden representation. With the introduction of EI and attention mechanism, sentiment representation and classification can be more effectively achieved by utilizing sentiment semantic information hidden in text topic and context. Experiments on real-world data sets show that our approach can improve sentiment classification performance effectively and outperform state-of-the-art deep learning-based methods significantly.

Recognizing Missing Electromyography Signal by Data Split Reorganization Strategy and Weight-Based Multiple Neural Network Voting Method

Surface electromyography (sEMG) signals have been applied widely in prosthetic hand controlling. In the sEMG signal acquisition, wireless devices bring convenience, but also introduce signal missing due to interference or failure during data transmission. The missing signal may only last for tens of milliseconds, but have a great impact on the recognition. Researchers have employed various methods to complete missing sEMG data, but the completed signal may not totally fit the origins, and more extra calculation time will be spent. When recognizing hand gestures by sEMG from few sensors, to recognize the slightly or not serious signal missing, this study proposed a data split reorganization (DSR) strategy and a weight-based multiple neural network voting (WMV) method. To validate the proposed methods, controllable missing sEMG signals are generated artificially. Three time domain features are extracted based on non-overlapping sliding windows. The DSR is employed to make full use of the features, and then the WMV is utilized to recognize them. Nine subjects participated in the experiments, and the results indicate that the accuracy of the proposed methods is higher. For 5%, 10%, and 15% data missing ratios, the accuracy is 93.66%, 92.55%, and 91.19%, respectively. The Wilcoxon signed-rank test also demonstrates that these results are significantly superior to the situations in which the proposed methods are not applied. In the future, we will optimize the proposed methods to recognize the seriously missing sEMG signal.

Comparison of Multilayer Neural Network Models in Terms of Success of Classifications Based on EmguCV, ML.NET and Tensorflow.Net

In this paper, we compare three different models of multilayer neural networks in terms of their success in the classification phase. These models were designed for EmguCV, ML.NET and Tensorflow.Net libraries, which are currently among the most widely used libraries in the implementation of an automatic recognition system. Using the EmguCV library, we achieved a success rate in the classification of human faces of 81.95% and with ML.NET, which was based on the pre-trained ResNet50 model using convolution layers, up to 91.15% accuracy. The result of the success of the classification process was influenced by the time required for training and also the time required for the classification itself. The Tensorflow.Net model did not show sufficient classification ability when classifying using vector distances; the highest success rate of classification was only 13.31%. Neural networks were trained on a dataset with 1454 photographs of faces involving 43 people. At a time when neural networks are becoming more and more used for applications of different natures, it is necessary to choose the right model in the classification process that will be able to achieve the required accuracy with the minimum time required for training. The application created by us allows the insertion of images and the creation of their own datasets, on the basis of which the user can train a model with its own parameters. Models can then be saved and integrated into other applications.

Neural Network-Based Information Transfer for Dynamic Optimization

In dynamic optimization problems (DOPs), as the environment changes through time, the optima also dynamically change. How to adapt to the dynamic environment and quickly find the optima in all environments is a challenging issue in solving DOPs. Usually, a new environment is strongly relevant to its previous environment. If we know how it changes from the previous environment to the new one, then we can transfer the information of the previous environment, e.g., past solutions, to get new promising information of the new environment, e.g., new high-quality solutions. Thus, in this paper, we propose a neural network (NN)-based information transfer method, named NNIT, to learn the transfer model of environment changes by NN and then use the learned model to reuse the past solutions. When the environment changes, NNIT first collects the solutions from both the previous environment and the new environment and then uses an NN to learn the transfer model from these solutions. After that, the NN is used to transfer the past solutions to new promising solutions for assisting the optimization in the new environment. The proposed NNIT can be incorporated into population-based evolutionary algorithms (EAs) to solve DOPs. Several typical state-of-the-art EAs for DOPs are selected for comprehensive study and evaluated using the widely used moving peaks benchmark. The experimental results show that the proposed NNIT is promising and can accelerate algorithm convergence.