An Adaptive Deep Belief Network With Sparse Restricted Boltzmann Machines

CLPREM: A real-time traffic prediction method for 5G mobile network

Network traffic prediction is an important network monitoring method, which is widely used in network resource optimization and anomaly detection. However, with the increasing scale of networks and the rapid development of 5-th generation mobile networks (5G), traditional traffic forecasting methods are no longer applicable. To solve this problem, this paper applies Long Short-Term Memory (LSTM) network, data augmentation, clustering algorithm, model compression, and other technologies, and proposes a Cluster-based Lightweight PREdiction Model (CLPREM), a method for real-time traffic prediction of 5G mobile networks. We have designed unique data processing and classification methods to make CLPREM more robust than traditional neural network models. To demonstrate the effectiveness of the method, we designed and conducted experiments in a variety of settings. Experimental results confirm that CLPREM can obtain higher accuracy than traditional prediction schemes with less time cost. To address the occasional anomaly prediction issue in CLPREM, we propose a preprocessing method that minimally impacts time overhead. This approach not only enhances the accuracy of CLPREM but also effectively resolves the real-time traffic prediction challenge in 5G mobile networks.

DRBM-ClustNet: A Deep Restricted Boltzmann-Kohonen Architecture for Data Clustering

A Bayesian deep restricted Boltzmann-Kohonen architecture for data clustering termed deep restricted Boltzmann machine (DRBM)-ClustNet is proposed. This core-clustering engine consists of a DRBM for processing unlabeled data by creating new features that are uncorrelated and have large variance with each other. Next, the number of clusters is predicted using the Bayesian information criterion (BIC), followed by a Kohonen network (KN)-based clustering layer. The processing of unlabeled data is done in three stages for efficient clustering of the nonlinearly separable datasets. In the first stage, DRBM performs nonlinear feature extraction by capturing the highly complex data representation by projecting the feature vectors of d dimensions into n dimensions. Most clustering algorithms require the number of clusters to be decided a priori; hence, here, to automate the number of clusters in the second stage, we use BIC. In the third stage, the number of clusters derived from BIC forms the input for the KN, which performs clustering of the feature-extracted data obtained from the DRBM. This method overcomes the general disadvantages of clustering algorithms, such as the prior specification of the number of clusters, convergence to local optima, and poor clustering accuracy on nonlinear datasets. In this research, we use two synthetic datasets, 15 benchmark datasets from the UCI Machine Learning repository, and four image datasets to analyze the DRBM-ClustNet. The proposed framework is evaluated based on clustering accuracy and ranked against other state-of-the-art clustering methods. The obtained results demonstrate that the DRBM-ClustNet outperforms state-of-the-art clustering algorithms.

Knowledge Transfer-Based Sparse Deep Belief Network

Deep learning has made remarkable achievements in various applications in recent years. With the increasing computing power and the "black box" problem of neural networks, however, the development of deep neural networks (DNNs) has entered a bottleneck period. This article proposes a novel deep belief network (DBN) based on knowledge transfer and optimization of the network structure. First, a neural-symbolic model is proposed to extract rules to describe the dynamic operation mechanism of the deep network. Second, knowledge fusion is proposed based on the merge and deletion of the extracted rules from the DBN model. Finally, a new DNN, knowledge transfer-based sparse DBN (KT-SDBN) is constructed to generate a sparse network without excessive information loss. In comparison with DBN, KT-SDBN has a more sparse network structure and better learning performance on the existing knowledge and data. The experimental results in the benchmark data indicate that KT-SDBN not only has effective feature learning performance with 30% of the original network parameters but also shows a large compression rate that is far larger than other structure optimization algorithms.

Evolutionary Weighted Broad Learning and Its Application to Fault Diagnosis in Self-Organizing Cellular Networks

As a novel neural network-based learning framework, a broad learning system (BLS) has attracted much attention due to its excellent performance on regression and balanced classification problems. However, it is found to be unsuitable for imbalanced data classification problems because it treats each class in an imbalanced dataset equally. To address this issue, this work proposes a weighted BLS (WBLS) in which the weight assigned to each class depends on the number of samples in it. In order to further boost its classification performance, an improved differential evolution algorithm is proposed to automatically optimize its parameters, including the ones in BLS and newly generated weights. We first optimize the parameters with a training dataset, and then apply them to WBLS on a test dataset. The experiments on 20 imbalanced classification problems have shown that our proposed method can achieve higher classification accuracy than the other methods in terms of several widely used performance metrics. Finally, it is applied to fault diagnosis in self-organizing cellular networks to further show its applicability to industrial application problems.

Learning Gaussian-Bernoulli RBMs Using Difference of Convex Functions Optimization

The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is a useful generative model that captures meaningful features from the given n -dimensional continuous data. The difficulties associated with learning GB-RBM are reported extensively in earlier studies. They indicate that the training of the GB-RBM using the current standard algorithms, namely contrastive divergence (CD) and persistent contrastive divergence (PCD), needs a carefully chosen small learning rate to avoid divergence which, in turn, results in slow learning. In this work, we alleviate such difficulties by showing that the negative log-likelihood for a GB-RBM can be expressed as a difference of convex functions if we keep the variance of the conditional distribution of visible units (given hidden unit states) and the biases of the visible units, constant. Using this, we propose a stochastic difference of convex (DC) functions programming (S-DCP) algorithm for learning the GB-RBM. We present extensive empirical studies on several benchmark data sets to validate the performance of this S-DCP algorithm. It is seen that S-DCP is better than the CD and PCD algorithms in terms of speed of learning and the quality of the generative model learned.

General function approximation of a class of cascade chaotic fuzzy systems

This paper presents an innovative architecture called cascade chaotic fuzzy system (CCFS) for the function approximation and chaotic modeling. The proposed model can dominate complications in the type-2 fuzzy systems and increase the chaotic performance of a whole framework. The proposed cascade structure is based on combining two or more one-dimensional chaotic maps. The combination provides a new chaotic map with more high nonlinearity than its grain maps. The fusion of cascade chaotic structure into the neurons of the membership layer of a conventional fuzzy system makes the CCFS more capable of confronting nonlinear problems. Based on the General Function Approximation and Stone-Weierstrass theorem, we show that the proposed model has the function approximation property. By analyzing the bifurcation diagram and applying the CCFS to the problem of chaotic modeling, the new model is investigated. Simulation results and analysis are demonstrated to illustrate the concept of general function approximation.

Evolutionary optimization assisted delayed deep cycle reservoir modeling method with its application to ship heave motion prediction

As one emerging reservoir modeling method, cycle reservoir with regular jumps (CRJ) provides one effective tool for many time series analysis tasks such as ship heave motion prediction. However, the shallow learning structure of single CRJ model limits its memory capacity and leads to unsatisfactory prediction performance. In order to pursue the stronger dynamic characteristic description of time series data, a delayed deep CRJ model is presented in this paper by integrating the deep learning framework with delay links and the evolutionary optimization for mixed-integer problem. Different from the basic CRJ model with only one reservoir, delayed deep CRJ builds multiple serial reservoirs with inserting the delay links between adjacent reservoirs. Due to the design of dynamic deep learning structure, the memory capacity is enlarged to improve ship heave motion prediction. Aiming at the mix-integer optimization problem in delayed deep CRJ model, a heuristic evolutionary optimization scheme based on the stepwise differential evolution algorithm is applied to determine the delayed deep CRJ parameters automatically. The stepwise differential evolution assisted delayed deep CRJ model can avoid the non-optimal solution resulted from the manual parameter setting effectively. Finally, one numerical example and the real experiment data are utilized to validate the methods and the results demonstrate that delayed deep CRJ model has better prediction performance in contrast to the basic CRJ method.

Deep Learning-Based Model Predictive Control for Continuous Stirred-Tank Reactor System

A continuous stirred-tank reactor (CSTR) system is widely applied in wastewater treatment processes. Its control is a challenging industrial-process-control problem due to great difficulty to achieve accurate system identification. This work proposes a deep learning-based model predictive control (DeepMPC) to model and control the CSTR system. The proposed DeepMPC consists of a growing deep belief network (GDBN) and an optimal controller. First, GDBN can automatically determine its size with transfer learning to achieve high performance in system identification, and it serves just as a predictive model of a controlled system. The model can accurately approximate the dynamics of the controlled system with a uniformly ultimately bounded error. Second, quadratic optimization is conducted to obtain an optimal controller. This work analyzes the convergence and stability of DeepMPC. Finally, the DeepMPC is used to model and control a second-order CSTR system. In the experiments, DeepMPC shows a better performance in modeling, tracking, and antidisturbance than the other state-of-the-art methods.