A self-organizing deep belief network for nonlinear system modeling

Hybrid deep learning diagonal recurrent neural network controller for nonlinear systems

In the present paper, a hybrid deep learning diagonal recurrent neural network controller (HDL-DRNNC) is proposed for nonlinear systems. The proposed HDL-DRNNC structure consists of a diagonal recurrent neural network (DRNN), whose initial values can be obtained through deep learning (DL). The DL algorithm, which is used in this study, is a hybrid algorithm that is based on a self-organizing map of the Kohonen procedure and restricted Boltzmann machine. The updating weights of the DRNN of the proposed algorithm are developed using the Lyapunov stability criterion. In this concern, simulation tasks such as disturbance signals and parameter variations are performed on mathematical and physical systems to improve the performance and the robustness of the proposed controller. It is clear from the results that the performance of the proposed controller is better than other existent controllers.

Deep Learning-Based Model Predictive Control for Continuous Stirred-Tank Reactor System

A continuous stirred-tank reactor (CSTR) system is widely applied in wastewater treatment processes. Its control is a challenging industrial-process-control problem due to great difficulty to achieve accurate system identification. This work proposes a deep learning-based model predictive control (DeepMPC) to model and control the CSTR system. The proposed DeepMPC consists of a growing deep belief network (GDBN) and an optimal controller. First, GDBN can automatically determine its size with transfer learning to achieve high performance in system identification, and it serves just as a predictive model of a controlled system. The model can accurately approximate the dynamics of the controlled system with a uniformly ultimately bounded error. Second, quadratic optimization is conducted to obtain an optimal controller. This work analyzes the convergence and stability of DeepMPC. Finally, the DeepMPC is used to model and control a second-order CSTR system. In the experiments, DeepMPC shows a better performance in modeling, tracking, and antidisturbance than the other state-of-the-art methods.

Statistical Approaches for Forecasting Primary Air Pollutants: A Review

Air pollutant forecasting can be used to quantitatively estimate pollutant reduction trends. Combining bibliometrics with the evolutionary tree and Markov chain methods can achieve a superior quantitative analysis of research hotspots and trends. In this work, we adopted a bibliometric method to review the research status of statistical prediction methods for air pollution, used evolutionary trees to analyze the development trend of such research, and applied the Markov chain to predict future research trends for major air pollutants. The results indicate that papers mainly focused on the effects of air pollution on human diseases, urban pollution exposure models, and land use regression (LUR) methods. Particulate matter (PM), nitrogen oxides (NOx), and ozone (O3) were the most investigated pollutants. Artificial neural network (ANN) methods were preferred in studies of PM and O3, while LUR were more widely used in studies of NOx. Additionally, multi-method hybrid techniques gradually became the most widely used approach between 2010 and 2018. In the future, the statistical prediction of air pollution is expected to be based on a mixed method to simultaneously predict multiple pollutants, and the interaction between pollutants will be the most challenging aspect of research on air pollution prediction. The research results summarized in this paper provide technical support for the accurate prediction of atmospheric pollution and the emergency management of regional air quality.

PM2.5 concentration modeling and prediction by using temperature-based deep belief network

Air quality prediction is a global hot issue, and PM_2.5 is an important factor affecting air quality. Due to complicated causes of formation, PM_2.5 prediction is a thorny and challenging task. In this paper, a novel deep learning model named temperature-based deep belief networks (TDBN) is proposed to predict the daily concentrations of PM_2.5 for the next day. Firstly, the location of PM_2.5 concentration prediction is Chaoyang Park in Beijing of China from January 1, 2018 to October 27, 2018. The auxiliary variables are selected as input variables of TDBN by Partial Least Square (PLS), and the corresponding data is divided into three independent sections: training samples, validating samples and testing samples. Secondly, the TDBN is composed of temperature-based restricted Boltzmann machine (RBM), where temperature is considered as an effective physical parameter in energy balance of training RBM. The structural parameters of TDBN are determined by minimizing the error in the training process, including hidden layers number, hidden neurons and value of temperature. Finally, the testing samples are used to test the performance of the proposed TDBN on PM_2.5 prediction, and the other similar models are tested by the same testing samples for convenience of comparison with TDBN. The experimental results demonstrate that TDBN performs better than its peers in root mean square error (RMSE), mean absolute error (MAE) and coefficient of determination (R²).

An Adaptive Deep Belief Network With Sparse Restricted Boltzmann Machines

Deep belief network (DBN) is an efficient learning model for unknown data representation, especially nonlinear systems. However, it is extremely hard to design a satisfactory DBN with a robust structure because of traditional dense representation. In addition, backpropagation algorithm-based fine-tuning tends to yield poor performance since its ease of being trapped into local optima. In this article, we propose a novel DBN model based on adaptive sparse restricted Boltzmann machines (AS-RBM) and partial least square (PLS) regression fine-tuning, abbreviated as ARP-DBN, to obtain a more robust and accurate model than the existing ones. First, the adaptive learning step size is designed to accelerate an RBM training process, and two regularization terms are introduced into such a process to realize sparse representation. Second, initial weight derived from AS-RBM is further optimized via layer-by-layer PLS modeling starting from the output layer to input one. Third, we present the convergence and stability analysis of the proposed method. Finally, our approach is tested on Mackey-Glass time-series prediction, 2-D function approximation, and unknown system identification. Simulation results demonstrate that it has higher learning accuracy and faster learning speed. It can be used to build a more robust model than the existing ones.

A Review of the Artificial Neural Network Models for Water Quality Prediction

Water quality prediction plays an important role in environmental monitoring, ecosystem sustainability, and aquaculture. Traditional prediction methods cannot capture the nonlinear and non-stationarity of water quality well. In recent years, the rapid development of artificial neural networks (ANNs) has made them a hotspot in water quality prediction. We have conducted extensive investigation and analysis on ANN-based water quality prediction from three aspects, namely feedforward, recurrent, and hybrid architectures. Based on 151 papers published from 2008 to 2019, 23 types of water quality variables were highlighted. The variables were primarily collected by the sensor, followed by specialist experimental equipment, such as a UV-visible photometer, as there is no mature sensor for measurement at present. Five different output strategies, namely Univariate-Input-Itself-Output, Univariate-Input-Other-Output, Multivariate-Input-Other(multi), Multivariate-Input-Itself-Other-Output, and Multivariate-Input-Itself-Other (multi)-Output, are summarized. From results of the review, it can be concluded that the ANN models are capable of dealing with different modeling problems in rivers, lakes, reservoirs, wastewater treatment plants (WWTPs), groundwater, ponds, and streams. The results of many of the review articles are useful to researchers in prediction and similar fields. Several new architectures presented in the study, such as recurrent and hybrid structures, are able to improve the modeling quality of future development.

A sparse deep belief network with efficient fuzzy learning framework

Deep belief network (DBN) is one of the most feasible ways to realize deep learning (DL) technique, and it has been attracting more and more attentions in nonlinear system modeling. However, DBN cannot provide satisfactory results in learning speed, modeling accuracy and robustness, which is mainly caused by dense representation and gradient diffusion. To address these problems and promote DBN's development in cross-models, we propose a Sparse Deep Belief Network with Fuzzy Neural Network (SDBFNN) for nonlinear system modeling. In this novel framework, the sparse DBN is considered as a pre-training technique to realize fast weight-initialization and to obtain feature vectors. It can balance the dense representation to improve its robustness. A fuzzy neural network is developed for supervised modeling so as to eliminate the gradient diffusion. Its input happens to be the obtained feature vector. As a novel cross-model, SDBFNN combines the advantages of both pre-training technique and fuzzy neural network to improve modeling capability. Its convergence is also analyzed as well. A benchmark problem and a practical problem in wastewater treatment are conducted to demonstrate the superiority of SDBFNN. The extensive experimental results show that SDBFNN achieves better performance than the existing methods in learning speed, modeling accuracy and robustness.