Locally Weighted Kernel Principal Component Regression Model for Soft Sensing of Nonlinear Time-Variant Processes

HyperVein: A Hyperspectral Image Dataset for Human Vein Detection

HyperSpectral Imaging (HSI) plays a pivotal role in various fields, including medical diagnostics, where precise human vein detection is crucial. HyperSpectral (HS) image data are very large and can cause computational complexities. Dimensionality reduction techniques are often employed to streamline HS image data processing. This paper presents a HS image dataset encompassing left- and right-hand images captured from 100 subjects with varying skin tones. The dataset was annotated using anatomical data to represent vein and non-vein areas within the images. This dataset is utilised to explore the effectiveness of dimensionality reduction techniques, namely: Principal Component Analysis (PCA), Folded PCA (FPCA), and Ward's Linkage Strategy using Mutual Information (WaLuMI) for vein detection. To generate experimental results, the HS image dataset was divided into train and test datasets. Optimum performing parameters for each of the dimensionality reduction techniques in conjunction with the Support Vector Machine (SVM) binary classification were determined using the Training dataset. The performance of the three dimensionality reduction-based vein detection methods was then assessed and compared using the test image dataset. Results show that the FPCA-based method outperforms the other two methods in terms of accuracy. For visualization purposes, the classification prediction image for each technique is post-processed using morphological operators, and results show the significant potential of HS imaging in vein detection.

Supervised Attention-Based Bidirectional Long Short-Term Memory Network for Nonlinear Dynamic Soft Sensor Application

Soft sensors are mathematical methods that describe the dependence of primary variables on secondary variables. A nonlinear characteristic commonly appears in modern industrial process data with increasing complexity and dynamics, which has brought challenges to soft sensor modeling. To solve these issues, a novel supervised attention-based bidirectional long short-term memory (SA-BiLSTM) is first proposed in this paper to handle the nonlinear industrial process modeling with dynamic features. In this SA-BiLSTM model, an attention mechanism is introduced to calculate the correlation between hidden features in each time step, thus avoiding the loss of important information. Furthermore, this approach combines historical quality information and a moving window through a supervised strategy of quality variables. Such manipulation not only extracts and exploits nonlinear dynamic latent information from the process and quality variables but also enhances the model's learning efficiency and overall prediction performance. Finally, two real industrial examples demonstrate the superiority of the proposed method compared to conventional methods.

Industrial Soft Sensor Optimized by Improved PSO: A Deep Representation-Learning Approach

Soft sensors based on deep learning approaches are growing in popularity due to their ability to extract high-level features from training, improving soft sensors' performance. In the training process of such a deep model, the set of hyperparameters is critical to archive generalization and reliability. However, choosing the training hyperparameters is a complex task. Usually, a random approach defines the set of hyperparameters, which may not be adequate regarding the high number of sets and the soft sensing purposes. This work proposes the RB-PSOSAE, a Representation-Based Particle Swarm Optimization with a modified evaluation function to optimize the hyperparameter set of a Stacked AutoEncoder-based soft sensor. The evaluation function considers the mean square error (MSE) of validation and the representation of the features extracted through mutual information (MI) analysis in the pre-training step. By doing this, the RB-PSOSAE computes hyperparameters capable of supporting the training process to generate models with improved generalization and relevant hidden features. As a result, the proposed method can generate more than 16.4% improvement in RMSE compared to another standard PSO-based method and, in some cases, more than 50% improvement compared to traditional methods applied to the same real-world nonlinear industrial process. Thus, the results demonstrate better prediction performance than traditional and state-of-the-art methods.

A Deep Supervised Learning Framework for Data-Driven Soft Sensor Modeling of Industrial Processes

Deep learning has been recently introduced for soft sensors in industrial processes. However, most of the existing deep networks, such as stacked autoencoder, are pretrained in a layerwise unsupervised way to learn feature representations for the raw input data itself. For soft sensors, it is necessary to extract quality-relevant features for quality prediction. Thus, a deep layerwise supervised pretraining framework is proposed for quality-relevant feature extraction and soft sensor modeling in this article, which is based on stacked supervised encoder-decoder (SSED). In SSED, hierarchical quality-relevant features are successively learned by a number of supervised encoder-decoder (SED) models. For each SED, the features from the previous hidden layer are served as new inputs to generate the high-level features that are learned with the constraint of predicting the quality data as good as possible at the output layer of this SED. With this new structure, the SED can learn quality-relevant features that can largely improve the prediction performance. By stacking multiple SEDs, hierarchical quality-relevant features can be progressively learned, and irrelevant information is gradually reduced by deep SSED network. The effectiveness of the proposed model is demonstrated on a numerical example and an industrial process of the debutanizer column.

Autoencoder-based nonlinear Bayesian locally weighted regression for soft sensor development

The framework of locally weighted learning (LWL) has established itself as a popular tool for developing nonlinear soft sensors in process industries. For LWL-based soft sensors, the key factor for achieving high performance is to construct accurate localized models. To this end, in this paper a nonlinear local model training algorithm called nonlinear Bayesian weighted regression (NBWR) is proposed. In the NBWR, the nonlinear features of process data are first extracted by the autoencoder; then, given a query sample a local dataset is selected on the feature space and a fully Bayesian regression model with differentiated sample weights is developed. The benefits of this approach, which include better consistency of correlation, stronger abilities to deal with process nonlinearities and uncertainties, overfitting and numerical issues, lead to superior performance. The NBWR is used for developing a soft sensor under the LWL framework, and a real-world industrial process is used to evaluate the performance of the NBWR-based soft sensor. The experimental results demonstrate that the proposed method outperforms several benchmarking soft sensing approaches.

Temperature prediction for roller kiln based on hybrid first-principle model and data-driven MW-DLWKPCR model

In this paper, a hybrid temperature prediction model is developed for an industrial roller kiln of lithium-ion battery cathode materials, which is based on first-principle model and moving window-double locally weighted kernel principal component regression (DLKWKPCR). First, the mechanism model is built for the roller kiln according to the energy conservation law and heat transfer mechanism. Since the first-principle model is based on some simplified assumptions, it often results in large estimation errors. Thus, a data-driven error compensation model is further constructed with real-time process running data. In order to handle the strongly nonlinear, highly redundant and gradually time-varying characteristics, the error compensation model is built with moving window based DLWKPCR. Finally, a hybrid temperature prediction model is obtained by combining the compensation model and the mechanism model. An industrial roller kiln is utilized to test the effectiveness of the hybrid prediction model, in which the modeling results demonstrate that the developed hybrid prediction model can correctly estimate the roller kiln temperature.

A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network

Deep learning networks have been recently utilized for fault detection and diagnosis (FDD) due to its effectiveness in handling industrial process data, which are often with high nonlinearities and strong correlations. However, the valuable information in the raw data may be filtered with the layer-wise feature compression in traditional deep networks. This cannot benefit for the subsequent fine-tuning phase of fault classification. To alleviate this problem, an extended deep belief network (EDBN) is proposed to fully exploit useful information in the raw data, in which raw data is combined with the hidden features as inputs to each extended restricted Boltzmann machine (ERBM) during the pre-training phase. Then, a dynamic EDBN-based fault classifier is constructed to take the dynamic characteristics of process data into consideration. Finally, to test the performance of the proposed method, it is applied to the Tennessee Eastman (TE) process for fault classification. By comparing EDBN and DBN under different network structures, the results show that EDBN has better feature extraction and fault classification performance than traditional DBN.

Evaluating the Performance of Newly Integrated Model in Nonlinear Chemical Process Against Missing Measurements

Application of data-driven soft sensors in manufacturing fields, for instance, chemical, pharmaceutical, and bioprocess have rapidly grown. The issue of missing measurements is common in chemical processing industries that involve data-driven soft sensors. Locally weighted Kernel partial least squares (LW-KPLS) algorithm has recently been proposed to develop adaptive soft sensors for nonlinear processes. This algorithm generally works well for complete datasets; however, it is unable to cope well with any datasets comprising missing measurements. Despite the above issue, limited studies can be found in assessing the effects of incomplete data and their treatment method on the predictive performances of LW-KPLS. To address these research gaps, therefore, a trimmed scores regression (TSR) based missing data imputation method was integrated to LW-KPLS to formulate trimmed scores regression assisted locally weighted Kernel partial least squares (TSR-LW-KPLS) model. In this study, this proposed TSR-LW-KPLS was employed to deal with missing measurements in nonlinear chemical process data. The performances of TSR-LW-KPLS were evaluated using three case studies having different percentages of missing measurements varying from 5 % to 40 %. The obtained results were then compared to the results from singular value decomposition assisted locally weighted Kernel partial least squares (SVD-LW-KPLS) model. SVD-LW-KPLS was also proposed by incorporating a singular value decomposition (SVD) based missing data treatment method into LW-KPLS. From the comparative studies, it is evident that the predictive accuracies of TSR-LW-KPLS are superior compared to the ones from SVD-LW-KPLS.

Development of Adaptive Soft Sensor Using Locally Weighted Kernel Partial Least Square Model

Locally weighted partial least square (LW-PLS) model has been commonly used to develop adaptive soft sensors and process monitoring for numerous industries which include pharmaceutical, petrochemical, semiconductor, wastewater treatment system and biochemical. The advantages of LW-PLS model are its ability to deal with a large number of input variables, collinearity among the variables and outliers. Nevertheless, since most industrial processes are highly nonlinear, a traditional LW-PLS which is based on a linear model becomes incapable of handling nonlinear processes. Hence, an improved LW-PLS model is required to enhance the adaptive soft sensors in dealing with data nonlinearity. In this work, Kernel function which has nonlinear features was incorporated into LW-PLS model and this proposed model is named locally weighted kernel partial least square (LW-KPLS). Comparisons between LW-PLS and LW-KPLS models in terms of predictive performance and their computational loads were carried out by evaluating both models using data generated from a simulated plant. From the results, it is apparent that in terms of predictive performance LW-KPLS is superior compared to LW-PLS. However, it is found that computational load of LW-KPLS is higher than LW-PLS. After adapting ensemble method with LW-KPLS, computational loads of both models were found to be comparable. These indicate that LW-KPLS performs better than LW-PLS in nonlinear process applications. In addition, evaluation on localization parameter in both LW-PLS and LW-KPLS is also carried out.