A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Age and medial compartmental OA were important predictors of the lateral compartmental OA in the discoid lateral meniscus: Analysis using machine learning approach

PURPOSE

The objective of this study was to develop a machine learning model that would predict lateral compartment osteoarthritis (OA) in the discoid lateral meniscus (DLM), from which to then identify factors contributing to lateral compartment OA, with a key focus on the patient's age.

METHODS

Data were collected from 611 patients with symptomatic DLM diagnosed using magnetic resonance imaging between April 2003 and May 2022. Twenty features, including demographic, clinical and radiological data and six algorithms were used to develop the predictive machine learning models. Shapley additive explanation (SHAP) analysis was performed on the best model, in addition to subgroup analyses according to age.

RESULTS

Extreme gradient boosting classifier was identified as the best prediction model, with an area under the receiver operating characteristic curve (AUROC) of 0.968, the highest among all the models, regardless of age (AUROC of 0.977 in young age and AUROC of 0.937 in old age). In the SHAP analysis, the most predictive feature was age, followed by the presence of medial compartment OA. In the subgroup analysis, the most predictive feature was age in young age, whereas the most predictive feature was the presence of medial compartment OA in old age.

CONCLUSION

The machine learning model developed in this study showed a high predictive performance with regard to predicting lateral compartment OA of the DLM. Age was identified as the most important factor, followed by medial compartment OA. In subgroup analysis, medial compartmental OA was found to be the most important factor in the older age group, whereas age remained the most important factor in the younger age group. These findings provide insights that may prove useful for the establishment of strategies for the treatment of patients with symptomatic DLM.

LEVEL OF EVIDENCE

Level III.

Supervised Manifold Learning Based on Multi-Feature Information Discriminative Fusion within an Adaptive Nearest Neighbor Strategy Applied to Rolling Bearing Fault Diagnosis

Rolling bearings are a key component for ensuring the safe and smooth operation of rotating machinery and are very prone to failure. Therefore, intelligent fault diagnosis research on rolling bearings has become a crucial task in the field of mechanical fault diagnosis. This paper proposes research on the fault diagnosis of rolling bearings based on an adaptive nearest neighbor strategy and the discriminative fusion of multi-feature information using supervised manifold learning (AN-MFIDFS-Isomap). Firstly, an adaptive nearest neighbor strategy is proposed using the Euclidean distance and cosine similarity to optimize the selection of neighboring points. Secondly, three feature space transformation and feature information extraction methods are proposed, among which an innovative exponential linear kernel function is introduced to provide new feature information descriptions for the data, enhancing feature sensitivity. Finally, under the adaptive nearest neighbor strategy, a novel AN-MFIDFS-Isomap algorithm is proposed for rolling bearing fault diagnosis by fusing various feature information and classifiers through discriminative fusion with label information. The proposed AN-MFIDFS-Isomap algorithm is validated on the CWRU open dataset and our experimental dataset. The experiments show that the proposed method outperforms other traditional manifold learning methods in terms of data clustering and fault diagnosis.

Exact Gaussian processes for massive datasets via non-stationary sparsity-discovering kernels

A Gaussian Process (GP) is a prominent mathematical framework for stochastic function approximation in science and engineering applications. Its success is largely attributed to the GP's analytical tractability, robustness, and natural inclusion of uncertainty quantification. Unfortunately, the use of exact GPs is prohibitively expensive for large datasets due to their unfavorable numerical complexity of [Formula: see text] in computation and [Formula: see text] in storage. All existing methods addressing this issue utilize some form of approximation-usually considering subsets of the full dataset or finding representative pseudo-points that render the covariance matrix well-structured and sparse. These approximate methods can lead to inaccuracies in function approximations and often limit the user's flexibility in designing expressive kernels. Instead of inducing sparsity via data-point geometry and structure, we propose to take advantage of naturally-occurring sparsity by allowing the kernel to discover-instead of induce-sparse structure. The premise of this paper is that the data sets and physical processes modeled by GPs often exhibit natural or implicit sparsities, but commonly-used kernels do not allow us to exploit such sparsity. The core concept of exact, and at the same time sparse GPs relies on kernel definitions that provide enough flexibility to learn and encode not only non-zero but also zero covariances. This principle of ultra-flexible, compactly-supported, and non-stationary kernels, combined with HPC and constrained optimization, lets us scale exact GPs well beyond 5 million data points.

Fault Diagnosis Using Dynamic Principal Component Analysis and GA Feature Selection Modeling for Industrial Processes

With the continuous expansion of industrial production scale, most of the chemical process variables are nonlinear, multi-modal and dynamic. For some traditional multivariate statistical monitoring and fault diagnosis algorithms, such as principal component analysis (PCA), the premise of its application is that the process data is time-independent. To this end, a dynamic principal component analysis (DPCA) method is proposed. However, since the input matrix of DPCA fault diagnosis needs to add an augmented matrix to the original data matrix, the number of eigenvalues of the augmented matrix is too large and there are many redundant eigenvectors. Therefore, this paper proposes a fault diagnosis and monitoring algorithm combining feature selection and DPCA, which considers the dynamic characteristics of multivariate data and reduces the dimension of the input matrix. At present, the average modeling and diagnostic accuracy of PCA-based fault diagnosis on T2 statistic is 65.49%, and that on Q statistic is 76.78%. The average modeling and diagnostic accuracy of fault diagnosis based on DPCA on T2 statistic is 63.17%, and the average modeling and diagnostic accuracy on Q statistic is 83.65%. Finally, through a TE simulation process, this paper proves that the accuracy is greatly improved when using the method proposed in this paper compared with PCA and DPCA.

A Comparative Study of Deep Neural Network-Aided Canonical Correlation Analysis-Based Process Monitoring and Fault Detection Methods

Multivariate analysis is an important kind of method in process monitoring and fault detection, in which the canonical correlation analysis (CCA) makes use of the correlation change between two groups of variables to distinguish the system status and has been greatly studied and applied. For the monitoring of nonlinear dynamic systems, the deep neural network-aided CCA (DNN-CCA) has received much attention recently, but it lacks a general definition and comparative study of different network structures. Therefore, this article first introduces four deep neural network (DNN) models that are suitable to combine with CCA, and the general form of DNN-CCA is given in detail. Then, the experimental comparison of these methods is conducted through three cases, so as to analyze the characteristics and distinctions of CCA aided by each DNN model. Finally, some suggestions on method selection are summarized, and the existed open issues in the current DNN-CCA form and future directions are discussed.

Unsupervised and semi-supervised learning: the next frontier in machine learning for plant systems biology

Advances in high-throughput omics technologies are leading plant biology research into the era of big data. Machine learning (ML) performs an important role in plant systems biology because of its excellent performance and wide application in the analysis of big data. However, to achieve ideal performance, supervised ML algorithms require large numbers of labeled samples as training data. In some cases, it is impossible or prohibitively expensive to obtain enough labeled training data; here, the paradigms of unsupervised learning (UL) and semi-supervised learning (SSL) play an indispensable role. In this review, we first introduce the basic concepts of ML techniques, as well as some representative UL and SSL algorithms, including clustering, dimensionality reduction, self-supervised learning (self-SL), positive-unlabeled (PU) learning and transfer learning. We then review recent advances and applications of UL and SSL paradigms in both plant systems biology and plant phenotyping research. Finally, we discuss the limitations and highlight the significance and challenges of UL and SSL strategies in plant systems biology.

Linear Chain Conditional Random Field for Operating Mode Identification and Multimode Process Monitoring

As a supervised machine learning algorithm, conditional random fields are mainly used for fault classification, which cannot detect new unknown faults. In addition, faulty variable location based on them has not been studied. In this paper, conditional random fields with a linear chain structure are utilized for modeling multimode processes with transitions. A linear chain conditional random field model is trained by normal data with mode label. This model is able to distinguish transitions from stable modes well. After mode identification, the expectation of state feature function is developed for fault detection and faulty variable location. Case studies on the Tennessee Eastman process and continuous stirred tank reactor (CSTR) testify the effectiveness of the proposed approach.

Criteria for optimizing kernel methods in fault monitoring process: A survey

Nowadays, how to select the kernel function and their parameters for ensuring high-performance indicators in fault diagnosis applications remains as two open research issues. This paper provides a comprehensive literature survey of kernel-preprocessing methods in condition monitoring tasks, with emphasis on the procedures for selecting their parameters. Accordingly, twenty kernel optimization criteria and sixteen kernel functions are analyzed. A kernel evaluation framework is further provided for helping in the selection and adjustment of kernel functions. The proposal is validated via a KPCA-based monitoring scheme and two well-known benchmark processes.

Nonlinear Dynamic Process Monitoring Based on Ensemble Kernel Canonical Variate Analysis and Bayesian Inference

By considering autocorrelation among process data, canonical variate analysis (CVA) can noticeably enhance fault detection performance. To monitor nonlinear dynamic processes, a kernel CVA (KCVA) model was developed by performing CVA in the kernel space generated by kernel principal component analysis (KPCA). The Gaussian kernel is widely adopted in KPCA for nonlinear process monitoring. In Gaussian kernel-based process monitoring, a single learner is represented by a certain selected kernel bandwidth. However, the selection of kernel bandwidth plays a pivotal role in the performance of process monitoring. Usually, the kernel bandwidth is determined manually. In this paper, a novel ensemble kernel canonical variate analysis (EKCVA) method is developed by integrating ensemble learning and kernel canonical variate analysis. Compared to a single learner, the ensemble learning method usually achieves greatly superior generalization performance through the combination of multiple base learners. Inspired by the ensemble learning method, KCVA models are established by using different kernel bandwidths. Further, two widely used T ² and Q monitoring statistics are constructed for each model. To improve process monitoring performance, these statistics are combined through Bayesian inference. A numerical example and two industrial benchmarks, the continuous stirred-tank reactor process and the Tennessee Eastman process, are carried out to demonstrate the superiority of the proposed method.

A roadmap for multi-omics data integration using deep learning

High-throughput next-generation sequencing now makes it possible to generate a vast amount of multi-omics data for various applications. These data have revolutionized biomedical research by providing a more comprehensive understanding of the biological systems and molecular mechanisms of disease development. Recently, deep learning (DL) algorithms have become one of the most promising methods in multi-omics data analysis, due to their predictive performance and capability of capturing nonlinear and hierarchical features. While integrating and translating multi-omics data into useful functional insights remain the biggest bottleneck, there is a clear trend towards incorporating multi-omics analysis in biomedical research to help explain the complex relationships between molecular layers. Multi-omics data have a role to improve prevention, early detection and prediction; monitor progression; interpret patterns and endotyping; and design personalized treatments. In this review, we outline a roadmap of multi-omics integration using DL and offer a practical perspective into the advantages, challenges and barriers to the implementation of DL in multi-omics data.

A p−V Diagram Based Fault Identification for Compressor Valve by Means of Linear Discrimination Analysis

The pressure-volume diagram (p−V diagram) is an established method for analyzing the thermodynamic process in the cylinder of a reciprocating compressor as well as the fault of its core components including valves. The failure of suction/discharge valves is the most common cause of unscheduled shutdowns, and undetected failure may lead to catastrophic accidents. Although researchers have investigated fault classification by various estimation techniques and case studies, few have looked deeper into the barriers and pathways to realize the level determination of faults. The initial stage of valve failure is characterized in the form of mild leakage; if this is identified at this period, more serious accidents can be prevented. This study proposes a fault diagnosis and severity estimation method of the reciprocating compressor valve by virtue of features extracted from the p−V diagram. Four-dimensional characteristic variables consisting of the pressure ratio, process angle coefficient, area coefficient, and process index coefficient are extracted from the p−V diagram. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) were applied to establish the diagnostic model, where PCA realizes feature amplification and projection, then LDA implements feature dimensionality reduction and failure prediction. The method was validated by the diagnosis of various levels of severity of valve leakage in a reciprocating compressor, and further, applied in the diagnosis of two actual faults: Mild leakage caused by the cracked valve plate in a reciprocating compressor, and serious leakage caused by the deformed valve in a hydraulically driven piston compressor for a hydrogen refueling station (HRS).

Fault Detection and Diagnosis for Plasticizing Process of Single-Base Gun Propellant Using Mutual Information Weighted MPCA under Limited Batch Samples Modelling

Aiming at the lack of reliable gradual fault detection and abnormal condition alarm and evaluation ability in the plasticizing process of single-base gun propellant, a fault detection and diagnosis method based on normalized mutual information weighted multiway principal component analysis (NMI-WMPCA) under limited batch samples modelling was proposed. In this method, the differences of coupling correlation among multi-dimensional process variables and the coupling characteristics of linear and nonlinear relationships in the process are considered. NMI-WMPCA utilizes the generalization ability of a multi-model to establish an accurate fault detection model in limited batch samples, and adopts fault diagnosis methods based on a multi-model SPE statistic contribution plot to identify the fault source. The experimental results demonstrate that the proposed method is effective, which can realize the rapid detection and diagnosis of multiple faults in the plasticizing process.

A Hybrid Intelligent Fault Diagnosis Strategy for Chemical Processes Based on Penalty Iterative Optimization

Process fault is one of the main reasons that a system may appear unreliable, and it affects the safety of a system. The existence of different degrees of noise in the industry also makes it difficult to extract the effective features of the data for the fault diagnosis method based on deep learning. In order to solve the above problems, this paper improves the deep belief network (DBN) and iterates the optimal penalty term by introducing a penalty factor, avoiding the local optimal situation of a DBN and improving the accuracy of fault diagnosis in order to minimize the impact of noise while improving fault diagnosis and process safety. Using the adaptive noise reduction capability of an adaptive lifting wavelet (ALW), a practical chemical process fault diagnosis model (ALW-DBN) is finally proposed. Then, according to the Tennessee–Eastman (TE) benchmark test process, the ALW-DBN model is compared with other methods, showing that the fault diagnosis performance of the enhanced DBN combined with adaptive wavelet denoising has been significantly improved. In addition, the ALW-DBN shows better performance under the influence of different noise levels in the acid gas absorption process, which proves its high adaptability to different noise levels.

Segmentation of Change in Surface Geometry Analysis for Cultural Heritage Applications

This work proposes a change-based segmentation method for applications to cultural heritage (CH) imaging to perform monitoring and assess changes at each surface point. It can be used as a support or component of the 3D sensors to analyze surface geometry changes. In this research, we proposed a new method to identify surface changes employing segmentation based on 3D geometrical data acquired at different time intervals. The geometrical comparison was performed by calculating point-to-point Euclidean distances for each pair of surface points between the target and source geometry models. Four other methods for local distance measurement were proposed and tested. In the segmentation method, we analyze the local histograms of the distances between the measuring points of the source and target models. Then the parameters of these histograms are determined, and predefined classes are assigned to target surface points. The proposed methodology was evaluated by considering two different case studies of restoration issues on CH surfaces and monitoring them over time. The results were presented with a colormap visualization for each category of the detected change in the analysis. The proposed segmentation method will help in the field of conservation and restoration for the documentation and quantification of geometrical surface change information. This analysis can help in decision-making for the assessment of damage and potential prevention of further damage, and the interpretation of measurement results.