Different Approaches to SCADA Data Completion in Water Networks

Decomposition Methods for Machine Learning with Small, Incomplete or Noisy Datasets

In many machine learning applications, measurements are sometimes incomplete or noisy resulting in missing features. In other cases, and for different reasons, the datasets are originally small, and therefore, more data samples are required to derive useful supervised or unsupervised classification methods. Correct handling of incomplete, noisy or small datasets in machine learning is a fundamental and classic challenge. In this article, we provide a unified review of recently proposed methods based on signal decomposition for missing features imputation (data completion), classification of noisy samples and artificial generation of new data samples (data augmentation). We illustrate the application of these signal decomposition methods in diverse selected practical machine learning examples including: brain computer interface, epileptic intracranial electroencephalogram signals classification, face recognition/verification and water networks data analysis. We show that a signal decomposition approach can provide valuable tools to improve machine learning performance with low quality datasets.

AI Based Real-Time Signal Reconstruction for Wind Farm with SCADA Sensor Failure

Supervisory Control and Data Acquisition (SCADA) systems used in wind turbines for monitoring the health and performance of a wind farm can suffer from data loss due to sensor failure, transmission link breakdown or network congestion. Sensory data is used for important control decisions and such data loss can make the failures harder to detect. This work proposes various solutions to reconstruct the lost information of important SCADA parameters using Linear and non-linear Artificial Intelligence (AI) algorithms. It comprises of three major contributions; (1) signal reconstruction from other available SCADA parameters, (2) comparison of linear and non-linear AI models, and (3) generalization of the AI algorithms between turbines. Experimental results demonstrate the effectiveness of the developed methodologies for reconstruction of the lost information for valuable planning decisions.

Double Tensor-Decomposition for SCADA Data Completion in Water Networks

Supervisory Control And Data Acquisition (SCADA) systems currently monitor and collect a huge among of data from all kind of processes. Ideally, they must run without interruption, but in practice, some data may be lost due to a sensor failure or a communication breakdown. When it happens, given the nature of these failures, information is lost in bursts, that is, sets of consecutive samples. When this occurs, it is necessary to fill out the gaps of the historical data with a reliable data completion method. This paper presents an ad hoc method to complete the data lost by a SCADA system in case of long bursts. The data correspond to levels of drinking water tanks of a Water Network company which present fluctuation patterns on a daily and a weekly scale. In this work, a new tensorization process and a novel completion algorithm mainly based on two tensor decompositions are presented. Statistical tests are realised, which consist of applying the data reconstruction algorithms, by deliberately removing bursts of data in verified historical databases, to be able to evaluate the real effectiveness of the tested methods. For this application, the presented approach outperforms the other techniques found in the literature.