Deep Time-Series Clustering: A Review

A Hierarchical RF-XGBoost Model for Short-Cycle Agricultural Product Sales Forecasting

Short-cycle agricultural product sales forecasting significantly reduces food waste by accurately predicting demand, ensuring producers match supply with consumer needs. However, the forecasting is often subject to uncertain factors, resulting in highly volatile and discontinuous data. To address this, a hierarchical prediction model that combines RF-XGBoost is proposed in this work. It adopts the Random Forest (RF) in the first layer to extract residuals and achieve initial prediction results based on correlation features from Grey Relation Analysis (GRA). Then, a new feature set based on residual clustering features is generated after the hierarchical clustering is applied to classify the characteristics of the residuals. Subsequently, Extreme Gradient Boosting (XGBoost) acts as the second layer that utilizes those residual clustering features to yield the prediction results. The final prediction is by incorporating the results from the first layer and second layer correspondingly. As for the performance evaluation, using agricultural product sales data from a supermarket in China from 1 July 2020 to 30 June 2023, the results demonstrate superiority over standalone RF and XGBoost, with a Mean Absolute Percentage Error (MAPE) reduction of 10% and 12%, respectively, and a coefficient of determination (R2) increase of 22% and 24%, respectively. Additionally, its generalization is validated across 42 types of agricultural products from six vegetable categories, showing its extensive practical ability. Such performances reveal that the proposed model beneficially enhances the precision of short-term agricultural product sales forecasting, with the advantages of optimizing the supply chain from producers to consumers and minimizing food waste accordingly.

Research on load clustering algorithm based on variational autoencoder and hierarchical clustering

Time series data complexity presents new challenges in clustering analysis across fields such as electricity, energy, industry, and finance. Despite advances in representation learning and clustering with Variational Autoencoders (VAE) based deep learning techniques, issues like the absence of discriminative power in feature representation, the disconnect between instance reconstruction and clustering objectives, and scalability challenges with large datasets persist. This paper introduces a novel deep time series clustering approach integrating VAE with metric learning. It leverages a VAE based on Gated Recurrent Units for temporal feature extraction, incorporates metric learning for joint optimization of latent space representation, and employs the sum of log likelihoods as the clustering merging criterion, markedly improving clustering accuracy and interpretability. Experimental findings demonstrate a 27.16% improvement in average clustering accuracy and a 47.15% increase in speed on industrial load data. This study offers novel insights and tools for the thorough analysis and application of time series data, with further exploration of VAE's potential in time series clustering anticipated in future research.

Unsupervised Time-Series Clustering of Left Atrial Strain for Cardiovascular Risk Assessment

BACKGROUND

Early identification of individuals at high risk for developing cardiovascular (CV) events is of paramount importance for efficient risk management. Here, the authors investigated whether using unsupervised machine learning methods on time-series data of left atrial (LA) strain could distinguish clinically meaningful phenogroups associated with the risk for developing adverse events.

METHODS

In 929 community-dwelling individuals (mean age, 51.6 years; 52.9% women), clinical and echocardiographic data were acquired, including LA strain traces, at baseline, and cardiac events were collected on average 6.3 years later. Two unsupervised learning techniques were used: (1) an ensemble of a deep convolutional neural network autoencoder with k-medoids and (2) a self-organizing map to cluster spatiotemporal patterns within LA strain curves. Clinical characteristics and cardiac outcome were used to evaluate the validity of the k clusters using the original cohort, while an external population cohort (n = 378) was used to validate the trained models.

RESULTS

In both approaches, the optimal number of clusters was five. The first three clusters had differences in sex distribution and heart rate but had a similar low CV risk profile. On the other hand, cluster 5 had the worst CV profile and a higher prevalence of left ventricular remodeling and diastolic dysfunction compared with the other clusters. The respective indexes of cluster 4 were between those of clusters 1 to 3 and 5. After adjustment for traditional risk factors, cluster 5 had the highest risk for cardiac events compared with clusters 1, 2, and 3 (hazard ratio, 1.36; 95% CI, 1.09-1.70; P = .0063). Similar LA strain patterns were obtained when the models were applied to the external validation cohort, and clinical characteristics revealed similar CV risk profiles across all clusters.

CONCLUSION

Unsupervised machine learning algorithms used in time-series LA strain curves identified clinically meaningful clusters of LA deformation and provide incremental prognostic information over traditional risk factors.

Spatialized temporal dynamics of daily ozone concentrations: Identification of the main spatial differences

Ground-level ozone (O₃) is one of the most worrisome air pollutants regarding environmental and health impacts. There is a need for a deeper understanding of its spatial and temporal dynamics. Models are needed to provide continuous temporal and spatial coverage in ozone concentration data with a fine resolution. However, the simultaneous influence of each determinant of ozone dynamics, their spatial and temporal variations, and their interaction make the resulting dynamics of O₃ concentrations difficult to understand. This study aimed to i) identify different classes of temporal dynamics of O₃ at daily and 9 km² resolution over a long-term period of 12 years, ii) identify the potential determinants of these dynamics and, iii) explore the spatial distribution of the potential classes of temporal dynamics on a spatial continuum and over about 1000 km². Thus, 126 time series of 12-year daily ozone concentrations were classified using dynamic time warping (DTW) and hierarchical clustering (study area centered on Besançon, eastern France). The different temporal dynamics obtained differed on elevation, ozone levels, proportions of urbanized and vegetated surfaces. We identified different daily ozone temporal dynamics, spatially structured, that overlapped areas called urban, suburban and rural. Urbanization, elevation and vegetation acted as determinants simultaneously. Individually, elevation and vegetated surface were positively correlated with O₃ concentrations (r = 0.84 and r = 0.41, respectively), while the proportion of urbanized area was negatively correlated with O₃ (r = -0.39). An increasing ozone concentration gradient was observed from urban to rural areas and was reinforced by the elevation gradient. Rural areas were both subject to higher ozone levels (p < 0.001), least monitoring and lower predictability. We identified main determinants of the temporal dynamics of ozone concentrations. The joint influence of determinants was also synthesized. This study proposed a systematic, and reproducible way to build exposure area mapping.

Online Monitoring of Sensor Calibration Status to Support Condition-Based Maintenance

Condition-Based Maintenance (CBM), based on sensors, can only be reliable if the data used to extract information are also reliable. Industrial metrology plays a major role in ensuring the quality of the data collected by the sensors. To guarantee that the values collected by the sensors are reliable, it is necessary to have metrological traceability made by successive calibrations from higher standards to the sensors used in the factories. To ensure the reliability of the data, a calibration strategy must be put in place. Usually, sensors are only calibrated on a periodic basis; so, they often go for calibration without it being necessary or collect data inaccurately. In addition, the sensors are checked often, increasing the need for manpower, and sensor errors are frequently overlooked when the redundant sensor has a drift in the same direction. It is necessary to acquire a calibration strategy based on the sensor condition. Through online monitoring of sensor calibration status (OLM), it is possible to perform calibrations only when it is really necessary. To reach this end, this paper aims to provide a strategy to classify the health status of the production equipment and of the reading equipment that uses the same dataset. A measurement signal from four sensors was simulated, for which Artificial Intelligence and Machine Learning with unsupervised algorithms were used. This paper demonstrates how, through the same dataset, it is possible to obtain distinct information. Because of this, we have a very important feature creation process, followed by Principal Component Analysis (PCA), K-means clustering, and classification based on Hidden Markov Models (HMM). Through three hidden states of the HMM, which represent the health states of the production equipment, we will first detect, through correlations, the features of its status. After that, an HMM filter is used to eliminate those errors from the original signal. Next, an equal methodology is conducted for each sensor individually and using statistical features in the time domain where we can obtain, through HMM, the failures of each sensor.

Application of streaming analytics for Artificial Lift systems: a human-in-the-loop approach for analysing clustered time-series data from progressive cavity pumps

Assessing real-time performance of Artificial Lift Pumps is a prevalent time-series problem to tackle for natural gas operators in Eastern Australia. Multiple physics, data-driven, and hybrid approaches have been investigated to analyse or predict pump performance. However, these methods present a challenge in running compute-heavy algorithms on streaming time-series data. As there is limited research on novel approaches to tackle multivariate time-series analytics for Artificial Lift systems, this paper introduces a human-in-the-loop approach, where petroleum engineers label clustered time-series data to aid in streaming analytics. We rely on our recently developed novel approach of converting streaming time-series data into heatmap images to assist with real-time pump performance analytics. During this study, we were able to automate the labelling of streaming time-series data, which helped petroleum and well surveillance engineers better manage Artificial Lift Pumps through machine learning supported exception-based surveillance. The streaming analytics system developed as part of this research used historical time-series data from three hundred and fifty-nine (359) coal seam gas wells. The developed method is currently used by two natural gas operators, where the operators can accurately detect ten (10) performance-related events and five (5) anomalous events. This paper serves a two-fold purpose; first, we describe a step-by-step methodology that readers can use to reproduce the clustering method for multivariate time-series data. Second, we demonstrate how a human-in-the-loop approach adds value to the proposed method and achieves real-world results.

Deep Temporal Iterative Clustering for Satellite Image Time Series Land Cover Analysis

The extensive amount of Satellite Image Time Series (SITS) data brings new opportunities and challenges for land cover analysis. Many supervised machine learning methods have been applied in SITS, but the labeled SITS samples are time- and effort-consuming to acquire. It is necessary to analyze SITS data with an unsupervised learning method. In this paper, we propose a new unsupervised learning method named Deep Temporal Iterative Clustering (DTIC) to deal with SITS data. The proposed method jointly learns a neural network’s parameters and the resulting features’ cluster assignments, which uses a standard clustering algorithm, K-means, to iteratively cluster the features produced by the feature extraction network and then uses the subsequent assignments as supervision to update the network’s weights. We apply DTIC to the unsupervised training of neural networks on both SITS datasets. Experimental results demonstrate that DTIC outperforms the state-of-the-art K-means clustering algorithm, which proves that the proposed approach successfully provides a novel idea for unsupervised training of SITS data.

Hydrological Time Series Clustering: A Case Study of Telemetry Stations in Thailand

Water level data from telemetry stations typically demonstrate diverse behaviors over time. Specific characteristics can be observed among distinct station groups that are different from others. Clustering time series data into a specified number of groups based on their similarity is an initial step for further analysis in water management analytics. Our main goal in this work is to develop a clustering framework based on a combination of feature representations, feature reduction techniques, as well as clustering algorithms. Thorough experiments on multiple combinations of these methods were conducted and compared. Based on collected water level data in Thailand, UMAP reduced representations of engineered features using HAC clustering with euclidean distance outperformed other methods. Its performance reached 0.8 Fowlkes-Mallows score. Out of 81 stations, only nine unclear cases were incorrectly clustered. Distinct behaviors with abrupt and frequent fluctuations could be perfectly identified.

LSTM-Based Prediction of Mediterranean Vegetation Dynamics Using NDVI Time-Series Data

Vegetation index time-series analysis of multitemporal satellite data is widely used to study vegetation dynamics in the present climate change era. This paper proposes a systematic methodology to predict the Normalized Difference Vegetation Index (NDVI) using time-series data extracted from the Moderate Resolution Imaging Spectroradiometer (MODIS). The key idea is to obtain accurate NDVI predictions by combining the merits of two effective computational intelligence techniques; namely, fuzzy clustering and long short-term memory (LSTM) neural networks under the framework of dynamic time warping (DTW) similarity measure. The study area is the Lesvos Island, located in the Aegean Sea, Greece, which is an insular environment in the Mediterranean coastal region. The algorithmic steps and the main contributions of the current work are described as follows. (1) A data reduction mechanism was applied to obtain a set of representative time series. (2) Since DTW is a similarity measure and not a distance, a multidimensional scaling approach was applied to transform the representative time series into points in a low-dimensional space, thus enabling the use of the Euclidean distance. (3) An efficient optimal fuzzy clustering scheme was implemented to obtain the optimal number of clusters that better described the underline distribution of the low-dimensional points. (4) The center of each cluster was mapped into time series, which were the mean of all representative time series that corresponded to the points belonging to that cluster. (5) Finally, the time series obtained in the last step were further processed in terms of LSTM neural networks. In particular, development and evaluation of the LSTM models was carried out considering a one-year period, i.e., 12 monthly time steps. The results indicate that the method identified unique time-series patterns of NDVI among different CORINE land-use/land-cover (LULC) types. The LSTM networks predicted the NDVI with root mean squared error (RMSE) ranging from 0.017 to 0.079. For the validation year of 2020, the difference between forecasted and actual NDVI was less than 0.1 in most of the study area. This study indicates that the synergy of the optimal fuzzy clustering based on DTW similarity of NDVI time-series data and the use of LSTM networks with clustered data can provide useful results for monitoring vegetation dynamics in fragmented Mediterranean ecosystems.

Non-Intrusive Load Monitoring Applied to AC Railways

Non-intrusive load monitoring takes place in residential and industrial contexts to disaggregate and identify loads connected to a distribution grid. This work studies the applicability and effectiveness for AC railways, considering the highly dynamic behavior of rolling stock as an electric load, immersed in varying contexts of moving loads. Both voltage–current diagrams and harmonic spectra were considered for identification and extraction of features relevant to classification and clustering. Principal components were extracted, approaching the problem using principal component analysis (PCA) and partial least square regression (PLSR). Clustering methods were then discussed, verifying separability performance and applicability to the railway context, checking the performance by means of the balanced accuracy index. Based on more than one hundred measured spectra, PLSR has been confirmed with superior performance and lower complexity. Independent verification based on dispersion and correlation were used to spot relevant spectrum components to use as clustering features and confirm the PLSR outcome.

Singular Spectrum Analysis of Tremorograms for Human Neuromotor Reaction Estimation

Singular spectrum analysis (SSA) is a method of time series analysis and is used in various fields, including medicine. A tremorogram is a biological signal that allows evaluation of a person’s neuromotor reactions in order to infer the state of the motor parts of the central nervous system (CNS). A tremorogram has a complex structure, and its analysis requires the use of advanced methods of signal processing and intelligent analysis. The paper’s novelty lies in the application of the SSA method to extract diagnostically significant features from tremorograms with subsequent evaluation of the state of the motor parts of the CNS. The article presents the application of a method of singular spectrum decomposition, comparison of known variants of classification, and grouping of principal components for determining the components of the tremorogram corresponding to the trend, periodic components, and noise. After analyzing the results of the SSA of tremorograms, we proposed a new algorithm of grouping based on the analysis of singular values of the trajectory matrix. An example of applying the SSA method to the analysis of tremorograms is shown. Comparison of known clustering methods and the proposed algorithm showed that there is a reasonable correspondence between the proposed algorithm and the traditional methods of classification and pairing in the set of periodic components.