Transformation-cost time-series method for analyzing irregularly sampled data

Interpolation and sampling effects on recurrence quantification measures

The recurrence plot and the recurrence quantification analysis (RQA) are well-established methods for the analysis of data from complex systems. They provide important insights into the nature of the dynamics, periodicity, regime changes, and many more. These methods are used in different fields of research, such as finance, engineering, life, and earth science. To use them, the data have usually to be uniformly sampled, posing difficulties in investigations that provide non-uniformly sampled data, as typical in medical data (e.g., heart-beat based measurements), paleoclimate archives (such as sediment cores or stalagmites), or astrophysics (supernova or pulsar observations). One frequently used solution is interpolation to generate uniform time series. However, this preprocessing step can introduce bias to the RQA measures, particularly those that rely on the diagonal or vertical line structure in the recurrence plot. Using prototypical model systems, we systematically analyze differences in the RQA measure average diagonal line length for data with different sampling and interpolation. For real data, we show that the course of this measure strongly depends on the choice of the sampling rate for interpolation. Furthermore, we suggest a correction scheme, which is capable of correcting the bias introduced by the prepossessing step if the interpolation ratio is an integer.

Power spectral estimate for discrete data

The identification of cycles in periodic signals is a ubiquitous problem in time series analysis. Many real-world datasets only record a signal as a series of discrete events or symbols. In some cases, only a sequence of (non-equidistant) times can be assessed. Many of these signals are furthermore corrupted by noise and offer a limited number of samples, e.g., cardiac signals, astronomical light curves, stock market data, or extreme weather events. We propose a novel method that provides a power spectral estimate for discrete data. The edit distance is a distance measure that allows us to quantify similarities between non-equidistant event sequences of unequal lengths. However, its potential to quantify the frequency content of discrete signals has so far remained unexplored. We define a measure of serial dependence based on the edit distance, which can be transformed into a power spectral estimate (EDSPEC), analogous to the Wiener-Khinchin theorem for continuous signals. The proposed method is applied to a variety of discrete paradigmatic signals representing random, correlated, chaotic, and periodic occurrences of events. It is effective at detecting periodic cycles even in the presence of noise and for short event series. Finally, we apply the EDSPEC method to a novel catalog of European atmospheric rivers (ARs). ARs are narrow filaments of extensive water vapor transport in the lower troposphere and can cause hazardous extreme precipitation events. Using the EDSPEC method, we conduct the first spectral analysis of European ARs, uncovering seasonal and multi-annual cycles along different spatial domains. The proposed method opens new research avenues in studying of periodic discrete signals in complex real-world systems.

Decline in seasonal predictability potentially destabilized Classic Maya societies

Classic Maya populations living in peri-urban states were highly dependent on seasonally distributed rainfall for reliable surplus crop yields. Despite intense study of the potential impact of decadal to centennial-scale climatic changes on the demise of Classic Maya sociopolitical institutions (750-950 CE), its direct importance remains debated. We provide a detailed analysis of a precisely dated speleothem record from Yok Balum cave, Belize, that reflects local hydroclimatic changes at seasonal scale over the past 1600 years. We find that the initial disintegration of Maya sociopolitical institutions and population decline occurred in the context of a pronounced decrease in the predictability of seasonal rainfall and severe drought between 700 and 800 CE. The failure of Classic Maya societies to successfully adapt to volatile seasonal rainfall dynamics likely contributed to gradual but widespread processes of sociopolitical disintegration. We propose that the complex abandonment of Classic Maya population centres was not solely driven by protracted drought but also aggravated by year-to-year decreases in rainfall predictability, potentially caused by a regional reduction in coherent Intertropical Convergence Zone-driven rainfall.

Multiple Sensors Data Integration for Traffic Incident Detection Using the Quadrant Scan

Non-recurrent congestion disrupts normal traffic operations and lowers travel time (TT) reliability, which leads to many negative consequences such as difficulties in trip planning, missed appointments, loss in productivity, and driver frustration. Traffic incidents are one of the six causes of non-recurrent congestion. Early and accurate detection helps reduce incident duration, but it remains a challenge due to the limitation of current sensor technologies. In this paper, we employ a recurrence-based technique, the Quadrant Scan, to analyse time series traffic volume data for incident detection. The data is recorded by multiple sensors along a section of urban highway. The results show that the proposed method can detect incidents better by integrating data from the multiple sensors in each direction, compared to using them individually. It can also distinguish non-recurrent traffic congestion caused by incidents from recurrent congestion. The results show that the Quadrant Scan is a promising algorithm for real-time traffic incident detection with a short delay. It could also be extended to other non-recurrent congestion types.

Sampling rate-corrected analysis of irregularly sampled time series

The analysis of irregularly sampled time series remains a challenging task requiring methods that account for continuous and abrupt changes of sampling resolution without introducing additional biases. The edit distance is an effective metric to quantitatively compare time series segments of unequal length by computing the cost of transforming one segment into the other. We show that transformation costs generally exhibit a nontrivial relationship with local sampling rate. If the sampling resolution undergoes strong variations, this effect impedes unbiased comparison between different time episodes. We study the impact of this effect on recurrence quantification analysis, a framework that is well suited for identifying regime shifts in nonlinear time series. A constrained randomization approach is put forward to correct for the biased recurrence quantification measures. This strategy involves the generation of a type of time series and time axis surrogates which we call sampling-rate-constrained (SRC) surrogates. We demonstrate the effectiveness of the proposed approach with a synthetic example and an irregularly sampled speleothem proxy record from Niue island in the central tropical Pacific. Application of the proposed correction scheme identifies a spurious transition that is solely imposed by an abrupt shift in sampling rate and uncovers periods of reduced seasonal rainfall predictability associated with enhanced El Niño-Southern Oscillation and tropical cyclone activity.

A complex network approach to study the extreme precipitation patterns in a river basin

The quantification of spatial propagation of extreme precipitation events is vital in water resources planning and disaster mitigation. However, quantifying these extreme events has always been challenging as many traditional methods are insufficient to capture the nonlinear interrelationships between extreme event time series. Therefore, it is crucial to develop suitable methods for analyzing the dynamics of extreme events over a river basin with a diverse climate and complicated topography. Over the last decade, complex network analysis emerged as a powerful tool to study the intricate spatiotemporal relationship between many variables in a compact way. In this study, we employ two nonlinear concepts of event synchronization and edit distance to investigate the extreme precipitation pattern in the Ganga river basin. We use the network degree to understand the spatial synchronization pattern of extreme rainfall and identify essential sites in the river basin with respect to potential prediction skills. The study also attempts to quantify the influence of precipitation seasonality and topography on extreme events. The findings of the study reveal that (1) the network degree is decreased in the southwest to northwest direction, (2) the timing of 50th percentile precipitation within a year influences the spatial distribution of degree, (3) the timing is inversely related to elevation, and (4) the lower elevation greatly influences connectivity of the sites. The study highlights that edit distance could be a promising alternative to analyze event-like data by incorporating event time and amplitude and constructing complex networks of climate extremes.

On detecting dynamical regime change using a transformation cost metric between persistent homology diagrams

This work outlines a pipeline for time series analysis that incorporates a measure of similarity not previously applied between homological summaries. Specifically, the well-established, but disparate, methods of persistent homology and TrAnsformation Cost Time Series (TACTS) are combined to provide a metric for tracking dynamics via changing homological features. TACTS allows subtle changes in dynamics to be accounted for, gives a quantitative output that can be directly interpreted, and is tunable to provide several complementary perspectives simultaneously. Our method is demonstrated first with known dynamical systems and then with a real-world electrocardiogram dataset. This paper highlights inadequacies in existing persistent homology metrics and describes circumstances where TACTS can be more sensitive and better suited to detecting a variety of regime changes.

Automatic detection of abrupt transitions in paleoclimate records

Bifurcations and tipping points (TPs) are an important part of the Earth system's behavior. These critical points represent thresholds at which small changes in the system's parameters or in the forcing abruptly switch it from one state or type of behavior to another. Current concern with TPs is largely due to the potential of slow anthropogenic forcing to bring about abrupt, and possibly irreversible, change to the physical climate system and impacted ecosystems. Paleoclimate proxy records have been shown to contain abrupt transitions, or "jumps," which may represent former instances of such dramatic climate change events. These transitions can provide valuable information for identifying critical TPs in current and future climate evolution. Here, we present a robust methodology for detecting abrupt transitions in proxy records that is applied to ice core and speleothem records of the last climate cycle. This methodology is based on the nonparametric Kolmogorov-Smirnov (KS) test for the equality, or not, of the probability distributions associated with two samples drawn from a time series, before and after any potential jump. To improve the detection of abrupt transitions in proxy records, the KS test is augmented by several other criteria and it is compared with recurrence analysis. The augmented KS test results show substantial skill when compared with more subjective criteria for jump detection. This test can also usefully complement recurrence analysis and improve upon certain aspects of its results.

Classification of Chaotic Signals of the Recurrence Matrix Using a Convolutional Neural Network and Verification through the Lyapunov Exponent

This study classified chaotic time series data, including smooth and nonsmooth problems in a dynamic system, using a convolutional neural network (CNN) and verified it through the Lyapunov exponent. For this, the classical nonlinear differential equation by the Lorenz model was used to analyze a smooth dynamic system. The vibro-impact model was used for the nonsmooth dynamic system. Recurrence is a fundamental property of a dynamic system, and a recurrence plot is a representative method to visualize the recurrence characteristics of reconstructed phase space. Therefore, this study calculated the Lyapunov exponent by parametric analysis and visualized the corresponding recurrence matrix to show the dynamic characteristics as an image. In addition, the dynamic characteristics were classified using the proposed CNN model. The proposed CNN model determined chaos with an accuracy of more than 92%.

Two efficient calculations of edit distance between marked point processes

In this paper, we propose to use linear programming methods or a more specialized method, namely, the Hungarian method, for speeding up the exact calculation of an edit distance for marked point processes [Y. Hirata and K. Aihara, Chaos 25, 123117 (2015)]. The key observation is that the problem of calculating the edit distance reduces to a matching problem on a bipartite graph. Our preliminary numerical results show that the proposed implementations are faster than the conventional ones by a factor of 10-1000.

Phase space reconstruction for non-uniformly sampled noisy time series

Analyzing data from paleoclimate archives such as tree rings or lake sediments offers the opportunity of inferring information on past climate variability. Often, such data sets are univariate and a proper reconstruction of the system's higher-dimensional phase space can be crucial for further analyses. In this study, we systematically compare the methods of time delay embedding and differential embedding for phase space reconstruction. Differential embedding relates the system's higher-dimensional coordinates to the derivatives of the measured time series. For implementation, this requires robust and efficient algorithms to estimate derivatives from noisy and possibly non-uniformly sampled data. For this purpose, we consider several approaches: (i) central differences adapted to irregular sampling, (ii) a generalized version of discrete Legendre coordinates, and (iii) the concept of Moving Taylor Bayesian Regression. We evaluate the performance of differential and time delay embedding by studying two paradigmatic model systems-the Lorenz and the Rössler system. More precisely, we compare geometric properties of the reconstructed attractors to those of the original attractors by applying recurrence network analysis. Finally, we demonstrate the potential and the limitations of using the different phase space reconstruction methods in combination with windowed recurrence network analysis for inferring information about past climate variability. This is done by analyzing two well-studied paleoclimate data sets from Ecuador and Mexico. We find that studying the robustness of the results when varying the analysis parameters is an unavoidable step in order to make well-grounded statements on climate variability and to judge whether a data set is suitable for this kind of analysis.

Multiplex recurrence networks

We have introduced a multiplex recurrence network approach by combining recurrence networks with the multiplex network approach in order to investigate multivariate time series. The potential use of this approach is demonstrated on coupled map lattices and a typical example from palaeobotany research. In both examples, topological changes in the multiplex recurrence networks allow for the detection of regime changes in their dynamics. The method goes beyond classical interpretation of pollen records by considering the vegetation as a whole and using the intrinsic similarity in the dynamics of the different regional vegetation elements. We find that the different vegetation types behave more similarly when one environmental factor acts as the dominant driving force.

Prediction of flow dynamics using point processes

Describing a time series parsimoniously is the first step to study the underlying dynamics. For a time-discrete system, a generating partition provides a compact description such that a time series and a symbolic sequence are one-to-one. But, for a time-continuous system, such a compact description does not have a solid basis. Here, we propose to describe a time-continuous time series using a local cross section and the times when the orbit crosses the local cross section. We show that if such a series of crossing times and some past observations are given, we can predict the system's dynamics with fine accuracy. This reconstructability neither depends strongly on the size nor the placement of the local cross section if we have a sufficiently long database. We demonstrate the proposed method using the Lorenz model as well as the actual measurement of wind speed.

Counting forbidden patterns in irregularly sampled time series. II. Reliability in the presence of highly irregular sampling

We are motivated by real-world data that exhibit severe sampling irregularities such as geological or paleoclimate measurements. Counting forbidden patterns has been shown to be a powerful tool towards the detection of determinism in noisy time series. They constitute a set of ordinal symbolic patterns that cannot be realised in time series generated by deterministic systems. The reliability of the estimator of the relative count of forbidden patterns from irregularly sampled data has been explored in two recent studies. In this paper, we explore highly irregular sampling frequency schemes. Using numerically generated data, we examine the reliability of the estimator when the sampling period has been drawn from exponential, Pareto and Gamma distributions of varying skewness. Our investigations demonstrate that some statistical properties of the sampling distribution are useful heuristics for assessing the estimator's reliability. We find that sampling in the presence of large chronological gaps can still yield relatively accurate estimates as long as the time series contains sufficiently many densely sampled areas. Furthermore, we show that the reliability of the estimator of forbidden patterns is poor when there is a high number of sampling intervals, which are larger than a typical correlation time of the underlying system.

See-saw relationship of the Holocene East Asian-Australian summer monsoon

The East Asian-Indonesian-Australian summer monsoon (EAIASM) links the Earth's hemispheres and provides a heat source that drives global circulation. At seasonal and inter-seasonal timescales, the summer monsoon of one hemisphere is linked via outflows from the winter monsoon of the opposing hemisphere. Long-term phase relationships between the East Asian summer monsoon (EASM) and the Indonesian-Australian summer monsoon (IASM) are poorly understood, raising questions of long-term adjustments to future greenhouse-triggered climate change and whether these changes could 'lock in' possible IASM and EASM phase relationships in a region dependent on monsoonal rainfall. Here we show that a newly developed nonlinear time series analysis technique allows confident identification of strong versus weak monsoon phases at millennial to sub-centennial timescales. We find a see-saw relationship over the last 9,000 years-with strong and weak monsoons opposingly phased and triggered by solar variations. Our results provide insights into centennial- to millennial-scale relationships within the wider EAIASM regime.