Statistical Mechanics and Information-Theoretic Perspectives on Complexity in the Earth System

Statistical Interdependence between Daily Precipitation and Extreme Daily Temperature in Regions of Mexico and Colombia

We study the statistical interdependence between daily precipitation and daily extreme temperature for regions of Mexico (14 climatic stations, period 1960-2020) and Colombia (7 climatic stations, period 1973-2020) using linear (cross-correlation and coherence) and nonlinear (global phase synchronization index, mutual information, and cross-sample entropy) synchronization metrics. The information shared between these variables is relevant and exhibits changes when comparing regions with different climatic conditions. We show that precipitation and temperature records from La Mojana are characterized by high persistence, while data from Mexico City exhibit lower persistence (less memory). We find that the information exchange and the level of coupling between the precipitation and temperature are higher for the case of the La Mojana region (Colombia) compared to Mexico City (Mexico), revealing that regions where seasonal changes are almost null and with low temperature gradients (less local variability) tend to display higher synchrony compared to regions where seasonal changes are very pronounced. The interdependence characterization between precipitation and temperature represents a robust option to characterize and analyze the collective dynamics of the system, applicable in climate change studies, as well as in changes not easily identifiable in future scenarios.

The Impact of COVID-19 on Weak-Form Efficiency in Cryptocurrency and Forex Markets

The COVID-19 pandemic has had an unprecedented impact on the global economy and financial markets. In this article, we explore the impact of the pandemic on the weak-form efficiency of the cryptocurrency and forex markets by conducting a comprehensive comparative analysis of the two markets. To estimate the weak-form of market efficiency, we utilize the asymmetric market deficiency measure (MDM) derived using the asymmetric multifractal detrended fluctuation analysis (A-MF-DFA) approach, along with fuzzy entropy, Tsallis entropy, and Fisher information. Initially, we analyze the temporal evolution of these four measures using overlapping sliding windows. Subsequently, we assess both the mean value and variance of the distribution for each measure and currency in two distinct time periods: before and during the pandemic. Our findings reveal distinct shifts in efficiency before and during the COVID-19 pandemic. Specifically, there was a clear increase in the weak-form inefficiency of traditional currencies during the pandemic. Among cryptocurrencies, BTC stands out for its behavior, which resembles that of traditional currencies. Moreover, our results underscore the significant impact of COVID-19 on weak-form market efficiency during both upward and downward market movements. These findings could be useful for investors, portfolio managers, and policy makers.

Investigating Dynamical Complexity and Fractal Characteristics of Bitcoin/US Dollar and Euro/US Dollar Exchange Rates around the COVID-19 Outbreak

This article investigates the dynamical complexity and fractal characteristics changes of the Bitcoin/US dollar (BTC/USD) and Euro/US dollar (EUR/USD) returns in the period before and after the outbreak of the COVID-19 pandemic. More specifically, we applied the asymmetric multifractal detrended fluctuation analysis (A-MF-DFA) method to investigate the temporal evolution of the asymmetric multifractal spectrum parameters. In addition, we examined the temporal evolution of Fuzzy entropy, non-extensive Tsallis entropy, Shannon entropy, and Fisher information. Our research was motivated to contribute to the comprehension of the pandemic's impact and the possible changes it caused in two currencies that play a key role in the modern financial system. Our results revealed that for the overall trend both before and after the outbreak of the pandemic, the BTC/USD returns exhibited persistent behavior while the EUR/USD returns exhibited anti-persistent behavior. Additionally, after the outbreak of COVID-19, there was an increase in the degree of multifractality, a dominance of large fluctuations, as well as a sharp decrease of the complexity (i.e., increase of the order and information content and decrease of randomness) of both BTC/USD and EUR/USD returns. The World Health Organization (WHO) announcement, in which COVID-19 was declared a global pandemic, appears to have had a significant impact on the sudden change in complexity. Our findings can help both investors and risk managers, as well as policymakers, to formulate a comprehensive response to the occurrence of such external events.

Evaluating Ecohydrological Model Sensitivity to Input Variability with an Information-Theory-Based Approach

Ecohydrological models vary in their sensitivity to forcing data and use available information to different extents. We focus on the impact of forcing precision on ecohydrological model behavior particularly by quantizing, or binning, time-series forcing variables. We use rate-distortion theory to quantize time-series forcing variables to different precisions. We evaluate the effect of different combinations of quantized shortwave radiation, air temperature, vapor pressure deficit, and wind speed on simulated heat and carbon fluxes for a multi-layer canopy model, which is forced and validated with eddy covariance flux tower observation data. We find that the model is more sensitive to radiation than meteorological forcing input, but model responses also vary with seasonal conditions and different combinations of quantized inputs. While any level of quantization impacts carbon flux similarly, specific levels of quantization influence heat fluxes to different degrees. This study introduces a method to optimally simplify forcing time series, often without significantly decreasing model performance, and could be applied within a sensitivity analysis framework to better understand how models use available information.

An information theory-based approach to characterize drivers of upstream salmon migration

The migration timing of Pacific salmon in the Columbia River basin is subject to multiple influences related to climate, human water resource management, and lagged effects such as oceanic conditions. We apply an information theory-based approach to analyze drivers of adult Chinook salmon migration within the spring and fall spawning seasons and between years based on salmon counts at dams along the Columbia and Snake Rivers. Time-lagged mutual information and information decomposition measures, which characterize lagged and nonlinear dependencies as reductions in uncertainty, are used to detect interactions between salmon counts and lagged streamflows, air and water temperatures, precipitation, snowpack, climate indices and downstream salmon counts. At a daily timescale, these interdependencies reflect migration timing and show differences between fall and spring run salmon, while dependencies based on variables at an annual resolution reflect long-term predictability. We also highlight several types of joint dependencies where predictability of salmon counts depends on the knowledge of multiple lagged sources. This study illustrates how co-varying human and natural drivers could propagate to influence salmon migration timing or overall returns, and how nonlinear types of dependencies between variables enhance predictability of a target. This information-based framework is broadly applicable to assess driving factors in other types of complex water resources systems or species life cycles.

Estimating the variance of Shannon entropy

The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly the most widespread tool is provided by the so-called plug-in estimator, whose statistical properties in terms of bias and variance were investigated since the first decade after the publication of Shannon's seminal works. In the case of an underlying multinomial distribution, while the bias can be evaluated by knowing support and data set size, variance is far more elusive. The aim of the present work is to investigate, in the multinomial case, the statistical properties of an estimator of a parameter that describes the variance of the plug-in estimator of Shannon entropy. We then exactly determine the probability distributions that maximize that parameter. The results presented here allow one to set upper limits to the uncertainty of entropy assessments under the hypothesis of memoryless underlying stochastic processes.

Causality and Information Transfer Between the Solar Wind and the Magnetosphere-Ionosphere System

An information-theoretic approach for detecting causality and information transfer is used to identify interactions of solar activity and interplanetary medium conditions with the Earth's magnetosphere-ionosphere systems. A causal information transfer from the solar wind parameters to geomagnetic indices is detected. The vertical component of the interplanetary magnetic field (Bz) influences the auroral electrojet (AE) index with an information transfer delay of 10 min and the geomagnetic disturbances at mid-latitudes measured by the symmetric field in the H component (SYM-H) index with a delay of about 30 min. Using a properly conditioned causality measure, no causal link between AE and SYM-H, or between magnetospheric substorms and magnetic storms can be detected. The observed causal relations can be described as linear time-delayed information transfer.

Dynamical Complexity of the 2015 St. Patrick's Day Magnetic Storm at Swarm Altitudes Using Entropy Measures

The continuously expanding toolbox of nonlinear time series analysis techniques has recently highlighted the importance of dynamical complexity to understand the behavior of the complex solar wind–magnetosphere–ionosphere–thermosphere coupling system and its components. Here, we apply new such approaches, mainly a series of entropy methods to the time series of the Earth’s magnetic field measured by the Swarm constellation. We show successful applications of methods, originated from information theory, to quantitatively study complexity in the dynamical response of the topside ionosphere, at Swarm altitudes, focusing on the most intense magnetic storm of solar cycle 24, that is, the St. Patrick’s Day storm, which occurred in March 2015. These entropy measures are utilized for the first time to analyze data from a low-Earth orbit (LEO) satellite mission flying in the topside ionosphere. These approaches may hold great potential for improved space weather nowcasts and forecasts.

An information-theoretic approach to extracting climate signals from deep polar ice cores

Paleoclimate records are rich sources of information about the past history of the Earth system. Information theory provides a new means for studying these records. We demonstrate that weighted permutation entropy of water-isotope data from the West Antarctica Ice Sheet (WAIS) Divide ice core reveals meaningful climate signals in this record. We find that this measure correlates with accumulation (meters of ice equivalent per year) and may record the influence of geothermal heating effects in the deepest parts of the core. Dansgaard-Oeschger and Antarctic Isotope Maxima events, however, do not appear to leave strong signatures in the information record, suggesting that these abrupt warming events may actually be predictable features of the climate's dynamics. While the potential power of information theory in paleoclimatology is significant, the associated methods require well-dated and high-resolution data. The WAIS Divide core is the first paleoclimate record that can support this kind of analysis. As more high-resolution records become available, information theory could become a powerful forensic tool in paleoclimate science.

Applications of Information Theory in Solar and Space Physics

Characterizing and modeling processes at the sun and space plasma in our solar system are difficult because the underlying physics is often complex, nonlinear, and not well understood. The drivers of a system are often nonlinearly correlated with one another, which makes it a challenge to understand the relative effects caused by each driver. However, entropy-based information theory can be a valuable tool that can be used to determine the information flow among various parameters, causalities, untangle the drivers, and provide observational constraints that can help guide the development of the theories and physics-based models. We review two examples of the applications of the information theoretic tools at the Sun and near-Earth space environment. In the first example, the solar wind drivers of radiation belt electrons are investigated using mutual information (MI), conditional mutual information (CMI), and transfer entropy (TE). As previously reported, radiation belt electron flux (J_e) is anticorrelated with solar wind density (n_sw) with a lag of 1 day. However, this lag time and anticorrelation can be attributed mainly to the J_e(t + 2 days) correlation with solar wind velocity (V_sw)(t) and n_sw(t + 1 day) anticorrelation with V_sw(t). Analyses of solar wind driving of the magnetosphere need to consider the large lag times, up to 3 days, in the (V_sw, n_sw) anticorrelation. Using CMI to remove the effects of V_sw, the response of J_e to n_sw is 30% smaller and has a lag time <24 h, suggesting that the loss mechanism due to n_sw or solar wind dynamic pressure has to start operating in <24 h. Nonstationarity in the system dynamics is investigated using windowed TE. The triangle distribution in J_e(t + 2 days) vs. V_sw(t) can be better understood with TE. In the second example, the previously identified causal parameters of the solar cycle in the Babcock-Leighton type model such as the solar polar field, meridional flow, polar faculae (proxy for polar field), and flux emergence are investigated using TE. The transfer of information from the polar field to the sunspot number (SSN) peaks at lag times of 3-4 years. Both the flux emergence and the meridional flow contribute to the polar field, but at different time scales. The polar fields from at least the last 3 cycles contain information about SSN.

Anomaly Detection in Paleoclimate Records Using Permutation Entropy

Permutation entropy techniques can be useful for identifying anomalies in paleoclimate data records, including noise, outliers, and post-processing issues. We demonstrate this using weighted and unweighted permutation entropy with water-isotope records containing data from a deep polar ice core. In one region of these isotope records, our previous calculations (See Garland et al. 2018) revealed an abrupt change in the complexity of the traces: specifically, in the amount of new information that appeared at every time step. We conjectured that this effect was due to noise introduced by an older laboratory instrument. In this paper, we validate that conjecture by reanalyzing a section of the ice core using a more advanced version of the laboratory instrument. The anomalous noise levels are absent from the permutation entropy traces of the new data. In other sections of the core, we show that permutation entropy techniques can be used to identify anomalies in the data that are not associated with climatic or glaciological processes, but rather effects occurring during field work, laboratory analysis, or data post-processing. These examples make it clear that permutation entropy is a useful forensic tool for identifying sections of data that require targeted reanalysis—and can even be useful for guiding that analysis.

Common solar wind drivers behind magnetic storm-magnetospheric substorm dependency

The dynamical relationship between magnetic storms and magnetospheric substorms is one of the most controversial issues of contemporary space research. Here, we address this issue through a causal inference approach to two corresponding indices in conjunction with several relevant solar wind variables. We find that the vertical component of the interplanetary magnetic field is the strongest and common driver of both storms and substorms. Further, our results suggest, at least based on the analyzed indices, that there is no statistical evidence for a direct or indirect dependency between substorms and storms and their statistical association can be explained by the common solar drivers. Given the powerful statistical tests we performed (by simultaneously taking into account time series of indices and solar wind variables), a physical mechanism through which substorms directly or indirectly drive storms or vice versa is, therefore, unlikely.

Dynamic process connectivity explains ecohydrologic responses to rainfall pulses and drought

Ecohydrologic fluxes within atmosphere, vegetation, and soil systems exhibit a joint variability that arises from forcing and feedback interactions. These interactions cause fluctuations to propagate between variables at many time scales. In an ecosystem, this connectivity dictates responses to climate change, land-cover change, and weather events and must be characterized to understand resilience and sensitivity. We use an information theory-based approach to quantify connectivity in the form of information flow associated with the propagation of fluctuations between variables. We apply this approach to study ecosystems that experience changes in dry-season moisture availability due to rainfall and drought conditions. We use data from two transects with flux towers located along elevation gradients and quantify redundant, synergistic, and unique flow of information between lagged sources and targets to characterize joint asynchronous time dependencies. At the Reynolds Creek Critical Zone Observatory in Idaho, a dry-season rainfall pulse leads to increased connectivity from soil and atmospheric variables to heat and carbon fluxes. At the Southern Sierra Critical Zone Observatory in California, separate sets of dominant drivers characterize two sites at which fluxes exhibit different drought responses. For both cases, our information flow-based connectivity characterizes dominant drivers and joint variability before, during, and after disturbances. This approach to gauge the responsiveness of ecosystem fluxes under multiple sources of variability furthers our understanding of complex ecohydrologic systems.

Temporal organization of magnetospheric fluctuations unveiled by recurrence patterns in the Dst index

Magnetic storms constitute the most remarkable large-scale phenomena of nonlinear magnetospheric dynamics. Studying the dynamical organization of macroscopic variability in terms of geomagnetic activity index data by means of complexity measures provides a promising approach for identifying the underlying processes and associated time scales. Here, we apply a suite of characteristics from recurrence quantification analysis (RQA) and recurrence network analysis (RNA) in order to unveil some key nonlinear features of the hourly Disturbance storm-time (Dst) index during periods with magnetic storms and such of normal variability. Our results demonstrate that recurrence-based measures can serve as excellent tracers for changes in the dynamical complexity along non-stationary records of geomagnetic activity. In particular, trapping time (characterizing the typical length of "laminar phases" in the observed dynamics) and recurrence network transitivity (associated with the number of the system's effective dynamical degrees of freedom) allow for a very good discrimination between magnetic storm and quiescence phases. In general, some RQA and RNA characteristics distinguish between storm and non-storm times equally well or even better than other previously considered nonlinear characteristics like Hurst exponent or symbolic dynamics based entropy concepts. Our results point to future potentials of recurrence characteristics for unveiling temporal changes in the dynamical complexity of the magnetosphere.

Detecting intermittent switching leadership in coupled dynamical systems

Leader-follower relationships are commonly hypothesized as a fundamental mechanism underlying collective behaviour in many biological and physical systems. Understanding the emergence of such behaviour is relevant in science and engineering to control the dynamics of complex systems toward a desired state. In prior works, due in part to the limitations of existing methods for dissecting intermittent causal relationships, leadership is assumed to be consistent in time and space. This assumption has been contradicted by recent progress in the study of animal behaviour. In this work, we leverage information theory and time series analysis to propose a novel and simple method for dissecting changes in causal influence. Our approach computes the cumulative influence function of a given individual on the rest of the group in consecutive time intervals and identify change in the monotonicity of the function as a change in its leadership status. We demonstrate the effectiveness of our approach to dissect potential changes in leadership on self-propelled particles where the emergence of leader-follower relationship can be controlled and on tandem flights of birds recorded in their natural environment. Our method is expected to provide a novel methodological tool to further our understanding of collective behaviour.

On the Relevance of Computing a Local Version of Sample Entropy in Cardiovascular Control Analysis

OBJECTIVE

Traditional definition of sample entropy (SampEn), here referred to as global SampEn (GSampEn), provides a conditional entropy estimate that blurs the local statistical properties of the time series. We hypothesized that a local version of SampEn (LSampEn) might be more powerful in the presence of determinism than GSampEn.

METHODS

LSampEn was computed by calculating the probability of the current sample conditioned on each reference pattern and averaging it over all reference patterns. The improved ability of LSampEn compared to GSampEn was demonstrated by simulating deterministic periodic, deterministic chaotic, and linear stochastic dynamics corrupted by additive noise and over real cardiovascular variability series recorded from 16 healthy subjects (max-min age range: 22-58 years) during incremental bicycle ergometer exercise.

RESULTS

We found that: i) LSampEn is more robust in describing deterministic periodic or nonlinear features in the presence of additive noise than GSampEn, ii) in association with a surrogate approach, LSampEn is more powerful in detecting nonlinear dynamics than GSampEn, iii) LSampEn and GSampEn are equivalent in the presence of stochastic linear dynamics, and iv) only LSampEn can detect the decrease of complexity of heart period variability during bicycle exercise being a likely hallmark of sympathetic activation.

CONCLUSION

LSampEn preserves the GSampEn capability in characterizing the complexity of short sequences but improves its reliability in the presence of deterministic patterns featuring sharp state transitions and nonlinear dynamics.

SIGNIFICANCE

Variations of complexity can be measured with a greater statistical power over short series using LSampEn, especially when nonlinear features are present.

Transductive Feature Selection Using Clustering-Based Sample Entropy for Temperature Prediction in Weather Forecasting

Entropy measures have been a major interest of researchers to measure the information content of a dynamical system. One of the well-known methodologies is sample entropy, which is a model-free approach and can be deployed to measure the information transfer in time series. Sample entropy is based on the conditional entropy where a major concern is the number of past delays in the conditional term. In this study, we deploy a lag-specific conditional entropy to identify the informative past values. Moreover, considering the seasonality structure of data, we propose a clustering-based sample entropy to exploit the temporal information. Clustering-based sample entropy is based on the sample entropy definition while considering the clustering information of the training data and the membership of the test point to the clusters. In this study, we utilize the proposed method for transductive feature selection in black-box weather forecasting and conduct the experiments on minimum and maximum temperature prediction in Brussels for 1-6 days ahead. The results reveal that considering the local structure of the data can improve the feature selection performance. In addition, despite the large reduction in the number of features, the performance is competitive with the case of using all features.

Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations

Entropy measures are widely applied to quantify the complexity of dynamical systems in diverse fields. However, the practical application of entropy methods is challenging, due to the variety of entropy measures and estimators and the complexity of real-world time series, including nonstationarities and long-range correlations (LRC). We conduct a systematic study on the performance, bias, and limitations of three basic measures (entropy, conditional entropy, information storage) and three traditionally used estimators (linear, kernel, nearest neighbor). We investigate the dependence of entropy measures on estimator- and process-specific parameters, and we show the effects of three types of nonstationarities due to artifacts (trends, spikes, local variance change) in simulations of stochastic autoregressive processes. We also analyze the impact of LRC on the theoretical and estimated values of entropy measures. Finally, we apply entropy methods on heart rate variability data from subjects in different physiological states and clinical conditions. We find that entropy measures can only differentiate changes of specific types in cardiac dynamics and that appropriate preprocessing is vital for correct estimation and interpretation. Demonstrating the limitations of entropy methods and shedding light on how to mitigate bias and provide correct interpretations of results, this work can serve as a comprehensive reference for the application of entropy methods and the evaluation of existing studies.

Recurrence measure of conditional dependence and applications

Identifying causal relations from observational data sets has posed great challenges in data-driven causality inference studies. One of the successful approaches to detect direct coupling in the information theory framework is transfer entropy. However, the core of entropy-based tools lies on the probability estimation of the underlying variables. Here we propose a data-driven approach for causality inference that incorporates recurrence plot features into the framework of information theory. We define it as the recurrence measure of conditional dependence (RMCD), and we present some applications. The RMCD quantifies the causal dependence between two processes based on joint recurrence patterns between the past of the possible driver and present of the potentially driven, excepting the contribution of the contemporaneous past of the driven variable. Finally, it can unveil the time scale of the influence of the sea-surface temperature of the Pacific Ocean on the precipitation in the Amazonia during recent major droughts.

Quantifying information transfer and mediation along causal pathways in complex systems

Measures of information transfer have become a popular approach to analyze interactions in complex systems such as the Earth or the human brain from measured time series. Recent work has focused on causal definitions of information transfer aimed at decompositions of predictive information about a target variable, while excluding effects of common drivers and indirect influences. While common drivers clearly constitute a spurious causality, the aim of the present article is to develop measures quantifying different notions of the strength of information transfer along indirect causal paths, based on first reconstructing the multivariate causal network. Another class of novel measures quantifies to what extent different intermediate processes on causal paths contribute to an interaction mechanism to determine pathways of causal information transfer. The proposed framework complements predictive decomposition schemes by focusing more on the interaction mechanism between multiple processes. A rigorous mathematical framework allows for a clear information-theoretic interpretation that can also be related to the underlying dynamics as proven for certain classes of processes. Generally, however, estimates of information transfer remain hard to interpret for nonlinearly intertwined complex systems. But if experiments or mathematical models are not available, then measuring pathways of information transfer within the causal dependency structure allows at least for an abstraction of the dynamics. The measures are illustrated on a climatological example to disentangle pathways of atmospheric flow over Europe.

Unified functional network and nonlinear time series analysis for complex systems science: The pyunicorn package

We introduce the pyunicorn (Pythonic unified complex network and recurrence analysis toolbox) open source software package for applying and combining modern methods of data analysis and modeling from complex network theory and nonlinear time series analysis. pyunicorn is a fully object-oriented and easily parallelizable package written in the language Python. It allows for the construction of functional networks such as climate networks in climatology or functional brain networks in neuroscience representing the structure of statistical interrelationships in large data sets of time series and, subsequently, investigating this structure using advanced methods of complex network theory such as measures and models for spatial networks, networks of interacting networks, node-weighted statistics, or network surrogates. Additionally, pyunicorn provides insights into the nonlinear dynamics of complex systems as recorded in uni- and multivariate time series from a non-traditional perspective by means of recurrence quantification analysis, recurrence networks, visibility graphs, and construction of surrogate time series. The range of possible applications of the library is outlined, drawing on several examples mainly from the field of climatology.