Entropy production and Kullback-Leibler divergence between stationary trajectories of discrete systems

Information-theoretical limit on the estimates of dissipation by molecular machines using single-molecule fluorescence resonance energy transfer experiments

Single-molecule fluorescence resonance energy transfer (FRET) experiments are commonly used to study the dynamics of molecular machines. While in vivo molecular processes often break time-reversal symmetry, the temporal directionality of cyclically operating molecular machines is often not evident from single-molecule FRET trajectories, especially in the most common two-color FRET studies. Solving a more quantitative problem of estimating the energy dissipation/entropy production by a molecular machine from single-molecule data is even more challenging. Here, we present a critical assessment of several practical methods of doing so, including Markov-model-based methods and a model-free approach based on an information-theoretical measure of entropy production that quantifies how (statistically) dissimilar observed photon sequences are from their time reverses. The Markov model approach is computationally feasible and may outperform model free approaches, but its performance strongly depends on how well the assumed model approximates the true microscopic dynamics. Markov models are also not guaranteed to give a lower bound on dissipation. Meanwhile, model-free, information-theoretical methods systematically underestimate entropy production at low photoemission rates, and long memory effects in the photon sequences make these methods demanding computationally. There is no clear winner among the approaches studied here, and all methods deserve to belong to a comprehensive data analysis toolkit.

Estimator of entropy production for partially accessible Markov networks based on the observation of blurred transitions

A central task in stochastic thermodynamics is the estimation of entropy production for partially accessible Markov networks. We establish an effective transition-based description for such networks with transitions that are not distinguishable and therefore blurred for an external observer. We demonstrate that, in contrast to a description based on fully resolved transitions, this effective description is typically non-Markovian at any point in time. Starting from an information-theoretic bound, we derive an operationally accessible entropy estimator for this observation scenario. We illustrate the operational relevance and the quality of this entropy estimator with a numerical analysis of various representative examples.

Milestoning estimators of dissipation in systems observed at a coarse resolution

Many nonequilibrium, active processes are observed at a coarse-grained level, where different microscopic configurations are projected onto the same observable state. Such "lumped" observables display memory, and in many cases, the irreversible character of the underlying microscopic dynamics becomes blurred, e.g., when the projection hides dissipative cycles. As a result, the observations appear less irreversible, and it is very challenging to infer the degree of broken time-reversal symmetry. Here we show, contrary to intuition, that by ignoring parts of the already coarse-grained state space we may-via a process called milestoning-improve entropy-production estimates. We present diverse examples where milestoning systematically renders observations "closer to underlying microscopic dynamics" and thereby improves thermodynamic inference from lumped data assuming a given range of memory, and we hypothesize that this effect is quite general. Moreover, whereas the correct general physical definition of time reversal in the presence of memory remains unknown, we here show by means of physically relevant examples that at least for semi-Markov processes of first and second order, waiting-time contributions arising from adopting a naive Markovian definition of time reversal generally must be discarded.

Emergence of Memory in Equilibrium versus Nonequilibrium Systems

Experiments often probe observables that correspond to low-dimensional projections of high-dimensional dynamics. In such situations distinct microscopic configurations become lumped into the same observable state. It is well known that correlations between the observable and the hidden degrees of freedom give rise to memory effects. However, how and under which conditions these correlations emerge remain poorly understood. Here we shed light on two fundamentally different scenarios of the emergence of memory in minimal stationary systems, where observed and hidden degrees of freedom either evolve cooperatively or are coupled by a hidden nonequilibrium current. In the reversible setting the strongest memory manifests when the timescales of hidden and observed dynamics overlap, whereas, strikingly, in the driven setting maximal memory emerges under a clear timescale separation. Our results hint at the possibility of fundamental differences in the way memory emerges in equilibrium versus driven systems that may be utilized as a "diagnostic" of the underlying hidden transport mechanism.

From physiological complexity to data interactions-A case study of recordings from exercise monitoring

The popularity of nonlinear analysis has been growing simultaneously with the technology of effort monitoring. Therefore, considering the simple methods of physiological data collection and the approaches from the information domain, we proposed integrating univariate and bivariate analysis for the rest and effort comparison. Two sessions separated by an intensive training program were studied. Nine subjects participated in the first session (S1) and seven in the second session (S2). The protocol included baseline (BAS), exercise, and recovery phase. During all phases, electrocardiogram (ECG) was recorded. For the analysis, we selected corresponding data lengths of BAS and exercise usually lasting less than 5 min. We found the utility of the differences between original data and their surrogates for sample entropy Sdiff and Kullback-Leibler divergence KLDdiff. Sdiff of heart rate variability was negative in BAS and exercise but its sensitivity for phases discrimination was not satisfactory. We studied the bivariate analysis of RR intervals and corresponding QT peaks by Interlayer Mutual Information (IMI) and average edge overlap (AVO) markers. While the IMI parameter decreases in exercise conditions, AVO increased in effort compared to BAS. These findings conclude that researchers should consider a bivariate analysis of extracted RR intervals and corresponding QT datasets, when only ECG is recorded during tests.

Generalized decomposition of multivariate information

Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its utility, the applicability of the PID is restricted by the need to assign elements as either "sources" or "targets", as well as the specific structure of the mutual information itself. Here, I introduce a generalized information decomposition that relaxes the source/target distinction while still satisfying the basic intuitions about information. This approach is based on the decomposition of the Kullback-Leibler divergence, and consequently allows for the analysis of any information gained when updating from an arbitrary prior to an arbitrary posterior. As a result, any information-theoretic measure that can be written as a linear combination of Kullback-Leibler divergences admits a decomposition in the style of Williams and Beer, including the total correlation, the negentropy, and the mutual information as special cases. This paper explores how the generalized information decomposition can reveal novel insights into existing measures, as well as the nature of higher-order synergies. We show that synergistic information is intimately related to the well-known Tononi-Sporns-Edelman (TSE) complexity, and that synergistic information requires a similar integration/segregation balance as a high TSE complexity. Finally, I end with a discussion of how this approach fits into other attempts to generalize the PID and the possibilities for empirical applications.

α-divergence improves the entropy production estimation via machine learning

Recent years have seen a surge of interest in the algorithmic estimation of stochastic entropy production (EP) from trajectory data via machine learning. A crucial element of such algorithms is the identification of a loss function whose minimization guarantees the accurate EP estimation. In this study we show that there exists a host of loss functions, namely, those implementing a variational representation of the α-divergence, which can be used for the EP estimation. By fixing α to a value between -1 and 0, the α-NEEP (Neural Estimator for Entropy Production) exhibits a much more robust performance against strong nonequilibrium driving or slow dynamics, which adversely affects the existing method based on the Kullback-Leibler divergence (α=0). In particular, the choice of α=-0.5 tends to yield the optimal results. To corroborate our findings, we present an exactly solvable simplification of the EP estimation problem, whose loss function landscape and stochastic properties give deeper intuition into the robustness of the α-NEEP.

Irreversible Mesoscale Fluctuations Herald the Emergence of Dynamical Phases

We study fluctuating field models with spontaneously emerging dynamical phases. We consider two typical transition scenarios associated with parity-time symmetry breaking: oscillatory instabilities and critical exceptional points. An analytical investigation of the low-noise regime reveals a drastic increase of the mesoscopic entropy production toward the transitions. For an illustrative model of two nonreciprocally coupled Cahn-Hilliard fields, we find physical interpretations in terms of actively propelled interfaces and a coupling of eigenmodes of the linearized dynamics near the critical exceptional point.

Irreversibility of symbolic time series: A cautionary tale

Many empirical time series are genuinely symbolic: Examples range from link activation patterns in network science, to DNA coding or firing patterns in neuroscience, to cryptography or combinatorics on words. In some other contexts, the underlying time series is actually real valued, and symbolization is applied subsequently, as in symbolic dynamics of chaotic systems. Among several time series quantifiers, time series irreversibility-the difference between forward and backward statistics in stationary time series-is of great relevance. However, the irreversible character of symbolized time series is not always equivalent to the one of the underlying real-valued signal, leading to some misconceptions and confusion on interpretability. Such confusion is even bigger for binary time series-a classical way to encode chaotic trajectories via symbolic dynamics. In this paper we aim to clarify some usual misconceptions and provide theoretical grounding for the practical analysis-and interpretation-of time irreversibility in symbolic time series. We outline sources of irreversibility in stationary symbolic sequences coming from frequency asymmetries of nonpalindromic pairs which we enumerate, and prove that binary time series cannot show any irreversibility based on words of length m<4, thus discussing the implications and sources of confusion. We also study irreversibility in the context of symbolic dynamics, and clarify why these can be reversible even when the underlying dynamical system is not, such as the case of the fully chaotic logistic map.

Kullback-Leibler Divergence of an Open-Queuing Network of a Cell-Signal-Transduction Cascade

Queuing networks (QNs) are essential models in operations research, with applications in cloud computing and healthcare systems. However, few studies have analyzed the cell's biological signal transduction using QN theory. This study entailed the modeling of signal transduction as an open Jackson's QN (JQN) to theoretically determine cell signal transduction, under the assumption that the signal mediator queues in the cytoplasm, and the mediator is exchanged from one signaling molecule to another through interactions between the signaling molecules. Each signaling molecule was regarded as a network node in the JQN. The JQN Kullback-Leibler divergence (KLD) was defined using the ratio of the queuing time (λ) to the exchange time (μ), λ/μ. The mitogen-activated protein kinase (MAPK) signal-cascade model was applied, and the KLD rate per signal-transduction-period was shown to be conserved when the KLD was maximized. Our experimental study on MAPK cascade supported this conclusion. This result is similar to the entropy-rate conservation of chemical kinetics and entropy coding reported in our previous studies. Thus, JQN can be used as a novel framework to analyze signal transduction.

Challenges in Inferring the Directionality of Active Molecular Processes from Single-Molecule Fluorescence Resonance Energy Transfer Trajectories

We discuss some of the practical challenges that one faces in using stochastic thermodynamics to infer directionality of molecular machines from experimental single-molecule trajectories. Because of the limited spatiotemporal resolution of single-molecule experiments and because both forward and backward transitions between the same pairs of states cannot always be detected, differentiating between the forward and backward directions of, e.g., an ATP-consuming molecular machine that operates periodically, turns out to be a nontrivial task. Using a simple extension of a Markov-state model that is commonly employed to analyze single-molecule transition-path measurements, we illustrate how irreversibility can be hidden from such measurements but in some cases can be uncovered when non-Markov effects in low-dimensional single-molecule trajectories are considered.

Model-Free Measurement of Local Entropy Production and Extractable Work in Active Matter

Time-reversal symmetry breaking and entropy production are universal features of nonequilibrium phenomena. Despite its importance in the physics of active and living systems, the entropy production of systems with many degrees of freedom has remained of little practical significance because the high dimensionality of their state space makes it difficult to measure. Here we introduce a local measure of entropy production and a numerical protocol to estimate it. We establish a connection between the entropy production and extractability of work in a given region of the system and show how this quantity depends crucially on the degrees of freedom being tracked. We validate our approach in theory, simulation, and experiments by considering systems of active Brownian particles undergoing motility-induced phase separation, as well as active Brownian particles and E.coli in a rectifying device in which the time-reversal asymmetry of the particle dynamics couples to spatial asymmetry to reveal its effects on a macroscopic scale.

Inferring entropy production rate from partially observed Langevin dynamics under coarse-graining

The entropy production rate (EPR) measures time-irreversibility in systems operating far from equilibrium. The challenge in estimating the EPR for a continuous variable system is the finite spatiotemporal resolution and the limited accessibility to all of the nonequilibrium degrees of freedom. Here, we estimate the irreversibility in partially observed systems following oscillatory dynamics governed by coupled overdamped Langevin equations. We coarse-grain an observed variable of a nonequilibrium driven system into a few discrete states and estimate a lower bound on the total EPR. As a model system, we use hair-cell bundle oscillations driven by molecular motors, such that the bundle tip position is observed, but the positions of the motors are hidden. In the observed variable space, the underlying driven process exhibits second-order semi-Markov statistics. The waiting time distributions (WTD), associated with transitions among the coarse-grained states, are non-exponential and convey the information on the broken time-reversal symmetry. By invoking the underlying time-irreversibility, we calculate a lower bound on the total EPR from the Kullback-Leibler divergence (KLD) between WTD. We show that the mean dwell-time asymmetry factor - the ratio between the mean dwell-times along the forward direction and the backward direction, can qualitatively measure the degree of broken time reversal symmetry and increases with finer spatial resolution. Finally, we apply our methodology to a continuous-time discrete Markov chain model, coarse-grained into a linear system exhibiting second-order semi-Markovian statistics, and demonstrate the estimation of a lower bound on the total EPR from irreversibility manifested only in the WTD.

Assigning Degrees of Stochasticity to Blazar Light Curves in the Radio Band Using Complex Networks

We focus on characterizing the high-energy emission mechanisms of blazars by analyzing the variability in the radio band of the light curves of more than a thousand sources. We are interested in assigning complexity parameters to these sources, modeling the time series of the light curves with the method of the Horizontal Visibility Graph (HVG), which allows us to obtain properties from degree distributions, such as a characteristic exponent to describe its stochasticity and the Kullback–Leibler Divergence (KLD), presenting a new perspective to the methods commonly used to study Active Galactic Nuclei (AGN). We contrast these parameters with the excess variance, which is an astronomical measurement of variability in light curves; at the same time, we use the spectral classification of the sources. While it is not possible to find significant correlations with the excess variance, the degree distributions extracted from the network are detecting differences related to the spectral classification of blazars. These differences suggest a chaotic behavior in the time series for the BL Lac sources and a correlated stochastic behavior in the time series for the FSRQ sources. Our results show that complex networks may be a valuable alternative tool to study AGNs according to the variability of their energy output.

Network Analysis of Time Series: Novel Approaches to Network Neuroscience

In the last two decades, there has been an explosion of interest in modeling the brain as a network, where nodes correspond variously to brain regions or neurons, and edges correspond to structural or statistical dependencies between them. This kind of network construction, which preserves spatial, or structural, information while collapsing across time, has become broadly known as "network neuroscience." In this work, we provide an alternative application of network science to neural data: network-based analysis of non-linear time series and review applications of these methods to neural data. Instead of preserving spatial information and collapsing across time, network analysis of time series does the reverse: it collapses spatial information, instead preserving temporally extended dynamics, typically corresponding to evolution through some kind of phase/state-space. This allows researchers to infer a, possibly low-dimensional, "intrinsic manifold" from empirical brain data. We will discuss three methods of constructing networks from nonlinear time series, and how to interpret them in the context of neural data: recurrence networks, visibility networks, and ordinal partition networks. By capturing typically continuous, non-linear dynamics in the form of discrete networks, we show how techniques from network science, non-linear dynamics, and information theory can extract meaningful information distinct from what is normally accessible in standard network neuroscience approaches.

Situational Understanding in the Human and the Machine

The Air Force research programs envision developing AI technologies that will ensure battlespace dominance, by radical increases in the speed of battlespace understanding and decision-making. In the last half century, advances in AI have been concentrated in the area of machine learning. Recent experimental findings and insights in systems neuroscience, the biophysics of cognition, and other disciplines provide converging results that set the stage for technologies of machine understanding and machine-augmented Situational Understanding. This paper will review some of the key ideas and results in the literature, and outline new suggestions. We define situational understanding and the distinctions between understanding and awareness, consider examples of how understanding-or lack of it-manifest in performance, and review hypotheses concerning the underlying neuronal mechanisms. Suggestions for further R&D are motivated by these hypotheses and are centered on the notions of Active Inference and Virtual Associative Networks.

Time-irreversibility test for random-length time series: The matching-time approach applied to DNA

In this work, we implement the so-called matching-time estimators for estimating the entropy rate as well as the entropy production rate for symbolic sequences. These estimators are based on recurrence properties of the system, which have been shown to be appropriate for testing irreversibility, especially when the sequences have large correlations or memory. Based on limit theorems for matching times, we derive a maximum likelihood estimator for the entropy rate by assuming that we have a set of moderately short symbolic time series of finite random duration. We show that the proposed estimator has several properties that make it adequate for estimating the entropy rate and entropy production rate (or for testing the irreversibility) when the sample sequences have different lengths, such as the coding sequences of DNA. We test our approach with controlled examples of Markov chains, non-linear chaotic maps, and linear and non-linear autoregressive processes. We also implement our estimators for genomic sequences to show that the degree of irreversibility of coding sequences in human DNA is significantly larger than that for the corresponding non-coding sequences.

Non-Equilibrium Entropy and Irreversibility in Generalized Stochastic Loewner Evolution from an Information-Theoretic Perspective

In this study, we theoretically investigated a generalized stochastic Loewner evolution (SLE) driven by reversible Langevin dynamics in the context of non-equilibrium statistical mechanics. Using the ability of Loewner evolution, which enables encoding of non-equilibrium systems into equilibrium systems, we formulated the encoding mechanism of the SLE by Gibbs entropy-based information-theoretic approaches to discuss its advantages as a means to better describe non-equilibrium systems. After deriving entropy production and flux for the 2D trajectories of the generalized SLE curves, we reformulated the system’s entropic properties in terms of the Kullback–Leibler (KL) divergence. We demonstrate that this operation leads to alternative expressions of the Jarzynski equality and the second law of thermodynamics, which are consistent with the previously suggested theory of information thermodynamics. The irreversibility of the 2D trajectories is similarly discussed by decomposing the entropy into additive and non-additive parts. We numerically verified the non-equilibrium property of our model by simulating the long-time behavior of the entropic measure suggested by our formulation, referred to as the relative Loewner entropy.

Breaking of time-reversal symmetry for a particle in a parabolic potential that is subjected to Lévy noise: Theory and an application to solar flare data

The noise in nonequilibrium systems commonly contains more outliers as compared to equilibrium systems and is often best described with a Lévy distribution. Many systems in which there are fluctuations around a steady-state throughput can be modeled as a Lévy-noise-subjected particle in a parabolic potential. We consider an overdamped particle in a parabolic potential that is subjected to noise. Microscopic reversibility and time-reversal symmetry apply if the particle is subject to Gaussian distributed noise, but are violated if the noise is Lévy. A parameter to detect this violation is formulated. We, furthermore, develop an understanding for how the time-reversal asymmetry depends on the time Δt between the sample points and on the stability index, α, of the Lévy noise. With solar flare data it is shown how the time-reversal asymmetry parameter of a signal can be used to obtain the α of the underlying noise.

Exploring dissipative sources of non-Markovian biochemical reaction systems

Many biological processes including important intracellular processes are governed by biochemical reaction networks. Usually, these reaction systems operate far from thermodynamic equilibrium, implying free-energy dissipation. On the other hand, single reaction events happen often in a memory manner, leading to non-Markovian kinetics. A question then arises: how do we calculate free-energy dissipation (defined as the entropy production rate) in this physically real case? We derive an analytical formula for calculating the energy consumption of a general reaction system with molecular memory characterized by nonexponential waiting-time distributions. It shows that this dissipation is composed of two parts: one from broken detailed balance of an equivalent Markovian system with the same topology and substrates, and the other from the direction-time dependence of waiting-time distributions. But, if the system is in a detailed balance and the waiting-time distribution is direction-time independent, there is no energy dissipation even in the non-Markovian case. These general results provide insights into the physical mechanisms underlying nonequilibrium processes. A continuous-time random-walk model and a generalized model of stochastic gene expression are chosen to clearly show dissipative sources and the relationship between energy dissipation and molecular memory.

Time-Reversibility, Causality and Compression-Complexity

Detection of the temporal reversibility of a given process is an interesting time series analysis scheme that enables the useful characterisation of processes and offers an insight into the underlying processes generating the time series. Reversibility detection measures have been widely employed in the study of ecological, epidemiological and physiological time series. Further, the time reversal of given data provides a promising tool for analysis of causality measures as well as studying the causal properties of processes. In this work, the recently proposed Compression-Complexity Causality (CCC) measure (by the authors) is shown to be free of the assumption that the "cause precedes the effect", making it a promising tool for causal analysis of reversible processes. CCC is a data-driven interventional measure of causality (second rung on the Ladder of Causation) that is based on Effort-to-Compress (ETC), a well-established robust method to characterize the complexity of time series for analysis and classification. For the detection of the temporal reversibility of processes, we propose a novel measure called the Compressive Potential based Asymmetry Measure. This asymmetry measure compares the probability of the occurrence of patterns at different scales between the forward-time and time-reversed process using ETC. We test the performance of the measure on a number of simulated processes and demonstrate its effectiveness in determining the asymmetry of real-world time series of sunspot numbers, digits of the transcedental number π and heart interbeat interval variability.

Unified approach to classical speed limit and thermodynamic uncertainty relation

The total entropy production quantifies the extent of irreversibility in thermodynamic systems, which is nonnegative for any feasible dynamics. When additional information such as the initial and final states or moments of an observable is available, it is known that tighter lower bounds on the entropy production exist according to the classical speed limits and the thermodynamic uncertainty relations. Here we obtain a universal lower bound on the total entropy production in terms of probability distributions of an observable in the time forward and backward processes. For a particular case, we show that our universal relation reduces to a classical speed limit, imposing a constraint on the speed of the system's evolution in terms of the Hatano-Sasa entropy production. Notably, the obtained classical speed limit is tighter than the previously reported bound by a constant factor. Moreover, we demonstrate that a generalized thermodynamic uncertainty relation can be derived from another particular case of the universal relation. Our uncertainty relation holds for systems with time-reversal symmetry breaking and recovers several existing bounds. Our approach provides a unified perspective on two closely related classes of inequality: classical speed limits and thermodynamic uncertainty relations.

Estimating entropy rate from censored symbolic time series: A test for time-irreversibility

In this work, we introduce a method for estimating the entropy rate and the entropy production rate from a finite symbolic time series. From the point of view of statistics, estimating entropy from a finite series can be interpreted as a problem of estimating parameters of a distribution with a censored or truncated sample. We use this point of view to give estimations of the entropy rate and the entropy production rate, assuming that they are parameters of a (limit) distribution. The last statement is actually a consequence of the fact that the distribution of estimations obtained from recurrence-time statistics satisfies the central limit theorem. We test our method using a time series coming from Markov chain models, discrete-time chaotic maps, and a real DNA sequence from the human genome.

Learning Entropy Production via Neural Networks

This Letter presents a neural estimator for entropy production (NEEP), that estimates entropy production (EP) from trajectories of relevant variables without detailed information on the system dynamics. For steady state, we rigorously prove that the estimator, which can be built up from different choices of deep neural networks, provides stochastic EP by optimizing the objective function proposed here. We verify the NEEP with the stochastic processes of the bead spring and discrete flashing ratchet models and also demonstrate that our method is applicable to high-dimensional data and can provide coarse-grained EP for Markov systems with unobservable states.

Quantifying the non-equilibrium activity of an active colloid

Active matter systems exhibit rich emergent behavior due to constant injection and dissipation of energy at the level of individual agents. Since these systems are far from equilibrium, their dynamics and energetics cannot be understood using the framework of equilibrium statistical mechanics. Recent developments in stochastic thermodynamics extend classical concepts of work, heat, and energy dissipation to fluctuating non-equilibrium systems. We use recent advances in experiment and theory to study the non-thermal dissipation of individual light-activated self-propelled colloidal particles. We focus on characterizing the transition from thermal to non-thermal fluctuations and show that energy dissipation rates on the order of ∼kBT s-1 are measurable from finite time series data.

Theory of Nonequilibrium Free Energy Transduction by Molecular Machines

Biomolecular machines are protein complexes that convert between different forms of free energy. They are utilized in nature to accomplish many cellular tasks. As isothermal nonequilibrium stochastic objects at low Reynolds number, they face a distinct set of challenges compared with more familiar human-engineered macroscopic machines. Here we review central questions in their performance as free energy transducers, outline theoretical and modeling approaches to understand these questions, identify both physical limits on their operational characteristics and design principles for improving performance, and discuss emerging areas of research.

Inferring broken detailed balance in the absence of observable currents

Identifying dissipation is essential for understanding the physical mechanisms underlying nonequilibrium processes. In living systems, for example, the dissipation is directly related to the hydrolysis of fuel molecules such as adenosine triphosphate (ATP). Nevertheless, detecting broken time-reversal symmetry, which is the hallmark of dissipative processes, remains a challenge in the absence of observable directed motion, flows, or fluxes. Furthermore, quantifying the entropy production in a complex system requires detailed information about its dynamics and internal degrees of freedom. Here we introduce a novel approach to detect time irreversibility and estimate the entropy production from time-series measurements, even in the absence of observable currents. We apply our technique to two different physical systems, namely, a partially hidden network and a molecular motor. Our method does not require complete information about the system dynamics and thus provides a new tool for studying nonequilibrium phenomena.

Non-equilibrium systems with hidden states are relevant for biological systems such as molecular motors. Here the authors introduce a method for quantifying irreversibility in such a system by exploiting the fluctuations in the waiting times of time series data.

Uncertainty relations for time-delayed Langevin systems

The thermodynamic uncertainty relation, which establishes a universal trade-off between nonequilibrium current fluctuations and dissipation, has been found for various Markovian systems. However, this relation has not been revealed for non-Markovian systems; therefore, we investigate the thermodynamic uncertainty relation for time-delayed Langevin systems. We prove that the fluctuation of arbitrary dynamical observables is constrained by the Kullback-Leibler divergence between the distributions of the forward path and its reversed counterpart. Specifically, for observables that are antisymmetric under time reversal, the fluctuation is bounded from below by a function of a quantity that can be identified as a generalization of the total entropy production in Markovian systems. We also provide a lower bound for arbitrary observables that are odd under position reversal. The term in this bound reflects the extent to which the position symmetry has been broken in the system and can be positive even in equilibrium. Our results hold for finite observation times and a large class of time-delayed systems because detailed underlying dynamics are not required for the derivation. We numerically verify the derived uncertainty relations using two single time-delay systems and one distributed time-delay system.

Visibility graph analysis of temporal irreversibility in sleep electroencephalograms

The study of sleep has continued to garner increased attention. However, most studies assume stationarity of sleep electroencephalogram (EEG) signals, whereas they are typically nonlinear and nonstationary. Little work has focused on the time irreversibility of sleep EEG signals. Hence, the aim of this work is to reveal the temporally irreversible structures of rapid-eye-movement (REM) and non-REM sleep using a visibility algorithm, which is robust to nonstationarity and finite-size effect. Results show that the temporal structure of non-REM sleep is more irreversible than that of REM sleep. The degree of irreversibility is highest in slow-wave sleep. Moreover, statistical analysis suggests that aging is the major factor that affects the irreversibility of sleep signals, while gender and body mass index contribute insignificantly. The dominant role of slow oscillations on the irreversible structures of the sleep signals is also indicated.

Non-Markovianity and negative entropy production rates

Entropy production plays a fundamental role in nonequilibrium thermodynamics to quantify the irreversibility of open systems. Its positivity can be ensured for a wide class of setups, but the entropy production rate can become negative sometimes. This is often taken as an indicator of non-Markovianity. We make this link precise by showing under which conditions a negative entropy production rate implies non-Markovianity and when it does not. For a system coupled to a single heat bath, this can be established within a unified language for two setups: (i) the dynamics resulting from a coarse-grained description of a Markovian master equation and (ii) the classical Hamiltonian dynamics of a system coupled to a bath. The quantum version of the latter result is shown not to hold despite the fact that the integrated thermodynamic description is formally equivalent to the classical case. The instantaneous fixed point of a non-Markovian dynamics plays an important role in our study. Our key contribution is to provide a consistent theoretical framework to study the finite-time thermodynamics of a large class of dynamics with a precise link to its non-Markovianity.

Information Thermodynamics for Time Series of Signal-Response Models

The entropy production in stochastic dynamical systems is linked to the structure of their causal representation in terms of Bayesian networks. Such a connection was formalized for bipartite (or multipartite) systems with an integral fluctuation theorem in [Phys. Rev. Lett. 111, 180603 (2013)]. Here we introduce the information thermodynamics for time series, that are non-bipartite in general, and we show that the link between irreversibility and information can only result from an incomplete causal representation. In particular, we consider a backward transfer entropy lower bound to the conditional time series irreversibility that is induced by the absence of feedback in signal-response models. We study such a relation in a linear signal-response model providing analytical solutions, and in a nonlinear biological model of receptor-ligand systems where the time series irreversibility measures the signaling efficiency.

Analysis of Cell Signal Transduction Based on Kullback-Leibler Divergence: Channel Capacity and Conservation of Its Production Rate during Cascade

Kullback–Leibler divergence (KLD) is a type of extended mutual entropy, which is used as a measure of information gain when transferring from a prior distribution to a posterior distribution. In this study, KLD is applied to the thermodynamic analysis of cell signal transduction cascade and serves an alternative to mutual entropy. When KLD is minimized, the divergence is given by the ratio of the prior selection probability of the signaling molecule to the posterior selection probability. Moreover, the information gain during the entire channel is shown to be adequately described by average KLD production rate. Thus, this approach provides a framework for the quantitative analysis of signal transduction. Moreover, the proposed approach can identify an effective cascade for a signaling network.

Universal bounds on current fluctuations

For current fluctuations in nonequilibrium steady states of Markovian processes, we derive four different universal bounds valid beyond the Gaussian regime. Different variants of these bounds apply to either the entropy change or any individual current, e.g., the rate of substrate consumption in a chemical reaction or the electron current in an electronic device. The bounds vary with respect to their degree of universality and tightness. A universal parabolic bound on the generating function of an arbitrary current depends solely on the average entropy production. A second, stronger bound requires knowledge both of the thermodynamic forces that drive the system and of the topology of the network of states. These two bounds are conjectures based on extensive numerics. An exponential bound that depends only on the average entropy production and the average number of transitions per time is rigorously proved. This bound has no obvious relation to the parabolic bound but it is typically tighter further away from equilibrium. An asymptotic bound that depends on the specific transition rates and becomes tight for large fluctuations is also derived. This bound allows for the prediction of the asymptotic growth of the generating function. Even though our results are restricted to networks with a finite number of states, we show that the parabolic bound is also valid for three paradigmatic examples of driven diffusive systems for which the generating function can be calculated using the additivity principle. Our bounds provide a general class of constraints for nonequilibrium systems.

Cycles, randomness, and transport from chaotic dynamics to stochastic processes

An overview of advances at the frontier between dynamical systems theory and nonequilibrium statistical mechanics is given. Sensitivity to initial conditions is a mechanism at the origin of dynamical randomness-alias temporal disorder-in deterministic dynamical systems. In spatially extended systems, sustaining transport processes, such as diffusion, relationships can be established between the characteristic quantities of dynamical chaos and the transport coefficients, bringing new insight into the second law of thermodynamics. With methods from dynamical systems theory, the microscopic time-reversal symmetry can be shown to be broken at the statistical level of description in nonequilibrium systems. In this way, the thermodynamic entropy production turns out to be related to temporal disorder and its time asymmetry away from equilibrium.

Time reversibility from visibility graphs of nonstationary processes

Visibility algorithms are a family of methods to map time series into networks, with the aim of describing the structure of time series and their underlying dynamical properties in graph-theoretical terms. Here we explore some properties of both natural and horizontal visibility graphs associated to several nonstationary processes, and we pay particular attention to their capacity to assess time irreversibility. Nonstationary signals are (infinitely) irreversible by definition (independently of whether the process is Markovian or producing entropy at a positive rate), and thus the link between entropy production and time series irreversibility has only been explored in nonequilibrium stationary states. Here we show that the visibility formalism naturally induces a new working definition of time irreversibility, which allows us to quantify several degrees of irreversibility for stationary and nonstationary series, yielding finite values that can be used to efficiently assess the presence of memory and off-equilibrium dynamics in nonstationary processes without the need to differentiate or detrend them. We provide rigorous results complemented by extensive numerical simulations on several classes of stochastic processes.

Stochastic thermodynamics of hidden pumps

We show that a reversible pumping mechanism operating between two states of a kinetic network can give rise to Poisson transitions between these two states. An external observer, for whom the pumping mechanism is not accessible, will observe a Markov chain satisfying local detailed balance with an emerging effective force induced by the hidden pump. Due to the reversibility of the pump, the actual entropy production turns out to be lower than the coarse-grained entropy production estimated from the flows and affinities of the resulting Markov chain. Moreover, in presence of a large time scale separation between the fast-pumping dynamics and the slow-network dynamics, a finite current with zero dissipation may be produced. We make use of these general results to build a synthetase-like kinetic scheme able to reversibly produce high free-energy molecules at a finite rate and a rotatory motor achieving 100% efficiency at finite speed.

Stochastic thermodynamics of Langevin systems under time-delayed feedback control: Second-law-like inequalities

Response lags are generic to almost any physical system and often play a crucial role in the feedback loops present in artificial nanodevices and biological molecular machines. In this paper, we perform a comprehensive study of small stochastic systems governed by an underdamped Langevin equation and driven out of equilibrium by a time-delayed continuous feedback control. In their normal operating regime, these systems settle in a nonequilibrium steady state in which work is permanently extracted from the surrounding heat bath. By using the Fokker-Planck representation of the dynamics, we derive a set of second-law-like inequalities that provide bounds to the rate of extracted work. These inequalities involve additional contributions characterizing the reduction of entropy production due to the continuous measurement process. We also show that the non-Markovian nature of the dynamics requires a modification of the basic relation linking dissipation to the breaking of time-reversal symmetry at the level of trajectories. The modified relation includes a contribution arising from the acausal character of the reverse process. This, in turn, leads to another second-law-like inequality. We illustrate the general formalism with a detailed analytical and numerical study of a harmonic oscillator driven by a linear feedback, which describes actual experimental setups.

Entropy production and fluctuation theorems for Langevin processes under continuous non-Markovian feedback control

Continuous feedback control of Langevin processes may be non-Markovian due to a time lag between the measurement and the control action. We show that this requires one to modify the basic relation between dissipation and time reversal and to include a contribution arising from the noncausal character of the reverse process. We then propose a new definition of the quantity measuring the irreversibility of a path in a nonequilibrium stationary state, which can also be regarded as the trajectory-dependent total entropy production. This leads to an extension of the second law, which takes a simple form in the long-time limit. As an illustration, we apply the general approach to linear systems that are both analytically tractable and experimentally relevant.

Information-theoretic versus thermodynamic entropy production in autonomous sensory networks

For sensory networks, we determine the rate with which they acquire information about the changing external conditions. Comparing this rate with the thermodynamic entropy production that quantifies the cost of maintaining the network, we find that there is no universal bound restricting the rate of obtaining information to be less than this thermodynamic cost. These results are obtained within a general bipartite model consisting of a stochastically changing environment that affects the instantaneous transition rates within the system. Moreover, they are illustrated with a simple four-states model motivated by cellular sensing. On the technical level, we obtain an upper bound on the rate of mutual information analytically and calculate this rate with a numerical method that estimates the entropy of a time series generated with a simulation.

Stochastic thermodynamics, fluctuation theorems and molecular machines

Stochastic thermodynamics as reviewed here systematically provides a framework for extending the notions of classical thermodynamics such as work, heat and entropy production to the level of individual trajectories of well-defined non-equilibrium ensembles. It applies whenever a non-equilibrium process is still coupled to one (or several) heat bath(s) of constant temperature. Paradigmatic systems are single colloidal particles in time-dependent laser traps, polymers in external flow, enzymes and molecular motors in single molecule assays, small biochemical networks and thermoelectric devices involving single electron transport. For such systems, a first-law like energy balance can be identified along fluctuating trajectories. For a basic Markovian dynamics implemented either on the continuum level with Langevin equations or on a discrete set of states as a master equation, thermodynamic consistency imposes a local-detailed balance constraint on noise and rates, respectively. Various integral and detailed fluctuation theorems, which are derived here in a unifying approach from one master theorem, constrain the probability distributions for work, heat and entropy production depending on the nature of the system and the choice of non-equilibrium conditions. For non-equilibrium steady states, particularly strong results hold like a generalized fluctuation-dissipation theorem involving entropy production. Ramifications and applications of these concepts include optimal driving between specified states in finite time, the role of measurement-based feedback processes and the relation between dissipation and irreversibility. Efficiency and, in particular, efficiency at maximum power can be discussed systematically beyond the linear response regime for two classes of molecular machines, isothermal ones such as molecular motors, and heat engines such as thermoelectric devices, using a common framework based on a cycle decomposition of entropy production.