The generalized Ornstein - Uhlenbeck process

Biped Robots Control in Gusty Environments with Adaptive Exploration Based DDPG

As technology rapidly evolves, the application of bipedal robots in various environments has widely expanded. These robots, compared to their wheeled counterparts, exhibit a greater degree of freedom and a higher complexity in control, making the challenge of maintaining balance and stability under changing wind speeds particularly intricate. Overcoming this challenge is critical as it enables bipedal robots to sustain more stable gaits during outdoor tasks, thereby increasing safety and enhancing operational efficiency in outdoor settings. To transcend the constraints of existing methodologies, this research introduces an adaptive bio-inspired exploration framework for bipedal robots facing wind disturbances, which is based on the Deep Deterministic Policy Gradient (DDPG) approach. This framework allows the robots to perceive their bodily states through wind force inputs and adaptively modify their exploration coefficients. Additionally, to address the convergence challenges posed by sparse rewards, this study incorporates Hindsight Experience Replay (HER) and a reward-reshaping strategy to provide safer and more effective training guidance for the agents. Simulation outcomes reveal that robots utilizing this advanced method can more swiftly explore behaviors that contribute to stability in complex conditions, and demonstrate improvements in training speed and walking distance over traditional DDPG algorithms.

Intermittent Kac's flights and the generalized telegrapher's equation

A generalized one-dimensional telegrapher equation associated with an intermittent change of sign in the velocity of a Kac's flight has been proposed. To solve this random differential equation, we used the enlarged master equation approach to obtain the exact differential equation for the evolution of a normalized positive distribution. This distribution is associated with a generalized finite-velocity diffusionlike process. We studied the robustness of the ballistic regime, the cutoff of its domain, and the time-dependent Gaussian convergence. The second moment for the evolution of the profile has been studied as a function of non-Poisson statistics (possibly intermittent) for the time intervals Δ_{ij} in the Kac's flight. Numerical results for the evolution of sharp and wide initial profiles have also been presented. In addition, for comparison with a non-Gaussian process at all times, we have revisited the non-Markov Poisson's flight with exponential pulses. A theory for generalized random flights with intermittent stochastic velocity and in the presence of a force is also presented, and the stationary distribution for two classes of potential has been obtained.

Ornstein-Uhlenbeck process and generalizations: Particle dynamics under comb constraints and stochastic resetting

The Ornstein-Uhlenbeck process is interpreted as Brownian motion in a harmonic potential. This Gaussian Markov process has a bounded variance and admits a stationary probability distribution, in contrast to the standard Brownian motion. It also tends to a drift towards its mean function, and such a process is called mean reverting. Two examples of the generalized Ornstein-Uhlenbeck process are considered. In the first one, we study the Ornstein-Uhlenbeck process on a comb model, as an example of the harmonically bounded random motion in the topologically constrained geometry. The main dynamical characteristics (as the first and the second moments) and the probability density function are studied in the framework of both the Langevin stochastic equation and the Fokker-Planck equation. The second example is devoted to the study of the effects of stochastic resetting on the Ornstein-Uhlenbeck process, including stochastic resetting in the comb geometry. Here the nonequilibrium stationary state is the main question in task, where the two divergent forces, namely, the resetting and the drift towards the mean, lead to compelling results in the cases of both the Ornstein-Uhlenbeck process with resetting and its generalization on the two-dimensional comb structure.

Fokker-Planck representations of non-Markov Langevin equations: application to delayed systems

Noise and time delays, or history-dependent processes, play an integral part in many natural and man-made systems. The resulting interplay between random fluctuations and time non-locality are essential features of the emerging complex dynamics in non-Markov systems. While stochastic differential equations in the form of Langevin equations with additive noise for such systems exist, the corresponding probabilistic formalism is yet to be developed. Here we introduce such a framework via an infinite hierarchy of coupled Fokker-Planck equations for the n-time probability distribution. When the non-Markov Langevin equation is linear, we show how the hierarchy can be truncated at n = 2 by converting the time non-local Langevin equation to a time-local one with additive coloured noise. We compare the resulting Fokker-Planck equations to an earlier version, solve them analytically and analyse the temporal features of the probability distributions that would allow to distinguish between Markov and non-Markov features. This article is part of the theme issue 'Nonlinear dynamics of delay systems'.

Weak Galilean invariance as a selection principle for coarse-grained diffusive models

How does the mathematical description of a system change in different reference frames? Galilei first addressed this fundamental question by formulating the famous principle of Galilean invariance. It prescribes that the equations of motion of closed systems remain the same in different inertial frames related by Galilean transformations, thus imposing strong constraints on the dynamical rules. However, real world systems are often described by coarse-grained models integrating complex internal and external interactions indistinguishably as friction and stochastic forces. Since Galilean invariance is then violated, there is seemingly no alternative principle to assess a priori the physical consistency of a given stochastic model in different inertial frames. Here, starting from the Kac-Zwanzig Hamiltonian model generating Brownian motion, we show how Galilean invariance is broken during the coarse-graining procedure when deriving stochastic equations. Our analysis leads to a set of rules characterizing systems in different inertial frames that have to be satisfied by general stochastic models, which we call "weak Galilean invariance." Several well-known stochastic processes are invariant in these terms, except the continuous-time random walk for which we derive the correct invariant description. Our results are particularly relevant for the modeling of biological systems, as they provide a theoretical principle to select physically consistent stochastic models before a validation against experimental data.

Superexponential fluctuation relation for dichotomous work reservoir systems

This paper introduces an analytical description of the probability density function of the dissipated and injected powers p(j_{dis}) and p(j_{inj}), respectively, in a paradigmatic nonequilibrium damped system in contact with a work reservoir that is analytically represented by telegraph noise and to which one can assign an effective temperature. This approach is able to overcome the well-known impossibility of obtaining closed solutions to steady-state distributions of this system and allows determining a superexponential fluctuation relation of the injected power, which is not even asymptotically exponential as for (shot-noise) Poissonian reservoirs. In the white-noise limit, that relation converges to the exponential formula that is standard in thermal systems; however, the distribution of the injected power remains quite different from that of the latter instance. Surprisingly, it is actually shown that a Gaussian distribution, which is archetypal of thermal systems, for the injected power can be achievable only for athermal reservoirs of this kind.

Active cage model of glassy dynamics

We build up a phenomenological picture in terms of the effective dynamics of a tracer confined in a cage experiencing random hops to capture some characteristics of glassy systems. This minimal description exhibits scale invariance properties for the small-displacement distribution that echo experimental observations. We predict the existence of exponential tails as a crossover between two Gaussian regimes. Moreover, we demonstrate that the onset of glassy behavior is controlled only by two dimensionless numbers: the number of hops occurring during the relaxation of the particle within a local cage and the ratio of the hopping length to the cage size.

Thermostatistics of a damped bimodal particle

We study the thermostatistics of a damped bimodal particle, i.e., a particle of mass m subject to a work reservoir that is analytically represented by the telegraph noise. Because of the colored nature of the noise, it does not fit the Lévy-Itô class of stochastic processes, making this system an instance of a nonequilibrium system in contact with a non-Gaussian external reservoir. We obtain the statistical description of the position and velocity, namely in the stationary state, as well as the (time-dependent) statistics of the energy fluxes in the system considering no constraints on the telegraph noise features. With that result we are able to give an account of the statistical properties of the large deviations of the injected and dissipated power that can change from sub-Gaussianity to super-Gaussianity depending on the color of the noise. By properly defining an effective temperature for this system, T, we are capable of obtaining an equivalent entropy production-exchange rate equal to the ratio between the dissipation of the medium, γ, and the mass of the particle, m, a relation that concurs with the case of a standard thermal reservoir at temperature, T=T.

Anomalous fluctuation properties

We complement and extend our work on fluctuations arising in nonequilibrium systems in steady states driven by Lévy noise [H. Touchette and E. G. D. Cohen, Phys. Rev. E 76, 020101(R) (2007)]. As a concrete example, we consider a particle subjected to a drag force and a Lévy white noise with tail index mu epsilon (0,2), and calculate the probability distribution of the work done on the particle by the drag force, as well as the probability distribution of the work dissipated by the dragged particle in a nonequilibrium steady state. For 0<mu<2, both distributions satisfy what we call an anomalous fluctuation property characterized by positive and negative fluctuations that asymptotically have the same probability. For mu=2, by contrast, the work and dissipated work distributions satisfy the known conventional and extended fluctuation relations, respectively, which are both characterized by positive fluctuations that are exponentially more probable than negative fluctuations. The difference between these different fluctuation behaviors is discussed in the context of large deviation theory. Experiments that could probe or reveal an anomalous fluctuation property are also discussed.

Steady-state work fluctuations of a dragged particle under external and thermal noise

We consider a particle, confined to a moving harmonic potential, under the influence of friction and external asymmetric Poissonian shot noise (PSN). We study the fluctuations of the work done to maintain this system in a nonequilibrium steady state. PSN generalizes the usual Gaussian noise and can be considered to be a paradigm of external noise, where fluctuation and dissipation originate from physically independent mechanisms. We consider two scenarios: (i) the noise is given purely by PSN and (ii) in addition to PSN the particle is subject to white Gaussian noise. In both cases we derive exact expressions for the large deviation form of the work distribution, which are characterized by the time scales of the system. We show that the usual steady-state fluctuation theorem does not apply in our model and that in a certain parameter regime large negative work fluctuations are more likely to occur than the corresponding positive ones, though the average work is always positive.

Fluctuation properties of an effective nonlinear system subject to Poisson noise

We study the work fluctuations of a particle, confined to a moving harmonic potential, under the influence of friction and external asymmetric Poissonian shot noise. This type of noise generalizes the usual Gaussian noise and induces an effective interaction between the noise and the potential, leading to an effectively nonlinear system with singular features. On the basis of an analytic solution we investigate the roles of time scales, symmetries, and singularities in the context of nonequilibrium fluctuations. Our results highlight the nonuniversality of the steady-state fluctuation theorem in stochastic systems.

Method to calculate the moments of the membrane voltage in a model neuron driven by multiplicative filtered shot noise

Neurons are subject to synaptic inputs from many other cells. These inputs consist of spikes changing the conductivity of the target cell, i.e., they enter the neural dynamics as multiplicative shot noise. Up to now, only for simplified models like current-based (additive-noise) point neurons or models with Gaussian white-noise input, exact solutions are available. We present a method to calculate the exact time-dependent moments for the voltage of a point neuron with conductance-based shot noise and a passive membrane. The exact solutions show features (for instance, maxima of the moments vs time) which are also confirmed by numerical simulations. The theoretical analysis of subthreshold membrane fluctuations may contribute to a better comprehension of neural noise in general. We also discuss how the analytical results may provide additional conditions for estimating parameters from experimental data.

Fluctuation relation for a Lévy particle

We study the work fluctuations of a particle subjected to a deterministic drag force plus a random forcing whose statistics is of the Lévy type. In the stationary regime, the probability density of the work is found to have "fat" power-law tails which assign a relatively high probability to large fluctuations compared with the case where the random forcing is Gaussian. These tails lead to a strong violation of existing fluctuation theorems, as the ratio of the probabilities of positive and negative work fluctuations of equal magnitude behaves in a nonmonotonic way. Possible experiments that could probe these features are proposed.

Functional characterization of linear delay Langevin equations

We present an exact functional characterization of linear delay Langevin equations driven by any noise structure defined through its characteristic functional. This method relies on the possibility of finding an explicitly analytical expression for each realization of the delayed stochastic process in terms of those of the driving noise. General properties of the transient dissipative dynamics are analyzed. The corresponding interplay with a color Gaussian noise is presented. As a full application of our functional method we study a model for population growth with non-Gaussian fluctuations: the Gompertz model driven by multiplicative white shot noise.

Computing a non-Maxwellian velocity distribution from first principles

We investigate a family of single-particle anomalous velocity distribution by solving a particular class of stochastic Liouville equations. The stationary state is obtained analytically and the Maxwell-Boltzmann distribution is reobtained in a particular limit. We discuss the comparison with other different methods to obtain the stationary state. Extensions when the models cannot be solved in an exact way are also pointed out in connection with the one-ficton approximation.

Harmonic potential driven by long-range correlated noise

The probability distribution of a particle in a quadratic potential driven by Gaussian long-range correlated noise has been obtained. The long-time asymptotic relaxation of the stochastic process has been characterized in terms of the long-range correlated noise appearing in the corresponding stochastic differential equation defining the process. The particular case when the particle is driven by a Gaussian color noise has also been revisited, in this way giving its exact probability distribution for all time. By using a characteristic functional technique reported previously, results for a non-Gaussian long-range correlated noise are also shown. Emphasis has been placed on solving the plane rotator in the presence of arbitrary random torques having long-range correlations. In order to show strong and weak non-Markovian effects coming from different sources of noise, the undamped free particle under the influence of arbitrary accelerations has been analyzed. We also analyze in detail the structure of the trajectories of the overdamped and undamped free particle.