Analytical solution of the off-equilibrium dynamics of a long-range spin-glass model

Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective

Recent years witnessed the development of powerful generative models based on flows, diffusion, or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical analysis of the performance and understanding of the limitations of these methods remain, however, challenging. In this paper, we undertake a step in this direction by analyzing the efficiency of sampling by these methods on a class of problems with a known probability distribution and comparing it with the sampling performance of more traditional methods such as the Monte Carlo Markov chain and Langevin dynamics. We focus on a class of probability distribution widely studied in the statistical physics of disordered systems that relate to spin glasses, statistical inference, and constraint satisfaction problems. We leverage the fact that sampling via flow-based, diffusion-based, or autoregressive networks methods can be equivalently mapped to the analysis of a Bayes optimal denoising of a modified probability measure. Our findings demonstrate that these methods encounter difficulties in sampling stemming from the presence of a first-order phase transition along the algorithm's denoising path. Our conclusions go both ways: We identify regions of parameters where these methods are unable to sample efficiently, while that is possible using standard Monte Carlo or Langevin approaches. We also identify regions where the opposite happens: standard approaches are inefficient while the discussed generative methods work well.

Real-space model for activated processes in rejuvenation and memory behavior of glassy systems

We offer an alternative real-space description, based purely on activated processes, for the understanding of relaxation dynamics in hierarchical landscapes. To this end, we use the cluster model, a coarse-grained lattice model of a jammed system, to analyze rejuvenation and memory effects during aging after a hard quench. In this model, neighboring particles on a lattice aggregate through local interactions into clusters that fragment with a probability based on their size. Despite the simplicity of the cluster model, it has been shown to reproduce salient observables of the aging dynamics in colloidal systems, such as those accounting for particle mobility and displacements. Here, we probe the model for more complex quench protocols and show that it exhibits rejuvenation and memory effects similar to those attributed to the complex hierarchical structure of a glassy energy landscape.

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

Is Stochastic Gradient Descent (SGD) substantially different from Metropolis Monte Carlo dynamics? This is a fundamental question at the time of understanding the most used training algorithm in the field of Machine Learning, but it received no answer until now. Here we show that in discrete optimization and inference problems, the dynamics of an SGD-like algorithm resemble very closely that of Metropolis Monte Carlo with a properly chosen temperature, which depends on the mini-batch size. This quantitative matching holds both at equilibrium and in the out-of-equilibrium regime, despite the two algorithms having fundamental differences (e.g. SGD does not satisfy detailed balance). Such equivalence allows us to use results about performances and limits of Monte Carlo algorithms to optimize the mini-batch size in the SGD-like algorithm and make it efficient at recovering the signal in hard inference problems.

The nature of non-phononic excitations in disordered systems

The frequency scaling exponent of low-frequency excitations in microscopically small glasses, which do not allow for the existence of waves (phonons), has been in the focus of the recent literature. The density of states g(ω) of these modes obeys an ωs scaling, where the exponent s, ranging between 2 and 5, depends on the quenching protocol. The orgin of these findings remains controversal. Here we show, using heterogeneous-elasticity theory, that in a marginally-stable glass sample g(ω) follows a Debye-like scaling (s = 2), and the associated excitations (type-I) are of random-matrix type. Further, using a generalisation of the theory, we demonstrate that in more stable samples, other, (type-II) excitations prevail, which are non-irrotational oscillations, associated with local frozen-in stresses. The corresponding frequency scaling exponent s is governed by the statistics of small values of the stresses and, therefore, depends on the details of the interaction potential.

Instantaneous normal modes of glass-forming liquids during the athermal relaxation process of the steepest descent algorithm

Understanding glass formation by quenching remains a challenge in soft condensed matter physics. Recent numerical studies on steepest descent dynamics, which is one of the simplest models of quenching, revealed that quenched liquids undergo slow relaxation with a power law towards mechanical equilibrium and that the late stage of this process is governed by local rearrangements of particles. These advances motivate the detailed study of instantaneous normal modes during the relaxation process because the glassy dynamics is considered to be governed by stationary points of the potential energy landscape. Here, we performed a normal mode analysis of configurations during the steepest descent dynamics and found that the dynamics is driven by almost flat directions of the potential energy landscape at long times. These directions correspond to localized modes and we characterized them in terms of their statistics and structure using methods developed in the study of local minima of the potential energy landscape.

A Computationally Informed Unified View on the Effect of Polarity and Sterics on the Glass Transition in Vinyl-based Polymer Melts

We unveil a unified view on the effect of side chains on the glass transition temperatures (Tg) in polymer melts by using molecular dynamics simulations, density functional theory calculations, and available experimental data. We use acrylates as a model system and evaluate the effect of n-alkyl side chains on Tg. We find that backbone dihedral angle fluctuations follow established patterns due to sterics, as expected. However, we also find that the dihedral angle orthogonal to the backbone, which normally is neglected when discussing the effect on Tg, introduces a secondary rotational degree of freedom which strongly impacts Tg. These results are in agreement with experiments and generalize to multiple other polymer systems, as is demonstrated using available experimental data. Conversely, n-alkyl pendant groups attached to the side group reduce Tg. Our work establishes a coherent framework that unifies previously established trends, emphasizing the polarity and size effects of n-alkyl chains on Tg.

Jamming, relaxation, and memory in a minimally structured glass former

Structural glasses form through various out-of-equilibrium processes, including temperature quenches, rapid compression (crunches), and shear. Although each of these processes should be formally understandable within the recently formulated dynamical mean-field theory (DMFT) of glasses, the numerical tools needed to solve the DMFT equations up to the relevant physical regime do not yet exist. In this context, numerical simulations of minimally structured (and therefore mean-field-like) model glass formers can aid the search for and understanding of such solutions, thanks to their ability to disentangle structural from dimensional effects. We study here the infinite-range Mari-Kurchan model under simple out-of-equilibrium processes, and we compare results with the random Lorentz gas [J. Phys. A 55, 334001 (2022)10.1088/1751-8121/ac7f06]. Because both models are mean-field-like and formally equivalent in the limit of infinite spatial dimensions, robust features are expected to appear in the DMFT as well. The comparison provides insight into temperature and density onsets, memory, as well as anomalous relaxation. This work also further enriches the algorithmic understanding of the jamming density.

Fluctuating landscapes and heavy tails in animal behavior

Animal behavior is shaped by a myriad of mechanisms acting on a wide range of scales. This immense variability hampers quantitative reasoning and renders the identification of universal principles elusive. Through data analysis and theory, we here show that slow non-ergodic drives generally give rise to heavy-tailed statistics in behaving animals. We leverage high-resolution recordings of C. elegans locomotion to extract a self-consistent reduced order model for an inferred reaction coordinate, bridging from sub-second chaotic dynamics to long-lived stochastic transitions among metastable states. The slow mode dynamics exhibits heavy-tailed first passage time distributions and correlation functions, and we show that such heavy tails can be explained by dynamics on a time-dependent potential landscape. Inspired by these results, we introduce a generic model in which we separate faster mixing modes that evolve on a quasi-stationary potential, from slower non-ergodic modes that drive the potential landscape, and reflect slowly varying internal states. We show that, even for simple potential landscapes, heavy tails emerge when barrier heights fluctuate slowly and strongly enough. In particular, the distribution of first passage times and the correlation function can asymptote to a power law, with related exponents that depend on the strength and nature of the fluctuations. We support our theoretical findings through direct numerical simulations.

Shear-induced criticality in glasses shares qualitative similarities with the Gardner phase

Although glass phases are ubiquitously found in various soft matter systems, we are still far from a complete understanding of them. The concept of marginal stability predicted by infinite-dimensional mean-field theories is drawing attention as a candidate for a universal and distinguishing unique feature of glasses. While among theoretical predictions, the non-Debye scaling has indeed been observed universally over various classes of glasses, the Gardner phase is found only in a limited portion of them. In this work, we numerically demonstrate that plastic events observed in two-dimensional Lennard-Jones glasses under quasistatic shear exhibit statistical properties that are qualitatively consistent with the picture of an infinitely hierarchical energy landscape associated with the Gardner phase.

Dynamic Gardner cross-over in a simple glass

The criticality of the jamming transition responsible for amorphous solidification has been theoretically linked to the marginal stability of a thermodynamic Gardner phase. While the critical exponents of jamming appear independent of the preparation history, the pertinence of Gardner physics far from equilibrium is an open question. To fill this gap, we numerically study the nonequilibrium dynamics of hard disks compressed toward the jamming transition using a broad variety of protocols. We show that dynamic signatures of Gardner physics can be disentangled from the aging relaxation dynamics. We thus define a generic dynamic Gardner cross-over regardless of the history. Our results show that the jamming transition is always accessed by exploring increasingly complex landscape, resulting in anomalous microscopic relaxation dynamics that remains to be understood theoretically.

How to count in hierarchical landscapes: A full solution to mean-field complexity

We derive the general solution for counting the stationary points of mean-field complex landscapes. It incorporates Parisi's solution for the ground state, as it should. Using this solution, we count the stationary points of two models: one with multistep replica symmetry breaking and one with full replica symmetry breaking.

Data-Driven Identification and Analysis of the Glass Transition in Polymer Melts

Understanding the nature of glass transition, as well as the precise estimation of the glass transition temperature for polymeric materials, remains open questions in both experimental and theoretical polymer sciences. We propose a data-driven approach, which utilizes the high-resolution details accessible through the molecular dynamics simulation and considers the structural information on individual chains. It clearly identifies the glass transition temperature of polymer melts of weakly semiflexible chains. By combining principal component analysis and clustering, we identify the glass transition temperature in the asymptotic limit even from relatively short time trajectories, which just reach into the Rouse-like monomer displacement regime. We demonstrate that fluctuations captured by the principal component analysis reflect the change in a chain's behavior: from conformational rearrangement above to small fluctuations below the glass transition temperature. Our approach is straightforward to apply and should be applicable to other polymeric glass-forming liquids.

Aging by Near-Extinctions in Many-Variable Interacting Populations

Models of many-species ecosystems, such as the Lotka-Volterra and replicator equations, suggest that these systems generically exhibit near-extinction processes, where population sizes go very close to zero for some time before rebounding, accompanied by a slowdown of the dynamics (aging). Here, we investigate the connection between near-extinction and aging by introducing an exactly solvable many-variable model, where the time derivative of each population size vanishes at both zero and some finite maximal size. We show that aging emerges generically when random interactions are taken between populations. Population sizes remain exponentially close (in time) to the absorbing values for extended periods of time, with rapid transitions between these two values. The mechanism for aging is different from the one at play in usual glassy systems: At long times, the system evolves in the vicinity of unstable fixed points rather than marginal ones.

Structural relaxation, dynamical arrest, and aging in soft-sphere liquids

We investigate the structural relaxation of a soft-sphere liquid quenched isochorically (ϕ = 0.7) and instantaneously to different temperatures T_f above and below the glass transition. For this, we combine extensive Brownian dynamics simulations and theoretical calculations based on the non-equilibrium self-consistent generalized Langevin equation (NE-SCGLE) theory. The response of the liquid to a quench generally consists of a sub-linear increase of the α-relaxation time with system's age. Approaching the ideal glass-transition temperature from above (T_f > T^a), sub-aging appears as a transient process describing a broad equilibration crossover for quenches to nearly arrested states. This allows us to empirically determine an equilibration timescale t^eq(T_f) that becomes increasingly longer as T_f approaches T^a. For quenches inside the glass (T_f ≤ T^a), the growth rate of the structural relaxation time becomes progressively larger as T_f decreases and, unlike the equilibration scenario, τ_α remains evolving within the whole observation time-window. These features are consistently found in theory and simulations with remarkable semi-quantitative agreement and coincide with those revealed in a previous and complementary study [P. Mendoza-Méndez et al., Phys. Rev. 96, 022608 (2017)] that considered a sequence of quenches with fixed final temperature T_f = 0 but increasing ϕ toward the hard-sphere dynamical arrest volume fraction ϕ_HS ^a=0.582. The NE-SCGLE analysis, however, unveils various fundamental aspects of the glass transition, involving the abrupt passage from the ordinary equilibration scenario to the persistent aging effects that are characteristic of glass-forming liquids. The theory also explains that, within the time window of any experimental observation, this can only be observed as a continuous crossover.

From Subaging to Hyperaging in Structural Glasses

We demonstrate nonequilibrium scaling laws for the aging and equilibration dynamics in glass formers that emerge from combining a relaxation equation for the static structure with the equilibrium scaling laws of glassy dynamics. Different scaling regimes are predicted for the evolution of the structural relaxation time τ with age (waiting time t_{w}), depending on the depth of the quench from the liquid into the glass: "simple" aging (τ∼t_{w}) applies for quenches close to the critical point of mode-coupling theory (MCT) and implies "subaging" (τ≈t_{w}^{δ} with δ<1) as a broad equilibration crossover for quenches to nearly arrested equilibrium states; "hyperaging" (or superaging, τ∼t_{w}^{δ^{'}} with δ^{'}>1) emerges for quenches deep into the glass. The latter is cut off by non-mean-field fluctuations that we account for within a recent extension of MCT, the stochastic β-relaxation theory (SBR). We exemplify the scaling laws with a schematic model that quantitatively fits simulation data.

Equivalence of Fluctuation-Dissipation and Edwards' Temperature in Cyclically Sheared Granular Systems

Using particle trajectory data obtained from x-ray tomography, we determine two kinds of effective temperatures in a cyclically sheared granular system. The first one is obtained from the fluctuation-dissipation theorem which relates the diffusion and mobility of lighter tracer particles immersed in the system. The second is the Edwards compactivity defined via the packing volume fluctuations. We find robust agreement between these two temperatures, independent of the type of the tracers, cyclic shear amplitudes, and particle surface roughness, giving therefore the first experimental evidence that the concept of effective temperature is valid in driven frictional granular systems.

An analytical theory of curriculum learning in teacher-student networks

In animals and humans, curriculum learning-presenting data in a curated order-is critical to rapid learning and effective pedagogy. A long history of experiments has demonstrated the impact of curricula in a variety of animals but, despite its ubiquitous presence, a theoretical understanding of the phenomenon is still lacking. Surprisingly, in contrast to animal learning, curricula strategies are not widely used in machine learning and recent simulation studies reach the conclusion that curricula are moderately effective or even ineffective in most cases. This stark difference in the importance of curriculum raises a fundamental theoretical question: when and why does curriculum learning help? In this work, we analyse a prototypical neural network model of curriculum learning in the high-dimensional limit, employing statistical physics methods. We study a task in which a sparse set of informative features are embedded amidst a large set of noisy features. We analytically derive average learning trajectories for simple neural networks on this task, which establish a clear speed benefit for curriculum learning in the online setting. However, when training experiences can be stored and replayed (for instance, during sleep), the advantage of curriculum in standard neural networks disappears, in line with observations from the deep learning literature. Inspired by synaptic consolidation techniques developed to combat catastrophic forgetting, we propose curriculum-aware algorithms that consolidate synapses at curriculum change points and investigate whether this can boost the benefits of curricula. We derive generalisation performance as a function of consolidation strength (implemented as an L 2 regularisation/elastic coupling connecting learning phases), and show that curriculum-aware algorithms can yield a large improvement in test performance. Our reduced analytical descriptions help reconcile apparently conflicting empirical results, trace regimes where curriculum learning yields the largest gains, and provide experimentally-accessible predictions for the impact of task parameters on curriculum benefits. More broadly, our results suggest that fully exploiting a curriculum may require explicit adjustments in the loss.

Theory and experiments for disordered elastic manifolds, depinning, avalanches, and sandpiles

Domain walls in magnets, vortex lattices in superconductors, contact lines at depinning, and many other systems can be modeled as an elastic system subject to quenched disorder. The ensuing field theory possesses a well-controlled perturbative expansion around its upper critical dimension. Contrary to standard field theory, the renormalization group (RG) flow involves a function, the disorder correlator Δ(w), and is therefore termed the functional RG. Δ(w) is a physical observable, the auto-correlation function of the center of mass of the elastic manifold. In this review, we give a pedagogical introduction into its phenomenology and techniques. This allows us to treat both equilibrium (statics), and depinning (dynamics). Building on these techniques, avalanche observables are accessible: distributions of size, duration, and velocity, as well as the spatial and temporal shape. Various equivalences between disordered elastic manifolds, and sandpile models exist: an elastic string driven at a point and the Oslo model; disordered elastic manifolds and Manna sandpiles; charge density waves and Abelian sandpiles or loop-erased random walks. Each of the mappings between these systems requires specific techniques, which we develop, including modeling of discrete stochastic systems via coarse-grained stochastic equations of motion, super-symmetry techniques, and cellular automata. Stronger than quadratic nearest-neighbor interactions lead to directed percolation, and non-linear surface growth with additional Kardar-Parisi-Zhang (KPZ) terms. On the other hand, KPZ without disorder can be mapped back to disordered elastic manifolds, either on the directed polymer for its steady state, or a single particle for its decay. Other topics covered are the relation between functional RG and replica symmetry breaking, and random-field magnets. Emphasis is given to numerical and experimental tests of the theory.

Competition between energy- and entropy-driven activation in glasses

In simplified models of glasses we clarify the existence of two different kinds of coexisting activated dynamics, with one of the two dominating over the other. One is the energy barrier hopping that is typically used to understand activation, and the other, which we call entropic activation, is driven by the scarcity of convenient directions in phase space. When entropic activation dominates, the height of the energy barriers is no longer the primary factor governing the system's slowdown. In our analysis, dominance of one mechanism over the other depends on temperature and the shape of the density of states. We also find that at low temperatures a phase transition between the two kinds of activation can occur. Our observations are used to provide a scenario that can harmonize the facilitation and thermodynamic pictures of the slowdown of glasses into a single description.

Molecular signatures of the glass transition in polymers

The glass transition temperature (T_{g}) is one of the most fundamental properties of polymers. T_{g} is predicted by some theories as a sudden change in a "macroscopic" quantity (e.g., compressibility). However, for systems with "soft" glass transitions where the change is gradual it becomes hard to pinpoint precisely the transition temperature as well as the set of molecular changes occurring during this transition. Here, we introduce two new molecular signatures for the glass transition of polymers that exhibit clear changes as one approaches T_{g}: (i) differential change of the probability distribution of dihedral angles as a function of temperature and (ii) the distribution of fractional of the time spent in the different torsional states. These new signatures provide insights into the glass transition in polymers by directly exhibiting the concept of spatial heterogeneity and dynamical ergodicity breaking in such systems, as well as provide a key step to quantitatively obtain the transition temperature from molecular characteristics of the polymeric systems.

"Inner clocks" of glass-forming liquids

Providing a physically sound explanation of aging phenomena in non-equilibrium amorphous materialsis a challenging problem in modern statistical thermodynamics. The slow evolution of physical propertiesafter quenches of control parameters is empirically well interpreted via the concept of material time (orinternal clock), based on the Tool-Narayanaswamy-Moynihan (TNM) model. Yet, the fundamental reasonsof its striking success remain unclear. We propose a microscopic rationale behind the material time onthe basis of the linear laws of irreversible thermodynamics and its extension that treats the correspondingkinetic coefficients as state functions of a slowly evolving material state. Our interpretation is based onthe recognition that the same mathematical structure governs both the Tool model and the recently devel-oped non-equilibrium extension of the self-consistent generalized Langevin equation theory (NE-SCGLE),guided by the universal principles of Onsager's theory of irreversible processes. This identification opensthe way for a generalization of the material-time concept to aging systems where several relaxation modeswith very different equilibration processes must be considered, and partially frozen glasses manifest theappearance of partial ergodicity breaking, and hence materials with multiple very distinct inner clocks.

Accelerating, to some extent, the p-spin dynamics

We consider a detailed-balance-violating dynamics whose stationary state is a prescribed Boltzmann distribution. Such dynamics have been shown to be faster than any equilibrium counterpart. We quantify the gain in convergence speed for a system whose energy landscape displays one and then an infinite number of energy barriers. In the latter case, we work with the mean-field disordered p spin and show that the convergence to equilibrium or to the nonergodic phase is accelerated during both the β- and α-relaxation stages. An interpretation in terms of trajectories in phase space and of an accidental fluctuation-dissipation theorem is provided.

Free-energy barriers in the Sherrington-Kirkpatrick model

The free-energy landscape of the Sherrington-Kirkpatrick (SK) Ising spin glass is simple in the framework of the Thouless-Anderson-Palmer (TAP) equations as each solution (which are minima of the free energy) has associated with it a nearby index-one saddle point. The free-energy barrier to escape the minimum is just the difference between the saddle point free energy and that at its associated minimum. This difference is calculated for the states with free energies f>f_{c}. It is very small for these states, decreasing as 1/N^{2}, where N is the number of spins in the system. These states are not marginally stable. We argue that such small barriers are why numerical studies never find these states when N is large. Instead, the states that are found are those that have marginal stability. For them the barriers are at least of O(1). f_{c} is the free energy per spin below which the states develop broken replica-symmetry-like overlaps with each other. In the regime f<f_{c} we can only offer some possibilities based around scaling arguments. One of these suggest that the barriers might become as large as N^{1/3}. That might be consistent with recent numerical studies on the Viana-Bray model, which were at variance with the expectations of Cugliandolo and Kurchan for the SK model.

Strong nonexponential relaxation and memory effects in a fluid with nonlinear drag

We analyze the dynamical evolution of a fluid with nonlinear drag, for which binary collisions are elastic, described at the kinetic level by the Enskog-Fokker-Planck equation. This model system, rooted in the theory of nonlinear Brownian motion, displays a really complex behavior when quenched to low temperatures. Its glassy response is controlled by a long-lived nonequilibrium state, independent of the degree of nonlinearity and also of the Brownian-Brownian collisions rate. The latter property entails that this behavior persists in the collisionless case, where the fluid is described by the nonlinear Fokker-Planck equation. The observed response, which includes nonexponential, algebraic, relaxation, and strong memory effects, presents scaling properties: the time evolution of the temperature-for both relaxation and memory effects-falls onto a master curve, regardless of the details of the experiment. To account for the observed behavior in simulations, it is necessary to develop an extended Sonine approximation for the kinetic equation-which considers not only the fourth cumulant but also the sixth one.

Observation of universal ageing dynamics in antibiotic persistence

Stress responses allow cells to adapt to changes in external conditions by activating specific pathways¹. Here we investigate the dynamics of single cells that were subjected to acute stress that is too strong for a regulated response but not lethal. We show that when the growth of bacteria is arrested by acute transient exposure to strong inhibitors, the statistics of their regrowth dynamics can be predicted by a model for the cellular network that ignores most of the details of the underlying molecular interactions. We observed that the same stress, applied either abruptly or gradually, can lead to totally different recovery dynamics. By measuring the regrowth dynamics after stress exposure on thousands of cells, we show that the model can predict the outcome of antibiotic persistence measurements. Our results may account for the ubiquitous antibiotic persistence phenotype², as well as for the difficulty in attempts to link it to specific genes³. More generally, our approach suggests that two different cellular states can be observed under stress: a regulated state, which prepares cells for fast recovery, and a disrupted cellular state due to acute stress, with slow and heterogeneous recovery dynamics. The disrupted state may be described by general properties of large random networks rather than by specific pathway activation. Better understanding of the disrupted state could shed new light on the survival and evolution of cells under stress.

Sherrington-Kirkpatrick model for spin glasses: Solution via the distributional zeta function method

We discuss the Sherrington-Kirkpatrick mean-field version of a spin glass within the distributional zeta function method (DZFM). In the DZFM, since the dominant contribution to the average free energy is written as a series of moments of the partition function of the model, the spin-glass multivalley structure is obtained. Also, an exact expression for the saddle points corresponding to each valley and a global critical temperature showing the existence of many stables or at least metastable equilibrium states is presented. Near the critical point, we obtain analytical expressions of the order parameters that are in agreement with phenomenological results. We evaluate the linear and nonlinear susceptibility and we find the expected singular behavior at the spin-glass critical temperature. Furthermore, we obtain a positive definite expression for the entropy and we show that ground-state entropy tends to zero as the temperature goes to zero. We show that our solution is stable for each term in the expansion. Finally, we analyze the behavior of the overlap distribution, where we find a general expression for each moment of the partition function.

Nonequilibrium Phase Transition to Anomalous Diffusion and Transport in a Basic Model of Nonlinear Brownian Motion

We investigate a basic model of nonlinear Brownian motion in a thermal environment, where nonlinear friction interpolates between viscous Stokes and dry Coulomb friction. We show that superdiffusion and supertransport emerge as a nonequilibrium critical phenomenon when such a Brownian motion is driven out of thermal equilibrium by a constant force. Precisely at the edge of a phase transition, velocity fluctuations diverge asymptotically and diffusion becomes superballistic. The autocorrelation function of velocity fluctuations in this nonergodic regime exhibits a striking aging behavior.

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

In this paper we investigate how gradient-based algorithms such as gradient descent (GD), (multi-pass) stochastic GD, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best generalization error at limited sample complexity. We consider the loss landscape of the high-dimensional phase retrieval problem as a prototypical highly non-convex example. We observe that for phase retrieval the stochastic variants of GD are able to reach perfect generalization for regions of control parameters where the GD algorithm is not. We apply dynamical mean-field theory from statistical physics to characterize analytically the full trajectories of these algorithms in their continuous-time limit, with a warm start, and for large system sizes. We further unveil several intriguing properties of the landscape and the algorithms such as that the GD can obtain better generalization properties from less informed initializations.

Counting equilibria of large complex systems by instability index

We consider a nonlinear autonomous system of [Formula: see text] degrees of freedom randomly coupled by both relaxational ("gradient") and nonrelaxational ("solenoidal") random interactions. We show that with increased interaction strength, such systems generically undergo an abrupt transition from a trivial phase portrait with a single stable equilibrium into a topologically nontrivial regime of "absolute instability" where equilibria are on average exponentially abundant, but typically, all of them are unstable, unless the dynamics is purely gradient. When interactions increase even further, the stable equilibria eventually become on average exponentially abundant unless the interaction is purely solenoidal. We further calculate the mean proportion of equilibria that have a fixed fraction of unstable directions.

Diffusions interacting through a random matrix: universality via stochastic Taylor expansion

Consider $$(X_{i}(t))$$ ( X i ( t ) ) solving a system of N stochastic differential equations interacting through a random matrix $${\mathbf {J}} = (J_{ij})$$ J = ( J ij ) with independent (not necessarily identically distributed) random coefficients. We show that the trajectories of averaged observables of $$(X_i(t))$$ ( X i ( t ) ) , initialized from some $$\mu $$ μ independent of $${\mathbf {J}}$$ J , are universal, i.e., only depend on the choice of the distribution $$\mathbf {J}$$ J through its first and second moments (assuming e.g., sub-exponential tails). We take a general combinatorial approach to proving universality for dynamical systems with random coefficients, combining a stochastic Taylor expansion with a moment matching-type argument. Concrete settings for which our results imply universality include aging in the spherical SK spin glass, and Langevin dynamics and gradient flows for symmetric and asymmetric Hopfield networks.

Properties of Equilibria and Glassy Phases of the Random Lotka-Volterra Model with Demographic Noise

We study a reference model in theoretical ecology, the disordered Lotka-Volterra model for ecological communities, in the presence of finite demographic noise. Our theoretical analysis, valid for symmetric interactions, shows that for sufficiently heterogeneous interactions and low demographic noise the system displays a multiple equilibria phase, which we fully characterize. In particular, we show that in this phase the number of locally stable equilibria is exponential in the number of species. Upon further decreasing the demographic noise, we unveil the presence of a second transition like the so-called "Gardner" transition to a marginally stable phase similar to that observed in the jamming of amorphous materials. We confirm and complement our analytical results by numerical simulations. Furthermore, we extend their relevance by showing that they hold for other interacting random dynamical systems such as the random replicant model. Finally, we discuss their extension to the case of asymmetric couplings.

Waiting-time dependent non-equilibrium phase diagram of simple glass- and gel-forming liquids

Under numerous circumstances, many soft and hard materials are present in a puzzling wealth of non-equilibrium amorphous states, whose properties are not stationary and depend on preparation. They are often summarized in unconventional "phase diagrams" that exhibit new "phases" and/or "transitions" in which time, however, is an essential variable. This work proposes a solution to the problem of theoretically defining and predicting these non-equilibrium phases and their time-evolving phase diagrams, given the underlying molecular interactions. We demonstrate that these non-equilibrium phases and the corresponding non-stationary (i.e., aging) phase diagrams can indeed be defined and predicted using the kinetic perspective of a novel non-equilibrium statistical mechanical theory of irreversible processes. This is illustrated with the theoretical description of the transient process of dynamic arrest into non-equilibrium amorphous solid phases of an instantaneously quenched simple model fluid involving repulsive hard-sphere plus attractive square well pair interactions.

Relaxation dynamics of the three-dimensional Coulomb glass model

In this paper, we analyze the dynamics of the Coulomb glass lattice model in three dimensions near a local equilibrium state by using mean-field approximations. We specifically focus on understanding the role of localization length (ξ) and the temperature (T) in the regime where the system is not far from equilibrium. We use the eigenvalue distribution of the dynamical matrix to characterize relaxation laws as a function of localization length at low temperatures. The variation of the minimum eigenvalue of the dynamical matrix with temperature and localization length is discussed numerically and analytically. Our results demonstrate the dominant role played by the localization length on the relaxation laws. For very small localization lengths, we find a crossover from exponential relaxation at long times to a logarithmic decay at intermediate times. No logarithmic decay at the intermediate times is observed for large localization lengths.

Glassy dynamics and equilibrium state on the honeycomb lattice: Role of surface diffusion and desorption on surface crowding

The phase behavior and adsorption kinetics of hard-core particles on a honeycomb lattice are studied by means of random sequential adsorption with surface diffusion. We concentrate on reversible adsorption by introducing a desorption process into our previous model and varying the equilibrium rate constant as a control parameter. We find that an exact prediction of the temporal evolution of fractional surface coverage and the surface pressure dynamics of reversible adsorption can be achieved by use of the blocking function of a system with irreversible adsorption of highly mobile particles. For systems out of equilibrium we observe several features of glassy dynamics, such as slow relaxation dynamics, the memory effect, and aging. In particular, the analysis of our system in the limit of small desorption probability shows simple aging behavior with a power-law decay. A detailed discussion of Gibbs adsorption isotherm for nonequilibrium adsorption is given, which exhibits a hysteresis between this system and its equilibrium counterpart.

Memory Formation in Jammed Hard Spheres

Liquids equilibrated below an onset condition share similar inherent states, while those above that onset have inherent states that markedly differ. Although this type of materials memory was first reported in simulations over 20 years ago, its physical origin remains controversial. Its absence from mean-field descriptions, in particular, has long cast doubt on its thermodynamic relevance. Motivated by a recent theoretical proposal, we reassess the onset phenomenology in simulations using a fast hard sphere jamming algorithm and find it to be both thermodynamically and dimensionally robust. Remarkably, we also uncover a second type of memory associated with a Gardner-like regime of the jamming algorithm.

Scaling Law Describes the Spin-Glass Response in Theory, Experiments, and Simulations

The correlation length ξ, a key quantity in glassy dynamics, can now be precisely measured for spin glasses both in experiments and in simulations. However, known analysis methods lead to discrepancies either for large external fields or close to the glass temperature. We solve this problem by introducing a scaling law that takes into account both the magnetic field and the time-dependent spin-glass correlation length. The scaling law is successfully tested against experimental measurements in a CuMn single crystal and against large-scale simulations on the Janus II dedicated computer.

Quantum Quenches in Isolated Quantum Glasses out of Equilibrium

In this work, we address the question of how a closed quantum system thermalizes in the presence of a random external potential. By investigating the quench dynamics of the isolated quantum spherical p-spin model, a paradigmatic model of a mean-field glass, we aim to shed new light on this complex problem. Employing a closed-time Schwinger-Keldysh path integral formalism, we first initialize the system in a random, infinite-temperature configuration and allow it to equilibrate in contact with a thermal bath before switching off the bath and performing a quench. We find evidence that increasing the strength of either the interactions or the quantum fluctuations can act to lower the effective temperature of the isolated system and stabilize glassy behavior.

Barriers, trapping times, and overlaps between local minima in the dynamics of the disordered Ising p-spin model

We study the low-temperature out-of-equilibrium Monte Carlo dynamics of the disordered Ising p-spin Model with p=3 and a small number of spin variables. We focus on sequences of configurations that are stable against single spin flips obtained by instantaneous gradient descent from persistent ones. We analyze the statistics of energy gaps, energy barriers, and trapping times on subsequences such that the overlap between consecutive configurations does not overcome a threshold. We compare our results to the predictions of various trap models finding the best agreement with the step model when the p-spin configurations are constrained to be uncorrelated.

Strong ergodicity breaking in aging of mean-field spin glasses

Out-of-equilibrium relaxation processes show aging if they become slower as time passes. Aging processes are ubiquitous and play a fundamental role in the physics of glasses and spin glasses and in other applications (e.g., in algorithms minimizing complex cost/loss functions). The theory of aging in the out-of-equilibrium dynamics of mean-field spin glass models has achieved a fundamental role, thanks to the asymptotic analytic solution found by Cugliandolo and Kurchan. However, this solution is based on assumptions (e.g., the weak ergodicity breaking hypothesis) which have never been put under a strong test until now. In the present work, we present the results of an extraordinary large set of numerical simulations of the prototypical mean-field spin glass models, namely the Sherrington-Kirkpatrick and the Viana-Bray models. Thanks to a very intensive use of graphics processing units (GPUs), we have been able to run the latter model for more than [Formula: see text] spin updates and thus safely extrapolate the numerical data both in the thermodynamical limit and in the large times limit. The measurements of the two-times correlation functions in isothermal aging after a quench from a random initial configuration to a temperature [Formula: see text] provides clear evidence that, at large times, such correlations do not decay to zero as expected by assuming weak ergodicity breaking. We conclude that strong ergodicity breaking takes place in mean-field spin glasses aging dynamics which, asymptotically, takes place in a confined configurational space. Theoretical models for the aging dynamics need to be revised accordingly.

Numerical solution of the dynamical mean field theory of infinite-dimensional equilibrium liquids

We present a numerical solution of the dynamical mean field theory of infinite-dimensional equilibrium liquids established by Maimbourg et al. [Phys. Rev. Lett. 116, 015902 (2016)]. For soft sphere interactions, we obtain the numerical solution by an iterative algorithm and a straightforward discretization of time. We also discuss the case of hard spheres for which we first derive analytically the dynamical mean field theory as a non-trivial limit of that of soft spheres. We present numerical results for the memory function and the mean square displacement. Our results reproduce and extend kinetic theory in the dilute or short-time limit, while they also describe dynamical arrest toward the glass phase in the dense strongly interacting regime.

Adaptive walks on high-dimensional fitness landscapes and seascapes with distance-dependent statistics

The dynamics of evolution is intimately shaped by epistasis - interactions between genetic elements which cause the fitness-effect of combinations of mutations to be non-additive. Analyzing evolutionary dynamics that involves large numbers of epistatic mutations is intrinsically difficult. A crucial feature is that the fitness landscape in the vicinity of the current genome depends on the evolutionary history. A key step is thus developing models that enable study of the effects of past evolution on future evolution. In this work, we introduce a broad class of high-dimensional random fitness landscapes for which the correlations between fitnesses of genomes are a general function of genetic distance. Their Gaussian character allows for tractable computational as well as analytic understanding. We study the properties of these landscapes focusing on the simplest evolutionary process: random adaptive (uphill) walks. Conventional measures of "ruggedness" are shown to not much affect such adaptive walks. Instead, the long-distance statistics of epistasis cause all properties to be highly conditional on past evolution, determining the statistics of the local landscape (the distribution of fitness-effects of available mutations and combinations of these), as well as the global geometry of evolutionary trajectories. In order to further explore the effects of conditioning on past evolution, we model the effects of slowly changing environments. At long times, such fitness "seascapes" cause a statistical steady state with highly intermittent evolutionary dynamics: populations undergo bursts of rapid adaptation, interspersed with periods in which adaptive mutations are rare and the population waits for more new directions to be opened up by changes in the environment. Finally, we discuss prospects for studying more complex evolutionary dynamics and on broader classes of high-dimensional landscapes and seascapes.

Universal non-mean-field scaling in the density of states of amorphous solids

Amorphous solids have excess soft modes in addition to the phonon modes described by the Debye theory. Recent numerical results show that if the phonon modes are carefully removed, the density of state of the excess soft modes exhibit universal quartic scaling, independent of the interaction potential, preparation protocol, and spatial dimensions. We hereby provide a theoretical framework to describe this universal scaling behavior. For this purpose, we extend the mean-field theory to include the effects of finite-dimensional fluctuation. Based on a semiphenomenological argument, we show that mean-field quadratic scaling is replaced by the quartic scaling in finite dimensions. Furthermore, we apply our formalism to explain the pressure and protocol dependence of the excess soft modes.

Quantum heat distribution in thermal relaxation processes

We analyze the heat exchange distribution of open quantum systems undergoing a thermal relaxation process with a time-dependent effective temperature. We show that such processes arise, for example, if the dynamics maximizes the entropy production. Using a two-point measurement scheme, we find an expression for the heat moment generating function that depends solely on the system's partition function and on the thermalization function (i.e., the law of cooling) describing the effective temperature. Applications include the relaxation of free bosonic and fermionic modes, for which closed-form expressions for the time-dependent heat distribution function are derived. Multiple free modes with arbitrary dispersion relations are also briefly discussed. In the semiclassical limit our formula agrees with previous results of the literature for the heat distribution of an optically trapped nanoscopic particle far from equilibrium.

Effects of frustration on fluctuation-dissipation relations

We study numerically the aging properties of the two-dimensional Ising model with quenched disorder considered in our recent paper [Phys. Rev. E 95, 062136 (2017)2470-004510.1103/PhysRevE.95.062136], where frustration can be tuned by varying the fraction of antiferromagnetic interactions. Specifically, we focus on the scaling properties of the autocorrelation and linear response functions after a quench of the model to a low temperature. We find that the interplay between equilibrium and aging occurs differently in the various regions of the phase diagram of the model. When the quench is made into the ferromagnetic phase the two-time quantities are made by the sum of an equilibrium and an aging part, whereas in the paramagnetic phase these parts combine in a multiplicative way. Scaling forms are shown to be obeyed with good accuracy, and the corresponding exponents and scaling functions are determined and discussed in the framework of what is known in clean and disordered systems.

Energy Fluctuations in Slowly Sheared Granular Materials

Here we show the first experimental measurement of the particle-scale energy fluctuations ΔE in a slowly sheared layer of photoelastic disks. Starting from an isotropically jammed state, applying shear causes the shear-induced stochastic strengthening and weakening of particle-scale energies, whose statistics and dynamics govern the evolution of the macroscopic stress-strain curve. We find that the ΔE behave as a temperaturelike noise field, showing a novel, Boltzmann-type, double-exponential distribution at any given shear strain γ. Following the framework of the soft glassy rheology theory, we extract an effective temperature χ from the statistics of the energy fluctuations to interpret the slow startup shear (shear starts from an isotropically jammed state) of granular materials as an "aging" process: Starting below one, χ gradually approaches one as γ increases, similar to those of spin glasses, thermal glasses, and bulk metallic glasses.

Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks

Networks of randomly connected neurons are among the most popular models in theoretical neuroscience. The connectivity between neurons in the cortex is however not fully random, the simplest and most prominent deviation from randomness found in experimental data being the overrepresentation of bidirectional connections among pyramidal cells. Using numerical and analytical methods, we investigate the effects of partially symmetric connectivity on the dynamics in networks of rate units. We consider the two dynamical regimes exhibited by random neural networks: the weak-coupling regime, where the firing activity decays to a single fixed point unless the network is stimulated, and the strong-coupling or chaotic regime, characterized by internally generated fluctuating firing rates. In the weak-coupling regime, we compute analytically, for an arbitrary degree of symmetry, the autocorrelation of network activity in the presence of external noise. In the chaotic regime, we perform simulations to determine the timescale of the intrinsic fluctuations. In both cases, symmetry increases the characteristic asymptotic decay time of the autocorrelation function and therefore slows down the dynamics in the network.

Activated dynamics: An intermediate model between the random energy model and the p-spin model

To study the activated dynamics of mean-field glasses, which takes place on times of order exp(N), where N is the system size, we introduce a new model, the correlated random energy model (CREM), that allows for a smooth interpolation between the REM and the p-spin models. We study numerically and analytically the CREM in the intermediate regime between REM and p-spin. We fully characterize its energy landscape, which is like a golf course but, at variance with the REM, has metabasins (or holes) containing several configurations. We find that an effective description for the dynamics, in terms of traps, emerges, provided that one identifies metabasins in the CREM with configurations in the trap model.