Dynamical Phase Transitions in a 2D Classical Nonequilibrium Model via 2D Tensor Networks

Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations

We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimization tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorizable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process, the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.

Topological phases in the dynamics of the simple exclusion process

We study the dynamical large deviations of the classical stochastic symmetric simple exclusion process (SSEP) by means of numerical matrix product states. We show that for half filling, long-time trajectories with a large enough imbalance between the number hops in even and odd bonds of the lattice belong to distinct symmetry-protected topological (SPT) phases. Using tensor network techniques, we obtain the large deviation (LD) phase diagram in terms of counting fields conjugate to the dynamical activity and the total hop imbalance. We show the existence of high activity trivial and nontrivial SPT phases (classified according to string order parameters) separated by either a critical phase or a critical point. Using the leading eigenstate of the tilted generator, obtained from infinite-system density-matrix renormalization group simulations, we construct a near-optimal dynamics for sampling the LDs, and show that the SPT phases manifest at the level of rare stochastic trajectories. We also show how to extend these results to other filling fractions, and discuss generalizations to asymmetric SEPs.

Learning nonequilibrium statistical mechanics and dynamical phase transitions

Nonequilibrium statistical mechanics exhibit a variety of complex phenomena far from equilibrium. It inherits challenges of equilibrium, including accurately describing the joint distribution of a large number of configurations, and also poses new challenges as the distribution evolves over time. Characterizing dynamical phase transitions as an emergent behavior further requires tracking nonequilibrium systems under a control parameter. While a number of methods have been proposed, such as tensor networks for one-dimensional lattices, we lack a method for arbitrary time beyond the steady state and for higher dimensions. Here, we develop a general computational framework to study the time evolution of nonequilibrium systems in statistical mechanics by leveraging variational autoregressive networks, which offer an efficient computation on the dynamical partition function, a central quantity for discovering the phase transition. We apply the approach to prototype models of nonequilibrium statistical mechanics, including the kinetically constrained models of structural glasses up to three dimensions. The approach uncovers the active-inactive phase transition of spin flips, the dynamical phase diagram, as well as new scaling relations. The result highlights the potential of machine learning dynamical phase transitions in nonequilibrium systems.

Efficient simulations of epidemic models with tensor networks: Application to the one-dimensional susceptible-infected-susceptible model

The contact process is an emblematic model of a nonequilibrium system, containing a phase transition between inactive and active dynamical regimes. In the epidemiological context, the model is known as the susceptible-infected-susceptible model, and it is widely used to describe contagious spreading. In this work, we demonstrate how accurate and efficient representations of the full probability distribution over all configurations of the contact process on a one-dimensional chain can be obtained by means of matrix product states (MPSs). We modify and adapt MPS methods from many-body quantum systems to study the classical distributions of the driven contact process at late times. We give accurate and efficient results for the distribution of large gaps, and illustrate the advantage of our methods over Monte Carlo simulations. Furthermore, we study the large deviation statistics of the dynamical activity, defined as the total number of configuration changes along a trajectory, and investigate quantum-inspired entropic measures, based on the second Rényi entropy.

Tensor network construction for lattice gas models: Hard-core and triangular lattice models

The representation of complex lattice models in the form of a tensor network is a promising approach to the analysis of the thermodynamics of such systems. Once the tensor network is built, various methods can be used to calculate the partition function of the corresponding model. However, it is possible to build the initial tensor network in different ways for the same model. In this work, we have proposed two ways of constructing tensor networks and demonstrated that the construction process affects the accuracy of calculations. For demonstration purposes, we have done a brief study of the 4 nearest-neighbor (NN) and 5NN models, where adsorbed particles exclude all sites up to the fourth and fifth nearest neighbors from being occupied by another particle. In addition, we have studied a 4NN model with finite repulsions with a fifth neighbor. In a sense, this model is intermediate between 4NN and 5NN models, so algorithms designed for systems with hard-core interactions may experience difficulties. We have obtained adsorption isotherms, as well as graphs of entropy and heat capacity for all models. The critical values of the chemical potential were determined from the position of the heat capacity peaks. As a result, we were able to improve our previous estimate of the position of the phase transition points for the 4NN and 5NN models. And in the model with finite interactions, we found the presence of two first-order phase transitions and made an estimate of the critical values of the chemical potential for them.

Optimal Sampling of Dynamical Large Deviations in Two Dimensions via Tensor Networks

We use projected entangled-pair states (PEPS) to calculate the large deviation statistics of the dynamical activity of the two-dimensional East model, and the two-dimensional symmetric simple exclusion process (SSEP) with open boundaries, in lattices of up to 40×40 sites. We show that at long times both models have phase transitions between active and inactive dynamical phases. For the 2D East model we find that this trajectory transition is of the first order, while for the SSEP we find indications of a second order transition. We then show how the PEPS can be used to implement a trajectory sampling scheme capable of directly accessing rare trajectories. We also discuss how the methods described here can be extended to study rare events at finite times.

Computing time-periodic steady-state currents via the time evolution of tensor network states

We present an approach based upon binary tree tensor network (BTTN) states for computing steady-state current statistics for a many-particle 1D ratchet subject to volume exclusion interactions. The ratcheted particles, which move on a lattice with periodic boundary conditions subject to a time-periodic drive, can be stochastically evolved in time to sample representative trajectories via a Gillespie method. In lieu of generating realizations of trajectories, a BTTN state can variationally approximate a distribution over the vast number of many-body configurations. We apply the density matrix renormalization group (DMRG) algorithm to initialize BTTN states, which are then propagated in time via the time-dependent variational principle (TDVP) algorithm to yield the steady-state behavior, including the effects of both typical and rare trajectories. The application of the methods to ratchet currents is highlighted, but the approach extends naturally to other interacting lattice models with time-dependent driving. Though trajectory sampling is conceptually and computationally simpler, we discuss situations for which the BTTN TDVP strategy can be beneficial.

Slow dynamics and large deviations in classical stochastic Fredkin chains

The Fredkin spin chain serves as an interesting theoretical example of a quantum Hamiltonian whose ground state exhibits a phase transition between three distinct phases, one of which violates the area law. Here we consider a classical stochastic version of the Fredkin model, which can be thought of as a simple exclusion process subject to additional kinetic constraints, and study its classical stochastic dynamics. The ground-state phase transition of the quantum chain implies an equilibrium phase transition in the stochastic problem, whose properties we quantify in terms of numerical matrix product states (MPSs). The stochastic model displays slow dynamics, including power-law decaying autocorrelation functions and hierarchical relaxation processes due to exponential localization. Like in other kinetically constrained models, the Fredkin chain has a rich structure in its dynamical large deviations-which we compute accurately via numerical MPSs-including an active-inactive phase transition and a hierarchy of trajectory phases connected to particular equilibrium states of the model. We also propose, via its height field representation, a generalization of the Fredkin model to two dimensions in terms of constrained dimer coverings of the honeycomb lattice.

Physics-informed graph neural networks enhance scalability of variational nonequilibrium optimal control

When a physical system is driven away from equilibrium, the statistical distribution of its dynamical trajectories informs many of its physical properties. Characterizing the nature of the distribution of dynamical observables, such as a current or entropy production rate, has become a central problem in nonequilibrium statistical mechanics. Asymptotically, for a broad class of observables, the distribution of a given observable satisfies a large deviation principle when the dynamics is Markovian, meaning that fluctuations can be characterized in the long-time limit by computing a scaled cumulant generating function. Calculating this function is not tractable analytically (nor often numerically) for complex, interacting systems, so the development of robust numerical techniques to carry out this computation is needed to probe the properties of nonequilibrium materials. Here, we describe an algorithm that recasts this task as an optimal control problem that can be solved variationally. We solve for optimal control forces using neural network ansätze that are tailored to the physical systems to which the forces are applied. We demonstrate that this approach leads to transferable and accurate solutions in two systems featuring large numbers of interacting particles.

Using tensor network states for multi-particle Brownian ratchets

The study of Brownian ratchets has taught how time-periodic driving supports a time-periodic steady state that generates nonequilibrium transport. When a single particle is transported in one dimension, it is possible to rationalize the current in terms of the potential, but experimental efforts have ventured beyond that single-body case to systems with many interacting carriers. Working with a lattice model of volume-excluding particles in one dimension, we analyze the impact of interactions on a flashing ratchet's current. To surmount the many-body problem, we employ the time-dependent variational principle (TDVP) applied to binary tree tensor networks (BTTN). Rather than propagating individual trajectories, the tensor network approach propagates a distribution over many-body configurations via a controllable variational approximation. The calculations, which reproduce Gillespie trajectory sampling, identify and explain a shift in the frequency of maximum current to higher driving frequency as the lattice occupancy increases.

Finite Time Large Deviations via Matrix Product States

Recent work has shown the effectiveness of tensor network methods for computing large deviation functions in constrained stochastic models in the infinite time limit. Here we show that these methods can also be used to study the statistics of dynamical observables at arbitrary finite time. This is a harder problem because, in contrast to the infinite time case, where only the extremal eigenstate of a tilted Markov generator is relevant, for finite time the whole spectrum plays a role. We show that finite time dynamical partition sums can be computed efficiently and accurately in one dimension using matrix product states and describe how to use such results to generate rare event trajectories on demand. We apply our methods to the Fredrickson-Andersen and East kinetically constrained models and to the symmetric simple exclusion process, unveiling dynamical phase diagrams in terms of counting field and trajectory time. We also discuss extensions of this method to higher dimensions.

Dynamical Large Deviations of Two-Dimensional Kinetically Constrained Models Using a Neural-Network State Ansatz

We use a neural-network ansatz originally designed for the variational optimization of quantum systems to study dynamical large deviations in classical ones. We use recurrent neural networks to describe the large deviations of the dynamical activity of model glasses, kinetically constrained models in two dimensions. We present the first finite size-scaling analysis of the large-deviation functions of the two-dimensional Fredrickson-Andersen model, and explore the spatial structure of the high-activity sector of the South-or-East model. These results provide a new route to the study of dynamical large-deviation functions, and highlight the broad applicability of the neural-network state ansatz across domains in physics.

Optimal sampling of dynamical large deviations via matrix product states

The large deviation statistics of dynamical observables is encoded in the spectral properties of deformed Markov generators. Recent works have shown that tensor network methods are well suited to compute accurately the relevant leading eigenvalues and eigenvectors. However, the efficient generation of the corresponding rare trajectories is a harder task. Here, we show how to exploit the matrix product state approximation of the dominant eigenvector to implement an efficient sampling scheme which closely resembles the optimal (so-called "Doob") dynamics that realizes the rare events. We demonstrate our approach on three well-studied lattice models, the Fredrickson-Andersen and East kinetically constrained models, and the symmetric simple exclusion process. We discuss how to generalize our approach to higher dimensions.

Phase Diagram of Active Brownian Spheres: Crystallization and the Metastability of Motility-Induced Phase Separation

Motility-induced phase separation (MIPS), the phenomenon in which purely repulsive active particles undergo a liquid-gas phase separation, is among the simplest and most widely studied examples of a nonequilibrium phase transition. Here, we show that states of MIPS coexistence are in fact only metastable for three-dimensional active Brownian particles over a very broad range of conditions, decaying at long times through an ordering transition we call active crystallization. At an activity just above the MIPS critical point, the liquid-gas binodal is superseded by the crystal-fluid coexistence curve, with solid, liquid, and gas all coexisting at the triple point where the two curves intersect. Nucleating an active crystal from a disordered fluid, however, requires a rare fluctuation that exhibits the nearly close-packed density of the solid phase. The corresponding barrier to crystallization is surmountable on a feasible timescale only at high activity, and only at fluid densities near maximal packing. The glassiness expected for such dense liquids at equilibrium is strongly mitigated by active forces, so that the lifetime of liquid-gas coexistence declines steadily with increasing activity, manifesting in simulations as a facile spontaneous crystallization at extremely high activity.

Dynamics and large deviation transitions of the XOR-Fredrickson-Andersen kinetically constrained model

We study a one-dimensional classical stochastic kinetically constrained model (KCM) inspired by Rydberg atoms in their "facilitated" regime, where sites can flip only if a single of their nearest neighbors is excited. We call this model "XOR-FA" to distinguish it from the standard Fredrickson-Andersen (FA) model. We describe the dynamics of the XOR-FA model, including its relation to simple exclusion processes in its domain wall representation. The interesting relaxation dynamics of the XOR-FA is related to the prominence of large dynamical fluctuations that lead to phase transitions between active and inactive dynamical phases as in other KCMs. By means of numerical tensor network methods we study in detail such transitions in the dynamical large deviation regime.

Transcorrelated density matrix renormalization group

We introduce the transcorrelated Density Matrix Renormalization Group (tcDMRG) theory for the efficient approximation of the energy for strongly correlated systems. tcDMRG encodes the wave function as a product of a fixed Jastrow or Gutzwiller correlator and a matrix product state. The latter is optimized by applying the imaginary-time variant of time-dependent (TD) DMRG to the non-Hermitian transcorrelated Hamiltonian. We demonstrate the efficiency of tcDMRG with the example of the two-dimensional Fermi-Hubbard Hamiltonian, a notoriously difficult target for the DMRG algorithm, for different sizes, occupation numbers, and interaction strengths. We demonstrate fast energy convergence of tcDMRG, which indicates that tcDMRG could increase the efficiency of standard DMRG beyond quasi-monodimensional systems and provides a generally powerful approach toward the dynamic correlation problem of DMRG.