Variational control forces for enhanced sampling of nonequilibrium molecular dynamics simulations

Sensitivity analysis of colored-noise-driven interacting particle systems

We propose an efficient sensitivity analysis method for a wide class of colored-noise-driven interacting particle systems (IPSs). Our method is based on unperturbed simulations and significantly extends the Malliavin weight sampling method proposed by Szamel [Europhys. Lett. 117, 50010 (2017)0295-507510.1209/0295-5075/117/50010] for evaluating sensitivities such as linear response functions of IPSs driven by simple Ornstein-Uhlenbeck processes. We show that the sensitivity index depends not only on two effective parameters that characterize the variance and correlation time of the noise, but also on the noise spectrum. In the case of a single particle in a harmonic potential, we obtain exact analytical formulas for two types of linear response functions. By applying our method to a system of many particles interacting via a repulsive screened Coulomb potential, we compute the mobility and effective temperature of the system. Our results show that the system dynamics depend, in a nontrivial way, on the noise spectrum.

Time-asymmetric fluctuation theorem and efficient free-energy estimation

The free-energy difference ΔF between two high-dimensional systems is notoriously difficult to compute but very important for many applications such as drug discovery. We demonstrate that an unconventional definition of work introduced by Vaikuntanathan and Jarzynski (2008) satisfies a microscopic fluctuation theorem that relates path ensembles that are driven by protocols unequal under time reversal. It has been shown before that counterdiabatic protocols-those having additional forcing that enforces the system to remain in instantaneous equilibrium, also known as escorted dynamics or engineered swift equilibration-yield zero-variance work measurements for this definition. We show that this time-asymmetric microscopic fluctuation theorem can be exploited for efficient free-energy estimation by developing a simple (i.e., neural-network free) and efficient adaptive time-asymmetric protocol optimization algorithm that yields ΔF estimates that are orders of magnitude lower in mean squared error than the generic linear interpolation protocol with which it is initialized.

Splitting probabilities as optimal controllers of rare reactive events

The committor constitutes the primary quantity of interest within chemical kinetics as it is understood to encode the ideal reaction coordinate for a rare reactive event. We show the generative utility of the committor in that it can be used explicitly to produce a reactive trajectory ensemble that exhibits numerically exact statistics as that of the original transition path ensemble. This is done by relating a time-dependent analog of the committor that solves a generalized bridge problem to the splitting probability that solves a boundary value problem under a bistable assumption. By invoking stochastic optimal control and spectral theory, we derive a general form for the optimal controller of a bridge process that connects two metastable states expressed in terms of the splitting probability. This formalism offers an alternative perspective into the role of the committor and its gradients in that they encode force fields that guarantee reactivity, generating trajectories that are statistically identical to the way that a system would react autonomously.

Variational time reversal for free-energy estimation in nonequilibrium steady states

Studying the structure of systems in nonequilibrium steady states necessitates tools that quantify population shifts and associated deformations of equilibrium free-energy landscapes under persistent currents. Within the framework of stochastic thermodynamics, we establish a variant of the Kawasaki-Crooks equality that relates nonequilibrium free-energy corrections in overdamped Langevin systems to heat dissipation statistics along time-reversed relaxation trajectories computable with molecular simulation. Using stochastic control theory, we arrive at a general variational approach to evaluate the Kawasaki-Crooks equality and use it to estimate distribution functions of order parameters in specific models of driven and active matter, attaining substantial improvement in accuracy over simple perturbative methods.

Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations

We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimization tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorizable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process, the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.

Nonequilibrium design strategies for functional colloidal assemblies

We use a nonequilibrium variational principle to optimize the steady-state, shear-induced interconversion of self-assembled nanoclusters of DNA-coated colloids. Employing this principle within a stochastic optimization algorithm allows us to identify design strategies for functional materials. We find that far-from-equilibrium shear flow can significantly enhance the flux between specific colloidal states by decoupling trade-offs between stability and reactivity required by systems in equilibrium. For isolated nanoclusters, we find nonequilibrium strategies for amplifying transition rates by coupling a given reaction coordinate to the background shear flow. We also find that shear flow can be made to selectively break detailed balance and maximize probability currents by coupling orientational degrees of freedom to conformational transitions. For a microphase consisting of many nanoclusters, we study the flux of colloids hopping between clusters. We find that a shear flow can amplify the flux without a proportional compromise on the microphase structure. This approach provides a general means of uncovering design principles for nanoscale, autonomous, functional materials driven far from equilibrium.

Dynamical large deviations of linear diffusions

Linear diffusions are used to model a large number of stochastic processes in physics, including small mechanical and electrical systems perturbed by thermal noise, as well as Brownian particles controlled by electrical and optical forces. Here we use techniques from large deviation theory to study the statistics of time-integrated functionals of linear diffusions, considering three classes of functionals or observables relevant for nonequilibrium systems which involve linear or quadratic integrals of the state in time. For these, we derive exact results for the scaled cumulant generating function and the rate function, characterizing the fluctuations of observables in the long-time limit, and study in an exact way the set of paths or effective process that underlies these fluctuations. The results give a complete description of how fluctuations arise in linear diffusions in terms of effective forces that remain linear in the state or, alternatively, in terms of fluctuating densities and currents that solve Riccati-type equations. We illustrate these results using two common nonequilibrium models, namely, transverse diffusions in two dimensions involving a nonconservative rotating force, and two interacting particles in contact with heat baths at different temperatures.

Physics-informed graph neural networks enhance scalability of variational nonequilibrium optimal control

When a physical system is driven away from equilibrium, the statistical distribution of its dynamical trajectories informs many of its physical properties. Characterizing the nature of the distribution of dynamical observables, such as a current or entropy production rate, has become a central problem in nonequilibrium statistical mechanics. Asymptotically, for a broad class of observables, the distribution of a given observable satisfies a large deviation principle when the dynamics is Markovian, meaning that fluctuations can be characterized in the long-time limit by computing a scaled cumulant generating function. Calculating this function is not tractable analytically (nor often numerically) for complex, interacting systems, so the development of robust numerical techniques to carry out this computation is needed to probe the properties of nonequilibrium materials. Here, we describe an algorithm that recasts this task as an optimal control problem that can be solved variationally. We solve for optimal control forces using neural network ansätze that are tailored to the physical systems to which the forces are applied. We demonstrate that this approach leads to transferable and accurate solutions in two systems featuring large numbers of interacting particles.

Finite Time Large Deviations via Matrix Product States

Recent work has shown the effectiveness of tensor network methods for computing large deviation functions in constrained stochastic models in the infinite time limit. Here we show that these methods can also be used to study the statistics of dynamical observables at arbitrary finite time. This is a harder problem because, in contrast to the infinite time case, where only the extremal eigenstate of a tilted Markov generator is relevant, for finite time the whole spectrum plays a role. We show that finite time dynamical partition sums can be computed efficiently and accurately in one dimension using matrix product states and describe how to use such results to generate rare event trajectories on demand. We apply our methods to the Fredrickson-Andersen and East kinetically constrained models and to the symmetric simple exclusion process, unveiling dynamical phase diagrams in terms of counting field and trajectory time. We also discuss extensions of this method to higher dimensions.

Learning nonequilibrium control forces to characterize dynamical phase transitions

Sampling the collective, dynamical fluctuations that lead to nonequilibrium pattern formation requires probing rare regions of trajectory space. Recent approaches to this problem, based on importance sampling, cloning, and spectral approximations, have yielded significant insight into nonequilibrium systems but tend to scale poorly with the size of the system, especially near dynamical phase transitions. Here we propose a machine learning algorithm that samples rare trajectories and estimates the associated large deviation functions using a many-body control force by leveraging the flexible function representation provided by deep neural networks, importance sampling in trajectory space, and stochastic optimal control theory. We show that this approach scales to hundreds of interacting particles and remains robust at dynamical phase transitions.

Direct Evaluation of Rare Events in Active Matter from Variational Path Sampling

Active matter represents a broad class of systems that evolve far from equilibrium due to the local injection of energy. Like their passive analogs, transformations between distinct metastable states in active matter proceed through rare fluctuations; however, their detailed balance violating dynamics renders these events difficult to study. Here, we present a simulation method for evaluating the rate and mechanism of rare events in generic nonequilibrium systems and apply it to study the conformational changes of a passive solute in an active fluid. The method employs a variational optimization of a control force that renders the rare event a typical one, supplying an exact estimate of its rate as a ratio of path partition functions. Using this method we find that increasing activity in the active bath can enhance the rate of conformational switching of the passive solute in a manner consistent with recent bounds from stochastic thermodynamics.

Modeling of biomolecular machines in non-equilibrium steady states

Numerical computations have become a pillar of all modern quantitative sciences. Any computation involves modeling-even if often this step is not made explicit-and any model has to neglect details while still being physically accurate. Equilibrium statistical mechanics guides both the development of models and numerical methods for dynamics obeying detailed balance. For systems driven away from thermal equilibrium, such a universal theoretical framework is missing. For a restricted class of driven systems governed by Markov dynamics and local detailed balance, stochastic thermodynamics has evolved to fill this gap and to provide fundamental constraints and guiding principles. The next step is to advance stochastic thermodynamics from simple model systems to complex systems with tens of thousands or even millions of degrees of freedom. Biomolecules operating in the presence of chemical gradients and mechanical forces are a prime example for this challenge. In this Perspective, we give an introduction to isothermal stochastic thermodynamics geared toward the systematic multiscale modeling of the conformational dynamics of biomolecular and synthetic machines, and we outline some of the open challenges.

Reinforcement learning of rare diffusive dynamics

We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, such as those relevant in the study of reactive events, and trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, such as those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.

Molecular Properties and Chemical Transformations Near Interfaces

The properties of bulk water and aqueous solutions are known to change in the vicinity of an interface and/or in a confined environment, including the thermodynamics of ion selectivity at interfaces, transition states and pathways of chemical reactions, and nucleation events and phase growth. Here we describe joint progress in identifying unifying concepts about how air, liquid, and solid interfaces can alter molecular properties and chemical reactivity compared to bulk water and multicomponent solutions. We also discuss progress made in interfacial chemistry through advancements in new theory, molecular simulation, and experiments.

Dissipation bounds the amplification of transition rates far from equilibrium

Complex systems can convert energy imparted by nonequilibrium forces to regulate how quickly they transition between long-lived states. While such behavior is ubiquitous in natural and synthetic systems, currently there is no general framework to relate the enhancement of a transition rate to the energy dissipated or to bound the enhancement achievable for a given energy expenditure. We employ recent advances in stochastic thermodynamics to build such a framework, which can be used to gain mechanistic insight into transitions far from equilibrium. We show that under general conditions, there is a basic speed limit relating the typical excess heat dissipated throughout a transition and the rate amplification achievable. We illustrate this tradeoff in canonical examples of diffusive barrier crossings in systems driven with autonomous and deterministic external forcing protocols. In both cases, we find that our speed limit tightly constrains the rate enhancement.

Long-ranged correlations in large deviations of local clustering

In systems of diffusing particles, we investigate large deviations of a time-averaged measure of clustering around one particle. We focus on biased ensembles of trajectories, which realize large-deviation events. The bias acts on a single particle, but elicits a response that spans the whole system. We analyze this effect through the lens of macroscopic fluctuation theory, focusing on the coupling of the bias to hydrodynamic modes. This explains that the dynamical free energy has nontrivial scaling relationships with the system size, in 1 and 2 spatial dimensions. We show that the long-ranged response to a bias on one particle also has consequences when biasing two particles.

Entropy production fluctuations encode collective behavior in active matter

We derive a general lower bound on distributions of entropy production in interacting active matter systems. The bound is tight in the limit that interparticle correlations are small and short-ranged, which we explore in four canonical active matter models. In all models studied, the bound is weak where collective fluctuations result in long-ranged correlations, which subsequently links the locations of phase transitions to enhanced entropy production fluctuations. We develop a theory for the onset of enhanced fluctuations and relate it to specific phase transitions in active Brownian particles. We also derive optimal control forces that realize the dynamics necessary to tune dissipation and manipulate the system between phases. In so doing, we uncover a general relationship between entropy production and pattern formation in active matter, as well as ways of controlling it.

Collective motion in large deviations of active particles

We analyze collective motion that occurs during rare (large deviation) events in systems of active particles, both numerically and analytically. We discuss the associated dynamical phase transition to collective motion, which occurs when the active work is biased towards larger values, and is associated with alignment of particles' orientations. A finite biasing field is needed to induce spontaneous symmetry breaking, even in large systems. Particle alignment is computed exactly for a system of two particles. For many-particle systems, we analyze the symmetry breaking by an optimal-control representation of the biased dynamics, and we propose a fluctuating hydrodynamic theory that captures the emergence of polar order in the biased state.

Dynamical Phase Transitions in a 2D Classical Nonequilibrium Model via 2D Tensor Networks

We demonstrate the power of 2D tensor networks for obtaining large deviation functions of dynamical observables in a classical nonequilibrium setting. Using these methods, we analyze the previously unstudied dynamical phase behavior of the fully 2D asymmetric simple exclusion process with biases in both the x and y directions. We identify a dynamical phase transition, from a jammed to a flowing phase, and characterize the phases and the transition, with an estimate of the critical point and exponents.