1
|
Singh AN, Limmer DT. Splitting probabilities as optimal controllers of rare reactive events. J Chem Phys 2024; 161:054113. [PMID: 39101534 DOI: 10.1063/5.0203840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 07/10/2024] [Indexed: 08/06/2024] Open
Abstract
The committor constitutes the primary quantity of interest within chemical kinetics as it is understood to encode the ideal reaction coordinate for a rare reactive event. We show the generative utility of the committor in that it can be used explicitly to produce a reactive trajectory ensemble that exhibits numerically exact statistics as that of the original transition path ensemble. This is done by relating a time-dependent analog of the committor that solves a generalized bridge problem to the splitting probability that solves a boundary value problem under a bistable assumption. By invoking stochastic optimal control and spectral theory, we derive a general form for the optimal controller of a bridge process that connects two metastable states expressed in terms of the splitting probability. This formalism offers an alternative perspective into the role of the committor and its gradients in that they encode force fields that guarantee reactivity, generating trajectories that are statistically identical to the way that a system would react autonomously.
Collapse
Affiliation(s)
- Aditya N Singh
- Department of Chemistry, University of California, Berkeley, California 94720, USA
- Chemical Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| | - David T Limmer
- Department of Chemistry, University of California, Berkeley, California 94720, USA
- Chemical Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
- Materials Science Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
- Kavli Energy Nanoscience Institute at Berkeley, Berkeley, California 94720, USA
| |
Collapse
|
2
|
Keller BG, Bolhuis PG. Dynamical Reweighting for Biased Rare Event Simulations. Annu Rev Phys Chem 2024; 75:137-162. [PMID: 38941527 DOI: 10.1146/annurev-physchem-083122-124538] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/30/2024]
Abstract
Dynamical reweighting techniques aim to recover the correct molecular dynamics from a simulation at a modified potential energy surface. They are important for unbiasing enhanced sampling simulations of molecular rare events. Here, we review the theoretical frameworks of dynamical reweighting for modified potentials. Based on an overview of kinetic models with increasing level of detail, we discuss techniques to reweight two-state dynamics, multistate dynamics, and path integrals. We explore the natural link to transition path sampling and how the effect of nonequilibrium forces can be reweighted. We end by providing an outlook on how dynamical reweighting integrates with techniques for optimizing collective variables and with modern potential energy surfaces.
Collapse
Affiliation(s)
- Bettina G Keller
- Department of Biology, Chemistry and Pharmacy, Freie Universität Berlin, Berlin, Germany;
| | - Peter G Bolhuis
- Van 't Hoff Institute for Molecular Sciences, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
3
|
Gillman E, Rose DC, Garrahan JP. Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations. PHYSICAL REVIEW LETTERS 2024; 132:197301. [PMID: 38804929 DOI: 10.1103/physrevlett.132.197301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 02/28/2024] [Accepted: 04/04/2024] [Indexed: 05/29/2024]
Abstract
We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimization tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorizable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process, the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.
Collapse
Affiliation(s)
- Edward Gillman
- School of Physics and Astronomy, University of Nottingham, Nottingham NG7 2RD, United Kingdom
- Centre for the Mathematics and Theoretical Physics of Quantum Non-Equilibrium Systems, University of Nottingham, Nottingham NG7 2RD, United Kingdom
| | - Dominic C Rose
- Department of Physics and Astronomy, University College London, Gower Street, London WC1E 6BT, United Kingdom
| | - Juan P Garrahan
- School of Physics and Astronomy, University of Nottingham, Nottingham NG7 2RD, United Kingdom
- Centre for the Mathematics and Theoretical Physics of Quantum Non-Equilibrium Systems, University of Nottingham, Nottingham NG7 2RD, United Kingdom
| |
Collapse
|
4
|
Das A, Limmer DT. Nonequilibrium design strategies for functional colloidal assemblies. Proc Natl Acad Sci U S A 2023; 120:e2217242120. [PMID: 37748070 PMCID: PMC10556551 DOI: 10.1073/pnas.2217242120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2022] [Accepted: 08/17/2023] [Indexed: 09/27/2023] Open
Abstract
We use a nonequilibrium variational principle to optimize the steady-state, shear-induced interconversion of self-assembled nanoclusters of DNA-coated colloids. Employing this principle within a stochastic optimization algorithm allows us to identify design strategies for functional materials. We find that far-from-equilibrium shear flow can significantly enhance the flux between specific colloidal states by decoupling trade-offs between stability and reactivity required by systems in equilibrium. For isolated nanoclusters, we find nonequilibrium strategies for amplifying transition rates by coupling a given reaction coordinate to the background shear flow. We also find that shear flow can be made to selectively break detailed balance and maximize probability currents by coupling orientational degrees of freedom to conformational transitions. For a microphase consisting of many nanoclusters, we study the flux of colloids hopping between clusters. We find that a shear flow can amplify the flux without a proportional compromise on the microphase structure. This approach provides a general means of uncovering design principles for nanoscale, autonomous, functional materials driven far from equilibrium.
Collapse
Affiliation(s)
- Avishek Das
- Department of Chemistry, University of California, Berkeley, CA94720
| | - David T. Limmer
- Department of Chemistry, University of California, Berkeley, CA94720
- Chemical Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA94720
- Material Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA94720
- Kavli Energy NanoSciences Institute, University of California, Berkeley, CA94720
| |
Collapse
|
5
|
Bolhuis PG, Brotzakis ZF, Keller BG. Optimizing molecular potential models by imposing kinetic constraints with path reweighting. J Chem Phys 2023; 159:074102. [PMID: 37581416 DOI: 10.1063/5.0151166] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2023] [Accepted: 06/19/2023] [Indexed: 08/16/2023] Open
Abstract
Empirical force fields employed in molecular dynamics simulations of complex systems are often optimized to reproduce experimentally determined structural and thermodynamic properties. In contrast, experimental knowledge about the interconversion rates between metastable states in such systems is hardly ever incorporated in a force field due to a lack of an efficient approach. Here, we introduce such a framework based on the relationship between dynamical observables, such as rate constants, and the underlying molecular model parameters using the statistical mechanics of trajectories. Given a prior ensemble of molecular dynamics trajectories produced with imperfect force field parameters, the approach allows for the optimal adaption of these parameters such that the imposed constraint of equally predicted and experimental rate constant is obeyed. To do so, the method combines the continuum path ensemble maximum caliber approach with path reweighting methods for stochastic dynamics. When multiple solutions are found, the method selects automatically the combination that corresponds to the smallest perturbation of the entire path ensemble, as required by the maximum entropy principle. To show the validity of the approach, we illustrate the method on simple test systems undergoing rare event dynamics. Next to simple 2D potentials, we explore particle models representing molecular isomerization reactions and protein-ligand unbinding. Besides optimal interaction parameters, the methodology gives physical insights into what parts of the model are most sensitive to the kinetics. We discuss the generality and broad implications of the methodology.
Collapse
Affiliation(s)
- Peter G Bolhuis
- van 't Hoff Institute for Molecular Sciences, University of Amsterdam, P.O. Box 94157, 1090 GD Amsterdam, The Netherlands
| | - Z Faidon Brotzakis
- Department of Chemistry, University of Cambridge, Cambridge CB2 1EW, United Kingdom
| | - Bettina G Keller
- Department of Biology, Chemistry, Pharmacy, Freie Universität Berlin, Arnimallee 22, D-14195 Berlin, Germany
| |
Collapse
|
6
|
Kuznets-Speck B, Limmer DT. Inferring equilibrium transition rates from nonequilibrium protocols. Biophys J 2023; 122:1659-1664. [PMID: 36964656 PMCID: PMC10183322 DOI: 10.1016/j.bpj.2023.03.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2022] [Revised: 01/08/2023] [Accepted: 03/21/2023] [Indexed: 03/26/2023] Open
Abstract
We develop a theory for inferring equilibrium transition rates from trajectories driven by a time-dependent force using results from stochastic thermodynamics. Applying the Kawasaki relation to approximate the nonequilibrium distribution function in terms of the equilibrium distribution function and the excess dissipation, we formulate a nonequilibrium transition state theory to estimate the rate enhancement over the equilibrium rate due to the nonequilibrium protocol. We demonstrate the utility of our theory in examples of pulling of harmonically trapped particles in one and two dimensions, as well as a semiflexible polymer with a reactive linker in three dimensions. We expect our purely thermodynamic approach will find use in both molecular simulation and force spectroscopy experiments.
Collapse
Affiliation(s)
| | - David T Limmer
- Chemistry Department, University of California, Berkeley, Berkeley, California; Chemical Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California; Material Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California; Kavli Energy NanoSciences Institute, University of California, Berkeley, Berkeley, California.
| |
Collapse
|
7
|
Cates ME, Nardini C. Classical Nucleation Theory for Active Fluid Phase Separation. PHYSICAL REVIEW LETTERS 2023; 130:098203. [PMID: 36930897 DOI: 10.1103/physrevlett.130.098203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Accepted: 02/10/2023] [Indexed: 06/18/2023]
Abstract
Classical Nucleation Theory (CNT), linking rare nucleation events to the free-energy landscape of a growing nucleus, is central to understanding phase-change kinetics in passive fluids. Nucleation in nonequilibrium systems is much harder to describe because there is no free energy, but instead a dynamics-dependent quasipotential that typically must be found numerically. Here we extend CNT to a class of active phase-separating systems governed by a minimal field-theoretic model (Active Model B+). In the small noise and supersaturation limits that CNT assumes, we compute analytically the quasipotential, and hence, nucleation barrier, for liquid-vapor phase separation. Crucial to our results, detailed balance, although broken microscopically by activity, is restored along the instanton trajectory, which in CNT involves the nuclear radius as the sole reaction coordinate.
Collapse
Affiliation(s)
- M E Cates
- DAMTP, Centre for Mathematical Sciences, University of Cambridge, Wilberforce Road, Cambridge CB3 0WA, United Kingdom
| | - C Nardini
- Service de Physique de l'Etat Condensé, CEA, CNRS Université Paris-Saclay, CEA-Saclay, 91191 Gif-sur-Yvette, France
- Sorbonne Université, CNRS, Laboratoire de Physique Théorique de la Matière Condensée, 75005 Paris, France
| |
Collapse
|
8
|
Zhong A, DeWeese MR. Limited-control optimal protocols arbitrarily far from equilibrium. Phys Rev E 2022; 106:044135. [PMID: 36397571 DOI: 10.1103/physreve.106.044135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Accepted: 09/16/2022] [Indexed: 06/16/2023]
Abstract
Recent studies have explored finite-time dissipation-minimizing protocols for stochastic thermodynamic systems driven arbitrarily far from equilibrium, when granted full external control to drive the system. However, in both simulation and experimental contexts, systems often may only be controlled with a limited set of degrees of freedom. Here, going beyond slow- and fast-driving approximations employed in previous studies, we obtain exact finite-time optimal protocols for this limited-control setting. By working with deterministic Fokker-Planck probability density time evolution, we can frame the work-minimizing protocol problem in the standard form of an optimal control theory problem. We demonstrate that finding the exact optimal protocol is equivalent to solving a system of Hamiltonian partial differential equations, which in many cases admit efficiently calculable numerical solutions. Within this framework, we reproduce analytical results for the optimal control of harmonic potentials and numerically devise optimal protocols for two anharmonic examples: varying the stiffness of a quartic potential and linearly biasing a double-well potential. We confirm that these optimal protocols outperform other protocols produced through previous methods, in some cases by a substantial amount. We find that for the linearly biased double-well problem, the mean position under the optimal protocol travels at a near-constant velocity. Surprisingly, for a certain timescale and barrier height regime, the optimal protocol is also nonmonotonic in time.
Collapse
Affiliation(s)
- Adrianne Zhong
- Department of Physics, University of California, Berkeley, Berkeley, California 94720, USA
| | - Michael R DeWeese
- Department of Physics, University of California, Berkeley, Berkeley, California 94720, USA and Redwood Center For Theoretical Neuroscience and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California 94720, USA
| |
Collapse
|
9
|
Yan J, Rotskoff G. Physics-informed graph neural networks enhance scalability of variational nonequilibrium optimal control. J Chem Phys 2022; 157:074101. [DOI: 10.1063/5.0095593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
When a physical system is driven away from equilibrium, the statistical distribution of its dynamical trajectories informs many of its physical properties. Characterizing the nature of the distribution of dynamical observables, such as a current or entropy production rate, has become a central problem in nonequilibrium statistical mechanics. Asymptotically, for a broad class of observables, the distribution of a given observable satisfies a large deviation principle when the dynamics is Markovian, meaning that fluctuations can be characterized in the long-time limit by computing a scaled cumulant generating function. Calculating this function is not tractable analytically (nor often numerically) for complex, interacting systems, so the development of robust numerical techniques to carry out this computation is needed to probe the properties of nonequilibrium materials. Here, we describe an algorithm that recasts this task as an optimal control problem that can be solved variationally. We solve for optimal control forces using neural network ansätze that are tailored to the physical systems to which the forces are applied. We demonstrate that this approach leads to transferable and accurate solutions in two systems featuring large numbers of interacting particles.
Collapse
Affiliation(s)
- Jiawei Yan
- Stanford University Department of Chemistry, United States of America
| | - Grant Rotskoff
- Chemistry, Stanford University Department of Chemistry, United States of America
| |
Collapse
|