Onsager-Machlup action-based path sampling and its combination with replica exchange for diffusive and multiple pathways

Reinforcement learning of rare diffusive dynamics

We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, such as those relevant in the study of reactive events, and trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, such as those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.

Multiscale aspects of molecular motions: From molecular vibrations, conformational changes of biomolecules to cellular dynamics

Molecular aspects of living systems are important because it is the most basic aspects of life as exemplified in biochemistry and structural biology. Since molecules move due to interactive forces between atoms, physics plays an important role to understand the dynamic phenomena of living systems. Here we review our multiscale approaches for computationally treating different levels of molecular motions: vibrational dynamics of molecules, conformational change of biomolecules, and cellular dynamics using statistical-mechanics-based models.

A modified nudged elastic band algorithm with adaptive spring lengths

We present a modified version of the nudged elastic band (NEB) algorithm to find minimum energy paths connecting two known configurations. We show that replacing the harmonic band-energy term with a discretized version of the Onsager-Machlup action leads to a NEB algorithm with adaptive spring lengths that automatically increase the resolution of the minimum energy path around the saddle point of the potential energy surface. The method has the same computational cost per optimization step of the standard NEB algorithm and does not introduce additional parameters. We present applications to the isomerization of alanine dipeptide, the elimination of hydrogen from ethane, and the healing of a 5-77-5 defect in graphene.

Path Ensembles for Pin1-Catalyzed Cis-Trans Isomerization of a Substrate Calculated by Weighted Ensemble Simulations

Pin1 enzyme protein recognizes specifically phosphorylated serine/threonine (pSer/pThr) and catalyzes the slow interconversion of the peptidyl-prolyl bond between cis and trans forms. Structural dynamics between the cis and trans forms are essential to reveal the underlying molecular mechanism of the catalysis. In this study, we apply the weighted ensemble (WE) simulation method to obtain comprehensive path ensembles for the Pin1-catalyzed isomerization process. Associated rate constants for both cis-to-trans and trans-to-cis isomerization are calculated to be submicroseconds time scales, which are in good agreement with the calculated free energy landscape where the cis form is slightly less favorable. The committor-like analysis indicates the shift of the transition state toward trans form (at the isomerization angle ω ∼ 110°) compared to the intrinsic position for the isolated substrate (ω ∼ 90°). The calculated structural ensemble clarifies a role of both the dual-histidine motif, His59/His157, and the basic residues, Lys63/Arg68/Arg69, to anchor both sides of the peptidyl-prolyl bond, the aromatic ring in Pro, and the phosphate in pSer, respectively. The rotation of the torsion angle is found to be facilitated by relaying the hydrogen-bond partner of the main-chain oxygen in pSer from Cys113 in the cis form to Arg68 in the trans form, through Ser154 at the transition state, which is really the cause of the shift in the transition state. The role of Ser154 as a driving force of the isomerization is confirmed by additional WE and free energy calculations for S154A mutant where the isomerization takes place slightly slower and the free energy barrier increases through the mutation. The present study shows the usefulness of the WE simulation for substantial path samplings between the reactant and product states, unraveling the molecular mechanism of the enzyme catalysis.

Metadynamics of Paths

We present a method to sample reactive pathways via biased molecular dynamics simulations in trajectory space. We show that the use of enhanced sampling techniques enables unconstrained exploration of multiple reaction routes. Time correlation functions are conveniently computed via reweighted averages along a single trajectory and kinetic rates are accessed at no additional cost. These abilities are illustrated analyzing a model potential and the umbrella inversion of NH_{3} in water. The algorithm allows a parallel implementation and promises to be a powerful tool for the study of rare events.

Exploring Configuration Space and Path Space of Biomolecules Using Enhanced Sampling Techniques-Searching for Mechanism and Kinetics of Biomolecular Functions

To understand functions of biomolecules such as proteins, not only structures but their conformational change and kinetics need to be characterized, but its atomistic details are hard to obtain both experimentally and computationally. Here, we review our recent computational studies using novel enhanced sampling techniques for conformational sampling of biomolecules and calculations of their kinetics. For efficiently characterizing the free energy landscape of a biomolecule, we introduce the multiscale enhanced sampling method, which uses a combined system of atomistic and coarse-grained models. Based on the idea of Hamiltonian replica exchange, we can recover the statistical properties of the atomistic model without any biases. We next introduce the string method as a path search method to calculate the minimum free energy pathways along a multidimensional curve in high dimensional space. Finally we introduce novel methods to calculate kinetics of biomolecules based on the ideas of path sampling: one is the Onsager⁻Machlup action method, and the other is the weighted ensemble method. Some applications of the above methods to biomolecular systems are also discussed and illustrated.

Stratified UWHAM and Its Stochastic Approximation for Multicanonical Simulations Which Are Far from Equilibrium

We describe a new analysis tool called Stratified unbinned Weighted Histogram Analysis Method (Stratified-UWHAM), which can be used to compute free energies and expectations from a multicanonical ensemble when a subset of the parallel simulations is far from being equilibrated because of barriers between free energy basins which are only rarely (or never) crossed at some states. The Stratified-UWHAM equations can be obtained in the form of UWHAM equations but with an expanded set of states. We also provide a stochastic solver, Stratified RE-SWHAM, for Stratified-UWHAM to remove its computational bottleneck. Stratified-UWHAM and Stratified RE-SWHAM are applied to study three test topics: the free energy landscape of alanine dipeptide, the binding affinity of a host-guest binding complex, and path sampling for a two-dimensional double well potential. The examples show that when some of the parallel simulations are only locally equilibrated, the estimates of free energies and equilibrium distributions provided by the conventional UWHAM (or MBAR) solutions exhibit considerable biases, but the estimates provided by Stratified-UWHAM and Stratified RE-SWHAM agree with the benchmark very well. Lastly, we discuss features of the Stratified-UWHAM approach which is based on coarse-graining in relation to two other maximum likelihood-based methods which were proposed recently, that also coarse-grain the multicanonical data.

Finding multiple reaction pathways via global optimization of action

Global searching for reaction pathways is a long-standing challenge in computational chemistry and biology. Most existing approaches perform only local searches due to computational complexity. Here we present a computational approach, Action-CSA, to find multiple diverse reaction pathways connecting fixed initial and final states through global optimization of the Onsager-Machlup action using the conformational space annealing (CSA) method. Action-CSA successfully overcomes large energy barriers via crossovers and mutations of pathways and finds all possible pathways of small systems without initial guesses on pathways. The rank order and the transition time distribution of multiple pathways are in good agreement with those of long Langevin dynamics simulations. The lowest action folding pathway of FSD-1 is consistent with recent experiments. The results show that Action-CSA is an efficient and robust computational approach to study the multiple pathways of complex reactions and large-scale conformational changes.

Role of Ito's lemma in sampling pinned diffusion paths in the continuous-time limit

We consider pinned diffusion paths that are explored by a particle moving via a conservative force while being in thermal equilibrium with its surroundings. To probe rare transitions, we use the Onsager-Machlup (OM) functional as a path probability distribution function for transition paths that are constrained to start and stop at predesignated points in different energy basins after a fixed time. The OM theory is based on a discrete-time version of Brownian dynamics, and thus it possesses a finite number of time steps. Here we explore the continuous-time limit where the number of time steps, and hence the dimensionality, becomes infinite. In this regime, the OM functional has been commonly regularized by using the Ito-Girsanov change of measure. This regularized form can then be used as a basis of a numerical algorithm to probe transition paths. In doing so, time again is discretized, progressing in fixed increments. When sampling paths, we find that numerical schemes based on this regularized continuous-time limit can fail catastrophically in describing the path of a particle moving in a potential with multiple wells. The origin of this behavior is traced to numerical instabilities in the discrete version of the continuous-time path measure that are not present in the infinite-dimensional limit. These instabilities arise because of the difficulty of satisfying, in finite dimensions, the conditions imposed by Ito's lemma that was an essential ingredient in the derivation of the regularized continuous-time measure. As an important consequence of this analysis, we conclude that the most probable diffusion path is not a physical entity because the thermodynamic action is effectively flat and cannot be minimized.

Multiscale enhanced path sampling based on the Onsager-Machlup action: application to a model polymer

We propose a novel path sampling method based on the Onsager-Machlup (OM) action by generalizing the multiscale enhanced sampling technique suggested by Moritsugu and co-workers [J. Chem. Phys. 133, 224105 (2010)]. The basic idea of this method is that the system we want to study (for example, some molecular system described by molecular mechanics) is coupled to a coarse-grained (CG) system, which can move more quickly and can be computed more efficiently than the original system. We simulate this combined system (original + CG system) using Langevin dynamics where different heat baths are coupled to the two systems. When the coupling is strong enough, the original system is guided by the CG system, and is able to sample the configuration and path space with more efficiency. We need to correct the bias caused by the coupling, however, by employing the Hamiltonian replica exchange, where we prepare many path replicas with different coupling strengths. As a result, an unbiased path ensemble for the original system can be found in the weakest coupling path ensemble. This strategy is easily implemented because a weight for a path calculated by the OM action is formally the same as the Boltzmann weight if we properly define the path "Hamiltonian." We apply this method to a model polymer with Asakura-Oosawa interaction, and compare the results with the conventional transition path sampling method.