Deep reinforcement learning of transition states

3DReact: Geometric Deep Learning for Chemical Reactions

Geometric deep learning models, which incorporate the relevant molecular symmetries within the neural network architecture, have considerably improved the accuracy and data efficiency of predictions of molecular properties. Building on this success, we introduce 3DReact, a geometric deep learning model to predict reaction properties from three-dimensional structures of reactants and products. We demonstrate that the invariant version of the model is sufficient for existing reaction data sets. We illustrate its competitive performance on the prediction of activation barriers on the GDB7-22-TS, Cyclo-23-TS, and Proparg-21-TS data sets in different atom-mapping regimes. We show that, compared to existing models for reaction property prediction, 3DReact offers a flexible framework that exploits atom-mapping information, if available, as well as geometries of reactants and products (in an invariant or equivariant fashion). Accordingly, it performs systematically well across different data sets, atom-mapping regimes, as well as both interpolation and extrapolation tasks.

Molecular Autonomous Pathfinder Using Deep Reinforcement Learning

Diffusion in solids is a slow process that dictates rate-limiting processes in key chemical reactions. Unlike crystalline solids that offer well-defined diffusion pathways, the lack of similar structural motifs in amorphous or glassy materials poses great challenges in bridging the slow diffusion process and material failures. To tackle this problem, we propose an AI-guided long-term atomistic simulation approach: molecular autonomous pathfinder (MAP) framework based on deep reinforcement learning (DRL), where the RL agent is trained to uncover energy efficient diffusion pathways. We employ a Deep Q-Network architecture with distributed prioritized replay buffer, enabling fully online agent training with accelerated experience sampling by an ensemble of asynchronous agents. After training, the agents provide atomistic configurations of diffusion pathways with their energy profile. We use a piecewise nudged elastic band to refine the energy profile of the obtained pathway and the corresponding diffusion time on the basis of transition-state theory. With the MAP framework, we demonstrate atomistic diffusion mechanisms in amorphous silica with time scales comparable to experiments.

Benchmarking machine-readable vectors of chemical reactions on computed activation barriers

In recent years, there has been a surge of interest in predicting computed activation barriers, to enable the acceleration of the automated exploration of reaction networks. Consequently, various predictive approaches have emerged, ranging from graph-based models to methods based on the three-dimensional structure of reactants and products. In tandem, many representations have been developed to predict experimental targets, which may hold promise for barrier prediction as well. Here, we bring together all of these efforts and benchmark various methods (Morgan fingerprints, the DRFP, the CGR representation-based Chemprop, SLATMd, B2Rl2, EquiReact and language model BERT + RXNFP) for the prediction of computed activation barriers on three diverse datasets.

Collective Variable-Based Enhanced Sampling: From Human Learning to Machine Learning

Enhanced-sampling algorithms relying on collective variables (CVs) are extensively employed to study complex (bio)chemical processes that are not amenable to brute-force molecular simulations. The selection of appropriate CVs characterizing the slow movement modes is of paramount importance for reliable and efficient enhanced-sampling simulations. In this Perspective, we first review the application and limitations of CVs obtained from chemical and geometrical intuition. We also introduce path-sampling algorithms, which can identify path-like CVs in a high-dimensional free-energy space. Machine-learning algorithms offer a viable approach to finding suitable CVs by analyzing trajectories from preliminary simulations. We discuss both the performance of machine-learning-derived CVs in enhanced-sampling simulations of experimental models and the challenges involved in applying these CVs to realistic, complex molecular assemblies. Moreover, we provide a prospective view of the potential advancements of machine-learning algorithms for the development of CVs in the field of enhanced-sampling simulations.

Reinforcement Learning for Traversing Chemical Structure Space: Optimizing Transition States and Minimum Energy Paths of Molecules

In recent years, deep learning has made remarkable strides, surpassing human capabilities in tasks, such as strategy games, and it has found applications in complex domains, including protein folding. In the realm of quantum chemistry, machine learning methods have primarily served as predictive tools or design aids using generative models, while reinforcement learning remains in its early stages of exploration. This work introduces an actor-critic reinforcement learning framework suitable for diverse optimization tasks, such as searching for molecular structures with specific properties within conformational spaces. As an example, we show an implementation of this scheme for calculating minimum energy pathways of a Claisen rearrangement reaction and a number of SN2 reactions. The results show that the algorithm is able to accurately predict minimum energy pathways and, thus, transition states, providing the first steps in using actor-critic methods to study chemical reactions.

Standard Binding Free-Energy Calculations: How Far Are We from Automation?

Recent success stories suggest that in silico protein-ligand binding free-energy calculations are approaching chemical accuracy. However, their widespread application remains limited by the extensive human intervention required, posing challenges for the neophyte. As such, it is critical to develop automated workflows for estimating protein-ligand binding affinities with minimum personal involvement. Key human efforts include setting up and tuning enhanced-sampling or alchemical-transformation algorithms as a preamble to computational binding free-energy estimations. Additionally, preparing input files, bookkeeping, and postprocessing represent nontrivial tasks. In this Perspective, we discuss recent progress in automating standard binding free-energy calculations, featuring the development of adaptive or parameter-free algorithms, standardization of binding free-energy calculation workflows, and the implementation of user-friendly software. We also assess the current state of automated standard binding free-energy calculations and evaluate the limitations of existing methods. Last, we outline the requirements for future algorithms and workflows to facilitate automated free-energy calculations for diverse protein-ligand complexes.

Accurate transition state generation with an object-aware equivariant elementary reaction diffusion model

Transition state search is key in chemistry for elucidating reaction mechanisms and exploring reaction networks. The search for accurate 3D transition state structures, however, requires numerous computationally intensive quantum chemistry calculations due to the complexity of potential energy surfaces. Here we developed an object-aware SE(3) equivariant diffusion model that satisfies all physical symmetries and constraints for generating sets of structures-reactant, transition state and product-in an elementary reaction. Provided reactant and product, this model generates a transition state structure in seconds instead of hours, which is typically required when performing quantum-chemistry-based optimizations. The generated transition state structures achieve a median of 0.08 Å root mean square deviation compared to the true transition state. With a confidence scoring model for uncertainty quantification, we approach an accuracy required for reaction barrier estimation (2.6 kcal mol-1) by only performing quantum chemistry-based optimizations on 14% of the most challenging reactions. We envision usefulness for our approach in constructing large reaction networks with unknown mechanisms.

A self-adapting first-principles exploration on the dissociation mechanism in sodiated aldohexose pyranoses assisted with neural network potentials

Understanding the mechanism of collision-induced dissociation (CID) in mono-saccharides with density functional theory (DFT) is challenging because of many possible reaction paths that originate from their high structural diversity. To search for the transition state (TS) from the huge number of conformers, we propose a three-step search scheme with the assistance of neural network potential (NNP). The search starts from a cross-checking of sugars, to a global search of all possible channels, and in the end, an exhaustive exploration around the low-lying channels. The cross-checking step quickly adapts the NNP from the studied molecules to the target ones. The other two steps utilize the adapted NNP to find the available pathways via random sampling of the structures. The study of the CID reactions in all eight types of aldohexose pyranoses was applied using the search scheme. The DFT calculations on AH-0 (Glc, Gal, and Man) in the previous study were utilized to construct an NNP and provide the TS structure database for searching AH-1 (All, Alt, Gul, Ido, and Tal). In total, we identified around 5200 TSs in AH-0 and AH-1, and the final NNP covers an energy range of more than 500 kJ mol^-1 with a mean absolute error of energy less than 4 kJ mol^-1. The search scheme is useful not only for saccharides but also for highly flexible bio-molecules.

Quantum Chemical Roots of Machine-Learning Molecular Similarity Descriptors

In this work, we explore the quantum chemical foundations of descriptors for molecular similarity. Such descriptors are key for traversing chemical compound space with machine learning. Our focus is on the Coulomb matrix and on the smooth overlap of atomic positions (SOAP). We adopt a basic framework that allows us to connect both descriptors to electronic structure theory. This framework enables us to then define two new descriptors that are more closely related to electronic structure theory, which we call Coulomb lists and smooth overlap of electron densities (SOED). By investigating their usefulness as molecular similarity descriptors, we gain new insights into how and why Coulomb matrix and SOAP work. Moreover, Coulomb lists avoid the somewhat mysterious diagonalization step of the Coulomb matrix and might provide a direct means to extract subsystem information that can be compared across Born-Oppenheimer surfaces of varying dimension. For the electron density, we derive the necessary formalism to create the SOED measure in close analogy to SOAP. Because this formalism is more involved than that of SOAP, we review the essential theory as well as introduce a set of approximations that eventually allow us to work with SOED in terms of the same implementation available for the evaluation of SOAP. We focus our analysis on elementary reaction steps, where transition state structures are more similar to either reactant or product structures than the latter two are with respect to one another. The prediction of electronic energies of transition state structures can, however, be more difficult than that of stable intermediates due to multi-configurational effects. The question arises to what extent molecular similarity descriptors rooted in electronic structure theory can resolve these intricate effects.

Locating Transition Zone in Phase Space

Understanding the reaction mechanism is required for better control of chemical reactions and is usually achieved by locating transition states (TSs) along a proper one-dimensional coordinate called reaction coordinate (RC). The identification of RC can be very difficult for high-dimensional realistic systems. A number of methods have been proposed to tackle this problem. A machine learning method is developed here to incorporate the influence of velocity on the reaction process. The method is also free of the unbalanced label problem resulting from the rather low fraction of configurations near the TS and can be easily extended to large systems. It locates the transition zone in the phase space and defines the dividing surface with a high transmission coefficient. Moreover, considering that the reaction environment can not only change the reaction path but also activate the reactive mode through energy transfer, we devise two measures to quantify the influence of these two factors on the reaction process and find that solvents can assist the reaction by directly doing work along the reactive mode. Not surprisingly, there is a positive correlation between the efficiency of energy transfer into the reactive mode and the reaction rate.

Explaining reaction coordinates of alanine dipeptide isomerization obtained from deep neural networks using Explainable Artificial Intelligence (XAI)

A method for obtaining appropriate reaction coordinates is required to identify transition states distinguishing product and reactant in complex molecular systems. Recently, abundant research has been devoted to obtaining reaction coordinates using artificial neural networks from deep learning literature, where many collective variables are typically utilized in the input layer. However, it is difficult to explain the details of which collective variables contribute to the predicted reaction coordinates owing to the complexity of the nonlinear functions in deep neural networks. To overcome this limitation, we used Explainable Artificial Intelligence (XAI) methods of the Local Interpretable Model-agnostic Explanation (LIME) and the game theory-based framework known as Shapley Additive exPlanations (SHAP). We demonstrated that XAI enables us to obtain the degree of contribution of each collective variable to reaction coordinates that is determined by nonlinear regressions with deep learning for the committor of the alanine dipeptide isomerization in vacuum. In particular, both LIME and SHAP provide important features to the predicted reaction coordinates, which are characterized by appropriate dihedral angles consistent with those previously reported from the committor test analysis. The present study offers an AI-aided framework to explain the appropriate reaction coordinates, which acquires considerable significance when the number of degrees of freedom increases.

Reinforcement learning of rare diffusive dynamics

We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, such as those relevant in the study of reactive events, and trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, such as those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.