Searching Configurations in Uncertainty Space: Active Learning of High-Dimensional Neural Network Reactive Potentials

In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back

Computational chemistry is an indispensable tool for understanding molecules and predicting chemical properties. However, traditional computational methods face significant challenges due to the difficulty of solving the Schrödinger equations and the increasing computational cost with the size of the molecular system. In response, there has been a surge of interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to in silico experiments. Integrating AI and ML into computational chemistry increases the scalability and speed of the exploration of chemical space. However, challenges remain, particularly regarding the reproducibility and transferability of ML models. This review highlights the evolution of ML in learning from, complementing, or replacing traditional computational chemistry for energy and property predictions. Starting from models trained entirely on numerical data, a journey set forth toward the ideal model incorporating or learning the physical laws of quantum mechanics. This paper also reviews existing computational methods and ML models and their intertwining, outlines a roadmap for future research, and identifies areas for improvement and innovation. Ultimately, the goal is to develop AI architectures capable of predicting accurate and transferable solutions to the Schrödinger equation, thereby revolutionizing in silico experiments within chemistry and materials science.

Collaboration on Machine-Learned Potentials with IPSuite: A Modular Framework for Learning-on-the-Fly

The field of machine learning potentials has experienced a rapid surge in progress, thanks to advances in machine learning theory, algorithms, and hardware capabilities. While the underlying methods are continuously evolving, the infrastructure for their deployment has lagged. The community, due to these rapid developments, frequently finds itself split into groups built around different implementations of machine-learned potentials. In this work, we introduce IPSuite, a Python-driven software package designed to connect different methods and algorithms from the comprehensive field of machine-learned potentials into a single platform while also providing a collaborative infrastructure, helping ensure reproducibility. Furthermore, the data management infrastructure of the IPSuite code enables simple model sharing and deployment in simulations. Currently, IPSuite supports six state-of-the-art machine learning approaches for the fitting of interatomic potentials as well as a variety of methods for the selection of training data, running of ab initio calculations, learning-on-the-fly strategies, model evaluation, and simulation deployment.

Local-environment-guided selection of atomic structures for the development of machine-learning potentials

Machine learning potentials (MLPs) have attracted significant attention in computational chemistry and materials science due to their high accuracy and computational efficiency. The proper selection of atomic structures is crucial for developing reliable MLPs. Insufficient or redundant atomic structures can impede the training process and potentially result in a poor quality MLP. Here, we propose a local-environment-guided screening algorithm for efficient dataset selection in MLP development. The algorithm utilizes a local environment bank to store unique local environments of atoms. The dissimilarity between a particular local environment and those stored in the bank is evaluated using the Euclidean distance. A new structure is selected only if its local environment is significantly different from those already present in the bank. Consequently, the bank is then updated with all the new local environments found in the selected structure. To demonstrate the effectiveness of our algorithm, we applied it to select structures for a Ge system and a Pd13H2 particle system. The algorithm reduced the training data size by around 80% for both without compromising the performance of the MLP models. We verified that the results were independent of the selection and ordering of the initial structures. We also compared the performance of our method with the farthest point sampling algorithm, and the results show that our algorithm is superior in both robustness and computational efficiency. Furthermore, the generated local environment bank can be continuously updated and can potentially serve as a growing database of feature local environments, aiding in efficient dataset maintenance for constructing accurate MLPs.

Combining Molecular Quantum Mechanical Modeling and Machine Learning for Accelerated Reaction Screening and Discovery

Molecular quantum mechanical modeling, accelerated by machine learning, has opened the door to high-throughput screening campaigns of complex properties, such as the activation energies of chemical reactions and absorption/emission spectra of materials and molecules; in silico. Here, we present an overview of the main principles, concepts, and design considerations involved in such hybrid computational quantum chemistry/machine learning screening workflows, with a special emphasis on some recent examples of their successful application. We end with a brief outlook of further advances that will benefit the field.

Modeling Equilibration Dynamics of Hyperthermal Products of Surface Reactions Using Scalable Neural Network Potential with First-Principles Accuracy

Equilibration dynamics of hot oxygen atoms following the dissociation of O2 on Pd(100) and Pd(111) surfaces are investigated by molecular dynamics simulations based on a scalable neural network potential enabling first-principles description of the interaction of O2 and O interacting with variable Pd supercells. By analyzing hundreds of trajectories with appropriate initial sampling, the measured distance distribution of equilibrated atom pairs on Pd(111) is well reproduced. However, our results on Pd(100) suggest that the ballistic motion of hot atoms predicted previously is a rare event under ideal conditions, while initial molecular orientation and surface thermal fluctuation could significantly affect the overall postdissociation dynamics. On both surfaces, dissociated hyperthermal oxygen atoms primarily locate their nascent positions and experience a similar random walk motion nearby.

Ring Polymer Molecular Dynamics Approach to Quantum Dissociative Chemisorption Rates

A ring polymer molecular dynamics (RPMD) method is proposed for the calculation of the dissociative chemisorption rate coefficient on surfaces. The RPMD rate theory is capable of handling quantum effects such as the zero-point energy and tunneling in dissociative chemisorption, while it relies on classical trajectories for the simulation. Applications to H2 dissociative chemisorption are demonstrated. For the highly activated process on Ag(111), strong deviations from Arrhenius behavior are found at low temperatures and attributed to tunneling. On Pt(111), where the dissociation has a barrierless pathway, the RPMD rate coefficient is found to agree with the experimentally derived thermal sticking coefficient within a factor of 2 over a large temperature range. Significant quantum effects are also found.

Parametrically Managed Activation Function for Fitting a Neural Network Potential with Physical Behavior Enforced by a Low-Dimensional Potential

Machine-learned representations of potential energy surfaces generated in the output layer of a feedforward neural network are becoming increasingly popular. One difficulty with neural network output is that it is often unreliable in regions where training data is missing or sparse. Human-designed potentials often build in proper extrapolation behavior by choice of functional form. Because machine learning is very efficient, it is desirable to learn how to add human intelligence to machine-learned potentials in a convenient way. One example is the well-understood feature of interaction potentials that they vanish when subsystems are too far separated to interact. In this article, we present a way to add a new kind of activation function to a neural network to enforce low-dimensional constraints. In particular, the activation function depends parametrically on all of the input variables. We illustrate the use of this step by showing how it can force an interaction potential to go to zero at large subsystem separations without either inputting a specific functional form for the potential or adding data to the training set in the asymptotic region of geometries where the subsystems are separated. In the process of illustrating this, we present an improved set of potential energy surfaces for the 14 lowest ³A' states of O₃. The method is more general than this example, and it may be used to add other low-dimensional knowledge or lower-level knowledge to machine-learned potentials. In addition to the O₃ example, we present a greater-generality method called parametrically managed diabatization by deep neural network (PM-DDNN) that is an improvement on our previously presented permutationally restrained diabatization by deep neural network (PR-DDNN).

First-principles surface reaction rates by ring polymer molecular dynamics and neural network potential: role of anharmonicity and lattice motion

Elementary gas-surface processes are essential steps in heterogeneous catalysis. A predictive understanding of catalytic mechanisms remains challenging due largely to difficulties in accurately characterizing the kinetics of such steps. Experimentally, thermal rates for elementary surface reactions can now be measured using a novel velocity imaging technique, providing a stringent testing ground for ab initio rate theories. Here, we propose to combine ring polymer molecular dynamics (RPMD) rate theory with state-of-the-art first-principles-determined neural network potential to calculate surface reaction rates. Taking NO desorption from Pd(111) as an example, we show that the harmonic approximation and the neglect of lattice motion in the commonly-used transition state theory overestimates and underestimates the entropy change during the desorption process, respectively, leading to opposite errors in rate coefficient predictions and artificial error cancellations. Including anharmonicity and lattice motion, our results reveal a generally neglected surface entropy change due to significant local structural change during desorption and obtain the right answer for the right reasons. Although quantum effects are found to be less important in this system, the proposed approach establishes a more reliable theoretical benchmark for accurately predicting the kinetics of elementary gas-surface processes.

Globally Accurate Gaussian Process Potential Energy Surface and Quantum Dynamics Studies on the Li(2S) + Na2 → LiNa + Na Reaction at Low Collision Energies

The LiNa2 reactive system has recently received great attention in the experimental study of ultracold chemical reactions, but the corresponding theoretical calculations have not been carried out. Here, we report the first globally accurate ground-state LiNa2 potential energy surface (PES) using a Gaussian process model based on only 1776 actively selected high-level ab initio training points. The constructed PES had high precision and strong generalization capability. On the new PES, the quantum dynamics calculations on the Li(2S) + Na2(v = 0, j = 0) → LiNa + Na reaction were carried out in the 0.001-0.01 eV collision energy range using an improved time-dependent wave packet method. The calculated results indicate that this reaction is dominated by a complex-forming mechanism at low collision energies. The presented dynamics data provide guidance for experimental research, and the newly constructed PES could be further used for ultracold reaction dynamics calculations on this reactive system.

Active sampling for neural network potentials: Accelerated simulations of shear-induced deformation in Cu-Ni multilayers

Neural network potentials (NNPs) can greatly accelerate atomistic simulations relative to ab initio methods, allowing one to sample a broader range of structural outcomes and transformation pathways. In this work, we demonstrate an active sampling algorithm that trains an NNP that is able to produce microstructural evolutions with accuracy comparable to those obtained by density functional theory, exemplified during structure optimizations for a model Cu-Ni multilayer system. We then use the NNP, in conjunction with a perturbation scheme, to stochastically sample structural and energetic changes caused by shear-induced deformation, demonstrating the range of possible intermixing and vacancy migration pathways that can be obtained as a result of the speedups provided by the NNP. The code to implement our active learning strategy and NNP-driven stochastic shear simulations is openly available at https://github.com/pnnl/Active-Sampling-for-Atomistic-Potentials.

Accelerated Quantum Mechanics/Molecular Mechanics Simulations via Neural Networks Incorporated with Mechanical Embedding Scheme

A powerful tool to study the mechanism of reactions in solutions or enzymes is to perform the ab initio quantum mechanical/molecular mechanical (QM/MM) molecular dynamics (MD) simulations. However, the computational cost is too high due to the explicit electronic structure calculations at every time step of the simulation. A neural network (NN) method can accelerate the QM/MM-MD simulations, but it has long been a problem to accurately describe the QM/MM electrostatic coupling by NN in the electrostatic embedding (EE) scheme. In this work, we developed a new method to accelerate QM/MM calculations in the mechanic embedding (ME) scheme. The potentials and partial point charges of QM atoms are first learned in vacuo by the embedded atom neural networks (EANN) approach. MD simulations are then performed on this EANN/MM potential energy surface (PES) to obtain free energy (FE) profiles for reactions, in which the QM/MM electrostatic coupling is treated in the mechanic embedding (ME) scheme. Finally, a weighted thermodynamic perturbation (wTP) corrects the FE profiles in the ME scheme to the EE scheme. For two reactions in water and one in methanol, our simulations reproduced the B3LYP/MM free energy profiles within 0.5 kcal/mol with a speed-up of 30-60-fold. The results show that the strategy of combining EANN potential in the ME scheme with the wTP correction is efficient and reliable for chemical reaction simulations in liquid. Another advantage of our method is that the QM PES is independent of the MM subsystem, so it can be applied to various MM environments as demonstrated by an S_N2 reaction studied in water and methanol individually, which used the same EANN PES. The free energy profiles are in excellent accordance with the results obtained from B3LYP/MM-MD simulations. In future, this method will be applied to the reactions of enzymes and their variants.

Atomic-Scale Mechanism of Platinum Catalyst Restructuring under a Pressure of Reactant Gas

Heterogeneous catalysis is key for chemical transformations. Understanding how catalysts' active sites dynamically evolve at the atomic scale under reaction conditions is a prerequisite for accurately determining catalytic mechanisms and predictably developing catalysts. We combine in situ time-dependent scanning tunneling microscopy observations and machine-learning-accelerated first-principles atomistic simulations to uncover the mechanism of restructuring of Pt catalysts under a pressure of carbon monoxide (CO). We show that a high CO coverage at a Pt step edge triggers the formation of atomic protrusions of low-coordination Pt atoms, which then detach from the step edge to create sub-nano-islands on the terraces, where under-coordinated sites are stabilized by the CO adsorbates. The fast and accurate machine-learning potential is key to enabling the exploration of tens of thousands of configurations for the CO-covered restructuring catalyst. These studies open an avenue to achieve an atomic-scale understanding of the structural dynamics of more complex metal nanoparticle catalysts under reaction conditions.

The quantum dynamics of H2 on Cu(111) at a surface temperature of 925 K: Comparing state-of-the-art theory to state-of-the-art experiments 2

State-of-the-art 6D quantum dynamics simulations for the dissociative chemisorption of H₂ on a thermally distorted Cu(111) surface, using the static corrugation model, were analyzed to produce several (experimentally available) observables. The expected error, especially important for lower reaction probabilities, was quantified using wavepackets on several different grids as well as two different analysis approaches to obtain more accurate results in the region where a slow reaction channel was experimentally shown to be dominant. The lowest reaction barrier sites for different thermally distorted surface slabs are shown to not just be energetically, but also geometrically, different between surface configurations, which can be used to explain several dynamical effects found when including surface temperature effects. Direct comparison of simulated time-of-flight spectra to those obtained from state-of-the-art desorption experiments showed much improved agreement compared to the perfect lattice BOSS approach. Agreement with experimental rotational and vibrational efficacies also somewhat improved when thermally excited surfaces were included in the theoretical model. Finally, we present clear quantum effects in the rotational quadrupole alignment parameters found for the lower rotationally excited states, which underlines the importance of careful quantum dynamical analyses of this system.

Accurate Description of the Quantum Dynamical Surface Temperature Effects on the Dissociative Chemisorption of H2 from Cu(111)

Accurately describing surface temperature effects for the dissociative scattering of H2 on a metal surface on a quantum dynamical level is currently one of the open challenges for theoretical surface scientists. We present the first quantum dynamical (QD) simulations of hydrogen dissociating on a Cu(111) surface which accurately describe all relevant surface temperature effects, using the static corrugation model (SCM). The reaction probabilities we obtain show very good agreement with those found using quasi-classical dynamics (QCD), both for individual surface slabs and for an averaged, thus Monte-Carlo sampled, set of thermally distorted surface configurations. Rovibrationally elastic scattering probabilities show a much clearer difference between the QCD and QD results, which appears to be traceable back towards thermally distorted surface configurations with very low dissociation probabilities and underlines the importance of investigating more observables than just dissociation. By reducing the number of distorted surface atoms included in the dynamical model, we also show that only including one, or even three, surface atoms is generally not enough to accurately describe the effects of surface temperature on dissociation and elastic scattering. These results are a major step forward in accurately describing hydrogen scattering from a thermally excited Cu(111) surface, and open up a pathway to better describe reaction and scattering from other relevant crystal facets, such as stepped surfaces, at moderately elevated surface temperatures where quantum effects are expected to play a more important role in the dissociation of H2 on Cu.

Neural Network Potentials: A Concise Overview of Methods

In the past two decades, machine learning potentials (MLPs) have reached a level of maturity that now enables applications to large-scale atomistic simulations of a wide range of systems in chemistry, physics, and materials science. Different machine learning algorithms have been used with great success in the construction of these MLPs. In this review, we discuss an important group of MLPs relying on artificial neural networks to establish a mapping from the atomic structure to the potential energy. In spite of this common feature, there are important conceptual differences among MLPs, which concern the dimensionality of the systems, the inclusion of long-range electrostatic interactions, global phenomena like nonlocal charge transfer, and the type of descriptor used to represent the atomic structure, which can be either predefined or learnable. A concise overview is given along with a discussion of the open challenges in the field.

Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Representing Globally Accurate Reactive Potential Energy Surfaces with Complex Topography by Combining Gaussian Process Regression and Neural Network

There has been increasing attention in using machine learning technologies, such as neural network (NN) and Gaussian process regression (GPR), to model multidimensional potential energy surfaces (PESs). NN PES features...

Automated Construction of Neural Network Potential Energy Surface: The Enhanced Self-Organizing Incremental Neural Network Deep Potential Method

In recent years, the use of deep learning (neural network) potential energy surface (NNPES) in molecular dynamics simulation has experienced explosive growth as it can be as accurate as quantum chemistry methods while being as efficient as classical mechanic methods. However, the development of NNPES is highly nontrivial. In particular, it has been troubling to construct a dataset that is as small as possible yet can cover the target chemical space. In this work, an ESOINN-DP method is developed, which has the enhanced self-organizing incremental neural network (ESOINN) and a newly proposed error indicator at its core. With ESOINN-DP, one can construct the NNPES with little human intervention, and this method ensures that the constructed reference dataset covers the target chemical space with minimum redundancy. The performance of the ESOINN-DP method has been well validated by developing neural network potential energy surfaces for water clusters, tripeptides, and by de-redundancy of a sub-dataset of the ANI-1 database. We believe that the ESOINN-DP method provides a novel idea for the construction of NNPES and, especially, the reference datasets, and it can be used for molecular dynamics (MD) simulations of various gas-phase and condensed-phase chemical systems.

Active learning of potential-energy surfaces of weakly bound complexes with regression-tree ensembles

Several pool-based active learning (AL) algorithms were employed to model potential-energy surfaces (PESs) with a minimum number of electronic structure calculations. Theoretical and empirical results suggest that superior strategies can be obtained by sampling molecular structures corresponding to large uncertainties in their predictions while at the same time not deviating much from the true distribution of the data. To model PESs in an AL framework, we propose to use a regression version of stochastic query by forest, a hybrid method that samples points corresponding to large uncertainties while avoiding collecting too many points from sparse regions of space. The algorithm is implemented with decision trees that come with relatively small computational costs. We empirically show that this algorithm requires around half the data to converge to the same accuracy in comparison to the uncertainty-based query-by-committee algorithm. Moreover, the algorithm is fully automatic and does not require any prior knowledge of the PES. Simulations on a 6D PES of pyrrole(H₂O) show that <15 000 configurations are enough to build a PES with a generalization error of 16 cm^-1, whereas the final model with around 50 000 configurations has a generalization error of 11 cm^-1.

Reduced-dimensional surface hopping with offline-online computations

Molecular dynamics simulations often classically evolve the nuclear geometry on adiabatic potential energy surfaces (PESs), punctuated by random hops between energy levels in regions of strong coupling, in an algorithm known as surface hopping. However, the computational expense of integrating the geometry on a full-dimensional PES and computing the required couplings can quickly become prohibitive as the number of atoms increases. In this work, we describe a method for surface hopping that uses only important reaction coordinates, performs all expensive evaluations of the true PESs and couplings only once before simulating dynamics (offline), and then queries the stored values during the surface hopping simulation (online). Our Python codes are freely available on GitHub. Using photodissociation of azomethane as a test case, this method is able to reproduce experimental results that have thus far eluded ab initio surface hopping studies.

Efficient Approximation of Potential Energy Surfaces with Mixed-Basis Interpolation

The potential energy surface (PES) describes the energy of a chemical system as a function of its geometry and is a fundamental concept in modern chemistry. A PES provides much useful information about the system, including the structures and energies of various stationary points, such as stable conformers (local minima) and transition states (first-order saddle points) connected by a minimum-energy path. Our group has previously produced surrogate reduced-dimensional PESs using sparse interpolation along chemically significant reaction coordinates, such as bond lengths, bond angles, and torsion angles. These surrogates used a single interpolation basis, either polynomials or trigonometric functions, in every dimension. However, relevant molecular dynamics (MD) simulations often involve some combination of both periodic and nonperiodic coordinates. Using a trigonometric basis on nonperiodic coordinates, such as bond lengths, leads to inaccuracies near the domain boundary. Conversely, polynomial interpolation on the periodic coordinates does not enforce the periodicity of the surrogate PES gradient, leading to nonconservation of total energy even in a microcanonical ensemble. In this work, we present an interpolation method that uses trigonometric interpolation on the periodic reaction coordinates and polynomial interpolation on the nonperiodic coordinates. We apply this method to MD simulations of possible isomerization pathways of azomethane between cis and trans conformers. This method is the only known interpolative method that appropriately conserves total energy in systems with both periodic and nonperiodic reaction coordinates. In addition, compared to all-polynomial interpolation, the mixed basis requires fewer electronic structure calculations to obtain a given level of accuracy, is an order of magnitude faster, and is freely available on GitHub.