Active learning in Gaussian process interpolation of potential energy surfaces

Training machine learning potentials for reactive systems: A Colab tutorial on basic models

In the last several years, there has been a surge in the development of machine learning potential (MLP) models for describing molecular systems. We are interested in a particular area of this field - the training of system-specific MLPs for reactive systems - with the goal of using these MLPs to accelerate free energy simulations of chemical and enzyme reactions. To help new members in our labs become familiar with the basic techniques, we have put together a self-guided Colab tutorial (https://cc-ats.github.io/mlp_tutorial/), which we expect to be also useful to other young researchers in the community. Our tutorial begins with the introduction of simple feedforward neural network (FNN) and kernel-based (using Gaussian process regression, GPR) models by fitting the two-dimensional Müller-Brown potential. Subsequently, two simple descriptors are presented for extracting features of molecular systems: symmetry functions (including the ANI variant) and embedding neural networks (such as DeepPot-SE). Lastly, these features will be fed into FNN and GPR models to reproduce the energies and forces for the molecular configurations in a Claisen rearrangement reaction.

AlF-AlF Reaction Dynamics between 200 K and 1000 K: Reaction Mechanisms and Intermediate Complex Characterization

AlF is a relevant molecule in astrochemistry as a tracer of F-bearing molecules. Additionally, AlF presents diagonal Franck-Condon factors and can be created very efficiently in the lab, which makes it a prototypical molecular for laser cooling. However, very little is known about the reaction dynamics of AlF. In this work, we report on the reaction dynamics of AlF-AlF between 200 and 1000 K using ab initio molecular dynamics and a highly efficient active learning approach for the potential energy surface, including all degrees of freedom. As a result, we identify the main reaction mechanisms and the lifetime of the intermediate complex AlF-AlF relevant to astrochemistry environments and regions in buffer gas cells.

Molecular dynamics-driven global potential energy surfaces: Application to the AlF dimer

In this work, we present a full-dimensional potential energy surface for AlF-AlF. We apply a general machine learning approach for full-dimensional potential energy surfaces, employing an active learning scheme trained on ab initio points, whose size grows based on the accuracy required. The training points are selected based on molecular dynamics simulations, choosing the most suitable configurations for different collision energy and mapping the most relevant part of the potential energy landscape of the system. The present approach does not require long-range information and is entirely general. As a result, it is possible to provide the full-dimensional AlF-AlF potential energy surface, requiring ≲0.01% of the configurations to be calculated ab initio. Furthermore, we analyze the general properties of the AlF-AlF system, finding critical differences with other reported results on CaF or bi-alkali dimers.

Bridging semiempirical and ab initio QM/MM potentials by Gaussian process regression and its sparse variants for free energy simulation

Free energy simulations that employ combined quantum mechanical and molecular mechanical (QM/MM) potentials at ab initio QM (AI) levels are computationally highly demanding. Here, we present a machine-learning-facilitated approach for obtaining AI/MM-quality free energy profiles at the cost of efficient semiempirical QM/MM (SE/MM) methods. Specifically, we use Gaussian process regression (GPR) to learn the potential energy corrections needed for an SE/MM level to match an AI/MM target along the minimum free energy path (MFEP). Force modification using gradients of the GPR potential allows us to improve configurational sampling and update the MFEP. To adaptively train our model, we further employ the sparse variational GP (SVGP) and streaming sparse GPR (SSGPR) methods, which efficiently incorporate previous sample information without significantly increasing the training data size. We applied the QM-(SS)GPR/MM method to the solution-phase SN2 Menshutkin reaction, NH3+CH3Cl→CH3NH3++Cl-, using AM1/MM and B3LYP/6-31+G(d,p)/MM as the base and target levels, respectively. For 4000 configurations sampled along the MFEP, the iteratively optimized AM1-SSGPR-4/MM model reduces the energy error in AM1/MM from 18.2 to 4.4 kcal/mol. Although not explicitly fitting forces, our method also reduces the key internal force errors from 25.5 to 11.1 kcal/mol/Å and from 30.2 to 10.3 kcal/mol/Å for the N-C and C-Cl bonds, respectively. Compared to the uncorrected simulations, the AM1-SSGPR-4/MM method lowers the predicted free energy barrier from 28.7 to 11.7 kcal/mol and decreases the reaction free energy from -12.4 to -41.9 kcal/mol, bringing these results into closer agreement with their AI/MM and experimental benchmarks.

Parallel Implementation of Nonadditive Gaussian Process Potentials for Monte Carlo Simulations

A strategy is presented to implement Gaussian process potentials in molecular simulations through parallel programming. Attention is focused on the three-body nonadditive energy, though all algorithms extend straightforwardly to the additive energy. The method to distribute pairs and triplets between processes is general to all potentials. Results are presented for a simulation box of argon, including full box and atom displacement calculations, which are relevant to Monte Carlo simulation. Data on speed-up are presented for up to 120 processes across four nodes. A 4-fold speed-up is observed over five processes, extending to 20-fold over 40 processes and 30-fold over 120 processes.

Globally Accurate Gaussian Process Potential Energy Surface and Quantum Dynamics Studies on the Li(2S) + Na2 → LiNa + Na Reaction at Low Collision Energies

The LiNa2 reactive system has recently received great attention in the experimental study of ultracold chemical reactions, but the corresponding theoretical calculations have not been carried out. Here, we report the first globally accurate ground-state LiNa2 potential energy surface (PES) using a Gaussian process model based on only 1776 actively selected high-level ab initio training points. The constructed PES had high precision and strong generalization capability. On the new PES, the quantum dynamics calculations on the Li(2S) + Na2(v = 0, j = 0) → LiNa + Na reaction were carried out in the 0.001-0.01 eV collision energy range using an improved time-dependent wave packet method. The calculated results indicate that this reaction is dominated by a complex-forming mechanism at low collision energies. The presented dynamics data provide guidance for experimental research, and the newly constructed PES could be further used for ultracold reaction dynamics calculations on this reactive system.

Accurate Prediction of Three-Body Intermolecular Interactions via Electron Deformation Density-Based Machine Learning

This work extends the electron deformation density-based descriptor, originally developed in the electron deformation density-based interaction energy machine learning (EDDIE-ML) algorithm to predict dimer interaction energies, to the prediction of three-body interactions in trimers. Using a sequential learning process to select the training data, the resulting Gaussian process regression (GPR) model predicts the three-body interaction energy within 0.2 kcal mol^-1 of the SRS-MP2/cc-pVTZ reference values for the 3B69 and S22-3 trimer data sets. A hybrid kernel function is introduced, which combines contributions from the average and individual atomic environments, allowing the total trimer interaction energy to be predicted in addition to the three-body contribution using the same descriptor. To extend the range and diversity of trimer interaction energies available in the literature, a new data set based on a protein-ligand crystal structure is introduced, consisting of 509 structures of a central ligand with two protein fragments. Benchmark calculations are provided for the new data set, which contains significantly larger molecular interactions than current databases in the literature in addition to charged fragments. Compared to density funtional theory (DFT)- and wavefunction-based methods for calculating the three-body interaction energy, our model makes predictions in a significantly shorter time frame by reducing the number of required SCF calculations from 7 to 4 performed at the PBE0 level of theory, showcasing the utility and efficiency of our Δ-ML method particularly when applied to larger systems.

Reliable emulation of complex functionals by active learning with error control

A statistical emulator can be used as a surrogate of complex physics-based calculations to drastically reduce the computational cost. Its successful implementation hinges on an accurate representation of the nonlinear response surface with a high-dimensional input space. Conventional "space-filling" designs, including random sampling and Latin hypercube sampling, become inefficient as the dimensionality of the input variables increases, and the predictive accuracy of the emulator can degrade substantially for a test input distant from the training input set. To address this fundamental challenge, we develop a reliable emulator for predicting complex functionals by active learning with error control (ALEC). The algorithm is applicable to infinite-dimensional mapping with high-fidelity predictions and a controlled predictive error. The computational efficiency has been demonstrated by emulating the classical density functional theory (cDFT) calculations, a statistical-mechanical method widely used in modeling the equilibrium properties of complex molecular systems. We show that ALEC is much more accurate than conventional emulators based on the Gaussian processes with "space-filling" designs and alternative active learning methods. In addition, it is computationally more efficient than direct cDFT calculations. ALEC can be a reliable building block for emulating expensive functionals owing to its minimal computational cost, controllable predictive error, and fully automatic features.

Fast atomic structure optimization with on-the-fly sparse Gaussian process potentials. JOURNAL OF PHYSICS. CONDENSED MATTER : AN INSTITUTE OF PHYSICS JOURNAL 2022;34:344007. [PMID: 35675808 DOI: 10.1088/1361-648x/ac76ff]

We apply on-the-fly machine learning potentials (MLPs) using the sparse Gaussian process regression (SGPR) algorithm for fast optimization of atomic structures. Great acceleration is achieved even in the context of a single local optimization. Although for finding the exact local minimum, due to limited accuracy of MLPs, switching to another algorithm may be needed. For random gold clusters, the forces are reduced to ∼0.1 eV Å^-1within less than ten first-principles (FP) calculations. Because of highly transferable MLPs, this algorithm is specially suitable for global optimization methods such as random or evolutionary structure searching or basin hopping. This is demonstrated by sequential optimization of random gold clusters for which, after only a few optimizations, FP calculations were rarely needed.

Machine learning for non-additive intermolecular potentials: quantum chemistry to first-principles predictions

Prediction of thermophysical properties from molecular principles requires accurate potential energy surfaces (PES). We present a widely-applicable method to produce first-principles PES from quantum chemistry calculations. Our approach accurately interpolates three-body non-additive interaction data, using the machine learning technique, Gaussian Processes (GP). The GP approach needs no bespoke modification when the number or type of molecules is changed. Our method produces highly accurate interpolation from significantly fewer training points than typical approaches, meaning ab initio calculations can be performed at higher accuracy. As an exemplar we compute the PES for all three-body cross interactions for CO₂-Ar mixtures. From these we calculate the CO₂-Ar virial coefficients up to 5th order. The resulting virial equation of state (EoS) is convergent for densities up to the critical density. Where convergent, the EoS makes accurate first-principles predictions for a range of thermophysical properties for CO₂-Ar mixtures, including the compressibility factor, speed of sound and Joule-Thomson coefficient. Our method has great potential to make wide-ranging first-principles predictions for mixtures of comparably sized molecules. Such predictions can replace the need for expensive, laborious and repetitive experiments and inform the continuum models required for applications.

Gaussian Process Regression and Machine Learning Methods for Carbon-Based Material Adsorption

Antibiotics have received a lot of attention as promising contaminants because of their ecotoxicological and long-term chemical stability in the atmosphere. Antibiotic adsorption on carbon-based materials (CBMs) such as charcoal and activated carbon has been identified as mainly effective for treating the wastewater strategies. Machine learning (ML) approaches were used to create generalized computation methods for tetracycline (TC) and sulfamethoxazole (SMX) adsorption in CBMs in this investigation. In the existing system, random forest and ANN methods were used for TC and SMX for predicting the quantities of antibiotics in the CBMs. For reducing the antibiotics from the industrial wastewater, the broadcast efforts of the experiments are a little complicated. In the proposed method, Gaussian process regression (GPR), active learning (AL), and ANN are used for predicting the antibiotic levels in the industrial wastewater. Below a variety of environmental parameters (e.g., warmth, solution pH) and adsorbent varieties, the created Ml algorithms outperformed classic isotherm models in conditions of generalisation. To evaluate TC and SMX adsorption on CBMs, we used comparative significance investigation and partial trust plots based on ML models. The proposed GPR reduces the antibiotics in wastewater; minimal experimental screening and the comparative significance and partial trust plot help in the treatment of wastewater.

Representing Globally Accurate Reactive Potential Energy Surfaces with Complex Topography by Combining Gaussian Process Regression and Neural Network

There has been increasing attention in using machine learning technologies, such as neural network (NN) and Gaussian process regression (GPR), to model multidimensional potential energy surfaces (PESs). NN PES features...

Gaussian process models of potential energy surfaces with boundary optimization

A strategy is outlined to reduce the number of training points required to model intermolecular potentials using Gaussian processes, without reducing accuracy. An asymptotic function is used at a long range, and the crossover distance between this model and the Gaussian process is learnt from the training data. The results are presented for different implementations of this procedure, known as boundary optimization, across the following dimer systems: CO-Ne, HF-Ne, HF-Na⁺, CO₂-Ne, and (CO₂)₂. The technique reduces the number of training points, at fixed accuracy, by up to ∼49%, compared to our previous work based on a sequential learning technique. The approach is readily transferable to other statistical methods of prediction or modeling problems.

Active learning of potential-energy surfaces of weakly bound complexes with regression-tree ensembles

Several pool-based active learning (AL) algorithms were employed to model potential-energy surfaces (PESs) with a minimum number of electronic structure calculations. Theoretical and empirical results suggest that superior strategies can be obtained by sampling molecular structures corresponding to large uncertainties in their predictions while at the same time not deviating much from the true distribution of the data. To model PESs in an AL framework, we propose to use a regression version of stochastic query by forest, a hybrid method that samples points corresponding to large uncertainties while avoiding collecting too many points from sparse regions of space. The algorithm is implemented with decision trees that come with relatively small computational costs. We empirically show that this algorithm requires around half the data to converge to the same accuracy in comparison to the uncertainty-based query-by-committee algorithm. Moreover, the algorithm is fully automatic and does not require any prior knowledge of the PES. Simulations on a 6D PES of pyrrole(H₂O) show that <15 000 configurations are enough to build a PES with a generalization error of 16 cm^-1, whereas the final model with around 50 000 configurations has a generalization error of 11 cm^-1.

A full-dimensional ab initio intermolecular potential energy surface and rovibrational spectra for OC-HF and OC-DF

We present a full-dimensional ab initio intermolecular potential energy surface (IPES) for the OC-HF van der Waals complex. 3167 ab initio points were computed at the frozen-core (FC) explicitly correlated coupled cluster [FC-CCSD(T)-F12b] level, with the augmented correlation-consistent polarized valence quadruple-zeta basis set plus bond functions. Basis set superposition error correction was also considered by the full counterpoise procedure. Gaussian process regression (GPR) was used to map out the potential energy surface, while a multipole expansion method was employed to smooth the ab initio noise of intermolecular potential in the long range. The global minimum of -1248.364 cm^-1 was located at the linear configuration with the C atom pointing toward the H atom of the HF molecule. In addition, a local minimum of -602.026 cm^-1 was found at another linear configuration with the O atom pointing toward the H atom of the HF molecule. The eigenstates were calculated on the vibrational averaged four-dimensional IPESs with the mixed radial discrete variable representation/angular finite basis representation method and Lanczos propagation algorithm. The dissociation energy D₀ was calculated to be 701.827 cm^-1, well reproducing the experimental value of 732 ± 2 cm^-1. The dipole moment surfaces were also fitted by GPR from 3132 ab initio points calculated using the coupled cluster method [CCSD(T)] with AVTZ basis set plus bond functions. The frequencies and relative line intensities of rovibrational transitions in the HF (DF) and CO stretching bands were further calculated and compared well with the experimental results. These results indicate the high fidelity of the new IPES.

Machine Learning for Electronically Excited States of Molecules

Electronically excited states of molecules are at the heart of photochemistry, photophysics, as well as photobiology and also play a role in material science. Their theoretical description requires highly accurate quantum chemical calculations, which are computationally expensive. In this review, we focus on not only how machine learning is employed to speed up such excited-state simulations but also how this branch of artificial intelligence can be used to advance this exciting research field in all its aspects. Discussed applications of machine learning for excited states include excited-state dynamics simulations, static calculations of absorption spectra, as well as many others. In order to put these studies into context, we discuss the promises and pitfalls of the involved machine learning techniques. Since the latter are mostly based on quantum chemistry calculations, we also provide a short introduction into excited-state electronic structure methods and approaches for nonadiabatic dynamics simulations and describe tricks and problems when using them in machine learning for excited states of molecules.

Machine Learning for Electronically Excited States of Molecules

Electronically excited states of molecules are at the heart of photochemistry, photophysics, as well as photobiology and also play a role in material science. Their theoretical description requires highly accurate quantum chemical calculations, which are computationally expensive. In this review, we focus on not only how machine learning is employed to speed up such excited-state simulations but also how this branch of artificial intelligence can be used to advance this exciting research field in all its aspects. Discussed applications of machine learning for excited states include excited-state dynamics simulations, static calculations of absorption spectra, as well as many others. In order to put these studies into context, we discuss the promises and pitfalls of the involved machine learning techniques. Since the latter are mostly based on quantum chemistry calculations, we also provide a short introduction into excited-state electronic structure methods and approaches for nonadiabatic dynamics simulations and describe tricks and problems when using them in machine learning for excited states of molecules.

Searching Configurations in Uncertainty Space: Active Learning of High-Dimensional Neural Network Reactive Potentials

Neural network (NN) potential energy surfaces (PESs) have been widely used in atomistic simulations with ab initio accuracy. While constructing NN PESs, their training data points are often sampled by molecular dynamics trajectories. This strategy can be however inefficient for reactive systems involving rare events. Here, we develop an uncertainty-driven active learning strategy to automatically and efficiently generate high-dimensional NN-based reactive potentials, taking a gas-surface reaction as an example. The difference between two independent NN models is used as a simple and differentiable uncertainty metric, allowing us to quickly search in the uncertainty space and place new samples at which the PES is less reliable. By interfacing this algorithm with the first-principles simulation package, we demonstrate that a globally accurate NN potential of the H₂ + Ag(111) system can be constructed with merely ∼150 data points. This PES can be further refined to describe H₂ dissociation on Ag(100) by adding ∼130 more configurations on this facet. The entire process is completely automatic and self-terminated once the relative error criterion is fulfilled. Impressively, data points sampled by this uncertainty-driven strategy are substantially fewer than by the traditional trajectory-based sampling. The final NN PES not only converges well the quantum dissociation probability of the molecule but also well-reproduces the phonon properties of the substrate and is capable of describing surface temperature effects. These results show the potential of this active learning approach in developing high-dimensional NN reactive potentials in gas and condensed phases.

Spectroscopy and Scattering Studies Using Interpolated Ab Initio Potentials

The Born-Oppenheimer potential energy surface (PES) has come a long way since its introduction in the 1920s, both conceptually and in predictive power for practical applications. Nevertheless, nearly 100 years later-despite astonishing advances in computational power-the state-of-the-art first-principles prediction of observables related to spectroscopy and scattering dynamics is surprisingly limited. For example, the water dimer, (H₂O)₂, with only six nuclei and 20 electrons, still presents a formidable challenge for full-dimensional variational calculations of bound states and is considered out of reach for rigorous scattering calculations. The extremely poor scaling of the most rigorous quantum methods is fundamental; however, recent progress in development of approximate methodologies has opened the door to fairly routine high-quality predictions, unthinkable 20 years ago. In this review, in relation to the workflow of spectroscopy and/or scattering studies, we summarize progress and challenges in the component areas of electronic structure calculations, PES fitting, and quantum dynamical calculations.

Kinetic study of the OH + HO2 → H2O + O2 reaction using ring polymer molecular dynamics and quantum dynamics

The reaction OH + HO2 → H2O + O2 is a prototype of radical-radical reactions. It plays an important role in interstellar/atmospheric chemistry and combustion, and considerable attention has thus been dedicated to its kinetics. In our previous work, we reported an accurate full-dimensional potential energy surface for the title reaction on the ground triplet electronic state. The quasi-classical trajectory (QCT) approach was employed to investigate its kinetics. Although the QCT rate coefficients were in good agreement with some experimental and theoretical results, QCT cannot account for the quantum mechanical effects, such as zero-point vibrational energy, recrossing, and tunneling, which may significantly affect the rate coefficients, particularly at low temperatures. In this work, the reduced-dimensional quantum dynamics and ring polymer molecular dynamics calculations were carried out to examine these effects and their impact on rate coefficients over the temperature range of 300-1300 K.

High-Fidelity Potential Energy Surfaces for Gas-Phase and Gas-Surface Scattering Processes from Machine Learning

In this Perspective, we review recent advances in constructing high-fidelity potential energy surfaces (PESs) from discrete ab initio points, using machine learning tools. Such PESs, albeit with substantial initial investments, provide significantly higher efficiency than direct dynamics methods and/or high accuracy at a level that is not affordable by on-the-fly approaches. These PESs not only are a necessity for quantum dynamical studies because of delocalization of wave packets but also enable the study of low-probability and long-time events in (quasi-)classical treatments. Our focus here is on inelastic and reactive scattering processes, which are more challenging than bound systems because of the involvement of continua. Relevant applications and developments for dynamical processes in both the gas phase and at gas-surface interfaces are discussed.

Quantum Chemistry in the Age of Machine Learning

As the quantum chemistry (QC) community embraces machine learning (ML), the number of new methods and applications based on the combination of QC and ML is surging. In this Perspective, a view of the current state of affairs in this new and exciting research field is offered, challenges of using machine learning in quantum chemistry applications are described, and potential future developments are outlined. Specifically, examples of how machine learning is used to improve the accuracy and accelerate quantum chemical research are shown. Generalization and classification of existing techniques are provided to ease the navigation in the sea of literature and to guide researchers entering the field. The emphasis of this Perspective is on supervised machine learning.

Predicting aggregation energy for single atom bimetallic catalysts on clean and O* adsorbed surfaces through machine learning models

Machine learning models are successfully developed for simultaneous prediction of stability and adsorption energy at single-atom bimetallic sites.

Bayesian machine learning for quantum molecular dynamics

This article discusses applications of Bayesian machine learning for quantum molecular dynamics.