Data-Efficient Machine Learning Potentials from Transfer Learning of Periodic Correlated Electronic Structure Methods: Liquid Water at AFQMC, CCSD, and CCSD(T) Accuracy

Periodic Local Coupled-Cluster Theory for Insulators and Metals

We describe the implementation details of periodic local coupled-cluster theory with single and double excitations (CCSD) and perturbative triple excitations [CCSD(T)] using local natural orbitals (LNOs) and k-point symmetry. We discuss and compare several choices for orbital localization, fragmentation, and LNO construction. By studying diamond and lithium, we demonstrate that periodic LNO-CC theory can be applied with equal success to both insulators and metals, achieving speedups of 2 to 3 orders of magnitude even for moderately sized k-point meshes. Our final predictions of the equilibrium cohesive energy, lattice constant, and bulk modulus for diamond and lithium are in good agreement with previous theoretical predictions and experimental results.

Transfer learning of hyperparameters for fast construction of anisotropic GPR models: design and application to the machine-learned force field FFLUX

The polarisable machine-learned force field FFLUX requires pre-trained anisotropic Gaussian process regression (GPR) models of atomic energies and multipole moments to propagate unbiased molecular dynamics simulations. The outcome of FFLUX simulations is highly dependent on the predictive accuracy of the underlying models whose training entails determining the optimal set of model hyperparameters. Unfortunately, traditional direct learning (DL) procedures do not scale well on this task, especially when the hyperparameter search is initiated from a (set of) random guess solution(s). Additionally, the complexity of the hyperparameter space (HS) increases with the number of geometrical input features, at least for anisotropic kernels, making the optimization of hyperparameters even more challenging. In this study, we propose a transfer learning (TL) protocol that accelerates the training process of anisotropic GPR models by facilitating access to promising regions of the HS. The protocol is based on a seeding-relaxation mechanism in which an excellent guess solution is identified by rapidly building one or several small source models over a subset of the target training set before readjusting the previous guess over the entire set. We demonstrate the performance of this protocol by building and assessing the performance of DL and TL models of atomic energies and charges in various conformations of benzene, ethanol, formic acid dimer and the drug fomepizole. Our experiments suggest that TL models can be built one order of magnitude faster while preserving the quality of their DL analogs. Most importantly, when deployed in FFLUX simulations, TL models compete with or even outperform their DL analogs when it comes to performing FFLUX geometry optimization and computing harmonic vibrational modes.

Five-Site Water Models for Ice and Liquid Water Generated by a Series-Parallel Machine Learning Strategy

Icing, a common natural phenomenon, always originates from a molecule. Molecular simulation is crucial for understanding the relevant process but still faces a great challenge in obtaining a uniform and accurate description of ice and liquid water with limited model parameters. Here, we propose a series-parallel machine learning (ML) approach consisting of a classification back-propagation neural network (BPNN), parallel regression BPNNs, and a genetic algorithm to establish conventional TIP5P-BG and temperature-dependent TIP5P-BGT models. The established water models exhibit a comprehensive balance among the crucial physical properties (melting point, density, vaporization enthalpy, self-diffusion coefficient, and viscosity) with mean absolute percentage errors of 2.65 and 2.40%, respectively, and excellent predictive performance on the related properties of liquid water. For ice, the simulation results on the critical nucleus size and growth rate are in good accordance with experiments. This work offers a powerful molecular model for phase transition and icing in nanoconfinement and a construction strategy for a complex molecular model in the extreme case.

Force-Free Identification of Minimum-Energy Pathways and Transition States for Stochastic Electronic Structure Theories

The accurate mapping of potential energy surfaces (PESs) is crucial to our understanding of the numerous physical and chemical processes mediated by atomic rearrangements, such as conformational changes and chemical reactions, and the thermodynamic and kinetic feasibility of these processes. Stochastic electronic structure theories, e.g., Quantum Monte Carlo (QMC) methods, enable highly accurate total energy calculations that in principle can be used to construct the PES. However, their stochastic nature poses a challenge to the computation and use of forces and Hessians, which are typically required in algorithms for minimum-energy pathway (MEP) and transition state (TS) identification, such as the nudged elastic band (NEB) algorithm and its climbing image formulation. Here, we present strategies that utilize the surrogate Hessian line-search method, previously developed for QMC structural optimization, to efficiently identify MEP and TS structures without requiring force calculations at the level of the stochastic electronic structure theory. By modifying the surrogate Hessian algorithm to operate in path-orthogonal subspaces and at saddle points, we show that it is possible to identify MEPs and TSs by using a force-free QMC approach. We demonstrate these strategies via two examples, the inversion of the ammonia (NH3) molecule and the nucleophilic substitution (SN2) reaction F- + CH3F → FCH3 + F-. We validate our results using Density Functional Theory (DFT)- and Coupled Cluster (CCSD, CCSD(T))-based NEB calculations. We then introduce a hybrid DFT-QMC approach to compute thermodynamic and kinetic quantities, free energy differences, rate constants, and equilibrium constants that incorporates stochastically optimized structures and their energies, and show that this scheme improves upon DFT accuracy. Our methods generalize straightforwardly to other systems and other high-accuracy theories that similarly face challenges computing energy gradients, paving the way for highly accurate PES mapping, transition state determination, and thermodynamic and kinetic calculations at significantly reduced computational expense.

Active sparse Bayesian committee machine potential for isothermal-isobaric molecular dynamics simulations

Recent advancements in machine learning potentials (MLPs) have significantly impacted the fields of chemistry, physics, and biology by enabling large-scale first-principles simulations. Among different machine learning approaches, kernel-based MLPs distinguish themselves through their ability to handle small datasets, quantify uncertainties, and minimize over-fitting. Nevertheless, their extensive computational requirements present considerable challenges. To alleviate these, sparsification methods have been developed, aiming to reduce computational scaling without compromising accuracy. In the context of isothermal and isobaric ML molecular dynamics (MD) simulations, achieving precise pressure estimation is crucial for reproducing reliable system behavior under constant pressure. Despite progress, sparse kernel MLPs struggle with precise pressure prediction. Here, we introduce a virial kernel function that significantly enhances the pressure estimation accuracy of MLPs. Additionally, we propose the active sparse Bayesian committee machine (BCM) potential, an on-the-fly MLP architecture that aggregates local sparse Gaussian process regression (SGPR) MLPs. The sparse BCM potential overcomes the steep computational scaling with the kernel size, and a predefined restriction on the size of kernel allows for fast and efficient on-the-fly training. Our advancements facilitate accurate and computationally efficient machine learning-enhanced MD (MLMD) simulations across diverse systems, including ice-liquid coexisting phases, Li10Ge(PS6)2 lithium solid electrolyte, and high-pressure liquid boron nitride.

Going for Gold(-Standard): Attaining Coupled Cluster Accuracy in Oxide-Supported Nanoclusters

The structure of oxide-supported metal nanoclusters plays an essential role in their sharply enhanced catalytic activity over that of bulk metals. Simulations provide the atomic-scale resolution needed to understand these systems. However, the sensitive mix of metal-metal and metal-support interactions, which govern their structure, puts stringent requirements on the method used, requiring calculations beyond standard density functional theory (DFT). The method of choice is coupled cluster theory [specifically CCSD(T)], but its computational cost has so far prevented its application to these systems. In this work, we showcase two approaches to make CCSD(T) accuracy readily achievable in oxide-supported nanoclusters. First, we leverage the SKZCAM protocol to provide the first benchmarks of oxide-supported nanoclusters, revealing that it is specifically metal-metal interactions that are challenging to capture with DFT. Second, we propose a CCSD(T) correction (ΔCC) to the metal-metal interaction errors in DFT, reaching accuracy comparable to that of the SKZCAM protocol at significantly lower cost. This approach forges a path toward studying larger systems at reliable accuracy, which we highlight by identifying a ground-state structure in agreement with experiments for Au20 on MgO, a challenging system where DFT models have yielded conflicting predictions.

To Pair or not to Pair? Machine-Learned Explicitly-Correlated Electronic Structure for NaCl in Water

The extent of ion pairing in solution is an important phenomenon to rationalize transport and thermodynamic properties of electrolytes. A fundamental measure of this pairing is the potential of mean force (PMF) between solvated ions. The relative stabilities of the paired and solvent shared states in the PMF and the barrier between them are highly sensitive to the underlying potential energy surface. However, direct application of accurate electronic structure methods is challenging, since long simulations are required. We develop wave function based machine learning potentials with the random phase approximation (RPA) and second order Møller-Plesset (MP2) perturbation theory for the prototypical system of Na and Cl ions in water. We show both methods in agreement, predicting the paired and solvent shared states to have similar energies (within 0.2 kcal/mol). We also provide the same benchmarks for different DFT functionals as well as insight into the PMF based on simple analyses of the interactions in the system.

Path Integral Simulations of Condensed-Phase Vibrational Spectroscopy

Recent theoretical and algorithmic developments have improved the accuracy with which path integral dynamics methods can include nuclear quantum effects in simulations of condensed-phase vibrational spectra. Such methods are now understood to be approximations to the delocalized classical Matsubara dynamics of smooth Feynman paths, which dominate the dynamics of systems such as liquid water at room temperature. Focusing mainly on simulations of liquid water and hexagonal ice, we explain how the recently developed quasicentroid molecular dynamics (QCMD), fast-QCMD, and temperature-elevated path integral coarse-graining simulations (Te PIGS) methods generate classical dynamics on potentials of mean force obtained by averaging over quantum thermal fluctuations. These new methods give very close agreement with one another, and the Te PIGS method has recently yielded excellent agreement with experimentally measured vibrational spectra for liquid water, ice, and the liquid-air interface. We also discuss the limitations of such methods.

A personal perspective of the present status and future challenges facing thermal reaction rate theory

Reaction rate theory has been at the center of physical chemistry for well over one hundred years. The evolution of the theory is not only of historical interest. Reliable and accurate computation of reaction rates remains a challenge to this very day, especially in view of the development of quantum chemistry methods, which predict the relevant force fields. It is still not possible to compute the numerically exact rate on the fly when the system has more than at most a few dozen anharmonic degrees of freedom, so one must consider various approximate methods, not only from the practical point of view of constructing numerical algorithms but also on conceptual and formal levels. In this Perspective, I present some of the recent analytical results concerning leading order terms in an ℏ2m series expansion of the exact rate and their implications on various approximate theories. A second aspect has to do with the crossover temperature between tunneling and thermal activation. Using a uniform semiclassical transmission probability rather than the "primitive" semiclassical theory leads to the conclusion that there is no divergence problem associated with a "crossover temperature." If one defines a semiclassical crossover temperature as the point at which the tunneling energy of the instanton equals the barrier height, then it is a factor of two higher than its previous estimate based on the "primitive" semiclassical approximation. In the low temperature tunneling regime, the uniform semiclassical theory as well as the "primitive" semiclassical theory were based on the classical Euclidean action of a periodic orbit on the inverted potential. The uniform semiclassical theory wrongly predicts that the "half-point," which is the energy at which the transmission probability equals 1/2, for any barrier potential, is always the barrier energy. We describe here how augmenting the Euclidean action with constant terms of order ℏ2 can significantly improve the accuracy of the semiclassical theory and correct this deficiency. This also leads to a deep connection with and improvement of vibrational perturbation theory. The uniform semiclassical theory also enables an extension of the quantum version of Kramers' turnover theory to temperatures below the "crossover temperature." The implications of these recent advances on various approximate methods used to date are discussed at length, leading to the conclusion that reaction rate theory will continue to challenge us both on conceptual and practical levels for years to come.

How to train a neural network potential

The introduction of modern Machine Learning Potentials (MLPs) has led to a paradigm change in the development of potential energy surfaces for atomistic simulations. By providing efficient access to energies and forces, they allow us to perform large-scale simulations of extended systems, which are not directly accessible by demanding first-principles methods. In these simulations, MLPs can reach the accuracy of electronic structure calculations, provided that they have been properly trained and validated using a suitable set of reference data. Due to their highly flexible functional form, the construction of MLPs has to be done with great care. In this Tutorial, we describe the necessary key steps for training reliable MLPs, from data generation via training to final validation. The procedure, which is illustrated for the example of a high-dimensional neural network potential, is general and applicable to many types of MLPs.

A Status Report on "Gold Standard" Machine-Learned Potentials for Water

Owing to the central importance of water to life as well as its unusual properties, potentials for water have been the subject of extensive research over the past 50 years. Recently, five potentials based on different machine learning approaches have been reported that are at or near the "gold standard" CCSD(T) level of theory. The development of such high-level potentials enables efficient and accurate simulations of water systems using classical and quantum dynamical approaches. This Perspective serves as a status report of these potentials, focusing on their methodology and applications to water systems across different phases. Their performances on the energies of gas phase water clusters, as well as condensed phase structural and dynamical properties, are discussed.

Developing machine-learned potentials to simultaneously capture the dynamics of excess protons and hydroxide ions in classical and path integral simulations

The transport of excess protons and hydroxide ions in water underlies numerous important chemical and biological processes. Accurately simulating the associated transport mechanisms ideally requires utilizing ab initio molecular dynamics simulations to model the bond breaking and formation involved in proton transfer and path-integral simulations to model the nuclear quantum effects relevant to light hydrogen atoms. These requirements result in a prohibitive computational cost, especially at the time and length scales needed to converge proton transport properties. Here, we present machine-learned potentials (MLPs) that can model both excess protons and hydroxide ions at the generalized gradient approximation and hybrid density functional theory levels of accuracy and use them to perform multiple nanoseconds of both classical and path-integral proton defect simulations at a fraction of the cost of the corresponding ab initio simulations. We show that the MLPs are able to reproduce ab initio trends and converge properties such as the diffusion coefficients of both excess protons and hydroxide ions. We use our multi-nanosecond simulations, which allow us to monitor large numbers of proton transfer events, to analyze the role of hypercoordination in the transport mechanism of the hydroxide ion and provide further evidence for the asymmetry in diffusion between excess protons and hydroxide ions.

Benchmark Phaseless Auxiliary-Field Quantum Monte Carlo Method for Small Molecules

We report a scalable Fortran implementation of the phaseless auxiliary-field quantum Monte Carlo (ph-AFQMC) and demonstrate its excellent performance and beneficial scaling with respect to system size. Furthermore, we investigate modifications of the phaseless approximation that can help to reduce the overcorrelation problems common to the ph-AFQMC. We apply the method to the 26 molecules in the HEAT set, the benzene molecule, and water clusters. We observe a mean absolute deviation of the total energy of 1.15 kcal/mol for the molecules in the HEAT set, close to chemical accuracy. For the benzene molecule, the modified algorithm despite using a single-Slater-determinant trial wavefunction yields the same accuracy as the original phaseless scheme with 400 Slater determinants. Despite these improvements, we find systematic errors for the CN, CO2, and O2 molecules that need to be addressed with more accurate trial wavefunctions. For water clusters, we find that the ph-AFQMC yields excellent binding energies that differ from CCSD(T) by typically less than 0.5 kcal/mol.

Machine Learning for Bridging the Gap between Density Functional Theory and Coupled Cluster Energies

Accurate electronic energies and properties are crucial for successful reaction design and mechanistic investigations. Computing energies and properties of molecular structures has proven extremely useful, and, with increasing computational power, the limits of high-level approaches (such as coupled cluster theory) are expanding to ever larger systems. However, because scaling is highly unfavorable, these methods are still not universally applicable to larger systems. To address the need for fast and accurate electronic energies of larger systems, we created a database of around 8000 small organic monomers (2000 dimers) optimized at the B3LYP-D3(BJ)/cc-pVTZ level of theory. This database also includes single-point energies computed at various levels of theory, including PBE1PBE, ωΒ97Χ, M06-2X, revTPSS, B3LYP, and BP86, for density functional theory as well as DLPNO-CCSD(T) and CCSD(T) for coupled cluster theory, all in conjunction with a cc-pVTZ basis. We used this database to train machine learning models based on graph neural networks using two different graph representations. Our models are able to make energy predictions from B3LYP-D3(BJ)/cc-pVTZ inputs to CCSD(T)/cc-pVTZ outputs with a mean absolute error of 0.78 and to DLPNO-CCSD(T)/cc-pVTZ with an mean absolute error of 0.50 and 0.18 kcal mol^-1 for monomers and dimers, respectively. The model for dimers was further validated on the S22 database, and the monomer model was tested on challenging systems, including those with highly conjugated or functionally complex molecules.

Are Neural Network Potentials Trained on Liquid States Transferable to Crystal Nucleation? A Test on Ice Nucleation in the mW Water Model

Neural network potentials (NNPs) are increasingly being used to study processes that happen on long time scales. A typical example is crystal nucleation, which rate is controlled by the occurrence of a rare fluctuation, i.e., the appearance of the critical nucleus. Because the properties of this nucleus are far from those of the bulk crystal, it is yet unclear whether NN potentials trained on equilibrium liquid states can accurately describe nucleation processes. So far, nucleation studies on NNPs have been limited to ab initio models whose nucleation properties are unknown, preventing an accurate comparison. Here we train a NN potential on the mW model of water─a classical three-body potential whose nucleation time scale is accessible in standard simulations. We show that a NNP trained only on a small number of liquid state points can reproduce with great accuracy the nucleation rates and free energy barriers of the original model, computed from both spontaneous and biased trajectories, strongly supporting the use of NNPs to study nucleation events.