Permutationally invariant polynomial regression for energies and gradients, using reverse differentiation, achieves orders of magnitude speed-up with high precision compared to other machine learning methods

Theoretical Study of the Thermal Rate Coefficients of the H3+ + C2H4 Reaction: Dynamics Study on a Full-Dimensional Potential Energy Surface

Ion-molecular reactions play a significant role in molecular evolution within the interstellar medium. In this study, the entrance channel reaction, H3+ + C2H4 → H2 + C2H5+, was investigated using classical molecular dynamic (classical MD) and ring polymer molecular dynamic (RPMD) simulation techniques. We developed an analytical potential energy surface function with a permutationally invariant polynomial basis, specifically employing the monomial symmetrized approach. Our dynamic simulations reproduced the rate coefficient of 300 K for H3+ + C2H4 → H2 + C2H5+, aligning reasonably well with the values in the kinetic database commonly utilized in astrochemistry. The thermal rate coefficients obtained using both the classical MD and RPMD techniques exhibited an increase from 100 K to 300 K as the temperature rose. Additionally, we analyzed the excess energy distribution of the C2H5+ fragment with respect to temperature to investigate the indirect reaction pathway of C2H5+ → H2 + C2H3+. This result suggests that the indirect reaction pathway of C2H5+ → H2 + C2H3+ holds minor significance, although the distribution highly depends on the collisional temperature.

Assessing Permutationally Invariant Polynomial and Symmetric Gradient Domain Machine Learning Potential Energy Surfaces for H3O2. J Phys Chem A 2024;128:3212-3219. [PMID: 38624168 PMCID: PMC11056970 DOI: 10.1021/acs.jpca.4c01044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

The singly hydrated hydroxide anion OH-(H2O) is of central importance to a detailed molecular understanding of water; therefore, there is strong motivation to develop a highly accurate potential to describe this anion. While this is a small molecule, it is necessary to have an extensive data set of energies and, if possible, forces to span several important stationary points. Here, we assess two machine-learned potentials, one using the symmetric gradient domain machine learning (sGDML) method and one based on permutationally invariant polynomials (PIPs). These are successors to a PIP potential energy surface (PES) reported in 2004. We describe the details of both fitting methods and then compare the two PESs with respect to precision, properties, and speed of evaluation. While the precision of the potentials is similar, the PIP PES is much faster to evaluate for energies and energies plus gradient than the sGDML one. Diffusion Monte Carlo calculations of the ground vibrational state, using both potentials, produce similar large anharmonic downshift of the zero-point energy compared to the harmonic approximation of the PIP and sGDML potentials. The computational time for these calculations using the sGDML PES is roughly 300 times greater than using the PIP one.

Tell Machine Learning Potentials What They Are Needed For: Simulation-Oriented Training Exemplified for Glycine

Machine learning potentials (MLPs) are widely applied as an efficient alternative way to represent potential energy surfaces (PESs) in many chemical simulations. The MLPs are often evaluated with the root-mean-square errors on the test set drawn from the same distribution as the training data. Here, we systematically investigate the relationship between such test errors and the simulation accuracy with MLPs on an example of a full-dimensional, global PES for the glycine amino acid. Our results show that the errors in the test set do not unambiguously reflect the MLP performance in different simulation tasks, such as relative conformer energies, barriers, vibrational levels, and zero-point vibrational energies. We also offer an easily accessible solution for improving the MLP quality in a simulation-oriented manner, yielding the most precise relative conformer energies and barriers. This solution also passed the stringent test by diffusion Monte Carlo simulations.

No Headache for PIPs: A PIP Potential for Aspirin Runs Much Faster and with Similar Precision Than Other Machine-Learned Potentials

Assessments of machine-learning (ML) potentials are an important aspect of the rapid development of this field. We recently reported an assessment of the linear-regression permutationally invariant polynomial (PIP) method for ethanol, using the widely used (revised) rMD17 data set. We demonstrated that the PIP approach outperformed numerous other methods, e.g., ANI, PhysNet, sGDML, and p-KRR, with respect to precision and notably with respect to speed [Houston et al., J. Chem. Phys. 2022, 156, 044120]. Here, we extend this assessment to the 21-atom aspirin molecule, using the rMD17 data set, with a focus on the speed of evaluation. Both energies and forces are used for training, and the precision of several PIPs is examined for both. Normal mode frequencies, the methyl torsional potential, and 1d vibrational energies for an OH stretch are presented. We show that the PIP approach achieves the level of precision obtained from other ML methods, e.g., atom-centered neural network methods, linear regression ACE, and kernel methods, as reported by Kovács et al. in J. Chem. Theory Comput. 2021, 17, 7696-7711. More significantly, we show that the PIP PESs run much faster than all other ML methods, whose timings were evaluated in that paper. We also show that the PIP PES extrapolates well enough to describe several internal motions of aspirin, including an OH stretch.

Formic Acid-Ammonia Heterodimer: A New Δ-Machine Learning CCSD(T)-Level Potential Energy Surface Allows Investigation of the Double Proton Transfer

The formic acid-ammonia dimer is an important example of a hydrogen-bonded complex in which a double proton transfer can occur. Its microwave spectrum has recently been reported and rotational constants and quadrupole coupling constants were determined. Calculated estimates of the double-well barrier and the internal barriers to rotation were also reported. Here, we report a full-dimensional potential energy surface (PES) for this complex, using two closely related Δ-machine learning methods to bring it to the CCSD(T) level of accuracy. The PES dissociates smoothly and accurately. Using a 2d quantum model the ground vibrational-state tunneling splitting is estimated to be less than 10-4 cm-1. The dipole moment along the intrinsic reaction coordinate is calculated along with a Mullikan charge analysis and supports the mildly ionic character of the minimum and strongly ionic character at the double-well barrier.

Degeneration of kernel regression with Matern kernels into low-order polynomial regression in high dimension

Kernel methods such as kernel ridge regression and Gaussian process regression with Matern-type kernels have been increasingly used, in particular, to fit potential energy surfaces (PES) and density functionals, and for materials informatics. When the dimensionality of the feature space is high, these methods are used with necessarily sparse data. In this regime, the optimal length parameter of a Matern-type kernel may become so large that the method effectively degenerates into a low-order polynomial regression and, therefore, loses any advantage over such regression. This is demonstrated theoretically as well as numerically in the examples of six- and fifteen-dimensional molecular PES using squared exponential and simple exponential kernels. The results shed additional light on the success of polynomial approximations such as PIP for medium-size molecules and on the importance of orders-of-coupling-based models for preserving the advantages of kernel methods with Matern-type kernels of on the use of physically motivated (reproducing) kernels.

Accurate fundamental invariant-neural network representation of ab initio potential energy surfaces

Highly accurate potential energy surfaces are critically important for chemical reaction dynamics. The large number of degrees of freedom and the intricate symmetry adaption pose a big challenge to accurately representing potential energy surfaces (PESs) for polyatomic reactions. Recently, our group has made substantial progress in this direction by developing the fundamental invariant-neural network (FI-NN) approach. Here, we review these advances, demonstrating that the FI-NN approach can represent highly accurate, global, full-dimensional PESs for reactive systems with even more than 10 atoms. These multi-channel reactions typically involve many intermediates, transition states, and products. The complexity and ruggedness of this potential energy landscape present even greater challenges for full-dimensional PES representation. These PESs exhibit a high level of complexity, molecular size, and accuracy of fit. Dynamics simulations based on these PESs have unveiled intriguing and novel reaction mechanisms, providing deep insights into the intricate dynamics involved in combustion, atmospheric, and organic chemistry.

Neural Network with Optimal Neuron Activation Functions Based on Additive Gaussian Process Regression

Feed-forward neural networks (NNs) are a staple machine learning method widely used in many areas of science and technology, including physical chemistry, computational chemistry, and materials informatics. While even a single-hidden-layer NN is a universal approximator, its expressive power is limited by the use of simple neuron activation functions (such as sigmoid functions) that are typically the same for all neurons. More flexible neuron activation functions would allow the use of fewer neurons and layers and thereby save computational cost and improve expressive power. We show that additive Gaussian process regression (GPR) can be used to construct optimal neuron activation functions that are individual to each neuron. An approach is also introduced that avoids nonlinear fitting of neural network parameters by defining them with rules. The resulting method combines the advantage of robustness of a linear regression with the higher expressive power of an NN. We demonstrate the approach by fitting the potential energy surfaces of the water molecule and formaldehyde. Without requiring any nonlinear optimization, the additive-GPR-based approach outperforms a conventional NN in the high-accuracy regime, where a conventional NN suffers more from overfitting.

A Status Report on "Gold Standard" Machine-Learned Potentials for Water

Owing to the central importance of water to life as well as its unusual properties, potentials for water have been the subject of extensive research over the past 50 years. Recently, five potentials based on different machine learning approaches have been reported that are at or near the "gold standard" CCSD(T) level of theory. The development of such high-level potentials enables efficient and accurate simulations of water systems using classical and quantum dynamical approaches. This Perspective serves as a status report of these potentials, focusing on their methodology and applications to water systems across different phases. Their performances on the energies of gas phase water clusters, as well as condensed phase structural and dynamical properties, are discussed.

Parallel Implementation of Nonadditive Gaussian Process Potentials for Monte Carlo Simulations

A strategy is presented to implement Gaussian process potentials in molecular simulations through parallel programming. Attention is focused on the three-body nonadditive energy, though all algorithms extend straightforwardly to the additive energy. The method to distribute pairs and triplets between processes is general to all potentials. Results are presented for a simulation box of argon, including full box and atom displacement calculations, which are relevant to Monte Carlo simulation. Data on speed-up are presented for up to 120 processes across four nodes. A 4-fold speed-up is observed over five processes, extending to 20-fold over 40 processes and 30-fold over 120 processes.

Parametrically Managed Activation Function for Fitting a Neural Network Potential with Physical Behavior Enforced by a Low-Dimensional Potential

Machine-learned representations of potential energy surfaces generated in the output layer of a feedforward neural network are becoming increasingly popular. One difficulty with neural network output is that it is often unreliable in regions where training data is missing or sparse. Human-designed potentials often build in proper extrapolation behavior by choice of functional form. Because machine learning is very efficient, it is desirable to learn how to add human intelligence to machine-learned potentials in a convenient way. One example is the well-understood feature of interaction potentials that they vanish when subsystems are too far separated to interact. In this article, we present a way to add a new kind of activation function to a neural network to enforce low-dimensional constraints. In particular, the activation function depends parametrically on all of the input variables. We illustrate the use of this step by showing how it can force an interaction potential to go to zero at large subsystem separations without either inputting a specific functional form for the potential or adding data to the training set in the asymptotic region of geometries where the subsystems are separated. In the process of illustrating this, we present an improved set of potential energy surfaces for the 14 lowest ³A' states of O₃. The method is more general than this example, and it may be used to add other low-dimensional knowledge or lower-level knowledge to machine-learned potentials. In addition to the O₃ example, we present a greater-generality method called parametrically managed diabatization by deep neural network (PM-DDNN) that is an improvement on our previously presented permutationally restrained diabatization by deep neural network (PR-DDNN).

Reconstructing Kernel-Based Machine Learning Force Fields with Superlinear Convergence

Kernel machines have sustained continuous progress in the field of quantum chemistry. In particular, they have proven to be successful in the low-data regime of force field reconstruction. This is because many equivariances and invariances due to physical symmetries can be incorporated into the kernel function to compensate for much larger data sets. So far, the scalability of kernel machines has however been hindered by its quadratic memory and cubical runtime complexity in the number of training points. While it is known that iterative Krylov subspace solvers can overcome these burdens, their convergence crucially relies on effective preconditioners, which are elusive in practice. Effective preconditioners need to partially presolve the learning problem in a computationally cheap and numerically robust manner. Here, we consider the broad class of Nyström-type methods to construct preconditioners based on successively more sophisticated low-rank approximations of the original kernel matrix, each of which provides a different set of computational trade-offs. All considered methods aim to identify a representative subset of inducing (kernel) columns to approximate the dominant kernel spectrum.

Neural network potentials for chemistry: concepts, applications and prospects

Artificial Neural Networks (NN) are already heavily involved in methods and applications for frequent tasks in the field of computational chemistry such as representation of potential energy surfaces (PES) and spectroscopic predictions. This perspective provides an overview of the foundations of neural network-based full-dimensional potential energy surfaces, their architectures, underlying concepts, their representation and applications to chemical systems. Methods for data generation and training procedures for PES construction are discussed and means for error assessment and refinement through transfer learning are presented. A selection of recent results illustrates the latest improvements regarding accuracy of PES representations and system size limitations in dynamics simulations, but also NN application enabling direct prediction of physical results without dynamics simulations. The aim is to provide an overview for the current state-of-the-art NN approaches in computational chemistry and also to point out the current challenges in enhancing reliability and applicability of NN methods on a larger scale.

PESPIP: Software to fit complex molecular and many-body potential energy surfaces with permutationally invariant polynomials

We wish to describe a potential energy surface by using a basis of permutationally invariant polynomials whose coefficients will be determined by numerical regression so as to smoothly fit a dataset of electronic energies as well as, perhaps, gradients. The polynomials will be powers of transformed internuclear distances, usually either Morse variables, exp(-r_i,j/λ), where λ is a constant range hyperparameter, or reciprocals of the distances, 1/r_i,j. The question we address is how to create the most efficient basis, including (a) which polynomials to keep or discard, (b) how many polynomials will be needed, (c) how to make sure the polynomials correctly reproduce the zero interaction at a large distance, (d) how to ensure special symmetries, and (e) how to calculate gradients efficiently. This article discusses how these questions can be answered by using a set of programs to choose and manipulate the polynomials as well as to write efficient Fortran programs for the calculation of energies and gradients. A user-friendly interface for access to monomial symmetrization approach results is also described. The software for these programs is now publicly available.

Δ-Machine Learned Potential Energy Surfaces and Force Fields

There has been great progress in developing machine-learned potential energy surfaces (PESs) for molecules and clusters with more than 10 atoms. Unfortunately, this number of atoms generally limits the level of electronic structure theory to less than the "gold standard" CCSD(T) level. Indeed, for the well-known MD17 dataset for molecules with 9-20 atoms, all of the energies and forces were obtained with DFT calculations (PBE). This Perspective is focused on a Δ-machine learning method that we recently proposed and applied to bring DFT-based PESs to close to CCSD(T) accuracy. This is demonstrated for hydronium, N-methylacetamide, acetyl acetone, and ethanol. For 15-atom tropolone, it appears that special approaches (e.g., molecular tailoring, local CCSD(T)) are needed to obtain the CCSD(T) energies. A new aspect of this approach is the extension of Δ-machine learning to force fields. The approach is based on many-body corrections to polarizable force field potentials. This is examined in detail using the TTM2.1 water potential. The corrections make use of our recent CCSD(T) datasets for 2-b, 3-b, and 4-b interactions for water. These datasets were used to develop a new fully ab initio potential for water, termed q-AQUA.

GPU-accelerated approximate kernel method for quantum machine learning

We introduce Quantum Machine Learning (QML)-Lightning, a PyTorch package containing graphics processing unit (GPU)-accelerated approximate kernel models, which can yield trained models within seconds. QML-Lightning includes a cost-efficient GPU implementation of FCHL19, which together can provide energy and force predictions with competitive accuracy on a microsecond per atom timescale. Using modern GPU hardware, we report learning curves of energies and forces as well as timings as numerical evidence for select legacy benchmarks from atomistic simulation including QM9, MD-17, and 3BPA.

Semiclassical and VSCF/VCI Calculations of the Vibrational Energies of trans- and gauche-Ethanol Using a CCSD(T) Potential Energy Surface

A recent full-dimensional Δ-Machine learning potential energy surface (PES) for ethanol is employed in semiclassical and vibrational self-consistent field (VSCF) and virtual-state configuration interaction (VCI) calculations, using MULTIMODE, to determine the anharmonic vibrational frequencies of vibration for both the trans and gauche conformers of ethanol. Both semiclassical and VSCF/VCI energies agree well with the experimental data. We find significant mixing between the VSCF basis states due to Fermi resonances between bending and stretching modes. The same effects are also accurately described by the full-dimensional semiclassical calculations. These are the first high-level anharmonic calculations using a PES, in particular a "gold-standard" CCSD(T) one.

Quantum Calculations on a New CCSD(T) Machine-Learned Potential Energy Surface Reveal the Leaky Nature of Gas-Phase Trans and Gauche Ethanol Conformers

Ethanol is a molecule of fundamental interest in combustion, astrochemistry, and condensed phase as a solvent. It is characterized by two methyl rotors and trans (anti) and gauche conformers, which are known to be very close in energy. Here we show that based on rigorous quantum calculations of the vibrational zero-point state, using a new ab initio potential energy surface (PES), the ground state resembles the trans conformer, but substantial delocalization to the gauche conformer is present. This explains experimental issues about identification and isolation of the two conformers. This “leak” effect is partially quenched when deuterating the OH group, which further demonstrates the need for a quantum mechanical approach. Diffusion Monte Carlo and full-dimensional semiclassical dynamics calculations are employed. The new PES is obtained by means of a Δ-machine learning approach starting from a pre-existing low level density functional theory surface. This surface is brought to the CCSD(T) level of theory using a relatively small number of ab initio CCSD(T) energies. Agreement between the corrected PES and direct ab initio results for standard tests is excellent. One- and two-dimensional discrete variable representation calculations focusing on the trans–gauche torsional motion are also reported, in reasonable agreement with experiment.

The MD17 datasets from the perspective of datasets for gas-phase “small” molecule potentials

There has been great progress in developing methods for machine-learned potential energy surfaces. There have also been important assessments of these methods by comparing so-called learning curves on datasets of electronic energies and forces, notably the MD17 database. The dataset for each molecule in this database generally consists of tens of thousands of energies and forces obtained from DFT direct dynamics at 500 K. We contrast the datasets from this database for three “small” molecules, ethanol, malonaldehyde, and glycine, with datasets we have generated with specific targets for the potential energy surfaces (PESs) in mind: a rigorous calculation of the zero-point energy and wavefunction, the tunneling splitting in malonaldehyde, and, in the case of glycine, a description of all eight low-lying conformers. We found that the MD17 datasets are too limited for these targets. We also examine recent datasets for several PESs that describe small-molecule but complex chemical reactions. Finally, we introduce a new database, “QM-22,” which contains datasets of molecules ranging from 4 to 15 atoms that extend to high energies and a large span of configurations.

Permutationally invariant polynomial representation of polarizability tensor surfaces for linear regression analysis

A linearly parameterized functional form for a Cartesian representation of molecular dipole polarizability tensor surfaces (PTS) is described. The proposed expression for the PTS is a linearization of the recently reported power series ansatz of the original Applequist model, which by construction is non-linear in parameter space. This new approach possesses (i) a unique solution to the least-squares fitting problem; (ii) a low level of the computational complexity of the resulting linear regression procedure, comparable to those of the potential energy and dipole moment surfaces; and (iii) a competitive level of accuracy compared to the non-linear PTS model. Calculations of CH₄ PTS, with polarizabilities fitted to 9000 training set points with the energies up to 14,000 cm^-1 show an impressive level of accuracy of the linear PTS model obtained with ~1600 parameters: ~1% versus 0.3% RMSE for the non-linear vs. linear model on a test set of 1000 configurations.

q-AQUA: A Many-Body CCSD(T) Water Potential, Including Four-Body Interactions, Demonstrates the Quantum Nature of Water from Clusters to the Liquid Phase

Many model potential energy surfaces (PESs) have been reported for water; however, none are strictly from "first-principles". Here we report such a potential, based on a many-body representation at the CCSD(T) level of theory up to the four-body interaction. The new PES is benchmarked for the isomers of the water hexamer for dissociation energies, harmonic frequencies, and unrestricted diffusion Monte Carlo (DMC) calculations of the zero-point energies of the Prism, Book, and Cage isomers. Dissociation energies of several isomers of the 20-mer agree well with recent benchmark energies. Exploratory DMC calculations on this cluster verify the robustness of the new PES for quantum simulations. The accuracy and speed of the new PES are demonstrated for standard condensed phase properties, i.e., the radial distribution function and the self-diffusion constant. Quantum effects are shown to be substantial for these observables and also needed to bring theory into excellent agreement with experiment.

Atomistic Simulations for Reactions and Vibrational Spectroscopy in the Era of Machine Learning─Quo Vadis?

Atomistic simulations using accurate energy functions can provide molecular-level insight into functional motions of molecules in the gas and in the condensed phase. This Perspective delineates the present status of the field from the efforts of others and some of our own work and discusses open questions and future prospects. The combination of physics-based long-range representations using multipolar charge distributions and kernel representations for the bonded interactions is shown to provide realistic models for the exploration of the infrared spectroscopy of molecules in solution. For reactions, empirical models connecting dedicated energy functions for the reactant and product states allow statistically meaningful sampling of conformational space whereas machine-learned energy functions are superior in accuracy. The future combination of physics-based models with machine-learning techniques and integration into all-purpose molecular simulation software provides a unique opportunity to bring such dynamics simulations closer to reality.