Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions

Current Status of the MB-pol Data-Driven Many-Body Potential for Predictive Simulations of Water Across Different Phases

Developing a molecular-level understanding of the properties of water is central to numerous scientific and technological applications. However, accurately modeling water through computer simulations has been a significant challenge due to the complex nature of the hydrogen-bonding network that water molecules form under different thermodynamic conditions. This complexity has led to over five decades of research and many modeling attempts. The introduction of the MB-pol data-driven many-body potential energy function marked a significant advancement toward a universal molecular model capable of predicting the structural, thermodynamic, dynamical, and spectroscopic properties of water across all phases. By integrating physics-based and data-driven (i.e., machine-learned) components, which correctly capture the delicate balance among different many-body interactions, MB-pol achieves chemical and spectroscopic accuracy, enabling realistic molecular simulations of water, from gas-phase clusters to liquid water and ice. In this review, we present a comprehensive overview of the data-driven many-body formalism adopted by MB-pol, highlight the main results and predictions made from computer simulations with MB-pol to date, and discuss the prospects for future extensions to data-driven many-body potentials of generic and reactive molecular systems.

Modeling Many-Body Interactions in Water with Gaussian Process Regression

We report a first-principles water dimer potential that captures many-body interactions through Gaussian process regression (GPR). Modeling is upgraded from previous work by using a custom kernel function implemented through the KeOps library, allowing for much larger GPR models to be constructed and interfaced with the next-generation machine learning force field FFLUX. A new synthetic water dimer data set, called WD24, is used for model training. The resulting models can predict 90% of dimer geometries within chemical accuracy for a test set and in a simulation. The curvature of the potential energy surface is captured by the models, and a successful geometry optimization is completed with a total energy error of just 2.6 kJ mol-1, from a starting structure where water molecules are separated by nearly 4.3 Å. Dimeric modeling of a flexible, noncrystalline system with FFLUX is shown for the first time.

Many-Body Function Corrected Neural Network with Atomic Attention (MBNN-att) for Molecular Property Prediction

Recent years have seen a surge of machine learning (ML) in chemistry for predicting chemical properties, but a low-cost, general-purpose, and high-performance model, desirable to be accessible on central processing unit (CPU) devices, remains not available. For this purpose, here we introduce an atomic attention mechanism into many-body function corrected neural network (MBNN), namely, MBNN-att ML model, to predict both the extensive and intensive properties of molecules and materials. The MBNN-att uses explicit function descriptors as the inputs for the atom-based feed-forward neural network (NN). The output of the NN is designed to be a vector to implement the multihead self-attention mechanism. This vector is split into two parts: the atomic attention weight part and the many-body-function part. The final property is obtained by summing the products of each atomic attention weight and the corresponding many-body function. We show that MBNN-att performs well on all QM9 properties, i.e., errors on all properties, below chemical accuracy, and, in particular, achieves the top performance for the energy-related extensive properties. By systematically comparing with other explicit-function-type descriptor ML models and the graph representation ML models, we demonstrate that the many-body-function framework and atomic attention mechanism are key ingredients for the high performance and the good transferability of MBNN-att in molecular property prediction.

Constructing and Evaluating Machine-Learned Interatomic Potentials for Li-Based Disordered Rocksalts

Lithium-based disordered rocksalts (LDRs), which are an important class of positive electrode materials that can increase the energy density of current Li-ion batteries, represent a significantly complex chemical and configurational space for conventional density functional theory (DFT)-based high-throughput screening approaches. Notably, atom-centered machine-learned interatomic potentials (MLIPs) are a promising pathway to accurately model the potential energy surface of highly disordered chemical spaces, such as LDRs, where the performance of such MLIPs has not been rigorously explored yet. Here, we represent a comprehensive evaluation of the accuracy, transferability, and ease of training of five atom-centered MLIPs, including the artificial neural network potentials developed by the atomic energy network (AENET), the Gaussian approximation potential (GAP), the spectral neighbor analysis potential (SNAP) and its quadratic extension (qSNAP), and the moment tensor potential (MTP), in modeling a 11-component LDR chemical space. Specifically, we generate a DFT-calculated data set of 10,842 configurations of disordered LiTMO2 and TMO2 compositions, where TM = Sc, Ti, V, Cr, Mn, Fe, Co, Ni, and/or Cu. To provide a point-of-comparison on the performance of atom-centered MLIPs, we also trained the neural equivariant interatomic potential (NequIP) on a subset of our data. Importantly, we find AENET to be the best potential in terms of accuracy and transferability for energy predictions, while MTP is the best for atomic forces. While AENET is the fastest to train among the MLIPs considered at low number of epochs (300), the training time increases significantly as epochs increase (3300), with a corresponding reduction in training errors (∼60%). Note that AENET and GAP tend to overfit in small data sets, with the extent of overfitting reducing with larger data sets. Finally, we observe AENET to provide reasonable predictions of average Li-intercalation voltages in layered, single-TM LiTMO2 frameworks, compared to DFT (∼10% error on average). Our study should pave the way both for discovering novel disordered rocksalt electrodes and for modeling other configurationally complex systems, such as high-entropy ceramics and alloys.

The Potential of Neural Network Potentials

In the next half-century, physical chemistry will likely undergo a profound transformation, driven predominantly by the combination of recent advances in quantum chemistry and machine learning (ML). Specifically, equivariant neural network potentials (NNPs) are a breakthrough new tool that are already enabling us to simulate systems at the molecular scale with unprecedented accuracy and speed, relying on nothing but fundamental physical laws. The continued development of this approach will realize Paul Dirac's 80-year-old vision of using quantum mechanics to unify physics with chemistry and providing invaluable tools for understanding materials science, biology, earth sciences, and beyond. The era of highly accurate and efficient first-principles molecular simulations will provide a wealth of training data that can be used to build automated computational methodologies, using tools such as diffusion models, for the design and optimization of systems at the molecular scale. Large language models (LLMs) will also evolve into increasingly indispensable tools for literature review, coding, idea generation, and scientific writing.

Comparing machine learning potentials for water: Kernel-based regression and Behler-Parrinello neural networks

In this paper, we investigate the performance of different machine learning potentials (MLPs) in predicting key thermodynamic properties of water using RPBE + D3. Specifically, we scrutinize kernel-based regression and high-dimensional neural networks trained on a highly accurate dataset consisting of about 1500 structures, as well as a smaller dataset, about half the size, obtained using only on-the-fly learning. This study reveals that despite minor differences between the MLPs, their agreement on observables such as the diffusion constant and pair-correlation functions is excellent, especially for the large training dataset. Variations in the predicted density isobars, albeit somewhat larger, are also acceptable, particularly given the errors inherent to approximate density functional theory. Overall, this study emphasizes the relevance of the database over the fitting method. Finally, this study underscores the limitations of root mean square errors and the need for comprehensive testing, advocating the use of multiple MLPs for enhanced certainty, particularly when simulating complex thermodynamic properties that may not be fully captured by simpler tests.

Parametrically Managed Activation Functions for Improved Global Potential Energy Surfaces for Six Coupled 5A' States and Fourteen Coupled 3A' States of O + O2

We report new potential energy surfaces for six coupled 5A' states and 14 coupled 3A' states of O3. The new surfaces are created by parametrically managed diabatization by deep neural network (PM-DDNN). The PM-DDNN method uses calculated adiabatic potential energy surfaces to discover and fit an underlying adiabatic-equivalent set of diabatic surfaces and their couplings and obtains the fit to the adiabatic surfaces by diagonalization of the diabatic potential energy matrix (DPEM). The procedure yields the adiabatic surfaces and their gradients, as well as the DPEM and its gradient. If desired one can also compute the nonadiabatic coupling due to the transformation. The present work improves on previous work by using a new coordinate to guide the decay of the neural network contribution to the many-body fit to the whole DPEM. The main objective was to obtain smoother potentials than the previous ones with better suitability for dynamics calculations, and this was achieved. Furthermore, we obtained suitably small deviations from the input reference data. For the six coupled 5A' surfaces, the 60,366 data below 10 eV are fit with a mean unsigned error (MUE) of 49 meV, and for the 14 coupled 3A' surfaces, the 76,733 data below 10 eV are fit with an MUE of 28 meV. The data below 5 eV fit even more accurately with MUEs of 37 meV (5A') and 20 meV (3A').

Machine Learning Prediction of the Experimental Transition Temperature of Fe(II) Spin-Crossover Complexes

Spin-crossover (SCO) complexes are materials that exhibit changes in the spin state in response to external stimuli, with potential applications in molecular electronics. It is challenging to know a priori how to design ligands to achieve the delicate balance of entropic and enthalpic contributions needed to tailor a transition temperature close to room temperature. We leverage the SCO complexes from the previously curated SCO-95 data set [Vennelakanti et al. J. Chem. Phys. 159, 024120 (2023)] to train three machine learning (ML) models for transition temperature (T1/2) prediction using graph-based revised autocorrelations as features. We perform feature selection using random forest-ranked recursive feature addition (RF-RFA) to identify the features essential to model transferability. Of the ML models considered, the full feature set RF and recursive feature addition RF models perform best, achieving moderate correlation to experimental T1/2 values. We then compare ML T1/2 predictions to those from three previously identified best-performing density functional approximations (DFAs) which accurately predict SCO behavior across SCO-95, finding that the ML models predict T1/2 more accurately than the best-performing DFAs. In addition, we study ML model predictions for a set of 18 SCO complexes for which only estimated T1/2 values are available. Upon excluding outliers from this set, the RF-RFA RF model shows a strong correlation to estimated T1/2 values with a Pearson's r of 0.82. In contrast, DFA-predicted T1/2 values have large errors and show no correlation to estimated T1/2 values over the same set of complexes. Overall, our study demonstrates slightly superior performance of ML models in comparison with some of the best-performing DFAs, and we expect ML models to improve further as larger data sets of SCO complexes are curated and become available for model training.

Exploring torsional conformer space with physical prior mean function-driven meta-Gaussian processes

We present a novel approach for systematically exploring the conformational space of small molecules with multiple internal torsions. Identifying unique conformers through a systematic conformational search is important for obtaining accurate thermodynamic functions (e.g., free energy), encompassing contributions from the ensemble of all local minima. Traditional geometry optimizers focus on one structure at a time, lacking transferability from the local potential-energy surface (PES) around a specific minimum to optimize other conformers. In this work, we introduce a physics-driven meta-Gaussian processes (meta-GPs) method that not only enables efficient exploration of target PES for locating local minima but, critically, incorporates physical surrogates that can be applied universally across the optimization of all conformers of the same molecule. Meta-GPs construct surrogate PESs based on the optimization history of prior conformers, dynamically selecting the most suitable prior mean function (representing prior knowledge in Bayesian learning) as a function of the optimization progress. We systematically benchmarked the performance of multiple GP variants for brute-force conformational search of amino acids. Our findings highlight the superior performance of meta-GPs in terms of efficiency, comprehensiveness of conformer discovery, and the distribution of conformers compared to conventional non-surrogate optimizers and other non-meta-GPs. Furthermore, we demonstrate that by concurrently optimizing, training GPs on the fly, and learning PESs, meta-GPs exhibit the capacity to generate high-quality PESs in the torsional space without extensive training data. This represents a promising avenue for physics-based transfer learning via meta-GPs with adaptive priors in exploring torsional conformer space.

Neural Network Water Model Based on the MB-Pol Many-Body Potential

The MB-pol many-body potential accurately predicts many properties of water, including cluster, liquid phase, and vapor-liquid equilibrium properties, but its high computational cost can make applying it in large-scale simulations quite challenging. In order to address this limitation, we developed a "deep potential" neural network (DPMD) model based on the MB-pol potential for water. We find that a DPMD model trained on mostly liquid configurations yields a good description of the bulk liquid phase but severely underpredicts vapor-liquid coexistence densities. By contrast, adding cluster configurations to the neural network training set leads to a good agreement for the vapor coexistence densities. Liquid phase densities under supercooled conditions are also represented well, even though they were not included in the training set. These results confirm that neural network models can combine accuracy and transferability if sufficient attention is given to the construction of a representative training set for the target system.

Graph-EAM: An Interpretable and Efficient Graph Neural Network Potential Framework

The development of deep learning interatomic potentials has enabled efficient and accurate computations in quantum chemistry and materials science, circumventing computationally expensive ab initio calculations. However, the huge number of learnable parameters in deep learning models and their complex architectures hinder physical interpretability and affect the robustness of the derived potential. In this work, we propose graph-EAM, a lightweight graph neural network (GNN) inspired by the empirical embedded atom method to model the interatomic potential of single-element structures. Four material systems: platinum, niobium, silicon, and amorphous-carbon, for which quantum simulation data sets are publicly available, are examined to demonstrate that graph-EAM can achieve high energy and force prediction accuracy─comparable or better than existing state-of-the-art machine learning models─with much fewer parameters. It is also shown that the explicit inclusion of the angular information via three-body atomic density increases the prediction accuracy. The accuracy and efficiency of potentials obtained from graph-EAM can help accelerate the molecular dynamics simulation.

Are Neural Network Potentials Trained on Liquid States Transferable to Crystal Nucleation? A Test on Ice Nucleation in the mW Water Model

Neural network potentials (NNPs) are increasingly being used to study processes that happen on long time scales. A typical example is crystal nucleation, which rate is controlled by the occurrence of a rare fluctuation, i.e., the appearance of the critical nucleus. Because the properties of this nucleus are far from those of the bulk crystal, it is yet unclear whether NN potentials trained on equilibrium liquid states can accurately describe nucleation processes. So far, nucleation studies on NNPs have been limited to ab initio models whose nucleation properties are unknown, preventing an accurate comparison. Here we train a NN potential on the mW model of water─a classical three-body potential whose nucleation time scale is accessible in standard simulations. We show that a NNP trained only on a small number of liquid state points can reproduce with great accuracy the nucleation rates and free energy barriers of the original model, computed from both spontaneous and biased trajectories, strongly supporting the use of NNPs to study nucleation events.

Many-Body Effects in Aqueous Systems: Synergies Between Interaction Analysis Techniques and Force Field Development

Interaction analysis techniques, including the many-body expansion (MBE), symmetry-adapted perturbation theory, and energy decomposition analysis, allow for an intuitive understanding of complex molecular interactions. We review these methods by first providing a historical context for the study of many-body interactions and discussing how nonadditivities emerge from Hamiltonians containing strictly pairwise-additive interactions. We then elaborate on the synergy between these interaction analysis techniques and the development of advanced force fields aimed at accurately reproducing the Born-Oppenheimer potential energy surface. In particular, we focus on ab initio-based force fields that aim to explicitly reproduce many-body terms and are fitted to high-level electronic structure results. These force fields generally incorporate many-body effects through (a) parameterization of distributed multipoles, (b) explicit fitting of the MBE, (c) inclusion of many-atom features in a neural network, and (d) coarse-graining of many-body terms into an effective two-body term. We also discuss the emerging use of the MBE to improve the accuracy and speed of ab initio molecular dynamics.

A neural network potential with self-trained atomic fingerprints: A test with the mW water potential

We present a neural network (NN) potential based on a new set of atomic fingerprints built upon two- and three-body contributions that probe distances and local orientational order, respectively. Compared with the existing NN potentials, the atomic fingerprints depend on a small set of tunable parameters that are trained together with the NN weights. In addition to simplifying the selection of the atomic fingerprints, this strategy can also considerably increase the overall accuracy of the network representation. To tackle the simultaneous training of the atomic fingerprint parameters and NN weights, we adopt an annealing protocol that progressively cycles the learning rate, significantly improving the accuracy of the NN potential. We test the performance of the network potential against the mW model of water, which is a classical three-body potential that well captures the anomalies of the liquid phase. Trained on just three state points, the NN potential is able to reproduce the mW model in a very wide range of densities and temperatures, from negative pressures to several GPa, capturing the transition from an open random tetrahedral network to a dense interpenetrated network. The NN potential also reproduces very well properties for which it was not explicitly trained, such as dynamical properties and the structure of the stable crystalline phases of mW.

Gaussian Process Regression Models for Predicting Atomic Energies and Multipole Moments

Developing a force field is a difficult task because its design is typically pulled in opposite directions by speed and accuracy. FFLUX breaks this trend by utilizing Gaussian process regression (GPR) to predict, at ab initio accuracy, atomic energies and multipole moments as obtained from the quantum theory of atoms in molecules (QTAIM). This work demonstrates that the in-house FFLUX training pipeline can generate successful GPR models for six representative molecules: peptide-capped glycine and alanine, glucose, paracetamol, aspirin, and ibuprofen. The molecules were sufficiently distorted to represent configurations from an AMBER-GAFF2 molecular dynamics run. All internal degrees of freedom were covered corresponding to 93 dimensions in the case of the largest molecule ibuprofen (33 atoms). Benefiting from active learning, the GPR models contain only about 2000 training points and return largely sub-kcal mol^-1 prediction errors for the validation sets. A proof of concept has been reached for transferring the model produced through active learning on one atomic property to that of the remaining atomic properties. The prediction of electrostatic interaction can be assessed at the intermolecular level, and the vast majority of interactions have a root-mean-square error of less than 0.1 kJ mol^-1 with a maximum value of ∼1 kJ mol^-1 for a glycine and paracetamol dimer.

Data-Driven Many-Body Potential Energy Functions for Generic Molecules: Linear Alkanes as a Proof-of-Concept Application

We present a generalization of the many-body energy (MB-nrg) theoretical/computational framework that enables the development of data-driven potential energy functions (PEFs) for generic covalently bonded molecules, with arbitrary quantum mechanical accuracy. The "nearsightedness of electronic matter" is exploited to define monomers as "natural building blocks" on the basis of their distinct chemical identity. The energy of generic molecules is then expressed as a sum of individual many-body energies of incrementally larger subsystems. The MB-nrg PEFs represent the low-order n-body energies, with n = 1-4, using permutationally invariant polynomials derived from electronic structure data carried out at an arbitrary quantum mechanical level of theory, while all higher-order n-body terms (n > 4) are represented by a classical many-body polarization term. As a proof-of-concept application of the general MB-nrg framework, we present MB-nrg PEFs for linear alkanes. The MB-nrg PEFs are shown to accurately reproduce reference energies, harmonic frequencies, and potential energy scans of alkanes, independently of their length. Since, by construction, the MB-nrg framework introduced here can be applied to generic covalently bonded molecules, we envision future computer simulations of complex molecular systems using data-driven MB-nrg PEFs, with arbitrary quantum mechanical accuracy.

Extending machine learning beyond interatomic potentials for predicting molecular properties

Machine learning (ML) is becoming a method of choice for modelling complex chemical processes and materials. ML provides a surrogate model trained on a reference dataset that can be used to establish a relationship between a molecular structure and its chemical properties. This Review highlights developments in the use of ML to evaluate chemical properties such as partial atomic charges, dipole moments, spin and electron densities, and chemical bonding, as well as to obtain a reduced quantum-mechanical description. We overview several modern neural network architectures, their predictive capabilities, generality and transferability, and illustrate their applicability to various chemical properties. We emphasize that learned molecular representations resemble quantum-mechanical analogues, demonstrating the ability of the models to capture the underlying physics. We also discuss how ML models can describe non-local quantum effects. Finally, we conclude by compiling a list of available ML toolboxes, summarizing the unresolved challenges and presenting an outlook for future development. The observed trends demonstrate that this field is evolving towards physics-based models augmented by ML, which is accompanied by the development of new methods and the rapid growth of user-friendly ML frameworks for chemistry.

Graph Theoretic Molecular Fragmentation for Multidimensional Potential Energy Surfaces Yield an Adaptive and General Transfer Machine Learning Protocol

Over a series of publications we have introduced a graph-theoretic description for molecular fragmentation. Here, a system is divided into a set of nodes, or vertices, that are then connected through edges, faces, and higher-order simplexes to represent a collection of spatially overlapping and locally interacting subsystems. Each such subsystem is treated at two levels of electronic structure theory, and the result is used to construct many-body expansions that are then embedded within an ONIOM-scheme. These expansions converge rapidly with many-body order (or graphical rank) of subsystems and have been previously used for ab initio molecular dynamics (AIMD) calculations and for computing multidimensional potential energy surfaces. Specifically, in all these cases we have shown that CCSD and MP2 level AIMD trajectories and potential surfaces may be obtained at density functional theory cost. The approach has been demonstrated for gas-phase studies, for condensed phase electronic structure, and also for basis set extrapolation-based AIMD. Recently, this approach has also been used to derive new quantum-computing algorithms that enormously reduce the quantum circuit depth in a circuit-based computation of correlated electronic structure. In this publication, we introduce (a) a family of neural networks that act in parallel to represent, efficiently, the post-Hartree-Fock electronic structure energy contributions for all simplexes (fragments), and (b) a new k-means-based tessellation strategy to glean training data for high-dimensional molecular spaces and minimize the extent of training needed to construct this family of neural networks. The approach is particularly useful when coupled cluster accuracy is desired and when fragment sizes grow in order to capture nonlocal interactions accurately. The unique multidimensional k-means tessellation/clustering algorithm used to determine our training data for all fragments is shown to be extremely efficient and reduces the needed training to only 10% of data for all fragments to obtain accurate neural networks for each fragment. These fully connected dense neural networks are then used to extrapolate the potential energy surface for all molecular fragments, and these are then combined as per our graph-theoretic procedure to transfer the learning process to a full system energy for the entire AIMD trajectory at less than one-tenth the cost as compared to a regular fragmentation-based AIMD calculation.

Molecular Dipole Moment Learning via Rotationally Equivariant Gaussian Process Regression with Derivatives in Molecular-orbital-based Machine Learning

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree--Fock computations. A molecular-orbital-based (MOB) pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of molecular orbitals (MOs).The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant Gaussian process regression (GPR) with derivatives algorithm is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly-accurate models for both dipole moment and energies on water and fourteen small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering (GMM/GPR) scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with literature results, MOB-ML provides the best test MAEs of 4.21 mDebye and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

Accurate Molecular-Orbital-Based Machine Learning Energies via Unsupervised Clustering of Chemical Space

We introduce an unsupervised clustering algorithm to improve training efficiency and accuracy in predicting energies using molecular-orbital-based machine learning (MOB-ML). This work determines clusters via the Gaussian mixture model (GMM) in an entirely automatic manner and simplifies an earlier supervised clustering approach [ J. Chem. Theory Comput. 2019, 15, 6668] by eliminating both the necessity for user-specified parameters and the training of an additional classifier. Unsupervised clustering results from GMM have the advantages of accurately reproducing chemically intuitive groupings of frontier molecular orbitals and exhibiting improved performance with an increasing number of training examples. The resulting clusters from supervised or unsupervised clustering are further combined with scalable Gaussian process regression (GPR) or linear regression (LR) to learn molecular energies accurately by generating a local regression model in each cluster. Among all four combinations of regressors and clustering methods, GMM combined with scalable exact GPR (GMM/GPR) is the most efficient training protocol for MOB-ML. The numerical tests of molecular energy learning on thermalized data sets of drug-like molecules demonstrate the improved accuracy, transferability, and learning efficiency of GMM/GPR over other training protocols for MOB-ML, i.e., supervised regression clustering combined with GPR (RC/GPR) and GPR without clustering. GMM/GPR also provides the best molecular energy predictions compared with ones from the literature on the same benchmark data sets. With a lower scaling, GMM/GPR has a 10.4-fold speedup in wall-clock training time compared with scalable exact GPR with a training size of 6500 QM7b-T molecules.

Fast Near Ab Initio Potential Energy Surfaces Using Machine Learning

A machine-learning based approach for evaluating potential energies for quantum mechanical studies of properties of the ground and excited vibrational states of small molecules is developed. This approach uses the molecular-orbital-based machine learning (MOB-ML) method to generate electronic energies with the accuracy of CCSD(T) calculations at the same cost as a Hartree-Fock calculation. To further reduce the computational cost of the potential energy evaluations without sacrificing the CCSD(T) level accuracy, GPU-accelerated Neural Network Potential Energy Surfaces (NN-PES) are trained to geometries and energies that are collected from small-scale Diffusion Monte Carlo (DMC) simulations, which are run using energies evaluated using the MOB-ML model. The combined NN+(MOB-ML) approach is used in variational calculations of the ground and low-lying vibrational excited states of water and in DMC calculations of the ground states of water, CH₅⁺, and its deuterated analogues. For both of these molecules, comparisons are made to the results obtained using potentials that were fit to much larger sets of electronic energies than were required to train the MOB-ML models. The NN+(MOB-ML) approach is also used to obtain a potential surface for C₂H₅⁺, which is a carbocation with a nonclassical equilibrium structure for which there is currently no available potential surface. This potential is used to explore the CH stretching vibrations, focusing on those of the bridging hydrogen atom. For both CH₅⁺ and C₂H₅⁺ the MOB-ML model is trained using geometries that were sampled from an AIMD trajectory, which was run at 350 K. By comparison, the structures sampled in the ground state calculations can have energies that are as much as ten times larger than those used to train the MOB-ML model. For water a higher temperature AIMD trajectory is needed to obtain accurate results due to the smaller thermal energy. A second MOB-ML model for C₂H₅⁺ was developed with additional higher energy structures in the training set. The two models are found to provide nearly identical descriptions of the ground state of C₂H₅⁺.

The MD17 datasets from the perspective of datasets for gas-phase “small” molecule potentials

There has been great progress in developing methods for machine-learned potential energy surfaces. There have also been important assessments of these methods by comparing so-called learning curves on datasets of electronic energies and forces, notably the MD17 database. The dataset for each molecule in this database generally consists of tens of thousands of energies and forces obtained from DFT direct dynamics at 500 K. We contrast the datasets from this database for three “small” molecules, ethanol, malonaldehyde, and glycine, with datasets we have generated with specific targets for the potential energy surfaces (PESs) in mind: a rigorous calculation of the zero-point energy and wavefunction, the tunneling splitting in malonaldehyde, and, in the case of glycine, a description of all eight low-lying conformers. We found that the MD17 datasets are too limited for these targets. We also examine recent datasets for several PESs that describe small-molecule but complex chemical reactions. Finally, we introduce a new database, “QM-22,” which contains datasets of molecules ranging from 4 to 15 atoms that extend to high energies and a large span of configurations.

Toward High-level Machine Learning Potential for Water Based on Quantum Fragmentation and Neural Networks

Accurate and efficient simulation of liquids, such as water and salt solutions, using high-level wave function theories is still a formidable task for computational chemists owing to the high computational costs. In this study, we develop a deep machine learning potential based on fragment-based second-order Møller-Plesset perturbation theory (DP-MP2) for water through neural networks. We show that the DP-MP2 potential predicts the structural, dynamical, and thermodynamic properties of liquid water in better agreement with the experimental data than previous studies based on density functional theory (DFT). The nuclear quantum effects (NQEs) on the properties of liquid water are also examined, which are noticeable in affecting the structural and dynamical properties of liquid water under ambient conditions. This work provides a general framework for quantitative predictions of the properties of condensed-phase systems with the accuracy of high-level wave function theory while achieving significant computational savings compared to ab initio simulations.

Many-Body Neural Network-Based Force Field for Structure-Based Coarse-Graining of Water

High-fidelity results from atomistic simulations can only be obtained by using accurate force-field (FF) parameters. Although empirical FFs are commonly used in the modeling of atomistic systems due to their simplicity, they have many limitations inherent in the crude approximations associated with their analytical form. Recent advances in neural network-based FFs have led to more accurate FFs by using symmetry functions or full many-body expansions. However, this approach leads to several issues including the arbitrariness of the symmetry functions, and the intangible and uninterpretable interactions which are only known once the positions of all atoms are set. More importantly, training is another bottleneck, as high-quality force and energy information is required, which is usually not accessible from experimental data. To solve these issues within the context of structure-based coarse-graining methods, we switch in this work to a local-search method to target the reference structure instead of using conventional backpropagation algorithms used to target the forces and energies of the reference structure. Our FF is decomposed into two-, three-, and higher-order terms, where each term is modeled with a separate neural network. To show the versatility of our method, we study four different systems, namely, Stillinger-Weber particles as an atomistic case and three water models, namely SPC/E, MB-pol, and ab initio, as coarse-graining cases. We show the successful application of our approach, by reproducing structural properties of different water models, followed by providing insight into the role of two-and three-body interactions. The results of all models indicate that the double-well isotropic pair potential, the signature of water-like behavior in an isotropic system, vanishes upon inclusion of the three-body interaction, showing dominance of the three-body interaction over the two-body interaction in water-like behavior with the single-well isotropic pair potential.

Local Kernel Regression and Neural Network Approaches to the Conformational Landscapes of Oligopeptides

The application of machine learning to theoretical chemistry has made it possible to combine the accuracy of quantum chemical energetics with the thorough sampling of finite-temperature fluctuations. To reach this goal, a diverse set of methods has been proposed, ranging from simple linear models to kernel regression and highly nonlinear neural networks. Here we apply two widely different approaches to the same, challenging problem: the sampling of the conformational landscape of polypeptides at finite temperature. We develop a local kernel regression (LKR) coupled with a supervised sparsity method and compare it with a more established approach based on Behler-Parrinello type neural networks. In the context of the LKR, we discuss how the supervised selection of the reference pool of environments is crucial to achieve accurate potential energy surfaces at a competitive computational cost and leverage the locality of the model to infer which chemical environments are poorly described by the DFTB baseline. We then discuss the relative merits of the two frameworks and perform Hamiltonian-reservoir replica-exchange Monte Carlo sampling and metadynamics simulations, respectively, to demonstrate that both frameworks can achieve converged and transferable sampling of the conformational landscape of complex and flexible biomolecules with comparable accuracy and computational cost.

Potential Application of Machine-Learning-Based Quantum Chemical Methods in Environmental Chemistry

It is an important topic in environmental sciences to understand the behavior and toxicology of chemical pollutants. Quantum chemical methodologies have served as useful tools for probing behavior and toxicology of chemical pollutants in recent decades. In recent years, machine learning (ML) techniques have brought revolutionary developments to the field of quantum chemistry, which may be beneficial for investigating environmental behavior and toxicology of chemical pollutants. However, the ML-based quantum chemical methods (ML-QCMs) have only scarcely been used in environmental chemical studies so far. To promote applications of the promising methods, this Perspective summarizes recent progress in the ML-QCMs and focuses on their potential applications in environmental chemical studies that could hardly be achieved by the conventional quantum chemical methods. Potential applications and challenges of the ML-QCMs in predicting degradation networks of chemical pollutants, searching global minima for atmospheric nanoclusters, discovering heterogeneous or photochemical transformation pathways of pollutants, as well as predicting environmentally relevant end points with wave functions as descriptors are introduced and discussed.

Representing Globally Accurate Reactive Potential Energy Surfaces with Complex Topography by Combining Gaussian Process Regression and Neural Network

There has been increasing attention in using machine learning technologies, such as neural network (NN) and Gaussian process regression (GPR), to model multidimensional potential energy surfaces (PESs). NN PES features...

A new permutation-symmetry-adapted machine learning diabatization procedure and its application in MgH2 system

This work introduces a new permutation-symmetry-adapted machine learning diabatization procedure, termed the diabatization by equivariant neural network (DENN). In this approach, the permutation symmetric and anti-symmetric elements in diabatic potential energy metrics (DPEMs) were simultaneously simulated by the equivariant neural network. The diabatization by deep neural network scheme was adopted for machine learning diabatization, and non-zero diabatic coupling was included to increase accuracy in the near degenerate region. Based on DENN, the global DPEMs for 1¹A' and 2¹A' states of MgH₂ have been constructed. To the best of our knowledge, these are the first global DPEMs for the MgH₂ system. The root-mean-square-errors (RMSEs) for diagonal elements (H₁₁ and H₂₂) and the off-diagonal element (H₁₂) around the conical intersection region were 5.824, 5.307, and 5.796 meV, respectively. The RMSEs of global adiabatic energies for two adiabatic states were 4.613 and 12.755 meV, respectively. The spectroscopic calculations of the 1¹A' state in the linear HMgH region are in good agreement with the experiment and previous theoretical results. The differences between calculated frequencies and corresponding experiment values are 1.38 and 1.08 cm^-1 for anti-symmetric stretching fundamental vibrational frequency and first overtone. The global DPEMs obtained in this work should be useful for further quantum mechanics dynamic simulations on the MgH₂ system.

Automated Construction of Neural Network Potential Energy Surface: The Enhanced Self-Organizing Incremental Neural Network Deep Potential Method

In recent years, the use of deep learning (neural network) potential energy surface (NNPES) in molecular dynamics simulation has experienced explosive growth as it can be as accurate as quantum chemistry methods while being as efficient as classical mechanic methods. However, the development of NNPES is highly nontrivial. In particular, it has been troubling to construct a dataset that is as small as possible yet can cover the target chemical space. In this work, an ESOINN-DP method is developed, which has the enhanced self-organizing incremental neural network (ESOINN) and a newly proposed error indicator at its core. With ESOINN-DP, one can construct the NNPES with little human intervention, and this method ensures that the constructed reference dataset covers the target chemical space with minimum redundancy. The performance of the ESOINN-DP method has been well validated by developing neural network potential energy surfaces for water clusters, tripeptides, and by de-redundancy of a sub-dataset of the ANI-1 database. We believe that the ESOINN-DP method provides a novel idea for the construction of NNPES and, especially, the reference datasets, and it can be used for molecular dynamics (MD) simulations of various gas-phase and condensed-phase chemical systems.

DL_FFLUX: A Parallel, Quantum Chemical Topology Force Field

DL_FFLUX is a force field based on quantum chemical topology that can perform molecular dynamics for flexible molecules endowed with polarizable atomic multipole moments (up to hexadecapole). Using the machine learning method kriging (aka Gaussian process regression), DL_FFLUX has access to atomic properties (energy, charge, dipole moment, etc.) with quantum mechanical accuracy. Newly optimized and parallelized using domain decomposition Message Passing Interface (MPI), DL_FFLUX is now able to deliver this rigorous methodology at scale while still in reasonable time frames. DL_FFLUX is delivered as an add-on to the widely distributed molecular dynamics code DL_POLY 4.08. For the systems studied here (10³–10⁵ atoms), DL_FFLUX is shown to add minimal computational cost to the standard DL_POLY package. In fact, the optimization of the electrostatics in DL_FFLUX means that, when high-rank multipole moments are enabled, DL_FFLUX is up to 1.25× faster than standard DL_POLY. The parallel DL_FFLUX preserves the quality of the scaling of MPI implementation in standard DL_POLY. For the first time, it is feasible to use the full capability of DL_FFLUX to study systems that are large enough to be of real-world interest. For example, a fully flexible, high-rank polarized (up to and including quadrupole moments) 1 ns simulation of a system of 10 125 atoms (3375 water molecules) takes 30 h (wall time) on 18 cores.

Gaussian process models of potential energy surfaces with boundary optimization

A strategy is outlined to reduce the number of training points required to model intermolecular potentials using Gaussian processes, without reducing accuracy. An asymptotic function is used at a long range, and the crossover distance between this model and the Gaussian process is learnt from the training data. The results are presented for different implementations of this procedure, known as boundary optimization, across the following dimer systems: CO-Ne, HF-Ne, HF-Na⁺, CO₂-Ne, and (CO₂)₂. The technique reduces the number of training points, at fixed accuracy, by up to ∼49%, compared to our previous work based on a sequential learning technique. The approach is readily transferable to other statistical methods of prediction or modeling problems.

General Many-Body Framework for Data-Driven Potentials with Arbitrary Quantum Mechanical Accuracy: Water as a Case Study

We present a general framework for the development of data-driven many-body (MB) potential energy functions (MB-QM PEFs) that represent the interactions between small molecules at an arbitrary quantum-mechanical (QM) level of theory. As a demonstration, a family of MB-QM PEFs for water is rigorously derived from density functionals belonging to different rungs across Jacob's ladder of approximations within density functional theory (MB-DFT) and from Møller-Plesset perturbation theory (MB-MP2). Through a systematic analysis of individual MB contributions to the interaction energies of water clusters, we demonstrate that all MB-QM PEFs preserve the same accuracy as the corresponding ab initio calculations, with the exception of those derived from density functionals within the generalized gradient approximation (GGA). The differences between the DFT and MB-DFT results are traced back to density-driven errors that prevent GGA functionals from accurately representing the underlying molecular interactions for different cluster sizes and hydrogen-bonding arrangements. We show that this shortcoming may be overcome, within the MB formalism, by using density-corrected functionals (DC-DFT) that provide a more consistent representation of each individual MB contribution. This is demonstrated through the development of a MB-DFT PEF derived from DC-PBE-D3 data, which more accurately reproduce the corresponding ab initio results.

A unified cost-effective method for the construction of reliable potential energy surfaces for H2S and H2O clusters

A DFT-based methodology has been used to construct the potential energy surface of H2S clusters up to pentamers. Geometrical parameters and energetics show very good agreement with the existing experimental and high-level theoretical results. Distinct stable conformers of three dimers, six trimers, eleven tetramers and twenty-three pentamers have been identified. Both S-HS H-bond and SS interactions are identified in dimers, trimers and pentamers, while no SS interactions could be found in any of the 11 tetramer conformers. The binding energies of the most stable dimer, trimer, tetramer and pentamer are -1.66, -5.21, -8.57 and -12.54 kcal mol-1, respectively. The PES has been found to be exceedingly flat and the energy gap between the most and the least stable conformers was found to be only 0.09, 2.13, 1.65 and 1.13 kcal mol-1, from the dimer to the pentamer, respectively. The proposed method has also been used for water clusters up to the pentamer. The results obtained were found to agree closely with the existing results. Only one conformer was found for the water dimer, whereas four, five and fifteen conformers were obtained for the trimer, tetramer and pentamer, respectively. Atoms in molecular calculations were found to corroborate with the geometric and energetic results for both clusters.

Multi-body effects in a coarse-grained protein force field

The use of coarse-grained (CG) models is a popular approach to study complex biomolecular systems. By reducing the number of degrees of freedom, a CG model can explore long time- and length-scales inaccessible to computational models at higher resolution. If a CG model is designed by formally integrating out some of the system's degrees of freedom, one expects multi-body interactions to emerge in the effective CG model's energy function. In practice, it has been shown that the inclusion of multi-body terms indeed improves the accuracy of a CG model. However, no general approach has been proposed to systematically construct a CG effective energy that includes arbitrary orders of multi-body terms. In this work, we propose a neural network based approach to address this point and construct a CG model as a multi-body expansion. By applying this approach to a small protein, we evaluate the relative importance of the different multi-body terms in the definition of an accurate model. We observe a slow convergence in the multi-body expansion, where up to five-body interactions are needed to reproduce the free energy of an atomistic model.

Physics-Inspired Structural Representations for Molecules and Materials

The first step in the construction of a regression model or a data-driven analysis, aiming to predict or elucidate the relationship between the atomic-scale structure of matter and its properties, involves transforming the Cartesian coordinates of the atoms into a suitable representation. The development of atomic-scale representations has played, and continues to play, a central role in the success of machine-learning methods for chemistry and materials science. This review summarizes the current understanding of the nature and characteristics of the most commonly used structural and chemical descriptions of atomistic structures, highlighting the deep underlying connections between different frameworks and the ideas that lead to computationally efficient and universally applicable models. It emphasizes the link between properties, structures, their physical chemistry, and their mathematical description, provides examples of recent applications to a diverse set of chemical and materials science problems, and outlines the open questions and the most promising research directions in the field.

Temperature-dependent dynamic structure factors for liquid water inferred from inelastic neutron scattering measurements

Temperature-dependent dynamic structure factors S(Q, ω) for liquid water have been calculated using a composite model, which is based on the decoupling approximation of the mean square displacement of the water molecules into diffusion and solid-like vibrational parts. The solid-like vibrational part S_vib(Q, ω) is calculated with the phonon expansion method established in the framework of the incoherent Gaussian approximation. The diffusion part S_diff(Q, ω) relies on the Egelstaff-Schofield translational diffusion model corrected for jump diffusions and rotational diffusions with the Singwi-Sjölander random model and Sears expansion, respectively. Systematics of the model parameters as a function of temperature were deduced from quasi-elastic neutron scattering data analysis reported in the literature and from molecular dynamics (MD) simulations relying on the TIP4P/2005f model. The resulting S(Q, ω) values are confronted by means of Monte Carlo simulations to inelastic neutron scattering data measured with IN4, IN5, and IN6 time-of-flight spectrometers of the Institut Laue-Langevin (ILL) (Grenoble, France). A modest range of temperatures (283-494 K) has been investigated with neutron wavelengths corresponding to incident neutron energies ranging from 0.57 to 67.6 meV. The neutron-weighted multiphonon spectra deduced from the ILL data indicate a slight overestimation by the MD simulations of the frequency shift and broadening of the librational band. The descriptive power of the composite model was suited for improving the comparison to experiments via Bayesian updating of prior model parameters inferred from MD simulations. The reported posterior temperature-dependent densities of state of hydrogen in H₂O would represent valuable insights for studying the collective coupling interactions in the water molecule between the inter- and intramolecular degrees of freedom.

GPU-Accelerated Neural Network Potential Energy Surfaces for Diffusion Monte Carlo

Diffusion Monte Carlo (DMC) provides a powerful method for understanding the vibrational landscape of molecules that are not well-described by conventional methods. The most computationally demanding step of these calculations is the evaluation of the potential energy. In this work, a general approach is developed in which a neural network potential energy surface is trained by using data generated from a small-scale DMC calculation. Once trained, the neural network can be evaluated by using highly parallelizable calls to a graphics processing unit (GPU). The power of this approach is demonstrated for DMC simulations on H₂O, CH₅⁺, and (H₂O)₂. The need to include permutation symmetry in the neural network potentials is explored and incorporated into the molecular descriptors of CH₅⁺ and (H₂O)₂. It is shown that the zero-point energies and wave functions obtained by using the neural network potentials are nearly identical to the results obtained when using the potential energy surfaces that were used to train the neural networks at a substantial savings in the computational requirements of the simulations.

Highly Accurate Many-Body Potentials for Simulations of N2O5 in Water: Benchmarks, Development, and Validation

Dinitrogen pentoxide (N₂O₅) is an important intermediate in the atmospheric chemistry of nitrogen oxides. Although there has been much research, the processes that govern the physical interactions between N₂O₅ and water are still not fully understood at a molecular level. Gaining a quantitative insight from computer simulations requires going beyond the accuracy of classical force fields while accessing length scales and time scales that are out of reach for high-level quantum-chemical approaches. To this end, we present the development of MB-nrg many-body potential energy functions for nonreactive simulations of N₂O₅ in water. This MB-nrg model is based on electronic structure calculations at the coupled cluster level of theory and is compatible with the successful MB-pol model for water. It provides a physically correct description of long-range many-body interactions in combination with an explicit representation of up to three-body short-range interactions in terms of multidimensional permutationally invariant polynomials. In order to further investigate the importance of the underlying interactions in the model, a TTM-nrg model was also devised. TTM-nrg is a more simplistic representation that contains only two-body short-range interactions represented through Born-Mayer functions. In this work, an active learning approach was employed to efficiently build representative training sets of monomer, dimer, and trimer structures, and benchmarks are presented to determine the accuracy of our new models in comparison to a range of density functional theory methods. By assessing the binding curves, distortion energies of N₂O₅, and interaction energies in clusters of N₂O₅ and water, we evaluate the importance of two-body and three-body short-range potentials. The results demonstrate that our MB-nrg model has high accuracy with respect to the coupled cluster reference, outperforms current density functional theory models, and thus enables highly accurate simulations of N₂O₅ in aqueous environments.

Ultra-coarse-graining of homopolymers in inhomogeneous systems

We develop coarse-grained (CG) models for simulating homopolymers in inhomogeneous systems, focusing on polymer films and droplets. If the CG polymers interact solely through two-body potentials, then the films and droplets either dissolve or collapse into small aggregates, depending on whether the effective polymer-polymer interactions have been determined from reference simulations in the bulk or at infinite dilution. To address this shortcoming, we include higher order interactions either through an additional three-body potential or a local density-dependent potential (LDP). We parameterize the two- and three-body potentials via force matching, and the LDP through relative entropy minimization. While the CG models with three-body interactions fail at reproducing stable polymer films and droplets, CG simulations with an LDP are able to do so. Minor quantitative differences between the reference and the CG simulations, namely a slight broadening of interfaces accompanied by a smaller surface tension in the CG simulations, can be attributed to the deformation of polymers near the interfaces, which cannot be resolved in the CG representation, where the polymers are mapped to spherical beads.

How Machine Learning Will Revolutionize Electrochemical Sciences

Electrochemical systems function via interconversion of electric charge and chemical species and represent promising technologies for our cleaner, more sustainable future. However, their development time is fundamentally limited by our ability to identify new materials and understand their electrochemical response. To shorten this time frame, we need to switch from the trial-and-error approach of finding useful materials to a more selective process by leveraging model predictions. Machine learning (ML) offers data-driven predictions and can be helpful. Herein we ask if ML can revolutionize the development cycle from decades to a few years. We outline the necessary characteristics of such ML implementations. Instead of enumerating various ML algorithms, we discuss scientific questions about the electrochemical systems to which ML can contribute.

Tunnelling splitting patterns in some partially deuterated water trimers

We apply our recently developed semiclassical method for calculating tunnelling splittings (TS) in asymmetric systems to make the first characterization of the ground-state TS pattern of some partially deuterated water trimers. Similarly to homoisotopic water trimers, the ground-state TS patterns are explained in terms of six distinct rearrangement mechanisms. TS patterns in (D2O)(H2O)2 and (H2O)(D2O)2 are composed of sextets induced by the dynamics of flips, and each of its levels is further finely split into a quartet of doublets and a doublet of quartets, respectively, due to various bifurcation dynamics. The TS pattern is obtained using 18 distinct tunnelling matrix elements. TS patterns of (HOD)(H2O)2 and (HOD)(D2O)2 each consists of two sextets, belonging to in-bond and out-of-bond substituted isomers. These sextet levels are further split into quartets by bifurcations. The TS pattern is computed in terms of 13 matrix elements. We also derive analytic expressions for bifurcation tunnelling splittings in terms of tunnelling matrix elements using symmetry. The present approach can be applied to other water clusters and also to the low-lying vibrationally excited states and should help in the interpretation and assignment of experimental spectra in the future.