Representing high-dimensional potential-energy surfaces for reactions at surfaces by neural networks

A simple approach to rotationally invariant machine learning of a vector quantity

Unlike with the energy, which is a scalar property, machine learning (ML) prediction of vector or tensor properties poses the additional challenge of achieving proper invariance (covariance) with respect to molecular rotation. For the energy gradients needed in molecular dynamics (MD), this symmetry is automatically fulfilled when taking analytic derivative of the energy, which is a scalar invariant (using properly invariant molecular descriptors). However, if the properties cannot be obtained by differentiation, other appropriate methods should be applied to retain the covariance. Several approaches have been suggested to properly treat this issue. For nonadiabatic couplings and polarizabilities, for example, it was possible to construct virtual quantities from which the above tensorial properties are obtained by differentiation and thus guarantee the covariance. Another possible solution is to build the rotational equivariance into the design of a neural network employed in the model. Here, we propose a simpler alternative technique, which does not require construction of auxiliary properties or application of special equivariant ML techniques. We suggest a three-step approach, using the molecular tensor of inertia. In the first step, the molecule is rotated using the eigenvectors of this tensor to its principal axes. In the second step, the ML procedure predicts the vector property relative to this orientation, based on a training set where all vector properties were in this same coordinate system. As the third step, it remains to transform the ML estimate of the vector property back to the original orientation. This rotate-predict-rotate (RPR) procedure should thus guarantee proper covariance of a vector property and is trivially extensible also to tensors such as polarizability. The RPR procedure has an advantage that the accurate models can be trained very fast for thousands of molecular configurations, which might be beneficial where many training sets are required (e.g., in active learning). We have implemented the RPR technique, using the MLatom and Newton-X programs for ML and MD, and performed its assessment on the dipole moment along MD trajectories of 1,2-dichloroethane.

A deep equivariant neural network approach for efficient hybrid density functional calculations

Hybrid density functional calculations are essential for accurate description of electronic structure, yet their widespread use is restricted by the substantial computational cost. Here we develop DeepH-hybrid, a deep equivariant neural network method for learning the hybrid-functional Hamiltonian as a function of material structure, which circumvents the time-consuming self-consistent field iterations and enables the study of large-scale materials with hybrid-functional accuracy. Our extensive experiments demonstrate good reliability as well as effective transferability and efficiency of the method. As a notable application, DeepH-hybrid is applied to study large-supercell Moiré-twisted materials, offering the first case study on how the inclusion of exact exchange affects flat bands in magic-angle twisted bilayer graphene. The work generalizes deep-learning electronic structure methods to beyond conventional density functional theory, facilitating the development of deep-learning-based ab initio methods.

Hybrid Quantum Mechanical, Molecular Mechanical, and Machine Learning Potential for Computing Aqueous-Phase Adsorption Free Energies on Metal Surfaces

Performing reliable computer simulations of elementary processes occurring at metal-water interfaces is pivotal for novel catalyst design in sustainable energy applications. Computational catalyst design hinges on the ability to reliably and efficiently compute the potential energy surface (PES) of the system. Due to the large system sizes needed for studying processes at liquid water-metal interfaces, these systems can currently not be described using density functional theory (DFT). In this work, we used a hybrid quantum mechanical, molecular mechanical, and machine learning potential for studying the adsorption behavior of phenol, atomic hydrogen, 2-butanol, and 2-butanone on the (0001) facet of Ru under reducing conditions when Ru is not oxidized. Specifically, we describe the adsorbate and the surrounding metal atoms at the DFT level of theory. Here, we also considered the electrostatic field effect of the water molecules on adsorbate-metal interactions. Next, for the water-water and water-adsorbate interactions, we used established classical force fields. Finally, for the water-Ru surface interaction, for which no reliable force fields have been published, we used Behler-Parrinello high-dimensional neural network potentials (HDNNPs). Employing this setup, we used our explicit solvation for metal surface (eSMS) approach to compute the aqueous-phase effect on the low-coverage adsorption of selected molecules and atoms on the (0001) facet of Ru. In agreement with previous experimental and computational studies of oxygenated molecules over transition metal facets, we found that liquid water destabilizes the tested adsorbates on Ru(0001). Interestingly, our findings indicate that adsorbates on Ru are less affected by the presence of an aqueous phase than on other transition metals (e.g., Pt), highlighting the necessity of experimental investigations of Ru-based catalytic systems in liquid water.

Improving Molecular-Dynamics Simulations for Solid-Liquid Interfaces with Machine-Learning Interatomic Potentials

Emerging developments in artificial intelligence have opened infinite possibilities for material simulation. Depending on the powerful fitting of machine learning algorithms to first-principles data, machine learning interatomic potentials (MLIPs) can effectively balance the accuracy and efficiency problems in molecular dynamics (MD) simulations, serving as powerful tools in various complex physicochemical systems. Consequently, this brings unprecedented enthusiasm for researchers to apply such novel technology in multiple fields to revisit the major scientific problems that have remained controversial owing to the limitations of previous computational methods. Herein, we introduce the evolution of MLIPs, provide valuable application examples for solid-liquid interfaces, and present current challenges. Driven by solving multitudinous difficulties in terms of the accuracy, efficiency, and versatility of MLIPs, this booming technique, combined with molecular simulation methods, will provide an underlying and valuable understanding of interdisciplinary scientific challenges, including materials, physics, and chemistry.

Statistical physical view of statistical inference in Bayesian linear regression model

This paper considers similarities between statistical physics and Bayes inference through the Bayesian linear regression model. Some similarities have been discussed previously, such as the analogy between the marginal likelihood in Bayes inference and the partition function in statistical mechanics. In particular, this paper considers the proposal to associate discrete sample size with inverse temperature [C. H. LaMont and P. A. Wiggins, Phys. Rev. E 99, 052140 (2019)2470-004510.1103/PhysRevE.99.052140]. The previous study suggested that incorporating this similarity motivates the derivation of analogs of thermodynamic functions such as energy and entropy. The study also anticipated that those analogous functions have potential to describe Bayes estimation from physical points of view and to provide physical insights into mechanisms of estimation. This paper incorporates a macroscopic perspective as an asymptotics similar to the thermodynamic limit into the previous suggestion. Its motivation stems from the statistical mechanical concept of deriving thermodynamic functions that characterize macroscopic properties of macroscopic systems. This incorporation not only allows analogs of macroscopic thermodynamic functions to be considered but also suggests a candidate for an analog of inverse temperature with continuity, which is partly consistent with the previous proposal to associate the discrete sample size with inverse temperature. On the basis of this suggestion, we analyze analogs of macroscopic thermodynamic functions for a Bayesian linear regression model which is the basis of various machine learning models. We further investigate, through the behavior of these functions, how Bayes estimation is described from the perspective of physics and what kind of physical insight is obtained. As a result, the estimation of regression coefficients, which is the primary task of regression, appears to be described by the physical picture of balance between decreasing energy and increasing entropy as in equilibrium states of thermodynamic systems. More specifically we observe the physical view of Bayes inference as follows: the estimation succeeds where the effect of decreasing energy is dominant at low temperature. On the other hand, the estimation fails where the effect of increasing entropy is dominant at high temperature.

Stable and accurate atomistic simulations of flexible molecules using conformationally generalisable machine learned potentials

Computational simulation methods based on machine learned potentials (MLPs) promise to revolutionise shape prediction of flexible molecules in solution, but their widespread adoption has been limited by the way in which training data is generated. Here, we present an approach which allows the key conformational degrees of freedom to be properly represented in reference molecular datasets. MLPs trained on these datasets using a global descriptor scheme are generalisable in conformational space, providing quantum chemical accuracy for all conformers. These MLPs are capable of propagating long, stable molecular dynamics trajectories, an attribute that has remained a challenge. We deploy the MLPs in obtaining converged conformational free energy surfaces for flexible molecules via well-tempered metadynamics simulations; this approach provides a hitherto inaccessible route to accurately computing the structural, dynamical and thermodynamical properties of a wide variety of flexible molecular systems. It is further demonstrated that MLPs must be trained on reference datasets with complete coverage of conformational space, including in barrier regions, to achieve stable molecular dynamics trajectories.

Compensation States Approach in the Hybrid Diabatization Scheme: Extension to Multidimensional Data and Properties

The diabatization of reactive systems for more than just a couple of states is a very demanding problem and generally requires advanced diabatization techniques. Especially for dissociative processes, the drastic changes in the adiabatic wave functions often would require large diabatic state bases, which quickly become impractical. Recently, we addressed this problem by the compensation states approach developed in the context of our hybrid diabatization scheme. This scheme utilizes wave function as well as energy data in combination with a diabatic potential model. In regions where the initial diabatic state basis becomes insufficient for an appropriate representation of the adiabatic states, new model states are generated. The new model states compensate for the state space not spanned by the initial diabatic basis. Such a compensation state is obtained by projecting the initial diabatic state space out of the adiabatic wave function. This yields a very efficient basis representation of the electronic Hamiltonian. The present work presents two new aspects. First, it is shown how other operators like the spin-orbit operator in the framework of the Effective Relativistic Coupling by Asymptotic Representation (ERCAR) can be evaluated in this compact model state space without losing the correct wave function information and accuracy. Second, the extension of the approach to multidimensional potential energy surface models is presented for methyl iodide including the C-I dissociation coordinate and the angular H3C-I bending coordinates.

Construction of High Accuracy Machine Learning Interatomic Potential for Surface/Interface of Nanomaterials-A Review

The inherent discontinuity and unique dimensional attributes of nanomaterial surfaces and interfaces bestow them with various exceptional properties. These properties, however, also introduce difficulties for both experimental and computational studies. The advent of machine learning interatomic potential (MLIP) addresses some of the limitations associated with empirical force fields, presenting a valuable avenue for accurate simulations of these surfaces/interfaces of nanomaterials. Central to this approach is the idea of capturing the relationship between system configuration and potential energy, leveraging the proficiency of machine learning (ML) to precisely approximate high-dimensional functions. This review offers an in-depth examination of MLIP principles and their execution and elaborates on their applications in the realm of nanomaterial surface and interface systems. The prevailing challenges faced by this potent methodology are also discussed.

Deep-Learning Density Functional Perturbation Theory

Calculating perturbation response properties of materials from first principles provides a vital link between theory and experiment, but is bottlenecked by the high computational cost. Here, a general framework is proposed to perform density functional perturbation theory (DFPT) calculations by neural networks, greatly improving the computational efficiency. Automatic differentiation is applied on neural networks, facilitating accurate computation of derivatives. High efficiency and good accuracy of the approach are demonstrated by studying electron-phonon coupling and related physical quantities. This work brings deep-learning density functional theory and DFPT into a unified framework, creating opportunities for developing ab initio artificial intelligence.

The effective relativistic coupling by asymptotic representation approach for molecules with multiple relativistic atoms

The Effective Relativistic Coupling by Asymptotic Representation (ERCAR) approach is a method to generate fully coupled diabatic potential energy surfaces (PESs) including relativistic effects, especially spin-orbit coupling. The spin-orbit coupling of a full molecule is determined only by the atomic states of selected relativistically treated atoms. The full molecular coupling effect is obtained by a diabatization with respect to asymptotic states, resulting in the correct geometry dependence of the spin-orbit effect. The ERCAR approach has been developed over the last decade and initially only for molecules with a single relativistic atom. This work presents its extension to molecules with more than a single relativistic atom using the iodine molecule as a proof-of-principle example. The theory for the general multiple atomic ERCAR approach is given. In this case, the diabatic basis is defined at the asymptote where all relativistic atoms are separated from the remaining molecular fragment. The effective spin-orbit operator is then a sum of spin-orbit operators acting on isolated relativistic atoms. PESs for the iodine molecule are developed within the new approach and it is shown that the resulting fine structure states are in good agreement with spin-orbit ab initio calculations.

Hydrogen atom scattering at the Al2O3(0001) surface: a combined experimental and theoretical study

Investigating atom-surface interactions is the key to an in-depth understanding of chemical processes at interfaces, which are of central importance in many fields - from heterogeneous catalysis to corrosion. In this work, we present a joint experimental and theoretical effort to gain insights into the atomistic details of hydrogen atom scattering at the α-Al2O3(0001) surface. Surprisingly, this system has been hardly studied to date, although hydrogen atoms as well as α-Al2O3 are omnipresent in catalysis as reactive species and support oxide, respectively. We address this system by performing hydrogen atom beam scattering experiments and molecular dynamics (MD) simulations based on a high-dimensional machine learning potential trained to density functional theory data. Using this combination of methods we are able to probe the properties of the multidimensional potential energy surface governing the scattering process. Specifically, we compare the angular distribution and the kinetic energy loss of the scattered atoms obtained in experiment with a large number of MD trajectories, which, moreover, allow to identify the underlying impact sites at the surface.

Machine Learning Prediction of the Experimental Transition Temperature of Fe(II) Spin-Crossover Complexes

Spin-crossover (SCO) complexes are materials that exhibit changes in the spin state in response to external stimuli, with potential applications in molecular electronics. It is challenging to know a priori how to design ligands to achieve the delicate balance of entropic and enthalpic contributions needed to tailor a transition temperature close to room temperature. We leverage the SCO complexes from the previously curated SCO-95 data set [Vennelakanti et al. J. Chem. Phys. 159, 024120 (2023)] to train three machine learning (ML) models for transition temperature (T1/2) prediction using graph-based revised autocorrelations as features. We perform feature selection using random forest-ranked recursive feature addition (RF-RFA) to identify the features essential to model transferability. Of the ML models considered, the full feature set RF and recursive feature addition RF models perform best, achieving moderate correlation to experimental T1/2 values. We then compare ML T1/2 predictions to those from three previously identified best-performing density functional approximations (DFAs) which accurately predict SCO behavior across SCO-95, finding that the ML models predict T1/2 more accurately than the best-performing DFAs. In addition, we study ML model predictions for a set of 18 SCO complexes for which only estimated T1/2 values are available. Upon excluding outliers from this set, the RF-RFA RF model shows a strong correlation to estimated T1/2 values with a Pearson's r of 0.82. In contrast, DFA-predicted T1/2 values have large errors and show no correlation to estimated T1/2 values over the same set of complexes. Overall, our study demonstrates slightly superior performance of ML models in comparison with some of the best-performing DFAs, and we expect ML models to improve further as larger data sets of SCO complexes are curated and become available for model training.

Accuracy Assessment of Atomistic Neural Network Potentials: The Impact of Cutoff Radius and Message Passing

Atomistic neural network potentials have achieved great success in accelerating atomistic simulations in complicated systems in recent years. They are typically based on the atomic decomposition of total properties, truncating the interatomic correlations to a local environment within a given cutoff radius. A more recently developed message passing (MP) neural network framework can, in principle, incorporate nonlocal effects through iteratively correlating some atoms outside the cutoff sphere with atoms inside, a process referred to as MP. However, how the model accuracy depends on the cutoff radius and the MP process has rarely been discussed. In this work, we investigate this dependence using a recursively embedded atom neural network method that possesses both local and MP features, in two representative systems: liquid H2O and solid Al2O3. We focus on how these settings influence predictions for structural and vibrational properties, namely, radial distribution functions (RDFs) and vibrational density of states (VDOSs). We find that while MP lowers test errors of energy and forces in general, it may not improve the prediction for RDFs and/or VDOSs if direct interatomic correlations in the local environment are insufficiently described. A cutoff radius exceeding the first neighbor shell is necessary, beyond which involving MP quickly enhances the model accuracy until convergence. This is a potentially more efficient way to increase the model accuracy than directly increasing the cutoff radius, especially with more memory savings in the GPU implementation. Our findings also suggest that using the mean test error as the measure of the model accuracy alone is inadequate.

Stress and heat flux via automatic differentiation

Machine-learning potentials provide computationally efficient and accurate approximations of the Born-Oppenheimer potential energy surface. This potential determines many materials properties and simulation techniques usually require its gradients, in particular forces and stress for molecular dynamics, and heat flux for thermal transport properties. Recently developed potentials feature high body order and can include equivariant semi-local interactions through message-passing mechanisms. Due to their complex functional forms, they rely on automatic differentiation (AD), overcoming the need for manual implementations or finite-difference schemes to evaluate gradients. This study discusses how to use AD to efficiently obtain forces, stress, and heat flux for such potentials, and provides a model-independent implementation. The method is tested on the Lennard-Jones potential, and then applied to predict cohesive properties and thermal conductivity of tin selenide using an equivariant message-passing neural network potential.

GeoT: A Geometry-Aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning

In recent years, molecular representation learning has emerged as a key area of focus in various chemical tasks. However, many existing models fail to fully consider the geometric information on molecular structures, resulting in less intuitive representations. Moreover, the widely used message passing mechanism is limited to providing the interpretation of experimental results from a chemical perspective. To address these challenges, we introduce a novel transformer-based framework for molecular representation learning, named the geometry-aware transformer (GeoT). The GeoT learns molecular graph structures through attention-based mechanisms specifically designed to offer reliable interpretability as well as molecular property prediction. Consequently, the GeoT can generate attention maps of the interatomic relationships associated with training objectives. In addition, the GeoT demonstrates performance comparable to that of MPNN-based models while achieving reduced computational complexity. Our comprehensive experiments, including an empirical simulation, reveal that the GeoT effectively learns chemical insights into molecular structures, bridging the gap between artificial intelligence and molecular sciences.

DFT-assisted low-dimensional carbon-based electrocatalysts design and mechanism study: a review

Low-dimensional carbon-based (LDC) materials have attracted extensive research attention in electrocatalysis because of their unique advantages such as structural diversity, low cost, and chemical tolerance. They have been widely used in a broad range of electrochemical reactions to relieve environmental pollution and energy crisis. Typical examples include hydrogen evolution reaction (HER), oxygen evolution reaction (OER), oxygen reduction reaction (ORR), carbon dioxide reduction reaction (CO2RR), and nitrogen reduction reaction (NRR). Traditional "trial and error" strategies greatly slowed down the rational design of electrocatalysts for these important applications. Recent studies show that the combination of density functional theory (DFT) calculations and experimental research is capable of accurately predicting the structures of electrocatalysts, thus revealing the catalytic mechanisms. Herein, current well-recognized collaboration methods of theory and practice are reviewed. The commonly used calculation methods and the basic functionals are briefly summarized. Special attention is paid to descriptors that are widely accepted as a bridge linking the structure and activity and the breakthroughs for high-volume accurate prediction of electrocatalysts. Importantly, correlated multiple descriptors are used to systematically describe the complicated interfacial electrocatalytic processes of LDC catalysts. Furthermore, machine learning and high-throughput simulations are crucial in assisting the discovery of new multiple descriptors and reaction mechanisms. This review will guide the further development of LDC electrocatalysts for extended applications from the aspect of DFT computations.

Optimizing the architecture of Behler-Parrinello neural network potentials

The architecture of neural network potentials is typically optimized at the beginning of the training process and remains unchanged throughout. Here, we investigate the accuracy of Behler-Parrinello neural network potentials for varying training set sizes. Using the QM9 and 3BPA datasets, we show that adjusting the network architecture according to the training set size improves the accuracy significantly. We demonstrate that both an insufficient and an excessive number of fitting parameters can have a detrimental impact on the accuracy of the neural network potential. Furthermore, we investigate the influences of descriptor complexity, neural network depth, and activation function on the model's performance. We find that for the neural network potentials studied here, two hidden layers yield the best accuracy and that unbounded activation functions outperform bounded ones.

MBX: A many-body energy and force calculator for data-driven many-body simulations

Many-Body eXpansion (MBX) is a C++ library that implements many-body potential energy functions (PEFs) within the "many-body energy" (MB-nrg) formalism. MB-nrg PEFs integrate an underlying polarizable model with explicit machine-learned representations of many-body interactions to achieve chemical accuracy from the gas to the condensed phases. MBX can be employed either as a stand-alone package or as an energy/force engine that can be integrated with generic software for molecular dynamics and Monte Carlo simulations. MBX is parallelized internally using Open Multi-Processing and can utilize Message Passing Interface when available in interfaced molecular simulation software. MBX enables classical and quantum molecular simulations with MB-nrg PEFs, as well as hybrid simulations that combine conventional force fields and MB-nrg PEFs, for diverse systems ranging from small gas-phase clusters to aqueous solutions and molecular fluids to biomolecular systems and metal-organic frameworks.

Accelerated Quantum Mechanics/Molecular Mechanics Simulations via Neural Networks Incorporated with Mechanical Embedding Scheme

A powerful tool to study the mechanism of reactions in solutions or enzymes is to perform the ab initio quantum mechanical/molecular mechanical (QM/MM) molecular dynamics (MD) simulations. However, the computational cost is too high due to the explicit electronic structure calculations at every time step of the simulation. A neural network (NN) method can accelerate the QM/MM-MD simulations, but it has long been a problem to accurately describe the QM/MM electrostatic coupling by NN in the electrostatic embedding (EE) scheme. In this work, we developed a new method to accelerate QM/MM calculations in the mechanic embedding (ME) scheme. The potentials and partial point charges of QM atoms are first learned in vacuo by the embedded atom neural networks (EANN) approach. MD simulations are then performed on this EANN/MM potential energy surface (PES) to obtain free energy (FE) profiles for reactions, in which the QM/MM electrostatic coupling is treated in the mechanic embedding (ME) scheme. Finally, a weighted thermodynamic perturbation (wTP) corrects the FE profiles in the ME scheme to the EE scheme. For two reactions in water and one in methanol, our simulations reproduced the B3LYP/MM free energy profiles within 0.5 kcal/mol with a speed-up of 30-60-fold. The results show that the strategy of combining EANN potential in the ME scheme with the wTP correction is efficient and reliable for chemical reaction simulations in liquid. Another advantage of our method is that the QM PES is independent of the MM subsystem, so it can be applied to various MM environments as demonstrated by an S_N2 reaction studied in water and methanol individually, which used the same EANN PES. The free energy profiles are in excellent accordance with the results obtained from B3LYP/MM-MD simulations. In future, this method will be applied to the reactions of enzymes and their variants.

Size and Quality of Quantum Mechanical Data Set for Training Neural Network Force Fields for Liquid Water

Molecular dynamics simulations have been used in different scientific fields to investigate a broad range of physical systems. However, the accuracy of calculation is based on the model considered to describe the atomic interactions. In particular, ab initio molecular dynamics (AIMD) has the accuracy of density functional theory (DFT) and thus is limited to small systems and a relatively short simulation time. In this scenario, Neural Network Force Fields (NNFFs) have an important role, since they provide a way to circumvent these caveats. In this work, we investigate NNFFs designed at the level of DFT to describe liquid water, focusing on the size and quality of the training data set considered. We show that structural properties are less dependent on the size of the training data set compared to dynamical ones (such as the diffusion coefficient), and a good sampling (selecting data reference for the training process) can lead to a small sample with good precision.

Learning Conductance: Gaussian Process Regression for Molecular Electronics

Experimental studies of charge transport through single molecules often rely on break junction setups, where molecular junctions are repeatedly formed and broken while measuring the conductance, leading to a statistical distribution of conductance values. Modeling this experimental situation and the resulting conductance histograms is challenging for theoretical methods, as computations need to capture structural changes in experiments, including the statistics of junction formation and rupture. This type of extensive structural sampling implies that even when evaluating conductance from computationally efficient electronic structure methods, which typically are of reduced accuracy, the evaluation of conductance histograms is too expensive to be a routine task. Highly accurate quantum transport computations are only computationally feasible for a few selected conformations and thus necessarily ignore the rich conformational space probed in experiments. To overcome these limitations, we investigate the potential of machine learning for modeling conductance histograms, in particular by Gaussian process regression. We show that by selecting specific structural parameters as features, Gaussian process regression can be used to efficiently predict the zero-bias conductance from molecular structures, reducing the computational cost of simulating conductance histograms by an order of magnitude. This enables the efficient calculation of conductance histograms even on the basis of computationally expensive first-principles approaches by effectively reducing the number of necessary charge transport calculations, paving the way toward their routine evaluation.

Neural network potentials for chemistry: concepts, applications and prospects

Artificial Neural Networks (NN) are already heavily involved in methods and applications for frequent tasks in the field of computational chemistry such as representation of potential energy surfaces (PES) and spectroscopic predictions. This perspective provides an overview of the foundations of neural network-based full-dimensional potential energy surfaces, their architectures, underlying concepts, their representation and applications to chemical systems. Methods for data generation and training procedures for PES construction are discussed and means for error assessment and refinement through transfer learning are presented. A selection of recent results illustrates the latest improvements regarding accuracy of PES representations and system size limitations in dynamics simulations, but also NN application enabling direct prediction of physical results without dynamics simulations. The aim is to provide an overview for the current state-of-the-art NN approaches in computational chemistry and also to point out the current challenges in enhancing reliability and applicability of NN methods on a larger scale.

Graph-convolutional neural networks for (QM)ML/MM molecular dynamics simulations

To accurately study the chemical reactions in the condensed phase or within enzymes, both quantum-mechanical description and sufficient configurational sampling are required to reach converged estimates. Here, quantum mechanics/molecular mechanics (QM/MM) molecular dynamics (MD) simulations play an important role, providing QM accuracy for the region of interest at a decreased computational cost. However, QM/MM simulations are still too expensive to study large systems on longer time scales. Recently, machine learning (ML) models have been proposed to replace the QM description. The main limitation of these models lies in the accurate description of long-range interactions present in condensed-phase systems. To overcome this issue, a recent workflow has been introduced combining a semi-empirical method (i.e. density functional tight binding (DFTB)) and a high-dimensional neural network potential (HDNNP) in a Δ-learning scheme. This approach has been shown to be capable of correctly incorporating long-range interactions within a cutoff of 1.4 nm. One of the promising alternative approaches to efficiently take long-range effects into account is the development of graph-convolutional neural networks (GCNNs) for the prediction of the potential-energy surface. In this work, we investigate the use of GCNN models - with and without a Δ-learning scheme - for (QM)ML/MM MD simulations. We show that the Δ-learning approach using a GCNN and DFTB as a baseline achieves competitive performance on our benchmarking set of solutes and chemical reactions in water. This method is additionally validated by performing prospective (QM)ML/MM MD simulations of retinoic acid in water and S-adenoslymethionine interacting with cytosine in water. The results indicate that the Δ-learning GCNN model is a valuable alternative for the (QM)ML/MM MD simulations of condensed-phase systems.

Protein Function Analysis through Machine Learning

Machine learning (ML) has been an important arsenal in computational biology used to elucidate protein function for decades. With the recent burgeoning of novel ML methods and applications, new ML approaches have been incorporated into many areas of computational biology dealing with protein function. We examine how ML has been integrated into a wide range of computational models to improve prediction accuracy and gain a better understanding of protein function. The applications discussed are protein structure prediction, protein engineering using sequence modifications to achieve stability and druggability characteristics, molecular docking in terms of protein-ligand binding, including allosteric effects, protein-protein interactions and protein-centric drug discovery. To quantify the mechanisms underlying protein function, a holistic approach that takes structure, flexibility, stability, and dynamics into account is required, as these aspects become inseparable through their interdependence. Another key component of protein function is conformational dynamics, which often manifest as protein kinetics. Computational methods that use ML to generate representative conformational ensembles and quantify differences in conformational ensembles important for function are included in this review. Future opportunities are highlighted for each of these topics.

Magnetic phase transition of monolayer chromium trihalides investigated with machine learning: toward a universal magnetic Hamiltonian

The prediction of magnetic phase transitions often requires model Hamiltonians to describe the necessary magnetic interactions. The advance of machine learning provides an opportunity to build a unified approach that can treat various magnetic systems without proposing new model Hamiltonians. Here, we develop such an approach by proposing a novel set of descriptors that describes the magnetic interactions and training the artificial neural network (ANN) that plays the role of a universal magnetic Hamiltonian. We then employ this approach and Monte Carlo simulation to investigate the magnetic phase transition of two-dimensional monolayer chromium trihalides using the trained ANNs as energy calculator. We show that the machine-learning-based approach shows advantages over traditional methods in the investigation of ferromagnetic and antiferromagnetic phase transitions, demonstrating its potential for other magnetic systems.

Grazing incidence fast atom and molecule diffraction: theoretical challenges

This perspective article reviews the state-of-the-art of grazing incidence fast atom and molecule diffraction (GIFAD and GIFMD) simulations and addresses the main challenges that theorists, aiming to provide useful inputs in this topic, are facing. We first discuss briefly the methods used to build accurate potential energy surfaces describing the interaction between the projectile and the surface. Subsequently, we focus on the dynamics simulation methods for GIFAD, a phenomenon that has received a lot of experimental attention since 2007, when the first measurements were published. Following this experimental effort, theorists have developed and adapted a bunch of methods able to simulate, analyze and extract information from the experimental outputs. We review these methods, from the very simple ones based on classical dynamics to the full quantum ones, paying special attention to more versatile semiclassical approaches, which include quantum ingredients in the dynamics at a computational cost only slightly higher than that required in classical dynamics. Within the semiclassical framework it is possible, for example, to include in the dynamics the surface phonons and the projectile coherence, two factors that may have a relevant influence on the experimental measurements, at a reasonable computational cost. Finally, we address GIFMD, a phenomenon that has received much less attention and for which there is still a lot of room for research. We review the few examples of GIFMD available in the literature, and we discuss new phenomena associated with the molecular internal degrees of freedom, which may have some impact in other closely related fields, such as molecular reactivity on metal surfaces. Finally, we point out opened questions, raised from the comparisons between theoretical and experimental results, which claim for further experimental efforts.

Predicting properties of periodic systems from cluster data: A case study of liquid water

The accuracy of the training data limits the accuracy of bulk properties from machine-learned potentials. For example, hybrid functionals or wave-function-based quantum chemical methods are readily available for cluster data but effectively out of scope for periodic structures. We show that local, atom-centered descriptors for machine-learned potentials enable the prediction of bulk properties from cluster model training data, agreeing reasonably well with predictions from bulk training data. We demonstrate such transferability by studying structural and dynamical properties of bulk liquid water with density functional theory and have found an excellent agreement with experimental and theoretical counterparts.

Neural Network Potentials: A Concise Overview of Methods

In the past two decades, machine learning potentials (MLPs) have reached a level of maturity that now enables applications to large-scale atomistic simulations of a wide range of systems in chemistry, physics, and materials science. Different machine learning algorithms have been used with great success in the construction of these MLPs. In this review, we discuss an important group of MLPs relying on artificial neural networks to establish a mapping from the atomic structure to the potential energy. In spite of this common feature, there are important conceptual differences among MLPs, which concern the dimensionality of the systems, the inclusion of long-range electrostatic interactions, global phenomena like nonlocal charge transfer, and the type of descriptor used to represent the atomic structure, which can be either predefined or learnable. A concise overview is given along with a discussion of the open challenges in the field.

Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Ab initio molecular dynamics with enhanced sampling in heterogeneous catalysis

Enhanced sampling ab initio simulations enable to study chemical phenomena in catalytic systems including thermal effects & anharmonicity, & collective dynamics describing enthalpic & entropic contributions, which can significantly impact on reaction free energy landscapes.

Predicting atomic-level reaction mechanisms for SN2 reactions via machine learning

Identifying atomic-level reaction mechanisms is an essential step in chemistry. In this study, we develop a joint-voting model based on three parallel machine-learning algorithms to predict atomic-level and dynamical mechanisms trained with 1700 trajectories. Three predictive experiments are carried out with the training trajectories divided into ten, seven, and five classes. The results indicate that, as the number of trajectories in each class increases from the ten- to five-class model, the five-class model converges the fastest and the prediction success rate increases. The number of trajectories in each experiment to get the predictive models converged is 100, 100, and 70, respectively. The prediction accuracy increases from 88.3% for the ten-class experiment, to 91.0% for the seven-class, and to 92.0% for the five-class. Our study demonstrates that machine learning can also be used to predict elementary dynamical processes of structural evolution along time, that is, atomic-level reaction mechanisms.

Thermally Averaged Magnetic Anisotropy Tensors via Machine Learning Based on Gaussian Moments

We propose a machine learning method to model molecular tensorial quantities, namely, the magnetic anisotropy tensor, based on the Gaussian moment neural network approach. We demonstrate that the proposed methodology can achieve an accuracy of 0.3-0.4 cm^-1 and has excellent generalization capability for out-of-sample configurations. Moreover, in combination with machine-learned interatomic potential energies based on Gaussian moments, our approach can be applied to study the dynamic behavior of magnetic anisotropy tensors and provide a unique insight into spin-phonon relaxation.

OrbNet Denali: A machine learning potential for biological and organic chemistry with semi-empirical cost and DFT accuracy

We present OrbNet Denali, a machine learning model for an electronic structure that is designed as a drop-in replacement for ground-state density functional theory (DFT) energy calculations. The model is a message-passing graph neural network that uses symmetry-adapted atomic orbital features from a low-cost quantum calculation to predict the energy of a molecule. OrbNet Denali is trained on a vast dataset of 2.3 × 10⁶ DFT calculations on molecules and geometries. This dataset covers the most common elements in biochemistry and organic chemistry (H, Li, B, C, N, O, F, Na, Mg, Si, P, S, Cl, K, Ca, Br, and I) and charged molecules. OrbNet Denali is demonstrated on several well-established benchmark datasets, and we find that it provides accuracy that is on par with modern DFT methods while offering a speedup of up to three orders of magnitude. For the GMTKN55 benchmark set, OrbNet Denali achieves WTMAD-1 and WTMAD-2 scores of 7.19 and 9.84, on par with modern DFT functionals. For several GMTKN55 subsets, which contain chemical problems that are not present in the training set, OrbNet Denali produces a mean absolute error comparable to those of DFT methods. For the Hutchison conformer benchmark set, OrbNet Denali has a median correlation coefficient of R² = 0.90 compared to the reference DLPNO-CCSD(T) calculation and R² = 0.97 compared to the method used to generate the training data (ωB97X-D3/def2-TZVP), exceeding the performance of any other method with a similar cost. Similarly, the model reaches chemical accuracy for non-covalent interactions in the S66x10 dataset. For torsional profiles, OrbNet Denali reproduces the torsion profiles of ωB97X-D3/def2-TZVP with an average mean absolute error of 0.12 kcal/mol for the potential energy surfaces of the diverse fragments in the TorsionNet500 dataset.

Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments

Artificial neural networks (NNs) are one of the most frequently used machine learning approaches to construct interatomic potentials and enable efficient large-scale atomistic simulations with almost ab initio accuracy. However, the simultaneous training of NNs on energies and forces, which are a prerequisite for, e.g., molecular dynamics simulations, can be demanding. In this work, we present an improved NN architecture based on the previous GM-NN model [Zaverkin V.; Kästner, J. J. Chem. Theory Comput. 2020, 16, 5410-5421], which shows an improved prediction accuracy and considerably reduced training times. Moreover, we extend the applicability of Gaussian moment-based interatomic potentials to periodic systems and demonstrate the overall excellent transferability and robustness of the respective models. The fast training by the improved methodology is a prerequisite for training-heavy workflows such as active learning or learning-on-the-fly.

MB-Fit: Software infrastructure for data-driven many-body potential energy functions

Many-body potential energy functions (MB-PEFs), which integrate data-driven representations of many-body short-range quantum mechanical interactions with physics-based representations of many-body polarization and long-range interactions, have recently been shown to provide high accuracy in the description of molecular interactions from the gas to the condensed phase. Here, we present MB-Fit, a software infrastructure for the automated development of MB-PEFs for generic molecules within the TTM-nrg (Thole-type model energy) and MB-nrg (many-body energy) theoretical frameworks. Besides providing all the necessary computational tools for generating TTM-nrg and MB-nrg PEFs, MB-Fit provides a seamless interface with the MBX software, a many-body energy and force calculator for computer simulations. Given the demonstrated accuracy of the MB-PEFs, particularly within the MB-nrg framework, we believe that MB-Fit will enable routine predictive computer simulations of generic (small) molecules in the gas, liquid, and solid phases, including, but not limited to, the modeling of quantum isomeric equilibria in molecular clusters, solvation processes, molecular crystals, and phase diagrams.

Efficient Approximation of Potential Energy Surfaces with Mixed-Basis Interpolation

The potential energy surface (PES) describes the energy of a chemical system as a function of its geometry and is a fundamental concept in modern chemistry. A PES provides much useful information about the system, including the structures and energies of various stationary points, such as stable conformers (local minima) and transition states (first-order saddle points) connected by a minimum-energy path. Our group has previously produced surrogate reduced-dimensional PESs using sparse interpolation along chemically significant reaction coordinates, such as bond lengths, bond angles, and torsion angles. These surrogates used a single interpolation basis, either polynomials or trigonometric functions, in every dimension. However, relevant molecular dynamics (MD) simulations often involve some combination of both periodic and nonperiodic coordinates. Using a trigonometric basis on nonperiodic coordinates, such as bond lengths, leads to inaccuracies near the domain boundary. Conversely, polynomial interpolation on the periodic coordinates does not enforce the periodicity of the surrogate PES gradient, leading to nonconservation of total energy even in a microcanonical ensemble. In this work, we present an interpolation method that uses trigonometric interpolation on the periodic reaction coordinates and polynomial interpolation on the nonperiodic coordinates. We apply this method to MD simulations of possible isomerization pathways of azomethane between cis and trans conformers. This method is the only known interpolative method that appropriately conserves total energy in systems with both periodic and nonperiodic reaction coordinates. In addition, compared to all-polynomial interpolation, the mixed basis requires fewer electronic structure calculations to obtain a given level of accuracy, is an order of magnitude faster, and is freely available on GitHub.

Gaussian Process Regression for Materials and Molecules

We provide an introduction to Gaussian process regression (GPR) machine-learning methods in computational materials science and chemistry. The focus of the present review is on the regression of atomistic properties: in particular, on the construction of interatomic potentials, or force fields, in the Gaussian Approximation Potential (GAP) framework; beyond this, we also discuss the fitting of arbitrary scalar, vectorial, and tensorial quantities. Methodological aspects of reference data generation, representation, and regression, as well as the question of how a data-driven model may be validated, are reviewed and critically discussed. A survey of applications to a variety of research questions in chemistry and materials science illustrates the rapid growth in the field. A vision is outlined for the development of the methodology in the years to come.

Machine Learning for Electronically Excited States of Molecules

Electronically excited states of molecules are at the heart of photochemistry, photophysics, as well as photobiology and also play a role in material science. Their theoretical description requires highly accurate quantum chemical calculations, which are computationally expensive. In this review, we focus on not only how machine learning is employed to speed up such excited-state simulations but also how this branch of artificial intelligence can be used to advance this exciting research field in all its aspects. Discussed applications of machine learning for excited states include excited-state dynamics simulations, static calculations of absorption spectra, as well as many others. In order to put these studies into context, we discuss the promises and pitfalls of the involved machine learning techniques. Since the latter are mostly based on quantum chemistry calculations, we also provide a short introduction into excited-state electronic structure methods and approaches for nonadiabatic dynamics simulations and describe tricks and problems when using them in machine learning for excited states of molecules.

Ab Initio Machine Learning in Chemical Compound Space

Chemical compound space (CCS), the set of all theoretically conceivable combinations of chemical elements and (meta-)stable geometries that make up matter, is colossal. The first-principles based virtual sampling of this space, for example, in search of novel molecules or materials which exhibit desirable properties, is therefore prohibitive for all but the smallest subsets and simplest properties. We review studies aimed at tackling this challenge using modern machine learning techniques based on (i) synthetic data, typically generated using quantum mechanics based methods, and (ii) model architectures inspired by quantum mechanics. Such Quantum mechanics based Machine Learning (QML) approaches combine the numerical efficiency of statistical surrogate models with an ab initio view on matter. They rigorously reflect the underlying physics in order to reach universality and transferability across CCS. While state-of-the-art approximations to quantum problems impose severe computational bottlenecks, recent QML based developments indicate the possibility of substantial acceleration without sacrificing the predictive power of quantum mechanics.

Machine Learning for Electronically Excited States of Molecules

Electronically excited states of molecules are at the heart of photochemistry, photophysics, as well as photobiology and also play a role in material science. Their theoretical description requires highly accurate quantum chemical calculations, which are computationally expensive. In this review, we focus on not only how machine learning is employed to speed up such excited-state simulations but also how this branch of artificial intelligence can be used to advance this exciting research field in all its aspects. Discussed applications of machine learning for excited states include excited-state dynamics simulations, static calculations of absorption spectra, as well as many others. In order to put these studies into context, we discuss the promises and pitfalls of the involved machine learning techniques. Since the latter are mostly based on quantum chemistry calculations, we also provide a short introduction into excited-state electronic structure methods and approaches for nonadiabatic dynamics simulations and describe tricks and problems when using them in machine learning for excited states of molecules.

Strategies for the construction of machine-learning potentials for accurate and efficient atomic-scale simulations

Recent advances in machine-learning interatomic potentials have enabled the efficient modeling of complex atomistic systems with an accuracy that is comparable to that of conventional quantum-mechanics based methods. At the same time, the construction of new machine-learning potentials can seem a daunting task, as it involves data-science techniques that are not yet common in chemistry and materials science. Here, we provide a tutorial-style overview of strategies and best practices for the construction of artificial neural network (ANN) potentials. We illustrate the most important aspects of (a) data collection, (b) model selection, (c) training and validation, and (d) testing and refinement of ANN potentials on the basis of practical examples. Current research in the areas of active learning and delta learning are also discussed in the context of ANN potentials. This tutorial review aims at equipping computational chemists and materials scientists with the required background knowledge for ANN potential construction and application, with the intention to accelerate the adoption of the method, so that it can facilitate exciting research that would otherwise be challenging with conventional strategies.

Machine learned Hückel theory: Interfacing physics and deep neural networks

The Hückel Hamiltonian is an incredibly simple tight-binding model known for its ability to capture qualitative physics phenomena arising from electron interactions in molecules and materials. Part of its simplicity arises from using only two types of empirically fit physics-motivated parameters: the first describes the orbital energies on each atom and the second describes electronic interactions and bonding between atoms. By replacing these empirical parameters with machine-learned dynamic values, we vastly increase the accuracy of the extended Hückel model. The dynamic values are generated with a deep neural network, which is trained to reproduce orbital energies and densities derived from density functional theory. The resulting model retains interpretability, while the deep neural network parameterization is smooth and accurate and reproduces insightful features of the original empirical parameterization. Overall, this work shows the promise of utilizing machine learning to formulate simple, accurate, and dynamically parameterized physics models.

The Rise of Neural Networks for Materials and Chemical Dynamics

Machine learning (ML) is quickly becoming a premier tool for modeling chemical processes and materials. ML-based force fields, trained on large data sets of high-quality electron structure calculations, are particularly attractive due their unique combination of computational efficiency and physical accuracy. This Perspective summarizes some recent advances in the development of neural network-based interatomic potentials. Designing high-quality training data sets is crucial to overall model accuracy. One strategy is active learning, in which new data are automatically collected for atomic configurations that produce large ML uncertainties. Another strategy is to use the highest levels of quantum theory possible. Transfer learning allows training to a data set of mixed fidelity. A model initially trained to a large data set of density functional theory calculations can be significantly improved by retraining to a relatively small data set of expensive coupled cluster theory calculations. These advances are exemplified by applications to molecules and materials.

Machine-learned potentials for next-generation matter simulations

The choice of simulation methods in computational materials science is driven by a fundamental trade-off: bridging large time- and length-scales with highly accurate simulations at an affordable computational cost. Venturing the investigation of complex phenomena on large scales requires fast yet accurate computational methods. We review the emerging field of machine-learned potentials, which promises to reach the accuracy of quantum mechanical computations at a substantially reduced computational cost. This Review will summarize the basic principles of the underlying machine learning methods, the data acquisition process and active learning procedures. We highlight multiple recent applications of machine-learned potentials in various fields, ranging from organic chemistry and biomolecules to inorganic crystal structure predictions and surface science. We furthermore discuss the developments required to promote a broader use of ML potentials, and the possibility of using them to help solve open questions in materials science and facilitate fully computational materials design.