General-Purpose Machine Learning Potentials Capturing Nonlocal Charge Transfer

Efficient determination of Born-effective charges, LO-TO splitting, and Raman tensors of solids with a real-space atom-centered deep learning approach

We introduce a deep neural network (DNN) framework called theReal-spaceAtomicDecompositionNETwork (radnet), which is capable of making accurate predictions of polarization and of electronic dielectric permittivity tensors in solids and aims to address limitations of previously available machine learning models for Raman predictions in periodic systems. This framework builds on previous, atom-centered approaches while utilizing deep convolutional neural networks. We report excellent accuracies on direct predictions for two prototypical examples: GaAs and BN. We then use automatic differentiation to efficiently calculate the Born-effective charges, longitudinal optical-transverse optical (LO-TO) splitting frequencies, and Raman tensors of these materials. We compute the Raman spectra, and find agreement withab initioresults. Lastly, we explore ways to generalize the predictions of polarization while taking into account periodic boundary conditions and symmetries.

Learning QM/MM potential using equivariant multiscale model

The machine learning (ML) method emerges as an efficient and precise surrogate model for high-level electronic structure theory. Its application has been limited to closed chemical systems without considering external potentials from the surrounding environment. To address this limitation and incorporate the influence of external potentials, polarization effects, and long-range interactions between a chemical system and its environment, the first two terms of the Taylor expansion of an electrostatic operator have been used as extra input to the existing ML model to represent the electrostatic environments. However, high-order electrostatic interaction is often essential to account for external potentials from the environment. The existing models based only on invariant features cannot capture significant distribution patterns of the external potentials. Here, we propose a novel ML model that includes high-order terms of the Taylor expansion of an electrostatic operator and uses an equivariant model, which can generate a high-order tensor covariant with rotations as a base model. Therefore, we can use the multipole-expansion equation to derive a useful representation by accounting for polarization and intermolecular interaction. Moreover, to deal with long-range interactions, we follow the same strategy adopted to derive long-range interactions between a target system and its environment media. Our model achieves higher prediction accuracy and transferability among various environment media with these modifications.

Experiment-Driven Atomistic Materials Modeling: A Case Study Combining X-Ray Photoelectron Spectroscopy and Machine Learning Potentials to Infer the Structure of Oxygen-Rich Amorphous Carbon

An important yet challenging aspect of atomistic materials modeling is reconciling experimental and computational results. Conventional approaches involve generating numerous configurations through molecular dynamics or Monte Carlo structure optimization and selecting the one with the closest match to experiment. However, this inefficient process is not guaranteed to succeed. We introduce a general method to combine atomistic machine learning (ML) with experimental observables that produces atomistic structures compatible with experiment by design. We use this approach in combination with grand-canonical Monte Carlo within a modified Hamiltonian formalism, to generate configurations that agree with experimental data and are chemically sound (low in energy). We apply our approach to understand the atomistic structure of oxygenated amorphous carbon (a-COx), an intriguing carbon-based material, to answer the question of how much oxygen can be added to carbon before it fully decomposes into CO and CO2. Utilizing an ML-based X-ray photoelectron spectroscopy (XPS) model trained from GW and density functional theory (DFT) data, in conjunction with an ML interatomic potential, we identify a-COx structures compliant with experimental XPS predictions that are also energetically favorable with respect to DFT. Employing a network analysis, we accurately deconvolve the XPS spectrum into motif contributions, both revealing the inaccuracies inherent to experimental XPS interpretation and granting us atomistic insight into the structure of a-COx. This method generalizes to multiple experimental observables and allows for the elucidation of the atomistic structure of materials directly from experimental data, thereby enabling experiment-driven materials modeling with a degree of realism previously out of reach.

Machine learning interatomic potential: Bridge the gap between small-scale models and realistic device-scale simulations

Machine learning interatomic potential (MLIP) overcomes the challenges of high computational costs in density-functional theory and the relatively low accuracy in classical large-scale molecular dynamics, facilitating more efficient and precise simulations in materials research and design. In this review, the current state of the four essential stages of MLIP is discussed, including data generation methods, material structure descriptors, six unique machine learning algorithms, and available software. Furthermore, the applications of MLIP in various fields are investigated, notably in phase-change memory materials, structure searching, material properties predicting, and the pre-trained universal models. Eventually, the future perspectives, consisting of standard datasets, transferability, generalization, and trade-off between accuracy and complexity in MLIPs, are reported.

DFTB Simulation of Charged Clusters Using Machine Learning Charge Inference

We present a modification to self-consistent charge density functional-based tight binding (SCC-DFTB), which allows computation based on approximate atomic charges. We obtain these charges by means of a machine learning (ML) process that combines a Coulomb model with a neural network. This allows us to avoid the SCC cycles in the SCC-DFTB calculation while maintaining its accuracy. The main input of the model is the atomic positions characterized by a set of atom-centered symmetry functions. The charge inference from our ML algorithm is as close as 10-2 units of charge from the exact SCC solution. Our ML-DFTB approach provides a good approximation of the density matrix and of the energy and forces with only a single diagonalization. This is a significant computational saving with respect to the complete SCC algorithm, which allows us to investigate a bigger ensemble of atoms. We show the quality of our approach in the case of charged silicon carbide (SiC) clusters. The ML-DFTB potential energy surface (PES) mimics the SCC-DFTB PES rather well, despite its simplicity. This allows us to obtain the same geometric structure ordering with respect to energy for small clusters. The dissociation barriers for ion emission are well-reproduced, which opens the way to investigating ion field emission and charged cluster stability. The ML-DFTB approach is obviously not limited to charged clusters or SiC materials. It opens a new route to investigate larger clusters than those investigated by standard SCC-DFTB, as well as surface and solid-state chemistry at the atomic level.

Construction of High Accuracy Machine Learning Interatomic Potential for Surface/Interface of Nanomaterials-A Review

The inherent discontinuity and unique dimensional attributes of nanomaterial surfaces and interfaces bestow them with various exceptional properties. These properties, however, also introduce difficulties for both experimental and computational studies. The advent of machine learning interatomic potential (MLIP) addresses some of the limitations associated with empirical force fields, presenting a valuable avenue for accurate simulations of these surfaces/interfaces of nanomaterials. Central to this approach is the idea of capturing the relationship between system configuration and potential energy, leveraging the proficiency of machine learning (ML) to precisely approximate high-dimensional functions. This review offers an in-depth examination of MLIP principles and their execution and elaborates on their applications in the realm of nanomaterial surface and interface systems. The prevailing challenges faced by this potent methodology are also discussed.

Machine Learning Potentials with the Iterative Boltzmann Inversion: Training to Experiment

Methodologies for training machine learning potentials (MLPs) with quantum-mechanical simulation data have recently seen tremendous progress. Experimental data have a very different character than simulated data, and most MLP training procedures cannot be easily adapted to incorporate both types of data into the training process. We investigate a training procedure based on iterative Boltzmann inversion that produces a pair potential correction to an existing MLP using equilibrium radial distribution function data. By applying these corrections to an MLP for pure aluminum based on density functional theory, we observe that the resulting model largely addresses previous overstructuring in the melt phase. Interestingly, the corrected MLP also exhibits improved performance in predicting experimental diffusion constants, which are not included in the training procedure. The presented method does not require autodifferentiating through a molecular dynamics solver and does not make assumptions about the MLP architecture. Our results suggest a practical framework for incorporating experimental data into machine learning models to improve the accuracy of molecular dynamics simulations.

DP/MM: A Hybrid Model for Zinc-Protein Interactions in Molecular Dynamics

Zinc-containing proteins are vital for many biological processes, yet accurately modeling them using classical force fields is hindered by complicated polarization and charge transfer effects. This study introduces DP/MM, a hybrid force field scheme that utilizes a deep potential model to correct the atomic forces of zinc ions and their coordinated atoms, elevating them from MM to QM levels of accuracy. Trained on the difference between MM and QM atomic forces across diverse zinc coordination groups, the DP/MM model faithfully reproduces structural characteristics of zinc coordination during simulations, such as the tetrahedral coordination of Cys4 and Cys3His1 groups. Furthermore, DP/MM allows water exchange in the zinc coordination environment. With its unique blend of accuracy, efficiency, flexibility, and transferability, DP/MM serves as a valuable tool for studying structures and dynamics of zinc-containing proteins and also represents a pioneering approach in the evolving landscape of machine learning potentials for molecular modeling.

Development of a machine learning finite-range nonlocal density functional

Kohn-Sham density functional theory has been the most popular method in electronic structure calculations. To fulfill the increasing accuracy requirements, new approximate functionals are needed to address key issues in existing approximations. It is well known that nonlocal components are crucial. Current nonlocal functionals mostly require orbital dependence such as in Hartree-Fock exchange and many-body perturbation correlation energy, which, however, leads to higher computational costs. Deviating from this pathway, we describe functional nonlocality in a new approach. By partitioning the total density to atom-centered local densities, a many-body expansion is proposed. This many-body expansion can be truncated at one-body contributions, if a base functional is used and an energy correction is approximated. The contribution from each atom-centered local density is a single finite-range nonlocal functional that is universal for all atoms. We then use machine learning to develop this universal atom-centered functional. Parameters in this functional are determined by fitting to data that are produced by high-level theories. Extensive tests on several different test sets, which include reaction energies, reaction barrier heights, and non-covalent interaction energies, show that the new functional, with only the density as the basic variable, can produce results comparable to the best-performing double-hybrid functionals, (for example, for the thermochemistry test set selected from the GMTKN55 database, BLYP based machine learning functional gives a weighted total mean absolute deviations of 3.33 kcal/mol, while DSD-BLYP-D3(BJ) gives 3.28 kcal/mol) with a lower computational cost. This opens a new pathway to nonlocal functional development and applications.

Force Field Analysis Software and Tools (FFAST): Assessing Machine Learning Force Fields under the Microscope

As the sophistication of machine learning force fields (MLFF) increases to match the complexity of extended molecules and materials, so does the need for tools to properly analyze and assess the practical performance of MLFFs. To go beyond average error metrics and into a complete picture of a model's applicability and limitations, we developed FFAST (force field analysis software and tools): a cross-platform software package designed to gain detailed insights into a model's performance and limitations, complete with an easy-to-use graphical user interface. The software allows the user to gauge the performance of any molecular force field,─such as popular state-of-the-art MLFF models, ─ on various popular data set types, providing general prediction error overviews, outlier detection mechanisms, atom-projected errors, and more. It has a 3D visualizer to find and picture problematic configurations, atoms, or clusters in a large data set. In this paper, the example of the MACE and NequIP models is used on two data sets of interest [stachyose and docosahexaenoic acid (DHA)]─to illustrate the use cases of the software. With this, it was found that carbons and oxygens involved in or near glycosidic bonds inside the stachyose molecule present increased prediction errors. In addition, prediction errors on DHA rise as the molecule folds, especially for the carboxylic group at the edge of the molecule. We emphasize the need for a systematic assessment of MLFF models for ensuring their successful application to the study of dynamics of molecules and materials.

Multilayer Perceptron Algorithm-Assisted Flexible Piezoresistive PDMS/Chitosan/cMWCNT Sponge Pressure Sensor for Sedentary Healthcare Monitoring

Recently, the health problems faced by sedentary workers have received increasing attention. In this study, a pressure sensor based on a poly(dimethylsiloxane) (PDMS)/carboxylated chitosan (CCS)/carboxylated multiwalled carbon nanotube (cMWCNT) sponge was prepared to realize a portable, sensitive, comfortable, and noninvasive healthcare monitoring system for sedentary workers. The proposed piezoresistive pressure sensor exhibited exceptional sensing performances with high sensitivity (147.74 kPa-1), an ultrawide detection range (22 Pa to 1.42 MPa), and reliable stability (over 3000 cycles). Furthermore, the obtained sensor displayed superior capability in detecting various human motion signals. Based on the 4 × 4 sensing array and multilayer perceptron (MLP) algorithm model, a smart cushion was developed to recognize five types of sitting postures and supply timely reminders to sedentary workers. The piezoresistive sponge pressure sensor proposed in this study reveals promising potential in the fields of wearable electronics, healthcare monitoring, and human-machine interface applications.

Global Neural Network Potential with Explicit Many-Body Functions for Improved Descriptions of Complex Potential Energy Surface

The high dimensional machine learning potential (MLP) that has developed rapidly in the past decade represents a giant step forward in large-scale atomic simulation for complex systems. The long-range interaction and the poor description of chemical reactions are typical problems of high dimensional MLP, which are mainly caused by the poor structure discrimination of the atom-centered ML model. Herein, we propose a low-cost neural-network-based MLP architecture for fitting global potential energy surface data, namely, G-MBNN, that can offer improved energy and force resolution on a complex potential energy surface. In G-MBNN, a set of many-body energy terms based on the local atomic environment are explicitly included in computing the total energy─the total energy of the system is written as the sum of atomic energy and many-body energy contributions. These extra many-body energy terms are computationally low-cost and, importantly, can provide easy access to delicate energy terms in complex systems such as very short repulsion, long-range attractions, and sensitive angular-dependent covalent interactions. We implement G-MBNN in the LASP code and demonstrate the improved accuracy of the new framework in representative systems, including ternary-element energy materials LiCoOx, TiO2 with defects, and a series of organic reactions.

Physics-Inspired Equivariant Descriptors of Nonbonded Interactions

One essential ingredient in many machine learning (ML) based methods for atomistic modeling of materials and molecules is the use of locality. While allowing better system-size scaling, this systematically neglects long-range (LR) effects such as electrostatic or dispersion interactions. We present an extension of the long distance equivariant (LODE) framework that can handle diverse LR interactions in a consistent way and seamlessly integrates with preexisting methods by building new sets of atom centered features. We provide a direct physical interpretation of these using the multipole expansion, which allows for simpler and more efficient implementations. The framework is applied to simple toy systems as proof of concept and a heterogeneous set of molecular dimers to push the method to its limits. By generalizing LODE to arbitrary asymptotic behaviors, we provide a coherent approach to treat arbitrary two- and many-body nonbonded interactions in the data-driven modeling of matter.

Development of scalable and generalizable machine learned force field for polymers

Understanding and predicting the properties of polymers is vital to developing tailored polymer molecules for desired applications. Classical force fields may fail to capture key properties, for example, the transport properties of certain polymer systems such as polyethylene glycol. As a solution, we present an alternative potential energy surface, a charge recursive neural network (QRNN) model trained on DFT calculations made on smaller atomic clusters that generalizes well to oligomers comprising larger atomic clusters or longer chains. We demonstrate the validity of the polymer QRNN workflow by modeling the oligomers of ethylene glycol. We apply two rounds of active learning (addition of new training clusters based on current model performance) and implement a novel model training approach that uses partial charges from a semi-empirical method. Our developed QRNN model for polymers produces stable molecular dynamics (MD) simulation trajectory and captures the dynamics of polymer chains as indicated by the striking agreement with experimental values. Our model allows working on much larger systems than allowed by DFT simulations, at the same time providing a more accurate force field than classical force fields which provides a promising avenue for large-scale molecular simulations of polymeric systems.

How to train a neural network potential

The introduction of modern Machine Learning Potentials (MLPs) has led to a paradigm change in the development of potential energy surfaces for atomistic simulations. By providing efficient access to energies and forces, they allow us to perform large-scale simulations of extended systems, which are not directly accessible by demanding first-principles methods. In these simulations, MLPs can reach the accuracy of electronic structure calculations, provided that they have been properly trained and validated using a suitable set of reference data. Due to their highly flexible functional form, the construction of MLPs has to be done with great care. In this Tutorial, we describe the necessary key steps for training reliable MLPs, from data generation via training to final validation. The procedure, which is illustrated for the example of a high-dimensional neural network potential, is general and applicable to many types of MLPs.

Operando Modeling of Zeolite-Catalyzed Reactions Using First-Principles Molecular Dynamics Simulations

Within this Perspective, we critically reflect on the role of first-principles molecular dynamics (MD) simulations in unraveling the catalytic function within zeolites under operating conditions. First-principles MD simulations refer to methods where the dynamics of the nuclei is followed in time by integrating the Newtonian equations of motion on a potential energy surface that is determined by solving the quantum-mechanical many-body problem for the electrons. Catalytic solids used in industrial applications show an intriguing high degree of complexity, with phenomena taking place at a broad range of length and time scales. Additionally, the state and function of a catalyst critically depend on the operating conditions, such as temperature, moisture, presence of water, etc. Herein we show by means of a series of exemplary cases how first-principles MD simulations are instrumental to unravel the catalyst complexity at the molecular scale. Examples show how the nature of reactive species at higher catalytic temperatures may drastically change compared to species at lower temperatures and how the nature of active sites may dynamically change upon exposure to water. To simulate rare events, first-principles MD simulations need to be used in combination with enhanced sampling techniques to efficiently sample low-probability regions of phase space. Using these techniques, it is shown how competitive pathways at operating conditions can be discovered and how broad transition state regions can be explored. Interestingly, such simulations can also be used to study hindered diffusion under operating conditions. The cases shown clearly illustrate how first-principles MD simulations reveal insights into the catalytic function at operating conditions, which could not be discovered using static or local approaches where only a few points are considered on the potential energy surface (PES). Despite these advantages, some major hurdles still exist to fully integrate first-principles MD methods in a standard computational catalytic workflow or to use the output of MD simulations as input for multiple length/time scale methods that aim to bridge to the reactor scale. First of all, methods are needed that allow us to evaluate the interatomic forces with quantum-mechanical accuracy, albeit at a much lower computational cost compared to currently used density functional theory (DFT) methods. The use of DFT limits the currently attainable length/time scales to hundreds of picoseconds and a few nanometers, which are much smaller than realistic catalyst particle dimensions and time scales encountered in the catalysis process. One solution could be to construct machine learning potentials (MLPs), where a numerical potential is derived from underlying quantum-mechanical data, which could be used in subsequent MD simulations. As such, much longer length and time scales could be reached; however, quite some research is still necessary to construct MLPs for the complex systems encountered in industrially used catalysts. Second, most currently used enhanced sampling techniques in catalysis make use of collective variables (CVs), which are mostly determined based on chemical intuition. To explore complex reactive networks with MD simulations, methods are needed that allow the automatic discovery of CVs or methods that do not rely on a priori definition of CVs. Recently, various data-driven methods have been proposed, which could be explored for complex catalytic systems. Lastly, first-principles MD methods are currently mostly used to investigate local reactive events. We hope that with the rise of data-driven methods and more efficient methods to describe the PES, first-principles MD methods will in the future also be able to describe longer length/time scale processes in catalysis. This might lead to a consistent dynamic description of all steps-diffusion, adsorption, and reaction-as they take place at the catalyst particle level.

Imperfections are not 0 K: free energy of point defects in crystals

Defects determine many important properties and applications of materials, ranging from doping in semiconductors, to conductivity in mixed ionic-electronic conductors used in batteries, to active sites in catalysts. The theoretical description of defect formation in crystals has evolved substantially over the past century. Advances in supercomputing hardware, and the integration of new computational techniques such as machine learning, provide an opportunity to model longer length and time-scales than previously possible. In this Tutorial Review, we cover the description of free energies for defect formation at finite temperatures, including configurational (structural, electronic, spin) and vibrational terms. We discuss challenges in accounting for metastable defect configurations, progress such as machine learning force fields and thermodynamic integration to directly access entropic contributions, and bottlenecks in going beyond the dilute limit of defect formation. Such developments are necessary to support a new era of accurate defect predictions in computational materials chemistry.

Toward a general neural network force field for protein simulations: Refining the intramolecular interaction in protein

Molecular dynamics (MD) is an extremely powerful, highly effective, and widely used approach to understanding the nature of chemical processes in atomic details for proteins. The accuracy of results from MD simulations is highly dependent on force fields. Currently, molecular mechanical (MM) force fields are mainly utilized in MD simulations because of their low computational cost. Quantum mechanical (QM) calculation has high accuracy, but it is exceedingly time consuming for protein simulations. Machine learning (ML) provides the capability for generating accurate potential at the QM level without increasing much computational effort for specific systems that can be studied at the QM level. However, the construction of general machine learned force fields, needed for broad applications and large and complex systems, is still challenging. Here, general and transferable neural network (NN) force fields based on CHARMM force fields, named CHARMM-NN, are constructed for proteins by training NN models on 27 fragments partitioned from the residue-based systematic molecular fragmentation (rSMF) method. The NN for each fragment is based on atom types and uses new input features that are similar to MM inputs, including bonds, angles, dihedrals, and non-bonded terms, which enhance the compatibility of CHARMM-NN to MM MD and enable the implementation of CHARMM-NN force fields in different MD programs. While the main part of the energy of the protein is based on rSMF and NN, the nonbonded interactions between the fragments and with water are taken from the CHARMM force field through mechanical embedding. The validations of the method for dipeptides on geometric data, relative potential energies, and structural reorganization energies demonstrate that the CHARMM-NN local minima on the potential energy surface are very accurate approximations to QM, showing the success of CHARMM-NN for bonded interactions. However, the MD simulations on peptides and proteins indicate that more accurate methods to represent protein-water interactions in fragments and non-bonded interactions between fragments should be considered in the future improvement of CHARMM-NN, which can increase the accuracy of approximation beyond the current mechanical embedding QM/MM level.

Challenges in modelling dynamic processes in realistic nanostructured materials at operating conditions

The question is addressed in how far current modelling strategies are capable of modelling dynamic phenomena in realistic nanostructured materials at operating conditions. Nanostructured materials used in applications are far from perfect; they possess a broad range of heterogeneities in space and time extending over several orders of magnitude. Spatial heterogeneities from the subnanometre to the micrometre scale in crystal particles with a finite size and specific morphology, impact the material's dynamics. Furthermore, the material's functional behaviour is largely determined by the operating conditions. Currently, there exists a huge length-time scale gap between attainable theoretical length-time scales and experimentally relevant scales. Within this perspective, three key challenges are highlighted within the molecular modelling chain to bridge this length-time scale gap. Methods are needed that enable (i) building structural models for realistic crystal particles having mesoscale dimensions with isolated defects, correlated nanoregions, mesoporosity, internal and external surfaces; (ii) the evaluation of interatomic forces with quantum mechanical accuracy albeit at much lower computational cost than the currently used density functional theory methods and (iii) derivation of the kinetics of phenomena taking place in a multi-length-time scale window to obtain an overall view of the dynamics of the process. This article is part of a discussion meeting issue 'Supercomputing simulations of advanced materials'.

A fully quantum-mechanical treatment for kaolinite

Neural network potentials for kaolinite minerals have been fitted to data extracted from density functional theory calculations that were performed using the revPBE + D3 and revPBE + vdW functionals. These potentials have then been used to calculate the static and dynamic properties of the mineral. We show that revPBE + vdW is better at reproducing the static properties. However, revPBE + D3 does a better job of reproducing the experimental IR spectrum. We also consider what happens to these properties when a fully quantum treatment of the nuclei is employed. We find that nuclear quantum effects (NQEs) do not make a substantial difference to the static properties. However, when NQEs are included, the dynamic properties of the material change substantially.

Symmetrized Drude Oscillator Force Fields Improve Numerical Performance of Polarizable Molecular Dynamics

Drude oscillator potentials are a popular and computationally efficient class of polarizable models that represent each polarizable atom as a positively charged Drude core harmonically bound to a negatively charged Drude shell. We show that existing force fields that place all non-Coulomb forces on the Drude core and none on the shell inadvertently couple the dipole to non-Coulombic forces. This introduces errors where interactions with neutral particles can erroneously induce atomic polarization, leading to spurious polarizations in the absence of an electric field, exacerbating violations of equipartition in the employed Carr-Parinello scheme. A suitable symmetrization of the interaction potential that correctly splits the force between the Drude core and shell can correct this shortcoming, improving the stability and numerical performance of Drude oscillator-based simulations. The symmetrization procedure is straightforward and only requires the rescaling of a few force field parameters.

Machine Learning Interatomic Potentials and Long-Range Physics

Advances in machine learned interatomic potentials (MLIPs), such as those using neural networks, have resulted in short-range models that can infer interaction energies with near ab initio accuracy and orders of magnitude reduced computational cost. For many atom systems, including macromolecules, biomolecules, and condensed matter, model accuracy can become reliant on the description of short- and long-range physical interactions. The latter terms can be difficult to incorporate into an MLIP framework. Recent research has produced numerous models with considerations for nonlocal electrostatic and dispersion interactions, leading to a large range of applications that can be addressed using MLIPs. In light of this, we present a Perspective focused on key methodologies and models being used where the presence of nonlocal physics and chemistry are crucial for describing system properties. The strategies covered include MLIPs augmented with dispersion corrections, electrostatics calculated with charges predicted from atomic environment descriptors, the use of self-consistency and message passing iterations to propagated nonlocal system information, and charges obtained via equilibration schemes. We aim to provide a pointed discussion to support the development of machine learning-based interatomic potentials for systems where contributions from only nearsighted terms are deficient.

Gaussian Process Regression Models for Predicting Atomic Energies and Multipole Moments

Developing a force field is a difficult task because its design is typically pulled in opposite directions by speed and accuracy. FFLUX breaks this trend by utilizing Gaussian process regression (GPR) to predict, at ab initio accuracy, atomic energies and multipole moments as obtained from the quantum theory of atoms in molecules (QTAIM). This work demonstrates that the in-house FFLUX training pipeline can generate successful GPR models for six representative molecules: peptide-capped glycine and alanine, glucose, paracetamol, aspirin, and ibuprofen. The molecules were sufficiently distorted to represent configurations from an AMBER-GAFF2 molecular dynamics run. All internal degrees of freedom were covered corresponding to 93 dimensions in the case of the largest molecule ibuprofen (33 atoms). Benefiting from active learning, the GPR models contain only about 2000 training points and return largely sub-kcal mol^-1 prediction errors for the validation sets. A proof of concept has been reached for transferring the model produced through active learning on one atomic property to that of the remaining atomic properties. The prediction of electrostatic interaction can be assessed at the intermolecular level, and the vast majority of interactions have a root-mean-square error of less than 0.1 kJ mol^-1 with a maximum value of ∼1 kJ mol^-1 for a glycine and paracetamol dimer.

High-dimensional neural network potentials for accurate vibrational frequencies: the formic acid dimer benchmark

In recent years, machine learning potentials (MLP) for atomistic simulations have attracted a lot of attention in chemistry and materials science. Many new approaches have been developed with the primary aim to transfer the accuracy of electronic structure calculations to large condensed systems containing thousands of atoms. In spite of these advances, the reliability of modern MLPs in reproducing the subtle details of the multi-dimensional potential-energy surface is still difficult to assess for such systems. On the other hand, moderately sized systems enabling the application of tools for thorough and systematic quality-control are nowadays rarely investigated. In this work we use benchmark-quality harmonic and anharmonic vibrational frequencies as a sensitive probe for the validation of high-dimensional neural network potentials. For the case of the formic acid dimer, a frequently studied model system for which stringent spectroscopic data became recently available, we show that high-quality frequencies can be obtained from state-of-the-art calculations in excellent agreement with coupled cluster theory and experimental data.

Producing chemically accurate atomic Gaussian process regression models by active learning for molecular simulation

Machine learning is becoming increasingly more important in the field of force field development. Never has it been more vital to have chemically accurate machine learning potentials because force fields become more sophisticated and their applications expand. In this study a method for developing chemically accurate Gaussian process regression models is demonstrated for an increasingly complex set of molecules. This work is an extension to previous work showing the progression of the active learning technique in producing more accurate models in much less CPU time than ever before. The per-atom active learning approach has unlocked the potential to generate chemically accurate models for molecules such as peptide-capped glycine.

Auto3D: Automatic Generation of the Low-Energy 3D Structures with ANI Neural Network Potentials

Computational programs accelerate the chemical discovery processes but often need proper three-dimensional molecular information as part of the input. Getting optimal molecular structures is challenging because it requires enumerating and optimizing a huge space of stereoisomers and conformers. We developed the Python-based Auto3D package for generating the low-energy 3D structures using SMILES as the input. Auto3D is based on state-of-the-art algorithms and can automatize the isomer enumeration and duplicate filtering process, 3D building process, geometry optimization, and ranking process. Tested on 50 molecules with multiple unspecified stereocenters, Auto3D is guaranteed to find the stereoconfiguration that yields the lowest-energy conformer. With Auto3D, we provide an extension of the ANI model. The new model, dubbed ANI-2xt, is trained on a tautomer-rich data set. ANI-2xt is benchmarked with DFT methods on geometry optimization and electronic and Gibbs free energy calculations. Compared with ANI-2x, ANI-2xt provides a 42% error reduction for tautomeric reaction energy calculations when using the gold-standard coupled-cluster calculation as the reference. ANI-2xt can accurately predict the energies and is several orders of magnitude faster than DFT methods.

Extending machine learning beyond interatomic potentials for predicting molecular properties

Machine learning (ML) is becoming a method of choice for modelling complex chemical processes and materials. ML provides a surrogate model trained on a reference dataset that can be used to establish a relationship between a molecular structure and its chemical properties. This Review highlights developments in the use of ML to evaluate chemical properties such as partial atomic charges, dipole moments, spin and electron densities, and chemical bonding, as well as to obtain a reduced quantum-mechanical description. We overview several modern neural network architectures, their predictive capabilities, generality and transferability, and illustrate their applicability to various chemical properties. We emphasize that learned molecular representations resemble quantum-mechanical analogues, demonstrating the ability of the models to capture the underlying physics. We also discuss how ML models can describe non-local quantum effects. Finally, we conclude by compiling a list of available ML toolboxes, summarizing the unresolved challenges and presenting an outlook for future development. The observed trends demonstrate that this field is evolving towards physics-based models augmented by ML, which is accompanied by the development of new methods and the rapid growth of user-friendly ML frameworks for chemistry.

Long-range dispersion-inclusive machine learning potentials for structure search and optimization of hybrid organic-inorganic interfaces

The computational prediction of the structure and stability of hybrid organic-inorganic interfaces provides important insights into the measurable properties of electronic thin film devices, coatings, and catalyst surfaces and plays an important role in their rational design. However, the rich diversity of molecular configurations and the important role of long-range interactions in such systems make it difficult to use machine learning (ML) potentials to facilitate structure exploration that otherwise requires computationally expensive electronic structure calculations. We present an ML approach that enables fast, yet accurate, structure optimizations by combining two different types of deep neural networks trained on high-level electronic structure data. The first model is a short-ranged interatomic ML potential trained on local energies and forces, while the second is an ML model of effective atomic volumes derived from atoms-in-molecules partitioning. The latter can be used to connect short-range potentials to well-established density-dependent long-range dispersion correction methods. For two systems, specifically gold nanoclusters on diamond (110) surfaces and organic π-conjugated molecules on silver (111) surfaces, we train models on sparse structure relaxation data from density functional theory and show the ability of the models to deliver highly efficient structure optimizations and semi-quantitative energy predictions of adsorption structures.

Ab initio machine learning of phase space averages

Equilibrium structures determine material properties and biochemical functions. We here propose to machine learn phase space averages, conventionally obtained by ab initio or force-field-based molecular dynamics (MD) or Monte Carlo (MC) simulations. In analogy to ab initio MD, our ab initio machine learning (AIML) model does not require bond topologies and, therefore, enables a general machine learning pathway to obtain ensemble properties throughout the chemical compound space. We demonstrate AIML for predicting Boltzmann averaged structures after training on hundreds of MD trajectories. The AIML output is subsequently used to train machine learning models of free energies of solvation using experimental data and to reach competitive prediction errors (mean absolute error ∼ 0.8 kcal/mol) for out-of-sample molecules-within milliseconds. As such, AIML effectively bypasses the need for MD or MC-based phase space sampling, enabling exploration campaigns of Boltzmann averages throughout the chemical compound space at a much accelerated pace. We contextualize our findings by comparison to state-of-the-art methods resulting in a Pareto plot for the free energy of solvation predictions in terms of accuracy and time.

Computer-aided drug design, quantum-mechanical methods for biological problems

Quantum chemistry enables to study systems with chemical accuracy (<1 kcal/mol from experiment) but is restricted to a handful of atoms due to its computational expense. This has led to ongoing interest to optimize and simplify these methods while retaining accuracy. Implementing quantum mechanical (QM) methods on modern hardware such as multiple-GPUs is one example of how the field is optimizing performance. Multiscale approaches like the so-called QM/molecular mechanical method are gaining popularity in drug discovery because they focus the application of QM methods on the region of choice (e.g., the binding site), while using efficient MM models to represent less relevant areas. The creation of simplified QM methods is another example, including the use of machine learning to create ultra-fast and accurate QM models. Herein, we summarize recent advancements in the development of optimized QM methods that enhance our ability to use these methods in computer aided drug discovery.

Machine learning potential era of zeolite simulation

Zeolites, owing to their great variety and complexity in structure and wide applications in chemistry, have long been the hot topic in chemical research. This perspective first presents a short retrospect of theoretical investigations on zeolites using the tools from classical force fields to quantum mechanics calculations and to the latest machine learning (ML) potential simulations. ML potentials as the next-generation technique for atomic simulation open new avenues to simulate and interpret zeolite systems and thus hold great promise for finally predicting the structure-functionality relation of zeolites. Recent advances using ML potentials are then summarized from two main aspects: the origin of zeolite stability and the mechanism of zeolite-related catalytic reactions. We also discussed the possible scenarios of ML potential application aiming to provide instantaneous and easy access of zeolite properties. These advanced applications could now be accomplished by combining cloud-computing-based techniques with ML potential-based atomic simulations. The future development of ML potentials for zeolites in the respects of improving the calculation accuracy, expanding the application scope and constructing the zeolite-related datasets is finally outlooked.

Improving the Accuracy of Atomistic Simulations of the Electrochemical Interface

Atomistic simulation of the electrochemical double layer is an ambitious undertaking, requiring quantum mechanical description of electrons, phase space sampling of liquid electrolytes, and equilibration of electrolytes over nanosecond time scales. All models of electrochemistry make different trade-offs in the approximation of electrons and atomic configurations, from the extremes of classical molecular dynamics of a complete interface with point-charge atoms to correlated electronic structure methods of a single electrode configuration with no dynamics or electrolyte. Here, we review the spectrum of simulation techniques suitable for electrochemistry, focusing on the key approximations and accuracy considerations for each technique. We discuss promising approaches, such as enhanced sampling techniques for atomic configurations and computationally efficient beyond density functional theory (DFT) electronic methods, that will push electrochemical simulations beyond the present frontier.

A deep potential model with long-range electrostatic interactions

Machine learning models for the potential energy of multi-atomic systems, such as the deep potential (DP) model, make molecular simulations with the accuracy of quantum mechanical density functional theory possible at a cost only moderately higher than that of empirical force fields. However, the majority of these models lack explicit long-range interactions and fail to describe properties that derive from the Coulombic tail of the forces. To overcome this limitation, we extend the DP model by approximating the long-range electrostatic interaction between ions (nuclei + core electrons) and valence electrons with that of distributions of spherical Gaussian charges located at ionic and electronic sites. The latter are rigorously defined in terms of the centers of the maximally localized Wannier distributions, whose dependence on the local atomic environment is modeled accurately by a deep neural network. In the DP long-range (DPLR) model, the electrostatic energy of the Gaussian charge system is added to short-range interactions that are represented as in the standard DP model. The resulting potential energy surface is smooth and possesses analytical forces and virial. Missing effects in the standard DP scheme are recovered, improving on accuracy and predictive power. By including long-range electrostatics, DPLR correctly extrapolates to large systems the potential energy surface learned from quantum mechanical calculations on smaller systems. We illustrate the approach with three examples: the potential energy profile of the water dimer, the free energy of interaction of a water molecule with a liquid water slab, and the phonon dispersion curves of the NaCl crystal.

A Hessian-based assessment of atomic forces for training machine learning interatomic potentials

In recent years, many types of machine learning potentials (MLPs) have been introduced, which are able to represent high-dimensional potential-energy surfaces (PESs) with close to first-principles accuracy. Most current MLPs rely on atomic energy contributions given as a function of the local chemical environments. Frequently, in addition to total energies, atomic forces are also used to construct the potentials, as they provide detailed local information about the PES. Since many systems are too large for electronic structure calculations, obtaining reliable reference forces from smaller subsystems, such as molecular fragments or clusters, can substantially simplify the construction of the training sets. Here, we propose a method to determine structurally converged molecular fragments, providing reliable atomic forces based on an analysis of the Hessian. The method, which serves as a locality test and allows us to estimate the importance of long-range interactions, is illustrated for a series of molecular model systems and the metal–organic framework MOF-5 as an example for a complex organic–inorganic hybrid material.

Transferable Neural Network Potential Energy Surfaces for Closed-Shell Organic Molecules: Extension to Ions

Transferable high dimensional neural network potentials (HDNNPs) have shown great promise as an avenue to increase the accuracy and domain of applicability of existing atomistic force fields for organic systems relevant to life science. We have previously reported such a potential (Schrödinger-ANI) that has broad coverage of druglike molecules. We extend that work here to cover ionic and zwitterionic druglike molecules expected to be relevant to drug discovery research activities. We report a novel HDNNP architecture, which we call QRNN, that predicts atomic charges and uses these charges as descriptors in an energy model that delivers conformational energies within chemical accuracy when measured against the reference theory it is trained to. Further, we find that delta learning based on a semiempirical level of theory approximately halves the errors. We test the models on torsion energy profiles, relative conformational energies, geometric parameters, and relative tautomer errors.

Molecular Simulations to Elucidate Transport Phenomena in Polymeric Membranes

Despite decades of dominance in separation technology, progress in the design and development of high-performance polymer-based membranes has been incremental. Recent advances in materials science and chemical synthesis provide opportunities for molecular-level design of next-generation membrane materials. Such designs necessitate a fundamental understanding of transport and separation mechanisms at the molecular scale. Molecular simulations are important tools that could lead to the development of fundamental structure-property-performance relationships for advancing membrane design. In this Perspective, we assess the application and capability of molecular simulations to understand the mechanisms of ion and water transport across polymeric membranes. Additionally, we discuss the reliability of molecular models in mimicking the structure and chemistry of nanochannels and transport pathways in polymeric membranes. We conclude by providing research directions for resolving key knowledge gaps related to transport phenomena in polymeric membranes and for the construction of structure-property-performance relationships for the design of next-generation membranes.

Kernel charge equilibration: efficient and accurate prediction of molecular dipole moments with a machine-learning enhanced electron density model

State-of-the-art machine learning (ML) interatomic potentials use local representations of atomic environments to ensure linear scaling and size-extensivity. This implies a neglect of long-range interactions, most prominently related to electrostatics. To overcome this limitation, we herein present a ML framework for predicting charge distributions and their interactions termed kernel charge equilibration (kQEq). This model is based on classical charge equilibration (QEq) models expanded with an environment-dependent electronegativity. In contrast to previously reported neural network models with a similar concept, kQEq takes advantage of the linearity of both QEq and Kernel Ridge Regression to obtain a closed-form linear algebra expression for training the models. Furthermore, we avoid the ambiguity of charge partitioning schemes by using dipole moments as reference data. As a first application, we show that kQEq can be used to generate accurate and highly data-efficient models for molecular dipole moments.

Learning Atomic Multipoles: Prediction of the Electrostatic Potential with Equivariant Graph Neural Networks

The accurate description of electrostatic interactions remains a challenging problem for classical potential-energy functions. The commonly used fixed partial-charge approximation fails to reproduce the electrostatic potential at short range due to its insensitivity to conformational changes and anisotropic effects. At the same time, possibly more accurate machine-learned (ML) potentials struggle with the long-range behavior due to their inherent locality ansatz. Employing a multipole expansion offers in principle an exact treatment of the electrostatic potential such that the long-range and short-range electrostatic interactions can be treated simultaneously with high accuracy. However, such an expansion requires the calculation of the electron density using computationally expensive quantum-mechanical (QM) methods. Here, we introduce an equivariant graph neural network (GNN) to address this issue. The proposed model predicts atomic multipoles up to the quadrupole, circumventing the need for expensive QM computations. By using an equivariant architecture, the model enforces the correct symmetry by design without relying on local reference frames. The GNN reproduces the electrostatic potential of various systems with high fidelity. Possible uses for such an approach include the separate treatment of long-range interactions in ML potentials, the analysis of electrostatic potential surfaces, and static multipoles in polarizable force fields.

NNAIMQ: A neural network model for predicting QTAIM charges

Atomic charges provide crucial information about the electronic structure of a molecular system. Among the different definitions of these descriptors, the one proposed by the Quantum Theory of Atoms in Molecules (QTAIM) is particularly attractive given its invariance against orbital transformations although the computational cost associated with their calculation limits its applicability. Given that Machine Learning (ML) techniques have been shown to accelerate orders of magnitude the computation of a number of quantum mechanical observables, in this work, we take advantage of ML knowledge to develop an intuitive and fast neural network model (NNAIMQ) for the computation of QTAIM charges for C, H, O, and N atoms with high accuracy. Our model has been trained and tested using data from quantum chemical calculations in more than 45 000 molecular environments of the near-equilibrium CHON chemical space. The reliability and performance of NNAIMQ have been analyzed in a variety of scenarios, from equilibrium geometries to molecular dynamics simulations. Altogether, NNAIMQ yields remarkably small prediction errors, well below the 0.03 electron limit in the general case, while accelerating the calculation of QTAIM charges by several orders of magnitude.

Neural Network Potentials: A Concise Overview of Methods

In the past two decades, machine learning potentials (MLPs) have reached a level of maturity that now enables applications to large-scale atomistic simulations of a wide range of systems in chemistry, physics, and materials science. Different machine learning algorithms have been used with great success in the construction of these MLPs. In this review, we discuss an important group of MLPs relying on artificial neural networks to establish a mapping from the atomic structure to the potential energy. In spite of this common feature, there are important conceptual differences among MLPs, which concern the dimensionality of the systems, the inclusion of long-range electrostatic interactions, global phenomena like nonlocal charge transfer, and the type of descriptor used to represent the atomic structure, which can be either predefined or learnable. A concise overview is given along with a discussion of the open challenges in the field.

Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

DL_FFLUX: A Parallel, Quantum Chemical Topology Force Field

DL_FFLUX is a force field based on quantum chemical topology that can perform molecular dynamics for flexible molecules endowed with polarizable atomic multipole moments (up to hexadecapole). Using the machine learning method kriging (aka Gaussian process regression), DL_FFLUX has access to atomic properties (energy, charge, dipole moment, etc.) with quantum mechanical accuracy. Newly optimized and parallelized using domain decomposition Message Passing Interface (MPI), DL_FFLUX is now able to deliver this rigorous methodology at scale while still in reasonable time frames. DL_FFLUX is delivered as an add-on to the widely distributed molecular dynamics code DL_POLY 4.08. For the systems studied here (10³–10⁵ atoms), DL_FFLUX is shown to add minimal computational cost to the standard DL_POLY package. In fact, the optimization of the electrostatics in DL_FFLUX means that, when high-rank multipole moments are enabled, DL_FFLUX is up to 1.25× faster than standard DL_POLY. The parallel DL_FFLUX preserves the quality of the scaling of MPI implementation in standard DL_POLY. For the first time, it is feasible to use the full capability of DL_FFLUX to study systems that are large enough to be of real-world interest. For example, a fully flexible, high-rank polarized (up to and including quadrupole moments) 1 ns simulation of a system of 10 125 atoms (3375 water molecules) takes 30 h (wall time) on 18 cores.

Gaussian Process Regression for Materials and Molecules

We provide an introduction to Gaussian process regression (GPR) machine-learning methods in computational materials science and chemistry. The focus of the present review is on the regression of atomistic properties: in particular, on the construction of interatomic potentials, or force fields, in the Gaussian Approximation Potential (GAP) framework; beyond this, we also discuss the fitting of arbitrary scalar, vectorial, and tensorial quantities. Methodological aspects of reference data generation, representation, and regression, as well as the question of how a data-driven model may be validated, are reviewed and critically discussed. A survey of applications to a variety of research questions in chemistry and materials science illustrates the rapid growth in the field. A vision is outlined for the development of the methodology in the years to come.

Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems

Machine learning models are poised to make a transformative impact on chemical sciences by dramatically accelerating computational algorithms and amplifying insights available from computational chemistry methods. However, achieving this requires a confluence and coaction of expertise in computer science and physical sciences. This Review is written for new and experienced researchers working at the intersection of both fields. We first provide concise tutorials of computational chemistry and machine learning methods, showing how insights involving both can be achieved. We follow with a critical review of noteworthy applications that demonstrate how computational chemistry and machine learning can be used together to provide insightful (and useful) predictions in molecular and materials modeling, retrosyntheses, catalysis, and drug design.

Machine learned Hückel theory: Interfacing physics and deep neural networks

The Hückel Hamiltonian is an incredibly simple tight-binding model known for its ability to capture qualitative physics phenomena arising from electron interactions in molecules and materials. Part of its simplicity arises from using only two types of empirically fit physics-motivated parameters: the first describes the orbital energies on each atom and the second describes electronic interactions and bonding between atoms. By replacing these empirical parameters with machine-learned dynamic values, we vastly increase the accuracy of the extended Hückel model. The dynamic values are generated with a deep neural network, which is trained to reproduce orbital energies and densities derived from density functional theory. The resulting model retains interpretability, while the deep neural network parameterization is smooth and accurate and reproduces insightful features of the original empirical parameterization. Overall, this work shows the promise of utilizing machine learning to formulate simple, accurate, and dynamically parameterized physics models.

The Rise of Neural Networks for Materials and Chemical Dynamics

Machine learning (ML) is quickly becoming a premier tool for modeling chemical processes and materials. ML-based force fields, trained on large data sets of high-quality electron structure calculations, are particularly attractive due their unique combination of computational efficiency and physical accuracy. This Perspective summarizes some recent advances in the development of neural network-based interatomic potentials. Designing high-quality training data sets is crucial to overall model accuracy. One strategy is active learning, in which new data are automatically collected for atomic configurations that produce large ML uncertainties. Another strategy is to use the highest levels of quantum theory possible. Transfer learning allows training to a data set of mixed fidelity. A model initially trained to a large data set of density functional theory calculations can be significantly improved by retraining to a relatively small data set of expensive coupled cluster theory calculations. These advances are exemplified by applications to molecules and materials.

Perspective on computational reaction prediction using machine learning methods in heterogeneous catalysis

Heterogeneous catalysis plays a significant role in the modern chemical industry. Towards the rational design of novel catalysts, understanding reactions over surfaces is the most essential aspect. Typical industrial catalytic processes such as syngas conversion and methane utilisation can generate a large reaction network comprising thousands of intermediates and reaction pairs. This complexity not only arises from the permutation of transformations between species but also from the extra reaction channels offered by distinct surface sites. Despite the success in investigating surface reactions at the atomic scale, the huge computational expense of ab initio methods hinders the exploration of such complicated reaction networks. With the proliferation of catalysis studies, machine learning as an emerging tool can take advantage of the accumulated reaction data to emulate the output of ab initio methods towards swift reaction prediction. Here, we briefly summarise the conventional workflow of reaction prediction, including reaction network generation, ab initio thermodynamics and microkinetic modelling. An overview of the frequently used regression models in machine learning is presented. As a promising alternative to full ab initio calculations, machine learning interatomic potentials are highlighted. Furthermore, we survey applications assisted by these methods for accelerating reaction prediction, exploring reaction networks, and computational catalyst design. Finally, we envisage future directions in computationally investigating reactions and implementing machine learning algorithms in heterogeneous catalysis.