Representing high-dimensional potential-energy surfaces for reactions at surfaces by neural networks

Quantum Machine Learning in Chemical Compound Space

Rather than numerically solving the computationally demanding equations of quantum or statistical mechanics, machine learning methods can infer approximate solutions, interpolating previously acquired property data sets of molecules and materials. The case is made for quantum machine learning: An inductive molecular modeling approach which can be applied to quantum chemistry problems.

On-the-Fly Machine Learning of Atomic Potential in Density Functional Theory Structure Optimization

Machine learning (ML) is used to derive local stability information for density functional theory calculations of systems in relation to the recently discovered SnO_{2}(110)-(4×1) reconstruction. The ML model is trained on (structure, total energy) relations collected during global minimum energy search runs with an evolutionary algorithm (EA). While being built, the ML model is used to guide the EA, thereby speeding up the overall rate by which the EA succeeds. Inspection of the local atomic potentials emerging from the model further shows chemically intuitive patterns.

Resolving Transition Metal Chemical Space: Feature Selection for Machine Learning and Structure-Property Relationships

Machine learning (ML) of quantum mechanical properties shows promise for accelerating chemical discovery. For transition metal chemistry where accurate calculations are computationally costly and available training data sets are small, the molecular representation becomes a critical ingredient in ML model predictive accuracy. We introduce a series of revised autocorrelation functions (RACs) that encode relationships of the heuristic atomic properties (e.g., size, connectivity, and electronegativity) on a molecular graph. We alter the starting point, scope, and nature of the quantities evaluated in standard ACs to make these RACs amenable to inorganic chemistry. On an organic molecule set, we first demonstrate superior standard AC performance to other presently available topological descriptors for ML model training, with mean unsigned errors (MUEs) for atomization energies on set-aside test molecules as low as 6 kcal/mol. For inorganic chemistry, our RACs yield 1 kcal/mol ML MUEs on set-aside test molecules in spin-state splitting in comparison to 15-20× higher errors for feature sets that encode whole-molecule structural information. Systematic feature selection methods including univariate filtering, recursive feature elimination, and direct optimization (e.g., random forest and LASSO) are compared. Random-forest- or LASSO-selected subsets 4-5× smaller than the full RAC set produce sub- to 1 kcal/mol spin-splitting MUEs, with good transferability to metal-ligand bond length prediction (0.004-5 Å MUE) and redox potential on a smaller data set (0.2-0.3 eV MUE). Evaluation of feature selection results across property sets reveals the relative importance of local, electronic descriptors (e.g., electronegativity, atomic number) in spin-splitting and distal, steric effects in redox potential and bond lengths.

First Principles Neural Network Potentials for Reactive Simulations of Large Molecular and Condensed Systems

Modern simulation techniques have reached a level of maturity which allows a wide range of problems in chemistry and materials science to be addressed. Unfortunately, the application of first principles methods with predictive power is still limited to rather small systems, and despite the rapid evolution of computer hardware no fundamental change in this situation can be expected. Consequently, the development of more efficient but equally reliable atomistic potentials to reach an atomic level understanding of complex systems has received considerable attention in recent years. A promising new development has been the introduction of machine learning (ML) methods to describe the atomic interactions. Once trained with electronic structure data, ML potentials can accelerate computer simulations by several orders of magnitude, while preserving quantum mechanical accuracy. This Review considers the methodology of an important class of ML potentials that employs artificial neural networks.

Energy-free machine learning force field for aluminum

We used the machine learning technique of Li et al. (PRL 114, 2015) for molecular dynamics simulations. Atomic configurations were described by feature matrix based on internal vectors, and linear regression was used as a learning technique. We implemented this approach in the LAMMPS code. The method was applied to crystalline and liquid aluminum and uranium at different temperatures and densities, and showed the highest accuracy among different published potentials. Phonon density of states, entropy and melting temperature of aluminum were calculated using this machine learning potential. The results are in excellent agreement with experimental data and results of full ab initio calculations.

Predicting electronic structure properties of transition metal complexes with neural networks

High-throughput computational screening has emerged as a critical component of materials discovery. Direct density functional theory (DFT) simulation of inorganic materials and molecular transition metal complexes is often used to describe subtle trends in inorganic bonding and spin-state ordering, but these calculations are computationally costly and properties are sensitive to the exchange-correlation functional employed. To begin to overcome these challenges, we trained artificial neural networks (ANNs) to predict quantum-mechanically-derived properties, including spin-state ordering, sensitivity to Hartree-Fock exchange, and spin-state specific bond lengths in transition metal complexes. Our ANN is trained on a small set of inorganic-chemistry-appropriate empirical inputs that are both maximally transferable and do not require precise three-dimensional structural information for prediction. Using these descriptors, our ANN predicts spin-state splittings of single-site transition metal complexes (i.e., Cr-Ni) at arbitrary amounts of Hartree-Fock exchange to within 3 kcal mol-1 accuracy of DFT calculations. Our exchange-sensitivity ANN enables improved predictions on a diverse test set of experimentally-characterized transition metal complexes by extrapolation from semi-local DFT to hybrid DFT. The ANN also outperforms other machine learning models (i.e., support vector regression and kernel ridge regression), demonstrating particularly improved performance in transferability, as measured by prediction errors on the diverse test set. We establish the value of new uncertainty quantification tools to estimate ANN prediction uncertainty in computational chemistry, and we provide additional heuristics for identification of when a compound of interest is likely to be poorly predicted by the ANN. The ANNs developed in this work provide a strategy for screening transition metal complexes both with direct ANN prediction and with improved structure generation for validation with first principles simulation.

Exploring surface landscapes with molecules: rotationally induced diffraction of H2 on LiF(001) under fast grazing incidence conditions

Atomic diffraction by surfaces under fast grazing incidence conditions has been used for almost a decade to characterize surface properties with more accuracy than with more traditional atomic diffraction methods. From six-dimensional solutions of the time-dependent Schrödinger equation, we show that diffraction of H₂ molecules under fast grazing incidence conditions could be even more informative for the characterization of ionic surfaces, due to the large anisotropic electrostatic interaction between the quadrupole moment of the molecule and the electric field created by the ionic crystal. Using the LiF(001) surface as a benchmark, we show that fast grazing incidence diffraction of H₂ strongly depends on the initial rotational state of the molecule, while rotationally inelastic processes are irrelevant. We demonstrate that, as a result of the anisotropy of the impinging projectile, initial rotational excitation leads to an increase in intensity of high-order diffraction peaks at incidence directions that satisfy precise symmetry constraints, thus providing a more detailed information on the surface characteristics than that obtained from low-order atomic diffraction peaks under fast grazing incidence conditions. As quadrupole-ion surface potentials are expected to accurately represent the interaction between H₂ and any surface with a marked ionic character, our analysis should be of general applicability to any of such surfaces. Finally, we show that a density functional theory description of the molecule-ion surface potential catches the main features observed experimentally.

Accurate Neural Network Description of Surface Phonons in Reactive Gas-Surface Dynamics: N2 + Ru(0001)

Ab initio molecular dynamics (AIMD) simulations enable the accurate description of reactive molecule-surface scattering especially if energy transfer involving surface phonons is important. However, presently, the computational expense of AIMD rules out its application to systems where reaction probabilities are smaller than about 1%. Here we show that this problem can be overcome by a high-dimensional neural network fit of the molecule-surface interaction potential, which also incorporates the dependence on phonons by taking into account all degrees of freedom of the surface explicitly. As shown for N₂ + Ru(0001), which is a prototypical case for highly activated dissociative chemisorption, the method allows an accurate description of the coupling of molecular and surface atom motion and accurately accounts for vibrational properties of the employed slab model of Ru(0001). The neural network potential allows reaction probabilities as low as 10^-5 to be computed, showing good agreement with experimental results.

Discovering charge density functionals and structure-property relationships with PROPhet: A general framework for coupling machine learning and first-principles methods

Modern ab initio methods have rapidly increased our understanding of solid state materials properties, chemical reactions, and the quantum interactions between atoms. However, poor scaling often renders direct ab initio calculations intractable for large or complex systems. There are two obvious avenues through which to remedy this problem: (i) develop new, less expensive methods to calculate system properties, or (ii) make existing methods faster. This paper describes an open source framework designed to pursue both of these avenues. PROPhet (short for PROPerty Prophet) utilizes machine learning techniques to find complex, non-linear mappings between sets of material or system properties. The result is a single code capable of learning analytical potentials, non-linear density functionals, and other structure-property or property-property relationships. These capabilities enable highly accurate mesoscopic simulations, facilitate computation of expensive properties, and enable the development of predictive models for systematic materials design and optimization. This work explores the coupling of machine learning to ab initio methods through means both familiar (e.g., the creation of various potentials and energy functionals) and less familiar (e.g., the creation of density functionals for arbitrary properties), serving both to demonstrate PROPhet’s ability to create exciting post-processing analysis tools and to open the door to improving ab initio methods themselves with these powerful machine learning techniques.

Perspective: Machine learning potentials for atomistic simulations

Nowadays, computer simulations have become a standard tool in essentially all fields of chemistry, condensed matter physics, and materials science. In order to keep up with state-of-the-art experiments and the ever growing complexity of the investigated problems, there is a constantly increasing need for simulations of more realistic, i.e., larger, model systems with improved accuracy. In many cases, the availability of sufficiently efficient interatomic potentials providing reliable energies and forces has become a serious bottleneck for performing these simulations. To address this problem, currently a paradigm change is taking place in the development of interatomic potentials. Since the early days of computer simulations simplified potentials have been derived using physical approximations whenever the direct application of electronic structure methods has been too demanding. Recent advances in machine learning (ML) now offer an alternative approach for the representation of potential-energy surfaces by fitting large data sets from electronic structure calculations. In this perspective, the central ideas underlying these ML potentials, solved problems and remaining challenges are reviewed along with a discussion of their current applicability and limitations.

Addressing uncertainty in atomistic machine learning

Machine-learning regression can precisely emulate the potential energy and forces of more expensive electronic-structure calculations, but to make useful predictions an assessment must be made of the prediction's credibility.

Quantum and classical dynamics of reactive scattering of H2 from metal surfaces

State-of-the-art theoretical models allow nowadays an accurate description of H₂/metal surface systems and phenomena relative to heterogeneous catalysis. Here we review the most relevant ones investigated during the last 10 years.

Quantum dynamics of polyatomic dissociative chemisorption on transition metal surfaces: mode specificity and bond selectivity

Recent advances in quantum dynamical characterization of polyatomic dissociative chemisorption on accurate global potential energy surfaces are critically reviewed.

Neural networks for local structure detection in polymorphic systems

The accurate identification and classification of local ordered and disordered structures is an important task in atomistic computer simulations. Here, we demonstrate that properly trained artificial neural networks can be used for this purpose. Based on a neural network approach recently developed for the calculation of energies and forces, the proposed method recognizes local atomic arrangements from a set of symmetry functions that characterize the environment around a given atom. The algorithm is simple and flexible and it does not rely on the definition of a reference frame. Using the Lennard-Jones system as well as liquid water and ice as illustrative examples, we show that the neural networks developed here detect amorphous and crystalline structures with high accuracy even in the case of complex atomic arrangements, for which conventional structure detection approaches are unreliable.

Generating accurate dipole moment surfaces using modified Shepard interpolation

We outline an approach for building molecular dipole moment surfaces using modified Shepard interpolation. Our approach is highly automated, requires minimal parameterization, and is iteratively improvable. Using the water molecule as a test case, we investigate how different aspects of the interpolation scheme affect the rate of convergence of calculated IR spectral line intensities. It is found that the interpolation scheme is sensitive to coordinate singularities present at linear geometries. Due to the generally monotonic nature of the dipole moment surface, the one-part weight function is found to be more effective than the more complicated two-part variant, with first-order interpolation also giving better-than-expected results. Almost all sensible schemes for choosing interpolation reference data points are found to exhibit acceptable convergence behavior.

Understanding the composition and activity of electrocatalytic nanoalloys in aqueous solvents: a combination of DFT and accurate neural network potentials

The shape, size, and composition of catalyst nanoparticles can have a significant influence on catalytic activity. Understanding such structure-reactivity relationships is crucial for the optimization of industrial catalysts and the design of novel catalysts with enhanced properties. In this letter, we employ a combination of first-principles computations and large-scale Monte-Carlo simulations with highly accurate neural network potentials to study the equilibrium surface structure and composition of bimetallic Au/Cu nanoparticles (NPs), which have recently been of interest as stable and efficient CO2 reduction catalysts. We demonstrate that the inclusion of explicit water molecules at a first-principles level of accuracy is necessary to predict experimentally observed trends in Au/Cu NP surface composition; in particular, we find that Au-coated core-shell NPs are thermodynamically favored in vacuum, independent of Au/Cu chemical potential and NP size, while NPs with mixed Au-Cu surfaces are preferred in aqueous solution. Furthermore, we show that both CO and O2 adsorption energies differ significantly for NPs with the equilibrium surface composition found in water and those with the equilibrium surface composition found in vacuum, suggesting large changes in CO2 reduction activity. Our results emphasize the importance of understanding and being able to predict the effects of catalytic environment on catalyst structure and activity. In addition, they demonstrate that first-principles-based neural network potentials provide a promising approach for accurately investigating the relationships between solvent, surface composition and morphology, surface electronic structure, and catalytic activity in systems composed of thousands of atoms.

Representing potential energy surfaces by high-dimensional neural network potentials

The development of interatomic potentials employing artificial neural networks has seen tremendous progress in recent years. While until recently the applicability of neural network potentials (NNPs) has been restricted to low-dimensional systems, this limitation has now been overcome and high-dimensional NNPs can be used in large-scale molecular dynamics simulations of thousands of atoms. NNPs are constructed by adjusting a set of parameters using data from electronic structure calculations, and in many cases energies and forces can be obtained with very high accuracy. Therefore, NNP-based simulation results are often very close to those gained by a direct application of first-principles methods. In this review, the basic methodology of high-dimensional NNPs will be presented with a special focus on the scope and the remaining limitations of this approach. The development of NNPs requires substantial computational effort as typically thousands of reference calculations are required. Still, if the problem to be studied involves very large systems or long simulation times this overhead is regained quickly. Further, the method is still limited to systems containing about three or four chemical elements due to the rapidly increasing complexity of the configuration space, although many atoms of each species can be present. Due to the ability of NNPs to describe even extremely complex atomic configurations with excellent accuracy irrespective of the nature of the atomic interactions, they represent a general and therefore widely applicable technique, e.g. for addressing problems in materials science, for investigating properties of interfaces, and for studying solvation processes.

Role of physisorption states in molecular scattering: a semilocal density-functional theory study on O2/Ag(111)

We simulate the scattering of O2 from Ag(111) with classical dynamics simulations performed on a six-dimensional potential energy surface calculated within semilocal density-functional theory. The enigmatic experimental trends that originally required the conjecture of two types of repulsive walls, arising from a physisorption and chemisorption part of the interaction potential, are fully reproduced. Given the inadequate description of the physisorption properties in semilocal density-functional theory, our work casts severe doubts on the prevalent notion to use molecular scattering data as indirect evidence for the existence of such states.

A density-functional theory-based neural network potential for water clusters including van der Waals corrections

The fundamental importance of water for many chemical processes has motivated the development of countless efficient but approximate water potentials for large-scale molecular dynamics simulations, from simple empirical force fields to very sophisticated flexible water models. Accurate and generally applicable water potentials should fulfill a number of requirements. They should have a quality close to quantum chemical methods, they should explicitly depend on all degrees of freedom including all relevant many-body interactions, and they should be able to describe molecular dissociation and recombination. In this work, we present a high-dimensional neural network (NN) potential for water clusters based on density-functional theory (DFT) calculations, which is constructed using clusters containing up to 10 monomers and is in principle able to meet all these requirements. We investigate the reliability of specific parametrizations employing two frequently used generalized gradient approximation (GGA) exchange-correlation functionals, PBE and RPBE, as reference methods. We find that the binding energy errors of the NN potentials with respect to DFT are significantly lower than the typical uncertainties of DFT calculations arising from the choice of the exchange-correlation functional. Further, we examine the role of van der Waals interactions, which are not properly described by GGA functionals. Specifically, we incorporate the D3 scheme suggested by Grimme (J. Chem. Phys. 2010, 132, 154104) in our potentials and demonstrate that it can be applied to GGA-based NN potentials in the same way as to DFT calculations without modification. Our results show that the description of small water clusters provided by the RPBE functional is significantly improved if van der Waals interactions are included, while in case of the PBE functional, which is well-known to yield stronger binding than RPBE, van der Waals corrections lead to overestimated binding energies.

Construction of high-dimensional neural network potentials using environment-dependent atom pairs

An accurate determination of the potential energy is the crucial step in computer simulations of chemical processes, but using electronic structure methods on-the-fly in molecular dynamics (MD) is computationally too demanding for many systems. Constructing more efficient interatomic potentials becomes intricate with increasing dimensionality of the potential-energy surface (PES), and for numerous systems the accuracy that can be achieved is still not satisfying and far from the reliability of first-principles calculations. Feed-forward neural networks (NNs) have a very flexible functional form, and in recent years they have been shown to be an accurate tool to construct efficient PESs. High-dimensional NN potentials based on environment-dependent atomic energy contributions have been presented for a number of materials. Still, these potentials may be improved by a more detailed structural description, e.g., in form of atom pairs, which directly reflect the atomic interactions and take the chemical environment into account. We present an implementation of an NN method based on atom pairs, and its accuracy and performance are compared to the atom-based NN approach using two very different systems, the methanol molecule and metallic copper. We find that both types of NN potentials provide an excellent description of both PESs, with the pair-based method yielding a slightly higher accuracy making it a competitive alternative for addressing complex systems in MD simulations.

A neural network potential-energy surface for the water dimer based on environment-dependent atomic energies and charges

Understanding the unique properties of water still represents a significant challenge for theory and experiment. Computer simulations by molecular dynamics require a reliable description of the atomic interactions, and in recent decades countless water potentials have been reported in the literature. Still, most of these potentials contain significant approximations, for instance a frozen internal structure of the individual water monomers. Artificial neural networks (NNs) offer a promising way for the construction of very accurate potential-energy surfaces taking all degrees of freedom explicitly into account. These potentials are based on electronic structure calculations for representative configurations, which are then interpolated to a continuous energy surface that can be evaluated many orders of magnitude faster. We present a full-dimensional NN potential for the water dimer as a first step towards the construction of a NN potential for liquid water. This many-body potential is based on environment-dependent atomic energy contributions, and long-range electrostatic interactions are incorporated employing environment-dependent atomic charges. We show that the potential and derived properties like vibrational frequencies are in excellent agreement with the underlying reference density-functional theory calculations.

Fast and accurate modeling of molecular atomization energies with machine learning

We introduce a machine learning model to predict atomization energies of a diverse set of organic molecules, based on nuclear charges and atomic positions only. The problem of solving the molecular Schrödinger equation is mapped onto a nonlinear statistical regression problem of reduced complexity. Regression models are trained on and compared to atomization energies computed with hybrid density-functional theory. Cross validation over more than seven thousand organic molecules yields a mean absolute error of ∼10  kcal/mol. Applicability is demonstrated for the prediction of molecular atomization potential energy curves.