Ab initio potential-energy surfaces for complex, multichannel systems using modified novelty sampling and feedforward neural networks

Representing potential energy surfaces by high-dimensional neural network potentials

The development of interatomic potentials employing artificial neural networks has seen tremendous progress in recent years. While until recently the applicability of neural network potentials (NNPs) has been restricted to low-dimensional systems, this limitation has now been overcome and high-dimensional NNPs can be used in large-scale molecular dynamics simulations of thousands of atoms. NNPs are constructed by adjusting a set of parameters using data from electronic structure calculations, and in many cases energies and forces can be obtained with very high accuracy. Therefore, NNP-based simulation results are often very close to those gained by a direct application of first-principles methods. In this review, the basic methodology of high-dimensional NNPs will be presented with a special focus on the scope and the remaining limitations of this approach. The development of NNPs requires substantial computational effort as typically thousands of reference calculations are required. Still, if the problem to be studied involves very large systems or long simulation times this overhead is regained quickly. Further, the method is still limited to systems containing about three or four chemical elements due to the rapidly increasing complexity of the configuration space, although many atoms of each species can be present. Due to the ability of NNPs to describe even extremely complex atomic configurations with excellent accuracy irrespective of the nature of the atomic interactions, they represent a general and therefore widely applicable technique, e.g. for addressing problems in materials science, for investigating properties of interfaces, and for studying solvation processes.

Six-dimensional quantum dynamics for dissociative chemisorption of H2 and D2 on Ag(111) on a permutation invariant potential energy surface

Quantum dynamics on a permutation invariant potential energy surface for H₂ dissociation on Ag(111) yield satisfactory agreement with experiment.

Studying chemical reactivity in a virtual environment

Chemical reactivity of a set of reactants is determined by its potential (electronic) energy (hyper)surface. The high dimensionality of this surface renders it difficult to efficiently explore reactivity in a large reactive system. Exhaustive sampling techniques and search algorithms are not straightforward to employ as it is not clear which explored path will eventually produce the minimum energy path of a reaction passing through a transition structure. Here, the chemist's intuition would be of invaluable help, but it cannot be easily exploited because (1) no intuitive and direct tool for the scientist to manipulate molecular structures is currently available and because (2) quantum chemical calculations are inherently expensive in terms of computational effort. In this work, we elaborate on how the chemist can be reintroduced into the exploratory process within a virtual environment that provides immediate feedback and intuitive tools to manipulate a reactive system. We work out in detail how this immersion should take place. We provide an analysis of modern semi-empirical methods which already today are candidates for the interactive study of chemical reactivity. Implications of manual structure manipulations for their physical meaning and chemical relevance are carefully analysed in order to provide sound theoretical foundations for the interpretation of the interactive reactivity exploration.

A density-functional theory-based neural network potential for water clusters including van der Waals corrections

The fundamental importance of water for many chemical processes has motivated the development of countless efficient but approximate water potentials for large-scale molecular dynamics simulations, from simple empirical force fields to very sophisticated flexible water models. Accurate and generally applicable water potentials should fulfill a number of requirements. They should have a quality close to quantum chemical methods, they should explicitly depend on all degrees of freedom including all relevant many-body interactions, and they should be able to describe molecular dissociation and recombination. In this work, we present a high-dimensional neural network (NN) potential for water clusters based on density-functional theory (DFT) calculations, which is constructed using clusters containing up to 10 monomers and is in principle able to meet all these requirements. We investigate the reliability of specific parametrizations employing two frequently used generalized gradient approximation (GGA) exchange-correlation functionals, PBE and RPBE, as reference methods. We find that the binding energy errors of the NN potentials with respect to DFT are significantly lower than the typical uncertainties of DFT calculations arising from the choice of the exchange-correlation functional. Further, we examine the role of van der Waals interactions, which are not properly described by GGA functionals. Specifically, we incorporate the D3 scheme suggested by Grimme (J. Chem. Phys. 2010, 132, 154104) in our potentials and demonstrate that it can be applied to GGA-based NN potentials in the same way as to DFT calculations without modification. Our results show that the description of small water clusters provided by the RPBE functional is significantly improved if van der Waals interactions are included, while in case of the PBE functional, which is well-known to yield stronger binding than RPBE, van der Waals corrections lead to overestimated binding energies.

Full-dimensional (15-dimensional) ab initio analytical potential energy surface for the H7+ cluster

Full-dimensional ab initio potential energy surface is constructed for the H(7)(+) cluster. The surface is a fit to roughly 160,000 interaction energies obtained with second-order MöllerPlesset perturbation theory and the cc-pVQZ basis set, using the invariant polynomial method [B. J. Braams and J. M. Bowman, Int. Rev. Phys. Chem. 28, 577 (2009)]. We employ permutationally invariant basis functions in Morse-type variables for all the internuclear distances to incorporate permutational symmetry with respect to interchange of H atoms into the representation of the surface. We describe how different configurations are selected in order to create the database of the interaction energies for the linear least squares fitting procedure. The root-mean-square error of the fit is 170 cm(-1) for the entire data set. The surface dissociates correctly to the H(5)(+) + H(2) fragments. A detailed analysis of its topology, as well as comparison with additional ab initio calculations, including harmonic frequencies, verify the quality and accuracy of the parameterized potential. This is the first attempt to present an analytical representation of the 15-dimensional surface of the H(7)(+) cluster for carrying out dynamics studies.

Construction of high-dimensional neural network potentials using environment-dependent atom pairs

An accurate determination of the potential energy is the crucial step in computer simulations of chemical processes, but using electronic structure methods on-the-fly in molecular dynamics (MD) is computationally too demanding for many systems. Constructing more efficient interatomic potentials becomes intricate with increasing dimensionality of the potential-energy surface (PES), and for numerous systems the accuracy that can be achieved is still not satisfying and far from the reliability of first-principles calculations. Feed-forward neural networks (NNs) have a very flexible functional form, and in recent years they have been shown to be an accurate tool to construct efficient PESs. High-dimensional NN potentials based on environment-dependent atomic energy contributions have been presented for a number of materials. Still, these potentials may be improved by a more detailed structural description, e.g., in form of atom pairs, which directly reflect the atomic interactions and take the chemical environment into account. We present an implementation of an NN method based on atom pairs, and its accuracy and performance are compared to the atom-based NN approach using two very different systems, the methanol molecule and metallic copper. We find that both types of NN potentials provide an excellent description of both PESs, with the pair-based method yielding a slightly higher accuracy making it a competitive alternative for addressing complex systems in MD simulations.

A neural network potential-energy surface for the water dimer based on environment-dependent atomic energies and charges

Understanding the unique properties of water still represents a significant challenge for theory and experiment. Computer simulations by molecular dynamics require a reliable description of the atomic interactions, and in recent decades countless water potentials have been reported in the literature. Still, most of these potentials contain significant approximations, for instance a frozen internal structure of the individual water monomers. Artificial neural networks (NNs) offer a promising way for the construction of very accurate potential-energy surfaces taking all degrees of freedom explicitly into account. These potentials are based on electronic structure calculations for representative configurations, which are then interpolated to a continuous energy surface that can be evaluated many orders of magnitude faster. We present a full-dimensional NN potential for the water dimer as a first step towards the construction of a NN potential for liquid water. This many-body potential is based on environment-dependent atomic energy contributions, and long-range electrostatic interactions are incorporated employing environment-dependent atomic charges. We show that the potential and derived properties like vibrational frequencies are in excellent agreement with the underlying reference density-functional theory calculations.

Practical training framework for fitting a function and its derivatives

This paper describes a practical framework for using multilayer feedforward neural networks to simultaneously fit both a function and its first derivatives. This framework involves two steps. The first step is to train the network to optimize a performance index, which includes both the error in fitting the function and the error in fitting the derivatives. The second step is to prune the network by removing neurons that cause overfitting and then to retrain it. This paper describes two novel types of overfitting that are only observed when simultaneously fitting both a function and its first derivatives. A new pruning algorithm is proposed to eliminate these types of overfitting. Experimental results show that the pruning algorithm successfully eliminates the overfitting and produces the smoothest responses and the best generalization among all the training algorithms that we have tested.

Communication: Hybrid ensembles for improved force matching

Force matching is a method for parameterizing empirical potentials in which the empirical parameters are fitted to a reference potential energy surface (PES). Typically, training data are sampled from a canonical ensemble generated with either the empirical potential or the reference PES. In this Communication, we show that sampling from either ensemble risks excluding critical regions of configuration space, leading to fitted potentials that deviate significantly from the reference PES. We present a hybrid ensemble which combines the Boltzmann probabilities of both potential surfaces into the fitting procedure, and we demonstrate that this technique improves the quality and stability of empirical potentials.

Toward a realistic density functional theory potential energy surface for the H5+ cluster

The potential energy surface of H(5)(+) is characterized using density functional theory. The hypersurface is evaluated at selected configurations employing different functionals, and compared with results obtained from ab initio CCSD(T) calculations. The lowest ten stationary points (minima and saddle-points) on the surface are located, and the features of the short-, intermediate-, and long-range intermolecular interactions are also investigated. A detailed analysis of the surface's topology, and comparisons with extensive CCSD(T) results, as well as a recent ab initio analytical surface, shows that density functional theory calculations using the B3(H) functional represent very well all aspects studied on the H(5)(+) potential. These include the tiny energy difference between the minimum at 1-C(2v) configuration and the 2-D(2d) one corresponding to the transition state for the proton transfer between the two equivalent C(2v) minima, and also the correct asymptotic behavior of the long-range interactions. The calculated binding energy and dissociation enthalpies compare very well with previous benchmark coupled-cluster ab initio data, and with experimental data available. Based on these results the use of such approach to perform first-principles molecular dynamics simulations could provide reliable information regarding the dynamics of protonated hydrogen clusters.

Input vector optimization of feed-forward neural networks for fitting ab initio potential-energy databases

The variation in the fitting accuracy of neural networks (NNs) when used to fit databases comprising potential energies obtained from ab initio electronic structure calculations is investigated as a function of the number and nature of the elements employed in the input vector to the NN. Ab initio databases for H(2)O(2), HONO, Si(5), and H(2)C[Double Bond]CHBr were employed in the investigations. These systems were chosen so as to include four-, five-, and six-body systems containing first, second, third, and fourth row elements with a wide variety of chemical bonding and whose conformations cover a wide range of structures that occur under high-energy machining conditions and in chemical reactions involving cis-trans isomerizations, six different types of two-center bond ruptures, and two different three-center dissociation reactions. The ab initio databases for these systems were obtained using density functional theory/B3LYP, MP2, and MP4 methods with extended basis sets. A total of 31 input vectors were investigated. In each case, the elements of the input vector were chosen from interatomic distances, inverse powers of the interatomic distance, three-body angles, and dihedral angles. Both redundant and nonredundant input vectors were investigated. The results show that among all the input vectors investigated, the set employed in the Z-matrix specification of the molecular configurations in the electronic structure calculations gave the lowest NN fitting accuracy for both Si(5) and vinyl bromide. The underlying reason for this result appears to be the discontinuity present in the dihedral angle for planar geometries. The use of trigometric functions of the angles as input elements produced significantly improved fitting accuracy as this choice eliminates the discontinuity. The most accurate fitting was obtained when the elements of the input vector were taken to have the form R(ij) (-n), where the R(ij) are the interatomic distances. When the Levenberg-Marquardt procedure was modified to permit error minimization with respect to n as well as the weights and biases of the NN, the optimum powers were all found to lie in the range of 1.625-2.38 for the four systems studied. No statistically significant increase in fitting accuracy was achieved for vinyl bromide when a different value of n was employed and optimized for each bond type. The rate of change in the fitting error with n is found to be very small when n is near its optimum value. Consequently, good fitting accuracy can be achieved by employing a value of n in the middle of the above range. The use of interparticle distances as elements of the input vector rather than the Z-matrix variables employed in the electronic structure calculations is found to reduce the rms fitting errors by factors of 8.86 and 1.67 for Si(5) and vinyl bromide, respectively. If the interparticle distances are replaced with input elements of the form R(ij) (-n) with n optimized, further reductions in the rms error by a factor of 1.31 to 2.83 for the four systems investigated are obtained. A major advantage of using this procedure to increase NN fitting accuracy rather than increasing the number of neurons or the size of the database is that the required increase in computational effort is very small.

Development of generalized potential-energy surfaces using many-body expansions, neural networks, and moiety energy approximations

A general method for the development of potential-energy hypersurfaces is presented. The method combines a many-body expansion to represent the potential-energy surface with two-layer neural networks (NN) for each M-body term in the summations. The total number of NNs required is significantly reduced by employing a moiety energy approximation. An algorithm is presented that efficiently adjusts all the coupled NN parameters to the database for the surface. Application of the method to four different systems of increasing complexity shows that the fitting accuracy of the method is good to excellent. For some cases, it exceeds that available by other methods currently in literature. The method is illustrated by fitting large databases of ab initio energies for Si(n) (n=3,4,...,7) clusters obtained from density functional theory calculations and for vinyl bromide (C(2)H(3)Br) and all products for dissociation into six open reaction channels (12 if the reverse reactions are counted as separate open channels) that include C-H and C-Br bond scissions, three-center HBr dissociation, and three-center H(2) dissociation. The vinyl bromide database comprises the ab initio energies of 71 969 configurations computed at MP4(SDQ) level with a 6-31G(d,p) basis set for the carbon and hydrogen atoms and Huzinaga's (4333/433/4) basis set augmented with split outer s and p orbitals (43321/4321/4) and a polarization f orbital with an exponent of 0.5 for the bromine atom. It is found that an expansion truncated after the three-body terms is sufficient to fit the Si(5) system with a mean absolute testing set error of 5.693x10(-4) eV. Expansions truncated after the four-body terms for Si(n) (n=3,4,5) and Si(n) (n=3,4,...,7) provide fits whose mean absolute testing set errors are 0.0056 and 0.0212 eV, respectively. For vinyl bromide, a many-body expansion truncated after the four-body terms provides fitting accuracy with mean absolute testing set errors that range between 0.0782 and 0.0808 eV. These errors correspond to mean percent errors that fall in the range 0.98%-1.01%. Our best result using the present method truncated after the four-body summation with 16 NNs yields a testing set error that is 20.3% higher than that obtained using a 15-dimensional (15-140-1) NN to fit the vinyl bromide database. This appears to be the price of the added simplicity of the many-body expansion procedure.

Using neural networks, optimized coordinates, and high-dimensional model representations to obtain a vinyl bromide potential surface

We demonstrate that it is possible to obtain good potentials using high-dimensional model representations (HDMRs) fitted with neural networks (NNs) from data in 12 dimensions and 15 dimensions. The HDMR represents the potential as a sum of lower-dimensional functions and our NN-based approach makes it possible to obtain all of these functions from one set of fitting points. To reduce the number of terms in the HDMR, we use optimized redundant coordinates. By using exponential neurons, one obtains a potential in sum-of-products form, which greatly facilitates quantum dynamics calculations. A 12-dimensional (reference) potential surface for vinyl bromide is first refitted to show that it can be represented as a sum of two-dimensional functions. To fit 3d functions of the original coordinates, to improve the potential, a huge amount of data would be required. Redundant coordinates avoid this problem. They enable us to bypass the combinatorial explosion of the number of terms which plagues all HDMR and multimode-type methods. We also fit to a set of approximately 70,000 ab initio points for vinyl bromide in 15 dimensions [M. Malshe et al., J. Chem. Phys. 127, 134105 (2007)] and show that it is possible to obtain a surface in sum-of-products form of quality similar to the quality of the full-dimensional fit. Although we obtain a full-dimensional surface, we limit the cost of the fitting by building it from fits of six-dimensional functions, each of which requires only a small NN.

Ab initio molecular dynamics of hydrogen dissociation on metal surfaces using neural networks and novelty sampling

We outline a hybrid multiscale approach for the construction of ab initio potential energy surfaces (PESs) useful for performing six-dimensional (6D) classical or quantum mechanical molecular dynamics (MD) simulations of diatomic molecules reacting at single crystal surfaces. The algorithm implements concepts from the corrugation reduction procedure, which reduces energetic variation in the PES, and uses neural networks for interpolation of smoothed ab initio data. A novelty sampling scheme is implemented and used to identify configurations that are most likely to be predicted inaccurately by the neural network. This hybrid multiscale approach, which couples PES construction at the electronic structure level to MD simulations at the atomistic scale, reduces the number of density functional theory (DFT) calculations needed to specify an accurate PES. Due to the iterative nature of the novelty sampling algorithm, it is possible to obtain a quantitative measure of the convergence of the PES with respect to the number of ab initio calculations used to train the neural network. We demonstrate the algorithm by first applying it to two analytic potentials, which model the H2/Pt(111) and H2/Cu(111) systems. These potentials are of the corrugated London-Eyring-Polanyi-Sato form, which are based on DFT calculations, but are not globally accurate. After demonstrating the convergence of the PES using these simple potentials, we use DFT calculations directly and obtain converged semiclassical trajectories for the H2/Pt(111) system at the PW91/generalized gradient approximation level. We obtain a converged PES for a 6D hydrogen-surface dissociation reaction using novelty sampling coupled directly to DFT. These results, in excellent agreement with experiments and previous theoretical work, are compared to previous simulations in order to explore the sensitivity of the PES (and therefore MD) to the choice of exchange and correlation functional. Despite having a lower energetic corrugation in our PES, we obtain a broader reaction probability curve than previous simulations, which is attributed to increased geometric corrugation in the PES and the effect of nonparallel dissociation pathways.

Representing molecule-surface interactions with symmetry-adapted neural networks

The accurate description of molecule-surface interactions requires a detailed knowledge of the underlying potential-energy surface (PES). Recently, neural networks (NNs) have been shown to be an efficient technique to accurately interpolate the PES information provided for a set of molecular configurations, e.g., by first-principles calculations. Here, we further develop this approach by building the NN on a new type of symmetry functions, which allows to take the symmetry of the surface exactly into account. The accuracy and efficiency of such symmetry-adapted NNs is illustrated by the application to a six-dimensional PES describing the interaction of oxygen molecules with the Al(111) surface.

Using redundant coordinates to represent potential energy surfaces with lower-dimensional functions

We propose a method for fitting potential energy surfaces with a sum of component functions of lower dimensionality. This form facilitates quantum dynamics calculations. We show that it is possible to reduce the dimensionality of the component functions by introducing new and redundant coordinates obtained with linear transformations. The transformations are obtained from a neural network. Different coordinates are used for different component functions and the new coordinates are determined as the potential is fitted. The quality of the fits and the generality of the method are illustrated by fitting reference potential surfaces of hydrogen peroxide and of the reaction OH+H(2)-->H(2)O+H.

Interpolating moving least-squares methods for fitting potential energy surfaces: Computing high-density potential energy surface data from low-density ab initio data points

A highly accurate and efficient method for molecular global potential energy surface (PES) construction and fitting is demonstrated. An interpolating-moving-least-squares (IMLS)-based method is developed using low-density ab initio Hessian values to compute high-density PES parameters suitable for accurate and efficient PES representation. The method is automated and flexible so that a PES can be optimally generated for classical trajectories, spectroscopy, or other applications. Two important bottlenecks for fitting PESs are addressed. First, high accuracy is obtained using a minimal density of ab initio points, thus overcoming the bottleneck of ab initio point generation faced in applications of modified-Shepard-based methods. Second, high efficiency is also possible (suitable when a huge number of potential energy and gradient evaluations are required during a trajectory calculation). This overcomes the bottleneck in high-order IMLS-based methods, i.e., the high cost/accuracy ratio for potential energy evaluations. The result is a set of hybrid IMLS methods in which high-order IMLS is used with low-density ab initio Hessian data to compute a dense grid of points at which the energy, Hessian, or even high-order IMLS fitting parameters are stored. A series of hybrid methods is then possible as these data can be used for neural network fitting, modified-Shepard interpolation, or approximate IMLS. Results that are indicative of the accuracy, efficiency, and scalability are presented for one-dimensional model potentials as well as for three-dimensional (HCN) and six-dimensional (HOOH) molecular PESs.