Low-Scaling Algorithm for Nudged Elastic Band Calculations Using a Surrogate Machine Learning Model

Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks

Neural network (NN) interatomic potentials provide fast prediction of potential energy surfaces, closely matching the accuracy of the electronic structure methods used to produce the training data. However, NN predictions are only reliable within well-learned training domains, and show volatile behavior when extrapolating. Uncertainty quantification methods can flag atomic configurations for which prediction confidence is low, but arriving at such uncertain regions requires expensive sampling of the NN phase space, often using atomistic simulations. Here, we exploit automatic differentiation to drive atomistic systems towards high-likelihood, high-uncertainty configurations without the need for molecular dynamics simulations. By performing adversarial attacks on an uncertainty metric, informative geometries that expand the training domain of NNs are sampled. When combined with an active learning loop, this approach bootstraps and improves NN potentials while decreasing the number of calls to the ground truth method. This efficiency is demonstrated on sampling of kinetic barriers, collective variables in molecules, and supramolecular chemistry in zeolite-molecule interactions, and can be extended to any NN potential architecture and materials system.

Machine-learning accelerated geometry optimization in molecular simulation

Geometry optimization is an important part of both computational materials and surface science because it is the path to finding ground state atomic structures and reaction pathways. These properties are used in the estimation of thermodynamic and kinetic properties of molecular and crystal structures. This process is slow at the quantum level of theory because it involves an iterative calculation of forces using quantum chemical codes such as density functional theory (DFT), which are computationally expensive and which limit the speed of the optimization algorithms. It would be highly advantageous to accelerate this process because then one could do either the same amount of work in less time or more work in the same time. In this work, we provide a neural network (NN) ensemble based active learning method to accelerate the local geometry optimization for multiple configurations simultaneously. We illustrate the acceleration on several case studies including bare metal surfaces, surfaces with adsorbates, and nudged elastic band for two reactions. In all cases, the accelerated method requires fewer DFT calculations than the standard method. In addition, we provide an Atomic Simulation Environment (ASE)-optimizer Python package to make the usage of the NN ensemble active learning for geometry optimization easier.

Perspective on integrating machine learning into computational chemistry and materials science

Machine learning (ML) methods are being used in almost every conceivable area of electronic structure theory and molecular simulation. In particular, ML has become firmly established in the construction of high-dimensional interatomic potentials. Not a day goes by without another proof of principle being published on how ML methods can represent and predict quantum mechanical properties-be they observable, such as molecular polarizabilities, or not, such as atomic charges. As ML is becoming pervasive in electronic structure theory and molecular simulation, we provide an overview of how atomistic computational modeling is being transformed by the incorporation of ML approaches. From the perspective of the practitioner in the field, we assess how common workflows to predict structure, dynamics, and spectroscopy are affected by ML. Finally, we discuss how a tighter and lasting integration of ML methods with computational chemistry and materials science can be achieved and what it will mean for research practice, software development, and postgraduate training.

Accelerating Metadynamics-Based Free-Energy Calculations with Adaptive Machine Learning Potentials

There is an increasing demand for free-energy calculations using ab initio molecular dynamics these days. Metadynamics (MetaD) is frequently utilized to reconstruct the free-energy surface, but it is often computationally intractable for the first-principles calculations. Machine learning potentials (MLPs) have become popular alternatives. However, the training could be a long and arduous process before using them in practical applications. To accelerate MetaD use with MLPs for the free-energy calculation in an easy manner, we propose the adaptive machine learning potential-accelerated metadynamics (AMLP-MetaD). In this method, the MLP in the form of a Gaussian approximation potential (GAP) can adapt itself based on its uncertainty estimation, which decides whether to accept the model prediction or recalculate it with a reference method (usually density functional theory) for further training during the MetaD simulation. We demonstrate that the free-energy landscape similar to the ab initio one can be obtained using AMLP-MetaD with a 10-time speedup. Moreover, the quality of the free-energy results can be deeply improved using Δ-MLP, which is the GAP-corrected density functional tight binding in our case. We exemplify this novel method with two model systems, CO adsorption on the Pt₁₃ cluster and the Pt(111) surface, which are of vital importance in heterogeneous catalysis. The successful application in these two tests highlights that our proposed method can be used in both cluster and periodic systems and for up to two collective variables.

Generalizable Trends in Electrochemical Protonation Barriers

Predicting activation energies for reaction steps is essential for modeling catalytic processes, but accurate barrier simulations often require considerable computational expense, especially for electrochemical reactions. Given the challenges of barrier computations and the growing promise of electrochemical routes for various processes, generalizable energetic trends in electrochemistry can significantly aid in analyzing reaction networks and building microkinetic models. Herein, we employ density functional theory and machine learning nudged elastic band models to simulate electrochemical protonation of *C, *N, and *O monatomic adsorbates from hydronium on a series of transition metal surfaces. We observe a consistent trend of decreasing protonation reaction energies yet increasing activation barriers from *O to *N to *C. Analysis of bond orders and reaction pathways provides insight into the origin of the observed trends in protonation energetics. We hypothesize that these results are relevant for polyatomic adsorbates, which can simplify analysis of reaction mechanisms and inform catalyst design.

Perspective on computational reaction prediction using machine learning methods in heterogeneous catalysis

Heterogeneous catalysis plays a significant role in the modern chemical industry. Towards the rational design of novel catalysts, understanding reactions over surfaces is the most essential aspect. Typical industrial catalytic processes such as syngas conversion and methane utilisation can generate a large reaction network comprising thousands of intermediates and reaction pairs. This complexity not only arises from the permutation of transformations between species but also from the extra reaction channels offered by distinct surface sites. Despite the success in investigating surface reactions at the atomic scale, the huge computational expense of ab initio methods hinders the exploration of such complicated reaction networks. With the proliferation of catalysis studies, machine learning as an emerging tool can take advantage of the accumulated reaction data to emulate the output of ab initio methods towards swift reaction prediction. Here, we briefly summarise the conventional workflow of reaction prediction, including reaction network generation, ab initio thermodynamics and microkinetic modelling. An overview of the frequently used regression models in machine learning is presented. As a promising alternative to full ab initio calculations, machine learning interatomic potentials are highlighted. Furthermore, we survey applications assisted by these methods for accelerating reaction prediction, exploring reaction networks, and computational catalyst design. Finally, we envisage future directions in computationally investigating reactions and implementing machine learning algorithms in heterogeneous catalysis.

Quantum and classical effects in DNA point mutations: Watson-Crick tautomerism in AT and GC base pairs

Proton transfer along the hydrogen bonds of DNA can lead to the creation of short-lived, but biologically relevant point mutations that can further lead to gene mutation and, potentially, cancer. In this work, the energy landscape of the canonical A-T and G-C base pairs (standard, amino-keto) to tautomeric A*-T* and G*-C* (non-standard, imino-enol) Watson-Crick DNA base pairs is modelled with density functional theory and machine-learning nudge-elastic band methods. We calculate the energy barriers and tunnelling rates of hydrogen transfer between and within each base monomer (A, T, G and C). We show that the role of tunnelling in A-T tautomerisation is statistically unlikely due to the presence of a small reverse reaction barrier. On the contrary, the thermal populations of the G*-C* point mutation could be non-trivial and propagate through the replisome. For the direct intramolecular transfer, the reaction is hindered by a substantial energy barrier. However, our calculations indicate that tautomeric bases in their monomeric form have remarkably long lifetimes.

Alkali metal adsorption on metal surfaces: new insights from new tools

The adsorption of sodium on Ru(0001) is studied using ³He spin-echo spectroscopy (HeSE), molecular dynamics simulations (MD) and density functional theory (DFT). In the multi-layer regime, an analysis of helium reflectivity, gives an electron-phonon coupling constant of λ = 0.64 ± 0.06. At sub-monolayer coverage, DFT calculations show that the preferred adsorption site changes from hollow site to top site as the supercell increases and the effective coverage, θ, is reduced from 0.25 to 0.0625 adsorbates per substrate atom. Energy barriers and adsorption geometries taken from DFT are used in molecular dynamics calculations to generate simulated data sets for comparison with measurements. We introduce a new Bayesian method of analysis that compares measurement and model directly, without assuming analytic lineshapes. The value of adsorbate-substrate energy exchange rate (friction) in the MD simulation is the sole variable parameter. Experimental data at a coverage θ = 0.028 compares well with the low-coverage DFT result, giving an effective activation barrier E_eff = 46 ± 4 meV with a friction γ = 0.3 ps^-1. Better fits to the data can be achieved by including additional variable parameters, but in all cases, the mechanism of diffusion is predominantly on a Bravais lattice, suggesting a single adsorption site in the unit cell, despite the close packed geometry.

Identification of earth-abundant materials for selective dehydrogenation of light alkanes to olefins

Selective ethane dehydrogenation (EDH) is an attractive on-purpose strategy for industrial ethylene production. Design of an effective, stable, and earth-abundant catalyst to replace noble metal Pt is the main obstacle for its large-scale application. Herein, we report an experimentally validated theoretical framework to discover promising catalysts for EDH, which combines descriptor-based microkinetic modeling, high-throughput computations, machine-learning concepts, and experiments. Our approach efficiently evaluates 1,998 bimetallic alloys by using accurately calculated C and CH₃ adsorption energies and identifies a small number of new promising noble-metal-free catalysts for selective EDH. A Ni₃Mo alloy predicted to be promising is successfully synthesized, and experimentally proven to outperform Pt in selective ethylene production from EDH, representing an important contribution to the improvement of light alkane dehydrogenation to olefins. These results will provide essential additions in the discovery and application of earth-abundant materials in catalysis.

Rationalization of promoted reverse water gas shift reaction by Pt3Ni alloy: Essential contribution from ensemble effect

Bimetallic alloys have attracted considerable attention due to the tunable catalytic activity and selectivity that can be different from those of pure metals. Here, we study the superior catalytic behaviors of the Pt₃Ni nanowire (NW) over each individual, Pt and Ni NWs during the reverse Water Gas Shift (rWGS) reaction, using density functional theory. The results show that the promoted rWGS activity by Pt₃Ni strongly depends on the ensemble effect (a particular arrangement of active sites introduced by alloying), while the contributions from ligand and strain effects, which are of great importance in electrocatalysis, are rather subtle. As a result, a unique Ni-Pt hybrid ensemble is observed at the 110/111 edge of the Pt₃Ni NW, where the synergy between Ni and Pt sites is active enough to stabilize carbon dioxide on the surface readily for the rWGS reaction but moderate enough to allow for the facile removal of carbon monoxide and hydrogenation of hydroxyl species. Our study highlights the importance of the ensemble effect in heterogeneous catalysis of metal alloys, enabling selective binding-tuning and promotion of catalytic activity.

Restricted-Variance Constrained, Reaction Path, and Transition State Molecular Optimizations Using Gradient-Enhanced Kriging

Gaussian process regression has recently been explored as an alternative to standard surrogate models in molecular equilibrium geometry optimization. In particular, the gradient-enhanced Kriging approach in association with internal coordinates, restricted-variance optimization, and an efficient and fast estimate of hyperparameters has demonstrated performance on par or better than standard methods. In this report, we extend the approach to constrained optimizations and transition states and benchmark it for a set of reactions. We compare the performance of the newly developed method with the standard techniques in the location of transition states and in constrained optimizations, both isolated and in the context of reaction path computation. The results show that the method outperforms the current standard in efficiency as well as in robustness.

Computational Methods in Heterogeneous Catalysis

The unprecedented ability of computations to probe atomic-level details of catalytic systems holds immense promise for the fundamentals-based bottom-up design of novel heterogeneous catalysts, which are at the heart of the chemical and energy sectors of industry. Here, we critically analyze recent advances in computational heterogeneous catalysis. First, we will survey the progress in electronic structure methods and atomistic catalyst models employed, which have enabled the catalysis community to build increasingly intricate, realistic, and accurate models of the active sites of supported transition-metal catalysts. We then review developments in microkinetic modeling, specifically mean-field microkinetic models and kinetic Monte Carlo simulations, which bridge the gap between nanoscale computational insights and macroscale experimental kinetics data with increasing fidelity. We finally review the advancements in theoretical methods for accelerating catalyst design and discovery. Throughout the review, we provide ample examples of applications, discuss remaining challenges, and provide our outlook for the near future.

Modified Single Iteration Synchronous-Transit Approach to Bound Diffusion Barriers for Solid-State Reactions

Herein, we detail an approach to accelerate the computational screening of materials for properties dictated by the kinetics of solid-state diffusion through reliably and rapidly identifying upper and lower bounds to the transition state (TS) energy through our proposed modified single iteration synchronous-transit (MSIST) approach. While this sacrifices providing detailed information of the explicit TS structure, it requires only 30% of the force evaluations of a full nudged elastic band (NEB) TS search and reduces the computational demand to compute estimated diffusion barriers by ∼70% on average. In all 53 cases in which we explicitly compared our results to those of an NEB calculation, the upper and lower bounds identified using this approach bracketed the TS energy calculated with explicit NEB calculations. We use the applications of diffusion of Na⁺ in potential sodium-ion battery electrodes and oxygen vacancy diffusion in solid-oxide fuel cell electrodes and redox mediators for solar thermochemical hydrogen production to demonstrate the power of MSIST for analyzing the kinetics of bulk diffusion. For Na⁺ diffusion through 13 proposed electrode materials in which the average diffusion barrier was 0.28 eV, the average difference between the upper and lower bounds was 0.08 eV. An iterative application of this approach to the three materials with the largest difference between their upper and lower bounds further narrowed the average range of the bounded TS energies to 0.04 eV while still requiring fewer force evaluations than an NEB TS calculation. When applied in a high-throughput manner to study 514 diffusion pathways in 97 different materials, the average difference between the upper and lower bounds was 0.33 eV and the average barrier, as calculated by the average of all upper and lower bounds, was ∼1.7 eV. Because the MSIST approach produces explicit errors, i.e., the difference between the upper and lower bounds energies, even predicted barrier ranges with large errors can be reliably modeled with weighted regression techniques. MSIST enables the analysis of the kinetics of solid-state diffusion across larger sets of materials and can thus efficiently provide data to train statistically learned models of diffusion and to develop physical insights into the diffusion process.

Structure prediction of surface reconstructions by deep reinforcement learning

We demonstrate how image recognition and reinforcement learning combined may be used to determine the atomistic structure of reconstructed crystalline surfaces. A deep neural network represents a reinforcement learning agent that obtains training rewards by interacting with an environment. The environment contains a quantum mechanical potential energy evaluator in the form of a density functional theory program. The agent handles the 3D atomistic structure as a series of stacked 2D images and outputs the next atom type to place and the atomic site to occupy. Agents are seen to require 1000-10 000 single point density functional theory evaluations, to learn by themselves how to build the optimal surface reconstructions of anatase TiO2(001)-(1 × 4) and rutile SnO2(110)-(4 × 1).

Revealing the structure of a catalytic combustion active-site ensemble combining uniform nanocrystal catalysts and theory insights

Supported metal catalysts are extensively used in industrial and environmental applications. To improve their performance, it is crucial to identify the most active sites. This identification is, however, made challenging by the presence of a large number of potential surface structures that complicate such an assignment. Often, the active site is formed by an ensemble of atoms, thus introducing further complications in its identification. Being able to produce uniform structures and identify the ones that are responsible for the catalyst performance is a crucial goal. In this work, we utilize a combination of uniform Pd/Pt nanocrystal catalysts and theory to reveal the catalytic active-site ensemble in highly active propene combustion materials. Using colloidal chemistry to exquisitely control nanoparticle size, we find that intrinsic rates for propene combustion in the presence of water increase monotonically with particle size on Pt-rich catalysts, suggesting that the reaction is structure dependent. We also reveal that water has a near-zero or mildly positive reaction rate order over Pd/Pt catalysts. Theory insights allow us to determine that the interaction of water with extended terraces present in large particles leads to the formation of step sites on metallic surfaces. These specific step-edge sites are responsible for the efficient combustion of propene at low temperature. This work reveals an elusive geometric ensemble, thus clearly identifying the active site in alkene combustion catalysts. These insights demonstrate how the combination of uniform catalysts and theory can provide a much deeper understanding of active-site geometry for many applications.

Restricted-Variance Molecular Geometry Optimization Based on Gradient-Enhanced Kriging

Machine learning techniques, specifically gradient-enhanced Kriging (GEK), have been implemented for molecular geometry optimization. GEK-based optimization has many advantages compared to conventional—step-restricted second-order truncated expansion—molecular optimization methods. In particular, the surrogate model given by GEK can have multiple stationary points, will smoothly converge to the exact model as the number of sample points increases, and contains an explicit expression for the expected error of the model function at an arbitrary point. Machine learning is, however, associated with abundance of data, contrary to the situation desired for efficient geometry optimizations. In this paper, we demonstrate how the GEK procedure can be utilized in a fashion such that in the presence of few data points, the surrogate surface will in a robust way guide the optimization to a minimum of a potential energy surface. In this respect, the GEK procedure will be used to mimic the behavior of a conventional second-order scheme but retaining the flexibility of the superior machine learning approach. Moreover, the expected error will be used in the optimizations to facilitate restricted-variance optimizations. A procedure which relates the eigenvalues of the approximate guessed Hessian with the individual characteristic lengths, used in the GEK model, reduces the number of empirical parameters to optimize to two: the value of the trend function and the maximum allowed variance. These parameters are determined using the extended Baker (e-Baker) and part of the Baker transition-state (Baker-TS) test suites as a training set. The so-created optimization procedure is tested using the e-Baker, full Baker-TS, and S22 test suites, at the density functional theory and second-order Møller–Plesset levels of approximation. The results show that the new method is generally of similar or better performance than a state-of-the-art conventional method, even for cases where no significant improvement was expected.

Deep Learning for Feynman's Path Integral in Strong-Field Time-Dependent Dynamics

Feynman's path integral approach is to sum over all possible spatiotemporal paths to reproduce the quantum wave function and the corresponding time evolution, which has enormous potential to reveal quantum processes in the classical view. However, the complete characterization of the quantum wave function with infinite paths is a formidable challenge, which greatly limits the application potential, especially in the strong-field physics and attosecond science. Instead of brute-force tracking every path one by one, here we propose a deep-learning-performed strong-field Feynman's formulation with a preclassification scheme that can predict directly the final results only with data of initial conditions, so as to attack unsurmountable tasks by existing strong-field methods and explore new physics. Our results build a bridge between deep learning and strong-field physics through Feynman's path integral, which would boost applications of deep learning to study the ultrafast time-dependent dynamics in strong-field physics and attosecond science and shed new light on the quantum-classical correspondence.

Quantum Chemistry in the Age of Machine Learning

As the quantum chemistry (QC) community embraces machine learning (ML), the number of new methods and applications based on the combination of QC and ML is surging. In this Perspective, a view of the current state of affairs in this new and exciting research field is offered, challenges of using machine learning in quantum chemistry applications are described, and potential future developments are outlined. Specifically, examples of how machine learning is used to improve the accuracy and accelerate quantum chemical research are shown. Generalization and classification of existing techniques are provided to ease the navigation in the sea of literature and to guide researchers entering the field. The emphasis of this Perspective is on supervised machine learning.

Predicting metal-metal interactions. I. The influence of strain on nanoparticle and metal adlayer stabilities

Strain-engineering of bimetallic nanomaterials is an important design strategy for developing new catalysts. Herein, we introduce an approach for including strain effects into a recently introduced, density functional theory (DFT)-based alloy stability model. The model predicts adsorption site stabilities in nanoparticles and connects these site stabilities with catalytic reactivity and selectivity. Strain-based dependencies will increase the model's accuracy for nanoparticles affected by finite-size effects. In addition to the stability of small nanoparticles, strain also influences the heat of adsorption of epitaxially grown metal-on-metal adlayers. In this respect, we successfully benchmark the strain-including alloy stability model with previous experimentally determined trends in the heats of adsorption of Au and Cu adlayers on Pt (111). For these systems, our model predicts stronger bimetallic interactions in the first monolayer than monometallic interactions in the second monolayer. We explicitly quantify the interplay between destabilizing strain effects and the energy gained by forming new metal-metal bonds. While tensile strain in the first Cu monolayer significantly destabilizes the adsorption strength, compressive strain in the first Au monolayer has a minimal impact on the heat of adsorption. Hence, this study introduces and, by comparison with previous experiments, validates an efficient DFT-based approach for strain-engineering the stability, and, in turn, the catalytic performance, of active sites in bimetallic alloys with atomic level resolution.

Efficient Global Structure Optimization with a Machine-Learned Surrogate Model

We propose a scheme for global optimization with first-principles energy expressions of atomistic structure. While unfolding its search, the method actively learns a surrogate model of the potential energy landscape on which it performs a number of local relaxations (exploitation) and further structural searches (exploration). Assuming Gaussian processes, deploying two separate kernel widths to better capture rough features of the energy landscape while retaining a good resolution of local minima, an acquisition function is used to decide on which of the resulting structures is the more promising and should be treated at the first-principles level. The method is demonstrated to outperform by 2 orders of magnitude a well established first-principles based evolutionary algorithm in finding surface reconstructions. Finally, global optimization with first-principles energy expressions is utilized to identify initial stages of the edge oxidation and oxygen intercalation of graphene sheets on the Ir(111) surface.

Benchmarking the acceleration of materials discovery by sequential learning

Sequential learning (SL) strategies, i.e. iteratively updating a machine learning model to guide experiments, have been proposed to significantly accelerate materials discovery and research. Applications on computational datasets and a handful of optimization experiments have demonstrated the promise of SL, motivating a quantitative evaluation of its ability to accelerate materials discovery, specifically in the case of physical experiments. The benchmarking effort in the present work quantifies the performance of SL algorithms with respect to a breadth of research goals: discovery of any "good" material, discovery of all "good" materials, and discovery of a model that accurately predicts the performance of new materials. To benchmark the effectiveness of different machine learning models against these goals, we use datasets in which the performance of all materials in the search space is known from high-throughput synthesis and electrochemistry experiments. Each dataset contains all pseudo-quaternary metal oxide combinations from a set of six elements (chemical space), the performance metric chosen is the electrocatalytic activity (overpotential) for the oxygen evolution reaction (OER). A diverse set of SL schemes is tested on four chemical spaces, each containing 2121 catalysts. The presented work suggests that research can be accelerated by up to a factor of 20 compared to random acquisition in specific scenarios. The results also show that certain choices of SL models are ill-suited for a given research goal resulting in substantial deceleration compared to random acquisition methods. The results provide quantitative guidance on how to tune an SL strategy for a given research goal and demonstrate the need for a new generation of materials-aware SL algorithms to further accelerate materials discovery.

Scaled and Dynamic Optimizations of Nudged Elastic Bands

We present a modified nudged elastic band routine that can reduce the number of force calls by more than 50% for bands with nonuniform convergence. The method, which we call "dyNEB", dynamically and selectively optimizes images on the basis of the perpendicular PES-derived forces and parallel spring forces acting on that region of the band. The convergence criteria are scaled to focus on the region of interest, i.e., the saddle point, while maintaining continuity of the band and avoiding truncation. We show that this method works well for solid state reaction barriers-nonelectrochemical in general and electrochemical in particular-and that the number of force calls can be significantly reduced without loss of resolution at the saddle point.

Catalysis-Hub.org, an open electronic structure database for surface reactions

We present a new open repository for chemical reactions on catalytic surfaces, available at https://www.catalysis-hub.org . The featured database for surface reactions contains more than 100,000 chemisorption and reaction energies obtained from electronic structure calculations, and is continuously being updated with new datasets. In addition to providing quantum-mechanical results for a broad range of reactions and surfaces from different publications, the database features a systematic, large-scale study of chemical adsorption and hydrogenation on bimetallic alloy surfaces. The database contains reaction specific information, such as the surface composition and reaction energy for each reaction, as well as the surface geometries and calculational parameters, essential for data reproducibility. By providing direct access via the web-interface as well as a Python API, we seek to accelerate the discovery of catalytic materials for sustainable energy applications by enabling researchers to efficiently use the data as a basis for new calculations and model generation.