Kinetic energy densities based on the fourth order gradient expansion: performance in different classes of materials and improvement via machine learning

Kohn-Sham accuracy from orbital-free density functional theory via Δ-machine learning

We present a Δ-machine learning model for obtaining Kohn-Sham accuracy from orbital-free density functional theory (DFT) calculations. In particular, we employ a machine-learned force field (MLFF) scheme based on the kernel method to capture the difference between Kohn-Sham and orbital-free DFT energies/forces. We implement this model in the context of on-the-fly molecular dynamics simulations and study its accuracy, performance, and sensitivity to parameters for representative systems. We find that the formalism not only improves the accuracy of Thomas-Fermi-von Weizsäcker orbital-free energies and forces by more than two orders of magnitude but is also more accurate than MLFFs based solely on Kohn-Sham DFT while being more efficient and less sensitive to model parameters. We apply the framework to study the structure of molten Al0.88Si0.12, the results suggesting no aggregation of Si atoms, in agreement with a previous Kohn-Sham study performed at an order of magnitude smaller length and time scales.

Machine learning of kinetic energy densities with target and feature smoothing: Better results with fewer training data

Machine learning (ML) of kinetic energy functionals (KEFs), in particular kinetic energy density (KED) functionals, is a promising way to construct KEFs for orbital-free density functional theory (DFT). Neural networks and kernel methods including Gaussian process regression (GPR) have been used to learn Kohn-Sham (KS) KED from density-based descriptors derived from KS DFT calculations. The descriptors are typically expressed as functions of different powers and derivatives of the electron density. This can generate large and extremely unevenly distributed datasets, which complicates effective application of ML techniques. Very uneven data distributions require many training datapoints, can cause overfitting, and can ultimately lower the quality of an ML KED model. We show that one can produce more accurate ML models from fewer data by working with smoothed density-dependent variables and KED. Smoothing palliates the issue of very uneven data distributions and associated difficulties of sampling while retaining enough spatial structure necessary for working within the paradigm of KEDF. We use GPR as a function of smoothed terms of the fourth order gradient expansion and KS effective potential and obtain accurate and stable (with respect to different random choices of training points) kinetic energy models for Al, Mg, and Si simultaneously from as few as 2000 samples (about 0.3% of the total KS DFT data). In particular, accuracies on the order of 1% in a measure of the quality of energy-volume dependence B'=EV0-ΔV-2EV0+E(V0+ΔV)ΔV/V02 (where V0 is the equilibrium volume and ΔV is a deviation from it) are obtained simultaneously for all three materials.

Orbital-Free Density Functional Theory: An Attractive Electronic Structure Method for Large-Scale First-Principles Simulations

Kohn-Sham Density Functional Theory (KSDFT) is the most widely used electronic structure method in chemistry, physics, and materials science, with thousands of calculations cited annually. This ubiquity is rooted in the favorable accuracy vs cost balance of KSDFT. Nonetheless, the ambitions and expectations of researchers for use of KSDFT in predictive simulations of large, complicated molecular systems are confronted with an intrinsic computational cost-scaling challenge. Particularly evident in the context of first-principles molecular dynamics, the challenge is the high cost-scaling associated with the computation of the Kohn-Sham orbitals. Orbital-free DFT (OFDFT), as the name suggests, circumvents entirely the explicit use of those orbitals. Without them, the structural and algorithmic complexity of KSDFT simplifies dramatically and near-linear scaling with system size irrespective of system state is achievable. Thus, much larger system sizes and longer simulation time scales (compared to conventional KSDFT) become accessible; hence, new chemical phenomena and new materials can be explored. In this review, we introduce the historical contexts of OFDFT, its theoretical basis, and the challenge of realizing its promise via approximate kinetic energy density functionals (KEDFs). We review recent progress on that challenge for an array of KEDFs, such as one-point, two-point, and machine-learnt, as well as some less explored forms. We emphasize use of exact constraints and the inevitability of design choices. Then, we survey the associated numerical techniques and implemented algorithms specific to OFDFT. We conclude with an illustrative sample of applications to showcase the power of OFDFT in materials science, chemistry, and physics.

KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory

Orbital-free density functional theory (OF-DFT) holds promise to compute ground state molecular properties at minimal cost. However, it has been held back by our inability to compute the kinetic energy as a functional of electron density alone. Here, we set out to learn the kinetic energy functional from ground truth provided by the more expensive Kohn-Sham density functional theory. Such learning is confronted with two key challenges: Giving the model sufficient expressivity and spatial context while limiting the memory footprint to afford computations on a GPU and creating a sufficiently broad distribution of training data to enable iterative density optimization even when starting from a poor initial guess. In response, we introduce KineticNet, an equivariant deep neural network architecture based on point convolutions adapted to the prediction of quantities on molecular quadrature grids. Important contributions include convolution filters with sufficient spatial resolution in the vicinity of nuclear cusp, an atom-centric sparse but expressive architecture that relays information across multiple bond lengths, and a new strategy to generate varied training data by finding ground state densities in the face of perturbations by a random external potential. KineticNet achieves, for the first time, chemical accuracy of the learned functionals across input densities and geometries of tiny molecules. For two-electron systems, we additionally demonstrate OF-DFT density optimization with chemical accuracy.

Neural Network with Optimal Neuron Activation Functions Based on Additive Gaussian Process Regression

Feed-forward neural networks (NNs) are a staple machine learning method widely used in many areas of science and technology, including physical chemistry, computational chemistry, and materials informatics. While even a single-hidden-layer NN is a universal approximator, its expressive power is limited by the use of simple neuron activation functions (such as sigmoid functions) that are typically the same for all neurons. More flexible neuron activation functions would allow the use of fewer neurons and layers and thereby save computational cost and improve expressive power. We show that additive Gaussian process regression (GPR) can be used to construct optimal neuron activation functions that are individual to each neuron. An approach is also introduced that avoids nonlinear fitting of neural network parameters by defining them with rules. The resulting method combines the advantage of robustness of a linear regression with the higher expressive power of an NN. We demonstrate the approach by fitting the potential energy surfaces of the water molecule and formaldehyde. Without requiring any nonlinear optimization, the additive-GPR-based approach outperforms a conventional NN in the high-accuracy regime, where a conventional NN suffers more from overfitting.

Symmetrized non-decomposable approximations of the non-additive kinetic energy functional

In subsystem density functional theory (DFT), the bottom-up strategy to approximate the multivariable functional of the non-additive kinetic energy (NAKE) makes it possible to impose exact properties on the corresponding NAKE potential (NAKEP). Such a construction might lead to a non-symmetric and non-homogeneous functional, which excludes the use of such approximations for the evaluation of the total energy. We propose a general formalism to construct a symmetric version based on a perturbation theory approach of the energy expression for the asymmetric part. This strategy is then applied to construct a symmetrized NAKE corresponding to the NAKEP developed recently [Polak et al., J. Chem. Phys. 156, 044103 (2022)], making it possible to evaluate consistently the energy. These functionals were used to evaluate the interaction energy in several model intermolecular complexes using the formal framework of subsystem DFT. The new symmetrized energy expression shows a superior qualitative performance over common decomposable models.

Automatic differentiation for orbital-free density functional theory

Differentiable programming has facilitated numerous methodological advances in scientific computing. Physics engines supporting automatic differentiation have simpler code, accelerating the development process and reducing the maintenance burden. Furthermore, fully differentiable simulation tools enable direct evaluation of challenging derivatives-including those directly related to properties measurable by experiment-that are conventionally computed with finite difference methods. Here, we investigate automatic differentiation in the context of orbital-free density functional theory (OFDFT) simulations of materials, introducing PROFESS-AD. Its automatic evaluation of properties derived from first derivatives, including functional potentials, forces, and stresses, facilitates the development and testing of new density functionals, while its direct evaluation of properties requiring higher-order derivatives, such as bulk moduli, elastic constants, and force constants, offers more concise implementations than conventional finite difference methods. For these reasons, PROFESS-AD serves as an excellent prototyping tool and provides new opportunities for OFDFT.

Machine learning in computational chemistry: interplay between (non)linearity, basis sets, and dimensionality

Machine learning (ML) based methods and tools have now firmly established themselves in physical chemistry and in particular in theoretical and computational chemistry and in materials chemistry. The generality of popular ML techniques such as neural networks or kernel methods (Gaussian process and kernel ridge regression and their flavors) permitted their application to diverse problems from prediction of properties of functional materials (catalysts, solid state ionic conductors, etc.) from descriptors to the building of interatomic potentials (where ML is currently routinely used in applications) and electron density functionals. These ML techniques are assumed to have superior expressive power of nonlinear methods, and are often used "as is", with concepts such as "non-parametric" or "deep learning" used without a clear justification for their need or advantage over simpler and more robust alternatives. In this Perspective, we highlight some interrelations between popular ML techniques and traditional linear regressions and basis expansions and demonstrate that in certain regimes (such as a very high dimensionality) these approximations might collapse. We also discuss ways to recover the expressive power of a nonlinear approach and to help select hyperparameters with the help of high-dimensional model representation and to obtain elements of insight while preserving the generality of the method.

Neural network-based pseudopotential: development of a transferable local pseudopotential

Transferable local pseudopotentials (LPPs) are essential for fast quantum simulations of materials. However, various types of LPPs suffer from low transferability, especially since they do not consider the norm-conserving condition. Here we propose a novel approach based on a deep neural network to produce transferable LPPs. We introduced a generalized Kerker method expressed with the deep neural network to represent the norm-conserving pseudo-wavefunctions. Its unique feature is that all necessary conditions of pseudopotentials can be explicitly considered in terms of a loss function. Then, it can be minimized using the back-propagation technique just with single point all-electron atom data. To assess the transferability and accuracy of the neural network-based LPPs (NNLPs), we carried out density functional theory calculations for the s- and p-block elements of the second to the fourth periods. The NNLPs outperformed other types of LPPs in both atomic and bulk calculations for most elements. In particular, they showed good transferability by predicting various properties of bulk systems including binary alloys with higher accuracy than LPPs tailored to bulk data.

Orbital-Free Methods for Plasmonics: Linear Response

Plasmonic systems, such as metal nanoparticles, are widely used in different application areas, going from biology to photovoltaics.The modeling of the optical response of such systems is of fundamental importance to analyze their behavior and to design new systems with required properties.When the characteristic sizes/distances reach a few nanometers, non-local and spill-out effects become relevant and conventional classical electrodynamics models are no more appropriate. Methods based on the Time-Dependent Density-Functional Theory (TD-DFT) represent the current reference for the description of quantum effects. However, TD-DFT is based on knowledge of all occupied orbitals whose calculation is computationally prohibitive to model large plasmonic systems of interest for applications.On the other hand, methods based on the Orbital-Free (OF) formulation of TD-DFT, can scale linearly with the system size.In this Review, OF methods ranging from semiclassical models to the quantum hydrodynamic theory, will be derived from the linear response TD-DFT, so that the key approximations and properties of each method can be clearly highlighted. The accuracy of the various approximations will be then validated for the linear optical properties of jellium nanoparticles, the most relevant model system in plasmonics. OF methods can describe the collective excitations in plasmonic systems with great accuracy andwithout system-tuned parameters. The accuracy on these methods depends only on the accuracy on the (universal) kinetic energy functional of the ground-state electronic density. Current approximations and future development directions will be indicated.

Kinetic Energy Density Functionals Based on a Generalized Screened Coulomb Potential: Linear Response and Future Perspectives

We consider kinetic energy functionals that depend, beside the usual semilocal quantities (density, gradient, Laplacian of the density), on a generalized Yukawa potential, that is the screened Coulomb potential of the density raised to some power. These functionals, named Yukawa generalized gradient approximations (yGGA), are potentially efficient real-space semilocal methods that include significant non-local effects and can describe different important exact properties of the kinetic energy. In this work, we focus in particular on the linear response behavior for the homogeneous electron gas (HEG). We show that such functionals are able to reproduce the exact Lindhard function behavior with a very good accuracy, outperforming all other semilocal kinetic functionals. These theoretical advances allow us to perform a detailed analysis of a special class of yGGAs, namely the linear yGGA functionals. Thus, we show how the present approach can generalize the yGGA functionals improving the HEG linear behavior and leading to an extended formula for the kinetic functional. Moreover, testing on several jellium cluster model systems allows highlighting advantages and limitations of the linear yGGA functionals and future perspectives for the development of yGGA kinetic functionals.

Easy representation of multivariate functions with low-dimensional terms via Gaussian process regression kernel design: applications to machine learning of potential energy surfaces and kinetic energy densities from sparse data

We show that Gaussian process regression (GPR) allows representing multivariate functions with low-dimensional terms via kernel design. When using a kernel built with high-dimensional model representation (HDMR), one obtains a similar type of representation as the previously proposed HDMR-GPR scheme while being faster and simpler to use. We tested the approach on cases where highly accurate machine learning is required from sparse data by fitting potential energy surfaces and kinetic energy densities.

Accurate parameterization of the kinetic energy functional for calculations using exact-exchange

Electronic structure calculations based on Kohn-Sham density functional theory (KSDFT) that incorporate exact-exchange or hybrid functionals are associated with a large computational expense, a consequence of the inherent cubic scaling bottleneck and large associated prefactor, which limits the length and time scales that can be accessed. Although orbital-free density functional theory (OFDFT) calculations scale linearly with system size and are associated with a significantly smaller prefactor, they are limited by the absence of accurate density-dependent kinetic energy functionals. Therefore, the development of accurate density-dependent kinetic energy functionals is important for OFDFT calculations of large realistic systems. To this end, we propose a method to train kinetic energy functional models at the exact-exchange level of theory by using a dictionary of physically relevant terms that have been proposed in the literature in conjunction with linear or nonlinear regression methods to obtain the fitting coefficients. For our dictionary, we use a gradient expansion of the kinetic energy nonlocal models proposed in the literature and their nonlinear combinations, such as a model that incorporates spatial correlations between higher order derivatives of electron density at two points. The predictive capabilities of these models are assessed by using a variety of model one-dimensional (1D) systems that exhibit diverse bonding characteristics, such as a chain of eight hydrogens, LiF, LiH, C₄H₂, C₄N₂, and C₃O₂. We show that by using the data from model 1D KSDFT calculations performed using the exact-exchange functional for only a few neutral structures, it is possible to generate models with high accuracy for charged systems and electron and kinetic energy densities during self-consistent field iterations. In addition, we show that it is possible to learn both the orbital dependent terms, i.e., the kinetic energy and the exact-exchange energy, and models that incorporate additional nonlinearities in spatial correlations, such as a quadratic model, are needed to capture subtle features of the kinetic energy density that are present in exact-exchange-based KSDFT calculations.

Accurate parameterization of the kinetic energy functional

The absence of a reliable formulation of the kinetic energy density functional has hindered the development of orbital free density functional theory. Using the data-aided learning paradigm, we propose a simple prescription to accurately model the kinetic energy density of any system. Our method relies on a dictionary of functional forms for local and nonlocal contributions, which have been proposed in the literature, and the appropriate coefficients are calculated via a linear regression framework. To model the nonlocal contributions, we explore two new nonlocal functionals-a functional that captures fluctuations in electronic density and a functional that incorporates gradient information. Since the analytical functional forms of the kernels present in these nonlocal terms are not known from theory, we propose a basis function expansion to model these seemingly difficult nonlocal quantities. This allows us to easily reconstruct kernels for any system using only a few structures. The proposed method is able to learn kinetic energy densities and total kinetic energies of molecular and periodic systems, such as H₂, LiH, LiF, and a one-dimensional chain of eight hydrogens using data from Kohn-Sham density functional theory calculations for only a few structures.

Perspective on integrating machine learning into computational chemistry and materials science

Machine learning (ML) methods are being used in almost every conceivable area of electronic structure theory and molecular simulation. In particular, ML has become firmly established in the construction of high-dimensional interatomic potentials. Not a day goes by without another proof of principle being published on how ML methods can represent and predict quantum mechanical properties-be they observable, such as molecular polarizabilities, or not, such as atomic charges. As ML is becoming pervasive in electronic structure theory and molecular simulation, we provide an overview of how atomistic computational modeling is being transformed by the incorporation of ML approaches. From the perspective of the practitioner in the field, we assess how common workflows to predict structure, dynamics, and spectroscopy are affected by ML. Finally, we discuss how a tighter and lasting integration of ML methods with computational chemistry and materials science can be achieved and what it will mean for research practice, software development, and postgraduate training.

Learning to Approximate Density Functionals

Density functional theory (DFT) calculations are used in over 40,000 scientific papers each year, in chemistry, materials science, and far beyond. DFT is extremely useful because it is computationally much less expensive than ab initio electronic structure methods and allows systems of considerably larger size to be treated. However, the accuracy of any Kohn-Sham DFT calculation is limited by the approximation chosen for the exchange-correlation (XC) energy. For more than half a century, humans have developed the art of such approximations, using general principles, empirical data, or a combination of both, typically yielding useful results, but with errors well above the chemical accuracy limit (1 kcal/mol). Over the last 15 years, machine learning (ML) has made major breakthroughs in many applications and is now being applied to electronic structure calculations. This recent rise of ML begs the question: Can ML propose or improve density functional approximations? Success could greatly enhance the accuracy and usefulness of DFT calculations without increasing the cost.In this work, we detail efforts in this direction, beginning with an elementary proof of principle from 2012, namely, finding the kinetic energy of several Fermions in a box using kernel ridge regression. This is an example of orbital-free DFT, for which a successful general-purpose scheme could make even DFT calculations run much faster. We trace the development of that work to state-of-the-art molecular dynamics simulations of resorcinol with chemical accuracy. By training on ab initio examples, one bypasses the need to find the XC functional explicitly. We also discuss how the exchange-correlation energy itself can be modeled with such methods, especially for strongly correlated materials. Finally, we show how deep neural networks with differentiable programming can be used to construct accurate density functionals from very few data points by using the Kohn-Sham equations themselves as a regularizer. All these cases show that ML can create approximations of greater accuracy than humans, and is capable of finding approximations that can deal with difficult cases such as strong correlation. However, such ML-designed functionals have not been implemented in standard codes because of one last great challenge: generalization. We discuss how effortlessly human-designed functionals can be applied to a wide range of situations, and how difficult that is for ML.

Neural Network Potential Energy Surfaces for Small Molecules and Reactions

We review progress in neural network (NN)-based methods for the construction of interatomic potentials from discrete samples (such as ab initio energies) for applications in classical and quantum dynamics including reaction dynamics and computational spectroscopy. The main focus is on methods for building molecular potential energy surfaces (PES) in internal coordinates that explicitly include all many-body contributions, even though some of the methods we review limit the degree of coupling, due either to a desire to limit computational cost or to limited data. Explicit and direct treatment of all many-body contributions is only practical for sufficiently small molecules, which are therefore our primary focus. This includes small molecules on surfaces. We consider direct, single NN PES fitting as well as more complex methods that impose structure (such as a multibody representation) on the PES function, either through the architecture of one NN or by using multiple NNs. We show how NNs are effective in building representations with low-dimensional functions including dimensionality reduction. We consider NN-based approaches to build PESs in the sums-of-product form important for quantum dynamics, ways to treat symmetry, and issues related to sampling data distributions and the relation between PES errors and errors in observables. We highlight combinations of NNs with other ideas such as permutationally invariant polynomials or sums of environment-dependent atomic contributions, which have recently emerged as powerful tools for building highly accurate PESs for relatively large molecular and reactive systems.

Deep Learning the Hohenberg-Kohn Maps of Density Functional Theory

A striking consequence of the Hohenberg-Kohn theorem of density functional theory is the existence of a bijection between the local density and the ground-state many-body wave function. Here we study the problem of constructing approximations to the Hohenberg-Kohn map using a statistical learning approach. Using supervised deep learning with synthetic data, we show that this map can be accurately constructed for a chain of one-dimensional interacting spinless fermions in different phases of this model including the charge ordered Mott insulator and metallic phases and the critical point separating them. However, we also find that the learning is less effective across quantum phase transitions, suggesting an intrinsic difficulty in efficiently learning nonsmooth functional relations. We further study the problem of directly reconstructing complex observables from simple local density measurements, proposing a scheme amenable to statistical learning from experimental data.

Quantum Chemistry in the Age of Machine Learning

As the quantum chemistry (QC) community embraces machine learning (ML), the number of new methods and applications based on the combination of QC and ML is surging. In this Perspective, a view of the current state of affairs in this new and exciting research field is offered, challenges of using machine learning in quantum chemistry applications are described, and potential future developments are outlined. Specifically, examples of how machine learning is used to improve the accuracy and accelerate quantum chemical research are shown. Generalization and classification of existing techniques are provided to ease the navigation in the sea of literature and to guide researchers entering the field. The emphasis of this Perspective is on supervised machine learning.

The Role of the Reduced Laplacian Renormalization in the Kinetic Energy Functional Development

The Laplacian of the electronic density diverges at the nuclear cusp, which complicates the development of Laplacian-level meta-GGA (LLMGGA) kinetic energy functionals for all-electron calculations. Here, we investigate some Laplacian renormalization methods, which avoid this divergence. We developed two different LLMGGA functionals, which improve the kinetic energy or the kinetic potential. We test these KE functionals in the context of Frozen-Density-Embedding (FDE), for a large palette of non-covalently interacting molecular systems. These functionals improve over the present state-of-the-art LLMGGA functionals for the FDE calculations.