Data-driven kinetic energy density fitting for orbital-free DFT: Linear vs Gaussian process regression

Semilocal Kinetic Energy Density Functionals on Atoms and Diatoms

An accurate semilocal kinetic energy density functional (KEDF) is crucial for reliable orbital-free density functional theory calculations. In our study, we assessed the performance of representative semilocal KEDFs using a more stringent indicator. Our findings highlight the superiority of the Perdew-Constantin (PC) functional in delivering energies close to the reference values. Upon analysis of the PC functional, we identified that enhancing its performance can be achieved through a more effective region selection regime. Experimenting with various region selection indicators, we discovered that the Laplacian-dependent reduced density gradient proves to be helpful. Subsequently, we empirically constructed an augmented variant of the PC functional, which not only yields energies close to the references but also, more importantly, demonstrates qualitative predictions for stable molecules and provides reasonable quantitative estimates for bond lengths in diatomic systems.

Degeneration of kernel regression with Matern kernels into low-order polynomial regression in high dimension

Kernel methods such as kernel ridge regression and Gaussian process regression with Matern-type kernels have been increasingly used, in particular, to fit potential energy surfaces (PES) and density functionals, and for materials informatics. When the dimensionality of the feature space is high, these methods are used with necessarily sparse data. In this regime, the optimal length parameter of a Matern-type kernel may become so large that the method effectively degenerates into a low-order polynomial regression and, therefore, loses any advantage over such regression. This is demonstrated theoretically as well as numerically in the examples of six- and fifteen-dimensional molecular PES using squared exponential and simple exponential kernels. The results shed additional light on the success of polynomial approximations such as PIP for medium-size molecules and on the importance of orders-of-coupling-based models for preserving the advantages of kernel methods with Matern-type kernels of on the use of physically motivated (reproducing) kernels.

Machine learning of kinetic energy densities with target and feature smoothing: Better results with fewer training data

Machine learning (ML) of kinetic energy functionals (KEFs), in particular kinetic energy density (KED) functionals, is a promising way to construct KEFs for orbital-free density functional theory (DFT). Neural networks and kernel methods including Gaussian process regression (GPR) have been used to learn Kohn-Sham (KS) KED from density-based descriptors derived from KS DFT calculations. The descriptors are typically expressed as functions of different powers and derivatives of the electron density. This can generate large and extremely unevenly distributed datasets, which complicates effective application of ML techniques. Very uneven data distributions require many training datapoints, can cause overfitting, and can ultimately lower the quality of an ML KED model. We show that one can produce more accurate ML models from fewer data by working with smoothed density-dependent variables and KED. Smoothing palliates the issue of very uneven data distributions and associated difficulties of sampling while retaining enough spatial structure necessary for working within the paradigm of KEDF. We use GPR as a function of smoothed terms of the fourth order gradient expansion and KS effective potential and obtain accurate and stable (with respect to different random choices of training points) kinetic energy models for Al, Mg, and Si simultaneously from as few as 2000 samples (about 0.3% of the total KS DFT data). In particular, accuracies on the order of 1% in a measure of the quality of energy-volume dependence B'=EV0-ΔV-2EV0+E(V0+ΔV)ΔV/V02 (where V0 is the equilibrium volume and ΔV is a deviation from it) are obtained simultaneously for all three materials.

Orbital-Free Density Functional Theory: An Attractive Electronic Structure Method for Large-Scale First-Principles Simulations

Kohn-Sham Density Functional Theory (KSDFT) is the most widely used electronic structure method in chemistry, physics, and materials science, with thousands of calculations cited annually. This ubiquity is rooted in the favorable accuracy vs cost balance of KSDFT. Nonetheless, the ambitions and expectations of researchers for use of KSDFT in predictive simulations of large, complicated molecular systems are confronted with an intrinsic computational cost-scaling challenge. Particularly evident in the context of first-principles molecular dynamics, the challenge is the high cost-scaling associated with the computation of the Kohn-Sham orbitals. Orbital-free DFT (OFDFT), as the name suggests, circumvents entirely the explicit use of those orbitals. Without them, the structural and algorithmic complexity of KSDFT simplifies dramatically and near-linear scaling with system size irrespective of system state is achievable. Thus, much larger system sizes and longer simulation time scales (compared to conventional KSDFT) become accessible; hence, new chemical phenomena and new materials can be explored. In this review, we introduce the historical contexts of OFDFT, its theoretical basis, and the challenge of realizing its promise via approximate kinetic energy density functionals (KEDFs). We review recent progress on that challenge for an array of KEDFs, such as one-point, two-point, and machine-learnt, as well as some less explored forms. We emphasize use of exact constraints and the inevitability of design choices. Then, we survey the associated numerical techniques and implemented algorithms specific to OFDFT. We conclude with an illustrative sample of applications to showcase the power of OFDFT in materials science, chemistry, and physics.

Tools for overcoming reliance on energy-based measures in chemistry: a tutorial review

The vast majority of literature in the chemical sciences describes fundamental chemical and physical phenomena using scalar measures, such as the energy, even though many phenomena are beyond the scope of scalar-based considerations. This problem exists no matter how accurately the associated energies are calculated. The solution that is explained in this work is to remove the reliance on scalar quantum chemical measures and instead utilize the vector-based and full symmetry-breaking nature of next generation quantum theory of atoms in molecules (NG-QTAIM). The connection with experiment on neutral chiral molecules is explained. A selection of non-energy-based explanations are provided: the functioning of molecular devices, why the cis-effect is the exception rather than the rule, stereochemical phenomena including chiral discrimination, quantifying chiral character of formally achiral molecules, mixed S and R stereoisomer character and the effect of an applied electric field. Current and future developments along with suggestions for future avenues of investigation are discussed. This tutorial review provides the practical details required to implement NG-QTAIM for a range of phenomena that are not accessible with energy-based measures. Step-by-step worked examples are included with data sets and instructions for use of commercial and open-source software along with examples of how to interpret the results.

Hybrid Density Functional Tight Binding (DFTB)─Molecular Mechanics Approach for a Low-Cost Expansion of DFTB Applicability

The density functional-based tight binding (DFTB) method has seen a rise in adoption for materials modeling, as it offers significant improvement in scalability with accuracy comparable to the density functional theory (DFT) when good parameterizations exist. The cost reduction in DFTB compared to DFT is achieved by the pre-parameterization of the elements of the Hamiltonian matrix as well as the repulsion potential between all pairs of atoms. Parameterization for new systems with accuracies competitive with DFT in specific applications requires specialized manpower and computational resources. This prevents the application of the DFTB method to systems for which it was not parameterized. In this work, we explore an approach to address the problem of missing parameters of DFTB by modeling the interactions with missing Slater-Koster parameters with an interatomic interaction potential. When the distance between two atoms modeled at the force-field level is sufficiently large, the approach results in accurate structural and electronic properties. The resulting calculation is therefore a hybrid between DFTB and molecular mechanics, a pure DFTB for atoms with a complete set of interatomic parameterizations, and a mix between DFTB and molecular mechanics for atoms with a missing interatomic parameterization. The approach is expected to be particularly useful for hybrid materials and interfaces. The method is tested on the examples of 2D materials, mixed oxides, and a large-scale calculation of an oxide-oxide interface.

Automatic differentiation for orbital-free density functional theory

Differentiable programming has facilitated numerous methodological advances in scientific computing. Physics engines supporting automatic differentiation have simpler code, accelerating the development process and reducing the maintenance burden. Furthermore, fully differentiable simulation tools enable direct evaluation of challenging derivatives-including those directly related to properties measurable by experiment-that are conventionally computed with finite difference methods. Here, we investigate automatic differentiation in the context of orbital-free density functional theory (OFDFT) simulations of materials, introducing PROFESS-AD. Its automatic evaluation of properties derived from first derivatives, including functional potentials, forces, and stresses, facilitates the development and testing of new density functionals, while its direct evaluation of properties requiring higher-order derivatives, such as bulk moduli, elastic constants, and force constants, offers more concise implementations than conventional finite difference methods. For these reasons, PROFESS-AD serves as an excellent prototyping tool and provides new opportunities for OFDFT.

The loss of the property of locality of the kernel in high-dimensional Gaussian process regression on the example of the fitting of molecular potential energy surfaces

Kernel-based methods, including Gaussian process regression (GPR) and generally kernel ridge regression, have been finding increasing use in computational chemistry, including the fitting of potential energy surfaces and density functionals in high-dimensional feature spaces. Kernels of the Matern family, such as Gaussian-like kernels (basis functions), are often used which allow imparting to them the meaning of covariance functions and formulating GPR as an estimator of the mean of a Gaussian distribution. The notion of locality of the kernel is critical for this interpretation. It is also critical to the formulation of multi-zeta type basis functions widely used in computational chemistry. We show, on the example of fitting of molecular potential energy surfaces of increasing dimensionality, the practical disappearance of the property of locality of a Gaussian-like kernel in high dimensionality. We also formulate a multi-zeta approach to the kernel and show that it significantly improves the quality of regression in low dimensionality but loses any advantage in high dimensionality, which is attributed to the loss of the property of locality.

Machine learning in computational chemistry: interplay between (non)linearity, basis sets, and dimensionality

Machine learning (ML) based methods and tools have now firmly established themselves in physical chemistry and in particular in theoretical and computational chemistry and in materials chemistry. The generality of popular ML techniques such as neural networks or kernel methods (Gaussian process and kernel ridge regression and their flavors) permitted their application to diverse problems from prediction of properties of functional materials (catalysts, solid state ionic conductors, etc.) from descriptors to the building of interatomic potentials (where ML is currently routinely used in applications) and electron density functionals. These ML techniques are assumed to have superior expressive power of nonlinear methods, and are often used "as is", with concepts such as "non-parametric" or "deep learning" used without a clear justification for their need or advantage over simpler and more robust alternatives. In this Perspective, we highlight some interrelations between popular ML techniques and traditional linear regressions and basis expansions and demonstrate that in certain regimes (such as a very high dimensionality) these approximations might collapse. We also discuss ways to recover the expressive power of a nonlinear approach and to help select hyperparameters with the help of high-dimensional model representation and to obtain elements of insight while preserving the generality of the method.

Kinetic Energy Density Functionals Based on a Generalized Screened Coulomb Potential: Linear Response and Future Perspectives

We consider kinetic energy functionals that depend, beside the usual semilocal quantities (density, gradient, Laplacian of the density), on a generalized Yukawa potential, that is the screened Coulomb potential of the density raised to some power. These functionals, named Yukawa generalized gradient approximations (yGGA), are potentially efficient real-space semilocal methods that include significant non-local effects and can describe different important exact properties of the kinetic energy. In this work, we focus in particular on the linear response behavior for the homogeneous electron gas (HEG). We show that such functionals are able to reproduce the exact Lindhard function behavior with a very good accuracy, outperforming all other semilocal kinetic functionals. These theoretical advances allow us to perform a detailed analysis of a special class of yGGAs, namely the linear yGGA functionals. Thus, we show how the present approach can generalize the yGGA functionals improving the HEG linear behavior and leading to an extended formula for the kinetic functional. Moreover, testing on several jellium cluster model systems allows highlighting advantages and limitations of the linear yGGA functionals and future perspectives for the development of yGGA kinetic functionals.

Easy representation of multivariate functions with low-dimensional terms via Gaussian process regression kernel design: applications to machine learning of potential energy surfaces and kinetic energy densities from sparse data

We show that Gaussian process regression (GPR) allows representing multivariate functions with low-dimensional terms via kernel design. When using a kernel built with high-dimensional model representation (HDMR), one obtains a similar type of representation as the previously proposed HDMR-GPR scheme while being faster and simpler to use. We tested the approach on cases where highly accurate machine learning is required from sparse data by fitting potential energy surfaces and kinetic energy densities.

Accurate parameterization of the kinetic energy functional for calculations using exact-exchange

Electronic structure calculations based on Kohn-Sham density functional theory (KSDFT) that incorporate exact-exchange or hybrid functionals are associated with a large computational expense, a consequence of the inherent cubic scaling bottleneck and large associated prefactor, which limits the length and time scales that can be accessed. Although orbital-free density functional theory (OFDFT) calculations scale linearly with system size and are associated with a significantly smaller prefactor, they are limited by the absence of accurate density-dependent kinetic energy functionals. Therefore, the development of accurate density-dependent kinetic energy functionals is important for OFDFT calculations of large realistic systems. To this end, we propose a method to train kinetic energy functional models at the exact-exchange level of theory by using a dictionary of physically relevant terms that have been proposed in the literature in conjunction with linear or nonlinear regression methods to obtain the fitting coefficients. For our dictionary, we use a gradient expansion of the kinetic energy nonlocal models proposed in the literature and their nonlinear combinations, such as a model that incorporates spatial correlations between higher order derivatives of electron density at two points. The predictive capabilities of these models are assessed by using a variety of model one-dimensional (1D) systems that exhibit diverse bonding characteristics, such as a chain of eight hydrogens, LiF, LiH, C₄H₂, C₄N₂, and C₃O₂. We show that by using the data from model 1D KSDFT calculations performed using the exact-exchange functional for only a few neutral structures, it is possible to generate models with high accuracy for charged systems and electron and kinetic energy densities during self-consistent field iterations. In addition, we show that it is possible to learn both the orbital dependent terms, i.e., the kinetic energy and the exact-exchange energy, and models that incorporate additional nonlinearities in spatial correlations, such as a quadratic model, are needed to capture subtle features of the kinetic energy density that are present in exact-exchange-based KSDFT calculations.

Accurate parameterization of the kinetic energy functional

The absence of a reliable formulation of the kinetic energy density functional has hindered the development of orbital free density functional theory. Using the data-aided learning paradigm, we propose a simple prescription to accurately model the kinetic energy density of any system. Our method relies on a dictionary of functional forms for local and nonlocal contributions, which have been proposed in the literature, and the appropriate coefficients are calculated via a linear regression framework. To model the nonlocal contributions, we explore two new nonlocal functionals-a functional that captures fluctuations in electronic density and a functional that incorporates gradient information. Since the analytical functional forms of the kernels present in these nonlocal terms are not known from theory, we propose a basis function expansion to model these seemingly difficult nonlocal quantities. This allows us to easily reconstruct kernels for any system using only a few structures. The proposed method is able to learn kinetic energy densities and total kinetic energies of molecular and periodic systems, such as H₂, LiH, LiF, and a one-dimensional chain of eight hydrogens using data from Kohn-Sham density functional theory calculations for only a few structures.

Machine learning meets chemical physics

Over recent years, the use of statistical learning techniques applied to chemical problems has gained substantial momentum. This is particularly apparent in the realm of physical chemistry, where the balance between empiricism and physics-based theory has traditionally been rather in favor of the latter. In this guest Editorial for the special topic issue on "Machine Learning Meets Chemical Physics," a brief rationale is provided, followed by an overview of the topics covered. We conclude by making some general remarks.