Kohn-Sham Equations as Regularizer: Building Prior Knowledge into Machine-Learned Physics

dxtb-An efficient and fully differentiable framework for extended tight-binding

Automatic differentiation (AD) emerged as an integral part of machine learning, accelerating model development by enabling gradient-based optimization without explicit analytical derivatives. Recently, the benefits of AD and computing arbitrary-order derivatives with respect to any variable were also recognized in the field of quantum chemistry. In this work, we present dxtb-an open-source, fully differentiable framework for semiempirical extended tight-binding (xTB) methods. Developed entirely in Python and leveraging PyTorch for array operations, dxtb facilitates extensibility and rapid prototyping while maintaining computational efficiency. Through comprehensive code vectorization and optimization, we essentially reach the speed of compiled xTB programs for high-throughput calculations of small molecules. The excellent performance also scales to large systems, and batch operability yields additional benefits for execution on parallel hardware. In particular, energy evaluations are on par with existing programs, whereas the speed of automatically differentiated nuclear derivatives is only 2 to 5 times slower compared to their analytical counterparts. We showcase the utility of AD in dxtb by calculating various molecular and spectroscopic properties, highlighting its capacity to enhance and simplify such evaluations. Furthermore, the framework streamlines optimization tasks and offers seamless integration of semiempirical quantum chemistry in machine learning, paving the way for physics-inspired end-to-end differentiable models. Ultimately, dxtb aims to further advance the capabilities of semiempirical methods, providing an extensible foundation for future developments and hybrid machine learning applications. The framework is accessible at https://github.com/grimme-lab/dxtb.

Quantum chemical package Jaguar: A survey of recent developments and unique features

This paper is dedicated to the quantum chemical package Jaguar, which is commercial software developed and distributed by Schrödinger, Inc. We discuss Jaguar's scientific features that are relevant to chemical research as well as describe those aspects of the program that are pertinent to the user interface, the organization of the computer code, and its maintenance and testing. Among the scientific topics that feature prominently in this paper are the quantum chemical methods grounded in the pseudospectral approach. A number of multistep workflows dependent on Jaguar are covered: prediction of protonation equilibria in aqueous solutions (particularly calculations of tautomeric stability and pKa), reactivity predictions based on automated transition state search, assembly of Boltzmann-averaged spectra such as vibrational and electronic circular dichroism, as well as nuclear magnetic resonance. Discussed also are quantum chemical calculations that are oriented toward materials science applications, in particular, prediction of properties of optoelectronic materials and organic semiconductors, and molecular catalyst design. The topic of treatment of conformations inevitably comes up in real world research projects and is considered as part of all the workflows mentioned above. In addition, we examine the role of machine learning methods in quantum chemical calculations performed by Jaguar, from auxiliary functions that return the approximate calculation runtime in a user interface, to prediction of actual molecular properties. The current work is second in a series of reviews of Jaguar, the first having been published more than ten years ago. Thus, this paper serves as a rare milestone on the path that is being traversed by Jaguar's development in more than thirty years of its existence.

Machine learning for accuracy in density functional approximations

Machine learning techniques have found their way into computational chemistry as indispensable tools to accelerate atomistic simulations and materials design. In addition, machine learning approaches hold the potential to boost the predictive power of computationally efficient electronic structure methods, such as density functional theory, to chemical accuracy and to correct for fundamental errors in density functional approaches. Here, recent progress in applying machine learning to improve the accuracy of density functional and related approximations is reviewed. Promises and challenges in devising machine learning models transferable between different chemistries and materials classes are discussed with the help of examples applying promising models to systems far outside their training sets.

Identifying and embedding transferability in data-driven representations of chemical space

Transferability, especially in the context of model generalization, is a paradigm of all scientific disciplines. However, the rapid advancement of machine learned model development threatens this paradigm, as it can be difficult to understand how transferability is embedded (or missed) in complex models developed using large training data sets. Two related open problems are how to identify, without relying on human intuition, what makes training data transferable; and how to embed transferability into training data. To solve both problems for ab initio chemical modelling, an indispensable tool in everyday chemistry research, we introduce a transferability assessment tool (TAT) and demonstrate it on a controllable data-driven model for developing density functional approximations (DFAs). We reveal that human intuition in the curation of training data introduces chemical biases that can hamper the transferability of data-driven DFAs. We use our TAT to motivate three transferability principles; one of which introduces the key concept of transferable diversity. Finally, we propose data curation strategies for general-purpose machine learning models in chemistry that identify and embed the transferability principles.

In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back

Computational chemistry is an indispensable tool for understanding molecules and predicting chemical properties. However, traditional computational methods face significant challenges due to the difficulty of solving the Schrödinger equations and the increasing computational cost with the size of the molecular system. In response, there has been a surge of interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to in silico experiments. Integrating AI and ML into computational chemistry increases the scalability and speed of the exploration of chemical space. However, challenges remain, particularly regarding the reproducibility and transferability of ML models. This review highlights the evolution of ML in learning from, complementing, or replacing traditional computational chemistry for energy and property predictions. Starting from models trained entirely on numerical data, a journey set forth toward the ideal model incorporating or learning the physical laws of quantum mechanics. This paper also reviews existing computational methods and ML models and their intertwining, outlines a roadmap for future research, and identifies areas for improvement and innovation. Ultimately, the goal is to develop AI architectures capable of predicting accurate and transferable solutions to the Schrödinger equation, thereby revolutionizing in silico experiments within chemistry and materials science.

Semilocal Kinetic Energy Density Functionals on Atoms and Diatoms

An accurate semilocal kinetic energy density functional (KEDF) is crucial for reliable orbital-free density functional theory calculations. In our study, we assessed the performance of representative semilocal KEDFs using a more stringent indicator. Our findings highlight the superiority of the Perdew-Constantin (PC) functional in delivering energies close to the reference values. Upon analysis of the PC functional, we identified that enhancing its performance can be achieved through a more effective region selection regime. Experimenting with various region selection indicators, we discovered that the Laplacian-dependent reduced density gradient proves to be helpful. Subsequently, we empirically constructed an augmented variant of the PC functional, which not only yields energies close to the references but also, more importantly, demonstrates qualitative predictions for stable molecules and provides reasonable quantitative estimates for bond lengths in diatomic systems.

STEP: extraction of underlying physics with robust machine learning

A prevalent class of challenges in modern physics are inverse problems, where physical quantities must be extracted from experimental measurements. End-to-end machine learning approaches to inverse problems typically require constructing sophisticated estimators to achieve the desired accuracy, largely because they need to learn the complex underlying physical model. Here, we discuss an alternative paradigm: by making the physical model auto-differentiable we can construct a neural surrogate to represent the unknown physical quantity sought, while avoiding having to relearn the known physics entirely. We dub this process surrogate training embedded in physics (STEP) and illustrate that it generalizes well and is robust against overfitting and significant noise in the data. We demonstrate how STEP can be applied to perform dynamic kernel deconvolution to analyse resonant inelastic X-ray scattering spectra and show that surprisingly simple estimator architectures suffice to extract the relevant physical information.

Leave No Photon Behind: Artificial Intelligence in Multiscale Physics of Photocatalyst and Photoreactor Design

Although solar fuels photocatalysis offers the promise of converting carbon dioxide directly with sunlight as commercially scalable solutions have remained elusive over the past few decades, despite significant advancements in photocatalysis band-gap engineering and atomic site activity. The primary challenge lies not in the discovery of new catalyst materials, which are abundant, but in overcoming the bottlenecks related to material-photoreactor synergy. These factors include achieving photogeneration and charge-carrier recombination at reactive sites, utilizing high mass transfer efficiency supports, maximizing solar collection, and achieving uniform light distribution within a reactor. Addressing this multi-dimensional problem necessitates harnessing machine learning techniques to analyze real-world data from photoreactors and material properties. In this perspective, the challenges are outlined associated with each bottleneck factor, review relevant data analysis studies, and assess the requirements for developing a comprehensive solution that can unlock the full potential of solar fuels photocatalysis technology. Physics-informed machine learning (or Physics Neural Networks) may be the key to advancing this important area from disparate data towards optimal reactor solutions.

Deep Mind 21 functional does not extrapolate to transition metal chemistry

The development of density functional approximations stands at a crossroads: while machine-learned functionals show potential to surpass their human-designed counterparts, their extrapolation to unseen chemistry lags behind. Here we assess how well the recent Deep Mind 21 (DM21) machine-learned functional [Science, 2021, 374, 1385-1389], trained on main-group chemistry, extrapolates to transition metal chemistry (TMC). We show that DM21 demonstrates comparable or occasionally superior accuracy to B3LYP for TMC, but consistently struggles with achieving self-consistent field convergence for TMC molecules. We also compare main-group and TMC machine-learning DM21 features to shed light on DM21's challenges in TMC. We finally propose strategies to overcome limitations in the extrapolative capabilities of machine-learned functionals in TMC.

Calculating Potential Energy Surfaces with Quantum Computers by Measuring Only the Density Along Adiabatic Transitions

We show that chemically accurate potential energy surfaces (PESs) can be generated from quantum computers by measuring only the density along an adiabatic transition between different molecular geometries. In lieu of using phase estimation, the energy is evaluated by performing line-integration using the inverted real-space time-dependent density functional theory Kohn-Sham (KS) potential obtained from the geometry-varying densities of the full wave function. The accuracy of this method depends on the validity of the adiabatic evolution itself and the potential inversion process (which is theoretically exact but can be numerically unstable), whereas the total evolution time is the defining factor for the precision of phase estimation. We examine the method with a one-dimensional system of two electrons for both the ground and first triplet states in first quantization, as well as the ground state of three- and four-electron systems in second quantization. It is shown that few accurate measurements can be utilized to obtain chemical accuracy across the full potential energy curve, with a shorter propagation time than may be required using phase estimation for a similar accuracy. We also show that an accurate potential energy curve can be calculated by making many imprecise density measurements (using a few shots) along the time evolution and smoothing the resulting density evolution. Finally, it is important to note that the method is able to classically provide a check of its own accuracy by comparing the density resulting from a time-independent KS calculation using the inverted potential with the measured density. This can be used to determine whether longer adiabatic evolution times are required to satisfy the adiabatic theorem.

Why neural functionals suit statistical mechanics

We describe recent progress in the statistical mechanical description of many-body systems via machine learning combined with concepts from density functional theory and many-body simulations. We argue that the neural functional theory by Sammülleret al(2023Proc. Natl Acad. Sci.120e2312484120) gives a functional representation of direct correlations and of thermodynamics that allows for thorough quality control and consistency checking of the involved methods of artificial intelligence. Addressing a prototypical system we here present a pedagogical application to hard core particle in one spatial dimension, where Percus' exact solution for the free energy functional provides an unambiguous reference. A corresponding standalone numerical tutorial that demonstrates the neural functional concepts together with the underlying fundamentals of Monte Carlo simulations, classical density functional theory, machine learning, and differential programming is available online athttps://github.com/sfalmo/NeuralDFT-Tutorial.

Overcoming the barrier of orbital-free density functional theory for molecular systems using deep learning

Orbital-free density functional theory (OFDFT) is a quantum chemistry formulation that has a lower cost scaling than the prevailing Kohn-Sham DFT, which is increasingly desired for contemporary molecular research. However, its accuracy is limited by the kinetic energy density functional, which is notoriously hard to approximate for non-periodic molecular systems. Here we propose M-OFDFT, an OFDFT approach capable of solving molecular systems using a deep learning functional model. We build the essential non-locality into the model, which is made affordable by the concise density representation as expansion coefficients under an atomic basis. With techniques to address unconventional learning challenges therein, M-OFDFT achieves a comparable accuracy to Kohn-Sham DFT on a wide range of molecules untouched by OFDFT before. More attractively, M-OFDFT extrapolates well to molecules much larger than those seen in training, which unleashes the appealing scaling of OFDFT for studying large molecules including proteins, representing an advancement of the accuracy-efficiency trade-off frontier in quantum chemistry.

Deep-Learning Density Functional Perturbation Theory

Calculating perturbation response properties of materials from first principles provides a vital link between theory and experiment, but is bottlenecked by the high computational cost. Here, a general framework is proposed to perform density functional perturbation theory (DFPT) calculations by neural networks, greatly improving the computational efficiency. Automatic differentiation is applied on neural networks, facilitating accurate computation of derivatives. High efficiency and good accuracy of the approach are demonstrated by studying electron-phonon coupling and related physical quantities. This work brings deep-learning density functional theory and DFPT into a unified framework, creating opportunities for developing ab initio artificial intelligence.

Machine Learning of a Density Functional for Anisotropic Patchy Particles

Anisotropic patchy particles have become an archetypical statistical model system for associating fluids. Here, we formulate an approach to the Kern-Frenkel model via the classical density functional theory to describe the positionally and orientationally resolved equilibrium density distributions in flat wall geometries. The density functional is split into a reference part for the orientationally averaged density and an orientational part in mean-field approximation. To bring the orientational part into a kernel form suitable for machine learning (ML) techniques, an expansion into orientational invariants and the proper incorporation of single-particle symmetries are formulated. The mean-field kernel is constructed via ML on the basis of hard wall simulation data. The results are compared to the well-known random-phase approximation, which strongly underestimates the orientational correlations close to the wall. Successes and shortcomings of the mean-field treatment of the orientational part are highlighted and perspectives are given for attaining a full-density functional via ML.

Development of a machine learning finite-range nonlocal density functional

Kohn-Sham density functional theory has been the most popular method in electronic structure calculations. To fulfill the increasing accuracy requirements, new approximate functionals are needed to address key issues in existing approximations. It is well known that nonlocal components are crucial. Current nonlocal functionals mostly require orbital dependence such as in Hartree-Fock exchange and many-body perturbation correlation energy, which, however, leads to higher computational costs. Deviating from this pathway, we describe functional nonlocality in a new approach. By partitioning the total density to atom-centered local densities, a many-body expansion is proposed. This many-body expansion can be truncated at one-body contributions, if a base functional is used and an energy correction is approximated. The contribution from each atom-centered local density is a single finite-range nonlocal functional that is universal for all atoms. We then use machine learning to develop this universal atom-centered functional. Parameters in this functional are determined by fitting to data that are produced by high-level theories. Extensive tests on several different test sets, which include reaction energies, reaction barrier heights, and non-covalent interaction energies, show that the new functional, with only the density as the basic variable, can produce results comparable to the best-performing double-hybrid functionals, (for example, for the thermochemistry test set selected from the GMTKN55 database, BLYP based machine learning functional gives a weighted total mean absolute deviations of 3.33 kcal/mol, while DSD-BLYP-D3(BJ) gives 3.28 kcal/mol) with a lower computational cost. This opens a new pathway to nonlocal functional development and applications.

Neural functional theory for inhomogeneous fluids: Fundamentals and applications

We present a hybrid scheme based on classical density functional theory and machine learning for determining the equilibrium structure and thermodynamics of inhomogeneous fluids. The exact functional map from the density profile to the one-body direct correlation function is represented locally by a deep neural network. We substantiate the general framework for the hard sphere fluid and use grand canonical Monte Carlo simulation data of systems in randomized external environments during training and as reference. Functional calculus is implemented on the basis of the neural network to access higher-order correlation functions via automatic differentiation and the free energy via functional line integration. Thermal Noether sum rules are validated explicitly. We demonstrate the use of the neural functional in the self-consistent calculation of density profiles. The results outperform those from state-of-the-art fundamental measure density functional theory. The low cost of solving an associated Euler-Lagrange equation allows to bridge the gap from the system size of the original training data to macroscopic predictions upon maintaining near-simulation microscopic precision. These results establish the machine learning of functionals as an effective tool in the multiscale description of soft matter.

Variational principle to regularize machine-learned density functionals: The non-interacting kinetic-energy functional

Practical density functional theory (DFT) owes its success to the groundbreaking work of Kohn and Sham that introduced the exact calculation of the non-interacting kinetic energy of the electrons using an auxiliary mean-field system. However, the full power of DFT will not be unleashed until the exact relationship between the electron density and the non-interacting kinetic energy is found. Various attempts have been made to approximate this functional, similar to the exchange-correlation functional, with much less success due to the larger contribution of kinetic energy and its more non-local nature. In this work, we propose a new and efficient regularization method to train density functionals based on deep neural networks, with particular interest in the kinetic-energy functional. The method is tested on (effectively) one-dimensional systems, including the hydrogen chain, non-interacting electrons, and atoms of the first two periods, with excellent results. For atomic systems, the generalizability of the regularization method is demonstrated by training also an exchange-correlation functional, and the contrasting nature of the two functionals is discussed from a machine-learning perspective.

Artificial Neural Network-Based Density Functional Approach for Adiabatic Energy Differences in Transition Metal Complexes

During the past decades, approximate Kohn-Sham density functional theory schemes have garnered many successes in computational chemistry and physics, yet the performance in the prediction of spin state energetics is often unsatisfactory. By means of a machine learning approach, an enhanced exchange and correlation functional is developed to describe adiabatic energy differences in transition metal complexes. The functional is based on the computationally efficient revision of the regularized, strongly constrained, and appropriately normed functional and improved by an artificial neural network correction trained over a small data set of electronic densities, atomization energies, and/or spin state energetics. The training process, performed using a bioinspired nongradient-based approach adapted for this work from the particle swarm optimization, is analyzed and discussed extensively. The resulting machine learned meta-generalized gradient approximation functional is shown to outperform most known density functionals in the prediction of adiabatic energy differences for a diverse set of transition metal complexes with varying local coordinations and metal choices.

Orbital-Free Density Functional Theory: An Attractive Electronic Structure Method for Large-Scale First-Principles Simulations

Kohn-Sham Density Functional Theory (KSDFT) is the most widely used electronic structure method in chemistry, physics, and materials science, with thousands of calculations cited annually. This ubiquity is rooted in the favorable accuracy vs cost balance of KSDFT. Nonetheless, the ambitions and expectations of researchers for use of KSDFT in predictive simulations of large, complicated molecular systems are confronted with an intrinsic computational cost-scaling challenge. Particularly evident in the context of first-principles molecular dynamics, the challenge is the high cost-scaling associated with the computation of the Kohn-Sham orbitals. Orbital-free DFT (OFDFT), as the name suggests, circumvents entirely the explicit use of those orbitals. Without them, the structural and algorithmic complexity of KSDFT simplifies dramatically and near-linear scaling with system size irrespective of system state is achievable. Thus, much larger system sizes and longer simulation time scales (compared to conventional KSDFT) become accessible; hence, new chemical phenomena and new materials can be explored. In this review, we introduce the historical contexts of OFDFT, its theoretical basis, and the challenge of realizing its promise via approximate kinetic energy density functionals (KEDFs). We review recent progress on that challenge for an array of KEDFs, such as one-point, two-point, and machine-learnt, as well as some less explored forms. We emphasize use of exact constraints and the inevitability of design choices. Then, we survey the associated numerical techniques and implemented algorithms specific to OFDFT. We conclude with an illustrative sample of applications to showcase the power of OFDFT in materials science, chemistry, and physics.

A Perspective on Sustainable Computational Chemistry Software Development and Integration

The power of quantum chemistry to predict the ground and excited state properties of complex chemical systems has driven the development of computational quantum chemistry software, integrating advances in theory, applied mathematics, and computer science. The emergence of new computational paradigms associated with exascale technologies also poses significant challenges that require a flexible forward strategy to take full advantage of existing and forthcoming computational resources. In this context, the sustainability and interoperability of computational chemistry software development are among the most pressing issues. In this perspective, we discuss software infrastructure needs and investments with an eye to fully utilize exascale resources and provide unique computational tools for next-generation science problems and scientific discoveries.

Nonlocal Functionals Inspired by the Strongly Interacting Limit of DFT: Exact Constraints and Implementation

Capturing strong correlation effects remains a key challenge for the development of improved exchange-correlation (XC) functionals in density functional theory. The recently proposed multiple radii functional (MRF) [J. Phys. Chem. Lett. 2017, 8, 2799; J. Chem. Theory Comput. 2019, 15, 3580] was designed to capture strong correlation effects seamlessly, as its mathematical structure draws from that of the exact XC functional in the limit of infinite correlations. The MRF functional provides a framework for building approximations along the density-fixed adiabatic connection, delivers accurate XC energy densities in the standard DFT gauge (same as that of the exact exchange energy density), and is free of one-electron self-interaction errors. To facilitate the development of XC functionals based on the MRF, we examine the behavior of the MRF functional when applied to uniform and scaled densities and consider how it can be made exact for the uniform electron gas. These theoretical insights are then used to build improved forms for the fluctuation function, an object that defines XC energy densities within the MRF framework. We also show how the MRF fluctuation function for physical correlation can be easily readjusted to accurately capture the XC functional in the limit of infinite correlations, demonstrating the versatility of MRF for building approximations for different correlation regimes. We describe the implementation of MRF using densities expanded on Gaussian basis sets, which improves the efficiency of previous grid-based MRF implementations.

GS-DeepNet: mastering tokamak plasma equilibria with deep neural networks and the Grad-Shafranov equation

The force-balanced state of magnetically confined plasmas heated up to 100 million degrees Celsius must be sustained long enough to achieve a burning-plasma state, such as in the case of ITER, a fusion reactor that promises a net energy gain. This force balance between the Lorentz force and the pressure gradient force, known as a plasma equilibrium, can be theoretically portrayed together with Maxwell's equations as plasmas are collections of charged particles. Nevertheless, identifying the plasma equilibrium in real time is challenging owing to its free-boundary and ill-posed conditions, which conventionally involves iterative numerical approach with a certain degree of subjective human decisions such as including or excluding certain magnetic measurements to achieve numerical convergence on the solution as well as to avoid unphysical solutions. Here, we introduce GS-DeepNet, which learns plasma equilibria through solely unsupervised learning, without using traditional numerical algorithms. GS-DeepNet includes two neural networks and teaches itself. One neural network generates a possible candidate of an equilibrium following Maxwell's equations and is taught by the other network satisfying the force balance under the equilibrium. Measurements constrain both networks. Our GS-DeepNet achieves reliable equilibria with uncertainties in contrast with existing methods, leading to possible better control of fusion-grade plasmas.

Open-Source Machine Learning in Computational Chemistry

The field of computational chemistry has seen a significant increase in the integration of machine learning concepts and algorithms. In this Perspective, we surveyed 179 open-source software projects, with corresponding peer-reviewed papers published within the last 5 years, to better understand the topics within the field being investigated by machine learning approaches. For each project, we provide a short description, the link to the code, the accompanying license type, and whether the training data and resulting models are made publicly available. Based on those deposited in GitHub repositories, the most popular employed Python libraries are identified. We hope that this survey will serve as a resource to learn about machine learning or specific architectures thereof by identifying accessible codes with accompanying papers on a topic basis. To this end, we also include computational chemistry open-source software for generating training data and fundamental Python libraries for machine learning. Based on our observations and considering the three pillars of collaborative machine learning work, open data, open source (code), and open models, we provide some suggestions to the community.

SchNetPack 2.0: A neural network toolbox for atomistic machine learning

SchNetPack is a versatile neural network toolbox that addresses both the requirements of method development and the application of atomistic machine learning. Version 2.0 comes with an improved data pipeline, modules for equivariant neural networks, and a PyTorch implementation of molecular dynamics. An optional integration with PyTorch Lightning and the Hydra configuration framework powers a flexible command-line interface. This makes SchNetPack 2.0 easily extendable with a custom code and ready for complex training tasks, such as the generation of 3D molecular structures.

Automatic differentiation for orbital-free density functional theory

Differentiable programming has facilitated numerous methodological advances in scientific computing. Physics engines supporting automatic differentiation have simpler code, accelerating the development process and reducing the maintenance burden. Furthermore, fully differentiable simulation tools enable direct evaluation of challenging derivatives-including those directly related to properties measurable by experiment-that are conventionally computed with finite difference methods. Here, we investigate automatic differentiation in the context of orbital-free density functional theory (OFDFT) simulations of materials, introducing PROFESS-AD. Its automatic evaluation of properties derived from first derivatives, including functional potentials, forces, and stresses, facilitates the development and testing of new density functionals, while its direct evaluation of properties requiring higher-order derivatives, such as bulk moduli, elastic constants, and force constants, offers more concise implementations than conventional finite difference methods. For these reasons, PROFESS-AD serves as an excellent prototyping tool and provides new opportunities for OFDFT.

Insights into the deviation from piecewise linearity in transition metal complexes from supervised machine learning models

Virtual high-throughput screening (VHTS) and machine learning (ML) with density functional theory (DFT) suffer from inaccuracies from the underlying density functional approximation (DFA). Many of these inaccuracies can be traced to the lack of derivative discontinuity that leads to a curvature in the energy with electron addition or removal. Over a dataset of nearly one thousand transition metal complexes typical of VHTS applications, we computed and analyzed the average curvature (i.e., deviation from piecewise linearity) for 23 density functional approximations spanning multiple rungs of "Jacob's ladder". While we observe the expected dependence of the curvatures on Hartree-Fock exchange, we note limited correlation of curvature values between different rungs of "Jacob's ladder". We train ML models (i.e., artificial neural networks or ANNs) to predict the curvature and the associated frontier orbital energies for each of these 23 functionals and then interpret differences in curvature among the different DFAs through analysis of the ML models. Notably, we observe spin to play a much more important role in determining the curvature of range-separated and double hybrids in comparison to semi-local functionals, explaining why curvature values are weakly correlated between these and other families of functionals. Over a space of 187.2k hypothetical compounds, we use our ANNs to pinpoint DFAs for which representative transition metal complexes have near-zero curvature with low uncertainty, demonstrating an approach to accelerate screening of complexes with targeted optical gaps.

Inverse molecular design and parameter optimization with Hückel theory using automatic differentiation

Semiempirical quantum chemistry has recently seen a renaissance with applications in high-throughput virtual screening and machine learning. The simplest semiempirical model still in widespread use in chemistry is Hückel's π-electron molecular orbital theory. In this work, we implemented a Hückel program using differentiable programming with the JAX framework based on limited modifications of a pre-existing NumPy version. The auto-differentiable Hückel code enabled efficient gradient-based optimization of model parameters tuned for excitation energies and molecular polarizabilities, respectively, based on as few as 100 data points from density functional theory simulations. In particular, the facile computation of the polarizability, a second-order derivative, via auto-differentiation shows the potential of differentiable programming to bypass the need for numeric differentiation or derivation of analytical expressions. Finally, we employ gradient-based optimization of atom identity for inverse design of organic electronic materials with targeted orbital energy gaps and polarizabilities. Optimized structures are obtained after as little as 15 iterations using standard gradient-based optimization algorithms.

Physics Perception in Sloshing Scenes With Guaranteed Thermodynamic Consistency

Physics perception very often faces the problem that only limited data or partial measurements on the scene are available. In this work, we propose a strategy to learn the full state of sloshing liquids from measurements of the free surface. Our approach is based on recurrent neural networks (RNN) that project the limited information available to a reduced-order manifold to not only reconstruct the unknown information but also be capable of performing fluid reasoning about future scenarios in real-time. To obtain physically consistent predictions, we train deep neural networks on the reduced-order manifold that, through the employ of inductive biases, ensure the fulfillment of the principles of thermodynamics. RNNs learn from history the required hidden information to correlate the limited information with the latent space where the simulation occurs. Finally, a decoder returns data to the high-dimensional manifold, to provide the user with insightful information in the form of augmented reality. This algorithm is connected to a computer vision system to test the performance of the proposed methodology with real information, resulting in a system capable of understanding and predicting future states of the observed fluid in real-time.

Regularized by Physics: Graph Neural Network Parametrized Potentials for the Description of Intermolecular Interactions

Simulations of molecular systems using electronic structure methods are still not feasible for many systems of biological importance. As a result, empirical methods such as force fields (FF) have become an established tool for the simulation of large and complex molecular systems. The parametrization of FF is, however, time-consuming and has traditionally been based on experimental data. Recent years have therefore seen increasing efforts to automatize FF parametrization or to replace FF with machine-learning (ML) based potentials. Here, we propose an alternative strategy to parametrize FF, which makes use of ML and gradient-descent based optimization while retaining a functional form founded in physics. Using a predefined functional form is shown to enable interpretability, robustness, and efficient simulations of large systems over long time scales. To demonstrate the strength of the proposed method, a fixed-charge and a polarizable model are trained on ab initio potential-energy surfaces. Given only information about the constituting elements, the molecular topology, and reference potential energies, the models successfully learn to assign atom types and corresponding FF parameters from scratch. The resulting models and parameters are validated on a wide range of experimentally and computationally derived properties of systems including dimers, pure liquids, and molecular crystals.

A transferable recommender approach for selecting the best density functional approximations in chemical discovery

Approximate density functional theory has become indispensable owing to its balanced cost-accuracy trade-off, including in large-scale screening. To date, however, no density functional approximation (DFA) with universal accuracy has been identified, leading to uncertainty in the quality of data generated from density functional theory. With electron density fitting and Δ-learning, we build a DFA recommender that selects the DFA with the lowest expected error with respect to the gold standard (but cost-prohibitive) coupled cluster theory in a system-specific manner. We demonstrate this recommender approach on the evaluation of vertical spin splitting energies of transition metal complexes. Our recommender predicts top-performing DFAs and yields excellent accuracy (about 2 kcal mol-1) for chemical discovery, outperforming both individual Δ-learning models and the best conventional single-functional approach from a set of 48 DFAs. By demonstrating transferability to diverse synthesized compounds, our recommender potentially addresses the accuracy versus scope dilemma broadly encountered in computational chemistry.

Reliable emulation of complex functionals by active learning with error control

A statistical emulator can be used as a surrogate of complex physics-based calculations to drastically reduce the computational cost. Its successful implementation hinges on an accurate representation of the nonlinear response surface with a high-dimensional input space. Conventional "space-filling" designs, including random sampling and Latin hypercube sampling, become inefficient as the dimensionality of the input variables increases, and the predictive accuracy of the emulator can degrade substantially for a test input distant from the training input set. To address this fundamental challenge, we develop a reliable emulator for predicting complex functionals by active learning with error control (ALEC). The algorithm is applicable to infinite-dimensional mapping with high-fidelity predictions and a controlled predictive error. The computational efficiency has been demonstrated by emulating the classical density functional theory (cDFT) calculations, a statistical-mechanical method widely used in modeling the equilibrium properties of complex molecular systems. We show that ALEC is much more accurate than conventional emulators based on the Gaussian processes with "space-filling" designs and alternative active learning methods. In addition, it is computationally more efficient than direct cDFT calculations. ALEC can be a reliable building block for emulating expensive functionals owing to its minimal computational cost, controllable predictive error, and fully automatic features.

Differentiable quantum chemistry with PySCF for molecules and materials at the mean-field level and beyond

We introduce an extension to the PySCF package, which makes it automatically differentiable. The implementation strategy is discussed, and example applications are presented to demonstrate the automatic differentiation framework for quantum chemistry methodology development. These include orbital optimization, properties, excited-state energies, and derivative couplings, at the mean-field level and beyond, in both molecules and solids. We also discuss some current limitations and directions for future work.

The fourth-order expansion of the exchange hole and neural networks to construct exchange–correlation functionals

The curvature Q σ of spherically averaged exchange (X) holes ρX, σ(r, u) is one of the crucial variables for the construction of approximations to the exchange–correlation energy of Kohn–Sham theory, the most prominent example being the Becke–Roussel model [A. D. Becke and M. R. Roussel, Phys. Rev. A 39, 3761 (1989)]. Here, we consider the next higher nonzero derivative of the spherically averaged X hole, the fourth-order term T σ. This variable contains information about the nonlocality of the X hole and we employ it to approximate hybrid functionals, eliminating the sometimes demanding calculation of the exact X energy. The new functional is constructed using machine learning; having identified a physical correlation between T σ and the nonlocality of the X hole, we employ a neural network to express this relation. While we only modify the X functional of the Perdew–Burke–Ernzerhof functional [Perdew et al., Phys. Rev. Lett. 77, 3865 (1996)], a significant improvement over this method is achieved.

Deep-learning density functionals for gradient descent optimization

Machine-learned regression models represent a promising tool to implement accurate and computationally affordable energy-density functionals to solve quantum many-body problems via density functional theory. However, while they can easily be trained to accurately map ground-state density profiles to the corresponding energies, their functional derivatives often turn out to be too noisy, leading to instabilities in self-consistent iterations and in gradient-based searches of the ground-state density profile. We investigate how these instabilities occur when standard deep neural networks are adopted as regression models, and we show how to avoid them by using an ad hoc convolutional architecture featuring an interchannel averaging layer. The main testbed we consider is a realistic model for noninteracting atoms in optical speckle disorder. With the interchannel average, accurate and systematically improvable ground-state energies and density profiles are obtained via gradient-descent optimization, without instabilities nor violations of the variational principle.

Machine learning, artificial intelligence, and chemistry: how smart algorithms are reshaping simulation and the laboratory

Machine learning and artificial intelligence are increasingly gaining in prominence through image analysis, language processing, and automation, to name a few applications. Machine learning is also making profound changes in chemistry. From revisiting decades-old analytical techniques for the purpose of creating better calibration curves, to assisting and accelerating traditional in silico simulations, to automating entire scientific workflows, to being used as an approach to deduce underlying physics of unexplained chemical phenomena, machine learning and artificial intelligence are reshaping chemistry, accelerating scientific discovery, and yielding new insights. This review provides an overview of machine learning and artificial intelligence from a chemist’s perspective and focuses on a number of examples of the use of these approaches in computational chemistry and in the laboratory.

Orbital Mixer: Using Atomic Orbital Features for Basis-Dependent Prediction of Molecular Wavefunctions

Leveraging ab initio data at scale has enabled the development of machine learning models capable of extremely accurate and fast molecular property prediction. A central paradigm of many previous studies focuses on generating predictions for only a fixed set of properties. Recent lines of research instead aim to explicitly learn the electronic structure via molecular wavefunctions, from which other quantum chemical properties can be directly derived. While previous methods generate predictions as a function of only the atomic configuration, in this work we present an alternate approach that directly purposes basis-dependent information to predict molecular electronic structure. Our model, Orbital Mixer, is composed entirely of multi-layer perceptrons (MLPs) using MLP-Mixer layers within a simple, intuitive, and scalable architecture that achieves competitive Hamiltonian and molecular orbital energy and coefficient prediction accuracies compared to the state-of-the-art.

Evolving symbolic density functionals

Systematic development of accurate density functionals has been a decades-long challenge for scientists. Despite emerging applications of machine learning (ML) in approximating functionals, the resulting ML functionals usually contain more than tens of thousands of parameters, leading to a huge gap in the formulation with the conventional human-designed symbolic functionals. We propose a new framework, Symbolic Functional Evolutionary Search (SyFES), that automatically constructs accurate functionals in the symbolic form, which is more explainable to humans, cheaper to evaluate, and easier to integrate to existing codes than other ML functionals. We first show that, without prior knowledge, SyFES reconstructed a known functional from scratch. We then demonstrate that evolving from an existing functional ωB97M-V, SyFES found a new functional, GAS22 (Google Accelerated Science 22), that performs better for most of the molecular types in the test set of Main Group Chemistry Database (MGCDB84). Our framework opens a new direction in leveraging computing power for the systematic development of symbolic density functionals.

Rapidly predicting Kohn-Sham total energy using data-centric AI

Predicting material properties by solving the Kohn-Sham (KS) equation, which is the basis of modern computational approaches to electronic structures, has provided significant improvements in materials sciences. Despite its contributions, both DFT and DFTB calculations are limited by the number of electrons and atoms that translate into increasingly longer run-times. In this work we introduce a novel, data-centric machine learning framework that is used to rapidly and accurately predicate the KS total energy of anatase \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{TiO}}_2$$\end{document}TiO2 nanoparticles (NPs) at different temperatures using only a small amount of theoretical data. The proposed framework that we call co-modeling eliminates the need for experimental data and is general enough to be used over any NPs to determine electronic structure and, consequently, more efficiently study physical and chemical properties. We include a web service to demonstrate the effectiveness of our approach.

Application of two-component neural network for exchange-correlation functional interpolation

Density functional theory (DFT) is one of the primary approaches to solving the many-body Schrodinger equation. The essential part of the DFT theory is the exchange-correlation (XC) functional, which can not be obtained in analytical form. Accordingly, the accuracy improvement of the DFT is mainly based on the development of XC functional approximations. Commonly, they are built upon analytic solutions in low- and high-density limits and result from quantum Monte Carlo or post-Hartree-Fock numerical calculations. However, there is no universal functional form to incorporate these data into XC functional. Instead, various parameterizations use heuristic rules to build a specific XC functional. The neural network (NN) approach to interpolate the data from higher precision theories can give a unified path to parametrize an XC functional. Moreover, data from many existing quantum chemical databases could provide the XC functional with improved accuracy. We develop NN XC functional, which gives exchange potential and energy density without direct derivatives of exchange-correlation energy density. Proposed NN architecture consists of two parts NN-E and NN-V, which could be trained in separate ways, adding new flexibility to XC functional. We also show that the developed NN XC functional converges in the self-consistent cycle and gives reasonable energies when applied to atoms, molecules, and crystals.

Large-scale distributed linear algebra with tensor processing units

We have repurposed Google tensor processing units (TPUs), application-specific chips developed for machine learning, into large-scale dense linear algebra supercomputers. The TPUs' fast intercore interconnects (ICIs), physically two-dimensional network topology, and high-bandwidth memory (HBM) permit distributed matrix multiplication algorithms to rapidly become computationally bound. In this regime, the matrix-multiply units (MXUs) dominate the runtime, yielding impressive scaling, performance, and raw size: Operating in float32 precision, a full 2,048-core pod of third-generation TPUs can multiply two matrices with linear size [Formula: see text] in about 2 min. Via curated algorithms emphasizing large, single-core matrix multiplications, other tasks in dense linear algebra can similarly scale. As examples, we present 1) QR decomposition; 2) resolution of linear systems; and 3) the computation of matrix functions by polynomial iteration, demonstrated by the matrix polar factorization.

Informing geometric deep learning with electronic interactions to accelerate quantum chemistry

Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. However, existing machine learning techniques are challenged by the scarcity of training data when exploring unknown chemical spaces. We overcome this barrier by systematically incorporating knowledge of molecular electronic structure into deep learning. By developing a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simulations and learned mappings to recover high-fidelity physical quantities. OrbNet-Equi accurately models a wide spectrum of target properties while being several orders of magnitude faster than density functional theory. Despite only using training samples collected from readily available small-molecule libraries, OrbNet-Equi outperforms traditional semiempirical and machine learning-based methods on comprehensive downstream benchmarks that encompass diverse main-group chemical processes. Our method also describes interactions in challenging charge-transfer complexes and open-shell systems. We anticipate that the strategy presented here will help to expand opportunities for studies in chemistry and materials science, where the acquisition of experimental or reference training data is costly.

Stability Analysis of Substituted Cobaltocenium [Bis(cyclopentadienyl)cobalt(III)] Employing Chemistry-Informed Neural Networks

Cationic cobaltocenium derivatives are promising components of the anion exchange membranes because of their excellent thermal and alkaline stability under the operating conditions of a fuel cell. Here, we present an efficient modeling approach to assessing the chemical stability of substituted cobaltocenium CoCp₂⁺, based on the computed electronic structure enhanced by machine learning techniques. Within the aqueous environment, the positive charge of the metal cation is balanced by the hydroxide anion through formation of the CoCp₂⁺OH^- complexes, whose dissociation is studied within the implicit solvent employing the density functional theory. The data set of about 118 the CoCp₂⁺OH^- complexes based on 42 substituent groups characterized by a range of electron-donating (ED) and electron-withdrawing (EW) properties is constructed and analyzed. Given 12 carefully chosen chemistry-informed descriptors of the complexes and relevant fragments, the stability of the complexes is found to strongly correlate with the energies of the highest occupied and lowest unoccupied molecular orbitals, modulated by a switching function of the Hirshfeld charge. The latter is used as a measure of the electron-withdrawing-donating character of the substituents. On the basis of this observation from the conventional regression analysis, two fully connected, feed-forward neural network (FNN) models with different unit structures, called the chemistry-informed (CINN) and the quadratic (QNN) neural networks, together with a support vector regression (SVR) model are developed. Both deep neural network models predict the bond dissociation energies of the cobaltocenium complexes with mean relative errors less than 10.56% and average absolute errors less than 1.63 kcal/mol, superior to the conventional regression and the SVR model prediction. The results show the potential of QNN to efficiently capture more complex relationships. The concept of incorporating the domain (chemical) knowledge/insight into the neural network structure paves the way to applications of machine learning techniques with small data sets, ultimately leading to better predictive models compared to the classical machine learning method SVR and conventional regression analysis.

Improving Results by Improving Densities: Density-Corrected Density Functional Theory

Density functional theory (DFT) calculations have become widespread in both chemistry and materials, because they usually provide useful accuracy at much lower computational cost than wavefunction-based methods. All practical DFT calculations require an approximation to the unknown exchange-correlation energy, which is then used self-consistently in the Kohn-Sham scheme to produce an approximate energy from an approximate density. Density-corrected DFT is simply the study of the relative contributions to the total energy error. In the vast majority of DFT calculations, the error due to the approximate density is negligible. But with certain classes of functionals applied to certain classes of problems, the density error is sufficiently large as to contribute to the energy noticeably, and its removal leads to much better results. These problems include reaction barriers, torsional barriers involving π-conjugation, halogen bonds, radicals and anions, most stretched bonds, etc. In all such cases, use of a more accurate density significantly improves performance, and often the simple expedient of using the Hartree-Fock density is enough. This Perspective explains what DC-DFT is, where it is likely to improve results, and how DC-DFT can produce more accurate functionals. We also outline challenges and prospects for the field.

PIGNet: a physics-informed deep learning model toward generalized drug-target interaction predictions

Recently, deep neural network (DNN)-based drug–target interaction (DTI) models were highlighted for their high accuracy with affordable computational costs. Yet, the models' insufficient generalization remains a challenging problem in the practice of in silico drug discovery. We propose two key strategies to enhance generalization in the DTI model. The first is to predict the atom–atom pairwise interactions via physics-informed equations parameterized with neural networks and provides the total binding affinity of a protein–ligand complex as their sum. We further improved the model generalization by augmenting a broader range of binding poses and ligands to training data. We validated our model, PIGNet, in the comparative assessment of scoring functions (CASF) 2016, demonstrating the outperforming docking and screening powers than previous methods. Our physics-informing strategy also enables the interpretation of predicted affinities by visualizing the contribution of ligand substructures, providing insights for further ligand optimization.

PIGNet, a deep neural network-based drug–target interaction model guided by physics and extensive data augmentation, shows significantly improved generalization ability and model performance.

How Well Does Kohn-Sham Regularizer Work for Weakly Correlated Systems?

Kohn-Sham regularizer (KSR) is a differentiable machine learning approach to finding the exchange-correlation functional in Kohn-Sham density functional theory that works for strongly correlated systems. Here we test KSR for a weak correlation. We propose spin-adapted KSR (sKSR) with trainable local, semilocal, and nonlocal approximations found by minimizing density and total energy loss. We assess the atoms-to-molecules generalizability by training on one-dimensional (1D) H, He, Li, Be, and Be²⁺ and testing on 1D hydrogen chains, LiH, BeH₂, and helium hydride complexes. The generalization error from our semilocal approximation is comparable to other differentiable approaches, but our nonlocal functional outperforms any existing machine learning functionals, predicting ground-state energies of test systems with a mean absolute error of 2.7 mH.

Transferable Neural Network Potential Energy Surfaces for Closed-Shell Organic Molecules: Extension to Ions

Transferable high dimensional neural network potentials (HDNNPs) have shown great promise as an avenue to increase the accuracy and domain of applicability of existing atomistic force fields for organic systems relevant to life science. We have previously reported such a potential (Schrödinger-ANI) that has broad coverage of druglike molecules. We extend that work here to cover ionic and zwitterionic druglike molecules expected to be relevant to drug discovery research activities. We report a novel HDNNP architecture, which we call QRNN, that predicts atomic charges and uses these charges as descriptors in an energy model that delivers conformational energies within chemical accuracy when measured against the reference theory it is trained to. Further, we find that delta learning based on a semiempirical level of theory approximately halves the errors. We test the models on torsion energy profiles, relative conformational energies, geometric parameters, and relative tautomer errors.

CIDER: An Expressive, Nonlocal Feature Set for Machine Learning Density Functionals with Exact Constraints

Machine learning (ML) has recently gained attention as a means to develop more accurate exchange-correlation (XC) functionals for density functional theory, but functionals developed thus far need to be improved on several metrics, including accuracy, numerical stability, and transferability across chemical space. In this work, we introduce a set of nonlocal features of the density called the CIDER formalism, which we use to train a Gaussian process model for the exchange energy that obeys the critical uniform scaling rule for exchange. The resulting CIDER exchange functional is significantly more accurate than any semilocal functional tested here, and it has good transferability across main-group molecules. This work therefore serves as an initial step toward more accurate exchange functionals, and it also introduces useful techniques for developing robust, physics-informed XC models via ML.

Machine learning the derivative discontinuity of density-functional theory

Machine learning is a powerful tool to design accurate, highly non-local, exchange-correlation functionals for density functional theory. So far, most of those machine learned functionals are trained for systems with an integer number of particles. As such, they are unable to reproduce some crucial and fundamental aspects, such as the explicit dependency of the functionals on the particle number or the infamous derivative discontinuity at integer particle numbers. Here we propose a solution to these problems by training a neural network as the universal functional of density-functional theory that (a) depends explicitly on the number of particles with a piece-wise linearity between the integer numbers and (b) reproduces the derivative discontinuity of the exchange-correlation energy. This is achieved by using an ensemble formalism, a training set containing fractional densities, and an explicitly discontinuous formulation.

DQC: A Python program package for differentiable quantum chemistry

Automatic differentiation represents a paradigm shift in scientific programming, where evaluating both functions and their derivatives is required for most applications. By removing the need to explicitly derive expressions for gradients, development times can be shortened and calculations can be simplified. For these reasons, automatic differentiation has fueled the rapid growth of a variety of sophisticated machine learning techniques over the past decade, but is now also increasingly showing its value to support ab initio simulations of quantum systems and enhance computational quantum chemistry. Here, we present an open-source differentiable quantum chemistry simulation code and explore applications facilitated by automatic differentiation: (1) calculating molecular perturbation properties, (2) reoptimizing a basis set for hydrocarbons, (3) checking the stability of self-consistent field wave functions, and (4) predicting molecular properties via alchemical perturbations.