Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties

Accurate Neural Network Fine-Tuning Approach for Transferable Ab Initio Energy Prediction across Varying Molecular and Crystalline Scales

Existing machine learning models attempt to predict the energies of large molecules by training small molecules, but eventually fail to retain high accuracy as the errors increase with system size. Through an orbital pairwise decomposition of the correlation energy, a pretrained neural network model on hundred-scale data containing small molecules is demonstrated to be sufficiently transferable for accurately predicting large systems, including molecules and crystals. Our model introduces a residual connection to explicitly learn the pairwise energy corrections, and employs various low-rank retraining techniques to modestly adjust the learned network parameters. We demonstrate that with as few as only one larger molecule retraining the base model originally trained on only small molecules of (H2O)6, the MP2 correlation energy of the large liquid water (H2O)64 in a periodic supercell can be predicted at chemical accuracy. Similar performance is observed for large protonated clusters and periodic poly glycine chains. A demonstrative application is presented to predict the energy ordering of symmetrically inequivalent sublattices for distinct hydrogen orientations in the ice XV phase. Our work represents an important step forward in the quest for cost-effective, highly accurate and transferable neural network models in quantum chemistry, bridging the electronic structure patterns between small and large systems.

Learning the Electrostatic Response of the Electron Density through a Symmetry-Adapted Vector Field Model

A current challenge in atomistic machine learning is that of efficiently predicting the response of the electron density under electric fields. We address this challenge with symmetry-adapted kernel functions that are specifically derived to account for the rotational symmetry of a three-dimensional vector field. We demonstrate the equivariance of the method on a set of rotated water molecules and show its high efficiency with respect to number of training configurations and features for liquid water and naphthalene crystals. We conclude showcasing applications for relaxed configurations of gold nanoparticles, reproducing the scaling law of the electronic polarizability with size, up to systems with more than 2000 atoms. By deriving a natural extension to equivariant learning models of the electron density, our method provides an accurate and inexpensive strategy to predict the electrostatic response of molecules and materials.

Machine learning Hubbard parameters with equivariant neural networks

Density-functional theory with extended Hubbard functionals (DFT + U + V) provides a robust framework to accurately describe complex materials containing transition-metal or rare-earth elements. It does so by mitigating self-interaction errors inherent to semi-local functionals which are particularly pronounced in systems with partially-filled d and f electronic states. However, achieving accuracy in this approach hinges upon the accurate determination of the on-site U and inter-site V Hubbard parameters. In practice, these are obtained either by semi-empirical tuning, requiring prior knowledge, or, more correctly, by using predictive but expensive first-principles calculations. Here, we present a machine learning model based on equivariant neural networks which uses atomic occupation matrices as descriptors, directly capturing the electronic structure, local chemical environment, and oxidation states of the system at hand. We target here the prediction of Hubbard parameters computed self-consistently with iterative linear-response calculations, as implemented in density-functional perturbation theory (DFPT), and structural relaxations. Remarkably, when trained on data from 12 materials spanning various crystal structures and compositions, our model achieves mean absolute relative errors of 3% and 5% for Hubbard U and V parameters, respectively. By circumventing computationally expensive DFT or DFPT self-consistent protocols, our model significantly expedites the prediction of Hubbard parameters with negligible computational overhead, while approaching the accuracy of DFPT. Moreover, owing to its robust transferability, the model facilitates accelerated materials discovery and design via high-throughput calculations, with relevance for various technological applications.

Bridging electronic and classical density-functional theory using universal machine-learned functional approximations

The accuracy of density-functional theory (DFT) calculations is ultimately determined by the quality of the underlying approximate functionals, namely the exchange-correlation functional in electronic DFT and the excess functional in the classical DFT formalism of fluids. For both electrons and fluids, the exact functional is highly nonlocal, yet most calculations employ approximate functionals that are semi-local or nonlocal in a limited weighted-density form. Machine-learned (ML) nonlocal density-functional approximations show promise in advancing applications of both electronic and classical DFTs, but so far these two distinct research areas have implemented disparate approaches with limited generality. Here, we formulate a universal ML framework and training protocol to learn nonlocal functionals that combine features of equivariant convolutional neural networks and the weighted-density approximation. We prototype this new approach for several 1D and quasi-1D problems and demonstrate that functionals with exactly the same hyperparameters achieve excellent accuracy for a diverse set of systems, including the hard-rod fluid, the inhomogeneous Ising model, the exact exchange energy of electrons, the electron kinetic energy for orbital-free DFT, as well as for liquid water with 1D inhomogeneities. These results lay the foundation for a universal ML approach to approximate exact 3D functionals spanning electronic and classical DFTs.

A deep equivariant neural network approach for efficient hybrid density functional calculations

Hybrid density functional calculations are essential for accurate description of electronic structure, yet their widespread use is restricted by the substantial computational cost. Here we develop DeepH-hybrid, a deep equivariant neural network method for learning the hybrid-functional Hamiltonian as a function of material structure, which circumvents the time-consuming self-consistent field iterations and enables the study of large-scale materials with hybrid-functional accuracy. Our extensive experiments demonstrate good reliability as well as effective transferability and efficiency of the method. As a notable application, DeepH-hybrid is applied to study large-supercell Moiré-twisted materials, offering the first case study on how the inclusion of exact exchange affects flat bands in magic-angle twisted bilayer graphene. The work generalizes deep-learning electronic structure methods to beyond conventional density functional theory, facilitating the development of deep-learning-based ab initio methods.

Deep learning tight-binding approach for large-scale electronic simulations at finite temperatures with ab initio accuracy

Simulating electronic behavior in materials and devices with realistic large system sizes remains a formidable task within the ab initio framework due to its computational intensity. Here we show DeePTB, an efficient deep learning-based tight-binding approach with ab initio accuracy to address this issue. By training on structural data and corresponding ab initio eigenvalues, the DeePTB model can efficiently predict tight-binding Hamiltonians for unseen structures, enabling efficient simulations of large-size systems under external perturbations such as finite temperatures and strain. This capability is vital for semiconductor band gap engineering and materials design. When combined with molecular dynamics, DeePTB facilitates efficient and accurate finite-temperature simulations of both atomic and electronic behavior simultaneously. This is demonstrated by computing the temperature-dependent electronic properties of a gallium phosphide system with 106 atoms. The availability of DeePTB bridges the gap between accuracy and scalability in electronic simulations, potentially advancing materials science and related fields by enabling large-scale electronic structure calculations.

Wigner kernels: Body-ordered equivariant machine learning without a basis

Machine-learning models based on a point-cloud representation of a physical object are ubiquitous in scientific applications and particularly well-suited to the atomic-scale description of molecules and materials. Among the many different approaches that have been pursued, the description of local atomic environments in terms of their discretized neighbor densities has been used widely and very successfully. We propose a novel density-based method, which involves computing "Wigner kernels." These are fully equivariant and body-ordered kernels that can be computed iteratively at a cost that is independent of the basis used to discretize the density and grows only linearly with the maximum body-order considered. Wigner kernels represent the infinite-width limit of feature-space models, whose dimensionality and computational cost instead scale exponentially with the increasing order of correlations. We present several examples of the accuracy of models based on Wigner kernels in chemical applications, for both scalar and tensorial targets, reaching an accuracy that is competitive with state-of-the-art deep-learning architectures. We discuss the broader relevance of these findings to equivariant geometric machine-learning.

Integrated workflows and interfaces for data-driven semi-empirical electronic structure calculations

Modern software engineering of electronic structure codes has seen a paradigm shift from monolithic workflows toward object-based modularity. Software objectivity allows for greater flexibility in the application of electronic structure calculations, with particular benefits when integrated with approaches for data-driven analysis. Here, we discuss different approaches to create deep modular interfaces that connect big-data workflows and electronic structure codes and explore the diversity of use cases that they can enable. We present two such interface approaches for the semi-empirical electronic structure package, DFTB+. In one case, DFTB+ is applied as a library and provides data to an external workflow; in another, DFTB+receives data via external bindings and processes the information subsequently within an internal workflow. We provide a general framework to enable data exchange workflows for embedding new machine-learning-based Hamiltonians within DFTB+ or enabling deep integration of DFTB+ in multiscale embedding workflows. These modular interfaces demonstrate opportunities in emergent software and workflows to accelerate scientific discovery by harnessing existing software capabilities.

In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back

Computational chemistry is an indispensable tool for understanding molecules and predicting chemical properties. However, traditional computational methods face significant challenges due to the difficulty of solving the Schrödinger equations and the increasing computational cost with the size of the molecular system. In response, there has been a surge of interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to in silico experiments. Integrating AI and ML into computational chemistry increases the scalability and speed of the exploration of chemical space. However, challenges remain, particularly regarding the reproducibility and transferability of ML models. This review highlights the evolution of ML in learning from, complementing, or replacing traditional computational chemistry for energy and property predictions. Starting from models trained entirely on numerical data, a journey set forth toward the ideal model incorporating or learning the physical laws of quantum mechanics. This paper also reviews existing computational methods and ML models and their intertwining, outlines a roadmap for future research, and identifies areas for improvement and innovation. Ultimately, the goal is to develop AI architectures capable of predicting accurate and transferable solutions to the Schrödinger equation, thereby revolutionizing in silico experiments within chemistry and materials science.

Electronic Excited States from Physically Constrained Machine Learning

Data-driven techniques are increasingly used to replace electronic-structure calculations of matter. In this context, a relevant question is whether machine learning (ML) should be applied directly to predict the desired properties or combined explicitly with physically grounded operations. We present an example of an integrated modeling approach in which a symmetry-adapted ML model of an effective Hamiltonian is trained to reproduce electronic excitations from a quantum-mechanical calculation. The resulting model can make predictions for molecules that are much larger and more complex than those on which it is trained and allows for dramatic computational savings by indirectly targeting the outputs of well-converged calculations while using a parametrization corresponding to a minimal atom-centered basis. These results emphasize the merits of intertwining data-driven techniques with physical approximations, improving the transferability and interpretability of ML models without affecting their accuracy and computational efficiency and providing a blueprint for developing ML-augmented electronic-structure methods.

Ensemble of local and global information for Protein-Ligand Binding Affinity Prediction

Accurately predicting protein-ligand binding affinities is crucial for determining molecular properties and understanding their physical effects. Neural networks and transformers are the predominant methods for sequence modeling, and both have been successfully applied independently for protein-ligand binding affinity prediction. As local and global information of molecules are vital for protein-ligand binding affinity prediction, we aim to combine bi-directional gated recurrent unit (BiGRU) and convolutional neural network (CNN) to effectively capture both local and global molecular information. Additionally, attention mechanisms can be incorporated to automatically learn and adjust the level of attention given to local and global information, thereby enhancing the performance of the model. To achieve this, we propose the PLAsformer approach, which encodes local and global information of molecules using 3DCNN and BiGRU with attention mechanism, respectively. This approach enhances the model's ability to encode comprehensive local and global molecular information. PLAsformer achieved a Pearson's correlation coefficient of 0.812 and a Root Mean Square Error (RMSE) of 1.284 when comparing experimental and predicted affinity on the PDBBind-2016 dataset. These results surpass the current state-of-the-art methods for binding affinity prediction. The high accuracy of PLAsformer's predictive performance, along with its excellent generalization ability, is clearly demonstrated by these findings.

Physics-Inspired Equivariant Descriptors of Nonbonded Interactions

One essential ingredient in many machine learning (ML) based methods for atomistic modeling of materials and molecules is the use of locality. While allowing better system-size scaling, this systematically neglects long-range (LR) effects such as electrostatic or dispersion interactions. We present an extension of the long distance equivariant (LODE) framework that can handle diverse LR interactions in a consistent way and seamlessly integrates with preexisting methods by building new sets of atom centered features. We provide a direct physical interpretation of these using the multipole expansion, which allows for simpler and more efficient implementations. The framework is applied to simple toy systems as proof of concept and a heterogeneous set of molecular dimers to push the method to its limits. By generalizing LODE to arbitrary asymptotic behaviors, we provide a coherent approach to treat arbitrary two- and many-body nonbonded interactions in the data-driven modeling of matter.

Universal machine learning for the response of atomistic systems to external fields

Machine learned interatomic interaction potentials have enabled efficient and accurate molecular simulations of closed systems. However, external fields, which can greatly change the chemical structure and/or reactivity, have been seldom included in current machine learning models. This work proposes a universal field-induced recursively embedded atom neural network (FIREANN) model, which integrates a pseudo field vector-dependent feature into atomic descriptors to represent system-field interactions with rigorous rotational equivariance. This "all-in-one" approach correlates various response properties like dipole moment and polarizability with the field-dependent potential energy in a single model, very suitable for spectroscopic and dynamics simulations in molecular and periodic systems in the presence of electric fields. Especially for periodic systems, we find that FIREANN can overcome the intrinsic multiple-value issue of the polarization by training atomic forces only. These results validate the universality and capability of the FIREANN method for efficient first-principles modeling of complicated systems in strong external fields.

Machine learning electronic structure methods based on the one-electron reduced density matrix

The theorems of density functional theory (DFT) establish bijective maps between the local external potential of a many-body system and its electron density, wavefunction and, therefore, one-particle reduced density matrix. Building on this foundation, we show that machine learning models based on the one-electron reduced density matrix can be used to generate surrogate electronic structure methods. We generate surrogates of local and hybrid DFT, Hartree-Fock and full configuration interaction theories for systems ranging from small molecules such as water to more complex compounds like benzene and propanol. The surrogate models use the one-electron reduced density matrix as the central quantity to be learned. From the predicted density matrices, we show that either standard quantum chemistry or a second machine-learning model can be used to compute molecular observables, energies, and atomic forces. The surrogate models can generate essentially anything that a standard electronic structure method can, ranging from band gaps and Kohn-Sham orbitals to energy-conserving ab-initio molecular dynamics simulations and infrared spectra, which account for anharmonicity and thermal effects, without the need to employ computationally expensive algorithms such as self-consistent field theory. The algorithms are packaged in an efficient and easy to use Python code, QMLearn, accessible on popular platforms.

General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian

The combination of deep learning and ab initio calculation has shown great promise in revolutionizing future scientific research, but how to design neural network models incorporating a priori knowledge and symmetry requirements is a key challenging subject. Here we propose an E(3)-equivariant deep-learning framework to represent density functional theory (DFT) Hamiltonian as a function of material structure, which can naturally preserve the Euclidean symmetry even in the presence of spin-orbit coupling. Our DeepH-E3 method enables efficient electronic structure calculation at ab initio accuracy by learning from DFT data of small-sized structures, making the routine study of large-scale supercells (>104 atoms) feasible. The method can reach sub-meV prediction accuracy at high training efficiency, showing state-of-the-art performance in our experiments. The work is not only of general significance to deep-learning method development but also creates opportunities for materials research, such as building a Moiré-twisted material database.

Prediction of Kinetic Product Ratios: Investigation of a Dynamically Controlled Case

Of the various factors influencing kinetically controlled product ratios, the role of nonstatistical dynamics is arguably the least well understood. In this paper, reactions were chosen in which dynamics played a dominant role in product selection, by design. Specifically, the reactions studied were the ring openings of cyclopropylidene to allene and tetramethylcyclopropylidene to tetramethylallene (2,4-dimethylpenta-2,3-diene). Both reactions have intrinsic reaction coordinates that bifurcate symmetrically, leading to products that are enantiomeric once the atoms are uniquely labeled. The question addressed in the study was whether the outcomes─that is, which product well on the potential energy surface was selected─could be predicted from their initial conditions for individual trajectories in quasiclassical dynamics simulations. Hybrid potentials were developed based on cooperative interaction between molecular mechanics and artificial neural networks, trained against data from electronic structure calculations. These potentials allowed simulations of both gas-phase and condensed-phase reactions. The outcome was that, for both reactions, prediction of initial selection of product wells could be made with >95% success from initial conditions of the trajectories in the gas phase. However, when trajectories were run for longer, looking for "final" products for each trajectory, the predictability dropped off dramatically. In the gas-phase simulations, this drop off was caused by trajectories hopping between product wells on the potential energy surface. That behavior could be suppressed in condensed phases, but then new uncertainty was introduced because the intermolecular interactions between solute and bath, necessary to permit intermolecular energy transfer and cooling of the hot initial products, often led to perturbations of the initial directions of trajectories on the potential energy surface. It would consequently appear that a general ability to predict outcomes for reactions in which nonstatistical dynamics dominate remains a challenge even in the age of sophisticated machine-learning capabilities.

Electronic-Structure Properties from Atom-Centered Predictions of the Electron Density

The electron density of a molecule or material has recently received major attention as a target quantity of machine-learning models. A natural choice to construct a model that yields transferable and linear-scaling predictions is to represent the scalar field using a multicentered atomic basis analogous to that routinely used in density fitting approximations. However, the nonorthogonality of the basis poses challenges for the learning exercise, as it requires accounting for all the atomic density components at once. We devise a gradient-based approach to directly minimize the loss function of the regression problem in an optimized and highly sparse feature space. In so doing, we overcome the limitations associated with adopting an atom-centered model to learn the electron density over arbitrarily complex data sets, obtaining very accurate predictions using a comparatively small training set. The enhanced framework is tested on 32-molecule periodic cells of liquid water, presenting enough complexity to require an optimal balance between accuracy and computational efficiency. We show that starting from the predicted density a single Kohn-Sham diagonalization step can be performed to access total energy components that carry an error of just 0.1 meV/atom with respect to the reference density functional calculations. Finally, we test our method on the highly heterogeneous QM9 benchmark data set, showing that a small fraction of the training data is enough to derive ground-state total energies within chemical accuracy.

Graph neural networks for materials science and chemistry

Machine learning plays an increasingly important role in many areas of chemistry and materials science, being used to predict materials properties, accelerate simulations, design new structures, and predict synthesis routes of new materials. Graph neural networks (GNNs) are one of the fastest growing classes of machine learning models. They are of particular relevance for chemistry and materials science, as they directly work on a graph or structural representation of molecules and materials and therefore have full access to all relevant information required to characterize materials. In this Review, we provide an overview of the basic principles of GNNs, widely used datasets, and state-of-the-art architectures, followed by a discussion of a wide range of recent applications of GNNs in chemistry and materials science, and concluding with a road-map for the further development and application of GNNs.

Modeling Chemical Reactions in Alkali Carbonate-Hydroxide Electrolytes with Deep Learning Potentials

We developed a deep potential machine learning model for simulations of chemical reactions in molten alkali carbonate-hydroxide electrolyte containing dissolved CO₂, using an active learning procedure. We tested the deep neural network (DNN) potential and training procedure against reaction kinetics, chemical composition, and diffusion coefficients obtained from density functional theory (DFT) molecular dynamics calculations. The DNN potential was found to match DFT results for the structural, transport, and short-time chemical reactions in the melt. Using the DNN potential, we extended the time scales of observation to 2 ns in systems containing thousands of atoms, while preserving quantum chemical accuracy. This allowed us to reach chemical equilibrium with respect to several chemical species in the melt. The approach can be generalized for a broad spectrum of chemically reactive systems.

Orbital Mixer: Using Atomic Orbital Features for Basis-Dependent Prediction of Molecular Wavefunctions

Leveraging ab initio data at scale has enabled the development of machine learning models capable of extremely accurate and fast molecular property prediction. A central paradigm of many previous studies focuses on generating predictions for only a fixed set of properties. Recent lines of research instead aim to explicitly learn the electronic structure via molecular wavefunctions, from which other quantum chemical properties can be directly derived. While previous methods generate predictions as a function of only the atomic configuration, in this work we present an alternate approach that directly purposes basis-dependent information to predict molecular electronic structure. Our model, Orbital Mixer, is composed entirely of multi-layer perceptrons (MLPs) using MLP-Mixer layers within a simple, intuitive, and scalable architecture that achieves competitive Hamiltonian and molecular orbital energy and coefficient prediction accuracies compared to the state-of-the-art.

UV-Visible Absorption Spectra of Solvated Molecules by Quantum Chemical Machine Learning

Predicting UV-visible absorption spectra is essential to understand photochemical processes and design energy materials. Quantum chemical methods can deliver accurate calculations of UV-visible absorption spectra, but they are computationally expensive, especially for large systems or when one computes line shapes from thermal averages. Here, we present an approach to predict UV-visible absorption spectra of solvated aromatic molecules by quantum chemistry (QC) and machine learning (ML). We show that a ML model, trained on the high-level QC calculation of the excitation energy of a set of aromatic molecules, can accurately predict the line shape of the lowest-energy UV-visible absorption band of several related molecules with less than 0.1 eV deviation with respect to reference experimental spectra. Applying linear decomposition analysis on the excitation energies, we unveil that our ML models probe vertical excitations of these aromatic molecules primarily by learning the atomic environment of their phenyl rings, which align with the physical origin of the π →π* electronic transition. Our study provides an effective workflow that combines ML with quantum chemical methods to accelerate the calculations of UV-visible absorption spectra for various molecular systems.

Unified theory of atom-centered representations and message-passing machine-learning schemes

Data-driven schemes that associate molecular and crystal structures with their microscopic properties share the need for a concise, effective description of the arrangement of their atomic constituents. Many types of models rely on descriptions of atom-centered environments, which are associated with an atomic property or with an atomic contribution to an extensive macroscopic quantity. Frameworks in this class can be understood in terms of atom-centered density correlations (ACDC), which are used as a basis for a body-ordered, symmetry-adapted expansion of the targets. Several other schemes that gather information on the relationship between neighboring atoms using “message-passing” ideas cannot be directly mapped to correlations centered around a single atom. We generalize the ACDC framework to include multi-centered information, generating representations that provide a complete linear basis to regress symmetric functions of atomic coordinates, and provide a coherent foundation to systematize our understanding of both atom-centered and message-passing and invariant and equivariant machine-learning schemes.

Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties

Symmetry considerations are at the core of the major frameworks used to provide an effective mathematical representation of atomic configurations that is then used in machine-learning models to predict the properties associated with each structure. In most cases, the models rely on a description of atom-centered environments and are suitable to learn atomic properties or global observables that can be decomposed into atomic contributions. Many quantities that are relevant for quantum mechanical calculations, however-most notably the single-particle Hamiltonian matrix when written in an atomic orbital basis-are not associated with a single center, but with two (or more) atoms in the structure. We discuss a family of structural descriptors that generalize the very successful atom-centered density correlation features to the N-center case and show, in particular, how this construction can be applied to efficiently learn the matrix elements of the (effective) single-particle Hamiltonian written in an atom-centered orbital basis. These N-center features are fully equivariant-not only in terms of translations and rotations but also in terms of permutations of the indices associated with the atoms-and are suitable to construct symmetry-adapted machine-learning models of new classes of properties of molecules and materials.