Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions

CHARMM at 45: Enhancements in Accessibility, Functionality, and Speed

Since its inception nearly a half century ago, CHARMM has been playing a central role in computational biochemistry and biophysics. Commensurate with the developments in experimental research and advances in computer hardware, the range of methods and applicability of CHARMM have also grown. This review summarizes major developments that occurred after 2009 when the last review of CHARMM was published. They include the following: new faster simulation engines, accessible user interfaces for convenient workflows, and a vast array of simulation and analysis methods that encompass quantum mechanical, atomistic, and coarse-grained levels, as well as extensive coverage of force fields. In addition to providing the current snapshot of the CHARMM development, this review may serve as a starting point for exploring relevant theories and computational methods for tackling contemporary and emerging problems in biomolecular systems. CHARMM is freely available for academic and nonprofit research at https://academiccharmm.org/program.

Universal materials model of deep-learning density functional theory Hamiltonian

Realizing large materials models has emerged as a critical endeavor for materials research in the new era of artificial intelligence, but how to achieve this fantastic and challenging objective remains elusive. Here, we propose a feasible pathway to address this paramount pursuit by developing universal materials models of deep-learning density functional theory Hamiltonian (DeepH), enabling computational modeling of the complicated structure-property relationship of materials in general. By constructing a large materials database and substantially improving the DeepH method, we obtain a universal materials model of DeepH capable of handling diverse elemental compositions and material structures, achieving remarkable accuracy in predicting material properties. We further showcase a promising application of fine-tuning universal materials models for enhancing specific materials models. This work not only demonstrates the concept of DeepH's universal materials model but also lays the groundwork for developing large materials models, opening up significant opportunities for advancing artificial intelligence-driven materials discovery.

Spiers Memorial Lecture: New directions in molecular scattering

The field of molecular scattering is reviewed as it pertains to gas-gas as well as gas-surface chemical reaction dynamics. We emphasize the importance of collaboration of experiment and theory, from which new directions of research are being pursued on increasingly complex problems. We review both experimental and theoretical advances that provide the modern toolbox available to molecular-scattering studies. We distinguish between two classes of work. The first involves simple systems and uses experiment to validate theory so that from the validated theory, one may learn far more than could ever be measured in the laboratory. The second class involves problems of great complexity that would be difficult or impossible to understand without a partnership of experiment and theory. Key topics covered in this review include crossed-beams reactive scattering and scattering at extremely low energies, where quantum effects dominate. They also include scattering from surfaces, reactive scattering and kinetics at surfaces, and scattering work done at liquid surfaces. The review closes with thoughts on future promising directions of research.

Neural-network Density Functional Theory Based on Variational Energy Minimization

Deep-learning density functional theory (DFT) shows great promise to significantly accelerate material discovery and potentially revolutionize materials research. However, current research in this field primarily relies on data-driven supervised learning, making the developments of neural networks and DFT isolated from each other. In this work, we present a theoretical framework of neural-network DFT, which unifies the optimization of neural networks with the variational computation of DFT, enabling physics-informed unsupervised learning. Moreover, we develop a differential DFT code incorporated with deep-learning DFT Hamiltonian, and introduce algorithms of automatic differentiation and backpropagation into DFT, demonstrating the capability of neural-network DFT. The physics-informed neural-network architecture not only surpasses conventional approaches in accuracy and efficiency, but also offers a new paradigm for developing deep-learning DFT methods.

Deep learning tight-binding approach for large-scale electronic simulations at finite temperatures with ab initio accuracy

Simulating electronic behavior in materials and devices with realistic large system sizes remains a formidable task within the ab initio framework due to its computational intensity. Here we show DeePTB, an efficient deep learning-based tight-binding approach with ab initio accuracy to address this issue. By training on structural data and corresponding ab initio eigenvalues, the DeePTB model can efficiently predict tight-binding Hamiltonians for unseen structures, enabling efficient simulations of large-size systems under external perturbations such as finite temperatures and strain. This capability is vital for semiconductor band gap engineering and materials design. When combined with molecular dynamics, DeePTB facilitates efficient and accurate finite-temperature simulations of both atomic and electronic behavior simultaneously. This is demonstrated by computing the temperature-dependent electronic properties of a gallium phosphide system with 106 atoms. The availability of DeePTB bridges the gap between accuracy and scalability in electronic simulations, potentially advancing materials science and related fields by enabling large-scale electronic structure calculations.

Accelerating the calculation of electron-phonon coupling strength with machine learning

The calculation of electron-phonon couplings (EPCs) is essential for understanding various fundamental physical properties, including electrical transport, optical and superconducting behaviors in materials. However, obtaining EPCs through fully first-principles methods is notably challenging, particularly for large systems or when employing advanced functionals. Here we introduce a machine learning framework to accelerate EPC calculations by utilizing atomic orbital-based Hamiltonian matrices and gradients predicted by an equivariant graph neural network. We demonstrate that our method not only yields EPC values in close agreement with first-principles results but also enhances calculation efficiency by several orders of magnitude. Application to GaAs using the Heyd-Scuseria-Ernzerhof functional reveals the necessity of advanced functionals for accurate carrier mobility predictions, while for the large Kagome crystal CsV3Sb5, our framework reproduces the experimentally observed double domes in pressure-induced superconducting phase diagrams. This machine learning framework offers a powerful and efficient tool for the investigation of diverse EPC-related phenomena in complex materials.

Fault-tolerant quantum chemical calculations with improved machine-learning models

Easy and effective usage of computational resources is crucial for scientific calculations. Following our recent work of machine-learning (ML) assisted scheduling optimization [J. Comput. Chem. 2023, 44, 1174], we further propose (1) the improved ML models for the better predictions of computational loads, and as such, more elaborate load-balancing calculations can be expected; (2) the idea of coded computation, that is, the integration of gradient coding, in order to introduce fault tolerance during the distributed calculations; and (3) their applications together with re-normalized exciton model with time-dependent density functional theory (REM-TDDFT) for calculating the excited states. Illustrated benchmark calculations include P38 protein, and solvent model with one or several excitable centers. The results show that the improved ML-assisted coded calculations can further improve the load-balancing and cluster utilization, owing primarily profit in fault tolerance that aims at the automated quantum chemical calculations for both ground and excited states.

Integrating chemistry knowledge in large language models via prompt engineering

This paper presents a study on the integration of domain-specific knowledge in prompt engineering to enhance the performance of large language models (LLMs) in scientific domains. The proposed domain-knowledge embedded prompt engineering method outperforms traditional prompt engineering strategies on various metrics, including capability, accuracy, F1 score, and hallucination drop. The effectiveness of the method is demonstrated through case studies on complex materials including the MacMillan catalyst, paclitaxel, and lithium cobalt oxide. The results suggest that domain-knowledge prompts can guide LLMs to generate more accurate and relevant responses, highlighting the potential of LLMs as powerful tools for scientific discovery and innovation when equipped with domain-specific prompts. The study also discusses limitations and future directions for domain-specific prompt engineering development.

Computational Investigation of the Potential and Limitations of Machine Learning with Neural Network Circuits Based on Synaptic Transistors

Synaptic transistors have been proposed to implement neuron activation functions of neural networks (NNs). While promising to enable compact, fast, inexpensive, and energy-efficient dedicated NN circuits, they also have limitations compared to digital NNs (realized as codes for digital processors), including shape choices of the activation function using particular types of transistor implementation, and instabilities due to noise and other factors present in analog circuits. We present a computational study of the effects of these factors on NN performance and find that, while accuracy competitive with traditional NNs can be realized for many applications, there is high sensitivity to the instability in the shape of the activation function, suggesting that, when highly accurate NNs are required, high-precision circuitry should be developed beyond what has been reported for synaptic transistors to date.

Integrated workflows and interfaces for data-driven semi-empirical electronic structure calculations

Modern software engineering of electronic structure codes has seen a paradigm shift from monolithic workflows toward object-based modularity. Software objectivity allows for greater flexibility in the application of electronic structure calculations, with particular benefits when integrated with approaches for data-driven analysis. Here, we discuss different approaches to create deep modular interfaces that connect big-data workflows and electronic structure codes and explore the diversity of use cases that they can enable. We present two such interface approaches for the semi-empirical electronic structure package, DFTB+. In one case, DFTB+ is applied as a library and provides data to an external workflow; in another, DFTB+receives data via external bindings and processes the information subsequently within an internal workflow. We provide a general framework to enable data exchange workflows for embedding new machine-learning-based Hamiltonians within DFTB+ or enabling deep integration of DFTB+ in multiscale embedding workflows. These modular interfaces demonstrate opportunities in emergent software and workflows to accelerate scientific discovery by harnessing existing software capabilities.

In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back

Computational chemistry is an indispensable tool for understanding molecules and predicting chemical properties. However, traditional computational methods face significant challenges due to the difficulty of solving the Schrödinger equations and the increasing computational cost with the size of the molecular system. In response, there has been a surge of interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to in silico experiments. Integrating AI and ML into computational chemistry increases the scalability and speed of the exploration of chemical space. However, challenges remain, particularly regarding the reproducibility and transferability of ML models. This review highlights the evolution of ML in learning from, complementing, or replacing traditional computational chemistry for energy and property predictions. Starting from models trained entirely on numerical data, a journey set forth toward the ideal model incorporating or learning the physical laws of quantum mechanics. This paper also reviews existing computational methods and ML models and their intertwining, outlines a roadmap for future research, and identifies areas for improvement and innovation. Ultimately, the goal is to develop AI architectures capable of predicting accurate and transferable solutions to the Schrödinger equation, thereby revolutionizing in silico experiments within chemistry and materials science.

Predicting the One-Particle Density Matrix with Machine Learning

Two of the most widely used electronic-structure theory methods, namely, Hartree-Fock and Kohn-Sham density functional theory, require the iterative solution of a set of Schrödinger-like equations. The speed of convergence of such a process depends on the complexity of the system under investigation, the self-consistent-field algorithm employed, and the initial guess for the density matrix. An initial density matrix close to the ground-state matrix will effectively allow one to cut out many of the self-consistent steps necessary to achieve convergence. Here, we predict the density matrix of Kohn-Sham density functional theory by constructing a neural network that uses only the atomic positions as information. Such a neural network provides an initial guess for the density matrix far superior to that of any other recipes available. Furthermore, the quality of such a neural-network density matrix is good enough for the evaluation of interatomic forces. This allows us to run accelerated ab initio molecular dynamics with little to no self-consistent steps.

Development of scoring-assisted generative exploration (SAGE) and its application to dual inhibitor design for acetylcholinesterase and monoamine oxidase B

De novo molecular design is the process of searching chemical space for drug-like molecules with desired properties, and deep learning has been recognized as a promising solution. In this study, I developed an effective computational method called Scoring-Assisted Generative Exploration (SAGE) to enhance chemical diversity and property optimization through virtual synthesis simulation, the generation of bridged bicyclic rings, and multiple scoring models for drug-likeness. In six protein targets, SAGE generated molecules with high scores within reasonable numbers of steps by optimizing target specificity without a constraint and even with multiple constraints such as synthetic accessibility, solubility, and metabolic stability. Furthermore, I suggested a top-ranked molecule with SAGE as dual inhibitors of acetylcholinesterase and monoamine oxidase B through multiple desired property optimization. Therefore, SAGE can generate molecules with desired properties by optimizing multiple properties simultaneously, indicating the importance of de novo design strategies in the future of drug discovery and development. SCIENTIFIC CONTRIBUTION: The scientific contribution of this study lies in the development of the Scoring-Assisted Generative Exploration (SAGE) method, a novel computational approach that significantly enhances de novo molecular design. SAGE uniquely integrates virtual synthesis simulation, the generation of complex bridged bicyclic rings, and multiple scoring models to optimize drug-like properties comprehensively. By efficiently generating molecules that meet a broad spectrum of pharmacological criteria-including target specificity, synthetic accessibility, solubility, and metabolic stability-within a reasonable number of steps, SAGE represents a substantial advancement over traditional methods. Additionally, the application of SAGE to discover dual inhibitors for acetylcholinesterase and monoamine oxidase B not only demonstrates its potential to streamline and enhance the drug development process but also highlights its capacity to create more effective and precisely targeted therapies. This study emphasizes the critical and evolving role of de novo design strategies in reshaping the future of drug discovery and development, providing promising avenues for innovative therapeutic discoveries.

Machine-Learned Kohn-Sham Hamiltonian Mapping for Nonadiabatic Molecular Dynamics

In this work, we report a simple, efficient, and scalable machine-learning (ML) approach for mapping non-self-consistent Kohn-Sham Hamiltonians constructed with one kind of density functional to the nearly self-consistent Hamiltonians constructed with another kind of density functional. This approach is designed as a fast surrogate Hamiltonian calculator for use in long nonadiabatic dynamics simulations of large atomistic systems. In this approach, the input and output features are Hamiltonian matrices computed from different levels of theory. We demonstrate that the developed ML-based Hamiltonian mapping method (1) speeds up the calculations by several orders of magnitude, (2) is conceptually simpler than alternative ML approaches, (3) is applicable to different systems and sizes and can be used for mapping Hamiltonians constructed with arbitrary density functionals, (4) requires a modest training data, learns fast, and generates molecular orbitals and their energies with the accuracy nearly matching that of conventional calculations, and (5) when applied to nonadiabatic dynamics simulation of excitation energy relaxation in large systems yields the corresponding time scales within the margin of error of the conventional calculations. Using this approach, we explore the excitation energy relaxation in C60 fullerene and Si75H64 quantum dot structures and derive qualitative and quantitative insights into dynamics in these systems.

An Unsupervised Machine Learning Approach for the Automatic Construction of Local Chemical Descriptors

Condensing the many physical variables defining a chemical system into a fixed-size array poses a significant challenge in the development of chemical Machine Learning (ML). Atom Centered Symmetry Functions (ACSFs) offer an intuitive featurization approach by means of a tedious and labor-intensive selection of tunable parameters. In this work, we implement an unsupervised ML strategy relying on a Gaussian Mixture Model (GMM) to automatically optimize the ACSF parameters. GMMs effortlessly decompose the vastness of the chemical and conformational spaces into well-defined radial and angular clusters, which are then used to build tailor-made ACSFs. The unsupervised exploration of the space has demonstrated general applicability across a diverse range of systems, spanning from various unimolecular landscapes to heterogeneous databases. The impact of the sampling technique and temperature on space exploration is also addressed, highlighting the particularly advantageous role of high-temperature Molecular Dynamics (MD) simulations. The reliability of the resulting features is assessed through the estimation of the atomic charges of a prototypical capped amino acid and a heterogeneous collection of CHON molecules. The automatically constructed ACSFs serve as high-quality descriptors, consistently yielding typical prediction errors below 0.010 electrons bound for the reported atomic charges. Altering the spatial distribution of the functions with respect to the cluster highlights the critical role of symmetry rupture in achieving significantly improved features. More specifically, using two separate functions to describe the lower and upper tails of the cluster results in the best performing models with errors as low as 0.006 electrons. Finally, the effectiveness of finely tuned features was checked across different architectures, unveiling the superior performance of Gaussian Process (GP) models over Feed Forward Neural Networks (FFNNs), particularly in low-data regimes, with nearly a 2-fold increase in prediction quality. Altogether, this approach paves the way toward an easier construction of local chemical descriptors, while providing valuable insights into how radial and angular spaces should be mapped. Finally, this work opens the possibility of encoding many-body information beyond angular terms into upcoming ML features.

Nondestructive measurement of terahertz optical thin films by machine learning based on physical consistency

Optical scattering measurement is one of the most commonly used methods for non-contact online measurement of film properties in industrial film manufacturing. Terahertz photons have low energy and are non-ionizing when measuring objects, so combining these two methods can enable online nondestructive testing of thin films. In the visible light band, some materials are transparent, and their thickness and material properties cannot be measured. Therefore, a method based on physical consistency modeling and machine learning is proposed in this paper, which realizes the method of obtaining high-precision thin film parameters through single-frequency terahertz wave measurement, and shows good performance. Through the experimental measurement of organic material thin films, it is proved that the proposed method is an effective terahertz online detection technology with high precision and high throughput.

Biomolecular dynamics with machine-learned quantum-mechanical force fields trained on diverse chemical fragments

The GEMS method enables molecular dynamics simulations of large heterogeneous systems at ab initio quality.

Electronic Excited States from Physically Constrained Machine Learning

Data-driven techniques are increasingly used to replace electronic-structure calculations of matter. In this context, a relevant question is whether machine learning (ML) should be applied directly to predict the desired properties or combined explicitly with physically grounded operations. We present an example of an integrated modeling approach in which a symmetry-adapted ML model of an effective Hamiltonian is trained to reproduce electronic excitations from a quantum-mechanical calculation. The resulting model can make predictions for molecules that are much larger and more complex than those on which it is trained and allows for dramatic computational savings by indirectly targeting the outputs of well-converged calculations while using a parametrization corresponding to a minimal atom-centered basis. These results emphasize the merits of intertwining data-driven techniques with physical approximations, improving the transferability and interpretability of ML models without affecting their accuracy and computational efficiency and providing a blueprint for developing ML-augmented electronic-structure methods.

AI in computational chemistry through the lens of a decade-long journey

This article gives a perspective on the progress of AI tools in computational chemistry through the lens of the author's decade-long contributions put in the wider context of the trends in this rapidly expanding field. This progress over the last decade is tremendous: while a decade ago we had a glimpse of what was to come through many proof-of-concept studies, now we witness the emergence of many AI-based computational chemistry tools that are mature enough to make faster and more accurate simulations increasingly routine. Such simulations in turn allow us to validate and even revise experimental results, deepen our understanding of the physicochemical processes in nature, and design better materials, devices, and drugs. The rapid introduction of powerful AI tools gives rise to unique challenges and opportunities that are discussed in this article too.

Depicted simulation model for removal of second-generation antipsychotic drugs adsorbed on Zn-MOF: adsorption locator assessment

CONTEXT

Electronic durable behavior on the material surface was accompanied by a class of antipsychotic drugs (APD) to describe the surface modification in the designed adsorption model. Hierarchically Zn-MOF system was utilized for estimating its capacity for drug molecule removal. Geometrically optimized strategy on the studied systems was performed using DFT/GGA/PBE. FMOs analysis was depicted based on the same level of calculations, and molecular electrostatic potential surface (MEP) was generated for unadsorbed and adsorbed systems to illustrate the variation in the surface-active sites. By interpreting the electronic density of states (DOS), the atomic orbital can be identified as a major or minor electronic distribution by PDOS graph. Adsorption locating behavior was considered to detect the significant surface interaction mode between APD and Zn-MOF surface based on lower adsorption energy. The stability of the adsorbed model was best described through dynamic simulation analysis with time through elevated temperatures. The non-covalent interactions were described using RDG/NCI analysis to show the major favorable surface interaction predicting the highly stable adsorption system.

METHODS

The most accurate geometrical computations were performed using the materials studio software followed by surface cleavage and vacuum slab generation. The first principle of DFT was used to apply CASTEP module with GGA/PBE method for band structure and DOS calculations. Three systems of antipsychotic drugs were computationally studied using CASTEP simulation package and adsorbed on an optimized Zn-MOF surface. Adsorption locator module predicted the preferred adsorption mechanistic models, in which the first model was arranged to be more stable, to confirm the occurrence of some interactions in the adsorption mechanism.

Exploration of the Two-Electron Excitation Space with Data-Driven Coupled Cluster

Computational cost limits the applicability of post-Hartree-Fock methods such as coupled-cluster on larger molecular systems. The data-driven coupled-cluster (DDCC) method applies machine learning to predict the coupled-cluster two-electron amplitudes (t2) using data from second-order perturbation theory (MP2). One major limitation of the DDCC models is the size of training sets that increases exponentially with the system size. Effective sampling of the amplitude space can resolve this issue. Five different amplitude selection techniques that reduce the amount of data used for training were evaluated, an approach that also prevents model overfitting and increases the portability of data-driven coupled-cluster singles and doubles to more complex molecules or larger basis sets. In combination with a localized orbital formalism to predict the CCSD t2 amplitudes, we have achieved a 10-fold error reduction for energy calculations.

SPAHM(a,b): Encoding the Density Information from Guess Hamiltonian in Quantum Machine Learning Representations

Recently, we introduced a class of molecular representations for kernel-based regression methods─the spectrum of approximated Hamiltonian matrices (SPAHM)─that takes advantage of lightweight one-electron Hamiltonians traditionally used as a self-consistent field initial guess. The original SPAHM variant is built from occupied-orbital energies (i.e., eigenvalues) and naturally contains all of the information about nuclear charges, atomic positions, and symmetry requirements. Its advantages were demonstrated on data sets featuring a wide variation of charge and spin, for which traditional structure-based representations commonly fail. SPAHM(a,b), as introduced here, expand the eigenvalue SPAHM into local and transferable representations. They rely upon one-electron density matrices to build fingerprints from atomic and bond density overlap contributions inspired from preceding state-of-the-art representations. The performance and efficiency of SPAHM(a,b) is assessed on the predictions for data sets of prototypical organic molecules (QM7) of different charges and azoheteroarene dyes in an excited state. Overall, both SPAHM(a) and SPAHM(b) outperform state-of-the-art representations on difficult prediction tasks such as the atomic properties of charged open-shell species and of π-conjugated systems.

Employing neural density functionals to generate potential energy surfaces

CONTEXT

With the union of machine learning (ML) and quantum chemistry, amid the debate between machine-learned functionals and human-designed functionals in density functional theory (DFT), this paper aims to demonstrate the generation of potential energy surfaces using computations with machine-learned density functional approximation (ML-DFA). A recent research trend is the application of ML in quantum sciences in the design of density functionals such as DeepMind's Deep Learning model (DeepMind21, DM21). Though science reported the state-of-the-art performance of DM21, the opportunity to utilize DeepMind's pretrained DM21 neural networks in computations in quantum chemistry has not yet been tapped. So far in the literature, the Deep Learning density functionals (DM21) have not been applied to generate potential energy surfaces. While the superior accuracy of DM21 has been reported, there is still a scarcity of publications that apply DM21 in calculations in the field. In this context, for the first time in literature, neural density functionals inferring 2D potential energy surfaces (ML-DFA-PES) based on machine-learned DFA-based computational method is contributed in this paper. This paper reports the ML-DFA-generated PES for C4H8, H2O, H2, and H2+ by employing a pretrained DM21m TensorFlow model with cc-pVDZ basis set. In addition, we also analyze the long-range behavior of DM21 based PES to investigate the ability to describe a system at long ranges. Furthermore, we compare PES diagrams from DM21 with popular DFT functionals (b3lyp/ PW6B95) and CCSD(T).

METHODS

In this method, 2D potential energy surfaces are obtained using a method that relies upon the neural network's ability to accurately learn the mapping between 3D electron density and exchange-correlation potential. By inserting Deep Learning inference in DFT with a pretrained neural network, self-consistent field (SCF) energy at different geometries along the coordinates of interest is computed, and then, potential energy surfaces are plotted. In this method, first, the electron density is computed mathematically, and this computed 3D electron density is used as a ML feature vector to predict the exchange correlation potential as a ML inference computed by a forward pass of pre-trained DM21 TensorFlow computational graph, followed by the computation of self-consistent field energy at multiple geometries, and then, SCF energies at different bond lengths/angles are plotted as 2D PES. We implement this in a python source code using frameworks such as PySCF and DM21. This paper contributes this implementation in open source. The source code and DM21-DFA-based PES are contributed at https://sites.google.com/view/MLfunctionals-DeepMind-PES .

Improved machine learning algorithm for predicting ground state properties

Finding the ground state of a quantum many-body system is a fundamental problem in quantum physics. In this work, we give a classical machine learning (ML) algorithm for predicting ground state properties with an inductive bias encoding geometric locality. The proposed ML model can efficiently predict ground state properties of an n-qubit gapped local Hamiltonian after learning from only [Formula: see text] data about other Hamiltonians in the same quantum phase of matter. This improves substantially upon previous results that require [Formula: see text] data for a large constant c. Furthermore, the training and prediction time of the proposed ML model scale as [Formula: see text] in the number of qubits n. Numerical experiments on physical systems with up to 45 qubits confirm the favorable scaling in predicting ground state properties using a small training dataset.

All-Purpose Measure of Electron Correlation for Multireference Diagnostics

We present an analytical relationship between two natural orbital occupancy-based indices, I N D ¯ and INDmax, and two established electron correlation metrics: the leading term of a configuration interaction expansion, c0, and the D2 diagnostic. Numerical validation revealed that I N D ¯ and INDmax can effectively substitute for c0 and D2, respectively. These indices offer three distinct advantages: (i) they are universally applicable across all electronic structure methods, (ii) their interpretation is more intuitive, and (iii) they can be readily incorporated into the development of hybrid electronic structure methods. Additionally, we draw a distinction between correlation measures and correlation diagnostics, establishing MP2 and CCSD numerical thresholds for INDmax, which are to be used as a multireference diagnostic. Our findings further demonstrate that establishing thresholds for other electronic structure methods can be easily accomplished using small data sets.

Reinforcement Learning for Traversing Chemical Structure Space: Optimizing Transition States and Minimum Energy Paths of Molecules

In recent years, deep learning has made remarkable strides, surpassing human capabilities in tasks, such as strategy games, and it has found applications in complex domains, including protein folding. In the realm of quantum chemistry, machine learning methods have primarily served as predictive tools or design aids using generative models, while reinforcement learning remains in its early stages of exploration. This work introduces an actor-critic reinforcement learning framework suitable for diverse optimization tasks, such as searching for molecular structures with specific properties within conformational spaces. As an example, we show an implementation of this scheme for calculating minimum energy pathways of a Claisen rearrangement reaction and a number of SN2 reactions. The results show that the algorithm is able to accurately predict minimum energy pathways and, thus, transition states, providing the first steps in using actor-critic methods to study chemical reactions.

Two-Step Machine Learning Approach for Charge-Transfer Coupling with Structurally Diverse Data

Electronic coupling is important in determining charge-transfer rates and dynamics. Coupling strength is sensitive to both intermolecular, e.g., orientation or distance, and intramolecular degrees of freedom. Hence, it is challenging to build an accurate machine learning model to predict electronic coupling of molecular pairs, especially for those derived from the amorphous phase, for which intermolecular configurations are much more diverse than those derived from crystals. In this work, we devise a new prediction algorithm that employs two consecutive KRR models. The first model predicts molecular orbitals (MOs) from structural variation for each fragment, and coupling is further predicted by using the overlap integral included in a second model. With our two-step procedure, we achieved mean absolute errors of 0.27 meV for an ethylene dimer and 1.99 meV for a naphthalene pair, much improved accuracy amounting to 14-fold and 3-fold error reductions, respectively. In addition, MOs from the first model can also be the starting point to obtain other quantum chemical properties from atomistic structures. This approach is also compatible with a MO predictor with sufficient accuracy.

Machine Learning Many-Body Green's Functions for Molecular Excitation Spectra

We present a machine learning (ML) framework for predicting Green's functions of molecular systems, from which photoemission spectra and quasiparticle energies at quantum many-body level can be obtained. Kernel ridge regression is adopted to predict self-energy matrix elements on compact imaginary frequency grids from static and dynamical mean-field electronic features, which gives direct access to real-frequency many-body Green's functions through analytic continuation and Dyson's equation. Feature and self-energy matrices are represented in a symmetry-adapted intrinsic atomic orbital plus projected atomic orbital basis to enforce rotational invariance. We demonstrate good transferability and high data efficiency of the proposed ML method across molecular sizes and chemical species by showing accurate predictions of density of states (DOS) and quasiparticle energies at the level of many-body perturbation theory (GW) or full configuration interaction. For the ML model trained on 48 out of 1995 molecules randomly sampled from the QM7 and QM9 data sets, we report the mean absolute errors of ML-predicted highest occupied and lowest unoccupied molecular orbital energies to be 0.13 and 0.10 eV, respectively, compared to GW@PBE0. We further showcase the capability of this method by applying the same ML model to predict DOS for significantly larger organic molecules with up to 44 heavy atoms.

Towards a transferable fermionic neural wavefunction for molecules

Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.

Leveraging Deep Neural Networks for Estimating Vickers Hardness from Nanoindentation Hardness

This research presents a comprehensive analysis of deep neural network models (DNNs) for the precise prediction of Vickers hardness (HV) in nitrided and carburized M50NiL steel samples, with hardness values spanning from 400 to 1000 HV. By conducting rigorous experimentation and obtaining corresponding nanoindentation data, we evaluated the performance of four distinct neural network architectures: Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory network (LSTM), and Transformer. Our findings reveal that MLP and LSTM models excel in predictive accuracy and efficiency, with MLP showing exceptional iteration efficiency and predictive precision. The study validates models for broad application in various steel types and confirms nanoindentation as an effective direct measure for HV hardness in thin films and gradient-variable regions. This work contributes a validated and versatile approach to the hardness assessment of thin-film materials and those with intricate microstructures, enhancing material characterization and potential application in advanced material engineering.

From intuition to AI: evolution of small molecule representations in drug discovery

Within drug discovery, the goal of AI scientists and cheminformaticians is to help identify molecular starting points that will develop into safe and efficacious drugs while reducing costs, time and failure rates. To achieve this goal, it is crucial to represent molecules in a digital format that makes them machine-readable and facilitates the accurate prediction of properties that drive decision-making. Over the years, molecular representations have evolved from intuitive and human-readable formats to bespoke numerical descriptors and fingerprints, and now to learned representations that capture patterns and salient features across vast chemical spaces. Among these, sequence-based and graph-based representations of small molecules have become highly popular. However, each approach has strengths and weaknesses across dimensions such as generality, computational cost, inversibility for generative applications and interpretability, which can be critical in informing practitioners' decisions. As the drug discovery landscape evolves, opportunities for innovation continue to emerge. These include the creation of molecular representations for high-value, low-data regimes, the distillation of broader biological and chemical knowledge into novel learned representations and the modeling of up-and-coming therapeutic modalities.

Physics-Inspired Equivariant Descriptors of Nonbonded Interactions

One essential ingredient in many machine learning (ML) based methods for atomistic modeling of materials and molecules is the use of locality. While allowing better system-size scaling, this systematically neglects long-range (LR) effects such as electrostatic or dispersion interactions. We present an extension of the long distance equivariant (LODE) framework that can handle diverse LR interactions in a consistent way and seamlessly integrates with preexisting methods by building new sets of atom centered features. We provide a direct physical interpretation of these using the multipole expansion, which allows for simpler and more efficient implementations. The framework is applied to simple toy systems as proof of concept and a heterogeneous set of molecular dimers to push the method to its limits. By generalizing LODE to arbitrary asymptotic behaviors, we provide a coherent approach to treat arbitrary two- and many-body nonbonded interactions in the data-driven modeling of matter.

Machine learning electronic structure methods based on the one-electron reduced density matrix

The theorems of density functional theory (DFT) establish bijective maps between the local external potential of a many-body system and its electron density, wavefunction and, therefore, one-particle reduced density matrix. Building on this foundation, we show that machine learning models based on the one-electron reduced density matrix can be used to generate surrogate electronic structure methods. We generate surrogates of local and hybrid DFT, Hartree-Fock and full configuration interaction theories for systems ranging from small molecules such as water to more complex compounds like benzene and propanol. The surrogate models use the one-electron reduced density matrix as the central quantity to be learned. From the predicted density matrices, we show that either standard quantum chemistry or a second machine-learning model can be used to compute molecular observables, energies, and atomic forces. The surrogate models can generate essentially anything that a standard electronic structure method can, ranging from band gaps and Kohn-Sham orbitals to energy-conserving ab-initio molecular dynamics simulations and infrared spectra, which account for anharmonicity and thermal effects, without the need to employ computationally expensive algorithms such as self-consistent field theory. The algorithms are packaged in an efficient and easy to use Python code, QMLearn, accessible on popular platforms.

Automatic identification of chemical moieties

In recent years, the prediction of quantum mechanical observables with machine learning methods has become increasingly popular. Message-passing neural networks (MPNNs) solve this task by constructing atomic representations, from which the properties of interest are predicted. Here, we introduce a method to automatically identify chemical moieties (molecular building blocks) from such representations, enabling a variety of applications beyond property prediction, which otherwise rely on expert knowledge. The required representation can either be provided by a pretrained MPNN, or be learned from scratch using only structural information. Beyond the data-driven design of molecular fingerprints, the versatility of our approach is demonstrated by enabling the selection of representative entries in chemical databases, the automatic construction of coarse-grained force fields, as well as the identification of reaction coordinates.

Ab initio quantum chemistry with neural-network wavefunctions

Deep learning methods outperform human capabilities in pattern recognition and data processing problems and now have an increasingly important role in scientific discovery. A key application of machine learning in molecular science is to learn potential energy surfaces or force fields from ab initio solutions of the electronic Schrödinger equation using data sets obtained with density functional theory, coupled cluster or other quantum chemistry (QC) methods. In this Review, we discuss a complementary approach using machine learning to aid the direct solution of QC problems from first principles. Specifically, we focus on quantum Monte Carlo methods that use neural-network ansatzes to solve the electronic Schrödinger equation, in first and second quantization, computing ground and excited states and generalizing over multiple nuclear configurations. Although still at their infancy, these methods can already generate virtually exact solutions of the electronic Schrödinger equation for small systems and rival advanced conventional QC methods for systems with up to a few dozen electrons.

Langevin Cooling for Unsupervised Domain Translation

Domain translation is the task of finding correspondence between two domains. Several deep neural network (DNN) models, e.g., CycleGAN and cross-lingual language models, have shown remarkable successes on this task under the unsupervised setting-the mappings between the domains are learned from two independent sets of training data in both domains (without paired samples). However, those methods typically do not perform well on a significant proportion of test samples. In this article, we hypothesize that many of such unsuccessful samples lie at the fringe-relatively low-density areas-of data distribution, where the DNN was not trained very well, and propose to perform the Langevin dynamics to bring such fringe samples toward high-density areas. We demonstrate qualitatively and quantitatively that our strategy, called Langevin cooling (L-Cool), enhances state-of-the-art methods in image translation and language translation tasks.

Treating Semiempirical Hamiltonians as Flexible Machine Learning Models Yields Accurate and Interpretable Results

Quantum chemistry provides chemists with invaluable information, but the high computational cost limits the size and type of systems that can be studied. Machine learning (ML) has emerged as a means to dramatically lower the cost while maintaining high accuracy. However, ML models often sacrifice interpretability by using components such as the artificial neural networks of deep learning that function as black boxes. These components impart the flexibility needed to learn from large volumes of data but make it difficult to gain insight into the physical or chemical basis for the predictions. Here, we demonstrate that semiempirical quantum chemical (SEQC) models can learn from large volumes of data without sacrificing interpretability. The SEQC model is that of density-functional-based tight binding (DFTB) with fixed atomic orbital energies and interactions that are one-dimensional functions of the interatomic distance. This model is trained to ab initio data in a manner that is analogous to that used to train deep learning models. Using benchmarks that reflect the accuracy of the training data, we show that the resulting model maintains a physically reasonable functional form while achieving an accuracy, relative to coupled cluster energies with a complete basis set extrapolation (CCSD(T)*/CBS), that is comparable to that of density functional theory (DFT). This suggests that trained SEQC models can achieve a low computational cost and high accuracy without sacrificing interpretability. Use of a physically motivated model form also substantially reduces the amount of ab initio data needed to train the model compared to that required for deep learning models.

Synergy of semiempirical models and machine learning in computational chemistry

Catalyzed by enormous success in the industrial sector, many research programs have been exploring data-driven, machine learning approaches. Performance can be poor when the model is extrapolated to new regions of chemical space, e.g., new bonding types, new many-body interactions. Another important limitation is the spatial locality assumption in model architecture, and this limitation cannot be overcome with larger or more diverse datasets. The outlined challenges are primarily associated with the lack of electronic structure information in surrogate models such as interatomic potentials. Given the fast development of machine learning and computational chemistry methods, we expect some limitations of surrogate models to be addressed in the near future; nevertheless spatial locality assumption will likely remain a limiting factor for their transferability. Here, we suggest focusing on an equally important effort-design of physics-informed models that leverage the domain knowledge and employ machine learning only as a corrective tool. In the context of material science, we will focus on semi-empirical quantum mechanics, using machine learning to predict corrections to the reduced-order Hamiltonian model parameters. The resulting models are broadly applicable, retain the speed of semiempirical chemistry, and frequently achieve accuracy on par with much more expensive ab initio calculations. These early results indicate that future work, in which machine learning and quantum chemistry methods are developed jointly, may provide the best of all worlds for chemistry applications that demand both high accuracy and high numerical efficiency.

Thermodynamic and kinetic modeling of electrocatalytic reactions using a first-principles approach

The computational modeling of electrochemical interfaces and their applications in electrocatalysis has attracted great attention in recent years. While tremendous progress has been made in this area, however, the accurate atomistic descriptions at the electrode/electrolyte interfaces remain a great challenge. The Computational Hydrogen Electrode (CHE) method and continuum modeling of the solvent and electrolyte interactions form the basis for most of these methodological developments. Several posterior corrections have been added to the CHE method to improve its accuracy and widen its applications. The most recently developed grand canonical potential approaches with the embedded diffuse layer models have shown considerable improvement in defining interfacial interactions at electrode/electrolyte interfaces over the state-of-the-art computational models for electrocatalysis. In this Review, we present an overview of these different computational models developed over the years to quantitatively probe the thermodynamics and kinetics of electrochemical reactions in the presence of an electrified catalyst surface under various electrochemical environments. We begin our discussion by giving a brief picture of the different continuum solvation approaches, implemented within the ab initio method to effectively model the solvent and electrolyte interactions. Next, we present the thermodynamic and kinetic modeling approaches to determine the activity and stability of the electrocatalysts. A few applications to these approaches are also discussed. We conclude by giving an outlook on the different machine learning models that have been integrated with the thermodynamic approaches to improve their efficiency and widen their applicability.

Combining Deep Learning Neural Networks with Genetic Algorithms to Map Nanocluster Configuration Spaces with Quantum Accuracy at Low Computational Cost

The configuration spaces for bimetallic AuPd nanoclusters of various sizes are explored efficiently and analyzed accurately by combining genetic algorithms with neural networks trained on density functional theory. The methodology demonstrated herein provides an optimizable solution to the problem of searching vast configuration spaces with quantum accuracy in a way that is computationally practical. We implement a machine learning algorithm which learns the density functional theory potential with increasing performance while simultaneously generating and relaxing structures within the system's global configuration space unbiasedly. As a result, the algorithm naturally converges onto the system's energy minima while mapping the configuration space as a function of energy. The algorithm's simple design applies not only to nanocluster configurations, as demonstrated, but to bulk, substrate, and adsorption sites as well, and it is designed to scale. To demonstrate its computational efficiency, we work with AuPd nanoclusters of sizes 15, 20, and 25 atoms. Results focus primarily on evaluating the algorithm's performance; however, several physical insights into possible configurations for these nanoclusters naturally emerge as well, such as geometric Au surface segregation and stoichiometric Au minimization as a function of stability.

Material symmetry recognition and property prediction accomplished by crystal capsule representation

Learning the global crystal symmetry and interpreting the equivariant information is crucial for accurately predicting material properties, yet remains to be fully accomplished by existing algorithms based on convolution networks. To overcome this challenge, here we develop a machine learning (ML) model, named symmetry-enhanced equivariance network (SEN), to build material representation with joint structure-chemical patterns, to encode important clusters embedded in the crystal structure, and to learn pattern equivariance in different scales via capsule transformers. Quantitative analyses of the intermediate matrices demonstrate that the intrinsic crystal symmetries and interactions between clusters have been exactly perceived by the SEN model and critically affect the prediction performances by reducing effective feature space. The mean absolute errors (MAEs) of 0.181 eV and 0.0161 eV/atom are obtained for predicting bandgap and formation energy in the MatBench dataset. The general and interpretable SEN model reveals the potential to design ML models by implicitly encoding feature relationship based on physical mechanisms.

Low-Data Deep Quantum Chemical Learning for Accurate MP2 and Coupled-Cluster Correlations

Accurate ab initio prediction of electronic energies is very expensive for macromolecules by explicitly solving post-Hartree-Fock equations. We here exploit the physically justified local correlation feature in a compact basis of small molecules and construct an expressive low-data deep neural network (dNN) model to obtain machine-learned electron correlation energies on par with MP2 and CCSD levels of theory for more complex molecules and different datasets that are not represented in the training set. We show that our dNN-powered model is data efficient and makes highly transferable predictions across alkanes of various lengths, organic molecules with non-covalent and biomolecular interactions, as well as water clusters of different sizes and morphologies. In particular, by training 800 (H2O)8 clusters with the local correlation descriptors, accurate MP2/cc-pVTZ correlation energies up to (H2O)128 can be predicted with a small random error within chemical accuracy from exact values, while a majority of prediction deviations are attributed to an intrinsically systematic error. Our results reveal that an extremely compact local correlation feature set, which is poor for any direct post-Hartree-Fock calculations, has however a prominent advantage in reserving important electron correlation patterns for making accurate transferable predictions across distinct molecular compositions, bond types, and geometries.

Machine Learning Frontier Orbital Energies of Nanodiamonds

Nanodiamonds have a wide range of applications including catalysis, sensing, tribology, and biomedicine. To leverage nanodiamond design via machine learning, we introduce the new data set ND5k, consisting of 5089 diamondoid and nanodiamond structures and their frontier orbital energies. ND5k structures are optimized via tight-binding density functional theory (DFTB) and their frontier orbital energies are computed using density functional theory (DFT) with the PBE0 hybrid functional. From this data set we derive a qualitative design suggestion for nanodiamonds in photocatalysis. We also compare recent machine learning models for predicting frontier orbital energies for similar structures as they have been trained on (interpolation on ND5k), and we test their abilities to extrapolate predictions to larger structures. For both the interpolation and extrapolation task, we find the best performance using the equivariant message passing neural network PaiNN. The second best results are achieved with a message passing neural network using a tailored set of atomic descriptors proposed here.

The central role of density functional theory in the AI age

Density functional theory (DFT) plays a pivotal role in chemical and materials science because of its relatively high predictive power, applicability, versatility, and computational efficiency. We review recent progress in machine learning (ML) model developments, which have relied heavily on DFT for synthetic data generation and for the design of model architectures. The general relevance of these developments is placed in a broader context for chemical and materials sciences. DFT-based ML models have reached high efficiency, accuracy, scalability, and transferability and pave the way to the routine use of successful experimental planning software within self-driving laboratories.

General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian

The combination of deep learning and ab initio calculation has shown great promise in revolutionizing future scientific research, but how to design neural network models incorporating a priori knowledge and symmetry requirements is a key challenging subject. Here we propose an E(3)-equivariant deep-learning framework to represent density functional theory (DFT) Hamiltonian as a function of material structure, which can naturally preserve the Euclidean symmetry even in the presence of spin-orbit coupling. Our DeepH-E3 method enables efficient electronic structure calculation at ab initio accuracy by learning from DFT data of small-sized structures, making the routine study of large-scale supercells (>104 atoms) feasible. The method can reach sub-meV prediction accuracy at high training efficiency, showing state-of-the-art performance in our experiments. The work is not only of general significance to deep-learning method development but also creates opportunities for materials research, such as building a Moiré-twisted material database.

Physics-inspired machine learning of localized intensive properties

Machine learning (ML) has been widely applied to chemical property prediction, most prominently for the energies and forces in molecules and materials. The strong interest in predicting energies in particular has led to a 'local energy'-based paradigm for modern atomistic ML models, which ensures size-extensivity and a linear scaling of computational cost with system size. However, many electronic properties (such as excitation energies or ionization energies) do not necessarily scale linearly with system size and may even be spatially localized. Using size-extensive models in these cases can lead to large errors. In this work, we explore different strategies for learning intensive and localized properties, using HOMO energies in organic molecules as a representative test case. In particular, we analyze the pooling functions that atomistic neural networks use to predict molecular properties, and suggest an orbital weighted average (OWA) approach that enables the accurate prediction of orbital energies and locations.

Harnessing Data Augmentation and Normalization Preprocessing to Improve the Performance of Chemical Reaction Predictions of Data-Driven Model

As a template-free, data-driven methodology, the molecular transformer model provides an alternative by which to predict the outcome of chemical reactions and design the route of the retrosynthetic plane in the field of organic synthesis and polymer chemistry. However, in consideration of the small datasets of chemical reactions, the data-driven model suffers from the difficulty of low accuracy in the prediction tasks of chemical reactions. In this contribution, we integrate the molecular transformer model with the strategies of data augmentation and normalization preprocessing to accomplish the three tasks of chemical reactions, including the forward predictions of chemical reactions, and single-step retrosynthetic predictions with and without the reaction classes. It is clearly demonstrated that the prediction accuracy of the molecular transformer model can be significantly raised by the use of proposed strategies for the three tasks of chemical reactions. Notably, after the introduction of the 40-level data augmentation and normalization preprocessing, the top-1 accuracy of the forward prediction increases markedly from 71.6% to 84.2% and the top-1 accuracy of the single-step retrosynthetic prediction with additional reaction class increases from 53.2% to 63.4%. Furthermore, it is found that the superior performance of the data-driven model originates from the correction of the grammatical errors of the SMILES strings, especially for the case of the reaction classes with small datasets.

SchNetPack 2.0: A neural network toolbox for atomistic machine learning

SchNetPack is a versatile neural network toolbox that addresses both the requirements of method development and the application of atomistic machine learning. Version 2.0 comes with an improved data pipeline, modules for equivariant neural networks, and a PyTorch implementation of molecular dynamics. An optional integration with PyTorch Lightning and the Hydra configuration framework powers a flexible command-line interface. This makes SchNetPack 2.0 easily extendable with a custom code and ready for complex training tasks, such as the generation of 3D molecular structures.

Deep-learning electronic-structure calculation of magnetic superstructures

Ab initio studies of magnetic superstructures are indispensable to research on emergent quantum materials, but are currently bottlenecked by the formidable computational cost. Here, to break this bottleneck, we have developed a deep equivariant neural network framework to represent the density functional theory Hamiltonian of magnetic materials for efficient electronic-structure calculation. A neural network architecture incorporating a priori knowledge of fundamental physical principles, especially the nearsightedness principle and the equivariance requirements of Euclidean and time-reversal symmetries ([Formula: see text]), is designed, which is critical to capture the subtle magnetic effects. Systematic experiments on spin-spiral, nanotube and moiré magnets were performed, making the challenging study of magnetic skyrmions feasible.