Machine Learning Methods to Predict Density Functional Theory B3LYP Energies of HOMO and LUMO Orbitals

Enhancing Carbon Acid pKa Prediction by Augmentation of Sparse Experimental Datasets with Accurate AIBL (QM) Derived Values

The prediction of the aqueous pK_a of carbon acids by Quantitative Structure Property Relationship or cheminformatics-based methods is a rather arduous problem. Primarily, there are insufficient high-quality experimental data points measured in homogeneous conditions to allow for a good global model to be generated. In our computationally efficient pK_a prediction method, we generate an atom-type feature vector, called a distance spectrum, from the assigned ionisation atom, and learn coefficients for those atom-types that show the impact each atom-type has on the pK_a of the ionisable centre. In the current work, we augment our dataset with pK_a values from a series of high performing local models derived from the Ab Initio Bond Lengths method (AIBL). We find that, in distilling the knowledge available from multiple models into one general model, the prediction error for an external test set is reduced compared to that using literature experimental data alone.

Transfer learning with graph neural networks for optoelectronic properties of conjugated oligomers

Despite the remarkable progress of machine learning (ML) techniques in chemistry, modeling the optoelectronic properties of long conjugated oligomers and polymers with ML remains challenging due to the difficulty in obtaining sufficient training data. Here, we use transfer learning to address the data scarcity issue by pre-training graph neural networks using data from short oligomers. With only a few hundred training data, we are able to achieve an average error of about 0.1 eV for the excited-state energy of oligothiophenes against time-dependent density functional theory (TDDFT) calculations. We show that the success of our transfer learning approach relies on the relative locality of low-lying electronic excitations in long conjugated oligomers. Finally, we demonstrate the transferability of our approach by modeling the lowest-lying excited-state energies of poly(3-hexylthiophene) in its single-crystal and solution phases using the transfer learning models trained with the data of gas-phase oligothiophenes. The transfer learning predicted excited-state energy distributions agree quantitatively with TDDFT calculations and capture some important qualitative features observed in experimental absorption spectra.

Big data and machine learning for materials science

Herein, we review aspects of leading-edge research and innovation in materials science that exploit big data and machine learning (ML), two computer science concepts that combine to yield computational intelligence. ML can accelerate the solution of intricate chemical problems and even solve problems that otherwise would not be tractable. However, the potential benefits of ML come at the cost of big data production; that is, the algorithms demand large volumes of data of various natures and from different sources, from material properties to sensor data. In the survey, we propose a roadmap for future developments with emphasis on computer-aided discovery of new materials and analysis of chemical sensing compounds, both prominent research fields for ML in the context of materials science. In addition to providing an overview of recent advances, we elaborate upon the conceptual and practical limitations of big data and ML applied to materials science, outlining processes, discussing pitfalls, and reviewing cases of success and failure.

Dipole Moment Variation Clears Up Electronic Excitations in the π-Stacked Complexes of Fluorescent Protein Chromophores

We propose a quantitative structure-property relationship (QSPR) model for prediction of spectral tuning in cyan, green, orange, and red fluorescent proteins, which are engineered by motifs of the green fluorescent protein. Protein variants, in which their chromophores are involved in the π-stacking interaction with amino acid residues tyrosine, phenylalanine, and histidine, are prospective markers useful in bioimaging and super-resolution microscopy. In this work, we constructed training sets of the π-stacked complexes of four fluorescent protein chromophores (of the green, orange, red, and cyan series) with various substituted benzenes and imidazoles and tested the use of dipole moment variation upon excitation (DMV) as a descriptor to evaluate the vertical excitation energies in these systems. To validate this approach, we computed and analyzed electron density distributions of the π-stacked complexes and correlated the QSPR predictions with the reference values of the transition energies obtained using the high-level ab initio quantum chemistry methods. According to our results, the use of the DMV descriptor allows one to predict excitation energies in the π-stacked complexes with errors not exceeding 0.1 eV, which makes this model a practically useful tool in the development of efficient fluorescent markers for in vivo imaging.

Deep Learning Total Energies and Orbital Energies of Large Organic Molecules Using Hybridization of Molecular Fingerprints

The ability to predict material properties without the need for resource-consuming experimental efforts can immensely accelerate material and drug discovery. Although ab initio methods can be reliable and accurate in making such predictions, they are computationally too expensive on a large scale. The recent advancements in artificial intelligence and machine learning as well as the availability of large quantum mechanics derived datasets enable us to train models on these datasets as a benchmark and to make fast predictions on much larger datasets. The success of these machine learning models highly depends on the machine-readable fingerprints of the molecules that capture their chemical properties as well as topological information. In this work, we propose a common deep learning-based framework to combine different types of molecular fingerprints to enhance prediction accuracy. A graph neural network (GNN), many-body tensor representation (MBTR), and a set of simple molecular descriptors (MD) were used to predict the total energies, highest occupied molecular orbital (HOMO) energies, and lowest unoccupied molecular orbital (LUMO) energies of a dataset containing ∼62k large organic molecules with complex aromatic rings and remarkably diverse functional groups. The results demonstrate that a combination of best performing molecular fingerprints can produce better results than the individual ones. The simple and flexible deep learning framework developed in this work can be easily adapted to incorporate other types of molecular fingerprints.

Predicting the DNA Conductance Using a Deep Feedforward Neural Network Model

Double-stranded DNA (dsDNA) has been established as an efficient medium for charge migration, bringing it to the forefront of the field of molecular electronics and biological research. The charge migration rate is controlled by the electronic couplings between the two nucleobases of DNA/RNA. These electronic couplings strongly depend on the intermolecular geometry and orientation. Estimating these electronic couplings for all the possible relative geometries of molecules using the computationally demanding first-principles calculations requires a lot of time and computational resources. In this article, we present a machine learning (ML)-based model to calculate the electronic coupling between any two bases of dsDNA/dsRNA and bypass the computationally expensive first-principles calculations. Using the Coulomb matrix representation which encodes the atomic identities and coordinates of the DNA base pairs to prepare the input dataset, we train a feedforward neural network model. Our neural network (NN) model can predict the electronic couplings between dsDNA base pairs with any structural orientation with a mean absolute error (MAE) of less than 0.014 eV. We further use the NN-predicted electronic coupling values to compute the dsDNA/dsRNA conductance.

PubChemQC PM6: Data Sets of 221 Million Molecules with Optimized Molecular Geometries and Electronic Properties

We report on optimized molecular geometries and electronic properties calculated by the PM6 method for 94.0% of the 91.6 million molecules cataloged in PubChem Compounds retrieved on August 29, 2016. In addition to neutral states, we also calculated those for cationic, anionic, and spin flipped electronic states of 56.2%, 49.7%, and 41.3% of the molecules, respectively. Thus, the grand total of the PM6 calculations amounted to 221 million. We compared the resulting molecular geometries with B3LYP/6-31G* optimized geometries for 2.6 million molecules. The root-mean-square deviations in bond length and bond angle were approximately 0.016 Å and 1.7°, respectively. Then, using linear regression to examine the HOMO energy levels E(HOMO) in the B3LYP and PM6 calculations, we found that E_B3LYP(HOMO) = 0.876E_PM6(HOMO) + 1.975 (eV) and calculated the coefficient of determination to be 0.803. Likewise, we examined the LUMO energy levels and found E_B3LYP(LUMO) = 1.069E_PM6(LUMO) - 0.420 (eV); the coefficient of determination was 0.842. We also generated four subdata sets, each of which was composed of molecules with molecular weights less than 500. Subdata set i contained C, H, O and N, ii contained C, H, N, O, P, and S, iii contained C, H, N, O, P, S, F, and Cl, and iv contained C, H, N, O, P, S, F, Cl, Na, K, Mg, and Ca. The data sets are available at http://pubchemqc.riken.jp/pm6_datasets.html under a Creative Commons Attribution 4.0 International license.

Using Machine Learning to Predict the Dissociation Energy of Organic Carbonyls

Bond dissociation energy (BDE), an indicator of the strength of chemical bonds, exhibits great potential for evaluating and screening high-performance materials and catalysts, which are of critical importance in industrial applications. However, the measurement or computation of BDE via conventional experimental or theoretical methods is usually costly and involved, substantially preventing the BDE from being applied to large-scale and high-throughput studies. Therefore, a potentially more efficient approach for estimating BDE is highly desirable. To this end, we combined first-principles calculations and machine learning techniques, including neural networks and random forest, to explore the inner relationships between carbonyl structure and its BDE. Results show that machine learning can not only effectively reproduce the computed BDEs of carbonyls but also in turn serve as guidance for the rational design of carbonyl structure aimed at optimizing performance.

Antimicrobial Isoflavones and Derivatives from Erythrina (Fabaceae): Structure Activity Perspective (Sar & Qsar) on Experimental and Mined Values Against Staphylococcus Aureus

Prenylated (iso)flavonoids, -flavans and pterocarpans from taxa in Erythrina are repeatedly flagged as potent antimicrobial compounds. In the current study, bark from E. lysistemon was extracted and seven isoflavone derivatives were purified: erybraedin A (1), phaseollidin (2), abyssinone V-4′ methyl ether (3), eryzerin C (4), alpumisoflavone (5), cristacarpin (6) and lysisteisoflavone (7). Minimum inhibition concentration (MIC) values were determined against a range of species of bacteria (skin pathogens), then values for another 67 derivatives from Erythrina, only against Staphylococcus aureus, were mined from the literature. Of the seven isolates, MIC values widely ranged from 1–600 μg/mL, with no obvious pattern of selectivity for Gram-types. Nevertheless, using the mined and experimentally determined values against S. aureus, Klekota-Roth fragments (Structure Activity Relationship: SAR) were determined then used as molecular descriptors to make a ‘decision tree’ based on structural characters inspired by the classes of antimicrobial potency (classes A-D). Furthermore, to make quantitative predictions of MIC values (Quantitative SAR: QSAR) ‘pace regression’ was utilized and validated (R² = 0.778, Q² = 0.727 and P² = 0.555). Evidently, the position and degree of prenylation is important; however, the presence of hydroxyl groups at positions 5 and 7 in ring A and 4′ in ring B is associated with lower MIC values. While antimicrobial results continue to validate the traditional use of E. lysistemon extracts (or Erythrina generally) in therapeutic applications consistent with anti-infection, it is surprising that this class of compound is not being utilized more often in general industry applications, such as food or cosmetic preservation, or in topical antimicrobial creams. Prenylated (iso)flavonoids are derived from several other Genera, such as Dorstenia (Moraceae), Ficus (Moraceae), Glycyrrhiza (Fabaceae), Paulownia (Lamiales) or Pomifera (Moraceae).

Spectral deep learning for prediction and prospective validation of functional groups

State-of-the-art identification of the functional groups present in an unknown chemical entity requires the expertise of a skilled spectroscopist to analyse and interpret Fourier transform infra-red (FTIR), mass spectroscopy (MS) and/or nuclear magnetic resonance (NMR) data. This process can be time-consuming and error-prone, especially for complex chemical entities that are poorly characterised in the literature, or inefficient to use with synthetic robots producing molecules at an accelerated rate. Herein, we introduce a fast, multi-label deep neural network for accurately identifying all the functional groups of unknown compounds using a combination of FTIR and MS spectra. We do not use any database, pre-established rules, procedures, or peak-matching methods. Our trained neural network reveals patterns typically used by human chemists to identify standard groups. Finally, we experimentally validated our neural network, trained on single compounds, to predict functional groups in compound mixtures. Our methodology showcases practical utility for future use in autonomous analytical detection.

What Does the Machine Learn? Knowledge Representations of Chemical Reactivity

In a departure from conventional chemical approaches, data-driven models of chemical reactions have recently been shown to be statistically successful using machine learning. These models, however, are largely black box in character and have not provided the kind of chemical insights that historically advanced the field of chemistry. To examine the knowledgebase of machine-learning models-what does the machine learn-this article deconstructs black-box machine-learning models of a diverse chemical reaction data set. Through experimentation with chemical representations and modeling techniques, the analysis provides insights into the nature of how statistical accuracy can arise, even when the model lacks informative physical principles. By peeling back the layers of these complicated models we arrive at a minimal, chemically intuitive model (and no machine learning involved). This model is based on systematic reaction-type classification and Evans-Polanyi relationships within reaction types which are easily visualized and interpreted. Through exploring this simple model, we gain deeper understanding of the data set and uncover a means for expert interactions to improve the model's reliability.

A Multiple Filter Based Neural Network Approach to the Extrapolation of Adsorption Energies on Metal Surfaces for Catalysis Applications

Computational catalyst discovery involves the development of microkinetic reactor models based on estimated parameters determined from density functional theory (DFT). For complex surface chemistries, the number of reaction intermediates can be very large, and the cost of calculating the adsorption energies by DFT for all surface intermediates even for one active site model can become prohibitive. In this paper, we have identified appropriate descriptors and machine learning models that can be used to predict a significant part of these adsorption energies given data on the rest of them. Moreover, our investigations also included the case when the species data used to train the predictive model are of different size relative to the species the model tries to predict-this is an extrapolation in the data space which is typically difficult with regular machine learning models. Due to the relative size of the available data sets, we have attempted to extrapolate from the larger species to the smaller ones in the current work. Here, we have developed a neural network based predictive model that combines an established additive atomic contribution based model with the concepts of a convolutional neural network that, when extrapolating, achieves a statistically significant improvement over the previous models.

Dataset's chemical diversity limits the generalizability of machine learning predictions

The QM9 dataset has become the golden standard for Machine Learning (ML) predictions of various chemical properties. QM9 is based on the GDB, which is a combinatorial exploration of the chemical space. ML molecular predictions have been recently published with an accuracy on par with Density Functional Theory calculations. Such ML models need to be tested and generalized on real data. PC9, a new QM9 equivalent dataset (only H, C, N, O and F and up to 9 “heavy” atoms) of the PubChemQC project is presented in this article. A statistical study of bonding distances and chemical functions shows that this new dataset encompasses more chemical diversity. Kernel Ridge Regression, Elastic Net and the Neural Network model provided by SchNet have been used on both datasets. The overall accuracy in energy prediction is higher for the QM9 subset. However, a model trained on PC9 shows a stronger ability to predict energies of the other dataset.

Hammett neural networks: prediction of frontier orbital energies of tungsten-benzylidyne photoredox complexes

The successful application of Hammett parameters as input features for regressive machine learning models is demonstrated and applied to predict energies of frontier orbitals of highly reducing tungsten–alkylidyne complexes of the form W( Created by potrace 1.16, written by Peter Selinger 2001-2019 ]]> CArR)L₄X.

The successful application of Hammett parameters as input features for regressive machine learning models is demonstrated and applied to predict energies of frontier orbitals of highly reducing tungsten–benzylidyne complexes of the form W( Created by potrace 1.16, written by Peter Selinger 2001-2019 ]]> CArR)L₄X. Using a reference molecular framework and the meta- and para-substituent Hammett parameters of the ligands, the models predict energies of frontier orbitals that correlate with redox potentials. The regressive models capture the multivariate character of electron-donating trends as influenced by multiple substituents even for non-aryl ligands, harnessing the breadth of Hammett parameters in a generalized model. We find a tungsten catalyst with tetramethylethylenediamine (tmeda) equatorial ligands and axial methoxyl substituents that should attract significant experimental interest since it is predicted to be highly reducing when photoactivated with visible light. The utilization of Hammett parameters in this study presents a generalizable and compact representation for exploring the effects of ligand substitutions.

Deep Learning Spectroscopy: Neural Networks for Molecular Excitation Spectra

Deep learning methods for the prediction of molecular excitation spectra are presented. For the example of the electronic density of states of 132k organic molecules, three different neural network architectures: multilayer perceptron (MLP), convolutional neural network (CNN), and deep tensor neural network (DTNN) are trained and assessed. The inputs for the neural networks are the coordinates and charges of the constituent atoms of each molecule. Already, the MLP is able to learn spectra, but the root mean square error (RMSE) is still as high as 0.3 eV. The learning quality improves significantly for the CNN (RMSE = 0.23 eV) and reaches its best performance for the DTNN (RMSE = 0.19 eV). Both CNN and DTNN capture even small nuances in the spectral shape. In a showcase application of this method, the structures of 10k previously unseen organic molecules are scanned and instant spectra predictions are obtained to identify molecules for potential applications.

Machine learning for the prediction of molecular dipole moments obtained by density functional theory

Machine learning (ML) algorithms were explored for the fast estimation of molecular dipole moments calculated by density functional theory (DFT) by B3LYP/6-31G(d,p) on the basis of molecular descriptors generated from DFT-optimized geometries and partial atomic charges obtained by empirical or ML schemes. A database was used with 10,071 structures, new molecular descriptors were designed and the models were validated with external test sets. Several ML algorithms were screened. Random forest regression models predicted an external test set of 3368 compounds achieving mean absolute error up to 0.44 D. The results represent a significant improvement of the dipole moments calculated using empirical point charges located at the nucleus, even assuming the DFT-optimized geometry (root mean square error, RMSE, of 0.68 D vs. 1.53 D and R² = 0.87 vs. 0.66).

Predictive Models for HOMO and LUMO Energies of N-Donor Heterocycles as Ligands for Lanthanides Separation

Quantum chemical calculations combined with QSPR methodology reveal challenging perspectives for the solution of a number of fundamental and applied problems. In this work, we performed the PM7 and DFT calculations and QSPR modeling of HOMO and LUMO energies for polydentate N-heterocyclic ligands promising for the extraction separation of lanthanides because these values are related to the ligands selectivity in the respect to the target cations. Data for QSPR modeling comprised the PM7 calculated HOMO and LUMO energies of N-donor heterocycles, including several types of both known and virtual undescribed polydentate ligands. Ensemble modeling included various molecular fragments as descriptors and different variable selection techniques to build consensus models (CMs) on a training set of 388 ligands using external cross-validation. CMs were then verified to make predictions for two external test sets: 45 ligands (T1) that were similar to the ligands of the training set, and 1546 structures (T2), which were substantially different from the ligands of the training set. The consensus models predict well in 5-fold cross-validation (RMSE_HOMO =0.097 eV, RMSE_LUMO =0.064 eV), and on the external test sets (T1: RMSE_HOMO =0.26 eV, RMSE_LUMO =0.24 eV; T2: RMSE_HOMO =0.26 eV, RMSE_LUMO =0.17 eV). An analysis of the results reveals that substituents in heteroaromatic rings of the ligands and at the amide nitrogens can deeply influence their metal binding properties.

Data Mining for Parameters Affecting Polymorph Selection in Contorted Hexabenzocoronene Derivatives

The macroscopic properties of molecular materials can be drastically influenced by their solid-state packing arrangements, of which there can be many (e.g., polymorphism). Strategies to controllably and predictively access select polymorphs are thus highly desired, but computationally predicting the conditions necessary to access a given polymorph is challenging with the current state of the art. Using derivatives of contorted hexabenzocoronene, cHBC, we employed data mining, rather than first-principles approaches, to find relationships between the crystallizing molecule, postdeposition solvent-vapor annealing conditions that induce polymorphic transformation, and the resulting polymorphs. This analysis yields a correlative function that can be used to successfully predict the appearance of either one of two polymorphs in thin films of cHBC derivatives. Within the postdeposition processing phase space of cHBC derivatives, we have demonstrated an approach to generate guidelines to select crystallization conditions to bias polymorph access. We believe this approach can be applied more broadly to accelerate the predictions of processing conditions to access desired molecular polymorphs, making progress toward one of the grand challenges identified by the Materials Genome Initiative.

Computational Methods for Ab Initio Molecular Dynamics

Ab initio molecular dynamics is an irreplaceable technique for the realistic simulation of complex molecular systems and processes from first principles. This paper proposes a comprehensive and self-contained review of ab initio molecular dynamics from a computational perspective and from first principles. Quantum mechanics is presented from a molecular dynamics perspective. Various approximations and formulations are proposed, including the Ehrenfest, Born–Oppenheimer, and Hartree–Fock molecular dynamics. Subsequently, the Kohn–Sham formulation of molecular dynamics is introduced as well as the afferent concept of density functional. As a result, Car–Parrinello molecular dynamics is discussed, together with its extension to isothermal and isobaric processes. Car–Parrinello molecular dynamics is then reformulated in terms of path integrals. Finally, some implementation issues are analysed, namely, the pseudopotential, the orbital functional basis, and hybrid molecular dynamics.

Empirical Classification of Trajectory Data: An Opportunity for the Use of Machine Learning in Molecular Dynamics

Classical Hamiltonian trajectories are initiated at random points in phase space on a fixed energy shell of a model two degrees of freedom potential, consisting of two interacting minima in an otherwise flat energy plane of infinite extent. Below the energy of the plane, the dynamics are demonstrably chaotic. However, most of the work in this paper involves trajectories at a fixed energy that is 1% above that of the plane, in which regime the dynamics exhibit behavior characteristic of chaotic scattering. The trajectories are analyzed without reference to the potential, as if they had been generated in a typical direct molecular dynamics simulation. The questions addressed are whether one can recover useful information about the structures controlling the dynamics in phase space from the trajectory data alone, and whether, despite the at least partially chaotic nature of the dynamics, one can make statistically meaningful predictions of trajectory outcomes from initial conditions. It is found that key unstable periodic orbits, which can be identified on the analytical potential, appear by simple classification of the trajectories, and that the specific roles of these periodic orbits in controlling the dynamics are also readily discerned from the trajectory data alone. Two different approaches to predicting trajectory outcomes from initial conditions are evaluated, and it is shown that the more successful of them has ∼90% success. The results are compared with those from a simple neural network, which has higher predictive success (97%) but requires the information obtained from the "by-hand" analysis to achieve that level. Finally, the dynamics, which occur partly on the very flat region of the potential, show characteristics of the much-studied phenomenon called "roaming." On this potential, it is found that roaming trajectories are effectively "failed" periodic orbits and that angular momentum can be identified as a key controlling factor, despite the fact that it is not a strictly conserved quantity. It is also noteworthy that roaming on this potential occurs in the absence of a "roaming saddle," which has previously been hypothesized to be a necessary feature for roaming to occur.