Future directions of chemical theory and computation

Theoretical and computational chemistry aims to develop chemical theory and to apply numerical computation and simulation to reveal the mechanism behind complex chemical phenomena via quantum theory and statistical mechanics. Computation is the third pillar of scientific research together with theory and experiment. Computation enables scientists to test, discover, and build models/theories of the corresponding chemical phenomena. Theoretical and computational chemistry has been advanced to a new era due to the development of high-performance computational facilities and artificial intelligence approaches. The tendency to merge electronic structural theory with quantum chemical dynamics and statistical mechanics is of increasing interest because of the rapid development of on-the-fly dynamic simulations for complex systems plus low-scaling electronic structural theory. Another challenging issue lies in the transition from order to disorder, from thermodynamics to dynamics, and from equilibrium to non-equilibrium. Despite an increasingly rapid emergence of advances in computational power, detailed criteria for databases, effective data sharing strategies, and deep learning workflows have yet to be developed. Here, we outline some challenges and limitations of the current artificial intelligence approaches with an outlook on the potential future directions for chemistry in the big data era.

Efficient hyperparameter tuning for kernel ridge regression with Bayesian optimization

Machine learning methods usually depend on internal parameters—so called hyperparameters—that need to be optimized for best performance. Such optimization poses a burden on machine learning practitioners, requiring expert knowledge, intuition or computationally demanding brute-force parameter searches. We here assess three different hyperparameter selection methods: grid search, random search and an efficient automated optimization technique based on Bayesian optimization (BO). We apply these methods to a machine learning problem based on kernel ridge regression in computational chemistry. Two different descriptors are employed to represent the atomic structure of organic molecules, one of which introduces its own set of hyperparameters to the method. We identify optimal hyperparameter configurations and infer entire prediction error landscapes in hyperparameter space that serve as visual guides for the hyperparameter performance. We further demonstrate that for an increasing number of hyperparameters, BO and random search become significantly more efficient in computational time than an exhaustive grid search, while delivering an equivalent or even better accuracy.

Evaluation of Categorical Matrix Completion Algorithms: Towards Improved Active Learning for Drug Discovery

MOTIVATION

High throughput and high content screening are extensively used to determine the effect of small molecule compounds and other potential therapeutics upon particular targets as part of the early drug development process. However, screening is typically used to find compounds that have a desired effect but not to identify potential undesirable side effects. This is because the size of the search space precludes measuring the potential effect of all compounds on all targets. Active machine learning has been proposed as a solution to this problem.

RESULTS

In this article, we describe an improved imputation method, Impute By Committee, for completion of matrices containing categorical values. We compare this method to existing approaches in the context of modeling the effects of many compounds on many targets using latent similarities between compounds and conditions. We also compare these methods for the task of driving active learning in well-characterized settings for synthetic and real datasets. Our new approach performed the best overall both in the accuracy of matrix completion itself and in the number of experiments needed to train an accurate predictive model compared to random selection of experiments. We further improved upon the performance of our new method by developing an adaptive switching strategy for active learning that iteratively chooses between different matrix completion methods.

AVAILABILITY

A Reproducible Research Archive containing all data and code will be made available upon acceptance at http://murphylab.cbd.cmu.edu/software.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Inferring experimental procedures from text-based representations of chemical reactions

The experimental execution of chemical reactions is a context-dependent and time-consuming process, often solved using the experience collected over multiple decades of laboratory work or searching similar, already executed, experimental protocols. Although data-driven schemes, such as retrosynthetic models, are becoming established technologies in synthetic organic chemistry, the conversion of proposed synthetic routes to experimental procedures remains a burden on the shoulder of domain experts. In this work, we present data-driven models for predicting the entire sequence of synthesis steps starting from a textual representation of a chemical equation, for application in batch organic chemistry. We generated a data set of 693,517 chemical equations and associated action sequences by extracting and processing experimental procedure text from patents, using state-of-the-art natural language models. We used the attained data set to train three different models: a nearest-neighbor model based on recently-introduced reaction fingerprints, and two deep-learning sequence-to-sequence models based on the Transformer and BART architectures. An analysis by a trained chemist revealed that the predicted action sequences are adequate for execution without human intervention in more than 50% of the cases.

Active discovery of organic semiconductors

The versatility of organic molecules generates a rich design space for organic semiconductors (OSCs) considered for electronics applications. Offering unparalleled promise for materials discovery, the vastness of this design space also dictates efficient search strategies. Here, we present an active machine learning (AML) approach that explores an unlimited search space through consecutive application of molecular morphing operations. Evaluating the suitability of OSC candidates on the basis of charge injection and mobility descriptors, the approach successively queries predictive-quality first-principles calculations to build a refining surrogate model. The AML approach is optimized in a truncated test space, providing deep methodological insight by visualizing it as a chemical space network. Significantly outperforming a conventional computational funnel, the optimized AML approach rapidly identifies well-known and hitherto unknown molecular OSC candidates with superior charge conduction properties. Most importantly, it constantly finds further candidates with highest efficiency while continuing its exploration of the endless design space.

Predicting Reaction Yields via Supervised Learning

Numerous disciplines, such as image recognition and language translation, have been revolutionized by using machine learning (ML) to leverage big data. In organic synthesis, providing accurate chemical reactivity predictions with supervised ML could assist chemists with reaction prediction, optimization, and mechanistic interrogation.To apply supervised ML to chemical reactions, one needs to define the object of prediction (e.g., yield, enantioselectivity, solubility, or a recommendation) and represent reactions with descriptive data. Our group's effort has focused on representing chemical reactions using DFT-derived physical features of the reacting molecules and conditions, which serve as features for building supervised ML models.In this Account, we present a review and perspective on three studies conducted by our group where ML models have been employed to predict reaction yield. First, we focus on a small reaction data set where 16 phosphine ligands were evaluated in a single Ni-catalyzed Suzuki-Miyaura cross-coupling reaction, and the reaction yield was modeled with linear regression. In this setting, where the regression complexity is strongly limited by the amount of available data, we emphasize the importance of identifying single features that are directly relevant to reactivity. Next, we focus on models trained on two larger data sets obtained with high-throughput experimentation (HTE). With hundreds to thousands of reactions available, more complex models can be explored, for example, models that algorithmically perform feature selection from a broad set of candidate features. We examine how a variety of ML algorithms model these data sets and how well these models generalize to out-of-sample substrates. Specifically, we compare the ML models that use DFT-based featurization to a baseline model that is obtained with features that carry no physical information, that is, random features, and to a naive non-ML model that averages yields of reactions that share the same conditions and substrate combinations. We find that for only one of the two data sets, DFT-based featurization leads to a significant, although moderate, out-of-sample prediction improvement. The source of this improvement was further isolated to specific features which allowed us to formulate a testable mechanistic hypothesis that was validated experimentally. Finally, we offer remarks on supervised ML model building on HTE data sets focusing on algorithmic improvements in model training.Statistical methods in chemistry have a rich history, but only recently has ML gained widespread attention in reaction development. As the untapped potential of ML is explored, novel tools are likely to arise from future research. Our studies suggest that supervised ML can lead to improved predictions of reaction yield over simpler modeling methods and facilitate mechanistic understanding of reaction dynamics. However, further research and development is required to establish ML as an indispensable tool in reactivity modeling.

Improving Data and Prediction Quality of High-Throughput Perovskite Synthesis with Model Fusion

Combinatorial fusion analysis (CFA) is an approach for combining multiple scoring systems using the rank-score characteristic function and cognitive diversity measure. One example is to combine diverse machine learning models to achieve better prediction quality. In this work, we apply CFA to the synthesis of metal halide perovskites containing organic ammonium cations via inverse temperature crystallization. Using a data set generated by high-throughput experimentation, four individual models (support vector machines, random forests, weighted logistic classifier, and gradient boosted trees) were developed. We characterize each of these scoring systems and explore 66 possible combinations of the models. When measured by the precision on predicting crystal formation, the majority of the combination models improves the individual model results. The best combination models outperform the best individual models by 3.9 percentage points in precision. In addition to improving prediction quality, we demonstrate how the fusion models can be used to identify mislabeled input data and address issues of data quality. In particular, we identify example cases where all single models and all fusion models do not give the correct prediction. Experimental replication of these syntheses reveals that these compositions are sensitive to modest temperature variations across the different locations of the heating element that can hinder or enhance the crystallization process. In summary, we demonstrate that model fusion using CFA can not only identify a previously unconsidered influence on reaction outcome but also be used as a form of quality control for high-throughput experimentation.

Data-driven machine learning model for the prediction of oxygen vacancy formation energy of metal oxide materials

Metal oxides are widely used in the fields of chemistry, physics and materials science. Oxygen vacancy formation energy is a key parameter to describe the chemical, mechanical, and thermodynamic properties of metal oxides. How to acquire quickly and accurately oxygen vacancy formation energy remains a challenge for both experimental and theoretical researchers. Herein, we propose a machine learning model for the prediction of oxygen vacancy formation energy via data-driven analysis and the definition of simple descriptors. Starting with the database containing oxygen vacancy formation energies for 1750 metal oxides with enough structural diversity, new descriptors that effectively avoid the defects of molecular fingerprints, molecular graphic descriptors and site descriptors are defined. The descriptors have obvious physical meanings and wide practicability. Multiple linear regression analysis is then used to screen important features for machine learning model development, and two strongly associated features are obtained. The selected descriptors are used as input for the training of 21 machine learning models to select and develop the most accurate machine learning model. Finally, it is shown that the least squares support vector regression method exhibits the best performance for accurate prediction of the targeted oxygen vacancy formation energy through systematic error analysis, and the prediction accuracy is also verified by the external dataset. Our work establishes a novel and simple computational approach for accurate prediction of the oxygen vacancy formation energy of metal oxides and highlights the availability of data-driven analysis for metal oxide material research.

Regio-selectivity prediction with a machine-learned reaction representation and on-the-fly quantum mechanical descriptors

Accurate and rapid evaluation of whether substrates can undergo the desired the transformation is crucial and challenging for both human knowledge and computer predictions. Despite the potential of machine learning in predicting chemical reactivity such as selectivity, popular feature engineering and learning methods are either time-consuming or data-hungry. We introduce a new method that combines machine-learned reaction representation with selected quantum mechanical descriptors to predict regio-selectivity in general substitution reactions. We construct a reactivity descriptor database based on ab initio calculations of 130k organic molecules, and train a multi-task constrained model to calculate demanded descriptors on-the-fly. The proposed platform enhances the inter/extra-polated performance for regio-selectivity predictions and enables learning from small datasets with just hundreds of examples. Furthermore, the proposed protocol is demonstrated to be generally applicable to a diverse range of chemical spaces. For three general types of substitution reactions (aromatic C-H functionalization, aromatic C-X substitution, and other substitution reactions) curated from a commercial database, the fusion model achieves 89.7%, 96.7%, and 97.2% top-1 accuracy in predicting the major outcome, respectively, each using 5000 training reactions. Using predicted descriptors, the fusion model is end-to-end, and requires approximately only 70 ms per reaction to predict the selectivity from reaction SMILES strings.

Deep learning and generative methods in cheminformatics and chemical biology: navigating small molecule space intelligently

The number of 'small' molecules that may be of interest to chemical biologists - chemical space - is enormous, but the fraction that have ever been made is tiny. Most strategies are discriminative, i.e. have involved 'forward' problems (have molecule, establish properties). However, we normally wish to solve the much harder generative or inverse problem (describe desired properties, find molecule). 'Deep' (machine) learning based on large-scale neural networks underpins technologies such as computer vision, natural language processing, driverless cars, and world-leading performance in games such as Go; it can also be applied to the solution of inverse problems in chemical biology. In particular, recent developments in deep learning admit the in silico generation of candidate molecular structures and the prediction of their properties, thereby allowing one to navigate (bio)chemical space intelligently. These methods are revolutionary but require an understanding of both (bio)chemistry and computer science to be exploited to best advantage. We give a high-level (non-mathematical) background to the deep learning revolution, and set out the crucial issue for chemical biology and informatics as a two-way mapping from the discrete nature of individual molecules to the continuous but high-dimensional latent representation that may best reflect chemical space. A variety of architectures can do this; we focus on a particular type known as variational autoencoders. We then provide some examples of recent successes of these kinds of approach, and a look towards the future.

Molecular representations in AI-driven drug discovery: a review and practical guide

The technological advances of the past century, marked by the computer revolution and the advent of high-throughput screening technologies in drug discovery, opened the path to the computational analysis and visualization of bioactive molecules. For this purpose, it became necessary to represent molecules in a syntax that would be readable by computers and understandable by scientists of various fields. A large number of chemical representations have been developed over the years, their numerosity being due to the fast development of computers and the complexity of producing a representation that encompasses all structural and chemical characteristics. We present here some of the most popular electronic molecular and macromolecular representations used in drug discovery, many of which are based on graph representations. Furthermore, we describe applications of these representations in AI-driven drug discovery. Our aim is to provide a brief guide on structural representations that are essential to the practice of AI in drug discovery. This review serves as a guide for researchers who have little experience with the handling of chemical representations and plan to work on applications at the interface of these fields.

Automation of Synthesis in Medicinal Chemistry: Progress and Challenges

Since the 1990s, concerted attempts have been made to improve the efficiency of medicinal chemistry synthesis tasks using automation. Although impacts have been seen in some tasks, such as small array synthesis and reaction optimization, many synthesis tasks in medicinal chemistry are still manual. As it has been shown that synthesis technology has a large effect on the properties of the compounds being tested, this review looks at recent research in automation relevant to synthesis in medicinal chemistry. A common theme has been the integration of tasks, as well as the use of increased computing power to access complex automation platforms remotely and to improve synthesis planning software. However, there has been more limited progress in modular tools for the medicinal chemist with a focus on autonomy rather than automation.

Automated, Accelerated Nanoscale Synthesis of Iminopyrrolidines

Miniaturization and acceleration of synthetic chemistry is an emerging area in pharmaceutical, agrochemical, and materials research and development. Herein, we describe the synthesis of iminopyrrolidine-2-carboxylic acid derivatives using chiral glutamine, oxo components, and isocyanide building blocks in an unprecedented Ugi-3-component reaction. We used I-DOT, a positive-pressure-based low-volume and non-contact dispensing technology to prepare more than 1000 different derivatives in a fully automated fashion. In general, the reaction is stereoselective, proceeds in good yields, and tolerates a wide variety of functional groups. We exemplify a pipeline of fast and efficient nanomole-scale scouting to millimole-scale synthesis for the discovery of a useful novel reaction with great scope.

Nanomaterial Synthesis Insights from Machine Learning of Scientific Articles by Extracting, Structuring, and Visualizing Knowledge

Nanomaterials of varying compositions and morphologies are of interest for many applications from catalysis to optics, but the synthesis of nanomaterials and their scale-up are most often time-consuming and Edisonian processes. Information gleaned from the scientific literature can help inform and accelerate nanomaterials development, but again, searching the literature and digesting the information are time-consuming manual processes for researchers. To help address these challenges, we developed scientific article-processing tools that extract and structure information from the text and figures of nanomaterials articles, thereby enabling the creation of a personalized knowledgebase for nanomaterials synthesis that can be mined to help inform further nanomaterials development. Starting with a corpus of ∼35k nanomaterials-related articles, we developed models to classify articles according to the nanomaterial composition and morphology, extract synthesis protocols from within the articles' text, and extract, normalize, and categorize chemical terms within synthesis protocols. We demonstrate the efficiency of the proposed pipeline on an expert-labeled set of nanomaterials synthesis articles, achieving 100% accuracy on composition prediction, 95% accuracy on morphology prediction, 0.99 AUC on protocol identification, and up to a 0.87 F1-score on chemical entity recognition. In addition to processing articles' text, microscopy images of nanomaterials within the articles are also automatically identified and analyzed to determine the nanomaterials' morphologies and size distributions. To enable users to easily explore the database, we developed a complementary browser-based visualization tool that provides flexibility in comparing across subsets of articles of interest. We use these tools and information to identify trends in nanomaterials synthesis, such as the correlation of certain reagents with various nanomaterial morphologies, which is useful in guiding hypotheses and reducing the potential parameter space during experimental design.

When robotics met fluidics

High-throughput fluidic technologies have increased the speed and accuracy of fluid processing to the extent that unlocking further gains will require replacing the human operator with a robotic counterpart. Recent advances in chemistry and biology, such as gene editing, have further exacerbated the need for smart, high-throughput experimentation. A growing number of innovations at the intersection of robotics and fluidics illustrate the tremendous opportunity in achieving fully self-driving fluid systems. We envision that the fields of synthetic chemistry and synthetic biology will be the first beneficiaries of AI-directed robotic and fluidic systems, and largely fall within two modalities: complex integrated centralized facilities that produce data, and distributed systems that synthesize products and conduct disease surveillance.

Benchmarking the acceleration of materials discovery by sequential learning

Sequential learning (SL) strategies, i.e. iteratively updating a machine learning model to guide experiments, have been proposed to significantly accelerate materials discovery and research. Applications on computational datasets and a handful of optimization experiments have demonstrated the promise of SL, motivating a quantitative evaluation of its ability to accelerate materials discovery, specifically in the case of physical experiments. The benchmarking effort in the present work quantifies the performance of SL algorithms with respect to a breadth of research goals: discovery of any "good" material, discovery of all "good" materials, and discovery of a model that accurately predicts the performance of new materials. To benchmark the effectiveness of different machine learning models against these goals, we use datasets in which the performance of all materials in the search space is known from high-throughput synthesis and electrochemistry experiments. Each dataset contains all pseudo-quaternary metal oxide combinations from a set of six elements (chemical space), the performance metric chosen is the electrocatalytic activity (overpotential) for the oxygen evolution reaction (OER). A diverse set of SL schemes is tested on four chemical spaces, each containing 2121 catalysts. The presented work suggests that research can be accelerated by up to a factor of 20 compared to random acquisition in specific scenarios. The results also show that certain choices of SL models are ill-suited for a given research goal resulting in substantial deceleration compared to random acquisition methods. The results provide quantitative guidance on how to tune an SL strategy for a given research goal and demonstrate the need for a new generation of materials-aware SL algorithms to further accelerate materials discovery.

A streamlined synthesis of the neurosteroid 3β-methoxypregnenolone assisted by a statistical experimental design and automation

The potential of integrating flow synthesizers, statistical design of experiments and automation has been exemplified to realize the streamlined etherification of pregnenolone to the neurosteroid 3β-methoxypregnenolone (MAP4343).

The good, the bad, and the ugly in chemical and biological data for machine learning

Machine learning and artificial intelligence (ML/AI) have become important research tools in molecular medicine and chemistry. Their rise and recent success in drug discovery promises a rapid progression of development pipelines while reshaping how fundamental and clinical research is conducted. By taking advantage of the ever-growing wealth of publicly available and proprietary data, learning algorithms now provide an attractive means to generate statistically motivated research hypotheses. Hitherto unknown data patterns may guide and prioritize experiments, and augment expert intuition. Therefore, data is a key component in the model building workflow. Herein, I aim to discuss types of chemical and biological data according to their quality and reemphasize general recommendations for their use in ML/AI.