101
|
Kovács DP, McCorkindale W, Lee AA. Quantitative interpretation explains machine learning models for chemical reaction prediction and uncovers bias. Nat Commun 2021; 12:1695. [PMID: 33727552 PMCID: PMC7966799 DOI: 10.1038/s41467-021-21895-w] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Accepted: 02/10/2021] [Indexed: 12/30/2022] Open
Abstract
Organic synthesis remains a major challenge in drug discovery. Although a plethora of machine learning models have been proposed as solutions in the literature, they suffer from being opaque black-boxes. It is neither clear if the models are making correct predictions because they inferred the salient chemistry, nor is it clear which training data they are relying on to reach a prediction. This opaqueness hinders both model developers and users. In this paper, we quantitatively interpret the Molecular Transformer, the state-of-the-art model for reaction prediction. We develop a framework to attribute predicted reaction outcomes both to specific parts of reactants, and to reactions in the training set. Furthermore, we demonstrate how to retrieve evidence for predicted reaction outcomes, and understand counterintuitive predictions by scrutinising the data. Additionally, we identify Clever Hans predictions where the correct prediction is reached for the wrong reason due to dataset bias. We present a new debiased dataset that provides a more realistic assessment of model performance, which we propose as the new standard benchmark for comparing reaction prediction models.
Collapse
Affiliation(s)
| | | | - Alpha A Lee
- Cavendish Laboratory, University of Cambridge, Cambridge, UK.
| |
Collapse
|
102
|
Li X, Fourches D. SMILES Pair Encoding: A Data-Driven Substructure Tokenization Algorithm for Deep Learning. J Chem Inf Model 2021; 61:1560-1569. [PMID: 33715361 DOI: 10.1021/acs.jcim.0c01127] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Simplified molecular input line entry system (SMILES)-based deep learning models are slowly emerging as an important research topic in cheminformatics. In this study, we introduce SMILES pair encoding (SPE), a data-driven tokenization algorithm. SPE first learns a vocabulary of high-frequency SMILES substrings from a large chemical dataset (e.g., ChEMBL) and then tokenizes SMILES based on the learned vocabulary for the actual training of deep learning models. SPE augments the widely used atom-level tokenization by adding human-readable and chemically explainable SMILES substrings as tokens. Case studies show that SPE can achieve superior performances on both molecular generation and quantitative structure-activity relationship (QSAR) prediction tasks. In particular, the SPE-based generative models outperformed the atom-level tokenization model in the aspects of novelty, diversity, and ability to resemble the training set distribution. The performance of SPE-based QSAR prediction models were evaluated using 24 benchmark datasets where SPE consistently either did match or outperform atom-level and k-mer tokenization. Therefore, SPE could be a promising tokenization method for SMILES-based deep learning models. An open-source Python package SmilesPE was developed to implement this algorithm and is now freely available at https://github.com/XinhaoLi74/SmilesPE.
Collapse
Affiliation(s)
- Xinhao Li
- Department of Chemistry, Bioinformatics Research Center, North Carolina State University, Raleigh, North Carolina 27695, United States
| | - Denis Fourches
- Department of Chemistry, Bioinformatics Research Center, North Carolina State University, Raleigh, North Carolina 27695, United States
| |
Collapse
|
103
|
Shen WX, Zeng X, Zhu F, Wang YL, Qin C, Tan Y, Jiang YY, Chen YZ. Out-of-the-box deep learning prediction of pharmaceutical properties by broadly learned knowledge-based molecular representations. NAT MACH INTELL 2021. [DOI: 10.1038/s42256-021-00301-6] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
|
104
|
Wu L, Huang R, Tetko IV, Xia Z, Xu J, Tong W. Trade-off Predictivity and Explainability for Machine-Learning Powered Predictive Toxicology: An in-Depth Investigation with Tox21 Data Sets. Chem Res Toxicol 2021; 34:541-549. [PMID: 33513003 DOI: 10.1021/acs.chemrestox.0c00373] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Selecting a model in predictive toxicology often involves a trade-off between prediction performance and explainability: should we sacrifice the model performance to gain explainability or vice versa. Here we present a comprehensive study to assess algorithm and feature influences on model performance in chemical toxicity research. We conducted over 5000 models for a Tox21 bioassay data set of 65 assays and ∼7600 compounds. Seven molecular representations as features and 12 modeling approaches varying in complexity and explainability were employed to systematically investigate the impact of various factors on model performance and explainability. We demonstrated that end points dictated a model's performance, regardless of the chosen modeling approach including deep learning and chemical features. Overall, more complex models such as (LS-)SVM and Random Forest performed marginally better than simpler models such as linear regression and KNN in the presented Tox21 data analysis. Since a simpler model with acceptable performance often also is easy to interpret for the Tox21 data set, it clearly was the preferred choice due to its better explainability. Given that each data set had its own error structure both for dependent and independent variables, we strongly recommend that it is important to conduct a systematic study with a broad range of model complexity and feature explainability to identify model balancing its predictivity and explainability.
Collapse
Affiliation(s)
- Leihong Wu
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, FDA, 3900 NCTR Road, Jefferson, Arkansas 72079, United States
| | - Ruili Huang
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences, National Institutes of Health, 9800 Medical Center Drive, Rockville, Maryland 20850, United States
| | - Igor V Tetko
- Institute of Structural Biology, Helmholtz Zentrum München-Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, 85764 Neuherberg, Germany.,BIGCHEM GmbH, Valerystraße 49, DE-85716 Unterschleißheim, Germany
| | - Zhonghua Xia
- Institute of Structural Biology, Helmholtz Zentrum München-Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, 85764 Neuherberg, Germany
| | - Joshua Xu
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, FDA, 3900 NCTR Road, Jefferson, Arkansas 72079, United States
| | - Weida Tong
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, FDA, 3900 NCTR Road, Jefferson, Arkansas 72079, United States
| |
Collapse
|
105
|
Semenyuta IV, Trush MM, Kovalishyn VV, Rogalsky SP, Hodyna DM, Karpov P, Xia Z, Tetko IV, Metelytsia LO. Structure-Activity Relationship Modeling and Experimental Validation of the Imidazolium and Pyridinium Based Ionic Liquids as Potential Antibacterials of MDR Acinetobacter Baumannii and Staphylococcus Aureus. Int J Mol Sci 2021; 22:ijms22020563. [PMID: 33429999 PMCID: PMC7827895 DOI: 10.3390/ijms22020563] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 12/29/2020] [Accepted: 01/05/2021] [Indexed: 12/31/2022] Open
Abstract
Online Chemical Modeling Environment (OCHEM) was used for QSAR analysis of a set of ionic liquids (ILs) tested against multi-drug resistant (MDR) clinical isolate Acinetobacter baumannii and Staphylococcus aureus strains. The predictive accuracy of regression models has coefficient of determination q2 = 0.66 - 0.79 with cross-validation and independent test sets. The models were used to screen a virtual chemical library of ILs, which was designed with targeted activity against MDR Acinetobacter baumannii and Staphylococcus aureus strains. Seven most promising ILs were selected, synthesized, and tested. Three ILs showed high activity against both these MDR clinical isolates.
Collapse
Affiliation(s)
- Ivan V. Semenyuta
- V.P. Kukhar Institute of Bioorganic Chemistry and Petrochemistry, National Academy of Science of Ukraine, 1 Murmanska Street, 02660 Kyiv, Ukraine; (I.V.S.); (M.M.T.); (V.V.K.); (S.P.R.); (D.M.H.); (L.O.M.)
| | - Maria M. Trush
- V.P. Kukhar Institute of Bioorganic Chemistry and Petrochemistry, National Academy of Science of Ukraine, 1 Murmanska Street, 02660 Kyiv, Ukraine; (I.V.S.); (M.M.T.); (V.V.K.); (S.P.R.); (D.M.H.); (L.O.M.)
| | - Vasyl V. Kovalishyn
- V.P. Kukhar Institute of Bioorganic Chemistry and Petrochemistry, National Academy of Science of Ukraine, 1 Murmanska Street, 02660 Kyiv, Ukraine; (I.V.S.); (M.M.T.); (V.V.K.); (S.P.R.); (D.M.H.); (L.O.M.)
| | - Sergiy P. Rogalsky
- V.P. Kukhar Institute of Bioorganic Chemistry and Petrochemistry, National Academy of Science of Ukraine, 1 Murmanska Street, 02660 Kyiv, Ukraine; (I.V.S.); (M.M.T.); (V.V.K.); (S.P.R.); (D.M.H.); (L.O.M.)
| | - Diana M. Hodyna
- V.P. Kukhar Institute of Bioorganic Chemistry and Petrochemistry, National Academy of Science of Ukraine, 1 Murmanska Street, 02660 Kyiv, Ukraine; (I.V.S.); (M.M.T.); (V.V.K.); (S.P.R.); (D.M.H.); (L.O.M.)
| | - Pavel Karpov
- Institute of Structural Biology, Helmholtz Zentrum München—German Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, D-85764 Neuherberg, Germany; (P.K.); (Z.X.)
| | - Zhonghua Xia
- Institute of Structural Biology, Helmholtz Zentrum München—German Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, D-85764 Neuherberg, Germany; (P.K.); (Z.X.)
| | - Igor V. Tetko
- Institute of Structural Biology, Helmholtz Zentrum München—German Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, D-85764 Neuherberg, Germany; (P.K.); (Z.X.)
- BIGCHEM GmbH, Unterschleißheim, Valerystr. 49, D-85716 Neuherberg, Germany
- Correspondence: ; Tel.: +49-89-3187-3575
| | - Larisa O. Metelytsia
- V.P. Kukhar Institute of Bioorganic Chemistry and Petrochemistry, National Academy of Science of Ukraine, 1 Murmanska Street, 02660 Kyiv, Ukraine; (I.V.S.); (M.M.T.); (V.V.K.); (S.P.R.); (D.M.H.); (L.O.M.)
| |
Collapse
|
106
|
Tetko IV, Engkvist O. From Big Data to Artificial Intelligence: chemoinformatics meets new challenges. J Cheminform 2020; 12:74. [PMID: 33339533 PMCID: PMC7747384 DOI: 10.1186/s13321-020-00475-y] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Accepted: 11/18/2020] [Indexed: 12/17/2022] Open
Abstract
The increasing volume of biomedical data in chemistry and life sciences requires development of new methods and approaches for their analysis. Artificial Intelligence and machine learning, especially neural networks, are increasingly used in the chemical industry, in particular with respect to Big Data. This editorial highlights the main results presented during the special session of the International Conference on Neural Networks organized by "Big Data in Chemistry" project and draws perspectives on the future progress of the field.
Collapse
Affiliation(s)
- Igor V Tetko
- Helmholtz Zentrum München-German Research Center for Environmental Health (GmbH), Institute of Structural Biology, Ingolstädter Landstraße 1, 85764, Neuherberg, Germany.
- BIGCHEM GmbH, Valerystr. 49, 85716, Unterschleißheim, Germany.
| | - Ola Engkvist
- Molecular AI, Discovery Sciences, R&D, AstraZeneca, Gothenburg, Sweden
| |
Collapse
|
107
|
Tetko IV, Karpov P, Van Deursen R, Godin G. State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis. Nat Commun 2020; 11:5575. [PMID: 33149154 PMCID: PMC7643129 DOI: 10.1038/s41467-020-19266-y] [Citation(s) in RCA: 124] [Impact Index Per Article: 31.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Accepted: 10/05/2020] [Indexed: 11/18/2022] Open
Abstract
We investigated the effect of different training scenarios on predicting the (retro)synthesis of chemical compounds using text-like representation of chemical reactions (SMILES) and Natural Language Processing (NLP) neural network Transformer architecture. We showed that data augmentation, which is a powerful method used in image processing, eliminated the effect of data memorization by neural networks and improved their performance for prediction of new sequences. This effect was observed when augmentation was used simultaneously for input and the target data simultaneously. The top-5 accuracy was 84.8% for the prediction of the largest fragment (thus identifying principal transformation for classical retro-synthesis) for the USPTO-50k test dataset, and was achieved by a combination of SMILES augmentation and a beam search algorithm. The same approach provided significantly better results for the prediction of direct reactions from the single-step USPTO-MIT test set. Our model achieved 90.6% top-1 and 96.1% top-5 accuracy for its challenging mixed set and 97% top-5 accuracy for the USPTO-MIT separated set. It also significantly improved results for USPTO-full set single-step retrosynthesis for both top-1 and top-10 accuracies. The appearance frequency of the most abundantly generated SMILES was well correlated with the prediction outcome and can be used as a measure of the quality of reaction prediction.
Collapse
Affiliation(s)
- Igor V Tetko
- Institute of Structural Biology, Helmholtz Zentrum München-Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, D-85764, Neuherberg, Germany.
- BIGCHEM GmbH, Valerystr. 49, D-85716, Unterschleißheim, Germany.
| | - Pavel Karpov
- Institute of Structural Biology, Helmholtz Zentrum München-Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, D-85764, Neuherberg, Germany
- BIGCHEM GmbH, Valerystr. 49, D-85716, Unterschleißheim, Germany
| | - Ruud Van Deursen
- Firmenich International SA, D-Lab by Firmenich, Rue de la Bergère 7, CH-1242, Meyrin-Satigny, Switzerland
| | - Guillaume Godin
- Firmenich International SA, D-Lab by Firmenich, Rue de la Bergère 7, CH-1242, Meyrin-Satigny, Switzerland.
| |
Collapse
|
108
|
Chakravarti SK. Reason Vectors: Abstract Representation of Chemistry–Biology Interaction Outcomes, for Reasoning and Prediction. J Chem Inf Model 2020; 60:4614-4628. [DOI: 10.1021/acs.jcim.0c00601] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Suman K. Chakravarti
- MultiCASE Inc., 23811 Chagrin Blvd., Suite 305, Beachwood, Ohio 44122, United States
| |
Collapse
|
109
|
Jiménez-Luna J, Grisoni F, Schneider G. Drug discovery with explainable artificial intelligence. NAT MACH INTELL 2020. [DOI: 10.1038/s42256-020-00236-4] [Citation(s) in RCA: 152] [Impact Index Per Article: 38.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
|
110
|
van Deursen R, Ertl P, Tetko IV, Godin G. GEN: highly efficient SMILES explorer using autodidactic generative examination networks. J Cheminform 2020; 12:22. [PMID: 33430998 PMCID: PMC7146994 DOI: 10.1186/s13321-020-00425-8] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2019] [Accepted: 03/23/2020] [Indexed: 12/31/2022] Open
Abstract
Recurrent neural networks have been widely used to generate millions of de novo molecules in defined chemical spaces. Reported deep generative models are exclusively based on LSTM and/or GRU units and frequently trained using canonical SMILES. In this study, we introduce Generative Examination Networks (GEN) as a new approach to train deep generative networks for SMILES generation. In our GENs, we have used an architecture based on multiple concatenated bidirectional RNN units to enhance the validity of generated SMILES. GENs autonomously learn the target space in a few epochs and are stopped early using an independent online examination mechanism, measuring the quality of the generated set. Herein we have used online statistical quality control (SQC) on the percentage of valid molecular SMILES as examination measure to select the earliest available stable model weights. Very high levels of valid SMILES (95–98%) can be generated using multiple parallel encoding layers in combination with SMILES augmentation using unrestricted SMILES randomization. Our trained models combine an excellent novelty rate (85–90%) while generating SMILES with strong conservation of the property space (95–99%). In GENs, both the generative network and the examination mechanism are open to other architectures and quality criteria.![]()
Collapse
Affiliation(s)
- Ruud van Deursen
- Firmenich SA, Research and Development, Rue des Jeunes 1, Les Acacias, 1227, Geneva, Switzerland.
| | - Peter Ertl
- Novartis Institutes for BioMedical Research, Novartis Campus, 4056, Basel, Switzerland
| | - Igor V Tetko
- Institute of Structural Biology, Helmholtz Zentrum München-German Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, 85764, Neuherberg, Germany.,BIGCHEM GmbH, Valerystr. 49, 85716, Unterschleißheim, Germany
| | - Guillaume Godin
- Firmenich SA, Research and Development, Rue des Jeunes 1, Les Acacias, 1227, Geneva, Switzerland.
| |
Collapse
|
111
|
Xu Y, Cai C, Wang S, Lai L, Pei J. Efficient molecular encoders for virtual screening. DRUG DISCOVERY TODAY. TECHNOLOGIES 2019; 32-33:19-27. [PMID: 33386090 DOI: 10.1016/j.ddtec.2020.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2020] [Revised: 08/23/2020] [Accepted: 08/28/2020] [Indexed: 06/12/2023]
Abstract
Molecular representations encoding molecular structure information play critical roles in molecular virtual screening (VS). In order to improve VS performance, an abundance of molecular encoders have been developed and tested by various VS challenges. Combinational strategies were also used to improve the performance. Deep learning (DL)-based molecular encoders have attracted much attention for their automatic information extraction ability. In this review, we present an overview of two-dimensional-, three-dimensional-, and DL-based molecular encoders, summarize recent progress of VS using DL technologies, and propose a general framework of DL molecular encoder-based VS. Perspectives on the future directions of molecular representations and applications in the prediction of active compounds are also provided.
Collapse
Affiliation(s)
- Youjun Xu
- BNLMS, Peking-Tsinghua Center for Life Sciences at the College of Chemistry and Molecular Engineering, Peking University, Beijing 100871, PR China
| | - Chenjing Cai
- Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, PR China
| | - Shiwei Wang
- PTN Graduate Program, Academy for Advanced Interdisciplinary Studies, Peking University, 100871, PR China
| | - Luhua Lai
- BNLMS, Peking-Tsinghua Center for Life Sciences at the College of Chemistry and Molecular Engineering, Peking University, Beijing 100871, PR China; Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, PR China.
| | - Jianfeng Pei
- Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, PR China.
| |
Collapse
|