Assessment and Reproducibility of Quantitative Structure–Activity Relationship Models by the Nonexpert

Making in silico predictive models for toxicology FAIR

In silico predictive models for toxicology include quantitative structure-activity relationship (QSAR) and physiologically based kinetic (PBK) approaches to predict physico-chemical and ADME properties, toxicological effects and internal exposure. Such models are used to fill data gaps as part of chemical risk assessment. There is a growing need to ensure in silico predictive models for toxicology are available for use and reproducible. This paper describes how the FAIR (Findable, Accessible, Interoperable, Reusable) principles, developed for data sharing, have been applied to in silico predictive models. In particular, this investigation has focussed on how the FAIR principles could be applied to improved regulatory acceptance of predictions from such models. Eighteen principles have been developed that cover all aspects of FAIR. It is intended that FAIRification of in silico predictive models for toxicology will increase their use and acceptance.

Risk assessment in the 21st century: where are we heading?

Reliance on animal tests for chemical safety assessment is increasingly being challenged, not only because of ethical reasons, but also because they procrastinate regulatory decisions and because of concerns over the transferability of results to humans. New approach methodologies (NAMs) need to be fit for purpose and new thinking is required to reconsider chemical legislation, validation of NAMs and opportunities to move away from animal tests. This article summarizes the presentations from a symposium at the 2022 Annual Congress of the British Toxicology Society on the topic of the future of chemical risk assessment in the 21st century. The symposium included three case-studies where NAMs have been used in safety assessments. The first case illustrated how read-across augmented with some in vitro tests could be used reliably to perform the risk assessment of analogues lacking data. The second case showed how specific bioactivity assays could identify an NAM point of departure (PoD) and how this could be translated through physiologically based kinetic modelling in an in vivo PoD for the risk assessment. The third case showed how adverse-outcome pathway (AOP) information, including molecular-initiating event and key events with their underlying data, established for certain chemicals could be used to produce an in silico model that is able to associate chemical features of an unstudied substance with specific AOPs or AOP networks. The manuscript presents the discussions that took place regarding the limitations and benefits of these new approaches, and what are the barriers and the opportunities for their increased use in regulatory decision making.

Chemical composition/pharmacophore modelling- based, virtual screening, molecular docking and dynamic simulation studies for the discovery of novel superoxide dismutase (SODs) of bioactive molecules from aerial parts of Inula Montana as antioxydant's agents

The accumulation of free radicals in the body develops chronic and degenerative diseases such as cancer, autoimmune diseases, rheumatoid arthritis, cardiovascular and neurodegenerative diseases. The first aim of this work was to study the chemical composition of Inula Montana essential oil using GC-FID and GC/MS analysis and the antioxidant activities using radical scavenging (DPPH) and the Ferric -Reducing Antioxidant Power (FRAP) tests. The second aim was to describe the assess the antioxidant activity and computational study of Superoxide Dismutase (SODs) and ctDNA inhibition. Sixty-nine compounds were identified in the essential oil of the aerial part of Inula montana. Shyobunol and α-Cadinol were the major compounds in the essential oil. The antioxidant power of the essential oil showed an important antioxidant effect compared to ascorbic acid and the methionine co-crystallized inhibitor. The results of the docking simulation revealed that E, E-Farnesyl acetate has an affinity to interact with binding models and the antioxidant activities of the ctDNA sequence and Superoxide Dismutase target. The penetration through the Blood-Brain Barrier came out to be best for E, E-Farnesyl acetate and E-Nerolidolacetate and was significantly higher than the control molecule and Lref. Finally, the application of ADMET filters gives us positive information on the compound E, E-Farnesyl acetate, which appears as a new inhibitor potentially more active towards ctDNA and SODs target. The active compounds, E,E-Farnesyl acetate can be used as templates for further development of more potent antioxidative agents.Communicated by Ramaswamy H. Sarma.

IVIVE: Facilitating the Use of In Vitro Toxicity Data in Risk Assessment and Decision Making

During the past few decades, the science of toxicology has been undergoing a transformation from observational to predictive science. New approach methodologies (NAMs), including in vitro assays, in silico models, read-across, and in vitro to in vivo extrapolation (IVIVE), are being developed to reduce, refine, or replace whole animal testing, encouraging the judicious use of time and resources. Some of these methods have advanced past the exploratory research stage and are beginning to gain acceptance for the risk assessment of chemicals. A review of the recent literature reveals a burst of IVIVE publications over the past decade. In this review, we propose operational definitions for IVIVE, present literature examples for several common toxicity endpoints, and highlight their implications in decision-making processes across various federal agencies, as well as international organizations, including those in the European Union (EU). The current challenges and future needs are also summarized for IVIVE. In addition to refining and reducing the number of animals in traditional toxicity testing protocols and being used for prioritizing chemical testing, the goal to use IVIVE to facilitate the replacement of animal models can be achieved through their continued evolution and development, including a strategic plan to qualify IVIVE methods for regulatory acceptance.

Pharmacokinetic Tools and Applications

Drug toxicity, as well as therapeutic activity, is contingent upon the parent drug, or a derivative thereof, reaching the relevant site of action in the body, at sufficient concentration, over a given period of time. Thus, the potential to truly elicit an effect is governed by both the intrinsic activity/toxicity of the drug (or its transformation products) and its pharmacokinetic profile. As the pharmaceutical industry has become increasingly aware of the role of pharmacokinetics in determining drug activity and toxicity, the range of software, both freely available and commercial, to predict relevant properties has proliferated. Such tools can be considered on three different levels, applicable at different stages within the drug development process and providing increasing detail and relevance of information. Level (i) is the prediction of fundamental physicochemical properties that can be used to screen vast virtual libraries of potential candidates. Level (ii), predicting the individual absorption, distribution, metabolism, and excretion (ADME) characteristics of potential drugs, can also be applied to many compounds simultaneously. Level (iii), predicting the concentration-time profile of a drug in blood or specific tissues/organs for individuals or a population, is the most sophisticated level of prediction, applied to fewer candidates. In this chapter, in silico tools for predicting ADME-relevant properties, across these three levels, and the applications of this information, are described using exemplar, freely available resources. Further resources are signposted but not all are considered in detail as the purpose here is more to provide an introduction to the capabilities and practicalities of the tools, rather than to provide an exhaustive review of all the tools available.

Development of Robust Quantitative Structure-Activity Relationship Models for CYP2C9, CYP2D6, and CYP3A4 Catalysis and Inhibition

Cytochrome P450 enzymes are responsible for the metabolism of >75% of marketed drugs, making it essential to identify the contributions of individual cytochromes P450 to the total clearance of a new candidate drug. Overreliance on one cytochrome P450 for clearance levies a high risk of drug-drug interactions; and considering that several human cytochrome P450 enzymes are polymorphic, it can also lead to highly variable pharmacokinetics in the clinic. Thus, it would be advantageous to understand the likelihood of new chemical entities to interact with the major cytochrome P450 enzymes at an early stage in the drug discovery process. Typical screening assays using human liver microsomes do not provide sufficient information to distinguish the specific cytochromes P450 responsible for clearance. In this regard, we experimentally assessed the metabolic stability of ∼5000 compounds for the three most prominent xenobiotic metabolizing human cytochromes P450, i.e., CYP2C9, CYP2D6, and CYP3A4, and used the data sets to develop quantitative structure-activity relationship models for the prediction of high-clearance substrates for these enzymes. Screening library included the NCATS Pharmaceutical Collection, comprising clinically approved low-molecular-weight compounds, and an annotated library consisting of drug-like compounds. To identify inhibitors, the library was screened against a luminescence-based cytochrome P450 inhibition assay; and through crossreferencing hits from the two assays, we were able to distinguish substrates and inhibitors of these enzymes. The best substrate and inhibitor models (balanced accuracies ∼0.7), as well as the data used to develop these models, have been made publicly available (https://opendata.ncats.nih.gov/adme) to advance drug discovery across all research groups. SIGNIFICANCE STATEMENT: In drug discovery and development, drug candidates with indiscriminate cytochrome P450 metabolic profiles are considered advantageous, since they provide less risk of potential issues with cytochrome P450 polymorphisms and drug-drug interactions. This study developed robust substrate and inhibitor quantitative structure-activity relationship models for the three major xenobiotic metabolizing cytochromes P450, i.e., CYP2C9, CYP2D6, and CYP3A4. The use of these models early in drug discovery will enable project teams to strategize or pivot when necessary, thereby accelerating drug discovery research.

Alignment-Free Antimicrobial Peptide Predictors: Improving Performance by a Thorough Analysis of the Largest Available Data Set

In the last two decades, a large number of machine-learning-based predictors for the activities of antimicrobial peptides (AMPs) have been proposed. These predictors differ from one another in the learning method and in the training and testing data sets used. Unfortunately, the training data sets present several drawbacks, such as a low representativeness regarding the experimentally validated AMP space, and duplicated peptide sequences between negative and positive data sets. These limitations give a low confidence to most of the approaches to be used in prospective studies. To address these weaknesses, we propose novel modeling and assessing data sets from the largest experimentally validated nonredundant peptide data set reported to date. From these novel data sets, alignment-free quantitative sequence-activity models (AF-QSAMs) based on Random Forest are created to identify general AMPs and their antibacterial, antifungal, antiparasitic, and antiviral functional types. An applicability domain analysis is carried out to determine the reliability of the predictions obtained, which, to the best of our knowledge, is performed for the first time for AMP recognition. A benchmarking is undertaken between the models proposed and several models from the literature that are freely available in 13 programs (ClassAMP, iAMP-2L, ADAM, MLAMP, AMPScanner v2.0, AntiFP, AMPfun, PEPred-suite, AxPEP, CAMPR3, iAMPpred, APIN, and Meta-iAVP). The models proposed are those with the best performance in all of the endpoints modeled, while most of the methods from the literature have weak-to-random predictive agreements. The models proposed are also assessed through Y-scrambling and repeated k-fold cross-validation tests, demonstrating that the outcomes obtained by them are not given by chance. Three chemometric analyses also confirmed the relevance of the peptides descriptors used in the modeling. Therefore, it can be concluded that the models built by fixing the drawbacks existing in the literature contribute to identifying antibacterial, antifungal, antiparasitic, and antiviral peptides with high effectivity and reliability. Models are freely available via the AMPDiscover tool at https://biocom-ampdiscover.cicese.mx/.

Flame: an open source framework for model development, hosting, and usage in production environments

This article describes Flame, an open source software for building predictive models and supporting their use in production environments. Flame is a web application with a web-based graphic interface, which can be used as a desktop application or installed in a server receiving requests from multiple users. Models can be built starting from any collection of biologically annotated chemical structures since the software supports structural normalization, molecular descriptor calculation, and machine learning model generation using predefined workflows. The model building workflow can be customized from the graphic interface, selecting the type of normalization, molecular descriptors, and machine learning algorithm to be used from a panel of state-of-the-art methods implemented natively. Moreover, Flame implements a mechanism allowing to extend its source code, adding unlimited model customization. Models generated with Flame can be easily exported, facilitating collaborative model development. All models are stored in a model repository supporting model versioning. Models are identified by unique model IDs and include detailed documentation formatted using widely accepted standards. The current version is the result of nearly 3 years of development in collaboration with users from the pharmaceutical industry within the IMI eTRANSAFE project, which aims, among other objectives, to develop high-quality predictive models based on shared legacy data for assessing the safety of drug candidates.

MAIP: a web service for predicting blood-stage malaria inhibitors

Malaria is a disease affecting hundreds of millions of people across the world, mainly in developing countries and especially in sub-Saharan Africa. It is the cause of hundreds of thousands of deaths each year and there is an ever-present need to identify and develop effective new therapies to tackle the disease and overcome increasing drug resistance. Here, we extend a previous study in which a number of partners collaborated to develop a consensus in silico model that can be used to identify novel molecules that may have antimalarial properties. The performance of machine learning methods generally improves with the number of data points available for training. One practical challenge in building large training sets is that the data are often proprietary and cannot be straightforwardly integrated. Here, this was addressed by sharing QSAR models, each built on a private data set. We describe the development of an open-source software platform for creating such models, a comprehensive evaluation of methods to create a single consensus model and a web platform called MAIP available at https://www.ebi.ac.uk/chembl/maip/ . MAIP is freely available for the wider community to make large-scale predictions of potential malaria inhibiting compounds. This project also highlights some of the practical challenges in reproducing published computational methods and the opportunities that open-source software can offer to the community.

Large-Scale Modeling of Multispecies Acute Toxicity End Points Using Consensus of Multitask Deep Learning Methods

Computational methods to predict molecular properties regarding safety and toxicology represent alternative approaches to expedite drug development, screen environmental chemicals, and thus significantly reduce associated time and costs. There is a strong need and interest in the development of computational methods that yield reliable predictions of toxicity, and many approaches, including the recently introduced deep neural networks, have been leveraged towards this goal. Herein, we report on the collection, curation, and integration of data from the public data sets that were the source of the ChemIDplus database for systemic acute toxicity. These efforts generated the largest publicly available such data set comprising > 80,000 compounds measured against a total of 59 acute systemic toxicity end points. This data was used for developing multiple single- and multitask models utilizing random forest, deep neural networks, convolutional, and graph convolutional neural network approaches. For the first time, we also reported the consensus models based on different multitask approaches. To the best of our knowledge, prediction models for 36 of the 59 end points have never been published before. Furthermore, our results demonstrated a significantly better performance of the consensus model obtained from three multitask learning approaches that particularly predicted the 29 smaller tasks (less than 300 compounds) better than other models developed in the study. The curated data set and the developed models have been made publicly available at https://github.com/ncats/ld50-multitask, https://predictor.ncats.io/, and https://cactus.nci.nih.gov/download/acute-toxicity-db (data set only) to support regulatory and research applications.

Current development of integrated web servers for preclinical safety and pharmacokinetics assessments in drug development

In drug development, preclinical safety and pharmacokinetics assessments of candidate drugs to ensure the safety profile are a must. While in vivo and in vitro tests are traditionally used, experimental determinations have disadvantages, as they are usually time-consuming and costly. In silico predictions of these preclinical endpoints have each been developed in the past decades. However, only a few web-based tools have integrated different models to provide a simple one-step platform to help researchers thoroughly evaluate potential drug candidates. To efficiently achieve this approach, a platform for preclinical evaluation must not only predict key ADMET (absorption, distribution, metabolism, excretion and toxicity) properties but also provide some guidance on structural modifications to improve the undesired properties. In this review, we organized and compared several existing integrated web servers that can be adopted in preclinical drug development projects to evaluate the subject of interest. We also introduced our new web server, Virtual Rat, as an alternative choice to profile the properties of drug candidates. In Virtual Rat, we provide not only predictions of important ADMET properties but also possible reasons as to why the model made those structural predictions. Multiple models were implemented into Virtual Rat, including models for predicting human ether-a-go-go-related gene (hERG) inhibition, cytochrome P450 (CYP) inhibition, mutagenicity (Ames test), blood-brain barrier penetration, cytotoxicity and Caco-2 permeability. Virtual Rat is free and has been made publicly available at https://virtualrat.cmdm.tw/.

QSPR/QSAR: State-of-Art, Weirdness, the Future

Ability of quantitative structure-property/activity relationships (QSPRs/QSARs) to serve for epistemological processes in natural sciences is discussed. Some weirdness of QSPR/QSAR state-of-art is listed. There are some contradictions in the research results in this area. Sometimes, these should be classified as paradoxes or weirdness. These points are often ignored. Here, these are listed and briefly commented. In addition, hypotheses on the future evolution of the QSPR/QSAR theory and practice are suggested. In particular, the possibility of extending of the QSPR/QSAR problematic by searching for the "statistical similarity" of different endpoints is suggested and illustrated by an example for relatively "distanced each from other" endpoints, namely (i) mutagenicity, (ii) anticancer activity, and (iii) blood-brain barrier.

Towards reproducible computational drug discovery

The reproducibility of experiments has been a long standing impediment for further scientific progress. Computational methods have been instrumental in drug discovery efforts owing to its multifaceted utilization for data collection, pre-processing, analysis and inference. This article provides an in-depth coverage on the reproducibility of computational drug discovery. This review explores the following topics: (1) the current state-of-the-art on reproducible research, (2) research documentation (e.g. electronic laboratory notebook, Jupyter notebook, etc.), (3) science of reproducible research (i.e. comparison and contrast with related concepts as replicability, reusability and reliability), (4) model development in computational drug discovery, (5) computational issues on model development and deployment, (6) use case scenarios for streamlining the computational drug discovery protocol. In computational disciplines, it has become common practice to share data and programming codes used for numerical calculations as to not only facilitate reproducibility, but also to foster collaborations (i.e. to drive the project further by introducing new ideas, growing the data, augmenting the code, etc.). It is therefore inevitable that the field of computational drug design would adopt an open approach towards the collection, curation and sharing of data/code.

Next generation physiologically based kinetic (NG-PBK) models in support of regulatory decision making

The fields of toxicology and chemical risk assessment seek to reduce, and eventually replace, the use of animals for the prediction of toxicity in humans. In this context, physiologically based kinetic (PBK) modelling based on in vitro and in silico kinetic data has the potential to a play significant role in reducing animal testing, by providing a methodology capable of incorporating in vitro human data to facilitate the development of in vitro to in vivo extrapolation of hazard information. In the present article, we discuss the challenges in: 1) applying PBK modelling to support regulatory decision making under the toxicology and risk-assessment paradigm shift towards animal replacement; 2) constructing PBK models without in vivo animal kinetic data, while relying solely on in vitro or in silico methods for model parameterization; and 3) assessing the validity and credibility of PBK models built largely using non-animal data. The strengths, uncertainties, and limitations of PBK models developed using in vitro or in silico data are discussed in an effort to establish a higher degree of confidence in the application of such models in a regulatory context. The article summarises the outcome of an expert workshop hosted by the European Commission Joint Research Centre (EC-JRC) - European Union Reference Laboratory for Alternatives to Animal Testing (EURL ECVAM), on "Physiologically-Based Kinetic modelling in risk assessment - reaching a whole new level in regulatory decision-making" held in Ispra, Italy, in November 2016, along with results from an international survey conducted in 2017 and recently reported activities occurring within the PBK modelling field. The discussions presented herein highlight the potential applications of next generation (NG)-PBK modelling, based on new data streams.

Study of the Applicability Domain of the QSAR Classification Models by Means of the Rivality and Modelability Indexes

The reliability of a QSAR classification model depends on its capacity to achieve confident predictions of new compounds not considered in the building of the model. The results of this external validation process show the applicability domain (AD) of the QSAR model and, therefore, the robustness of the model to predict the property/activity of new molecules. In this paper we propose the use of the rivality and modelability indexes for the study of the characteristics of the datasets to be correctly modeled by a QSAR algorithm and to predict the reliability of the built model to prognosticate the property/activity of new molecules. The calculation of these indexes has a very low computational cost, not requiring the building of a model, thus being good tools for the analysis of the datasets in the first stages of the building of QSAR classification models. In our study, we have selected two benchmark datasets with similar number of molecules but with very different modelability and we have corroborated the capacity of the predictability of the rivality and modelability indexes regarding the classification models built using Support Vector Machine and Random Forest algorithms with 5-fold cross-validation and leave-one-out techniques. The results have shown the excellent ability of both indexes to predict outliers and the applicability domain of the QSAR classification models. In all cases, these values accurately predicted the statistic parameters of the QSAR models generated by the algorithms.

How Precise Are Our Quantitative Structure-Activity Relationship Derived Predictions for New Query Chemicals?

Quantitative structure-activity relationship (QSAR) models have long been used for making predictions and data gap filling in diverse fields including medicinal chemistry, predictive toxicology, environmental fate modeling, materials science, agricultural science, nanoscience, food science, and so forth. Usually a QSAR model is developed based on chemical information of a properly designed training set and corresponding experimental response data while the model is validated using one or more test set(s) for which the experimental response data are available. However, it is interesting to estimate the reliability of predictions when the model is applied to a completely new data set (true external set) even when the new data points are within applicability domain (AD) of the developed model. In the present study, we have categorized the quality of predictions for the test set or true external set into three groups (good, moderate, and bad) based on absolute prediction errors. Then, we have used three criteria [(a) mean absolute error of leave-one-out predictions for 10 most close training compounds for each query molecule; (b) AD in terms of similarity based on the standardization approach; and (c) proximity of the predicted value of the query compound to the mean training response] in different weighting schemes for making a composite score of predictions. It was found that using the most frequently appearing weighting scheme 0.5-0-0.5, the composite score-based categorization showed concordance with absolute prediction error-based categorization for more than 80% test data points while working with 5 different datasets with 15 models for each set derived in three different splitting techniques. These observations were also confirmed with true external sets for another four endpoints suggesting applicability of the scheme to judge the reliability of predictions for new datasets. The scheme has been implemented in a tool "Prediction Reliability Indicator" available at http://dtclab.webs.com/software-tools and http://teqip.jdvu.ac.in/QSAR_Tools/DTCLab/, and the tool is presently valid for multiple linear regression models only.