The utility of structure-activity relationship (SAR) models for prediction and covariate selection in developmental toxicity: comparative analysis of logistic regression and decision tree models

A QSAR study for predicting malformation in zebrafish embryo

BACKGROUND

Developmental toxicity tests are extremely expensive, require a large number of animals, and are time-consuming. It is necessary to develop a new approach to simplify the analysis of developmental endpoints. One of these endpoints is malformation, and one group of ongoing methods for simplifying is in silico models. In this study, we aim to develop a quantitative structure-activity relationship (QSAR) model and identify the best algorithm for predicting malformations, as well as the most important and effective physicochemical properties associated with malformation.

METHODS

The dataset was extracted from a reliable database called COMPTOX. Physicochemical properties (descriptors) were calculated using Mordred and RDKit chemoinformatics software. The data were cleaned, preprocessed, and then split into training and testing sets. Machine learning algorithms, such as gradient boosting model (GBM) and logistic regression (LR), as well as deep learning models, including multilayer perceptron (MLP) and neural networks (NNs) trained with train set data and different sets of descriptors. The models were then validated with test set and various statistical parameters, such as Matthew's correlation coefficient (MCC) and balanced accuracy (BAC) score, were used to compare the models.

RESULTS

A set of descriptors containing with 78% AUC was identified as the best set of descriptors. Gradient boosting was determined to be the best algorithm with 78% predictive power.

CONCLUSIONS

The descriptors that were the most effective for developing models directly impact the mechanism of malformation, and GBM is the best model due to its MCC and BAC.

Where developmental toxicity meets explainable artificial intelligence: state-of-the-art and perspectives

INTRODUCTION

The application of Artificial Intelligence (AI) to predictive toxicology is rapidly increasing, particularly aiming to develop non-testing methods that effectively address ethical concerns and reduce economic costs. In this context, Developmental Toxicity (Dev Tox) stands as a key human health endpoint, especially significant for safeguarding maternal and child well-being.

AREAS COVERED

This review outlines the existing methods employed in Dev Tox predictions and underscores the benefits of utilizing New Approach Methodologies (NAMs), specifically focusing on eXplainable Artificial Intelligence (XAI), which proves highly efficient in constructing reliable and transparent models aligned with recommendations from international regulatory bodies.

EXPERT OPINION

The limited availability of high-quality data and the absence of dependable Dev Tox methodologies render XAI an appealing avenue for systematically developing interpretable and transparent models, which hold immense potential for both scientific evaluations and regulatory decision-making.

In silico QSAR modeling to predict the safe use of antibiotics during pregnancy

The use of medicines during pregnancy is a growing public health concern due to the risk of developmental toxicity. Healthcare providers heavily rely on the FDA pregnancy risk categories (A, B, C, D, and X). Antibiotics are among the most prescribed drugs during pregnancy and are often listed under category B or C. However, the risk-benefit assessment may be lacking due to challenges in the clinical toxicology studies on pregnant women, such as ethical concerns. The primary focus of this study is to generate a model that predicts the safe use of antibiotics during pregnancy by using in silico approaches. Thus, a QSAR model was created to assess the FDA pregnancy category (B or C) of antibiotics. The dataset consisted of 97 antibiotics obtained from the FDA. A total of 6420 molecular descriptors were determined via multiple software and various machine learning algorithms were utilized. The performance of the models was measured using internal and external validation. The accuracy (ACC) values of the most successful model were 83.82% for the internal and 94.11% for the external validation. Sensitivity (SE), specificity (SP), MCC, and ROC values were 0.878, 0.778, 0.68, and 0.892 for the internal validation and 0.9, 1, 0.887, and 0.936 for the external validation, respectively. Kappa statistics also indicate that there was a substantial agreement for internal validation with 0.6765 and an almost perfect agreement for external validation with 0.8811. In conclusion, our model can be used as an initial step before pre-clinical and clinical studies to predict the safe use of antibiotics in pregnancy.

2D-Quantitative structure-activity relationship modeling for risk assessment of pharmacotherapy applied during pregnancy

The risk evaluation for pharmacological therapy during pregnancy is critical for maternal and fetal health. The initial risk assessment stage, the risk measurement, begins with pregnancy-labeling categories (A, B, C, D, and X) for pharmaceuticals defined by the US Food and Drug Administration (FDA). Recently, in silico methods have been preferred in toxicology studies to eliminate ethical issues before conducting clinical toxicology studies and animal experiments. Quantitative structure-activity relationship (QSAR) modeling is one of the in silico methodologies. The research focuses on creating a QSAR model that predicts the five FDA pregnancy categories of medications. Our dataset included 868 pharmaceuticals, containing nearly every pharmacological group collected from the FDA. 2D-molecular descriptors were calculated using PaDEL software. Twenty-four QSAR models were developed, and the best four models were discussed. The results of the models were compared according to sensitivity, accuracy, F-score, specificity, receiver operating characteristic (ROC) values, and Matthews correlation coefficient. Considering the statistical results, random forest is the best model for determining the pregnancy risk category of drugs. The accuracy of the model was 76.49% for internal and 93.58% for external validation. According to the kappa statistics, there is an average agreement of 0.583 for internal validation and a perfect agreement of 0.893 for external validation. Because the error rates of the model are very close to 0, the model is highly accurate. Consequently, our novel QSAR model gives guidance on the safe use of pharmaceuticals during pregnancy without requiring animal tests or clinical trials on pregnant women.

Review of the state of science and evaluation of currently available in silico prediction models for reproductive and developmental toxicity: A case study on pesticides

BACKGROUND

In silico methods for toxicity prediction have increased significantly in recent years due to the 3Rs principle. This also applies to predicting reproductive toxicology, which is one of the most critical factors in pesticide approval. The widely used quantitative structure-activity relationship (QSAR) models use experimental toxicity data to create a model that relates experimentally observed toxicity to molecular structures to predict toxicity. Aim of the study was to evaluate the available prediction models for developmental and reproductive toxicity regarding their strengths and weaknesses in a pesticide database.

METHODS

The reproductive toxicity of 315 pesticides, which have a GHS classification by ECHA, was compared with the prediction of different in silico models: VEGA, OECD (Q)SAR Toolbox, Leadscope Model Applier, and CASE Ultra by MultiCASE.

RESULTS

In all models, a large proportion (up to 77%) of all pesticides were outside the chemical space of the model. Analysis of the prediction of remaining pesticides revealed a balanced accuracy of the models between 0.48 and 0.66.

CONCLUSION

Overall, predictions were only meaningful in rare cases and therefore always require evaluation by an expert. The critical factors were the underlying data and determination of molecular similarity, which offer great potential for improvement.

Predicting Prenatal Developmental Toxicity Based On the Combination of Chemical Structures and Biological Data

For hazard identification, classification, and labeling purposes, animal testing guidelines are required by law to evaluate the developmental toxicity potential of new and existing chemical products. However, guideline developmental toxicity studies are costly, time-consuming, and require many laboratory animals. Computational modeling has emerged as a promising, animal-sparing, and cost-effective method for evaluating the developmental toxicity potential of chemicals, such as endocrine disruptors, without the use of animals. We aimed to develop a predictive and explainable computational model for developmental toxicants. To this end, a comprehensive dataset of 1244 chemicals with developmental toxicity classifications was curated from public repositories and literature sources. Data from 2140 toxicological high-throughput screening assays were extracted from PubChem and the ToxCast program for this dataset and combined with information about 834 chemical fragments to group assays based on their chemical-mechanistic relationships. This effort revealed two assay clusters containing 83 and 76 assays, respectively, with high positive predictive rates for developmental toxicants identified with animal testing guidelines (PPV = 72.4 and 77.3% during cross-validation). These two assay clusters can be used as developmental toxicity models and were applied to predict new chemicals for external validation. This study provides a new strategy for constructing alternative chemical developmental toxicity evaluations that can be replicated for other toxicity modeling studies.

A new method for predicting the acute toxicity of carbamate pesticides based on the perspective of binding information with carrier protein

Toxicity is one of the most important factors limiting the success of new drug development. In this paper, we built a fast and convenient new method (Carrier protein binding information-toxicity relationship, CPBITR) for predicting drug acute toxicity based on the perspective of binding information with carrier protein. First, we studied the binding information between carbamate pesticides and human serum albumin (HSA) through various spectroscopic methods and molecular docking. Then a total of 16 models were established to clarify the relationship between binding information with HSA and drug toxicity. The results showed that the binding information was related to toxicity. Finally we obtained the effective toxicity prediction model for carbamate pesticides. And the "Platform for Predicting Drug Toxicity Based on the Information of Binding with Carrier Protein" was established with the Back-propagation neural network model. We proposed and proved that it was feasible to predict drug toxicity from this new perspective: binding with carrier protein. According to this new perspective, toxicity prediction model of other drugs can also be established. This new method has the advantages of convenience and fast, and can be used to screen out low-toxic drugs quickly in the early stage. It is helpful for drug research and development.

In Silico Models for Developmental Toxicity

Modeling developmental toxicity has been a challenge for (Q)SAR model developers due to the complexity of the endpoint. Recently, some new in silico methods have been developed introducing the possibility to evaluate the integration of existing methods by taking advantage of various modeling perspectives. It is important that the model user is aware of the underlying basis of the different models in general, as well as the considerations and assumptions relative to the specific predictions that are obtained from these different models for the same chemical. The evaluation on the predictions needs to be done on a case-by-case basis, checking the analogues (possibly using structural, physicochemical, and toxicological information); for this purpose, the assessment of the applicability domain of the models provides further confidence in the model prediction. In this chapter, we present some examples illustrating an approach to combine human-based rules and statistical methods to support the prediction of developmental toxicity; we also discuss assumptions and uncertainties of the methodology.

Comparison of predictions of developmental toxicity for compounds of solvent data set

We have considered a series of 235 compounds technically classified as solvents. Chemically, they belong to different classes. Their potential developmental toxicity was evaluated using two models available on platform VEGA HUB; model CAESAR and the Developmental/ Reproductive Toxicity library (PG) model. Models provide beside the prediction of developmental toxicity additional information on similar compounds from models' training sets. In the report, first, we compare the predictions. Second, the sets of similar compounds have been used to implement the clustering scheme. The Kohonen artificial neural network method has been applied as a clustering method. The clusters obtained have been discussed for both models.

The Tox21 10K Compound Library: Collaborative Chemistry Advancing Toxicology

Since 2009, the Tox21 project has screened ∼8500 chemicals in more than 70 high-throughput assays, generating upward of 100 million data points, with all data publicly available through partner websites at the United States Environmental Protection Agency (EPA), National Center for Advancing Translational Sciences (NCATS), and National Toxicology Program (NTP). Underpinning this public effort is the largest compound library ever constructed specifically for improving understanding of the chemical basis of toxicity across research and regulatory domains. Each Tox21 federal partner brought specialized resources and capabilities to the partnership, including three approximately equal-sized compound libraries. All Tox21 data generated to date have resulted from a confluence of ideas, technologies, and expertise used to design, screen, and analyze the Tox21 10K library. The different programmatic objectives of the partners led to three distinct, overlapping compound libraries that, when combined, not only covered a diversity of chemical structures, use-categories, and properties but also incorporated many types of compound replicates. The history of development of the Tox21 "10K" chemical library and data workflows implemented to ensure quality chemical annotations and allow for various reproducibility assessments are described. Cheminformatics profiling demonstrates how the three partner libraries complement one another to expand the reach of each individual library, as reflected in coverage of regulatory lists, predicted toxicity end points, and physicochemical properties. ToxPrint chemotypes (CTs) and enrichment approaches further demonstrate how the combined partner libraries amplify structure-activity patterns that would otherwise not be detected. Finally, CT enrichments are used to probe global patterns of activity in combined ToxCast and Tox21 activity data sets relative to test-set size and chemical versus biological end point diversity, illustrating the power of CT approaches to discern patterns in chemical-activity data sets. These results support a central premise of the Tox21 program: A collaborative merging of programmatically distinct compound libraries would yield greater rewards than could be achieved separately.

The Tox21 10K Compound Library: Collaborative Chemistry Advancing Toxicology

Since 2009, the Tox21 project has screened ∼8500 chemicals in more than 70 high-throughput assays, generating upward of 100 million data points, with all data publicly available through partner websites at the United States Environmental Protection Agency (EPA), National Center for Advancing Translational Sciences (NCATS), and National Toxicology Program (NTP). Underpinning this public effort is the largest compound library ever constructed specifically for improving understanding of the chemical basis of toxicity across research and regulatory domains. Each Tox21 federal partner brought specialized resources and capabilities to the partnership, including three approximately equal-sized compound libraries. All Tox21 data generated to date have resulted from a confluence of ideas, technologies, and expertise used to design, screen, and analyze the Tox21 10K library. The different programmatic objectives of the partners led to three distinct, overlapping compound libraries that, when combined, not only covered a diversity of chemical structures, use-categories, and properties but also incorporated many types of compound replicates. The history of development of the Tox21 "10K" chemical library and data workflows implemented to ensure quality chemical annotations and allow for various reproducibility assessments are described. Cheminformatics profiling demonstrates how the three partner libraries complement one another to expand the reach of each individual library, as reflected in coverage of regulatory lists, predicted toxicity end points, and physicochemical properties. ToxPrint chemotypes (CTs) and enrichment approaches further demonstrate how the combined partner libraries amplify structure-activity patterns that would otherwise not be detected. Finally, CT enrichments are used to probe global patterns of activity in combined ToxCast and Tox21 activity data sets relative to test-set size and chemical versus biological end point diversity, illustrating the power of CT approaches to discern patterns in chemical-activity data sets. These results support a central premise of the Tox21 program: A collaborative merging of programmatically distinct compound libraries would yield greater rewards than could be achieved separately.

An automated framework for compiling and integrating chemical hazard data

Comparative chemical hazard assessment, which compares hazards for several endpoints across several chemicals, can be used for a variety of purposes including alternatives assessment and the prioritization of chemicals for further assessment. A new framework was developed to compile and integrate chemical hazard data for several human health and ecotoxicity endpoints from public online sources including hazardous chemical lists, Globally Harmonized System hazard codes (H-codes) or hazard categories from government health agencies, experimental quantitative toxicity values, and predicted values using Quantitative Structure-Activity Relationship (QSAR) models. QSAR model predictions were obtained using EPA's Toxicity Estimation Software Tool. Java programming was used to download hazard data, convert data from each source into a consistent score record format, and store the data in a database. Scoring criteria based on the EPA's Design for the Environment Program Alternatives Assessment Criteria for Hazard Evaluation were used to determine ordinal hazard scores (i.e., low, medium, high, or very high) for each score record. Different methodologies were assessed for integrating data from multiple sources into one score for each hazard endpoint for each chemical. The chemical hazard assessment (CHA) Database developed in this study currently contains more than 990,000 score records for more than 85,000 chemicals. The CHA Database and the methods used in its development may contribute to several cheminformatics, public health, and environmental activities.

Leveraging complementary computational models for prioritizing chemicals of developmental and reproductive toxicity concern: an example of food contact materials

The evaluation of developmental and reproductive toxicity of food contact materials (FCMs) is an important task for food safety. Since traditional experiments are both time-consuming and labor-intensive, only a small number of FCMs have sufficient toxicological data for evaluating their effects on human health. While computational methods such as structural alerts and quantitative structure-activity relationships can serve as first-line tools for the identification of chemicals of high toxicity concern, models with binary outputs and unsatisfied accuracy and coverage prevent the use of computational methods for prioritizing chemicals of high concern. This study proposed a genetic algorithm-based method to develop a weight-of-evidence (WoE) model leveraging complementary methods of structural alerts, quantitative structure-activity relationships and in silico toxicogenomics models for chemical prioritization. The WoE model was applied to evaluate 623 food contact chemicals and identify 26 chemicals of high toxicity concern, where 13 chemicals have been reported to be developmental or reproductive toxic and further experiments are suggested for the remaining 13 chemicals without toxicity data related to developmental and reproductive effects. The proposed WoE model is potentially useful for prioritizing chemicals of high toxicity concern and the methodology may be applied to toxicities other than developmental and reproductive toxicity.

In silico prediction of drug-induced developmental toxicity by using machine learning approaches

Some drugs and xenobiotics have the potential to disturb homeostasis, normal growth, differentiation, development or behavior during prenatal development or postnatally until puberty. Assessment of the developmental toxicity is one of the important safety considerations incorporated by international regulatory agencies. In this investigation, seven machine learning methods, including naïve Bayes, support vector machine, recursive partitioning, k-nearest neighbor, C4.5 decision tree, random forest and Adaboost, were used to build binary classification models for developmental toxicity. Among these models, the naïve Bayes classifier represented the best predictive performance and stability, which gave 91.11% overall prediction accuracy, 91.50% balanced accuracy and 0.818 MCC for the training set, and generated 83.93% concordance, 81.85% balanced accuracy and 0.627 MCC for the test set. The application domains were analyzed, and only one chemical in the test set was identified as outside the application domain. In addition, 10 important molecular descriptors related to developmental toxicity were selected by the genetic algorithm, which may contribute to explanation of the mechanisms of developmental toxicants. The best naïve Bayes classification model should be employed as alternative method for qualitative prediction of chemical-induced developmental toxicity in early stages of drug development.

Grouping of CoMPARA data with respect to compounds from the carcinogenic potency database

Methods for clustering and measures of similarity of chemical structures have become an important supporting tool in chemoinformatics. They represent the basis for categorization of chemicals and read-across, where a molecular property is estimated from 'similar molecules'. This study proposes a clustering scheme within the given dataset with respect to a reference dataset. The scheme was applied on two datasets ToxCast_AR_Agonist and ToxCast_AR_Antagonists with 1654 and 1522 compounds, respectively. The compounds are tested to androgen receptor activity (AR) in 11 high throughput screening assays. The carcinogenic dataset was used as the reference set.

A SAR-based mechanistic study on the combined toxicities of sulfonamides and quorum sensing inhibitors on Escherichia coli

Quorum sensing inhibitors (QSIs) are promising alternatives to antibiotics, but they are discharged into the environment after their use cycle. This poses joint effects on the organisms in the environment. Therefore, it is of great importance to study the combined toxicities of QSIs and antibiotics. In this study, we investigated the single and combined toxicities of four potential QSIs and 11 sulfonamides (SAs) on Escherichia coli. The results revealed that the single toxicities of SAs were greater than those of QSIs, and the toxicities were found positively related to the binding energies (E_bind) with their target proteins, for both antibiotics and QSIs. The combined toxicities of the binary mixtures were observed to be either antagonism or addition. The antagonism could be explained by the phenomenon that QSIs changed SAs molecules into ionic forms, preventing the SA molecules entering the bacteria. Furthermore, it was found that the ratios of the effective concentration (the actual concentration involved in the interaction with the proteins) in the antagonistic cases were higher than those in the additive cases. This study would benefit both rational use of the drug combination and ecological risk assessment of antibiotics and QSIs in the real environment.

Development of novel in silico model for developmental toxicity assessment by using naïve Bayes classifier method

Toxicological testing associated with developmental toxicity endpoints are very expensive, time consuming and labor intensive. Thus, developing alternative approaches for developmental toxicity testing is an important and urgent task in the drug development filed. In this investigation, the naïve Bayes classifier was applied to develop a novel prediction model for developmental toxicity. The established prediction model was evaluated by the internal 5-fold cross validation and external test set. The overall prediction results for the internal 5-fold cross validation of the training set and external test set were 96.6% and 82.8%, respectively. In addition, four simple descriptors and some representative substructures of developmental toxicants were identified. Thus, we hope the established in silico prediction model could be used as alternative method for toxicological assessment. And these obtained molecular information could afford a deeper understanding on the developmental toxicants, and provide guidance for medicinal chemists working in drug discovery and lead optimization.

In silico prediction of toxicity of phenols to Tetrahymena pyriformis by using genetic algorithm and decision tree-based modeling approach

Risk assessment of chemicals is an important issue in environmental protection; however, there is a huge lack of experimental data for a large number of end-points. The experimental determination of toxicity of chemicals involves high costs and time-consuming process. In silico tools such as quantitative structure-toxicity relationship (QSTR) models, which are constructed on the basis of computational molecular descriptors, can predict missing data for toxic end-points for existing or even not yet synthesized chemicals. Phenol derivatives are known to be aquatic pollutants. With this background, we aimed to develop an accurate and reliable QSTR model for the prediction of toxicity of 206 phenols to Tetrahymena pyriformis. A multiple linear regression (MLR)-based QSTR was obtained using a powerful descriptor selection tool named Memorized_ACO algorithm. Statistical parameters of the model were 0.72 and 0.68 for R_training² and R_test², respectively. To develop a high-quality QSTR model, classification and regression tree (CART) was employed. Two approaches were considered: (1) phenols were classified into different modes of action using CART and (2) the phenols in the training set were partitioned to several subsets by a tree in such a manner that in each subset, a high-quality MLR could be developed. For the first approach, the statistical parameters of the resultant QSTR model were improved to 0.83 and 0.75 for R_training² and R_test², respectively. Genetic algorithm was employed in the second approach to obtain an optimal tree, and it was shown that the final QSTR model provided excellent prediction accuracy for the training and test sets (R_training² and R_test² were 0.91 and 0.93, respectively). The mean absolute error for the test set was computed as 0.1615.

Literature Review of (Q)SAR Modelling of Nanomaterial Toxicity

Despite the clear benefits that nanotechnology can bring to various sectors of industry, there are serious concerns about the potential health risks associated with engineered nanomaterials (ENMs), intensified by the limited understanding of what makes ENMs toxic and how to make them safe. As the use of ENMs for commercial purposes and the number of workers/end-users being exposed to these materials on a daily basis increases, the need for assessing the potential adverse effects of multifarious ENMs in a time- and cost-effective manner becomes more apparent. One strategy to alleviate the problem of testing a large number and variety of ENMs in terms of their toxicological properties is through the development of computational models that decode the relationships between the physicochemical features of ENMs and their toxicity. Such data-driven models can be used for hazard screening, early identification of potentially harmful ENMs and the toxicity-governing physicochemical properties, and accelerating the decision-making process by maximising the use of existing data. Moreover, these models can also support industrial, regulatory and public needs for designing inherently safer ENMs. This chapter is mainly concerned with the investigation of the applicability of (quantitative) structure-activity relationship ((Q)SAR) methods to modelling of ENMs' toxicity. It summarizes the key components required for successful application of data-driven toxicity prediction techniques to ENMs, the published studies in this field and the current limitations of this approach.

The toxicity of crude 4-methylcyclohexanemethanol (MCHM): review of experimental data and results of predictive models for its constituents and a putative metabolite

Crude 4-methylcyclohexanemethanol (MCHM) is an industrial solvent used to clean coal. Approximately 10 000 gallons of a liquid mixture containing crude MCHM were accidently released into the Elk River in West Virginia in January 2014. Because of the proximity to a water treatment facility, the contaminated water was distributed to approximately 300 000 residents. In this review, experimental data and computational predictions for the toxicity for crude MCHM, distilled MCHM, its other components and its putative metabolites are presented. Crude MCHM, its other constituents and its metabolites have low to moderate acute and subchronic oral toxicity. Crude MCHM has been shown not to be a skin sensitizer below certain doses, indicating that at plausible human exposures it does not cause an allergic response. Crude MCHM and its constituents cause slight to moderate skin and eye irritation in rodents at high concentrations. These chemicals are not mutagenic and are not predicted to be carcinogenic. Several of the constituents were predicted through modeling to be possible developmental toxicants; however, 1,4-cyclohexanedimethanol, 1,4-cyclohexanedicarboxylic acid and dimethyl 1,4-cyclohexanedicarboxylate did not demonstrate developmental toxicity in rat studies. Following the spill, the Centers for Disease Control and Prevention recommended a short-term health advisory level of 1 ppm for drinking water that it determined was unlikely to be associated with adverse health effects. Crude MCHM has an odor threshold lower than 10 ppb, indicating that it could be detected at concentrations at least 100-fold less than this risk criterion. Collectively, the findings and predictions indicate that crude MCHM poses no apparent toxicological risk to humans at 1 ppm in household water.

Special endpoint and product specific considerations in pharmaceutical acceptable daily exposure derivation

Recently, a guideline has been published by the European Medicines Agency (EMA) on setting safe limits, permitted daily exposures (PDE) [also called acceptable daily exposures (ADE)], for medicines manufactured in multi-product facilities. The ADE provides a safe exposure limit for inadvertent exposure of a drug due to cross-contamination in manufacturing. The ADE determination encompasses a standard risk assessment, requiring an understanding of the toxicological and pharmacological effects, the mechanism of action, drug compound class, and the dose-response as well as the pharmacokinetic properties of the compound. While the ADE concept has broad application in pharmaceutical safety there are also nuances and specific challenges associated with some toxicological endpoints or drug product categories. In this manuscript we discuss considerations for setting ADEs when the following specific adverse health endpoints may constitute the critical effect: genotoxicity, developmental and reproductive toxicity (DART), and immune system modulation (immunostimulation or immunosuppression), and for specific drug classes, including antibody drug conjugates (ADCs), emerging medicinal therapeutic compounds, and compounds with limited datasets. These are challenging toxicological scenarios that require a careful evaluation of all of the available information in order to establish a health-based safe level.

In Silico Model for Developmental Toxicity: How to Use QSAR Models and Interpret Their Results

Modeling developmental toxicity has been a challenge for (Q)SAR model developers due to the complexity of the endpoint. Recently, some new in silico methods have been developed introducing the possibility to evaluate the integration of existing methods by taking advantage of various modeling perspectives. It is important that the model user is aware of the underlying basis of the different models in general, as well as the considerations and assumptions relative to the specific predictions that are obtained from these different models for the same chemical. The evaluation on the predictions needs to be done on a case-by-case basis, checking the analogs (possibly using structural, physicochemical, and toxicological information); for this purpose, the assessment of the applicability domain of the models provides further confidence in the model prediction. In this chapter, we present some examples illustrating an approach to combine human-based rules and statistical methods to support the prediction of developmental toxicity; we also discuss assumptions and uncertainties of the methodology.

Evaluation of toxic endpoints for a set of cosmetic ingredients with CAESAR models

The randomly selected set of 558 chemicals from Cosmetic inventory was studied with internet accessible program package CAESAR. Four toxic endpoints were considered: mutagenicity, carcinogenicity, developmental toxicity and skin sensitization. The CAESAR program provides beside the predictions comprehensive information on applicability domain and the similarity between the considered compound and the compounds from model's training set. This information was used to implement for clustering and classification of chemicals. As the technique the Self Organizing Maps was applied. This technique also enables us to define to each cluster the cluster indicator, i.e., the characteristic compound, which is considered as a representative for a cluster.

A novel approach to generate robust classification models to predict developmental toxicity from imbalanced datasets

Computational models to predict the developmental toxicity of compounds are built on imbalanced datasets wherein the toxicants outnumber the non-toxicants. Consequently, the results are biased towards the majority class (toxicants). To overcome this problem and to obtain sensitive but also accurate classifiers, we followed an integrated approach wherein (i) Synthetic Minority Over Sampling (SMOTE) is used for re-sampling, (ii) genetic algorithm (GA) is used for variable selection and (iii) support vector machines (SVM) is used for model development. The best model, M3, has (i) sensitivity (SE) = 85.54% and specificity (SP) = 85.62% in leave-one-out validation, (ii) classification accuracy of the training set = 99.67%, (iii) classification accuracy of the test set = 92.59%; and (iv) sensitivity = 92.68, specificity = 92.31 on the test set. Consensus prediction based on models M3-M5 improved these percentages by 5% over M3. From the analysis of results we infer that data imbalance in toxicity studies can be effectively addressed by the application of re-sampling techniques.

Random generalized linear model: a highly accurate and interpretable ensemble predictor

BACKGROUND

Ensemble predictors such as the random forest are known to have superior accuracy but their black-box predictions are difficult to interpret. In contrast, a generalized linear model (GLM) is very interpretable especially when forward feature selection is used to construct the model. However, forward feature selection tends to overfit the data and leads to low predictive accuracy. Therefore, it remains an important research goal to combine the advantages of ensemble predictors (high accuracy) with the advantages of forward regression modeling (interpretability). To address this goal several articles have explored GLM based ensemble predictors. Since limited evaluations suggested that these ensemble predictors were less accurate than alternative predictors, they have found little attention in the literature.

RESULTS

Comprehensive evaluations involving hundreds of genomic data sets, the UCI machine learning benchmark data, and simulations are used to give GLM based ensemble predictors a new and careful look. A novel bootstrap aggregated (bagged) GLM predictor that incorporates several elements of randomness and instability (random subspace method, optional interaction terms, forward variable selection) often outperforms a host of alternative prediction methods including random forests and penalized regression models (ridge regression, elastic net, lasso). This random generalized linear model (RGLM) predictor provides variable importance measures that can be used to define a "thinned" ensemble predictor (involving few features) that retains excellent predictive accuracy.

CONCLUSION

RGLM is a state of the art predictor that shares the advantages of a random forest (excellent predictive accuracy, feature importance measures, out-of-bag estimates of accuracy) with those of a forward selected generalized linear model (interpretability). These methods are implemented in the freely available R software package randomGLM.

Estimating misclassification error: a closer look at cross-validation based methods

Background

To estimate a classifier’s error in predicting future observations, bootstrap methods have been proposed as reduced-variation alternatives to traditional cross-validation (CV) methods based on sampling without replacement. Monte Carlo (MC) simulation studies aimed at estimating the true misclassification error conditional on the training set are commonly used to compare CV methods. We conducted an MC simulation study to compare a new method of bootstrap CV (BCV) to k-fold CV for estimating clasification error.

Findings

For the low-dimensional conditions simulated, the modest positive bias of k-fold CV contrasted sharply with the substantial negative bias of the new BCV method. This behavior was corroborated using a real-world dataset of prognostic gene-expression profiles in breast cancer patients. Our simulation results demonstrate some extreme characteristics of variance and bias that can occur due to a fault in the design of CV exercises aimed at estimating the true conditional error of a classifier, and that appear not to have been fully appreciated in previous studies. Although CV is a sound practice for estimating a classifier’s generalization error, using CV to estimate the fixed misclassification error of a trained classifier conditional on the training set is problematic. While MC simulation of this estimation exercise can correctly represent the average bias of a classifier, it will overstate the between-run variance of the bias.

Conclusions

We recommend k-fold CV over the new BCV method for estimating a classifier’s generalization error. The extreme negative bias of BCV is too high a price to pay for its reduced variance.

CAESAR models for developmental toxicity

Background

The new REACH legislation requires assessment of a large number of chemicals in the European market for several endpoints. Developmental toxicity is one of the most difficult endpoints to assess, on account of the complexity, length and costs of experiments. Following the encouragement of QSAR (in silico) methods provided in the REACH itself, the CAESAR project has developed several models.

Results

Two QSAR models for developmental toxicity have been developed, using different statistical/mathematical methods. Both models performed well. The first makes a classification based on a random forest algorithm, while the second is based on an adaptive fuzzy partition algorithm. The first model has been implemented and inserted into the CAESAR on-line application, which is java-based software that allows everyone to freely use the models.

Conclusions

The CAESAR QSAR models have been developed with the aim to minimize false negatives in order to make them more usable for REACH. The CAESAR on-line application ensures that both industry and regulators can easily access and use the developmental toxicity model (as well as the models for the other four endpoints).

QSAR models for reproductive toxicity and endocrine disruption activity

Reproductive toxicity is an important regulatory endpoint, which is required in registration procedures of chemicals used for different purposes (for example pesticides). The in vivo tests are expensive, time consuming and require large numbers of animals, which must be sacrificed. Therefore an effort is ongoing to develop alternative In vitro and in silico methods to evaluate reproductive toxicity. In this review we describe some modeling approaches. In the first example we describe the CAESAR model for prediction of reproductive toxicity; the second example shows a classification model for endocrine disruption potential based on counter propagation artificial neural networks; the third example shows a modeling of relative binding affinity to rat estrogen receptor, and the fourth one shows a receptor dependent modeling experiment.

Integrated Decision-tree Testing Strategies for Developmental and Reproductive Toxicity with Respect to the Requirements of the EU REACH Legislation

Liverpool John Moores University and FRAME conducted a research project, sponsored by Defra, on the status of alternatives to animal testing with regard to the European Union REACH (Registration, Evaluation and Authorisation of Chemicals) system for the safety testing and risk assessment of chemicals. The project covered all the main toxicity endpoints associated with the REACH system. This paper focuses on the prospects for the use of alternative methods (both in vitro and in silico) in developmental and reproductive toxicity testing. It considers many tests based on primary cells and cell lines, and the available expert systems and QSARs for developmental and reproductive toxicity, and also covers tests for endocrine disruption. Ways in which reduction and refinement measures can be used are also discussed, particularly the use of an enhanced one-generation reproductive study, which could potentially replace the two-generation study, and therefore considerably reduce the number of animals required in reproductive toxicity. Decision-tree style integrated testing strategies are also proposed for developmental and reproductive toxicity and for endocrine disruption, followed by a number of recommendations for the future facilitation of developmental and reproductive toxicity testing, with respect to human risk assessment.

Integrated Decision-tree Testing Strategies for Developmental and Reproductive Toxicity with Respect to the Requirements of the EU REACH Legislation

Liverpool John Moores University and FRAME conducted a research project, sponsored by Defra, on the status of alternatives to animal testing with regard to the European Union REACH (Registration, Evaluation and Authorisation of Chemicals) system for the safety testing and risk assessment of chemicals. The project covered all the main toxicity endpoints associated with the REACH system. This paper focuses on the prospects for the use of alternative methods (both in vitro and in silico) in developmental and reproductive toxicity testing. It considers many tests based on primary cells and cell lines, and the available expert systems and QSARs for developmental and reproductive toxicity, and also covers tests for endocrine disruption. Ways in which reduction and refinement measures can be used are also discussed, particularly the use of an enhanced one-generation reproductive study, which could potentially replace the two-generation study, and therefore considerably reduce the number of animals required in reproductive toxicity. Decision-tree style integrated testing strategies are also proposed for developmental and reproductive toxicity and for endocrine disruption, followed by a number of recommendations for the future facilitation of developmental and reproductive toxicity testing, with respect to human risk assessment.

Facilitating compound progression of antiretroviral agents via modeling and simulation

Pharmacotherapy in human immunodeficiency virus (HIV)-infected patients and the development of safe and effective antiretroviral dosing regimens has been hindered by numerous issues, including the rapid development of viral resistance to drug therapy, the narrow therapeutic window of the drug compounds, and lack of fundamental knowledge concerning the sources of variation in exposure and response to antiretroviral agents. Sources of variation may include factors such as interpatient differences in genetic expression, immunological response, pathogenesis, epidemiologic and socioeconomic factors, and demographics. Modeling and simulation (M&S) techniques have become valuable tools to identify and quantify variability in exposure and response to antiretroviral agents throughout the drug development process. Before actual entry into human safety and pharmacokinetic (PK) trials, in vitro screening and in vivo pharmacology studies conducted to assess compound potency and compatibility with agents included in acceptable antiretroviral therapy (ART) regimens can be characterized via quantitative relationships. In addition, physiochemical data is initially used to screen drug candidates based on favorable PK and biopharmaceutic properties. Compound progression can likewise be supported with M&S exercises to ensure the traceability of key assumptions and decisions. The underlying techniques utilize nonlinear mixed effect modeling, Monte Carlo simulation, Neural networks, several regression-based approaches, and less computationally intensive techniques. The application of such an approach promises to be an essential component in the development of new agents to treat HIV-1 and is being implemented in the context of evaluating Nk1r antagonists as potential candidates to treat NeuroAIDS.

A comparison of regression trees, logistic regression, generalized additive models, and multivariate adaptive regression splines for predicting AMI mortality

Clinicians and health service researchers are frequently interested in predicting patient-specific probabilities of adverse events (e.g. death, disease recurrence, post-operative complications, hospital readmission). There is an increasing interest in the use of classification and regression trees (CART) for predicting outcomes in clinical studies. We compared the predictive accuracy of logistic regression with that of regression trees for predicting mortality after hospitalization with an acute myocardial infarction (AMI). We also examined the predictive ability of two other types of data-driven models: generalized additive models (GAMs) and multivariate adaptive regression splines (MARS). We used data on 9484 patients admitted to hospital with an AMI in Ontario. We used repeated split-sample validation: the data were randomly divided into derivation and validation samples. Predictive models were estimated using the derivation sample and the predictive accuracy of the resultant model was assessed using the area under the receiver operating characteristic (ROC) curve in the validation sample. This process was repeated 1000 times-the initial data set was randomly divided into derivation and validation samples 1000 times, and the predictive accuracy of each method was assessed each time. The mean ROC curve area for the regression tree models in the 1000 derivation samples was 0.762, while the mean ROC curve area of a simple logistic regression model was 0.845. The mean ROC curve areas for the other methods ranged from a low of 0.831 to a high of 0.851. Our study shows that regression trees do not perform as well as logistic regression for predicting mortality following AMI. However, the logistic regression model had performance comparable to that of more flexible, data-driven models such as GAMs and MARS.

Database composition can affect the structure-activity relationship prediction

The percent active (A) and inactive (I) chemicals in a database can directly affect the sensitivity (% active chemicals predicted correctly) and specificity (% inactive chemicals predicted correctly) of structure-activity relationship (SAR) analyses. Subdividing the National Center for Toxicological Research (NCTR) liver cancer database (NCTRlcdb) into various A/I ratios, which varied from 0.2 to 5.5, resulted in sensitivity/specificity ratios that varied from 0.1 to 6.5. As percent active chemicals increased (increasing A/I ratio), the sensitivity rose, the specificity decreased, and the concordance (% total chemicals predicted correctly) remained fairly constant. The numbers of chemicals in the various data sets ranged from 187 to 999 and appeared to have no affect on any of the 3 predictors of sensitivity, specificity, or concordance.

Decision threshold adjustment in class prediction

Standard classification algorithms are generally designed to maximize the number of correct predictions (concordance). The criterion of maximizing the concordance may not be appropriate in certain applications. In practice, some applications may emphasize high sensitivity (e.g., clinical diagnostic tests) and others may emphasize high specificity (e.g., epidemiology screening studies). This paper considers effects of the decision threshold on sensitivity, specificity, and concordance for four classification methods: logistic regression, classification tree, Fisher's linear discriminant analysis, and a weighted k-nearest neighbor. We investigated the use of decision threshold adjustment to improve performance of either sensitivity or specificity of a classifier under specific conditions. We conducted a Monte Carlo simulation showing that as the decision threshold increases, the sensitivity decreases and the specificity increases; but, the concordance values in an interval around the maximum concordance are similar. For specified sensitivity and specificity levels, an optimal decision threshold might be determined in an interval around the maximum concordance that meets the specified requirement. Three example data sets were analyzed for illustrations.

Classification ensembles for unbalanced class sizes in predictive toxicology

This paper investigates the effects of the ratio of positive-to-negative samples on the sensitivity, specificity, and concordance. When the class sizes in the training samples are not equal, the classification rule derived will favor the majority class and result in a low sensitivity on the minority class prediction. We propose an ensemble classification approach to adjust for differential class sizes in a binary classifier system. An ensemble classifier consists of a set of base classifiers; its prediction rule is based on a summary measure of individual classifications by the base classifiers. Two re-sampling methods, augmentation and abatement, are proposed to generate different bootstrap samples of equal class size to build the base classifiers. The augmentation method balances the two class sizes by bootstrapping additional samples from the minority class, whereas the abatement method balances the two class sizes by sampling only a subset of samples from the majority class. The proposed procedure is applied to a data set to predict estrogen receptor binding activity and to a data set to predict animal liver carcinogenicity using SAR (structure-activity relationship) models as base classifiers. The abatement method appears to perform well in balancing sensitivity and specificity.