Improving Opportunities for Regulatory Acceptance of QSARs: The Importance of Model Domain, Uncertainty, Validity and Predictability

Multi-Strategy Assessment of Different Uses of QSAR under REACH Analysis of Alternatives to Advance Information Transparency

Under the Registration, Evaluation, Authorization, and Restriction of Chemicals (REACH) analysis of alternatives (AoA) process, quantitative structure–activity relationship (QSAR) models play an important role in expanding information gathering and organizing frameworks. Increasingly recognized as an alternative to testing under registration. QSARs have become a relevant tool in bridging data gaps and supporting weight of evidence (WoE) when assessing alternative substances. Additionally, QSARs are growing in importance in integrated testing strategies (ITS). For example, the REACH ITS framework for specific endpoints directs registrants to consider non-testing results, including QSAR predictions, when deciding if further animal testing is needed. Despite the raised profile of QSARs in these frameworks, a gap exists in the evaluation of QSAR use and QSAR documentation under authorization. An assessment of the different uses (e.g., WoE and ITS) in which QSAR predictions play a role in evidence gathering and organizing remains unaddressed for AoA. This study approached the disparity in information for QSAR predictions by conducting a substantive review of 24 AoA through May 2017, which contained higher-tier endpoints under REACH. Understanding the manner in which applicants manage QSAR prediction information in AoA and assessing their potential within ITS will be valuable in promoting regulatory use of QSARs and building out future platforms in the face of rapidly evolving technology while advancing information transparency.

Whether the Validation of the Predictive Potential of Toxicity Models is a Solved Task?

Different kinds of biological activities are defined by complex biochemical interactions, which are termed as a "mathematical function" not only of the molecular structure but also for some additional circumstances, such as physicochemical conditions, interactions via energy and information effects between a substance and organisms, organs, cells. These circumstances lead to the great complexity of prediction for biochemical endpoints, since all "details" of corresponding phenomena are practically unavailable for the accurate registration and analysis. Researchers have not a possibility to carry out and analyse all possible ways of the biochemical interactions, which define toxicological or therapeutically attractive effects via direct experiment. Consequently, a compromise, i.e. the development of predictive models of the above phenomena, becomes necessary. However, the estimation of the predictive potential of these models remains a task that is solved only partially. This mini-review presents a collection of attempts to be used for the above-mentioned task, two special statistical indices are proposed, which may be a measure of the predictive potential of models. These indices are (i) Index of Ideality of Correlation; and (ii) Correlation Contradiction Index.

The study of the index of ideality of correlation as a new criterion of predictive potential of QSPR/QSAR-models

Acetylcholinesterase (AChE) inhibitors, dihydrofolate reductase inhibitors (DHFR), Toxicity in Tetrahymena pyriformis (TP), Acute Toxicity in fathead minnow (TFat), Water solubility (WS), and Acute Aquatic Toxicity in Daphnia magna (DM) are examined as endpoints to establish quantitative structure - property/activity relationships (QSPRs/QSARs). The Index of Ideality of Correlation (IIC) is a measure of predictive potential. The IIC has been studied in a few recent works. The comparison of models for the six endpoints above confirms that the index can be a useful tool for building up and validation of QSPR/QSAR models. All examined endpoints are important from an ecologic point of view. The diversity of examined endpoints confirms that the IIC is real criterion of the predictive potential of a model.

Comparison of Multiple Linear Regressions and Neural Networks based QSAR models for the design of new antitubercular compounds

The performance of two QSAR methodologies, namely Multiple Linear Regressions (MLR) and Neural Networks (NN), towards the modeling and prediction of antitubercular activity was evaluated and compared. A data set of 173 potentially active compounds belonging to the hydrazide family and represented by 96 descriptors was analyzed. Models were built with Multiple Linear Regressions (MLR), single Feed-Forward Neural Networks (FFNNs), ensembles of FFNNs and Associative Neural Networks (AsNNs) using four different data sets and different types of descriptors. The predictive ability of the different techniques used were assessed and discussed on the basis of different validation criteria and results show in general a better performance of AsNNs in terms of learning ability and prediction of antitubercular behaviors when compared with all other methods. MLR have, however, the advantage of pinpointing the most relevant molecular characteristics responsible for the behavior of these compounds against Mycobacterium tuberculosis. The best results for the larger data set (94 compounds in training set and 18 in test set) were obtained with AsNNs using seven descriptors (R(2) of 0.874 and RMSE of 0.437 against R(2) of 0.845 and RMSE of 0.472 in MLRs, for test set). Counter-Propagation Neural Networks (CPNNs) were trained with the same data sets and descriptors. From the scrutiny of the weight levels in each CPNN and the information retrieved from MLRs, a rational design of potentially active compounds was attempted. Two new compounds were synthesized and tested against M. tuberculosis showing an activity close to that predicted by the majority of the models.

Uncertainty in QSAR predictions

It is relevant to consider uncertainty in individual predictions when quantitative structure-activity (or property) relationships (QSARs) are used to support decisions of high societal concern. Successful communication of uncertainty in the integration of QSARs in chemical safety assessment under the EU Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) system can be facilitated by a common understanding of how to define, characterise, assess and evaluate uncertainty in QSAR predictions. A QSAR prediction is, compared to experimental estimates, subject to added uncertainty that comes from the use of a model instead of empirically-based estimates. A framework is provided to aid the distinction between different types of uncertainty in a QSAR prediction: quantitative, i.e. for regressions related to the error in a prediction and characterised by a predictive distribution; and qualitative, by expressing our confidence in the model for predicting a particular compound based on a quantitative measure of predictive reliability. It is possible to assess a quantitative (i.e. probabilistic) predictive distribution, given the supervised learning algorithm, the underlying QSAR data, a probability model for uncertainty and a statistical principle for inference. The integration of QSARs into risk assessment may be facilitated by the inclusion of the assessment of predictive error and predictive reliability into the "unambiguous algorithm", as outlined in the second OECD principle.

Evaluating the applicability domain in the case of classification predictive models for carcinogenicity based on the counter propagation artificial neural network

The applicability domain (AD) of models developed for regulatory use has attached great attention recently. The AD of quantitative structure-activity relationship (QSAR) models is the response and chemical structure space in which the model makes predictions with a given reliability. The evaluation of AD of regressions QSAR models for congeneric sets of chemicals can be find in many papers and books while the issue about metrics for the evaluation of an AD for the non-linear models (like neural networks) for the diverse set of chemicals represents the new field of investigations in QSAR studies. The scientific society is standing before the challenge to find out reliable way for the evaluation of an AD of non linear models. The new metrics for the evaluation of the AD of the counter propagation artificial neural network (CP ANN) models are discussed in the article: the Euclidean distances between an object (molecule) and the corresponding excited neuron of the neural network and between an object (molecule) and the representative object (vector of average values of descriptors). The investigation of the training and test sets chemicals coverage in the descriptors space was made with the respect to false predicted chemicals. The leverage approach was used to compare non linear (CP ANN) models with linear ones.

Selectivity-based QSAR approach for screening and evaluation of TRH analogs for TRH-R1 and TRH-R2 receptors subtypes

Design and development of therapeutically useful CNS selective thyrotropin-releasing hormone (TRH) analogs acting on TRH-R2 receptor subtype, exerting weak or no TRH-R1-mediated TSH-releasing side effects has gained imagination of researchers in the recent past. The present study reports the development and implementation of a selectivity-based QSAR approach for screening selective agonists of TRH-R2 receptor subtype. The statistically significant predictive models were thoroughly validated using an external validation set whose activity was previously unknown. The model was able to predict preference for either of the receptor subtypes successfully.

Chapter 9 Molecular Similarity: Advances in Methods, Applications and Validations in Virtual Screening and QSAR

This chapter discusses recent developments in some of the areas that exploit the molecular similarity principle, novel approaches to capture molecular properties by the use of novel descriptors, focuses on a crucial aspect of computational models-their validity, and discusses additional ways to examine data available, such as those from high-throughput screening (HTS) campaigns and to gain more knowledge from this data. The chapter also presents some of the recent applications of methods discussed focusing on the successes of virtual screening applications, database clustering and comparisons (such as drug- and in-house-likeness), and the recent large-scale validations of docking and scoring programs. While a great number of descriptors and modeling methods has been proposed until today, the recent trend toward proper model validation is very much appreciated. Although some of their limitations are surely because of underlying principles and limitations of fundamental concepts, others will certainly be eliminated in the future.

Database composition can affect the structure-activity relationship prediction

The percent active (A) and inactive (I) chemicals in a database can directly affect the sensitivity (% active chemicals predicted correctly) and specificity (% inactive chemicals predicted correctly) of structure-activity relationship (SAR) analyses. Subdividing the National Center for Toxicological Research (NCTR) liver cancer database (NCTRlcdb) into various A/I ratios, which varied from 0.2 to 5.5, resulted in sensitivity/specificity ratios that varied from 0.1 to 6.5. As percent active chemicals increased (increasing A/I ratio), the sensitivity rose, the specificity decreased, and the concordance (% total chemicals predicted correctly) remained fairly constant. The numbers of chemicals in the various data sets ranged from 187 to 999 and appeared to have no affect on any of the 3 predictors of sensitivity, specificity, or concordance.

A topological substructural approach for the prediction of P-glycoprotein substrates

A topological substructural molecular design approach (TOPS-MODE) has been used to predict whether a given compound is a P-glycoprotein (P-gp) substrate or not. A linear discriminant model was developed to classify a data set of 163 compounds as substrates or nonsubstrates (91 substrates and 72 nonsubstrates). The final model fit the data with sensitivity of 82.42% and specificity of 79.17%, for a final accuracy of 80.98%. The model was validated through the use of an external validation set (40 compounds, 22 substrates and 18 nonsubstrates) with a 77.50% of prediction accuracy; fivefold full cross-validation (removing 40 compounds in each cycle, 80.50% of good prediction) and the prediction of an external test set of marketed drugs (35 compounds, 71.43% of good prediction). This methodology evidenced that the standard bond distance, the polarizability and the Gasteiger-Marsilli atomic charge affect the interaction with the P-gp; suggesting the capacity of the TOPS-MODE descriptors to estimate the P-gp substrates for new drug candidates. The potentiality of the TOPS-MODE approach was assessed with a family of compounds not covered by the original training set (6-fluoroquinolones), and the final prediction had a 77.7% of accuracy. Finally, the positive and negative substructural contributions to the classification of 6-fluoroquinolones, as P-gp substrates, were identified; evidencing the possibilities of the present approach in the lead generation and optimization processes.

Intelligent testing strategies for chemicals testing -- a case of more haste, less speed?

The prospects for using (Q)SAR modelling, read-across (chemical) and other non-animal approaches as part of integrated testing strategies for chemical risk assessment, within the framework of the EU REACH legislation, are considered. The potential advantages and limitations of (Q)SAR modelling and read-across methods for chemical regulatory risk assessment are reviewed. It is concluded that it would be premature to base a testing strategy on chemical-based computational modelling approaches, until such time as criteria to validate them for their reliability and relevance by using independent and transparent procedures, have been agreed. This is mainly because of inherent problems in validating and accepting (Q)SARs for regulatory use in ways that are analogous to those that have been developed and applied for in vitro tests. Until this issue has been resolved, it is recommended that testing strategies should be developed which comprise the integrated use of computational and read-across approaches. These should be applied in a cautious and judicious way, in association with available tissue culture methods, and in conjunction with metabolism and biokinetic studies. Such strategies should be intelligently applied by being driven by exposure information (based on bioavailability, not merely on production volume) and hazard information needs, in preference to a tick-box approach. In the meantime, there should be increased efforts to develop improved (Q)SARs, expert systems and new in vitro methods, and, in particular, ways to expedite their validation and acceptance must be found and prospectively agreed with all major stakeholders.

Ranking of aquatic toxicity of esters modelled by QSAR

Alternative methods like predictions based on Quantitative Structure-Activity Relationships (QSARs) are now accepted to fill data gaps and define priority lists for more expensive and time consuming assessments. A heterogeneous data set of 74 esters was studied for their aquatic toxicity, and available experimental toxicity data on algae, Daphnia and fish were used to develop statistically validated QSAR models, obtained using multiple linear regression (MLR) by the OLS (Ordinary Least Squares) method and GA-VSS (Variable Subset Selection by Genetic Algorithms) to predict missing values. An ESter Aquatic Toxicity INdex (ESATIN) was then obtained by combining, by PCA, experimental and predicted toxicity data, from which model outliers and esters highly influential due to their structure had been eliminated. Finally this integrated aquatic toxicity index, defined by the PC1 score, was modelled using only a few theoretical molecular descriptors. This last QSAR model, statistically validated for its predictive power, could be proposed as a preliminary evaluative method for screening/prioritising esters according to their integrated aquatic toxicity, just starting from their molecular structure.

Animal testing and alternative approaches for the human health risk assessment under the proposed new European chemicals regulation

During the past 20 years the EU legislation for the notification of chemicals has focussed on new chemicals and at the same time failed to cover the evaluation of existing chemicals in Europe. Therefore, in a new EU chemicals policy (REACH, Registration, Evaluation and Authorization of Chemicals) the European Commission proposes to evaluate 30,000 chemicals within a period of 15 years. We are providing estimates of the testing requirements based on our personal experiences during the past 20 years. A realistic scenario based on an in-depth discussion of potential toxicological developments and an optimised "tailor-made" testing strategy shows that to meet the goals of the REACH policy, animal numbers may be significantly reduced below 10 million if industry would use in-house data from toxicity testing, which are confidential, if non-animal tests would be used, and if information from quantitative structure activity relationships (QSARs) would be applied in substance-tailored testing schemes. The procedures for evaluating the reproductive toxicity of chemicals have the strongest impact on the total number of animals bred for testing under REACH. We are assuming both an active collaboration with our colleagues in industry and substantial funding of the development and validation of advanced non-animal methods by the EU Commission, specifically in reproductive and developmental toxicity.

In silico modelling of hazard endpoints: current problems and perspectives

Major scientific hurdles in the acceptance of quantitative structure-activity relationships (QSAR) for regulatory purposes have been identified. First, when quantifying important features of chemical structure complexities of molecular structure have often been ignored. More mechanistic modelling of chemical structure should proceed on two fronts: by developing a more in-depth understanding and representation of the multiple states possible for a single chemical by achieving greater rigor in understanding of conformational flexibility of chemicals; and, by considering families of activated metabolites that are derived in biological systems from an initial chemical substrate. Second, QSAR research is severely limited by the lack of systematic databases for important risk assessment endpoints, and despite many decades of research, the ability to cluster reactive chemicals by common toxicity pathways is in its infancy. Finally, computational tools are lacking for defining where a specific QSAR is applicable within the domain (universe) of chemical structures that are to be regulated. This paper describes some of the approaches being taken to address these needs. Applications of some of these new approaches are demonstrated for the prediction of chemical mutagenicity, where considerations of both molecular flexibility and metabolic activation improved the QSAR predictability and interpretations. Lastly, the applicability domain for a specific QSAR predicting estrogen receptor binding is presented in the context of a mechanistically-defined chemical structure space for large heterogeneous chemical datasets of regulatory concern. A strategic approach is discussed to selecting chemicals for model improvement and validation until regulatory acceptance criteria for risk assessment applications are met.