Dempster-Shafer theory for combining in silico evidence and estimating uncertainty in chemical risk assessment

In Silico Prediction of Skin Sensitization for Compounds via Flexible Evidence Combination Based on Machine Learning and Dempster-Shafer Theory

Skin sensitization is increasingly becoming a significant concern in the development of drugs and cosmetics due to consumer safety and occupational health problems. In silico methods have emerged as alternatives to traditional in vivo animal testing due to ethical and economic considerations. In this study, machine learning methods were used to build quantitative structure-activity relationship (QSAR) models on five skin sensitization data sets (GPMT, LLNA, DPRA, KeratinoSens, and h-CLAT), achieving effective predictive accuracies (correct classification rates of 0.688-0.764 on test sets). To address the complex mechanisms of human skin sensitization, the Dempster-Shafer theory was applied to merge multiple QSAR models, resulting in an evidence-based integrated decision model. Various evidence combinations and combination rules were explored, with the self-defined Q3 rule showing superior balance. The combination of evidence such as GPMT and KeratinoSens and h-CLAT achieved a correct classification rate (CCR) of 0.880 and coverage of 0.893 while maintaining the competitiveness of other combinations. Additionally, the Shapley additive explanations (SHAP) method was used to interpret important features and substructures related to skin sensitization. A comparative analysis of an external human test set demonstrated the superior performance of the proposed method. Finally, to enhance accessibility, the workflow was implemented into a user-friendly software named HSkinSensDS.

A problem formulation framework for the application of in silico toxicology methods in chemical risk assessment

The first step in the hazard or risk assessment of chemicals should be to formulate the problem through a systematic and iterative process aimed at identifying and defining factors critical to the assessment. However, no general agreement exists on what components an in silico toxicology problem formulation (PF) should include. The present work aims to develop a PF framework relevant to the application of in silico models for chemical toxicity prediction. We modified and applied a PF framework from the general risk assessment literature to peer reviewed papers describing PFs associated with in silico toxicology models. Important gaps between the general risk assessment literature and the analyzed PF literature associated with in silico toxicology methods were identified. While the former emphasizes the need for PFs to address higher-level conceptual questions, the latter does not. There is also little consistency in the latter regarding the PF components addressed, reinforcing the need for a PF framework that enable users of in silico toxicology models to answer the central conceptual questions aimed at defining components critical to the model application. Using the developed framework, we highlight potential areas of uncertainty manifestation in in silico toxicology PF in instances where particular components are missing or implicitly described. The framework represents the next step in standardizing in silico toxicology PF component. The framework can also be used to improve the understanding of how uncertainty is apparent in an in silico toxicology PF, thus facilitating ways to address uncertainty.

Is read-across for chemicals comparable to medical device equivalence and where to use it for conformity assessment?

Novel medical devices must conform to medical device regulation (MDR) for European market entry. Likewise, chemicals must comply with the Registration, Evaluation, Authorization and Restriction of Chemicals (REACh) regulation. Both pose regulatory challenges for manufacturers, but concordantly provide an approach for transferring data from an already registered device or compound to the one undergoing accreditation. This is called equivalence for medical devices and read-across for chemicals. Although read-across is not explicitly prohibited in the process of medical device accreditation, it is usually not performed due to a lack of guidance and acceptance criteria from the authorities. Nonetheless, a scientifically justified read-across of material-based endpoints, as well as toxicological assessment of chemical aspects, such as extractables and leachables, can prevent failure of MDR device equivalence if data is lacking. Further, read-across, if applied correctly can facilitate the standard MDR conformity assessment. The need for read-across within medical device registration should let authorities to reconsider device accreditation and the formulation of respective guidance documents. Acceptance criteria like in the European Chemicals Agency (ECHA) read-across assessment framework (RAAF) are needed. This can reduce the impact of the MDR and help with keeping high European innovation device rate, beneficial for medical device patients.

On QSAR-based cardiotoxicity modeling with the expressiveness-enhanced graph learning model and dual-threshold scheme

Introduction: Given the direct association with malignant ventricular arrhythmias, cardiotoxicity is a major concern in drug design. In the past decades, computational models based on the quantitative structure-activity relationship have been proposed to screen out cardiotoxic compounds and have shown promising results. The combination of molecular fingerprint and the machine learning model shows stable performance for a wide spectrum of problems; however, not long after the advent of the graph neural network (GNN) deep learning model and its variant (e.g., graph transformer), it has become the principal way of quantitative structure-activity relationship-based modeling for its high flexibility in feature extraction and decision rule generation. Despite all these progresses, the expressiveness (the ability of a program to identify non-isomorphic graph structures) of the GNN model is bounded by the WL isomorphism test, and a suitable thresholding scheme that relates directly to the sensitivity and credibility of a model is still an open question. Methods: In this research, we further improved the expressiveness of the GNN model by introducing the substructure-aware bias by the graph subgraph transformer network model. Moreover, to propose the most appropriate thresholding scheme, a comprehensive comparison of the thresholding schemes was conducted. Results: Based on these improvements, the best model attains performance with 90.4% precision, 90.4% recall, and 90.5% F1-score with a dual-threshold scheme (active: < 1 μ M ; non-active: > 30 μ M ). The improved pipeline (graph subgraph transformer network model and thresholding scheme) also shows its advantages in terms of the activity cliff problem and model interpretability.

A read-across case study on chronic toxicity of branched carboxylic acids (1): Integration of mechanistic evidence from new approach methodologies (NAMs) to explore a common mode of action

This read-across case study characterises thirteen, structurally similar carboxylic acids demonstrating the application of in vitro and in silico human-based new approach methods, to determine biological similarity. Based on data from in vivo animal studies, the read-across hypothesis is that all analogues are steatotic and so should be considered hazardous. Transcriptomic analysis to determine differentially expressed genes (DEGs) in hepatocytes served as first tier testing to confirm a common mode-of-action and identify differences in the potency of the analogues. An adverse outcome pathway (AOP) network for hepatic steatosis, informed the design of an in vitro testing battery, targeting AOP relevant MIEs and KEs, and Dempster-Shafer decision theory was used to systematically quantify uncertainty and to define the minimal testing scope. The case study shows that the read-across hypothesis is the critical core to designing a robust, NAM-based testing strategy. By summarising the current mechanistic understanding, an AOP enables the selection of NAMs covering MIEs, early KEs, and late KEs. Experimental coverage of the AOP in this way is vital since MIEs and early KEs alone are not confirmatory of progression to the AO. This strategy exemplifies the workflow previously published by the EUTOXRISK project driving a paradigm shift towards NAM-based NGRA.

COSMOS next generation - A public knowledge base leveraging chemical and biological data to support the regulatory assessment of chemicals

The COSMOS Database (DB) was originally established to provide reliable data for cosmetics-related chemicals within the COSMOS Project funded as part of the SEURAT-1 Research Initiative. The database has subsequently been maintained and developed further into COSMOS Next Generation (NG), a combination of database and in silico tools, essential components of a knowledge base. COSMOS DB provided a cosmetics inventory as well as other regulatory inventories, accompanied by assessment results and in vitro and in vivo toxicity data. In addition to data content curation, much effort was dedicated to data governance - data authorisation, characterisation of quality, documentation of meta information, and control of data use. Through this effort, COSMOS DB was able to merge and fuse data of various types from different sources. Building on the previous effort, the COSMOS Minimum Inclusion (MINIS) criteria for a toxicity database were further expanded to quantify the reliability of studies. COSMOS NG features multiple fingerprints for analysing structure similarity, and new tools to calculate molecular properties and screen chemicals with endpoint-related public profilers, such as DNA and protein binders, liver alerts and genotoxic alerts. The publicly available COSMOS NG enables users to compile information and execute analyses such as category formation and read-across. This paper provides a step-by-step guided workflow for a simple read-across case, starting from a target structure and culminating in an estimation of a NOAEL confidence interval. Given its strong technical foundation, inclusion of quality-reviewed data, and provision of tools designed to facilitate communication between users, COSMOS NG is a first step towards building a toxicological knowledge hub leveraging many public data systems for chemical safety evaluation. We continue to monitor the feedback from the user community at support@mn-am.com.

Toward regulatory acceptance and improving the prediction confidence of in silico approaches: a case study of genotoxicity

Introduction: Genotoxicity is an imperative component of the human health safety assessment of chemicals. Its secure forecast is of the utmost importance for all health prevention strategies and regulations.Areas covered: We surveyed several types of alternative, animal-free approaches ((quantitative) structure-activity relationship (Q)SAR, read-across, Adverse Outcome Pathway, Integrated Approaches to Testing and Assessment) for genotoxicity prediction within the needs of regulatory frameworks, putting special emphasis on data quality and uncertainties issues.Expert opinion: (Q)SAR models and read-across approaches for in vitro bacterial mutagenicity have sufficient reliability for use in prioritization processes, and as support in regulatory decisions in combination with other types of evidence. (Q)SARs and read-across methodologies for other genotoxicity endpoints need further improvements and should be applied with caution. It appears that there is still large room for improvement of genotoxicity prediction methods. Availability of well-curated high-quality databases, covering a broader chemical space, is one of the most important needs. Integration of in silico predictions with expert knowledge, weight-of-evidence-based assessment, and mechanistic understanding of genotoxicity pathways are other key points to be addressed for the generation of more accurate and trustable results.

Development of a Battery of In Silico Prediction Tools for Drug-Induced Liver Injury from the Vantage Point of Translational Safety Assessment

Drug-induced liver injury (DILI) remains a challenge when translating knowledge from the preclinical stage to human use cases. Attempts to model human DILI directly based on the information from drug labels have had some success; however, the approach falls short of providing insights or addressing uncertainty due to the difficulty of decoupling the idiosyncratic nature of human DILI outcomes. Our approach in this comparative analysis is to leverage existing preclinical and clinical data as well as information on metabolism to better translate mammalian to human DILI. The human DILI knowledge base from the United States Food and Drug Administration (U.S. FDA) National Center for Toxicology Research contains 1036 pharmaceuticals from diverse therapeutic categories. A human DILI training set of 305 oral marketed drugs was prepared and a binary classification scheme applied. The second knowledge base consists of mammalian repeated dose toxicity with liver toxicity data from various regulatory sources. Within this knowledge base, we identified 278 pharmaceuticals containing 198 marketed or withdrawn oral drugs with data from the U.S. FDA new drug application and 98 active pharmaceutical ingredients from ToxCast. From this collection, a set of 225 oral drugs was prepared as the mammalian hepatotoxicity training set with particular end points of pathology findings in the liver and bile duct. Both human and mammalian data sets were processed using various learning algorithms, including artificial intelligence approaches. The external validations for both models were comparable to the training statistics. These data sets were also used to extract species-differentiating chemotypes that differentiate DILI effects on humans from mammals. A systematic workflow was devised to predict human DILI and provide mechanistic insights. For a given query molecule, both human and mammalian models are run. If the predictions are discordant, both metabolites and parents are investigated for quantitative structure-activity relationship and species-differentiating chemotypes. Their results are combined using the Dempster-Shafer decision theory to yield a final outcome prediction for human DILI with estimated uncertainty. Finally, these tools are implementable within an in silico platform for systematic evaluation.

Quantifying Uncertainty in Ecotoxicological Risk Assessment: MUST, a Modular Uncertainty Scoring Tool

Whether conducting a risk, hazard, or alternatives assessment, one invariably struggles with the task of reconciling multiple available values of toxicological thresholds into a single outcome. When combining multiple pieces of evidence from many different sources, it is important to consider the role of data uncertainty. Uncertainty is inherent to all scientific data. However, in toxicological assessments, controversies and uncertainties are typically understated; they lack methodological transparency; or they poorly integrate qualitative and quantitative sources of information. Similarly, in model development, data curation is rarely performed with sufficient rigor, particularly when applying big data statistics. To overcome the hurdles of a decision process that must reconcile divergent data, we developed an uncertainty scoring tool that can be trained to reproduce specific decision-making paradigms and ensure consistency in the practitioner's judgment across complex scenarios. While designed to aid with ecotoxicological assessments and predictive model development, the tool's applicability extends to any decision-making process that calls for synthesis of incongruent data. Here, we highlight the development process, as well as demonstrate the method's utility in several prototypical ecotoxicological case studies.

Setting the stage for next-generation risk assessment with non-animal approaches: the EU-ToxRisk project experience

In 2016, the European Commission launched the EU-ToxRisk research project to develop and promote animal-free approaches in toxicology. The 36 partners of this consortium used in vitro and in silico methods in the context of case studies (CSs). These CSs included both compounds with a highly defined target (e.g. mitochondrial respiratory chain inhibitors) as well as compounds with poorly defined molecular initiation events (e.g. short-chain branched carboxylic acids). The initial project focus was on developing a science-based strategy for read-across (RAx) as an animal-free approach in chemical risk assessment. Moreover, seamless incorporation of new approach method (NAM) data into this process (= NAM-enhanced RAx) was explored. Here, the EU-ToxRisk consortium has collated its scientific and regulatory learnings from this particular project objective. For all CSs, a mechanistic hypothesis (in the form of an adverse outcome pathway) guided the safety evaluation. ADME data were generated from NAMs and used for comprehensive physiological-based kinetic modelling. Quality assurance and data management were optimized in parallel. Scientific and Regulatory Advisory Boards played a vital role in assessing the practical applicability of the new approaches. In a next step, external stakeholders evaluated the usefulness of NAMs in the context of RAx CSs for regulatory acceptance. For instance, the CSs were included in the OECD CS portfolio for the Integrated Approach to Testing and Assessment project. Feedback from regulators and other stakeholders was collected at several stages. Future chemical safety science projects can draw from this experience to implement systems toxicology-guided, animal-free next-generation risk assessment.

Evaluation of the applicability of existing (Q)SAR models for predicting the genotoxicity of pesticides and similarity analysis related with genotoxicity of pesticides for facilitating of grouping and read across: An EFSA funded project

To facilitate the practical implementation of the guidance on the residue definition for dietary risk assessment, EFSA has organized an evaluation of applicability of existing in silico models for predicting the genotoxicity of pesticides and their metabolites, including literature survey, application of QSARs and development of Read Across methodologies. This paper summarizes the main results. For the Ames test, all (Q)SAR models generated statistically significant predictions, comparable with the experimental variability of the test. The reliability of the models for other assays/endpoints appears to be still far from optimality. Two new Read Across approaches were evaluated: Read Across was largely successful for predicting the Ames test results, but less for in vitro Chromosomal Aberrations. The worse results for non-Ames endpoints may be attributable to the several revisions of experimental protocols and evaluation criteria of results, that have made the databases qualitatively non-homogeneous and poorly suitable for modeling. Last, Parent/Metabolite structural differences (besides known Structural Alerts) that may, or may not cause changes in the Ames mutagenicity were identified and catalogued. The findings from this work are suitable for being integrated into Weight-of-Evidence and Tiered evaluation schemes. Areas needing further developments are pointed out.

Unlocking the potential of in silico chemical safety assessment - A report on a cross-sector symposium on current opportunities and future challenges

In silico chemical safety assessment can support the evaluation of hazard and risk following potential exposure to a substance. A symposium identified a number of opportunities and challenges to implement in silico methods, such as quantitative structure-activity relationships (QSARs) and read-across, to assess the potential harm of a substance in a variety of exposure scenarios, e.g. pharmaceuticals, personal care products, and industrial chemicals. To initiate the process of in silico safety assessment, clear and unambiguous problem formulation is required to provide the context for these methods. These approaches must be built on data of defined quality, while acknowledging the possibility of novel data resources tapping into on-going progress with data sharing. Models need to be developed that cover appropriate toxicity and kinetic endpoints, and that are documented appropriately with defined uncertainties. The application and implementation of in silico models in chemical safety requires a flexible technological framework that enables the integration of multiple strands of data and evidence. The findings of the symposium allowed for the identification of priorities to progress in silico chemical safety assessment towards the animal-free assessment of chemicals.