Toward an evidence-based toxicology

Artificial intelligence (AI)-it's the end of the tox as we know it (and I feel fine)

The rapid progress of AI impacts diverse scientific disciplines, including toxicology, and has the potential to transform chemical safety evaluation. Toxicology has evolved from an empirical science focused on observing apical outcomes of chemical exposure, to a data-rich field ripe for AI integration. The volume, variety and velocity of toxicological data from legacy studies, literature, high-throughput assays, sensor technologies and omics approaches create opportunities but also complexities that AI can help address. In particular, machine learning is well suited to handle and integrate large, heterogeneous datasets that are both structured and unstructured-a key challenge in modern toxicology. AI methods like deep neural networks, large language models, and natural language processing have successfully predicted toxicity endpoints, analyzed high-throughput data, extracted facts from literature, and generated synthetic data. Beyond automating data capture, analysis, and prediction, AI techniques show promise for accelerating quantitative risk assessment by providing probabilistic outputs to capture uncertainties. AI also enables explanation methods to unravel mechanisms and increase trust in modeled predictions. However, issues like model interpretability, data biases, and transparency currently limit regulatory endorsement of AI. Multidisciplinary collaboration is needed to ensure development of interpretable, robust, and human-centered AI systems. Rather than just automating human tasks at scale, transformative AI can catalyze innovation in how evidence is gathered, data are generated, hypotheses are formed and tested, and tasks are performed to usher new paradigms in chemical safety assessment. Used judiciously, AI has immense potential to advance toxicology into a more predictive, mechanism-based, and evidence-integrated scientific discipline to better safeguard human and environmental wellbeing across diverse populations.

Applying a modified systematic review and integrated assessment framework (SYRINA) - a case study on triphenyl phosphate

This work presents a case study in applying a systematic review framework (SYRINA) to the identification of chemicals as endocrine disruptors. The suitability and performance of the framework is tested with regard to the widely accepted World Health Organization definition of an endocrine disruptor (ED). The endocrine disrupting potential of triphenyl phosphate (TPP), a well-studied flame retardant reported to exhibit various endocrine related effects was assessed. We followed the 7 steps of the SYRINA framework, articulating the research objective via Populations, Exposures, Comparators, Outcomes (PECO) statements, performed literature search and screening, conducted study evaluation, performed data extraction and summarized and integrated the evidence. Overall, 66 studies, consisting of in vivo, in vitro and epidemiological data, were included. We concluded that triphenyl phosphate could be identified as an ED based on metabolic disruption and reproductive function. We found that the tools used in this case study and the optimizations performed on the framework were suitable to assess properties of EDs. A number of challenges and areas for methodological development in systematic appraisal of evidence relating to endocrine disrupting potential were identified; significant time and effort were needed for the analysis of in vitro mechanistic data in this case study, thus increasing the workload and time needed to perform the systematic review process. Further research and development of this framework with regards to grey literature (non-peer-reviewed literature) search, harmonization of study evaluation methods, more consistent evidence integration approaches and a pre-defined method to assess links between adverse effect and endocrine activity are recommended. It would also be advantageous to conduct more case studies for a chemical with less data than TPP.

Principles of risk decision-making

Risk management decisions in public health require consideration of a number of complex, often conflicting factors. The aim of this review was to propose a set of 10 fundamental principles to guide risk decision-making. Although each of these principles is sound in its own right, the guidance provided by different principles might lead the decision-maker in different directions. For example, where the precautionary principle advocates for preemptive risk management action under situations of scientific uncertainty and potentially catastrophic consequences, the principle of risk-based decision-making encourages decision-makers to focus on established and modifiable risks, where a return on the investment in risk management is all but guaranteed in the near term. To evaluate the applicability of the 10 principles in practice, one needs to consider 10 diverse risk issues of broad concern and explore which of these principles are most appropriate in different contexts. The 10 principles presented here afford substantive insight into the process of risk management decision-making, although decision-makers will ultimately need to exercise judgment in reaching appropriate risk decisions, accounting for all of the scientific and extra-scientific factors relevant to the risk decision at hand.

Replacement of animal testing by integrated approaches to testing and assessment (IATA): a call for in vivitrosi

Alternative methods to animal use in toxicology are evolving with new advanced tools and multilevel approaches, to answer from one side to 3Rs requirements, and on the other side offering relevant and valid tests for drugs and chemicals, considering also their combination in test strategies, for a proper risk assessment.While stand-alone methods, have demonstrated to be applicable for some specific toxicological predictions with some limitations, the new strategy for the application of New Approach Methods (NAM), to solve complex toxicological endpoints is addressed by Integrated Approaches for Testing and Assessment (IATA), aka Integrated Testing Strategies (ITS) or Defined Approaches for Testing and Assessment (DA). The central challenge of evidence integration is shared with the needs of risk assessment and systematic reviews of an evidence-based Toxicology. Increasingly, machine learning (aka Artificial Intelligence, AI) lends itself to integrate diverse evidence streams.In this article, we give an overview of the state of the art of alternative methods and IATA in toxicology for regulatory use for various hazards, outlining future orientation and perspectives. We call on leveraging the synergies of integrated approaches and evidence integration from in vivo, in vitro and in silico as true in vivitrosi.

The ECOTOXicology Knowledgebase: A Curated Database of Ecologically Relevant Toxicity Tests to Support Environmental Research and Risk Assessment

The need for assembled existing and new toxicity data has accelerated as the amount of chemicals introduced into commerce continues to grow and regulatory mandates require safety assessments for a greater number of chemicals. To address this evolving need, the ECOTOXicology Knowledgebase (ECOTOX) was developed starting in the 1980s and is currently the world's largest compilation of curated ecotoxicity data, providing support for assessments of chemical safety and ecological research through systematic and transparent literature review procedures. The recently released version of ECOTOX (Ver 5, www.epa.gov/ecotox) provides single-chemical ecotoxicity data for over 12,000 chemicals and ecological species with over one million test results from over 50,000 references. Presented is an overview of ECOTOX, detailing the literature review and data curation processes within the context of current systematic review practices and discussing how recent updates improve the accessibility and reusability of data to support the assessment, management, and research of environmental chemicals. Relevant and acceptable toxicity results are identified from studies in the scientific literature, with pertinent methodological details and results extracted following well-established controlled vocabularies and newly extracted toxicity data added quarterly to the public website. Release of ECOTOX, Ver 5, included an entirely redesigned user interface with enhanced data queries and retrieval options, visualizations to aid in data exploration, customizable outputs for export and use in external applications, and interoperability with chemical and toxicity databases and tools. This is a reliable source of curated ecological toxicity data for chemical assessments and research and continues to evolve with accessible and transparent state-of-the-art practices in literature data curation and increased interoperability to other relevant resources. Environ Toxicol Chem 2022;41:1520-1539. © 2022 SETAC. This article has been contributed to by US Government employees and their work is in the public domain in the USA.

A Survey of Systematic Evidence Mapping Practice and the Case for Knowledge Graphs in Environmental Health and Toxicology

Systematic evidence mapping offers a robust and transparent methodology for facilitating evidence-based approaches to decision-making in chemicals policy and wider environmental health (EH). Interest in the methodology is growing; however, its application in EH is still novel. To facilitate the production of effective systematic evidence maps for EH use cases, we survey the successful application of evidence mapping in other fields where the methodology is more established. Focusing on issues of “data storage technology,” “data integrity,” “data accessibility,” and “transparency,” we characterize current evidence mapping practice and critically review its potential value for EH contexts. We note that rigid, flat data tables and schema-first approaches dominate current mapping methods and highlight how this practice is ill-suited to the highly connected, heterogeneous, and complex nature of EH data. We propose this challenge is overcome by storing and structuring data as “knowledge graphs.” Knowledge graphs offer a flexible, schemaless, and scalable model for systematically mapping the EH literature. Associated technologies, such as ontologies, are well-suited to the long-term goals of systematic mapping methodology in promoting resource-efficient access to the wider EH evidence base. Several graph storage implementations are readily available, with a variety of proven use cases in other fields. Thus, developing and adapting systematic evidence mapping for EH should utilize these graph-based resources to ensure the production of scalable, interoperable, and robust maps to aid decision-making processes in chemicals policy and wider EH.

Knowledge Organization Systems for Systematic Chemical Assessments

BACKGROUND

Although the implementation of systematic review and evidence mapping methods stands to improve the transparency and accuracy of chemical assessments, they also accentuate the challenges that assessors face in ensuring they have located and included all the evidence that is relevant to evaluating the potential health effects an exposure might be causing. This challenge of information retrieval can be characterized in terms of "semantic" and "conceptual" factors that render chemical assessments vulnerable to the streetlight effect.

OBJECTIVES

This commentary presents how controlled vocabularies, thesauruses, and ontologies contribute to overcoming the streetlight effect in information retrieval, making up the key components of Knowledge Organization Systems (KOSs) that enable more systematic access to assessment-relevant information than is currently achievable. The concept of Adverse Outcome Pathways is used to illustrate what a general KOS for use in chemical assessment could look like.

DISCUSSION

Ontologies are an underexploited element of effective knowledge organization in the environmental health sciences. Agreeing on and implementing ontologies in chemical assessment is a complex but tractable process with four fundamental steps. Successful implementation of ontologies would not only make currently fragmented information about health risks from chemical exposures vastly more accessible, it could ultimately enable computational methods for chemical assessment that can take advantage of the full richness of data described in natural language in primary studies. https://doi.org/10.1289/EHP6994.

Real world-like simulations show efficient predictive power of in vitro skin corrosion tests used as stand-alone and in combination and how can toxicologists take advantage of them

Historically, performance of in vitro toxicology test methods has been evaluated and considered for regulatory purposes based on sensitivity & specificity values derived from validation studies. Other indicators are however useful to evaluate in vitro tests such as positive & negative predictive values (PPV & NPV), likelihood ratios (LRs) or odds ratio (OR). These indicators, which are routinely used in diagnostic tests, if adapted and adequately applied to in vitro tests would help determine their realistic predictive power in real world-like situation and help risk assessors know how they can rely on in vitro tests used for their safety assessments. This paper performs a series of simulations considering the actual distribution in ECHA C&L inventory of skin corrosive chemicals to calculate several of these indicators of performance (PPV, NPV, LRs, OR). It shows applied examples of predictive power on EpiSkin™ and SkinEthic™ RHE two validated in vitro skin corrosive tests, explains how to build testing strategies based on these examples, compares so called 'bottom-up' and 'top-down' approaches, and demonstrates the number of tests required, and how risk assessors can practically take advantage of this.

Perspectives on In Vitro to In Vivo Extrapolations

Quantitative in vitro to in vivo extrapolation (QIVIVE) is broadly considered a prerequisite bridge from in vitro findings to a dose paradigm. Quality and relevance of cell systems are the first prerequisite for QIVIVE. Information-rich and mechanistic endpoints (biomarkers) improve extrapolations, but a sophisticated endpoint does not make a bad cell model a good one. The next need is reverse toxicokinetics (TK), which estimates the dose necessary to reach a tissue concentration that is active in vitro. The Johns Hopkins Center for Alternatives to Animal Testing (CAAT) has created a roadmap for animal-free systemic toxicity testing, in which the needs and opportunities for TK are elaborated, in the context of different systemic toxicities. The report was discussed at two stakeholder forums in Brussels in 2012 and in Washington in 2013; the key recommendations are summarized herein. Contrary to common belief and the Paracelsus paradigm of everything is toxic, the majority of industrial chemicals do not exhibit toxicity. Strengthening the credibility of negative results of alternative approaches for hazard identification, therefore, avoids the need for QIVIVE. Here, especially the combination of methods in integrated testing strategies is most promising. Two further but very different approaches aim to overcome the problem of modeling in vivo complexity: The human-on-a-chip movement aims to reproduce large parts of living organism's complexity via microphysiological systems, that is, organ equivalents combined by microfluidics. At the same time, the Toxicity Testing in the 21st Century (Tox-21c) movement aims for mechanistic approaches (adverse outcome pathways as promoted by Organisation for Economic Co-operation and Development (OECD) or pathways of toxicity in the Human Toxome Project) for high-throughput screening, biological phenotyping, and ultimately a systems toxicology approach through integration with computer modeling. These 21st century approaches also require 21st century validation, for example, by evidence-based toxicology. Ultimately, QIVIVE is a prerequisite for extrapolating Tox-21c such approaches to human risk assessment.

Computational approaches to chemical hazard assessment

Computational prediction of toxicity has reached new heights as a result of decades of growth in the magnitude and diversity of biological data. Public packages for statistics and machine learning make model creation faster. New theory in machine learning and cheminformatics enables integration of chemical structure, toxicogenomics, simulated and physical data in the prediction of chemical health hazards, and other toxicological information. Our earlier publications have characterized a toxicological dataset of unprecedented scale resulting from the European REACH legislation (Registration Evaluation Authorisation and Restriction of Chemicals). These publications dove into potential use cases for regulatory data and some models for exploiting this data. This article analyzes the options for the identification and categorization of chemicals, moves on to the derivation of descriptive features for chemicals, discusses different kinds of targets modeled in computational toxicology, and ends with a high-level perspective of the algorithms used to create computational toxicology models.

Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned with the Zebrafish Embryotoxicity Test

Systematic review methodology is a means of addressing specific questions through structured, consistent, and transparent examinations of the relevant scientific evidence. This methodology has been used to advantage in clinical medicine, and is being adapted for use in other disciplines. Although some applications to toxicology have been explored, especially for hazard identification, the present preparatory study is, to our knowledge, the first attempt to adapt it to the assessment of toxicological test methods. As our test case, we chose the zebrafish embryotoxicity test (ZET) for developmental toxicity and its mammalian counterpart, the standard mammalian prenatal development toxicity study, focusing the review on how well the ZET predicts the presence or absence of chemical-induced prenatal developmental toxicity observed in mammalian studies. An interdisciplinary team prepared a systematic review protocol and adjusted it throughout this piloting phase, where needed. The final protocol was registered and will guide the main study (systematic review), which will execute the protocol to comprehensively answer the review question. The goal of this preparatory study was to translate systematic review methodology to the assessment of toxicological test method performance. Consequently, it focused on the methodological issues encountered, whereas the main study will report substantive findings. These relate to numerous systematic review steps, but primarily to searching and selecting the evidence. Applying the lessons learned to these challenges can improve not only our main study, but may also be helpful to others seeking to use systematic review methodology to compare toxicological test methods. We conclude with a series of recommendations that, if adopted, would help improve the quality of the published literature, and make conducting systematic reviews of toxicological studies faster and easier over time.

3S - Systematic, systemic, and systems biology and toxicology

A biological system is more than the sum of its parts - it accomplishes many functions via synergy. Deconstructing the system down to the molecular mechanism level necessitates the complement of reconstructing functions on all levels, i.e., in our conceptualization of biology and its perturbations, our experimental models and computer modelling. Toxicology contains the somewhat arbitrary subclass "systemic toxicities"; however, there is no relevant toxic insult or general disease that is not systemic. At least inflammation and repair are involved that require coordinated signaling mechanisms across the organism. However, the more body components involved, the greater the challenge to reca-pitulate such toxicities using non-animal models. Here, the shortcomings of current systemic testing and the development of alternative approaches are summarized. We argue that we need a systematic approach to integrating existing knowledge as exemplified by systematic reviews and other evidence-based approaches. Such knowledge can guide us in modelling these systems using bioengineering and virtual computer models, i.e., via systems biology or systems toxicology approaches. Experimental multi-organ-on-chip and microphysiological systems (MPS) provide a more physiological view of the organism, facilitating more comprehensive coverage of systemic toxicities, i.e., the perturbation on organism level, without using substitute organisms (animals). The next challenge is to establish disease models, i.e., micropathophysiological systems (MPPS), to expand their utility to encompass biomedicine. Combining computational and experimental systems approaches and the chal-lenges of validating them are discussed. The suggested 3S approach promises to leverage 21st century technology and systematic thinking to achieve a paradigm change in studying systemic effects.

A primer on systematic reviews in toxicology

Systematic reviews, pioneered in the clinical field, provide a transparent, methodologically rigorous and reproducible means of summarizing the available evidence on a precisely framed research question. Having matured to a well-established approach in many research fields, systematic reviews are receiving increasing attention as a potential tool for answering toxicological questions. In the larger framework of evidence-based toxicology, the advantages and obstacles of, as well as the approaches for, adapting and adopting systematic reviews to toxicology are still being explored. To provide the toxicology community with a starting point for conducting or understanding systematic reviews, we herein summarized available guidance documents from various fields of application. We have elaborated on the systematic review process by breaking it down into ten steps, starting with planning the project, framing the question, and writing and publishing the protocol, and concluding with interpretation and reporting. In addition, we have identified the specific methodological challenges of toxicological questions and have summarized how these can be addressed. Ultimately, this primer is intended to stimulate scientific discussions of the identified issues to fuel the development of toxicology-specific methodology and to encourage the application of systematic review methodology to toxicological issues.

Adaptation of the ToxRTool to Assess the Reliability of Toxicology Studies Conducted with Genetically Modified Crops and Implications for Future Safety Testing

To determine the reliability of food safety studies carried out in rodents with genetically modified (GM) crops, a Food Safety Study Reliability Tool (FSSRTool) was adapted from the European Centre for the Validation of Alternative Methods' (ECVAM) ToxRTool. Reliability was defined as the inherent quality of the study with regard to use of standardized testing methodology, full documentation of experimental procedures and results, and the plausibility of the findings. Codex guidelines for GM crop safety evaluations indicate toxicology studies are not needed when comparability of the GM crop to its conventional counterpart has been demonstrated. This guidance notwithstanding, animal feeding studies have routinely been conducted with GM crops, but their conclusions on safety are not always consistent. To accurately evaluate potential risks from GM crops, risk assessors need clearly interpretable results from reliable studies. The development of the FSSRTool, which provides the user with a means of assessing the reliability of a toxicology study to inform risk assessment, is discussed. Its application to the body of literature on GM crop food safety studies demonstrates that reliable studies report no toxicologically relevant differences between rodents fed GM crops or their non-GM comparators.

The Emergence of Systematic Review in Toxicology

The Evidence-based Toxicology Collaboration hosted a workshop on "The Emergence of Systematic Review and Related Evidence-based Approaches in Toxicology," on November 21, 2014 in Baltimore, Maryland. The workshop featured speakers from agencies and organizations applying systematic review approaches to questions in toxicology, speakers with experience in conducting systematic reviews in medicine and healthcare, and stakeholders in industry, government, academia, and non-governmental organizations. Based on the workshop presentations and discussion, here we address the state of systematic review methods in toxicology, historical antecedents in both medicine and toxicology, challenges to the translation of systematic review from medicine to toxicology, and thoughts on the way forward. We conclude with a recommendation that as various agencies and organizations adapt systematic review methods, they continue to work together to ensure that there is a harmonized process for how the basic elements of systematic review methods are applied in toxicology.

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations

Systematic review (SR) is a rigorous, protocol-driven approach designed to minimise error and bias when summarising the body of research evidence relevant to a specific scientific question. Taking as a comparator the use of SR in synthesising research in healthcare, we argue that SR methods could also pave the way for a "step change" in the transparency, objectivity and communication of chemical risk assessments (CRA) in Europe and elsewhere. We suggest that current controversies around the safety of certain chemicals are partly due to limitations in current CRA procedures which have contributed to ambiguity about the health risks posed by these substances. We present an overview of how SR methods can be applied to the assessment of risks from chemicals, and indicate how challenges in adapting SR methods from healthcare research to the CRA context might be overcome. Regarding the latter, we report the outcomes from a workshop exploring how to increase uptake of SR methods, attended by experts representing a wide range of fields related to chemical toxicology, risk analysis and SR. Priorities which were identified include: the conduct of CRA-focused prototype SRs; the development of a recognised standard of reporting and conduct for SRs in toxicology and CRA; and establishing a network to facilitate research, communication and training in SR methods. We see this paper as a milestone in the creation of a research climate that fosters communication between experts in CRA and SR and facilitates wider uptake of SR methods into CRA.

Toward Good Read-Across Practice (GRAP) guidance

Grouping of substances and utilizing read-across of data within those groups represents an important data gap filling technique for chemical safety assessments. Categories/analogue groups are typically developed based on structural similarity and, increasingly often, also on mechanistic (biological) similarity. While read-across can play a key role in complying with legislation such as the European REACH regulation, the lack of consensus regarding the extent and type of evidence necessary to support it often hampers its successful application and acceptance by regulatory authorities. Despite a potentially broad user community, expertise is still concentrated across a handful of organizations and individuals. In order to facilitate the effective use of read-across, this document presents the state of the art, summarizes insights learned from reviewing ECHA published decisions regarding the relative successes/pitfalls surrounding read-across under REACH, and compiles the relevant activities and guidance documents. Special emphasis is given to the available existing tools and approaches, an analysis of ECHA's published final decisions associated with all levels of compliance checks and testing proposals, the consideration and expression of uncertainty, the use of biological support data, and the impact of the ECHA Read-Across Assessment Framework (RAAF) published in 2015.

Environmental Chemical Assessment in Clinical Practice: Unveiling the Elephant in the Room

A growing body of evidence suggests chemicals present in air, water, soil, food, building materials and household products are toxicants that contribute to the many chronic diseases typically seen in routine medical practice. Yet, despite calls from numerous organisations to provide clinicians with more training and awareness in environmental health, there are multiple barriers to the clinical assessment of toxic environmental exposures. Recent developments in the fields of systems biology, innovative breakthroughs in biomedical research encompassing the "-omics" fields, and advances in mobile sensing, peer-to-peer networks and big data, provide tools that future clinicians can use to assess environmental chemical exposures in their patients. There is also a need for concerted action at all levels, including actions by individual patients, clinicians, medical educators, regulators, government and non-government organisations, corporations and the wider civil society, to understand the "exposome" and minimise the extent of toxic exposures on current and future generations. Clinical environmental chemical risk assessment may provide a bridge between multiple disciplines that uses new technologies to herald in a new era in personalised medicine that unites clinicians, patients and civil society in the quest to understand and master the links between the environment and human health.

Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools

This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the test method for a given purpose. Relevance encapsulates the scientific basis of the test method, its capacity to predict adverse effects in the "target system" (i.e. human health or the environment) as well as its applicability for the intended purpose. In this chapter we focus on the validation of non-animal in vitro alternative testing methods and review the concepts, challenges, processes and tools fundamental to the validation of in vitro methods intended for hazard testing of chemicals. We explore major challenges and peculiarities of validation in this area. Based on the notion that validation per se is a scientific endeavour that needs to adhere to key scientific principles, namely objectivity and appropriate choice of methodology, we examine basic aspects of study design and management, and provide illustrations of statistical approaches to describe predictive performance of validated test methods as well as their reliability.

Evidence-based causation in toxicology

We introduced Evidence-based Toxicology (EBT) in 2005 to address the disparities that exist between the various Weight-of-Evidence (WOE) methods typically applied in the regulatory hazard decision-making arena and urged toxicologists to adopt the evidence-based guidelines long-utilized in medicine (i.e., Evidence-Based Medicine or EBM). This review of the activities leading to the adoption of evidence-based methods and EBT during the last decade demonstrates how fundamental concepts that form EBT, such as the use of systematic reviews to capture and consider all available information, are improving toxicological evaluations performed by various groups and agencies. We reiterate how the EBT framework, a process that provides a method for performing human chemical causation analyses in an objective, transparent and reproducible manner, differs significantly from past and current regulatory WOE approaches. We also discuss why the uncertainties associated with regulatory WOE schemes lead to a definition of the term “risk” that contains unquantifiable uncertainties not present in this term as it is used in epidemiology and medicine. We believe this distinctly different meaning of “risk” should be clearly conveyed to those not familiar with this difference (e.g., the lay public), when theoretical/nomologic risks associated with chemical-induced toxicities are presented outside of regulatory and related scientific parlance.

From "weight of evidence" to quantitative data integration using multicriteria decision analysis and Bayesian methods

“Weighing” available evidence in the process of decision-making is unavoidable, yet it is one step that routinely raises suspicions: what evidence should be used, how much does it weigh, and whose thumb may be tipping the scales? This commentary aims to evaluate the current state and future roles of various types of evidence for hazard assessment as it applies to environmental health. In its recent evaluation of the US Environmental Protection Agency’s Integrated Risk Information System assessment process, the National Research Council committee singled out the term “weight of evidence” (WoE) for critique, deeming the process too vague and detractive to the practice of evaluating human health risks of chemicals. Moving the methodology away from qualitative, vague and controversial methods towards generalizable, quantitative and transparent methods for appropriately managing diverse lines of evidence is paramount for both regulatory and public acceptance of the hazard assessments. The choice of terminology notwithstanding, a number of recent Bayesian WoE-based methods, the emergence of multi criteria decision analysis for WoE applications, as well as the general principles behind the foundational concepts of WoE, show promise in how to move forward and regain trust in the data integration step of the assessments. We offer our thoughts on the current state of WoE as a whole and while we acknowledge that many WoE applications have been largely qualitative and subjective in nature, we see this as an opportunity to turn WoE towards a quantitative direction that includes Bayesian and multi criteria decision analysis.

The human toxome project

The Human Toxome Project, funded as an NIH Transformative Research grant 2011-2016, is focused on developing the concepts and the means for deducing, validating and sharing molecular pathways of toxicity (PoT). Using the test case of estrogenic endocrine disruption, the responses of MCF-7 human breast cancer cells are being phenotyped by transcriptomics and mass-spectroscopy-based metabolomics. The bioinformatics tools for PoT deduction represent a core deliverable. A number of challenges for quality and standardization of cell systems, omics technologies and bioinformatics are being addressed. In parallel, concepts for annotation, validation and sharing of PoT information, as well as their link to adverse outcomes, are being developed. A reasonably comprehensive public database of PoT, the Human Toxome Knowledge-base, could become a point of reference for toxicological research and regulatory test strategies.

Developmental neurotoxicity - challenges in the 21st century and in vitro opportunities

In recent years neurodevelopmental problems in children have increased at a rate that suggests lifestyle factors and chemical exposures as likely contributors. When environmental chemicals contribute to neurodevelopmental disorders developmental neurotoxicity (DNT) becomes an enormous concern. But how can it be tackled? Current animal test- based guidelines are prohibitively expensive, at $ 1.4 million per substance, while their predictivity for human health effects may be limited, and mechanistic data that would help species extrapolation are not available. A broader screening for substances of concern requires a reliable testing strategy, applicable to larger numbers of substances, and sufficiently predictive to warrant further testing. This review discusses the evidence for possible contributions of environmental chemicals to DNT, limitations of the current test paradigm, emerging concepts and technologies pertinent to in vitro DNT testing and assay evaluation, as well as the prospect of a paradigm shift based on 21st century technologies.

Biological and medical applications of a brain-on-a-chip

The desire to develop and evaluate drugs as potential countermeasures for biological and chemical threats requires test systems that can also substitute for the clinical trials normally crucial for drug development. Current animal models have limited predictivity for drug efficacy in humans as the large majority of drugs fails in clinical trials. We have limited understanding of the function of the central nervous system and the complexity of the brain, especially during development and neuronal plasticity. Simple in vitro systems do not represent physiology and function of the brain. Moreover, the difficulty of studying interactions between human genetics and environmental factors leads to lack of knowledge about the events that induce neurological diseases. Microphysiological systems (MPS) promise to generate more complex in vitro human models that better simulate the organ's biology and function. MPS combine different cell types in a specific three-dimensional (3D) configuration to simulate organs with a concrete function. The final aim of these MPS is to combine different "organoids" to generate a human-on-a-chip, an approach that would allow studies of complex physiological organ interactions. The recent discovery of induced pluripotent stem cells (iPSCs) gives a range of possibilities allowing cellular studies of individuals with different genetic backgrounds (e.g., human disease models). Application of iPSCs from different donors in MPS gives the opportunity to better understand mechanisms of the disease and can be a novel tool in drug development, toxicology, and medicine. In order to generate a brain-on-a-chip, we have established a 3D model from human iPSCs based on our experience with a 3D rat primary aggregating brain model. After four weeks of differentiation, human 3D aggregates stain positive for different neuronal markers and show higher gene expression of various neuronal differentiation markers compared to 2D cultures. Here we present the applications and challenges of this emerging technology.

Mechanistic validation

Validation of new approaches in regulatory toxicology is commonly defined as the independent assessment of the reproducibility and relevance (the scientific basis and predictive capacity) of a test for a particular purpose. In large ring trials, the emphasis to date has been mainly on reproducibility and predictive capacity (comparison to the traditional test) with less attention given to the scientific or mechanistic basis. Assessing predictive capacity is difficult for novel approaches (which are based on mechanism), such as pathways of toxicity or the complex networks within the organism (systems toxicology). This is highly relevant for implementing Toxicology for the 21st Century, either by high-throughput testing in the ToxCast/Tox21 project or omics-based testing in the Human Toxome Project. This article explores the mostly neglected assessment of a test's scientific basis, which moves mechanism and causality to the foreground when validating/qualifying tests. Such mechanistic validation faces the problem of establishing causality in complex systems. However, pragmatic adaptations of the Bradford Hill criteria, as well as bioinformatic tools, are emerging. As critical infrastructures of the organism are perturbed by a toxic mechanism we argue that by focusing on the target of toxicity and its vulnerability, in addition to the way it is perturbed, we can anchor the identification of the mechanism and its verification.

Look back in anger - what clinical studies tell us about preclinical work

Misled by animal studies and basic research? Whenever we take a closer look at the outcome of clinical trials in a field such as, most recently, stroke or septic shock, we see how limited the value of our preclinical models was. For all indications, 95% of drugs that enter clinical trials do not make it to the market, despite all promise of the (animal) models used to develop them. Drug development has started already to decrease its reliance on animal models: In Europe, for example, despite increasing R&D expenditure, animal use by pharmaceutical companies dropped by more than 25% from 2005 to 2008. In vitro studies are likewise limited: questionable cell authenticity, over-passaging, mycoplasma infections, and lack of differentiation as well as non-homeostatic and non-physiologic culture conditions endanger the relevance of these models. The standards of statistics and reporting often are poor, further impairing reliability. Alarming studies from industry show miserable reproducibility of landmark studies. This paper discusses factors contributing to the lack of reproducibility and relevance of pre-clinical research.

THE CONCLUSION

Publish less but of better quality and do not rely on the face value of animal studies.

Integrated testing strategies for safety assessments

Despite the fact that toxicology uses many stand-alone tests, a systematic combination of several information sources very often is required: Examples include: when not all possible outcomes of interest (e.g., modes of action), classes of test substances (applicability domains), or severity classes of effect are covered in a single test; when the positive test result is rare (low prevalence leading to excessive false-positive results); when the gold standard test is too costly or uses too many animals, creating a need for prioritization by screening. Similarly, tests are combined when the human predictivity of a single test is not satisfactory or when existing data and evidence from various tests will be integrated. Increasingly, kinetic information also will be integrated to make an in vivo extrapolation from in vitro data. Integrated Testing Strategies (ITS) offer the solution to these problems. ITS have been discussed for more than a decade, and some attempts have been made in test guidance for regulations. Despite their obvious potential for revamping regulatory toxicology, however, we still have little guidance on the composition, validation, and adaptation of ITS for different purposes. Similarly, Weight of Evidence and Evidence-based Toxicology approaches require different pieces of evidence and test data to be weighed and combined. ITS also represent the logical way of combining pathway-based tests, as suggested in Toxicology for the 21st Century. This paper describes the state of the art of ITS and makes suggestions as to the definition, systematic combination, and quality assurance of ITS.

Design of a testing strategy using non-animal based test methods: lessons learnt from the ACuteTox project

In the framework of toxicology, a testing strategy can be viewed as a series of steps which are taken to come to a final prediction about a characteristic of a compound under study. The testing strategy is performed as a single-step procedure, usually called a test battery, using simultaneously all information collected on different endpoints, or as tiered approach in which a decision tree is followed. Design of a testing strategy involves statistical considerations, such as the development of a statistical prediction model. During the EU FP6 ACuteTox project, several prediction models were proposed on the basis of statistical classification algorithms which we illustrate here. The final choice of testing strategies was not based on statistical considerations alone. However, without thorough statistical evaluations a testing strategy cannot be identified. We present here a number of observations made from the statistical viewpoint which relate to the development of testing strategies. The points we make were derived from problems we had to deal with during the evaluation of this large research project. A central issue during the development of a prediction model is the danger of overfitting. Procedures are presented to deal with this challenge.

Validation and quality control of replacement alternatives – current status and future challenges

Alternatives to animal testing have been developed mainly in the fields of toxicology and vaccine testing. Typical examples are the evaluation of phototoxicity, eye irritation or skin corrosion/irritation of cosmetics and industrial chemicals. However, examples can also be found in other biomedical areas, such the control of the quality of drug preparations for pyrogens or for the control of the production process of biologics, such as botulinum neurotoxin. For regulatory purposes, the quality, transferability and predictivity of an alternative method needs to be evaluated. This procedure is called the “validation process” of a new method. It follows defined rules, and several governmental institutions have been established to perform, supervise or advise on this process. As this often results in a delay of method implementation, different alternatives for the evaluation of a method's suitability and quality are under discussion. We describe here the principles of model development and quality control. We also give an overview on methods that have undergone validation. Strengths and shortcomings of traditional approaches are discussed, and new developments and challenges are outlined.

The use of integrated and intelligent testing strategies in the prediction of toxic hazard and in risk assessment

There is increasing concern that insurmountable differences between humans and laboratory animals limit the relevance and reliability for hazard identification and risk assessment purposes of animal data produced by traditional toxicity test procedures. A way forward is offered by the emerging new technologies, which can be directly applied to human material or even to human beings themselves. This promises to revolutionise the evaluation of the safety of chemicals and chemical products of various kinds and, in particular, pharmaceuticals. The available and developing technologies are summarised and it is emphasised that they will need to be used selectively, in integrated and intelligent testing strategies, which, in addition to being scientifically sound, must be manageable and affordable. Examples are given of proposed testing strategies for general chemicals, cosmetic ingredients, candidate pharmaceuticals, inhaled substances, nanoparticles and neurotoxicity.

Expert review for GHS classification of chemicals on health effects

Intoxication as a result of chemical accidents is a major issue in industrial health. The Globally Harmonized System of Classification and Labelling of Chemicals (GHS) provides a framework for hazard communication on chemicals using labelling or safety data sheets. The GHS will be expected to reduce the number of chemical accidents by communicating the hazards posed and prompting safety measures to be taken. One of the issues which may be a barrier to effective implementation of the GHS results from discrepancies in GHS classifications of chemicals across countries/regions. The main reasons are the differences in information sources used and in the expertise of people making the classification (Classifiers). The GHS requests expert judgment in a weight of evidence (WOE) approach in the application of the criteria of classification. A WOE approach is an assessment method that considers all available information bearing on the determination of toxicity. The quality and consistency of the data, study design, mechanism or mode of action, dose-effect relationships and biological relevance should be taken into account. Therefore, expert review should be necessary to classify chemicals accurately. However, the GHS does not provide any information on the required level of expertise of the Classifiers, definition of who qualifies as an expert, evaluation methods of WOE or data quality, and the timing of expert judgment and the need for updating/re-classification as new information becomes available. In this paper, key methods and issues in expert reviews are discussed. Examples of expert reviews and recommendations for harmonized classification are also presented.

Alternative in vitro assays in nanomaterial toxicology

Nanomaterials are acclaimed for their novel properties, for which broad new uses are being discovered with increasing frequency. It is obvious that, as the properties change, unwanted properties (toxicity) are to be expected as well. Current toxicology, however, is already overwhelmed with the challenge of addressing new chemicals, not to mention the enormous number of old chemicals never properly assessed. Limitations of traditional approaches range from animal welfare issues, which were a strong driving force for alternative approaches (the 3Rs concept) over the last two decades, to aspects of throughput and accuracy of the predicted toxicities. The latter has prompted discussion about a revolutionary change in chemical safety assessment, now known as Toxicology for the 21st Century (Tox-21c). The multitude of possible formulations of nanomaterials to be assessed for novel toxic properties makes these alternative approaches especially attractive, given the well recognized limitations of traditional animal-based approaches--limitations that might be even more pronounced for nanomaterials, which have notably altered biokinetics.

Enhancing credibility of chemical safety studies: emerging consensus on key assessment criteria

OBJECTIVES

We examined the extent to which consensus exists on the criteria that should be used for assessing the credibility of a scientific work, regardless of its funding source, and explored how these criteria might be implemented.

DATA SOURCES

Three publications, all presented at a session of the 2009 annual meeting of the Society for Risk Analysis, have proposed a range of criteria for evaluating the credibility of scientific studies. At least two other similar sets of criteria have recently been proposed elsewhere.

DATA EXTRACTION/SYNTHESIS

In this article we review these criteria, highlight the commonalities among them, and integrate them into a list of 10 criteria. We also discuss issues inherent in any attempt to implement the criteria systematically.

CONCLUSIONS

Recommendations by many scientists and policy experts converge on a finite list of criteria for assessing the credibility of a scientific study without regard to funding source. These criteria should be formalized through a consensus process or a governmental initiative that includes discussion and pilot application of a system for reproducibly implementing them. Formal establishment of such a system should enable the debate regarding chemical studies to move beyond funding issues and focus on scientific merit.

Moving forward in human cancer risk assessment

BACKGROUND

The current safety paradigm for assessing carcinogenic properties of drugs, cosmetics, industrial chemicals, and environmental exposures relies mainly on in vitro genotoxicity testing followed by 2-year rodent bioassays. This testing battery is extremely sensitive but has low specificity. Furthermore, rodent bioassays are associated with high costs, high animal burden, and limited predictive value for human risks.

OBJECTIVES

We provide a response to a growing appeal for a paradigm change in human cancer risk assessment.

METHODS

To facilitate development of a road map for this needed paradigm change in carcinogenicity testing, a workshop titled "Genomics in Cancer Risk Assessment" brought together toxicologists from academia and industry and government regulators and risk assessors from the United States and the European Union. Participants discussed the state-of-the-art in developing alternative testing strategies for carcinogenicity, with emphasis on potential contributions from omics technologies.

RESULTS AND CONCLUSIONS

The goal of human risk assessment is to decide whether a given exposure to an agent is acceptable to human health and to provide risk management measures based on evaluating and predicting the effects of exposures on human health. Although exciting progress is being made using genomics approaches, a new paradigm that uses these methods and human material when possible would provide mechanistic insights that may inform new predictive approaches (e.g., in vitro assays) and facilitate the development of genomics-derived biomarkers. Regulators appear to be willing to accept such approaches where use is clearly defined, evidence is strong, and approaches are qualified for regulatory use.

Computational science in drug metabolism and toxicology

Computational scientific tools involving construction and testing of models, screening and data mining for drug and chemical induced toxicities and metabolism have significantly grown in experimental use to help guide product development and assist by enhancing certain areas of regulatory decision making. This themed issue of the journal entitled Computational Science in Drug Metabolism & Toxicology contains state-of-the-art review articles and perspectives covering a diversity of in silico approaches. Computational science tools have a strong potential for expediting our further understanding of drug metabolism and toxicity and are continually being developed and validated. The reader will gain an understanding of the current state of in silico tools and modeling approaches aimed at reducing these liabilities. In addition, how these tools are tested and developed for use in drug safety to support drug development efforts and a review of how they are used to predict genotoxic liabilities are covered in this issue. Computational science tools when properly validated and used judiciously can lend themselves as enablers to support drug safety assessment in investigative and applied settings.

Successful drug development despite adverse preclinical findings part 1: processes to address issues and most important findings

Unexpected adverse preclinical findings (APFs) are not infrequently encountered during drug development. Such APFs can be functional disturbances such as QT prolongation, morphological toxicity or carcinogenicity. The latter is of particular concern in conjunction with equivocal genotoxicity results. The toxicologic pathologist plays an important role in recognizing these effects, in helping to characterize them, to evaluate their risk for man, and in proposing measures to mitigate the risk particularly in early clinical trials. A careful scientific evaluation is crucial while termination of the development of a potentially useful drug must be avoided. This first part of the review discusses processes to address unexpected APFs and provides an overview over typical APFs in particular classes of drugs. If the mode of action (MoA) by which a drug candidate produces an APF is known, this supports evaluation of its relevance for humans. Tailor-made mechanistic studies, when needed, must be planned carefully to test one or several hypotheses regarding the potential MoA and to provide further data for risk evaluation. Safety considerations are based on exposure at no-observed-adverse-effect levels (NOAEL) of the most sensitive and relevant animal species and guide dose escalation in clinical trials. The availability of early markers of toxicity for monitoring of humans adds further safety to clinical studies. Risk evaluation is concluded by a weight of evidence analysis (WoE) with an array of parameters including drug use, medical need and alternatives on the market. In the second part of this review relevant examples of APFs will be discussed in more detail.

Comparing in vivo, in vitro and in silico methods and integrated strategies for chemical assessment: problems and prospects

The RAINBOW workshop addressed the background for the integration of in vivo, in vitro and computer-based (in silico) methods, to facilitate the study of the toxic properties of chemicals. On the basis of these discussions, we prepared the present paper, outlining the strengths, weaknesses, opportunities and threats of each approach, both alone and integrated into a single testing strategy. The current scheme for evaluation of chemicals needs to be reshaped, in the face of the much larger numbers of chemicals which need to be examined and the availability of a diversified set of tools.

Acute oral toxicity: variability, reliability, relevance and interspecies comparison of rodent LD50 data from literature surveyed for the ACuteTox project

The ACuteTox project has aimed to optimise and prevalidate an in vitro testing strategy for predicting human acute toxicity. Ninety-seven reference substances were selected and an in vivo acute toxicity database was compiled. Comprehensive statistical analyses of the in vivo LD50 data to evaluate variability and reliability, interspecies correlation, predictive capacities with regard to EU and GHS toxicity categories, and deduction of performance criteria for in vitro methods is presented. For the majority of substances variability among rodent data followed a log normal distribution where good reproducibility was found. Rat and mouse interspecies comparison of LD50 studies by ordinary regression showed high correlation, with coefficients of determination, ranging between 0.8 and 0.9. Substance specific differences were only significant for warfarin and cycloheximide. No correlation of compound LD50 range with presumed study quality rank (by assigning Klimisch reliability scores) was found. Modelling based on LD50 variability showed that with at least 90% probability ∼54% of the substances would fall into only one GHS category and ∼44% would fall within two adjacent categories. These results could form the basis for deriving a predictive capacity that should be expected from alternative approaches to the conventional in vivo acute oral toxicity test.