The importance of the prediction model in the validation of alternative tests

Avoiding a reproducibility crisis in regulatory toxicology-on the fundamental role of ring trials

The ongoing transition from chemical hazard and risk assessment based on animal studies to assessment relying mostly on non-animal data, requires a multitude of novel experimental methods, and this means that guidance on the validation and standardisation of test methods intended for international applicability and acceptance, needs to be updated. These so-called new approach methodologies (NAMs) must be applicable to the chemical regulatory domain and provide reliable data which are relevant to hazard and risk assessment. Confidence in and use of NAMs will depend on their reliability and relevance, and both are thoroughly assessed by validation. Validation is, however, a time- and resource-demanding process. As updates on validation guidance are conducted, the valuable components must be kept: Reliable data are and will remain fundamental. In 2016, the scientific community was made aware of the general crisis in scientific reproducibility-validated methods must not fall into this. In this commentary, we emphasize the central importance of ring trials in the validation of experimental methods. Ring trials are sometimes considered to be a major hold-up with little value added to the validation. Here, we clarify that ring trials are indispensable to demonstrate the robustness and reproducibility of a new method. Further, that methods do fail in method transfer and ring trials due to different stumbling blocks, but these provide learnings to ensure the robustness of new methods. At the same time, we identify what it would take to perform ring trials more efficiently, and how ring trials fit into the much-needed update to the guidance on the validation of NAMs.

Establishment of a human cell-based in vitro battery to assess developmental neurotoxicity hazard of chemicals

Developmental neurotoxicity (DNT) is a major safety concern for all chemicals of the human exposome. However, DNT data from animal studies are available for only a small percentage of manufactured compounds. Test methods with a higher throughput than current regulatory guideline methods, and with improved human relevance are urgently needed. We therefore explored the feasibility of DNT hazard assessment based on new approach methods (NAMs). An in vitro battery (IVB) was assembled from ten individual NAMs that had been developed during the past years to probe effects of chemicals on various fundamental neurodevelopmental processes. All assays used human neural cells at different developmental stages. This allowed us to assess disturbances of: (i) proliferation of neural progenitor cells (NPC); (ii) migration of neural crest cells, radial glia cells, neurons and oligodendrocytes; (iii) differentiation of NPC into neurons and oligodendrocytes; and (iv) neurite outgrowth of peripheral and central neurons. In parallel, cytotoxicity measures were obtained. The feasibility of concentration-dependent screening and of a reliable biostatistical processing of the complex multi-dimensional data was explored with a set of 120 test compounds, containing subsets of pre-defined positive and negative DNT compounds. The battery provided alerts (hit or borderline) for 24 of 28 known toxicants (82% sensitivity), and for none of the 17 negative controls. Based on the results from this screen project, strategies were developed on how IVB data may be used in the context of risk assessment scenarios employing integrated approaches for testing and assessment (IATA).

Evaluation of Skin Irritation of Acids Commonly Used in Cleaners in 3D-Reconstructed Human Epidermis Model, KeraSkinTM

Cleaners such as dishwashing liquids contain various chemicals that cause skin damage. Alkaline agents used in cleaners alter the lipid composition of the skin and damage the skin barrier. However, little is known about the effects of acids used in cleaners on the skin. Here, we investigated the effects of acidic pH on the skin and evaluated the skin irritation of acids commonly used in cleaners with a 3D-reconstructed human epidermis model, KeraSkin^TM, according to OECD TG439. First, to examine the effects of acidic pH, we evaluated the skin irritation of citrate buffers (0.1 M, McIlvaine buffer) prepared in a wide pH range (pH 1.5-6.0). Surprisingly, cell viability was not significantly affected even at pH 1.5, reflecting that the acidity alone may not be sufficient to induce skin irritation. Even after longer exposure (180 min), the cell viability was not reduced below 50%, a cutoff to determine an irritant. To examine the effect of the anionic part, several organic acids used in cleaners (citric acid, glycolic acid, lactic acid, malic acid, and succinic acid) were examined. These organic acids also failed to reduce viability at 0.1 M. However, at 1 M, most of the acids tested, except lactic acid, were determined to be skin irritants. Histology further supported the skin irritancy of acids at 1 M. Similarly, inorganic acids (hydrogen bromide, hydrogen chloride, nitric acid, and sulfuric acid) were determined to be irritants only at 1 M. In the case of alkaline agents, pH and concentrations were also important factors to determine the skin irritancy, although the epidermal structure and lipids were more damaged than acids. Collectively, we demonstrated that both the pH and concentration are important factors for the skin irritancy of acids, shedding an important insight into the mechanism of skin irritation.

Guidance on risk assessment of nanomaterials to be applied in the food and feed chain: human and animal health

The EFSA has updated the Guidance on risk assessment of the application of nanoscience and nanotechnologies in the food and feed chain, human and animal health. It covers the application areas within EFSA's remit, including novel foods, food contact materials, food/feed additives and pesticides. The updated guidance, now Scientific Committee Guidance on nano risk assessment (SC Guidance on Nano-RA), has taken account of relevant scientific studies that provide insights to physico-chemical properties, exposure assessment and hazard characterisation of nanomaterials and areas of applicability. Together with the accompanying Guidance on Technical requirements for regulated food and feed product applications to establish the presence of small particles including nanoparticles (Guidance on Particle-TR), the SC Guidance on Nano-RA specifically elaborates on physico-chemical characterisation, key parameters that should be measured, methods and techniques that can be used for characterisation of nanomaterials and their determination in complex matrices. The SC Guidance on Nano-RA also details aspects relating to exposure assessment and hazard identification and characterisation. In particular, nanospecific considerations relating to in vitro/in vivo toxicological studies are discussed and a tiered framework for toxicological testing is outlined. Furthermore, in vitro degradation, toxicokinetics, genotoxicity, local and systemic toxicity as well as general issues relating to testing of nanomaterials are described. Depending on the initial tier results, additional studies may be needed to investigate reproductive and developmental toxicity, chronic toxicity and carcinogenicity, immunotoxicity and allergenicity, neurotoxicity, effects on gut microbiome and endocrine activity. The possible use of read-across to fill data gaps as well as the potential use of integrated testing strategies and the knowledge of modes or mechanisms of action are also discussed. The Guidance proposes approaches to risk characterisation and uncertainty analysis.

Guidance on risk assessment of the application of nanoscience and nanotechnologies in the food and feed chain: Part 1, human and animal health

The European Food Safety Authority has produced this Guidance on human and animal health aspects (Part 1) of the risk assessment of nanoscience and nanotechnology applications in the food and feed chain. It covers the application areas within EFSA's remit, e.g. novel foods, food contact materials, food/feed additives and pesticides. The Guidance takes account of the new developments that have taken place since publication of the previous Guidance in 2011. Potential future developments are suggested in the scientific literature for nanoencapsulated delivery systems and nanocomposites in applications such as novel foods, food/feed additives, biocides, pesticides and food contact materials. Therefore, the Guidance has taken account of relevant new scientific studies that provide more insights to physicochemical properties, exposure assessment and hazard characterisation of nanomaterials. It specifically elaborates on physicochemical characterisation of nanomaterials in terms of how to establish whether a material is a nanomaterial, the key parameters that should be measured, the methods and techniques that can be used for characterisation of nanomaterials and their determination in complex matrices. It also details the aspects relating to exposure assessment and hazard identification and characterisation. In particular, nanospecific considerations relating to in vivo/in vitro toxicological studies are discussed and a tiered framework for toxicological testing is outlined. It describes in vitro degradation, toxicokinetics, genotoxicity as well as general issues relating to testing of nanomaterials. Depending on the initial tier results, studies may be needed to investigate reproductive and developmental toxicity, immunotoxicity, allergenicity, neurotoxicity, effects on gut microbiome and endocrine activity. The possible use of read‐across to fill data gaps as well as the potential use of integrated testing strategies and the knowledge of modes/mechanisms of action are also discussed. The Guidance proposes approaches to risk characterisation and uncertainty analysis, and provides recommendations for further research in this area.

This publication is linked to the following EFSA Supporting Publications article: http://onlinelibrary.wiley.com/doi/10.2903/sp.efsa.2018.EN-1430/full

Integrated Approaches to Testing and Assessment

In this chapter, we explain how Integrated Approaches to Testing and Assessment (IATA) offer a means of integrating and translating the data generated by toxicity testing methods, thereby serving as flexible and suitable tools for toxicological decision making in the twenty-first century. In addition to traditional in vitro and in vivo testing methods, IATA are increasingly incorporating newly developed in vitro systems and measurement technologies such as high throughput screening and high content imaging. Computational approaches are also being used in IATA development, both as a means of generating data (e.g. QSARs), interpreting data (bioinformatics and chemoinformatics), and as a means of integrating multiple sources of data (e.g. expert systems, bayesian models). Decision analytic methods derived from socioeconomic theory can also play a role in developing flexible and optimal IATA solutions. Some of the challenges involved in the development, validation and implementation of IATA are also discussed.

Mechanistic validation

Validation of new approaches in regulatory toxicology is commonly defined as the independent assessment of the reproducibility and relevance (the scientific basis and predictive capacity) of a test for a particular purpose. In large ring trials, the emphasis to date has been mainly on reproducibility and predictive capacity (comparison to the traditional test) with less attention given to the scientific or mechanistic basis. Assessing predictive capacity is difficult for novel approaches (which are based on mechanism), such as pathways of toxicity or the complex networks within the organism (systems toxicology). This is highly relevant for implementing Toxicology for the 21st Century, either by high-throughput testing in the ToxCast/Tox21 project or omics-based testing in the Human Toxome Project. This article explores the mostly neglected assessment of a test's scientific basis, which moves mechanism and causality to the foreground when validating/qualifying tests. Such mechanistic validation faces the problem of establishing causality in complex systems. However, pragmatic adaptations of the Bradford Hill criteria, as well as bioinformatic tools, are emerging. As critical infrastructures of the organism are perturbed by a toxic mechanism we argue that by focusing on the target of toxicity and its vulnerability, in addition to the way it is perturbed, we can anchor the identification of the mechanism and its verification.

Biological Read-Across: Mechanistically-Based Species–Species and Endpoint–Endpoint Extrapolations

This chapter describes the development and use of relationships between toxicity data, for the same chemicals, derived from different species. These relationships provide simple models to predict toxicity of, potentially, a higher species from that of a lower species. Approaches to the formation of these models are described, notably inter-species relationships, quantitative structure-activity-activity relationships and prediction models. It is noted that the best extrapolations are for closely related species i.e. within taxa. In addition, forming groups or categories of compounds according to common mechanisms of toxic action improves the correlation for extrapolations from lower to higher species. A freely available software package, Web-ICE, is introduced as a suitable tool to apply these methods.

Diagnosis: Toxic! – Trying to Apply Approaches of Clinical Diagnostics and Prevalence in Toxicology Considerations

The assessment of relevance of toxicological testing was compared with approaches of diagnostic medicine, a discipline that faces a comparable situation. Considering the work of a toxicologist as setting a diagnosis for compounds, assessment tools for diagnostic tests were transferred to toxicological tests. In clinical diagnostics, test uncertainty is well accepted and incorporated in this assessment. Furthermore, prevalence information is considered to evaluate the gain in information resulting from the application of a test. Several common toxicological scenarios, in which test uncertainty and prevalence are combined, are discussed including the interdependence of test accuracy, prevalence and predictive values or the sequential application of a screening and a confirmatory test. In addition, real prevalences derived from prevalences determined by an imperfect test are presented. We conclude that information on prevalences of toxic health effects is required to allow a complete assessment of the relevance of toxicological test. In this process, lessons can be learned from evidence-based approaches in clinical diagnostics.

Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs

This article provides an overview of methods for reliability assessment of quantitative structure-activity relationship (QSAR) models in the context of regulatory acceptance of human health and environmental QSARs. Useful diagnostic tools and data analytical approaches are highlighted and exemplified. Particular emphasis is given to the question of how to define the applicability borders of a QSAR and how to estimate parameter and prediction uncertainty. The article ends with a discussion regarding QSAR acceptability criteria. This discussion contains a list of recommended acceptability criteria, and we give reference values for important QSAR performance statistics. Finally, we emphasize that rigorous and independent validation of QSARs is an essential step toward their regulatory acceptance and implementation.

Prediction models for eye irritation potential based on endpoints of the HETCAM and neutral red uptake tests

The aim of this study was to explore the possibility of distinguishing between eye irritants (I; EU risk phrases R36 and R41) and nonirritants (NI), by using in vitro endpoints of the hen's egg test on the chorioallantoic membrane (the HETCAM test) and the neutral red uptake (NRU) test. Prediction models were derived by applying binary logistic regression to the in vitro data for these endpoints, which were taken from the report of a German validation study on the use of the HETCAM and 3T3 NRU tests as alternatives to the Draize eye irritation test. Whereas the validation study led to the conclusion that the combined use of the two tests enables a satisfactory discrimination between severe (R41) and nonsevere (NI, R36) eye irritants, the results of the present study indicate that the two in vitro tests can also be used to discriminate between nonirritants (NI) and irritants (R36 and R41).

The use of pH measurements to predict the potential of chemicals to cause acute dermal and ocular toxicity

Regulatory guidelines for the assessment of acute dermal and ocular toxicity refer to the need to take the pH values of chemicals into consideration, since the acidic and basic properties of chemicals are known to play a role in the generation of acute dermal and ocular lesions. However, not all test guidelines provide an objective interpreting pH measurements in terms of acute skin or eye toxicity. The aim of this study was to develop classification models based on pH data for predicting the potential of chemicals to cause skin corrosion, skin irritation and eye irritation. The possible application of these models in the context of tiered testing strategies is discussed.