Validation of alternative methods for toxicity testing

Avoiding a reproducibility crisis in regulatory toxicology-on the fundamental role of ring trials

The ongoing transition from chemical hazard and risk assessment based on animal studies to assessment relying mostly on non-animal data, requires a multitude of novel experimental methods, and this means that guidance on the validation and standardisation of test methods intended for international applicability and acceptance, needs to be updated. These so-called new approach methodologies (NAMs) must be applicable to the chemical regulatory domain and provide reliable data which are relevant to hazard and risk assessment. Confidence in and use of NAMs will depend on their reliability and relevance, and both are thoroughly assessed by validation. Validation is, however, a time- and resource-demanding process. As updates on validation guidance are conducted, the valuable components must be kept: Reliable data are and will remain fundamental. In 2016, the scientific community was made aware of the general crisis in scientific reproducibility-validated methods must not fall into this. In this commentary, we emphasize the central importance of ring trials in the validation of experimental methods. Ring trials are sometimes considered to be a major hold-up with little value added to the validation. Here, we clarify that ring trials are indispensable to demonstrate the robustness and reproducibility of a new method. Further, that methods do fail in method transfer and ring trials due to different stumbling blocks, but these provide learnings to ensure the robustness of new methods. At the same time, we identify what it would take to perform ring trials more efficiently, and how ring trials fit into the much-needed update to the guidance on the validation of NAMs.

Accounting for Precision Uncertainty of Toxicity Testing: Methods to Define Borderline Ranges and Implications for Hazard Assessment of Chemicals

For hazard classifications of chemicals, continuous data from animal- or nonanimal testing methods are often dichotomized into binary positive/negative outcomes by defining classification thresholds (CT). Experimental data are, however, subject to biological and technical variability. Each test method's precision is limited resulting in uncertainty of the positive/negative outcome if the experimental result is close to the CT. Borderline ranges (BR) around the CT were suggested, which represent ranges in which the study result is ambiguous, that is, positive or negative results are equally likely. The BR reflects a method's precision uncertainty. This article explores and compares different approaches to quantify the BR. Besides using the pooled standard deviation, we determine the BR by means of the median absolute deviation (MAD), with a sequential combination of both methods, and by using nonparametric bootstrapping. Furthermore, we quantify the BR for different hazardous effects, including nonanimal tests for skin corrosion, eye irritation, skin irritation, and skin sensitization as well as for an animal test on skin sensitization (the local lymph node assay, LLNA). Additionally, for one method (direct peptide reactivity assay) the BR was determined experimentally and compared to calculated BRs. Our results demonstrate that (i) the precision of the methods is determining the size of their BRs, (ii) there is no "perfect" method to derive a BR, alas, (iii) a consensus on BR is needed to account for the limited precision of testing methods.

Comparison of the predictive nature of the Genomic Allergen Rapid Detection (GARD) assay with mammalian assays in determining the skin sensitisation potential of agrochemical active ingredients

Alternatives to mammalian testing are highly desirable to predict the skin sensitisation potential of agrochemical active ingredients (AI). The GARD assay, a stimulated, dendritic cell-like, cell line measuring genomic signatures, was evaluated using twelve AIs (seven sensitisers and five non-sensitisers) and the results compared with historical results from guinea pig or local lymph node assay (LLNA) studies. Initial GARD results suggested 11/12 AIs were sensitisers and six concurred with mammalian data. Conformal predictions changed one AI to a non-sensitiser. An AI identified as non-sensitising in the GARD assay was considered a potent sensitiser in the LLNA. In total 7/12 GARD results corresponded with mammalian data. AI chemistries might not be comparable to the GARD training set in terms of applicability domains. Whilst the GARD assay can replace mammalian tests for skin sensitisation evaluation for compounds including cosmetic ingredients, further work in agrochemical chemistries is needed for this assay to be a viable replacement to animal testing. The work conducted here is, however, considered exploratory research and the methodology needs further development to be validated for agrochemicals. Mammalian and other alternative assays for regulatory safety assessments of AIs must provide confidence to assign the appropriate classification for human health protection.

Progress toward the Validation of Alternative Tests

ECVAM's role in the practical validation of replacement alternative methods for use in regulatory testing is reviewed, including an outline of the criteria which have been used in determining ECVAM's priorities. Some of the difficulties which have arisen in validation studies are discussed, and solutions to these are proposed, with particular emphasis on ensuring that methods are sufficiently well-developed to enter the validation process, and on the ECVAM prevalidation scheme for encouraging protocol optimisation and the prior assessment of interlaboratory transferability. Comments are made on problems encountered in selecting test materials backed by adequate in vivo data and in undertaking appropriate in vivo/in vitro comparisons.

The Validation of Toxicological Prediction Models

An alternative method is shown to consist of two parts: the test system itself; and a prediction model for converting in vitro endpoints into predictions of in vivo toxicity. For the alternative method to be relevant and reliable, it is important that its prediction model component is of high predictive power and is sufficiently robust against sources of data variability. In other words, the prediction model must be subjected to criticism, leading successful models to the state of confirmation. It is shown that there are certain circumstances in which a new prediction model may be introduced without the necessity to generate new test system data.

Discovery of Ezrin Expression as a Potential Biomarker for Chemically Induced Ocular Irritation Using Human Corneal Epithelium Cell Line and a Reconstructed Human Cornea-like Epithelium Model

Numerous studies have attempted to develop a new in vitro eye irritation test (EIT). To obtain more reliable results from EIT, potential new biomarkers that reflect eye irritation by chemicals must be identified. We investigated candidate biomarkers for eye irritation, using a proteomics approach. Sodium lauryl sulfate (SLS) or benzalkonium chloride (BAC) was applied on a reconstructed human cornea-like epithelium model, MCTT HCE, and corneal protein expression was examined by two-dimensional gel electrophoresis. We found that ezrin (EZR) was significantly upregulated by SLS or BAC. In addition, upregulation of EZR in immortalized human corneal cells treated with SLS or BAC was confirmed by quantitative reverse transcription-PCR and western blot analysis. Furthermore, other well-known eye irritants such as cetylpyridinium bromide, Triton X-100, cyclohexanol, ethanol, 2-methyl-1-pentanol, and sodium hydroxide significantly increased EZR expression in immortalized human corneal cells. Induction of EZR promoter activity in irritant-treated human corneal cells was confirmed by a luciferase gene reporter assay. In conclusion, EZR expression may be a potential biomarker for detecting eye irritation, which may substantially improve the performance of in vitro EIT.

A Critical Assessment of the OECD Collaborative Study to Validate the Uterotrophic Assay for the Detection of Oestrogenic and Anti-oestrogenic Chemicals

The design and execution of a recently completed validation study on the rat uterotrophic assay for detecting oestrogens and anti-oestrogens, managed by the OECD, are critically assessed with respect to internationally agreed criteria for the validation of new in vitro and in vivo toxicity test methods. It is concluded that, while the design of the study did not take account of several important criteria for validation, the uterotrophic assay appears to reliably detect the strong and weak oestrogenic substances used in the study, which act via binding to the oestrogen receptor in vivo. However, the reliability of the assay has not been substantiated for detecting anti-oestrogens that act as antagonists, due to the involvement of an insufficient number of experiments and test chemicals. Moreover, the data do not permit an assessment of the accuracy of the prediction of oestrogenicity, and the protocols have not been sufficiently optimised with regard to controlling variables. This problem has been exacerbated by a wish to introduce as much flexibility as possible into the protocols during the formal validation phase of the study, rather than during a separate prevalidation stage. In addition, the choice between surgically treated and/or immature animals, and details of housing and husbandry conditions that are necessary for increasing the sensitivity and efficiency of the assay, need to be clarified. The assay also lacks a well-defined prediction model by which the overall relevance of the data to toxicity, and especially to human hazard, can be assessed, and no performance criteria have been established. The results of this analysis of the study indicate that it would be premature to produce an OECD test guideline for the uterotrophic assay at this time, before some of the above issues have been satisfactorily resolved.

Biostatistical Methods for the Validation of Alternative Methods for In Vitro Toxicity Testing

Statistical methods for the validation of toxicological in vitro test assays are developed and applied. Validation is performed either in comparison with in vivo assays or in comparison with other in vitro assays of established validity. Biostatistical methods are presented which are of potential use and benefit for the validation of alternative methods for the risk assessment of chemicals, providing at least an equivalent level of protection through in vitro toxicity testing to that obtained through the use of current in vivo methods. Characteristic indices are developed and determined. Qualitative outcomes are characterised by the rates of false-positive and false-negative predictions, sensitivity and specificity, and predictive values. Quantitative outcomes are characterised by regression coefficients derived from predictive models. The receiver operating characteristics (ROC) technique, applicable when a continuum of cut-off values is considered, is discussed in detail, in relation to its use for statistical modelling and statistical inference. The methods presented are examined for their use for the proof of safety and for toxicity detection and testing. We emphasise that the final validation of toxicity testing is human toxicity, and that the in vivo test itself is only a predictor with an inherent uncertainty. Therefore, the validation of the in vitro test has to account for the vagueness and uncertainty of the "gold standard" in vivo test. We address model selection and model validation, and a four-step scheme is proposed for the conduct of validation studies. Gaps and research needs are formulated to improve the validation of alternative methods for in vitro toxicity testing.

The Principles of Validation and the ECVAM Validation Process

The validation of a test method is the process by which the relevance and reliability of the method are assessed for a particular purpose. It is an essential stage in the evolution of the method from its development to its acceptance and application for regulatory purposes. The principles according to which alternative tests should be validated have been agreed upon at an international level, although the actual process by which the validation process is conducted varies between different validation authorities. This paper summarises the principles of alternative test development and validation and describes how the principles have been applied to the validation of in vitro tests by the European Centre for the Validation of Alternative Methods (ECVAM).

Perspectives on:In Vitro Evaluation of Biomedical Polymers

Recently polymeric materials have gained tremendous attention in a wide variety of applications spanning from electronics to environmental and biomedical fields. In this paper, current in vitro methods for polymers biocompatibility assessment are reviewed in combination with new concepts and techniques that appear promising for the development and improvement of in vitro methods with the purpose of reducing animal experimentation. The utilization of medical devices, for example, has always been subordinate to the assessment of their biocompatibility. This aspect, as well as the methods for evaluating biocompatibility have changed over the years as a result of new developments in cell biology that have revolutionized in vitro techniques for assaying polymeric materials for bioapplications.

Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools

This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the test method for a given purpose. Relevance encapsulates the scientific basis of the test method, its capacity to predict adverse effects in the "target system" (i.e. human health or the environment) as well as its applicability for the intended purpose. In this chapter we focus on the validation of non-animal in vitro alternative testing methods and review the concepts, challenges, processes and tools fundamental to the validation of in vitro methods intended for hazard testing of chemicals. We explore major challenges and peculiarities of validation in this area. Based on the notion that validation per se is a scientific endeavour that needs to adhere to key scientific principles, namely objectivity and appropriate choice of methodology, we examine basic aspects of study design and management, and provide illustrations of statistical approaches to describe predictive performance of validated test methods as well as their reliability.

Performance of the Neutral Red uptake assay in the COLIPA international validation study on alternatives to the rabbit eye irritation test

The neutral red uptake (NRU) assay was included as part of the COLIPA international validation trial of in vitro alternatives to the Draize eye irritation test. In a blind trial, 55 substances were tested at four laboratories. Following testing, a prediction of the in vivo Draize modified maximum average score (MMAS) for each substance was made by each laboratory using a prediction model relating mean NR(50) value (concentration causing 50% reduction in NRU from that of untreated control cells) to MMAS. Following statistical analysis of the results and breaking of the code, assessment of the results and further analysis was carried out by the participating laboratories. This paper presents the conclusions with regard to the NRU assay. The initial trial analysis indicated that the interlaboratory reproducibility of results of the NRU assay was good. However, there was a poor correlation between observed and predicted MMAS (using the proposed prediction model) when all the test substances were analysed together (r=0.246). Data analysis of subsets of substances indicated that the best predictions were for pure surfactants only (r=0.843) although this data did not fit within the limits of the prediction model. The NRU assay therefore appears to have limited use as a complete Draize replacement. A further examination of the COLIPA trial data may identify combinations of assays which may be more useful than the individual assays which, like NRU, have been shown to be poor predictors of eye irritation.

The Performance of the Tissue Equivalent Assay using the Skin(2)(TM) ZK1200 Model in the COLIPA International Validation Study on Alternatives to the Draize Eye Irritation Test

The tissue equivalent assay (TEA) (Osborne et al., 1995) was used to evaluate 55 mixed ingredients and formulations in the COLIPA International Validation Study on Alternatives to the Draize Rabbit Eye Irritation Test (Brantom et al., 1997). The TEA can be used to test all types of materials since it uses a topical application approach and is not limited to only testing liquid or soluble materials. A prediction model (PM) for the test was developed using historical eye irritation data from a total of 132 materials on which in vivo and in vitro data were available. A regression model was derived from these data and used to relate the in vitro endpoint (t(50)) obtained in the study to a Draize MMAS (modified maximum average score). This provided a measure of the predicted in vivo eye irritation scores. In the current study, two separate laboratories used the same protocol to test the same set of coded materials and the results of both laboratories were compared to the initial PM. The TEA met the reliability criteria of the validation study in reproducing the predefined PM in both laboratories, and a good relationship between predicted and observed Draize MMAS values was obtained (r=0.906 and r=0.850). Good correlations were maintained when separate analyses were made of the formulations and ingredients included in the test set. Good relationships between the in vitro endpoint and individual Draize tissue scores (r>0.8) were also exhibited. Although insufficient data were available to make an assessment of interlaboratory variation, some difference in the reproducibility of the assay was noted between the two laboratories, particularly for the highly irritating materials. However, the consistency of data was encouraging and the discrepancies seen between the laboratories suggested a sensitivity of the model to subtle differences in application techniques, and in handling and timing. Taken together, these results indicate the utility of the TEA test for these types of substances and the need to more fully address the issue of interlaboratory reproducibility.

Measurement of cytotoxicity by ATP-based luminescence assay in primary cell cultures and cell lines

Drug discovery and toxicological safety testing share a need for dependable in vitro cellular toxicity tests. Ideally such tests should be objective, quantitative, reproducible and able to lend themselves to automation. A number of assays fulfil these criteria well, but recently it has become clear that the molecular phenotype of the cell tested and the complex interplay between different cell types can radically alter the response to individual agents. The differences observed between primary cell cultures and cell lines make it preferable to use primary cultures for assessment of toxicity, yet the problems of using primary cell cultures are considerable as the number of cells available for testing is often small. Recently, we have developed a short-term cell culture assay based on the detection of ATP by the luciferin-luciferase reaction. Four drugs/agents can be tested in triplicate at seven dilutions in one 96-well microplate with 1000 cells/well in the case of cell lines, or 10,000 cells/well for primary tumour tissue. The small number of cells required is a major advantage of this method. Initially developed as a tumour chemosensitivity assay, the assay has shown considerable promise as a general in vitro toxicity assay allowing both cell lines and primary tissue cultures to be tested. Heterogeneity of sensitivity is present in benign tissue biopsies as well as tumours. Molecular alterations within the cell and the interplay of different cell types have been addressed in a number of different model systems using the assay, suggesting that this technology may have more general application.

Comparing and evaluating alternative (in vitro) tests on their ability to predict the Draize maximum average score

The Cosmetic, Toiletry, and Fragrance Association (CTFA) Evaluation of Alternatives Program comprised a multi-phased study of the relationship between Draize eye irritation test data and comparable data from a selection of promising alternative (in vitro) tests. The CTFA Program was designed to determine the effectiveness and limitations of several in vitro tests over a range of different cosmetic and personal-care product types. Test materials constituted experimental formulations representative of three distinct product types. Each material was tested in vivo (according to a modified Draize eye irritation test protocol) and in vitro (according to one of up to forty different protocols). A statistical ranking and selection procedure ("concordance analysis") was used to identify those in vitro tests where the relationships between in vitro and in vivo score was sufficiently well defined to warrant further statistical analysis. In vitro test performance was then evaluated by regression modelling of these relationships. Maximum average Draize score (MAS) was utilized as the primary quantitative measure of eye irritation potential in vivo. The goodness-of-fit of the observed data to the regression model and comparison of the magnitude of upper and lower prediction-bounds on the range of probable MAS values associated with the regression model fit (prediction intervals) provide a means by which the performance of each in vitro test may be measured relative to Draize test outcome. The narrower the prediction interval (i.e. the more precise the fit), the more predictive of in vivo score (MAS) is the in vitro test result. The prediction interval thus represents uncertainty associated with Draize test prediction. Such uncertainty depends heavily on the degree of irritancy. In Phases I and II, the widths of the prediction intervals were narrowest in the region corresponding to low irritation potential; increasing widths were observed as irritation potential increased. In Phase III, relatively narrow prediction interval widths were observed at both the low and high end of the observed range of irritation potential; wider intervals were observed in the middle of the observed range. In general, the selected endpoints in each phase had similar average prediction interval widths and thereby differed only slightly in their ability to predict MAS to a given level of precision; any differences between endpoints tended to occur at the low and/or high ends of the observed range of irritation potential. The primary contributor to total variability associated with prediction of MAS is the deviation between the Draize score as observed in the laboratory and what is predicted by the model for a given formulation. Consistently, this component is responsible for 70% to 95% of the total variability. The other components (i.e. variability among replicate MAS and in vitro scores) could be reduced simply by increasing the number of replicate tests performed on each test formulation. However, this would have relatively little impact on the overall precision of prediction.

FRAME and the validation process

FRAME's historical involvement in the development of the principles of validation, whereby the reliability and relevance of a procedure are established for a specific purpose, and in the practical application of the process, is summarised, and examples of participation in various validation studies on in vitro tests are reviewed. Emphasis is placed on the need for a parallel invalidation process, and on the role of ATLA as a forum for objective reporting and discussion on all aspects of the validation process.

In vitro alternatives to the use of animals in ocular toxicology testing

Chemical substances, including household products, industrial chemicals, and cosmetics, must be tested for ocular toxicity or irritancy so that the public can be assured of their safety or warned of dangers associated with their use. The in vivo Draize test is the standard method used to meet this requirement; however, this test is coming under increasing criticism on scientific and ethical grounds. This has led to the development of a large number of proposed in vitro tests, some of which are routinely used to screen chemicals in toxicology laboratories. This review addresses regulations governing ocular irritancy testing and the current status of the movement toward use of alternative methods. Such methods include the use of cultured cells, hen's eggs, isolated animal eyes and corneas, human corneal epithelial cell lines, and the recently developed in vitro corneal equivalent models. The protocols for these methods are outlined, and their endpoints are described with respect to prediction of in vivo responses. The tests are evaluated in the context of the outcomes of validation studies and acceptance by regulatory agencies. While several of these tests yield useful information concerning ocular irritancy, to date, no in vitro alternative test has been validated as a replacement for the Draize test. If the goal of replacing the in vivo test while protecting the public from chemical eye injury is to be achieved, further development and improvement of alternative tests, as well as establishment of a human ocular toxicity data base, are required.

The local lymph node assay and the assessment of relative potency: status of validation

For the prediction of skin sensitization potential, the local lymph node assay (LLNA) is a fully validated alternative to guinea-pig tests. More recently, information from LLNA dose-response analyses has been used to assess the relative potency of skin sensitizing chemicals. These data are then deployed for risk assessment and risk management. In this commentary, the utility and validity of these relative potency measurements are reviewed. It is concluded that the LLNA does provide a valuable assessment of relative sensitizing potency in the form of the estimated concentration of a chemical required to produce a threefold stimulation of draining lymph node cell proliferation compared with concurrent controls (EC3 value) and that all reasonable validation requirements have been addressed successfully. EC3 measurements are reproducible in both intra- and interlaboratory evaluations and are stable over time. It has been shown also, by several independent groups, that EC3 values correlate closely with data on relative human skin sensitization potency. Consequently, the recommendation made here is that LLNA EC3 measurements should now be regarded as a validated method for the determination of the relative potency of skin sensitizing chemicals, a conclusion that has already been reached by a number of independent expert groups.

The need for a formal invalidation process for animal and non-animal tests

A plethora of regulations require that many chemicals and chemical products are tested for efficacy and/or toxicity. When permitted to operate effectively and without bias, the ECVAM/ICCVAM/OECD validation process can be used to independently establish that new animal and non-animal test procedures are sufficiently relevant and reliable for their stated purposes and should be considered for regulatory use. However, the validation process is under threat because of vested interests of various kinds, and it is clear that many currently-accepted animal tests and candidate animal and non-animal tests do not, and could never, meet the agreed criteria for necessity, test development, prevalidation, validation and acceptance. We therefore need an invalidation process to parallel and protect the validation process, so that such methods could be independently reviewed and declared irrelevant and/or unreliable for their claimed purposes. An additional advantage of such a process would be that valuable resources would no longer be wasted in attempts to secure the acceptance of inherently inadequate tests.

The Evaluation of the Local Tolerance of Vaginal Formulations, With or Without Nonoxynol-9, Using the Slug Mucosal Irritation Test

BACKGROUND

Frequent use of some vaginal formulations can induce mucosal irritation. Therefore, it is important to evaluate their vaginal tolerance.

GOAL

The goal of this study was to optimize the mucosal irritation test using slugs for the evaluation of the local tolerance of vaginal gels and investigate the relevance of the test.

STUDY DESIGN

The irritation potential of the gels was assessed by the amount of mucus produced during a repeated 30-minute contact period. Membrane damage was estimated from the release of proteins and enzymes. After optimization of the procedure, the local tolerance of several vaginal gels was evaluated.

RESULTS

Hydroxyethyl cellulose gel induced no irritation, because the mucus production and the protein release were low and no enzyme release was detected. Replens and K-Y jelly resulted in an increased mucus production; however, no increased protein and no enzyme release were detected. The nonoxynol-9-containing gels Protectaid, Advantage S, and Conceptrol caused a higher mucus production and an increased protein release and/or enzyme release, indicating severe irritation.

CONCLUSION

The mucosal irritation test using slugs can be used for local tolerance testing of vaginal formulations.

Future improvements: replacement in vitro methods

Revolutions in thinking and practice are essential in regulatory toxicology if genuine protection of human beings and the environment is truly to be improved. New test development is the key: Tests should have greater relevance than the current animal procedures based on (1) a mechanistic understanding of the basis of the test method itself and of the toxic phenomenon of concern, (2) taking advantage of new developments in cell and molecular biology and computer systems of various kinds, and (3) a clear understanding of the value of good prediction models. In the not-too-distant future, current research in genomics, proteomics, and metabolomics should provide opportunities for the development of valuable new tests. An inescapable requirement of tests intended to be used for regulatory purposes is validation (i.e., an independent assessment of relevance and reliability for stated purposes according to internationally agreed-upon criteria). However, there is no standard validation scheme; a case-by-case approach is essential. It is important to take advantage of experience, which reveals that prevalidation makes formal validation studies faster, less expensive, and more likely to succeed, and that the procedures for independent assessment used by the European Centre for the Validation of Alternative Methods (ECVAM) and the Interagency Coordinating Committee for the Validation of Alternative Methods (ICCVAM) are effective in practice.

The principles of validation and the ECVAM validation process

The validation of a test method is the process by which the relevance and reliability of the method are assessed for a particular purpose. It is an essential stage in the evolution of the method from its development to its acceptance and application for regulatory purposes. The principles according to which alternative tests should be validated have been agreed at an international level, although the actual process by which the validation process is conducted varies between different validation authorities. This paper summarises the principles of alternative test development and validation, and describes how the principles have been applied to the validation of in vitro tests by ECVAM.

Ocular safety: a silent (in vitro) success story

Ocular irritation testing has been one of the animal test methods most criticised by animal welfare advocates. Additional criticism has arisen from within the scientific community, based on the variability of the animal test results and the questionable relevance of the extremely high dose levels employed. As a result, the Draize eye irritation test has been one of the main targets for in vitro replacement. Despite extensive efforts, however, there is still no in vitro method that is fully validated as a regulatory replacement. In spite of this, many individual companies are using diverse in vitro ocular irritation tests to gain important safety and efficacy information about their products and raw materials, eliminating the need for animal testing in the process. This is done in a safe fashion by applying intelligent testing paradigms. ECVAM has played a major role in this success, through its many programmes that have emphasised the importance of understanding the true toxicological need, and then using in vitro tests to provide that information. Thus, even in the absence of a successfully validated regulatory assay, the desired result of reducing animal testing is being met.

An investigation of new toxicity test method performance in validation studies: 1. Toxicity test methods that have predictive capacity no greater than chance

An approach commonly used to measure new toxicity test method (NTM) performance in validation studies is to divide toxicity results into positive and negative classifications, and the identify true positive (TP), true negative (TN), false positive (FP) and false negative (FN) results. After this step is completed, the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are calculated. Although these statistics are widely used and often the only statistics used to assess the performance of toxicity test methods, there is little specific guidance in the validation literature on what values for these statistics indicate adequate performance. The purpose of this study was to begin developing data-based answers to this question by characterizing the CPS obtained from an NTM whose data have a completely random association with a reference test method (RTM). Determining the CPS of this worst-case scenario is useful because it provides a lower baseline from which the performance of an NTM can be judged in future validation studies. It also provides an indication of relationships in the CPS that help identify random or near-random relationships in the data. The results from this study of randomly associated tests show that the values obtained for the statistics vary significantly depending on the cut-offs chosen, that high values can be obtained for individual statistics, and that the different measures cannot be considered independently when evaluating the performance of an NTM. When the association between results of an NTM and RTM is random the sum of the complementary pairs of statistics (sensitivity + specificity, NPV + PPV) is approximately 1, and the prevalence (i.e., the proportion of toxic chemicals in the population of chemicals) and PPV are equal. Given that combinations of high sensitivity-low specificity or low specificity-high sensitivity (i.e., the sum of the sensitivity and specificity equal to approximately 1) indicate lack of predictive capacity, an NTM having these performance characteristics should be considered no better for predicting toxicity than by chance alone.

An investigation of new toxicity test method performance in validation studies: 3. Sensitivity and specificity are not independent of prevalence or distribution of toxicity

Often, the only measures of toxicity test performance provided in validation studies are the contingent probability statistics (CPS) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity and specificity are generally used in preference to NPV and PPV since NPV and PPV are assumed to vary with changes in prevalence while sensitivity and specificity are assumed to be independent of changes in prevalence. The purpose of the studies reported here was to test whether or not sensitivity and specificity are actually independent of changes in prevalence. Results derived from these studies indicate that sensitivity and specificity vary significantly depending on the prevalence of toxic substances in the set of chemicals being tested. This means sensitivity and specificity should not always be considered constant indicators of toxicity test performance.

An investigation of new toxicity test method performance in validation studies: 2. Comparison of three measures of toxicity test performance

An area that requires further research is how best to measure test method performance in validation studies and how to set criteria that should be used to judge the adequacy of this performance. The studies reported here were designed to begin an investigation of these questions. Computer simulations were used to generate data sets similar to those that might be obtained from a large validation study. These data were then analysed using three procedures including determination of the 95% prediction interval (PI), calculation of Pearson's correlation coefficient and calculation of the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The results of this work suggest that of the three approaches examined, quantitative measurements with calculation of the 95% PI provide the most information to allow discrimination between the performance of several different NTMs. The results also suggest that dividing data sets into positive and negative toxicity classifications followed by the calculation of CPS leads to considerable information loss. This loss of information may be so significant that it is not possible in certain circumstances to distinguish between NTMs that are adequate and those that are not.

The importance of the prediction model in the validation of alternative tests

An overview is presented of the validation process adopted by the European Centre for the Validation of Alternative Methods, with particular emphasis on the central role of the prediction model (PM). The development of an adequate PM is considered to be just as important as the development of an adequate test system, since the validity of an alternative test can only be established when both components (the test system and the PM) have successfully undergone validation. It is argued, however, that alternative tests and their associated PMs do not necessarily need to undergo validation at the same time, and that retrospective validation may be appropriate when a test system is found to be reliable, but the case for its relevance remains to be demonstrated. For an alternative test to be considered "scientifically valid", it is necessary for three conditions to be fulfilled, referred to here as the criteria for scientific relevance, predictive relevance, and reliability. A minimal set of criteria for the acceptance of any PM is defined, but it should be noted that required levels of predictive ability need to be established on a case-by-case basis, taking into account the inherent variability of the alternative and in vivo test data. Finally, in view of the growing shift in emphasis from the use of stand-alone alternative tests to alternative testing strategies, the importance of making the PM an integral part of the testing strategy is discussed.

Evaluation (not validation) of quantitative models

The present regulatory climate has led to increasing demands for scientists to attest to the predictive reliability of numerical simulation models used to help set public policy, a process frequently referred to as model validation. But while model validation may reveal useful information, this paper argues that it is not possible to demonstrate the predictive reliability of any model of a complex natural system in advance of its actual use. All models embed uncertainties, and these uncertainties can and frequently do undermine predictive reliability. In the case of lead in the environment, we may categorize model uncertainties as theoretical, empirical, parametrical, and temporal. Theoretical uncertainties are aspects of the system that are not fully understood, such as the biokinetic pathways of lead metabolism. Empirical uncertainties are aspects of the system that are difficult (or impossible) to measure, such as actual lead ingestion by an individual child. Parametrical uncertainties arise when complexities in the system are simplified to provide manageable model input, such as representing longitudinal lead exposure by cross-sectional measurements. Temporal uncertainties arise from the assumption that systems are stable in time. A model may also be conceptually flawed. The Ptolemaic system of astronomy is a historical example of a model that was empirically adequate but based on a wrong conceptualization. Yet had it been computerized--and had the word then existed--its users would have had every right to call it validated. Thus, rather than talking about strategies for validation, we should be talking about means of evaluation. That is not to say that language alone will solve our problems or that the problems of model evaluation are primarily linguistic. The uncertainties inherent in large, complex models will not go away simply because we change the way we talk about them. But this is precisely the point: calling a model validated does not make it valid. Modelers and policymakers must continue to work toward finding effective ways to evaluate and judge the quality of their models, and to develop appropriate terminology to communicate these judgments to the public whose health and safety may be at stake.

Applications of reconstructed skin models in pharmaco-toxicological trials

The development of new cosmetic formulations requires precise assessment of their safety and efficacy. Today, legislation demands quality control combined with severe safety measures, as well as a limited use of animals for such testing (European Community directive 93/35/EEC). Consequently, safety assessment protocols are oriented towards in vivo tests on human volunteers and in vitro alternative methods to animal use, especially tissue engineered skin substitutes. In this paper, dermal and skin equivalents developed in the laboratory are described. The applications of reconstructed epidermis and skin substitutes for pharmaco-toxicological trials are also discussed. These tissue models have been shown to be very useful tools to assess cutaneous irritation, phototoxicity, photoprotection and to perform efficacy tests of cosmetic molecules and finished products. In conclusion, the authors are confident that these in vitro models can contribute to reduce animal use for routine toxicity testing.

The local lymph node assay: status of validation

For the prediction of skin sensitization potential of substances, the local lymph node assay (LLNA) is an alternative to the widely used guinea pig tests. Over a 10-yr period this method has undergone extensive development, evaluation and validation. In this commentary, the quality of this validation is examined. It is concluded that the LLNA has successfully passed through all reasonable validation stages. It provides a reliable and relevant source of predictive skin sensitization data which, unlike results from guinea pig tests, are reproducible from laboratory to laboratory. Thus, it is now ready for acceptance as a viable and complete alternative to traditional methods, offering substantial opportunities for reduction in animal usage and improved animal welfare without compromising standards for the identification of significant skin sensitizers.