Predicting Rodent Carcinogenicity from Four in vitro Genetic Toxicity Assays: An Evaluation of 114 Chemicals Studied by the National Toxicology Program

Biostatistical Methods for the Validation of Alternative Methods for In Vitro Toxicity Testing

Statistical methods for the validation of toxicological in vitro test assays are developed and applied. Validation is performed either in comparison with in vivo assays or in comparison with other in vitro assays of established validity. Biostatistical methods are presented which are of potential use and benefit for the validation of alternative methods for the risk assessment of chemicals, providing at least an equivalent level of protection through in vitro toxicity testing to that obtained through the use of current in vivo methods. Characteristic indices are developed and determined. Qualitative outcomes are characterised by the rates of false-positive and false-negative predictions, sensitivity and specificity, and predictive values. Quantitative outcomes are characterised by regression coefficients derived from predictive models. The receiver operating characteristics (ROC) technique, applicable when a continuum of cut-off values is considered, is discussed in detail, in relation to its use for statistical modelling and statistical inference. The methods presented are examined for their use for the proof of safety and for toxicity detection and testing. We emphasise that the final validation of toxicity testing is human toxicity, and that the in vivo test itself is only a predictor with an inherent uncertainty. Therefore, the validation of the in vitro test has to account for the vagueness and uncertainty of the "gold standard" in vivo test. We address model selection and model validation, and a four-step scheme is proposed for the conduct of validation studies. Gaps and research needs are formulated to improve the validation of alternative methods for in vitro toxicity testing.

FEMA GRAS assessment of natural flavor complexes: Citrus-derived flavoring ingredients

In 2015, the Expert Panel of the Flavor and Extract Manufacturers Association (FEMA) initiated a re-evaluation of the safety of over 250 natural flavor complexes (NFCs) used as flavoring ingredients. This publication is the first in a series and summarizes the evaluation of 54 Citrus-derived NFCs using the procedure outlined in Smith et al. (2005) and updated in Cohen et al. (2018) to evaluate the safety of naturally-occurring mixtures for their intended use as flavoring ingredients. The procedure relies on a complete chemical characterization of each NFC intended for commerce and organization of each NFC's chemical constituents into well-defined congeneric groups. The safety of the NFC is evaluated using the well-established and conservative threshold of toxicological concern (TTC) concept in addition to data on absorption, metabolism and toxicology of members of the congeneric groups and the NFC under evaluation. As a result of the application of the procedure, 54 natural flavor complexes derived from botanicals of the Citrus genus were affirmed as generally recognized as safe (GRAS) under their conditions of intended use as flavoring ingredients based on an evaluation of each NFC and the constituents and congeneric groups therein.

Combining machine learning models of in vitro and in vivo bioassays improves rat carcinogenicity prediction

In vitro genotoxicity bioassays are cost-efficient methods of assessing potential carcinogens. However, many genotoxicity bioassays are inappropriate for detecting chemicals eliciting non-genotoxic mechanisms, such as tumour promotion, this necessitates the use of in vivo rodent carcinogenicity (IVRC) assays. In silico IVRC modelling could potentially address the low throughput and high cost of this assay. We aimed to develop and combine computational QSAR models of novel bioassays for the prediction of IVRC results and compare with existing software. QSAR models were generated from existing Ames (n = 6512), Syrian Hamster Embryonic (SHE, n = 410), ISSCAN rodent carcinogenicity (ISC, n = 834) and GreenScreen GADD45a-GFP (n = 1415) chemical datasets. These models mapped the molecular descriptors of each compound to their respective assay result using machine learning algorithms (adaboost, k-Nearest Neighbours, C.45 Decision Tree, Multilayer Perceptron, Random Forest). The best performing models were combined with k-Nearest Neighbours to create a cascade model for IVRC prediction. High QSAR model performance was observed from ten time 10-fold cross-validation with above 80% accuracy and 0.85 AUC for each assay dataset. The cascade model predicted rat carcinogenicity with 69.3% accuracy and 0.700 AUC. This study demonstrates the novelty of a combined approach for IVRC prediction, with higher performance than existing software.

RIFM fragrance ingredient safety assessment, isoeugenol, CAS Registry Number 97-54-1

The use of this material under current use conditions is supported by the existing information. This material was evaluated for genotoxicity, repeated dose toxicity, developmental toxicity, reproductive toxicity, local respiratory toxicity, phototoxicity, skin sensitization potential, as well as, environmental safety. Repeated dose toxicity was determined to have the most conservative systemic exposure derived NO[A]EL of 37.5 mg/kg/day. A gavage 13-week subchronic toxicity study conducted in mice resulted in a MOE of 5769 while considering 38.4% absorption from skin contact and 100% from inhalation. A MOE of >100 is deemed acceptable.

Historical perspective on the development of the genetic toxicity test battery in the United States

The currently used genetic toxicity testing battery (the Ames Salmonella test, the in vitro mammalian cell mouse lymphoma assay and/or the in vitro mammalian cell chromosome assay, and the rodent bone marrow chromosome aberration or micronucleus assay) had its origins in the early-to-mid 1970s. By the late 1970s, a large number of genetic tests had been proposed or recommended by the US-EPA for identifying germ cell mutagens and carcinogens. After a number of modifications that were primarily directed toward minimizing the number of tests used, the test battery reached its current state in the mid-1980s. This test battery, with some minor modifications in the timing or ordering of the tests is mandated by regulatory authorities worldwide. Although it would be intellectually satisfying to presume that this compendium of tests was developed and selected for regulatory screening based solely on scientific grounds, it was actually based on a combination of scientific data, theoretical considerations, chance, and advocacy, and not always in equal proportions. The evolution of the current genetic toxicity test battery, and some of the activities and considerations that directed this evolution are described.

Genotoxicity of water from the Songhua River, China, in 1994-1995 and 2002-2003: Potential risks for human health

A previous study showed that the cancer mortalities are higher for residents who lived nearby the Songhua River heavily polluted by organic contamination. It is important to determine its risk of carcinogenic potential. Short-term genotoxic bio-assays using Salmonella, Sister Chromatid Exchange (SCE), and Micronuclei (MN) assays were employed to examine the genotoxic activity of ether extracts of water samples taken from the Songhua River. The results of the Salmonella bioassay indicated that there were indirect frame-shift mutagens in the water samples. A dose-response relationship for the SCE and MN assays was obtained. These results showed that organic extracts of water samples have genotoxic activity and the risk of carcinogenic potential to human health. The mutagenesis of water samples had changed compared to the results in 1994-1995. An increasing trend of risk of carcinogenic potential in the Songhua River after ten years should be noted and needs to be studied further.

Evaluation of the ability of a battery of three in vitro genotoxicity tests to discriminate rodent carcinogens and non-carcinogens I. Sensitivity, specificity and relative predictivity

The performance of a battery of three of the most commonly used in vitro genotoxicity tests--Ames+mouse lymphoma assay (MLA)+in vitro micronucleus (MN) or chromosomal aberrations (CA) test--has been evaluated for its ability to discriminate rodent carcinogens and non-carcinogens, from a large database of over 700 chemicals compiled from the CPDB ("Gold"), NTP, IARC and other publications. We re-evaluated many (113 MLA and 30 CA) previously published genotoxicity results in order to categorise the performance of these assays using the response categories we established. The sensitivity of the three-test battery was high. Of the 553 carcinogens for which there were valid genotoxicity data, 93% of the rodent carcinogens evaluated in at least one assay gave positive results in at least one of the three tests. Combinations of two and three test systems had greater sensitivity than individual tests resulting in sensitivities of around 90% or more, depending on test combination. Only 19 carcinogens (out of 206 tested in all three tests, considering CA and MN as alternatives) gave consistently negative results in a full three-test battery. Most were either carcinogenic via a non-genotoxic mechanism (liver enzyme inducers, peroxisome proliferators, hormonal carcinogens) considered not necessarily relevant for humans, or were extremely weak (presumed) genotoxic carcinogens (e.g. N-nitrosodiphenylamine). Two carcinogens (5-chloro-o-toluidine, 1,1,2,2-tetrachloroethane) may have a genotoxic element to their carcinogenicity and may have been expected to produce positive results somewhere in the battery. We identified 183 chemicals that were non-carcinogenic after testing in both male and female rats and mice. There were genotoxicity data on 177 of these. The specificity of the Ames test was reasonable (73.9%), but all mammalian cell tests had very low specificity (i.e. below 45%), and this declined to extremely low levels in combinations of two and three test systems. When all three tests were performed, 75-95% of non-carcinogens gave positive (i.e. false positive) results in at least one test in the battery. The extremely low specificity highlights the importance of understanding the mechanism by which genotoxicity may be induced (whether it is relevant for the whole animal or human) and using weight of evidence approaches to assess the carcinogenic risk from a positive genotoxicity signal. It also highlights deficiencies in the current prediction from and understanding of such in vitro results for the in vivo situation. It may even signal the need for either a reassessment of the conditions and criteria for positive results (cytotoxicity, solubility, etc.) or the development and use of a completely new set of in vitro tests (e.g. mutation in transgenic cell lines, systems with inherent metabolic activity avoiding the use of S9, measurement of genetic changes in more cancer-relevant genes or hotspots of genes, etc.). It was very difficult to assess the performance of the in vitro MN test, particularly in combination with other assays, because the published database for this assay is relatively small at this time. The specificity values for the in vitro MN assay may improve if data from a larger proportion of the known non-carcinogens becomes available, and a larger published database of results with the MN assay is urgently needed if this test is to be appreciated for regulatory use. However, specificity levels of <50% will still be unacceptable. Despite these issues, by adopting a relative predictivity (RP) measure (ratio of real:false results), it was possible to establish that positive results in all three tests indicate the chemical is greater than three times more likely to be a rodent carcinogen than a non-carcinogen. Likewise, negative results in all three tests indicate the chemical is greater than two times more likely to be a rodent non-carcinogen than a carcinogen. This RP measure is considered a useful tool for industry to assess the likelihood of a chemical possessing carcinogenic potential from batteries of positive or negative results.

An investigation of new toxicity test method performance in validation studies: 1. Toxicity test methods that have predictive capacity no greater than chance

An approach commonly used to measure new toxicity test method (NTM) performance in validation studies is to divide toxicity results into positive and negative classifications, and the identify true positive (TP), true negative (TN), false positive (FP) and false negative (FN) results. After this step is completed, the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are calculated. Although these statistics are widely used and often the only statistics used to assess the performance of toxicity test methods, there is little specific guidance in the validation literature on what values for these statistics indicate adequate performance. The purpose of this study was to begin developing data-based answers to this question by characterizing the CPS obtained from an NTM whose data have a completely random association with a reference test method (RTM). Determining the CPS of this worst-case scenario is useful because it provides a lower baseline from which the performance of an NTM can be judged in future validation studies. It also provides an indication of relationships in the CPS that help identify random or near-random relationships in the data. The results from this study of randomly associated tests show that the values obtained for the statistics vary significantly depending on the cut-offs chosen, that high values can be obtained for individual statistics, and that the different measures cannot be considered independently when evaluating the performance of an NTM. When the association between results of an NTM and RTM is random the sum of the complementary pairs of statistics (sensitivity + specificity, NPV + PPV) is approximately 1, and the prevalence (i.e., the proportion of toxic chemicals in the population of chemicals) and PPV are equal. Given that combinations of high sensitivity-low specificity or low specificity-high sensitivity (i.e., the sum of the sensitivity and specificity equal to approximately 1) indicate lack of predictive capacity, an NTM having these performance characteristics should be considered no better for predicting toxicity than by chance alone.

An investigation of new toxicity test method performance in validation studies: 3. Sensitivity and specificity are not independent of prevalence or distribution of toxicity

Often, the only measures of toxicity test performance provided in validation studies are the contingent probability statistics (CPS) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity and specificity are generally used in preference to NPV and PPV since NPV and PPV are assumed to vary with changes in prevalence while sensitivity and specificity are assumed to be independent of changes in prevalence. The purpose of the studies reported here was to test whether or not sensitivity and specificity are actually independent of changes in prevalence. Results derived from these studies indicate that sensitivity and specificity vary significantly depending on the prevalence of toxic substances in the set of chemicals being tested. This means sensitivity and specificity should not always be considered constant indicators of toxicity test performance.

Predicting rodent carcinogenicity from mutagenic potency measured in the Ames Salmonella assay

Many in vitro tests have been developed to identify chemicals that can damage cellular DNA or cause mutations, and secondarily to identify potential carcinogens. The test receiving by far the most use and attention has been the Salmonella (SAL) mutagenesis test developed by Ames and colleagues [(1973): Proc Natl Acad Sci USA 70:2281-2285; (1975): Mutat Res 31:347-364], because of its initial promise of high qualitative (YES/NO) predictivity for cancer in rodents and, by extension, in humans. In addition to the initial reports of high qualitative predictivity, there was also an early report by Meselson and Russell [in Hiatt HH et al (1977): "Origins of Human Cancer, Book C: Human Risk Assessment," pp 1473-1481] of a quantitative relationship between mutagenic potency measured in SAL and carcinogenic potency measured in rodents, for a small number of chemicals. However, other reports using larger numbers of chemicals have found only very weak correlations. The primary purpose of this study was to determine whether mutagenic potency, as measured in a number of different ways, could be used to improve predictivity of carcinogenicity, either qualitatively or quantitatively. To this end, eight measures of SAL mutagenic potency were used. This study firmly establishes that the predictive relationship between mutagenic potency in SAL and rodent carcinogenicity is, at best, weak. When predicting qualitative carcinogenicity, only qualitative mutagenicity is useful; none of the quantitative measures of potency considered improves the carcinogenicity prediction. In fact, when qualitative mutagenicity is forced out of the model, the quantitative measures are still not predictive of carcinogenicity. When predicting quantitative carcinogenicity, several possible methods were considered for summarizing potency over all experiments; however, in all cases, the relationship between mutagenic potency predictors and quantitative carcinogenicity is very weak.