Use of statistical decision rules for evaluating laboratory animal carcinogenicity studies

Proper interpretation of chronic toxicity studies and their statistics: A critique of "Which level of evidence does the US National Toxicology Program provide? Statistical considerations using the Technical Report 578 on Ginkgo biloba as an example"

A recent article by Gaus (2014) demonstrates a serious misunderstanding of the NTP's statistical analysis and interpretation of rodent carcinogenicity data as reported in Technical Report 578 (Ginkgo biloba) (NTP, 2013), as well as a failure to acknowledge the abundant literature on false positive rates in rodent carcinogenicity studies. The NTP reported Ginkgo biloba extract to be carcinogenic in mice and rats. Gaus claims that, in this study, 4800 statistical comparisons were possible, and that 209 of them were statistically significant (p<0.05) compared with 240 (4800×0.05) expected by chance alone; thus, the carcinogenicity of Ginkgo biloba extract cannot be definitively established. However, his assumptions and calculations are flawed since he incorrectly assumes that the NTP uses no correction for multiple comparisons, and that significance tests for discrete data operate at exactly the nominal level. He also misrepresents the NTP's decision making process, overstates the number of statistical comparisons made, and ignores the fact that the mouse liver tumor effects were so striking (e.g., p<0.0000000000001) that it is virtually impossible that they could be false positive outcomes. Gaus' conclusion that such obvious responses merely "generate a hypothesis" rather than demonstrate a real carcinogenic effect has no scientific credibility. Moreover, his claims regarding the high frequency of false positive outcomes in carcinogenicity studies are misleading because of his methodological misconceptions and errors.

Analysis of chloroethane toxicity and carcinogenicity including a comparison with bromoethane

Chloroethane (CE) gas carcinogenicity is analyzed and determined from a National Toxicology Program (NTP) bioassay where an inhalation concentration of 15,000 ppm CE gas in air produced the highest incidence of an uncommon-to-rare tumor ever observed by the NTP. Persistently inhaled CE produces endometrial cancers in female mice. The first-tumor-corrected uterine endometrial incidence (I) in B6C3F1 mice is 90%, but no significant tumors occurred in F344 rats. The endometrial cancers dispersed by 1) migrating locally to the adjacent myometrium, 2) then migrating to the bloodstream by intravasation, 3) entering 17 distal organs by extravasation and adapting to the new tissue environment. Distal cancers retained sufficient endometrial cell features to be recognized at each metastatic site. CE produced one of the highest metastasis rates ever observed by NTP of 79%. Comparing CE with bromoethane (BE), a structural analogue, it was found that BE too produced rare murine endometrial cancers yielding the second highest NTP incidence rate of I = 58% with a similar high malignancy rate of 56%. Because of the historical rarity of endometrial tumors in the B6C3F1 mouse, both of these SAR haloethanes seem to be evoking a strong, related carcinogenic potential in B6C3F1 mice, but not in F344 rats. The question of whether humans are similar to mice or to rats is addressed here and in Gargas, et al., 2008. The powerful carcinogenesis caused by these halohydrocarbons may have been caused by excessive and metabolically unresolved acetaldehyde (AC) which is directly generated by Cyp2E1 in the oxidative elimination of CE. With >95% AC metabolic production, as predicted from pharmacokinetic (PK) studies depending on CE exposure, AC is the main elimination intermediate. AC is a known animal carcinogen and a strongly suspected human carcinogen. Also, CE causes incipient decreases of tissue essential glutathione pools [GSH] by Phase II conjugation metabolic elimination of CE (and BE), by glutantione transferase (GST), in most organs (except brain) exposed to high circulating CE and it metabolites. In three laboratories, an excessive stress reaction of hyperkinesis was observed only during 15,000 ppm gas exposure but not when the exposure ceased or when exposure was presented at 150 ppm. Test rodents other than the female mice did not exhibit a pattern of visible stress nor did they have a carcinogenic response to CE gas. Unremitting stress has been documented to contribute a feedback to the hypothalamus which stimulates the hypothalamic-pituitary-axis (HPA), which in turn, induces the adrenal glands. Because estrus and estrogen and progesterone levels were unaltered by CE gas, the adrenal over stimulation, causing high steroid output, may be the penultimate step in this extraordinary carcinogenic response. High adrenal production of corticosteroids could adversely promote endometrial cells to cancers in mice − a mechanism that has already been observed in humans.

On use of the multistage dose-response model for assessing laboratory animal carcinogenicity

We explore how well a statistical multistage model describes dose-response patterns in laboratory animal carcinogenicity experiments from a large database of quantal response data. The data are collected from the US EPA's publicly available IRIS data warehouse and examined statistically to determine how often higher-order values in the multistage predictor yield significant improvements in explanatory power over lower-order values. Our results suggest that the addition of a second-order parameter to the model only improves the fit about 20% of the time, while adding even higher-order terms apparently does not contribute to the fit at all, at least with the study designs we captured in the IRIS database. Also included is an examination of statistical tests for assessing significance of higher-order terms in a multistage dose-response model. It is noted that bootstrap testing methodology appears to offer greater stability for performing the hypothesis tests than a more-common, but possibly unstable, "Wald" test.

Unexpected tumour findings in lifetime rodent bioassay studies--what to do?

Currently, the majority of substances tested in lifetime bioassays in rodents are not mutagenic and, therefore, at the most weakly carcinogenic, generally by epigenetic mechanisms. It thus appears obvious that only marginal increases of tumour incidences can be expected in lifetime bioassays and that, therefore, every aspect of a potential carcinogenic effect must be thoroughly evaluated. This paper describes a series of key factors, which should be looked at in order to exclude that the lifetime bioassay in question is flawed for design, technical or qualification reasons. It also provides some hints whether there is indeed a real effect and not just a variation of the spontaneous tumour incidences. Tumour findings must be seen in the context of the animal model, the pharmcokinetics and pharmcodynamics of the test substance, as well as any other observation in the present or other studies with the test substance, including non-tumour findings and--in particular--potential precursor lesions and effects on feed intake and survival. The possibility that the observed carcinogenic effects may be species-specific and not relevant for man is discussed. It is also important to check what findings are reported with similar substances or substances with the same pharmacological effect. Data from additional investigations on material of the same study and/or mechanistic studies are often needed to support the final risk assessment.

Using the NTP database to assess the value of rodent carcinogenicity studies for determining human cancer risk

The large database of carcinogenicity results generated by the National Toxicology Program (NTP) provides a unique opportunity to critically evaluate important scientific issues such as (1) the frequency of positive outcomes, (2) the interspecies correlation in carcinogenic response between rats and mice, (3) the correlation between body weight and tumor incidence, (4) estimates of the false-positive and false-negative rates, and (5) the frequency of decreasing tumor incidences. Such database evaluations enable us to better understand the value and limitations of rodent carcinogenicity studies for determining human cancer risk. However, as the NTP database becomes increasingly accessible to the general scientific community, there is also increased opportunity for misuse of the database. This article reexamines and updates previous database evaluations, presents four scientific principles that should be employed by anyone attempting to use this database, and illustrates how failure to apply these principles can lead to misleading results.

Short- and intermediate-term carcinogenicity testing--a review. Part 1: the prototypes mouse skin tumour assay and rat liver focus assay

Carcinogenicity testing is by far the most expensive and time-consuming study type of toxicology. For many years, the lifetime exposure with the maximum tolerated dose in two rodent species has been the gold standard of carcinogenicity testing of pharmaceuticals. Major change was introduced by the Fourth International Conference on Harmonization in July 1997; a chronic rodent bioassay in one species and a short-term carcinogenicity assay are regarded as sufficient for registration. Such requirements provide the opportunity to redirect the vast resources previously spent on the lifetime study in the second species. Numerous experimental protocols for short- and intermediate-term carcinogenicity testing in many target tissues have been available for years. The first part of this review describes the basic principles of short- and intermediate-term carcinogenicity testing using the examples of the widely used mouse skin tumour assay and the rat liver foci assay. In the context of these experimental models, the discrimination and quantification of initiating and promoting activity and the use of preneoplastic lesions as endpoints in carcinogenicity testing are described. The review includes the limitations of the models with regard to the extrapolation from effects observed in animal experiments to a potential exposure of humans.

Estimates of the number of liver carcinogens in bioassays conducted by the National Toxicology Program

Estimates were made of the numbers of liver carcinogens in 390 long-term bioassays conducted by the National Toxicology Program (NTP). These estimates were obtained from examination of the global pattern of p-values obtained from statistical tests applied to individual bioassays. Representative estimates of the number of liver carcinogens (90% confidence interval in parentheses) obtained in our analysis compared to NTP's determination are as follows: female rats--49 (23, 76), NTP = 30; male rats--88 (59, 116), NTP = 35; female mice--131 (105, 157), NTP = 81; male mice--100 (73, 126), NTP = 61; overall--166 (135, 197), NTP = 108. The estimator from which these estimates were obtained is biased low by an unknown amount. Consequently, this study provides persuasive evidence of the existence of more rodent liver carcinogens than were identified by the NTP.

Overall false positive rates in tests for linear trend in tumor incidence in animal carcinogenicity studies of new drugs

Based on results of simulation and empirical studies conducted within the Divisions of Biometrics, Center for Drug Evaluation and Research, Food and Drug Administration, and in collaboration with the National Toxicology Program, the Center has recently changed the significance levels for testing positive linear trend in incidence rate for common and rare tumors, respectively, from 0.01 and 0.05 to 0.005 and 0.025. The overall false positive rate resulting from the use of this new rule in the tests for linear trend in a two-species-two-sex study is about 10%, the rate that is judged as the most appropriate in a regulatory setting by the Center. This paper describes two of the studies.

Mesenchymal tumors of the mouse urinary bladder with vascular and smooth muscle differentiation

Bifenthrin, a synthetic pyrethroid insecticide/miticide, has been fed to male and female Swiss Webster mice at levels of 0, 50, 200, 500, and 600 ppm in the diet for between 604 and 644 days. Tumors of the urinary bladder were observed and initially reported as leiomyosarcomas. Subsequently, the bladders were reviewed and the tumors showed a pattern of both epithelioid cells and spindle cells forming irregular vascular channels. The tumors appeared to arise from the trigone of the bladder and, in some cases, invaded the bladder wall. No metastases were recorded. The tumor is usually considered rare; however, in this study, it was commonly observed in all groups but predominantly in males. The histogenesis of the tumor is uncertain, but from its pleomorphic histological features, including smooth muscle and vascularity, it is probably derived from vascular mesenchyme.

Sources of variability in data from a positive selection lacZ transgenic mouse mutation assay: an interlaboratory study

Experimental features of a positive selection transgenic mouse mutation assay based on a lambda lacZ transgene are considered in detail, with emphasis on results using germ cells as the target tissue. Sources of variability in the experimental protocol that can affect the statistical nature of the observations are examined, with the goal of identifying sources of excess variation in the observed mutant frequencies. The sources include plate-to-plate (within packages), package-to-package (within animals), and animal-to-animal variability. Data from five laboratories are evaluated in detail. Results suggest only scattered patterns of excess variability below the animal-to-animal level, but, generally, significant excess variability at the animal-to-animal level. Using source of variability analyses to guide the choice of statistical methods, control-vs-treatment comparisons are performed for assessing the male germ cell mutagenicity of ethylnitrosourea (ENU), isopropyl methanesulfonate (iPMS), and methyl methanesulfonate (MMS). Results on male germ cell mutagenesis of ethyl methanesulfonate (EMS) and methylnitrosourea (MNU) are also reported.

The utility of two rodent species in carcinogenic risk assessment of pharmaceuticals in Europe

For the past 20-30 years, lifespan carcinogenicity studies for pharmaceuticals have been required to be carried out in two rodent species. Due to scientific progress, the necessity/justification of lifespan studies in two species for the assessment of carcinogenic risk of pharmaceuticals is currently under discussion. A study in one species (either rat or mouse) might suffice. To appraise the need for a study in a second species, a database was compiled of all pharmaceuticals tested for carcinogenicity for which a marketing authorization was applied for in Germany and The Netherlands since 1980. The incidence of treatment-related tumor findings was determined in either rat or mouse or in both. Tumor findings occurred for nearly 50% of all compounds, with the rat being more sensitive than the mouse. Specific attention was given to the question whether tumor findings in mice ever caused the regulatory authorities to refuse registration, to restrict the proposed therapeutic indication of a pharmaceutical, or to apply a cautionary label. It was found that no tumor findings in mice alone ever led to such a regulatory action. In addition, whether mouse studies had been important in interpreting the results of rat studies was determined. A negative mouse study (no tumors found) was rarely used to declare the rat findings irrelevant to humans. A mechanistic explanation was used as a much more important argument in the assessment of tumor findings in rats. In case of transspecies findings, the target organs were the usual ones, such as lung and liver, or the tumors occurred as a result of an exaggerated pharmacodynamic action expected from the pharmacology of the compound. The results of the database thus question the need of maintaining the requirement of rodent carcinogenicity studies in two species.

Evaluation of false positive and false negative outcomes in NTP long-term rodent carcinogenicity studies

The decision-making process used by the National Toxicology Program (NTP) in its evaluation of long-term rodent carcinogenicity studies was investigated to determine whether or not this procedure resulted in an excessive number of false positive or false negative outcomes. All site-specific tumor incidences that were found to be significantly (p < 0.05) increased either by a trend test or by pairwise comparisons of each dosed group against the controls in 218 NTP 2-year studies with Fischer 344 rats and/or B6C3F1 mice were tabulated and compared to the number of statistically significant tumor increases expected to occur by chance. Our evaluation suggests that false positive rates are fairly low in NTP long-term studies. Assessing false negative rates is more difficult because of the limited sensitivity of the bioassay for detecting subtle carcinogenic effects. Moreover, reduced body weights frequently occur in dosed animals, and the positive correlation between the incidences of certain site-specific tumors and body weight may mask the detection of carcinogenic effects. Despite these difficulties, our analysis did identify one tumor showing evidence of false negative outcomes: interstitial cell tumors of the testis in male Fischer 344 (F344) rats. This tumor showed considerably more significant (p < 0.05) increased incidences than expected by chance, yet none were considered to be chemically-related. However, the biological significance of interstitial cell tumor increases in F344 rats is uncertain because of the high background rate of neoplasia (> 90%) for this target site.

Estimating multiple tumor transition rates based on data from survival-sacrifice experiments

The development of two types of tumors in an animal survival-sacrifice experiment can be described by a Markov illness-and-death process with four unknown tumor transition rates and four unknown death rates. Due to the occult nature of tumors, these transition rates cannot be identified through the use of a completely nonparametric model. We propose a semiparametric model that assumes that the four tumor transition rates are proportional to each other and can be described by a known parametric function. We show that these tumor transition rates can be estimated using an ECM algorithm and that inferences about these parameters can be drawn using the likelihood ratio test. We illustrate this model and the algorithm with data from the ED01 study.

Fat, frail and dying young: survival, body weight and pathology of the Charles River Sprague-Dawley-derived rat prior to and since the introduction of the VAFR variant in 1988

Trends in survival and body weight were evaluated in 2140 control Sprague-Dawley-derived [Crl: COBS-CD(SD)BR and Crl: COBS-VAF CD(SD)BR] rats used for 24-month rat carcinogenicity studies between 1979 and 1991. Body weight and survival were remarkably stable in the CD-COBS rats used during 1979-1987: at 24 months, the mean survival in males was 68 +/- 5%, and 60 +/- 5% in females. With the CD-COBS-VAF rat, a variant of the CD-COBS strain used between 1988 and 1991, the survival at 24 months dropped to 41 +/- 3% in males, and 44 +/- 7% in females compared to the CD-COBS. The CD-COBS-VAF rat had a significantly reduced life span (P < 0.001 at 24 months), a significant increase in mean body weight (males at 6 months: 672 +/- 24 g vs. 536 +/- 6 g; females: 359 +/- 7 g vs 308 +/- 3 g; P < 0.001) and food consumption (males at 6 months: 31.3 +/- 3.3 vs. 25.4 +/- 2.1 g d-1; females: 22.0 +/- 2.7 g v. 20.3 +/- 2.0 g d-1; P < 0.001). CD-COBS-VAF rats which failed to survive up to study termination had individual body weights at 3, 6 and 12 months which were significantly higher (P < 0.001) than those which survived until 24 months. Our historical data base of control rats (CD-COBS and CD-COBS-VAF) in carcinogenicity studies revealed a significant (males: P < 0.001); females: P < 0.01) and inverse linear relation between mean 3-month body weight and 24-month survival. When compared to CD-COBS animals, CD-COBS-VAF rats showed an increase in the incidence of pituitary tumours in males, mammary fibroadenomas in females, an increase in the incidence of severity of glomerulonephrosis, and a greater incidence of animals which died without any obvious pathology. It is concluded that, in our Sprague-Dawley substrains, both the individual and the group mean body weights in early adult life appear predictive for the individual and group life expectancy. The decrease in longevity in the CD-COBS-VAF rat is principally due to disease and degeneration processes associated with fast growth and high body weight.

Pathology reevaluation of the Kociba et al. (1978) bioassay of 2,3,7,8-TCDD: implications for risk assessment

The chronic bioassay of 2,3,7,8-tetrachlorodibenzo-p-dioxin (2,3,7,8-TCDD) reported in 1978 by Kociba et al. has been considered to be the primary evidence supporting its carcinogenicity, and is the basis for most dioxin regulations in North America and Western Europe. Because the histopathological criteria for proliferative lesions in the rat liver have changed significantly since 1978, a reevaluation of the liver slides was conducted recently by an independent panel of pathologists. Using current National Toxicology Program criteria, their study showed, in contrast to the original findings, that about two-thirds fewer tumors were present in the livers of female Sprague-Dawley rats. The no-observed-adverse-effect level (NOAEL) for hepatocellular carcinomas was 0.01 micrograms/kg/d rather than 0.001 micrograms/kg/d, which had been reported in 1978. In light of these significant findings, a quantitative dose-response assessment of 2,3,7,8-TCDD was undertaken to predict the potential carcinogenic risks to humans. Risk-specific doses (RsDs) and cancer potency factors (CPFs) were calculated by applying the linearized multistage (LMS) model to the combined incidences of hepatocellular carcinomas and adenomas, classified in accordance with the 1990 histopathological criteria. Based on the weight of evidence regarding the mechanism of action of 2,3,7,8-TCDD, body weight rather than surface area was selected as the appropriate means for scaling rodent data to predict the human response. Using the survival-adjusted data, the RsD for a 1 in 1,000,000 (10(-6)) plausible upper bound (95%) lifetime incremental cancer risk was 370 fg/kg/d based only on the incidence of hepatocellular carcinomas, and 100 fg/kg/d when hepatocellular carcinomas and adenomas were combined. The corresponding upper-bound (95%) CPFs were 2700 and 9700 (mg/kg/d)-1, respectively. These results indicate that the carcinogenic risk to humans from exposure to 2,3,7,8-TCDD is at least 16-fold lower than previous estimates derived from the Kociba et al. (1978) bioassay.