Proof of Hazard and Proof of Safety in Toxicological Studies Using Simultaneous Confidence Intervals for Differences and Ratios to Control

Using historical control data in bioassays for regulatory toxicology

Historical control data (HCD) consist of pooled control group responses from bioassays. These data must be collected and are often used or reported in regulatory toxicology studies for multiple purposes: as quality assurance for the test system, to help identify toxicological effects and their effect-size relevance and to address the statistical multiple comparison problem. The current manuscript reviews the various classical and potential new approaches for using HCD. Issues in current practice are identified and recommendations for improved use and discussion are provided. Furthermore, stakeholders are invited to discuss whether it is necessary to consider uncertainty when using HCD formally and statistically in toxicological discussions and whether binary inclusion/exclusion criteria for HCD should be revised to a tiered information contribution to assessments. Overall, the critical value of HCD in toxicological bioassays is highlighted when used in a weight-of-evidence assessment.

Expanding the toxicologist's statistical toolbox: Using effect size estimation and dose-response modelling for holistic assessments instead of generic testing

It is tempting to base (eco-)toxicological assay evaluation solely on statistical significance tests. The approach is stringent, objective and facilitates binary decisions. However, tests according to null hypothesis statistical testing (NHST) are thought experiments that rely heavily on assumptions. The generic and unreflected application of statistical tests has been called "mindless" by Gigerenzer. While statistical tests have an appropriate application domain, the present work investigates how unreflected testing may affect toxicological assessments. Dunnett multiple-comparison and Williams trend testing and their compatibility intervals are compared with dose-response-modelling in case studies, where data do not follow textbook behavior, nor behave as expected from a toxicological point of view. In such cases, toxicological assessments based only on p-values may be biased and biological evaluations based on plausibility may be prioritized. If confidence in a negative assay outcome cannot be established, further data may be needed for a robust toxicological assessment.

Alternatives to statistical decision trees in regulatory (eco-)toxicological bioassays

The goal of (eco-) toxicological testing is to experimentally establish a dose or concentration-response and to identify a threshold with a biologically relevant and probably non-random deviation from "normal". Statistical tests aid this process. Most statistical tests have distributional assumptions that need to be satisfied for reliable performance. Therefore, most statistical analyses used in (eco-)toxicological bioassays use subsequent pre- or assumption-tests to identify the most appropriate main test, so-called statistical decision trees. There are however several deficiencies with the approach, based on study design, type of tests used and subsequent statistical testing in general. When multiple comparisons are used to identify a non-random change against negative control, we propose to use robust testing, which can be generically applied without the need of decision trees. Visualization techniques and reference ranges also offer advantages over the current pre-testing approaches. We aim to promulgate the concepts in the (eco-) toxicological community and initiate a discussion for regulatory acceptance.

Comparison of false-positive rates of 2 hypothesis-test approaches in relation to laboratory toxicity test performance

We compared 2 statistical hypothesis-test approaches (no-observed-effect concentration [NOEC] and test of significant toxicity [TST]) to determine the influence of laboratory test performance on the false-positive error rate using the US Environmental Protection Agency's Ceriodaphnia dubia reproduction whole-effluent toxicity (WET) test endpoint. Simulation and power calculations were used to determine error rates based on observed control coefficients of variation (CV) for 8 laboratories over a range of effect levels. Average C. dubia control reproduction among laboratories was 20 to 40 offspring per female, and the 75th percentile CV was 0.10 to 0.31, reflecting a range in laboratory performance. The 2 approaches behave similarly for CVs of 0.2 to 0.3. At effects <10%, as CV decreases, TST is less likely to declare toxicity and NOEC is more likely to do so. Laboratory performance affects whether a sample is declared toxic and influences the probability of false-positive (and -negative) error rates using either approach. At the 75th percentile control CV observed for each laboratory, 4 laboratories would achieve approximately a 5% false-positive rate using 13 or fewer replicates for this test method. For the remaining 4 laboratories, more replicates would be needed to achieve a 5% false-positive rate. The present analyses demonstrate how false-positive rates are influenced by laboratory performance and WET test design. Environ Toxicol Chem 2019;38:511-523. Published 2019 Wiley Periodicals Inc. on behalf of SETAC. This article is a US government work and, as such, is in the public domain in the United States of America.

Carcinogenicity in rats of the SGLT2 inhibitor canagliflozin

The carcinogenicity potential of canagliflozin, an inhibitor of SGLT2, was evaluated in a 2-year rat study (10, 30, and 100 mg/kg). Rats showed an increase in pheochromocytomas, renal tubular tumors, and testicular Leydig cell tumors. Systemic exposure multiples at the highest dose relative to the maximum clinical dose were 12- to 21-fold. Pheochromocytomas and renal tubular tumors were noted in both sexes at 100 mg/kg. Leydig cell tumors were observed in males in all dose groups and were associated with increased luteinizing hormone levels. Hyperplasia was increased in the adrenal medulla at 100 mg/kg, but only a limited increase in simple tubular hyperplasia was observed in the kidney of males at 100 mg/kg. Hyperostosis occurred and was accompanied by substantial effects on calcium metabolism, including increased urinary calcium excretion and decreased levels of calcium regulating hormones (1,25-dihydroxyvitamin D and parathyroid hormone). A separate study with radiolabeled calcium confirmed that increased urinary calcium excretion was mediated via increased calcium absorption from the gastrointestinal tract. It was hypothesized that, at high doses, canagliflozin might have inhibited glucose absorption in the intestine via SGLT1 inhibition that resulted in glucose malabsorption, which increased calcium absorption by stimulating colonic glucose fermentation and reducing intestinal pH. Pheochromocytomas and adrenal medullary hyperplasia were attributed to altered calcium homeostasis, which have a known relationship in the rat. In conclusion, Leydig cell tumors were associated with increased luteinizing hormone levels and pheochromocytomas were most likely related to glucose malabsorption and altered calcium homeostasis. Renal tubular tumors may also have been linked to glucose malabsorption.

Statistical evaluation of toxicological assays: Dunnett or Williams test-take both

The US National Toxicology Program recommends the use of the parametric multiple comparison procedures of Dunnett and Williams for the evaluation of repeated toxicity studies. For endpoints where either increasing or decreasing effects are of toxicological relevance, we recommend the use of the two-sided Dunnett test exclusively. For the many other endpoints, where a priori only one direction is of toxicological relevance, however, we recommend the combination of Dunnett and Williams test. In particular, we recommend the so-called Umbrella-protected Williams test which offers insights for all interesting monotone and non-monotone alternatives while only suffering a marginal loss in power compared to the Dunnett test. We illustrate the power difference analytically and compare the approach for different endpoint types using three real data examples to alternative tests available. Nonparametric tests, which are suitable for the evaluation of skewed distributed or scores data, are also considered. Particular attention is given to the different interpretations of the findings revealed by the different test. R programs used for the analyses are provided.

Simultaneous confidence intervals on multivariate non-inferiority

We discuss the analysis of non-inferiority trials based on a multi-armed design with multiple correlated endpoints. We consider five different scenarios: (1) global non-inferiority; (2) non-inferiority for subsets of comparisons; (3) global non-inferiority for a treatment group; (4) global non-inferiority for an endpoint; and (5) local non-inferiority. We describe and compare related approaches, which are based on the union-intersection and intersection-union test principles - alone and in combination. Because non-inferiority thresholds for many differently scaled endpoints are rarely available in practice, the approaches described focus on the estimation of simultaneous confidence limits and their post hoc interpretation on non-inferiority. We discuss and demonstrate pros and cons by means of a real data example.