A studentized permutation test for three-arm trials in the ‘gold standard’ design

On the role of benchmarking data sets and simulations in method comparison studies

Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral but favor a novel method. Apart from the choice of design and a proper reporting of the findings, there are different approaches concerning the underlying data for such method comparison studies. Most manuscripts on statistical methodology rely on simulation studies and provide a single real-world data set as an example to motivate and illustrate the methodology investigated. In the context of supervised learning, in contrast, methods are often evaluated using so-called benchmarking data sets, that is, real-world data that serve as gold standard in the community. Simulation studies, on the other hand, are much less common in this context. The aim of this paper is to investigate differences and similarities between these approaches, to discuss their advantages and disadvantages, and ultimately to develop new approaches to the evaluation of methods picking the best of both worlds. To this aim, we borrow ideas from different contexts such as mixed methods research and Clinical Scenario Evaluation.

Optimization of the two-stage group sequential three-arm gold-standard design for non-inferiority trials

If design parameters are chosen appropriately, group sequential trial designs are known to be able to reduce the expected sample size under the alternative hypothesis compared to single-stage designs. The same holds true for the so-called 'gold-standard' design for non-inferiority trials, a design involving an experimental group, an active control group, and a placebo group. However, choosing design parameters that maximize the advantages of a two-stage approach for the three-arm gold-standard design for non-inferiority trials is not a straightforward task. In particular, optimal choices of futility boundaries for this design have not been thoroughly discussed in existing literature. We present a variation of the hierarchical testing procedure, which allows for the incorporation of binding futility boundaries at interim analyses. We show that this procedure maintains strong control of the family-wise type I error rate. Within this framework, we consider the futility and efficacy boundaries as well as the sample size allocation ratios as optimization parameters. This allows the investigation of the efficiency gain from including the option to stop for futility in addition to the ability to stop for efficacy. To analyze the extended designs, optimality criteria that include the design's performance under the alternative as well as the null hypothesis are introduced. On top of this, we discuss methods to limit the allocation of placebo patients in the trial while maintaining relatively good operating characteristics. The results of our numerical optimization procedure are discussed and a comparison of different approaches to designing a three-arm gold-standard non-inferiority trial is provided.

Single-stage, three-arm, adaptive test strategies for non-inferiority trials with an unstable reference

For indications where only unstable reference treatments are available and use of placebo is ethically justified, three-arm "gold standard" designs with an experimental, reference and placebo arm are recommended for non-inferiority trials. In such designs, the demonstration of efficacy of the reference or experimental treatment is a requirement. They have the disadvantage that only little can be concluded from the trial if the reference fails to be efficacious. To overcome this, we investigate novel single-stage, adaptive test strategies where non-inferiority is tested only if the reference shows sufficient efficacy and otherwise δ $$ \delta $$ -superiority of the experimental treatment over placebo is tested. With a properly chosen superiority margin, δ $$ \delta $$ -superiority indirectly shows non-inferiority. We optimize the sample size for several decision rules and find that the natural, data driven test strategy, which tests non-inferiority if the reference's efficacy test is significant, leads to the smallest overall and placebo sample sizes. We proof that under specific constraints on the sample sizes, this procedure controls the family-wise error rate. All optimal sample sizes are found to meet this constraint. We finally show how to account for a relevant placebo drop-out rate in an efficient way and apply the new test strategy to a real life data set.

Bayesian sample size determination in a three-arm non-inferiority trial with binary endpoints

A three-arm non-inferiority trial including a test treatment, a reference treatment, and a placebo is recommended to assess the assay sensitivity and internal validity of a trial when applicable. Existing methods for designing and analyzing three-arm trials with binary endpoints are mainly developed from a frequentist viewpoint. However, these methods largely depend on large sample theories. To alleviate this problem, we propose two fully Bayesian approaches, the posterior variance approach and Bayes factor approach, to determine sample size required in a three-arm non-inferiority trial with binary endpoints. Simulation studies are conducted to investigate the performance of the proposed Bayesian methods. An example is illustrated by the proposed methodologies. Bayes factor method always leads to smaller sample sizes than the posterior variance method, utilizing the historical data can reduce the required sample size, simultaneous test requires more sample size to achieve the desired power than the non-inferiority test, the selection of the hyperparameters has a relatively large effect on the required sample size. When only controlling the posterior variance, the posterior variance criterion is a simple and effective option for obtaining a rough outcome. When conducting a previous clinical trial, it is recommended to use the Bayes factor criterion in practical applications.

Confidence Intervals for Assessing Non-Inferiority with Assay Sensitivity in a Three-Arm Trial with Normally Distributed Endpoints

Various approaches including hypothesis test and confidence interval (CI) construction have been proposed to assess non-inferiority and assay sensitivity via a known fraction or pre-specified margin in three-arm trials with continuous or discrete endpoints. However, there is little work done on the construction of the non-inferiority margin from historical data and simultaneous generalized CIs (SGCIs) in a three-arm trial with the normally distributed endpoints. Based on the generalized fiducial method and the square-and-add method, we propose two simultaneous CIs for assessing non-inferiority and assay sensitivity in a three-arm trial. For comparison, we also consider the Wald-type Bonferroni simultaneous CI and parametric bootstrap simultaneous CI. An algorithm for evaluating the optimal sample size for attaining the pre-specified power is given. Simulation studies are conducted to investigate the performance of the proposed CIs in terms of their empirical coverage probabilities. An example taken from the mildly asthmatic study is illustrated using the proposed simultaneous CIs. Empirical results show that the proposed generalized fiducial method and the square-and-add method behave better than other two compared CIs.

Simultaneous confidence interval for assessing non-inferiority with assay sensitivity in a three-arm trial with binary endpoints

A three-arm trial including an experimental treatment, an active reference treatment and a placebo is often used to assess the non-inferiority (NI) with assay sensitivity of an experimental treatment. Various hypothesis-test-based approaches via a fraction or pre-specified margin have been proposed to assess the NI with assay sensitivity in a three-arm trial. There is little work done on confidence interval in a three-arm trial. This paper develops a hybrid approach to construct simultaneous confidence interval for assessing NI and assay sensitivity in a three-arm trial. For comparison, we present normal-approximation-based and bootstrap-resampling-based simultaneous confidence intervals. Simulation studies evidence that the hybrid approach with the Wilson score statistic performs better than other approaches in terms of empirical coverage probability and mesial-non-coverage probability. An example is used to illustrate the proposed approaches.

The association of TNF-α -308G/A and -238G/A polymorphisms with type 2 diabetes mellitus: a meta-analysis

Tumor necrosis factor-α (TNF-α) is involved in insulin resistance and has long been a candidate gene implicated in type 2 diabetes mellitus (T2DM), however the association between TNF-α polymorphisms -308G/A and -238G/A and T2DM remains controversial. The present study sought to verify associations between these polymorphisms and T2DM susceptibility using a meta-analysis approach. A total of 49 case-control studies were selected up to October 2018. Statistical analyses were performed by STATA 15.0 software. The odds ratios (ORs) and 95% confidence intervals were calculated to estimate associations. Meta-analyses revealed significant associations between TNF-α -308G/A and T2DM in the allele model (P=0.000); the dominant model (P=0.000); the recessive model (P=0.001); the overdominant model (P=0.008) and the codominant model (P=0.000). Subgroup analyses also showed associations in the allele model (P=0.006); the dominant model (P=0.004) and the overdominant model (P=0.005) in the Caucasian and in the allele model (P=0.007); the dominant model (P=0.014); the recessive model (P=0.000) and the codominant model (P=0.000) in the Asian. There were no associations between TNF-α -238G/A and T2DM in the overall and subgroup populations. Meta-regression, sensitivity analysis and publication bias analysis confirmed that results and data were statistically robust. Our meta-analysis suggests that TNF-α -308G/A is a risk factor for T2DM in Caucasian and Asian populations. It also indicates that TNF-α -238G/A may not be a risk factor for T2DM. More comprehensive studies will be required to confirm these associations.

Sequential parallel comparison design for "gold standard" noninferiority trials with a prespecified margin

Three-arm noninferiority trials (involving an experimental treatment, a reference treatment, and a placebo)-called the "gold standard" noninferiority trials-are conducted in patients with mental disorders whenever feasible, but often fail to show superiority of the experimental treatment and/or the reference treatment over the placebo. One possible reason is that some of the patients receiving the placebo show apparent improvement in the clinical condition. An approach to addressing this problem is the use of the sequential parallel comparison design (SPCD). Nonetheless, the SPCD has not yet been discussed in relation to gold standard noninferiority trials. In this article, our aim was to develop a hypothesis-testing method and its corresponding sample size calculation method for gold standard noninferiority trials with the SPCD. In a simulation, we show that the proposed hypothesis-testing method achieves the nominal type I error rate and power and that the proposed sample size calculation method has adequate power accuracy.

NONPARAMETRIC TESTING FOR MULTIPLE SURVIVAL FUNCTIONS WITH NON-INFERIORITY MARGINS

New nonparametric tests for the ordering of multiple survival functions are developed with the possibility of right censorship taken into account. The motivation comes from non-inferiority trials with multiple treatments. The proposed tests are based on nonparametric likelihood ratio statistics, which are known to provide more powerful tests than Wald-type procedures, but in this setting have only been studied for pairs of survival functions or in the absence of censoring. We introduce a novel type of pool adjacent violator algorithm that leads to a complete solution of the problem. The limit distributions can be expressed as weighted sums of squares involving projections of certain Gaussian processes onto the given ordered alternative. A simulation study shows that the new procedures have superior power to a competing combined-pairwise Cox model approach. We illustrate the proposed methods using data from a three-arm non-inferiority trial.

Group sequential designs for negative binomial outcomes

Count data and recurrent events in clinical trials, such as the number of lesions in magnetic resonance imaging in multiple sclerosis, the number of relapses in multiple sclerosis, the number of hospitalizations in heart failure, and the number of exacerbations in asthma or in chronic obstructive pulmonary disease (COPD) are often modeled by negative binomial distributions. In this manuscript, we study planning and analyzing clinical trials with group sequential designs for negative binomial outcomes. We propose a group sequential testing procedure for negative binomial outcomes based on Wald statistics using maximum likelihood estimators. The asymptotic distribution of the proposed group sequential test statistics is derived. The finite sample size properties of the proposed group sequential test for negative binomial outcomes and the methods for planning the respective clinical trials are assessed in a simulation study. The simulation scenarios are motivated by clinical trials in chronic heart failure and relapsing multiple sclerosis, which cover a wide range of practically relevant settings. Our research assures that the asymptotic normal theory of group sequential designs can be applied to negative binomial outcomes when the hypotheses are tested using Wald statistics and maximum likelihood estimators. We also propose two methods, one based on Student's t-distribution and one based on resampling, to improve type I error rate control in small samples. The statistical methods studied in this manuscript are implemented in the R package gscounts, which is available for download on the Comprehensive R Archive Network (CRAN).

Blinded sample size re-estimation in three-arm trials with 'gold standard' design

In this article, we study blinded sample size re-estimation in the 'gold standard' design with internal pilot study for normally distributed outcomes. The 'gold standard' design is a three-arm clinical trial design that includes an active and a placebo control in addition to an experimental treatment. We focus on the absolute margin approach to hypothesis testing in three-arm trials at which the non-inferiority of the experimental treatment and the assay sensitivity are assessed by pairwise comparisons. We compare several blinded sample size re-estimation procedures in a simulation study assessing operating characteristics including power and type I error. We find that sample size re-estimation based on the popular one-sample variance estimator results in overpowered trials. Moreover, sample size re-estimation based on unbiased variance estimators such as the Xing-Ganju variance estimator results in underpowered trials, as it is expected because an overestimation of the variance and thus the sample size is in general required for the re-estimation procedure to eventually meet the target power. To overcome this problem, we propose an inflation factor for the sample size re-estimation with the Xing-Ganju variance estimator and show that this approach results in adequately powered trials. Because of favorable features of the Xing-Ganju variance estimator such as unbiasedness and a distribution independent of the group means, the inflation factor does not depend on the nuisance parameter and, therefore, can be calculated prior to a trial. Moreover, we prove that the sample size re-estimation based on the Xing-Ganju variance estimator does not bias the effect estimate. Copyright © 2017 John Wiley & Sons, Ltd.