Optimal Conditional Error Functions for the Control of Conditional Power

An Adaptive Three-Arm Comparative Clinical Endpoint Bioequivalence Study Design With Unblinded Sample Size Re-Estimation and Optimized Allocation Ratio

A three-arm comparative clinical endpoint bioequivalence (BE) study is often used to establish bioequivalence (BE) between a locally acting generic drug (T) and reference drug (R), where superiority needs to be established for T and R over Placebo (P) and equivalence needs to be established for T vs. R. Sometimes, when study design parameters are uncertain, a fixed design study may be under- or over-powered and result in study failure or unnecessary cost. In this paper, we propose a two-stage adaptive clinical endpoint BE study with unblinded sample size re-estimation, standard or maximum combination method, optimized allocation ratio, optional re-estimation of the effect size based on likelihood estimation, and optional re-estimation of the R and P treatment means at interim analysis, which have not been done previously. Our proposed method guarantees control of Type 1 error rate analytically. It helps to reduce the average sample size when the original fixed design is overpowered and increases the sample size and power when the original study and group sequential design are under-powered. Our proposed adaptive design can help generic drug sponsors cut cost and improve success rate, making clinical study endpoint BE studies more affordable and more generic drugs accessible to the public.

New results on optimal conditional error functions for adaptive two-stage designs

Unblinded interim analyses in clinical trials with adaptive designs are gaining increasing popularity. Here, the type I error rate is controlled by defining an appropriate conditional error function. Since various approaches to the selection of the conditional error function exist, the question of an optimal choice arises. In this article, we extend existing work on optimal conditional error functions by two results. Firstly, we prove that techniques from variational calculus can be applied to derive existing optimal conditional error functions. Secondly, we answer the question of optimizing the conditional error function of an optimal promising zone design and investigate the efficiency gain.

Optimization of adaptive designs with respect to a performance score

Adaptive designs are an increasingly popular method for the adaptation of design aspects in clinical trials, such as the sample size. Scoring different adaptive designs helps to make an appropriate choice among the numerous existing adaptive design methods. Several scores have been proposed to evaluate adaptive designs. Moreover, it is possible to determine optimal two-stage adaptive designs with respect to a customized objective score by solving a constrained optimization problem. In this paper, we use the conditional performance score by Herrmann et al. (2020) as the optimization criterion to derive optimal adaptive two-stage designs. We investigate variations of the original performance score, for example, by assigning different weights to the score components and by incorporating prior assumptions on the effect size. We further investigate a setting where the optimization framework is extended by a global power constraint, and additional optimization of the critical value function next to the stage-two sample size is performed. Those evaluations with respect to the sample size curves and the resulting design's performance can contribute to facilitate the score's usage in practice.

Optimal adaptive promising zone designs

We develop optimal decision rules for sample size re-estimation in two-stage adaptive group sequential clinical trials. It is usual for the initial sample size specification of such trials to be adequate to detect a realistic treatment effect δ a with good power, but not sufficient to detect the smallest clinically meaningful treatment effect δ min . Moreover it is difficult for the sponsors of such trials to make the up-front commitment needed to adequately power a study to detect δ min . It is easier to justify increasing the sample size if the interim data enter a so-called "promising zone" that ensures with high probability that the trial will succeed. We have considered promising zone designs that optimize unconditional power and promising zone designs that optimize conditional power and have discussed the tension that exists between these two objectives. Where there is reluctance to base the sample size re-estimation rule on the parameter δ min we propose a Bayesian option whereby a prior distribution is assigned to the unknown treatment effect δ , which is then integrated out of the objective function with respect to its posterior distribution at the interim analysis.

Conditional power and friends: The why and how of (un)planned, unblinded sample size recalculations in confirmatory trials

Adapting the final sample size of a trial to the evidence accruing during the trial is a natural way to address planning uncertainty. Since the sample size is usually determined by an argument based on the power of the trial, an interim analysis raises the question of how the final sample size should be determined conditional on the accrued information. To this end, we first review and compare common approaches to estimating conditional power, which is often used in heuristic sample size recalculation rules. We then discuss the connection of heuristic sample size recalculation and optimal two-stage designs, demonstrating that the latter is the superior approach in a fully preplanned setting. Hence, unplanned design adaptations should only be conducted as reaction to trial-external new evidence, operational needs to violate the originally chosen design, or post hoc changes in the optimality criterion but not as a reaction to trial-internal data. We are able to show that commonly discussed sample size recalculation rules lead to paradoxical adaptations where an initially planned optimal design is not invariant under the adaptation rule even if the planning assumptions do not change. Finally, we propose two alternative ways of reacting to newly emerging trial-external evidence in ways that are consistent with the originally planned design to avoid such inconsistencies.

Optimal planning of adaptive two-stage designs

Adaptive designs are playing an increasingly important role in the planning of clinical trials. While there exists various research on the optimal determination of a two-stage design, non-optimal versions still are frequently applied in clinical research. In this article, we strive to motivate the application of optimal adaptive designs and give guidance on how to determine them. It is demonstrated that optimizing a trial design with respect to particular objective criteria can have a substantial benefit over the application of conventional adaptive sample size recalculation rules. Furthermore, we show that in many practical situations, optimal group-sequential designs show an almost negligible performance loss compared to optimal adaptive designs. Finally, we illustrate how optimal designs can be tailored to specific operational requirements by customizing the underlying optimization problem.

Controlling the type I error rate in two-stage sequential adaptive designs when testing for average bioequivalence

In a 2×2 crossover trial for establishing average bioequivalence (ABE) of a generic agent and a currently marketed drug, the recommended approach to hypothesis testing is the two one-sided test (TOST) procedure, which depends, among other things, on the estimated within-subject variability. The power of this procedure, and therefore the sample size required to achieve a minimum power, depends on having a good estimate of this variability. When there is uncertainty, it is advisable to plan the design in two stages, with an interim sample size reestimation after the first stage, using an interim estimate of the within-subject variability. One method and 3 variations of doing this were proposed by Potvin et al. Using simulation, the operating characteristics, including the empirical type I error rate, of the 4 variations (called Methods A, B, C, and D) were assessed by Potvin et al and Methods B and C were recommended. However, none of these 4 variations formally controls the type I error rate of falsely claiming ABE, even though the amount of inflation produced by Method C was considered acceptable. A major disadvantage of assessing type I error rate inflation using simulation is that unless all possible scenarios for the intended design and analysis are investigated, it is impossible to be sure that the type I error rate is controlled. Here, we propose an alternative, principled method of sample size reestimation that is guaranteed to control the type I error rate at any given significance level. This method uses a new version of the inverse-normal combination of p-values test, in conjunction with standard group sequential techniques, that is more robust to large deviations in initial assumptions regarding the variability of the pharmacokinetic endpoints. The sample size reestimation step is based on significance levels and power requirements that are conditional on the first-stage results. This necessitates a discussion and exploitation of the peculiar properties of the power curve of the TOST testing procedure. We illustrate our approach with an example based on a real ABE study and compare the operating characteristics of our proposed method with those of Method B of Povin et al.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

An approach to the conditional error rate principle with nuisance parameters

In the presence of nuisance parameters, the conditional error rate principle is difficult to apply because of the dependency of the conditional error function of the preplanned test on nuisance parameters. To use the conditional error rate principle with nuisance parameters, we propose to search among tests that guarantee overall error control for the test that maximizes a weighted combination of the conditional error rates over possible values of the nuisance parameters. We show that the optimization problem that defines such a test can be solved efficiently by existing algorithms.

Flexible design of two-stage adaptive procedures for phase III clinical trials

The recent popularity of two-stage adaptive designs has fueled a number of proposals for their use in phase III clinical trials. Many of these designs assign certain restrictive functional forms to the design elements of stage 2, such as sample size, critical value and conditional power functions. We propose a more flexible method of design without imposing any particular functional forms on these design elements. Our methodology permits specification of a design based on either conditional or unconditional characteristics, and allows accommodation of sample size limit. Furthermore, we show how to compute the P value, confidence interval and a reasonable point estimate for any design that can be placed under the proposed framework.

Performance of adaptive sample size adjustment with respect to stopping criteria and time of interim analysis

The benefit of adjusting the sample size in clinical trials on the basis of treatment effects observed in interim analysis has been the subject of several recent papers. Different conclusions were drawn about the usefulness of this approach for gaining power or saving sample size, because of differences in trial design and setting. We examined the benefit of sample size adjustment in relation to trial design parameters such as 'time of interim analysis' and 'choice of stopping criteria'. We compared the adaptive weighted inverse normal method with classical group sequential methods for the most common and for optimal stopping criteria in early, half-time and late interim analyses. We found that reacting to interim data might significantly reduce average sample size in some situations, while classical approaches can out-perform the adaptive designs under other circumstances. We characterized these situations with respect to time of interim analysis and choice of stopping criteria.

An overview of statistical approaches for adaptive designs and design modifications

With adaptive design methods for clinical trials, design elements such as sample size or further interim sample sizes may be changed during the course of the trial depending on all previously collected data. The focus of the overview is on group sequential approaches where the types of adaptations need not be specified in advance. An overview of the different statistical approaches for adaptive design methods proposed in the literature is given, relations between these methods are described and some perspectives of application and for future research are discussed.

Using the partitioning principle to adaptively design dose-response studies

An adaptive two-stage design is proposed for dose-response studies to find the minimum effective dose. The procedure is conducted in a stepwise fashion based on the partition testing principle with familywise error rate controlled strongly. We examine a wide dose range vs. a placebo in the first stage. Then an interim analysis is conducted with potential modification of design features of the experiment. Ineffective and/or unsafe dose treatments are terminated, and selected doses are further investigated in the second stage. Inference is based on a pre-chosen conditional error function. Several conditional error functions are discussed and compared.

An Adaptive Two-stage Design with Treatment Selection Using the Conditional Error Function Approach

As an approach to combining the phase II dose finding trial and phase III pivotal trials, we propose a two-stage adaptive design that selects the best among several treatments in the first stage and tests significance of the selected treatment in the second stage. The approach controls the type I error defined as the probability of selecting a treatment and claiming its significance when the selected treatment is indifferent from placebo, as considered in Bischoff and Miller (2005). Our approach uses the conditional error function and allows determining the conditional type I error function for the second stage based on information observed at the first stage in a similar way to that for an ordinary adaptive design without treatment selection. We examine properties such as expected sample size and stage-2 power of this design with a given type I error and a maximum stage-2 sample size under different hypothesis configurations. We also propose a method to find the optimal conditional error function of a simple parametric form to improve the performance of the design and have derived optimal designs under some hypothesis configurations. Application of this approach is illustrated by a hypothetical example.

A general formulation for a one-sided group sequential design

BACKGROUND

A major contribution to the statistical literature on group sequential designs was provided by Pampallona and Tsiatis who developed closed form functions that can be used to iteratively calculate the boundary points of a family of popular group sequential designs. A related area of interest is the use of conditional probability calculations to make interim decisions in stochastic curtailment procedures.

PURPOSE

The purpose of the paper is to develop group sequential designs based on conditional probabilities, to compare our results to the general closed form family of designs developed by Pampallona and Tsiatis, and to relate these to commonly used stochastic curtailment procedures.

METHODS

The problem and its solution are formulated and derived mathematically. A graphical interpretation of the results provides the reader with an alternative mechanism to understand the results and their significance.

RESULTS

One-sided group sequential design boundary points, as closed form functions, are derived from conditional probability statements. These conditional probability statements can be interpreted as the probability, at the final analysis, of reversing the conclusion reached at an interim state. Under mild constraints, these boundary points are identical to the Pampallona and Tsiatis boundary points. At any interim stage when a boundary point is attained or surpassed we suggest a graphical approach to examine the conditional probability of reversing the interim decision at the final stage versus a range of possible parameter values. For stochastic curtailment procedures, we recommend relaxing (increasing) the conditional probability levels to at least 0.50 so that early stopping is at least as likely as for the O'Brien-Fleming procedure.

LIMITATIONS

The results are limited to one-sided group sequential designs.

CONCLUSIONS

Conditional probabilities of reversing interim decisions provides a useful concept to develop group sequential designs and to evaluate stochastic curtailment procedures.

Estimation in flexible two stage designs

Adaptive test designs for clinical trials allow for a wide range of data driven design adaptations using all information gathered until an interim analysis. The basic principle is to use a test statistics which is invariant with respect to the design adaptations under the null hypothesis. This allows for a control of the type I error rate for the primary hypothesis even for adaptations not specified a priori in the study protocol. Estimation is usually another important part of a clinical trial, however, is more difficult in adaptive designs. In this research paper we give an overview of point and interval estimates for flexible designs and compare methods for typical sample size rules. We also make some proposals for confidence intervals which have nominal coverage probability also after an unforeseen design adaptation and which contain the maximum likelihood estimate and the usual unadjusted confidence interval.

Adaptive designs based on the truncated product method

Background

Adaptive designs are becoming increasingly important in clinical research. One approach subdivides the study into several (two or more) stages and combines the p-values of the different stages using Fisher's combination test.

Methods

Alternatively to Fisher's test, the recently proposed truncated product method (TPM) can be applied to combine the p-values. The TPM uses the product of only those p-values that do not exceed some fixed cut-off value. Here, these two competing analyses are compared.

Results

When an early termination due to insufficient effects is not appropriate, such as in dose-response analyses, the probability to stop the trial early with the rejection of the null hypothesis is increased when the TPM is applied. Therefore, the expected total sample size is decreased. This decrease in the sample size is not connected with a loss in power. The TPM turns out to be less advantageous, when an early termination of the study due to insufficient effects is possible. This is due to a decrease of the probability to stop the trial early.

Conclusion

It is recommended to apply the TPM rather than Fisher's combination test whenever an early termination due to insufficient effects is not suitable within the adaptive design.

The reassessment of trial perspectives from interim data—a critical view

If an interim analysis is performed during a trial it is tempting to determine the conditional power to reach a rejection in the trial given the observed results in the interim analysis. Since the true effect size is unknown the conditional power may be calculated by using the effect size, which the study has been powered for in the planning phase or by using an interim estimate of the true size (or a combination of both). In either case the conditional power is a random variable and its density is investigated depending on the analysis time and the true effect size. Under the null hypothesis, in early interim analyses after a small proportion of sample units, the conditional power typically will be close to the overall power when the effect size from the planning stage is used for calculation. In this case the majority of observations must still be made and the small first-stage sample in general will be dominated by the hypothetical second-stage chance based on the wrong parameter value. It is shown that the conditional power in moderately underpowered studies can have a distribution symmetric around 0.5. When using the interim estimate for calculating the conditional power the density in general will be u-shaped. The impact of using conditional power to reassess the sample size using flexible two-stage combination tests is shown for a specific example in terms of overall power and average sample size as compared to the corresponding group sequential design. For small true effect sizes mid-trial sample size recalculation based on an interim estimate may lead to an overly large price to be paid in average sample size in relation to the gain in overall power. Finally, the problem is discussed in terms of estimating the true conditional power.