Mid-trial design reviews for sequential clinical trials

Blinded and unblinded internal pilot study designs for clinical trials with count data

Internal pilot studies are a popular design feature to address uncertainties in the sample size calculations caused by vague information on nuisance parameters. Despite their popularity, only very recently blinded sample size reestimation procedures for trials with count data were proposed and their properties systematically investigated. Although blinded procedures are favored by regulatory authorities, practical application is somewhat limited by fears that blinded procedures are prone to bias if the treatment effect was misspecified in the planning. Here, we compare unblinded and blinded procedures with respect to bias, error rates, and sample size distribution. We find that both procedures maintain the desired power and that the unblinded procedure is slightly liberal whereas the actual significance level of the blinded procedure is close to the nominal level. Furthermore, we show that in situations where uncertainty about the assumed treatment effect exists, the blinded estimator of the control event rate is biased in contrast to the unblinded estimator, which results in differences in mean sample sizes in favor of the unblinded procedure. However, these differences are rather small compared to the deviations of the mean sample sizes from the sample size required to detect the true, but unknown effect. We demonstrate that the variation of the sample size resulting from the blinded procedure is in many practically relevant situations considerably smaller than the one of the unblinded procedures. The methods are extended to overdispersed counts using a quasi-likelihood approach and are illustrated by trials in relapsing multiple sclerosis.

A practical comparison of blinded methods for sample size reviews in survival data clinical trials

This paper presents practical approaches to the problem of sample size re-estimation in the case of clinical trials with survival data when proportional hazards can be assumed. When data are readily available at the time of the review, on a full range of survival experiences across the recruited patients, it is shown that, as expected, performing a blinded re-estimation procedure is straightforward and can help to maintain the trial's pre-specified error rates. Two alternative methods for dealing with the situation where limited survival experiences are available at the time of the sample size review are then presented and compared. In this instance, extrapolation is required in order to undertake the sample size re-estimation. Worked examples, together with results from a simulation study are described. It is concluded that, as in the standard case, use of either extrapolation approach successfully protects the trial error rates.

Sample size re-estimation for survival data in clinical trials with an adaptive design

In clinical trials with survival data, investigators may wish to re-estimate the sample size based on the observed effect size while the trial is ongoing. Besides the inflation of the type-I error rate due to sample size re-estimation, the method for calculating the sample size in an interim analysis should be carefully considered because the data in each stage are mutually dependent in trials with survival data. Although the interim hazard estimate is commonly used to re-estimate the sample size, the estimate can sometimes be considerably higher or lower than the hypothesized hazard by chance. We propose an interim hazard ratio estimate that can be used to re-estimate the sample size under those circumstances. The proposed method was demonstrated through a simulation study and an actual clinical trial as an example. The effect of the shape parameter for the Weibull survival distribution on the sample size re-estimation is presented.

Combining an Internal Pilot with an Interim Analysis for Single Degree of Freedom Tests

An internal pilot with interim analysis (IPIA) design combines interim power analysis (an internal pilot) with interim data analysis (two stage group sequential). We provide IPIA methods for single df hypotheses within the Gaussian general linear model, including one and two group t tests. The design allows early stopping for efficacy and futility while also re-estimating sample size based on an interim variance estimate. Study planning in small samples requires the exact and computable forms reported here. The formulation gives fast and accurate calculations of power, type I error rate, and expected sample size.

Group sequential trials revisited: simple implementation using SAS

The methodology of group sequential trials is now well established and widely implemented. The benefits of the group sequential approach are generally acknowledged, and its use, when applied properly, is accepted by researchers and regulators. This article describes how a wide range of group sequential designs can easily be implemented using two accessible SAS functions. One of these, PROBBNRM is a standard function, while the other, SEQ, is part of the interactive matrix language of SAS, PROC IML. The account focuses on the essentials of the approach and reveals how straightforward it can be. The design of studies is described, including their evaluation in terms of the distribution of final sample size. The conduct of the interim analyses is discussed, with emphasis on the consequences of inevitable departures from the planned schedule of information accrual. The computations required for the final analysis, allowing for the sequential design, are closely related to those conducted at the design stage. Illustrative examples are given and listings of suitable of SAS code are provided.

The use of mid-trial reviews for design modifications in small scale clinical studies

Many clinical studies such as those in the areas of toxicology, early phase clinical trials and bioequivalence studies use small samples due to the high cost of experimentation. These studies test hypotheses based on small samples. These small samples result in low power and therefore even if the alternative hypotheses may be true the chance of it being rejected is low. The sample size is determined in an ad-hoc way and no proper scientific approach is used. Sample size calculations for clinical studies are usually conducted to determine the total number of patients needed to satisfy a specified power requirement, and their validity is dependent on pre-trial knowledge of nuisance parameters and distributional and modelling assumptions. Another short coming is that often hypotheses are tested without checking the assumptions required by the test. This paper looks at design reviews in the context of small samples. It examines several design modifications done with a small internal pilot study. In the past similar techniques have been applied to large scale studies but its performance is yet to be established in small scale clinical studies thus the contribution of this paper is in justifying the validity of these techniques for small samples too. The methodology is illustrated on an uncontrolled observational toxicology study. In this paper simulations will be presented showing that the design modifications would not influence the type-I error rate and that these would be successful in preserving the power, and the implementation of the design review procedure will be described.

Adaptive clinical trials: progress and challenges

Adaptive designs promise the flexibility to redesign clinical trials at interim stages. This flexibility would provide greater efficiency in drug development. However, despite this promise, many hesitate to implement such designs. Here we explore three possible reasons for the hesitation: (i) confusion with respect to the definition of an 'adaptive design'; (ii) controversy surrounding the use of sample size re-estimation methods; and (iii) logistical barriers that must be overcome in order to use adaptive designs within existing trial frameworks.The large volume of recent work has created confusion with respect to the definition of an 'adaptive design'. Unfortunately, this has resulted in reduced usage of many acceptable methods because of guilt by association with the more controversial methods. This review attempts to clarify the differences among many common types of proposed adaptive designs. Once the differences are noted, it becomes apparent that some adaptive designs are well accepted while others remain very controversial. In fact, much of the controversy and criticism surrounding adaptive designs has focused on their use for sample size re-estimation. Hence, this review also examines the different types of adaptive designs for sample size re-estimation in order to clarify the controversy surrounding the use of these methods. Specifically, separating the controversial from good practice requires clarifying differences between adaptive designs with sample size re-estimation based on a revised treatment effect and re-estimation based only on nuisance parameters (internal pilot designs). Finally, many logistical barriers must be overcome in order to use adaptive designs within existing trial frameworks.If the promise of adaptive designs is to be achieved, it will be important to bring together large groups of individuals from funding sources and regulatory agencies to address these limitations. Very few discussions of these issues have appeared in journals that are targeted to clinical audiences. In fact, current use of adaptive designs is not really hindered by the lack of statistical methods to accommodate the adaptations. Rather, there is a need for education as to which adaptive designs are acceptable and which are not acceptable. These discussions will require the involvement of many individuals outside the statistical community. In this review, we summarize the existing methods and current controversies with the intent of providing a clarification that will enable these individuals to participate in these much-needed discussions.

Efficient adaptive designs with mid-course sample size adjustment in clinical trials

Adaptive designs have been proposed for clinical trials in which the nuisance parameters or alternative of interest are unknown or likely to be misspecified before the trial. Although most previous works on adaptive designs and mid-course sample size re-estimation have focused on two-stage or group-sequential designs in the normal case, we consider here a new approach that involves at most three stages and is developed in the general framework of multiparameter exponential families. This approach not only maintains the prescribed type I error probability but also provides a simple but asymptotically efficient sequential test whose finite-sample performance, measured in terms of the expected sample size and power functions, is shown to be comparable to the optimal sequential design, determined by dynamic programming, in the simplified normal mean case with known variance and prespecified alternative, and superior to the existing two-stage designs and also to adaptive group-sequential designs when the alternative or nuisance parameters are unknown or misspecified.

Randomised controlled trials in orthopaedic surgery and traumatology: overview of parameters and pitfalls

There is a growing consensus that randomised controlled clinical trial (RCT) provide a secure basis for determining treatment effects. Prospective randomised clinical trials can be a powerful tool in medical science and evidence-based medicine. A well-defined study hypothesis, with a prospectively applied study design, blinded and randomised treatment allocation and assessment, with appropriate control groups can provide strong evidence in support of treatment decisions. However, the recent reviews of the medical literature indicate that the study design itself does not ensure the quality of science or useful and valid scientific data. Thus, regardless of the study design or level of evidence, it remains imperative for the physician and surgeon to critically evaluate a scientific report. Moreover, as randomisation, concealment of treatment allocation and blinding are difficult issues to resolve in orthopaedic surgery, future trials should focus on detailed and correct reporting of outcome measures.

Efficient ways exist to obtain the optimal sample size in clinical trials in rare diseases

OBJECTIVE

Recruitment of pediatric patients in randomized clinical trials is hampered by the rarity of many conditions and by ethical constraints. The objective of this paper is to give an overview of design options to obtain a statistically valid result while including a minimum number of subjects.

STUDY DESIGN AND SETTING

Overview and discussion of several approaches to conduct valid randomized clinical trials in rare diseases and vulnerable populations.

RESULTS

Sequential designs have been developed as efficient ways to evaluate accumulating information from a clinical trial, thereby reducing the average size of trials. Different sequential procedures exist, including group sequential designs, boundaries designs, and adaptive designs. The sample size attained at the end of the trial is unknown at the start. The sample size for a given set of alpha, beta, and effect size may turn out to be larger than with a classical fixed sample size approach. Simulations have shown that on average, sample sizes are smaller.

CONCLUSION

There are several possibilities to optimize the number of subjects in a clinical trial. The rarity of many disorders in children and the ethical requirements in this patient population should not obstruct the performance of well-designed research to support clinical decision making.

Flexible two-stage design with sample size reassessment for survival trials

Flexible trials with adaptive design modification at interim analyses have been recently proposed as an answer to cope with some limitations of traditional designs for phase III trials. Actually, the sample size and duration of fixed design trials strongly depend on the determination, prior to the study, of key parameters such as the expected treatment effect and the event rates. A misspecification of these parameters may result in an underpowered or overpowered trial. In the flexible framework, the remainder of a design can be modified at an interim analysis, while preserving the initially specified global error rates. In this work, we present a flexible design with sample size re-evaluation for survival trials and study its properties in practical settings. The results show that, if parameters are initially misspecified, the proposed method allows an improved power control with a reasonable increase in sample size, if any. Practical guidelines concerning the choice of the trial parameters are also given.

An adaptive approach to designing comparative diagnostic accuracy studies

Comparative diagnostic studies usually involve comparison of the area under receiver operating characteristic curves when biomarkers are measured on a continuous or ordinal scales. In designing such studies, specification of a number of nuisance parameters is often required to compute sample sizes. When these parameters are incorrectly specified, statistical power to detect a meaningful difference in area can be substantially adversely affected. We propose an adaptive method to calculate the sample size and show these procedures to be effective in controlling error rates.

Robustness of an odds-ratio test in a stratified group sequential trial with a binary outcome measure

Many clinical trials compare two or more treatment groups by using a binary outcome measure. For example, the goal could be to determine whether the frequency of pain episodes is significantly reduced in the treatment group (arm A) as compared to the control group (arm B). However, for ethical or regulatory reasons, group sequential designs are commonly employed. Then, based on a binomial distribution, the stopping boundaries for the interim analyses are constructed for assessing the difference in the response probabilities between the two groups. This is easily accomplished by using any of the standard procedures, e.g., those discussed by Jennison and Turnbull (2000), and using one of the most commonly used software packages, East (2000). Several factors are known to often affect the primary outcome of interest, but their true distributions are not known in advance. In addition, these factors may cause heterogeneous treatment responses among individuals in a group, and their exact effect size may be unknown. To limit the effect of such factors on the comparison of the two arms, stratified randomization is used in the actual conduct of the trial. Then, a stratified analysis based on the odds ratio proposed in Jennison and Turnbull (2000, pages 251-252) and consistent with the stratified design is undertaken. However, the stopping rules used for the interim analyses are those obtained for determining the differences in response rates in a design that was not stratified. The purpose of this paper is to assess the robustness of such an approach on the performance of the odds ratio test when the underlying distribution and effect size of the factors that influence the outcome may vary. The simulation studies indicate that, in general, the stratified approach offers consistently better results than does the unstratified approach, as long as the difference in the weighted average of the response probabilities across strata between the two groups remains closer to the hypothesized values, irrespective of the differences in the (allocation) distributions and heterogeneous response rate. However, if the response probabilities deviate significantly from the hypothesized values so that the difference in the weighted average is less than the hypothesized value, then the proposed study could be significantly underpowered.

An adaptive approach for bivariate phase II clinical trial designs

In designing a phase II cancer clinical trial monitoring simultaneously the response and toxicity rate of a therapeutic agent, the odds ratio has to be specified. The false positive or Type I error rate, however, can be substantially inflated if the specified value is considerably larger than the true odds ratio. To overcome the sensitivity of the error rates to the odds ratio, an adaptive procedure is proposed which allows the sample size to be re-estimated based on observed odds ratio. Simulation results show that the procedure is robust against the odds ratio assumptions and controls effectively the Type I error rate.

Sample Size Recalculation in Internal Pilot Study Designs: A Review

The adequacy of sample size is important to clinical trials. In the planning phase of a trial, however, the investigators are often quite uncertain about the sizes of parameters which are needed for sample size calculations. A solution to this problem is mid-course recalculation of the sample size during the ongoing trial. In internal pilot study designs, nuisance parameters are estimated on the basis of interim data and the sample size is adjusted accordingly. This review attempts to give an overview on the available methods. It is written not only for biometricians who are already familar with the the topic and wish to update their knowledge but also for users new to the subject.

A 25-year review of sequential methodology in clinical studies

This paper explores the theoretical developments and subsequent uptake of sequential methodology in clinical studies in the 25 years since Statistics in Medicine was launched. The review examines the contributions which have been made to all four phases into which clinical trials are traditionally classified and highlights major statistical advancements, together with assessing application of the techniques. The vast majority of work has been in the setting of phase III clinical trials and so emphasis will be placed here. Finally, comments are given indicating how the subject area may develop in the future.

A practical comparison of group-sequential and adaptive designs

Sequential methods provide a formal framework by which clinical trial data can be monitored as they accumulate. The results from interim analyses can be used either to modify the design of the remainder of the trial or to stop the trial as soon as sufficient evidence of either the presence or absence of a treatment effect is available. The circumstances under which the trial will be stopped with a claim of superiority for the experimental treatment, must, however, be determined in advance so as to control the overall type I error rate. One approach to calculating the stopping rule is the group-sequential method. A relatively recent alternative to group-sequential approaches is the adaptive design method. This latter approach provides considerable flexibility in changes to the design of a clinical trial at an interim point. However, a criticism is that the method by which evidence from different parts of the trial is combined means that a final comparison of treatments is not based on a sufficient statistic for the treatment difference, suggesting that the method may lack power. The aim of this paper is to compare two adaptive design approaches with the group-sequential approach. We first compare the form of the stopping boundaries obtained using the different methods. We then focus on a comparison of the power of the different trials when they are designed so as to be as similar as possible. We conclude that all methods acceptably control type I error rate and power when the sample size is modified based on a variance estimate, provided no interim analysis is so small that the asymptotic properties of the test statistic no longer hold. In the latter case, the group-sequential approach is to be preferred. Provided that asymptotic assumptions hold, the adaptive design approaches control the type I error rate even if the sample size is adjusted on the basis of an estimate of the treatment effect, showing that the adaptive designs allow more modifications than the group-sequential method.

Sample size re-estimation in group-sequential response-adaptive clinical trials

In clinical trials where the variances of the response variables are unknown, in accurate estimates of these can affect the type II error rate considerably. More accurate estimates of the variances may be obtained by taking a look at the data available part way through the trial and re-calculating the required sample size based on these new estimates. The main impetus for sample size re-estimation came from a two-stage procedure developed by Stein in 1945 and the literature is now replete with variations on this approach. In this paper, existing sample size re-estimation methods for both fixed sample and sequential clinical trial models will be reviewed. These will then be extended for use in group-sequential response-adaptive designs. In particular, a test for a recently developed group-sequential response-adaptive design, which compares two treatments with immediate normally distributed responses and unknown variances, is presented based on a modified version of Stein's test. The principal modifications involve updating the required sample size at each interim analysis and calculating the test statistic based on the current estimates of the variances. Hence, all the available information is used at each stage. Simulation is used to assess to what extent the updating of the required sample size at each interim analysis in the new test helps to attain the nominal error rates. The test is compared to modified versions of a simple test and a Stein-type group sequential t-test studied in the recent literature. These tests calculate the required sample sizes based on less accurate estimates of the variances. The type I error rate is close to the nominal value and the power is more accurately maintained in the new test.

Mid-course sample size modification in clinical trials based on the observed treatment effect

It is not uncommon to set the sample size in a clinical trial to attain specified power at a value for the treatment effect deemed likely by the experimenters, even though a smaller treatment effect would still be clinically important. Recent papers have addressed the situation where such a study produces only weak evidence of a positive treatment effect at an interim stage and the organizers wish to modify the design in order to increase the power to detect a smaller treatment effect than originally expected. Raising the power at a small treatment effect usually leads to considerably higher power than was first specified at the original alternative. Several authors have proposed methods which are not based on sufficient statistics of the data after the adaptive redesign of the trial. We discuss these proposals and show in an example how the same objectives can be met while maintaining the sufficiency principle, as long as the eventuality that the treatment effect may be small is considered at the design stage. The group sequential designs we suggest are quite standard in many ways but unusual in that they place emphasis on reducing the expected sample size at a parameter value under which extremely high power is to be achieved. Comparisons of power and expected sample size show that our proposed methods can out-perform L. Fisher's 'variance spending' procedure. Although the flexibility to redesign an experiment in mid-course may be appealing, the cost in terms of the number of observations needed to correct an initial design may be substantial.

Simple procedures for blinded sample size adjustment that do not affect the type I error rate

For normally distributed data, determination of the appropriate sample size requires a knowledge of the variance. Because of the uncertainty in the planning phase, two-stage procedures are attractive where the variance is reestimated from a subsample and the sample size is adjusted if necessary. From a regulatory viewpoint, preserving blindness and maintaining the ability to calculate or control the type I error rate are essential. Recently, a number of proposals have been made for sample size adjustment procedures in the t-test situation. Unfortunately, none of these methods satisfy both these requirements. We show through analytical computations that the type I error rate of the t-test is not affected if simple blind variance estimators are used for sample size recalculation. Furthermore, the results for the expected power of the procedures demonstrate that the methods are effective in ensuring the desired power even under initial misspecification of the variance. A method is discussed that can be applied in a more general setting and that assumes analysis with a permutation test. This procedure maintains the significance level for any design situation and arbitrary blind sample size recalculation strategy.

Modification of the sample size and the schedule of interim analyses in survival trials based on data inspections

A method is presented which allows us to adapt the sample size as well as the number and time points of interim analyses to the treatment difference observed at an interim look during the course of a clinical trial with censored survival time as the endpoint. The method allows the inclusion of data inspections during the course of the trial and redesign of the trial on the basis of the observed treatment difference without affecting the type I error risk. Formulae for recalculating the required number of events and the number of further patients to be randomized as a function of the observed hazard rates and the detectable hazard ratio are given.