Internal pilot studies I: type I error rate of the naive t-test

Study in Parkinson's disease of exercise phase 3 (SPARX3): study protocol for a randomized controlled trial

BACKGROUND

To date, no medication has slowed the progression of Parkinson's disease (PD). Preclinical, epidemiological, and experimental data on humans all support many benefits of endurance exercise among persons with PD. The key question is whether there is a definitive additional benefit of exercising at high intensity, in terms of slowing disease progression, beyond the well-documented benefit of endurance training on a treadmill for fitness, gait, and functional mobility. This study will determine the efficacy of high-intensity endurance exercise as first-line therapy for persons diagnosed with PD within 3 years, and untreated with symptomatic therapy at baseline.

METHODS

This is a multicenter, randomized, evaluator-blinded study of endurance exercise training. The exercise intervention will be delivered by treadmill at 2 doses over 18 months: moderate intensity (4 days/week for 30 min per session at 60-65% maximum heart rate) and high intensity (4 days/week for 30 min per session at 80-85% maximum heart rate). We will randomize 370 participants and follow them at multiple time points for 24 months. The primary outcome is the Movement Disorders Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) motor score (Part III) with the primary analysis assessing the change in MDS-UPDRS motor score (Part III) over 12 months, or until initiation of symptomatic antiparkinsonian treatment if before 12 months. Secondary outcomes are striatal dopamine transporter binding, 6-min walk distance, number of daily steps, cognitive function, physical fitness, quality of life, time to initiate dopaminergic medication, circulating levels of C-reactive protein (CRP), and brain-derived neurotrophic factor (BDNF). Tertiary outcomes are walking stride length and turning velocity.

DISCUSSION

SPARX3 is a Phase 3 clinical trial designed to determine the efficacy of high-intensity, endurance treadmill exercise to slow the progression of PD as measured by the MDS-UPDRS motor score. Establishing whether high-intensity endurance treadmill exercise can slow the progression of PD would mark a significant breakthrough in treating PD. It would have a meaningful impact on the quality of life of people with PD, their caregivers and public health.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04284436 . Registered on February 25, 2020.

Bayesian updating: increasing sample size during the course of a study

Background

A priori sample size calculation requires an a priori estimate of the size of the effect. An incorrect estimate may result in a sample size that is too low to detect effects or that is unnecessarily high. An alternative to a priori sample size calculation is Bayesian updating, a procedure that allows increasing sample size during the course of a study until sufficient support for a hypothesis is achieved. This procedure does not require and a priori estimate of the effect size. This paper introduces Bayesian updating to researchers in the biomedical field and presents a simulation study that gives insight in sample sizes that may be expected for two-group comparisons.

Methods

Bayesian updating uses the Bayes factor, which quantifies the degree of support for a hypothesis versus another one given the data. It can be re-calculated each time new subjects are added, without the need to correct for multiple interim analyses. A simulation study was conducted to study what sample size may be expected and how large the error rate is, that is, how often the Bayes factor shows most support for the hypothesis that was not used to generate the data.

Results

The results of the simulation study are presented in a Shiny app and summarized in this paper. Lower sample size is expected when the effect size is larger and the required degree of support is lower. However, larger error rates may be observed when a low degree of support is required and/or when the sample size at the start of the study is small. Furthermore, it may occur sufficient support for neither hypothesis is achieved when the sample size is bounded by a maximum.

Conclusions

Bayesian updating is a useful alternative to a priori sample size calculation, especially so in studies where additional subjects can be recruited easily and data become available in a limited amount of time. The results of the simulation study show how large a sample size can be expected and how large the error rate is.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01334-6.

Controlling type 1 error rate for sequential, bioequivalence studies with crossover designs

Sample size reestimation in a crossover, bioequivalence study can be a useful adaptive design tool, particularly when the intrasubject variability of the drug formulation under investigation is not well understood. When sample size reestimation is done based on an interim estimate of the intrasubject variability and bioequivalence is tested using the pooled estimate of intrasubject variability, type 1 error inflation will occur. Type 1 error inflation is caused by the pooled estimate being a biased estimator of the intrasubject variability. The type 1 error inflation and bias of the pooled estimator of variability are well characterized in the setting of a two-arm, parallel study. The purpose of this work is to extend this characterization to the setting of a crossover, bioequivalence study with sample size reestimation and to propose an estimator of the intrasubject variability that will prevent type 1 error inflation.

Estimation after blinded sample size reassessment

Blinded sample size reassessment is a popular means to control the power in clinical trials if no reliable information on nuisance parameters is available in the planning phase. We investigate how sample size reassessment based on blinded interim data affects the properties of point estimates and confidence intervals for parallel group superiority trials comparing the means of a normal endpoint. We evaluate the properties of two standard reassessment rules that are based on the sample size formula of the z-test, derive the worst case reassessment rule that maximizes the absolute mean bias and obtain an upper bound for the mean bias of the treatment effect estimate.

Internal pilot design for balanced repeated measures

Repeated measures are common in clinical trials and epidemiological studies. Designing studies with repeated measures requires reasonably accurate specifications of the variances and correlations to select an appropriate sample size. Underspecifying the variances leads to a sample size that is inadequate to detect a meaningful scientific difference, while overspecifying the variances results in an unnecessarily large sample size. Both lead to wasting resources and placing study participants in unwarranted risk. An internal pilot design allows sample size recalculation based on estimates of the nuisance parameters in the covariance matrix. We provide the theoretical results that account for the stochastic nature of the final sample size in a common class of linear mixed models. The results are useful for designing studies with repeated measures and balanced design. Simulations examine the impact of misspecification of the covariance matrix and demonstrate the accuracy of the approximations in controlling the type I error rate and achieving the target power. The proposed methods are applied to a longitudinal study assessing early antiretroviral therapy for youth living with HIV.

Sample Size Reassessment in Non-inferiority Trials

Objectives: Analysis of covariance (ANCOVA) is widely applied in practice and its use is recommended by regulatory guidelines. However, the required sample size for ANCOVA depends on parameters that are usually uncertain in the planning phase of a study. Sample size recalculation within the internal pilot study design allows to cope with this problem. From a regulatory viewpoint it is preferable that the treatment group allocation remains masked and that the type I error is controlled at the specified significance level. The characteristics of blinded sample size reassessment for ANCOVA in non-inferiority studies have not been investigated yet. We propose an appropriate method and evaluate its performance.

Methods: In a simulation study, the characteristics of the proposed method with respect to type I error rate, power and sample size are investigated. It is illustrated by a clinical trial example how strict control of the significance level can be achieved.

Results: A slight excess of the type I error rate beyond the nominal significance level was observed. The extent of exceedance increases with increasing non-inferiority margin and increasing correlation between outcome and covariate. The procedure assures the desired power over a wide range of scenarios even if nuisance parameters affecting the sample size are initially mis-specified.

Conclusions: The proposed blinded sample size recalculation procedure protects from insufficient sample sizes due to incorrect assumptions about nuisance parameters in the planning phase. The original procedure may lead to an elevated type I error rate, but methods are available to control the nominal significance level.

Blinded sample size re-estimation in three-arm trials with 'gold standard' design

In this article, we study blinded sample size re-estimation in the 'gold standard' design with internal pilot study for normally distributed outcomes. The 'gold standard' design is a three-arm clinical trial design that includes an active and a placebo control in addition to an experimental treatment. We focus on the absolute margin approach to hypothesis testing in three-arm trials at which the non-inferiority of the experimental treatment and the assay sensitivity are assessed by pairwise comparisons. We compare several blinded sample size re-estimation procedures in a simulation study assessing operating characteristics including power and type I error. We find that sample size re-estimation based on the popular one-sample variance estimator results in overpowered trials. Moreover, sample size re-estimation based on unbiased variance estimators such as the Xing-Ganju variance estimator results in underpowered trials, as it is expected because an overestimation of the variance and thus the sample size is in general required for the re-estimation procedure to eventually meet the target power. To overcome this problem, we propose an inflation factor for the sample size re-estimation with the Xing-Ganju variance estimator and show that this approach results in adequately powered trials. Because of favorable features of the Xing-Ganju variance estimator such as unbiasedness and a distribution independent of the group means, the inflation factor does not depend on the nuisance parameter and, therefore, can be calculated prior to a trial. Moreover, we prove that the sample size re-estimation based on the Xing-Ganju variance estimator does not bias the effect estimate. Copyright © 2017 John Wiley & Sons, Ltd.

A framework for prospectively defining progression rules for internal pilot studies monitoring recruitment

Just over half of publicly funded trials recruit their target sample size within the planned study duration. When recruitment targets are missed, the funder of a trial is faced with the decision of either committing further resources to the study or risk that a worthwhile treatment effect may be missed by an underpowered final analysis. To avoid this challenging situation, when there is insufficient prior evidence to support predicted recruitment rates, funders now require feasibility assessments to be performed in the early stages of trials. Progression criteria are usually specified and agreed with the funder ahead of time. To date, however, the progression rules used are typically ad hoc. In addition, rules routinely permit adaptations to recruitment strategies but do not stipulate criteria for evaluating their effectiveness. In this paper, we develop a framework for planning and designing internal pilot studies which permit a trial to be stopped early if recruitment is disappointing or to continue to full recruitment if enrolment during the feasibility phase is adequate. This framework enables a progression rule to be pre-specified and agreed upon prior to starting a trial. The novel two-stage designs stipulate that if neither of these situations arises, adaptations to recruitment should be made and subsequently evaluated to establish whether they have been successful. We derive optimal progression rules for internal pilot studies which minimise the expected trial overrun and maintain a high probability of completing the study when the recruitment rate is adequate. The advantages of this procedure are illustrated using a real trial example.

Blinded versus unblinded estimation of a correlation coefficient to inform interim design adaptations

Regulatory authorities require that the sample size of a confirmatory trial is calculated prior to the start of the trial. However, the sample size quite often depends on parameters that might not be known in advance of the study. Misspecification of these parameters can lead to under- or overestimation of the sample size. Both situations are unfavourable as the first one decreases the power and the latter one leads to a waste of resources. Hence, designs have been suggested that allow a re-assessment of the sample size in an ongoing trial. These methods usually focus on estimating the variance. However, for some methods the performance depends not only on the variance but also on the correlation between measurements. We develop and compare different methods for blinded estimation of the correlation coefficient that are less likely to introduce operational bias when the blinding is maintained. Their performance with respect to bias and standard error is compared to the unblinded estimator. We simulated two different settings: one assuming that all group means are the same and one assuming that different groups have different means. Simulation results show that the naïve (one-sample) estimator is only slightly biased and has a standard error comparable to that of the unblinded estimator. However, if the group means differ, other estimators have better performance depending on the sample size per group and the number of groups.

Alternative designs for clinical trials in rare diseases

Evidence-based medicine requires strong scientific evidence upon which to base treatment. In rare diseases, study populations are often small, and thus this evidence is difficult to accrue. Investigators, though, should be creative and develop a flexible toolkit of methods to deal with the problems inherent in the study of rare disease. This narrative review presents alternative clinical trial designs for studying treatments of rare diseases, including cross-over and n-of-1 trials, randomized placebo-phase design, enriched enrollment, randomized withdrawal design, and classes of adaptive designs. Examples of applications of these designs are presented along with their advantages and disadvantages. Additional analytical considerations such as Bayesian analysis, internal pilots, and use of biomarkers as surrogate outcomes are further discussed. A framework for selecting appropriate clinical trial design is proposed to guide investigators in the process of selecting alternative designs for rare diseases. © 2016 Wiley Periodicals, Inc.

Distribution of the two-sample t-test statistic following blinded sample size re-estimation

We consider the blinded sample size re-estimation based on the simple one-sample variance estimator at an interim analysis. We characterize the exact distribution of the standard two-sample t-test statistic at the final analysis. We describe a simulation algorithm for the evaluation of the probability of rejecting the null hypothesis at given treatment effect. We compare the blinded sample size re-estimation method with two unblinded methods with respect to the empirical type I error, the empirical power, and the empirical distribution of the standard deviation estimator and final sample size. We characterize the type I error inflation across the range of standardized non-inferiority margin for non-inferiority trials, and derive the adjusted significance level to ensure type I error control for given sample size of the internal pilot study. We show that the adjusted significance level increases as the sample size of the internal pilot study increases. Copyright © 2016 John Wiley & Sons, Ltd.

An internal pilot design for prospective cancer screening trials with unknown disease prevalence

BACKGROUND

For studies that compare the diagnostic accuracy of two screening tests, the sample size depends on the prevalence of disease in the study population, and on the variance of the outcome. Both parameters may be unknown during the design stage, which makes finding an accurate sample size difficult.

METHODS

To solve this problem, we propose adapting an internal pilot design. In this adapted design, researchers will accrue some percentage of the planned sample size, then estimate both the disease prevalence and the variances of the screening tests. The updated estimates of the disease prevalence and variance are used to conduct a more accurate power and sample size calculation.

RESULTS

We demonstrate that in large samples, the adapted internal pilot design produces no Type I inflation. For small samples (N less than 50), we introduce a novel adjustment of the critical value to control the Type I error rate. We apply the method to two proposed prospective cancer screening studies: 1) a small oral cancer screening study in individuals with Fanconi anemia and 2) a large oral cancer screening trial.

CONCLUSION

Conducting an internal pilot study without adjusting the critical value can cause Type I error rate inflation in small samples, but not in large samples. An internal pilot approach usually achieves goal power and, for most studies with sample size greater than 50, requires no Type I error correction. Further, we have provided a flexible and accurate approach to bound Type I error below a goal level for studies with small sample size.

Options and Considerations for Adaptive Laboratory Experiments

Motivated by laboratory experiments that fail to reach significance, we developed a small sample size approach to designing a subsequent experiment that controls overall type I error and achieves sufficient conditional power. We focus on experiments with leukemia cells, and use a specific example in Chronic Lymphocytic Leukemia to discuss unanticipated patient variance and difficult to predict interaction effect sizes. We emphasize the importance of achieving significance in the first run of an experiment, which results in simplifying the multiple considerations usually associated with interim analysis and decision making in adaptive clinical trials. Within the context of combination testing for an adaptive laboratory experiment, we show that a range of reasonable options for the futility cut-off, effect size estimation, and significance level for the first run provide similar power and expected overall sample size. We contrast this approach to a naive procedure in which a second unplanned experiment is run based on non-significance in the first experiment, and data are combined as if they were obtained from one run.

Two-stage designs in bioequivalence trials

PURPOSE

The aim of this study is to assess the current status of non-fixed sample size designs in bioequivalence trials with a focus on two-stage adaptive approaches.

METHODS

We searched PubMed and Google Scholar from inception to October 2014. Regulatory guidelines were obtained from the public domain. Different methods were compared by Monte Carlo simulations for their impact on the patient's and producer's risks.

RESULTS

Add-on designs, group sequential designs and adaptive two-stage sequential designs are currently accepted to demonstrate bioequivalence in various regulations. All three approaches may inflate the patient's risk if applied inconsiderately. Direct transfer of methods developed for superiority testing to bioequivalence is not warranted. Published two-stage frameworks maintain the type I error and generally the desired power. Adaptation based on the observed T/R ratio observed in the first stage should be applied with caution. Monte Carlo simulations are an efficient tool to explore the operating characteristics of methods.

CONCLUSIONS

Validated two-stage frameworks can be applied without requiring the sponsor to perform own simulations-which could further improve power based on additional assumptions. Two-stage designs are both ethical and economical alternatives to fixed sample designs.

Connections between permutation and t-tests: relevance to adaptive methods

A permutation test assigns a p-value by conditioning on the data and treating the different possible treatment assignments as random. The fact that the conditional type I error rate given the data is controlled at level α ensures validity of the test even if certain adaptations are made. We show the connection between permutation and t-tests, and use this connection to explain why certain adaptations are valid in a t-test setting as well. We illustrate this with an example of blinded sample size recalculation.

Blinded sample size re-estimation in superiority and noninferiority trials: bias versus variance in variance estimation

The internal pilot study design allows for modifying the sample size during an ongoing study based on a blinded estimate of the variance thus maintaining the trial integrity. Various blinded sample size re-estimation procedures have been proposed in the literature. We compare the blinded sample size re-estimation procedures based on the one-sample variance of the pooled data with a blinded procedure using the randomization block information with respect to bias and variance of the variance estimators, and the distribution of the resulting sample sizes, power, and actual type I error rate. For reference, sample size re-estimation based on the unblinded variance is also included in the comparison. It is shown that using an unbiased variance estimator (such as the one using the randomization block information) for sample size re-estimation does not guarantee that the desired power is achieved. Moreover, in situations that are common in clinical trials, the variance estimator that employs the randomization block length shows a higher variability than the simple one-sample estimator and in turn the sample size resulting from the related re-estimation procedure. This higher variability can lead to a lower power as was demonstrated in the setting of noninferiority trials. In summary, the one-sample estimator obtained from the pooled data is extremely simple to apply, shows good performance, and is therefore recommended for application.

Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy

There is growing interest in how best to adapt and readapt treatments to individuals to maximize clinical benefit. In response, adaptive treatment strategies (ATS), which operationalize adaptive, sequential clinical decision making, have been developed. From a patient's perspective an ATS is a sequence of treatments, each individualized to the patient's evolving health status. From a clinician's perspective, an ATS is a sequence of decision rules that input the patient's current health status and output the next recommended treatment. Sequential multiple assignment randomized trials (SMART) have been developed to address the sequencing questions that arise in the development of ATSs, but SMARTs are relatively new in clinical research. This article provides an introduction to ATSs and SMART designs. This article also discusses the design of SMART pilot studies to address feasibility concerns, and to prepare investigators for a full-scale SMART. We consider an example SMART for the development of an ATS in the treatment of pediatric generalized anxiety disorders. Using the example SMART, we identify and discuss design issues unique to SMARTs that are best addressed in an external pilot study prior to the full-scale SMART. We also address the question of how many participants are needed in a SMART pilot study. A properly executed pilot study can be used to effectively address concerns about acceptability and feasibility in preparation for (that is, prior to) executing a full-scale SMART.

A power calculation guide for fMRI studies

In the past, power analyses were not that common for fMRI studies, but recent advances in power calculation techniques and software development are making power analyses much more accessible. As a result, power analyses are more commonly expected in grant applications proposing fMRI studies. Even though the software is somewhat automated, there are important decisions to be made when setting up and carrying out a power analysis. This guide provides tips on carrying out power analyses, including obtaining pilot data, defining a region of interest and other choices to help create reliable power calculations.

Effect of alendronate and vitamin D₃ on fractional calcium absorption in a double-blind, randomized, placebo-controlled trial in postmenopausal osteoporotic women

Menopause and increasing age are associated with a decrease in calcium absorption that can contribute to the pathogenesis of osteoporosis. We hypothesized that alendronate plus vitamin D(3) (ALN + D) would increase fractional calcium absorption (FCA). In this randomized, double-blind, placebo-controlled multicenter clinical trial, 56 postmenopausal women with 25-hydroxyvitamin D [25(OH)D] concentrations of 25 ng/mL or less and low bone mineral density (BMD) received 5 weekly doses of placebo or alendronate 70 mg plus vitamin D(3) 2800 IU (ALN + D). Calcium intake was stabilized to approximately 1200 mg/d prior to randomization. FCA was determined using a dual-tracer stable-calcium isotope method. FCA and 25(OH)D were similar between treatment groups at baseline (0.31 ± 0.12 ng/mL and 19.8 ± 4.7 ng/mL, respectively). After 1 month of treatment, subjects randomized to ALN + D experienced a significant least squares (LS) mean [95% confidence interval (CI)] increase in FCA [0.070 (0.042, 0.098)], whereas FCA did not change significantly in the placebo group [-0.016 (-0.044, 0.012)]. After ALN + D treatment, patients had higher 25(OH)D levels (LS mean difference 7.3 ng/mL, p < .001). The rise in serum 1,25-dihydroxyvitamin D(3) (p < .02) and parathyroid hormone (p < .001) were greater in the ALN + D group than in placebo-treated patients. ALN + D was associated with an increase in FCA of 0.07. To our knowledge, there is no other trial showing such a marked rise in calcium absorption owing to treatment with a bisphosphonate or owing to a small rise in 25(OH)D. This unique response of ALN + D is important for the treatment of osteoporosis, but the exact mechanism requires further study.

Understanding the FDA guidance on adaptive designs: historical, legal, and statistical perspectives

The recent Food and Drug Administration (FDA) guidance for industry on adaptive designs is perhaps one of the important undertakings by CDER/CBER Office of Biostatistics. Undoubtedly, adaptive designs may affect almost all phases of clinical development and impact nearly all aspects of clinical trial planning, execution and statistical inference. Thus, it is a significant accomplishment for the Office of Biostatistics to develop this well-thought-out and all-encompassing guidance document. In this paper, we discuss some critical topical issues of adaptive designs with supporting methodological work from either existing literature, additional technical notes, or accompanying papers. In particular, we provide numerous sources of design, conduct, analysis, and interpretation bias that arise from statistical procedures. We illustrate, as a result, and caution that substantial research is necessary for many adaptive designs to meet required scientific standards prior to their applications in clinical trials.

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating "nuisance" parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error - leading to better resource utilization - by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre-determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure-group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re-estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.

Optimal sampling in retrospective logistic regression via two-stage method

Case-control sampling is popular in epidemiological research because of its cost and time saving. In a logistic regression model, with limited knowledge on the covariance matrix of the point estimator of the regression coefficients a priori, there exists no fixed sample size analysis. In this study, we propose a two-stage sequential analysis, in which the optimal sample fraction and the required sample size to achieve a predetermined volume of a joint confidence set are estimated in an interim analysis. Additionally required observations are collected in the second stage according to the estimated optimal sample fraction. At the end of the experiment, data from these two stages are combined and analyzed for statistical inference. Simulation studies are conducted to justify the proposed two-stage procedure and an example is presented for illustration. It is found that the proposed two-stage procedure performs adequately in the sense that the resultant joint confidence set has a well-controlled volume and achieves the required coverage probability. Furthermore, the optimal sample fractions among all the selected scenarios are close to one. Hence, the proposed procedure can be simplified by always considering a balance design.

Statistical design of personalized medicine interventions: the Clarification of Optimal Anticoagulation through Genetics (COAG) trial

Background

There is currently much interest in pharmacogenetics: determining variation in genes that regulate drug effects, with a particular emphasis on improving drug safety and efficacy. The ability to determine such variation motivates the application of personalized drug therapies that utilize a patient's genetic makeup to determine a safe and effective drug at the correct dose. To ascertain whether a genotype-guided drug therapy improves patient care, a personalized medicine intervention may be evaluated within the framework of a randomized controlled trial. The statistical design of this type of personalized medicine intervention requires special considerations: the distribution of relevant allelic variants in the study population; and whether the pharmacogenetic intervention is equally effective across subpopulations defined by allelic variants.

Methods

The statistical design of the Clarification of Optimal Anticoagulation through Genetics (COAG) trial serves as an illustrative example of a personalized medicine intervention that uses each subject's genotype information. The COAG trial is a multicenter, double blind, randomized clinical trial that will compare two approaches to initiation of warfarin therapy: genotype-guided dosing, the initiation of warfarin therapy based on algorithms using clinical information and genotypes for polymorphisms in CYP2C9 and VKORC1; and clinical-guided dosing, the initiation of warfarin therapy based on algorithms using only clinical information.

Results

We determine an absolute minimum detectable difference of 5.49% based on an assumed 60% population prevalence of zero or multiple genetic variants in either CYP2C9 or VKORC1 and an assumed 15% relative effectiveness of genotype-guided warfarin initiation for those with zero or multiple genetic variants. Thus we calculate a sample size of 1238 to achieve a power level of 80% for the primary outcome. We show that reasonable departures from these assumptions may decrease statistical power to 65%.

Conclusions

In a personalized medicine intervention, the minimum detectable difference used in sample size calculations is not a known quantity, but rather an unknown quantity that depends on the genetic makeup of the subjects enrolled. Given the possible sensitivity of sample size and power calculations to these key assumptions, we recommend that they be monitored during the conduct of a personalized medicine intervention.

Trial Registration

clinicaltrials.gov: NCT00839657

Blinded sample size recalculation in multicentre trials with normally distributed outcome

The internal pilot study design enables to estimate nuisance parameters required for sample size calculation on the basis of data accumulated in an ongoing trial. By this, misspecifications made when determining the sample size in the planning phase can be corrected employing updated knowledge. According to regulatory guidelines, blindness of all personnel involved in the trial has to be preserved and the specified type I error rate has to be controlled when the internal pilot study design is applied. Especially in the late phase of drug development, most clinical studies are run in more than one centre. In these multicentre trials, one may have to deal with an unequal distribution of the patient numbers among the centres. Depending on the type of the analysis (weighted or unweighted), unequal centre sample sizes may lead to a substantial loss of power. Like the variance, the magnitude of imbalance is difficult to predict in the planning phase. We propose a blinded sample size recalculation procedure for the internal pilot study design in multicentre trials with normally distributed outcome and two balanced treatment groups that are analysed applying the weighted or the unweighted approach. The method addresses both uncertainty with respect to the variance of the endpoint and the extent of disparity of the centre sample sizes. The actual type I error rate as well as the expected power and sample size of the procedure is investigated in simulation studies. For the weighted analysis as well as for the unweighted analysis, the maximal type I error rate was not or only minimally exceeded. Furthermore, application of the proposed procedure led to an expected power that achieves the specified value in many cases and is throughout very close to it.

Sequential design approaches for bioequivalence studies with crossover designs

The planning of bioequivalence (BE) studies, as for any clinical trial, requires a priori specification of an effect size for the determination of power and an assumption about the variance. The specified effect size may be overly optimistic, leading to an underpowered study. The assumed variance can be either too small or too large, leading, respectively, to studies that are underpowered or overly large. There has been much work in the clinical trials field on various types of sequential designs that include sample size reestimation after the trial is started, but these have seen only little use in BE studies. The purpose of this work was to validate at least one such method for crossover design BE studies. Specifically, we considered sample size reestimation for a two-stage trial based on the variance estimated from the first stage. We identified two methods based on Pocock's method for group sequential trials that met our requirement for at most negligible increase in type I error rate.

Sample size re-estimation in clinical trials

Adaptive clinical trials are becoming very popular because of their flexibility in allowing mid-stream changes of sample size, endpoints, populations, etc. At the same time, they have been regarded with mistrust because they can produce bizarre results in very extreme settings. Understanding the advantages and disadvantages of these rapidly developing methods is a must. This paper reviews flexible methods for sample size re-estimation when the outcome is continuous.

A SAS macro for sample size adjustment and randomization test for internal pilot study

An unnecessarily high or inadequately low sample size often occurs in clinical trials if the planned variance of the trials is overestimated or underestimated. Internal pilot study which utilizes the information from the patients recruited up to interim stage can solve this problem well by re-estimating the variance and re-calculating the sample size. The trial may get a satisfactory power but the type I error rate may be inflated while the t-test is adopted to make hypothesis test because condition of t-distribution is not sufficed any more with variation of the sample size resulted from internal pilot design. If blind variance estimators of the internal pilot are used for sample size recalculation and randomization test is used to accomplish the final hypothesis test, not only the blindness of the internal pilot is preserved but also the ability to control the type I error rate is guaranteed. A SAS macro is programmed to simulate the process of sample size adjustment and randomization test.

Nonconventional clinical trial designs: approaches to provide more precise estimates of treatment effects with a smaller sample size, but at a cost

Statistical sciences have recently made advancements that allow improved precision or reduced sample size in clinical research studies. Herein, we review 4 of the more promising: (1) improvements in approaches for dose selection trials, (2) approaches for sample size adjustment, (3) selection of study end point and associated statistical methods, and (4) frequentist versus Bayesian statistical methods. Whereas each of these holds the opportunity for more efficient trials, each are associated with the need for more stringent assumptions or increased complexity in the interpretation of results. The opportunities for these promising approaches, and their associated "costs," are reviewed.

Two-Stage Sample Size Re-Estimation Based on a Nuisance Parameter: A Review

Sample size calculations are important and difficult in clinical trails because they depend on the nuisance parameter and treatment effect. Recently, much attention has been focused on two-stage methods whereby the first stage constitutes an internal pilot study used to estimate parameters and revise the final sample size. This paper reviews two-stage methods based on estimation of nuisance parameters in either a continuous or dichotomous outcome setting.

Assessment of the Gould-Shih procedure for sample size re-estimation

The power of a clinical trial is partly dependent upon its sample size. With continuous data, the sample size needed to attain a desired power is a function of the within-group standard deviation. An estimate of this standard deviation can be obtained during the trial itself based upon interim data; the estimate is then used to re-estimate the sample size. Gould and Shih proposed a method, based on the EM algorithm, which they claim produces a maximum likelihood estimate of the within-group standard deviation while preserving the blind, and that the estimate is quite satisfactory. However, others have claimed that the method can produce non-unique and/or severe underestimates of the true within-group standard deviation. Here the method is thoroughly examined to resolve the conflicting claims and, via simulation, to assess its validity and the properties of its estimates. The results show that the apparent non-uniqueness of the method's estimate is due to an apparently innocuous alteration that Gould and Shih made to the EM algorithm. When this alteration is removed, the method is valid in that it produces the maximum likelihood estimate of the within-group standard deviation (and also of the within-group means). However, the estimate is negatively biased and has a large standard deviation. The simulations show that with a standardized difference of 1 or less, which is typical in most clinical trials, the standard deviation from the combined samples ignoring the groups is a better estimator, despite its obvious positive bias.

Opening the adaptive toolbox

This is a discussion of the following three papers appearing in this special issue on adaptive designs: 'A regulatory view on adaptive/flexible clinical trial design' by H. M. James Hung, Robert T. O'Neill, Sue-Jane Wang and John Lawrence; 'Confirmatory clinical trials with an adaptive design' by Armin Koch; and 'FDA's critical path initiative: A perspective on contributions of biostatistics' by Robert T. O'Neill.

Sample Size Recalculation in Internal Pilot Study Designs: A Review

The adequacy of sample size is important to clinical trials. In the planning phase of a trial, however, the investigators are often quite uncertain about the sizes of parameters which are needed for sample size calculations. A solution to this problem is mid-course recalculation of the sample size during the ongoing trial. In internal pilot study designs, nuisance parameters are estimated on the basis of interim data and the sample size is adjusted accordingly. This review attempts to give an overview on the available methods. It is written not only for biometricians who are already familar with the the topic and wish to update their knowledge but also for users new to the subject.

Variance Estimation in Clinical Studies with Interim Sample Size Reestimation

We consider clinical studies with a sample size re-estimation based on the unblinded variance estimation at some interim point of the study. Because the sample size is determined in such a flexible way, the usual variance estimator at the end of the trial is biased. We derive sharp bounds for this bias. These bounds have a quite simple form and can help for the decision if this bias is negligible for the actual study or if a correction should be done. An exact formula for the bias is also provided. We discuss possibilities to get rid of this bias or at least to reduce the bias substantially. For this purpose, we propose a certain additive correction of the bias. We see in an example that the significance level of the test can be controlled when this additive correction is used.

Topical quinolone vs. antiseptic for treating chronic suppurative otitis media: a randomized controlled trial

OBJECTIVE

To compare a topical quinolone antibiotic (ciprofloxacin) with a cheaper topical antiseptic (boric acid) for treating chronic suppurative otitis media in children.

DESIGN

Randomized controlled trial.

SETTING AND PARTICIPANTS

A total of 427 children with chronic suppurative otitis media enrolled from 141 schools following screening of 39 841 schoolchildren in Kenya. Intervention Topical ciprofloxacin (n = 216) or boric acid in alcohol (n = 211); child-to-child treatment twice daily for 2 weeks.

MAIN OUTCOME MEASURES

Resolution of discharge (at 2 weeks for primary outcome), healing of the tympanic membrane, and change in hearing threshold from baseline, all at 2 and 4 weeks.

RESULTS

At 2 weeks, discharge was resolved in 123 of 207 (59%) children given ciprofloxacin, and in 65 of 204 (32%) given boric acid (relative risk 1.86; 95% CI 1.48-2.35; P < 0.0001). This effect was also significant at 4 weeks, and ciprofloxacin was associated with better hearing at both visits. No difference with respect to tympanic membrane healing was detected. There were significantly fewer adverse events of ear pain, irritation, and bleeding on mopping with ciprofloxacin than boric acid.

CONCLUSIONS

Ciprofloxacin performed better than boric acid and alcohol for treating chronic suppurative otitis media in children in Kenya.

Design and analysis of pilot studies: recommendations for good practice

Pilot studies play an important role in health research, but they can be misused, mistreated and misrepresented. In this paper we focus on pilot studies that are used specifically to plan a randomized controlled trial (RCT). Citing examples from the literature, we provide a methodological framework in which to work, and discuss reasons why a pilot study might be undertaken. A well-conducted pilot study, giving a clear list of aims and objectives within a formal framework will encourage methodological rigour, ensure that the work is scientifically valid and publishable, and will lead to higher quality RCTs. It will also safeguard against pilot studies being conducted simply because of small numbers of available patients.

The advantages and disadvantages of adaptive designs for clinical trials

Flexible designs for clinical trials permit mid-trial design modifications, which are based on interim information from inside or outside the trial while meeting (regulatory) requirements for the control of the type I error rate. The basic principle is to combine stage standardized test statistics such as p-values or z-scores in a pre-specified way. The flexibility covers changes of sample sizes, treatment allocation ratios and the number of interim analyses, as well as the selection of treatments, doses and end points. The price to be paid is that non-standard test statistics must be used after an adaptation.

Increasing the sample size when the unblinded interim result is promising

Increasing the sample size based on unblinded interim result may inflate the type I error rate and appropriate statistical adjustments may be needed to control the type I error rate at the nominal level. We briefly review the existing approaches which allow early stopping due to futility, or change the test statistic by using different weights, or adjust the critical value for final test, or enforce rules for sample size recalculation. The implication of early stopping due to futility and a simple modification to the weighted Z-statistic approach are discussed. In this paper, we show that increasing the sample size when the unblinded interim result is promising will not inflate the type I error rate and therefore no statistical adjustment is necessary. The unblinded interim result is considered promising if the conditional power is greater than 50 per cent or equivalently, the sample size increment needed to achieve a desired power does not exceed an upper bound. The actual sample size increment may be determined by important factors such as budget, size of the eligible patient population and competition in the market. The 50 per cent-conditional-power approach is extended to a group sequential trial with one interim analysis where a decision may be made at the interim analysis to stop the trial early due to a convincing treatment benefit, or to increase the sample size if the interim result is not as good as expected. The type I error rate will not be inflated if the sample size may be increased only when the conditional power is greater than 50 per cent. If there are two or more interim analyses in a group sequential trial, our simulation study shows that the type I error rate is also well controlled.

Properties of internal pilots with the univariate approach to repeated measures

Uncertainty surrounding the error covariance matrix often presents the biggest barrier to achieving accurate power analysis in the 'univariate' approach to repeated measures analysis of variance (UNIREP). A poor choice gives either an overpowered study which wastes resources, or an underpowered study with little chance of success. Internal pilot designs were introduced to resolve such uncertainty about error variance for t-tests. In earlier papers, we extended the use of internal pilots to any univariate linear model with fixed predictors and independent Gaussian errors. Here we further extend our exact and approximate results to UNIREP analysis. For a fixed treatment effect, the inaccuracy in a power calculation depends only on the ratio of the true variance to the value used for planning. The greater complexity of repeated measures requires generalizing misspecification of error variance to the misspecification of the eigenvalues of the error covariance. We recommend approximating the misspecification in terms of the first and second moments of the eigenvalues, for both fixed sample and internal pilot designs. We also describe an unadjusted approach for internal pilots with repeated measures. Simulations illustrate the fact that both positive and negative properties in the univariate setting extend to repeated measures analysis. In particular, internal pilots allow maintaining power or reducing expected sample size when the covariance matrix used for planning differs from the true value. However, an unadjusted approach can inflate test size, at least with small to moderate sample sizes. Hence new, adjusted methods must be developed for small samples. At this time, we caution against using an internal pilot design with repeated measures without first conducting simulations to document the amount of test size inflation possible for the conditions of interest.