Sample size re-estimation in an on-going NIH-sponsored clinical trial: the secondary prevention of small subcortical strokes experience

BACKGROUND AND PURPOSE

When planning clinical trials, decisions regarding sample size are often based on educated guesses of parameters, which may in fact prove to be over- or under-estimates. For example, after initiation of the SPS3 study, published data indicated that the recurrent stroke rates might be lower than initially planned for the study. Failure to account for this could result in an under-powered study. Thus, we performed a sample size re-estimation, and describe the experience herein.

METHODS

We evaluated different scenarios based on a re-estimated overall event rate, including increasing the sample size and increasing the follow-up time, to determine their impact on both type I error and the power to detect the initially planned treatment difference.

RESULTS

We found that by increasing the sample size from 2500 to 3000 and by following the patients for one year after the end of recruitment, we would maintain our planned type I error rate, and increase the power to detect the prespecified clinically meaningful difference to between 67% and 87%, depending on the rate of recruitment.

CONCLUSIONS

We successfully implemented this unplanned design modification in the SPS3 study, in order to allow for sufficient power to detect the planned treatment differences.

CLINICAL TRIALS REGISTRATION INFORMATION

Clinical Trials Registration - http://clinicaltrials.gov/show/NCT00059306. Unique identifier: NCT00059306.

Systems pharmacology, pharmacogenetics, and clinical trial design in network medicine

The rapidly growing disciplines of systems biology and network science are now poised to meet the fields of clinical medicine and pharmacology. Principles of systems pharmacology can be applied to drug design and, ultimately, testing in human clinical trials. Rather than focusing exclusively on single drug targets, systems pharmacology examines the holistic response of a phenotype-dependent pathway or pathways to drug perturbation. Knowledge of individual pharmacogenetic profiles further modulates the responses to these drug perturbations, moving the field toward more individualized ('personalized') drug development. The speed with which the information required to assess these system responses and their genomic underpinnings is changing and the importance of identifying the optimal drug or drug combinations for maximal benefit and minimal risk require that clinical trial design strategies be adaptable. In this paper, we review the tenets of adaptive clinical trial design as they may apply to an era of expanding knowledge of systems pharmacology and pharmacogenomics, and clinical trail design in network medicine.

Response-adaptive designs for continuous treatment responses in phase III clinical trials: A review

A variety of response-adaptive randomization procedures have been proposed in literature assuming binary outcomes. However, the list is not so long for continuous outcomes though many real clinical trials deal with continuous treatment responses. In this paper, we attempt to explore the available procedures together with a comparison of their performances. Some real-life adaptive trial is also reviewed.

Adaptive trial design: could we use this approach to improve clinical trials in the field of global health?

We need more clinical trials in the world's poorest regions to evaluate new drugs and vaccines, and also to find better ways to manage health issues. Clinical trials are expensive, time consuming, and cumbersome. However, in wealthier regions these limiting factors are being addressed to make trials less administrative and improve the designs. A good example is adaptive trial design. This innovation is becoming accepted by the regulators and has been taken up by the pharmaceutical industry to reduce product development times and costs. If this approach makes trials easier and less expensive surely we should be implementing this approach in the field of tropical medicine and international health? As yet this has rarely been proposed and there are few examples. There is a need for raising the awareness of these design approaches because they could be used to make dramatic improvements to clinical research in developing countries.

A "SMART" design for building individualized treatment sequences

Interventions often involve a sequence of decisions. For example, clinicians frequently adapt the intervention to an individual's outcomes. Altering the intensity and type of intervention over time is crucial for many reasons, such as to obtain improvement if the individual is not responding or to reduce costs and burden when intensive treatment is no longer necessary. Adaptive interventions utilize individual variables (severity, preferences) to adapt the intervention and then dynamically utilize individual outcomes (response to treatment, adherence) to readapt the intervention. The Sequential Multiple Assignment Randomized Trial (SMART) provides high-quality data that can be used to construct adaptive interventions. We review the SMART and highlight its advantages in constructing and revising adaptive interventions as compared to alternative experimental designs. Selected examples of SMART studies are described and compared. A data analysis method is provided and illustrated using data from the Extending Treatment Effectiveness of Naltrexone SMART study.

Adaptive methods: when and how should they be used in clinical trials?

Adaptive clinical trial designs are defined as designs that use data cumulated during trial to possibly modify certain aspects without compromising the validity and integrity of the said trial. Compared to more traditional trials, in theory, adaptive designs allow the same information to be generated but in a more efficient manner. The advantages and limits of this type of design together with the weight of the constraints, in particular of a logistic nature, that their use implies, differ depending on whether the trial is exploratory or confirmatory with a view to registration. One of the key elements ensuring trial integrity is the involvement of an independent committee to determine adaptations in terms of experimental design during the study. Adaptive methods for clinical trials are appealing and may be accepted by the relevant authorities. However, the constraints that they impose must be determined well in advance.

Confounding due to changing background risk in adaptively randomized trials

BACKGROUND

While adaptive trials tend to improve efficiency, they are also subject to some unique biases.

PURPOSE

We address a bias that arises from adaptive randomization in the setting of a time trend in disease incidence.

METHODS

We use a potential-outcome model and directed acyclic graphs to illustrate the bias that arises from a changing subject allocation ratio with a concurrent change in background risk.

RESULTS

In a trial that uses adaptive randomization, time trends in risk can bias the crude effect estimate obtained by naively combining the data from the different stages of the trial. We illustrate how the bias arises from an interplay of departures from exchangeability among groups and the changing randomization proportions.

LIMITATIONS

We focus on risk-ratio and risk-difference analysis.

CONCLUSIONS

Analysis of trials using adaptive randomization should involve attention to or adjustment for possible trends in background risk. Numerous modeling strategies are available for that purpose, including stratification, trend modeling, inverse-probability-of-treatment weighting, and hierarchical regression.

From trial and error to trial simulation III: a framework for interim analysis in efficacy trials with antidepressant drugs

Clinical trials with antidepressant drugs often fail to detect drug effect, even with drugs that are known to be efficacious. In a previous publication, we showed that a model-based approach is required to address some of the existing challenges in the design of clinical trial protocols. Here, we illustrate how the implementation of an interim analysis (IA) may help to identify studies that are headed for failure, early in the trial before completion of treatment. In contrast to traditional IA procedures, an adaptive Bayesian approach is proposed to optimize the timing of analysis and decision criteria for futility and efficacy, taking into account enrollment rate and treatment response at intermediate visits in the trial. Validation procedures involving re-enrollment of patients confirmed the performance of the method. Our findings reveal that optimization of the timing and decision criteria at the interim stage is critical for the accuracy of the conclusions about treatment efficacy or futility.

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating "nuisance" parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error - leading to better resource utilization - by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre-determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure-group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re-estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.

An introduction to adaptive designs and adaptation in CNS trials

Adaptive designs learn from accumulating trial data in real time and apply this knowledge to optimize subsequent study execution. A set of design rules define a priori which modifications may be incorporated into the trial design. Judicious use of adaptive designs may increase the information value per resource unit invested by avoiding allocation of patients to non-efficacious/unsafe therapies and allowing stopping decisions to be made at the earliest possible time point. Ultimately this may accelerate the development of promising therapies.

Bayesian adaptive dose-finding studies with delayed responses

In recent years, Bayesian response-adaptive designs have been used to improve the efficiency of learning in dose-finding studies. Many current methods for analyzing the data at the time of the interim analysis only use the data from patients who have completed the study. Therefore, data collected at intermediate time points are not used for decision making in these studies. However, in some disease areas such as diabetes and obesity, patients may need to be studied for several weeks or months for a drug effect to emerge. Additionally, slow enrollment rates can limit the number of patients who complete the study in a given period of time. Consequently, at the time of an interim analysis, there may be only a small proportion (e.g., 20%) of patients who have completed the study. In this paper, we propose a new Bayesian prediction model to incorporate all the data (from patients who have completed the study and those who have not completed) to make decisions about the study at the interim analysis. Examples of decisions made at the interim analysis include adaptive treatment allocation, dropping nonefficacious dose arms, stopping the study for positive efficacy, and stopping the study for futility. The model is able to handle incomplete longitudinal data including missing data considered missing at random (MAR). A utility-function-based decision rule is also discussed. The benefit of our new method is demonstrated through trial simulations. Three scenarios are examined, and the simulation results demonstrate that this new method outperforms traditional design with the same sample size in each of these scenarios.

The dual test: safeguarding p-value combination tests for adaptive designs

Many modern adaptive designs apply an analysis where p-values from different stages are weighted together to an overall hypothesis test. One merit of this combination approach is that the design can be made very flexible. However, combination tests violate the sufficiency and conditionality principles. As a consequence, combination tests may lead to absurd conclusions, such as 'proving' a positive effect while the average effect is negative. We explore the possibility of modifying the test so that such illogical conclusions are no longer possible. The dual test requires both the weighted combination test and a naïve test, ignoring the adaptations, to be statistically significant. The result is that the flexibility and type I error level control of the combination test are preserved, while the naïve test adds a safeguard against unconvincing results. The dual test is, by construction, at least as conservative as the combination test. However, many design changes will not lead to any power loss. A typical situation where the combination approach can be used is two-stage sample size reestimation (SSR). For this case, we give a complete specification of all sample size modifications for which the two tests are equally powerful. We also study the overall power loss for some suggested SSR rules. Rules based on conditional power generally lead to ignorable power loss while a decision analytic approach exhibits clear discrepancies between the two tests.

A Bayesian dose-finding trial with adaptive dose expansion to flexibly assess efficacy and safety of an investigational drug

BACKGROUND

Adaptive dose-ranging trials are more efficient than traditional approaches and may be designed to explicitly address the goals and decisions inherent in learn-phase drug development. We report the design, implementation, and outcome of an innovative Bayesian, response-adaptive, dose-ranging trial of an investigational drug in patients with diabetes, incorporating a dose expansion approach to flexibly address both efficacy and safety.

PURPOSE

The design was developed to assess whether one or more doses of an investigational drug demonstrated superior efficacy to an active control while maintaining an acceptable safety profile.

METHODS

The trial used a two-stage design, in which patients were initially allocated equally to placebo, investigational drug at a low and a medium dose, and an active control. Movement to the second stage was contingent upon evidence of efficacy (measured by change in fasting blood glucose) to add a very low dose of the investigational drug and of safety (measured by weight gain) to add a high dose of the investigational drug. The design incorporated a longitudinal model to maximize use of incomplete data, predictive probabilities to guide the decisions to terminate the trial for futility or move on to Stage 2, and a dose-response model in Stage 2 to borrow information across adjacent doses. Extensive simulations were used to fine tune trial parameters, to define operating characteristics, and to determine the required sample sizes. A data monitoring committee was provided with frequent reports to aid in trial oversight.

RESULTS

In Stage 1, as trial data accrued, the predictive probability that either the low or medium dose of the investigational drug was superior to the active control fell to low values. Stage 1 termination was recommended after 199 subjects were randomized, out of a maximum trial size of 500 subjects, and the final sample size was 218. Thus the trial did not progress to Stage 2.

LIMITATIONS

Because of the relatively narrow dose range to be assessed, and the inability to utilize the highest dose at the beginning of the trial, a fully responsive-adaptive design incorporating dose-response modeling was not considered a viable option. This limited the efficiency gains possible with a full set of adaptive design elements.

CONCLUSIONS

The two-stage dose-expansion design functioned as designed, recommending early termination based on a low probability that the tested doses had efficacy greater than the active control.

Design of adaptive two-stage double-arm clinical trials for dichotomous variables

It is well known that flexibility is one of the major advantages of an adaptive two-stage design, and the intended adaptation should be as preplanned as possible to maintain the integrity of the clinical trial. The design of adaptive two-stage double-arm clinical trials for dichotomous variables was proposed by simulation and forecasting procedure at the planning stage. To further ensure the integrity of the clinical trial, the sample size scheme for each scenario, which was supposed to be based on the first stage, was provided in the protocol by Monte Carlo simulation. In addition, the study parameters were determined by comparing the assessment indexes such as total sample size, expected sample size and the test power at the first stage. Furthermore, Fisher's combination test and pooled data analysis were considered and compared through the simulation. The latter, which has the larger overall power and the better overall type I error control, with the same sample size was adopted for further simulation and statistical analysis in the clinical trial.

Adaptive designs for randomized trials in public health

In this article, we present a discussion of two general ways in which the traditional randomized trial can be modified or adapted in response to the data being collected. We use the term adaptive design to refer to a trial in which characteristics of the study itself, such as the proportion assigned to active intervention versus control, change during the trial in response to data being collected. The term adaptive sequence of trials refers to a decision-making process that fundamentally informs the conceptualization and conduct of each new trial with the results of previous trials. Our discussion below investigates the utility of these two types of adaptations for public health evaluations. Examples are provided to illustrate how adaptation can be used in practice. From these case studies, we discuss whether such evaluations can or should be analyzed as if they were formal randomized trials, and we discuss practical as well as ethical issues arising in the conduct of these new-generation trials.

Adaptive clinical trials: progress and challenges

Adaptive designs promise the flexibility to redesign clinical trials at interim stages. This flexibility would provide greater efficiency in drug development. However, despite this promise, many hesitate to implement such designs. Here we explore three possible reasons for the hesitation: (i) confusion with respect to the definition of an 'adaptive design'; (ii) controversy surrounding the use of sample size re-estimation methods; and (iii) logistical barriers that must be overcome in order to use adaptive designs within existing trial frameworks.The large volume of recent work has created confusion with respect to the definition of an 'adaptive design'. Unfortunately, this has resulted in reduced usage of many acceptable methods because of guilt by association with the more controversial methods. This review attempts to clarify the differences among many common types of proposed adaptive designs. Once the differences are noted, it becomes apparent that some adaptive designs are well accepted while others remain very controversial. In fact, much of the controversy and criticism surrounding adaptive designs has focused on their use for sample size re-estimation. Hence, this review also examines the different types of adaptive designs for sample size re-estimation in order to clarify the controversy surrounding the use of these methods. Specifically, separating the controversial from good practice requires clarifying differences between adaptive designs with sample size re-estimation based on a revised treatment effect and re-estimation based only on nuisance parameters (internal pilot designs). Finally, many logistical barriers must be overcome in order to use adaptive designs within existing trial frameworks.If the promise of adaptive designs is to be achieved, it will be important to bring together large groups of individuals from funding sources and regulatory agencies to address these limitations. Very few discussions of these issues have appeared in journals that are targeted to clinical audiences. In fact, current use of adaptive designs is not really hindered by the lack of statistical methods to accommodate the adaptations. Rather, there is a need for education as to which adaptive designs are acceptable and which are not acceptable. These discussions will require the involvement of many individuals outside the statistical community. In this review, we summarize the existing methods and current controversies with the intent of providing a clarification that will enable these individuals to participate in these much-needed discussions.

Group sequential and adaptive designs - a review of basic concepts and points of discussion

In recent times, group sequential and adaptive designs for clinical trials have attracted great attention from industry, academia and regulatory authorities. These designs allow analyses on accumulating data - as opposed to classical, "fixed-sample" statistics. The rapid development of a great variety of statistical procedures is accompanied by a lively debate on their potential merits and shortcomings. The purpose of this review article is to ease orientation in both respects. First, we provide a concise overview of the essential technical concepts, with special emphasis on their interrelationships. Second, we give a structured review of the current controversial discussion on practical issues, opportunities and challenges of these new designs.

Sequential analysis applied to clinical trials in dentistry: a systematic review

Clinical trials employ sequential analysis for the ethical and economic benefits it brings. In dentistry, as in other fields, resources are scarce and efforts are made to ensure that patients are treated ethically. The objective of this systematic review was to characterise the use of sequential analysis for clinical trials in dentistry. We searched various databases from 1900 through to January 2008. Articles were selected for review if they were clinical trials in the field of dentistry that had applied some form of sequential analysis. Selection was carried out independently by two of the authors. We included 18 trials from various specialties, which involved many different interventions. We conclude that sequential analysis seems to be underused in this field but that there are sufficient methodological resources in place for future applications.Evidence-Based Dentistry (2008) 9, 55-62. doi:10.1038/sj.ebd.6400587.

Combining proof-of-concept with dose-finding: utilization of adaptive designs in migraine clinical trials

There is an obvious need to improve clinical trial designs with respect to efficiency, duration and the number of patients recruited. Adaptive (flexible) designs may be valuable in this respect. We simulated the properties of a two-stage adaptive proof-of-concept and dose-finding trial design in adult migraine patients with moderate to severe headache, with or without aura. We also assessed the usefulness of a combined Bayesian and frequentist approach in the estimation of the probability of success of subsequent Phase III studies. Applying such an innovative approach would result in a reduction of the required sample size by 30 patients and no prolongation of the trial duration. The probability of success in Phase III is > 81%. An innovative adaptive design can facilitate testing of investigational migraine medications by reducing patient numbers and improving predictivity of success in Phase III.

Adaptive designs in clinical drug development: opportunities, challenges, and scope reflections following PhRMA's November 2006 workshop

This paper provides reflections on the opportunities, scope and challenges of adaptive design as discussed at PhRMA's workshop held in November 2006. We also provide a status report of workstreams within PhRMA's working group on adaptive designs, which were triggered by the November workshop. Rather than providing a comprehensive review of the presentations given, we limit ourselves to a selection of key statements. The authors reflect the position of PhRMA's working group on adaptive designs.

Flexible design and efficient implementation of adaptive dose-finding studies

A dose-finding study with an adaptive design generates three computational problems: fitting the dose-response curve given the current data, identifying the dose to be given to the next patient that is optimal for learning about the dose-response curve, and pretrial simulation in order to establish operating characteristics of alternative designs. Identifying the 'optimal' dose is the rate-limiting step since conventional methods, estimating the full posterior predictive distribution of some utility function under each of the possible doses, are very slow. We explore a simpler strategy based on importance sampling, whereby the posterior mean of the utility at each candidate dose is estimated by taking its average across an empirical distribution for the model parameters from the current Markov chain Monte Carlo (MCMC) run, weighted according to the likelihood of one or more predicted observations. We identify appropriate settings for this algorithm and illustrate its application in the context of a normal dynamic linear model used in a dose-finding clinical trial of a neutrophil inhibitory factor in acute ischaemic stroke.

Multiplicity and flexibility in clinical trials

Flexible designs offer a large amount of flexibility in clinical trials with control of the type I error rate. This allows the combination of trials from different clinical phases of a drug development process. Such combinations require designs where hypotheses are selected and/or added at interim analysis without knowing the selection rule in advance so that both flexibility and multiplicity issues arise. The paper reviews the basic principles and some of the common methods for reaching flexibility while controlling the family-wise error rate in the strong sense. Flexible designs have been criticized because they may lead to different weights for the patients from the different stages when reassessing sample sizes. Analyzing the data in a conventional way avoids such unequal weighting but may inflate the multiple type I error rate. In cases where the conditional type I error rates of the new design (and conventional analysis) are below the conditional type I error rates of the initial design the conventional analysis may, however, be done without inflating the type I error rate. Focusing on a parallel group design with two treatments and a common control, we use this principle to investigate when we can select one treatment, reassess sample sizes and test the corresponding null hypotheses by the conventional level alpha z-test without compromising on the multiple type I error rate.