Stopping a clinical trial early: frequentist and Bayesian approaches applied to a CALGB trial in non-small-cell lung cancer

Bayesian clinical trials at The University of Texas MD Anderson Cancer Center: An update

BACKGROUND/AIMS

In our 2009 article, we showed that Bayesian methods had established a foothold in developing therapies in our institutional oncology trials. In this article, we will document what has happened since that time. In addition, we will describe barriers to implementing Bayesian clinical trials, as well as our experience overcoming them.

METHODS

We reviewed MD Anderson Cancer Center clinical trials submitted to the institutional protocol office for scientific and ethical review between January 2009 and December 2013, the same length time period as the previous article. We tabulated Bayesian methods implemented for design or analyses for each trial and then compared these to our previous findings.

RESULTS

Overall, we identified 1020 trials and found that 283 (28%) had Bayesian components so we designated them as Bayesian trials. Among MD Anderson-only and multicenter trials, 56% and 14%, respectively, were Bayesian, higher rates than our previous study. Bayesian trials were more common in phase I/II trials (34%) than in phase III/IV (6%) trials. Among Bayesian trials, the most commonly used features were for toxicity monitoring (65%), efficacy monitoring (36%), and dose finding (22%). The majority (86%) of Bayesian trials used non-informative priors. A total of 75 (27%) trials applied Bayesian methods for trial design and primary endpoint analysis. Among this latter group, the most commonly used methods were the Bayesian logistic regression model (N = 22), the continual reassessment method (N = 20), and adaptive randomization (N = 16). Median institutional review board approval time from protocol submission was the same 1.4 months for Bayesian and non-Bayesian trials. Since the previous publication, the Biomarker-Integrated Approaches of Targeted Therapy for Lung Cancer Elimination (BATTLE) trial was the first large-scale decision trial combining multiple treatments in a single trial. Since then, two regimens in breast cancer therapy have been identified and published from the cooperative Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging and Molecular Analysis (I-SPY 2), enhancing cooperation among investigators and drug developers across the nation, as well as advancing information needed for personalized medicine. Many software programs and Shiny applications for Bayesian trial design and calculations are available from our website which has had more than 21,000 downloads worldwide since 2004.

CONCLUSION

Bayesian trials have the increased flexibility in trial design needed for personalized medicine, resulting in more cooperation among researchers working to fight against cancer. Some disadvantages of Bayesian trials remain, but new methods and software are available to improve their function and incorporation into cancer clinical research.

A comparison of two worlds: How does Bayes hold up to the status quo for the analysis of clinical trials?

BACKGROUND

There is a paucity of literature comparing Bayesian analytic techniques with traditional approaches for analyzing clinical trials using real trial data.

METHODS

We compared Bayesian and frequentist group sequential methods using data from two published clinical trials. We chose two widely accepted frequentist rules, O'Brien-Fleming and Lan-DeMets, and conjugate Bayesian priors. Using the nonparametric bootstrap, we estimated a sampling distribution of stopping times for each method. Because current practice dictates the preservation of an experiment-wise false positive rate (Type I error), we approximated these error rates for our Bayesian and frequentist analyses with the posterior probability of detecting an effect in a simulated null sample. Thus for the data-generated distribution represented by these trials, we were able to compare the relative performance of these techniques.

RESULTS

No final outcomes differed from those of the original trials. However, the timing of trial termination differed substantially by method and varied by trial. For one trial, group sequential designs of either type dictated early stopping of the study. In the other, stopping times were dependent upon the choice of spending function and prior distribution.

CONCLUSIONS

Results indicate that trialists ought to consider Bayesian methods in addition to traditional approaches for analysis of clinical trials. Though findings from this small sample did not demonstrate either method to consistently outperform the other, they did suggest the need to replicate these comparisons using data from varied clinical trials in order to determine the conditions under which the different methods would be most efficient.

Bayesian design of single-arm phase II clinical trials with continuous monitoring

BACKGROUND

Bayesian designs are increasingly used to conduct phase II clinical trials. However, stopping boundaries in most Bayesian designs are defined from posterior credible intervals. The use of designs based on posterior credible intervals results in a loss of efficiency when compared to formal stopping rules based on Bayesian hypothesis tests. Such designs also introduce an unnecessary element of subjectivity in the interpretation of trial results.

METHODS

We derive a new class of Bayesian designs based on formal hypothesis tests. The prior densities used to define the alternative hypotheses in these tests assign no mass to parameter values that are consistent with the null hypotheses and are called nonlocal alternative prior densities.

RESULTS

We show that Bayesian designs based on hypothesis tests and nonlocal alternative prior densities are more efficient than common Bayesian designs based on posterior credible intervals and common frequentist designs. In contrast to trial designs based on Bayesian credible intervals, we demonstrate that the mis-specification of the prior densities used to describe the anticipated effect of the experimental treatment in designs based on hypothesis tests cannot increase the expected weight of evidence in favor of the trial agent.

LIMITATIONS

Extension of test-based designs to phase I-II designs and randomized phase II designs remains an open research question.

CONCLUSIONS

Phase II single-arm trials designed using Bayesian hypothesis tests with nonlocal alternatives provide better operating characteristics, use fewer patients per correct decision, and provide more directly interpretable results than other commonly used Bayesian and frequentist designs. Because the mis-specification of the prior density on the effect of the experimental agent decreases the expected weight of evidence that is collected in favor of the experimental treatment, the use of Bayesian hypothesis tests to design clinical trials also eliminates a potential source of bias often associated with trials designed using posterior credible intervals.

Release of data from an ongoing randomized clinical trial for sample size adjustment or planning

The determination of an appropriate sample size is a key issue in planning and designing randomized clinical trials. In settings with time-to-event or binary outcomes, the required sample size depends on the control-arm event (response) rate. An accurate estimate of this rate is not often available at the planning stage. Therefore, non-comparative control-arm or pooled-arm event rates from an ongoing trial are sometimes released for sample size adjustment or planning purposes. It is shown that such non-comparative data release may still contain information on the relative treatment benefit and may thus adversely affect the ongoing trial. A simple approach to minimizing the effect of the data release is suggested.

Bayesian statistics in medicine: a 25 year review

This review examines the state of Bayesian thinking as Statistics in Medicine was launched in 1982, reflecting particularly on its applicability and uses in medical research. It then looks at each subsequent five-year epoch, with a focus on papers appearing in Statistics in Medicine, putting these in the context of major developments in Bayesian thinking and computation with reference to important books, landmark meetings and seminal papers. It charts the growth of Bayesian statistics as it is applied to medicine and makes predictions for the future. From sparse beginnings, where Bayesian statistics was barely mentioned, Bayesian statistics has now permeated all the major areas of medical statistics, including clinical trials, epidemiology, meta-analyses and evidence synthesis, spatial modelling, longitudinal modelling, survival modelling, molecular genetics and decision-making in respect of new technologies.

Use of a Bayesian approach to decide when to stop a therapeutic trial: the case of a chemoprophylaxis trial in human immunodeficiency virus infection

From 1996 to 1998, a phase III, placebo-controlled, therapeutic trial was conducted in Abidjan, Ivory Coast, to assess the efficacy of cotrimoxazole prophylaxis in reducing severe morbidity in adults at early stages of human immunodeficiency virus infection. The authors used the real data from this trial to simulate three Bayesian interim analyses. Three prior distributions were considered: a noninformative one, a skeptical one, and one based on external information. The posterior distribution was calculated by using directed acyclic graphs and Gibbs sampling. This Bayesian approach showed different results according to the prior distribution chosen. Although use of the noninformative prior would have led to stopping the trial at the same time that the frequentist approach would have, the skeptical prior would have led to continuing it, and the prior based on external data would have led to stopping it 1 year earlier.

Comparison of the dose-response relationships of 2 lipid-lowering agents: a Bayesian meta-analysis

BACKGROUND

Comparing the dose-response of a new drug to that of a previously studied drug can aid in understanding their relative potencies. Two dose-finding studies addressed the effect of a new drug, rosuvastatin, on its ability to decrease low-density lipoprotein cholesterol (LDL-C) levels. One of these studies included 2 doses of atorvastatin, and substantial additional information is available in the literature about the effect of atorvastatin on LDL-C level lowering.

METHODS

The 2 dose-finding studies of rosuvastatin considered otherwise healthy patients who had hypercholesterolemia. Comparable studies of atorvastatin were identified via a MEDLINE search in December 1999. Multiple reviewer consensus identified 15 of 41 studies on atorvastatin published since 1996 that met these selection criteria: reporting of LDL-C level change from baseline at least 6 weeks after treatment initiation, doses administered, and treatment group sizes. Eligible populations had clinical evidence of hypercholesterolemia. We excluded studies with patients who had severe illness or a previous history of transplantation. Data extraction of the mean, sample sizes, and SDs (or CIs) by dose was carried out independently by multiple reviewers. We combined the results from the various studies with Bayesian hierarchical modeling and analyzed them with Markov chain Monte Carlo techniques.

RESULTS

Combining this study and literature results substantially increased the power to compare the dose-response relationships of rosuvastatin and atorvastatin. Rosuvastatin reduced LDL-C level by an estimated 10 to 17 percentage points more than atorvastatin when both were given at the same dose. Approximately one quarter of the dose of rosuvastatin achieved about the same magnitude of LDL-C level reduction as atorvastatin at dosages as high as 80 mg. This finding does not imply a 4-fold difference in efficacy overall and specifically does not describe the results at higher dosage levels.

CONCLUSIONS

Bayesian meta-analysis of results from related studies allows the comparison of the dose-response relationships of 2 drugs, better estimates of a particular dose-response relationship within an individual study, and the expression of relative benefits (of dose and drug) in terms of probabilities. Explicitly comparing a study's results with historical data using Bayesian meta-analysis allows clinicians to view the study in the larger context of medical research.