A case for Bayesianism in clinical trials

QRISK3 underestimates the risk of cardiovascular events in patients with COPD

BACKGROUND

Patients with chronic obstructive pulmonary disease (COPD) are at increased risk of cardiovascular disease (CVD). The extent to which the excess CVD risk is captured by risk factors in QRISK, a widely used CVD risk scoring tool, is not well studied.

METHODS

We created an incidence cohort of diagnosed COPD patients from the United Kingdom (UK) Clinical Practice Research Datalink GOLD database (January 1998-July 2018). The outcome was a composite of fatal or non-fatal CVD events. Sex-specific age-standardised incidence ratios (SIR) were compared with values for the UK primary-care population. The observed 10-year CVD risk was derived using the Kaplan-Meier estimator and was compared with predicted 10-year risk from the QRISK3 tool.

RESULTS

13 208 patients (mean age 64.9 years, 45% women) were included. CVD incidence was 3.53 events per 100 person-years. The SIR of CVD was 1.71 (95% CI 1.61 to 1.75) in women and 1.62 (95%CI 1.54-1.64) in men. SIR was particularly high among patients younger than 65 years (women=2.13 (95% CI 1.94 to 2.19); men=1.86 (95% CI 1.74 to 1.90)). On average, the observed 10-year risk was 52% higher than QRISK predicted score (33.5% vs 22.1%). The difference was higher in patients younger than 65 years (observed risk 82% higher than predicted).

CONCLUSION

People living with COPD are at a significantly heightened risk of CVD over and beyond their predicted risk. This is particularly the case for younger people whose 10-year CVD risk can be >80% higher than predicted. Risk scoring tools must be validated and revised to provide accurate CVD predictions in patients with COPD.

Interpreting a Bayesian phase II futility clinical trial

Background

A resurgence of research into phase II trial design in the mid-2000s led to the use of futility designs in a wide variety of disease areas. Phase II futility studies differ from efficacy studies in that their null hypothesis is that treatment, relative to control, does not meet or exceed the level of benefit required to justify additional study. A rejection of the null hypothesis indicates that the treatment should not proceed to a larger confirmatory trial.

Methods

Bayesian approaches to the design of phase II futility clinical trials are presented and allow for the quantification of key probabilities, such as the predictive probability of current trial success or even the predictive probability of a future trial’s success.

Results

We provide an illustration of the design and interpretation of a phase II futility study constructed in a Bayesian framework. We focus on the operating characteristics of our motivating trial based on a simulation study, as well as the general interpretation of trial outcomes, type I, and type II errors in this framework.

Conclusions

Phase II futility clinical trials, when designed under in a Bayesian framework, offer an alternative approach to the design of mid-phase studies which provide unique benefits relative to trials designed in a frequentist framework and designs which focus on treatment efficacy.

One view of the next decade of research on behavioral and biobehavioral approaches to cancer prevention and control: intervention optimization

As a new decade begins, we propose that the time is right to reexamine current methods and procedures and look for opportunities to accelerate progress in cancer prevention and control. In this article we offer our view of the next decade of research on behavioral and biobehavioral interventions for cancer prevention and control. We begin by discussing and questioning several implicit conventions. We then briefly introduce an alternative research framework: the multiphase optimization strategy (MOST). MOST, a principled framework for intervention development, optimization, and evaluation, stresses not only intervention effectiveness, but also intervention affordability, scalability, and efficiency. We review some current limitations of MOST along with future directions for methodological work in this area, and suggest some changes in the scientific environment we believe would permit wider adoption of intervention optimization. We propose that wider adoption of intervention optimization would have a positive impact on development and successful implementation of interventions for cancer prevention and control and on intervention science more broadly, including accumulation of a coherent base of knowledge about what works and what does not; establishment of an empirical basis for adaptation of interventions to different settings with different levels and types of resources; and, in the long run, acceleration of progress from Stage 0 to Stage V in the National Institutes of Health Model of Stages of Intervention Development.

Bayesian adaptive design of early-phase clinical trials for precision medicine based on cancer biomarkers

Cancer tissue samples obtained via biopsy or surgery were examined for specific gene mutations by genetic testing to inform treatment. Precision medicine, which considers not only the cancer type and location, but also the genetic information, environment, and lifestyle of each patient, can be applied for disease prevention and treatment in individual patients. The number of patient-specific characteristics, including biomarkers, has been increasing with time; these characteristics are highly correlated with outcomes. The number of patients at the beginning of early-phase clinical trials is often limited. Moreover, it is challenging to estimate parameters of models that include baseline characteristics as covariates such as biomarkers. To overcome these issues and promote personalized medicine, we propose a dose-finding method that considers patient background characteristics, including biomarkers, using a model for phase I/II oncology trials. We built a Bayesian neural network with input variables of dose, biomarkers, and interactions between dose and biomarkers and output variables of efficacy outcomes for each patient. We trained the neural network to select the optimal dose based on all background characteristics of a patient. Simulation analysis showed that the probability of selecting the desirable dose was higher using the proposed method than that using the naïve method.

Bayesian Analyses of Cardiovascular Trials-Bringing Added Value to the Table

The limitations of traditional statistical analyses of randomised clinical trials that follow the frequentist inference paradigm have been increasingly noted. This article discusses the Bayesian approach to statistical inference in randomised clinical trials, demonstrating its functioning, utility, and limitations through an examination of current cardiovascular examples. A simplified overview of the mechanics of Bayesian inference and a glossary of the Bayesian terminology is first provided. The duality of the Bayesian approach, providing both an evidential calculus based on the likelihood ratio and a belief calculus that incorporates our prior beliefs with the current data, is presented. Specific cardiovascular trials are reanalysed with Bayesian methods. It is claimed that the Bayesian approach, by providing an enhanced ability to appreciate and model uncertainty, leads to an enriched understanding of the strength and quantification of the evidence, of the distinction between statistical and clinical significance, of the within- and between-trial variability, of subgroup analyses, of the utility of informative priors, and of our ability to synthesise and update our knowledge base. Ultimately, it is argued that the Bayesian approach is more intuitive and transparent, permits enhanced data analysis and interpretation, and may lead to improved decision making not only by trialists but also by practicing clinicians, guideline writers, and even expert regulatory advisory consultants.

Bayesian Meta-Analysis of Myopia Control with Multifocal Lenses

The aim of this study is to provide reliable guidelines for the mean percentage efficacy together with the 95% credibility interval in slowing down progression of myopia by a specific intervention over defined time periods, derived from a substantial number of randomised controlled clinical trials (RCTs) with consistent outcomes. Multifocal spectacles and contact lenses represent interventions with the largest number of RCTs carried out. Our meta-analyses considered 10 RCTs involving 1662 children which have tested the efficacy of progressive addition spectacle lenses (PALs). In a separate model for comparison purposes nine RCTs with 982 children trialling soft multifocal contact lenses (MFCLs) were analysed. Bayesian random-effects hierarchical models were fitted. The highest efficacy in retarding progression of the scaled sphere equivalent refraction was achieved after 12 M follow-up with the mean 28% reduction in progression and the 95% credibility interval between 21% and 35%. For comparison, the 95% credibility interval for the mean efficacy of soft MFCLs at 12 M follow up is 21% to 37%. We conclude that both multifocal spectacle and contact lenses moderately slow down progression of myopia, relative to single-vision spectacle lenses (SVLs) in the first 12 months after intervention. The relative efficacy of PALs tends to weaken after the first 12 months.

Do we need to adjust for interim analyses in a Bayesian adaptive trial design?

Background

Bayesian adaptive methods are increasingly being used to design clinical trials and offer several advantages over traditional approaches. Decisions at analysis points are usually based on the posterior distribution of the treatment effect. However, there is some confusion as to whether control of type I error is required for Bayesian designs as this is a frequentist concept.

Methods

We discuss the arguments for and against adjusting for multiplicities in Bayesian trials with interim analyses. With two case studies we illustrate the effect of including interim analyses on type I/II error rates in Bayesian clinical trials where no adjustments for multiplicities are made. We propose several approaches to control type I error, and also alternative methods for decision-making in Bayesian clinical trials.

Results

In both case studies we demonstrated that the type I error was inflated in the Bayesian adaptive designs through incorporation of interim analyses that allowed early stopping for efficacy and without adjustments to account for multiplicity. Incorporation of early stopping for efficacy also increased the power in some instances. An increase in the number of interim analyses that only allowed early stopping for futility decreased the type I error, but also decreased power. An increase in the number of interim analyses that allowed for either early stopping for efficacy or futility generally increased type I error and decreased power.

Conclusions

Currently, regulators require demonstration of control of type I error for both frequentist and Bayesian adaptive designs, particularly for late-phase trials. To demonstrate control of type I error in Bayesian adaptive designs, adjustments to the stopping boundaries are usually required for designs that allow for early stopping for efficacy as the number of analyses increase. If the designs only allow for early stopping for futility then adjustments to the stopping boundaries are not needed to control type I error. If one instead uses a strict Bayesian approach, which is currently more accepted in the design and analysis of exploratory trials, then type I errors could be ignored and the designs could instead focus on the posterior probabilities of treatment effects of clinically-relevant values.

Frequentist operating characteristics of Bayesian optimal designs via simulation

Bayesian adaptive designs have become popular because of the possibility of increasing the number of patients treated with more beneficial treatments, while still providing sufficient evidence for treatment efficacy comparisons. It can be essential, for regulatory and other purposes, to conduct frequentist analyses both before and after a Bayesian adaptive trial, and these remain challenging. In this paper, we propose a general simulation-based approach to compare frequentist designs with Bayesian adaptive designs based on frequentist criteria such as power and to compute valid frequentist p-values. We illustrate our approach by comparing the power of an equal randomization (ER) design with that of an optimal Bayesian adaptive (OBA) design. The Bayesian design considered here is the dynamic programming solution of the optimization of a specific utility function defined by the number of successes in a patient horizon, including patients whose treatment will be affected by the trial's results after the end of the trial. While the power of an ER design depends on treatment efficacy and the sample size, the power of the OBA design also depends on the patient horizon size. Our results quantify the trade-off between power and the optimal assignment of patients to treatments within the trial. We show that, for large patient horizons, the two criteria are in agreement, while for small horizons, differences can be substantial. This has implications for precision medicine, where patient horizons are decreasing as a result of increasing stratification of patients into subpopulations defined by molecular markers.

Bayesian sample-size determination methods considering both worthwhileness and unpromisingness for exploratory two-arm randomized clinical trials with binary endpoints

A randomized exploratory clinical trial comparing an experimental treatment with a control treatment on a binary endpoint is often conducted to make a go or no-go decision. Such an exploratory trial needs to have an adequate sample size such that it will provide convincing evidence that the experimental treatment is either worthwhile or unpromising relative to the control treatment. In this paper, we propose three new sample-size determination methods for an exploratory trial, which utilize the posterior probabilities calculated from predefined efficacy and inefficacy criteria leading to a declaration of the worthwhileness or unpromisingness of the experimental treatment. Simulation studies, including numerical investigation, showed that all three methods could declare the experimental treatment as worthwhile or unpromising with a high probability when the true response probability of the experimental treatment group is higher or lower, respectively, than that of the control treatment group.

Application of Bayesian evidence synthesis to modelling the effect of ketogenic therapy on survival of high grade glioma patients

Background

Ketogenic therapy in the form of ketogenic diets or calorie restriction has been proposed as a metabolic treatment of high grade glioma (HGG) brain tumors based on mechanistic reasoning obtained mainly from animal experiments. Given the paucity of clinical studies of this relatively new approach, our goal is to extrapolate evidence from the greater number of animal studies and synthesize it with the available human data in order to estimate the expected effects of ketogenic therapy on survival in HGG patients. At the same time we are using this analysis as an example for demonstrating how Bayesianism can be applied in the spirit of a circular view of evidence.

Results

A Bayesian hierarchical model was developed. Data from three human cohort studies and 17 animal experiments were included to estimate the effects of four ketogenic interventions (calorie restriction/ketogenic diets as monotherapy/combination therapy) on the restricted mean survival time ratio in humans using various assumptions for the relationships between humans, rats and mice. The impact of different biological assumptions about the relevance of animal data for humans as well as external information based on mechanistic reasoning or case studies was evaluated by specifying appropriate priors. We provide statistical and philosophical arguments for why our approach is an improvement over existing (frequentist) methods for evidence synthesis as it is able to utilize evidence from a variety of sources. Depending on the prior assumptions, a 30–70% restricted mean survival time prolongation in HGG patients was predicted by the models. The highest probability of a benefit (> 90%) for all four ketogenic interventions was obtained when adopting an enthusiastic prior based on previous case reports together with assuming synergism between ketogenic therapies with other forms of treatment. Combinations with other treatments were generally found more effective than ketogenic monotherapy.

Conclusions

Combining evidence from both human and animal studies is statistically possible using a Bayesian approach. We found an overall survival-prolonging effect of ketogenic therapy in HGG patients. Our approach is best compatible with a circular instead of hierarchical view of evidence and easy to update once more data become available.

Electronic supplementary material

The online version of this article (10.1186/s12976-018-0084-y) contains supplementary material, which is available to authorized users.

Clinical trial endpoints for patients with gliomas

Malignant glioma represents a diverse set of molecularly heterogeneous diseases. Few therapeutic agents have been approved despite decades of clinical trials research and pre-clinical investigation. Attempts to refine neuroimaging criteria and recent discovery of the genomic profiles linking tumor subsets to survival outcomes have spurred discussion on a variety of new approaches in clinical trial design and relevant endpoints. Here we focus on those endpoints in clinical trial design for patients with primary glioma and related issues still to be resolved.

Bayesian randomized clinical trials: From fixed to adaptive design

Randomized controlled studies are the gold standard for phase III clinical trials. Using α-spending functions to control the overall type I error rate, group sequential methods are well established and have been dominating phase III studies. Bayesian randomized design, on the other hand, can be viewed as a complement instead of competitive approach to the frequentist methods. For the fixed Bayesian design, the hypothesis testing can be cast in the posterior probability or Bayes factor framework, which has a direct link to the frequentist type I error rate. Bayesian group sequential design relies upon Bayesian decision-theoretic approaches based on backward induction, which is often computationally intensive. Compared with the frequentist approaches, Bayesian methods have several advantages. The posterior predictive probability serves as a useful and convenient tool for trial monitoring, and can be updated at any time as the data accrue during the trial. The Bayesian decision-theoretic framework possesses a direct link to the decision making in the practical setting, and can be modeled more realistically to reflect the actual cost-benefit analysis during the drug development process. Other merits include the possibility of hierarchical modeling and the use of informative priors, which would lead to a more comprehensive utilization of information from both historical and longitudinal data. From fixed to adaptive design, we focus on Bayesian randomized controlled clinical trials and make extensive comparisons with frequentist counterparts through numerical studies.

Advantages of Bayesian monitoring methods in deciding whether and when to stop a clinical trial: an example of a neonatal cooling trial

BACKGROUND

Decisions to stop randomized trials are often based on traditional P value thresholds and are often unconvincing to clinicians. To familiarize clinical investigators with the application and advantages of Bayesian monitoring methods, we illustrate the steps of Bayesian interim analysis using a recent major trial that was stopped based on frequentist analysis of safety and futility.

METHODS

We conducted Bayesian reanalysis of a factorial trial in newborn infants with hypoxic-ischemic encephalopathy that was designed to investigate whether outcomes would be improved by deeper (32 °C) or longer cooling (120 h), as compared with those achieved by standard whole body cooling (33.5 °C for 72 h). Using prior trial data, we developed neutral and enthusiastic prior probabilities for the effect on predischarge mortality, defined stopping guidelines for a clinically meaningful effect, and derived posterior probabilities for predischarge mortality.

RESULTS

Bayesian relative risk estimates for predischarge mortality were closer to 1.0 than were frequentist estimates. Posterior probabilities suggested increased predischarge mortality (relative risk > 1.0) for the three intervention groups; two crossed the Bayesian futility threshold.

CONCLUSIONS

Bayesian analysis incorporating previous trial results and different pre-existing opinions can help interpret accruing data and facilitate informed stopping decisions that are likely to be meaningful and convincing to clinicians, meta-analysts, and guideline developers.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01192776 . Registered on 31 August 2010.

Idle thoughts of a 'well-calibrated' Bayesian in clinical drug development

The use of Bayesian approaches in the regulated world of pharmaceutical drug development has not been without its difficulties or its critics. The recent Food and Drug Administration regulatory guidance on the use of Bayesian approaches in device submissions has mandated an investigation into the operating characteristics of Bayesian approaches and has suggested how to make adjustments in order that the proposed approaches are in a sense calibrated. In this paper, I present examples of frequentist calibration of Bayesian procedures and argue that we need not necessarily aim for perfect calibration but should be allowed to use procedures, which are well-calibrated, a position supported by the guidance.

Changing the Paradigm for the Treatment and Development of New Therapies for FSGS

Focal segmental glomerulosclerosis (FSGS) is a renal pathology finding that represents a constellation of rare kidney diseases, which manifest as proteinuria, edema nephrotic syndrome, hypertension, and increased risk for kidney failure. Therapeutic options for FSGS are reviewed displaying the expected efficacy from 25 to 69% depending on specific therapy, patient characteristics, cost, and common side effects. This variability in treatment response is likely caused, in part, by the heterogeneity in the etiology and active molecular mechanisms of FSGS. Clinical trials in FSGS have been scant in number and slow to recruit, which may stem, in part, from reliance on classic clinical trial design paradigms. Traditional clinical trial designs based on the "learn and confirm" paradigm may not be appropriate for rare diseases, such as FSGS. Future drug development and testing will require novel approaches to trial designs that have the capacity to enrich study populations and adapt the trial in a planned way to gain efficiencies in trial completion timelines. A clinical trial simulation is provided that compares a classical and more modern design to determine the maximum tolerated dose in FSGS.

Emerging innovations in clinical trial design

Designs of clinical trials have changed little since the advent of randomization in the 1940s. Modern innovations in designs are being driven by the increasing recognition in clinical research that diseases are heterogeneous and patients who apparently have the same disease require different therapies. This article describes some innovations in clinical trial design across therapeutic areas but with a focus on oncology. No one knows what the future holds for clinical trial design but the status quo of large trials that pretend the patient population is homogeneous is not sustainable, either economically or scientifically/medically. No one knows what the eventual business model and regulatory model will be, but they will be very different from today's.

Bayesian methodology for the design and interpretation of clinical trials in critical care medicine: a primer for clinicians

OBJECTIVES

To review Bayesian methodology and its utility to clinical decision making and research in the critical care field.

DATA SOURCE AND STUDY SELECTION

Clinical, epidemiological, and biostatistical studies on Bayesian methods in PubMed and Embase from their inception to December 2013.

DATA SYNTHESIS

Bayesian methods have been extensively used by a wide range of scientific fields, including astronomy, engineering, chemistry, genetics, physics, geology, paleontology, climatology, cryptography, linguistics, ecology, and computational sciences. The application of medical knowledge in clinical research is analogous to the application of medical knowledge in clinical practice. Bedside physicians have to make most diagnostic and treatment decisions on critically ill patients every day without clear-cut evidence-based medicine (more subjective than objective evidence). Similarly, clinical researchers have to make most decisions about trial design with limited available data. Bayesian methodology allows both subjective and objective aspects of knowledge to be formally measured and transparently incorporated into the design, execution, and interpretation of clinical trials. In addition, various degrees of knowledge and several hypotheses can be tested at the same time in a single clinical trial without the risk of multiplicity. Notably, the Bayesian technology is naturally suited for the interpretation of clinical trial findings for the individualized care of critically ill patients and for the optimization of public health policies.

CONCLUSIONS

We propose that the application of the versatile Bayesian methodology in conjunction with the conventional statistical methods is not only ripe for actual use in critical care clinical research but it is also a necessary step to maximize the performance of clinical trials and its translation to the practice of critical care medicine.

A Bayesian Stopping Rule for Sequential Monitoring of Serious Adverse Events

In an ongoing clinical trial, there will always be a risk for unanticipated critical safety problems, such as excessive occurrence of serious adverse events. When such a problem arises, the trial administrators must conduct an immediate evaluation to determine whether the trial should be terminated to protect patients. This decision is complicated but may be aided by statistical stopping rules. Sequential stopping rules are appropriate for immediate decisions, but frequentist approaches may not be useful because the unknown truncated end of the trial makes it impossible to define type I errors. Thus, a Bayesian stopping rule is proposed that is based on the posterior distribution with an informative prior distribution, and a guideline to construct this stopping rule is presented. Some operating characteristics are evaluated and compared with those of the modified sequential probability ratio test (SPRT), the maximized SPRT, and Pocock's test. The proposed method has flexibility for construction and could provide a more desirable performance than the other compared methods.

A comparison of Bayesian adaptive randomization and multi-stage designs for multi-arm clinical trials

When several experimental treatments are available for testing, multi-arm trials provide gains in efficiency over separate trials. Including interim analyses allows the investigator to effectively use the data gathered during the trial. Bayesian adaptive randomization (AR) and multi-arm multi-stage (MAMS) designs are two distinct methods that use patient outcomes to improve the efficiency and ethics of the trial. AR allocates a greater proportion of future patients to treatments that have performed well; MAMS designs use pre-specified stopping boundaries to determine whether experimental treatments should be dropped. There is little consensus on which method is more suitable for clinical trials, and so in this paper, we compare the two under several simulation scenarios and in the context of a real multi-arm phase II breast cancer trial. We compare the methods in terms of their efficiency and ethical properties. We also consider the practical problem of a delay between recruitment of patients and assessment of their treatment response. Both methods are more efficient and ethical than a multi-arm trial without interim analyses. Delay between recruitment and response assessment attenuates this efficiency gain. We also consider futility stopping rules for response adaptive trials that add efficiency when all treatments are ineffective. Our comparisons show that AR is more efficient than MAMS designs when there is an effective experimental treatment, whereas if none of the experimental treatments is effective, then MAMS designs slightly outperform AR.

The utility of Bayesian predictive probabilities for interim monitoring of clinical trials

BACKGROUND

Bayesian predictive probabilities can be used for interim monitoring of clinical trials to estimate the probability of observing a statistically significant treatment effect if the trial were to continue to its predefined maximum sample size.

PURPOSE

We explore settings in which Bayesian predictive probabilities are advantageous for interim monitoring compared to Bayesian posterior probabilities, p-values, conditional power, or group sequential methods.

RESULTS

For interim analyses that address prediction hypotheses, such as futility monitoring and efficacy monitoring with lagged outcomes, only predictive probabilities properly account for the amount of data remaining to be observed in a clinical trial and have the flexibility to incorporate additional information via auxiliary variables.

LIMITATIONS

Computational burdens limit the feasibility of predictive probabilities in many clinical trial settings. The specification of prior distributions brings additional challenges for regulatory approval.

CONCLUSIONS

The use of Bayesian predictive probabilities enables the choice of logical interim stopping rules that closely align with the clinical decision-making process.

Biomarker-based adaptive trials for patients with glioblastoma--lessons from I-SPY 2

The traditional clinical trials infrastructure may not be ideally suited to evaluate the numerous therapeutic hypotheses that result from the increasing number of available targeted agents combined with the various methodologies to molecularly subclassify patients with glioblastoma. Additionally, results from smaller screening studies are rarely translated to successful larger confirmatory studies, potentially related to a lack of efficient control arms or the use of unvalidated surrogate endpoints. Streamlining clinical trials and providing a flexible infrastructure for biomarker development is clearly needed for patients with glioblastoma. The experience developing and implementing the I-SPY studies in breast cancer may serve as a guide to developing such trials in neuro-oncology.

A two-stage Bayesian design for co-development of new drugs and companion diagnostics

Most new drug development in oncology is based on targeting specific molecules. Genomic profiles and deregulated drug targets vary from patient to patient making new treatments likely to benefit only a subset of patients traditionally grouped in the same clinical trials. Predictive biomarkers are being developed to identify patients who are most likely to benefit from a particular treatment; however, their biological basis is not always conclusive. The inclusion of marker-negative patients in a trial is therefore sometimes necessary for a more informative evaluation of the therapy. In this paper, we present a two-stage Bayesian design that includes both marker-positive and marker-negative patients in a clinical trial. We formulate a family of prior distributions that represent the degree of a priori confidence in the predictive biomarker. To avoid exposing patients to a treatment to which they may not be expected to benefit, we perform an interim analysis that may stop accrual of marker-negative patients or accrual of all patients. We demonstrate with simulations that the design and priors used control type I errors, give adequate power, and enable the early futility analysis of test-negative patients to be based on prior specification on the strength of evidence in the biomarker.

Bayesian approaches for comparative effectiveness research

BACKGROUND

A hallmark of comparative effectiveness research is the analysis of all the available evidence from different studies addressing a given question of medical risk versus benefit. The Bayesian statistical approach is ideally suited for such investigations because it is inherently synthetic and because it is philosophically uninhibited regarding the ability to analyze all the available evidence.

PURPOSE

To consider a variety of comparative effectiveness research settings and show how the Bayesian approach applies.

METHODS

The Bayesian approach is described as it has been applied to the comparative analysis of implantable cardioverter defibrillators and mammographic screening, in the Cancer Intervention and Surveillance Modeling Network, in comparisons of patient outcomes data from different sources, and in designing adaptive clinical trials to support the development of 'personalized medicine.'

RESULTS

Bayesian methods allow for continued learning as data accrue and for cumulating meta-analyses and the comparison of heterogeneous studies. Bayesian methods enable predictive probability distributions of the results of future studies.

LIMITATIONS

Bayesian posterior distributions are subject to potential bias - in the selection of 'available' evidence and in the choice of a likelihood model. Sensitivity analyses help to control this bias.

CONCLUSIONS

The Bayesian approach has much to offer comparative effectiveness research. It provides a mechanism for synthesizing various sources of information and for updating knowledge in an online fashion as evidence accumulates.

Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Part 3: Design considerations

Ongoing safety monitoring of clinical trials of investigational treatments must operate at levels that range from the minute and detailed - namely, mathematical treatment of trial data - to the philosophical and societal - namely, ethical concerns for individuals and populations. Between those two poles lies a realm of environmental and pragmatic considerations that reflect the goals, biases, risk-tolerance, and constraints of study sponsors and organizers. These factors, while more difficult to quantify or, at times, to justify, also have a meaningful impact on the approach to safety monitoring and the resulting actions and outcomes. This paper considers the influence and interaction of two such factors, study design and statistical framework, on continuous safety monitoring procedures. Group sequential designs have been generally preferred for clinical trials over continuous sequential designs because of practical considerations. The group means and greater time for deliberation when using a group sequential procedure, as opposed to a continuous sequential procedure, can improve the quality of the analyses with minimal loss in sensitivity. However, undertaking any sequential analysis within a frequentist framework provokes considerable theoretical and practical difficulties. Continuous monitoring with a likelihood based method, on the other hand, has the advantages that all available information, including new data, can be used; sample sizes need not be fixed; and decisions can be made at any time without statistical penalty, irrespective of trial design. Such responsive statistical rules are needed to provide guidance to the human beings charged with trial monitoring.

Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Part 2: Statistical considerations

If the primary objective of a trial is to learn about the ability of a new treatment to help future patients without sacrificing the safe and effective treatment of the current patients, then a Bayesian design with frequent assessments of the accumulating data should be considered. Unfortunately, Bayesian analyses typically do not have standard approaches, and because of the subjectivity of prior probabilities and the possibility for introducing bias, statisticians have developed other methods for statistical inference that only depend on deductive probabilities. However, these frequentist probabilities are just theories about how certain relative frequencies will develop over time. They have no real meaning in a single experiment. Designed to work well in the long run, p-values become hard to explain for individual experiments. Fortunately, the controversy surrounding Bayes' theorem comes, not from the representation of evidence, but from the use of probabilities to measure belief. A prior distribution is not necessary. The likelihood function contains all of the information in a trial relevant for making inferences about the parameters. Monitoring clinical trials is a dynamic process which requires flexibility to respond to unforeseen developments. Likelihood ratios allow the data to speak for themselves, without regard for the probability of observing weak or misleading evidence, and decisions to stop, or continue, a trial can be made at any time, with all of the available information. A likelihood based method is needed.

Hydroxyurea suppresses HCV replication in humans: a Phase I trial of oral hydroxyurea in chronic hepatitis C patients

BACKGROUND

HCV is the main causative agent of chronic liver disease, which could progress to liver cirrhosis and hepatocellular carcinoma. By using a recently developed genome-length HCV RNA replication reporter assay system, we found that hydroxyurea (HU), an inhibitor of DNA synthesis, inhibited HCV RNA replication.

METHODS

To test the hypothesis that HU suppresses HCV replication in humans, we conducted a Phase I trial involving Japanese patients with chronic hepatitis C (CHC) and investigated the safety and effectiveness of a 4-week course of oral HU.

RESULTS

A total of nine patients were treated with an HU dose level of 500 mg three times daily. Dose-limiting toxicity was not observed at this dose level. Of the nine patients, eight exhibited a moderate decrease in serum HCV RNA levels during the trial. A decrease in HCV RNA levels to nadir levels was achieved for the eight patients (median -0.27 log(10) IU/ml [range -0.08--0.44]) at various times during the 4 weeks after therapy initiation.

CONCLUSIONS

The results of this Phase I trial suggest that HU has potential as an anti-HCV agent that could be effective for the treatment of CHC patients.

Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials

Bayesian clinical trial designs offer the possibility of a substantially reduced sample size, increased statistical power, and reductions in cost and ethical hazard. However when prior and current information conflict, Bayesian methods can lead to higher than expected type I error, as well as the possibility of a costlier and lengthier trial. This motivates an investigation of the feasibility of hierarchical Bayesian methods for incorporating historical data that are adaptively robust to prior information that reveals itself to be inconsistent with the accumulating experimental data. In this article, we present several models that allow for the commensurability of the information in the historical and current data to determine how much historical information is used. A primary tool is elaborating the traditional power prior approach based upon a measure of commensurability for Gaussian data. We compare the frequentist performance of several methods using simulations, and close with an example of a colon cancer trial that illustrates a linear models extension of our adaptive borrowing approach. Our proposed methods produce more precise estimates of the model parameters, in particular, conferring statistical significance to the observed reduction in tumor size for the experimental regimen as compared to the control regimen.

Bayesian analysis of a mastitis control plan to investigate the influence of veterinary prior beliefs on clinical interpretation

The fundamental objective for health research is to determine whether changes should be made to clinical decisions. Decisions made by veterinary surgeons in the light of new research evidence are known to be influenced by their prior beliefs, especially their initial opinions about the plausibility of possible results. In this paper, clinical trial results for a bovine mastitis control plan were evaluated within a Bayesian context, to incorporate a community of prior distributions that represented a spectrum of clinical prior beliefs. The aim was to quantify the effect of veterinary surgeons’ initial viewpoints on the interpretation of the trial results.

A Bayesian analysis was conducted using Markov chain Monte Carlo procedures. Stochastic models included a financial cost attributed to a change in clinical mastitis following implementation of the control plan. Prior distributions were incorporated that covered a realistic range of possible clinical viewpoints, including scepticism, enthusiasm and uncertainty. Posterior distributions revealed important differences in the financial gain that clinicians with different starting viewpoints would anticipate from the mastitis control plan, given the actual research results. For example, a severe sceptic would ascribe a probability of 0.50 for a return of <£5 per cow in an average herd that implemented the plan, whereas an enthusiast would ascribe this probability for a return of >£20 per cow. Simulations using increased trial sizes indicated that if the original study was four times as large, an initial sceptic would be more convinced about the efficacy of the control plan but would still anticipate less financial return than an initial enthusiast would anticipate after the original study. In conclusion, it is possible to estimate how clinicians’ prior beliefs influence their interpretation of research evidence. Further research on the extent to which different interpretations of evidence result in changes to clinical practice would be worthwhile.

Interpreting trial results in light of conflicting evidence: a Bayesian analysis of adjuvant chemotherapy for non-small-cell lung cancer

PURPOSE

When successive randomized trials contradict prior evidence, clinicians may be unsure how to evaluate them: Does accumulating evidence warrant changing practice? An increasingly popular solution, Bayesian statistics quantitatively evaluate new results in context. This study provides a clinically relevant example of Bayesian methods.

METHODS

Three recent non-small-cell lung cancer adjuvant chemotherapy trials were evaluated in light of prior conflicting data. Results were used from International Adjuvant Lung Trial (IALT), JBR.10, and Adjuvant Navelbine International Trialist Association (ANITA). Prior evidence was sequentially updated to calculate the probability of each survival benefit level (overall and by stage) and variance. Sensitivity analysis was performed using expert opinion and uninformed estimates of survival benefit prior probability.

RESULTS

The probability of a 4% survival benefit increased from 33% before IALT to 64% after IALT. After sequential updating with JBR.10 and ANITA, this probability was 82% (hazard ratio = 0.84; 95% CI, 0.77 to 0.91). IALT produced the largest decrease in variance (61%) and decreased the chance of survival decrement to 0%. Sensitivity analysis did not support a survival benefit after IALT. However, sequential updating substantiated a 4% survival benefit and, for stage II and III, more than 90% probability of a 6% benefit and 50% probability of a 12% benefit.

CONCLUSION

When evaluated in context with prior data, IALT did not support a 4% survival benefit. However, sequential updating with JBR.10 and ANITA did. A model for future assessments, this study demonstrates the unique ability of Bayesian analysis to evaluate results that contradict prior evidence.

Optimal Bayesian design for patient selection in a clinical study

Bayesian experimental design for a clinical trial involves specifying a utility function that models the purpose of the trial, in this case the selection of patients for a diagnostic test. The best sample of patients is selected by maximizing expected utility. This optimization task poses difficulties due to a high-dimensional discrete design space and, also, to an expected utility formula of high complexity. A simulation-based optimal design method is feasible in this case. In addition, two deterministic algorithms that perform a systematic search over the design space are developed to address the computational issues.

Preventing brain injury in newborns with congenital heart disease: brain imaging and innovative trial designs

BACKGROUND AND PURPOSE

Newborns with congenital heart disease are at high risk for brain injury and adverse neurodevelopmental outcomes. MRI enables the objective determination of the severity of brain injury in critically ill newborns with congenital heart disease. We will rationalize the use of MRI as a surrogate for neurodevelopmental outcome and describe novel randomization techniques that can be used in trials in this population.

METHODS

This article describes the evidence for the use of MRI and the link with neurodevelopmental outcome established in newborns. We also discuss the use of adaptive randomization techniques for future clinical trials in newborns with congenital heart disease. These strategies will be highlighted using an example.

RESULTS

Brain injuries occur with high frequency in newborns with congenital heart disease. It is not until school age that the full extent of neurological sequelae becomes apparent and the rapid pace of innovation in neonatal cardiac surgery prevents timely evaluation of changes in care. MRI provides a timely, safe, and reliable outcome measure and has been extensively studied in newborns with other conditions in which the link between brain injury and neurodevelopmental outcome has been established. Clinical trials using MRI as an outcome measure as well as adaptive randomization can improve the efficiency of such trials.

CONCLUSIONS

Clinical trials of brain protection are urgently needed in newborns with congenital heart disease given the unacceptable frequency of brain injury in this population; MRI provides an early surrogate marker of long-term neurodevelopmental outcome and adaptive randomization can be used to improve the efficiency of these clinical trials.

A randomised controlled trial of intramuscular vs. intravenous antivenom for latrodectism--the RAVE study

BACKGROUND

Widow spider-bite causes latrodectism and is associated with significant morbidity worldwide. Antivenom is given by both the intravenous (IV) and intramuscular (IM) routes and it is unclear which is more effective.

AIM

To compare the effectiveness of IV vs. IM redback spider antivenom.

DESIGN

Randomized controlled trial.

METHODS

Patients with latrodectism were given either IV or IM antivenom according to a randomized double-dummy, double-blind protocol. The first antivenom treatment was followed by another identical treatment after two hours if required. The primary outcome was a clinically significant reduction in pain two hours after the last treatment. A fully Bayesian analysis was used to estimate the probability of the desired treatment effect, predetermined as an absolute difference of 20%.

RESULTS

We randomly allocated 126 patients to receive antivenom IV (64) and IM (62). After antivenom treatment pain improved in 40/64(62%) in the IV group vs. 33/62(53%) in the IM group (+9%; 95% Credible Interval [CrI]: -8% to +26%). The probability of a difference greater than zero (IV superior) was 85% but the probability of a difference >20% was only 10%. In 55 patients with systemic effects, these improved in 58% after IV antivenom vs. 65% after IM antivenom (-8%; 95% CrI: -32% to +17%). Twenty-four hours after antivenom pain had improved in 84% in the IV group vs. 71% in the IM group (+13%; 95% CrI: -2% to +27%). A meta-analysis including data from a previous trial found no difference in the primary outcome between IV and IM administration.

DISCUSSION

The difference between IV and IM routes of administration of widow spider antivenom is, at best, small and does not justify routinely choosing one route over the other. Furthermore, antivenom may provide no benefit over placebo.

Bayesian designs to account for patient heterogeneity in phase II clinical trials

PURPOSE OF REVIEW

Between-patient heterogeneity is very common in clinical trials. This complicates treatment evaluation, due to known prognostic subgroup effects or potential treatment-subgroup interactions. We review two Bayesian phase II clinical trial designs that account explicitly for patient heterogeneity. The first design uses analysis of covariance to assess treatment effects in subgroups known to have different prognoses. The second design uses a hierarchical model for settings where, a priori, the experimental treatment effects in the subgroups are assumed to be exchangeable.

RECENT FINDINGS

Compared with simpler designs that ignore patient heterogeneity, each design provides substantial improvements by reducing both false positive and false negative rates and focusing resources on subgroups where an experimental treatment is more likely to provide an advance over standard therapy. In either case, accounting for potential treatment-subgroup interactions is extremely important.

SUMMARY

Due to the rapidly increasing number of potential new treatments to be evaluated clinically, increasing costs and limited resources, it is critically important to perform early phase clinical trials efficiently. The new Bayesian methods described here and other related methods provide efficient, broadly applicable tools to address these problems.

Bayesian interim analysis in clinical trials

We propose a Bayesian approach to monitor clinical trials with clustered binary outcomes using multivariate probit models. Our monitoring is based on the calculated probability of the reduced incidence rate using a new treatment compared with the standard treatment greater than a target improvement under different prior scenarios for the treatment effect. We develop a Bayesian sampling algorithm for posterior inference allowing missing values in the outcomes. We illustrate our method using a published early trail of inhaled nitric oxide therapy in premature infants.

Bayesians in clinical trials: Asleep at the switch

The refreshing Bayes perspective has much to offer biostatistics. Yet, from its 225-year-old roots sprung difficulties that blocked its growth at the beginning of the 20th century. Computational obstacles in concert with an inability to identify the best indifferent prior revealed a weakness on which frequentists capitalized. It took Bayesians 40 years to recover, allowing the infant field of biostatistics to fall firmly in the hands of the frequentists. Recent disillusionment with the frequentist perspective, and its hegemony of p-values, has produced a second opportunity for the Bayesian philosophy to make solid contributions to clinical trials.However, difficulty with the applicability of the likelihood principle, problems with prevalent prior 'disinformation' in clinical medicine, in concert with the complexity of truly representative loss functions threaten again to thwart the Bayesian march into biostatistics. Seven suggestions are offered to the Bayesians to help them adapt to the rigors of clinical research.

Bayesian design and conduct of phase II single-arm clinical trials with binary outcomes: a tutorial

The aim of phase II single-arm clinical trials of a new drug is to determine whether it has sufficient promising activity to warrant its further development. For the last several years Bayesian statistical methods have been proposed and used. Bayesian approaches are ideal for earlier phase trials as they take into account information that accrues during a trial. Predictive probabilities are then updated and so become more accurate as the trial progresses. Suitable priors can act as pseudo samples, which make small sample clinical trials more informative. Thus patients have better chances to receive better treatments. The goal of this paper is to provide a tutorial for statisticians who use Bayesian methods for the first time or investigators who have some statistical background. In addition, real data from three clinical trials are presented as examples to illustrate how to conduct a Bayesian approach for phase II single-arm clinical trials with binary outcomes.

A hybrid Bayesian-frequentist approach to evaluate clinical trial designs for tests of superiority and non-inferiority

Specification of the study objective of superiority or non-inferiority at the design stage of a phase III clinical trial can sometimes be very difficult due to the uncertainty that surrounds the efficacy level of the experimental treatment. This uncertainty makes it tempting for investigators to design a trial that would allow testing of both superiority and non-inferiority hypotheses. However, when a conventional single-stage design is used to test both hypotheses, the sample size is based on the chosen primary objective of either superiority or non-inferiority. In this situation, the power of the test for the secondary objective can be low, which may lead to a large loss of resources. Potentially low reproducibility is another major concern for the single-stage design in phase III trials, because significant findings of confirmatory trials are required to be reproducible. In this paper, we propose a hybrid Bayesian-frequentist approach to evaluate reproducibility and power in single-stage designs for phase III trials to test both superiority and non-inferiority. The essence of the proposed approach is to express the uncertainty that surrounds the efficacy of the experimental treatment as a probability distribution. Then one can use Bayes formula with simple graphical techniques to evaluate reproducibility and power adequacy.

Economic evaluation of a Bayesian model to predict late-phase success of new chemical entities

OBJECTIVE

To evaluate the economic impact of a Bayesian network model designed to predict clinical success of a new chemical entity (NCE) based on pre-phase III data.

METHODS

We trained our Bayesian network model on publicly accessible data on 503 NCEs, stratified by therapeutic class. We evaluated the sensitivity, specificity and accuracy of our model on an independent data set of 18 NCE-indication pairs, using prior probability data for the antineoplastic NCEs within the training set. We performed Monte Carlo simulations to evaluate the economic performance of our model relative to reported pharmaceutical industry performance, taking into account reported capitalized phase costs, cumulative revenues for a postapproval period of 7 years, and the range of possible false negative and true negative rates for terminated NCEs within the pharmaceutical industry.

RESULTS

Our model predicted outcomes on the independent validation set of oncology agents with 78% accuracy (80%sensitivity and 76% specificity). In comparison with the pharmaceutical industry's reported success rates, on average our model significantly reduced capitalized expenditures from $727 million/successful NCE to $444 million/successful NCE (P < 0.001), and significantly improved revenues from $347 million/phase III trial to $507 million/phase III trial (P < 0.001) during the first 7 years post launch. These results indicate that our model identified successful NCEs more efficiently than currently reported pharmaceutical industry performances.

CONCLUSIONS

Accurate prediction of NCE outcomes is computationally feasible, significantly increasing the proportion of successful NCEs, and likely eliminating ineffective and unsafe NCEs.

Bayesian decision-theoretic group sequential clinical trial design based on a quadratic loss function: a frequentist evaluation

The decision to terminate a controlled clinical trial at the time of an interim analysis is perhaps best made by weighing the value of the likely additional information to be gained if further subjects are enrolled against the various costs of that further enrollment. The most commonly used statistical plans for interim analysis (eg, O'Brien-Fleming), however, are based on a frequentist approach that makes no such comparison. A two-armed Bayesian decision-theoretic clinical trial design is developed for a disease with two possible outcomes, incorporating a quadratic decision loss function and using backward induction to quantify the cost of future enrollment. Monte Carlo simulation is used to compare frequentist error rates and mean required sample sizes for these Bayesian designs with the two-tailed frequentist group-sequential designs of, O'Brien-Fleming and Pocock. When the terminal decision loss function is chosen to yield typical frequentist error rates, the mean sample sizes required by the Bayesian designs are smaller than those of the corresponding O'Brien-Fleming frequentist designs, largely due to the more frequent interim analyses typically used with the Bayesian designs and the ability of the Bayesian designs to terminate early and conclude equivalence. Adding stochastic curtailment to the frequentist designs and using the same number of interim analyses results in largely equivalent trials. An example of a Bayesian design for the data safety monitoring of a clinical trial is given. Our design assumes independence of the probabilities of success in the two trial arms. Additionally, we have chosen non-informative priors and selected loss functions to produce trials with appealing frequentist error rates, rather than choosing priors that reflect realistic prior information and loss functions that reflect true costs. Our Bayesian designs allow interpretation of the final results along either Bayesian or frequentist lines. For the Bayesian, they minimize the total cost and allow the direct calculation of the probability density function for the difference in efficacy. For the frequentist, they have well-characterized type I and II error rates and in some cases lead to a reduction in the mean sample size.