Bayesian predictive approach to interim monitoring in clinical trials

Sample Size Reestimation in Stochastic Curtailment Tests With Time-to-Events Outcome in the Case of Nonproportional Hazards Utilizing Two Weibull Distributions With Unknown Shape Parameters

Stochastic curtailment tests for Phase II two-arm trials with time-to-event end points are traditionally performed using the log-rank test. Recent advances in designing time-to-event trials have utilized the Weibull distribution with a known shape parameter estimated from historical studies. As sample size calculations depend on the value of this shape parameter, these methods either cannot be used or likely underperform/overperform when the natural variation around the point estimate is ignored. We demonstrate that when the magnitude of the Weibull shape parameters changes, unblinded interim information on the shape of the survival curves can be useful to enrich the final analysis for reestimation of the sample size. For such scenarios, we propose two Bayesian solutions to estimate the natural variations of the Weibull shape parameter. We implement these approaches under the framework of the newly proposed relative time method that allows nonproportional hazards and nonproportional time. We also demonstrate the sample size reestimation for the relative time method using three different approaches (internal pilot study approach, conditional power, and predictive power approach) at the interim stage of the trial. We demonstrate our methods using a hypothetical example and provide insights regarding the practical constraints for the proposed methods.

Sample size reestimation and Bayesian predictive probability for single-arm clinical trials with a time-to-event endpoint using Weibull distribution with unknown shape parameter

This manuscript consists of two topics. Firstly, we explore the utility of internal pilot study (IPS) approach for reestimating sample size at an interim stage when a reliable estimate of the nuisance shape parameter of the Weibull distribution for modeling survival data is unavailable during the planning phase of a study. Although IPS approach can help rescue the study power, it is noted that the adjusted sample size can be as much as twice the initially planned sample size, which may put substantial practical constraints to continue the study. Secondly, we discuss Bayesian predictive probability for conducting interim analyses to obtain preliminary evidence of efficacy or futility of an experimental treatment warranting early termination of a clinical trial. In the context of single-arm clinical trials with time-to-event endpoints following Weibull distribution, we present the calculation of the Bayesian predictive probability when the shape parameter of the Weibull distribution is unknown. Based on the data accumulated at the interim, we propose two approaches which rely on the posterior mode or the entire posterior distribution of the shape parameter. To account for uncertainty in the shape parameter, it is recommended to incorporate its entire posterior distribution in our calculation.

Stochastic curtailment tests for phase II trial with time-to-event outcome using the concept of relative time in the case of non-proportional hazards

As part of the drug development process, interim analysis is frequently used to design efficient phase II clinical trials. A stochastic curtailment framework is often deployed wherein a decision to continue or curtail the trial is taken at each interim look based on the likelihood of observing a positive or negative treatment effect if the trial were to continue to its anticipated end. Thus, curtailment can take place due to evidence of early efficacy or futility. Traditionally, in the case of time-to-event endpoints, interim monitoring is conducted in a two-arm clinical trial using the log-rank test, often with the assumption of proportional hazards. However, when this is violated, the log-rank test may not be appropriate, resulting in loss of power and subsequently inaccurate sample sizes. In this paper, we propose stochastic curtailment methods for two-arm phase II trial with the flexibility to allow non-proportional hazards. The proposed methods are built utilizing the concept of relative time assuming that the survival times in the two treatment arms follow two different Weibull distributions. Three methods - conditional power, predictive power and Bayesian predictive probability - are discussed along with corresponding sample size calculations. The monitoring strategy is discussed with a real-life example.

Upstrapping to determine futility: predicting future outcomes nonparametrically from past data

BACKGROUND

Clinical trials often involve some form of interim monitoring to determine futility before planned trial completion. While many options for interim monitoring exist (e.g., alpha-spending, conditional power), nonparametric based interim monitoring methods are also needed to account for more complex trial designs and analyses. The upstrap is one recently proposed nonparametric method that may be applied for interim monitoring.

METHODS

Upstrapping is motivated by the case resampling bootstrap and involves repeatedly sampling with replacement from the interim data to simulate thousands of fully enrolled trials. The p-value is calculated for each upstrapped trial and the proportion of upstrapped trials for which the p-value criteria are met is compared with a pre-specified decision threshold. To evaluate the potential utility for upstrapping as a form of interim futility monitoring, we conducted a simulation study considering different sample sizes with several different proposed calibration strategies for the upstrap. We first compared trial rejection rates across a selection of threshold combinations to validate the upstrapping method. Then, we applied upstrapping methods to simulated clinical trial data, directly comparing their performance with more traditional alpha-spending and conditional power interim monitoring methods for futility.

RESULTS

The method validation demonstrated that upstrapping is much more likely to find evidence of futility in the null scenario than the alternative across a variety of simulations settings. Our three proposed approaches for calibration of the upstrap had different strengths depending on the stopping rules used. Compared to O'Brien-Fleming group sequential methods, upstrapped approaches had type I error rates that differed by at most 1.7% and expected sample size was 2-22% lower in the null scenario, while in the alternative scenario power fluctuated between 15.7% lower and 0.2% higher and expected sample size was 0-15% lower.

CONCLUSIONS

In this proof-of-concept simulation study, we evaluated the potential for upstrapping as a resampling-based method for futility monitoring in clinical trials. The trade-offs in expected sample size, power, and type I error rate control indicate that the upstrap can be calibrated to implement futility monitoring with varying degrees of aggressiveness and that performance similarities can be identified relative to considered alpha-spending and conditional power futility monitoring methods.

Using Bayesian statistics in confirmatory clinical trials in the regulatory setting: a tutorial review

Bayesian statistics plays a pivotal role in advancing medical science by enabling healthcare companies, regulators, and stakeholders to assess the safety and efficacy of new treatments, interventions, and medical procedures. The Bayesian framework offers a unique advantage over the classical framework, especially when incorporating prior information into a new trial with quality external data, such as historical data or another source of co-data. In recent years, there has been a significant increase in regulatory submissions using Bayesian statistics due to its flexibility and ability to provide valuable insights for decision-making, addressing the modern complexity of clinical trials where frequentist trials are inadequate. For regulatory submissions, companies often need to consider the frequentist operating characteristics of the Bayesian analysis strategy, regardless of the design complexity. In particular, the focus is on the frequentist type I error rate and power for all realistic alternatives. This tutorial review aims to provide a comprehensive overview of the use of Bayesian statistics in sample size determination, control of type I error rate, multiplicity adjustments, external data borrowing, etc., in the regulatory environment of clinical trials. Fundamental concepts of Bayesian sample size determination and illustrative examples are provided to serve as a valuable resource for researchers, clinicians, and statisticians seeking to develop more complex and innovative designs.

Epistemic uncertainty in Bayesian predictive probabilities

Bayesian predictive probabilities have become a ubiquitous tool for design and monitoring of clinical trials. The typical procedure is to average predictive probabilities over the prior or posterior distributions. In this paper, we highlight the limitations of relying solely on averaging, and propose the reporting of intervals or quantiles for the predictive probabilities. These intervals formalize the intuition that uncertainty decreases with more information. We present four different applications (Phase 1 dose escalation, early stopping for futility, sample size re-estimation, and assurance/probability of success) to demonstrate the practicality and generality of the proposed approach.

Seamless phase 2/3 design for trials with multiple co-primary endpoints using Bayesian predictive power

Seamless phase 2/3 design has become increasingly popular in clinical trials with a single endpoint. Trials that define success based on the achievement of all co-primary endpoints (CPEs) encounter the challenge of inflated type 2 error rates, often leading to an overly large sample size. To tackle this challenge, we introduced a seamless phase 2/3 design strategy that employs Bayesian predictive power (BPP) for futility monitoring and sample size re-estimation at interim analysis. The correlations among multiple CPEs are incorporated using a Dirichlet-multinomial distribution. An alternative approach based on conditional power (CP) was also discussed for comparison. A seamless phase 2/3 vaccine trial employing four binary endpoints under the non-inferior hypothesis serves as an example. Our results spotlight that, in scenarios with relatively small phase 2 sample sizes (e.g., 50 or 100 subjects), the BPP approach either outperforms or matches the CP approach in terms of overall power. Particularly, with n1 = 50 and ρ = 0, BPP showcases an overall power advantage over CP by as much as 8.54%. Furthermore, when the phase 2 stage enrolled more subjects (e.g., 150 or 200), especially with a phase 2 sample size of 200 and ρ = 0, the BPP approach evidences a peak difference of 5.76% in early stop probability over the CP approach, emphasizing its better efficiency in terminating futile trials. It's noteworthy that both BPP and CP methodologies maintained type 1 error rates under 2.5%. In conclusion, the integration of the Dirichlet-Multinominal model with the BPP approach offers improvement in certain scenarios over the CP approach for seamless phase 2/3 trials with multiple CPEs.

Futility monitoring for randomized clinical trials with non-proportional hazards: An optimal conditional power approach

BACKGROUND

Standard futility analyses designed for a proportional hazards setting may have serious drawbacks when non-proportional hazards are present. One important type of non-proportional hazards occurs when the treatment effect is delayed. That is, there is little or no early treatment effect but a substantial later effect.

METHODS

We define optimality criteria for futility analyses in this setting and propose simple search procedures for deriving such rules in practice.

RESULTS

We demonstrate the advantages of the optimal rules over commonly used rules in reducing the average number of events, the average sample size, or the average study duration under the null hypothesis with minimal power loss under the alternative hypothesis.

CONCLUSION

Optimal futility rules can be derived for a non-proportional hazards setting that control the loss of power under the alternative hypothesis while maximizing the gain in early stopping under the null hypothesis.

Using short-term endpoints to improve interim decision making and trial duration in two-stage phase II trials with nested binary endpoints

In oncology, phase II clinical trials are often planned as single-arm two-stage designs with a binary endpoint, for example, progression-free survival after 12 months, and the option to stop for futility after the first stage. Simon's two-stage design is a very popular approach but depending on the follow-up time required to measure the patients' outcomes the trial may have to be paused undesirably long. To shorten this forced interruption, it was proposed to use a short-term endpoint for the interim decision, such as progression-free survival after 3 months. We show that if the assumptions for the short-term endpoint are misspecified, the decision-making in the interim can be misleading, resulting in a great loss of statistical power. For the setting of a binary endpoint with nested measurements, such as progression-free survival, we propose two approaches that utilize all available short-term and long-term assessments of the endpoint to guide the interim decision. One approach is based on conditional power and the other is based on Bayesian posterior predictive probability of success. In extensive simulations, we show that both methods perform similarly, when appropriately calibrated, and can greatly improve power compared to the existing approach in settings with slow patient recruitment. Software code to implement the methods is made publicly available.

Sample size re-estimation in Phase 2 dose-finding: Conditional power versus Bayesian predictive power

Unblinded sample size re-estimation (SSR) is often planned in a clinical trial when there is large uncertainty about the true treatment effect. For Proof-of Concept (PoC) in a Phase II dose finding study, contrast test can be adopted to leverage information from all treatment groups. In this article, we propose two-stage SSR designs using frequentist conditional power (CP) and Bayesian predictive power (PP) for both single and multiple contrast tests. The Bayesian SSR can be implemented under a wide range of prior settings to incorporate different prior knowledge. Taking the adaptivity into account, all type I errors of final analysis in this paper are rigorously protected. Simulation studies are carried out to demonstrate the advantages of unblinded SSR in multi-arm trials.

Optimal predictive probability designs for randomized biomarker-guided oncology trials

Introduction

Efforts to develop biomarker-targeted anti-cancer therapies have progressed rapidly in recent years. With efforts to expedite regulatory reviews of promising therapies, several targeted cancer therapies have been granted accelerated approval on the basis of evidence acquired in single-arm phase II clinical trials. And yet, in the absence of randomization, patient prognosis for progression-free survival and overall survival may not have been studied under standard of care chemotherapies for emerging biomarker subpopulations prior to the submission of an accelerated approval application. Historical control rates used to design and evaluate emerging targeted therapies often arise as population averages, lacking specificity to the targeted genetic or immunophenotypic profile. Thus, historical trial results are inherently limited for inferring the potential "comparative efficacy" of novel targeted therapies. Consequently, randomization may be unavoidable in this setting. Innovations in design methodology are needed, however, to enable efficient implementation of randomized trials for agents that target biomarker subpopulations.

Methods

This article proposes three randomized designs for early phase biomarker-guided oncology clinical trials. Each design utilizes the optimal efficiency predictive probability method to monitor multiple biomarker subpopulations for futility. Only designs with type I error between 0.05 and 0.1 and power of at least 0.8 were considered when selecting an optimal efficiency design from among the candidate designs formed by different combinations of posterior and predictive threshold. A simulation study motivated by the results reported in a recent clinical trial studying atezolizumab treatment in patients with locally advanced or metastatic urothelial carcinoma is used to evaluate the operating characteristics of the various designs.

Results

Out of a maximum of 300 total patients, we find that the enrichment design has an average total sample size under the null of 101.0 and a total average sample size under the alternative of 218.0, as compared to 144.8 and 213.8 under the null and alternative, respectively, for the stratified control arm design. The pooled control arm design enrolled a total of 113.2 patients under the null and 159.6 under the alternative, out of a maximum of 200. These average sample sizes that are 23-48% smaller under the alternative and 47-64% smaller under the null, as compared to the realized sample size of 310 patients in the phase II study of atezolizumab.

Discussion

Our findings suggest that potentially smaller phase II trials to those used in practice can be designed using randomization and futility stopping to efficiently obtain more information about both the treatment and control groups prior to phase III study.

Basket Trials: Review of Current Practice and Innovations for Future Trials

Advances in biology and immunology have elucidated genetic and immunologic origins of cancer. Innovations in sequencing technologies revealed that distinct cancer histologies shared common genetic and immune phenotypic traits. Pharmacologic developments made it possible to target these alterations, yielding novel classes of targeted agents whose therapeutic potential span multiple tumor types. Basket trials, one type of master protocol, emerged as a tool for evaluating biomarker-targeted therapies among multiple tumor histologies. Conventionally conducted within the phase II setting and designed to estimate high and durable objective responses, basket trials pose challenges to statistical design and interpretation of results. This article reviews basket trials implemented in oncology studies and discusses issues related to their statistical design and analysis.

Bayesian and frequentist approaches to sequential monitoring for futility in oncology basket trials: A comparison of Simon's two-stage design and Bayesian predictive probability monitoring with information sharing across baskets

This article discusses and compares statistical designs of basket trial, from both frequentist and Bayesian perspectives. Baskets trials are used in oncology to study interventions that are developed to target a specific feature (often genetic alteration or immune phenotype) that is observed across multiple tissue types and/or tumor histologies. Patient heterogeneity has become pivotal to the development of non-cytotoxic treatment strategies. Treatment targets are often rare and exist among several histologies, making prospective clinical inquiry challenging for individual tumor types. More generally, basket trials are a type of master protocol often used for label expansion. Master protocol is used to refer to designs that accommodates multiple targets, multiple treatments, or both within one overarching protocol. For the purpose of making sequential decisions about treatment futility, Simon's two-stage design is often embedded within master protocols. In basket trials, this frequentist design is often applied to independent evaluations of tumor histologies and/or indications. In the tumor agnostic setting, rarer indications may fail to reach the sample size needed for even the first evaluation for futility. With recent innovations in Bayesian methods, it is possible to evaluate for futility with smaller sample sizes, even for rarer indications. Novel Bayesian methodology for a sequential basket trial design based on predictive probability is introduced. The Bayesian predictive probability designs allow interim analyses with any desired frequency, including continual assessments after each patient observed. The sequential design is compared with and without Bayesian methods for sharing information among a collection of discrete, and potentially non-exchangeable tumor types. Bayesian designs are compared with Simon's two-stage minimax design.

Early Termination of Clinical Trials for Futility - Considerations for a Data and Safety Monitoring Board

Early Termination of Clinical Trials for FutilityClinical trials may be stopped for futility if there is little or no chance of demonstrating the hoped-for effect. Reasons include evidence of no treatment effect, substantial missing data that would unacceptably undermine trial conclusions, or event rates too low to support meaningful comparisons. This review examines issues faced by DSMBs in such settings.

Optimal unplanned design modification in adaptive two-stage trials

Adaptive planning of clinical trials allows modifying the entire trial design at any time point mid-course. In this paper, we consider the case when a trial-external update of the planning assumptions during the ongoing trial makes an unforeseen design adaptation necessary. We take up the idea to construct adaptive designs with defined features by solving an optimization problem and apply it to the situation of unplanned design reassessment. By using the conditional error principle, we present an approach on how to optimally modify the trial design at an unplanned interim analysis while at the same time strictly protecting the type I error rate. This linking of optimal design planning and the conditional error principle allows sound reactions to unforeseen events that make a design reassessment necessary.

Optimal Sequential Predictive Probability Designs for Early-Phase Oncology Expansion Cohorts

PURPOSE

The customary approach to early-phase clinical trial design, where the focus is on identification of the maximum tolerated dose, is not always suitable for noncytotoxic or other targeted therapies. Many trials have continued to follow the 3 + 3 dose-escalation design, but with the addition of phase I dose-expansion cohorts to further characterize safety and assess efficacy. Dose-expansion cohorts are not always planned in advance nor rigorously designed. We introduce an approach to the design of phase I expansion cohorts on the basis of sequential predictive probability monitoring.

METHODS

Two optimization criteria are proposed that allow investigators to stop for futility to preserve limited resources while maintaining traditional control of type I and type II errors. We demonstrate the use of these designs through simulation, and we elucidate their implementation with a redesign of the phase I expansion cohort for atezolizumab in metastatic urothelial carcinoma.

RESULTS

A sequential predictive probability design outperforms Simon's two-stage designs and posterior probability monitoring with respect to both proposed optimization criteria. The Bayesian sequential predictive probability design yields increased power while significantly reducing the average sample size under the null hypothesis in the context of the case study, whereas the original study design yields too low type I error and power. The optimal efficiency design tended to have more desirable properties, subject to constraints on type I error and power, compared with the optimal accuracy design.

CONCLUSION

The optimal efficiency design allows investigators to preserve limited financial resources and to maintain ethical standards by halting potentially large dose-expansion cohorts early in the absence of promising efficacy results, while maintaining traditional control of type I and II error rates.

The Futility of Futility Analyses in Adjuvant Trials in Hormone Receptor Positive Breast Cancer

An interim analysis is commonly used in phase III superiority trials to compare treatment arms, with the goal of terminating exposure of patients to ineffective or unsafe drugs, or to identify highly effective therapies for earlier public disclosure. Traditionally, interim analyses have been designed to identify early evidence of extremely large benefit of the experimental approach, potentially leading to early dissemination of effective treatments. Increasingly, interim analysis has also involved analysis of futility which may lead to early termination of a trial that will not yield additional useful information This presents an important challenge in early-stage hormone receptor positive breast cancer, where recurrence often occurs late, with a steady annual event rate up to 20 years. Early analysis of events may miss late treatment effects that can be observed only with longer follow-up. We discuss approaches to futility analysis in adjuvant clinical trials in hormone receptor positive breast cancer, the role of the Data Safety Monitoring Committee in such analyses, considerations of the potential harms versus benefits of treatment, and the risks of continuing versus early stopping of a trial.

The use of external control data for predictions and futility interim analyses in clinical trials

BACKGROUND

External control (EC) data from completed clinical trials and electronic health records can be valuable for the design and analysis of future clinical trials. We discuss the use of EC data for early stopping decisions in randomized clinical trials (RCTs).

METHODS

We specify interim analyses (IAs) approaches for RCTs, which allow investigators to integrate external data into early futility stopping decisions. IAs utilize predictions based on early data from the RCT, possibly combined with external data. These predictions at IAs express the probability that the trial will generate significant evidence of positive treatment effects. The trial is discontinued if this predictive probability becomes smaller than a prespecified threshold. We quantify efficiency gains and risks associated with the integration of external data into interim decisions. We then analyze a collection of glioblastoma (GBM) data sets, to investigate if the balance of efficiency gains and risks justify the integration of external data into the IAs of future GBM RCTs.

RESULTS

Our analyses illustrate the importance of accounting for potential differences between the distributions of prognostic variables in the RCT and in the external data to effectively leverage external data for interim decisions. Using GBM data sets, we estimate that the integration of external data increases the probability of early stopping of ineffective experimental treatments by up to 25% compared to IAs that do not leverage external data. Additionally, we observe a reduction of the probability of early discontinuation for effective experimental treatments, which improves the RCT power.

CONCLUSION

Leveraging external data for IAs in RCTs can support early stopping decisions and reduce the number of enrolled patients when the experimental treatment is ineffective.

Bayesian Sequential Monitoring of Single-Arm Trials: A Comparison of Futility Rules Based on Binary Data

In clinical trials, futility rules are widely used to monitor the study while it is in progress, with the aim of ensuring early termination if the experimental treatment is unlikely to provide the desired level of efficacy. In this paper, we focus on Bayesian strategies to perform interim analyses in single-arm trials based on a binary response variable. Designs that exploit both posterior and predictive probabilities are described and a slight modification of the futility rules is introduced when a fixed historical response rate is used, in order to add uncertainty in the efficacy probability of the standard treatment through the use of prior distributions. The stopping boundaries of the designs are compared under the same trial settings and simulation studies are performed to evaluate the operating characteristics when analogous procedures are used to calibrate the probability cut-offs of the different decision rules.

Stopping rules for multi-category computerized classification testing

Computerized classification testing (CCT) aims to classify persons into one of two or more possible categories to make decisions such as mastery/non-mastery or meet most/meet all/exceed. A defining feature of CCT is its stopping criterion: the test terminates when there is enough confidence to make a decision. There is abundant research on CCT with a single cut-off, and two common stopping criteria are the sequential probability ratio test (SPRT) statistic and the generalized likelihood ratio statistic (GLR). However, there is a relative scarcity of research extending the SPRT to the multi-hypothesis case for when there is more than one cut-off. In this paper, we propose a new multi-category GLR (mGLR) statistic as well as a stochastically curtailed version of the CCT with three or more categories. A simulation study was conducted to show that the mGLR statistic outperformed the existing stopping rules by generating shorter average test length without sacrificing classification accuracy. Results also revealed that the stochastically curtailed mGLR successfully increased test efficiency in certain testing conditions.

Improving sample size recalculation in adaptive clinical trials by resampling

Sample size calculations in clinical trials need to be based on profound parameter assumptions. Wrong parameter choices may lead to too small or too high sample sizes and can have severe ethical and economical consequences. Adaptive group sequential study designs are one solution to deal with planning uncertainties. Here, the sample size can be updated during an ongoing trial based on the observed interim effect. However, the observed interim effect is a random variable and thus does not necessarily correspond to the true effect. One way of dealing with the uncertainty related to this random variable is to include resampling elements in the recalculation strategy. In this paper, we focus on clinical trials with a normally distributed endpoint. We consider resampling of the observed interim test statistic and apply this principle to several established sample size recalculation approaches. The resulting recalculation rules are smoother than the original ones and thus the variability in sample size is lower. In particular, we found that some resampling approaches mimic a group sequential design. In general, incorporating resampling of the interim test statistic in existing sample size recalculation rules results in a substantial performance improvement with respect to a recently published conditional performance score.

Some design considerations incorporating early futility for single-arm clinical trials with time-to-event primary endpoints using Weibull distribution

Sample size calculation is an essential component of the planning phase of a clinical trial. In the context of single-arm clinical trials with time-to-event (TTE) endpoints, only a few options with limited design features are available. Motivated from ethical or practical considerations, two-stage designs are implemented for single-arm studies to obtain early evidence of futility. A major drawback of such designs is that early stopping may only occur at the conclusion of the first stage, even if lack of efficacy becomes apparent at any other time point over the course of the clinical trial. In this manuscript, we attempt to fill some existing gaps in the literature related to single-arm clinical trials with TTE endpoints. We propose a parametric maximum likelihood estimate-based test whose variance component accounts for the expected proportion of loss to follow-up and different accrual patterns (early, late, or uniform accrual). For the proposed method, we present three stochastic curtailment methods (conditional power, predictive power, Bayesian predictive probability) which can be employed for efficacy or futility testing purposes. Finally, we discuss the implementation of group sequential designs for obtaining an early evidence of efficacy or futility at pre-planned timings of interim analyses. Through extensive simulations, it is shown that our proposed method performs well for designing these studies with moderate to large sample sizes. Some examples are presented to demonstrate various aspects of the stochastic curtailment and repeated significance testing methods presented in this manuscript.

Functional data analysis and prediction tools for continuous glucose-monitoring studies

Introduction

To identify phenotypes of type 1 diabetes based on glucose curves from continuous glucose-monitoring (CGM) using functional data (FD) analysis to account for longitudinal glucose patterns. We present a reliable prediction model that can accurately predict glycemic levels based on past data collected from the CGM sensor and real-time risk of hypo-/hyperglycemic for individuals with type 1 diabetes.

Methods

A longitudinal cohort study of 443 type 1 diabetes patients with CGM data from a completed trial. The FD analysis approach, sparse functional principal components (FPCs) analysis was used to identify phenotypes of type 1 diabetes glycemic variation. We employed a nonstationary stochastic linear mixed-effects model (LME) that accommodates between-patient and within-patient heterogeneity to predict glycemic levels and real-time risk of hypo-/hyperglycemic by creating specific target functions for these excursions.

Results

The majority of the variation (73%) in glucose trajectories was explained by the first two FPCs. Higher order variation in the CGM profiles occurred during weeknights, although variation was higher on weekends. The model has low prediction errors and yields accurate predictions for both glucose levels and real-time risk of glycemic excursions.

Conclusions

By identifying these distinct longitudinal patterns as phenotypes, interventions can be targeted to optimize type 1 diabetes management for subgroups at the highest risk for compromised long-term outcomes such as cardiac disease or stroke. Further, the estimated change/variability in an individual's glucose trajectory can be used to establish clinically meaningful and patient-specific thresholds that, when coupled with probabilistic predictive inference, provide a useful medical-monitoring tool.

Do we need to adjust for interim analyses in a Bayesian adaptive trial design?

Background

Bayesian adaptive methods are increasingly being used to design clinical trials and offer several advantages over traditional approaches. Decisions at analysis points are usually based on the posterior distribution of the treatment effect. However, there is some confusion as to whether control of type I error is required for Bayesian designs as this is a frequentist concept.

Methods

We discuss the arguments for and against adjusting for multiplicities in Bayesian trials with interim analyses. With two case studies we illustrate the effect of including interim analyses on type I/II error rates in Bayesian clinical trials where no adjustments for multiplicities are made. We propose several approaches to control type I error, and also alternative methods for decision-making in Bayesian clinical trials.

Results

In both case studies we demonstrated that the type I error was inflated in the Bayesian adaptive designs through incorporation of interim analyses that allowed early stopping for efficacy and without adjustments to account for multiplicity. Incorporation of early stopping for efficacy also increased the power in some instances. An increase in the number of interim analyses that only allowed early stopping for futility decreased the type I error, but also decreased power. An increase in the number of interim analyses that allowed for either early stopping for efficacy or futility generally increased type I error and decreased power.

Conclusions

Currently, regulators require demonstration of control of type I error for both frequentist and Bayesian adaptive designs, particularly for late-phase trials. To demonstrate control of type I error in Bayesian adaptive designs, adjustments to the stopping boundaries are usually required for designs that allow for early stopping for efficacy as the number of analyses increase. If the designs only allow for early stopping for futility then adjustments to the stopping boundaries are not needed to control type I error. If one instead uses a strict Bayesian approach, which is currently more accepted in the design and analysis of exploratory trials, then type I errors could be ignored and the designs could instead focus on the posterior probabilities of treatment effects of clinically-relevant values.

Determining a Bayesian predictive power stopping rule for futility in a non-inferiority trial with binary outcomes

Background/Aims

Non-inferiority trials investigate whether a novel intervention, which typically has other benefits (i.e., cheaper or safer), has similar clinical effectiveness to currently available treatments. In situations where interim evidence in a non-inferiority trial suggests that the novel treatment is truly inferior, ethical concerns with continuing randomisation to the “inferior” intervention are raised. Thus, if interim data indicate that concluding non-inferiority at the end of the trial is unlikely, stopping for futility should be considered. To date, limited examples are available to guide the development of stopping rules for non-inferiority trials.

Methods

We used a Bayesian predictive power approach to develop a stopping rule for futility for a trial collecting binary outcomes. We evaluated the frequentist operating characteristics of the stopping rule to ensure control of the Type I and Type II error. Our case study is the Intranasal Ketamine for Procedural Sedation trial (INK trial), a non-inferiority trial designed to assess the sedative properties of ketamine administered using two alternative routes.

Results

We considered implementing our stopping rule after the INK trial enrols 140 patients out of 560. The trial would be stopped if 12 more patients experience a failure on the novel treatment compared to standard care. This trial has a type I error rate of 2.2% and a power of 80%.

Conclusions

Stopping for futility in non-inferiority trials reduces exposure to ineffective treatments and preserves resources for alternative research questions. Futility stopping rules based on Bayesian predictive power are easy to implement and align with trial aims.

Trial registration

ClinicalTrials.gov NCT02828566 July 11, 2016.

•

It is important to consider stopping for futility in non-inferiority trials.

•

We develop a rule to stop a non-inferiority trial using Bayesian predictive power.

•

We provide code and an online application to implement this method.

•

We reduce the complexity of developing stopping rules in non-inferiority trials.

Dynamic predictive probabilities to monitor rapid cystic fibrosis disease progression

Cystic fibrosis (CF) is a progressive, genetic disease characterized by frequent, prolonged drops in lung function. Accurately predicting rapid underlying lung-function decline is essential for clinical decision support and timely intervention. Determining whether an individual is experiencing a period of rapid decline is complicated due to its heterogeneous timing and extent, and error component of the measured lung function. We construct individualized predictive probabilities for "nowcasting" rapid decline. We assume each patient's true longitudinal lung function, S(t), follows a nonlinear, nonstationary stochastic process, and accommodate between-patient heterogeneity through random effects. Corresponding lung-function decline at time t is defined as the rate of change, S'(t). We predict S'(t) conditional on observed covariate and measurement history by modeling a measured lung function as a noisy version of S(t). The method is applied to data on 30 879 US CF Registry patients. Results are contrasted with a currently employed decision rule using single-center data on 212 individuals. Rapid decline is identified earlier using predictive probabilities than the center's currently employed decision rule (mean difference: 0.65 years; 95% confidence interval (CI): 0.41, 0.89). We constructed a bootstrapping algorithm to obtain CIs for predictive probabilities. We illustrate real-time implementation with R Shiny. Predictive accuracy is investigated using empirical simulations, which suggest this approach more accurately detects peak decline, compared with a uniform threshold of rapid decline. Median area under the ROC curve estimates (Q1-Q3) were 0.817 (0.814-0.822) and 0.745 (0.741-0.747), respectively, implying reasonable accuracy for both. This article demonstrates how individualized rate of change estimates can be coupled with probabilistic predictive inference and implementation for a useful medical-monitoring approach.

Interim Monitoring for Futility in Clinical Trials with Two Co-primary Endpoints Using Prediction

We discuss using prediction as a flexible and practical approach for monitoring futility in clinical trials with two co-primary endpoints. This approach is appealing in that it provides quantitative evaluation of potential effect sizes and associated precision, and can be combined with flexible error-spending strategies. We extend prediction of effect size estimates and the construction of predicted intervals to the two co-primary endpoints case, and illustrate interim futility monitoring of treatment effects using prediction with an example. We also discuss alternative approaches based on the conditional and predictive powers, compare these methods and provide some guidance on the use of prediction for better decision in clinical trials with co-primary endpoints.

A Randomized, Placebo-Controlled Trial to Assess the Effects of 8 Weeks of Administration of GSK256073, a Selective GPR109A Agonist, on High-Density Lipoprotein Cholesterol in Subjects With Dyslipidemia

GPR109A (HM74A), a G-protein-coupled receptor, is hypothesized to mediate lipid and lipoprotein changes and dermal flushing associated with niacin administration. GSK256073 (8-chloro-3-pentyl-1H-purine-2,6[3H,7H]-dione) is a selective GPR109A agonist shown to suppress fatty acid levels and produce mild flushing in short-term clinical studies. This study evaluated the effects of GSK256073 on lipids in subjects with low high-density lipoprotein cholesterol (HDLc). Subjects (n = 80) were randomized (1:1:1:1) to receive GSK256073 5, 50, or 150 mg/day or matching placebo for 8 weeks. The primary end point was determining the GSK256073 exposure-response relationship for change from baseline in HDLc. No significant exposure response was observed between GSK256073 and HDLc levels. GSK256073 did not significantly alter HDLc levels versus placebo, but rather revealed a trend at the 150-mg dose for a nonsignificant decrease in HDLc (-6.31%; P = .12) and an increase in triglycerides (median, 24.4%; 95% confidence interval, 7.3%-41.6%). Flushing was reported in 21%, 25%, and 60% of subjects (5, 50, and 150 mg, respectively) versus 24% for placebo. Results indicated that selective activation of the GPR109A receptor with GSK256073 did not produce niacin-like lipid effects. These findings add to the increasing evidence that niacin-mediated lipoprotein changes occur predominantly via GPR109A-independent pathways.

Trial Refresh: A Case for an Adaptive Platform Trial for Pulmonary Exacerbations of Cystic Fibrosis

Cystic fibrosis is a genetic disease typically characterized by progressive lung damage and premature mortality. Pulmonary exacerbations, or flare-ups of the lung disease, often require hospitalization for intensive treatment. Approximately 25% of patients with cystic fibrosis do not recover their baseline lung function after pulmonary exacerbations. There is a relative paucity of evidence to inform treatment strategies for exacerbations. Compounding this lack of evidence, there are a large number of treatment options already as well as becoming available. This results in significant variability between medication regimens prescribed by different physicians, treatment centers and regions with potentially adverse impact to patients. The conventional strategy is to undertake essential randomized clinical trials to inform treatment decisions and improve outcomes for patients with exacerbations. However, over the past several decades, clinical trials have generally failed to provide information critical to improved treatment and management of exacerbations. Bayesian adaptive platform trials hold the promise of addressing clinical uncertainties and informing treatment. Using modeling and response adaptive randomization, they allow for the evaluation of multiple treatments across different management domains, and progressive improvement in patient outcomes throughout the course of the trial. Bayesian adaptive platform trials require substantial amounts of preparation. Basic preparation includes extensive stakeholder involvement including elicitation of consumer preferences and clinician understanding of the research topic, defining the research questions, determining the best outcome measures, delineating study sub-groups, in depth statistical modeling, designing end-to-end digital solutions seamlessly supporting clinicians, researchers and patients, constructing randomisation algorithms and importantly, defining pre-determined intra-study end-points. This review will discuss the motivation and necessary steps required to embark on a Bayesian adaptive platform trial to optimize medication regimens for the treatment of pulmonary exacerbations of cystic fibrosis.

A predictive probability interim design for phase II clinical trials with continuous endpoints

Molecular targeted therapies come often with lower toxicity profiles than traditional cytotoxic treatments, thus shifting drug development paradigm into establishing evidence of biological activity, target modulation, and pharmacodynamics effects of these therapies in early phase trials. Therefore, these trials need to address simultaneous evaluation of safety, proof-of-concept biological marker activity, or changes in continuous tumor size instead of binary response rate. Interim analyses are typically incorporated in the trial due to concerns regarding excessive toxicity and ineffective new treatment. There is a lack of interim strategies developed to monitor futility and/or efficacy for these types of continuous outcomes, especially in single-arm phase II trials. We propose a 2-stage design based on predictive probability to accommodate continuous endpoints, assuming a normal distribution with known variance. Simulation results and case study demonstrated that the proposed design can incorporate an interim stop for futility as well as for efficacy while maintaining desirable design properties. As expected, using continuous tumor size resulted in reduced sample sizes for both optimal and minimax designs. A limited exploration of various priors was performed and shown to be robust. As research rapidly moves to incorporate more molecular targeted therapies, it will accommodate new types of outcomes while allowing for flexible stopping rules to continue optimizing trial resources and prioritize agents with compelling early phase data.

Predictive probability methods for interim monitoring in clinical trials with longitudinal outcomes

In clinical research and development, interim monitoring is critical for better decision-making and minimizing the risk of exposing patients to possible ineffective therapies. For interim futility or efficacy monitoring, predictive probability methods are widely adopted in practice. Those methods have been well studied for univariate variables. However, for longitudinal studies, predictive probability methods using univariate information from only completers may not be most efficient, and data from on-going subjects can be utilized to improve efficiency. On the other hand, leveraging information from on-going subjects could allow an interim analysis to be potentially conducted once a sufficient number of subjects reach an earlier time point. For longitudinal outcomes, we derive closed-form formulas for predictive probabilities, including Bayesian predictive probability, predictive power, and conditional power and also give closed-form solutions for predictive probability of success in a future trial and the predictive probability of success of the best dose. When predictive probabilities are used for interim monitoring, we study their distributions and discuss their analytical cutoff values or stopping boundaries that have desired operating characteristics. We show that predictive probabilities utilizing all longitudinal information are more efficient for interim monitoring than that using information from completers only. To illustrate their practical application for longitudinal data, we analyze 2 real data examples from clinical trials.

Finding the Optimal volume and intensity of Resistance Training Exercise for Type 2 Diabetes: The FORTE Study, a Randomized Trial

AIM

To compare different volumes and intensities of resistance training (RT) combined with aerobic training (AT) for improvements in glycemic control and cardiovascular health for persons with type 2 diabetes (T2DM).

METHODS

Participants with T2DM were stratified by HbA1c and randomized: "usual care" (RT1), which consisted of moderate intensity (50% 1-repetition maximum [1-RM]), low volumeRT (initiated half-way through program); higher intensity (75% 1-RM) and higher volume (initiated at program onset) RT (RT2); or moderate intensity but higher volume RT (RT3). RT sets and repetitions were adjusted to maintain similar work and volume between RT2 and RT3. Walking or cycling (60-80% aerobic capacity)was prescribed 5 times/week, and RT was prescribed 2 times/week. An ANCOVA, adjusted for baseline and gender, assessed changes post-6months in glycemic control (HbA1c- primary outcome), aerobic capacity and anthropometrics.

RESULTS

Sixty-two participants (52.3±1.2years, 48% female) were randomized (RT1, n=20; RT2, n=20; RT3, n=22). Only post-training fasting glucose, without significant HbA1c change, was different between groups (RT1-RT3=-1.7mmol/L, p=0.046). Pre-post differences were found in pooled HbA1c (7.4±0.2%[57±2.2mmol/mol] vs. 6.7±0.2%[50±2.2mmol/mol], p<0.001), aerobic capacity (21.5±0.8vs. 25.2±0.8ml/kg/min, p<0.001), body mass (84.0±2.7vs. 83.0±2.7kg, p=0.022[DXA]), body mass index (30.8±0.9vs. 30.3±0.8kg/m², p=0.02) and body fat (32.3±1.1vs. 31.3±1.2%, p<0.001). The trial was discontinued early; no HbA1c advantage was found with either RT2 or RT3 over RT1.

CONCLUSIONS

Combined AT+RT exercise improved glycemic control, cardiovascular risk factors and body composition after 6months for participants with T2DM, but differential effects between the prescribed intensities and volumes of RT were not found to effect HbA1c.

Treatment of spontaneous preterm labour with retosiban: a phase II pilot dose-ranging study

Aims

The aims of the present study were to investigate the maternal, fetal and neonatal safety and tolerability, pharmacodynamics and pharmacokinetics of intravenous (IV) retosiban in pregnant women with spontaneous preterm labour (PTL) between 34^0/7 and 35^6/7 weeks' gestation.

Methods

In parts A and B of a three‐part, double‐blind, placebo‐controlled, multicentre study, women were randomized 3:1 (Part A) or 2:1 (Part B) to either 12‐h IV retosiban followed by a single dose of oral placebo (R‐P) or 12‐h IV placebo followed by single‐dose oral retosiban (P‐R).

Results

A total of 29 women were randomized; 20 to R‐P and nine to P‐R. An integrated analysis found that adverse events were infrequent in mothers/newborns and consistent with events expected in the population under study or associated with confounding factors. Retosiban was rapidly absorbed after oral administration, with an observed half‐life of 1.45 h. Efficacy analyses included 19 women. While not statistically significant, those receiving R‐P more frequently achieved uterine quiescence in 6 h (R‐P, 63%; 95% credible interval [CrI]: 38, 84; P‐R, 43%; 95% CrI: 12, 78) and more achieved a reduction of ≥50% in uterine contractions in 6 h (R‐P, 63%; 95% CrI: 38, 84; P‐R, 29%; 95% CrI: 4, 64). The number of days to delivery was increased in women receiving R‐P (median 26 days for R‐P vs. 13 days for P‐R).

Conclusions

Intravenous retosiban has a favourable safety and tolerability profile and might prolong pregnancies in women with PTL. The study provides the rationale and dosing strategy for further evaluation of the efficacy of retosiban in the treatment of PTL.

Medical informatics research trend analysis: A text mining approach

The objective of this research is to identify major subject areas of medical informatics and explore the time-variant changes therein. As such it can inform the field about where medical informatics research has been and where it is heading. Furthermore, by identifying subject areas, this study identifies the development trends and the boundaries of medical informatics as an academic field. To conduct the study, first we identified 26,307 articles in PubMed archives which were published in the top medical informatics journals within the timeframe of 2002 to 2013. And then, employing a text mining -based semi-automated analytic approach, we clustered major research topics by analyzing the most frequently appearing subject terms extracted from the abstracts of these articles. The results indicated that some subject areas, such as biomedical, are declining, while other research areas such as health information technology (HIT), Internet-enabled research, and electronic medical/health records (EMR/EHR), are growing. The changes within the research subject areas can largely be attributed to the increasing capabilities and use of HIT. The Internet, for example, has changed the way medical research is conducted in the health care field. While discovering new medical knowledge through clinical and biological experiments is important, the utilization of EMR/EHR enabled the researchers to discover novel medical insight buried deep inside massive data sets, and hence, data analytics research has become a common complement in the medical field, rapidly growing in popularity.

Oral Vancomycin Followed by Fecal Transplantation Versus Tapering Oral Vancomycin Treatment for Recurrent Clostridium difficile Infection: An Open-Label, Randomized Controlled Trial

Background

Fecal transplantation (FT) is a promising treatment for recurrent Clostridium difficile infection (CDI), but its true effectiveness remains unknown. We compared 14 days of oral vancomycin followed by a single FT by enema with oral vancomycin taper (standard of care) in adult patients experiencing acute recurrence of CDI.

Methods

In a phase 2/3, single-center, open-label trial, participants from Ontario, Canada, experiencing recurrence of CDI were randomly assigned in a 1:1 ratio to 14 days of oral vancomycin treatment followed by a single 500-mL FT by enema, or a 6-week taper of oral vancomycin. Patients with significant immunocompromise, history of fulminant CDI, or irreversible bleeding disorders were excluded. The primary endpoint was CDI recurrence within 120 days. Microbiota analysis was performed on fecal filtrate from donors and stool samples from FT recipients, as available.

Results

The study was terminated at the interim analysis after randomizing 30 patients. Nine of 16 (56.2%) patients who received FT and 5 of 12 (41.7%) in the vancomycin taper group experienced recurrence of CDI, corresponding with symptom resolution in 43.8% and 58.3%, respectively. Fecal microbiota analysis of 3 successful FT recipients demonstrated increased diversity. A futility analysis did not support continuing the study. Adverse events were similar in both groups and uncommon.

Conclusions

In patients experiencing an acute episode of recurrent CDI, a single FT by enema was not significantly different from oral vancomycin taper in reducing recurrent CDI. Further research is needed to explore optimal donor selection, FT preparation, route, timing, and number of administrations.

Clinical Trials Registration

NCT01226992.

Advantages of Bayesian monitoring methods in deciding whether and when to stop a clinical trial: an example of a neonatal cooling trial

BACKGROUND

Decisions to stop randomized trials are often based on traditional P value thresholds and are often unconvincing to clinicians. To familiarize clinical investigators with the application and advantages of Bayesian monitoring methods, we illustrate the steps of Bayesian interim analysis using a recent major trial that was stopped based on frequentist analysis of safety and futility.

METHODS

We conducted Bayesian reanalysis of a factorial trial in newborn infants with hypoxic-ischemic encephalopathy that was designed to investigate whether outcomes would be improved by deeper (32 °C) or longer cooling (120 h), as compared with those achieved by standard whole body cooling (33.5 °C for 72 h). Using prior trial data, we developed neutral and enthusiastic prior probabilities for the effect on predischarge mortality, defined stopping guidelines for a clinically meaningful effect, and derived posterior probabilities for predischarge mortality.

RESULTS

Bayesian relative risk estimates for predischarge mortality were closer to 1.0 than were frequentist estimates. Posterior probabilities suggested increased predischarge mortality (relative risk > 1.0) for the three intervention groups; two crossed the Bayesian futility threshold.

CONCLUSIONS

Bayesian analysis incorporating previous trial results and different pre-existing opinions can help interpret accruing data and facilitate informed stopping decisions that are likely to be meaningful and convincing to clinicians, meta-analysts, and guideline developers.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01192776 . Registered on 31 August 2010.

Bayesian predictive power: choice of prior and some recommendations for its use as probability of success in drug development

Bayesian predictive power, the expectation of the power function with respect to a prior distribution for the true underlying effect size, is routinely used in drug development to quantify the probability of success of a clinical trial. Choosing the prior is crucial for the properties and interpretability of Bayesian predictive power. We review recommendations on the choice of prior for Bayesian predictive power and explore its features as a function of the prior. The density of power values induced by a given prior is derived analytically and its shape characterized. We find that for a typical clinical trial scenario, this density has a u-shape very similar, but not equal, to a β-distribution. Alternative priors are discussed, and practical recommendations to assess the sensitivity of Bayesian predictive power to its input parameters are provided. Copyright © 2016 John Wiley & Sons, Ltd.