A unified Bayesian framework for bias adjustment in multiple comparisons from clinical trials

In clinical trials, multiple comparisons arising from various treatments/doses, subgroups, or endpoints are common. Typically, trial teams focus on the comparison showing the largest observed treatment effect, often involving a specific treatment pair and endpoint within a subgroup. These findings frequently lead to follow-up pivotal studies, many of which do not confirm the initial positive results. Selection bias occurs when the most promising treatment, subgroup, or endpoint is chosen for further development, potentially skewing subsequent investigations. Such bias can be defined as the deviation in the observed treatment effects from the underlying truth. In this article, we propose a general and unified Bayesian framework to address selection bias in clinical trials with multiple comparisons. Our approach does not require a priori specification of a parametric distribution for the prior, offering a more flexible and generalized solution. The proposed method facilitates a more accurate interpretation of clinical trial results by adjusting for such selection bias. Through simulation studies, we compared several methods and demonstrated their superior performance over the normal shrinkage estimator. We recommended the use of Bayesian Model Averaging estimator averaging over Gaussian Mixture Models as the prior distribution based on its performance and flexibility. We applied the method to a multicenter, randomized, double-blind, placebo-controlled study investigating the cardiovascular effects of dulaglutide.

Adjusting for treatment selection in phase II/III clinical trials with time to event data

Phase II/III clinical trials are efficient two-stage designs that test multiple experimental treatments. In stage 1, patients are allocated to the control and all experimental treatments, with the data collected from them used to select experimental treatments to continue to stage 2. Patients recruited in stage 2 are allocated to the selected treatments and the control. Combined data of stage 1 and stage 2 are used for a confirmatory phase III analysis. Appropriate analysis needs to adjust for selection bias of the stage 1 data. Point estimators exist for normally distributed outcome data. Extending these estimators to time to event data is not straightforward because treatment selection is based on correlated treatment effects and stage 1 patients who do not get events in stage 1 are followed-up in stage 2. We have derived an approximately uniformly minimum variance conditional unbiased estimator (UMVCUE) and compared its biases and mean squared errors to existing bias adjusted estimators. In simulations, one existing bias adjusted estimator has similar properties as the practically unbiased UMVCUE while the others can have noticeable biases but they are less variable than the UMVCUE. For confirmatory phase II/III clinical trials where unbiased estimators are desired, we recommend the UMVCUE or the existing estimator with which it has similar properties.

Point estimation for adaptive trial designs I: A methodological review

Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value," and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realized trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This article is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this article can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.

Estimation of the treatment effect following a clinical trial that stopped early for benefit

When a clinical trial stops early for benefit, the maximum likelihood estimate (MLE) of the treatment effect may be subject to overestimation bias. Several authors have proposed adjusting for this bias using the conditional MLE, which is obtained by conditioning on early stopping. However, this approach has a fundamental problem in that the adjusted estimate may not be in the direction of benefit, even though the study has stopped early due to benefit. In this paper, we address this problem by embedding both the MLE and the conditional MLE within a broader class of penalised likelihood estimates, and choosing a member of the class that is a favourable compromise between the two. This penalised MLE, and its associated confidence interval, always lie in the direction of benefit when the study stops early for benefit. We study its properties using both simulations and analyses of the ENZAMET trial in metastatic prostate cancer. Conditional on stopping early for benefit, the method is found to have good unbiasedness and coverage properties, along with very favourable efficiency at earlier interim analyses. We recommend the penalised MLE as a supplementary analysis to a conventional primary analysis when a clinical trial stops early for benefit.

The benefits of covariate adjustment for adaptive multi-arm designs

Covariate adjustment via a regression approach is known to increase the precision of statistical inference when fixed trial designs are employed in randomized controlled studies. When an adaptive multi-arm design is employed with the ability to select treatments, it is unclear how covariate adjustment affects various aspects of the study. Consider the design framework that relies on pre-specified treatment selection rule(s) and a combination test approach for hypothesis testing. It is our primary goal to evaluate the impact of covariate adjustment on adaptive multi-arm designs with treatment selection. Our secondary goal is to show how the Uniformly Minimum Variance Conditionally Unbiased Estimator can be extended to account for covariate adjustment analytically. We find that adjustment with different sets of covariates can lead to different treatment selection outcomes and hence probabilities of rejecting hypotheses. Nevertheless, we do not see any negative impact on the control of the familywise error rate when covariates are included in the analysis model. When adjusting for covariates that are moderately or highly correlated with the outcome, we see various benefits to the analysis of the design. Conversely, there is negligible impact when including covariates that are uncorrelated with the outcome. Overall, pre-specification of covariate adjustment is recommended for the analysis of adaptive multi-arm design with treatment selection. Having the statistical analysis plan in place prior to the interim and final analyses is crucial, especially when a non-collapsible measure of treatment effect is considered in the trial.

Point and interval estimation in two-stage adaptive designs with time to event data and biomarker-driven subpopulation selection

In personalized medicine, it is often desired to determine if all patients or only a subset of them benefit from a treatment. We consider estimation in two-stage adaptive designs that in stage 1 recruit patients from the full population. In stage 2, patient recruitment is restricted to the part of the population, which, based on stage 1 data, benefits from the experimental treatment. Existing estimators, which adjust for using stage 1 data for selecting the part of the population from which stage 2 patients are recruited, as well as for the confirmatory analysis after stage 2, do not consider time to event patient outcomes. In this work, for time to event data, we have derived a new asymptotically unbiased estimator for the log hazard ratio and a new interval estimator with good coverage probabilities and probabilities that the upper bounds are below the true values. The estimators are appropriate for several selection rules that are based on a single or multiple biomarkers, which can be categorical or continuous.

Efficient Adaptive Designs for Clinical Trials of Interventions for COVID-19

The COVID-19 pandemic has led to an unprecedented response in terms of clinical research activity. An important part of this research has been focused on randomized controlled clinical trials to evaluate potential therapies for COVID-19. The results from this research need to be obtained as rapidly as possible. This presents a number of challenges associated with considerable uncertainty over the natural history of the disease and the number and characteristics of patients affected, and the emergence of new potential therapies. These challenges make adaptive designs for clinical trials a particularly attractive option. Such designs allow a trial to be modified on the basis of interim analysis data or stopped as soon as sufficiently strong evidence has been observed to answer the research question, without compromising the trial's scientific validity or integrity. In this article, we describe some of the adaptive design approaches that are available and discuss particular issues and challenges associated with their use in the pandemic setting. Our discussion is illustrated by details of four ongoing COVID-19 trials that have used adaptive designs.

Adaptive multiarm multistage clinical trials

Two methods for designing adaptive multiarm multistage (MAMS) clinical trials, originating from conceptually different group sequential frameworks are presented, and their operating characteristics are compared. In both methods pairwise comparisons are made, stage-by-stage, between each treatment arm and a common control arm with the goal of identifying active treatments and dropping inactive ones. At any stage one may alter the future course of the trial through adaptive changes to the prespecified decision rules for treatment selection and sample size reestimation, and notwithstanding such changes, both methods guarantee strong control of the family-wise error rate. The stage-wise MAMS approach was historically the first to be developed and remains the standard method for designing inferentially seamless phase 2-3 clinical trials. In this approach, at each stage, the data from each treatment comparison are summarized by a single multiplicity adjusted P-value. These stage-wise P-values are combined by a prespecified combination function and the resultant test statistic is monitored with respect to the classical two-arm group sequential efficacy boundaries. The cumulative MAMS approach is a more recent development in which a separate test statistic is constructed for each treatment comparison from the cumulative data at each stage. These statistics are then monitored with respect to multiplicity adjusted group sequential efficacy boundaries. We compared the powers of the two methods for designs with two and three active treatment arms, under commonly utilized decision rules for treatment selection, sample size reestimation and early stopping. In our investigations, which were carried out over a reasonably exhaustive exploration of the parameter space, the cumulative MAMS designs were more powerful than the stage-wise MAMS designs, except for the homogeneous case of equal treatment effects, where a small power advantage was discernable for the stage-wise MAMS designs.

Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods, simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so‐called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked‐up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.

Physiotherapy rehabilitation for osteoporotic vertebral fracture-a randomised controlled trial and economic evaluation (PROVE trial)

The trial compared three physiotherapy approaches: manual or exercise therapy compared with a single session of physiotherapy education (SSPT) for people with osteoporotic vertebral fracture(s). At 1 year, there were no statistically significant differences between the groups meaning there is inadequate evidence to support manual or exercise therapy.

INTRODUCTION

To evaluate the clinical and cost-effectiveness of different physiotherapy approaches for people with osteoporotic vertebral fracture(s) (OVF).

METHODS

>Prospective, multicentre, adaptive, three-arm randomised controlled trial. Six hundred fifteen adults with back pain, osteoporosis, and at least 1 OVF participated.

INTERVENTIONS

7 individual physiotherapy sessions over 12 weeks focused on either manual therapy or home exercise compared with a single session of physiotherapy education (SSPT). The co-primary outcomes were quality of life and back muscle endurance measured by the QUALEFFO-41 and timed loaded standing (TLS) test at 12 months.

RESULTS

At 12 months, there were no statistically significant differences between groups. Mean QUALEFFO-41: - 1.3 (exercise), - 0.15 (manual), and - 1.2 (SSPT), a mean difference of - 0.2 (95% CI, - 3.2 to 1.6) for exercise and 1.3 (95% CI, - 1.8 to 2.9) for manual therapy. Mean TLS: 9.8 s (exercise), 13.6 s (manual), and 4.2 s (SSPT), a mean increase of 5.8 s (95% CI, - 4.8 to 20.5) for exercise and 9.7 s (95% CI, 0.1 to 24.9) for manual therapy. Exercise provided more quality-adjusted life years than SSPT but was more expensive. At 4 months, significant changes above SSPT occurred in endurance and balance in manual therapy, and in endurance for those ≤ 70 years, in balance, mobility, and walking in exercise.

CONCLUSIONS

Adherence was problematic. Benefits at 4 months did not persist and at 12 months, we found no significant differences between treatments. There is inadequate evidence a short physiotherapy intervention of either manual therapy or home exercise provides long-term benefits, but arguably short-term benefits are valuable.

TRIAL REGISTRATION

ISRCTN 49117867.

Exercise or manual physiotherapy compared with a single session of physiotherapy for osteoporotic vertebral fracture: three-arm PROVE RCT

BACKGROUND

A total of 25,000 people in the UK have osteoporotic vertebral fracture (OVF). Evidence suggests that physiotherapy may have an important treatment role.

OBJECTIVE

The objective was to investigate the clinical effectiveness and cost-effectiveness of two different physiotherapy programmes for people with OVF compared with a single physiotherapy session.

DESIGN

This was a prospective, adaptive, multicentre, assessor-blinded randomised controlled trial (RCT) with nested qualitative and health economic studies.

SETTING

This trial was based in 21 NHS physiotherapy departments.

PARTICIPANTS

The participants were people with symptomatic OVF.

INTERVENTIONS

Seven sessions of either manual outpatient physiotherapy or exercise outpatient physiotherapy compared with the best practice of a 1-hour single session of physiotherapy (SSPT).

MAIN OUTCOME MEASURES

Outcomes were measured at 4 and 12 months. The primary outcomes were quality of life and muscle endurance, which were measured by the disease-specific QUALEFFO-41 (Quality of Life Questionnaire of the European Foundation for Osteoporosis - 41 items) and timed loaded standing (TLS) test, respectively. Secondary outcomes were (1) thoracic kyphosis angle, (2) balance, evaluated via the functional reach test (FRT), and (3) physical function, assessed via the Short Physical Performance Battery (SPPB), 6-minute walk test (6MWT), Physical Activity Scale for the Elderly, a health resource use and falls diary, and the EuroQol-5 Dimensions, five-level version.

RESULTS

A total of 615 participants were enrolled, with 216, 203 and 196 randomised by a computer-generated program to exercise therapy, manual therapy and a SSPT, respectively. Baseline data were available for 613 participants, 531 (86.6%) of whom were women; the mean age of these participants was 72.14 years (standard deviation 9.09 years). Primary outcome data were obtained for 69% of participants (429/615) at 12 months: 175 in the exercise therapy arm, 181 in the manual therapy arm and 173 in the SSPT arm. Interim analysis met the criteria for all arms to remain in the study. For the primary outcomes at 12 months, there were no significant benefits over SSPT of exercise [QUALEFFO-41, difference -0.23 points, 95% confidence interval (CI) -3.20 to 1.59 points; p = 1.000; and TLS test, difference 5.77 seconds, 95% CI -4.85 to 20.46 seconds; p = 0.437] or of manual therapy (QUALEFFO-41, difference 1.35 points, 95% CI -1.76 to 2.93 points; p = 0.744; TLS test, difference 9.69 seconds (95% CI 0.09 to 24.86 seconds; p = 0.335). At 4 months, there were significant gains for both manual therapy and exercise therapy over SSPT in the TLS test in participants aged < 70 years. Exercise therapy was superior to a SSPT at 4 months in the SPPB, FRT and 6MWT and manual therapy was superior to a SSPT at 4 months in the TLS test and FRT. Neither manual therapy nor exercise therapy was cost-effective relative to a SSPT using the threshold of £20,000 per quality-adjusted life-year. There were no treatment-related serious adverse events.

CONCLUSIONS

This is the largest RCT to date assessing physiotherapy in participants with OVFs. At 1 year, neither treatment intervention conferred more benefit than a single 1-hour physiotherapy advice session. The focus of future work should be on the intensity and duration of interventions to determine if changes to these would demonstrate more sustained effects.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN49117867.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 44. See the NIHR Journals Library website for further project information.

To add or not to add a new treatment arm to a multiarm study: A decision-theoretic framework

Multiarm clinical trials, which compare several experimental treatments against control, are frequently recommended due to their efficiency gain. In practise, all potential treatments may not be ready to be tested in a phase II/III trial at the same time. It has become appealing to allow new treatment arms to be added into on‐going clinical trials using a “platform” trial approach. To the best of our knowledge, many aspects of when to add arms to an existing trial have not been explored in the literature. Most works on adding arm(s) assume that a new arm is opened whenever a new treatment becomes available. This strategy may prolong the overall duration of a study or cause reduction in marginal power for each hypothesis if the adaptation is not well accommodated. Within a two‐stage trial setting, we propose a decision‐theoretic framework to investigate when to add or not to add a new treatment arm based on the observed stage one treatment responses. To account for different prospect of multiarm studies, we define utility in two different ways; one for a trial that aims to maximise the number of rejected hypotheses; the other for a trial that would declare a success when at least one hypothesis is rejected from the study. Our framework shows that it is not always optimal to add a new treatment arm to an existing trial. We illustrate a case study by considering a completed trial on knee osteoarthritis.

Adjusting for selection bias in assessing treatment effect estimates from multiple subgroups

This paper discusses a number of methods for adjusting treatment effect estimates in clinical trials where differential effects in several subpopulations are suspected. In such situations, the estimates from the most extreme subpopulation are often overinterpreted. The paper focusses on the construction of simultaneous confidence intervals intended to provide a more realistic assessment regarding the uncertainty around these extreme results. The methods from simultaneous inference are compared with shrinkage estimates arising from Bayesian hierarchical models by discussing salient features of both approaches in a typical application.

Point estimation following two-stage adaptive threshold enrichment clinical trials

Recently, several study designs incorporating treatment effect assessment in biomarker-based subpopulations have been proposed. Most statistical methodologies for such designs focus on the control of type I error rate and power. In this paper, we have developed point estimators for clinical trials that use the two-stage adaptive enrichment threshold design. The design consists of two stages, where in stage 1, patients are recruited in the full population. Stage 1 outcome data are then used to perform interim analysis to decide whether the trial continues to stage 2 with the full population or a subpopulation. The subpopulation is defined based on one of the candidate threshold values of a numerical predictive biomarker. To estimate treatment effect in the selected subpopulation, we have derived unbiased estimators, shrinkage estimators, and estimators that estimate bias and subtract it from the naive estimate. We have recommended one of the unbiased estimators. However, since none of the estimators dominated in all simulation scenarios based on both bias and mean squared error, an alternative strategy would be to use a hybrid estimator where the estimator used depends on the subpopulation selected. This would require a simulation study of plausible scenarios before the trial.