Optimizing randomized trial designs to distinguish which subpopulations benefit from treatment

Efficient testing of the biomarker positive and negative subgroups in a biomarker-stratified trial

In many randomized placebo-controlled trials with a biomarker defined subgroup, it is believed that this subgroup has the same or higher treatment effect compared with its complement. These subgroups are often referred to as the biomarker positive and negative subgroups. Most biomarker-stratified pivotal trials are aimed at demonstrating a significant treatment effect either in the biomarker positive subgroup or in the overall population. A major shortcoming of this approach is that the treatment can be declared effective in the overall population even though it has no effect in the biomarker negative subgroup. We use the isotonic assumption about the treatment effects in the two subgroups to construct an efficient way to test for a treatment effect in both the biomarker positive and negative subgroups. A substantial reduction in the required sample size for such a trial compared with existing methods makes evaluating the treatment effect in both the biomarker positive and negative subgroups feasible in pivotal trials especially when the prevalence of the biomarker positive subgroup is less than 0.5.

Considerations for identifying the "right" subgroup in adaptive enrichment trials

Adaptive Enrichment Trials aim to make efficient use of data in a pivotal trial of a new targeted therapy to both (a) more precisely identify who benefits from that therapy and (b) improve the likelihood of successfully concluding that the drug is effective, while controlling the probability of false positives. There are a number of frameworks for conducting such a trial and decisions that must be made regarding how to identify that target subgroup. Among those decisions, one must choose how aggressively to restrict enrollment criteria based on the accumulating evidence in the trial. In this article, we empirically evaluate the impact of aggressive versus conservative enrollment restrictions on the power of the trial to detect an effect of treatment. We identify that, in some cases, a more aggressive strategy can substantially improve power. This additionally raises an important question regarding label indication: To what degree do we need a formal test of the hypothesis of no treatment effect in the exact population implied by the label indication? We discuss this question and evaluate how our answer for adaptive enrichment trials may relate to the answer implied by current practice for broad eligibility trials.

New insights into adaptive enrichment designs

The transition towards personalized medicine is happening and the new experimental framework is raising several challenges, from a clinical, ethical, logistical, regulatory, and statistical perspective. To face these challenges, innovative study designs with increasing complexity have been proposed. In particular, adaptive enrichment designs are becoming more attractive for their flexibility. However, these procedures rely on an increasing number of parameters that are unknown at the planning stage of the clinical trial, so the study design requires particular care. This review is dedicated to adaptive enrichment studies with a focus on design aspects. While many papers deal with methods for the analysis, the sample size determination and the optimal allocation problem have been overlooked. We discuss the multiple aspects involved in adaptive enrichment designs that contribute to their advantages and disadvantages. The decision-making process of whether or not it is worth enriching should be driven by clinical and ethical considerations as well as scientific and statistical concerns.

IBIS: identify biomarker-based subgroups with a Bayesian enrichment design for targeted combination therapy

BACKGROUND

Combination therapies directed at multiple targets have potentially improved treatment effects for cancer patients. Compared to monotherapy, targeted combination therapy leads to an increasing number of subgroups and complicated biomarker-based efficacy profiles, making it more difficult for efficacy evaluation in clinical trials. Therefore, it is necessary to develop innovative clinical trial designs to explore the efficacy of targeted combination therapy in different subgroups and identify patients who are more likely to benefit from the investigational combination therapy.

METHODS

We propose a statistical tool called 'IBIS' to Identify BIomarker-based Subgroups and apply it to the enrichment design framework. The IBIS contains three main elements: subgroup division, efficacy evaluation and subgroup identification. We first enumerate all possible subgroup divisions based on biomarker levels. Then, Jensen-Shannon divergence is used to distinguish high-efficacy and low-efficacy subgroups, and Bayesian hierarchical model (BHM) is employed to borrow information within these two subsets for efficacy evaluation. Regarding subgroup identification, a hypothesis testing framework based on Bayes factors is constructed. This framework also plays a key role in go/no-go decisions and enriching specific population. Simulation studies are conducted to evaluate the proposed method.

RESULTS

The accuracy and precision of IBIS could reach a desired level in terms of estimation performance. In regard to subgroup identification and population enrichment, the proposed IBIS has superior and robust characteristics compared with traditional methods. An example of how to obtain design parameters for an adaptive enrichment design under the IBIS framework is also provided.

CONCLUSIONS

IBIS has the potential to be a useful tool for biomarker-based subgroup identification and population enrichment in clinical trials of targeted combination therapy.

ANALYSIS OF "LEARN-AS-YOU-GO" (LAGO) STUDIES

In Learn-As-you-GO (LAGO) adaptive studies, the intervention is a complex multicomponent package, and is adapted in stages during the study based on past outcome data. This design formalizes standard practice in public health intervention studies. An effective intervention package is sought, while minimizing intervention package cost. In LAGO study data, the interventions in later stages depend upon the outcomes in the previous stages, violating standard statistical theory. We develop an estimator for the intervention effects, and prove consistency and asymptotic normality using a novel coupling argument, ensuring the validity of the test for the hypothesis of no overall intervention effect. We develop a confidence set for the optimal intervention package and confidence bands for the success probabilities under alternative package compositions. We illustrate our methods in the BetterBirth Study, which aimed to improve maternal and neonatal outcomes among 157,689 births in Uttar Pradesh, India through a multicomponent intervention package.

Using Bayesian modeling in frequentist adaptive enrichment designs

Our increased understanding of the mechanistic heterogeneity of diseases has pushed the development of targeted therapeutics. We do not expect all patients with a given disease to benefit from a targeted drug; only those in the target population. That is, those with sufficient dysregulation in the biomolecular pathway targeted by treatment. However, due to complexity of the pathway, and/or technical issues with our characterizing assay, it is often hard to characterize the target population until well into large-scale clinical trials. This has stimulated the development of adaptive enrichment trials; clinical trials in which the target population is adaptively learned; and enrollment criteria are adaptively updated to reflect this growing understanding. This paper proposes a framework for group-sequential adaptive enrichment trials. Building on the work of Simon & Simon (2013). Adaptive enrichment designs for clinical trials. Biostatistics 14(4), 613-625), it includes a frequentist hypothesis test at the end of the trial. However, it uses Bayesian methods to optimize the decisions required during the trial (regarding how to restrict enrollment) and Bayesian methods to estimate effect size, and characterize the target population at the end of the trial. This joint frequentist/Bayesian design combines the power of Bayesian methods for decision making with the use of a formal hypothesis test at the end of the trial to preserve the studywise probability of a type I error.

Lactobacillus rhamnosus GG versus Placebo for Acute Gastroenteritis in Children

BACKGROUND

Acute gastroenteritis develops in millions of children in the United States every year, and treatment with probiotics is common. However, data to support the use of probiotics in this population are limited.

METHODS

We conducted a prospective, randomized, double-blind trial involving children 3 months to 4 years of age with acute gastroenteritis who presented to one of 10 U.S. pediatric emergency departments. Participants received a 5-day course of Lactobacillus rhamnosus GG at a dose of 1×10¹⁰ colony-forming units twice daily or matching placebo. Follow-up surveys were conducted daily for 5 days and again 14 days after enrollment and 1 month after enrollment. The primary outcome was moderate-to-severe gastroenteritis, which was defined as an illness episode with a total score on the modified Vesikari scale of 9 or higher (scores range from 0 to 20, with higher scores indicating more severe disease), within 14 days after enrollment. Secondary outcomes included the duration and frequency of diarrhea and vomiting, the duration of day-care absenteeism, and the rate of household transmission (defined as the development of symptoms of gastroenteritis in previously asymptomatic household contacts).

RESULTS

Among the 971 participants, 943 (97.1%) completed the trial. The median age was 1.4 years (interquartile range, 0.9 to 2.3), and 513 participants (52.9%) were male. The modified Vesikari scale score for the 14-day period after enrollment was 9 or higher in 55 of 468 participants (11.8%) in the L. rhamnosus GG group and in 60 of 475 participants (12.6%) in the placebo group (relative risk, 0.96; 95% confidence interval, 0.68 to 1.35; P=0.83). There were no significant differences between the L. rhamnosus GG group and the placebo group in the duration of diarrhea (median, 49.7 hours in the L. rhamnosus GG group and 50.9 hours in the placebo group; P=0.26), duration of vomiting (median, 0 hours in both groups; P=0.17), or day-care absenteeism (median, 2 days in both groups; P=0.67) or in the rate of household transmission (10.6% and 14.1% in the two groups, respectively; P=0.16).

CONCLUSIONS

Among preschool children with acute gastroenteritis, those who received a 5-day course of L. rhamnosus GG did not have better outcomes than those who received placebo. (Funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development and others; ClinicalTrials.gov number, NCT01773967 .).

Inference for multimarker adaptive enrichment trials

Identification of treatment selection biomarkers has become very important in cancer drug development. Adaptive enrichment designs have been developed for situations where a unique treatment selection biomarker is not apparent based on the mechanism of action of the drug. With such designs, the eligibility rules may be adaptively modified at interim analysis times to exclude patients who are unlikely to benefit from the test treatment.We consider a recently proposed, particularly flexible approach that permits development of model-based multifeature predictive classifiers as well as optimized cut-points for continuous biomarkers. A single significance test, including all randomized patients, is performed at the end of the trial of the strong null hypothesis that the expected outcome on the test treatment is no better than control for any of the subset populations of patients accrued in the K stages of the clinical trial. In this paper, we address 2 issues involving inference following an adaptive enrichment design as described above. The first is specification of the intended use population and estimation of treatment effect for that population following rejection of the strong null hypothesis. The second issue is defining conditions in which rejection of the strong null hypothesis implies rejection of the null hypothesis for the intended use population.

Treatment evaluation for a data-driven subgroup in adaptive enrichment designs of clinical trials

Adaptive enrichment designs (AEDs) of clinical trials allow investigators to restrict enrollment to a promising subgroup based on an interim analysis. Most of the existing AEDs deal with a small number of predefined subgroups, which are often unknown at the design stage. The newly developed Simon design offers a great deal of flexibility in subgroup selection (without requiring pre-defined subgroups) but does not provide a procedure for estimating and testing treatment efficacy for the selected subgroup. This article proposes a 2-stage AED which does not require predefined subgroups but requires a prespecified algorithm for choosing a subgroup on the basis of baseline covariate information. Having a prespecified algorithm for subgroup selection makes it possible to use cross-validation and bootstrap methods to correct for the resubstitution bias in estimating treatment efficacy for the selected subgroup. The methods are evaluated and compared in a simulation study mimicking actual clinical trials of human immunodeficiency virus infection.

Randomised controlled trial of Lactobacillus rhamnosus (LGG) versus placebo in children presenting to the emergency department with acute gastroenteritis: the PECARN probiotic study protocol

INTRODUCTION

Acute gastroenteritis (AGE) is a common and burdensome condition that affects millions of children worldwide each year. Currently available strategies are limited to symptomatic management, treatment and prevention of dehydration and infection control; no disease-modifying interventions exist. Probiotics, defined as live microorganisms beneficial to the host, have shown promise in improving AGE outcomes, but existing studies have sufficient limitations such that the use of probiotics cannot currently be recommended with confidence. Here we present the methods of a large, rigorous, randomised, double-blind placebo-controlled study to assess the effectiveness and side effect profile of Lactobacillus rhamnosus GG (LGG) (ATCC 53103) in children with AGE.

METHODS AND ANALYSIS

The study is being conducted in 10 US paediatric emergency departments (EDs) within the federally funded Pediatric Emergency Care Applied Research Network, in accordance with current SPIRIT and CONSORT statement recommendations. We will randomise 970 children presenting to participating EDs with AGE to either 5 days of treatment with LGG (10¹⁰colony-forming unit twice a day) or placebo between July 2014 to December 2017. The main outcome is the occurrence of moderate-to-severe disease over time, as defined by the Modified Vesikari Scale. We also record adverse events and side effects related to the intervention. We will conduct intention-to-treat analyses and use an enrichment design to restore the statistical power in case the presence of a subpopulation with a substantially low treatment effect is identified.

ETHICS AND DISSEMINATION

Institutional review board approval has been obtained at all sites, and data and material use agreements have been established between the participating sites. The results of the trial will be published in peer-reviewed journals. A deidentified public data set will be made available after the completion of all study procedures.

TRIAL REGISTRATION NUMBER

NCT01773967.

Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans health Administration

OBJECTIVES

The US Veterans Health Administration (VHA) has begun using predictive modeling to identify Veterans at high suicide risk to target care. Initial analyses are reported here.

METHODS

A penalized logistic regression model was compared with an earlier proof-of-concept logistic model. Exploratory analyses then considered commonly-used machine learning algorithms. Analyses were based on electronic medical records for all 6,360 individuals classified in the National Death Index as having died by suicide in fiscal years 2009-2011 who used VHA services the year of their death or prior year and a 1% probability sample of time-matched VHA service users alive at the index date (n = 2,112,008).

RESULTS

A penalized logistic model with 61 predictors had sensitivity comparable to the proof-of-concept model (which had 381 predictors) at target thresholds. The machine learning algorithms had relatively similar sensitivities, the highest being for Bayesian additive regression trees, with 10.7% of suicides occurred among the 1.0% of Veterans with highest predicted risk and 28.1% among the 5.0% of with highest predicted risk.

CONCLUSIONS

Based on these results, VHA is using penalized logistic regression in initial intervention implementation. The paper concludes with a discussion of other practical issues that might be explored to increase model performance.

Recursive partitioning for heterogeneous causal effects

In this paper we propose methods for estimating heterogeneity in causal effects in experimental and observational studies and for conducting hypothesis tests about the magnitude of differences in treatment effects across subsets of the population. We provide a data-driven approach to partition the data into subpopulations that differ in the magnitude of their treatment effects. The approach enables the construction of valid confidence intervals for treatment effects, even with many covariates relative to the sample size, and without "sparsity" assumptions. We propose an "honest" approach to estimation, whereby one sample is used to construct the partition and another to estimate treatment effects for each subpopulation. Our approach builds on regression tree methods, modified to optimize for goodness of fit in treatment effects and to account for honest estimation. Our model selection criterion anticipates that bias will be eliminated by honest estimation and also accounts for the effect of making additional splits on the variance of treatment effect estimates within each subpopulation. We address the challenge that the "ground truth" for a causal effect is not observed for any individual unit, so that standard approaches to cross-validation must be modified. Through a simulation study, we show that for our preferred method honest estimation results in nominal coverage for 90% confidence intervals, whereas coverage ranges between 74% and 84% for nonhonest approaches. Honest estimation requires estimating the model with a smaller sample size; the cost in terms of mean squared error of treatment effects for our preferred method ranges between 7-22%.

An adaptive design for updating the threshold value of a continuous biomarker

Potential predictive biomarkers are often measured on a continuous scale, but in practice, a threshold value to divide the patient population into biomarker ‘positive’ and ‘negative’ is desirable. Early phase clinical trials are increasingly using biomarkers for patient selection, but at this stage, it is likely that little will be known about the relationship between the biomarker and the treatment outcome. We describe a single-arm trial design with adaptive enrichment, which can increase power to demonstrate efficacy within a patient subpopulation, the parameters of which are also estimated. Our design enables us to learn about the biomarker and optimally adjust the threshold during the study, using a combination of generalised linear modelling and Bayesian prediction. At the final analysis, a binomial exact test is carried out, allowing the hypothesis that ‘no population subset exists in which the novel treatment has a desirable response rate’ to be tested. Through extensive simulations, we are able to show increased power over fixed threshold methods in many situations without increasing the type-I error rate. We also show that estimates of the threshold, which defines the population subset, are unbiased and often more precise than those from fixed threshold studies. We provide an example of the method applied (retrospectively) to publically available data from a study of the use of tamoxifen after mastectomy by the German Breast Study Group, where progesterone receptor is the biomarker of interest.

Group sequential designs with prospectively planned rules for subpopulation enrichment

We propose a class of randomized trial designs aimed at gaining the advantages of wider generalizability and faster recruitment while mitigating the risks of including a population for which there is greater a priori uncertainty. We focus on testing null hypotheses for the overall population and a predefined subpopulation. Our designs have preplanned rules for modifying enrollment criteria based on data accrued at interim analyses. For example, enrollment can be restricted if the participants from a predefined subpopulation are not benefiting from the new treatment. Our designs have the following features: the multiple testing procedure fully leverages the correlation among statistics for different populations; the asymptotic familywise Type I error rate is strongly controlled; for outcomes that are binary or normally distributed, the decision rule and multiple testing procedure are functions of the data only through minimal sufficient statistics. Our designs incorporate standard group sequential boundaries for each population of interest; this may be helpful in communicating the designs, because many clinical investigators are familiar with such boundaries, which can be summarized succinctly in a single table or graph. We demonstrate these designs through simulations of a Phase III trial of a new treatment for stroke. User-friendly, free software implementing these designs is described. Copyright © 2016 John Wiley & Sons, Ltd.

Recent Statistical Contributions to Medical Device Development

Clinical trials in the development of new medical device products are in many ways analogous to clinical trials in the development of new drug or biologic products. However, the differences are important and not always intuitive to a statistician with only experience supporting development of drug and biologic products. In this paper we discuss some of the interesting differences with focus on the statistical innovation that is coming out of the medical device area. We discuss examples of the differences in clinical trial design and effects of these differences on clinical development programs.

Uniformly Most Powerful Tests for Simultaneously Detecting a Treatment Effect in the Overall Population and at Least One Subpopulation

We take the perspective of a researcher planning a randomized trial of a new treatment, where it is suspected that certain subpopulations may benefit more than others. These subpopulations could be defined by a risk factor or biomarker measured at baseline. We focus on situations where the overall population is partitioned into two, predefined subpopulations. When the true average treatment effect for the overall population is positive, it logically follows that it must be positive for at least one subpopulation. Our goal is to construct multiple testing procedures that maximize power for simultaneously rejecting the overall population null hypothesis and at least one subpopulation null hypothesis. We show that uniformly most powerful tests exist for this problem, in the case where outcomes are normally distributed. We construct new multiple testing procedures that, to the best of our knowledge, are the first to have this property. These procedures have the advantage of not requiring any sacrifice for detecting a treatment effect in the overall population, compared to the uniformly most powerful test of the overall population null hypothesis.

Observational research on NCDs in HIV-positive populations: conceptual and methodological considerations

Noncommunicable diseases (NCDs) account for a growing burden of morbidity and mortality among people living with HIV in low- and middle-income countries (LMICs). HIV infection and antiretroviral therapy interact with NCD risk factors in complex ways, and research into this "web of causation" has so far been largely based on data from high-income countries. However, improving the understanding, treatment, and prevention of NCDs in LMICs requires region-specific evidence. Priority research areas include: (1) defining the burden of NCDs among people living with HIV, (2) understanding the impact of modifiable risk factors, (3) evaluating effective and efficient care strategies at individual and health systems levels, and (4) evaluating cost-effective prevention strategies. Meeting these needs will require observational data, both to inform the design of randomized trials and to replace trials that would be unethical or infeasible. Focusing on Sub-Saharan Africa, we discuss data resources currently available to inform this effort and consider key limitations and methodological challenges. Existing data resources often lack population-based samples; HIV-negative, HIV-positive, and antiretroviral therapy-naive comparison groups; and measurements of key NCD risk factors and outcomes. Other challenges include loss to follow-up, competing risk of death, incomplete outcome ascertainment and measurement of factors affecting clinical decision making, and the need to control for (time-dependent) confounding. We review these challenges and discuss strategies for overcoming them through augmented data collection and appropriate analysis. We conclude with recommendations to improve the quality of data and analyses available to inform the response to HIV and NCD comorbidity in LMICs.

Testing for efficacy in adaptive clinical trials with enrichment

Adaptive design of clinical trials has attracted considerable interest because of its potential of reducing costs and saving time in the clinical development process. In this paper, we consider the problem of assessing the effectiveness of a test treatment over a control by a two-arm randomized clinical trial in a potentially heterogenous patient population. In particular, we study enrichment designs that use accumulating data from a clinical trial to adaptively determine patient subpopulation in which the treatment effect is eventually assessed. A hypothesis testing procedure and a lower confidence limit are presented for the treatment effect in the selected patient subgroups. The performances of the new methods are compared with existing approaches through a simulation study.

Adaptive designs for confirmatory clinical trials with subgroup selection

Growing interest in stratified medicine is leading to increasing importance of subgroup analyses in confirmatory clinical trials. Conventionally, confirmatory clinical trials either focus on a subgroup identified in advance or assess subgroup effects once the trial is completed. The focus of this article is methodology for adaptive clinical trials that both identify whether a treatment is particularly effective in a predefined subgroup, potentially enabling alteration of recruitment, and assess the effectiveness in the subgroup and/or whole population. Methods for such adaptive trials are described and compared, and the logistical and regulatory issues associated with such approaches are discussed.

Drug-diagnostics co-development in oncology

Developments in genomics are providing a biological basis for the heterogeneity of clinical course and response to treatment that have long been apparent to clinicians. The ability to molecularly characterize human diseases presents new opportunities to develop more effective treatments and new challenges for the design and analysis of clinical trials. In oncology, treatment of broad populations with regimens that benefit a minority of patients is less economically sustainable with expensive molecularly targeted therapeutics. The established molecular heterogeneity of human diseases requires the development of new paradigms for the design and analysis of randomized clinical trials as a reliable basis for predictive medicine. We review prospective designs for the development of new therapeutics and predictive biomarkers to inform their use. We cover designs for a wide range of settings. At one extreme is the development of a new drug with a single candidate biomarker and strong biological evidence that marker negative patients are unlikely to benefit from the new drug. At the other extreme are phase III clinical trials involving both genome-wide discovery of a predictive classifier and internal validation of that classifier. We have outlined a prediction based approach to the analysis of randomized clinical trials that both preserves the type I error and provides a reliable internally validated basis for predicting which patients are most likely or unlikely to benefit from a new regimen.

Confidence intervals for the selected population in randomized trials that adapt the population enrolled

It is a challenge to design randomized trials when it is suspected that a treatment may benefit only certain subsets of the target population. In such situations, trial designs have been proposed that modify the population enrolled based on an interim analysis, in a preplanned manner. For example, if there is early evidence during the trial that the treatment only benefits a certain subset of the population, enrollment may then be restricted to this subset. At the end of such a trial, it is desirable to draw inferences about the selected population. We focus on constructing confidence intervals for the average treatment effect in the selected population. Confidence interval methods that fail to account for the adaptive nature of the design may fail to have the desired coverage probability. We provide a new procedure for constructing confidence intervals having at least 95% coverage probability, uniformly over a large class Q of possible data generating distributions. Our method involves computing the minimum factor c by which a standard confidence interval must be expanded in order to have, asymptotically, at least 95% coverage probability, uniformly over Q. Computing the expansion factor c is not trivial, since it is not a priori clear, for a given decision rule, for which data generating distribution leads to the worst-case coverage probability. We give an algorithm that computes c, and then prove an optimality property for the resulting confidence interval procedure.

Adaptive enrichment designs for clinical trials

Modern medicine has graduated from broad spectrum treatments to targeted therapeutics. New drugs recognize the recently discovered heterogeneity of many diseases previously considered to be fairly homogeneous. These treatments attack specific genetic pathways which are only dysregulated in some smaller subset of patients with the disease. Often this subset is only rudimentarily understood until well into large-scale clinical trials. As such, standard practice has been to enroll a broad range of patients and run post hoc subset analysis to determine those who may particularly benefit. This unnecessarily exposes many patients to hazardous side effects, and may vastly decrease the efficiency of the trial (especially if only a small subset of patients benefit). In this manuscript, we propose a class of adaptive enrichment designs that allow the eligibility criteria of a trial to be adaptively updated during the trial, restricting entry to patients likely to benefit from the new treatment. We show that our designs both preserve the type 1 error, and in a variety of cases provide a substantial increase in power.

Potential of adaptive clinical trial designs in pharmacogenetic research

Adaptive trial designs can be beneficial in pharmacogenetic research when prior uncertainty exists regarding the exact role and clinical relevance of genetic variability in drug response. This type of design enables us to learn about the effect of the genetic variability on drug response and to immediately use this information for the remainder of the study. For different types of adaptive trial designs, we discuss when and how the designs are suitable for pharmacogenetic research: adaptation of randomization, adaptation of patient enrollment and adaptive enrichment. To illustrate the potential benefits of an adaptive design over a fixed design, we simulated an adaptive trial based on the results of the IPASS trial. With a simple model we show that for this example an adaptive enrichment design would have led to a smaller trial, with less EGF receptor mutation-negative patients unnecessarily exposed to the drug, without compromising the α level or reducing power.