Optimizing Trial Designs for Targeted Therapies

Adaptive enrichment designs with a continuous biomarker

A popular design for clinical trials assessing targeted therapies is the two-stage adaptive enrichment design with recruitment in stage 2 limited to a biomarker-defined subgroup chosen based on data from stage 1. The data-dependent selection leads to statistical challenges if data from both stages are used to draw inference on treatment effects in the selected subgroup. If subgroups considered are nested, as when defined by a continuous biomarker, treatment effect estimates in different subgroups follow the same distribution as estimates in a group-sequential trial. This result is used to obtain tests controlling the familywise type I error rate (FWER) for six simple subgroup selection rules, one of which also controls the FWER for any selection rule. Two approaches are proposed: one based on multivariate normal distributions suitable if the number of possible subgroups, k, is small, and one based on Brownian motion approximations suitable for large k. The methods, applicable in the wide range of settings with asymptotically normal test statistics, are illustrated using survival data from a breast cancer trial.

A framework for assessing the impact of accelerated approval

The FDA's Accelerated Approval program (AA) is a regulatory program to expedite availability of products to treat serious or life-threatening illnesses that lack effective treatment alternatives. Ideally, all of the many stakeholders such as patients, physicians, regulators, and health technology assessment [HTA] agencies that are affected by AA should benefit from it. In practice, however, there is intense debate over whether evidence supporting AA is sufficient to meet the needs of the stakeholders who collectively bring an approved product into routine clinical care. As AAs have become more common, it becomes essential to be able to determine their impact objectively and reproducibly in a way that provides for consistent evaluation of therapeutic decision alternatives. We describe the basic features of an approach for evaluating AA impact that accommodates stakeholder-specific views about potential benefits, risks, and costs. The approach is based on a formal decision-analytic framework combining predictive distributions for therapeutic outcomes (efficacy and safety) based on statistical models that incorporate findings from AA trials with stakeholder assessments of various actions that might be taken. The framework described here provides a starting point for communicating the value of a treatment granted AA in the context of what is important to various stakeholders.

Selecting predictive biomarkers from genomic data

Recently there have been tremendous efforts to develop statistical procedures which allow to determine subgroups of patients for which certain treatments are effective. This article focuses on the selection of prognostic and predictive genetic biomarkers based on a relatively large number of candidate Single Nucleotide Polymorphisms (SNPs). We consider models which include prognostic markers as main effects and predictive markers as interaction effects with treatment. We compare different high-dimensional selection approaches including adaptive lasso, a Bayesian adaptive version of the Sorted L-One Penalized Estimator (SLOBE) and a modified version of the Bayesian Information Criterion (mBIC2). These are compared with classical multiple testing procedures for individual markers. Having identified predictive markers we consider several different approaches how to specify subgroups susceptible to treatment. Our main conclusion is that selection based on mBIC2 and SLOBE has similar predictive performance as the adaptive lasso while including substantially fewer biomarkers.

Development of a model to demonstrate the impact of National Institute of Health and Care Excellence cost-effectiveness assessment on health utility for targeted medicines

Advances in medical technology have led to a better understanding of heterogeneity of diseases and patients, and to the development of targeted medicines. This development is beneficial to society but can come at an increased cost to pharmaceutical manufacturers due to the costs associated with developing and manufacturing a diagnostic test. For such medicines, the conventional pricing structure, where a therapy is approved if it is deemed cost-effective, may not appropriately incentivize targeted drug development. We model the decision-making processes for both the healthcare provider and the pharmaceutical manufacturer, capturing their main priorities, and populate it with information from a recent appraisal by the National Institute of Health and Care Excellence. Healthcare providers prefer a stratified drug to be developed for a subgroup of the population when the drug is on average effective in the subgroup but with a detrimental effect in the complement. Whilst pharmaceutical manufacturers' preferences are similar, regions of disagreement exist. We show how preferences can be aligned by either penalizing the development of a non-stratified drug or rewarding the development of a stratified drug. The cost and position of alignment depends on the true value of health to the healthcare provider, among other parameters.

Optimizing subgroup selection in two-stage adaptive enrichment and umbrella designs

We design two‐stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision‐theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per‐comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.

Collaborative Platform Trials to Fight COVID-19: Methodological and Regulatory Considerations for a Better Societal Outcome

For the development of coronavirus disease 2019 (COVID‐19) drugs during the ongoing pandemic, speed is of essence whereas quality of evidence is of paramount importance. Although thousands of COVID‐19 trials were rapidly started, many are unlikely to provide robust statistical evidence and meet regulatory standards (e.g., because of lack of randomization or insufficient power). This has led to an inefficient use of time and resources. With more coordination, the sheer number of patients in these trials might have generated convincing data for several investigational treatments. Collaborative platform trials, comparing several drugs to a shared control arm, are an attractive solution. Those trials can utilize a variety of adaptive design features in order to accelerate the finding of life‐saving treatments. In this paper, we discuss several possible designs, illustrate them via simulations, and also discuss challenges, such as the heterogeneity of the target population, time‐varying standard of care, and the potentially high number of false hypothesis rejections in phase II and phase III trials. We provide corresponding regulatory perspectives on approval and reimbursement, and note that the optimal design of a platform trial will differ with our societal objective and by stakeholder. Hasty approvals may delay the development of better alternatives, whereas searching relentlessly for the single most efficacious treatment may indirectly diminish the number of lives saved as time is lost. We point out the need for incentivizing developers to participate in collaborative evidence‐generation initiatives when a positive return on investment is not met.

Optimized multiple testing procedures for nested sub-populations based on a continuous biomarker

An important step in the development of targeted therapies is the identification and confirmation of sub-populations where the treatment has a positive treatment effect compared to a control. These sub-populations are often based on continuous biomarkers, measured at baseline. For example, patients can be classified into biomarker low and biomarker high subgroups, which are defined via a threshold on the continuous biomarker. However, if insufficient information on the biomarker is available, the a priori choice of the threshold can be challenging and it has been proposed to consider several thresholds and to apply appropriate multiple testing procedures to test for a treatment effect in the corresponding subgroups controlling the family-wise type 1 error rate. In this manuscript we propose a framework to select optimal thresholds and corresponding optimized multiple testing procedures that maximize the expected power to identify at least one subgroup with a positive treatment effect. Optimization is performed over a prior on a family of models, modelling the relation of the biomarker with the expected outcome under treatment and under control. We find that for the considered scenarios 3 to 4 thresholds give the optimal power. If there is a prior belief on a small subgroup where the treatment has a positive effect, additional optimization of the spacing of thresholds may result in a large benefit. The procedure is illustrated with a clinical trial example in depression.

Optimal Decision Criteria for the Study Design and Sample Size of a Biomarker-Driven Phase III Trial

BACKGROUND

The design and sample size of a phase III study for new medical technologies were historically determined within the framework of frequentist hypothesis testing. Recently, drug development using predictive biomarkers, which can predict efficacy based on the status of biomarkers, has attracted attention, and various study designs using predictive biomarkers have been suggested. Additionally, when choosing a study design, considering economic factors, such as the risk of development, expected revenue, and cost, is important.

METHODS

Here, we propose a method to determine the optimal phase III design and sample size and judge whether the phase III study will be conducted using the expected net present value (eNPV). The eNPV is defined using the probability of success of the study calculated based on historical data, the revenue that will be obtained after the success of the phase III study, and the cost of the study. Decision procedures of the optimal phase III design and sample size considering historical data obtained up to the start of the phase III study were considered using numerical examples.

RESULTS

Based on the numerical examples, the optimal study design and sample size depend on the mean treatment effect in the biomarker-positive and biomarker-negative populations obtained from historical data, the between-trial variance of response, the prevalence of the biomarker-positive population, and the threshold value of probability of success required to go to phase III study.

CONCLUSIONS

Thus, the design and sample size of a biomarker-driven phase III study can be appropriately determined based on the eNPV.

A conditional error function approach for adaptive enrichment designs with continuous endpoints

Adaptive enrichment designs offer an efficient and flexible way to demonstrate the efficacy of a treatment in a clinically defined full population or in, eg, biomarker-defined subpopulations while controlling the family-wise Type I error rate in the strong sense. Frequently used testing strategies in designs with two or more stages include the combination test and the conditional error function approach. Here, we focus on the latter and present some extensions. In contrast to previous work, we allow for multiple subgroups rather than one subgroup only. For nested as well as nonoverlapping subgroups with normally distributed endpoints, we explore the effect of estimating the variances in the subpopulations. Instead of using a normal approximation, we derive new t-distribution-based methods for two different scenarios. First, in the case of equal variances across the subpopulations, we present exact results using a multivariate t-distribution. Second, in the case of potentially varying variances across subgroups, we provide some improved approximations compared to the normal approximation. The performance of the proposed conditional error function approaches is assessed and compared to the combination test in a simulation study. The proposed methods are motivated by an example in pulmonary arterial hypertension.

Exploratory analyses of effect modifiers in the antidepressant treatment of major depression: Individual-participant data meta-analysis of 2803 participants in seven placebo-controlled randomized trials

BACKGROUND

It is clinically important to know who are likely to respond more or less to antidepressants. However, meaningful effect modifiers (variables associated with differential response depending on the treatment) are yet to be identified.

METHODS

We conducted individual participant data (IPD) meta-analysis and meta-regression to explore effect modifiers in placebo-controlled antidepressant trials conducted so far in Japan.

RESULTS

We obtained access to IPD from seven placebo-controlled trials comparing bupropion, duloxetine, escitalopram, mirtazapine, paroxetine or venlafaxine with placebo in the acute phase treatment of major depression (total n = 2803). The higher the guilt subscale score of the baseline Hamilton Rating Scale for Depression (HRSD), the greater the difference in reduction in depression severity between the antidepressants and placebo at week 6, while the older the current age or the age at onset, the smaller the difference. At week 8, the guilt subscale score of HRSD and presence of suicidal ideation at baseline predicted greater, and the anhedonia subscale and insomnia subscale scores of HRSD and early response at week 2 predicted smaller, difference in reduction.

LIMITATIONS

Different studies measured different sets of baseline variables and we were able to analyze only a limited set of candidate variables for effect modification.

CONCLUSION

Age, age at onset, several HRSD subscales including guilt, anhedonia and insomnia, presence of suicidal ideation at baseline and early response are potential effect modifiers for response to antidepressants in the acute phase antidepressant treatment of major depression. Future trials need to measure these and additional variables in concerted efforts to enable matching of treatments with individual characteristics in depression.

Design of experiments for a confirmatory trial of precision medicine

Precision medicine, aka stratified/personalized medicine, is becoming more pronounced in the medical field due to advancement in computational ability to learn about patient genomic backgrounds. A biomaker, i.e. a type of biological process indicator, is often used in precision medicine to classify patient population into several subgroups. The aim of precision medicine is to tailor treatment regimes for different patient subgroups who suffer from the same disease. A multi-arm design could be conducted to explore the effect of treatment regimes on different biomarker subgroups. However, if treatments work only on certain subgroups, which is often the case, enrolling all patient subgroups in a confirmatory trial would increase the burden of a study. Having observed a phase II trial, we propose a design framework for finding an optimal design that could be implemented in a phase III study or a confirmatory trial. We consider two elements in our approach: Bayesian data analysis of observed data, and design of experiments. The first tool selects subgroups and treatments to be enrolled in the future trial whereas the second tool provides an optimal treatment randomization scheme for each selected/enrolled subgroups. Considering two independent treatments and two independent biomarkers, we illustrate our approach using simulation studies. We demonstrate efficiency gain, i.e. high probability of recommending truly effective treatments in the right subgroup, of the optimal design found by our framework over a randomized controlled trial and a biomarker–treatment linked trial.

•

A classical randomized controlled trial fails to identify subgroup treatment effect.

•

Standard enriched designs may miss out potential patient subgroups.

•

A standard multi-arm design could be inefficient for a trial of precision medicine.

•

A data-driven design framework could provide efficient designs for future trials.

Recent advances in methodology for clinical trials in small populations: the InSPiRe project

Where there are a limited number of patients, such as in a rare disease, clinical trials in these small populations present several challenges, including statistical issues. This led to an EU FP7 call for proposals in 2013. One of the three projects funded was the Innovative Methodology for Small Populations Research (InSPiRe) project. This paper summarizes the main results of the project, which was completed in 2017.The InSPiRe project has led to development of novel statistical methodology for clinical trials in small populations in four areas. We have explored new decision-making methods for small population clinical trials using a Bayesian decision-theoretic framework to compare costs with potential benefits, developed approaches for targeted treatment trials, enabling simultaneous identification of subgroups and confirmation of treatment effect for these patients, worked on early phase clinical trial design and on extrapolation from adult to pediatric studies, developing methods to enable use of pharmacokinetics and pharmacodynamics data, and also developed improved robust meta-analysis methods for a small number of trials to support the planning, analysis and interpretation of a trial as well as enabling extrapolation between patient groups. In addition to scientific publications, we have contributed to regulatory guidance and produced free software in order to facilitate implementation of the novel methods.

Point estimation following two-stage adaptive threshold enrichment clinical trials

Recently, several study designs incorporating treatment effect assessment in biomarker-based subpopulations have been proposed. Most statistical methodologies for such designs focus on the control of type I error rate and power. In this paper, we have developed point estimators for clinical trials that use the two-stage adaptive enrichment threshold design. The design consists of two stages, where in stage 1, patients are recruited in the full population. Stage 1 outcome data are then used to perform interim analysis to decide whether the trial continues to stage 2 with the full population or a subpopulation. The subpopulation is defined based on one of the candidate threshold values of a numerical predictive biomarker. To estimate treatment effect in the selected subpopulation, we have derived unbiased estimators, shrinkage estimators, and estimators that estimate bias and subtract it from the naive estimate. We have recommended one of the unbiased estimators. However, since none of the estimators dominated in all simulation scenarios based on both bias and mean squared error, an alternative strategy would be to use a hybrid estimator where the estimator used depends on the subpopulation selected. This would require a simulation study of plausible scenarios before the trial.

Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

BACKGROUND

IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers.

METHOD

The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials.

RESULTS

The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials.

CONCLUSION

IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.

Approaches to sample size calculation for clinical trials in rare diseases

We discuss 3 alternative approaches to sample size calculation: traditional sample size calculation based on power to show a statistically significant effect, sample size calculation based on assurance, and sample size based on a decision-theoretic approach. These approaches are compared head-to-head for clinical trial situations in rare diseases. Specifically, we consider 3 case studies of rare diseases (Lyell disease, adult-onset Still disease, and cystic fibrosis) with the aim to plan the sample size for an upcoming clinical trial. We outline in detail the reasonable choice of parameters for these approaches for each of the 3 case studies and calculate sample sizes. We stress that the influence of the input parameters needs to be investigated in all approaches and recommend investigating different sample size approaches before deciding finally on the trial size. Highly influencing for the sample size are choice of treatment effect parameter in all approaches and the parameter for the additional cost of the new treatment in the decision-theoretic approach. These should therefore be discussed extensively.

Optimized adaptive enrichment designs

Based on a Bayesian decision theoretic approach, we optimize frequentist single- and adaptive two-stage trial designs for the development of targeted therapies, where in addition to an overall population, a pre-defined subgroup is investigated. In such settings, the losses and gains of decisions can be quantified by utility functions that account for the preferences of different stakeholders. In particular, we optimize expected utilities from the perspectives both of a commercial sponsor, maximizing the net present value, and also of the society, maximizing cost-adjusted expected health benefits of a new treatment for a specific population. We consider single-stage and adaptive two-stage designs with partial enrichment, where the proportion of patients recruited from the subgroup is a design parameter. For the adaptive designs, we use a dynamic programming approach to derive optimal adaptation rules. The proposed designs are compared to trials which are non-enriched (i.e. the proportion of patients in the subgroup corresponds to the prevalence in the underlying population). We show that partial enrichment designs can substantially improve the expected utilities. Furthermore, adaptive partial enrichment designs are more robust than single-stage designs and retain high expected utilities even if the expected utilities are evaluated under a different prior than the one used in the optimization. In addition, we find that trials optimized for the sponsor utility function have smaller sample sizes compared to trials optimized under the societal view and may include the overall population (with patients from the complement of the subgroup) even if there is substantial evidence that the therapy is only effective in the subgroup.

A decision theoretical modeling for Phase III investments and drug licensing

For a new candidate drug to become an approved medicine, several decision points have to be passed. In this article, we focus on two of them: First, based on Phase II data, the commercial sponsor decides to invest (or not) in Phase III. Second, based on the outcome of Phase III, the regulator determines whether the drug should be granted market access. Assuming a population of candidate drugs with a distribution of true efficacy, we optimize the two stakeholders' decisions and study the interdependence between them. The regulator is assumed to seek to optimize the total public health benefit resulting from the efficacy of the drug and a safety penalty. In optimizing the regulatory rules, in terms of minimal required sample size and the Type I error in Phase III, we have to consider how these rules will modify the commercial optimization made by the sponsor. The results indicate that different Type I errors should be used depending on the rarity of the disease.

PIPELINEs: Creating Comparable Clinical Knowledge Efficiently by Linking Trial Platforms

Adaptive, seamless, multisponsor, multitherapy clinical trial designs executed as large scale platforms, could create superior evidence more efficiently than single-sponsor, single-drug trials. These trial PIPELINEs also could diminish barriers to trial participation, increase the representation of real-world populations, and create systematic evidence development for learning throughout a therapeutic life cycle, to continually refine its use. Comparable evidence could arise from multiarm design, shared comparator arms, and standardized endpoints-aiding sponsors in demonstrating the distinct value of their innovative medicines; facilitating providers and patients in selecting the most appropriate treatments; assisting regulators in efficacy and safety determinations; helping payers make coverage and reimbursement decisions; and spurring scientists with translational insights. Reduced trial times and costs could enable more indications, reduced development cycle times, and improved system financial sustainability. Challenges to overcome range from statistical to operational to collaborative governance and data exchange.