Aiding the Adoption of Master Protocols by Optimizing Patient Engagement

Master protocols (MPs) are an important addition to the clinical trial repertoire. As defined by the U.S. Food and Drug Administration (FDA), this term means "a protocol designed with multiple sub-studies, which may have different objectives (goals) and involve coordinated efforts to evaluate one or more investigational drugs in one or more disease subtypes within the overall trial structure." This means we now have a unique, scientifically based MP that describes how a clinical trial will be conducted using one or more potential candidate therapies to treat patients in one or more diseases. Patient engagement (PE) is also a critical factor that has been recognized by FDA through its Patient-Focused Drug Development (PFDD) initiative, and by the European Medicines Agency (EMA), which states on its website that it has been actively interacting with patients since the creation of the Agency in 1995. We propose that utilizing these PE principles in MPs can make them more successful for sponsors, providers, and patients. Potential benefits of MPs for patients awaiting treatment can include treatments that better fit a patient's needs; availability of more treatments; and faster access to treatments. These make it possible to develop innovative therapies (especially for rare diseases and/or unique subpopulations, e.g., pediatrics), to minimize untoward side effects through careful dose escalation practices and, by sharing a control arm, to lower the probability of being assigned to a placebo arm for clinical trial participants. This paper is authored by select members of the American Statistical Association (ASA)/DahShu Master Protocol Working Group (MPWG) People and Patient Engagement (PE) Subteam. DahShu is a 501(c)(3) non-profit organization, founded to promote research and education in data science. This manuscript does not include direct feedback from US or non-US regulators, though multiple regulatory-related references are cited to confirm our observation that improving patient engagement is supported by regulators. This manuscript represents the authors' independent perspective on the Master Protocol; it does not represent the official policy or viewpoint of FDA or any other regulatory organization or the views of the authors' employers. The objective of this manuscript is to provide drug developers, contract research organizations (CROs), third party capital investors, patient advocacy groups (PAGs), and biopharmaceutical executives with a better understanding of how including the patient voice throughout MP development and conduct creates more efficient clinical trials. The PE Subteam also plans to publish a Plain Language Summary (PLS) of this publication for clinical trial participants, patients, caregivers, and the public as they seek to understand the risks and benefits of MP clinical trial participation.

How Electronic Medical Record Integration Can Support More Efficient Critical Care Clinical Trials

Large volumes of data are collected on critically ill patients, and using data science to extract information from the electronic medical record (EMR) and to inform the design of clinical trials represents a new opportunity in critical care research. Using improved methods of phenotyping critical illnesses, subject identification and enrollment, and targeted treatment group assignment alongside newer trial designs such as adaptive platform trials can increase efficiency while lowering costs. Some tools such as the EMR to automate data collection are already in use. Refinement of data science approaches in critical illness research will allow for better clinical trials and, ultimately, improved patient outcomes.

Platform trials to evaluate the benefit-risk of COVID-19 therapeutics: Successes, learnings, and recommendations for future pandemics

BACKGROUND

In response to the COVID-19 global pandemic, multiple platform trials were initiated to accelerate evidence generation of potential therapeutic interventions. Given a rapidly evolving and dynamic pandemic, platform trials have a key advantage over traditional randomized trials: multiple interventions can be investigated under a master protocol sharing a common infrastructure.

METHODS

This paper focuses on nine platform trials that were instrumental in advancing care in COVID-19 in the hospital and community setting. A semi-structured qualitative interview was conducted with the principal investigators and lead statisticians of these trials. Information from the interviews and public sources were tabulated and summarized across trials, and recommendations for best practice for the next health crisis are provided.

RESULTS

Based on the information gathered takeaways were identified as 1) the existence of some aspect of trial design or conduct (e.g., existing network of investigators or colleagues, infrastructure for data capture and relevant statistical expertise) was a key success factor; 2) the choice of treatments (e.g., repurposed drugs) had major impact on the trials as did the choice of primary endpoint; and 3) the lack of coordination across trials was flagged as an area for improvement.

CONCLUSION

These trials deployed during the COVID-19 pandemic demonstrate how to achieve both speed and quality of evidence generation regarding clinical benefit (or not) of existing therapies to treat new pathogens in a pandemic setting. As a group, these trials identified treatments that worked, and many that did not, in a matter of months.

On the use of non-concurrent controls in platform trials: a scoping review

BACKGROUND

Platform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met.

METHODS

We conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites.

RESULTS

Only 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive.

CONCLUSIONS

Statistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.

Bayes' Theorem in Neurocritical Care: Principles and Practice

Patients with critical neurological illness are diverse. As a result of the heterogeneity of this patient population, standardized approaches to patient management might not confer benefit. A precision medicine approach to neurocritical care is therefore urgently needed to improve our understanding of neurocritical illness and the care provided to this vulnerable cohort. Research designs and approaches based on Bayesian models have the potential to meet this need, as they are specifically designed to evolve with emerging evidence. This adaptability provides a benefit over the popular frequentist statistical approach, as it provides a way of adjusting hypotheses and trial procedures to maximize efficacy. This review summarizes the current state of knowledge on Bayes' theorem, and its potential applications to the field of neurocritical care. We review the basic principles underlying Bayes' theorem, compare the use of Bayesian versus frequentist statistics in medicine, and discuss the relevance of Bayesian statistics to the field of neuroscience and to clinical research. Finally, we explore the potential benefits of employing Bayesian methods within the field of neurocritical care as a steppingstone toward implementing precision medicine approaches to improve patient outcomes for complex, heterogeneous disorders.

Integrating non-concurrent controls in the analyses of late-entry experimental arms in multi-arm trials with a shared control group in the presence of parameter drift

BACKGROUND

Under a master protocol, open platform trials allow new experimental treatments to enter an existing clinical trial. Whether late-entry experimental treatments should be compared to all available or concurrently randomized controls is not well established. Using all available data can increase power and precision; however, drift in population parameters can yield biased estimates and impact type I error rate.

METHODS

We explored the application of methods developed to incorporate historical controls in two-arm trials to the analysis of a late-entry arm in a simulated open platform trial under varying scenarios of parameter drift. Methods explored include test-then-pool, fixed power prior, dynamic power prior, and multi-source exchangeability model approaches.

RESULTS/CONCLUSIONS

Simulated trial results confirm that in the presence of no drift, naively pooling all controls increases power and produces more precise, unbiased estimates when compared to using concurrent controls only. However, under drift, pooling can result in type I error rate inflation or deflation and biased estimates. In the presence of parameter drift, methods that partially borrow non-concurrent data, either through a static weighting mechanism or through methods that allow the heterogeneity between non-concurrent and concurrent data to determine the degree of borrowing, are superior to naively pooling the data. However, compared to using concurrent controls only, these approaches cannot guarantee type I error control or unbiased estimates. Thus, concurrent controls should be used as comparators in confirmatory studies.

On model-based time trend adjustments in platform trials with non-concurrent controls

BACKGROUND

Platform trials can evaluate the efficacy of several experimental treatments compared to a control. The number of experimental treatments is not fixed, as arms may be added or removed as the trial progresses. Platform trials are more efficient than independent parallel group trials because of using shared control groups. However, for a treatment entering the trial at a later time point, the control group is divided into concurrent controls, consisting of patients randomised to control when that treatment arm is in the platform, and non-concurrent controls, patients randomised before. Using non-concurrent controls in addition to concurrent controls can improve the trial's efficiency by increasing power and reducing the required sample size, but can introduce bias due to time trends.

METHODS

We focus on a platform trial with two treatment arms and a common control arm. Assuming that the second treatment arm is added at a later time, we assess the robustness of recently proposed model-based approaches to adjust for time trends when utilizing non-concurrent controls. In particular, we consider approaches where time trends are modeled either as linear in time or as a step function, with steps at time points where treatments enter or leave the platform trial. For trials with continuous or binary outcomes, we investigate the type 1 error rate and power of testing the efficacy of the newly added arm, as well as the bias and root mean squared error of treatment effect estimates under a range of scenarios. In addition to scenarios where time trends are equal across arms, we investigate settings with different time trends or time trends that are not additive in the scale of the model.

RESULTS

A step function model, fitted on data from all treatment arms, gives increased power while controlling the type 1 error, as long as the time trends are equal for the different arms and additive on the model scale. This holds even if the shape of the time trend deviates from a step function when patients are allocated to arms by block randomisation. However, if time trends differ between arms or are not additive to treatment effects in the scale of the model, the type 1 error rate may be inflated.

CONCLUSIONS

The efficiency gained by using step function models to incorporate non-concurrent controls can outweigh potential risks of biases, especially in settings with small sample sizes. Such biases may arise if the model assumptions of equality and additivity of time trends are not satisfied. However, the specifics of the trial, scientific plausibility of different time trends, and robustness of results should be carefully considered.

Combining factorial and multi-arm multi-stage platform designs to evaluate multiple interventions efficiently

BACKGROUND

Factorial-MAMS design platform designs have many advantages, but the practical advantages and disadvantages of combining the two designs have not been explored.

METHODS

We propose practical methods for a combined design within the platform trial paradigm where some interventions are not expected to interact and could be given together.

RESULTS

We describe the combined design and suggest diagrams that can be used to represent it. Many properties are common both to standard factorial designs, including the need to consider interactions between interventions and the impact of intervention efficacy on power of other comparisons, and to standard multi-arm multi-stage designs, including the need to pre-specify procedures for starting and stopping intervention comparisons. We also identify some specific features of the factorial-MAMS design: timing of interim and final analyses should be determined by calendar time or total observed events; some non-factorial modifications may be useful; eligibility criteria should be broad enough to include any patient eligible for any part of the randomisation; stratified randomisation may conveniently be performed sequentially; and analysis requires special care to use only concurrent controls.

CONCLUSION

A combined factorial-MAMS design can combine the efficiencies of factorial trials and multi-arm multi-stage platform trials. It allows us to address multiple research questions under one protocol and to test multiple new treatment options, which is particularly important when facing a new emergent infection such as COVID-19.

Implementation of platform trials in the COVID-19 pandemic: A rapid review

Motivation

Platform designs - master protocols that allow for new treatment arms to be added over time - have gained considerable attention in recent years. Between 2001 and 2019, 16 platform trials were initiated globally. The COVID-19 pandemic seems to have provided a new motivation for these designs. We conducted a rapid review to quantify and describe platform trials used in COVID-19.

Methods

We cross-referenced PubMed, ClinicalTrials.gov, and the Cytel COVID-19 Clinical Trials Tracker to identify platform trials, defined by their stated ability to add future arms.

Results

We identified 58 COVID-19 platform trials globally registered between January 2020 and May 2021. According to trial registries, 16 trials have added new therapies (median 3, IQR 4) and 11 have dropped arms (median 3, IQR 2.5). About 50% of trials publicly share their protocol, and 31 trials (53%) intend to share trial data. Forty-nine trials (84%) explicitly report adaptive features, and 21 trials (36%) state Bayesian methods.

Conclusions

During the pandemic, there has been a surge in the number of platform trials compared to historical use. While transparency in statistical methods and clarity of data sharing policies needs improvement, platform trials appear particularly well-suited for rapid evidence generation. Trials secured funding quickly and many succeeded in adding new therapies in a short time period, thus demonstrating the potential for these trial designs to be implemented beyond the pandemic. The evidence gathered here may provide ample insight to further inform operational, statistical, and regulatory aspects of future platform trial conduct.

Stratified randomization for platform trials with differing experimental arm eligibility

BACKGROUND

Platform trials facilitate efficient use of resources by comparing multiple experimental agents to a common standard of care arm. They can accommodate a changing scientific paradigm within a single trial protocol by adding or dropping experimental arms-critical features for trials in rapidly developing disease areas such as COVID-19 or cancer therapeutics. However, in these trials, efficacy and safety issues may render certain participant subgroups ineligible to some experimental arms, and methods for stratified randomization do not readily apply to this setting.

METHODS

We propose extensions for conventional methods of stratified randomization for platform trials whose experimental arms may differ in eligibility criteria. These methods balance on a prespecified set of stratification variables observable prior to arm assignment that remains the same across experimental arms. To do so, we suggest modifying block randomization by including experimental arm eligibility as a stratifying variable, and we suggest modifying the imbalance score calculation in dynamic balancing by performing pairwise comparisons between each eligible experimental arm and standard of care arm participants eligible to that experimental arm.

RESULTS

We provide worked examples to illustrate the proposed extensions. In addition, we also provide a formula to quantify the relative efficiency loss of platform trials with varying eligibility compared with trials with non-varying eligibility as measured by the size of the common standard of care arm.

CONCLUSIONS

This article presents important extensions to conventional methods for stratified randomization in order to facilitate the implementation of platform trials with differing experimental arm eligibility.

Statistical consideration when adding new arms to ongoing clinical trials: the potentials and the caveats

BACKGROUND

Platform trials improve the efficiency of the drug development process through flexible features such as adding and dropping arms as evidence emerges. The benefits and practical challenges of implementing novel trial designs have been discussed widely in the literature, yet less consideration has been given to the statistical implications of adding arms. MAIN: We explain different statistical considerations that arise from allowing new research interventions to be added in for ongoing studies. We present recent methodology development on addressing these issues and illustrate design and analysis approaches that might be enhanced to provide robust inference from platform trials. We also discuss the implication of changing the control arm, how patient eligibility for different arms may complicate the trial design and analysis, and how operational bias may arise when revealing some results of the trials. Lastly, we comment on the appropriateness and the application of platform trials in phase II and phase III settings, as well as publicly versus industry-funded trials.

CONCLUSION

Platform trials provide great opportunities for improving the efficiency of evaluating interventions. Although several statistical issues are present, there are a range of methods available that allow robust and efficient design and analysis of these trials.

Infectious diseases epidemiology, quantitative methodology, and clinical research in the midst of the COVID-19 pandemic: Perspective from a European country

Starting from historic reflections, the current SARS-CoV-2 induced COVID-19 pandemic is examined from various perspectives, in terms of what it implies for the implementation of non-pharmaceutical interventions, the modeling and monitoring of the epidemic, the development of early-warning systems, the study of mortality, prevalence estimation, diagnostic and serological testing, vaccine development, and ultimately clinical trials. Emphasis is placed on how the pandemic had led to unprecedented speed in methodological and clinical development, the pitfalls thereof, but also the opportunities that it engenders for national and international collaboration, and how it has simplified and sped up procedures. We also study the impact of the pandemic on clinical trials in other indications. We note that it has placed biostatistics, epidemiology, virology, infectiology, and vaccinology, and related fields in the spotlight in an unprecedented way, implying great opportunities, but also the need to communicate effectively, often amidst controversy.

Including non-concurrent control patients in the analysis of platform trials: is it worth it?

Background

Platform trials allow adding new experimental treatments to an on-going trial. This feature is attractive to practitioners due to improved efficiency. Nevertheless, the operating characteristics of a trial that adds arms have not been well-studied. One controversy is whether just the concurrent control data (i.e. of patients who are recruited after a new arm is added) should be used in the analysis of the newly added treatment(s), or all control data (i.e. non-concurrent and concurrent).

Methods

We investigate the benefits and drawbacks of using non-concurrent control data within a two-stage setting. We perform simulation studies to explore the impact of a linear and a step trend on the inference of the trial. We compare several analysis approaches when one includes all the control data or only concurrent control data in the analysis of the newly added treatment.

Results

When there is a positive trend and all the control data are used, the marginal power of rejecting the corresponding hypothesis and the type one error rate can be higher than the nominal value. A model-based approach adjusting for a stage effect is equivalent to using concurrent control data; an adjustment with a linear term may not guarantee valid inference when there is a non-linear trend.

Conclusions

If strict error rate control is required then non-concurrent control data should not be used; otherwise it may be beneficial if the trend is sufficiently small. On the other hand, the root mean squared error of the estimated treatment effect can be improved through using non-concurrent control data.

An overview of platform trials with a checklist for clinical readers

OBJECTIVES

The objective of the study was to outline key considerations for general clinical readers when critically evaluating publications on platform trials and for researchers when designing these types of clinical trials.

STUDY DESIGN AND SETTING

In this review, we describe key concepts of platform trials with case study discussion of two hallmark platform trials in STAMPEDE and I-SPY2. We provide reader's guide to platform trials with a critical appraisal checklist.

RESULTS

Platform trials offer flexibilities of dropping ineffective arms early based on interim data and introducing new arms into the trial. For platform trials, it is important to consider how interventions are compared and evaluated throughout and how new interventions are introduced. For intervention comparisons, it is important to consider what the primary analysis is, what and how many interventions are active simultaneously, and allocation between different arms. Interim evaluation considerations should include the number and timing of interim evaluations and outcomes and statistical rules used to drop interventions. New interventions are usually introduced based on scientific merits, so consideration of these merits is important, together with the timing and mechanisms in which new interventions are added.

CONCLUSION

More efforts are needed to improve the scientific literacy of platform trials. Our review provides an overview of the important concepts of platform trials.

Adding new experimental arms to randomised clinical trials: Impact on error rates

BACKGROUND

Experimental treatments pass through various stages of development. If a treatment passes through early-phase experiments, the investigators may want to assess it in a late-phase randomised controlled trial. An efficient way to do this is adding it as a new research arm to an ongoing trial while the existing research arms continue, a so-called multi-arm platform trial. The familywise type I error rate is often a key quantity of interest in any multi-arm platform trial. We set out to clarify how it should be calculated when new arms are added to a trial some time after it has started.

METHODS

We show how the familywise type I error rate, any-pair and all-pairs powers can be calculated when a new arm is added to a platform trial. We extend the Dunnett probability and derive analytical formulae for the correlation between the test statistics of the existing pairwise comparison and that of the newly added arm. We also verify our analytical derivation via simulations.

RESULTS

Our results indicate that the familywise type I error rate depends on the shared control arm information (i.e. individuals in continuous and binary outcomes and primary outcome events in time-to-event outcomes) from the common control arm patients and the allocation ratio. The familywise type I error rate is driven more by the number of pairwise comparisons and the corresponding (pairwise) type I error rates than by the timing of the addition of the new arms. The familywise type I error rate can be estimated using Šidák's correction if the correlation between the test statistics of pairwise comparisons is less than 0.30.

CONCLUSIONS

The findings we present in this article can be used to design trials with pre-planned deferred arms or to add new pairwise comparisons within an ongoing platform trial where control of the pairwise error rate or familywise type I error rate (for a subset of pairwise comparisons) is required.

Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols

BACKGROUND

Master protocols, classified as basket trials, umbrella trials, and platform trials, are novel designs that investigate multiple hypotheses through concurrent sub-studies (e.g., multiple treatments or populations or that allow adding/removing arms during the trial), offering enhanced efficiency and a more ethical approach to trial evaluation. Despite the many advantages of these designs, they are infrequently used.

METHODS

We conducted a landscape analysis of master protocols using a systematic literature search to determine what trials have been conducted and proposed for an overall goal of improving the literacy in this emerging concept. On July 8, 2019, English-language studies were identified from MEDLINE, EMBASE, and CENTRAL databases and hand searches of published reviews and registries.

RESULTS

We identified 83 master protocols (49 basket, 18 umbrella, and 16 platform trials). The number of master protocols has increased rapidly over the last five years. Most have been conducted in the US (n = 44/83) and investigated experimental drugs (n = 82/83) in the field of oncology (n = 76/83). The majority of basket trials were exploratory (i.e., phase I/II; n = 47/49) and not randomized (n = 44/49), and more than half (n = 28/48) investigated only a single intervention. The median sample size of basket trials was 205 participants (interquartile range, Q3-Q1 [IQR]: 500-90 = 410), and the median study duration was 22.3 (IQR: 74.1-42.9 = 31.1) months. Similar to basket trials, most umbrella trials were exploratory (n = 16/18), but the use of randomization was more common (n = 8/18). The median sample size of umbrella trials was 346 participants (IQR: 565-252 = 313), and the median study duration was 60.9 (IQR: 81.3-46.9 = 34.4) months. The median number of interventions investigated in umbrella trials was 5 (IQR: 6-4 = 2). The majority of platform trials were randomized (n = 15/16), and phase III investigation (n = 7/15; one did not report information on phase) was more common in platform trials with four of them using seamless II/III design. The median sample size was 892 (IQR: 1835-255 = 1580), and the median study duration was 58.9 (IQR: 101.3-36.9 = 64.4) months.

CONCLUSIONS

We anticipate that the number of master protocols will continue to increase at a rapid pace over the upcoming decades. More efforts to improve awareness and training are needed to apply these innovative trial design methods to fields outside of oncology.

Reporting of master protocols towards a standardized approach: A systematic review

BACKGROUND

In September 2018 the FDA provided a draft guidance on master protocols reflecting an increased interest in these designs by industry. Master protocols refer to a single overarching protocol developed to evaluate multiple hypotheses and may be further categorized as basket, umbrella, and platform trials. However, inconsistencies in reporting persist in the literature. We conducted a systematic review to describe master protocol reporting with the goal of facilitating the further development and spread of these innovative trial designs.

METHODS

We searched MEDLINE, EMBASE, and CENTRAL from inception to April 25, 2019 for English articles on master protocols. This was supplemented by hand searches of trial registries and of the bibliographies of published reviews. We used the FDA's definitions of master protocols as references and compared them to self-reported master protocols.

RESULTS

We identified 278 master protocol publications, consisting of 228 protocols and 50 reviews. Sixty-six records provided unique definitions of master protocol types. We observed considerable heterogeneity in definitions of master protocols, and over half (54%) used oncology-specific language. The majority of self-classified master protocols (57%) were consistent with the FDA's definitions of master protocols.

CONCLUSION

The terms 'master protocol', 'basket trial', 'umbrella trial', and 'platform trial' are inconsistently described. Careful treatment of these terms and adherence to the definitions set forth by the FDA will facilitate better understanding of these trial designs and allow them to be used broadly and to their full potential in clinical research. We encourage trial methodologists to use these trial designations when applicable.

The Brave New World of clinical cancer research: Adaptive biomarker-driven trials integrating clinical practice with clinical research

Clinical trials are the final links in the chains of knowledge and for determining the roles of therapeutic advances. Unfortunately, in an important sense they are the weakest links. This article describes two designs that are being explored today: platform trials and basket trials. Both are attempting to merge clinical research and clinical practice.