Online control of the False Discovery Rate in group-sequential platform trials

Why and how should we simulate platform trials? Learnings from EU-PEARL

BACKGROUND

Platform trials are innovative clinical trials governed by a master protocol that allows for the evaluation of multiple investigational treatments that enter and leave the trial over time. Interest in platform trials has been steadily increasing over the last decade. Due to their highly adaptive nature, platform trials provide sufficient flexibility to customize important trial design aspects to the requirements of both the specific disease under investigation and the different stakeholders. The flexibility of platform trials, however, comes with complexities when designing such trials. In the past, we reviewed existing software for simulating clinical trials and found that none of them were suitable for simulating platform trials as they do not accommodate the design features and flexibility inherent to platform trials, such as staggered entry of treatments over time.

RESULTS

We argued that simulation studies are crucial for the design of efficient platform trials. We developed and proposed an iterative, simulation-guided "vanilla and sprinkles" framework, i.e. from a basic to a more complex design, for designing platform trials. We addressed the functionality limitations of existing software as well as the unavailability of the coding therein by developing a suite of open-source software to use in simulating platform trials based on the R programming language. To give some examples, the newly developed software supports simulating staggered entry of treatments throughout the trial, choosing different options for control data sharing, specifying different platform stopping rules and platform-level operating characteristics. The software we developed is available through open-source licensing to enable users to access and modify the code. The separate use of two of these software packages to implement the same platform design by independent teams obtained the same results.

CONCLUSION

We provide a framework, as well as open-source software for the design and simulation of platform trials. The software tools provide the flexibility necessary to capture the complexity of platform trials.

A Bayesian Hybrid Design With Borrowing From Historical Study

In early phase drug development of combination therapy, the primary objective is to preliminarily assess whether there is additive activity from a novel agent when combined with an established monotherapy. Due to potential feasibility issues for conducting a large randomized study, uncontrolled single-arm trials have been the mainstream approach in cancer clinical trials. However, such trials often present significant challenges in deciding whether to proceed to the next phase of development due to the lack of randomization in traditional two-arm trials. A hybrid design, leveraging data from a completed historical clinical study of the monotherapy, offers a valuable option to enhance study efficiency and improve informed decision-making. Compared to traditional single-arm designs, the hybrid design may significantly enhance power by borrowing external information, enabling a more robust assessment of activity. The primary challenge of hybrid design lies in handling information borrowing. We introduce a Bayesian dynamic power prior (DPP) framework with three components of controlling amount of dynamic borrowing. The framework offers flexible study design options with explicit interpretation of borrowing, allowing customization according to specific needs. Furthermore, the posterior distribution in the proposed framework has a closed form, offering significant advantages in computational efficiency. The proposed framework's utility is demonstrated through simulations and a case study.

Regulatory Issues of Platform Trials: Learnings from EU-PEARL

Although platform trials have many benefits, the complexity of these designs may result not only in increased methodological but also regulatory and ethical challenges. These aspects were addressed as part of the IMI project EU Patient-Centric Clinical Trial Platforms (EU-PEARL). We reviewed the available guidelines on platform trials in the European Union and the United States. This is supported and complemented by feedback received from regulatory interactions with the European Medicines Agency and the US Food and Drug Administration. Throughout the project we collected the needs of all relevant stakeholders including ethics committees, regulators, and health technology assessment bodies through active dialog and dedicated stakeholder workshops. Furthermore, we focused on methodological aspects and where applicable identified the corresponding guidance. Learnings from the guideline review, regulatory interactions, and workshops are provided. Based on these, a master protocol template was developed. Issues that still need harmonization or clarification in guidelines or where further methodological research is needed are also presented. These include questions around clinical trial submissions in Europe, the need for multiplicity control across the whole master protocol, the use of non-concurrent controls, and the impact of different randomization schemes. Master protocols are an efficient and patient-centered clinical trial design that can expedite drug development. However, they can also introduce additional operational and regulatory complexities. It is important to understand the different requirements of stakeholders upfront and address them in the trial. While relevant guidance is increasing, early dialog with relevant stakeholders can help to further support such designs.

Current state-of-the-art and gaps in platform trials: 10 things you should know, insights from EU-PEARL

Platform trials bring the promise of making clinical research more efficient and more patient centric. While their use has become more widespread, including their prominent role during the COVID-19 pandemic response, broader adoption of platform trials has been limited by the lack of experience and tools to navigate the critical upfront planning required to launch such collaborative studies. The European Union-Patient-cEntric clinicAl tRial pLatform (EU-PEARL) initiative has produced new methodologies to expand the use of platform trials with an overarching infrastructure and services embedded into Integrated Research Platforms (IRPs), in collaboration with patient representatives and through consultation with U.S. Food and Drug Administration and European Medicines Agency stakeholders. In this narrative review, we discuss the outlook for platform trials in Europe, including challenges related to infrastructure, design, adaptations, data sharing and regulation. Documents derived from the EU-PEARL project, alongside a literature search including PubMed and relevant grey literature (e.g., guidance from regulatory agencies and health technology agencies) were used as sources for a multi-stage collaborative process through which the 10 more important points based on lessons drawn from the EU-PEARL project were developed and summarised as guidance for the setup of platform trials. We conclude that early involvement of critical stakeholder such as regulatory agencies or patients are critical steps in the implementation and later acceptance of platform trials. Addressing these gaps will be critical for attaining the full potential of platform trials for patients.

Funding

Innovative Medicines Initiative 2 Joint Undertaking with support from the European Union's Horizon 2020 research and innovation programme and EFPIA.

Online false discovery rate control for LORD++ and SAFFRON under positive, local dependence

Online testing procedures assume that hypotheses are observed in sequence, and allow the significance thresholds for upcoming tests to depend on the test statistics observed so far. Some of the most popular online methods include alpha investing, LORD++, and SAFFRON. These three methods have been shown to provide online control of the "modified" false discovery rate (mFDR) under a condition known as CS. However, to our knowledge, LORD++ and SAFFRON have only been shown to control the traditional false discovery rate (FDR) under an independence condition on the test statistics. Our work bolsters these results by showing that SAFFRON and LORD++ additionally ensure online control of the FDR under a "local" form of nonnegative dependence. Further, FDR control is maintained under certain types of adaptive stopping rules, such as stopping after a certain number of rejections have been observed. Because alpha investing can be recovered as a special case of the SAFFRON framework, our results immediately apply to alpha investing as well. In the process of deriving these results, we also formally characterize how the conditional super-uniformity assumption implicitly limits the allowed p-value dependencies. This implicit limitation is important not only to our proposed FDR result, but also to many existing mFDR results.

Online multiple hypothesis testing

Modern data analysis frequently involves large-scale hypothesis testing, which naturally gives rise to the problem of maintaining control of a suitable type I error rate, such as the false discovery rate (FDR). In many biomedical and technological applications, an additional complexity is that hypotheses are tested in an online manner, one-by-one over time. However, traditional procedures that control the FDR, such as the Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. To address these challenges, a new field of methodology has developed over the past 15 years showing how to control error rates for online multiple hypothesis testing. In this framework, hypotheses arrive in a stream, and at each time point the analyst decides whether to reject the current hypothesis based both on the evidence against it, and on the previous rejection decisions. In this paper, we present a comprehensive exposition of the literature on online error rate control, with a review of key theory as well as a focus on applied examples. We also provide simulation results comparing different online testing algorithms and an up-to-date overview of the many methodological extensions that have been proposed.

Online error rate control for platform trials

Platform trials evaluate multiple experimental treatments under a single master protocol, where new treatment arms are added to the trial over time. Given the multiple treatment comparisons, there is the potential for inflation of the overall type I error rate, which is complicated by the fact that the hypotheses are tested at different times and are not necessarily pre-specified. Online error rate control methodology provides a possible solution to the problem of multiplicity for platform trials where a relatively large number of hypotheses are expected to be tested over time. In the online multiple hypothesis testing framework, hypotheses are tested one-by-one over time, where at each time-step an analyst decides whether to reject the current null hypothesis without knowledge of future tests but based solely on past decisions. Methodology has recently been developed for online control of the false discovery rate as well as the familywise error rate (FWER). In this article, we describe how to apply online error rate control to the platform trial setting, present extensive simulation results, and give some recommendations for the use of this new methodology in practice. We show that the algorithms for online error rate control can have a substantially lower FWER than uncorrected testing, while still achieving noticeable gains in power when compared with the use of a Bonferroni correction. We also illustrate how online error rate control would have impacted a currently ongoing platform trial.

Designing an exploratory phase 2b platform trial in NASH with correlated, co-primary binary endpoints

Non-alcoholic steatohepatitis (NASH) is the progressive form of nonalcoholic fatty liver disease (NAFLD) and a disease with high unmet medical need. Platform trials provide great benefits for sponsors and trial participants in terms of accelerating drug development programs. In this article, we describe some of the activities of the EU-PEARL consortium (EU Patient-cEntric clinicAl tRial pLatforms) regarding the use of platform trials in NASH, in particular the proposed trial design, decision rules and simulation results. For a set of assumptions, we present the results of a simulation study recently discussed with two health authorities and the learnings from these meetings from a trial design perspective. Since the proposed design uses co-primary binary endpoints, we furthermore discuss the different options and practical considerations for simulating correlated binary endpoints.