A decision theoretical modeling for Phase III investments and drug licensing

Quantifying the Benefits of Digital Biomarkers and Technology-Based Study Endpoints in Clinical Trials: Project Moneyball

Introduction

Digital biomarkers have significant potential to transform drug development, but only a few have contributed meaningfully to bring new treatments to market. There are uncertainties in how they will generate quantifiable benefits in clinical trial performance and ultimately to the chances of phase 3 success. Here we have proposed a statistical framework and ran a proof-of-concept model with hypothetical digital biomarkers and visualized them in a familiar manner to study power calculation.

Methods

A Monte Carlo simulation for Parkinson's disease (PD) was performed using the Captario SUM® platform and illustrative study technology impact calculations were generated. We took inspiration from the EMA-qualified wearable-derived digital endpoint stride velocity 95^th centile (SV95C) for Duchenne muscular dystrophy, and we imagined a similar measurement for PD would be available in the future. DaTscan enrichment and “SV95C-like” endpoint biomarkers were assumed on a hypothetical disease-modifying drug pivotal trial aiming for an 80% probability of achieving a study p value of less than 0.05.

Results

Four scenarios with different combinations of technologies were illustrated. The model illustrated a way to quantify the magnitude of the contributions that enrichment and endpoint technologies could make to drug development studies.

Discussion/Conclusion

Quantitative models could be valuable not only for the study sponsors but also as an interactive and collaborative engagement tool for technology players and multi-stakeholder consortia. Establishing values of digital biomarkers could also facilitate business cases and financial investments.

A framework for assessing the impact of accelerated approval

The FDA's Accelerated Approval program (AA) is a regulatory program to expedite availability of products to treat serious or life-threatening illnesses that lack effective treatment alternatives. Ideally, all of the many stakeholders such as patients, physicians, regulators, and health technology assessment [HTA] agencies that are affected by AA should benefit from it. In practice, however, there is intense debate over whether evidence supporting AA is sufficient to meet the needs of the stakeholders who collectively bring an approved product into routine clinical care. As AAs have become more common, it becomes essential to be able to determine their impact objectively and reproducibly in a way that provides for consistent evaluation of therapeutic decision alternatives. We describe the basic features of an approach for evaluating AA impact that accommodates stakeholder-specific views about potential benefits, risks, and costs. The approach is based on a formal decision-analytic framework combining predictive distributions for therapeutic outcomes (efficacy and safety) based on statistical models that incorporate findings from AA trials with stakeholder assessments of various actions that might be taken. The framework described here provides a starting point for communicating the value of a treatment granted AA in the context of what is important to various stakeholders.

Do strict decision criteria hamper productivity in the pharmaceutical industry?

The discouragingly high rates of attrition in drug development, and in particular in Phase 2, warrant a closer look at the decision criteria applied for investment in the next phase (Phase 3). We have in this article evaluated Stop/Go criteria after Phase 2, based on a model encompassing both Phase 2 and 3, as well as the eventual outcome on the market. The results indicate that the value of a drug project is often maximized if rather liberal decision criteria are applied. The routine adherence to standard criteria, e.g. requiring significance at 5% level, may lead to an unduly high rate of false negative decisions. This might ultimately hamper the productivity of drug development and leading to potentially useful drugs not being taken forward to benefit the intended patients.

Selection bias, investment decisions and treatment effect distributions

When making decisions regarding the investment and design for a Phase 3 programme in the development of a new drug, the results from preceding Phase 2 trials are an important source of information. However, only projects in which the Phase 2 results show promising treatment effects will typically be considered for a Phase 3 investment decision. This implies that, for those projects where Phase 3 is pursued, the underlying Phase 2 estimates are subject to selection bias. We will in this article investigate the nature of this selection bias based on a selection of distributions for the treatment effect. We illustrate some properties of Bayesian estimates, providing shrinkage of the Phase 2 estimate to counteract the selection bias. We further give some empirical guidance regarding the choice of prior distribution and comment on the consequences for decision-making in investment and planning for Phase 3 programmes.

Collaborative Platform Trials to Fight COVID-19: Methodological and Regulatory Considerations for a Better Societal Outcome

For the development of coronavirus disease 2019 (COVID‐19) drugs during the ongoing pandemic, speed is of essence whereas quality of evidence is of paramount importance. Although thousands of COVID‐19 trials were rapidly started, many are unlikely to provide robust statistical evidence and meet regulatory standards (e.g., because of lack of randomization or insufficient power). This has led to an inefficient use of time and resources. With more coordination, the sheer number of patients in these trials might have generated convincing data for several investigational treatments. Collaborative platform trials, comparing several drugs to a shared control arm, are an attractive solution. Those trials can utilize a variety of adaptive design features in order to accelerate the finding of life‐saving treatments. In this paper, we discuss several possible designs, illustrate them via simulations, and also discuss challenges, such as the heterogeneity of the target population, time‐varying standard of care, and the potentially high number of false hypothesis rejections in phase II and phase III trials. We provide corresponding regulatory perspectives on approval and reimbursement, and note that the optimal design of a platform trial will differ with our societal objective and by stakeholder. Hasty approvals may delay the development of better alternatives, whereas searching relentlessly for the single most efficacious treatment may indirectly diminish the number of lives saved as time is lost. We point out the need for incentivizing developers to participate in collaborative evidence‐generation initiatives when a positive return on investment is not met.

Efficient two-stage sequential arrays of proof of concept studies for pharmaceutical portfolios

Previous work has shown that individual randomized "proof-of-concept" (PoC) studies may be designed to maximize cost-effectiveness, subject to an overall PoC budget constraint. Maximizing cost-effectiveness has also been considered for arrays of simultaneously executed PoC studies. Defining Type III error as the opportunity cost of not performing a PoC study, we evaluate the common pharmaceutical practice of allocating PoC study funds in two stages. Stage 1, or the first wave of PoC studies, screens drugs to identify those to be permitted additional PoC studies in Stage 2. We investigate if this strategy significantly improves efficiency, despite slowing development. We quantify the benefit, cost, benefit-cost ratio, and Type III error given the number of Stage 1 PoC studies. Relative to a single stage PoC strategy, significant cost-effective gains are seen when at least one of the drugs has a low probability of success (10%) and especially when there are either few drugs (2) with a large number of indications allowed per drug (10) or a large portfolio of drugs (4). In these cases, the recommended number of Stage 1 PoC studies ranges from 2 to 4, tracking approximately with an inflection point in the minimization curve of Type III error.

A modelling framework for improved design and decision-making in drug development

The development of a new drug is an extremely high-risk enterprise. The attrition rates of development projects and the average costs for each launched product are daunting, and the completion of a development program requires a very long time horizon. These facts imply that there are huge potential gains, should one be able to improve efficiency and enhance decision-making capabilities. In this paper, we argue that substantial gains can be achieved by adapting a holistic view of drug development. Historically, too much planning, design and decision-making in the pharmaceutical development has been based on locally optimising separate parts of the development program, and too often important sources of uncertainty are ignored. We propose instead a model-based approach built on two essential pillars; (1) an integrated holistic view of the development program, including post-launch marketing and sales, with all parts evaluated simultaneously; (2) an explicit appreciation of all relevant sources of uncertainty. Computer simulations are utilised to evaluate the properties of the program options at hand, and to provide valuable quantitative decision support. Applications of this modelling approach have proven to add large value to development projects in terms of better program options being generated and more value-adding decisions taken.

Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

BACKGROUND

IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers.

METHOD

The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials.

RESULTS

The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials.

CONCLUSION

IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.