Optimal Cost-Effective Designs of Phase II Proof of Concept Trials and Associated Go–No Go Decisions

Aiding the Adoption of Master Protocols by Optimizing Patient Engagement

Master protocols (MPs) are an important addition to the clinical trial repertoire. As defined by the U.S. Food and Drug Administration (FDA), this term means "a protocol designed with multiple sub-studies, which may have different objectives (goals) and involve coordinated efforts to evaluate one or more investigational drugs in one or more disease subtypes within the overall trial structure." This means we now have a unique, scientifically based MP that describes how a clinical trial will be conducted using one or more potential candidate therapies to treat patients in one or more diseases. Patient engagement (PE) is also a critical factor that has been recognized by FDA through its Patient-Focused Drug Development (PFDD) initiative, and by the European Medicines Agency (EMA), which states on its website that it has been actively interacting with patients since the creation of the Agency in 1995. We propose that utilizing these PE principles in MPs can make them more successful for sponsors, providers, and patients. Potential benefits of MPs for patients awaiting treatment can include treatments that better fit a patient's needs; availability of more treatments; and faster access to treatments. These make it possible to develop innovative therapies (especially for rare diseases and/or unique subpopulations, e.g., pediatrics), to minimize untoward side effects through careful dose escalation practices and, by sharing a control arm, to lower the probability of being assigned to a placebo arm for clinical trial participants. This paper is authored by select members of the American Statistical Association (ASA)/DahShu Master Protocol Working Group (MPWG) People and Patient Engagement (PE) Subteam. DahShu is a 501(c)(3) non-profit organization, founded to promote research and education in data science. This manuscript does not include direct feedback from US or non-US regulators, though multiple regulatory-related references are cited to confirm our observation that improving patient engagement is supported by regulators. This manuscript represents the authors' independent perspective on the Master Protocol; it does not represent the official policy or viewpoint of FDA or any other regulatory organization or the views of the authors' employers. The objective of this manuscript is to provide drug developers, contract research organizations (CROs), third party capital investors, patient advocacy groups (PAGs), and biopharmaceutical executives with a better understanding of how including the patient voice throughout MP development and conduct creates more efficient clinical trials. The PE Subteam also plans to publish a Plain Language Summary (PLS) of this publication for clinical trial participants, patients, caregivers, and the public as they seek to understand the risks and benefits of MP clinical trial participation.

Bayesian Complex Innovative Trial Designs (CIDs) and Their Use in Drug Development for Rare Disease

As the temporal, financial, and ethical cost of randomized clinical trials (RCTs) continues to rise, researchers and regulators in drug discovery and development face increasing pressure to make better use of existing data sources. This pressure is especially high in rare disease, where traditionally designed RCTs are often infeasible due to the inability to recruit enough patients or the unwillingness of patients or trial leaders to randomly assign anyone to placebo. Bayesian statistical methods have recently been recommended in such settings for their ability to combine disparate data sources, increasing overall study power. The use of these methods has received a boost in the United States thanks to a new willingness by regulators at the Food and Drug Administration to consider complex innovative trial designs. These designs allow trialists to change the nature of the trial (eg, stop early for success or futility, drop an underperforming trial arm, incorporate data on historical controls, etc) while it is still running. In this article, we review a broad collection of Bayesian techniques useful in rare disease research, indicating the benefits and risks associated with each. We begin with relatively innocuous methods for combining information from RCTs and proceed on through increasingly innovative approaches that borrow strength from increasingly heterogeneous and less carefully curated data sources. We also offer 2 examples from the very recent literature illustrating how clinical pharmacology principles can make important contributions to such designs, confirming the interdisciplinary nature of this work.

Innovations in Clinical Development in Rare Diseases of Children and Adults: Small Populations and/or Small Patients

Many of the afflictions of children are rare diseases. This creates numerous drug development challenges related to small populations, including limited information about the disease state, enrollment challenges, and diminished incentives for pediatric development of novel therapies by pharmaceutical and biotechnology sponsors. We review selected innovations in clinical development that may partially mitigate some of these difficulties, starting with the concept of development efficiency for individual clinical trials, clinical programs (involving multiple trials for a single drug), and clinical portfolios of multiple drugs, and decision analysis as a tool to optimize efficiency. Development efficiency is defined as the ability to reach equally rigorous or more rigorous conclusions in less time, with fewer trial participants, or with fewer resources. We go on to discuss efficient methods for matching targeted therapies to biomarker-defined subgroups, methods for eliminating or reducing the need for natural history data to guide rare disease development, the use of basket trials to enhance efficiency by grouping multiple similar disease applications in a single clinical trial, and the use of alternative data sources including historical controls to augment or replace concurrent controls in clinical studies. Greater understanding and broader application of these methods could lead to improved therapies and/or more widespread and rapid access to novel therapies for rare diseases in both children and adults.

A digital intervention targeting cognitive control network dysfunction in middle age and older adults with major depression

Nonpharmacological interventions targeting putative network mechanisms of major depressive disorder (MDD) may represent novel treatments. This mechanistic study investigates how a video game-like intervention, designed to improve cognitive control network (CCN) functioning by targeting multitasking, influences the CCN of middle-aged and older adults with MDD. The sample consisted of 34 adults aged 45-75 with SCID-defined diagnosis of MDD, Hamilton depression rating scale scores ≥20, and a deficit in cognitive control. Participants were instructed to play at home for 20-25 min per day, at least 5 times per week, for 4 weeks. Evidence of target engagement was defined a priori as >2/3 of participants showing CCN improvement. CCN engagement was defined as a change in a Z score of ≥0.5 on functional magnetic resonance imaging (fMRI) in activation and functional connectivity of the CCN during task-based and resting-state fMRI, respectively. 74% of participants showed a change in activation of the CCN, and 72% showed an increase in resting-state functional connectivity. Sixty-eight percent demonstrated improved cognitive control function, measured as either improvement on sustained attention or working memory performance or reduced self-reported symptoms of apathy on the frontal systems behavioral scale (FrsBe). Participants also reported a significant reduction in mood symptoms measured by PHQ-9. A remotely deployed neuroscience-informed video game-like intervention improves both CCN functions and mood in middle-aged and older adults with MDD. This easily-disseminated intervention may rescue CCN dysfunction present in a substantial subset of middle-aged and older adults with MDD.

Friction and Decision Rules in Portfolio Decision Analysis

In portfolio decision analysis, features comprise the objectives, alternatives, physics, and information that define a decision context. By modeling features, decision analysts forecast the expected utilities of the alternatives. A model is complete if it contains all the features. A model is well-calibrated if it correctly predicts the probability distributions of each alternative’s utility, whereas ill-calibrated models, like those that suffer the optimizer’s curse, do not. Friction identifies qualities of a situation that prevent decision analysts from creating complete, well-calibrated models. When friction is significant, can maximizing expected utility be a suboptimal decision rule? Is satisfying decision theory’s axioms a necessary or sufficient condition for good decision making? Can rules that violate the axioms outperform rules that satisfy them? A simulation study of how unbiased, imprecise forecasts of payoffs affect project selection finds that, for the example tested, the answers are yes, no, and yes, which suggests that further studies of friction may be worthwhile. Discussions of friction bookend the study, starting the paper by defining friction and concluding by presenting three frameworks, each one from a different field of study, that provide mathematical tools for studying friction.

Efficient two-stage sequential arrays of proof of concept studies for pharmaceutical portfolios

Previous work has shown that individual randomized "proof-of-concept" (PoC) studies may be designed to maximize cost-effectiveness, subject to an overall PoC budget constraint. Maximizing cost-effectiveness has also been considered for arrays of simultaneously executed PoC studies. Defining Type III error as the opportunity cost of not performing a PoC study, we evaluate the common pharmaceutical practice of allocating PoC study funds in two stages. Stage 1, or the first wave of PoC studies, screens drugs to identify those to be permitted additional PoC studies in Stage 2. We investigate if this strategy significantly improves efficiency, despite slowing development. We quantify the benefit, cost, benefit-cost ratio, and Type III error given the number of Stage 1 PoC studies. Relative to a single stage PoC strategy, significant cost-effective gains are seen when at least one of the drugs has a low probability of success (10%) and especially when there are either few drugs (2) with a large number of indications allowed per drug (10) or a large portfolio of drugs (4). In these cases, the recommended number of Stage 1 PoC studies ranges from 2 to 4, tracking approximately with an inflection point in the minimization curve of Type III error.

Decision-theoretic designs for a series of trials with correlated treatment effects using the Sarmanov multivariate beta-binomial distribution

The motivation for the work in this article is the setting in which a number of treatments are available for evaluation in phase II clinical trials and where it may be infeasible to try them concurrently because the intended population is small. This paper introduces an extension of previous work on decision-theoretic designs for a series of phase II trials. The program encompasses a series of sequential phase II trials with interim decision making and a single two-arm phase III trial. The design is based on a hybrid approach where the final analysis of the phase III data is based on a classical frequentist hypothesis test, whereas the trials are designed using a Bayesian decision-theoretic approach in which the unknown treatment effect is assumed to follow a known prior distribution. In addition, as treatments are intended for the same population it is not unrealistic to consider treatment effects to be correlated. Thus, the prior distribution will reflect this. Data from a randomized trial of severe arthritis of the hip are used to test the application of the design. We show that the design on average requires fewer patients in phase II than when the correlation is ignored. Correspondingly, the time required to recommend an efficacious treatment for phase III is quicker.

PIPELINEs: Creating Comparable Clinical Knowledge Efficiently by Linking Trial Platforms

Adaptive, seamless, multisponsor, multitherapy clinical trial designs executed as large scale platforms, could create superior evidence more efficiently than single-sponsor, single-drug trials. These trial PIPELINEs also could diminish barriers to trial participation, increase the representation of real-world populations, and create systematic evidence development for learning throughout a therapeutic life cycle, to continually refine its use. Comparable evidence could arise from multiarm design, shared comparator arms, and standardized endpoints-aiding sponsors in demonstrating the distinct value of their innovative medicines; facilitating providers and patients in selecting the most appropriate treatments; assisting regulators in efficacy and safety determinations; helping payers make coverage and reimbursement decisions; and spurring scientists with translational insights. Reduced trial times and costs could enable more indications, reduced development cycle times, and improved system financial sustainability. Challenges to overcome range from statistical to operational to collaborative governance and data exchange.

The Impact of Adaptive Design on Portfolio Optimization

BACKGROUND

Pharmaceutical portfolios are optimized by improved allocation of a fixed budget into individual trials that leads to an improved value of a portfolio. This paper investigates how flexibility of adaptive design contributes to portfolio optimization.

METHODS

An example portfolio was designed, and strategies that did or did not include trials with adaptive designs were specified. Operating characteristics of a traditional portfolio were compared to that of an adaptive portfolio. Adaptive portfolios offer potential advantages over traditional ones. Its flexibility largely increases the number of decision points, and as such it allows for a much more frequent reassessment of portfolios. Additionally, an adaptive portfolio can correct itself if initial decisions were made incorrectly.

RESULTS

Despite all these advantages, the adaptive portfolio did not outperform the traditional portfolio. The main reason is that in this case, adaptive designs allowed for increases in sample size to the point where improvements per unit increase were minimal, instead of allocating this budget to additional trials.

CONCLUSIONS

It is critical to minimize missed opportunities to initiate new promising trials, and to increase sample size only in regions that promise meaningful improvements in power.

Translating predictive biomarkers within oncology clinical development programs

Predictive biomarkers provide essential information to enable personalized medicine, and hold the promise for enhancing the effectiveness and value of cancer therapies. However, they do not always work. This review provides a framework for managing the risk of predictive biomarkers and maximally harvesting their benefit. Methods are provided which permit data-driven, adaptive decision making about the use of predictive biomarkers during clinical development, applying them to the extent they are validated by the clinical data. Techniques for optimizing overall development efficiency, measured as the number of successful drug indications approved per patient utilized, are also presented.

Decision-theoretic designs for small trials and pilot studies: A review

Pilot studies and other small clinical trials are often conducted but serve a variety of purposes and there is little consensus on their design. One paradigm that has been suggested for the design of such studies is Bayesian decision theory. In this article, we review the literature with the aim of summarizing current methodological developments in this area. We find that decision-theoretic methods have been applied to the design of small clinical trials in a number of areas. We divide our discussion of published methods into those for trials conducted in a single stage, those for multi-stage trials in which decisions are made through the course of the trial at a number of interim analyses, and those that attempt to design a series of clinical trials or a drug development programme. In all three cases, a number of methods have been proposed, depending on the decision maker’s perspective being considered and the details of utility functions that are used to construct the optimal design.

Emerging drugs for neuropathic pain

INTRODUCTION

Neuropathic pain is a costly and disabling condition, which affects up to 8% of the population. Available therapies often provide incomplete pain relief and treatment-related side effects are common. Preclinical neuropathic pain models have facilitated identification of several promising targets, which have progressed to human clinical phases of evaluation.

AREAS COVERED

A systematic database search yielded 25 new molecular entities with specified pharmacological mechanisms that have reached Phase II or III clinical trials. These include calcium channel antagonists, vanilloid receptor antagonists, potassium channel agonists, NMDA antagonists, novel opioid receptor agonists, histamine H3 receptor antagonists, a novel sodium channel antagonist, serotonin modulators, a novel acetylcholine receptor agonist, α-2b adrenoreceptor agonist, cannabinoid CB2 receptor agonist, nitric oxide synthase inhibitor, orexin receptor antagonist, angiotensin II 2 antagonist, imidazoline I2 receptor agonist, apoptosis inhibitor and fatty acid amide hydrolase inhibitor.

EXPERT OPINION

Although the diversity of pharmacological mechanisms of interest emphasise the complexity of neuropathic pain transmission, the considerable number of agents under development reflect a continued enthusiasm in drug development for neuropathic pain. Ongoing enhancements in methodology of both preclinical and clinical research and closer translation in both directions are expected to more efficiently identify new agents, which will improve the management of neuropathic pain.

Maximizing return on socioeconomic investment in phase II proof-of-concept trials

Phase II proof-of-concept (POC) trials play a key role in oncology drug development, determining which therapeutic hypotheses will undergo definitive phase III testing according to predefined Go-No Go (GNG) criteria. The number of possible POC hypotheses likely far exceeds available public or private resources. We propose a design strategy for maximizing return on socioeconomic investment in phase II trials that obtains the greatest knowledge with the minimum patient exposure. We compare efficiency using the benefit-cost ratio, defined to be the risk-adjusted number of truly active drugs correctly identified for phase III development divided by the risk-adjusted total sample size in phase II and III development, for different POC trial sizes, powering schemes, and associated GNG criteria. It is most cost-effective to conduct small POC trials and set the corresponding GNG bars high, so that more POC trials can be conducted under socioeconomic constraints. If δ is the minimum treatment effect size of clinical interest in phase II, the study design with the highest benefit-cost ratio has approximately 5% type I error rate and approximately 20% type II error rate (80% power) for detecting an effect size of approximately 1.5δ. A Go decision to phase III is made when the observed effect size is close to δ. With the phenomenal expansion of our knowledge in molecular biology leading to an unprecedented number of new oncology drug targets, conducting more small POC trials and setting high GNG bars maximize the return on socioeconomic investment in phase II POC trials.

An Analysis of Methodologies That Can Be Used to Validate if a Perioperative Surgical Home Improves the Patient-centeredness, Evidence-based Practice, Quality, Safety, and Value of Patient Care

Approximately 80 million inpatient and outpatient surgeries are performed annually in the United States. Widely variable and fragmented perioperative care exposes these surgical patients to lapses in expected standard of care, increases the chance for operational mistakes and accidents, results in unnecessary and potentially detrimental care, needlessly drives up costs, and adversely affects the patient healthcare experience. The American Society of Anesthesiologists and other stakeholders have proposed a more comprehensive model of perioperative care, the Perioperative Surgical Home (PSH), to improve current care of surgical patients and to meet the future demands of increased volume, quality standards, and patient-centered care. To justify implementation of this new healthcare delivery model to surgical colleagues, administrators, and patients and maintain the integrity of evidenced-based practice, the nascent PSH model must be rigorously evaluated. This special article proposes comparative effectiveness research aims or objectives and an optimal study design for the novel PSH model.

Evaluation of early efficacy endpoints for proof-of-concept trials

A Phase II proof-of-concept (POC) trial usually uses an early efficacy endpoint other than a clinical endpoint as the primary endpoint. Because of the advancement in bioscience and technology, which has yielded a number of new surrogate biomarkers, drug developers often have more candidate endpoints to choose from than they can handle. As a result, selection of endpoint and its effect size as well as choice of type I/II error rates are often at the center of heated debates in design of POC trials. While optimization of the trade-off between benefit and cost is the implicit objective in such a decision-making process, it is seldom explicitly accounted for in practice. In this research note, motivated by real examples from the oncology field, we provide practical measures for evaluation of early efficacy endpoints (E4) for POC trials. We further provide optimal design strategies for POC trials that include optimal Go-No Go decision criteria for initiation of Phase III and optimal resource allocation strategies for conducting multiple POC trials in a portfolio under fixed resources. Although oncology is used for illustration purpose, the same idea developed in this research note also applies to similar situations in other therapeutic areas or in early-stage drug development in that a Go-No Go decision has to rely on limited data from an early efficacy endpoint and cost-effectiveness is the main concern.

New evidence-based adaptive clinical trial methods for optimally integrating predictive biomarkers into oncology clinical development programs

Predictive biomarkers are important to the future of oncology; they can be used to identify patient populations who will benefit from therapy, increase the value of cancer medicines, and decrease the size and cost of clinical trials while increasing their chance of success. But predictive biomarkers do not always work. When unsuccessful, they add cost, complexity, and time to drug development. This perspective describes phases 2 and 3 development methods that efficiently and adaptively check the ability of a biomarker to predict clinical outcomes. In the end, the biomarker is emphasized to the extent that it can actually predict.

A mathematical model for maximizing the value of phase 3 drug development portfolios incorporating budget constraints and risk

We describe a value-driven approach to optimizing pharmaceutical portfolios. Our approach incorporates inputs from research and development and commercial functions by simultaneously addressing internal and external factors. This approach differentiates itself from current practices in that it recognizes the impact of study design parameters, sample size in particular, on the portfolio value. We develop an integer programming (IP) model as the basis for Bayesian decision analysis to optimize phase 3 development portfolios using expected net present value as the criterion. We show how this framework can be used to determine optimal sample sizes and trial schedules to maximize the value of a portfolio under budget constraints. We then illustrate the remarkable flexibility of the IP model to answer a variety of 'what-if' questions that reflect situations that arise in practice. We extend the IP model to a stochastic IP model to incorporate uncertainty in the availability of drugs from earlier development phases for phase 3 development in the future. We show how to use stochastic IP to re-optimize the portfolio development strategy over time as new information accumulates and budget changes occur.

Current issues in oncology drug development, with a focus on Phase II trials

In this commentary we discuss several challenges that are of current relevance to the design of clinical trials in oncology. We argue that the compartmentalization of trials into the three standard phases, with non overlapping aims, is not necessary and in fact may slow the clinical development of agents. Combined Phase I/II trials and/or Phase I trials that at minimum collect efficacy data and more optimally include a preliminary measure of efficacy in dosing determination should be more widely utilized. Similarly, we posit that randomized Phase II trials should be used more frequently, as opposed to the traditional historical single arm Phase II trial that usually does not have a valid comparison group. The use of non binary endpoints is a simple modification that can improve the efficiency of early phase trials. The heterogeneity in scientific goals and contexts in early phase oncology trials is considerable, and the potential to improve the design to match these goals is great. We review these and other issues in the context of 5 manuscripts related to Phase II trials published in this volume. Our overall premise is that the potential benefits associated with the oncology clinical trial community moving away from the one size fits all paradigm of trial design are great, and that more flexible and efficient designs tailored to match the goals of each study are currently available and being used successfully.