Advantages of a wholly Bayesian approach to assessing efficacy in early drug development: a case study

Delta Inflation, Optimism Bias, and Uncertainty in Clinical Trials

The phenomenon of delta inflation, in which design treatment effects tend to exceed observed treatment effects, has been documented in several therapeutic areas. Delta inflation has often been attributed to investigators' optimism bias, or an unwarranted belief in the efficacy of new treatments. In contrast, we argue that delta inflation may be a natural consequence of clinical equipoise, that is, genuine uncertainty about the relative benefits of treatments before a trial is initiated. We review alternative methodologies that can offer more direct evidence about investigators' beliefs, including Bayesian priors and forecasting analysis. The available evidence for optimism bias appears to be mixed, and can be assessed only where uncertainty is expressed explicitly at the trial design stage.

Comparison of Quality of Recovery between Modified Thoracoabdominal Nerves Block through Perichondrial Approach versus Oblique Subcostal Transversus Abdominis Plane Block in Patients Undergoing Total Laparoscopic Hysterectomy: A Pilot Randomized Controlled Trial

Modified thoracoabdominal nerves block through a perichondrial approach (M-TAPA) provides a wide analgesic range. Herein, we examined the quality of recovery (QoR) of M-TAPA for total laparoscopic hysterectomy (TLH) compared with oblique subcostal transversus abdominis plane block (OSTAPB) and measured plasma levobupivacaine concentrations (PClevo). Forty female patients undergoing TLH were randomized to each group. Nerve blocks were performed bilaterally with 25 mL of 0.25% levobupivacaine administered per side. The primary outcome was changes in QoR-15 scores on postoperative days (POD) 1 and 2 from the preoperative baseline. The main secondary outcomes were PClevo at 15, 30, 45, 60, and 120 min after performing nerve block. Group differences (M-TAPA-OSTAPB) in mean changes from baseline in QoR-15 scores on POD 1 and 2 were -11.3 (95% confidence interval (CI), -24.9 to 2.4, p = 0.104; standard deviation (SD), 22.8) and -7.0 (95% CI, -20.5 to 6.6, p = 0.307; SD, 18.7), respectively. Changes in PClevo were similar in both groups. The post hoc analysis using Bayesian statistics revealed that posterior probabilities of M-TAPA being clinically more effective than OSTAPB were up to 22.4 and 24.4% for POD 1 and 2, respectively. In conclusion, M-TAPA may not be superior to OSTAPB for TLH.

Concept and application of the probability of pharmacological success (PoPS) as a decision tool in drug development: a position paper

BACKGROUND

In drug development, few molecules from a large pool of early candidates become successful medicines after demonstrating a favourable benefit-risk ratio. Many decisions are made along the way to continue or stop the development of a molecule. The probability of pharmacological success, or PoPS, is a tool for informing early-stage decisions based on benefit and risk data available at the time.

RESULTS

The PoPS is the probability that most patients can achieve adequate pharmacology for the intended indication while minimising the number of subjects exposed to safety risk. This probability is usually a function of dose; hence its computation typically requires exposure-response models for pharmacology and safety. The levels of adequate pharmacology and acceptable risk must be specified. The uncertainties in these levels, in the exposure-response relationships, and in relevant translation all need to be identified. Several examples of different indications are used to illustrate how this approach can facilitate molecule progression decisions for preclinical and early clinical development. The examples show that PoPS assessment is an effective mechanism for integrating multi-source data, identifying knowledge gaps, and forcing transparency of assumptions. With its application, translational modelling becomes more meaningful and dose prediction more rigorous. Its successful implementation calls for early planning, sound understanding of the disease-drug system, and cross-discipline collaboration. Furthermore, the PoPS evolves as relevant knowledge grows.

CONCLUSION

The PoPS is a powerful evidence-based framework to formally capture multiple uncertainties into a single probability term for assessing benefit-risk ratio. In GSK, it is now expected for governance review at all early-phase decision gates.

A Bayesian three-tier quantitative decision-making framework for single arm studies in early phase oncology

In early phase oncology drug development, single arm proof-of-concept (POC) studies are increasingly being used to drive the early decisions for future development of the drug. Decision-makings based on such studies, typically involving small sample size and early surrogate efficacy endpoints, are extremely challenging. In particular, given the tremendous competition in the development of immunotherapies, expedition of the most promising programs is desired. To this end, we have proposed a Bayesian three-tier approach to facilitate the decision-making process, inheriting all the benefits of Bayesian decision-making approaches and formally allowing the option of acceleration. With pre-specified Bayesian decision criteria, three types of decisions regarding the future development of the drug can be made: (1) terminating the program, (2) further investigation, considering totality of evidence or additional POC studies, and (3) accelerating the program. We further proposed a Bayesian adaptive three-tier (BAT) design, extending the decision-making approach to incorporate adaptive thresholds and allow for continuous monitoring of the study. We compare the performance of the proposed methods with some other existing methods through simulations.

Quantitative decision making for investment in global health intervention trials: Case study of the NEWBORN study on emollient therapy in preterm infants in Kenya

Background

Partners from an NGO, academia, industry and government applied a tool originating in the private sector – Quantitative Decision Making (QDM) – to rigorously assess whether to invest in testing a global health intervention. The proposed NEWBORN study was designed to assess whether topical emollient therapy with sunflower seed oil in infants with very low birthweight <1500 g in Kenya would result in a significant reduction in neonatal mortality compared to standard of care.

Methods

The QDM process consisted of prior elicitation, modelling of prior distributions, and simulations to assess Probability of Success (PoS) via assurance calculations. Expert opinion was elicited on the probability that emollient therapy with sunflower seed oil will have any measurable benefit on neonatal mortality based on available evidence. The distribution of effect sizes was modelled and trial data simulated using Statistical Analysis System to obtain the overall assurance which represents the PoS for the planned study. A decision-making framework was then applied to characterise the ability of the study to meet pre-selected decision-making endpoints.

Results

There was a 47% chance of a positive outcome (defined as a significant relative reduction in mortality of ≥15%), a 45% chance of a negative outcome (defined as a significant relative reduction in mortality <10%), and an 8% chance of ending in the consider zone (ie, a mortality reduction of 10 to <15%) for infants <1500 g.

Conclusions

QDM is a novel tool from industry which has utility for prioritisation of investments in global health, complementing existing tools [eg, Child Health and Nutrition Research Initiative]. Results from application of QDM to the NEWBORN study suggests that it has a high probability of producing clear results. Findings encourage future formation of public-private partnerships for health.

A New Comprehensive Approach to Assess the Probability of Success of Development Programs Before Pivotal Trials

The point at which clinical development programs transition from early phase to pivotal trials is a critical milestone. Substantial uncertainty about the outcome of pivotal trials may remain even after seeing positive early phase data, and companies may need to make difficult prioritization decisions for their portfolio. The probability of success (PoS) of a program, a single number expressed as a percentage reflecting the multitude of risks that may influence the final program outcome, is a key decision-making tool. Despite its importance, companies often rely on crude industry benchmarks that may be "adjusted" by experts based on undocumented criteria and which are typically misaligned with the definition of success used to drive commercial forecasts, leading to overly optimistic expected net present value calculations. We developed a new framework to assess the PoS of a program before pivotal trials begin. Our definition of success encompasses the successful outcome of pivotal trials, regulatory approval and meeting the requirements for market access as outlined in the target product profile. The proposed approach is organized in four steps and uses an innovative Bayesian approach to synthesize all relevant evidence. The new PoS framework is systematic and transparent. It will help organizations to make more informed decisions. In this paper, we outline the rationale and elaborate on the structure of the proposed framework, provide examples, and discuss the benefits and challenges associated with its adoption.

Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study

BACKGROUND

Many decisions in health care aim to maximise health, requiring judgements about interventions that may have higher health effects but potentially incur additional costs (cost-effectiveness framework). The evidence used to establish cost-effectiveness is typically uncertain and it is important that this uncertainty is characterised. In situations in which evidence is uncertain, the experience of experts is essential. The process by which the beliefs of experts can be formally collected in a quantitative manner is structured expert elicitation. There is heterogeneity in the existing methodology used in health-care decision-making. A number of guidelines are available for structured expert elicitation; however, it is not clear if any of these are appropriate for health-care decision-making.

OBJECTIVES

The overall aim was to establish a protocol for structured expert elicitation to inform health-care decision-making. The objectives are to (1) provide clarity on methods for collecting and using experts' judgements, (2) consider when alternative methodology may be required in particular contexts, (3) establish preferred approaches for elicitation on a range of parameters, (4) determine which elicitation methods allow experts to express uncertainty and (5) determine the usefulness of the reference protocol developed.

METHODS

A mixed-methods approach was used: systemic review, targeted searches, experimental work and narrative synthesis. A review of the existing guidelines for structured expert elicitation was conducted. This identified the approaches used in existing guidelines (the 'choices') and determined if dominant approaches exist. Targeted review searches were conducted for selection of experts, level of elicitation, fitting and aggregation, assessing accuracy of judgements and heuristics and biases. To sift through the available choices, a set of principles that underpin the use of structured expert elicitation in health-care decision-making was defined using evidence generated from the targeted searches, quantities to elicit experimental evidence and consideration of constraints in health-care decision-making. These principles, including fitness for purpose and reflecting individual expert uncertainty, were applied to the set of choices to establish a reference protocol. An applied evaluation of the developed reference protocol was also undertaken.

RESULTS

For many elements of structured expert elicitation, there was a lack of consistency across the existing guidelines. In almost all choices, there was a lack of empirical evidence supporting recommendations, and in some circumstances the principles are unable to provide sufficient justification for discounting particular choices. It is possible to define reference methods for health technology assessment. These include a focus on gathering experts with substantive skills, eliciting observable quantities and individual elicitation of beliefs. Additional considerations are required for decision-makers outside health technology assessment, for example at a local level, or for early technologies. Access to experts may be limited and in some circumstances group discussion may be needed to generate a distribution.

LIMITATIONS

The major limitation of the work conducted here lies not in the methods employed in the current work but in the evidence available from the wider literature relating to how appropriate particular methodological choices are.

CONCLUSIONS

The reference protocol is flexible in many choices. This may be a useful characteristic, as it is possible to apply this reference protocol across different settings. Further applied studies, which use the choices specified in this reference protocol, are required.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 37. See the NIHR Journals Library website for further project information. This work was also funded by the Medical Research Council (reference MR/N028511/1).

Conditional assurance: the answer to the questions that should be asked within drug development

In this paper, we extend the use of assurance for a single study to explore how meeting a study's pre-defined success criteria could update our beliefs about the true treatment effect and impact the assurance of subsequent studies. This concept of conditional assurance, the assurance of a subsequent study conditional on success in an initial study, can be used assess the de-risking potential of the study requiring immediate investment, to ensure it provides value within the overall development plan. If the planned study does not discharge sufficient later phase risk, alternative designs and/or success criteria should be explored. By transparently laying out the different design options and the risks associated, this allows for decision makers to make quantitative investment choices based on their risk tolerance levels and potential return on investment. This paper lays out the derivation of conditional assurance, discusses how changing the design of a planned study will impact the conditional assurance of a future study, as well as presenting a simple illustrative example of how this methodology could be used to transparently compare development plans to aid decision making within an organisation.

Bayesian design and analysis of external pilot trials for complex interventions

External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention, and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre-specified thresholds known as progression criteria, although the statistical properties of such decision rules are rarely assessed. Such assessment is complicated by several methodological challenges, including the simultaneous evaluation of multiple endpoints, complex multi-level models, small sample sizes, and uncertainty in nuisance parameters. In response to these challenges, we describe a Bayesian approach to the design and analysis of external pilot trials. We show how progression decisions can be made by minimizing the expected value of a loss function, defined over the whole parameter space to allow for preferences and trade-offs between multiple parameters to be articulated and used in the decision-making process. The assessment of preferences is kept feasible by using a piecewise constant parametrization of the loss function, the parameters of which are chosen at the design stage to lead to desirable operating characteristics. We describe a flexible, yet computationally intensive, nested Monte Carlo algorithm for estimating operating characteristics. The method is used to revisit the design of an external pilot trial of a complex intervention designed to increase the physical activity of care home residents.

Optimising the trade-off between type I and II error rates in the Bayesian context

For any decision-making study, there are two sorts of errors that can be made, declaring a positive result when the truth is negative, and declaring a negative result when the truth is positive. Traditionally, the primary analysis of a study is a two-sided hypothesis test, the type I error rate will be set to 5% and the study is designed to give suitably low type II error - typically 10 or 20% - to detect a given effect size. These values are standard, arbitrary and, other than the choice between 10 and 20%, do not reflect the context of the study, such as the relative costs of making type I and II errors and the prior belief the drug will be placebo-like. Several authors have challenged this paradigm, typically for the scenario where the planned analysis is frequentist. When resource is limited, there will always be a trade-off between the type I and II error rates, and this article explores optimising this trade-off for a study with a planned Bayesian statistical analysis. This work provides a scientific basis for a discussion between stakeholders as to what type I and II error rates may be appropriate and some algebraic results for normally distributed data.

Implementing Historical Controls in Oncology Trials

Drug development in oncology has broadened from mainly considering randomized clinical trials to also including single-arm trials tailored for very specific subtypes of cancer. They often use historical controls, and this article discusses benefits and risks of this paradigm and provide various regulatory and statistical considerations. While leveraging the information brought by historical controls could potentially shorten development time and reduce the number of patients enrolled, a careful selection of the past studies, a prespecified statistical analysis accounting for the heterogeneity between studies, and early engagement with regulators will be key to success. Although both the European Medicines Agency and the U.S. Food and Drug Administration have already approved medicines based on nonrandomized experiments, the evidentiary package can be perceived as less comprehensive than randomized experiments. Use of historical controls, therefore, is better suited for cases of high unmet clinical need, where the disease course is well characterized and the primary endpoint is objective. IMPLICATIONS FOR PRACTICE: Incorporating historical data in single-arm oncology trials has the potential to accelerate drug development and to reduce the number of patients enrolled, compared with standard randomized controlled clinical trials. Given the lack of blinding and randomization, such an approach is better suited for cases of high unmet clinical need and/or difficult experimental situations, in which the trajectory of the disease is well characterized and the endpoint can be measured objectively. Careful pre-specification and selection of the historical data, matching of the patient characteristics with the concurrent trial data, and innovative statistical methodologies accounting for between-study variation will be needed. Early engagement with regulators (e.g., via Scientific Advice) is highly recommended.

Current status of Bayesian clinical trials for oncology, 2020

Bayesian methods had established a foothold in developing therapies in oncology trials. Methods: We identified clinical trials posted on the ClinicalTrials.gov database focused on Oncology trials with a Bayesian approach in their design. Differences in study characteristics such as design, study phase, randomization, masking, purpose of study, main outcomes, gender, age and funding involvement according to Bayesian approach were assessed using Chi-squared or Fisher's exact tests. Results: We identified 225 studies with Bayesian components in their design addressing oncological diseases. The most common designs were Bayesian Toxicity Monitoring (26.4%), Model-based designs (36%) Model-assisted designs (8%). Statistical methods such as Bayesian logistic regression model (59.4%), Bayesian piecewise exponential survival regression (10.9%) and the Continual reassessment method (9.4%) were the most used. Conclusions: Bayesian trials are more common in the early phases of drug development specifically in phase II trials (43.6%). Cancer institutes or Hospitals funded most of the studies retrieved. This type of design has increased over time and represent an innovative means of increasing trial efficiency.

Leveraging historical data into oncology development programs: Two case studies of phase 2 Bayesian augmented control trial designs

Leveraging historical data into the design and analysis of phase 2 randomized controlled trials can improve efficiency of drug development programs. Such approaches can reduce sample size without loss of power. Potential issues arise when the current control arm is inconsistent with historical data, which may lead to biased estimates of treatment efficacy, loss of power, or inflated type 1 error. Consideration as to how to borrow historical information is important, and in particular, adjustment for prognostic factors should be considered. This paper will illustrate two motivating case studies of oncology Bayesian augmented control (BAC) trials. In the first example, a glioblastoma study, an informative prior was used for the control arm hazard rate. Sample size savings were 15% to 20% by using a BAC design. In the second example, a pancreatic cancer study, a hierarchical model borrowing method was used, which enabled the extent of borrowing to be determined by consistency of observed study data with historical studies. Supporting Bayesian analyses also adjusted for prognostic factors. Incorporating historical data via Bayesian trial design can provide sample size savings, reduce study duration, and enable a more scientific approach to development of novel therapies by avoiding excess recruitment to a control arm. Various sensitivity analyses are necessary to interpret results. Current industry efforts for data transparency have meaningful implications for access to patient-level historical data, which, while not critical, is helpful to adjust for potential imbalances in prognostic factors.

Clustered allocation as a way of understanding historical controls: Components of variation and regulatory considerations

There has been increasing interest in recent years in the possibility of increasing the efficiency of clinical trials by using historical controls. There has been a general recognition that in replacing concurrent by historical controls, the potential for bias is serious and requires some down-weighting to the apparent amount of historical information available. However, such approaches have generally assumed that what is required is some modification to the standard inferential model offered by the parallel group trial. In our opinion, the correct starting point that requires modification is a trial in which treatments are allocated to clusters. This immediately shows that the amount of information available is governed not just by the number of historical patients but also by the number of centres and of historical studies. Furthermore, once one accepts that external patients may be used as controls, this raises the issue as to which patients should be used. Thus, abandoning concurrent control has implications for many aspects of design and analysis of trials, including (a) identification, pre-specification and agreement on a suitable historical dataset; (b) an agreed, enforceable and checkable plan for recruiting the experimental arm; (c) a finalised analysis plan prior to beginning the trial and (d) use of a hierarchical model with sufficient complexity. We discuss these issues and suggest approaches to design and analysis making extensive reference to the partially randomised Therapeutic Arthritis Research and Gastrointestinal Event Trial study. We also compare some Bayesian and frequentist approaches and provide some important regulatory considerations. We conclude that effective use of historical data will require considerable circumspection and discipline.

A tutorial on Bayes Factor Design Analysis using an informed prior

Well-designed experiments are likely to yield compelling evidence with efficient sample sizes. Bayes Factor Design Analysis (BFDA) is a recently developed methodology that allows researchers to balance the informativeness and efficiency of their experiment (Schönbrodt & Wagenmakers, Psychonomic Bulletin & Review, 25(1), 128-142 2018). With BFDA, researchers can control the rate of misleading evidence but, in addition, they can plan for a target strength of evidence. BFDA can be applied to fixed-N and sequential designs. In this tutorial paper, we provide an introduction to BFDA and analyze how the use of informed prior distributions affects the results of the BFDA. We also present a user-friendly web-based BFDA application that allows researchers to conduct BFDAs with ease. Two practical examples highlight how researchers can use a BFDA to plan for informative and efficient research designs.

Using the Data Agreement Criterion to Rank Experts' Beliefs

Experts’ beliefs embody a present state of knowledge. It is desirable to take this knowledge into account when making decisions. However, ranking experts based on the merit of their beliefs is a difficult task. In this paper, we show how experts can be ranked based on their knowledge and their level of (un)certainty. By letting experts specify their knowledge in the form of a probability distribution, we can assess how accurately they can predict new data, and how appropriate their level of (un)certainty is. The expert’s specified probability distribution can be seen as a prior in a Bayesian statistical setting. We evaluate these priors by extending an existing prior-data (dis)agreement measure, the Data Agreement Criterion, and compare this approach to using Bayes factors to assess prior specification. We compare experts with each other and the data to evaluate their appropriateness. Using this method, new research questions can be asked and answered, for instance: Which expert predicts the new data best? Is there agreement between my experts and the data? Which experts’ representation is more valid or useful? Can we reach convergence between expert judgement and data? We provided an empirical example ranking (regional) directors of a large financial institution based on their predictions of turnover.

Effect of PF-04634817, an Oral CCR2/5 Chemokine Receptor Antagonist, on Albuminuria in Adults with Overt Diabetic Nephropathy

Introduction

Inflammatory cell recruitment, which is potentially mediated by the monocyte chemoattractant protein 1/C-C chemokine receptor type 2 (CCR2) system and by C-C chemokine receptor type 5 (CCR5) activity, may play a role in the development and progression of diabetic nephropathy. PF-04634817 is a dual chemokine CCR2/5 receptor antagonist that is being developed for the treatment of diabetic nephropathy.

Methods

We evaluated the efficacy of PF-04634817 compared with matching placebo for reduction of albuminuria after 12 weeks of treatment in subjects with type 2 diabetes who received standard of care (SOC; angiotensin-converting enzyme inhibitor or angiotensin receptor blocker therapy), in a randomized, double-blind, placebo-controlled, parallel-group phase 2 study.

Results

A total of 226 subjects who received SOC with baseline estimated glomerular filtration rates between 20 and 75 ml/min per 1.73 m² and a baseline urinary albumin-to-creatinine ratio (UACR) of ≥300 mg/g were randomly assigned 3:1 to receive PF-04634817 (150 or 200 mg orally, once daily) or placebo. The primary analysis was Bayesian, with an informative prior for placebo response (equivalent to including an additional 80 subjects in the placebo arm). We observed a placebo-adjusted reduction in UACR of 8.2% (ratio 0.918; 95% credible interval: 0.75–1.09) at week 12 in the PF-04634817 arm. PF-04634817 appeared to be safe and well-tolerated.

Conclusion

Despite the good safety profile shown by PF-04634817, clinical development for this indication was discontinued in light of the modest efficacy observed.

Quantitative decision-making in randomized Phase II studies with a time-to-event endpoint

One of the most critical decision points in clinical development is Go/No-Go decision-making after a proof-of-concept study. Traditional decision-making relies on a formal hypothesis testing with control of type I and type II error rates, which is limited by assessing the strength of efficacy evidence in a small isolated trial. In this article, we propose a quantitative Bayesian/frequentist decision framework for Go/No-Go criteria and sample size evaluation in Phase II randomized studies with a time-to-event endpoint. By taking the uncertainty of treatment effect into consideration, we propose an integrated quantitative approach for a program when both the Phase II and Phase III trials share a common endpoint while allowing a discount of the observed Phase II data. Our results confirm the argument that an increase in the sample size of a Phase II trial will result in greater increase in the probability of success of a Phase III trial than increasing the Phase III trial sample size by equal amount. We illustrate the steps in quantitative decision-making with a real example of a randomized Phase II study in metastatic pancreatic cancer.

First-in-man-proof of concept study with molidustat: a novel selective oral HIF-prolyl hydroxylase inhibitor for the treatment of renal anaemia

AIMS

Insufficient erythropoietin (EPO) synthesis is a relevant cause of renal anaemia in patients with chronic kidney disease. Molidustat, a selective hypoxia-inducible factor prolyl hydroxylase (HIF-PH) inhibitor, increases endogenous EPO levels dose dependently in preclinical models. We examined the pharmacokinetics, safety, tolerability and effect on EPO levels of single oral doses of molidustat in healthy male volunteers.

METHODS

This was a single-centre, randomized, single-blind, placebo-controlled, group-comparison, dose-escalation study. Molidustat was administered at doses of 5, 12.5, 25, 37.5 or 50 mg as a polyethylene glycol-based solution.

RESULTS

In total, 45 volunteers received molidustat and 14 received placebo. Molidustat was absorbed rapidly, and the mean maximum plasma concentration and area under the concentration-time curve increased dose dependently. The mean terminal half-life was 4.64-10.40 h. A significant increase in endogenous EPO was observed following single oral doses of molidustat of 12.5 mg and above. Geometric mean peak EPO levels were 14.8 IU l^-1 (90% confidence interval 13.0, 16.9) for volunteers who received placebo and 39.8 IU l^-1 (90% confidence interval: 29.4, 53.8) for those who received molidustat 50 mg. The time course of EPO levels resembled the normal diurnal variation in EPO. Maximum EPO levels were observed approximately 12 h postdose and returned to baseline after approximately 24-48 h. All doses of molidustat were well tolerated and there were no significant changes in vital signs or laboratory safety parameters.

CONCLUSIONS

Oral administration of molidustat to healthy volunteers elicited a dose-dependent increase in endogenous EPO. These results support the ongoing development of molidustat as a potential new treatment for patients with renal anaemia.

Practical experiences of adopting assurance as a quantitative framework to support decision making in drug development

All clinical trials are designed for success of their primary objectives. Hence, evaluating the probability of success (PoS) should be a key focus at the design stage both to support funding approval from sponsor governance boards and to inform trial design itself. Use of assurance-that is, expected success probability averaged over a prior probability distribution for the treatment effect-to quantify PoS of a planned study has grown across the industry in recent years, and has now become routine within the authors' company. In this paper, we illustrate some of the benefits of systematically adopting assurance as a quantitative framework to support decision making in drug development through several case-studies where evaluation of assurance has proved impactful in terms of trial design and in supporting governance-board reviews of project proposals. In addition, we describe specific features of how the assurance framework has been implemented within our company, highlighting the critical role that prior elicitation plays in this process, and illustrating how the overall assurance calculation may be decomposed into a sequence of conditional PoS estimates which can provide greater insight into how and when different development options are able to discharge risk.

Better decision making in drug development through adoption of formal prior elicitation

With the continued increase in the use of Bayesian methods in drug development, there is a need for statisticians to have tools to develop robust and defensible informative prior distributions. Whilst relevant empirical data should, where possible, provide the basis for such priors, it is often the case that limitations in data and/or our understanding may preclude direct construction of a data-based prior. Formal expert elicitation methods are a key technique that can be used to determine priors in these situations. Within GlaxoSmithKline, we have adopted a structured approach to prior elicitation on the basis of the SHELF elicitation framework and routinely use this in conjunction with calculation of probability of success (assurance) of the next study(s) to inform internal decision making at key project milestones. The aim of this paper is to share our experiences of embedding the use of prior elicitation within a large pharmaceutical company, highlighting both the benefits and challenges of prior elicitation through a series of case studies. We have found that putting team beliefs into the shape of a quantitative probability distribution provides a firm anchor for all internal decision making, enabling teams to provide investment boards with formally appropriate estimates of the probability of trial success as well as robust plans for interim decision rules where appropriate. As an added benefit, the elicitation process provides transparency about the beliefs and risks of the potential medicine, ultimately enabling better portfolio and company-wide decision making.

Integrating dose estimation into a decision-making framework for model-based drug development

Model-informed drug discovery and development offers the promise of more efficient clinical development, with increased productivity and reduced cost through scientific decision making and risk management. Go/no-go development decisions in the pharmaceutical industry are often driven by effect size estimates, with the goal of meeting commercially generated target profiles. Sufficient efficacy is critical for eventual success, but the decision to advance development phase is also dependent on adequate knowledge of appropriate dose and dose-response. Doses which are too high or low pose risk of clinical or commercial failure. This paper addresses this issue and continues the evolution of formal decision frameworks in drug development. Here, we consider the integration of both efficacy and dose-response estimation accuracy into the go/no-go decision process, using a model-based approach. Using prespecified target and lower reference values associated with both efficacy and dose accuracy, we build a decision framework to more completely characterize development risk. Given the limited knowledge of dose response in early development, our approach incorporates a set of dose-response models and uses model averaging. The approach and its operating characteristics are illustrated through simulation. Finally, we demonstrate the decision approach on a post hoc analysis of the phase 2 data for naloxegol (a drug approved for opioid-induced constipation).

Bayesian Phase II optimization for time-to-event data based on historical information

After exploratory drug development, companies face the decision whether to initiate confirmatory trials based on limited efficacy information. This proof-of-concept decision is typically performed after a Phase II trial studying a novel treatment versus either placebo or an active comparator. The article aims to optimize the design of such a proof-of-concept trial with respect to decision making. We incorporate historical information and develop pre-specified decision criteria accounting for the uncertainty of the observed treatment effect. We optimize these criteria based on sensitivity and specificity, given the historical information. Specifically, time-to-event data are considered in a randomized 2-arm trial with additional prior information on the control treatment. The proof-of-concept criterion uses treatment effect size, rather than significance. Criteria are defined on the posterior distribution of the hazard ratio given the Phase II data and the historical control information. Event times are exponentially modeled within groups, allowing for group-specific conjugate prior-to-posterior calculation. While a non-informative prior is placed on the investigational treatment, the control prior is constructed via the meta-analytic-predictive approach. The design parameters including sample size and allocation ratio are then optimized, maximizing the probability of taking the right decision. The approach is illustrated with an example in lung cancer.

Phosphodiesterase Type 5 Inhibition Reduces Albuminuria in Subjects with Overt Diabetic Nephropathy

Diabetic nephropathy (DN) is the leading cause of ESRD worldwide. Reduced bioavailability or uncoupling of nitric oxide in the kidney, leading to decreased intracellular levels of the nitric oxide pathway effector molecule cyclic guanosine monophosphate (cGMP), has been implicated in the progression of DN. Preclinical studies suggest that elevating the cGMP intracellular pool through inhibition of the cGMP-hydrolyzing enzyme phosphodiesterase type 5 (PDE5) might exert renoprotective effects in DN. To test this hypothesis, the novel, highly specific, and long-acting PDE5 inhibitor, PF-00489791, was assessed in a multinational, multicenter, randomized, double-blind, placebo-controlled, parallel group trial of subjects with type 2 diabetes mellitus and overt nephropathy receiving angiotensin converting enzyme inhibitor or angiotensin receptor blocker background therapy. In total, 256 subjects with an eGFR between 25 and 60 ml/min per 1.73 m² and macroalbuminuria defined by a urinary albumin-to-creatinine ratio >300 mg/g, were randomly assigned 3:1, respectively, to receive PF-00489791 (20 mg) or placebo orally, once daily for 12 weeks. Using the predefined primary assessment of efficacy (Bayesian analysis with informative prior), we observed a significant reduction in urinary albumin-to-creatinine ratio of 15.7% (ratio 0.843; 95% credible interval 0.73 to 0.98) in response to the 12-week treatment with PF-00489791 compared with placebo. PF-00489791 was safe and generally well tolerated in this patient population. Most common adverse events were mild in severity and included headache and upper gastrointestinal events. In conclusion, the safety and efficacy profile of PDE5 inhibitor PF-00489791 supports further investigation as a novel therapy to improve renal outcomes in DN.

Big Data: transforming drug development and health policy decision making

The explosion of data sources, accompanied by the evolution of technology and analytical techniques, has created considerable challenges and opportunities for drug development and healthcare resource utilization. We present a systematic overview these phenomena, and suggest measures to be taken for effective integration of the new developments in the traditional medical research paradigm and health policy decision making. Special attention is paid to pertinent issues in emerging areas, including rare disease drug development, personalized medicine, Comparative Effectiveness Research, and privacy and confidentiality concerns.

Addressing potential prior-data conflict when using informative priors in proof-of-concept studies

Bayesian methods are increasingly used in proof-of-concept studies. An important benefit of these methods is the potential to use informative priors, thereby reducing sample size. This is particularly relevant for treatment arms where there is a substantial amount of historical information such as placebo and active comparators. One issue with using an informative prior is the possibility of a mismatch between the informative prior and the observed data, referred to as prior-data conflict. We focus on two methods for dealing with this: a testing approach and a mixture prior approach. The testing approach assesses prior-data conflict by comparing the observed data to the prior predictive distribution and resorting to a non-informative prior if prior-data conflict is declared. The mixture prior approach uses a prior with a precise and diffuse component. We assess these approaches for the normal case via simulation and show they have some attractive features as compared with the standard one-component informative prior. For example, when the discrepancy between the prior and the data is sufficiently marked, and intuitively, one feels less certain about the results, both the testing and mixture approaches typically yield wider posterior-credible intervals than when there is no discrepancy. In contrast, when there is no discrepancy, the results of these approaches are typically similar to the standard approach. Whilst for any specific study, the operating characteristics of any selected approach should be assessed and agreed at the design stage; we believe these two approaches are each worthy of consideration.