A behavioral Bayes method to determine the sample size of a clinical trial considering efficacy and safety

Optimal Decision Criteria for the Study Design and Sample Size of a Biomarker-Driven Phase III Trial

BACKGROUND

The design and sample size of a phase III study for new medical technologies were historically determined within the framework of frequentist hypothesis testing. Recently, drug development using predictive biomarkers, which can predict efficacy based on the status of biomarkers, has attracted attention, and various study designs using predictive biomarkers have been suggested. Additionally, when choosing a study design, considering economic factors, such as the risk of development, expected revenue, and cost, is important.

METHODS

Here, we propose a method to determine the optimal phase III design and sample size and judge whether the phase III study will be conducted using the expected net present value (eNPV). The eNPV is defined using the probability of success of the study calculated based on historical data, the revenue that will be obtained after the success of the phase III study, and the cost of the study. Decision procedures of the optimal phase III design and sample size considering historical data obtained up to the start of the phase III study were considered using numerical examples.

RESULTS

Based on the numerical examples, the optimal study design and sample size depend on the mean treatment effect in the biomarker-positive and biomarker-negative populations obtained from historical data, the between-trial variance of response, the prevalence of the biomarker-positive population, and the threshold value of probability of success required to go to phase III study.

CONCLUSIONS

Thus, the design and sample size of a biomarker-driven phase III study can be appropriately determined based on the eNPV.

Bayesian sample size determination for longitudinal studies with continuous response based on different scientific questions of interest

Longitudinal study designs are commonly applied in much scientific research, especially in the medical, social, and economic sciences. Longitudinal studies allow researchers to measure changes in each individual's responses over time and often have higher statistical power than cross-sectional studies. Choosing an appropriate sample size is a crucial step in a successful study. In longitudinal studies, because of the complexity of their design, including the selection of the number of individuals and the number of repeated measurements, sample size determination is less studied. This paper uses a simulation-based method to determine the sample size from a Bayesian perspective. For this purpose, several Bayesian criteria for sample size determination are used, of which the most important one is the Bayesian power criterion. We determine the sample size of a longitudinal study based on the scientific question of interest, by the choice of an appropriate model. Most of the methods of determining sample size are based on the definition of a single hypothesis. In this paper, in addition to using this method, we determine the sample size using multiple hypotheses. Using several examples, the proposed Bayesian methods are illustrated and discussed.

Value of information methods to design a clinical trial in a small population to optimise a health economic utility function

Background

Most confirmatory randomised controlled clinical trials (RCTs) are designed with specified power, usually 80% or 90%, for a hypothesis test conducted at a given significance level, usually 2.5% for a one-sided test. Approval of the experimental treatment by regulatory agencies is then based on the result of such a significance test with other information to balance the risk of adverse events against the benefit of the treatment to future patients. In the setting of a rare disease, recruiting sufficient patients to achieve conventional error rates for clinically reasonable effect sizes may be infeasible, suggesting that the decision-making process should reflect the size of the target population.

Methods

We considered the use of a decision-theoretic value of information (VOI) method to obtain the optimal sample size and significance level for confirmatory RCTs in a range of settings. We assume the decision maker represents society. For simplicity we assume the primary endpoint to be normally distributed with unknown mean following some normal prior distribution representing information on the anticipated effectiveness of the therapy available before the trial. The method is illustrated by an application in an RCT in haemophilia A. We explicitly specify the utility in terms of improvement in primary outcome and compare this with the costs of treating patients, both financial and in terms of potential harm, during the trial and in the future.

Results

The optimal sample size for the clinical trial decreases as the size of the population decreases. For non-zero cost of treating future patients, either monetary or in terms of potential harmful effects, stronger evidence is required for approval as the population size increases, though this is not the case if the costs of treating future patients are ignored.

Conclusions

Decision-theoretic VOI methods offer a flexible approach with both type I error rate and power (or equivalently trial sample size) depending on the size of the future population for whom the treatment under investigation is intended. This might be particularly suitable for small populations when there is considerable information about the patient population.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0475-0) contains supplementary material, which is available to authorized users.

Approaches to sample size calculation for clinical trials in rare diseases

We discuss 3 alternative approaches to sample size calculation: traditional sample size calculation based on power to show a statistically significant effect, sample size calculation based on assurance, and sample size based on a decision-theoretic approach. These approaches are compared head-to-head for clinical trial situations in rare diseases. Specifically, we consider 3 case studies of rare diseases (Lyell disease, adult-onset Still disease, and cystic fibrosis) with the aim to plan the sample size for an upcoming clinical trial. We outline in detail the reasonable choice of parameters for these approaches for each of the 3 case studies and calculate sample sizes. We stress that the influence of the input parameters needs to be investigated in all approaches and recommend investigating different sample size approaches before deciding finally on the trial size. Highly influencing for the sample size are choice of treatment effect parameter in all approaches and the parameter for the additional cost of the new treatment in the decision-theoretic approach. These should therefore be discussed extensively.

Determination of the optimal sample size for a clinical trial accounting for the population size

The problem of choosing a sample size for a clinical trial is a very common one. In some settings, such as rare diseases or other small populations, the large sample sizes usually associated with the standard frequentist approach may be infeasible, suggesting that the sample size chosen should reflect the size of the population under consideration. Incorporation of the population size is possible in a decision-theoretic approach either explicitly by assuming that the population size is fixed and known, or implicitly through geometric discounting of the gain from future patients reflecting the expected population size. This paper develops such approaches. Building on previous work, an asymptotic expression is derived for the sample size for single and two-arm clinical trials in the general case of a clinical trial with a primary endpoint with a distribution of one parameter exponential family form that optimizes a utility function that quantifies the cost and gain per patient as a continuous function of this parameter. It is shown that as the size of the population, N, or expected size, N∗ in the case of geometric discounting, becomes large, the optimal trial size is O(N1/2) or O(N∗1/2). The sample size obtained from the asymptotic expression is also compared with the exact optimal sample size in examples with responses with Bernoulli and Poisson distributions, showing that the asymptotic approximations can also be reasonable in relatively small sample sizes.

Late-stage pharmaceutical R&D and pricing policies under two-stage regulation

We present a model combining the two regulatory stages relevant to the approval of a new health technology: the authorisation of its commercialisation and the insurer's decision about whether to reimburse its cost. We show that the degree of uncertainty concerning the true value of the insurer's maximum willingness to pay for a unit increase in effectiveness has a non-monotonic impact on the optimal price of the innovation, the firm's expected profit and the optimal sample size of the clinical trial. A key result is that there exists a range of values of the uncertainty parameter over which a reduction in uncertainty benefits the firm, the insurer and patients. We consider how different policy parameters may be used as incentive mechanisms, and the incentives to invest in R&D for marginal projects such as those targeting rare diseases. The model is calibrated using data on a new treatment for cystic fibrosis.

Decision-theoretic designs for small trials and pilot studies: A review

Pilot studies and other small clinical trials are often conducted but serve a variety of purposes and there is little consensus on their design. One paradigm that has been suggested for the design of such studies is Bayesian decision theory. In this article, we review the literature with the aim of summarizing current methodological developments in this area. We find that decision-theoretic methods have been applied to the design of small clinical trials in a number of areas. We divide our discussion of published methods into those for trials conducted in a single stage, those for multi-stage trials in which decisions are made through the course of the trial at a number of interim analyses, and those that attempt to design a series of clinical trials or a drug development programme. In all three cases, a number of methods have been proposed, depending on the decision maker’s perspective being considered and the details of utility functions that are used to construct the optimal design.

Pricing of medical devices under coverage uncertainty--a modelling approach

Product vendors and manufacturers are increasingly aware that purchasers of health care will fund new clinical treatments only if they are perceived to deliver value-for-money. This influences companies' internal commercial decisions, including the price they set for their products. Other things being equal, there is a price threshold, which is the maximum price at which the device will be funded and which, if its value were known, would play a central role in price determination. This paper examines the problem of pricing a medical device from the vendor's point of view in the presence of uncertainty about what the price threshold will be. A formal solution is obtained by maximising the expected value of the net revenue function, assuming a Bayesian prior distribution for the price threshold. A least admissible price is identified. The model can also be used as a tool for analysing proposed pricing policies when no formal prior specification of uncertainty is available.

Value of information and pricing new healthcare interventions

Previous application of value-of-information methods to optimal clinical trial design have predominantly taken a societal decision-making perspective, implicitly assuming that healthcare costs are covered through public expenditure and trial research is funded by government or donation-based philanthropic agencies. In this paper, we consider the interaction between interrelated perspectives of a societal decision maker (e.g. the National Institute for Health and Clinical Excellence [NICE] in the UK) charged with the responsibility for approving new health interventions for reimbursement and the company that holds the patent for a new intervention. We establish optimal decision making from societal and company perspectives, allowing for trade-offs between the value and cost of research and the price of the new intervention. Given the current level of evidence, there exists a maximum (threshold) price acceptable to the decision maker. Submission for approval with prices above this threshold will be refused. Given the current level of evidence and the decision maker's threshold price, there exists a minimum (threshold) price acceptable to the company. If the decision maker's threshold price exceeds the company's, then current evidence is sufficient since any price between the thresholds is acceptable to both. On the other hand, if the decision maker's threshold price is lower than the company's, then no price is acceptable to both and the company's optimal strategy is to commission additional research. The methods are illustrated using a recent example from the literature.

A behavioural Bayes approach to the determination of sample size for clinical trials considering efficacy and safety: imbalanced sample size in treatment groups

The behavioural Bayes approach to sample size determination for clinical trials assumes that the number of subsequent patients switching to a new drug from the current drug depends on the strength of the evidence for efficacy and safety that was observed in the clinical trials. The optimal sample size is the one which maximises the expected net benefit of the trial. The approach has been developed in a series of papers by Pezeshk and the present authors (Gittins JC, Pezeshk H. A behavioral Bayes method for determining the size of a clinical trial. Drug Information Journal 2000; 34: 355-63; Gittins JC, Pezeshk H. How Large should a clinical trial be? The Statistician 2000; 49(2): 177-87; Gittins JC, Pezeshk H. A decision theoretic approach to sample size determination in clinical trials. Journal of Biopharmaceutical Statistics 2002; 12(4): 535-51; Gittins JC, Pezeshk H. A fully Bayesian approach to calculating sample sizes for clinical trials with binary responses. Drug Information Journal 2002; 36: 143-50; Kikuchi T, Pezeshk H, Gittins J. A Bayesian cost-benefit approach to the determination of sample size in clinical trials. Statistics in Medicine 2008; 27(1): 68-82; Kikuchi T, Gittins J. A behavioral Bayes method to determine the sample size of a clinical trial considering efficacy and safety. Statistics in Medicine 2009; 28(18): 2293-306; Kikuchi T, Gittins J. A Bayesian procedure for cost-benefit evaluation of a new drug in multi-national clinical trials. Statistics in Medicine 2009 (Submitted)). The purpose of this article is to provide a rationale for experimental designs which allocate more patients to the new treatment than to the control group. The model uses a logistic weight function, including an interaction term linking efficacy and safety, which determines the number of patients choosing the new drug, and hence the resulting benefit. A Monte Carlo simulation is employed for the calculation. Having a larger group of patients on the new drug in general makes it easier to recruit patients to the trial and may also be ethically desirable. Our results show that this can be done with very little if any reduction in expected net benefit.