A simple way to unify multicriteria decision analysis (MCDA) and stochastic multicriteria acceptability analysis (SMAA) using a Dirichlet distribution in benefit-risk assessment

A comparison of various aggregation functions in multi-criteria decision analysis for drug benefit-risk assessment

Multi-criteria decision analysis is a quantitative approach to the drug benefit-risk assessment which allows for consistent comparisons by summarising all benefits and risks in a single score. The multi-criteria decision analysis consists of several components, one of which is the utility (or loss) score function that defines how benefits and risks are aggregated into a single quantity. While a linear utility score is one of the most widely used approach in benefit-risk assessment, it is recognised that it can result in counter-intuitive decisions, for example, recommending a treatment with extremely low benefits or high risks. To overcome this problem, alternative approaches to the scores construction, namely, product, multi-linear and Scale Loss Score models, were suggested. However, to date, the majority of arguments concerning the differences implied by these models are heuristic. In this work, we consider four models to calculate the aggregated utility/loss scores and compared their performance in an extensive simulation study over many different scenarios, and in a case study. It is found that the product and Scale Loss Score models provide more intuitive treatment recommendation decisions in the majority of scenarios compared to the linear and multi-linear models, and are more robust to the correlation in the criteria.

Comprehensive Benefit-Risk Assessment of Noninferior Treatments Using Multicriteria Decision Analysis

OBJECTIVES

To develop a simple approach for evaluating the overall benefit-risk of a new noninferiority treatment compared with a standard of care.

METHODS

We propose using multicriteria decision analysis that accounts for uncertainty associated with both clinical outcomes and patient preference data. Because patients' preferences are likely to be influenced by their baseline characteristics, we suggest carrying out a preference study at the beginning of a trial. To reduce the burden of an additional study questionnaire, preference elicitation could be done on a small sample of trial participants. To restore preferences for all trial participants, we propose using multiple imputation (MI). Using simulations, we examine whether 3 different MI procedures lead to the same benefit-risk assessment conclusion, as if all trial participant preferences were obtained. We also compare MI results to complete case analysis, where only preferences of the small sample of trial participants are considered.

RESULTS

We show that the MI procedure successfully restores patients' preferences for the trial participants using different outcome criteria and preferences. For example, using 3 outcome criteria with only 10% of the trial participants providing their preferences, complete case analysis demonstrated a new noninferior treatment as favorable only 5.1% of the time, whereas MI procedures did so between 16.2% and 17.9% of the time. Given that 17.6% correspond to the fully observed weights, the MI methods demonstrate favorable results.

CONCLUSIONS

The MI procedure can help facilitate a simple comprehensive benefit-risk assessment for new noninferior treatments.

A flexible design for advanced Phase I/II clinical trials with continuous efficacy endpoints

There is growing interest in integrated Phase I/II oncology clinical trials involving molecularly targeted agents (MTA). One of the main challenges of these trials are nontrivial dose-efficacy relationships and administration of MTAs in combination with other agents. While some designs were recently proposed for such Phase I/II trials, the majority of them consider the case of binary toxicity and efficacy endpoints only. At the same time, a continuous efficacy endpoint can carry more information about the agent's mechanism of action, but corresponding designs have received very limited attention in the literature. In this work, an extension of a recently developed information-theoretic design for the case of a continuous efficacy endpoint is proposed. The design transforms the continuous outcome using the logistic transformation and uses an information-theoretic argument to govern selection during the trial. The performance of the design is investigated in settings of single-agent and dual-agent trials. It is found that the novel design leads to substantial improvements in operating characteristics compared to a model-based alternative under scenarios with nonmonotonic dose/combination-efficacy relationships. The robustness of the design to missing/delayed efficacy responses and to the correlation in toxicity and efficacy endpoints is also investigated.

From Individual to Population Preferences: Comparison of Discrete Choice and Dirichlet Models for Treatment Benefit-Risk Tradeoffs

Introduction. The Dirichlet distribution has been proposed for representing preference heterogeneity, but there is limited evidence on its suitability for modeling population preferences on treatment benefits and risks. Methods. We conducted a simulation study to compare how the Dirichlet and standard discrete choice models (multinomial logit [MNL] and mixed logit [MXL]) differ in their convergence to stable estimates of population benefit-risk preferences. The source data consisted of individual-level tradeoffs from an existing 3-attribute patient preference study (N = 560). The Dirichlet population model was fit directly to the attribute weights in the source data. The MNL and MXL population models were fit to the outcomes of a simulated discrete choice experiment in the same sample of 560 patients. Convergence to the parameter values of the Dirichlet and MNL population models was assessed with sample sizes ranging from 20 to 500 (100 simulations per sample size). Model variability was also assessed with coefficient P values. Results. Population preference estimates of all models were very close to the sample mean, and the MNL and MXL models had good fit (McFadden's adjusted R² = 0.12 and 0.13). The Dirichlet model converged reliably to within 0.05 distance of the population preference estimates with a sample size of 100, where the MNL model required a sample size of 240 for this. The MNL model produced consistently significant coefficient estimates with sample sizes of 100 and higher. Conclusion. The Dirichlet model is likely to have smaller sample size requirements than standard discrete choice models in modeling population preferences for treatment benefit-risk tradeoffs and is a useful addition to health preference analyst's toolbox.

A novel measure of drug benefit-risk assessment based on Scale Loss Score

Quantitative methods have been proposed to assess and compare the benefit-risk balance of treatments. Among them, multicriteria decision analysis (MCDA) is a popular decision tool as it permits to summarise the benefits and the risks of a drug in a single utility score, accounting for the preferences of the decision-makers. However, the utility score is often derived using a linear model which might lead to counter-intuitive conclusions; for example, drugs with no benefit or extreme risk could be recommended. Moreover, it assumes that the relative importance of benefits against risks is constant for all levels of benefit or risk, which might not hold for all drugs. We propose Scale Loss Score (SLoS) as a new tool for the benefit-risk assessment, which offers the same advantages as the linear multicriteria decision analysis utility score but has, in addition, desirable properties permitting to avoid recommendations of non-effective or extremely unsafe treatments, and to tolerate larger increases in risk for a given increase in benefit when the amount of benefit is small than when it is high. We present an application to a real case study on telithromycin in Community Acquired Pneumonia and Acute Bacterial Sinusitis, and we investigated the patterns of behaviour of Scale Loss Score, as compared to the linear multicriteria decision analysis, in a comprehensive simulation study.

A Bayesian approach for individual-level drug benefit-risk assessment

In existing benefit-risk assessment (BRA) methods, benefit and risk criteria are usually identified and defined separately based on aggregated clinical data and therefore ignore the individual-level differences as well as the association among the criteria. We proposed a Bayesian multicriteria decision-making method for BRA of drugs using individual-level data. We used a multidimensional latent trait model to account for the heterogeneity of treatment effects with latent variables introducing the dependencies among outcomes. We then applied the stochastic multicriteria acceptability analysis approach for BRA incorporating imprecise and heterogeneous patient preference information. We adopted an efficient Markov chain Monte Carlo algorithm when implementing the proposed method. We applied our method to a case study to illustrate how individual-level benefit-risk profiles could inform decision-making.

Periodic benefit-risk assessment using Bayesian stochastic multi-criteria acceptability analysis

Benefit-risk (BR) assessment is essential to ensure the best decisions are made for a medical product in the clinical development process, regulatory marketing authorization, post-market surveillance, and coverage and reimbursement decisions. One challenge of BR assessment in practice is that the benefit and risk profile may keep evolving while new evidence is accumulating. Regulators and the International Conference on Harmonization (ICH) recommend performing periodic benefit-risk evaluation report (PBRER) through the product's lifecycle. In this paper, we propose a general statistical framework for periodic benefit-risk assessment, in which Bayesian meta-analysis and stochastic multi-criteria acceptability analysis (SMAA) will be combined to synthesize the accumulating evidence. The proposed approach allows us to compare the acceptability of different drugs dynamically and effectively and accounts for the uncertainty of clinical measurements and imprecise or incomplete preference information of decision makers. We apply our approaches to two real examples in a post-hoc way for illustration purpose. The proposed method may easily be modified for other pre and post market settings, and thus be an important complement to the current structured benefit-risk assessment (sBRA) framework to improve the transparent and consistency of the decision-making process.

Bayesian joint modelling of benefit and risk in drug development

To gain regulatory approval, a new medicine must demonstrate that its benefits outweigh any potential risks, ie, that the benefit-risk balance is favourable towards the new medicine. For transparency and clarity of the decision, a structured and consistent approach to benefit-risk assessment that quantifies uncertainties and accounts for underlying dependencies is desirable. This paper proposes two approaches to benefit-risk evaluation, both based on the idea of joint modelling of mixed outcomes that are potentially dependent at the subject level. Using Bayesian inference, the two approaches offer interpretability and efficiency to enhance qualitative frameworks. Simulation studies show that accounting for correlation leads to a more accurate assessment of the strength of evidence to support benefit-risk profiles of interest. Several graphical approaches are proposed that can be used to communicate the benefit-risk balance to project teams. Finally, the two approaches are illustrated in a case study using real clinical trial data.