Evaluation of antibody-based preventive alternatives for respiratory syncytial virus: a novel multi-criteria decision analysis framework and assessment of nirsevimab in Spain

BACKGROUND

Respiratory syncytial virus (RSV) is a highly infectious disease that poses a significant clinical and medical burden, as well as social disruption and economic costs, recognized by the World Health Organization as a public health issue. After several failed attempts to find preventive candidates (compounds, products, including vaccines), new alternatives might be available, one being nirsevimab, the first and only option approved for RSV prevention in neonates and infants during their first RSV season. The objective of this study was to develop a novel multi-criteria decision analysis (MCDA) framework for RSV antibody-based preventive alternatives and to use it to assess the value of nirsevimab vs. placebo as a systematic immunization approach to prevent RSV in neonates and infants during their first RSV season in Spain.

METHODS

Based on a pre-established model called Vaccinex, an ad-hoc MCDA framework was created to reflect relevant attributes for the assessment of current and future antibody-based preventive measures for RSV. The estimated value of nirsevimab was obtained by means of an additive linear model combining weights and scores assigned by a multidisciplinary committee of 9 experts. A retest and three sensitivity analyses were conducted.

RESULTS

Nirsevimab was evaluated through a novel framework with 26 criteria by the committee as a measure that adds value (positive final estimated value: 0.56 ± 0.11) to the current RSV scenario in Spain, by providing a high efficacy for prevention of neonates and infants. In addition, its implementation might generate cost savings in hospitalizations and to the healthcare system and increase the level of public health awareness among the general population, while reducing health inequities.

CONCLUSIONS

Under a methodology with increasing use in the health field, nirsevimab has been evaluated as a measure which adds value for RSV prevention in neonates and infants during their first RSV season in Spain.

Quantitative Benefit-Risk Assessment of COVID-19 Vaccines Using the Multi-Criteria Decision Analysis

In the early SARS-CoV-2 (COVID-19) pandemic, four major vaccines were approved despite limited efficacy and safety data through short regulatory review periods. Thus, it is necessary to assess the benefit-risk (BR) profiles of the COVID-19 vaccines. We conducted a quantitative BR assessment for four COVID-19 vaccines (mRNA-based: mRNA-1273 and BNT162b2; viral vector-based: Ad26.COV.2 and ChAdOx1-S) using multi-criteria decision analysis. Three benefit criteria and two risk criteria were considered: preventing COVID-19 infection for (1) adults aged ≥18 years; (2) seniors aged 60 years or older; and (3) severe COVID-19, adverse events (AEs), and serious AEs. Data were retrieved from clinical trials, observational studies, and county-specific AE monitoring reports. Based on the collected data, vaccines were scored for each criterion. 22 professionals weighted each criterion. The overall BR score was calculated using scores and weights. mRNA-1273 was the most preferred vaccine in pre-authorization and BNT162b2 in post-authorization. We found that the mRNA vaccine had a good balance between the benefits and risks. Using this BR assessment, the benefit-risk profile of COVID-19 vaccines can be updated with cumulated data. It will contribute to building evidence for decision making by policy makers and health professionals.

A comparison of various aggregation functions in multi-criteria decision analysis for drug benefit-risk assessment

Multi-criteria decision analysis is a quantitative approach to the drug benefit-risk assessment which allows for consistent comparisons by summarising all benefits and risks in a single score. The multi-criteria decision analysis consists of several components, one of which is the utility (or loss) score function that defines how benefits and risks are aggregated into a single quantity. While a linear utility score is one of the most widely used approach in benefit-risk assessment, it is recognised that it can result in counter-intuitive decisions, for example, recommending a treatment with extremely low benefits or high risks. To overcome this problem, alternative approaches to the scores construction, namely, product, multi-linear and Scale Loss Score models, were suggested. However, to date, the majority of arguments concerning the differences implied by these models are heuristic. In this work, we consider four models to calculate the aggregated utility/loss scores and compared their performance in an extensive simulation study over many different scenarios, and in a case study. It is found that the product and Scale Loss Score models provide more intuitive treatment recommendation decisions in the majority of scenarios compared to the linear and multi-linear models, and are more robust to the correlation in the criteria.

Multi-Criteria Decision Analysis for Benefit-Risk Analysis by National Regulatory Authorities

The approval process for pharmaceuticals has always included a consideration of the trade-offs between benefits and risks. Until recently, these trade-offs have been made in panel discussions without using a decision model to explicitly consider what these trade-offs might be. Recently, the EMA and the FDA have embraced Multi-Criteria Decision Analysis (MCDA) as a methodology for making approval decisions. MCDA offers an approach for improving the quality of these decisions and, in particular, by using quantitative and qualitative data in a structured decision model to make trade-offs in a logical, transparent and auditable way. This paper will review the recent use of MCDA by the FDA and EMA and recommend its wider adoption by other National Regulatory Authorities (NRAs) and the pharmaceutical industry.

Multicriteria decision analysis in health care decision in oncology: a systematic review

INTRODUCTION

Multicriteria decision analysis (MCDA) has been used to inform health decisions in health technology assessments (HTA) processes. This is particularly important to complex treatment decisions in oncology.

AREAS COVERED

Five databases (PubMed, EMBASE, LILACS, Web of Science and CRD's NHS Economic Evaluation Database) were searched for studies comparing health technologies in oncology, involving the concept MCDA. The ISPOR MCDA Good Practices Guidelines were used to assess the reporting quality. Study selection, appraisal, and data extraction were performed by two reviewers. Fifteen studies were included. The main decision problem was related to health technology assessment of cancer treatments. Clinicians and public health experts were the most frequent stakeholders. The most frequently included criteria comprised therapeutic benefit, and socio-economic impact. Value measurement approach, direct rating techniques, and additive model for aggregation were used in most studies. Uncertainty analysis revealed the impact of posology and costs on the studies' results. All studies showed some level of overlapping decision criteria.

EXPERT OPINION

There is considerable diversity of methods in MCDA for healthcare decision-making in oncology. The evidence presented can serve as a resource when considering which stakeholders, criteria, and techniques to include in future MCDA studies in oncology.

Benefit-Risk Assessment of Vaccines. Part I: A Systematic Review to Identify and Describe Studies About Quantitative Benefit-Risk Models Applied to Vaccines

Introduction

Understanding the balance between the benefits and risks of vaccination is essential to ensure informed and adequate public health decision making. Quantitative benefit–risk models (qBRm) represent useful tools to help decision makers with supporting benefit–risk assessment throughout the lifecycle of a medical product. However, few initiatives have been launched to harmonise qBRm approaches, specifically for vaccines.

Objectives

The aim of this paper was to identify publications about qBRm applied to vaccines through a systematic literature review, and to describe their characteristics.

Methods

Medline, Scopus and Institute for Scientific Information Web of Knowledge databases were searched to identify articles in English, published from database inceptions up to December 2019. The search strategy included the combination of three key concepts: ‘benefit–risk’, ‘modelling’ and ‘vaccines’. Data extracted included the modelling context and the methodological approaches used.

Results

Of 3172 publications screened, 48 original publications were included. Most of the selected studies were published over the past decade and focused on rotavirus (15), dengue (10) and influenza (6) vaccines. The majority (30) of studies reported analyses related to high-income countries. The methodology of the studies differed, particularly in modelling techniques, benefit–risk measures, and sensitivity analyses. The present work also pointed out a high level of variability in the quality of reporting across studies, with particular regard to input parameters and methodological approaches.

Conclusions

This review provides an extensive list of qBRm applied to vaccines. Discrepancies across studies were identified during our review. While the number of published qBRm studies is increasing, no reporting guidance for qBRm applied to vaccines is currently available. This may affect decision makers’ confidence in the results and their benefit–risk assessment(s); therefore, the development of such reporting guidance is highly needed.

Electronic supplementary material

The online version of this article (10.1007/s40264-020-00984-7) contains supplementary material, which is available to authorized users.

Structured benefit-risk evaluation for medicinal products: review of quantitative benefit-risk assessment findings in the literature

A favorable benefit–risk profile remains an essential requirement for marketing authorization of medicinal drugs and devices. Furthermore, prior subjective, implicit and inconsistent ad hoc benefit–risk assessment methods have rightly evolved towards more systematic, explicit or “structured” approaches. Contemporary structured benefit–risk evaluation aims at providing an objective assessment of the benefit–risk profile of medicinal products and a higher transparency for decision making purposes. The use of a descriptive framework should be the preferred starting point for a structured benefit–risk assessment. In support of more precise assessments, quantitative and semi-quantitative methodologies have been developed and utilized to complement descriptive or qualitative frameworks in order to facilitate the structured evaluation of the benefit–risk profile of medicinal products. In addition, quantitative structured benefit–risk analysis allows integration of patient preference data. Collecting patient perspectives throughout the medical product development process has become increasingly important and key to the regulatory decision-making process. Both industry and regulatory authorities increasingly rely on descriptive structured benefit–risk evaluation and frameworks in drug, vaccine and device evaluation and comparison. Although varied qualitative methods are more commonplace, quantitative approaches have recently been emphasized. However, it is unclear how frequently these quantitative frameworks have been used by pharmaceutical companies to support submission dossiers for drug approvals or to respond to the health authorities’ requests. The objective of this study has been to identify and review, for the first time, currently available, published, structured, quantitative benefit–risk evaluations which may have informed health care professionals and/or payor as well as contributed to decision making purposes in the regulatory setting for drug, vaccine and/or device approval.

ADVANCE system testing: Benefit-risk analysis of a marketed vaccine using multi-criteria decision analysis and individual-level state transition modelling

BACKGROUND

The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using electronic health record (eHR) databases in Europe. Proof-of-concept studies were designed to assess the proposed processes and system for generating the required evidence to perform B/R assessment and near-real time monitoring of vaccines. We aimed to test B/R methodologies for vaccines, using the comparison of the B/R profiles of whole-cell (wP) and acellular pertussis (aP) vaccine formulations in children as an example.

METHODS

We used multi-criteria decision analysis (MCDA) to structure the B/R assessment combined with individual-level state transition modelling to build the B/R effects table. In the state transition model, we simulated the number of events in two hypothetical cohorts of 1 million children followed from first pertussis dose till pre-school-entry booster (or six years of age, whichever occurred first), with one cohort receiving wP, and the other aP. The benefits were reductions in pertussis incidence and complications. The risks were increased incidences of febrile convulsions, fever, hypotonic-hyporesponsive episodes, injection-site reactions and persistent crying. Most model parameters were informed by estimates (coverage, background incidences, relative risks) from eHR databases from Denmark (SSI), Spain (BIFAP and SIDIAP), Italy (Pedianet) and the UK (RCGP-RSC and THIN). Preferences were elicited from clinical and epidemiological experts.

RESULTS

Using state transition modelling to build the B/R effects table facilitated the comparison of different vaccine effects (e.g. immediate vaccine risks vs long-term vaccine benefits). Estimates from eHR databases could be used to inform the simulation model. The model results could be easily combined with preference weights to obtain B/R scores.

CONCLUSION

Existing B/R methodology, modelling and estimates from eHR databases can be successfully used for B/R assessment of vaccines.

A novel measure of drug benefit-risk assessment based on Scale Loss Score

Quantitative methods have been proposed to assess and compare the benefit-risk balance of treatments. Among them, multicriteria decision analysis (MCDA) is a popular decision tool as it permits to summarise the benefits and the risks of a drug in a single utility score, accounting for the preferences of the decision-makers. However, the utility score is often derived using a linear model which might lead to counter-intuitive conclusions; for example, drugs with no benefit or extreme risk could be recommended. Moreover, it assumes that the relative importance of benefits against risks is constant for all levels of benefit or risk, which might not hold for all drugs. We propose Scale Loss Score (SLoS) as a new tool for the benefit-risk assessment, which offers the same advantages as the linear multicriteria decision analysis utility score but has, in addition, desirable properties permitting to avoid recommendations of non-effective or extremely unsafe treatments, and to tolerate larger increases in risk for a given increase in benefit when the amount of benefit is small than when it is high. We present an application to a real case study on telithromycin in Community Acquired Pneumonia and Acute Bacterial Sinusitis, and we investigated the patterns of behaviour of Scale Loss Score, as compared to the linear multicriteria decision analysis, in a comprehensive simulation study.

Economic Analysis of Vaccination Programs: An ISPOR Good Practices for Outcomes Research Task Force Report

This report provides recommendations for budget holders and decision makers in high-, middle, and low-income countries requiring economic analyses of new vaccination programs to allocate scarce resources given budget constraints. ISPOR's Economic Evaluation of Vaccines Designed to Prevent Infectious Disease: Good Practices Task Force wrote guidelines for three analytic methods and solicited comments on them from external reviewers. Cost-effectiveness analyses use decision-analytic models to estimate cumulative changes in resource use, costs, and changes in quality- or disability-adjusted life-years attributable to changes in disease outcomes. Constrained optimization modeling uses a mathematical objective function to be optimized (e.g. disease cases avoided) for a target population for a set of interventions including vaccination programs within established constraints. Fiscal health modeling estimates changes in net present value of government revenues and expenditures attributable to changes in disease outcomes. The task force recommends that those designing economic analyses for new vaccination programs take into account the decision maker's policy objectives and country-specific decision context when estimating: uptake rate in the target population; vaccination program's impact on disease cases in the population over time using a dynamic transmission epidemiologic model; vaccination program implementation and operating costs; and the changes in costs and health outcomes of the target disease(s). The three approaches to economic analysis are complementary and can be used alone or together to estimate a vaccination program's economic value for national, regional, or subregional decision makers in high-, middle-, and low-income countries.

Benefit-Risk Monitoring of Vaccines Using an Interactive Dashboard: A Methodological Proposal from the ADVANCE Project

INTRODUCTION

New vaccines are launched based on their benefit-risk (B/R) profile anticipated from clinical development. Proactive post-marketing surveillance is necessary to assess whether the vaccination uptake and the B/R profile are as expected and, ultimately, whether further public health or regulatory actions are needed. There are several, typically not integrated, facets of post-marketing vaccine surveillance: the surveillance of vaccination coverage, vaccine safety, effectiveness and impact.

OBJECTIVE

With this work, we aim to assess the feasibility and added value of using an interactive dashboard as a potential methodology for near real-time monitoring of vaccine coverage and pre-specified health benefits and risks of vaccines.

METHODS

We developed a web application with an interactive dashboard for B/R monitoring. The dashboard is demonstrated using simulated electronic healthcare record data mimicking the introduction of rotavirus vaccination in the UK. The interactive dashboard allows end users to select certain parameters, including expected vaccine effectiveness, age groups, and time periods and allows calculation of the incremental net health benefit (INHB) as well as the incremental benefit-risk ratio (IBRR) for different sets of preference weights. We assessed the potential added value of the dashboard by user testing amongst a range of stakeholders experienced in the post-marketing monitoring of vaccines.

RESULTS

The dashboard was successfully implemented and demonstrated. The feedback from the potential end users was generally positive, although reluctance to using composite B/R measures was expressed.

CONCLUSION

The use of interactive dashboards for B/R monitoring is promising and received support from various stakeholders. In future research, the use of such an interactive dashboard will be further tested with real-life data as opposed to simulated data.