Multiple Criteria Decision Analysis for Health Care Decision Making--An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force

Value contribution of etranacogene dezaparvovec gene therapy in moderately severe and severe haemophilia B through multi-criteria decision analysis

INTRODUCTION

The value of gene therapies for haemophilia needs to be assessed holistically.

AIM

To determine the value of etranacogene dezaparvovec (ED) compared to current extended half-life (EHL) recombinant factors (rFIX), using multi-criteria decision analysis (MCDA).

METHOD

MCDA EVIDEM methodology adapted to orphan drugs was used, with nine quantitative criteria and four contextual criteria. The MCDA framework was rated by 28 multidisciplinary experts. Descriptive statistics were performed for quantitative and qualitative criteria.

RESULTS

Haemophilia B (HB) was considered a severe disease (mean ± SD: 4.3 ± 0.7) with some unmet needs (mean ± SD 3.3 ± 0.9). Experts found ED more effective (mean ± SD 2.0 ± 2.3) and provide better quality of life (QoL) (mean ± SD: 1.8 ± 1.5) than the comparative HB treatments but with safety uncertainties (mean ± SD -1.2 ± 1.8). ED could lead to medical cost and non-medical cost savings over time (mean ± SD: 1.6 ± 2.0 and 2.0 ± 1.5, respectively). The quality of the evidence was high (mean ± SD: 3.9 ± 0.9). ED was considered aligned with the priorities of the National Health System (NHS) and the specific interests of patients. ED's value contribution was 0.45 (+1 = highest value).

CONCLUSIONS

ED brings added value in the treatment of moderately severe and severe HB (sHB) compared to current EHL rFIX, addressing the severity of the disease and increasing efficacy and patients' QoL especially related to the single dose and low bleeding rate. Concerns about long-term safety need to be addressed.

Scenario analysis and multi-criteria decision analysis to explore alternative reimbursement pathways for whole genome sequencing for blood cancer patients

BACKGROUND

Whole genome sequencing (WGS) has transformative potential for blood cancer management, but reimbursement is hindered by uncertain benefits relative to added costs. This study employed scenario planning and multi-criteria decision analysis (MCDA) to evaluate stakeholders' preferences for alternative reimbursement pathways, informing future health technology assessment (HTA) submission of WGS in blood cancer.

METHODS

Key factors influencing WGS reimbursement in blood cancers were identified through a literature search. Hypothetical scenarios describing various evidential characteristics of WGS for HTA were developed using the morphological approach. An online survey, incorporating MCDA weights, was designed to gather stakeholder preferences (consumers/patients, clinicians/health professionals, industry representatives, health economists, and HTA committee members) for these scenarios. The survey assessed participants' approval of WGS reimbursement for each scenario, and scenario preferences were determined using the geometric mean method, applying an algorithm to improve reliability and precision by addressing inconsistent responses.

RESULTS

Nineteen participants provided complete survey responses, primarily clinicians or health professionals (n = 6; 32 %), consumers/patients and industry representatives (both at n = 5; 26 %). "Clinical impact of WGS results on patient care" was the most critical criterion (criteria weight of 0.25), followed by "diagnostic accuracy of WGS" (0.21), "cost-effectiveness of WGS" (0.19), "availability of reimbursed treatment after WGS" (0.16), and "eligibility criteria for reimbursed treatment based on actionable WGS results" and "cost comparison of WGS" (both at 0.09). Participants preferred a scenario with substantial clinical evidence, high access to reimbursed targeted treatment, cost-effectiveness below $50,000 per quality-adjusted life year (QALY) gained, and affordability relative to standard molecular tests. Reimbursement was initially opposed until criteria such as equal cost to standard tests and better treatment accessibility were met.

CONCLUSION

Payers commonly emphasize acceptable cost-effectiveness, but strong clinical evidence for many variants and comparable costs to standard tests are likely to drive positive reimbursement decisions for WGS.

Using a Patient-Centered Multicriteria Decision Analysis to Assess the Value of Multiple Sclerosis Treatments in the US: A Study Protocol

OBJECTIVE

The engagement of patients and family caregivers in value assessment is pivotal since they provide valuable contributions to assessment acceptability and relevance. The proposed study aims to use patient-centered techniques and multicriteria decision analysis (MCDA) to evaluate the values of disease-modifying therapies (DMTs) for multiple sclerosis (MS) from the perspectives of patients and family caregivers living in three 'Deep South' States of the US-Alabama, Louisiana, and Mississippi.

METHODS

This study will follow guidance from the Patient-Centered Outcomes Research Institute (PCORI) for patient engagement and two best practice reports for MCDA from the Professional Society for Health Economics and Outcomes Research (ISPOR) to complete value assessment. Throughout the study, we will engage multiple stakeholders, including patients, family caregivers, healthcare providers, and payers. Forty patients with MS and their family caregivers from Alabama, Louisiana, and Mississippi will be invited to participate in this study. We will intensively train them for value assessment knowledge and MCDA before we engage them in MCDA to determine the value of DMTs for MS.

DISCUSSIONS

Our approach differs from common MCDA since we incorporated a patient-centered framework in this study. Unlike previous studies only briefly inform or prepare participants before the MCDA process, in this study, we will provide basic value assessment trainings for patients and family caregivers to ensure they can effectively engage throughout the patient-centered MCDA process. We expect this study will demonstrate that the patient-centered MCDA approach is feasible and likely leads to improved patients' and family caregivers' engagement in value assessment.

Development of a Prioritization Framework to Aid Healthcare Funding Decision Making in Health Technology Assessment in Australia: Application of Multicriteria Decision Analysis

OBJECTIVES

This study develops a prioritization framework to aid healthcare funding decision making in health technology assessment (HTA) in Australia using a multiple criteria decision analysis (MCDA) approach.

METHODS

MCDA frameworks for HTAs were reviewed through literature survey to identify the initial criteria and levels within each criterion. Key stakeholders and experts were consulted to confirm these criteria and levels. A conjoint analysis using 1000Minds was undertaken with policy makers from the Department of Health to establish ranking criteria and weighting scores. Monte Carlo simulations were used to examine the sensitivity of findings to factors affecting the ranking and weighting scores. The MCDA was then applied to 6 examples of chronic care models or technologies projects to demonstrate the performance of this approach.

RESULTS

Five criteria (clinical efficacy/effectiveness, safety and tolerability, severity of the condition, quality/uncertainty, and direct impact on healthcare costs) were consistently ranked highest by healthcare decision makers. Among the criteria, patient-level health outcomes were considered the most important, followed by social and ethical values. The analyses were robust to inform the uncertainty in the parameter.

CONCLUSIONS

This study has developed an MCDA tool that effectively integrates key priorities for HTA reviews, reflecting the values and preferences of healthcare stakeholders in Australia. Although this tool aims to align the assessment process more closely with health benefits, it also highlights the importance of considering other criteria.

A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment

Background

There are increasing demands on Emergency Medical Services. More efficient treatment pathways are required to support conveyance decision making and patient referral in prehospital care. Point of Care testing is increasingly available and utilised across the NHS to support optimal ways of working. We aimed to design and conduct a Multiple Criteria Decision Analysis to prioritise in vitro point of care tests and use cases for inclusion in a platform trial of in vitro point of care testing in UK Emergency Medical Services.

Methods

We designed a Multiple Criteria Decision Analysis that included systematic scoping reviews stakeholder recruitment, two stakeholder surveys and two stakeholder workshops to scope the use cases, explore criteria and map use cases, evaluate the criteria and measure the use cases against the criteria.

Results

We recruited 32 stakeholders. We developed a scoring matrix with 4 criteria for scoring the use cases and 8 criteria for scoring the point of care tests and applied weighting determined from survey results. Use cases were scored by the stakeholders against 4 criteria. The 3 highest scoring use cases were point of care troponin testing in: possible Acute Myocardial Infarction, lactate testing in suspected sepsis and in trauma. We developed the process for scoring the point of care tests to be completed close to a proposed trial to allow for a changes in technology.

Conclusions

We successfully designed a Multiple Criteria Decision Analysis to identify use cases and candidate tests for inclusion in a future platform trial of in vitro point of care testing in UK Emergency Medical Services. We identified 3 use cases for evaluation in a platform trial of in vitro point of care testing: troponin testing in possible acute myocardial infarction, lactate testing in suspected sepsis and lactate testing to identify occult haemorrhage in trauma.

Development of a Value Assessment Framework for Pediatric Health Technologies Using Multicriteria Decision Analysis: Expanding the Value Lens for Funding Decision Making

OBJECTIVES

A health technology assessment (HTA) does not systematically account for the circumstances and needs of children and youth. To supplement HTA processes, we aimed to develop a child-tailored value assessment framework using a multicriteria decision analysis approach.

METHODS

We constructed a multicriteria-decision-analysis-based model in multiple phases to create the Comprehensive Assessment of Technologies for Child Health (CATCH) framework. Using a modified Delphi process with stakeholders having broad disciplinary and geographic variation (N = 23), we refined previously generated criteria and developed rank-based weights. We established a criterion-pertinent scoring rubric for assessing incremental benefits of new drugs. Three clinicians independently assessed comprehension by pilotscoring 9 drugs. We then validated CATCH for 2 childhood cancer therapies through structured deliberation with an expert panel (N = 10), obtaining individual scores, consensus scores, and verbal feedback. Analyses included descriptive statistics, thematic analysis, exploratory disagreement indices, and sensitivity analysis.

RESULTS

The modified Delphi process yielded 10 criteria, based on absolute importance/relevance and agreed importance (median disagreement indices = 0.34): Effectiveness, Child-specific Health-related Quality of Life, Disease Severity, Unmet Need, Therapeutic Safety, Equity, Family Impacts, Life-course Development, Rarity, and Fair Share of Life. Pilot scoring resulted in adjusted criteria definitions and more precise score-scaling guidelines. Validation panelists endorsed the framework's key modifiers of value. Modes of their individual prescores aligned closely with deliberative consensus scores.

CONCLUSIONS

We iteratively developed a value assessment framework that captures dimensions of child-specific health and nonhealth gains. CATCH could improve the richness and relevance of HTA decision making for children in Canada and comparable health systems.

Eumycetoma causative agents: A systematic review to inform the World Health Organization priority list of fungal pathogens

The World Health Organization, in response to the growing burden of fungal disease, established a process to develop a fungal priority pathogens list. This systematic review aimed to evaluate the epidemiology and impact of eumycetoma. PubMed and Web of Science were searched to identify studies published between 1 January 2011 and 19 February 2021. Studies reporting on mortality, inpatient care, complications and sequelae, antifungal susceptibility, risk factors, preventability, annual incidence, global distribution, and emergence during the study time frames were selected. Overall, 14 studies were eligible for inclusion. Morbidity was frequent with moderate to severe impairment of quality of life in 60.3%, amputation in up to 38.5%, and recurrent or long-term disease in 31.8%-73.5% of patients. Potential risk factors included male gender (56.6%-79.6%), younger age (11-30 years; 64%), and farming occupation (62.1%-69.7%). Mycetoma was predominantly reported in Sudan, particularly in central Sudan (37%-76.6% of cases). An annual incidence of 0.1/100 000 persons and 0.32/100  000 persons/decade was reported in the Philippines and Uganda, respectively. In Uganda, a decline in incidence from 3.37 to 0.32/100  000 persons between two consecutive 10-year periods (2000-2009 and 2010-2019) was detected. A community-based, multi-pronged prevention programme was associated with a reduction in amputation rates from 62.8% to 11.9%. With the pre-specified criteria, no studies of antifungal drug susceptibility, mortality, and hospital lengths of stay were identified. Future research should include larger cohort studies, greater drug susceptibility testing, and global surveillance to develop evidence-based treatment guidelines and to determine more accurately the incidence and trends over time.

Aspergillus fumigatus-a systematic review to inform the World Health Organization priority list of fungal pathogens

Recognizing the growing global burden of fungal infections, the World Health Organization established a process to develop a priority list of fungal pathogens (FPPL). In this systematic review, we aimed to evaluate the epidemiology and impact of invasive infections caused by Aspergillus fumigatus to inform the first FPPL. The pre-specified criteria of mortality, inpatient care, complications and sequelae, antifungal susceptibility, risk factors, preventability, annual incidence, global distribution, and emergence were used to search for relevant articles between 1 January 2016 and 10 June 2021. Overall, 49 studies were eligible for inclusion. Azole antifungal susceptibility varied according to geographical regions. Voriconazole susceptibility rates of 22.2% were reported from the Netherlands, whereas in Brazil, Korea, India, China, and the UK, voriconazole susceptibility rates were 76%, 94.7%, 96.9%, 98.6%, and 99.7%, respectively. Cross-resistance was common with 85%, 92.8%, and 100% of voriconazole-resistant A. fumigatus isolates also resistant to itraconazole, posaconazole, and isavuconazole, respectively. The incidence of invasive aspergillosis (IA) in patients with acute leukemia was estimated at 5.84/100 patients. Six-week mortality rates in IA cases ranged from 31% to 36%. Azole resistance and hematological malignancy were poor prognostic factors. Twelve-week mortality rates were significantly higher in voriconazole-resistant than in voriconazole-susceptible IA cases (12/22 [54.5%] vs. 27/88 [30.7%]; P = .035), and hematology patients with IA had significantly higher mortality rates compared with solid-malignancy cases who had IA (65/217 [30%] vs. 14/78 [18%]; P = .04). Carefully designed surveillance studies linking laboratory and clinical data are required to better inform future FPPL.

One Health Risk and Disease (OHRAD): a tool to prioritise the risks for epidemic-prone diseases from One Health perspective

BACKGROUND

The rise in epidemic-prone diseases daily poses a serious concern globally. Evidence suggests that many of these diseases are of animal origin and contribute to economic loss. Considering the limited time and other resources available for the animal and human health sectors, selecting the most urgent and significant risk factors and diseases is vital, even though all epidemic-prone diseases and associated risk factors should be addressed. The main aim of developing this tool is to provide a readily accessible instrument for prioritising risk factors and diseases that could lead to disease emergence, outbreak or epidemic.

METHODS

This tool uses a quantitative and semi-quantitative multi-criteria decision analysis (MCDA) method that involves five steps: Identifying risk factors and diseases, Weighting the criteria, Risk and disease scoring, Calculating risk impact and disease burden score, and Ranking risks and diseases. It is intended to be implemented through a co-creation workshop and involves individual and group activities. The last two steps are automated in the MS Excel score sheet.

RESULTS

This One Health Risk and Disease (OHRAD) prioritisation tool starts with an individual activity of identifying the risks and diseases from the more extensive list. This, then, leads to a group activity of weighing the criteria and providing scores for each risk and disease. Finally, the individual risk and disease scores with the rankings are generated in this tool.

CONCLUSIONS

The outcome of this OHRAD prioritisation tool is that the top risks and diseases are prioritised for the particular context from One Health perspective. This prioritised list will help experts and officials decide which epidemic-prone diseases to focus on and for which to develop and design prevention and control measures.

Methods to Include Environmental Impacts in Health Economic Evaluations and Health Technology Assessments: A Scoping Review

OBJECTIVES

The environmental impacts of healthcare are important factors that should be considered during health technology assessments. This study aims to summarize the evidence that exists about methods to include environmental impacts in health economic evaluations and health technology assessments.

METHODS

We identified records for screening using an existing scoping review and a systematic search of academic databases and gray literature up to September 2023. We screened the identified records for eligibility and extracted data using a narrative synthesis approach. The review was conducted following the JBI Manual for Evidence Synthesis and reported according to the Preferred Reporting Items for Systematic Reviews and Meta Analyses Extension for Scoping Reviews checklist.

RESULTS

We identified 2898 records and assessed the full text of 114, of which 54 were included in this review. Ten methods were identified to include environmental impacts in health economic evaluations and health technology assessments. Methods included converting environmental impacts to dollars or disability-adjusted life years and including them in a cost-effectiveness, cost-utility, or cost-benefit analysis, calculating an incremental carbon footprint effectiveness ratio or incremental carbon footprint cost ratio, incorporating impacts as one criteria of a multi-criteria decision analysis, and freely considering impacts during health technology assessment deliberation processes.

CONCLUSIONS

Methods to include environmental impacts in health economic evaluations and health technology assessments exist but have not been tested for widespread use by health technology assessment agencies. Further research and implementation work is needed to determine which method can best aid decision makers to choose low environmental impact healthcare interventions.

Incorporating Prior Data in Quantitative Benefit-Risk Assessments: Case Study of a Bayesian Method

BACKGROUND

Multiple criteria decision analysis (MCDA) and stochastic multi-criteria acceptability analysis (SMAA) in their current implementation cannot incorporate prior or external information on benefits and risks. We demonstrate how to incorporate prior data using a Bayesian mixture model approach while conducting quantitative benefit-risk assessments (qBRA) for medical products.

METHODS

We implemented MCDA and SMAA in a Bayesian framework. To incorporate information from a prior study, we use mixture priors on each benefit and risk attribute that mixes information from a previous study with a vague prior distribution. The degree of borrowing is varied using a mixing proportion parameter.

RESULTS

A demonstration case study for qBRA using the supplementary New Drug Application (sNDA) filing for Rivaroxaban for the indication of reduction in the risk of major thrombotic vascular events in patients with peripheral artery disease (PAD) was used to illustrate the method. Net utility scores, obtained from the randomized controlled trial data to support the sNDA, from the MCDA for Rivaraxoban and comparator were 0.48 and 0.56, respectively, with Rivaroxaban being the preferred alternative only 33% of the time. We show that with only 30% borrowing from a previous RCT, the MCDA and SMAA results are favorable for Rivaroxaban, accounting for the seemingly aberrant results on all-cause death in the trial data used to support the sNDA.

CONCLUSION

Our method to formally incorporate prior data in MCDA and SMAA is easy to use and interpret. Software in the form of an RShiny App is available here: https://sai-dharmarajan.shinyapps.io/BayesianMCDA_SMAA/ .

Nurses' preferences regarding MenACWY conjugate vaccines attributes: a discrete choice experiment in Spain

OBJECTIVES

Immunisation against preventable diseases as meningitis is crucial from a public health perspective to face challenges posed by these infections. Nurses hold a great responsibility for these programs, which highlights the importance of understanding their preferences and needs to improve the success of campaigns. This study aimed to investigate nurses' preferences regarding Meningococcus A, C, W, and Y (MenACWY) conjugate vaccines commercialised in Spain.

STUDY DESIGN

A national-level discrete choice experiment (DCE) was conducted.

METHODS

A literature review and a focus group informed the DCE design. Six attributes were included: pharmaceutical form, coadministration evidence, shelf-life, package contents, single-doses per package, and package volume. Conditional logit models quantified preferences and relative importance (RI).

RESULTS

Thirty experienced primary care nurses participated in this study. Evidence of coadministration with other vaccines was the most important attribute (RI = 43.78%), followed by package size (RI = 22.17%), pharmaceutical form (RI = 19.07%), and package content (RI = 11.80%). There was a preference for evidence of coadministration with routine vaccines (odds ratio [OR] = 2.579, 95% confidence interval [95%CI] = 2.210-3.002), smaller volumes (OR = 1.494, 95%CI = 1.264-1.767), liquid formulations (OR = 1.283, 95%CI = 1.108-1.486) and package contents including only vial/s (OR = 1.283, 95%CI = 1.108-1.486). No statistical evidence was found for the remaining attributes.

CONCLUSIONS

Evidence of coadministration with routine vaccines, easy-to-store packages, and fully liquid formulations were drivers of nurses' preferences regarding MenACWY conjugate vaccines. These findings provide valuable insights for decision-makers to optimize current campaigns.

Rationing in an Era of Multiple Tight Constraints: Is Cost-Utility Analysis Still Fit for Purpose?

Cost-utility analysis may not be sufficient to support reimbursement decisions when the assessed health intervention requires a large proportion of the healthcare budget or when the monetary healthcare budget is not the only resource constraint. Such cases include joint replacement, coronavirus disease 2019 (COVID-19) interventions and settings where all resources are constrained (e.g. post-COVID-19 or in low/middle-income countries). Using literature on health technology assessment, rationing and reimbursement in healthcare, we identified seven alternative frameworks for simultaneous decisions about (dis)investment and proposed modifications to deal with multiple resource constraints. These frameworks comprised constrained optimisation; cost-effectiveness league table; 'step-in-the-right-direction' approach; heuristics based on effective gradients; weighted cost-effectiveness ratios; multicriteria decision analysis (MCDA); and programme budgeting and marginal analysis (PBMA). We used numerical examples to demonstrate how five of these alternative frameworks would operate. The modified frameworks we propose could be used in local commissioning and/or health technology assessment to supplement standard cost-utility analysis for interventions that have large budget impact and/or are subject to additional constraints.

Exploring health preference heterogeneity in the UK: Using the online elicitation of personal utility functions approach to construct EQ-5D-5L value functions on societal, group and individual level

A new method has recently been developed for valuing health states, called 'Online elicitation of Personal Utility Functions' (OPUF). In contrast to established methods, such as time trade-off or discrete choice experiments, OPUF does not require hundreds of respondents, but allows estimating utility functions for small groups and even at the individual level. In this study, we used OPUF to elicit EQ-5D-5L health state preferences from a (not representative) sample of the UK general population, and then compared utility functions on the societal-, group-, and individual level. A demo version of the survey is available at: https://eq5d5l.me. Data from 874 respondents were included in the analysis. For each respondent, we constructed a personal EQ-5D-5L value set. These personal value sets predicted respondents' choices in three hold-out discrete choice tasks with an accuracy of 78%. Overall, preferences varied greatly between individuals. However, PERMANOVA analysis showed that demographic characteristics explained only a small proportion of the variability between subgroups. While OPUF is still under development, it has important strengths: it can be used to construct value sets for patient reported outcome instruments such as EQ-5D-5L, while also allowing examination of underlying preferences in an unprecedented level of detail. In the future, OPUF could be used to complement existing methods, allowing valuation studies in smaller samples, and providing more detailed insights into the heterogeneity of preferences across subgroups.

Innovative decision making tools using discrete mathematics for stress urinary incontinence treatment

In this study, we applied graph theory to clinical decision-making for Stress Urinary Incontinence (SUI) treatment. Utilizing discrete mathematics, we developed a system to visually understand the shortest path to the desired treatment outcomes by considering various patient variables. Focusing on women aged 35-50, we examined the effectiveness of Tension-free Vaginal Tape (TVT) surgery and Vaginal Erbium Laser (VEL) treatment for over 15 years. The TVT group consisted of 102 patients who underwent surgery using either the Advantage Fit mid-urethral sling system (Boston Scientific Co., MA, USA) or the GYNECARE TVT retropubic system (Ethicon Inc., NJ, USA). The VEL group included 113 patients treated with a non-ablative Erbium: YAG laser (FotonaSmooth™ XS; Fotona d.o.o., Ljubljana, Slovenia), and there were 112 patients in the control group. We constructed a network diagram analyzing the correlations between health, demographic factors, treatment methods, and patient outcomes. By calculating the shortest path using heuristic functions, we identified significant correlations and treatment effects. This approach supports patient decision making by choosing between TVT and VEL treatments based on individual objectives. Our findings provide new insights into SUI treatment, highlighting the value of a data-driven personalized approach for clinical decision-making. This interdisciplinary study bridges the gap between mathematics and medicine, demonstrating the importance of a data-centric approach in clinical decisions.

Value contribution of blood-based neurofilament light chain as a biomarker in multiple sclerosis using multi-criteria decision analysis

Introduction

Multiple sclerosis (MS) is a chronic autoimmune demyelinating disease that represents a leading cause of non-traumatic disability among young and middle-aged adults. MS is characterized by neurodegeneration caused by axonal injury. Current clinical and radiological markers often lack the sensitivity and specificity required to detect inflammatory activity and neurodegeneration, highlighting the need for better approaches. After neuronal injury, neurofilament light chains (NfL) are released into the cerebrospinal fluid, and eventually into blood. Thus, blood-based NfL could be used as a potential biomarker for inflammatory activity, neurodegeneration, and treatment response in MS. The objective of this study was to determine the value contribution of blood-based NfL as a biomarker in MS in Spain using the Multi-Criteria Decision Analysis (MCDA) methodology.

Materials and methods

A literature review was performed, and the results were synthesized in the evidence matrix following the criteria included in the MCDA framework. The study was conducted by a multidisciplinary group of six experts. Participants were trained in MCDA and scored the evidence matrix. Results were analyzed and discussed in a group meeting through reflective MCDA discussion methodology.

Results

MS was considered a severe condition as it is associated with significant disability. There are unmet needs in MS as a disease, but also in terms of biomarkers since no blood biomarker is available in clinical practice to determine disease activity, prognostic assessment, and response to treatment. The results of the present study suggest that quantification of blood-based NfL may represent a safe option to determine inflammation, neurodegeneration, and response to treatments in clinical practice, as well as to complement data to improve the sensitivity of the diagnosis. Participants considered that blood-based NfL could result in a lower use of expensive tests such as magnetic resonance imaging scans and could provide cost-savings by avoiding ineffective treatments. Lower indirect costs could also be expected due to a lower impact of disability consequences. Overall, blood-based NfL measurement is supported by high-quality evidence.

Conclusion

Based on MCDA methodology and the experience of a multidisciplinary group of six stakeholders, blood-based NfL measurement might represent a high-value-option for the management of MS in Spain.

Decision discovery using clinical decision support system decision log data for supporting the nurse decision-making process

BACKGROUND

Decision-making in healthcare is increasingly complex; notably in hospital environments where the information density is high, e.g., emergency departments, oncology departments, and psychiatry departments. This study aims to discover decisions from logged data to improve the decision-making process.

METHODS

The Design Science Research Methodology (DSRM) was chosen to design an artifact (algorithm) for the discovery and visualization of decisions. The DSRM's different activities are explained, from the definition of the problem to the evaluation of the artifact. During the design and development activities, the algorithm itself is created. During the demonstration and evaluation activities, the algorithm was tested with an authentic synthetic dataset.

RESULTS

The results show the design and simulation of an algorithm for the discovery and visualization of decisions. A fuzzy classifier algorithm was adapted for (1) discovering decisions from a decision log and (2) visualizing the decisions using the Decision Model and Notation standard.

CONCLUSIONS

In this paper, we show that decisions can be discovered from a decision log and visualized for the improvement of the decision-making process of healthcare professionals or to support the periodic evaluation of protocols and guidelines.

Prioritizing resource allocation to reduce adverse effects of pesticide risk for endangered species

The use of pesticides promotes food security because of the multiple benefits it brings to agriculture, such as reduction in crop losses. However, the use of pesticides can be potentially harmful to non-target species. In the U.S., the Environmental Protection Agency regulates the use of pesticides to manage the risks associated with these agents and to protect species under the Endangered Species Act. As part of these regulations, pesticides must be registered and then reviewed every 15 years to ensure the use conditions are updated with the best available data. The registration and review process can invoke corrective measures to ensure protection of endangered species. However, the registration review process is highly resource and time consuming. There is currently a backlog of unreviewed pesticides, leaving a large quantity of pesticides without updated use conditions to protect species. Identifying ways to streamline this process is urgently needed. We develop a sequencing approach to address the risk assessment bottleneck in the pesticide registration and review process and identify species that would benefit most from detailed assessments. We then demonstrate the magnitude of potential efficiencies using this sequencing process for 61 terrestrial listed species in the state of California. Our results show a consistent ranking of listed species according to their relative benefits from assessment, with 90 % of the species being robustly classified across scenarios in the sensitivity analysis. We found that prioritizing the assessment of a small group of species could potentially result in high conservation benefits, and identify species in need of more detailed data for a robust sequencing. We examine how a sequencing approach can guide decisions about what species might benefit most from different levels of assessment. Our results demonstrate the conservation benefits of employing a sequencing approach to prioritize the allocation of limited resources for endangered species.

Design of a multiple criteria decision analysis framework for prioritizing high-impact health technologies in a regional health service

OBJECTIVES

This study aims to develop a framework for establishing priorities in the regional health service of Murcia, Spain, to facilitate the creation of a comprehensive multiple criteria decision analysis (MCDA) framework. This framework will aid in decision-making processes related to the assessment, reimbursement, and utilization of high-impact health technologies.

METHOD

Based on the results of a review of existing frameworks for MCDA of health technologies, a set of criteria was proposed to be used in the context of evaluating high-impact health technologies. Key stakeholders within regional healthcare services, including clinical leaders and management personnel, participated in a focus group (n = 11) to discuss the proposed criteria and select the final fifteen. To elicit the weights of the criteria, two surveys were administered, one to a small sample of healthcare professionals (n = 35) and another to a larger representative sample of the general population (n = 494).

RESULTS

The responses obtained from health professionals in the weighting procedure exhibited greater consistency compared to those provided by the general public. The criteria more highly weighted were "Need for intervention" and "Intervention outcomes." The weights finally assigned to each item in the multicriteria framework were derived as the equal-weighted sum of the mean weights from the two samples.

CONCLUSIONS

A multi-attribute function capable of generating a composite measure (multicriteria) to assess the value of high-impact health interventions has been developed. Furthermore, it is recommended to pilot this procedure in a specific decision context to evaluate the efficacy, feasibility, usefulness, and reliability of the proposed tool.

Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

Priority setting in times of crises: an analysis of priority setting for the COVID-19 response in the Western Pacific Region

BACKGROUND

While priority setting is recognized as critical for promoting accountability and transparency in health system planning, its role in supporting rational, equitable and fair pandemic planning and responses is less well understood. This study aims to describe how priority setting was used to support planning in the initial stage of the pandemic response in a subset of countries in the Western Pacific Region (WPR).

METHODS

We purposively sampled a subset of countries from WPR and undertook a critical document review of the initial national COVID-19 pandemic response plans. A pre-specified tool guided data extraction and the analysis examined the use of quality parameters of priority setting, and equity considerations.

RESULTS

Nine plans were included in this analysis, from the following countries: Papua New Guinea, Tonga, The Philippines, Fiji, China, Australia, New Zealand, Japan, and Taiwan. Most commonly the plans described strong political will to respond swiftly, resource needs, stakeholder engagement, and defined the roles of institutions that guided COVID-19 response decision-making. The initial plans did not reflect strong evidence of public engagement or considerations of equity informing the early responses to the pandemic.

CONCLUSION

This study advances an understanding of how priority setting and equity considerations were integrated to support the development of the initial COVID-19 responses in nine countries in WPR and contributes to the literature on health system planning during emergencies. This baseline assessment reveals evidence of the common priority setting parameters that were deployed in the initial responses, the prioritized resources and equity considerations and reinforces the importance of strengthening health system capacity for priority setting to support future pandemic preparedness.

Synthesis of relevant information around non-core domains to support Multi-Criteria Decision Analysis (MCDA) for decision making

Countries fundamentally base macro and micro decision making in the field of health on economic considerations, the budgetary impact of technologies being a major criterion. Nevertheless, the value of the technology of interest and its dimensions are more complex if we seek to take decisions based on the value itself. The use of structured and explicit approaches that require the assessment of multiple criteria that reflect the dimensions of this value may significantly improve the quality of the decision making. Multi-criteria decision analysis (MCDA) is a complementary decision-making tool that is able to systematically incorporate dimensions or domains such as ethical, organisational, legal, environmental and social considerations, as well as costs and benefits of medical interventions, together with the distinct perspectives of the interested parties. The objective of this article is to propose the implementation of analysis of non-core domains, in reports of Health Technology Assessment (HTA) agencies/units. To assess the scientific evidence on MCDA techniques a systematic review was conducted using structured searches in biomedical databases and websites of various HTA organisations. A consensus group was held using the nominal group technique and involving users of healthcare services, providers, managers and academics. Complementary, a survey was sent to HTA agencies to ascertain the degree of implementation of MCDA in their methods. 42 articles reporting the use of non-core criteria for the assessment of health technologies were included in the analysis. From these articles, a total of 216 non-core criteria were retrieved and categorised into domains by the researchers, and of these, 56 were classified as socioeconomic, 59 as organisational, 10 as legal, 8 as environmental and 47 as ethical, while 36 were considered to relate to other domains. The consensus group, based on the 216 non-core criteria obtained from the systematic review, proposed, and defined 26 criteria that participants considered necessary for decision making in healthcare. The consensus group did not consider that any of the domains should be given more weight than others or that any individual criteria should dominate. These approaches can serve as a framework of reference for a well-structured systematic discussion concerning the basis of individual criteria and the evidence supporting them.

Participatory Value Evaluation (PVE): A New Preference-Elicitation Method for Decision Making in Healthcare

Participatory value evaluation (PVE) has recently been introduced in the field of health as a new method to elicit stated preferences for public policies. PVE is a method in which respondents in a choice experiment are presented with various policy options and their attributes, and are asked to compose their portfolio of preference given a public-resource constraint. This paper aims to illustrate PVE's potential for informing healthcare decision making and to position it relative to established preference-elicitation methods. We first describe PVE and its theoretical background. Next, by means of a narrative review of the eight existing PVE applications within and outside the health domain, we illustrate the different implementations of the main features of the method. We then compare PVE to several established preference-elicitation methods in terms of the structure and nature of the choice tasks presented to respondents. The portfolio-based choice task in a PVE requires respondents to consider a set of policy alternatives in relation to each other and to make trade-offs subject to one or more constraints, which more closely resembles decision making by policymakers. When using a flexible budget constraint, respondents can trade-off their private income with public expenditures. Relative to other methods, a PVE may be cognitively more demanding and is less efficient; however, it seems a promising complementary method for the preference-based assessment of health policies. Further research into the feasibility and validity of the method is required before researchers and policymakers can fully appreciate the advantages and disadvantages of the PVE as a preference-elicitation method.

Alternative approaches to measuring value: an update on innovative methods in the context of the United States Medicare drug price negotiation program

INTRODUCTION

The United States has begun assessing the value of pharmaceuticals to inform negotiated prices in the Medicare program. Given strong political objections in the United States to the use of QALYs, Medicare will need to adopt an alternative approach to measuring value.

AREAS COVERED

In this narrative review, we identified six alternative approaches to measuring value (equal value life-years, health years in total, generalized risk-adjusted cost-effectiveness, severity weighting based on absolute or proportional shortfall, comparative effectiveness based on conventional clinical endpoints, and comparative effectiveness based on both conventional endpoints and patient-centric value elements) and five criteria for assessing these approaches (responsiveness to concerns about discrimination, feasibility, transparency, flexibility, and the ability to incorporate factors beyond traditional value elements).

EXPERT OPINION

Four of the alternatives are broadly aligned with the cost-effectiveness framework, but none fully addresses all aspects of the stated concerns that QALYs may be used to unintentionally implement discrimination. We note, however, that the extent to which these concerns lead to discrimination in practice is unknown. Finally, we recommend an approach for measuring value in terms of comparative effectiveness that combines quantitative ranking and weighting of distinct criteria (including patient-centric value elements) with deliberation.

Selection of appropriate on-site household sanitation options for rural communities of Zimbabwe - case of Mbire district, Zimbabwe

Selecting an appropriate sanitation option involves multiple stakeholders with often conflicting objectives. A multiple criteria decision analysis (MCDA) framework was developed to inform decision makers on selecting appropriate sanitation options for rural communities. Criteria established from literature were evaluated and weighted on-line by stakeholders. A performance matrix was developed by assigning weights to criteria and scoring alternatives. Selection of alternatives was based on a composite appropriateness index from a rank using the simple multi-attribute ranking technique. The framework was evaluated by verification, validation and sensitivity analysis. Five alternatives were evaluated on 14 decision criteria. The first preferred alternative was the urine diverting dry toilet (72.54) then the Blair ventilated improved pit latrine (67.10). The framework was commented as reasonable and robust. A simple and transparent MCDA framework was developed considering local conditions in a participatory manner to select appropriate alternatives for rural sanitation where a single option is encouraged.

A Step Toward the Development of the First National Multi-Criteria Decision Analysis Framework to Support Healthcare Decision Making in Saudi Arabia

OBJECTIVES

To assess the feasibility of implementing multi-criteria decision analysis (MCDA) and to select the criteria for preparing a national MCDA framework for health technology assessment of orphan drugs in the Kingdom of Saudi Arabia (KSA).

METHODS

The study was conducted in 3 phases. In phase I, a targeted literature review was performed to gather relevant information on the implementation of MCDA in healthcare decision making. Phase II was a cross-sectional survey, conducted to obtain insights from different stakeholders and key opinion leaders on specific topics from the KSA perspective. Phase III included a round-table discussion involving experts to validate the results obtained in the phase II survey and further elaborate on specific requirements that may be critical for developing the first national MCDA framework in the KSA.

RESULTS

All the key opinion leaders involved in the study acknowledged the importance of implementing MCDA in the KSA. The Ministry of Health was assigned the responsibility of chairing the MCDA decision process. The experts selected the quantitative, qualitative, and economic criteria to be considered for the MCDA framework. The stakeholders decided to initiate a pilot phase using the deliberative MCDA methodology for the assessment of orphan drugs based on the selected criteria for a period of 1 year and then reevaluate the need to adapt the pragmatic MCDA model.

CONCLUSION

This article describes the novel initiative that examined the feasibility and process required for the development of the first MCDA framework in the KSA to support healthcare decision making.

Evaluation of antibody-based preventive alternatives for respiratory syncytial virus: a novel multi-criteria decision analysis framework and assessment of nirsevimab in Spain

BACKGROUND

Respiratory syncytial virus (RSV) is a highly infectious disease that poses a significant clinical and medical burden, as well as social disruption and economic costs, recognized by the World Health Organization as a public health issue. After several failed attempts to find preventive candidates (compounds, products, including vaccines), new alternatives might be available, one being nirsevimab, the first and only option approved for RSV prevention in neonates and infants during their first RSV season. The objective of this study was to develop a novel multi-criteria decision analysis (MCDA) framework for RSV antibody-based preventive alternatives and to use it to assess the value of nirsevimab vs. placebo as a systematic immunization approach to prevent RSV in neonates and infants during their first RSV season in Spain.

METHODS

Based on a pre-established model called Vaccinex, an ad-hoc MCDA framework was created to reflect relevant attributes for the assessment of current and future antibody-based preventive measures for RSV. The estimated value of nirsevimab was obtained by means of an additive linear model combining weights and scores assigned by a multidisciplinary committee of 9 experts. A retest and three sensitivity analyses were conducted.

RESULTS

Nirsevimab was evaluated through a novel framework with 26 criteria by the committee as a measure that adds value (positive final estimated value: 0.56 ± 0.11) to the current RSV scenario in Spain, by providing a high efficacy for prevention of neonates and infants. In addition, its implementation might generate cost savings in hospitalizations and to the healthcare system and increase the level of public health awareness among the general population, while reducing health inequities.

CONCLUSIONS

Under a methodology with increasing use in the health field, nirsevimab has been evaluated as a measure which adds value for RSV prevention in neonates and infants during their first RSV season in Spain.

Modelling Informal Carers' Health-Related Quality of Life: Challenges for Economic Evaluation

There has been increasing interest in including carers' health-related qualify of life (HRQoL) in decision models, but currently there is no best practice guidance as to how to do so. Models thus far have typically assumed that carers' HRQoL can be predicted from patient health states, as we illustrate with three examples of disease-modifying treatments. However, this approach limits the mechanisms that influence carers' HRQoL solely to patient health and may not accurately reflect carers' outcomes. In this article, we identify and discuss challenges associated with modelling intervention effects on carers' HRQoL: attaching carer utilities to patient disease states, the size of the caring network, aggregation of carer and patient HRQoL, patient death, and modelling longer-term carer HRQoL. We review and critique potential alternatives to modelling carers' HRQoL in decision models: trial-based analyses, qualitative consideration, cost-consequence analysis, and multicriteria decision analysis, noting that each of these also has its own challenges. We provide a framework of issues to consider when modelling carers' HRQoL and suggest how these can be addressed in current practice and future research.

Adding new childhood vaccines to China's National Immunization Program: evidence, benefits, and priorities

China's National Immunization Program has made remarkable achievements but does not include several important childhood vaccines that are readily available in the private market, such as pneumococcal conjugate vaccine (PCV), rotavirus vaccine, Haemophilus influenzae serotype b (Hib) vaccine, and varicella vaccine. We reviewed the literature to assess these four non-National Immunization Program vaccines in terms of their disease burdens, coverage, inequalities, and cost-effectiveness in China and aimed to recommend priorities for introducing them to the National Immunization Program. Based on our calculations using the available evidence, incorporating these vaccines into China's National Immunization Program in 2019 could have averted 11 761 deaths among children younger than 5 years, accounting for 10·29% of the total deaths in children younger than 5 years and reducing the mortality rate from 7·8 per 1000 to 7·0 per 1000. The review showed that 13-valent PCV (PCV13) had the lowest and most inequitable coverage but could prevent the highest number of deaths. In a budgetary analysis for the cohort of newborns in 2023, we estimated that the projected aggregate government costs were US$1954·92 million for PCV13, $1273·13 million for pentavalent rotavirus vaccine, $415·30 million for Hib vaccine, and $221·64 million for varicella vaccine. Our overall multicriteria decision analysis suggested the following priority order for introducing these four non-programme vaccines to the National Immunization Program to benefit the Chinese population: PCV13, rotavirus vaccine, Hib vaccine, and varicella vaccine.

Clinical comprehensive evaluation of direct oral anticoagulants for patients with atrial fibrillation in China

BACKGROUND

Direct oral anticoagulants (DOACs) are increasingly recommended over warfarin in stroke prevention for patients with non-valvular atrial fibrillation (AF). However, there is an important evidence gap in choosing the most appropriate DOAC for Chinese patients in clinical practice.

METHODS

A multi-criteria decision analysis (MCDA) was adopted to build a scoring framework. Attributes and criteria were identified and determined by a scoping literature review, two rounds of Delphi surveys, and a consensus meeting. Weights of each attribute and criterion in the framework were determined using analytic hierarchy process (AHP). Evidence was collected based on the domestic or at least Asian data. Scoring methods for each criterion were developed depended on their characteristics and determined with an expert consensus meeting. Comprehensive scores of each DOAC were calculated based on the utility scores of each criterion and their corresponding weights.

RESULTS

A total of 5 attributes, including safety, efficacy, costs/cost-effectiveness, suitability, and accessibility, were determined, and 16 criteria were under the 5 attributes. The safety and efficacy were ranked as the top two important attributes with the weights of 38.8% and 35.9%, respectively, while the suitability received the lowest weight of 7.9%. The comprehensive score for edoxaban was the highest (72.3), followed by dabigatran (49.7), rivaroxaban (37.9), and apixaban (35.8).

CONCLUSIONS

This study provided a scoring framework developed for comprehensive evaluation of DOACs in China. The ranking of DOACs could help to support the decision-making in clinical practice. The framework could provide a reference for comprehensive evaluation of other drugs.

Care Pathway Analysis to Inform the Earliest Stages of Technology Development: Scoping Oncological Indications in Need of Innovation

OBJECTIVES

Identifying unmet needs for innovative solutions across disease contexts is challenging but important for directing funding and research efforts and informing early-stage decisions during the innovation process. Our aim was to study the merits of care pathway analysis to scope disease contexts and guide the development of innovative devices. We used oncologic surgery as a case study, for which many intraoperative imaging techniques are under development.

METHODS

Care pathway analysis is a mapping process, which produces graphical maps of clinical pathways using important outcomes and subsequent consequences. We performed care pathway analyses for glioblastoma, breast, bladder, prostate, renal, pancreatic, and oral cavity cancer. Differences between a "perfect" care pathway and the current care pathway in terms of percentage of inadequate margins, associated recurrences, quality of life, and 5-year overall survival were calculated to determine unmet needs. Data from The Netherlands Cancer Registry and literature were used.

RESULTS

Care pathway analysis showed that highest percentages of inadequate margins were found in oral cavity cancer (72.5%), glioblastoma (48.7%), and pancreatic cancer (43.9%). Inadequate margins showed the strongest increase in recurrences in cancer of oral cavity, and bladder (absolute increases of 43.5% and 41.2%, respectively). Impact on survival was largest for bladder and oral cavity cancer with positive margins.

CONCLUSIONS

Care pathway analysis provides overviews of current clinical paths in multiple indications. Disease contexts can be compared via effectiveness gaps that show the potential need for innovative solutions. This information can be used as basis for stakeholder involvement processes to prioritize care pathways in need of innovation.

Assessing neuro-oncology clinical trial impact and value: Testing a novel multi-criteria decision analysis app

BACKGROUND

Many clinical trials are conducted globally, creating challenges in deciding which trial outcomes deserve a clinician's focus and where to direct limited resources. Determining the 'value' of a clinical trial relative to others could be useful in this context. The aim of this study was to test a novel web-based application using multi-criteria decision analysis (MCDA) to rank clinical trial value.

METHODS

The MCDA tool combines seven metrics: unmet need; target population size; access; outcomes; cost; academic impact and use of results. Clinical trials were ranked according to their calculated 'value' - meaning the importance or worth of a trial. We determined face validity of the app using a set of ten published Phase 3 neuro-oncology clinical trials. A survey of neuro-oncology clinicians asked them to rank the same ten clinical trials, and to rank the seven metrics in terms of importance.

RESULTS

The two highest app-ranked trials were in concordance with that of the survey respondents, and consistent with the two studies that have had the most impact on routine clinical practice in neuro-oncology. Of the seven metrics, surveyed clinicians considered patient outcomes and unmet need to be the most important when determining clinical trial value.

CONCLUSIONS

The metrics app was able to rank and produce a numerical 'value' for existing phase 3 neuro-oncology clinical trials. In the future, a related app to prospectively rank future trials at the startup stage could be developed to help centers determine which should be prioritized to be conducted at their site.

Delphi Plus: A novel methodology for identifying evidence-based data standards for health service decision-making

The underlying tenet of evidence-based decision-making in health services is assessing all the relevant evidence. Using the traditional qualitative and quantitative approaches to identifying evidence may not capture the full spectrum of factors that need to be addressed. A selective mixed-method approach may provide a comprehensive assessment of the relevant knowledge. This paper adds to the methodological literature by outlining a novel sequential, mixed-method, exploratory process for identifying evidence-based data standards that may be used for health service decision-making. The three-phase process, entitled Delphi Plus, engages peer-nominated topic-specific experts to assess all publicly available and practice-based items and, through a series of reviews, reach an evidence-based consensus on standards for decision-making. Each process phase is outlined in-depth and supplemented by practical learnings gained through its implementation. The Delphi Plus methodology provides the first comprehensive process for combining the published and practised data to develop evidence-based data standards. The routine use of Delphi Plus would provide a framework for benchmarking in health services, enabling greater monitoring and evaluation of client outcomes and improving quality care. This manuscript describes the process of implementing Delphi Plus and provides an example of data standards generated from its use, which directly inform the Australian Government's Primary Health Care 10 Year Plan.

Developing and validating a multi-criteria decision analytic tool to assess the value of cancer clinical trials: evaluating cancer clinical trial value

BACKGROUND

Demonstrating safety and efficacy of new medical treatments requires clinical trials but clinical trials are costly and may not provide value proportionate to their costs. As most health systems have limited resources, it is therefore important to identify the trials with the highest value. Tools exist to assess elements of a clinical trial such as statistical validity but are not wholistic in their valuation of a clinical trial. This study aims to develop a measure of clinical trials value and provide an online tool for clinical trial prioritisation.

METHODS

A search of the academic and grey literature and stakeholder consultation was undertaken to identify a set of criteria to aid clinical trial valuation using multi-criteria decision analysis. Swing weighting and ranking exercises were used to calculate appropriate weights of each of the included criteria and to estimate the partial-value function for each underlying metric. The set of criteria and their respective weights were applied to the results of six different clinical trials to calculate their value.

RESULTS

Seven criteria were identified: 'unmet need', 'size of target population', 'eligible participants can access the trial', 'patient outcomes', 'total trial cost', 'academic impact' and 'use of trial results'. The survey had 80 complete sets of responses (51% response rate). A trial designed to address an 'Unmet Need' was most commonly ranked as the most important with a weight of 24.4%, followed by trials demonstrating improved 'Patient Outcomes' with a weight of 21.2%. The value calculated for each trial allowed for their clear delineation and thus a final value ranking for each of the six trials.

CONCLUSION

We confirmed that the use of the decision tool for valuing clinical trials is feasible and that the results are face valid based on the evaluation of six trials. A proof-of-concept applying this tool to a larger set of trials with an external validation is currently underway.

Value of concerted and hospital hemodialysis through a multi-criteria decision analysis

OBJECTIVE

To evaluate the value of the provision of contracted versus hospital dialysis services for the treatment of chronic kidney disease in Spain using the multicriteria decision analysis methodology.

METHOD

The EVIDEM (Evidence and Value: Impact on Decision Making) evaluation framework was used to calculate the estimated value of both dialysis delivery models (arranged vs. hospital) through a virtual workshop in which different profiles participated: directors and managers, professionals and heads of units and representatives of patients and relatives. The scores were combined using an additive lineal model, which combined the weight of the model with the individual score of the criteria, and each value was transformed to a scale between 0 and 1.

RESULTS

The estimated value for arranged dialysis was 0.29 (DS: ±0.2) and 0.39 (DS: ±0.2) for hospital dialysis. All profiles gave a higher value to hospital hemodialysis compared to contracted hemodialysis. The highest value for hospital dialysis was for patients (0.44), with the lowest mean value for directors (0.36) and the range for arranged dialysis being between patients (0.31) and intermediate positions (0.27).

CONCLUSIONS

Hospital hemodialysis obtained a higher value than concerted dialysis. In general, the panelists affirmed that it is a useful and interesting exercise and that, to a certain extent, it provides security in decision-making, since it allows ordering, rationalizing and considering, in an explicit and transparent manner, the different criteria involved.

Priority setting in the German healthcare system: results from a discrete choice experiment

Worldwide, social healthcare systems must face the challenges of a growing scarcity of resources and of its inevitable distributional effects. Explicit criteria are needed to define the boundaries of public reimbursement decisions. As Germany stands at the beginning of such a discussion, more formalised priority setting procedures seem in order. Recent research identified multi-criteria decision analysis (MCDA) as a promising approach to inform and to guide decision-making in healthcare systems. In that regard, this paper aims to analyse the relative weight assigned to various criteria in setting priority interventions in Germany. A discrete choice experiment (DCE) was employed in 2015 to elicit equity and efficiency preferences of 263 decision makers, through six attributes. The experiment allowed us to rate different policy interventions based on their features in a composite league table (CLT). As number of potential beneficiaries, severity of disease, individual health benefits and cost-effectiveness are the most relevant criteria for German decision makers within the sample population, the results display an overall higher preference towards efficiency criteria. Specific high priority interventions are mental disorders and cardiovascular diseases.

Comparing Patient and Provider Priorities Around Amputation Level Outcomes Using Multiple Criteria Decision Analysis

BACKGROUND

Patients with chronic limb threatening ischemia may require a transmetatarsal amputation (TMA) or a transtibial amputation. When making an amputation-level decision, these patients face a tradeoff-a TMA preserves more limb and may provide better mobility but has a lower probability of primary wound healing and may therefore result in additional same or higher level amputation surgeries with an associated negative impact on function. Understanding differences in how patients and providers prioritize these tradeoffs and other outcomes may enhance shared decision-making.

OBJECTIVES

Compare patient priorities with provider perceptions of patient priorities using Multiple Criteria Decision Analysis (MCDA).

METHODS

The MCDA Analytic Hierarchy Process was chosen due to its low cognitive burden and ease of implementation. We included 5 criteria (outcomes): ability to walk, healing after amputation surgery, rehabilitation program intensity, limb length, and ease of use of prosthetic/orthotic device. A national sample of dysvascular lower-limb amputees and providers were recruited from the Veterans Health Administration with the MCDA administered online to providers and telephonically to patients.

RESULTS

Twenty-six dysvascular amputees and 38 providers participated. Fifty percent of patients had undergone a TMA; 50%, a transtibial amputation. When compared to providers, patients placed higher value on TMA (72% vs. 63%). Patient versus provider priorities were ability to walk (47% vs. 42%), healing (18% vs. 28%), ease of prosthesis use (17% vs. 13%), limb length (11% vs. 13%), and then rehabilitation intensity (7% vs. 6%).

LIMITATIONS

Our sample may not generalize to other populations.

CONCLUSIONS

Provider perceptions aligned with patient values on amputation level but varied around the importance of each outcome.

IMPLICATIONS

These findings illuminate some differences between patients' values and provider perceptions of patient values, suggesting a role for shared decision-making. Embedding this MCDA framework into a future decision aid may facilitate these discussions.

Developing a combined framework for priority setting in integrated health and social care systems

BACKGROUND

There is an international move towards greater integration of health and social care to cope with the increasing demand on services.. In Scotland, legislation was passed in 2014 to integrate adult health and social care services resulting in the formation of 31 Health and Social Care Partnerships (HSCPs). Greater integration does not eliminate resource scarcity and the requirement to make (resource) allocation decisions to meet the needs of local populations. There are different perspectives on how to facilitate and improve priority setting in health and social care organisations with limited resources, but structured processes at the local level are still not widely implemented. This paper reports on work with new HSCPs in Scotland to develop a combined multi-disciplinary priority setting and resource allocation framework.

METHODS

To develop the combined framework, a scoping review of the literature was conducted to determine the key principles and approaches to priority setting from economics, decision-analysis, ethics and law, and attempts to combine such approaches. Co-production of the combined framework involved a multi-disciplinary workshop including local, and national-level stakeholders and academics to discuss and gather their views.

RESULTS

The key findings from the literature review and the stakeholder workshop were taken to produce a final combined framework for priority setting and resource allocation. This is underpinned by principles from economics (opportunity cost), decision science (good decisions), ethics (justice) and law (fair procedures). It outlines key stages in the priority setting process, including: framing the question, looking at current use of resources, defining options and criteria, evaluating options and criteria, and reviewing each stage. Each of these has further sub-stages and includes a focus on how the combined framework interacts with the consultation and involvement of patients, public and the wider staff.

CONCLUSIONS

The integration agenda for health and social care is an opportunity to develop and implement a combined framework for setting priorities and allocating resources fairly to meet the needs of the population. A key aim of both integration and the combined framework is to facilitate the shifting of resources from acute services to the community.

The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs

BACKGROUND

Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering services to clients (program start-up) and competency in treatment delivery. The present study evaluates the extent to which implementation process fidelity at different implementation stages predicts achievement of those milestones.

METHODS

Implementation process fidelity data-as measured by the Stages of Implementation Completion (SIC)-from 1287 implementing sites across 27 evidence-informed programs were examined in mixed effects regression models with sites nested within programs. Implementation process fidelity, as measured by the proportion of implementation activities completed during the three stages of the SIC Pre-Implementation phase and overall Pre-Implementation (Phase 1) and Implementation (Phase 2) proportion scores, was assessed as a predictor of sites achieving program start-up (i.e., delivering services) and competency in program delivery.

RESULTS

The predicted probability of start-up across all sites was low at 35% (95% CI [33%, 38%]). When considering the evidence-informed program being implemented, that probability was nearly twice as high (64%; 95% CI [42%, 82%]), and 57% of the total variance in program start-up was attributable to the program. Implementation process fidelity was positively and significantly associated with achievement of program start-up and competency. The magnitude of this relationship varied significantly across programs for Pre-Implementation Stage 1 (i.e., Engagement) only. Compared to other stages, completing more Pre-Implementation Stage 3 (Readiness Planning) activities resulted in the most rapid gains in probability of achieving program start-up. The predicted probability of achieving competency was very low unless sites had high scores in both Pre-Implementation and Implementation phases.

CONCLUSIONS

Strong implementation process fidelity-as measured by SIC Pre-Implementation and Implementation phase proportion scores-was associated with sites' achievement of program start-up and competency in program delivery, with early implementation process fidelity being especially potent. These findings highlight the importance of a rigorous Pre-Implementation process.

Value contribution of cenobamate for the treatment of Focal-Onset Seizures (FOS) in patients with drug-resistant epilepsy (DRE) in Spain through reflective Multi-Criteria Decision Analysis (MCDA)

INTRODUCTION

Epilepsy is one of the most common neurological conditions worldwide. The main goal of its treatment is to achieve seizure freedom without intolerable adverse effects. However, despite the availability of many anti-seizure medications, including the latest options, called third-generation anti-seizure medications (ASMs), approximately 40% of people with epilepsy present drug-resistant epilepsy (DRE). Cenobamate is the first ASM approved in Spain for the adjunctive treatment of Focal-Onset Seizures (FOS) in adult patients with DRE. In a chronic disease with a portfolio of available ASMs, the decision to introduce a new therapeutic alternative must follow a holistic evaluation of value provided. Reflective Multi-Criteria Decision Analysis (MCDA) methodology allows to determine the value contribution of a treatment in a given indication considering all relevant criteria for healthcare decision-making in a transparent and systematic manner from the perspective of relevant stakeholders.

PURPOSE

The aim of this study was to determine the relative value contribution of cenobamate in the treatment of FOS in patients with DRE compared with third-generation ASMs using reflective MCDA-based methodology.

METHODS

A systematic literature review (combining biomedical databases and grey literature sources) was performed to populate the Evidence and Value: Impact on DEcisionMaking (EVIDEM) MCDA framework adapted to determine what represents value in the management of FOS in patients with DRE in Spain. The study was conducted in two phases. The first took place in 2021 with a multi-stakeholder group of eight participants. The second phase was conducted in 2022 with a multi-stakeholder group of 32 participants. Participants were trained in MCDA methodology and scored four evidence matrices (cenobamate vs. brivaracetam, vs. perampanel, vs. lacosamide and vs. eslicarbazepine acetate). Results were analyzed and discussed in a group meeting through reflective MCDA discussion methodology.

RESULTS

DRE is considered a very severe condition associated with many important unmet needs, mainly with regard to the lack of more effective treatments to achieve the ultimate goal of treatment. Compared to third-generation ASMs, cenobamate is perceived to have a better efficacy profile based on improvements in responder rate and seizure freedom. Regarding safety, it is considered to have a similar profile to alternatives and a positive quality-of-life profile. Cenobamate results in lower direct medical costs (excluding pharmacological) and indirect costs. Overall, cenobamate is regarded as providing a high therapeutic impact and supported by high-quality evidence.

CONCLUSIONS

Based on reflective MCDA methodology and stakeholders' experience in clinical management of epilepsy in Spain, cenobamate is perceived as a value-added option for the treatment of patients with DRE when compared with third-generation ASMs.

The Full Value of Vaccine Assessments (FVVA): a framework for assessing and communicating the value of vaccines for investment and introduction decision-making

BACKGROUND

Several economic obstacles can deter the development and use of vaccines. This can lead to limited product options for some diseases, delays in new product development, and inequitable access to vaccines. Although seemingly distinct, these obstacles are actually interrelated and therefore need to be addressed through a single over-arching strategy encompassing all stakeholders.

METHODS

To help overcome these obstacles, we propose a new approach, the Full Value of Vaccines Assessments (FVVA) framework, to guide the assessment and communication of the value of a vaccine. The FVVA framework is designed to facilitate alignment across key stakeholders and to enhance decision-making around investment in vaccine development, policy-making, procurement, and introduction, particularly for vaccines intended for use in low- and middle-income countries.

RESULTS

The FVVA framework has three key elements. First, to enhance assessment, existing value-assessment methods and tools are adapted to include broader benefits of vaccines as well as opportunity costs borne by stakeholders. Second, to improve decision-making, a deliberative process is required to recognize the agency of stakeholders and to ensure country ownership of decision-making and priority setting. Third, the FVVA framework provides a consistent and evidence-based approach that facilitates communication about the full value of vaccines, helping to enhance alignment and coordination across diverse stakeholders.

CONCLUSIONS

The FVVA framework provides guidance for stakeholders organizing global-level efforts to promote investment in vaccines that are priorities for LMICs. By providing a more holistic view of the benefits of vaccines, its application also has the potential to encourage greater take-up by countries, thereby leading to more sustainable and equitable impacts of vaccines and immunization programmes.

Using multi-criteria decision analysis to describe stakeholder preferences for new quality improvement initiatives that could optimise prescribing in England

Background

Hospital decision-makers have limited resources to implement quality improvement projects. To decide which interventions to take forward, trade-offs must be considered that inevitably turn on stakeholder preferences. The multi-criteria decision analysis (MCDA) approach could make this decision process more transparent.

Method

An MCDA was conducted to rank-order four types of interventions that could optimise medication use in England's National Healthcare System (NHS) hospitals, including Computerised Interface, Built Environment, Written Communication, and Face-to-Face Interactions. Initially, a core group of quality improvers (N = 10) was convened to determine criteria that could influence which interventions are taken forward according to the Consolidated Framework for Implementation Research. Next, to determine preference weightings, a preference survey was conducted with a diverse group of quality improvers (N = 356) according to the Potentially All Pairwise Ranking of All Possible Alternatives method. Then, rank orders of four intervention types were calculated according to models with criteria unweighted and weighted according to participant preferences using an additive function. Uncertainty was estimated by probabilistic sensitivity analysis using 1,000 Monte Carlo Simulation iterations.

Results

The most important criteria influencing what interventions were preferred was whether they addressed "patient needs" (17.6%)' and their financial "cost (11.5%)". The interventions' total scores (unweighted score out of 30 | weighted out of 100%) were: Computerised Interface (25 | 83.8%), Built Environment (24 | 79.6%), Written Communication (22 | 71.6%), and Face-to-Face (22 | 67.8%). The probabilistic sensitivity analysis revealed that the Computerised Interface would be the most preferred intervention over various degrees of uncertainty.

Conclusions

An MCDA was conducted to rank order intervention types that stand to increase medication optimisation across hospitals in England. The top-ranked intervention type was the Computerised Interface. This finding does not imply Computerised Interface interventions are the most effective interventions but suggests that successfully implementing lower-ranked interventions may require more conversations that acknowledge stakeholder concerns.

Treatment goals for rheumatoid arthritis: patient engagement and goal collection

Aim: We developed the Patient-Engaged Health Technology Assessment strategy for survey-based goal collection from patients to yield patient-important outcomes suitable for use in multi-criteria decision analysis. Methods: Rheumatoid arthritis patients were recruited from online patient networks for proof-of-concept testing of goal collection and prioritization using a survey. A Project Steering Committee and Expert Panel rated the feasibility of scaling to larger samples. Results: Survey respondents (n = 47) completed the goal collection exercise. Finding effective treatments was rated by respondents as the most important goal, and reducing stiffness was rated as the least important. Feedback from our steering committee and expert panel support the approach's feasibility for goal identification and ranking. Conclusion: Goals relevant for treatment evaluation can be identified and rated for importance by patients to permit wide input from patients with lived experience of disease.

The Use of Multicriteria Decision Analysis to Support Decision Making in Healthcare: An Updated Systematic Literature Review

OBJECTIVES

Multicriteria decision analysis (MCDA) is increasingly used for decision making in healthcare. However, its application in different decision-making contexts is still unclear. This study aimed to provide a comprehensive review of MCDA studies performed to inform decisions in healthcare and to summarize its application in different decision contexts.

METHODS

We updated a systematic review conducted in 2013 by searching Embase, MEDLINE, and Google Scholar for MCDA studies in healthcare, published in English between August 2013 and November 2020. We also expanded the search by reviewing grey literature found via Trip Medical Database and Google, published between January 1990 and November 2020. A comprehensive template was developed to extract information about the decision context, criteria, methods, stakeholders involved, and sensitivity analyses conducted.

RESULTS

From the 4295 identified studies, 473 studies were eligible for full-text review after assessing titles and abstracts. Of those, 228 studies met the inclusion criteria and underwent data extraction. The use of MCDA continues to grow in healthcare literature, with most of the studies (49%) informing priority-setting decisions. Safety, cost, and quality of care delivery are the most frequently used criteria, although there are considerable differences across decision contexts. Almost half of the MCDA studies used the linear additive model whereas scales and the analytical hierarchy process were the most used techniques for scoring and weighting, respectively. Not all studies report on each one of the MCDA steps, consider axiomatic properties, or justify the methods used.

CONCLUSIONS

A guide on how to conduct and report MCDA that acknowledges the particularities of the different decision contexts and methods needs to be developed.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Barriers and facilitators to implementing priority setting and resource allocation tools in hospital decisions: A systematic review

Health care budgets in high-income countries are having issues coping with unsustainable growth in demand, particularly in the hospital setting. Despite this, implementing tools systematising priority setting and resource allocation decisions has been challenging. This study answers two questions: (1) what are the barriers and facilitators to implementing priority setting tools in the hospital setting of high-income countries? and (2) what is their fidelity? A systematic review using the Cochrane methods was conducted including studies of hospital-related priority setting tools reporting barriers or facilitators for implementation, published after the year 2000. Barriers and facilitators were classified using the Consolidated Framework for Implementation Research (CFIR). Fidelity was assessed using priority setting tool's standards. Out of thirty studies, ten reported program budgeting and marginal analysis (PBMA), twelve multi-criteria decision analysis (MCDA), six health technology assessment (HTA) related frameworks, and two, an ad hoc tool. Barriers and facilitators were outlined across all CFIR domains. Implementation factors not frequently observed, such as 'evidence of previous successful tool application', 'knowledge and beliefs about the intervention' or 'external policy and incentives' were reported. Conversely, some constructs did not yield any barrier or facilitator including 'intervention source' or 'peer pressure'. PBMA studies satisfied the fidelity criteria between 86% and 100%, for MCDA it varied between 36% and 100%, and for HTA it was between 27% and 80%. However, fidelity was not related to implementation. This study is the first to use an implementation science approach. Results represent the starting point for organisations wishing to use priority setting tools in the hospital setting by providing an overview of barriers and facilitators. These factors can be used to assess readiness for implementation or to serve as the foundation for process evaluations. Through our findings, we aim to improve the uptake of priority setting tools and support their sustainable use.

High-energy devices in different surgical settings: lessons learnt from a full health technology assessment report developed by SICE (Società Italiana di Chirurgia Endoscopica)

BACKGROUND

The present paper aims at evaluating the potential benefits of high-energy devices (HEDs) in the Italian surgical practice, defining the comparative efficacy and safety profiles, as well as the potential economic and organizational advantages for hospitals and patients, with respect to standard monopolar or bipolar devices.

METHODS

A Health Technology Assessment was conducted in 2021 assuming the hospital perspective, comparing HEDs and standard monopolar/bipolar devices, within eleven surgical settings: appendectomy, hepatic resections, colorectal resections, cholecystectomy, splenectomy, hemorrhoidectomy, thyroidectomy, esophago-gastrectomy, breast surgery, adrenalectomy, and pancreatectomy. The nine EUnetHTA Core Model dimensions were deployed considering a multi-methods approach. Both qualitative and quantitative methods were used: (1) a systematic literature review for the definition of the comparative efficacy and safety data; (2) administration of qualitative questionnaires, completed by 23 healthcare professionals (according to 7-item Likert scale, ranging from - 3 to + 3); and (3) health-economics tools, useful for the economic evaluation of the clinical pathway and budget impact analysis, and for the definition of the organizational and accessibility advantages, in terms of time or procedures' savings.

RESULTS

The literature declared a decrease in operating time and length of stay in using HEDs in most surgical settings. While HEDs would lead to a marginal investment for the conduction of 178,619 surgeries on annual basis, their routinely implementation would generate significant organizational savings. A decrease equal to - 5.25/-9.02% of operating room time and to - 5.03/-30.73% of length of stay emerged. An advantage in accessibility to surgery could be hypothesized in a 9% of increase, due to the gaining in operatory slots. Professionals' perceptions crystallized and confirmed literature evidence, declaring a better safety and effectiveness profile. An improvement in both patients and caregivers' quality-of-life emerged.

CONCLUSIONS

The results have demonstrated the strategic relevance related to HEDs introduction, their economic sustainability, and feasibility, as well as the potentialities in process improvement.