The Use of MCDA in HTA: Great Potential, but More Effort Needed

Scenario analysis and multi-criteria decision analysis to explore alternative reimbursement pathways for whole genome sequencing for blood cancer patients

BACKGROUND

Whole genome sequencing (WGS) has transformative potential for blood cancer management, but reimbursement is hindered by uncertain benefits relative to added costs. This study employed scenario planning and multi-criteria decision analysis (MCDA) to evaluate stakeholders' preferences for alternative reimbursement pathways, informing future health technology assessment (HTA) submission of WGS in blood cancer.

METHODS

Key factors influencing WGS reimbursement in blood cancers were identified through a literature search. Hypothetical scenarios describing various evidential characteristics of WGS for HTA were developed using the morphological approach. An online survey, incorporating MCDA weights, was designed to gather stakeholder preferences (consumers/patients, clinicians/health professionals, industry representatives, health economists, and HTA committee members) for these scenarios. The survey assessed participants' approval of WGS reimbursement for each scenario, and scenario preferences were determined using the geometric mean method, applying an algorithm to improve reliability and precision by addressing inconsistent responses.

RESULTS

Nineteen participants provided complete survey responses, primarily clinicians or health professionals (n = 6; 32 %), consumers/patients and industry representatives (both at n = 5; 26 %). "Clinical impact of WGS results on patient care" was the most critical criterion (criteria weight of 0.25), followed by "diagnostic accuracy of WGS" (0.21), "cost-effectiveness of WGS" (0.19), "availability of reimbursed treatment after WGS" (0.16), and "eligibility criteria for reimbursed treatment based on actionable WGS results" and "cost comparison of WGS" (both at 0.09). Participants preferred a scenario with substantial clinical evidence, high access to reimbursed targeted treatment, cost-effectiveness below $50,000 per quality-adjusted life year (QALY) gained, and affordability relative to standard molecular tests. Reimbursement was initially opposed until criteria such as equal cost to standard tests and better treatment accessibility were met.

CONCLUSION

Payers commonly emphasize acceptable cost-effectiveness, but strong clinical evidence for many variants and comparable costs to standard tests are likely to drive positive reimbursement decisions for WGS.

Assessing the Carbon Footprint of Telemedicine: A Systematic Review

Background

Healthcare is responsible for 4% to 10% of carbon emissions worldwide, of which 22% is related to transport. Telemedicine emerged as a potential solution to reduce the footprint, for example, by reducing travel. However, a need to understand which variables to include in carbon footprint estimations in telemedicine limits our understanding of the beneficial impact telemedicine might have on our environment. This paper aims to systematically assess the reported carbon footprint and include variables assessed by the literature, comparing telemedicine with usual care.

Methods

The systematic review followed the PRISMA guidelines in PubMed, Medline, Embase and Scopus. A quality assessment was performed using a transparency checklist for carbon footprint calculators. Carbon emissions were evaluated based on four categories, including patient travel, and streamlined life cycle assessment (LCA) for assessing included variables relevant to telemedicine.

Results

We included 33 articles from 1117 records for analysis. The average transparency score was 38% (range 18%-68%). The median roundtrip travel distance for each patient was 131 km (interquartile range [IQR]: 60.8-351), or 25.6 kgCO2 (IQR: 10.6-105.6) emissions. There is high variance among included variables. Saved emissions are structurally underestimated by not including external factors such as a streamlined LCA.

Conclusions

Telemedicine aids in reducing emissions, with travel distance being the most significant contributor. Additionally, we recommend accounting for the LCA since it highlights important nuances. This review furthers the debate on assessing carbon footprint savings due to telemedicine.

Unravelling Elements of Value of Healthcare and Assessing their Importance Using Evidence from Two Discrete-Choice Experiments in England

BACKGROUND

Health systems are moving towards value-based care, implementing new care models that allegedly aim beyond patient outcomes. Therefore, a policy and academic debate is underway regarding the definition of value in healthcare, the inclusion of costs in value metrics, and the importance of each value element. This study aimed to define healthcare value elements and assess their relative importance (RI) to the public in England.

METHOD

Using data from 26 semi-structured interviews and a literature review, and applying decision-theory axioms, we selected a comprehensive and applicable set of value-based elements. Their RI was determined using two discrete choice experiments (DCEs) based on Bayesian D-efficient DCE designs, with one DCE incorporating healthcare costs expressed as income tax rise. Respondent preferences were analysed using mixed logit models.

RESULTS

Six value elements were identified: additional life-years, health-related quality of life, patient experience, target population size, equity, and cost. The DCE surveys were completed by 402 participants. All utility coefficients had the expected signs and were statistically significant (p < 0.05). Additional life-years (25.3%; 95% confidence interval [CI] 22.5-28.6%) and patient experience (25.2%; 95% CI 21.6-28.9%) received the highest RI, followed by target population size (22.4%; 95% CI 19.1-25.6%) and quality of life (17.6%; 95% CI 15.0-20.3%). Equity had the lowest RI (9.6%; 95% CI 6.4-12.1%), decreasing by 8.8 percentage points with cost inclusion. A similar reduction was observed in the RI of quality of life when cost was included.

CONCLUSION

The public prioritizes value elements not captured by conventional metrics, such as quality-adjusted life-years. Although cost inclusion did not alter the preference ranking, its inclusion in the value metric warrants careful consideration.

Development of a Prioritization Framework to Aid Healthcare Funding Decision Making in Health Technology Assessment in Australia: Application of Multicriteria Decision Analysis

OBJECTIVES

This study develops a prioritization framework to aid healthcare funding decision making in health technology assessment (HTA) in Australia using a multiple criteria decision analysis (MCDA) approach.

METHODS

MCDA frameworks for HTAs were reviewed through literature survey to identify the initial criteria and levels within each criterion. Key stakeholders and experts were consulted to confirm these criteria and levels. A conjoint analysis using 1000Minds was undertaken with policy makers from the Department of Health to establish ranking criteria and weighting scores. Monte Carlo simulations were used to examine the sensitivity of findings to factors affecting the ranking and weighting scores. The MCDA was then applied to 6 examples of chronic care models or technologies projects to demonstrate the performance of this approach.

RESULTS

Five criteria (clinical efficacy/effectiveness, safety and tolerability, severity of the condition, quality/uncertainty, and direct impact on healthcare costs) were consistently ranked highest by healthcare decision makers. Among the criteria, patient-level health outcomes were considered the most important, followed by social and ethical values. The analyses were robust to inform the uncertainty in the parameter.

CONCLUSIONS

This study has developed an MCDA tool that effectively integrates key priorities for HTA reviews, reflecting the values and preferences of healthcare stakeholders in Australia. Although this tool aims to align the assessment process more closely with health benefits, it also highlights the importance of considering other criteria.

Societal Preferences in Health Technology Assessments for Rare Diseases and Orphan Drugs: A Systematic Literature Review of New Analytic Approaches

OBJECTIVES

This systematic literature review aimed to explore experiences worldwide of societal preferences integration into health technology assessments (HTAs) for rare diseases (RDs) and orphan drugs (ODs) through the implementation of multicriteria decision analysis (MCDA), discrete choice experiments (DCEs), and person trade-off (PTO) methods, among others.

METHODS

A systematic search of the literature was conducted in April 2021 using PubMed, Cochrane, Embase, and Scopus databases. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses approach was used for the review phases. Finally, the Promoting Action on Research Implementation in Health Services framework was used to discuss the implementation of these instruments in the RD context.

RESULTS

A total of 33 articles met the inclusion criteria. The studies measured societal preferences for RD and OD as part of HTA using MCDA (n = 17), DCE (n = 8), and PTO (n = 4), among other methods (n = 4). These found that patients and clinicians do not prioritize funding based on rarity. The public is willing to allocate funds only if the OD demonstrates effectiveness and improves the quality of life, considering as relevant factors disease severity, unmet health needs, and quality of life. Conversely, HTA agency experts preferred their current approach, placing more weight on cost-effectiveness and evidence quality, even though they expressed concern about the fairness of the drug review process.

CONCLUSIONS

MCDA, PTO, and DCE are helpful and transparent methods for assessing societal preferences in HTA for RD and OD. However, their methodological limitations, such as arbitrary criteria selection, subjective scoring methods, framing effects, weighting adaptation, and value measurement models, could make implementation challenging.

A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment

Background

There are increasing demands on Emergency Medical Services. More efficient treatment pathways are required to support conveyance decision making and patient referral in prehospital care. Point of Care testing is increasingly available and utilised across the NHS to support optimal ways of working. We aimed to design and conduct a Multiple Criteria Decision Analysis to prioritise in vitro point of care tests and use cases for inclusion in a platform trial of in vitro point of care testing in UK Emergency Medical Services.

Methods

We designed a Multiple Criteria Decision Analysis that included systematic scoping reviews stakeholder recruitment, two stakeholder surveys and two stakeholder workshops to scope the use cases, explore criteria and map use cases, evaluate the criteria and measure the use cases against the criteria.

Results

We recruited 32 stakeholders. We developed a scoring matrix with 4 criteria for scoring the use cases and 8 criteria for scoring the point of care tests and applied weighting determined from survey results. Use cases were scored by the stakeholders against 4 criteria. The 3 highest scoring use cases were point of care troponin testing in: possible Acute Myocardial Infarction, lactate testing in suspected sepsis and in trauma. We developed the process for scoring the point of care tests to be completed close to a proposed trial to allow for a changes in technology.

Conclusions

We successfully designed a Multiple Criteria Decision Analysis to identify use cases and candidate tests for inclusion in a future platform trial of in vitro point of care testing in UK Emergency Medical Services. We identified 3 use cases for evaluation in a platform trial of in vitro point of care testing: troponin testing in possible acute myocardial infarction, lactate testing in suspected sepsis and lactate testing to identify occult haemorrhage in trauma.

Values Beyond "Health" in Budget-Constrained Healthcare Systems

OBJECTIVES

Most current methods to value healthcare treatments only incorporate measures such as quality-adjusted life-years, combining gains in health-related quality of life and life expectancy in specific ways. Failure of these methods to recognize other dimensions of value has led to calls for methods to include additional values that are associated with the healthcare treatments but not captured directly by quality-adjusted life-years. This article seeks to provide methodologically sound ways to incorporate additional health-related outcomes, focusing on budget-constrained healthcare systems, in which using standard welfare economics methods are often eschewed.

METHODS

The analysis develops standard extra-welfarist approaches to maximizing aggregate health, subject to fixed-budget constraints, using Lagrange multiplier methods. Then, additional valuable health-related outcomes, eg, reduced caregiver burden, real option value, and market- and non-market productivity are introduced. The article also introduces a social welfare function approach to illuminate how disability, disease severity and other equity-related issues can be incorporated into complete welfare measures.

RESULTS

Resulting analysis, fully developed in an Appendix in Supplemental Materials found at https://doi.org/10.1016/j.jval.2024.02.005 and summarized in the main text, show that understanding how average and marginal healthcare costs increase with output and how health augments "additional values" provides ways to assess willingness to pay for them in these fixed-budget situations.

CONCLUSIONS

In budget-constrained healthcare systems, only from actual budget allocations can values both of health itself and "additional values" be inferred. These methods, combined with methodologically sound social welfare functions, demonstrate how to move from "health" to "welfare" in measuring the value of increased healthcare use.

Rationing in an Era of Multiple Tight Constraints: Is Cost-Utility Analysis Still Fit for Purpose?

Cost-utility analysis may not be sufficient to support reimbursement decisions when the assessed health intervention requires a large proportion of the healthcare budget or when the monetary healthcare budget is not the only resource constraint. Such cases include joint replacement, coronavirus disease 2019 (COVID-19) interventions and settings where all resources are constrained (e.g. post-COVID-19 or in low/middle-income countries). Using literature on health technology assessment, rationing and reimbursement in healthcare, we identified seven alternative frameworks for simultaneous decisions about (dis)investment and proposed modifications to deal with multiple resource constraints. These frameworks comprised constrained optimisation; cost-effectiveness league table; 'step-in-the-right-direction' approach; heuristics based on effective gradients; weighted cost-effectiveness ratios; multicriteria decision analysis (MCDA); and programme budgeting and marginal analysis (PBMA). We used numerical examples to demonstrate how five of these alternative frameworks would operate. The modified frameworks we propose could be used in local commissioning and/or health technology assessment to supplement standard cost-utility analysis for interventions that have large budget impact and/or are subject to additional constraints.

The application of multi-criteria decision analysis in evaluating the value of drug-oriented intervention: a literature review

Objectives: Multi-Criteria Decision Analysis (MCDA) has gained increasing attention in supporting drug risk-benefit assessment, pricing and reimbursement, as well as optimization of clinical interventions. The objective of this study was to systematically collect and categorize evaluation criteria and techniques of weighting and scoring of MCDA for drug value assessment. Methods: A systematic review of the literature was conducted across seven databases to identify articles utilizing the MCDA frameworks for the evaluation of drug value. Evaluation criteria mentioned in the included studies were extracted and assigned to 5 dimensions including clinical, economic, innovative, societal and humanistic value. A descriptive statistical analysis was performed on the identified drug value evaluation criteria, as well as the weighting and scoring techniques employed. The more a criterion or technique were mentioned in articles, the more important we consider it. Results: Out of the 82 articles included, 111 unique criteria were identified to evaluate the value of drug. Among the 56 unique criteria (448 times) used to measure clinical value, the most frequently mentioned were "comparative safety/tolerability" (58 times), "comparative effectiveness/efficacy" (56 times), "comparative patient-perceived health/patient reported outcomes" (37 times), "disease severity" (34 times), and "unmet needs" (25 times). Regarding economic value measurement, out of the 20 unique criteria (124 times), the most frequently utilized criteria were "cost of intervention" (17 times), "comparative other medical costs" (16 times), and "comparative non-medical costs" (18 times). Out of the 10 criteria (18 times) for assessing innovative value, "a novel pharmacological mechanism" was the most frequently mentioned criterion (5 times). Among the 22 criteria (73 times) used to measure societal value, "system capacity and appropriate use of intervention" was the most frequently cited criterion (14 times). Out of the 3 criteria (15 times) utilized to measure humanistic value, "political/historical/cultural context" was the most frequently mentioned criterion (9 times). Furthermore, 11 scoring and 11 weighting techniques were found from various MCDA frameworks. "Swing weighting" and "a direct rating scale" were the most frequently used techniques in included articles. Conclusion: This study comprehensively presented the current evaluation dimensions, criteria, and techniques for scoring and weighting in drug-oriented MCDA articles. By highlighting the frequently cited evaluation criteria and techniques for scoring and weighting, this analysis will provide a foundation to reasonably select appropriate evaluation criteria and technique in constructing the MCDA framework that aligns with research objectives.

Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

Synthesis of relevant information around non-core domains to support Multi-Criteria Decision Analysis (MCDA) for decision making

Countries fundamentally base macro and micro decision making in the field of health on economic considerations, the budgetary impact of technologies being a major criterion. Nevertheless, the value of the technology of interest and its dimensions are more complex if we seek to take decisions based on the value itself. The use of structured and explicit approaches that require the assessment of multiple criteria that reflect the dimensions of this value may significantly improve the quality of the decision making. Multi-criteria decision analysis (MCDA) is a complementary decision-making tool that is able to systematically incorporate dimensions or domains such as ethical, organisational, legal, environmental and social considerations, as well as costs and benefits of medical interventions, together with the distinct perspectives of the interested parties. The objective of this article is to propose the implementation of analysis of non-core domains, in reports of Health Technology Assessment (HTA) agencies/units. To assess the scientific evidence on MCDA techniques a systematic review was conducted using structured searches in biomedical databases and websites of various HTA organisations. A consensus group was held using the nominal group technique and involving users of healthcare services, providers, managers and academics. Complementary, a survey was sent to HTA agencies to ascertain the degree of implementation of MCDA in their methods. 42 articles reporting the use of non-core criteria for the assessment of health technologies were included in the analysis. From these articles, a total of 216 non-core criteria were retrieved and categorised into domains by the researchers, and of these, 56 were classified as socioeconomic, 59 as organisational, 10 as legal, 8 as environmental and 47 as ethical, while 36 were considered to relate to other domains. The consensus group, based on the 216 non-core criteria obtained from the systematic review, proposed, and defined 26 criteria that participants considered necessary for decision making in healthcare. The consensus group did not consider that any of the domains should be given more weight than others or that any individual criteria should dominate. These approaches can serve as a framework of reference for a well-structured systematic discussion concerning the basis of individual criteria and the evidence supporting them.

Scoring the Value Assessment Framework for China: A Factorial Survey

OBJECTIVES

This study aimed to develop the scoring functions for the recently developed value assessment framework (VAF) for China, which comprises 12 attributes.

METHODS

We implemented a factorial survey among Chinese healthcare stakeholders from July to September 2022. A total of 240 hypothetical drug value profiles described by the VAF were grouped into 60 blocks and randomly assigned to respondents. Each respondent was assigned with 1 block, each presented in 3 disease scenarios of different levels of severity. For each profile, respondents were asked to assess the drug's value on a scale from 0 (lowest) to 10 (highest) and make 1 of the 3 insurance recommendations: cover, to be negotiated for coverage, or reject. Linear and logistic mixed-effects models were used to develop scoring functions for aggregating the value attributes.

RESULTS

A total of 365 respondents participated in the survey. 3968 responses from 331 respondents were included in the analysis. Most of the included respondents were under 45 (n = 256, 77.3%), females (n = 208, 62.8%), living in urban areas (n = 296, 89.4%), and with a bachelor's degree or higher (n = 303, 91.5%). Health benefits and safety carried more weights than other attributes in the scoring functions across disease scenarios. The value and probability of entering negotiation or receiving insurance coverage for the attribute profiles for severe/critical disease were higher than for mild/moderate disease.

CONCLUSIONS

The scoring functions of the VAF can be used to assess the value of a drug and its probability of entering negotiation or receiving insurance coverage in China.

A Step Toward the Development of the First National Multi-Criteria Decision Analysis Framework to Support Healthcare Decision Making in Saudi Arabia

OBJECTIVES

To assess the feasibility of implementing multi-criteria decision analysis (MCDA) and to select the criteria for preparing a national MCDA framework for health technology assessment of orphan drugs in the Kingdom of Saudi Arabia (KSA).

METHODS

The study was conducted in 3 phases. In phase I, a targeted literature review was performed to gather relevant information on the implementation of MCDA in healthcare decision making. Phase II was a cross-sectional survey, conducted to obtain insights from different stakeholders and key opinion leaders on specific topics from the KSA perspective. Phase III included a round-table discussion involving experts to validate the results obtained in the phase II survey and further elaborate on specific requirements that may be critical for developing the first national MCDA framework in the KSA.

RESULTS

All the key opinion leaders involved in the study acknowledged the importance of implementing MCDA in the KSA. The Ministry of Health was assigned the responsibility of chairing the MCDA decision process. The experts selected the quantitative, qualitative, and economic criteria to be considered for the MCDA framework. The stakeholders decided to initiate a pilot phase using the deliberative MCDA methodology for the assessment of orphan drugs based on the selected criteria for a period of 1 year and then reevaluate the need to adapt the pragmatic MCDA model.

CONCLUSION

This article describes the novel initiative that examined the feasibility and process required for the development of the first MCDA framework in the KSA to support healthcare decision making.

Modelling Informal Carers' Health-Related Quality of Life: Challenges for Economic Evaluation

There has been increasing interest in including carers' health-related qualify of life (HRQoL) in decision models, but currently there is no best practice guidance as to how to do so. Models thus far have typically assumed that carers' HRQoL can be predicted from patient health states, as we illustrate with three examples of disease-modifying treatments. However, this approach limits the mechanisms that influence carers' HRQoL solely to patient health and may not accurately reflect carers' outcomes. In this article, we identify and discuss challenges associated with modelling intervention effects on carers' HRQoL: attaching carer utilities to patient disease states, the size of the caring network, aggregation of carer and patient HRQoL, patient death, and modelling longer-term carer HRQoL. We review and critique potential alternatives to modelling carers' HRQoL in decision models: trial-based analyses, qualitative consideration, cost-consequence analysis, and multicriteria decision analysis, noting that each of these also has its own challenges. We provide a framework of issues to consider when modelling carers' HRQoL and suggest how these can be addressed in current practice and future research.

Challenges in the Evaluation of Emerging Highly Specialised Technologies: Is There a Role for Living HTA?

There is currently deep uncertainty about the clinical benefits and cost effectiveness of highly specialised technologies (HSTs), like gene and cell therapies. These treatments are novel, typically have high upfront costs, the patient populations are small and heterogenous, there is minimal information on their long-term safety and effectiveness, and data are limited and often of poor quality. With the increasing number of these technologies and their high cost burden on governments and health care providers, policy makers are currently walking a decision tightrope. On the one hand, an unfavourable funding decision could potentially limit patient access to life-saving treatments, while on the other, a favourable decision could result in unsustainable budget impacts and perhaps poorer patient health outcomes. Health technology assessment (HTA) is meant to determine the value of a health technology in order to promote an equitable, efficient, and high-quality health system. However, standard HTA processes have failed to mitigate the deep uncertainties associated with these technologies. In this paper, we propose a Living HTA framework to address these challenges. This framework includes a one-off process for making explicit the societal values associated with HSTs. These would inform the decision-making approach, data collection and the development of disease-specific reference models to be used by industry sponsors as the basis for their submissions for public funding. Coverage with an evidence development mechanism is also proposed by which data can be collected in real time to update the reference model on a rolling basis, thereby allowing re-assessment of the clinical and cost effectiveness of individual HSTs. The HTA would be 'live' until the results indicate there is sufficient certainty for the funding decision to be confirmed, the price changed or the funding removed.

Reconstructing the value puzzle in health technology assessment: a pragmatic review to determine which modelling methods can account for additional value elements

Health technology assessment (HTA) has traditionally relied on cost-effectiveness analysis (CEA) as a cornerstone of evaluation of new therapies, assessing the clinical validity and utility, the efficacy, and the cost-effectiveness of new interventions. The current format of cost-effectiveness analysis, however, does not allow for inclusion of more holistic aspects of health and, therefore, value elements for new technologies such as the impact on patients and society beyond its pure clinical and economic value. This study aimed to review the recent modelling attempts to expand the traditional cost-effectiveness analysis approach by incorporating additional elements of value in health technology assessment. A pragmatic literature review was conducted for articles published between 2012 and 2022 reporting cost-effectiveness analysis including value aspects beyond the clinical and cost-effectiveness estimates; searches identified 13 articles that were eligible for inclusion. These expanded modelling approaches mainly focused on integrating the impact of societal values and health equity in cost-effectiveness analysis, both of which were championed as important aspects of health technology assessment that should be incorporated into future technology assessments. The reviewed cost-effectiveness analysis methods included modification of the current cost-effectiveness analysis methodology (distributional cost-effectiveness analysis, augmented cost-effectiveness analysis, extended cost-effectiveness analysis) or the use of multi-criteria decision analysis. Of these approaches, augmented cost-effectiveness analysis appears to have the most potential by expanding traditional aspects of value, as it uses techniques already familiar to health technology assessment agencies but also allows space for incorporation of qualitative aspects of a product's value. This review showcases that methods to unravel additional value elements for technology assessment exist, therefore, patient access to promising technologies can be improved by moving the discussion from "if" to "how" additional value elements can inform decision-making.

The Use of Multicriteria Decision Analysis to Support Decision Making in Healthcare: An Updated Systematic Literature Review

OBJECTIVES

Multicriteria decision analysis (MCDA) is increasingly used for decision making in healthcare. However, its application in different decision-making contexts is still unclear. This study aimed to provide a comprehensive review of MCDA studies performed to inform decisions in healthcare and to summarize its application in different decision contexts.

METHODS

We updated a systematic review conducted in 2013 by searching Embase, MEDLINE, and Google Scholar for MCDA studies in healthcare, published in English between August 2013 and November 2020. We also expanded the search by reviewing grey literature found via Trip Medical Database and Google, published between January 1990 and November 2020. A comprehensive template was developed to extract information about the decision context, criteria, methods, stakeholders involved, and sensitivity analyses conducted.

RESULTS

From the 4295 identified studies, 473 studies were eligible for full-text review after assessing titles and abstracts. Of those, 228 studies met the inclusion criteria and underwent data extraction. The use of MCDA continues to grow in healthcare literature, with most of the studies (49%) informing priority-setting decisions. Safety, cost, and quality of care delivery are the most frequently used criteria, although there are considerable differences across decision contexts. Almost half of the MCDA studies used the linear additive model whereas scales and the analytical hierarchy process were the most used techniques for scoring and weighting, respectively. Not all studies report on each one of the MCDA steps, consider axiomatic properties, or justify the methods used.

CONCLUSIONS

A guide on how to conduct and report MCDA that acknowledges the particularities of the different decision contexts and methods needs to be developed.

Identifying Attributes for a Value Assessment Framework in China: A Qualitative Study

BACKGROUND

Value assessment frameworks (VAFs) are promising tools for measuring the value of health technologies and informing coverage policymaking; however, most published VAFs were developed for high-income countries. This study aimed to identify value attributes as part of the development of a VAF in China.

METHODS

We used the qualitative description approach. Specifically, we conducted open-ended semi-structured interviews with Chinese stakeholders, as well as a review and analysis of publicly available government documents related to health technology assessment (HTA) and coverage policies in China. Conventional content analysis and the constant comparison technique were used to generate value attributes. Multiple criteria were used to determine the inclusion of a value attribute, with response levels of included attributes finalized via consensus meetings among the research team.

RESULTS

Thirty-four stakeholders living or working in China completed the semi-structured interview. These stakeholders included policymakers (n = 4), healthcare providers (n = 8), HTA researchers (n = 6), patients and members of the general public (n = 9), and industry representatives (n = 7). In addition, 16 government documents were included for analysis. Twelve value attributes grouped in eight categories are included in the VAF: (1) severity of disease; (2) health benefit, including survival, clinical outcomes, and patient-reported outcomes; (3) safety; (4) economic impact, including budget impact to payer and patients, and cost effectiveness; (5) innovation; (6) organizational impact; (7) health equity; and (8) quality of evidence.

CONCLUSION

These 12 value attributes were identified for the development of a VAF to support health technologies' value assessment and coverage policymaking in China.

The Value of New: Consideration of Product Novelty in Health Technology Assessments of Pharmaceuticals

BACKGROUND

Efforts to understand how treatments affect patients and society have broadened the criteria that health technology assessment (HTA) organizations apply to value assessments. We examined whether HTA agencies in eight countries consider treatment novelty in methods and deliberations.

METHODS

We defined a novel pharmaceutical product to be one that offers a new approach to treatment (e.g., new mechanism of action), addresses an unmet need (e.g., targets a rare condition without effective treatments), or has a broader impact beyond what is typically measured in an HTA. We reviewed peer-reviewed publications and technical guidance materials from HTA organizations in Australia, Canada, England, France, The Netherlands, Norway, Sweden, and the United States (US). In addition, we explored how HTA organizations integrated novelty considerations into deliberations and recommendations related to two newer therapies-voretigene neparvovec for an inherited retinal disorder and ocrelizumab for multiple sclerosis.

RESULTS

None of the HTA organizations acknowledge treatment novelty as an explicit value criterion in their assessments of pharmaceutical products. However, drugs that have novel characteristics are given special consideration, particularly when they address an unmet need. Several organizations document a willingness to expend more resources and accept greater evidence uncertainty for such treatments. Qualitative deliberations about the additional unquantified potential benefits of treatment may also influence HTA recommendations.

CONCLUSION

Major HTA organizations do not recognize novelty as an explicit value criterion, although drugs with novel characteristics may receive special consideration. There is an opportunity for organizations to codify their approach to evaluating novelty in value assessment.

Assessing the potential of HTA to inform resource allocation decisions in low-income settings: The case of Malawi

Health technology assessment (HTA) offers a set of analytical tools to support health systems' decisions about resource allocation. Although there is increasing interest in these tools across the world, including in some middle-income countries, they remain rarely used in low-income countries (LICs). In general, the focus of HTA is narrow, mostly limited to assessments of efficacy and cost-effectiveness. However, the principles of HTA can be used to support a broader series of decisions regarding new health technologies. We examine the potential for this broad use of HTA in LICs, with a focus on Malawi. We develop a framework to classify the main decisions on health technologies within health systems. The framework covers decisions on identifying and prioritizing technologies for detailed assessment, deciding whether to adopt an intervention, assessing alternative investments for implementation and scale-up, and undertaking further research activities. We consider the relevance of the framework to policymakers in Malawi and we use two health technologies as examples to investigate the main barriers and enablers to the use of HTA methods. Although the scarcity of local data, expertise, and other resources could risk limiting the operationalisation of HTA in LICs, we argue that even in highly resource constrained health systems, such as in Malawi, the use of HTA to support a broad range of decisions is feasible and desirable.

Moving Beyond Quality-Adjusted Life-Years in Elderly Care: How Can Multicriteria Decision Analysis Complement Cost-Effectiveness Analysis in Local-Level Decision Making

OBJECTIVES

This study aimed to investigate how multicriteria decision analysis (MCDA) could complement cost-effectiveness analysis (CEA) to support investment decisions in elderly care at local level.

METHODS

We used an integrated elderly care program in The Netherlands as a case study to demonstrate the application of both methods. In a 12-month quasi-experimental study (n = 384), data on the following outcome measures were collected: quality-adjusted life-years (CEA) and physical functioning, psychological well-being, social relationships and participation, enjoyment of life, resilience, person centeredness, continuity of care, and costs (MCDA). We performed regression analysis on inversed probability weighted data and controlled for potential confounders to obtain a double robust estimate of the outcomes. Probabilistic sensitivity analyses determined uncertainty for both methods.

RESULTS

The integrated elderly care program was not likely (ie, 36%) to be cost-effective according to the CEA (incremental cost-effectiveness ratios: €88 249 from a societal perspective) using the conventional Dutch willingness-to-pay threshold (ie, €50 000). The MCDA demonstrated that informal caregivers and professionals slightly preferred the intervention over usual care, driven by enjoyment of life and person centeredness. Patients did not prefer either the intervention or usual care, whereas payers and policy makers slightly preferred usual care, mainly due to higher costs of the intervention.

CONCLUSIONS

MCDA could provide local-level decision makers with a broader measurement of effectiveness by including outcomes beyond health and longevity and the preferences of multiple stakeholders. This additional information could foster the acceptability and implementability of cost-effective innovations in elderly care.

Modelling perceived risk in blockchain enabled supply chain utilizing fuzzy-AHP

Purpose

Blockchain has been considered a disrupting technology that can add value in various supply chains differently. The provenance framework matches the four blockchain capabilities of traceability, certifiability, trackability and verifiability to the five generic risks, namely, the financial risk, psychological risk, social risk, physical risk and performance risk. This will help in uncording which specific risk gets mitigated by the use of blockchain in a specific supply chain.

Design/methodology/approach

This study illustrates four supply chains, namely, pharmaceutical industry, fast moving consumer goods industry, precious metal and automotive industry, and maps the risks associated with them to the provenance framework wherein the applicability of blockchain is mapped. Fuzzy analytical hierarchical processing (F-AHP) is used to rank the risks in the supply chain.

Findings

Blockchain capabilities can elevate the provenance knowledge leading to assurance in terms of origin, authenticity, custody and integrity to mitigate the supply chain risks. Present work highlights the thrust areas across various supply chains and identifies the risk priority tasks aligning the contextual supply chain risks. This study has covered five major risk perceptions. This study contributes to the literature on blockchain, customer perceived risk, provenance and supply chain.

Practical implications

This methodology can be adopted to understand and market the application of blockchain in a supply chain. It brings the marketers and marketing perspective to the supply chain. Exhaustive risk perception can be included to get more comprehensive data on mapping the risks along different supply chains. Vertical extensions of this work can be consideration of other supply chains including dairy, fruits and vegetables, electronics and component assemblies to derive the comprehensive framework for mapping risk perceptions and thereby supply chain risk mitigation through blockchain technology.

Originality/value

This linkage between blockchain, perceived risk, applications in the supply chain and a tool to convince the customers about the blockchain applicability has not been discussed in the literature. Adopting the multi-criteria decision-making F-AHP approach, this study attempt to rank the risks and stimulate conversations around a common framework for multiple sectors.

Health economic evaluation of digital nursing technologies: a review of methodological recommendations

BACKGROUND

Health economic evaluation of digital nursing technologies (DNT) is important to provide information that helps avoid undesirable developments and implementations as well as increase the chances of success of developed applications. At the same time, studies and evidence on cost-effectiveness are still very rare in this field. Review studies in related technology areas such as telemedicine frequently criticise the quality and comparability of health economic evaluations conducted in this field. Based on a content analysis of methodological literature on the economic evaluation of innovative (digital) technologies in health and nursing, this article aims to identify specific challenges in this research area and offers recommendations on how to address these challenges to promote more sound health economic evaluations in the future.

METHODS

A rapid review was conducted, consisting of a systematic search in the Pubmed database as well as Google Scholar. In addition, the literature lists of the analysed texts were scoured for additional texts to be included. Methodological literature, single studies, and reviews were included. A total of 536 studies were screened, of which 29 were included in the full text analysis.

RESULTS

Based on the systematic content analysis of the studies under consideration, 10 specific methodological challenges are identified, and the methodological recommendations were examined for consideration. A particular focus was given to whether specific methodological approaches might be needed in the context of evaluating the efficiency of DNT.

CONCLUSION

Many of the challenges identified for the health economic evaluations of digital nursing technologies are comparable to those of other complex health care interventions. The recommendations discussed can help to alleviate those challenges. Future research should focus on alternative approaches to assessing causality in different phases of technology development while maintaining high evidence standards. High-evidence economic assessment of technologies in nursing care should be carried out in routine use, especially if they are intended to be reimbursed by the social insurance.

A comparison of various aggregation functions in multi-criteria decision analysis for drug benefit-risk assessment

Multi-criteria decision analysis is a quantitative approach to the drug benefit-risk assessment which allows for consistent comparisons by summarising all benefits and risks in a single score. The multi-criteria decision analysis consists of several components, one of which is the utility (or loss) score function that defines how benefits and risks are aggregated into a single quantity. While a linear utility score is one of the most widely used approach in benefit-risk assessment, it is recognised that it can result in counter-intuitive decisions, for example, recommending a treatment with extremely low benefits or high risks. To overcome this problem, alternative approaches to the scores construction, namely, product, multi-linear and Scale Loss Score models, were suggested. However, to date, the majority of arguments concerning the differences implied by these models are heuristic. In this work, we consider four models to calculate the aggregated utility/loss scores and compared their performance in an extensive simulation study over many different scenarios, and in a case study. It is found that the product and Scale Loss Score models provide more intuitive treatment recommendation decisions in the majority of scenarios compared to the linear and multi-linear models, and are more robust to the correlation in the criteria.

Value-based person-centred integrated care for frail elderly living at home: a quasi-experimental evaluation using multicriteria decision analysis

OBJECTIVE

To evaluate the value of the person-centred, integrated care programme Care Chain Frail Elderly (CCFE) compared with usual care, using multicriteria decision analysis (MCDA).

DESIGN

In a 12-month quasi-experimental study, triple-aim outcomes were measured at 0, 6 and 12 months by trained interviewers during home-visits.

SETTING

Primary care, community-based elderly care.

PARTICIPANTS

384 community-dwelling frail elderly were enrolled. The 12-month completion rate was 70% in both groups. Propensity score matching was used to balance age, gender, marital status, living situation, education, smoking status and 3 month costs prior to baseline between the two groups.

INTERVENTION

The CCFE is an integrated care programme with unique features like the presence of the elderly and informal caregiver at the multidisciplinary team meetings, and a bundled payment.

PRIMARY AND SECONDARY OUTCOMES MEASURES

The MCDA results in weighted overall value scores that combines the performance on physical functioning, psychological well-being, social relationships and participation, enjoyment of life, resilience, person-centredness, continuity of care and costs, with importance weights of patients, informal caregivers, professionals, payers and policy-makers.

RESULTS

At 6 months, the overall value scores of CCFE were higher in all stakeholder groups, driven by enjoyment of life (standardised performance scores 0.729 vs 0.685) and person-centredness (0.749 vs 0.663). At 12 months, the overall value scores in both groups were similar from a patient's perspective, slightly higher for CCFE from an informal caregiver's and professional's perspective, and lower for CCFE from a payer's and policy-maker's perspective. The latter was driven by a worse performance on physical functioning (0.682 vs 0.731) and higher costs (€22 816 vs €20 680).

CONCLUSIONS

The MCDA indicated that the CCFE is the preferred way of delivering care to frail elderly at 6 months. However, at 12 months, MCDA results showed little difference from the perspective of patients, informal caregivers and professionals, while payers and policy-makers seemed to prefer usual care.

Priority setting of vaccine introduction in Bangladesh: a multicriteria decision analysis study

OBJECTIVE

To prioritise vaccines for introduction in Bangladesh.

METHODS

Multicriteria decision analysis (MCDA) process was used to prioritise potential vaccines for introduction in Bangladesh. A set of criteria were identified, weighted and assigned scores by relevant stakeholders (n=14) during workshop A. The performance matrix of the data of vaccines against the criteria set was constructed and validated with the experts (n=6) in workshop B. The vaccines were ranked and appraised by another group of stakeholders (n=10) in workshop C, and the final workshop D involved the dissemination of the findings to decision-makers (n=28).

RESULTS

Five criteria including incidence rate, case fatality rate, vaccine efficacy, size of the population at risk and type of population at risk were used quantitatively to evaluate and to score the vaccines. Two other criteria, cost-effectiveness and outbreak potentiality, were considered qualitatively. On deliberation, the Japanese encephalitis (JE) vaccine was ranked top to be recommended for introduction in Bangladesh.

CONCLUSIONS

Based on the MCDA results, JE vaccine is planned to be recommended to the decision-makers for introduction into the national vaccine benefit package. The policymakers support the use of systematic evidence-based decision-making processes such as MCDA for vaccine introduction in Bangladesh, and to prioritise health interventions in the country.

Topic selection process in health technology assessment agencies around the world: a systematic review

Objective

The purpose of this study was to systematically review the process for topic selection by health technology assessment (HTA) agencies around the world to provide the knowledge base for the improvement of topic selection frameworks in HTA agencies.

Methods

A systematic search was conducted in PubMed and EMBASE to identify papers up to February 2019. Gray literature was identified by screening the Web sites of HTA agencies on the nonprofit member list of the International Network of Agencies for Health Technology Assessment (INAHTA). Data were extracted for each HTA agency and synthesized, with issues including general contextual information about each agency and the process of topic selection.

Results

Out of forty-nine nonprofit members of INAHTA, a total of seventeen HTA agencies with a framework for topic selection were identified from twenty-two included papers/documents. Multiple criteria were used for topic selection in all frameworks and agencies undertook multiple steps, which could include the specification of criteria for topic selection, identification of topics, short listing of potential topics, scoping of potential topics, scoring and ranking of potential topics, and deliberation and decision on final topics for HTA. Shortcomings were found in relation to methods of scoring and ranking as well as lack of monitoring and the evaluation of the process.

Conclusions

Our study provides insights into the current practice of topic selection in HTA agencies. Multiple criteria decision analysis methodology appears highly relevant to these processes. A consensus approach for the development of methods of topic selection would be valuable for the HTA community.

Prioritizing Healthcare Interventions: A Comparison of Multicriteria Decision Analysis and Cost-Effectiveness Analysis

OBJECTIVES

To investigate the extent to which stated preferences for treatment criteria elicited using multicriteria decision analysis (MCDA) methods are consistent with the trade-offs (implicitly) applied in cost-effectiveness analysis (CEA), and the impact of any differences on the prioritization of treatments.

METHODS

We used existing MCDA and CEA models developed to evaluate interventions for knee osteoarthritis in the New Zealand population. We established equivalent input parameters for each model, for the criteria "treatment effectiveness," "cost," "risk of serious harms," and "risk of mild-to-moderate harms" across a comprehensive range of (hypothetical) interventions to produce a complete ranking of interventions from each model. We evaluated the consistency of these rankings between the 2 models and investigated any systematic differences between the (implied) weight placed on each criterion in determining rankings.

RESULTS

There was an overall moderate-to-strong correlation in intervention rankings between the MCDA and CEA models (Spearman correlation coefficient = 0.51). Nevertheless, there were systematic differences in the evaluation of trade-offs between intervention attributes and the resulting weights placed on each criterion. The CEA model placed lower weights on risks of harm and much greater weight on cost (at all accepted levels of willingness-to-pay per quality-adjusted life-year than did respondents to the MCDA survey.

CONCLUSIONS

MCDA and CEA approaches to inform intervention prioritization may give systematically different results, even when considering the same criteria and input data. These differences should be considered when designing and interpreting such studies to inform treatment prioritization decisions.

QALYs and ambulatory status: societal preferences for healthcare decision making

BACKGROUND

This research aimed to review the theoretical and methodological aspects of the quality-adjusted life year (QALY) which give rise to potential for bias against certain patient populations, including those with problems with walking or an inability to walk (ambulatory disabilities), when health technology assessment decisions rely on QALY gain to show cost-effectiveness. Societal preferences for treating ambulatory versus non-ambulatory patients were also investigated.

METHODS

We reviewed published literature to identify information on theoretical underpinnings of the QALY, measurement of utilities for QALY assessment, and empirical evidence of societal preferences for the treatment of ambulatory and non-ambulatory patients.

RESULTS AND DISCUSSION

Health states which represent mobility impairment and the inability to walk receive low valuation from general public preferences. Non-ambulatory patients, for example those with advanced neuromuscular disease, have lower utilities determined by standardized preference-based measurement (PBM) tools. Any treatment that increases survival but could not restore ambulation would result in lower lifetime QALY gains for non-ambulatory versus ambulatory patients. Treatments could therefore potentially be deemed less cost-effective, or not cost-effective at all for this patient population.Empirical research indicates a societal preference for equal treatment of patients regardless of ambulatory status. The main limitation of our review was the non-systematic approach to evidence search and review, however, given the broad scope of content required to meet the aims of the review, we believe that the targeted approach was appropriate. The evidence presented in this article highlights the need for alternatives to strict QALY-based approaches to prevent avoidable health inequities when determining cost-effectiveness of healthcare interventions for non-ambulatory populations against fixed cost-effectiveness thresholds. An alternative metric, the Equal Value of Life Years Gained (evLYG), has been proposed as a supplementary measure for use alongside the QALY for its potential to alleviate bias against disabled patient populations during the assessment of healthcare treatments.

Multicriteria decision analysis in health care decision in oncology: a systematic review

INTRODUCTION

Multicriteria decision analysis (MCDA) has been used to inform health decisions in health technology assessments (HTA) processes. This is particularly important to complex treatment decisions in oncology.

AREAS COVERED

Five databases (PubMed, EMBASE, LILACS, Web of Science and CRD's NHS Economic Evaluation Database) were searched for studies comparing health technologies in oncology, involving the concept MCDA. The ISPOR MCDA Good Practices Guidelines were used to assess the reporting quality. Study selection, appraisal, and data extraction were performed by two reviewers. Fifteen studies were included. The main decision problem was related to health technology assessment of cancer treatments. Clinicians and public health experts were the most frequent stakeholders. The most frequently included criteria comprised therapeutic benefit, and socio-economic impact. Value measurement approach, direct rating techniques, and additive model for aggregation were used in most studies. Uncertainty analysis revealed the impact of posology and costs on the studies' results. All studies showed some level of overlapping decision criteria.

EXPERT OPINION

There is considerable diversity of methods in MCDA for healthcare decision-making in oncology. The evidence presented can serve as a resource when considering which stakeholders, criteria, and techniques to include in future MCDA studies in oncology.

Application of Multicriteria Decision Analysis to Determine the Value of Prophylaxis Relative to On-Demand Treatment in Hemophilia A and Emicizumab versus Replacement Therapy in the Greek Healthcare Setting

BACKGROUND AND OBJECTIVE

Hemophilia A (HA) is a rare disease that is characterized by congenital underproduction or dysfunction of the endogenous coagulation factor VIII (FVIII). The aim of the present study was to determine the value of prophylaxis versus on-demand treatment strategies for moderate to severe HA (MtSHA) patients and the value of emicizumab in the prophylaxis of MtSHA in Greece, compared with short half-life (SHL) FVIII and extended half-life (EHL) FVIII through multicriteria decision analysis (MCDA).

METHODS

A literature review was performed to identify a set of criteria relevant to the therapeutic approaches and therapies under investigation. A performance matrix was populated by two literature reviews and meta-analyses. The criteria selected were hierarchically classified by allocating weights on a 0-100 scale. The performances of therapies were scored at the 100-point scale. The value judgments utilized for weighing and scoring were sourced via a survey among independent multidisciplinary system stakeholders. A linear additive value function was used for the calculation of total value estimates.

RESULTS

The participants ranked 'annual number of bleedings per patient' and 'percentage of target joint bleeds' as the most important criteria, while the least important criterion was the 'annual treatment cost' for both assessments. Based on the weights elicited and the performance in each criterion, the overall value score was higher for prophylaxis treatment (58.27) compared with on-demand treatment (40.13). In the other comparison, the most valued treatment was emicizumab (77.05) followed by EHL FVIII (71.52) and SHL FVIII (19.88). According to the participants, the most important factors for managing MtSHA patients are those related to successful management of bleeding events given their contribution to improved quality of life (QoL) and reduced morbidity.

CONCLUSIONS

This MCDA has shown that the prophylaxis strategy was perceived as a more valuable option for managing MtSHA patients when compared with the on-demand strategy. Moreover, emicizumab adds higher value and improves patient QoL compared with replacement therapy for MtSHA in Greece. Emicizumab addresses important unmet needs due to its improved efficacy and mode of administration.

Resource Allocation in a National Dental Service Using Program Budgeting Marginal Analysis

INTRODUCTION

In any health system, choices must be made about the allocation of resources (budget), which are often scarce. Economics has defined frameworks to aid resource allocation, and program budgeting marginal analysis (PBMA) is one such framework. In principle, patient and public values can be incorporated into these frameworks, using techniques such as willingness to pay (WTP). However, this has not been done before, and few formal resource allocation processes have been undertaken in dentistry. This study aimed to undertake a PBMA with embedded WTP values in a national dental setting.

METHODS

The PBMA process was undertaken by a panel of participant-researchers representing commissioners, dentists, dental public health staff, and academics. The panel reviewed current allocations and generated a set of weighted criteria to evaluate services against. Services to be considered for removal and investment were determined by the panel and wider discussion and then scored against the criteria. Values from a nationally representative WTP survey of the public contributed to the scores for interventions. Final decisions on removal and investment were taken after panel discussion using individual anonymous electronic voting.

RESULTS

The PBMA process resulted in recommendations to invest in new program components to improve access to general dentists, care home dentistry, and extra support for dental public health input into local government decisions. Disinvestments were recommended in orthodontics and to remove routine scaling and polishing of teeth.

DISCUSSION

The PBMA process was successful in raising awareness of resource allocation issues. Implementation of findings will depend on the ability of decision makers to find ways of operationalizing the decisions. The process illustrates practical aspects of the process that future dental PBMAs could learn from.

KNOWLEDGE TRANSFER STATEMENT

This study illustrates a framework for resource allocation in dental health services and will aid decision makers in implementing their own resource allocation systems.

Evaluating an integrated care pathway for frail elderly patients in Norway using multi-criteria decision analysis

Background

To provide value-based care for patients with multi-morbidity, innovative integrated care programmes and comprehensive evaluations of such programmes are required. In Norway, a new programme called “Holistic Continuity of Patient Care” (HCPC) addresses the issue of multi-morbidity by providing integrated care within learning networks for frail elderly patients who receive municipal home care services or a short-term stay in a nursing home. This study conducts a multi-criteria decision analysis (MCDA) to evaluate whether the HCPC programme performs better on a large set of outcomes corresponding to the ‘triple aim’ compared to usual care.

Methods

Prospective longitudinal survey data were collected at baseline and follow-up after 6-months. The assessment of HCPC was implemented by a novel MCDA framework. The relative weights of importance of the outcomes used in the MCDA were obtained from a discrete choice experiment among five different groups of stakeholders. The performance score was estimated using a quasi-experimental design and linear mixed methods. Performance scores were standardized and multiplied by their weights of importance to obtain the overall MCDA value by stakeholder group.

Results

At baseline in the HCPC and usual care groups, respectively, 120 and 89 patients responded, of whom 87 and 41 responded at follow-up. The average age at baseline was 80.0 years for HCPC and 83.6 for usual care. Matching reduced the standardized differences between the groups for patient background characteristics and outcome variables. The MCDA results indicated that HCPC was preferred to usual care irrespective of stakeholders. The better performance of HCPC was mostly driven by improvements in enjoyment of life, psychological well-being, and social relationships and participation. Results were consistent with sensitivity analyses using Monte Carlo simulation.

Conclusion

Frail elderly with multi-morbidity represent complex health problems at large costs for society in terms of health- and social care. This study is a novel contribution to assessing and understanding HCPC programme performance respecting the multi-dimensionality of desired outcomes. Integrated care programmes like HCPC may improve well-being of patients, be cost-saving, and contribute to the pursuit of evidence based gradual reforms in the care of frail elderly.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06805-6.

How has implementation been incorporated in health technology assessments in the United Kingdom? A systematic rapid review

OBJECTIVES

Health interventions in a clinical setting may be complex. This is particularly true of clinical interventions which require systems reorganization or behavioural change, and/or when implementation involves additional challenges not captured within a clinical trial setting. Medical Research Council guidance on complex interventions highlights the need to consider economic evaluation alongside implementation. However, the extent to which this guidance has been adhered to, and how, is unclear. The failure to incorporate implementation within the evaluation of an intervention may hinder the translation of research findings into routine practice. This will have consequences for patient care. This study examined the methods used to address implementation within health research conducted through funding from the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme.

METHODS

We conducted a rapid review using a systematic approach. We included all NIHR HTA monographs which contained the word "implementation" within the title or abstract published between 2014 and 2020. We assessed the studies according to existing recommendations for specifying and reporting implementation approaches in research. Additional themes which were not included in the recommendation, but were of particular relevance to our research question, were also identified and summarized in a narrative synthesis.

RESULTS

The extent to which implementation was formally incorporated, and defined, varied among studies. Methods for examining implementation ranged from single stakeholder engagement events to the more comprehensive process evaluation. There was no obvious pattern as to whether approaches to implementation had evolved over recent years. Approximately 50% (22/42) of studies included an economic evaluation. Of these, two studies included the use of qualitative data obtained within the study to quantitatively inform aspects relating to implementation and economic evaluation in their study.

DISCUSSION

A variety of approaches were identified for incorporating implementation within an HTA. However, they did not go far enough in terms of incorporating implementation into the actual design and evaluation. To ensure the implementation of clinically effective and cost-effective interventions, we propose that further guidance on how to incorporate implementation within complex interventions is required. Incorporating implementation into economic evaluation provides a step in this direction.

Feasibility of Measuring Preferences for Chemotherapy Among Early-Stage Breast Cancer Survivors Using a Direct Rank Ordering Multicriteria Decision Analysis Versus a Time Trade-Off

OBJECTIVES

Chemotherapy is increasingly a preference-based choice among women diagnosed with early-stage breast cancer. Multicriteria decision analysis (MCDA) is a promising but underutilized method to facilitate shared decision making. We explored the feasibility of conducting an MCDA using direct rank ordering versus a time trade-off (TTO) to assess chemotherapy choice in a large population-based sample.

METHODS

We surveyed 904 early-stage breast cancer survivors who were within 5 years of diagnosis and reported to the Western Washington State Cancer System and Kaiser Permanente Northern California registries. Direct rank ordering of 11 criteria and TTO surveys were conducted from September 2015 to July 2016; clinical data were obtained from registries or medical records. Multivariable regressions estimated post hoc associations between the MCDA, TTO, and self-reported chemotherapy receipt, considering covariates.

RESULTS

Survivors ranged in age from 25 to 74 years and 73.9% had stage I tumors. The response rate for the rank ordering was 81.0%; TTO score was 94.2%. A one-standard deviation increase in the difference between the chemotherapy and no chemotherapy MCDA scores was associated with a 75.1% (95% confidence interval 43.9-109.7%; p < 0.001) increase in the adjusted odds of having received chemotherapy; no association was found between the TTO score and chemotherapy receipt.

CONCLUSIONS

A rank-order-based MCDA was feasible and was associated with chemotherapy choice. Future research should consider developing and testing this MCDA for use in clinical encounters. Additional research is required to develop a TTO-based model and test its properties against a pragmatic MCDA to inform future shared decision-making tools.

How to integrate evidence from patient preference studies into health technology assessment: a critical review and recommendations

Health technology assessment (HTA) agencies vary in their use of quantitative patient preference data (PP) and the extent to which they have formalized this use in their guidelines. Based on the authors' knowledge of the literature, we identified six different PP "use cases" that integrate PP into HTA in five different ways: through endpoint selection, clinical benefit rating, predicting uptake, input into economic evaluation, and a means to weight all HTA criteria. Five types of insight are distinguished across the use cases: understanding what matters to patients, predicting patient choices, estimating the utility generated by treatment benefits, estimating the willingness to pay for treatment benefits, and informing distributional considerations. Summarizing the literature on these use cases, we recommend circumstances in which PP can add value to HTA and the further research and guidance that is required to support the integration of PP in HTA. Where HTA places more emphasis on clinical outcomes, novel endpoints are available; or where there are already many treatment options, PP can add value by helping decision makers to understand what matters to patients. Where uptake is uncertain, PP can be used to estimate uptake probability. Where indication-specific utility functions are required or where existing utility measures fail to capture the value of treatments, PP can be used to generate or supplement existing utility estimates. Where patients are paying out of pocket, PP can be used to estimate willingness to pay.

Evaluating Complex Health and Social Care Program Using Multi-Criteria Decision Analysis: A Case Study of "Better Together in Amsterdam North"

OBJECTIVES

Multi-criteria decision analysis (MCDA) has been recommended to support policy making in healthcare. However, practical applications of MCDA are sparse. One potential use for MCDA is for the evaluation of programs for complex and vulnerable patients. These complex patients benefit from integrated care programs that span healthcare and social care and aim to improve more than just health outcomes. MCDA can evaluate programs that aim to improve broader outcomes because it allows the evaluation of multiple outcomes alongside each other. In this study, we evaluate an innovative integrated care program in the Netherlands using MCDA.

METHODS

We used an innovative MCDA framework with broad outcomes of health, well-being, and cost to evaluate the Better Together in Amsterdam North (BSiN) program using preferences of patients, partners, providers, payers, and policy makers in the Netherlands. BSiN provides case management support for a period of 6 months. Seven outcomes that previous research has deemed important to complex patients were measured, including physical functioning and social relationships and participation.

RESULTS

We find that the program improved the overall MCDA score marginally, and, thus, after 6 and after 12 months, BSiN was preferred to usual care by all stakeholders. BSiN was preferred to usual care, mostly owing to improvements in psychological well-being and social relationships and participation.

CONCLUSIONS

The integrated healthcare and social care program BSiN in the Netherlands was preferred to usual care according to an MCDA evaluation. MCDA seems a useful method to evaluate complex programs with benefits beyond health.

Case studies for implementing MCDA for tender and purchasing decisions in hospitals in Indonesia and Thailand

BACKGROUND

A multi-criteria decision analysis (MCDA) approach has been suggested for helping purchasers in low- and middle-income countries in an evidence-based assessment of multi-source pharmaceuticals to mitigate potential adverse consequences of price-based decisions on patient access to effective medicines. Six workshops for developing MCDA-instruments for purchasing were conducted in Indonesia, Kazakhstan, Thailand, and Kuwait in 2017-2020. In Indonesia and Thailand, two pilot-initiatives aimed to implement the instruments for hospital drug purchasing decisions.

OBJECTIVE

By analysing and comparing the experiences and progress from the MCDA-workshops and the two case-examples for hospital implementation in Indonesia and Thailand, we aim to gain insights, which will support future implementation.

METHODS

The selection of criteria and their average weight were compared quantitatively across the MCDA-instruments developed in all four countries and settings. Implementation experiences from two case-examples were studied, which included (1) testing the instrument across a variety of drugs in seven hospitals in Thailand and (2) implementation in one specialty hospital in Indonesia. Semi-structured interviews were conducted via web-conferences with four diverse stakeholders in the pilot implementation projects in Thailand and Indonesia. The open responses were evaluated through qualitative content analysis and synthesis using grounded theory coding.

RESULTS

Drivers for implementation were making 'better' decisions, achieving transparency and a rational selection process, reducing drug shortages, and assuring consistent quality. Challenges were seen on the technical level (definition or of criteria, scoring methods, access to data) or change-related challenges (resistance, perception of increased workload, lack of competencies or capabilities, lack of resources). The comparison of the MCDA instruments revealed high similarity, but also clear need for local adaptations in each specific case.

CONCLUSION

A set a of measures targeting challenges related to utility, methodology, data requirements, capacity building and training as well as the broader societal impact can help to overcome challenges in the implementation. Careful planning of implementation and organizational change is recommended for ensuring commitment and fit to local context and culture. Designing a collaborative change program for each application of MCDA-based purchasing will enable healthcare stakeholders to maximally benefit in terms of quality and effectiveness of care and access for patients.

Economic Evaluation of New Models of Care: Does the Decision Change Between Cost-Utility Analysis and Multi-Criteria Decision Analysis?

OBJECTIVES

To experiment with new approaches of collaboration in healthcare delivery, local authorities implement new models of care. Regarding the local decision context of these models, multi-criteria decision analysis (MCDA) may be of added value to cost-utility analysis (CUA), because it covers a wider range of outcomes. This study compares the 2 methods using a side-by-side application.

METHODS

A new Dutch model of care, Primary Care Plus (PC+), was used as a case study to compare the results of CUA and MCDA. Data of patients referred to PC+ or care-as-usual were retrieved by questionnaires and administrative databases with a 3-month follow-up. Propensity score matching together with generalized linear regression models was used to reduce confounding. Univariate and probabilistic sensitivity analyses were performed to explore uncertainty in the results.

RESULTS

Although both methods indicated PC+ as the dominant alternative, complementary differences were observed. MCDA provided additional evidence that PC+ improved access to care (standardized performance score of 0.742 vs 0.670) and that improvement in health-related quality of life was driven by the psychological well-being component (standardized performance score of 0.710 vs 0.704). Furthermore, MCDA estimated the budget required for PC+ to be affordable in addition to preferable (€521.42 per patient). Additionally, MCDA was less sensitive to the utility measures used.

CONCLUSIONS

MCDA may facilitate an auditable and transparent evaluation of new models of care by providing additional information on a wider range of outcomes and incorporating affordability. However, more effort is needed to increase the usability of MCDA among local decision makers.

Program Evaluation of Population- and System-Level Policies: Evidence for Decision Making

BACKGROUND

Policy evaluations often focus on ex post estimation of causal effects on short-term surrogate outcomes. The value of such information is limited for decision making, as the failure to reflect policy-relevant outcomes and disregard for opportunity costs prohibits the assessment of value for money. Further, these evaluations do not always consider all relevant evidence, other courses of action, or decision uncertainty.

METHODS

In this article, we explore how policy evaluation could better meet the needs of decision making. We begin by defining the evidence required to inform decision making. We then conduct a literature review of challenges in evaluating policies. Finally, we highlight potential methods available to help address these challenges.

RESULTS

The evidence required to inform decision making includes the impacts on the policy-relevant outcomes, the costs and associated opportunity costs, and the consequences of uncertainty. Challenges in evaluating health policies are described using 8 categories: 1) valuation space; 2) comparators; 3) time of evaluation; 4) mechanisms of action; 5) effects; 6) resources, constraints, and opportunity costs; 7) fidelity, adaptation, and level of implementation; and 8) generalizability and external validity. Methods from a broad set of disciplines are available to improve policy evaluation, relating to causal inference, decision-analytic modeling, theory of change, realist evaluation, and structured expert elicitation.

LIMITATIONS

The targeted review may not identify all possible challenges, and the methods covered are not exhaustive.

CONCLUSIONS

Evaluations should provide appropriate evidence to inform decision making. There are challenges in evaluating policies, but methods from multiple disciplines are available to address these challenges.

IMPLICATIONS

Evaluators need to carefully consider the decision being informed, the necessary evidence to inform it, and the appropriate methods.[Box: see text].

Geospatial modelling to estimate the territory at risk of establishment of influenza type A in Mexico - An ecological study

The aim of this study was to estimate the territory at risk of establishment of influenza type A (EOITA) in Mexico, using geospatial models. A spatial database of 1973 outbreaks of influenza worldwide was used to develop risk models accounting for natural (natural threat), anthropic (man-made) and environmental (combination of the above) transmission. Then, a virus establishment risk model; an introduction model of influenza A developed in another study; and the three models mentioned were utilized using multi-criteria spatial evaluation supported by geographically weighted regression (GWR), receiver operating characteristic analysis and Moran's I. The results show that environmental risk was concentrated along the Gulf and Pacific coasts, the Yucatan Peninsula and southern Baja California. The identified risk for EOITA in Mexico were: 15.6% and 4.8%, by natural and anthropic risk, respectively, while 18.5% presented simultaneous environmental, natural and anthropic risk. Overall, 28.1% of localities in Mexico presented a High/High risk for the establishment of influenza type A (area under the curve=0.923, P<0.001; GWR, r2=0.840, P<0.001; Moran's I =0.79, P<0.001). Hence, these geospatial models were able to robustly estimate those areas susceptible to EOITA, where the results obtained show the relation between the geographical area and the different effects on health. The information obtained should help devising and directing strategies leading to efficient prevention and sound administration of both human and financial resources.

How to Value Orphan Drugs? A Review of European Value Assessment Frameworks

Background: Decision-makers have implemented a variety of value assessment frameworks (VAFs) for orphan drugs in European jurisdictions, which has contributed to variations in access for rare disease patients. This review provides an overview of the strengths and limitations of VAFs for the reimbursement of orphan drugs in Europe, and may serve as a guide for decision-makers.

Methods: A narrative literature review was conducted using the databases Pubmed, Scopus and Web of Science. Only publications in English were included. Publications known to the authors were added, as well as conference or research papers, or information published on the website of reimbursement and health technology assessment (HTA) agencies. Additionally, publications were included through snowballing or focused searches.

Results: Although a VAF that applies a standard economic evaluation treats both orphan drugs and non-orphan drugs equally, its focus on cost-effectiveness discards the impact of disease rarity on data uncertainty, which influences an accurate estimation of an orphan drug’s health benefit in terms of quality-adjusted life-years (QALYs). A VAF that weighs QALYs or applies a variable incremental cost-effectiveness (ICER) threshold, allows the inclusion of value factors beyond the QALY, although their methodologies are flawed. Multi-criteria decision analysis (MCDA) incorporates a flexible set of value factors and involves multiple stakeholders’ perspectives. Nevertheless, its successful implementation relies on decision-makers’ openness toward transparency and a pragmatic approach, while allowing the flexibility for continuous improvement.

Conclusion: The frameworks listed above each have multiple strengths and weaknesses. We advocate that decision-makers apply the concept of accountability for reasonableness (A4R) to justify their choice for a specific VAF for orphan drugs and to strive for maximum transparency concerning the decision-making process. Also, in order to manage uncertainty and feasibility of funding, decision-makers may consider using managed-entry agreements rather than implementing a separate VAF for orphan drugs.

Fostering Responsible Innovation in Health: An Evidence-Informed Assessment Tool for Innovation Stakeholders

BACKGROUND

Responsible innovation in health (RIH) emphasizes the importance of developing technologies that are responsive to system-level challenges and support equitable and sustainable healthcare. To help decision-makers identify whether an innovation fulfills RIH requirements, we developed and validated an evidence-informed assessment tool comprised of 4 inclusion and exclusion criteria, 9 assessment attributes and a scoring system.

METHODS

We conducted an inter-rater reliability assessment to establish the extent to which 2 raters agree when applying the RIH Tool to a diversified sample of health innovations (n=25). Following the Tool's 3-step process, sources of information were collected and cross-checked to ensure their clarity and relevance. Ratings were reported independently in a spreadsheet to generate the study's database. To measure inter-rater reliability, we used: a non-adjusted index (percent agreement), a chance-adjusted index (Gwet's AC) and the Pearson's correlation coefficient. Results of the Tool's application to the whole sample of innovations are summarized through descriptive statistics.

RESULTS

Our findings show complete agreement for the screening criteria, "almost perfect" agreement for 7 assessment attributes, "substantial" agreement for 2 attributes and "almost perfect" agreement for the RIH overall score. A large portion of the sample obtained high scores for 6 attributes (health relevance, health inequalities, responsiveness, level and intensity of care and frugality) and low scores for 3 attributes (ethical, legal, and social issues [ELSIs], inclusiveness and eco-responsibility). At the rating step, 88% of the innovations had a sufficient number of attributes documented (≥ 7/9), but the assessment was based on sources of moderate to high quality (mean score ≥ 2 points) for 36% of the sample. While "Almost all RIH features" were present for 24% of the innovations (RIH mean score between 4.1-5.0 points), "Many RIH features" were present for 52% of the sample (3.1-4.0 points) and "Few RIH features" were present for 24% of the innovations (2.1-3.0 points).

CONCLUSION

By confirming key aspects of the RIH Tool's reliability and applicability, our study brings its development to completion. It can be jointly put into action by innovation stakeholders who want to foster innovations with greater social, economic and environmental value.

Multiple criteria decision analysis for medicine reimbursement in the Lebanese context

Objective

The objective of this exploratory analysis is to reflect and discuss which criteria of multiple criteria decision analysis (MCDA) would be relevant as part of value determination when appraising healthcare interventions in the Lebanese context.

Methods

A workshop was conducted as part of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Lebanon Chapter and included the two frameworks: Evidence and Value: Impact on Decision Making (EVIDEM) and Advance Value Framework. Thirty-seven participants expressed their individual preferences through a qualitative and a quantitative exercise.

Results

In the qualitative analysis of both frameworks, participants unanimously agreed on the relevance of comparative efficacy, safety, and impact of medical costs. In EVIDEM, disease severity and unmet needs were also considered to be important criteria by more than 90 percent of the participants. In the quantitative analysis of both frameworks, disease severity ranked first (a mean normalized weight of .1 in EVIDEM and .27 in Advance Value Framework), followed by the size of the population (.09), the type of therapeutic benefit at the patient level (.09) and population level (.08), and the efficacy (.07) in EVIDEM. In the Advance Value Framework, the combined unmet need/disease severity criteria were followed by direct and meaningful end points (.15), safety (.12), contraindications (.08), and indirect surrogate end points (.07).

Conclusions

The results were concordant with those reported in countries that have conducted similar surveys such as France, Italy, and Spain. The MCDA methodology could be used as a cornerstone to enhance evidence-based discussions among Lebanese stakeholders involved in evaluation and decision-making purposes.

HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies

This last decade has been marked by significant advances in the development of cell and gene (C&G) therapies, such as gene targeting or stem cell-based therapies. C&G therapies offer transformative benefits to patients but present a challenge to current health technology decision-making systems because they are typically reviewed when clinical efficacy data are very limited and when there is uncertainty about the long-term durability of outcomes. These challenges are not unique to C&G therapies, but they face more of these barriers, reflecting the need for adapting existing value assessment frameworks. Still, C&G therapies have the potential to be cost-effective even at very high price points. The impact on healthcare budgets will depend on the success rate of pipeline assets and on the extent to which C&G therapies will expand to wider pathologies beyond rare or ultra-rare diseases. Getting pricing and reimbursement models right is important for incentivising research and development investment while not jeopardising the sustainability of healthcare systems. Payers and manufacturers therefore need to acknowledge each other's constraints-limitations in the evidence generation on the manufacturer side, budget considerations on the payer side-and embrace innovative thinking and approaches to ensure timely delivery of therapies to patients. Several experts in health technology assessment and clinical experts have worked together to produce this publication and identify methodological and policy options to improve the assessment of C&G therapies, and make it happen better, faster and sustainably in the coming years.

Overcoming Challenges with the Adoption of Point-of-Care Testing: From Technology Push and Clinical Needs to Value Propositions

Major technical challenges often prevent developers from producing new point-of-care technologies that deliver the required clinical performance in the intended settings of use. But even when devices meet clinical requirements, they can fail to be adopted and successfully implemented. Adoption barriers occur when decision makers do not understand the "value proposition" of new technologies. Current discussions of value in the context of point-of-care testing focus predominantly on the intended use and performance of the device from the manufacturer's point-of-view. However, the perspective of potential adopters in determining whether new devices provide value is also important, as is the opinion of all stakeholders who will be impacted. Incorporating value concepts into decisions made across the full development-to-adoption continuum can increase the likelihood that point-of-care testing will have the desired impact on health care delivery and patient outcomes. This article discusses how various approaches to technology development impact adoption and compares the characteristics of these approaches to emerging value concepts. It also provides an overview of value initiatives and tools that are being developed to support the evaluation of value propositions. These are presented for a range of technology adoption decision contexts, with particular applicability to point-of-care testing. Expanding the focus of research to address gaps in both the creation and evaluation of value propositions is imperative in order for value concepts to positively influence the adoption of point-of-care testing.

Potential Criteria for Frameworks to Support the Evaluation of Innovative Medicines in Upper Middle-Income Countries-A Systematic Literature Review on Value Frameworks and Multi-Criteria Decision Analyses

Background

Multicriteria Decision Analysis (MCDA), a formal decision support framework, has been growing in popularity recently in the field of health care. MCDA can support pricing and reimbursement decisions on the macro level, which is of great importance especially in countries with more limited resources.

Objectives

The aim of this systematic review was to facilitate the development of future MCDA frameworks, by proposing a set of criteria focusing on the purchasing decisions of single-source innovative pharmaceuticals in upper middle-income countries.

Methods

A systematic literature review was conducted on the decision criteria included in value frameworks (VFs) or MCDA tools. Scopus, Medline, databases of universities, websites of Health Technology Assessment Agencies, and other relevant organizations were included in the search. Double title-abstract screening and double full-text review were conducted, and all extracted data were double-checked. A team of researchers performed the merging and selection process of the extracted criteria.

Results

A total of 1,878 articles entered the title and abstract screening. From these, 341 were eligible to the full-text review, and 36 were included in the final data extraction phase. From these articles 394 criteria were extracted in total. After deduplication and clustering, 26 different criteria were identified. After the merging and selection process, a set of 16 general criteria was proposed.

Conclusion

Based on the results of the systematic literature review, a pool of 16 criteria was selected. This can serve as a starting point for constructing MCDA frameworks in upper middle-income countries after careful adaptation to the local context.

Early Health Technology Assessment during Nonalcoholic Steatohepatitis Drug Development: A Two-Round, Cross-Country, Multicriteria Decision Analysis

Background. The assessment of value along the clinical development of new biopharmaceutical compounds is a challenging task. Complex and uncertain evidence has to be analyzed, considering a multitude of value preferences from different stakeholders. Objective. To investigate the use of multicriteria decision analysis (MCDA) to support decision making during drug development while considering payer and health technology assessment (HTA) value concerns, by applying the Advance Value Framework in nonalcoholic steatohepatitis (NASH) and testing for the consistency of the results. Design. A multiattribute value theory methodology was applied and 2 rounds of decision conferences (DCs) were organized in 3 countries (England, France, and Germany), with the participation of national key experts and stakeholders using the MACBETH questioning protocol and algorithm. A total of 51 health care professionals, patient advocates, and methodologists, including (ex-) committee members or assessors from national HTA bodies, participated in 6 DCs in the study countries. Target Population. NASH patients in fibrosis stages F2 to 3 were considered. Interventions. The value of a hypothetical product profile was assessed against 3 compounds under development using their phase 2 results. Outcome Measures. DC participants' value preferences were elicited involving criteria selection, options scoring, and criteria weighting. Results. Highly consistent valuation rankings were observed in all DCs, always favoring the same compound. Highly consistent rankings of criteria clusters were observed, favoring therapeutic benefit criteria, followed by safety profile and innovation level criteria. Limitations. There was a lack of comparative treatment effects, early evidence on surrogate endpoints was used, and stakeholder representativeness was limited in some DCs. Conclusions. The use of MCDA is promising in supporting early HTA, illustrating high consistency in results across countries and between study rounds.