How to use multi-criteria decision analysis methods for reimbursement decision-making in healthcare: a step-by-step guide

Development of a Value Assessment Framework for Pediatric Health Technologies Using Multicriteria Decision Analysis: Expanding the Value Lens for Funding Decision Making

OBJECTIVES

A health technology assessment (HTA) does not systematically account for the circumstances and needs of children and youth. To supplement HTA processes, we aimed to develop a child-tailored value assessment framework using a multicriteria decision analysis approach.

METHODS

We constructed a multicriteria-decision-analysis-based model in multiple phases to create the Comprehensive Assessment of Technologies for Child Health (CATCH) framework. Using a modified Delphi process with stakeholders having broad disciplinary and geographic variation (N = 23), we refined previously generated criteria and developed rank-based weights. We established a criterion-pertinent scoring rubric for assessing incremental benefits of new drugs. Three clinicians independently assessed comprehension by pilotscoring 9 drugs. We then validated CATCH for 2 childhood cancer therapies through structured deliberation with an expert panel (N = 10), obtaining individual scores, consensus scores, and verbal feedback. Analyses included descriptive statistics, thematic analysis, exploratory disagreement indices, and sensitivity analysis.

RESULTS

The modified Delphi process yielded 10 criteria, based on absolute importance/relevance and agreed importance (median disagreement indices = 0.34): Effectiveness, Child-specific Health-related Quality of Life, Disease Severity, Unmet Need, Therapeutic Safety, Equity, Family Impacts, Life-course Development, Rarity, and Fair Share of Life. Pilot scoring resulted in adjusted criteria definitions and more precise score-scaling guidelines. Validation panelists endorsed the framework's key modifiers of value. Modes of their individual prescores aligned closely with deliberative consensus scores.

CONCLUSIONS

We iteratively developed a value assessment framework that captures dimensions of child-specific health and nonhealth gains. CATCH could improve the richness and relevance of HTA decision making for children in Canada and comparable health systems.

Economic evaluation in cardiac electrophysiology: Determining the value of emerging technologies

Cardiac electrophysiology is a constantly evolving speciality that has benefited from technological innovation and refinements over the past several decades. Despite the potential of these technologies to reshape patient care, their upfront costs pose a challenge to health policymakers who are responsible for the assessment of the novel technology in the context of increasingly limited resources. In this context, it is critical for new therapies or technologies to demonstrate that the measured improvement in patients' outcomes for the cost of achieving that improvement is within conventional benchmarks for acceptable health care value. The field of Health Economics, specifically economic evaluation methods, facilitates this assessment of value in health care. In this review, we provide an overview of the basic principles of economic evaluation and provide historical applications within the field of cardiac electrophysiology. Specifically, the cost-effectiveness of catheter ablation for both atrial fibrillation (AF) and ventricular tachycardia, novel oral anticoagulants for stroke prevention in AF, left atrial appendage occlusion devices, implantable cardioverter defibrillators, and cardiac resynchronization therapy will be reviewed.

Assessing educational methods for tomorrow's supply chain leaders with the integration of skill development priorities: a fuzzy decision-making approach

Purpose

Effective educational methods are critical for successfully training future supply chain talent. The paper proposes a fuzzy multi-criteria decision-making model to evaluate and select the best educational method for tomorrow's supply chain leaders integrating skill development priorities in an uncertain environment.

Design/methodology/approach

The Grounded theory scheme is used to identify SC leaders' skillsets criteria and educational method alternatives. Fuzzy step-wise weight assessment ratio analysis sets the priority and determines the weight of 17 criteria. Eight decision-makers evaluate 13 alternatives using fuzzy linguistic terms. Fuzzy technique for order preference by similarity to ideal solution ranks and shows the most effective educational method. Sensitivity analysis presents the applicability of this study.

Findings

Its implementation in a university-industry collaboration case in Brazil, Mentored learning from industry experts is the best educational method. The skill development priorities are data analytics ability, end-to-end supply chain vision and problem-solving. Technical skills are the most important criteria that influence the selection of the optimal option and educational methods related to learning from others rank in the top teaching pool, including multidisciplinary cross-cultural training.

Originality/value

This paper is among the first to evaluate educational methods with skill development priorities integration for supply chain students using fuzzy SWARA–fuzzy TOPSIS. It provides actionable insights: a decision-making procedure for educational method selection, a broad skills profile for supply chain professional success and educational methods that professors can bring to in classroom/virtual environment.

How innovation can be defined, evaluated and rewarded in health technology assessment

BACKGROUND

What constitutes innovation in health technologies can be defined and measured in a number of ways and it has been widely researched and published about. However, while many countries mention it as a criterion for pricing or reimbursement of health technologies, countries differ widely in how they define and operationalise it.

METHODS

We performed a literature review, using a snowballing search. In this paper, we explore how innovation has been defined in the literature in relation to health technology assessment. We also describe how a selection of countries (England, France, Italy, Spain and Japan) take account of innovation in their health technology assessment frameworks and explore the key methodologies that can capture it as a dimension of value in a new health technology. We propose a way of coming to, and incorporating into health technology assessment systems, a definition of innovation for health technologies that is independent of other dimensions of value that they already account for in their systems, such as clinical benefit. We use Spain as an illustrative example of how innovation might be operationalised as a criterion for decision making in health technology assessment.

RESULTS

The countries analysed here can be divided into 2 groups with respect to how they define innovation. France, Japan and Italy use features such as severity, unmet need and therapeutic added value as indicators of the degree of innovation of a health technology, while England, Spain consider the degree of innovation as a separate and additional criterion from others. In the case of Spain, a notion of innovation might be constructed around concepts of `step-change', `convenience', `strength of evidence base' and `impact on future research & development'.

CONCLUSIONS

If innovation is to be used as operational criteria for adoption, pricing and reimbursement of health technologies, the concept must be clearly defined, and it ought to be independent from other value dimensions already captured in their health technology assessment systems.

Development of a multi-criteria decision-making-based assessment model for dental material selection: Engine-driven nickel-titanium instruments case study

OBJECTIVES

The aims of this study are (i) to propose specific selection criteria related to NiTi instruments for dental practitioners and (ii) to objectively assess the NiTi instruments.

MATERIALS AND METHODS

The steps of the methodology are as follows: Step 1: "Delphi method" was employed to reach a consensus on criteria defined according to the literature review and a group of panelists. Step 2: "Smart pairwise comparisons" were employed to rank the proposed criteria. Step 3: "Borda voting" was employed to determine the weights of the proposed criteria. Step 4: To determine assessment scores, "Simple Additive Weighting" was employed. Step 5: Reliability and validity checks were made by "simulation."

RESULTS

Specific criteria classified under dimensions were proposed and weighted for the NiTi instrument assessment. In this context, an assessment model was proposed and validated.

CONCLUSIONS

The proposed assessment model for NiTi instruments could aid to make the decision-making process as systematic, transparent, and reproducible as possible not only for dental practitioners but also for healthcare professionals. Also, this proposed model can represent a reference framework for further MCDM studies which can rank or classify materials in medical science.

CLINICAL RELEVANCE

The model proposed in this study can be used to aid decision-making in clinical practice by means assessing the NiTi instrumentation system alternatives for practitioners.

Dynamic evaluation method for planning sustainable landfills using GIS and multi-criteria in areas of urban sprawl with land-use conflicts

Landfill site selection is problematic in many countries, especially developing nations where there is rapid population growth, which leads to high levels of inadequate waste disposal. To find sustainable landfill sites in sprawling cities, this study presents an approach that combines geographic information system (GIS) with multi-criteria (social, environmental and, technical criteria) and the population growth projection. The greater Maputo area in Mozambique was selected as a representative city for the study, which is undergoing rapid urbanization. Six criteria, i.e., land use, transport networks, hydrology, conservation reserve, geology and slope, were considered and overlaid in the GIS using an analytic hierarchy process (AHP). The arithmetic projection of the population trend suggests that the greater Maputo area is experiencing a rapid and uncontrolled population growth, especially in Matola city. These pronounced changes in population then significantly change the landfill placement decision making. Dynamic and static scenarios were created, based on the analysis of multi-criteria and the areas likely to undergo future increased population growth. A comparative evaluation in a scenario of dynamic behavior considering future population showed that suitable areas for landfill sites have been drastically modified due to social and environmental factors affected by population distribution in some regions. The results indicate that some suitable areas can generate land use conflicts due to population growth with unplanned land use expansion. Finally, the western part of Matola city is recognized as the most recommendable landfill site, which can serve both Maputo and Matola city with affordable costs. This study provides an effective landfill placement decision making approach, which is possible to be applied anywhere, especially in developing countries to improve sustainable municipal solid waste management systems.

The Inflection Point Model: a Model to Explore the Hidden Burdens of Non-Cancerous Genitourinary Conditions

OBJECTIVE

To propose a conceptual model to identify points along the condition course where actions or inaction affect downstream burdens of non-cancerous genitourinary conditions (NCGUC).

MATERIALS AND METHODS

The National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) convened an interdisciplinary meeting to comprehensively consider the burdens of NCGUCs. Subsequently, the authors met monthly to conceptualize the model.

RESULTS

Inflection points (IP) describe time points during a condition course that are sensitive to change. Our proposed Inflection Point Model (IPM) helps conceptualize burden/benefit trade-offs in any related decision and provides a platform to identify the downstream aggregate burden of a NCGUC across multiple socio-ecological levels at a single time point, which may be summed across the condition course to measure cumulative burden. Two personae demonstrate the utility of this model to better understand impacts of two common NCGUCs.

CONCLUSIONS

The IPM may be applied in multiple contexts: narrowly to explore burden of a single NCGUC at a single IP; or more broadly, to address multiple conditions, multiple IPs, or multiple domains/levels of social ecology. Applying the IPM may entail combining population data describing prevalence of NCGUCs, associated behaviors, and resulting outcome patterns that can be combined with suitable mathematical models to quantify aggregate and cumulative burden. The IPM challenges stakeholders to expand from the individual to include broader levels of social ecology. Application of the IPM will undoubtedly identify data gaps and research needs that must be fulfilled to delineate and address the burden of NCGUCs.

Performance of Fuzzy Multi-Criteria Decision Analysis of Emergency System in COVID-19 Pandemic. An Extensive Narrative Review

The actual coronavirus disease 2019 (COVID-19) pandemic has led to the limit of emergency systems worldwide, leading to the collapse of health systems, police, first responders, as well as other areas. Various ways of dealing with this world crisis have been proposed from many aspects, with fuzzy multi-criteria decision analysis being a method that can be applied to a wide range of emergency systems and professional groups, aiming to confront several associated issues and challenges. The purpose of this critical review was to discuss the basic principles, present current applications during the first pandemic wave, and propose future implications of this methodology. For this purpose, both primary sources, such as scientific articles, and secondary ones, such as bibliographic indexes, web pages, and databases, were used. The main search engines were PubMed, SciELO, and Google Scholar. The method was a systematic literature review of the available literature regarding the performance of the fuzzy multi-criteria decision analysis of emergency systems in the COVID-19 pandemic. The results of this study highlight the importance of the fuzzy multi-criteria decision analysis method as a beneficial tool for healthcare workers and first responders’ emergency professionals to face this pandemic as well as to manage the created uncertainty and its related risks.

Priority setting in the Brazilian emergency medical service: a multi-criteria decision analysis (MCDA)

BACKGROUND

Despite the proven value of multicriteria decision analysis in the health field, there is a lack of studies focused on prioritising victims in the Emergency Medical Service, EMS. With this, and knowing that the decision maker needs a direction on which choice may be the most appropriate, based on different and often conflicting criteria. The current work developed a new model for prioritizing victims of SAMU/192, based on the multicriteria decision methodology, taking into account the scarcity of resources.

METHODS

An expert panel and a discussion group were formed, which defined the limits of the problem, and identified the evaluation criteria for choosing a victim, amongst four alternatives illustrated from hypothetical scenarios of emergency situations-clinical and traumatic diseases of absolute priority. For prioritization, an additive mathematical method was used that aggregates criteria in a flexible and interactive version, FITradeoff.

RESULTS

The structuring of the problem led the researchers to identify twenty-five evaluation criteria, amongst which ten were essential to guide decisions. As a result, in the simulation of prioritization of four requesting victims in view of the availability of only one ambulance, the proposed model supported the decision by suggesting the prioritization of one of the victims.

CONCLUSIONS

This work contributed to the prioritization of victims using multicriteria decision support methodology. Selecting and weighing the criteria in this study indicated that the protocols that guide regulatory physicians do not consider all the criteria for prioritizing victims in an environment of scarcity of resources. Finally, the proposed model can support crucial decision based on a rational and transparent decision-making process that can be applied in other EMS.

Reconciling ACEA and MCDA: is there a way forward for measuring cost-effectiveness in the U.S. healthcare setting?

Background

The ISPOR Special Task Force (STF) on US Value Assessment Frameworks was agnostic about exactly how to implement the quality-adjusted life year (QALY) as a key element in an overall cost-effectiveness evaluation. But the STF recommended using the cost-per-QALY gained as a starting point in deliberations about including a new technology in a health plan benefit. The STF offered two major alternative approaches—augmented cost-effectiveness analysis (ACEA) and multi-criteria decision analysis (MCDA)—while emphasizing the need to apply either a willingness-to-pay (WTP) or opportunity cost threshold rule to operationalize the inclusion decision.

Methods

The MCDA model uses the multi-attribute utility function. The ACEA model is based on the expected utility theory. In both ACEA and MCDA models, value trade-offs are derived in a hierarchical model with two high-level objectives which measure overall health gain separately from financial attributes affecting consumption.

Results

Even though value trade-offs can be elicited or revealed without considering budget constraints, we demonstrate that they can be used similarly to WTP-based cost-effectiveness thresholds for resource allocation decisions. The consideration of how costs of medical technology, income, and severity of disease affect value trade-offs demonstrates, however, that reconciling decisions in ACEA and MCDA requires that health and consumption are either complements or independent attributes.

Conclusions

We conclude that value trade-offs derived either from ACEA or MCDA move similarly with changes in main factors considered by enrollees and decision makers—costs of the medical technology, income, and severity of disease. Consequently, this complementarity between health and consumption is a necessary condition for reconciling ACEA and MCDA. Moreover, their similarity would be further enhanced if the QALY is used as the key attribute or anchor in the MCDA value function: the choice between the two is a pragmatic question that is still open.

Selecting Image-Guided Surgical Technologies in Oncology: A Surgeon's Perspective

BACKGROUND

To improve surgical performance, image-guided (IG) technologies are increasingly introduced. Yet, it is unknown which oncological procedures yield most value from these technologies. This study aimed to select the most promising IG technology per oncologic indication.

METHODS

An Analytic Hierarchical Process was used to evaluate three IG technologies: navigation, optical imaging, and augmented reality, in five oncologic indications compared with usual care. Sixteen decision criteria were selected. The relative importance of the criteria and the expected performance of the technologies were evaluated among surgeons. The combination of these scores gives the expected value per technology.

RESULTS

On criteria level, sparing critical tissue (9%-18%) and reducing the risk of local recurrence (11%-27%) were most important. Navigation was preferred in three indications-removal of lymph nodes (42%), liver (47%), and rectal tumors (33%). In removing rectal tumors, optical imaging was equally preferred (34%). In removing breast and tongue tumors, no technology was clearly preferred.

CONCLUSIONS

In selecting IG technologies, especially optical and navigation technologies are expected to add value in addition to usual care. Further development of those technologies for the preferred indications seems valuable. Multi-attribute analysis showed to be useful in prioritization of conducting clinical studies and steer research and development initiatives.

A New Decision Model Approach for Health Technology Assessment and A Case Study for Dialysis Alternatives in Turkey

Background: This paper presents a generic Multi-Criteria Decision Analysis (MCDA) model for Health Technology Assessment (HTA) decision-making, which can be applied to a wide range of HTA studies, regardless of the healthcare technology type under consideration. Methods: The HTA Core Model^® of EUnetHTA was chosen as a basis for the development of the MCDA model because of its common acceptance among European Union countries. Validation of MCDA4HTA was carried out by an application with the HTA study group of the Turkish Ministry of Health. The commitment of the decision-making group is completed via an online application of 10 different questionnaires. The Analytic Hierarchy Process (AHP) is used to determine the weights. Scores of the criteria in MCDA4HTA are gathered directly from the HTA report. The performance matrix in this application is run with fuzzy Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), fuzzy Vise Kriterijumska Optimizacija I Kompromisno Resenje (VIKOR), and goal programming MCDA techniques. Results: Results for fuzzy VIKOR, fuzzy TOPSIS, and goal programming are 0.018, 0.309, and 0.191 for peritoneal dialysis and 0.978, 0.677, and 0.327 for hemodialysis, respectively. Conclusions: Peritoneal dialysis is found to be the best choice under the given circumstances, despite its higher costs to society. As an integrated decision-making model for HTA, MCDA4HTA supports both evidence-based decision policy and the transparent commitment of multi-disciplinary stakeholders.

Assessing the Preferences for Criteria in Multi-Criteria Decision Analysis in Treatments for Rare Diseases

Background: Increasingly, multi-criteria decision analysis has gained importance as a method by which to assess the value of orphan drugs. However, very little attention has been given to the weight (relative preferences) of the individual criteria used in a framework. Aims: This study sought to gain an understanding of the preferential weights that should be allocated in a multi-criteria decision analysis framework for orphan drugs, from a multi-stakeholder perspective. Method: Using key MCDA criteria for orphan drugs reported in the literature, we developed an interactive web-based survey tool to capture preferences for different criteria from a general stakeholder sample who were requested to assign weights from a reimbursement perspective. Each criterion could be assigned a weight on a sliding scale from 0 to 100% as long as the sum of all the criteria was 100%. We subsequently used the interactive tool with an expert focus group, followed up with a group discussion regarding each criterion and their perspectives on the weight that each criterion should be allocated when assessing an orphan drug. The expert focus group participants were then able to adjust their weights, if the group discussion had changed their perspectives. Results: The interactive tool was completed by 120 general stakeholder sample from a wide range of countries and professional backgrounds and an expert focus group of ten members. The results showed the differences in perspectives on the importance of criteria. Both groups considered Treatment efficacy to be the most important criterion. The general stakeholder sample weighted Treatment safety at 12.03% compared to the expert focus group's average of 20%. The results also demonstrated the value of the group discussion, which provided additional insights into the perspectives on the importance of criteria in assessing orphan drugs. Conclusion: This study aimed to contribute to the important aspect of preferences for different criteria in MCDA. This study sheds light on the important aspect of the preferences of the different criteria. All respondents agreed on the relative importance of Treatment efficacy and Treatment safety, criteria that are captured in conventional cost-effectiveness studies, but they also expressed the view that in addition to those, several disease-related and drug-related criteria should be included in MCDA frameworks for assessing orphan drugs.

Multi-criteria analysis in the health area: selection of the most appropriate triage system for the emergency care units in natal

Background

Multiobjective decision-making processes present a high degree of complexity in their solution, and tools such as multicriteria decision analysis appear as a way to facilitate the decision-makers’ solution and ensure that the decision is made cohesively and efficiently. In the public health sector, decisions are even more delicate because they work not only with the direct influence of human needs, but also with limited financial resources. An important point for the emergency care units is the triage system, which consists of a pre-evaluation of the patients, classifying them according to the degree of life risk. Through triage, the patient can be attended more quickly and efficiently, streamlining the whole process. Thus, the present research endeavored to determine the most appropriate triage protocol for emergency healthcare units in Natal-RN city in Brazil and may help others less advanced countries to determine the most appropriate triage protocol for emergency healthcare.

Methods

In this study, we used the multicriteria analysis method known as FITradeoff. In addition, interviews and structured questionnaires applied with nurses, specialists and directors.

Results

Based on the questionnaires and preferences presented by the decision-makers, the Spanish Triage System was the most suitable protocol for the emergency care units, which presented with high ease of use and implementation.

Conclusions

This study reached its main objective, which was to determine the most appropriate triage protocol. In addition, it was observed the possibility of new research, such as the development of a specific protocol for this emergency care units and the creation of an application software for this new protocol.

PARS, a system combining semantic technologies with multiple criteria decision aiding for supporting antibiotic prescriptions

OBJECTIVE

Motivated by the well documented worldwide spread of adverse drug events, as well as the increased danger of antibiotic resistance (caused mainly by inappropriate prescribing and overuse), we propose a novel recommendation system for antibiotic prescription (PARS).

METHOD

Our approach is based on the combination of semantic technologies with MCDA (Multiple Criteria Decision Aiding) that allowed us to build a two level decision support model. Given a specific domain, the approach assesses the adequacy of an alternative/action (prescription of antibiotic) for a specific subject (patient) with an issue (bacterial infection) in a given context (medical). The goal of the first level of the decision support model is to select the set of alternatives which have the potential to be suitable. Then the second level sorts the alternatives into categories according to their adequacy using an MCDA sorting method (MR-Sort with Veto) and a structured set of description logic queries.

RESULTS

We applied this approach in the domain of antibiotic prescriptions, working closely with the EpiCura Hospital Center (BE). Its performance was compared to the EpiCura recommendation guidelines which are currently in use. The results showed that the proposed system is more consistent in its recommendations when compared with the static EpiCura guidelines. Moreover, with PARS the antibiotic prescribing workflow becomes more flexible. PARS allows the user (physician) to update incrementally and dynamically a patient's profile with more information, or to input knowledge modifications that accommodate the decision context (like the introduction of new side effects and antibiotics, the development of germs that are resistant, etc). At the end of our evaluation, we detail a number of limitations of the current version of PARS and discuss future perspectives.

Hospital-based Health Technology Assessment in Kazakhstan: 3 years' experience of one unit

OBJECTIVES

The aim of this study was to describe the development and activities of the Hospital-Based Health Technology Assessment (HB-HTA) Unit in the Hospital of the President's Affairs Administration, one of the first examples of the implementation of HB-HTA into the practice of Kazakhstani hospitals.

METHODS

Details of the development of the Unit were obtained from the hospital's administrative records. The Unit's own records were used to describe the reports prepared and the clinical areas that were covered. Responses to recommendations in the Unit's reports were obtained from hospital administration and individual departments. Estimates of savings and payback periods were based on data from the hospital information system, and data submitted by manufacturers and distributors of medical equipment.

RESULTS

Fifty-one rapid- and mini-HTA reports were prepared by the Unit from 2015 to 2017. Seventeen health technologies (33 percent) were not recommended for implementation in hospital practice. Refusal to implement sixteen of these technologies saved approximately 1,053,500 USD. Of the thirty-four recommended health technologies, twenty-four were implemented to treat or diagnose 1,376 patients, and eight others were included in plans for 2018-20. Of the twenty-four implemented health technologies, twelve did not require additional investments. The payback period of investments for the other twelve implemented technologies is not more than 3 years for six, less than 5 years for four, and more than 10 years for two technologies.

CONCLUSIONS

Establishment of the HB-HTA Unit in the hospital created the basis for making informed managerial decisions; identifying key directions for strategic development; and improving hospital management.

Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps

Background

The health area is one of the most affected systems on the perspective of decision-making with multiobjectives, thus becoming prone to errors in the final solution, however, multicriteria decision analysis (MDCA) appears as an aid tool for this process decision-making. Therefore,the present study aims to analyze and synthesize articles found in the literature, involing MCDA in health care, evaluating general issues and methodological aspects, structuring them in a single work.

Methods

Surveys in the bibliographic databases SCOPUS and PUBMED indicated 1852 documents on the subject, however after a careful verificatios, 66 studies were selected to be analyzed completely. The data extracted from the included articles were organized into a spreadsheet for the preparation of analysis, and the technique used was descriptive statistics.

Results

It was possible to identify a growth trend in the application of the MCDA in the health area, but no dominance was identified in relation to the authors of the publication and the periodicals where they are published, but some countries stood out in terms of the number of published researches, such as: Canada and Turkey. In defining the decision problem, and in defining criteria, the “literature” presented the greatest demand for those who wish to structure their decision problem. Finally, it was verified by the analysis of the problem, that the MCDA to solve the problems of ranking has comprehensive application and that there is a greater incidence in the use of the AHP and Logic methods Fuzzy.

Conclusion

With this, it is possible to observe, through the data of this review, that more than the multicriteria methods, the multicriteria decision model has been highlighted, also in the health area. In addition, the study can guide new applications and techniques using MCDA in the health care.

Revealed preferences towards the appraisal of orphan drugs in Poland - multi criteria decision analysis

Background

A Multi Criteria Decision Analysis (MCDA) technique was adopted to reveal the preferences of the Appraisal Body of the Polish HTA agency towards orphan drugs (OMPs).

Results

There were 34 positive and 23 negative HTA recommendations out of 54 distinctive drug-indication pairs. The MCDA matrix consisted of 13 criteria, seven of which made the most impact on the HTA process. Appraisal of clinical evidence, cost of therapy, and safety considerations were the main contributors to the HTA guidance, whilst advancement of technology and manufacturing costs made the least impact.

Conclusions

MCDA can be regarded as a valuable tool for revealing decision makers’ preferences in the healthcare sector. Given that only roughly half of all criteria included in the MCDA matrix were deemed to make an impact on the HTA process, there is certainly some room for improvement with respect to the adaptation of a new approach towards the value assessment of OMPs in Poland.

Electronic supplementary material

The online version of this article (10.1186/s13023-018-0803-9) contains supplementary material, which is available to authorized users.

Using multi-criteria decision analysis to appraise orphan drugs: a systematic review

INTRODUCTION

Multi-criteria decision analysis (MCDA) could potentially solve current methodological difficulties in the appraisal of orphan drugs.

AREAS COVERED

We provide an overview of the existing evidence regarding the use of MCDA in the appraisal of orphan drugs worldwide. Three databases (Pubmed, Embase, Web of Science) were searched for English, French and German literature published between January 2000 and April 2017. Full-text articles were supplemented with conference abstracts. A total of seven articles and six abstracts were identified.

EXPERT COMMENTARY

The literature suggests that MCDA is increasingly being used in the context of appraising orphan drugs. It has shown itself to be a flexible approach with the potential to assist in decision-making regarding reimbursement for orphan drugs. However, further research regarding its application must be conducted.

Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework

Escalating drug prices have catalysed the generation of numerous "value frameworks" with the aim of informing payers, clinicians and patients on the assessment and appraisal process of new medicines for the purpose of coverage and treatment selection decisions. Although this is an important step towards a more inclusive Value Based Assessment (VBA) approach, aspects of these frameworks are based on weak methodologies and could potentially result in misleading recommendations or decisions. In this paper, a Multiple Criteria Decision Analysis (MCDA) methodological process, based on Multi Attribute Value Theory (MAVT), is adopted for building a multi-criteria evaluation model. A five-stage model-building process is followed, using a top-down "value-focused thinking" approach, involving literature reviews and expert consultations. A generic value tree is structured capturing decision-makers' concerns for assessing the value of new medicines in the context of Health Technology Assessment (HTA) and in alignment with decision theory. The resulting value tree (Advance Value Tree) consists of three levels of criteria (top level criteria clusters, mid-level criteria, bottom level sub-criteria or attributes) relating to five key domains that can be explicitly measured and assessed: (a) burden of disease, (b) therapeutic impact, (c) safety profile (d) innovation level and (e) socioeconomic impact. A number of MAVT modelling techniques are introduced for operationalising (i.e. estimating) the model, for scoring the alternative treatment options, assigning relative weights of importance to the criteria, and combining scores and weights. Overall, the combination of these MCDA modelling techniques for the elicitation and construction of value preferences across the generic value tree provides a new value framework (Advance Value Framework) enabling the comprehensive measurement of value in a structured and transparent way. Given its flexibility to meet diverse requirements and become readily adaptable across different settings, the Advance Value Framework could be offered as a decision-support tool for evaluators and payers to aid coverage and reimbursement of new medicines.

The use of non-economic criteria in pricing and reimbursement decisions in Central and Eastern Europe: issues, trends and recommendations

INTRODUCTION

According to some experts, there is still room for improvement with regard to the inclusion of ethical considerations in Health Technology Assessment (HTA).

AREAS COVERED

The pros and cons of the introduction of non-economic criteria in the HTA process in Central and Eastern Europe (CEE) are discussed. In comparison to Western Europe, financial considerations are even more important in CEE settings; however, it could also be said that attachment to equity and justice is part of CEE's heritage. Therefore, the trade-off between conflicting principles is evaluated. Expert commentary: To ensure the right balance between equity and efficiency in decision making, the current HTA framework has to be further augmented to allow all conflicting criteria to be addressed to a satisfactory degree. Following other examples, the applicability of multi criteria decision analysis technique to CEE settings should be further investigated.

The Value of Medicines: A Crucial but Vague Concept

Health Technology Assessment is increasingly used to evaluate the value of healthcare products and to prioritize resources; however, defining exactly what value is and how it should be measured remains a challenge. In this article, we report the results of a literature review, focusing on nine European countries, with the aim of investigating how value is defined from the perspective of different stakeholders, how definitions of value are used, and how value is incorporated into decision making. Only three articles were identified that presented definitions of value, and there was no single shared definition of value in healthcare, which appears to be a highly subjective concept. The majority of the countries investigated combine clinical assessment with economic evaluation to make reimbursement recommendations; the quality-adjusted life-year is the most commonly used measure of value but does not capture broader aspects of value that may be important to patients and healthcare systems. We describe the use of value-based pricing and multi-criteria decision analysis, two approaches to the incorporation of broader aspects of value into decision making. Overall, we have identified considerable variation in how a product's value is defined by different stakeholders. Although a universal understanding of value in healthcare is important, it is clear that current definitions are insufficient, potentially leading to inconsistent reimbursement decisions. Ultimately, the establishment of clearer policies for defining and measuring value in healthcare is needed, and is likely to lead to improvements in the consistency of decision making.

CHANGING HEALTH TECHNOLOGY ASSESSMENT PARADIGMS?

Objectives: Health technology assessment (HTA) has to innovate to best support changing health system environments and to help provide access to valuable innovation under fiscal constraint.

Methods: Issues associated with changing HTA paradigms were identified through scoping and explored through deliberation at a meeting of industry and HTA leaders.

Results: Five broad areas of change (engagement, scientific dialogue, research prioritization, adaptive approaches, and real world data) were identified. The meeting focused on two themes derived from these: re-thinking scientific dialogue and multi-stakeholder engagement, and re-thinking value, affordability, and access. Earlier and ongoing engagement to steer the innovation process and help achieve appropriate use across the technology lifecycle was perceived as important but would be resource intensive and would require priority setting. Patients need to be involved throughout, and particularly at the early stages. Further discussion is needed on the type of body best suited to convening the dialogue required. There was agreement that HTA must continue to assess value, but views differed on the role that HTA should play in assessing affordability and on appropriate responses to challenges around affordability. Enhanced horizon scanning could play an important role in preparing for significant future investments.

Conclusions: Early and ongoing multi-stakeholder engagement and revisiting approaches to valuing innovation are required. Questions remain as to the most appropriate role for HTA bodies. Changing HTA paradigms extend HTA's traditional remit of being responsive to decision-makers demands to being more proactive and considering whole system value.

Value-Based Assessment of New Medical Technologies: Towards a Robust Methodological Framework for the Application of Multiple Criteria Decision Analysis in the Context of Health Technology Assessment

In recent years, multiple criteria decision analysis (MCDA) has emerged as a likely alternative to address shortcomings in health technology assessment (HTA) by offering a more holistic perspective to value assessment and acting as an alternative priority setting tool. In this paper, we argue that MCDA needs to subscribe to robust methodological processes related to the selection of objectives, criteria and attributes in order to be meaningful in the context of healthcare decision making and fulfil its role in value-based assessment (VBA). We propose a methodological process, based on multi-attribute value theory (MAVT) methods comprising five distinct phases, outline the stages involved in each phase and discuss their relevance in the HTA process. Importantly, criteria and attributes need to satisfy a set of desired properties, otherwise the outcome of the analysis can produce spurious results and misleading recommendations. Assuming the methodological process we propose is adhered to, the application of MCDA presents three very distinct advantages to decision makers in the context of HTA and VBA: first, it acts as an instrument for eliciting preferences on the performance of alternative options across a wider set of explicit criteria, leading to a more complete assessment of value; second, it allows the elicitation of preferences across the criteria themselves to reflect differences in their relative importance; and, third, the entire process of preference elicitation can be informed by direct stakeholder engagement, and can therefore reflect their own preferences. All features are fully transparent and facilitate decision making.

Making Good Decisions in Healthcare with Multi-Criteria Decision Analysis: The Use, Current Research and Future Development of MCDA

Healthcare decision making is usually characterized by a low degree of transparency. The demand for transparent decision processes can be fulfilled only when assessment, appraisal and decisions about health technologies are performed under a systematic construct of benefit assessment. The benefit of an intervention is often multidimensional and, thus, must be represented by several decision criteria. Complex decision problems require an assessment and appraisal of various criteria; therefore, a decision process that systematically identifies the best available alternative and enables an optimal and transparent decision is needed. For that reason, decision criteria must be weighted and goal achievement must be scored for all alternatives. Methods of multi-criteria decision analysis (MCDA) are available to analyse and appraise multiple clinical endpoints and structure complex decision problems in healthcare decision making. By means of MCDA, value judgments, priorities and preferences of patients, insurees and experts can be integrated systematically and transparently into the decision-making process. This article describes the MCDA framework and identifies potential areas where MCDA can be of use (e.g. approval, guidelines and reimbursement/pricing of health technologies). A literature search was performed to identify current research in healthcare. The results showed that healthcare decision making is addressing the problem of multiple decision criteria and is focusing on the future development and use of techniques to weight and score different decision criteria. This article emphasizes the use and future benefit of MCDA.

Estimating the value of medical treatments to patients using probabilistic multi criteria decision analysis

BACKGROUND

Estimating the value of medical treatments to patients is an essential part of healthcare decision making, but is mostly done implicitly and without consulting patients. Multi criteria decision analysis (MCDA) has been proposed for the valuation task, while stated preference studies are increasingly used to measure patient preferences. In this study we propose a methodology for using stated preferences to weigh clinical evidence in an MCDA model that includes uncertainty in both patient preferences and clinical evidence explicitly.

METHODS

A probabilistic MCDA model with an additive value function was developed and illustrated using a case on hypothetical treatments for depression. The patient-weighted values were approximated with Monte Carlo simulations and compared to expert-weighted results. Decision uncertainty was calculated as the probability of rank reversal for the first rank. Furthermore, scenario analyses were done to assess the relative impact of uncertainty in preferences and clinical evidence, and of assuming uniform preference distributions.

RESULTS

The patient-weighted values for drug A, drug B, drug C, and placebo were 0.51 (95% CI: 0.48 to 0.54), 0.51 (95% CI: 0.48 to 0.54), 0.54 (0.49 to 0.58), and 0.15 (95% CI: 0.13 to 0.17), respectively. Drug C was the most preferred treatment and the rank reversal probability for first rank was 27%. This probability decreased to 18% when uncertainty in performances was not included and increased to 41% when uncertainty in criterion weights was not included. With uniform preference distributions, the first rank reversal probability increased to 61%. The expert-weighted values for drug A, drug B, drug C, and placebo were 0.67 (95% CI: 0.65 to 0.68), 0.57 (95% CI: 0.56 to 0.59), 0.67 (95% CI: 0.61 to 0.71), and 0.19 (95% CI: 0.17 to 0.21). The rank reversal probability for the first rank according to experts was 49%.

CONCLUSIONS

Preferences elicited from patients can be used to weigh clinical evidence in a probabilistic MCDA model. The resulting treatment values can be contrasted to results from experts, and the impact of uncertainty can be quantified using rank probabilities. Future research should focus on integrating the model with regulatory decision frameworks and on including other types of uncertainty.

Addressing preference heterogeneity in public health policy by combining Cluster Analysis and Multi-Criteria Decision Analysis: Proof of Method

The use of subgroups based on biological-clinical and socio-demographic variables to deal with population heterogeneity is well-established in public policy. The use of subgroups based on preferences is rare, except when religion based, and controversial. If it were decided to treat subgroup preferences as valid determinants of public policy, a transparent analytical procedure is needed. In this proof of method study we show how public preferences could be incorporated into policy decisions in a way that respects both the multi-criterial nature of those decisions, and the heterogeneity of the population in relation to the importance assigned to relevant criteria. It involves combining Cluster Analysis (CA), to generate the subgroup sets of preferences, with Multi-Criteria Decision Analysis (MCDA), to provide the policy framework into which the clustered preferences are entered. We employ three techniques of CA to demonstrate that not only do different techniques produce different clusters, but that choosing among techniques (as well as developing the MCDA structure) is an important task to be undertaken in implementing the approach outlined in any specific policy context. Data for the illustrative, not substantive, application are from a Randomized Controlled Trial of online decision aids for Australian men aged 40-69 years considering Prostate-specific Antigen testing for prostate cancer. We show that such analyses can provide policy-makers with insights into the criterion-specific needs of different subgroups. Implementing CA and MCDA in combination to assist in the development of policies on important health and community issues such as drug coverage, reimbursement, and screening programs, poses major challenges -conceptual, methodological, ethical-political, and practical - but most are exposed by the techniques, not created by them.

A review and classification of approaches for dealing with uncertainty in multi-criteria decision analysis for healthcare decisions

Multi-criteria decision analysis (MCDA) is increasingly used to support decisions in healthcare involving multiple and conflicting criteria. Although uncertainty is usually carefully addressed in health economic evaluations, whether and how the different sources of uncertainty are dealt with and with what methods in MCDA is less known. The objective of this study is to review how uncertainty can be explicitly taken into account in MCDA and to discuss which approach may be appropriate for healthcare decision makers. A literature review was conducted in the Scopus and PubMed databases. Two reviewers independently categorized studies according to research areas, the type of MCDA used, and the approach used to quantify uncertainty. Selected full text articles were read for methodological details. The search strategy identified 569 studies. The five approaches most identified were fuzzy set theory (45% of studies), probabilistic sensitivity analysis (15%), deterministic sensitivity analysis (31%), Bayesian framework (6%), and grey theory (3%). A large number of papers considered the analytic hierarchy process in combination with fuzzy set theory (31%). Only 3% of studies were published in healthcare-related journals. In conclusion, our review identified five different approaches to take uncertainty into account in MCDA. The deterministic approach is most likely sufficient for most healthcare policy decisions because of its low complexity and straightforward implementation. However, more complex approaches may be needed when multiple sources of uncertainty must be considered simultaneously.

Multi-criteria decision analysis for health technology assessment in Canada: insights from an expert panel discussion

Multi-criteria decision analysis (MCDA), a decision-making tool, has received increasing attention in recent years, notably in the healthcare field. For Canada, it is unclear whether and how MCDA should be incorporated into the existing health technology assessment (HTA) decision-making process. To facilitate debate on improving HTA decision-making in Canada, a workshop was held in conjunction with the 8th World Congress on Health Economics of the International Health Economics Association in Toronto, Canada in July 2011. The objective of the workshop was to discuss the potential benefits and challenges related to the use of MCDA for HTA decision-making in Canada. This paper summarizes and discusses the recommendations of an expert panel convened at the workshop to discuss opportunities and concerns with reference to the implementation of MCDA in Canada.