Overcoming barriers to priority setting using interdisciplinary methods

Criteria for the procedural fairness of health financing decisions: a scoping review

Due to constraints on institutional capacity and financial resources, the road to universal health coverage (UHC) involves difficult policy choices. To assist with these choices, scholars and policy makers have done extensive work on criteria to assess the substantive fairness of health financing policies: their impact on the distribution of rights, duties, benefits and burdens on the path towards UHC. However, less attention has been paid to the procedural fairness of health financing decisions. The Accountability for Reasonableness Framework (A4R), which is widely applied to assess procedural fairness, has primarily been used in priority-setting for purchasing decisions, with revenue mobilization and pooling receiving limited attention. Furthermore, the sufficiency of the A4R framework's four criteria (publicity, relevance, revisions and appeals, and enforcement) has been questioned. Moreover, research in political theory and public administration (including deliberative democracy), public finance, environmental management, psychology, and health financing has examined the key features of procedural fairness, but these insights have not been synthesized into a comprehensive set of criteria for fair decision-making processes in health financing. A systematic study of how these criteria have been applied in decision-making situations related to health financing and in other areas is also lacking. This paper addresses these gaps through a scoping review. It argues that the literature across many disciplines can be synthesized into 10 core criteria with common philosophical foundations. These go beyond A4R and encompass equality, impartiality, consistency over time, reason-giving, transparency, accuracy of information, participation, inclusiveness, revisability and enforcement. These criteria can be used to evaluate and guide decision-making processes for financing UHC across different country income levels and health financing arrangements. The review also presents examples of how these criteria have been applied to decisions in health financing and other sectors.

Developing a combined framework for priority setting in integrated health and social care systems

BACKGROUND

There is an international move towards greater integration of health and social care to cope with the increasing demand on services.. In Scotland, legislation was passed in 2014 to integrate adult health and social care services resulting in the formation of 31 Health and Social Care Partnerships (HSCPs). Greater integration does not eliminate resource scarcity and the requirement to make (resource) allocation decisions to meet the needs of local populations. There are different perspectives on how to facilitate and improve priority setting in health and social care organisations with limited resources, but structured processes at the local level are still not widely implemented. This paper reports on work with new HSCPs in Scotland to develop a combined multi-disciplinary priority setting and resource allocation framework.

METHODS

To develop the combined framework, a scoping review of the literature was conducted to determine the key principles and approaches to priority setting from economics, decision-analysis, ethics and law, and attempts to combine such approaches. Co-production of the combined framework involved a multi-disciplinary workshop including local, and national-level stakeholders and academics to discuss and gather their views.

RESULTS

The key findings from the literature review and the stakeholder workshop were taken to produce a final combined framework for priority setting and resource allocation. This is underpinned by principles from economics (opportunity cost), decision science (good decisions), ethics (justice) and law (fair procedures). It outlines key stages in the priority setting process, including: framing the question, looking at current use of resources, defining options and criteria, evaluating options and criteria, and reviewing each stage. Each of these has further sub-stages and includes a focus on how the combined framework interacts with the consultation and involvement of patients, public and the wider staff.

CONCLUSIONS

The integration agenda for health and social care is an opportunity to develop and implement a combined framework for setting priorities and allocating resources fairly to meet the needs of the population. A key aim of both integration and the combined framework is to facilitate the shifting of resources from acute services to the community.

Barriers and facilitators to implementing priority setting and resource allocation tools in hospital decisions: A systematic review

Health care budgets in high-income countries are having issues coping with unsustainable growth in demand, particularly in the hospital setting. Despite this, implementing tools systematising priority setting and resource allocation decisions has been challenging. This study answers two questions: (1) what are the barriers and facilitators to implementing priority setting tools in the hospital setting of high-income countries? and (2) what is their fidelity? A systematic review using the Cochrane methods was conducted including studies of hospital-related priority setting tools reporting barriers or facilitators for implementation, published after the year 2000. Barriers and facilitators were classified using the Consolidated Framework for Implementation Research (CFIR). Fidelity was assessed using priority setting tool's standards. Out of thirty studies, ten reported program budgeting and marginal analysis (PBMA), twelve multi-criteria decision analysis (MCDA), six health technology assessment (HTA) related frameworks, and two, an ad hoc tool. Barriers and facilitators were outlined across all CFIR domains. Implementation factors not frequently observed, such as 'evidence of previous successful tool application', 'knowledge and beliefs about the intervention' or 'external policy and incentives' were reported. Conversely, some constructs did not yield any barrier or facilitator including 'intervention source' or 'peer pressure'. PBMA studies satisfied the fidelity criteria between 86% and 100%, for MCDA it varied between 36% and 100%, and for HTA it was between 27% and 80%. However, fidelity was not related to implementation. This study is the first to use an implementation science approach. Results represent the starting point for organisations wishing to use priority setting tools in the hospital setting by providing an overview of barriers and facilitators. These factors can be used to assess readiness for implementation or to serve as the foundation for process evaluations. Through our findings, we aim to improve the uptake of priority setting tools and support their sustainable use.

Knowledge Mapping of Multicriteria Decision Analysis in Healthcare: A Bibliometric Analysis

Objective

Multicriteria decision analysis (MCDA) is a useful tool in complex decision-making situations, and has been used in medical fields to evaluate treatment options and drug selection. This study aims to provide valuable insights into MCDA in healthcare through examining the research focus of existing studies, major fields, major applications, most productive authors and countries, and most common journals in the domain.

Methods

A bibliometric analysis was conducted on the publication related to MCDA in healthcare from the Web of Science Core Collection (WoSCC) database on 14 July 2021. Three bibliometric software (VOSviewer, R-bibliometrix, and CiteSpace) were used to conduct the analysis including years, countries, institutes, authors, journals, co-citation references, and keywords.

Results

A total of 410 publications were identified with an average yearly growth rate of 32% (1999–2021), from 196 academic journals with 23,637 co-citation references by 871 institutions from 70 countries/regions. The United States was the most productive country (n = 80). Universiti Pendidikan Sultan Idris (n = 16), Université de Montréal (n = 13), and Syreon Research Institute (n = 12) were the top productive institutions. A A Zaidan, Mireille Goetghebeur and Zoltan Kalo were the biggest nodes in every cluster of authors' networks. The top journals in terms of the number of articles (n = 17) and citations (n = 1,673) were Value in Health and Journal of Medical Systems, respectively. The extant literature has focused on four aspects, including the analytic hierarchy process (AHP), decision-making, health technology assessment, and healthcare waste management. COVID-19 and fuzzy TOPSIS received careful attention from MCDA applications recently. MCDA in big data, telemedicine, TOPSIS, and fuzzy AHP is well-developed and an important theme, which may be the trend in future research.

Conclusion

This study uncovers a holistic picture of the performance of MCDA-related literature published in healthcare. MCDA has a broad application on different topics and would be helpful for practitioners, researchers, and decision-makers working in healthcare to advance the wheel of medical complex decision-making. It can be argued that the door is still open for improving the role of MCDA in healthcare, whether in its methodology (e.g., fuzzy TOPSIS) or application (e.g., telemedicine).

The Application of Multicriteria Decision Analysis Methods in Health Care: A Literature Review

OBJECTIVES

The main objectives of this article are 2-fold. First, we explore the application of multicriteria decision analysis (MCDA) methods in different areas of health care, particularly the adoption of various MCDA methods across health care decision making problems. Second, we report on the publication trends on the application of MCDA methods in health care.

METHOD

PubMed was searched for literature from 1960 to 2019 in the English language. A wide range of keywords was used to retrieve relevant studies. The literature search was performed in September 2019. Articles were included only if they have reported an MCDA case in health care.

RESULTS AND CONCLUSION

The search yielded 8,318 abstracts, of which 158 fulfilled the inclusion criteria and were considered for further analysis. Hybrid methods are the most widely used methods in health care decision making problems. When it comes to single methods, analytic hierarchy process (AHP) is the most widely used method followed by TOPSIS (technique for order preference by similarity to ideal solution), multiattribute utility theory, goal programming, EVIDEM (evidence and value: impact on decision making), evidential reasoning, discrete choice experiment, and so on. Interestingly, the usage of hybrid methods has been high in recent years. AHP is most widely applied in screening and diagnosing and followed by treatment, medical devices, resource allocation, and so on. Furthermore, treatment, screening and diagnosing, medical devices, and drug development and assessment got more attention in the MCDA context. It is indicated that the application of MCDA methods to health care decision making problem is determined by the nature and complexity of the health care problem. However, guidelines and tools exist that assist in the selection of an MCDA method.

More than cost-effectiveness? Applying a second-stage filter to improve policy decision making

Background

Apart from cost‐effectiveness, considerations like equity and acceptability may affect health‐care priority setting. Preferably, priority setting combines evidence evaluation with an appraisal procedure, to elicit and weigh these considerations.

Objective

To demonstrate a structured approach for eliciting and evaluating a broad range of assessment criteria, including key stakeholders’ values, aiming to support decision makers in priority setting.

Methods

For a set of cost‐effective substitute interventions for depression care, the appraisal criteria were adopted from the Australian Assessing Cost‐Effectiveness initiative. All substitute interventions were assessed in an appraisal, using focus group discussions and semi‐structured interviews conducted among key stakeholders.

Results

Appraisal of the substitute cost‐effective interventions yielded an overview of considerations and an overall recommendation for decision makers. Two out of the thirteen pairs were deemed acceptable and realistic, that is investment in therapist‐guided and Internet‐based cognitive behavioural therapy instead of cognitive behavioural therapy in mild depression, and investment in combination therapy rather than individual psychotherapy in severe depression. In the remaining substitution pairs, substantive issues affected acceptability. The key issues identified were as follows: workforce capacity, lack of stakeholder support and the need for change in clinicians’ attitude.

Conclusions

Systematic identification of stakeholders’ considerations allows decision makers to prioritize among cost‐effective policy options. Moreover, this approach entails an explicit and transparent priority‐setting procedure and provides insights into the intended and unintended consequences of using a certain health technology.

Patient contribution

Patients were involved in the conduct of the study for instance, by sharing their values regarding considerations relevant for priority setting.

Comparison of Decision Modeling Approaches for Health Technology and Policy Evaluation

We discuss tradeoffs and errors associated with approaches to modeling health economic decisions. Through an application in pharmacogenomic (PGx) testing to guide drug selection for individuals with a genetic variant, we assessed model accuracy, optimal decisions, and computation time for an identical decision scenario modeled 4 ways: using 1) coupled-time differential equations (DEQ), 2) a cohort-based discrete-time state transition model (MARKOV), 3) an individual discrete-time state transition microsimulation model (MICROSIM), and 4) discrete event simulation (DES). Relative to DEQ, the net monetary benefit for PGx testing (v. a reference strategy of no testing) based on MARKOV with rate-to-probability conversions using commonly used formulas resulted in different optimal decisions. MARKOV was nearly identical to DEQ when transition probabilities were embedded using a transition intensity matrix. Among stochastic models, DES model outputs converged to DEQ with substantially fewer simulated patients (1 million) v. MICROSIM (1 billion). Overall, properly embedded Markov models provided the most favorable mix of accuracy and runtime but introduced additional complexity for calculating cost and quality-adjusted life year outcomes due to the inclusion of "jumpover" states after proper embedding of transition probabilities. Among stochastic models, DES offered the most favorable mix of accuracy, reliability, and speed.

Priority Setting in the Polish Health Care System According to Patients' Perspective

Identification of health priorities is concerned with equitable distribution of resources and is an important part of strategic planning in the health care system. The aim of this article is to describe health priorities in the Polish health care system from the patients’ perspective. The study included 533 patients hospitalized in the Lodz region. The average age of the respondents was 48.5 years and one third (36.6%) had university education. Most of the respondents (64.9%) negatively assessed the functioning of the health care system in Poland. Most of them claimed the following aspects require improvements: financing health services (85.8%), determining priorities in health care (80.3%), the role of health insurance (80.3%), and medical education (70.8%). Over 70% of the respondents agreed the role of politicians in designing and implementing health system reforms should be limited. The fact that the respondents so negatively assessed the Polish health care system implies there is a need for full discussion on redefining health priorities.

A Participatory Action Research Approach to Mental Health Interventions among Corrections Officers: Standardizing Priorities and Maintaining Design Autonomy

A central dilemma in Participatory Action Research (PAR) is to establish participant decision authority on interventions while adhering to rigorous research practices. We faced this dilemma as part of an ongoing multi-site field research project in the corrections sector, where semi-autonomous union-based Design Teams (DTs) address worker health issues and design interventions. Employee focus groups and surveys elicited areas of concern, pointing to four topics in particular: overtime and sleep, work-family balance, physical fitness, and mental health; these were later expanded to eight priority areas. Quantitative rankings were generated by focus groups of line-level employees and supervisors. A multi-level, iterative priority selection process averaged focus group ratings of topic importance and also difficulty to address separately. Areas of job stress and mental health had highest importance but were also considered most difficult to address. A labor-management steering committee reviewed and endorsed the rankings and transmitted these to newly formed DTs. In principle, each DT was free to establish a different topic for initial intervention but they all chose the most important and difficult to address topics. This structured multi-tiered participatory process preserved ownership by all parties. Balancing participant autonomy and efficient prioritization of topics among multiple interest groups in this PAR effort met research methods needs and also made it easier for DTs to focus on the difficult and stigmatized area of mental health in the correctional workforce.

Use of programme budgeting and marginal analysis to set priorities for local NHS dental services: learning from the north east of England

Background

Priority setting is necessary where competing demands exceed the finite resources available. The aim of the study was to develop and test a prioritization framework based upon programme budgeting and marginal analysis (PBMA) as a tool to assist National Health Service (NHS) commissioners in their management of resources for local NHS dental services.

Methods

Twenty-seven stakeholders (5 dentists, 8 commissioners and 14 patients) participated in a case-study based in a former NHS commissioning organization in the north of England. Stakeholders modified local decision-making criteria and applied them to a number of different scenarios.

Results

The majority of financial resources for NHS dental services in the commissioning organization studied were allocated to primary care dental practitioners' contracts in perpetuity, potentially constraining commissioners' abilities to shift resources. Compiling the programme budget was successful, but organizational flux and difficulties engaging local NHS commissioners significantly impacted upon the marginal analysis phase.

Conclusions

NHS dental practitioners' contracts resemble budget-silos which do not facilitate local resource reallocation. 'Context-specific' factors significantly challenged the successful implementation and impact of PBMA. A local PBMA champion embedded within commissioning organizations should be considered. Participants found visual depiction of the cost-value ratio helpful during their initial priority setting deliberations.

Evidence, values, and funding decisions in Canadian cancer systems

Expenditure on cancer therapies is rising rapidly in many countries, particularly for cancer drugs. In recent years, this has stimulated a global debate among the public, patients, clinicians, decision-makers, and the pharmaceutical industry on value, affordability, and sustainability propositions relating to cancer therapies. In this article, we discuss some recent developments in evidence-based approaches to priority setting and resource allocation in Canadian cancer systems. These developments include new methods for deliberative public engagement, generating and using real-world evidence, multi-criteria decision analysis, and handling uncertainty with evidence for gene therapies.

Using co-creation and multi-criteria decision analysis to close service gaps for underserved populations

BACKGROUND

Navigating treatment pathways remains a challenge for populations with complex needs due to bottlenecks, service gaps and access barriers. The application of novel methods may be required to identify and remedy such problems.

OBJECTIVE

To demonstrate a novel approach to identifying persistent service gaps, generating potential solutions and prioritizing action.

DESIGN

Co-creation and multi-criteria decision analysis in the context of a larger, mixed methods study.

SETTING AND PARTICIPANTS

Community-dwelling sample of older women living alone (OWLA), residing in Melbourne, Australia (n = 13-37). Convenience sample of (n = 11) representatives from providers and patient organizations.

INTERVENTIONS

Novel interventions co-created to support health, well-being and independence for OWLA and bridge missing links in pathways to care.

MAIN OUTCOME MEASURES

Performance criteria, criterion weights , performance ratings, summary scores and ranks reflecting the relative value of interventions to OWLA.

RESULTS

The co-creation process generated a list of ten interventions. Both OWLA and stakeholders considered a broad range of criteria when evaluating the relative merits of these ten interventions and a "Do Nothing" alternative. Combining criterion weights with performance ratings yielded a consistent set of high priority interventions, with "Handy Help," "Volunteer Drivers" and "Exercise Buddies" most highly ranked by both OWLA and stakeholder samples.

DISCUSSION AND CONCLUSIONS

The present study described and demonstrated the use of multi-criteria decision analysis to prioritize a set of novel interventions generated via a co-creation process. Application of this approach can add community voice to the policy debate and begin to bridge the gap in service provision for underserved populations.

Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art

BACKGROUND

Multi-criteria decision analysis (MCDA) concepts, models and tools have been used increasingly in health technology assessment (HTA), with several studies pointing out practical and theoretical issues related to its use. This study provides a critical review of published studies on MCDA in the context of HTA by assessing their methodological quality and summarising methodological challenges.

METHODS

A systematic review was conducted to identify studies discussing, developing or reviewing the use of MCDA in HTA using aggregation approaches. Studies were classified according to publication time and type, country of study, technology type and study type. The PROACTIVE-S approach was constructed and used to analyse methodological quality. Challenges and limitations reported in eligible studies were collected and summarised; this was followed by a critical discussion on research requirements to address the identified challenges.

RESULTS

129 journal articles were eligible for review, 56% of which were published in 2015-2017; 42% focused on pharmaceuticals; 36, 26 and 18% reported model applications, issues regarding MCDA implementation analyses, and proposing frameworks, respectively. Poor compliance with good methodological practice (< 25% complying studies) was found regarding behavioural analyses, discussion of model assumptions and uncertainties, modelling of value functions, and dealing with judgment inconsistencies. The five most reported challenges related to evidence and data synthesis; value system differences and participant selection issues; participant difficulties; methodological complexity and resource balance; and criteria and attributes modelling. A critical discussion on ways to address these challenges ensues.

DISCUSSION

Results highlight the need for advancement in robust methodologies, procedures and tools to improve methodological quality of MCDA in HTA studies. Research pathways include developing new model features, good practice guidelines, technologies to enable participation and behavioural research.

Addressing the affordability of cancer drugs: using deliberative public engagement to inform health policy

BACKGROUND

Health system expenditure on cancer drugs is rising rapidly in many OECD countries given the costly new treatments and increased rates of use due to a growing and ageing population. These factors put considerable strain on the sustainability of health systems worldwide, sparking public debate among clinicians, pharmaceutical companies, policy-makers and citizens on issues of affordability and equity. We engaged Canadians through a series of deliberative public engagement events to determine their priorities for making cancer drug funding decisions fair and sustainable in Canada's publicly financed health system.

METHODS

An approach to deliberation was developed based on the McMaster Health Forum's citizen panels and the established Burgess and O'Doherty model of deliberative public engagement. Six deliberations were held across Canada in 2016. Transcripts were coded in NVivo and analysed to determine where participants' views converged and diverged. Recommendations were grouped thematically.

RESULTS

A total of 115 Canadians participated in the deliberative events and developed 86 recommendations. Recommendations included the review and regular re-review of approved drugs using 'real-world' evidence on effectiveness and cost-effectiveness; prioritisation of treatments that restore patients' independence, mental health and general well-being; ensuring that decision processes, results and their rationales are transparent; and commitment to people with similar needs receiving the same care regardless of where in Canada they live.

CONCLUSIONS

The next steps for policy-makers should be to develop mechanisms for (1) re-reviewing effectiveness and cost-effectiveness data for all cancer drugs; (2) making disinvestments in cancer drugs that satisfy requirements relating to grandfathering and compassionate access; (3) ensuring fair and equitable access to cancer drugs for all Canadians; and (4) fostering a pan-Canadian approach to cancer drug funding decisions.

A multicriteria resource allocation model for the redesign of services following birth

BACKGROUND

Many healthcare services are under considerable pressure to reduce costs while improving quality. This is particularly true in the United Kingdom's National Health Service where postnatal care is sometimes viewed as having a low priority. There is much debate about the service's redesign and the reallocation of resources, both along care pathways and between groups of mothers and babies with different needs. The aim of this study was to develop a decision support tool that would encourage a systemic approach to service redesign and that could assess the various quality and financial implications of service change options making the consequent trade-offs explicit. The paper describes the development process and an initial implementation as a preliminary exploration of the possible merits of this approach.

METHODS

Other studies have suggested that combining multicriteria decision analysis with programme budgeting and marginal analysis might offer a suitable basis for resource allocation decisions in healthcare systems. The Postnatal care Resource Allocation Model incorporated this approach in a decision support tool to analyse the consequences of varying design parameters, notably staff contacts and time, on the various quality domains and costs. The initial phase of the study focussed on mapping postnatal care, involving interviews and workshops with a variety of stakeholders. This was supplemented with a literature review and the resultant knowledge base was encoded in the decision support tool. The model was then tested with various stakeholders before being used in an NHS Trust in England.

RESULTS

The model provides practical support, helping staff explore options and articulate their proposals for the redesign of postnatal care. The integration of cost and quality domains facilitates trade-offs, allowing staff to explore the benefits of reallocating resources between hospital and community-based care, and different patient-categories.

CONCLUSIONS

The main benefits of the model include its structure for assembling the key data, sharing evidence amongst multi-professional teams and encouraging constructive, systemic debate. Although the model was developed in the context of the routine maternity services for mothers and babies in the days following birth it could be adapted for use in other health care services.

The development of neural stimulators: a review of preclinical safety and efficacy studies

OBJECTIVE

Given the rapid expansion of the field of neural stimulation and the rigorous regulatory approval requirements required before these devices can be applied clinically, it is important that there is clarity around conducting preclinical safety and efficacy studies required for the development of this technology.

APPROACH

The present review examines basic design principles associated with the development of a safe neural stimulator and describes the suite of preclinical safety studies that need to be considered when taking a device to clinical trial.

MAIN RESULTS

Neural stimulators are active implantable devices that provide therapeutic intervention, sensory feedback or improved motor control via electrical stimulation of neural or neuro-muscular tissue in response to trauma or disease. Because of their complexity, regulatory bodies classify these devices in the highest risk category (Class III), and they are therefore required to go through a rigorous regulatory approval process before progressing to market. The successful development of these devices is achieved through close collaboration across disciplines including engineers, scientists and a surgical/clinical team, and the adherence to clear design principles. Preclinical studies form one of several key components in the development pathway from concept to product release of neural stimulators. Importantly, these studies provide iterative feedback in order to optimise the final design of the device. Key components of any preclinical evaluation include: in vitro studies that are focussed on device reliability and include accelerated testing under highly controlled environments; in vivo studies using animal models of the disease or injury in order to assess efficacy and, given an appropriate animal model, the safety of the technology under both passive and electrically active conditions; and human cadaver and ex vivo studies designed to ensure the device's form factor conforms to human anatomy, to optimise the surgical approach and to develop any specialist surgical tooling required.

SIGNIFICANCE

The pipeline from concept to commercialisation of these devices is long and expensive; careful attention to both device design and its preclinical evaluation will have significant impact on the duration and cost associated with taking a device through to commercialisation. Carefully controlled in vitro and in vivo studies together with ex vivo and human cadaver trials are key components of a thorough preclinical evaluation of any new neural stimulator.

Don't Discount Societal Value in Cost-Effectiveness Comment on "Priority Setting for Universal Health Coverage: We Need Evidence-Informed Deliberative Processes, Not Just More Evidence on Cost-Effectiveness"

As healthcare resources become increasingly scarce due to growing demand and stagnating budgets, the need for effective priority setting and resource allocation will become ever more critical to providing sustainable care to patients. While societal values should certainly play a part in guiding these processes, the methodology used to capture these values need not necessarily be limited to multi-criterion decision analysis (MCDA)-based processes including 'evidence-informed deliberative processes.' However, if decision-makers intend to not only incorporates the values of the public they serve into decisions but have the decisions enacted as well, consideration should be given to more direct involvement of stakeholders. Based on the examples provided by Baltussen et al, MCDA-based processes like 'evidence-informed deliberative processes' could be one way of achieving this laudable goal.

Decommissioning health care: identifying best practice through primary and secondary research – a prospective mixed-methods study

Background

Decommissioning – defined as the planned process of removing, reducing or replacing health-care services – is an important component of current reforms in the NHS. However, the evidence base on which to guide policy and practice in this area is weak.

Aim

This study aims to formulate theoretically grounded, evidence-informed guidance to support best practice in effective decommissioning of NHS services.

Design

The overall approach is a sequential, multimethod research design. The study involves (1) a literature synthesis summarising what is known about decommissioning, an international expert Delphi study, 12 interviews with national/regional bodies and seven narrative vignettes from NHS leaders; (2) a survey of Clinical Commissioning Groups (CCGs) in England (n = 56/211, 27%); (3) longitudinal, prospective case studies of four purposively sampled decommissioning projects comprising 59 semistructured interviews, 18 non-participant observations and documentary analysis; and (4) research with citizens, patient/service user representatives, carers, third-sector organisations and local community groups, including three focus groups (30 participants) and a second Delphi study (26 participants). The study took place over the period 2013–16.

Setting

The English NHS.

Results

There is a lack of robust evidence to guide decommissioning, but among experts there is a high level of consensus for the following good-practice principles: establish a strong leadership team, engage clinical leaders from an early stage and establish a clear rationale for change. The most common type of CCG decommissioning activity was ‘relocation or replacement of a service from an acute to a community setting’ (28% of all activities) and the majority of responding CCGs (77%) were planning to decommission services. Case studies demonstrate the need to (1) draw on evidence, reviews and policies to frame the problem; (2) build alliances in order to legitimise decommissioning as a solution; (3) seek wider acceptance, including among patients and community groups, of decommissioning; and (4) devise implementation plans that recognise the additional challenges of removal and replacement. Citizens, patient/service user representatives, carers, third-sector organisations and local community groups were more likely to believe that decommissioning is driven by financial and political concerns than by considerations of service quality and efficiency, and to distrust and/or resent decision-makers. Overall, the study suggests that failure rates in decommissioning are likely to be higher than in other forms of service change, suggesting the need for tailored design and implementation approaches.

Limitations

There were few opportunities for patient and public engagement in early phases of the research; however, this was mitigated by the addition of work package 4. We were unable to track outcomes of decommissioning activities within the time scales of the project and the survey response rate was lower than anticipated.

Conclusions

Decommissioning is shaped by change management and implementation, evidence and information, and relationships and politics. We propose an expanded understanding, encompassing organisational and political factors, of how avoidance of loss affects the delivery of decommissioning programmes. Future work should explore the relationships between contexts, mechanisms and outcomes in decommissioning, develop the understanding of how loss affects decisions and explore the long-term impact of decommissioning and its impact on patient care and outcomes.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Value Assessment Frameworks for HTA Agencies: The Organization of Evidence-Informed Deliberative Processes

Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, health technology assessment agencies (HTAs) presently employ value assessment frameworks that are ill fitted to capture the range and diversity of stakeholder values and thereby risk compromising the legitimacy of their recommendations. We propose "evidence-informed deliberative processes" as an alternative framework with the aim to enhance this legitimacy. This framework integrates two increasingly popular and complementary frameworks for priority setting: multicriteria decision analysis and accountability for reasonableness. Evidence-informed deliberative processes are, on one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making through evidence-informed evaluation of the identified values. The framework has important implications for how HTA agencies should ideally organize their processes. First, HTA agencies should take the responsibility of organizing stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phases, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria and to provide argumentation for how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal. The framework should not be considered a blueprint for HTA agencies but rather an aspirational goal-agencies can take incremental steps toward achieving this goal.

Implementing priority setting frameworks: Insights from leading researchers

In spite of a substantial literature developing frameworks for policymakers to use in resource allocation decisions in healthcare, there remains limited published work reporting on the implementation or evaluation of such frameworks in practice. This paper presents findings of a targeted survey of 18 leading researchers around the implementation and evaluation of priority-setting exercises. Approximately one third of respondents knew of situations where recommendations of priority-setting exercises had been implemented, one third knew that recommendations had not been implemented and the final third responded that they did not know whether recommendations had been adopted. The lack of evidence linking the implementation of priority-setting recommendations to equity and efficiency outcomes was highlighted by all respondents. Features identified as facilitating successful implementation of priority-setting recommendations included having a climate ready to accept priority-setting, good leadership or a 'champion' for the priority-setting process and having a health economist to guide the process. Successful disinvestment was very uncommon in the experience of the researchers surveyed. Recommendations emerging from Program Budgeting and Marginal Analysis exercises appeared to be more widely implemented than those coming from alternative processes. Identifying if the process was repeated following the initial process was suggested as a means to measure success.

What criteria do decision makers in Thailand use to set priorities for vaccine introduction?

Background

There is a need to identify rational criteria and set priorities for vaccines. In Thailand, many licensed vaccines are being considering for introduction into the Expanded Program on Immunization; thus, the government has to make decisions about which vaccines should be adopted. This study aimed to set priorities for new vaccines and to facilitate decision analysis.

Methods

We used a best-worst scaling study for rank-ordering of vaccines. The candidate vaccines were determined by a set of criteria, including burden of disease, target age group, budget impact, side effect, effectiveness, severity of disease, and cost of vaccine. The criteria were identified from a literature review and by in-depth, open-ended interviews with experts. The priority-setting model was conducted among three groups of stakeholders, including policy makers, healthcare professionals and healthcare administrators. The vaccine data were mapped and then calculated for the probability of selection.

Results

From the candidate vaccines, the probability of hepatitis B vaccine being selected by all respondents (96.67 %) was ranked first. This was followed, respectively, by pneumococcal conjugate vaccine-13 (95.09 %) and Haemophilus influenzae type b vaccine (90.87 %). The three groups of stakeholders (policy makers, healthcare professionals and healthcare administrators) showed the same ranking trends. Most severe disease, high fever rate and high disease burden showed the highest coefficients for criterion levels being selected by all respondents. This result can be implied that a vaccine which can prevent most severe disease with high disease burden and has low safety has a greater chance of being selected by respondents in this study.

Conclusions

The priority setting of vaccines through a multiple-criteria approach could contribute to transparency and accountability in the decision-making process. This is a step forward in the development of an evidence-based approach that meets the need of developing country. The methodology is generalizable but its application to another country would require the criteria as relevant to that country.

Electronic supplementary material

The online version of this article (doi:10.1186/s12889-016-3382-5) contains supplementary material, which is available to authorized users.

Using the Nine Common Themes of Good Practice checklist as a tool for evaluating the research priority setting process of a provincial research and program evaluation program

BACKGROUND

Given the context-specific nature of health research prioritization and the obligation to effectively allocate resources to initiatives that will achieve the greatest impact, evaluation of priority setting processes can refine and strengthen such exercises and their outcomes. However, guidance is needed on evaluation tools that can be applied to research priority setting. This paper describes the adaption and application of a conceptual framework to evaluate a research priority setting exercise operating within the public health sector in Ontario, Canada.

METHODS

The Nine Common Themes of Good Practice checklist, described by Viergever et al. (Health Res Policy Syst 8:36, 2010) was used as the conceptual framework to evaluate the research priority setting process developed for the Locally Driven Collaborative Projects (LDCP) program in Ontario, Canada. Multiple data sources were used to inform the evaluation, including a review of selected priority setting approaches, surveys with priority setting participants, document review, and consultation with the program advisory committee.

RESULTS

The evaluation assisted in identifying improvements to six elements of the LDCP priority setting process. The modifications were aimed at improving inclusiveness, information gathering practices, planning for project implementation, and evaluation. In addition, the findings identified that the timing of priority setting activities and level of control over the process were key factors that influenced the ability to effectively implement changes.

CONCLUSIONS

The findings demonstrate the novel adaptation and application of the 'Nine Common Themes of Good Practice checklist' as a tool for evaluating a research priority setting exercise. The tool can guide the development of evaluation questions and enables the assessment of key constructs related to the design and delivery of a research priority setting process.

Making Good Decisions in Healthcare with Multi-Criteria Decision Analysis: The Use, Current Research and Future Development of MCDA

Healthcare decision making is usually characterized by a low degree of transparency. The demand for transparent decision processes can be fulfilled only when assessment, appraisal and decisions about health technologies are performed under a systematic construct of benefit assessment. The benefit of an intervention is often multidimensional and, thus, must be represented by several decision criteria. Complex decision problems require an assessment and appraisal of various criteria; therefore, a decision process that systematically identifies the best available alternative and enables an optimal and transparent decision is needed. For that reason, decision criteria must be weighted and goal achievement must be scored for all alternatives. Methods of multi-criteria decision analysis (MCDA) are available to analyse and appraise multiple clinical endpoints and structure complex decision problems in healthcare decision making. By means of MCDA, value judgments, priorities and preferences of patients, insurees and experts can be integrated systematically and transparently into the decision-making process. This article describes the MCDA framework and identifies potential areas where MCDA can be of use (e.g. approval, guidelines and reimbursement/pricing of health technologies). A literature search was performed to identify current research in healthcare. The results showed that healthcare decision making is addressing the problem of multiple decision criteria and is focusing on the future development and use of techniques to weight and score different decision criteria. This article emphasizes the use and future benefit of MCDA.

An evaluation tool for assessing performance in priority setting and resource allocation: multi-site application to identify strengths and weaknesses

Introduction

An evaluation tool should help improve formal priority setting and resource allocation (PSRA) processes in Canada and elsewhere. These are crucial to maximizing value from limited resources.

Methods

On the basis of case studies, balanced scorecard development protocols and use-focused evaluation principles, an evaluation tool was developed based on an existing framework for high PSRA performance and implemented in two health care organizations in British Columbia, Canada.

Results

Implementation of the tool identified areas of strength, improvement and weakness in the pilot organizations’ processes for PSRA including: communication, staff engagement and culture. Refinements were identified and incorporated into the tool for future application.

Conclusion

This is the first documented multi-site application of such an evaluation tool. Broader dissemination should have use both in further refining the basis of the tool and in catalysing improved performance of PSRA practice.

'Real-world' health care priority setting using explicit decision criteria: a systematic review of the literature

Background

Health care decision making requires making resource allocation decisions among programs, services, and technologies that all compete for a finite resource pool. Methods of priority setting that use explicitly defined criteria can aid health care decision makers in arriving at funding decisions in a transparent and systematic way. The purpose of this paper is to review the published literature and examine the use of criteria-based methods in ‘real-world’ health care allocation decisions.

Methods

A systematic review of the published literature was conducted to find examples of ‘real-world’ priority setting exercises that used explicit criteria to guide decision-making.

Results

We found thirty-three examples in the peer-reviewed and grey literature, using a variety of methods and criteria. Program effectiveness, equity, affordability, cost-effectiveness, and the number of beneficiaries emerged as the most frequently-used decision criteria. The relative importance of criteria in the ‘real-world’ trials differed from the frequency in preference elicitation exercises. Neither the decision-making method used, nor the relative economic strength of the country in which the exercise took place, appeared to have a strong effect on the type of criteria chosen.

Conclusions

Health care decisions are made based on criteria related both to the health need of the population and the organizational context of the decision. Following issues related to effectiveness and affordability, ethical issues such as equity and accessibility are commonly identified as important criteria in health care resource allocation decisions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-0814-3) contains supplementary material, which is available to authorized users.

Equity and efficiency preferences of health policy makers in China--a stated preference analysis

BACKGROUND

Macroeconomic growth in China enables significant progress in health care and public health. It faces difficult choices regarding access, quality and affordability, while dealing with the increasing burden of chronic diseases. Policymakers are pressured to make complex decisions while implementing health strategies. This study shows how this process could be structured and reports the specific equity and efficiency preferences among Chinese policymakers.

METHODS

In total, 78 regional, provincial and national level policymakers with considerable experience participated in a discrete choice experiment, weighting the relative importance of six policy attributes describing equity and efficiency. Results from a conditional logistic model are presented for the six criteria, measuring the associated weights. Observed and unobserved heterogeneities were incorporated and tested in the model. Findings are used to give an example of ranking health interventions in relation to the present disease burden in China.

RESULTS

In general, respondents showed strong preference for efficiency criteria i.e. total beneficiaries and cost-effectiveness as the most important attributes in decision making over equity criteria. Hence, priority interventions would be those conditions that are most prevalent in the country and cost least per health gain.

CONCLUSION

Although efficiency criteria override equity ones, major health threats in China would be targeted. Multicriteria decision analysis makes explicit important trade-offs between efficiency and equity, leading to explicit, transparent and rational policy making.

STAR—People-Powered Prioritization

The aim of cost effectiveness analysis (CEA) is to inform the allocation of scarce resources. CEA is routinely used in assessing the cost-effectiveness of specific health technologies by agencies such as the National Institute for Health and Clinical Excellence (NICE) in England and Wales. But there is extensive evidence that because of barriers of accessibility and acceptability, CEA has not been used by local health planners in their annual task of allocating fixed budgets to a wide range of types of health care. This paper argues that these planners can use Socio Technical Allocation of Resources (STAR) for that task. STAR builds on the principles of CEA and the practice of program budgeting and marginal analysis. STAR uses requisite models to assess the cost-effectiveness of all interventions considered for resource reallocation by explicitly applying the theory of health economics to evidence of scale, costs, and benefits, with deliberation facilitated through an interactive social process of engaging key stakeholders. In that social process, the stakeholders generate missing estimates of scale, costs, and benefits of the interventions; develop visual models of their relative cost-effectiveness; and interpret the results. We demonstrate the feasibility of STAR by showing how it was used by a local health planning agency of the English National Health Service, the Isle of Wight Primary Care Trust, to allocate a fixed budget in 2008 and 2009.

Multi-criteria decision analysis of breast cancer control in low- and middle- income countries: development of a rating tool for policy makers

BACKGROUND

The objective of this study was to develop a rating tool for policy makers to prioritize breast cancer interventions in low- and middle- income countries (LMICs), based on a simple multi-criteria decision analysis (MCDA) approach. The definition and identification of criteria play a key role in MCDA, and our rating tool could be used as part of a broader priority setting exercise in a local setting. This tool may contribute to a more transparent priority-setting process and fairer decision-making in future breast cancer policy development.

METHODS

First, an expert panel (n = 5) discussed key considerations for tool development. A literature review followed to inventory all relevant criteria and construct an initial set of criteria. A Delphi study was then performed and questionnaires used to discuss a final list of criteria with clear definitions and potential scoring scales. For this Delphi study, multiple breast cancer policy and priority-setting experts from different LMICs were selected and invited by the World Health Organization. Fifteen international experts participated in all three Delphi rounds to assess and evaluate each criterion.

RESULTS

This study resulted in a preliminary rating tool for assessing breast cancer interventions in LMICs. The tool consists of 10 carefully crafted criteria (effectiveness, quality of the evidence, magnitude of individual health impact, acceptability, cost-effectiveness, technical complexity, affordability, safety, geographical coverage, and accessibility), with clear definitions and potential scoring scales.

CONCLUSIONS

This study describes the development of a rating tool to assess breast cancer interventions in LMICs. Our tool can offer supporting knowledge for the use or development of rating tools as part of a broader (MCDA based) priority setting exercise in local settings. Further steps for improving the tool are proposed and should lead to its useful adoption in LMICs.

Health sector priority setting at meso-level in lower and middle income countries: lessons learned, available options and suggested steps

Setting priority for health programming and budget allocation is an important issue, but there is little consensus on related processes. It is particularly relevant in low resource settings and at province- and district- or "meso-level", where contextual influences may be greater, information scarce and capacity lower. Although recent changes in disease epidemiology and health financing suggest even greater need to allocate resources effectively, the literature is relatively silent on evidence-based priority-setting in low and middle income countries (LMICs). We conducted a comprehensive review of the peer-reviewed and grey literature on health resource priority-setting in LMICs, focussing on meso-level and the evidence-based priority-setting processes (PSPs) piloted or suggested there. Our objective was to assess PSPs according to whether they have influenced resource allocation and impacted the outcome indicators prioritised. An exhaustive search of the peer-reviewed and grey literature published in the last decade yielded 57 background articles and 75 reports related to priority-setting at meso-level in LMICs. Although proponents of certain PSPs still advocate their use, other experts instead suggest broader elements to guide priority-setting. We conclude that currently no process can be confidently recommended for such settings. We also assessed the common reasons for failure at all levels of priority-setting and concluded further that local authorities should additionally consider contextual and systems limitations likely to prevent a satisfactory process and outcomes, particularly at meso-level. Recent literature proposes a list of related attributes and warning signs, and facilitated our preparation of a simple decision-tree or roadmap to help determine whether or not health systems issues should be improved in parallel to support for needed priority-setting; what elements of the PSP need improving; monitoring, and evaluation. Health priority-setting at meso-level in LMICs can involve common processes, but will often require additional attention to local health systems.

Balancing efficiency, equity and feasibility of HIV treatment in South Africa - development of programmatic guidance

South Africa, the country with the largest HIV epidemic worldwide, has been scaling up treatment since 2003 and is rapidly expanding its eligibility criteria. The HIV treatment programme has achieved significant results, and had 1.8 million people on treatment per 2011. Despite these achievements, it is now facing major concerns regarding (i) efficiency: alternative treatment policies may save more lives for the same budget; (ii) equity: there are large inequalities in who receives treatment; (iii) feasibility: still only 52% of the eligible population receives treatment.Hence, decisions on the design of the present HIV treatment programme in South Africa can be considered suboptimal. We argue there are two fundamental reasons to this. First, while there is a rapidly growing evidence-base to guide priority setting decisions on HIV treatment, its included studies typically consider only one criterion at a time and thus fail to capture the broad range of values that stakeholders have. Second, priority setting on HIV treatment is a highly political process but it seems no adequate participatory processes are in place to incorporate stakeholders' views and evidences of all sorts.We propose an alternative approach that provides a better evidence base and outlines a fair policy process to improve priority setting in HIV treatment. The approach integrates two increasingly important frameworks on health care priority setting: accountability for reasonableness (A4R) to foster procedural fairness, and multi-criteria decision analysis (MCDA) to construct an evidence-base on the feasibility, efficiency, and equity of programme options including trade-offs. The approach provides programmatic guidance on the choice of treatment strategies at various decisions levels based on a sound conceptual framework, and holds large potential to improve HIV priority setting in South Africa.

Multi-criteria development and incorporation into decision tools for health technology adoption

PURPOSE

When introducing new health technologies, decision makers must integrate research evidence with local operational management information to guide decisions about whether and under what conditions the technology will be used. Multi-criteria decision analysis can support the adoption or prioritization of health interventions by using criteria to explicitly articulate the health organization's needs, limitations, and values in addition to evaluating evidence for safety and effectiveness. This paper seeks to describe the development of a framework to create agreed-upon criteria and decision tools to enhance a pre-existing local health technology assessment (HTA) decision support program.

DESIGN/METHODOLOGY/APPROACH

The authors compiled a list of published criteria from the literature, consulted with experts to refine the criteria list, and used a modified Delphi process with a group of key stakeholders to review, modify, and validate each criterion. In a workshop setting, the criteria were used to create decision tools.

FINDINGS

A set of user-validated criteria for new health technology evaluation and adoption was developed and integrated into the local HTA decision support program. Technology evaluation and decision guideline tools were created using these criteria to ensure that the decision process is systematic, consistent, and transparent.

PRACTICAL IMPLICATIONS

This framework can be used by others to develop decision-making criteria and tools to enhance similar technology adoption programs.

ORIGINALITY/VALUE

The development of clear, user-validated criteria for evaluating new technologies adds a critical element to improve decision-making on technology adoption, and the decision tools ensure consistency, transparency, and real-world relevance.

Allocating limited resources in a time of fiscal constraints: a priority setting case study from Dalhousie University Faculty of Medicine

Facing a projected $1.4M deficit on a $35M operating budget for fiscal year 2011/2012, members of the Dalhousie University Faculty of Medicine developed and implemented an explicit, transparent, criteria-based priority setting process for resource reallocation. A task group that included representatives from across the Faculty of Medicine used a program budgeting and marginal analysis (PBMA) framework, which provided an alternative to the typical public-sector approaches to addressing a budget deficit of across-the-board spending cuts and political negotiation. Key steps to the PBMA process included training staff members and department heads on priority setting and resource reallocation, establishing process guidelines to meet immediate and longer-term fiscal needs, developing a reporting structure and forming key working groups, creating assessment criteria to guide resource reallocation decisions, assessing disinvestment proposals from all departments, and providing proposal implementation recommendations to the dean. All departments were required to submit proposals for consideration. The task group approved 27 service reduction proposals and 28 efficiency gains proposals, totaling approximately $2.7M in savings across two years. During this process, the task group faced a number of challenges, including a tight timeline for development and implementation (January to April 2011), a culture that historically supported decentralized planning, at times competing interests (e.g., research versus teaching objectives), and reductions in overall health care and postsecondary education government funding. Overall, faculty and staff preferred the PBMA approach to previous practices. Other institutions should use this example to set priorities in times of fiscal constraints.

Broader economic evaluation of disease management programs using multi-criteria decision analysis

OBJECTIVES

The aim of this paper is to develop a methodological framework to facilitate the application of Multi-Criteria Decision Analysis (MCDA) for a comprehensive economic evaluation of disease management programs (DMPs).

METHODS

We studied previously developed frameworks for the evaluation of DMPs and different methods of MCDA and we used practical field experience in the economic evaluation of DMPs and personal discussions with stakeholders in chronic care.

RESULTS

The framework includes different objectives and criteria that are relevant for the evaluation of DMPs, indicators that can be used to measure how DMPs perform on these criteria, and distinguishes between the development and implementation phase of DMPs. The objectives of DMPs are categorised into a) changes in the process of care delivery, b) changes in patient lifestyle and self-management behaviour, c) changes in biomedical, physiological and clinical health outcomes, d) changes in health-related quality of life, and e) changes in final health outcomes. All relevant costs of DMPs are also included in the framework. Based on this framework we conducted a MCDA of a hypothetical DMP versus usual care.

CONCLUSIONS

We call for a comprehensive economic evaluation of DMPs that is not just based on a single criterion but takes into account multiple relevant criteria simultaneously. The framework we presented here is a step towards standardising such an evaluation.

Priority-setting for evidence-based health outreach in community-based organizations: A mixed-methods study in three Massachusetts communities

Priority setting, or determining how to best allocate limited resources, is an important first step for evidence-based public health approaches in community-based organizations (CBOs), but guidance for such work is limited. This study aims to study drivers of priority setting and the way CBOs use data for this work. Data come from PLANET MassCONECT, a Community-Based Participatory Research project focused on knowledge translation among CBOs targeting the underserved in Boston, Lawrence, and Worcester, MA. We conducted four focus group discussions with CBO staff members (31 participants) in 2008 and a survey of 214 CBO staff members in 2009. Multiple, often competing factors appear to drive priority setting, including data, funding, partnerships, and community preferences. The process may be hindered by challenges related to finding, evaluating, and utilizing data for priority setting. Supporting CBOs in efforts to use data effectively and incorporate context into systematic priority-setting processes is vital.

Disease control priorities for neglected tropical diseases: lessons from priority ranking based on the quality of evidence, cost effectiveness, severity of disease, catastrophic health expenditures, and loss of productivity

BACKGROUND

In the context of limited health care budgets in countries where Neglected Tropical Diseases (NTDs) are endemic, scaling up disease control interventions entails the setting of priorities. However, solutions based solely on cost-effectiveness analyses may lead to biased and insufficiently justified priorities.

OBJECTIVES

The objectives of this paper are to 1) demonstrate how a range of equity concerns can be used to identify feasible priority setting criteria, 2) show how these criteria can be fed into a multi-criteria decision-making matrix, and 3) discuss the conditions under which this decision-making procedure should be carried out in a real-world decision-making context.

METHODS

This paper draws on elements from theories of decision analysis and ethical theories of fair resource allocation. We explore six typical NTD interventions by employing a modified multi-criteria decision analysis model with predefined criteria, drawn from a priority setting guide under development by the WHO. To identify relevant evidence for the six chosen interventions, we searched the PubMed and Cochrane databases.

DISCUSSION

Our in vitro multi-criteria decision analysis suggested that case management for visceral leishmaniasis should be given a higher priority than mass campaigns to prevent soil-transmitted helminthic infections. This seems to contradict current health care priorities and recommendations in the literature. We also consider procedural conditions that should be met in a contextualised decision-making process and we stress the limitations of this study exercise.

CONCLUSION

By exploring how several criteria relevant to the multi-facetted characteristics of NTDs can be taken into account simultaneously, we are able to suggest how improved priority settings among NTDs can be realised.

The challenge of obtaining information necessary for multi-criteria decision analysis implementation: the case of physiotherapy services in Canada

BACKGROUND

As fiscal constraints dominate health policy discussions across Canada and globally, priority-setting exercises are becoming more common to guide the difficult choices that must be made. In this context, it becomes highly desirable to have accurate estimates of the value of specific health care interventions.Economic evaluation is a well-accepted method to estimate the value of health care interventions. However, economic evaluation has significant limitations, which have lead to an increase in the use of Multi-Criteria Decision Analysis (MCDA). One key concern with MCDA is the availability of the information necessary for implementation. In the Fall 2011, the Canadian Physiotherapy Association embarked on a project aimed at providing a valuation of physiotherapy services that is both evidence-based and relevant to resource allocation decisions. The framework selected for this project was MCDA. We report on how we addressed the challenge of obtaining some of the information necessary for MCDA implementation.

METHODS

MCDA criteria were selected and areas of physiotherapy practices were identified. The building up of the necessary information base was a three step process. First, there was a literature review for each practice area, on each criterion. The next step was to conduct interviews with experts in each of the practice areas to critique the results of the literature review and to fill in gaps where there was no or insufficient literature. Finally, the results of the individual interviews were validated by a national committee to ensure consistency across all practice areas and that a national level perspective is applied.

RESULTS

Despite a lack of research evidence on many of the considerations relevant to the estimation of the value of physiotherapy services (the criteria), sufficient information was obtained to facilitate MCDA implementation at the local level.

CONCLUSIONS

The results of this research project serve two purposes: 1) a method to obtain information necessary to implement MCDA is described, and 2) the results in terms of information on the benefits provided by each of the twelve areas of physiotherapy practice can be used by decision-makers as a starting point in the implementation of MCDA at the local level.

Priority-setting and rationing in healthcare: evidence from the English experience

In a context of ever increasing demand, the recent economic downturn has placed further pressure on decision-makers to effectively target healthcare resources. Over recent years there has been a push to develop more explicit evidence-based priority-setting processes, which aim to be transparent and inclusive in their approach and a number of analytical tools and sources of evidence have been developed and utilised at national and local levels. This paper reports findings from a qualitative research study which investigated local priority-setting activity across five English Primary Care Trusts, between March and November 2012. Findings demonstrate the dual aims of local decision-making processes: to improve the overall effectiveness of priority-setting (i.e. reaching 'correct' resource allocation decisions); and to increase the acceptability of priority-setting processes for those involved in both decision-making and implementation. Respondents considered priority-setting processes to be compartmentalised and peripheral to resource planning and allocation. Further progress was required with regard to disinvestment and service redesign with respondents noting difficulty in implementing decisions. While local priority-setters had begun to develop more explicit processes, public awareness and input remained limited. The leadership behaviours required to navigate the political complexities of working within and across organisations with differing incentives systems and cultures remained similarly underdeveloped.

Priority setting and policy advocacy by nursing associations: a scoping review and implications using a socio-ecological whole systems lens

OBJECTIVE

We undertook an interpretative scoping review to examine organizational priority setting and policy advocacy and the factors that influence nursing associations' cross-sector public policy choices and actions.

METHOD

Evidence was drawn from research, narrative, and theoretical sources that described priority setting and policy advocacy undertaken by non-governmental, non-profit, and nursing associations. Text was extracted from selected papers, imported into NVivo 8, coded, and analyzed using a descriptive-analytical narrative method.

RESULTS

Many internal and external factors are shown to shape organizations' policy choices and actions including governance and governance structures, membership arrangements, legislative, professional, and jurisdictional mandates, perceived credibility, and external system disruptions.

CONCLUSIONS

Internal and external factors are identified in the literature as critical to how organizations succeed or fail to set achievable priorities and advance their advocacy goals. Case comparisons and longitudinal research are needed to understand nursing associations' policy choices and actions for cross-sector public policy given their complex organizational structures and dynamic professional-legal-social-economic-political-ecological environments. A socio-ecological systems perspective can inform the development of theoretical frameworks and research to understand leverage points and blockages to guide nursing associations' public policy choices and actions at varying points in time.

Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada

BACKGROUND

Consistent healthcare decision making requires systematic consideration of decision criteria and evidence available to inform them. This can be tackled by combining multicriteria decision analysis (MCDA) and Health Technology Assessment (HTA). The objective of this study was to field-test a decision support framework (EVIDEM), explore its utility to a drug advisory committee and test its reliability over time.

METHODS

Tramadol for chronic non-cancer pain was selected by the health plan as a case study relevant to their context. Based on extensive literature review, a by-criterion HTA report was developed to provide synthesized evidence for each criterion of the framework (14 criteria for the MCDA Core Model and 6 qualitative criteria for the Contextual Tool). During workshop sessions, committee members tested the framework in three steps by assigning: 1) weights to each criterion of the MCDA Core Model representing individual perspective; 2) scores for tramadol for each criterion of the MCDA Core Model using synthesized data; and 3) qualitative impacts of criteria of the Contextual Tool on the appraisal. Utility and reliability of the approach were explored through discussion, survey and test-retest. Agreement between test and retest data was analyzed by calculating intra-rater correlation coefficients (ICCs) for weights, scores and MCDA value estimates.

RESULTS

The framework was found useful by the drug advisory committee in supporting systematic consideration of a broad range of criteria to promote a consistent approach to appraising healthcare interventions. Directly integrated in the framework as a "by-criterion" HTA report, synthesized evidence for each criterion facilitated its consideration, although this was sometimes limited by lack of relevant data. Test-retest analysis showed fair to good consistency of weights, scores and MCDA value estimates at the individual level (ICC ranging from 0.676 to 0.698), thus lending some support for the reliability of the approach. Overall, committee members endorsed the inclusion of most framework criteria and revealed important areas of discussion, clarification and adaptation of the framework to the needs of the committee.

CONCLUSIONS

By promoting systematic consideration of all decision criteria and the underlying evidence, the framework allows a consistent approach to appraising healthcare interventions. Further testing and validation are needed to advance MCDA approaches in healthcare decisionmaking.

Bridging Health Technology Assessment (HTA) and Efficient Health Care Decision Making with Multicriteria Decision Analysis (MCDA)

Background. Health care decision making is complex and requires efficient and explicit processes to ensure transparency and consistency of factors considered. Objectives. To pilot an adaptable decision-making framework incorporating multicriteria decision analysis (MCDA) in health technology assessment (HTA) with a pan-Canadian group of policy and clinical decision makers and researchers appraising 10 medicines covering 6 therapeutic areas. Methods. An appraisal group was convened and participants were asked to express their individual perspectives, independently of the medicines, by assigning weights to each criterion of the MCDA core model: disease severity, size of population, current practice and unmet needs, intervention outcomes (efficacy, safety, patient reported), type of health benefit, economics, and quality of evidence. Participants then assigned performance scores for each medicine using available evidence synthesized in a “by-criterion” HTA report covering each of the MCDA CORE model criteria. MCDA estimates of perceived value were calculated by combining normalized weights and scores. Feedback on the approach was collected through structured discussion. Results. Relative weights on criteria varied widely, reflecting the diverse perspectives of participants. Scores for each criterion provided a performance measure, highlighting strengths and weaknesses of each medicine. MCDA estimates of perceived value ranged from 0.42 to 0.64 across medicines, providing comprehensive measures incorporating a large spectrum of criteria. Participants reported that the framework provided an efficient approach to systematic consideration in a pragmatic format of the multiple elements guiding decision, including criteria and values (MCDA core model) and evidence (HTA “by-criterion” report). Conclusions. This proof-of-concept study demonstrated the usefulness of incorporating MCDA in HTA to support transparent and systematic appraisal of health care interventions. Further research is needed to advance MCDA-based approaches to more effective healthcare decision making.

Institutionalizing telemedicine applications: the challenge of legitimizing decision-making

During the last decades a variety of telemedicine applications have been trialed worldwide. However, telemedicine is still an example of major potential benefits that have not been fully attained. Health care regulators are still debating why institutionalizing telemedicine applications on a large scale has been so difficult and why health care professionals are often averse or indifferent to telemedicine applications, thus preventing them from becoming part of everyday clinical routines. We believe that the lack of consolidated procedures for supporting decision making by health care regulators is a major weakness. We aim to further the current debate on how to legitimize decision making about the institutionalization of telemedicine applications on a large scale. We discuss (1) three main requirements--rationality, fairness, and efficiency--that should underpin decision making so that the relevant stakeholders perceive them as being legitimate, and (2) the domains and criteria for comparing and assessing telemedicine applications--benefits and sustainability. According to these requirements and criteria, we illustrate a possible reference process for legitimate decision making about which telemedicine applications to implement on a large scale. This process adopts the health care regulators' perspective and is made up of 2 subsequent stages, in which a preliminary proposal and then a full proposal are reviewed.

Priority setting in healthcare: towards guidelines for the program budgeting and marginal analysis framework

Economists' approaches to priority setting focus on the principles of opportunity cost, marginal analysis and choice under scarcity. These approaches are based on the premise that it is possible to design a rational priority setting system that will produce legitimate changes in resource allocation. However, beyond issuing guidance at the national level, economic approaches to priority setting have had only a moderate impact in practice. In particular, local health service organizations - such as health authorities, health maintenance organizations, hospitals and healthcare trusts - have had difficulty implementing evidence from economic appraisals. Yet, in the context of making decisions between competing claims on scarce health service resources, economic tools and thinking have much to offer. The purpose of this article is to describe and discuss ten evidence-based guidelines for the successful design and implementation of a program budgeting and marginal analysis (PBMA) priority setting exercise. PBMA is a framework that explicitly recognizes the need to balance pragmatic and ethical considerations with economic rationality when making resource allocation decisions. While the ten guidelines are drawn from the PBMA framework, they may be generalized across a range of economic approaches to priority setting.

Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decision-making framework to growth hormone for Turner syndrome patients

OBJECTIVES

To test and further develop a healthcare policy and clinical decision support framework using growth hormone (GH) for Turner syndrome (TS) as a complex case study.

METHODS

The EVIDEM framework was further developed to complement the multicriteria decision analysis (MCDA) Value Matrix, that includes 15 quantifiable components of decision clustered in four domains (quality of evidence, disease, intervention and economics), with a qualitative tool including six ethical and health system-related components of decision. An extensive review of the literature was performed to develop a health technology assessment report (HTA) tailored to each component of decision, and content was validated by experts. A panel of representative stakeholders then estimated the MCDA value of GH for TS in Canada by assigning weights and scores to each MCDA component of decision and then considered the impact of non-quantifiable components of decision.

RESULTS

Applying the framework revealed significant data gaps and the importance of aligning research questions with data needs to truly inform decision. Panelists estimated the value of GH for TS at 41% of maximum value on the MCDA scale, with good agreement at the individual level (retest value 40%; ICC: 0.687) and large variation across panelists. Main contributors to this panel specific value were "Improvement of efficacy", "Disease severity" and "Quality of evidence". Ethical considerations on utility, efficiency and fairness as well as potential misuse of GH had mixed effects on the perceived value of the treatment.

CONCLUSIONS

This framework is proposed as a pragmatic step beyond the current cost-effectiveness model, combining HTA, MCDA, values and ethics. It supports systematic consideration of all components of decision and available evidence for greater transparency. Further testing and validation is needed to build up MCDA approaches combined with pragmatic HTA in healthcare decision-making.

Multi-criteria decision analysis to prioritize health interventions: Capitalizing on first experiences

This paper capitalizes on a first set of experiences on the application of multi-criteria decision analysis (MCDA) in seven low- and middle-income settings. It thereby reacts to a recent paper by Peacock et al., highlighting the potential of MCDA to guide policy makers in highly specific decision-making contexts. We argue that MCDA also has a broader application in setting priorities in health, i.e. to indicate general perceptions on priorities without defining the allocation of resources in a precise fashion. This use of MCDA can have far-reaching and constructive influences on policy formulation.