Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers

Priority setting to support a public health research agenda: a modified Delphi study with public health stakeholders in Germany

BACKGROUND

Research priority setting (RPS) studies are necessary to close the significant gap between the scientific evidence produced and the evidence stakeholders need. Their findings can make resource allocation in research more efficient. However, no general framework for conducting an RPS study among public health stakeholders exists. RPS studies in public health are rare and no such study has been previously conducted and published in Germany. Therefore, we aimed to investigate which research topics in public health are prioritised by relevant stakeholders in Germany.

METHODS

Our RPS study consisted of a scoping stage and a Delphi stage each split into two rounds. Firstly, we invited members of the German Public Health Association to gather expert insights during two initial workshops. Next, we defined the relevant stakeholder groups and recruited respondents. Thereafter, we collected research topics and assessment criteria with the respondents in the first Delphi round and aggregated the responses through content analysis. Finally, we asked the respondents to rate the research topics with the assessment criteria in the second Delphi round.

RESULTS

In total, 94 out of the 140 invited public health organisations nominated 230 respondents for the Delphi study of whom almost 90% participated in both Delphi rounds. We compiled a comprehensive list of 76 research topics that were rated and ranked by several assessment criteria. We split the research topics into two types, substantive research topics and methodological-theoretical research topics respectively, to ensure the comparability among the research topics. In both types of research topics-substantive research topics and methodological-theoretical research topics-the respective top five ranked research topics hardly differed between public health researchers and public health practitioners. However, clear differences exist in the priority ranking of many (non-top priority) research topics between the stakeholder groups.

CONCLUSIONS

This research demonstrates that it is possible, with limited resources, to prioritise research topics for public health at the national level involving a wide range of pertinent stakeholders. The results can be used by research funding institutions to initiate calls for research projects with an increased relevance for health and/or scientific progress.

Decision-making processes for essential packages of health services: experience from six countries

Many countries around the world strive for universal health coverage, and an essential packages of health services (EPHS) is a central policy instrument for countries to achieve this. It defines the coverage of services that are made available, as well as the proportion of the costs that are covered from different financial schemes and who can receive these services. This paper reports on the development of an analytical framework on the decision-making process of EPHS revision, and the review of practices of six countries (Afghanistan, Ethiopia, Pakistan, Somalia, Sudan and Zanzibar-Tanzania).The analytical framework distinguishes the practical organisation, fairness and institutionalisation of decision-making processes. The review shows that countries: (1) largely follow a similar practical stepwise process but differ in their implementation of some steps, such as the choice of decision criteria; (2) promote fairness in their EPHS process by involving a range of stakeholders, which in the case of Zanzibar included patients and community members; (3) are transparent in terms of at least some of the steps of their decision-making process and (4) in terms of institutionalisation, express a high degree of political will for ongoing EPHS revision with almost all countries having a designated governing institute for EPHS revision.We advise countries to organise meaningful stakeholder involvement and foster the transparency of the decision-making process, as these are key to fairness in decision-making. We also recommend countries to take steps towards the institutionalisation of their EPHS revision process.

Evidence-Informed Deliberative Processes for Health Benefit Package Design - Part II: A Practical Guide

BACKGROUND

Countries around the world are using health technology assessment (HTA) for health benefit package design. Evidence-informed deliberative processes (EDPs) are a practical and stepwise approach to enhance legitimate health benefit package design based on deliberation between stakeholders to identify, reflect and learn about the meaning and importance of values, informed by evidence on these values. This paper reports on the development of practical guidance on EDPs, while the conceptual framework of EDPs is described in a companion paper.

METHODS

The first guide on EDPs (2019) is further developed based on academic knowledge exchange, surveying 27 HTA bodies and 66 experts around the globe, and the implementation of EDPs in several countries. We present the revised steps of EDPs and how selected HTA bodies (in Australia, Brazil, Canada, France, Germany, Scotland, Thailand and the United Kingdom) organize key issues of legitimacy in their processes. This is based on a review of literature via PubMed and HTA bodies' websites.

RESULTS

HTA bodies around the globe vary considerable in how they address legitimacy (stakeholder involvement ideally through participation with deliberation; evidence-informed evaluation; transparency; and appeal) in their processes. While there is increased attention for improving legitimacy in decision-making processes, we found that the selected HTA bodies are still lacking or just starting to develop activities in this area. We provide recommendations on how HTA bodies can improve on this.

CONCLUSION

The design and implementation of EDPs is in its infancy. We call for a systematic analysis of experiences of a variety of countries, from which general principles on EDPs might subsequently be inferred.

The value of vaccines

Optimizing vaccine spending depends on recognizing the full value of vaccination (VoV). Existing taxonomies of such value are not comprehensive because they are not guided by general theories. I rely on two such theories: subjective-value theory claims that what has value is determined by what people actually or ideally want in life. A welfarist theory of government states that a fundamental objective of government is to promote social value (or social welfare). These jointly imply that any aspect of life that individuals actually or ideally value and that could be negatively affected by vaccine-preventable diseases (and therefore positively affected by preventive vaccines) is an element of VoV. I build a more comprehensive-value taxonomy than currently exists based on this implication.

Priorities for rheumatic and musculoskeletal disease research in Ireland

Background

Research priority setting is a useful approach to decide which unanswered questions are most worth trying to solve through research. The aim is to reduce bias in the research agenda. Traditionally, research was decided by funders, policymakers, and academics with limited influence from other stakeholders like people living with health conditions, caregivers, or the community. This can lead to research gaps that fail to address these important stakeholder needs. The objective of this study is to identify the top research priorities for Rheumatic and Musculoskeletal Disease (RMD) research in Ireland.

Methods

The process framework included a design workshop, two online surveys and a review of the literature.

Participants

545 people completed the first survey to identify RMD research topics relevant to Ireland, of which 72% identified as a person living with RMD. 460 people completed the second survey to prioritise these research topics.

Results

The first survey had 2185 research topics submitted. These were analysed and grouped into 38 topic areas which were ranked in the second survey. The top three research priorities for RMD research in Ireland focused on preventing RMD progression, RMD diagnosis and its impact, and pain management.

Conclusions

The prioritised research topics indicate important areas of RMD research for Ireland. Research funded in response to these co-created research priorities will have increased relevance and impact.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41927-022-00285-9.

Prioritising child health and maternity evidence-based interventions or service models: a stakeholder-driven process

Aim

A UK programme, led by the National Institute for Health Research (NIHR) (https://www.nihr.ac.uk) and coordinated by Applied Research Collaborations (ARC), (https://www.nihr.ac.uk/explore-nihr/support/collaborating-in-applied-health-research.htm) aimed to identify and select evidence-based, implementation-ready service innovations for evaluation. The programme focused on seven areas of health provision. We report on a prioritisation process designed to identify and assess innovations in one of these areas: child and maternal health (CH&M).

Methods

We developed a three-stage, online, stakeholder driven process to 1) identify, 2) assess and prioritise and 3) select evidence-based interventions or service models, using crowdsourcing to identify projects and the APEASE criteria to assess and select projects. A brief evidence review was conducted for all initial suggestions to identify those with the largest evidence-base to take forward for ranking by stakeholders. Stakeholder workshops considered and ranked these suggestions using the APEASE criteria. We then conducted in-depth evidence reviews for the highest ranked suggestions. The Project Management Group and Advisory Board used these reviews and the APEASE criteria to select the final projects.

Results

We received 32 initial suggestions from a range of clinicians, practitioners and researchers. Fourteen of the most evidence-based suggestions were considered and ranked at four themed stakeholder workshops. Nine suggestions were ranked for further in-depth evidence review and a final four projects were selected for implementation evaluation using the APEASE criteria. These were: 1. Maternal Mental Health Services Multidisciplinary Teams 2. Early years tooth brushing programme 3. Trauma-focused CBT for young people in care and 4. Independent Domestic Violence Advisors in maternity settings. Feedback from participants suggested that having public representatives participating in all stakeholder meetings, rather than being consulted separately, focused discussions clearly on patient benefit rather than research aims.

Conclusions

The stakeholder-driven process achieved its aim of identifying, prioritising and assessing and selecting, evidence-based projects for wider implementation and evaluation. The concurrent process could be adapted by other researchers or policy makers.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08110-2.

Guidelines that use the GRADE approach often fail to provide complete economic information for recommendations: A systematic survey

OBJECTIVE

Little is known about how developers and panel members report cost and cost effectiveness considerations in GRADE guideline Evidence-to-Decision (EtD) frameworks. A systematic survey was conducted to explore approaches and factors contributing to variability in economic information reporting.

STUDY DESIGN AND SETTING

Guideline organization websites were systematically searched to create a convenience sample of guidelines. Reviewers screened published EtD frameworks and generated frequencies of reporting approaches. We used thematic analysis to summarize factors related to variability of economic information reporting.

RESULTS

We included 142 guidelines. The overall rate of reporting economic information was high (91%); however, there was variability across completion of predefined EtD Likert-type judgments (70%), noting information as not identified across EtD framework domains (57%), and providing remarks to justify recommendations (38%). Six themes contributing to variability emerged, related to: intervention, population, payor, provider, healthcare resource use, and economic model building factors. Only 2 guidelines performed a GRADE certainty appraisal of economic outcomes.

CONCLUSION

Completing predefined EtD Likert-type judgments, specifically reporting a literature review approach, study selection criteria and economic model building limitations, as well as linking these to recommendation justification remarks are potential areas for improved use, adoption and adaptation of recommendation, and transparency of GRADE EtD frameworks.

A SBM-DEA based performance evaluation and optimization for social organizations participating in community and home-based elderly care services

The community and home-based elderly care service system has been proved an effective pattern to mitigate the elderly care dilemma under the background of accelerating aging in China. In particular, the participation of social organizations in community and home-based elderly care service has powerfully fueled the multi-supply of elderly care. As the industry of the elderly care service is in the ascendant, the management lags behind, resulting in the waste of significant social resources. Therefore, performance evaluation is proposed to resolve this problem. However, a systematic framework for evaluating performance of community and home-based elderly care service centers (CECSCs) is absent. To overcome this limitation, the SBM-DEA model is introduced in this paper to evaluate the performance of CECSCs. 186 social organizations in Nanjing were employed as an empirical study to develop the systematic framework for performance evaluation. Through holistic analysis of previous studies and interviews with experts, a systematic framework with 33 indicators of six dimensions (i.e., financial management, hardware facilities, team building, service management, service object and organization construction) was developed. Then, Sensitivity Analysis is used to screen the direction of performance optimization and specific suggestions were put forward for government, industrial associations and CECSCs to implement. The empirical study shows the proposed framework using SBM-DEA and sensitivity analysis is viable for conducting performance evaluation and improvement of CECSCs, which is conducive to the sustainable development of CECSCs.

Building resource constraints and feasibility considerations in mathematical models for infectious disease: A systematic literature review

•

Mathematical model capabilities to explore complex systems now enable priority-setting to consider local resource constraints.

•

Common objectives of model-based analyses incorporating constraints are to assess real-world feasibility or allocate resources efficiently.

•

Constraints may be incorporated via (i) model-based estimation; (ii) linkage of mathematical and health system models; or (iii) optimisation.

•

Models can then project constrained intervention effects and costs and resource requirement s for delivering interventions at full scale.

•

'Health system constraints' should be systematically defined for routine operationalisation in model-based priority-setting.

Priority setting for infectious disease control is increasingly concerned with physical input constraints and other real-world restrictions on implementation and on the decision process. These health system constraints determine the ‘feasibility’ of interventions and hence impact. However, considering them within mathematical models places additional demands on model structure and relies on data availability. This review aims to provide an overview of published methods for considering constraints in mathematical models of infectious disease.

We systematically searched the literature to identify studies employing dynamic transmission models to assess interventions in any infectious disease and geographical area that included non-financial constraints to implementation. Information was extracted on the types of constraints considered and how these were identified and characterised, as well as on the model structures and techniques for incorporating the constraints.

A total of 36 studies were retained for analysis. While most dynamic transmission models identified were deterministic compartmental models, stochastic models and agent-based simulations were also successfully used for assessing the effects of non-financial constraints on priority setting. Studies aimed to assess reductions in intervention coverage (and programme costs) as a result of constraints preventing successful roll-out and scale-up, and/or to calculate costs and resources needed to relax these constraints and achieve desired coverage levels. We identified three approaches for incorporating constraints within the analyses: (i) estimation within the disease transmission model; (ii) linking disease transmission and health system models; (iii) optimising under constraints (other than the budget).

The review highlighted the viability of expanding model-based priority setting to consider health system constraints. We show strengths and limitations in current approaches to identify and quantify locally-relevant constraints, ranging from simple assumptions to structured elicitation and operational models. Overall, there is a clear need for transparency in the way feasibility is defined as a decision criteria for its systematic operationalisation within models.

"Rehabilitation Research Framework for Patients With COVID-19" Defined by Cochrane Rehabilitation and the World Health Organization Rehabilitation Programme

The coronavirus disease 2019 (COVID-19) pandemic resulted in a surge of research activity. Since its outset, efforts have been made to guide the rapid generation of research in medicine. There are gaps in some areas of rehabilitation research for patients with COVID-19. The development of a specific research framework might serve to help monitor the status of research (mapping), shape and strengthen research by pointing to under-investigated areas, and promote rehabilitation research in this context. This article introduces and discusses the COVID-19 Rehabilitation Research Framework (CRRF) and presents the methodology used for its development. The questions have been developed among the World Health Organization (WHO) Rehabilitation Programme, Cochrane Rehabilitation, and the experts of its Rehabilitation-COVID-19 Evidence-based Response Action International Multiprofessional Steering Committee. The framework is divided into 2 parts and includes 20 questions organized in 4 groups: epidemiology, and evidence at the micro- (individual), meso- (health services), and macro- (health systems) levels. The CRRF offers a comprehensive view of the research areas relevant to COVID-19 and rehabilitation that are necessary to inform best practice and ensure rehabilitation services and health systems can best serve the population with COVID-19. The collaboration between Cochrane Rehabilitation and the WHO Rehabilitation Programme in establishing the CRRF brought together perspectives from the health systems, health management, and clinical evidence. The authors encourage researchers to use the CRRF when planning studies on rehabilitation in the context of COVID-19.

Multi-criteria decision analysis approach for strategy scale-up with application to Chagas disease management in Bolivia

OBJECTIVE

Design and build a strategy construction and evaluation software system to help stakeholders to develop viable strategies to expand (and adapt) the Chagas Platform healthcare model through the primary healthcare system in Bolivia.

METHODS

The software was built based on a ranking of medical Interventions and Actions (needed to support Interventions' implementation) needed for comprehensive management of Chagas Disease in Bolivia. The ranking was performed using a Multi Criteria Decision Analysis (MCDA) methodology adapted to the WHO's building blocks framework. Data regarding the criteria and the rankings was obtained through surveys and interviews with health care professionals working on Chagas disease. The Analytical Hierarchy Process was used to construct the decision criteria weights. Data Envelopment Analysis was used to identify the Interventions that lay on the efficiency frontier of outcomes and the complexity of associated Actions. These techniques were combined with integer programing tools using the open-source software R to build a decision-making tool to assess the outcomes and complexity of any combination of Interventions and Actions. This model and tool were applied to data concerning the care of Chagas disease in Bolivia collected through surveys of experts. The tool works by loading the data from each specific context.

RESULTS

The initial set of Interventions and Actions recommended after analysis of the survey data was further refined through face-to-face interviews with field experts in Bolivia, resulting in a strategy of 18 Interventions and 15 Actions. Within the WHO model the Leadership and Governance building block came up as the one needing more support with Actions such as the inclusion of Chagas into Annual Municipal Operational Plans by appointing local and provincial coordinators.

CONCLUSION

This project established the suitability of the model for constructing healthcare strategies. The model could be developed further resulting in a decision-making tool for program managers in a wide range of healthcare related issues, including neglected and/ or prevalent diseases. The tool has the potential to be used at different stages of decision making by diverse stakeholders in order to coordinate activities needed to address a health problem.

Can We Incorporate Societal Values in Resource Allocation Decisions Among Disease Categories? An Empirical Approach

OBJECTIVES

Historically, resource allocation decisions in healthcare are based on univariate approaches, inevitably overlooking value dimensions that are essential from a societal welfare maximization perspective. This article aims to present a wider perspective on decision making that incorporates societal values when prioritizing future resource allocation among disease areas.

METHODS

Sociotechnical application of multiple-criteria decision analysis with a set of criteria (value judgments) that are based on positive as well as normative dimensions of resource allocation. We use Greece as a case study. Societal value judgments were sourced via a multidisciplinary panel of experts who collectively provided criteria weights and scores for each alternative (16 disease categories, classified according to the Global Burden of Disease study) against each criterion. An additive value function provided the total value in priority preference for each alternative.

RESULTS

The criteria that were deemed relevant to the decision-making process and their respective relative weights were burden of disease (0.245), capacity to benefit (0.190), direct cost and projected changes in the next 5 years (0.160), indirect cost (0.132), intensity of unmet needs (0.109), incidence of catastrophic expenditure (0.091), and caring externalities (0.073). The additive value function revealed that the top 5 priorities in highest total value scores were neoplasms, circulatory diseases, injuries, neurologic diseases, and musculoskeletal diseases.

CONCLUSIONS

Incorporation of societal value criteria in resource allocation decisions can highlight priorities and lead to different sets of planning decisions than solely demand-driven allocation.

Creating a priority list of non-communicable diseases to support health research funding decision-making

OBJECTIVE

To develop and pilot a framework based on multi-criteria decision analysis (MCDA) to prioritize non-communicable diseases (NCDs) to support health research funding decision-making.

METHODS

The framework involves identifying NCDs to be prioritized, specifying prioritization criteria and determining their weights from a survey of stakeholders. The mean weights from the survey are applied to the NCDs' ratings on the criteria to generate a 'total score' for each NCD, by which the NCDs are prioritized.

RESULTS

Nineteen NCDs and five criteria were included. The criteria, in decreasing order of importance (mean weights in parentheses), are: deaths across the population (27.7 %), loss of quality-of-life across the population (23.0 %), cost to patients and families (18.6 %), cost to the health system (17.2 %), and whether vulnerable groups are disproportionately affected (13.4 %). The priority list of NCDs, stratified into four tiers of importance, is: 'Very critical' priority: coronary heart disease, back and neck pain, diabetes mellitus; 'Critical' priority: dementia and Alzheimer's disease, stroke; 'High' priority: colon and rectum cancer, depressive disorders, chronic obstructive pulmonary disease, chronic kidney disease, breast cancer, prostate cancer, arthritis, lung cancer; and 'Medium' priority: asthma, hearing loss, melanoma skin cancer, addictive disorders, non-melanoma skin cancer, headaches.

CONCLUSION

The results indicate the framework for prioritizing NCDs for research funding is feasible and effective. The framework could also be used for other health conditions.

Pathway towards an ideal and sustainable framework agreement for the public procurement of vaccines in Spain: a multi-criteria decision analysis

Objective: To advance the development of an ideal and sustainable framework agreement for the public procurement of vaccines in Spain, and to agree on the desirable award criteria and their relative weight. Methods: A multidisciplinary committee of seven health-care professionals and managers developed a partial multi-criteria decision analysis to determine the award criteria that should be considered and their specific weights for the public procurement of routine vaccines and seasonal influenza vaccines, considering their legal viability. A re-test of the results was carried out. The current situation was analyzed through 118 tender specifications and compared to the ideal framework. Results: Price is the prevailing award criterion for the public procurement of both routine (weighting of 60% versus 40% for all other criteria) and influenza (36% versus 64%) vaccines. Ideally, 22 criteria should be considered for routine vaccines, grouped and weighted into five domains: efficacy (weighting of 29%), economic aspects (27%), vaccine characteristics (22%), presentation form and packaging (13%), and others (9%). Per criteria set, price was the most important criterion (22%), followed by effectiveness (9%), and composition/formulation (7%). Regarding influenza vaccines, 20 criteria were selected, grouped, and weighted: efficacy (29%), economic aspects (25%), vaccine characteristics (20%), presentation form and packaging (16%), and others (11%). Per criteria set, price was also the most relevant criterion (19%), followed by composition/formulation (8%), and effectiveness (8%). Conclusions: Contrary to the current approach, technical award criteria should prevail over economic criteria in an ideal and sustainable framework agreement for the public procurement of vaccines.

Organizational aspect in healthcare decision-making: a literature review

BACKGROUND

Organizational aspect is rarely considered in healthcare. However, it is gradually seen as one of the key aspects of the decision-making process as well as clinical and economic dimensions. Our primary objective was to identify criteria already used to assess the organizational impact of medical innovations. Our secondary objective was to structure them into an inventory to support decision-makers to select the relevant criteria for their complex decision-making issues.

MATERIALS AND METHODS

A search using the Medline database was conducted in June 2019. The records published between January, 1990 and December, 2018 were identified. The publications cited by the authors of the included articles and the websites of health technology assessment agencies, units or learned societies identified during the search were also consulted. The identified criteria were structured in an inventory.

RESULTS

We selected 107 records of a wide range of evidence mostly published after the 2000s. We identified 636 criteria that we classified into five categories: people, task, structure, technology, and surroundings.

CONCLUSION

Criteria selection is a crucial step in any multi-criteria decision analysis (MCDA). This work is the first step in the development of a validated MCDA method to assess the organizational impact of medical innovations.

Development of a composite indicator to prioritize districts for implementation of human immunodeficiency virus programmes in Maharashtra, India

A key recommendation of the National AIDS Control Programme-IV of India was to develop new strategies for geo-prioritization of the human immunodeficiency virus (HIV) epidemic. We conducted this study to categorize the districts in Maharashtra (India) based on a multidimensional framework for geo-prioritization of services. Programmatic data on trends of HIV prevalence, coverage of marginalized populations and vulnerability factors were included. A composite indicator based on these was developed, and the cumulative score was calculated for each district. HIV prevalence among general population has declined steadily from 0.60% in 2007 to 0.33% in 2017. The programme coverage was stable but inadequate for men who have sex with men (MSM). The coverage for female sex workers (FSWs) was inadequate and reduced over time. Nine districts were categorized as high priority, 13 as moderate priority and 11 were classified as low-priority districts based on burden and vulnerability for HIV. The high-priority districts were Pune, Solapur and Yavatmal for FSW interventions and Pune, Thane and Latur for MSM interventions. This multidimensional indicator is based on existing programmatic data, dynamic and can be made state-specific. It is useful to categorize and prioritize districts for allocation of resources and geo-prioritization of services in resource limited settings.

Identifying Key Factors Related to the Resource Allocation in the Health Sector of the Iranian Oil Industry: Application of DEMATEL Method

Background: Resource allocation is one of the major challenges of health systems. Various criteria are used to allocate health resources worldwide. Objectives: The current study aimed to identify and prioritize key factors related to the resource allocation in the health sector of the Iranian Oil Industry using the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method. Methods: This cross-sectional case study was conducted in two main phases during 2017. In the first phase, the literature review and interviews with experts (n = 6) were used to identify initial criteria for resource allocation. Then, considering the views of high and middle managers in the health sector of the Iranian Oil Industry, the final criteria were selected. In the second phase, the DEMATEL technique was used to assess the interactions among the selected criteria. Results: A total of nine criteria were entered into the DEMATEL final questionnaire. Criteria were divided into two main groups of influencing and influenced factors. The high-level documents and general policies of the health system (C1), the burden of diseases (C2), the number of population covered (C4), the infrastructure of the covered regions (C5), and population health needs (C8) were among factors that influenced other factors. Factors such as current budget (C3), the expected benefits of people (C6), the expected benefits of policymakers (C7), and the financial returns of the resource allocation (C9) were among the factors that were influenced by other factors. The number of covered population (C4) and the infrastructure of the regions (C5) were not influenced by other factors but could affect other factors. High-level documents (C1) and the burden of diseases (C2) were identified as independent factors. Conclusions: While allocating resources, policymakers should pay particular attention to influencing factors. The number of population covered in each region and the infrastructure available in each region were identified as key and important criteria in the resource allocation process. It is suggested that these factors be considered while allocating resources to different regions of the country.

Three-Way Decisions Making Using Covering Based Fractional Orthotriple Fuzzy Rough Set Model

On the basis of decision-theoretical rough sets (DTRSs), the three-way decisions give new model of decision approach for deal with the problem of decision. This proposed model of decision method is based on the loss function of DTRSs. First, the concept of fractional orthotriple fuzzy β -covering (FOF β -covering) and fractional orthotriple fuzzy β -neighborhood (FOF β -neighborhood) was introduced. We combined loss feature of DTRSs with covering-based fractional orthotriple fuzzy rough sets (CFOFSs) under the fractional orthotriple fuzzy condition. Secondly, we proposed a new FOF-covering decision-theoretical rough sets model (FOFCDTRSs) and developed related properties. Then, based on the grade of positive, neutral and negative membership of fractional orthotriple fuzzy numbers (FOFNs), five methods are established for addressing the expected loss expressed in the form of FOFNs and the corresponding three-way decisions are also derived. Based on this, we presented a FOFCDTRS-based algorithm for multi-criteria decision making (MCDM). Then, an example verifies the feasibility of the five methods for solving the MCDM problem. Finally, by comparing the results of the decisions of five methods with different loss functions.

[GRADE: Evidence to Decision (EtD) framework for coverage decisions]

BACKGROUND

Coverage decisions are decisions by third party payers about whether and how much to pay for technologies or services, and under what conditions. Given their complexity, a systematic and transparent approach is needed. The DECIDE project, a GRADE working group initiative funded by the European Union, has developed GRADE Evidence to Decision (EtD) frameworks for different types of decisions, including coverage ones.

METHODS

We used an iterative approach, including brainstorming to generate ideas, consultation with stakeholders, user testing, and pilot testing of the framework.

RESULTS

The general structure of the EtD includes formulation of the question, an assessment using 12 criteria, and conclusions. Criteria that are relevant for coverage decisions are similar to those for clinical recommendations from a population perspective. Important differences between the two include the decision-making processes, accountability, and the nature of the judgments that need to be made for some criteria. Although cost-effectiveness is a key consideration when making coverage decisions, it may not be the determining factor. Strength of recommendation is not directly linked to the type of coverage decisions, but when there are important uncertainties, it may be possible to cover an intervention for a subgroup, in the context of research, with price negotiation, or with restrictions.

CONCLUSION

The EtD provides a systematic and transparent approach for making coverage decisions. It helps ensure consideration of key criteria that determine whether a technology or service should be covered and that judgments are informed by the best available evidence.

Use of programme budgeting and marginal analysis to set priorities for local NHS dental services: learning from the north east of England

Background

Priority setting is necessary where competing demands exceed the finite resources available. The aim of the study was to develop and test a prioritization framework based upon programme budgeting and marginal analysis (PBMA) as a tool to assist National Health Service (NHS) commissioners in their management of resources for local NHS dental services.

Methods

Twenty-seven stakeholders (5 dentists, 8 commissioners and 14 patients) participated in a case-study based in a former NHS commissioning organization in the north of England. Stakeholders modified local decision-making criteria and applied them to a number of different scenarios.

Results

The majority of financial resources for NHS dental services in the commissioning organization studied were allocated to primary care dental practitioners' contracts in perpetuity, potentially constraining commissioners' abilities to shift resources. Compiling the programme budget was successful, but organizational flux and difficulties engaging local NHS commissioners significantly impacted upon the marginal analysis phase.

Conclusions

NHS dental practitioners' contracts resemble budget-silos which do not facilitate local resource reallocation. 'Context-specific' factors significantly challenged the successful implementation and impact of PBMA. A local PBMA champion embedded within commissioning organizations should be considered. Participants found visual depiction of the cost-value ratio helpful during their initial priority setting deliberations.

Multicriteria Decision Analysis to Support Health Technology Assessment Agencies: Benefits, Limitations, and the Way Forward

OBJECTIVE

Recent years have witnessed an increased interest in the use of multicriteria decision analysis (MCDA) to support health technology assessment (HTA) agencies for setting healthcare priorities. However, its implementation to date has been criticized for being "entirely mechanistic," ignoring opportunity costs, and not following best practice guidelines. This article provides guidance on the use of MCDA in this context.

METHODS

The present study was based on a systematic review and consensus development. We developed a typology of MCDA studies and good implementation practice. We reviewed 36 studies over the period 1990 to 2018 on their compliance with good practice and developed recommendations. We reached consensus among authors over the course of several review rounds.

RESULTS

We identified 3 MCDA study types: qualitative MCDA, quantitative MCDA, and MCDA with decision rules. The types perform differently in terms of quality, consistency, and transparency of recommendations on healthcare priorities. We advise HTA agencies to always include a deliberative component. Agencies should, at a minimum, undertake qualitative MCDA. The use of quantitative MCDA has additional benefits but also poses design challenges. MCDA with decision rules, used by HTA agencies in The Netherlands and the United Kingdom and typically referred to as structured deliberation, has the potential to further improve the formulation of recommendations but has not yet been subjected to broad experimentation and evaluation.

CONCLUSION

MCDA holds large potential to support HTA agencies in setting healthcare priorities, but its implementation needs to be improved.

Systematic review for the development of a pharmaceutical and medical products prioritization framework

Objective

To identify and analyze the criteria, approaches, and conceptual frameworks, used for national/international priority setting.

Data sources

We performed a search of the main biomedical databases (Medline/PubMed, Embase, Centre for Reviews and Dissemination, and Cochrane), and we reviewed assessment agency websites, among other sources.

Study design

An systematic review of the literature was carried out.

Data collection

Eligibility criteria for inclusion were based on set of predefined criteria. Systematic reviews and/or qualitative studies (interviews, surveys, expert consensus, etc) that aimed to identify prioritization criteria or develop general operational frameworks for the selection of health priorities were included. A critical analysis is made of all the aspects that may be useful for any public body that intends to establish priorities in health.

Principal findings

We found that there are no standardized criteria for priority setting, although common trends have been identified regarding key elements. Eight key domains were identified: 1) need for intervention; 2) health outcomes; 3) type of benefit of the intervention; 4) economic consequences; 5) existing knowledge on the intervention/quality and uncertainties of the regarding evidence; 6) implementation and complexity of the intervention/feasibility; 7) justice and ethics; and 8) overall context.

Conclusions

Our review provides a thorough analysis of the relevant issues and offers key recommendations regarding considerations for developing a national prioritization framework. Findings are envisioned to be useful for different public organizations that are aiming to establish healthcare priorities.

Electronic supplementary material

The online version of this article (10.1186/s40545-019-0181-2) contains supplementary material, which is available to authorized users.

The Pan American Health Organization-adapted Hanlon method for prioritization of health programs

Objectives.

To document the underlying science of how the Pan American Health Organization (PAHO) adapted the Hanlon method, which prioritizes disease control programs, to its wider range of program areas and used it to implement the PAHO Strategic Plan 2014 – 2019.

Methods.

In 2014, PAHO established a Strategic Plan Advisory Group (SPAG) with representatives from 12 Member States to work closely with the PAHO Technical Team to adapt the Hanlon method to disease and non-disease control programs. Three meetings were held in 2015 – 2016 during which SPAG reviewed existing priority-setting methods, assessed the original Hanlon method and subsequent revisions, and developed the adapted method. This project was initiated by Member States, facilitated by PAHO, and conducted jointly in transparent and horizontal technical cooperation.

Results.

From the original Hanlon equation, the PAHO-adapted method maintains components A (size of problem), B (seriousness of problem), and C (effectiveness of intervention), drops component D (PEARL – Propriety, Economics, Acceptability, Resources, and Legality), and adds component E (inequity) and F (institutional positioning). The PEARL score was dropped because it serves a purpose for pre-screening process, but not in the priority-setting process for PAHO.

Conclusions.

The PAHO-adapted Hanlon method provides a refined approach for prioritizing public health programs that include disease and non-disease control areas. The method may be useful for the World Health Organization and country governments with similar needs.

Use of multi-attribute decision-making to inform prioritization of Cochrane review topics relevant to rehabilitation

BACKGROUND

Limited resources imply the need for prioritization; this also applies to the conduct of Cochrane Reviews. Therefore, processes for identifying the most important topics for review should be determined.

AIM

The aim of this study was to describe some examples of prioritization approaches used within Cochrane; to introduce the concept of multi-criteria decision analysis, provide an example of how this is being used to determine the relative importance of research questions in musculoskeletal health by the Australasian Musculoskeletal Clinical Trials (ANZMUSC) network and how a similar approach could be used to inform Cochrane Rehabilitation priorities.

METHODS

A narrative overview of the Cochrane Methods Priority Setting Group and new guidance from Cochrane on priority setting; a description of the ANZMUSC prioritization project and the proposed outline of a prioritization process that could be undertaken by Review Groups related to rehabilitation.

RESULTS

There are no explicit processes for prioritization for Cochrane Reviews that involve multi-criteria decision making, even though such approaches appear to be potentially useful and may overcome some disadvantages of alternative approaches.

DISCUSSION

Although the ANZMUSC prioritization project is not yet complete nor shown to be successful, it may offer a useful road-map for developing a transparent method of prioritizing which research topics to pursue, in a way that could be easily implemented and updated.

The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective

INTRODUCTION

Evidence-to-decision (EtD) frameworks intend to ensure that all criteria of relevance to a health decision are systematically considered. This paper, part of a series commissioned by the WHO, reports on the development of an EtD framework that is rooted in WHO norms and values, reflective of the changing global health landscape, and suitable for a range of interventions and complexity features. We also sought to assess the value of this framework to decision-makers at global and national levels, and to facilitate uptake through suggestions on how to prioritise criteria and methods to collect evidence.

METHODS

In an iterative, principles-based approach, we developed the framework structure from WHO norms and values. Preliminary criteria were derived from key documents and supplemented with comprehensive subcriteria obtained through an overview of systematic reviews of criteria employed in health decision-making. We assessed to what extent the framework can accommodate features of complexity, and conducted key informant interviews among WHO guideline developers. Suggestions on methods were drawn from the literature and expert consultation.

RESULTS

The new WHO-INTEGRATE (INTEGRATe Evidence) framework comprises six substantive criteria-balance of health benefits and harms, human rights and sociocultural acceptability, health equity, equality and non-discrimination, societal implications, financial and economic considerations, and feasibility and health system considerations-and the meta-criterion quality of evidence. It is intended to facilitate a structured process of reflection and discussion in a problem-specific and context-specific manner from the start of a guideline development or other health decision-making process. For each criterion, the framework offers a definition, subcriteria and example questions; it also suggests relevant primary research and evidence synthesis methods and approaches to assessing quality of evidence.

CONCLUSION

The framework is deliberately labelled version 1.0. We expect further modifications based on focus group discussions in four countries, example applications and input across concerned disciplines.

Doing 'Upstream' Priority-Setting for Global Health with Justice: Moving from Vision to Practice?

The vision of global health with justice which Larry Gostin articulates in his book Global Health Law envisages a switch to 'upstream' priority-setting for expenditure on health, with a focus upon social determinants and a goal of redressing health inequalities. This article explores what is meant by this proposal and offers a critical evaluation of it. It is argued that difficulties arise in respect of the ethical and evidential bases for such an approach to the setting of priorities, while significant challenges may also arise in the necessary modification of structures of governance.

Stakeholder Participation for Legitimate Priority Setting: A Checklist

Accountable decision-makers are required to legitimize their priority setting decisions in health to members of society. In this perspective we stress the point that fair, legitimate processes should reflect efforts of authorities to treat all stakeholders as moral equals in terms of providing all people with well-justified, reasonable reasons to endorse the decisions. We argue there is a special moral concern for being accountable to those who are potentially adversely affected by decisions. Health authorities need to operationalize this requirement into real world action. In this perspective, we operationalize five key steps in doing so, in terms of (i) proactively identifying potentially adversely affected stakeholders; (ii) comprehensively including them in the decision-making process; (iii) ensuring meaningful participation; (iv) communication of recommendations or decisions; and (v) the organization of evaluation and appeal mechanisms. Health authorities are advised to use a checklist in the form of 29 reflective questions, aligned with these five key steps, to assist them in the practical organization of legitimate priority setting in healthcare.

Developing Evidence to Decision Frameworks and an Interactive Evidence to Decision Tool for Making and Using Decisions and Recommendations in Health Care

Evidence-informed health care decisions and recommendations need to be made systematically and transparently. Mediating technology can help manage boundaries between groups making decisions and target audiences, enhancing salience, credibility, and legitimacy for all. This article describes the development of the Evidence to Decision (EtD) framework and an interactive tool to create and use frameworks (iEtD) to support communication in decision making. Methods: Using a human-centered design approach, we created prototypes employing a broad range of methods to iteratively develop EtD framework content and iEtD tool functionality. Results: We developed tailored EtD frameworks for making evidence-informed decisions and recommendations about clinical practice interventions, diagnostic and screening tests, coverage, and health system and public health options. The iEtD tool provides functionality for preparing frameworks, using them in group discussions, and publishing output for implementation or adaption. EtD and iEtD are intuitive and useful for producers and users of frameworks, and flexible for use across different types of topics, decisions, and organizations. They bring valued structure to panel discussions and transparency to published output. Conclusion: EtD and iEtD can resolve some of the challenges inherent in multicriteria, multistakeholder decision systems. They are freely available online for all to use at https://ietd.epistemonikos.org/ and https://gradepro.org.

[GRADE: Evidence to Decision (EtD) frameworks - a systematic and transparent approach to making well informed healthcare choices. 1: Introduction]

In healthcare, the processes, criteria, and evidence that decision makers use to reach their judgments often remain unclear. Decision makers sometimes neglect important criteria, give undue weight to criteria, or do not use the best available evidence to inform their judgments. Thus, the GRADE (Grading of Recommendations Assessment, Development and Evaluation) working group developed a system to support transparent decision making. The purpose of the Evidence-to-Decision (EtD) framework is to help people use evidence in a structured and transparent way to inform decisions in the context of clinical recommendations, coverage decisions, and health system, or public health recommendations and decisions. EtD frameworks include the formulation of the question, an assessment of the evidence, and drawing conclusions. EtD frameworks inform users of recommendations about judgments that were made and the evidence supporting these judgments by making the basis for decisions transparent to target audiences. EtD frameworks also facilitate dissemination of recommendations.

Value judgment of health interventions from different perspectives: arguments and criteria

Background

The healthcare sector is evolving while life expectancy is increasing. These trends put greater pressure on healthcare resources, prompt healthcare reforms, and demand transparent arguments and criteria to assess the overall value of health interventions. There is no consensus on the core criteria by which to value and prioritize interventions, and individual stakeholders might value specific elements differently. The present study is based on a literature review that retrieved the most widely recognized arguments and criteria used in decision-making. The aim was to compile a smaller set of arguments and criteria that would seem most relevant to different stakeholders.

Methods

A literature review was performed in Medline and EMBASE. The initial search retrieved over 2000 articles and documents of relevant committees. A selection was made based on their reference to healthcare, policy issues, or social justice. Finally, 84 papers were included. Data extraction took place after appraisal of the articles. A full table was made, including all arguments and criteria found; next, identical or largely overlapping arguments were excluded. The remaining arguments and criteria were assessed for relevance and a reduced set was compiled.

Results

The final set included 25 arguments and criteria, categorized by type (clinical, social justice, ethical, and policy). For each argument and criterion, relevance to stakeholders was scored on three levels (not, partly, and completely relevant).

Conclusions

Many arguments and criteria play a role in making value judgments on health interventions, but not all are relevant to all interventions. Moreover, they may interact with each other. A viable way to deal with interacting and possibly conflicting arguments and criteria might be to arrange public discussions that would evoke different stakeholders’ perspectives.

Improving health promotion through central rating of interventions: the need for Responsive Guidance

BACKGROUND

In several countries, attempts are made to improve health promotion by centrally rating the effectiveness of health promotion interventions. The Dutch Effectiveness Rating System (ERS) for health promotion interventions is an improvement-oriented approach in which multi-disciplinary expert committees rate available health promotion interventions as 'theoretically sound', 'probably effective' or 'proven effective'. The aim of this study is to explore the functioning of the ERS and the perspective of researchers, policy-makers and practitioners regarding its contribution to improvement.

METHODS

We interviewed 53 selected key informants from research, policy and practice in the Netherlands and observed the assessment of 12 interventions.

RESULTS

Between 2008 and 2012, a total of 94 interventions were submitted to the ERS, of which 23 were rejected, 58 were rated as 'theoretically sound', 10 were rated as 'probably effective' and 3 were rated as 'proven effective'. According to participants, the ERS was intended to facilitate both the improvement of available interventions and the improvement of health promotion in practice. While participants expected that describing and rating interventions promoted learning and enhanced the transferability of interventions, they were concerned that the ERS approach was not suitable for guiding intervention development and improving health promotion in practice. The expert committees that assessed the interventions struggled with a lack of norms for the relevance of effects and questions about how effects should be studied and rated. Health promotion practitioners were concerned that the ERS neglected the local adaptation of interventions and did not encourage the improvement of aspects like applicability and costs. Policy-makers and practitioners were worried that the lack of proven effectiveness legitimised cutbacks rather than learning and advancing health promotion.

CONCLUSION

While measuring and centrally rating the effectiveness of interventions can be beneficial, the evidence based-inspired ERS approach is too limited to guide both intervention development and the improvement of health promotion in practice. To better contribute to improving health promotion, a more reflexive and responsive guidance approach is required, namely one which stimulates the improvement of different intervention aspects, provides targeted recommendations to practitioners and provides feedback to those who develop and rate interventions.

Towards fair and effective North-South collaboration: realising a programme for demand-driven and locally led research

BACKGROUND

At the turn of the 90s, studies showed that health research contributed little to health and development in low- and middle-income countries because it was oriented towards international priorities and dominated by researchers from the North. A new approach to North-South collaboration was required that would support demand-driven and locally led research in the South. The aim of this study was to analyse the development and functioning of a programme for demand-driven and locally led research in Ghana that was supported by a North-South collaboration.

METHODS

For this mixed-method case study, we combined document analysis, key informant interviews and observation of programme events.

RESULTS

The development of the research programme started with constructing a sponsorship constellation in the Netherlands. After highlighting the problems with traditional research collaboration, an advisory council formulated a vision for a more equal and effective approach to North-South collaboration. Together with Ghanaian partners, this vision was turned into a proposal for a Ghanaian-led programme for demand-driven and locally led research, which was funded by the Netherlands government. Research priority setting showed that the Ghanaian research needs were very different from the priorities of foreign funders and researchers. After a slow start, the number of locally submitted proposals increased from 13 in 2001 to 94 in 2005, revealing the existence of a substantial, but partly latent reservoir of research capacity. In total, 79 studies were funded. An impact evaluation showed that the results of the majority of the studies were used to contribute to action. Despite its success, the research programme came to an end in 2008 after the sponsorship constellation in the Netherlands fell apart.

CONCLUSION

Our study shows that realising a programme for demand-driven and locally led research in the South provides an effective approach to North-South collaboration in which results are used and local capacities and institutions are strengthened.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation

Background

This is the tenth in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. After more than a decade of research, there is little published evidence of active and successful disinvestment. The paucity of frameworks, methods and tools is reported to be a factor in the lack of success. However there are clear and consistent messages in the literature that can be used to inform development of a framework for operationalising disinvestment. This paper, along with the conceptual review of disinvestment in Paper 9 of this series, aims to integrate the findings of the SHARE Program with the existing disinvestment literature to address the lack of information regarding systematic organisation-wide approaches to disinvestment at the local health service level.

Discussion

A framework for disinvestment in a local healthcare setting is proposed. Definitions for essential terms and key concepts underpinning the framework have been made explicit to address the lack of consistent terminology. Given the negative connotations of the word ‘disinvestment’ and the problems inherent in considering disinvestment in isolation, the basis for the proposed framework is ‘resource allocation’ to address the spectrum of decision-making from investment to disinvestment. The focus is positive: optimising healthcare, improving health outcomes, using resources effectively.

The framework is based on three components: a program for decision-making, projects to implement decisions and evaluate outcomes, and research to understand and improve the program and project activities. The program consists of principles for decision-making and settings that provide opportunities to introduce systematic prompts and triggers to initiate disinvestment. The projects follow the steps in the disinvestment process. Potential methods and tools are presented, however the framework does not stipulate project design or conduct; allowing application of any theories, methods or tools at each step. Barriers are discussed and examples illustrating constituent elements are provided.

Conclusions

The framework can be employed at network, institutional, departmental, ward or committee level. It is proposed as an organisation-wide application, embedded within existing systems and processes, which can be responsive to needs and priorities at the level of implementation. It can be used in policy, management or clinical contexts.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2506-7) contains supplementary material, which is available to authorized users.

Progressive realisation of universal health coverage: what are the required processes and evidence?

Progressive realisation is invoked as the guiding principle for countries on their own path to universal health coverage (UHC). It refers to the governmental obligations to immediately and progressively move towards the full realisation of UHC. This paper provides procedural guidance for countries, that is, how they can best organise their processes and evidence collection to make decisions on what services to provide first under progressive realisation. We thereby use 'evidence-informed deliberative processes', a generic value assessment framework to guide decision making on the choice of health services. We apply this to the concept of progressive realisation of UHC. We reason that countries face two important choices to achieve UHC. First, they need to define which services they consider as high priority, on the basis of their social values, including cost-effectiveness, priority to the worse off and financial risk protection. Second, they need to make tough choices whether they should first include more priority services, first expand coverage of existing priority services or first reduce co-payments of existing priority services. Evidence informed deliberative processes can facilitate these choices for UHC, and are also essential to the progressive realisation of the right to health. The framework informs health authorities on how they can best organise their processes in terms of composition of an appraisal committee including stakeholders, of decision-making criteria, collection of evidence and development of recommendations, including their communication. In conclusion, this paper fills in an important gap in the literature by providing procedural guidance for countries to progressively realise UHC.

[GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction]

Clinicians, guideline developers, and policymakers sometimes neglect important criteria, give undue weight to criteria, and do not use the best available evidence to inform their judgments. Explicit and transparent systems for decision making can help to ensure that all important criteria are considered and that decisions are informed by the best available research evidence. The GRADE Working Group has developed Evidence to Decision (EtD) frameworks for the different type of recommendations or decisions. The purpose of EtD frameworks is to help people use evidence in a structured and transparent way to inform decisions in the context of clinical recommendations, coverage decisions, and health system or public health recommendations and decisions. EtD frameworks have a common structure that includes formulation of the question, an assessment of the evidence, and drawing conclusions, though there are some differences between frameworks for each type of decision. EtD frameworks inform users about the judgments that were made and the evidence supporting those judgments by making the basis for decisions transparent to target audiences. EtD frameworks also facilitate dissemination of recommendations and enable decision makers in other jurisdictions to adopt recommendations or decisions, or adapt them to their context. This article is a translation of the original article published in British Medical Journal. The EtD frameworks are currently used in the Clinical Practice Guideline Programme of the Spanish National Health System, co-ordinated by GuíaSalud.

GRADE EVIDENCE TO DECISION (EtD) FRAMEWORK FOR COVERAGE DECISIONS

Objectives: Coverage decisions are decisions by third party payers about whether and how much to pay for technologies or services, and under what conditions. Given their complexity, a systematic and transparent approach is needed. The DECIDE (Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence) Project, a GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group initiative funded by the European Union, has developed GRADE Evidence to Decision (EtD) framework for different types of decisions, including coverage ones.

Methods: We used an iterative approach, including brainstorming to generate ideas, consultation with stakeholders, user testing, and pilot testing of the framework.

Results: The general structure of the EtD includes formulation of the question, an assessment using twelve criteria, and conclusions. Criteria that are relevant for coverage decisions are similar to those for clinical recommendations from a population perspective. Important differences between the two include the decision-making processes, accountability, and the nature of the judgments that need to be made for some criteria. Although cost-effectiveness is a key consideration when making coverage decisions, it may not be the determining factor. Strength of recommendation is not directly linked to the type of coverage decisions, but when there are important uncertainties, it may be possible to cover an intervention for a subgroup, in the context of research, with price negotiation, or with restrictions.

Conclusions: The EtD provides a systematic and transparent approach for making coverage decisions. It helps ensure consideration of key criteria that determine whether a technology or service should be covered and that judgments are informed by the best available evidence.

Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework

Escalating drug prices have catalysed the generation of numerous "value frameworks" with the aim of informing payers, clinicians and patients on the assessment and appraisal process of new medicines for the purpose of coverage and treatment selection decisions. Although this is an important step towards a more inclusive Value Based Assessment (VBA) approach, aspects of these frameworks are based on weak methodologies and could potentially result in misleading recommendations or decisions. In this paper, a Multiple Criteria Decision Analysis (MCDA) methodological process, based on Multi Attribute Value Theory (MAVT), is adopted for building a multi-criteria evaluation model. A five-stage model-building process is followed, using a top-down "value-focused thinking" approach, involving literature reviews and expert consultations. A generic value tree is structured capturing decision-makers' concerns for assessing the value of new medicines in the context of Health Technology Assessment (HTA) and in alignment with decision theory. The resulting value tree (Advance Value Tree) consists of three levels of criteria (top level criteria clusters, mid-level criteria, bottom level sub-criteria or attributes) relating to five key domains that can be explicitly measured and assessed: (a) burden of disease, (b) therapeutic impact, (c) safety profile (d) innovation level and (e) socioeconomic impact. A number of MAVT modelling techniques are introduced for operationalising (i.e. estimating) the model, for scoring the alternative treatment options, assigning relative weights of importance to the criteria, and combining scores and weights. Overall, the combination of these MCDA modelling techniques for the elicitation and construction of value preferences across the generic value tree provides a new value framework (Advance Value Framework) enabling the comprehensive measurement of value in a structured and transparent way. Given its flexibility to meet diverse requirements and become readily adaptable across different settings, the Advance Value Framework could be offered as a decision-support tool for evaluators and payers to aid coverage and reimbursement of new medicines.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 6: investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting

BACKGROUND

This is the sixth in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE program was established to investigate a systematic, integrated, evidence-based approach to disinvestment within a large Australian health service. This paper describes the methods employed in undertaking pilot disinvestment projects. It draws a number of lessons regarding the strengths and weaknesses of these methods; particularly regarding the crucial first step of identifying targets for disinvestment.

METHODS

Literature reviews, survey, interviews, consultation and workshops were used to capture and process the relevant information. A theoretical framework was adapted for evaluation and explication of disinvestment projects, including a taxonomy for the determinants of effectiveness, process of change and outcome measures. Implementation, evaluation and costing plans were developed.

RESULTS

Four literature reviews were completed, surveys were received from 15 external experts, 65 interviews were conducted, 18 senior decision-makers attended a data gathering workshop, 22 experts and local informants were consulted, and four decision-making workshops were undertaken. Mechanisms to identify disinvestment targets and criteria for prioritisation and decision-making were investigated. A catalogue containing 184 evidence-based opportunities for disinvestment and an algorithm to identify disinvestment projects were developed. An Expression of Interest process identified two potential disinvestment projects. Seventeen additional projects were proposed through a non-systematic nomination process. Four of the 19 proposals were selected as pilot projects but only one reached the implementation stage. Factors with potential influence on the outcomes of disinvestment projects are discussed and barriers and enablers in the pilot projects are summarised.

CONCLUSION

This study provides an in-depth insight into the experience of disinvestment in one local healthcare service. To our knowledge, this is the first paper to report the process of disinvestment from identification, through prioritisation and decision-making, to implementation and evaluation, and finally explication of the processes and outcomes.

Sustainability in health care by allocating resources effectively (SHARE) 3: examining how resource allocation decisions are made, implemented and evaluated in a local healthcare setting

Background

This is the third in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. Leaders in a large Australian health service planned to establish an organisation-wide, systematic, integrated, evidence-based approach to disinvestment. In order to introduce new systems and processes for disinvestment into existing decision-making infrastructure, we aimed to understand where, how and by whom resource allocation decisions were made, implemented and evaluated. We also sought the knowledge and experience of staff regarding previous disinvestment activities.

Methods

Structured interviews, workshops and document analysis were used to collect information from multiple sources in an environmental scan of decision-making systems and processes. Findings were synthesised using a theoretical framework.

Results

Sixty-eight respondents participated in interviews and workshops. Eight components in the process of resource allocation were identified: Governance, Administration, Stakeholder engagement, Resources, Decision-making, Implementation, Evaluation and, where appropriate, Reinvestment of savings. Elements of structure and practice for each component are described and a new framework was developed to capture the relationships between them. A range of decision-makers, decision-making settings, type and scope of decisions, criteria used, and strengths, weaknesses, barriers and enablers are outlined. The term ‘disinvestment’ was not used in health service decision-making. Previous projects that involved removal, reduction or restriction of current practices were driven by quality and safety issues, evidence-based practice or a need to find resource savings and not by initiatives where the primary aim was to disinvest. Measuring resource savings is difficult, in some situations impossible. Savings are often only theoretical as resources released may be utilised immediately by patients waiting for beds, clinic appointments or surgery. Decision-making systems and processes for resource allocation are more complex than assumed in previous studies.

Conclusion

There is a wide range of decision-makers, settings, scope and type of decisions, and criteria used for allocating resources within a single institution. To our knowledge, this is the first paper to report this level of detail and to introduce eight components of the resource allocation process identified within a local health service.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2207-2) contains supplementary material, which is available to authorized users.

Developing a decision support system to link health technology assessment (HTA) reports to the health system policies in Iran

The recent increase of 'Health Technology Assessment' (HTA)-related activities in Iran has necessitated the clarification of policy-making process based on the HTA reports. This study aimed to develop a Decision Support System (DSS) in order to adopt evidence-informed policies regarding health technologies in Iran. The study can be classified as Health Policy and Systems Research. A core panel of seven experts conducted two separate reviews of relevant literature for: 1- Determining the potential technology-related policies. 2- Listing the criteria influencing those policy decisions. The policies and criteria were separately discussed and subsequently rated for appropriateness and necessity during two expert meetings in 2013. In the next step, The 'Discrete Choice Experiment' (DCE) method was employed to develop the DSS for the final technology-related policies. Accordingly, the core panel members independently rated the appropriateness of each policy for 30 virtual technologies based on the random values assigned to all the criteria for each technology. The obtained data for each policy were separately analysed using stepwise regression model, resulting in a minimal set of independent and statistically significant criteria contributing in the experts' judgments about the appropriateness of that policy. The obtained regression coefficients were used as the relative weights of the different levels of the final criteria of any policy statement, shaping the decision support scoring tool for each policy. The study has outlined 64 policy decisions under 7 macro policy areas concerning a health technology. Also, 34 criteria used for making those policy decisions have been organized within a portfolio. DCE, using stepwise regression, resulted in 64 scoring tools shaping the DSS for all HTA-related policies. Both the results and methodology of the study may serve as a guide for policy makers (researchers), particularly in low and middle income countries, in developing decision aids for their own context-specific HTA-related policies.

Stated and Revealed Preferences for Funding New High-Cost Cancer Drugs: A Critical Review of the Evidence from Patients, the Public and Payers

BACKGROUND

The growing focus on patient-centred care has encouraged the inclusion of patient and public input into payer drug reimbursement decisions. Yet, little is known about patient/public priorities for funding high-cost medicines, and how they compare to payer priorities applied in public funding decisions for new cancer drugs.

OBJECTIVES

The aim was to identify and compare the funding preferences of cancer patients and the general public against the criteria used by payers making cancer drug funding decisions.

METHODS

A thorough review of the empirical, peer-reviewed English literature was conducted. Information sources were PubMed, EMBASE, MEDLINE, Web of Science, Business Source Complete, and EconLit. Eligible studies (1) assessed the cancer drug funding preferences of patients, the general public or payers, (2) had pre-defined measures of funding preference, and (3) had outcomes with attributes or measures of 'value'. The quality of included studies was evaluated using a health technology assessment-based assessment tool, followed by extraction of general study characteristics and funding preferences, which were categorized using an established WHO-based framework.

RESULTS

Twenty-five preference studies were retrieved (11 quantitative, seven qualitative, seven mixed-methods). Most studies were published from 2005 onward, with the oldest dating back to 1997. Two studies evaluated both patient and public perspectives, giving 27 total funding perspectives (41 % payer, 33 % public, 26 % patients). Of 41 identified funding criteria, payers consider the most (35), the general public considers fewer (23), and patients consider the fewest (12). We identify four unique patient criteria: financial protection, access to medical information, autonomy in treatment decision making, and the 'value of hope'. Sixteen countries/jurisdictions were represented.

CONCLUSIONS

Our results suggest that (1) payers prioritize efficiency (health gains per dollar), while citizens (patients and the general public) prioritize equity (equal access to cancer medicines independent of cost or effectiveness), (2) citizens prioritize few criteria relevant to payers, and (3) citizens prioritize several criteria not considered by payers. This can explain why payer and citizen priorities clash when new cancer medicines are denied public funding.

Value Assessment Frameworks for HTA Agencies: The Organization of Evidence-Informed Deliberative Processes

Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, health technology assessment agencies (HTAs) presently employ value assessment frameworks that are ill fitted to capture the range and diversity of stakeholder values and thereby risk compromising the legitimacy of their recommendations. We propose "evidence-informed deliberative processes" as an alternative framework with the aim to enhance this legitimacy. This framework integrates two increasingly popular and complementary frameworks for priority setting: multicriteria decision analysis and accountability for reasonableness. Evidence-informed deliberative processes are, on one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making through evidence-informed evaluation of the identified values. The framework has important implications for how HTA agencies should ideally organize their processes. First, HTA agencies should take the responsibility of organizing stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phases, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria and to provide argumentation for how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal. The framework should not be considered a blueprint for HTA agencies but rather an aspirational goal-agencies can take incremental steps toward achieving this goal.

Fair Processes for Priority Setting: Putting Theory into Practice Comment on "Expanded HTA: Enhancing Fairness and Legitimacy"

Embedding health technology assessment (HTA) in a fair process has great potential to capture societal values relevant to public reimbursement decisions on health technologies. However, the development of such processes for priority setting has largely been theoretical. In this paper, we provide further practical lead ways on how these processes can be implemented. We first present the misconception about the relation between facts and values that is since long misleading the conduct of HTA and underlies the current assessment-appraisal split. We then argue that HTA should instead be explicitly organized as an ongoing evidence-informed deliberative process, that facilitates learning among stakeholders. This has important consequences for whose values to consider, how to deal with vested interests, how to consider all values in the decision-making process, and how to communicate decisions. This is in stark contrast to how HTA processes are implemented now. It is time to set the stage for HTA as learning.

Core values for vaccine evaluation

Currently, most health economic modelling approaches tend to inadequately incorporate crucial disease-specific criteria and other attributes of benefit resulting from vaccination, which limits their utility for evaluating vaccines and, in consequence, for optimally guiding vaccine decision-making. Additionally, vaccine evaluation methods are frequently poorly standardised and non-transparent, leading to a potentially low level of accountability that can hinder acceptance of resulting decisions. To address these issues, we have considered whether it is possible to identify a set of universal vaccine-disease considerations, which we have called Core Values. To begin to identify such a set of criteria, and to establish whether strong agreement around such core values exists, we conducted two studies based on the Delphi technique. Both studies surveyed a cohort consisting of expert members of the global vaccine community with diverse professional backgrounds. Formal statistical analysis of both studies identified four attributes with strong agreement: 1. Incidence disease cases prevented per year, 2. Cost-effectiveness (including cost-benefit and cost-utility analysis), 3. High mortality disease (case-fatality-rate), and 4. Severity of target disease (risk of morbidity and mortality). These results suggest the feasibility of identifying a clear consensus on a specific set of Core Values for Vaccine Evaluation.

IMPORTANCE OF MULTIPLE CRITERIA FOR PRIORITY SETTING OF HIV/AIDS INTERVENTIONS

Objectives: This study describes the views of various stakeholders on the importance of different criteria for priority setting of HIV/AIDS interventions in Indonesia.

Methods: Based on a general list of criteria and a focus group discussion with stakeholders (n = 6), a list was developed of thirty-two criteria that play a role in priority setting in HIV/AIDS control in West-Java province. Criteria were categorized according to the World Health Organization's health system goals and building block frameworks. People living with HIV/AIDS (n = 49), healthcare workers (HCW) (n = 41), the general population (n = 43), and policy makers (n = 22) rated the importance of thirty-two criteria on a 5-point Likert-scale. Thereafter, respondents ranked the highest rated criteria to express more detailed preferences.

Results: Stakeholders valued the following criteria as most important for the priority setting of HIV/AIDS interventions: an intervention's impact on the HIV/AIDS epidemic, reduction of stigma, quality of care, effectiveness on individual level, and feasibility in terms of current capacity of the health system (i.e., HCW, product, information, and service requirements), financial sustainability, and acceptance by donors. Overall, stakeholders’ preferences for the importance of criteria are similar.

Conclusions: Our study design outlines an approach for other settings to identify which criteria are important for priority setting of health interventions. For Indonesia, these study results may be used in priority setting processes for HIV/AIDS control and may contribute to more transparent and systematic allocation of resources.

Priority Setting for Universal Health Coverage: We Need Evidence-Informed Deliberative Processes, Not Just More Evidence on Cost-Effectiveness

Priority setting of health interventions is generally considered as a valuable approach to support low- and middle-income countries (LMICs) in their strive for universal health coverage (UHC). However, present initiatives on priority setting are mainly geared towards the development of more cost-effectiveness information, and this evidence does not sufficiently support countries to make optimal choices. The reason is that priority setting is in reality a value-laden political process in which multiple criteria beyond cost-effectiveness are important, and stakeholders often justifiably disagree about the relative importance of these criteria. Here, we propose the use of 'evidence-informed deliberative processes' as an approach that does explicitly recognise priority setting as a political process and an intrinsically complex task. In these processes, deliberation between stakeholders is crucial to identify, reflect and learn about the meaning and importance of values, informed by evidence on these values. Such processes then result in the use of a broader range of explicit criteria that can be seen as the product of both international learning ('core' criteria, which include eg, cost-effectiveness, priority to the worse off, and financial protection) and learning among local stakeholders ('contextual' criteria). We believe that, with these evidence-informed deliberative processes in place, priority setting can provide a more meaningful contribution to achieving UHC.