Stakeholder Participation for Legitimate Priority Setting: A Checklist

Stakeholder participation in the COVID-19 pandemic preparedness and response plans: A synthesis of findings from 70 countries

Stakeholder participation is a key component of a fair and equitable priority-setting in health. The COVID-19 pandemic highlighted the need for fair and equitable priority setting, and hence, stakeholder participation. To date, there is limited literature on stakeholder participation in the development of the pandemic plans (including the priority setting plans) that were rapidly developed during the pandemic. Drawing on a global study of national COVID-19 preparedness and response plans, we present a secondary analysis of COVID-19 national plans from 70 countries from the six WHO regions, focusing on stakeholder participation. We found that most plans were prepared by the Ministry of Health and acknowledged WHO guidance, however less than half mentioned that additional stakeholders were involved. Few plans described a strategy for stakeholder participation and/or accounted for public participation in the plan preparation. However, diverse stakeholders (including multiple governmental, non-governmental, and international organizations) were proposed to participate in the implementation of the plans. Overall, there was a lack of transparency about who participated in decision-making and limited evidence of meaningful participation of the community, including marginalized groups. The critical relevance of stakeholder participation in priority setting requires that governments develop strategies for meaningful participation of diverse stakeholders during pandemics such as COVID-19, and in routine healthcare priority setting.

Procedural fairness in benefit package design: inclusion of pre-exposure prophylaxis of HIV in Universal Coverage Scheme in Thailand

Since 2002, Thailand's Universal Coverage Scheme (UCS) has adopted a comprehensive benefits package with few exclusions. A positive-list approach has gradually been applied, with pre-exposure prophylaxis (PrEP) of HIV recently being included. Disagreements resulting from competing values and diverging interests necessitate an emphasis on procedural fairness when making any decisions. This qualitative study analyses agenda setting, policy formulation and early implementation of PrEP from a procedural fairness lens. Literature reviews and in-depth interviews with 13 key stakeholders involved in PrEP policy processes were conducted. Civil society organizations (CSOs) and academia piloted PrEP service models and co-produced evidence on programmatic feasibility and outcomes. Through a broad stakeholder representation process, the Department of Disease Control proposed PrEP for inclusion in UCS benefits package in 2017. PrEP was shown to be cost-effective and affordable through rigorous health technology assessment, peer review, use of up-to-date evidence and safe-guards against conflicts of interest. In 2021, Thailand's National Health Security Board decided to include PrEP as a prevention and promotion package, free of charge, for the populations at risk. Favourable conditions for procedural fairness were created by Thailand's legislative provisions that enable responsive governance, notably inclusiveness, transparency, safeguarding public interest and accountable budget allocations; longstanding institutional capacity to generate local evidence; and implementation capacity for realisation of procedural fairness criteria. Multiple stakeholders including CSOs, academia and the government deliberated in the policy process through working groups and sub-committees. However, a key lesson from Thailand's deliberative process concerns a possible 'over interpretation' of conflicts of interest, intended to promote impartial decision-making, which inadvertently limited the voices of key populations represented in the decision processes. Finally, this case study underscores the value of examining the full policy cycle when assessing procedural fairness, since some stages of the process may be more amenable to certain procedural criteria than others.

Considering equity in priority setting using transmission models: Recommendations and data needs

OBJECTIVES

Disease transmission models are used in impact assessment and economic evaluations of infectious disease prevention and treatment strategies, prominently so in the COVID-19 response. These models rarely consider dimensions of equity relating to the differential health burden between individuals and groups. We describe concepts and approaches which are useful when considering equity in the priority setting process, and outline the technical choices concerning model structure, outputs, and data requirements needed to use transmission models in analyses of health equity.

METHODS

We reviewed the literature on equity concepts and approaches to their application in economic evaluation and undertook a technical consultation on how equity can be incorporated in priority setting for infectious disease control. The technical consultation brought together health economists with an interest in equity-informative economic evaluation, ethicists specialising in public health, mathematical modellers from various disease backgrounds, and representatives of global health funding and technical assistance organisations, to formulate key areas of consensus and recommendations.

RESULTS

We provide a series of recommendations for applying the Reference Case for Economic Evaluation in Global Health to infectious disease interventions, comprising guidance on 1) the specification of equity concepts; 2) choice of evaluation framework; 3) model structure; and 4) data needs. We present available conceptual and analytical choices, for example how correlation between different equity- and disease-relevant strata should be considered dependent on available data, and outline how assumptions and data limitations can be reported transparently by noting key factors for consideration.

CONCLUSIONS

Current developments in economic evaluations in global health provide a wide range of methodologies to incorporate equity into economic evaluations. Those employing infectious disease models need to use these frameworks more in priority setting to accurately represent health inequities. We provide guidance on the technical approaches to support this goal and ultimately, to achieve more equitable health policies.

Evidence-Informed Deliberative Processes for Legitimate Health Benefit Package Design - Part I: Conceptual Framework

BACKGROUND

Countries around the world are increasingly rethinking the design of their health benefit packages to achieve universal health coverage (UHC). Health technology assessment (HTA) bodies support governments in these decisions, but employ value frameworks that do not sufficiently account for the intrinsically complex and value-laden political reality of benefit package design.

METHODS

Several years ago, evidence-informed deliberative processes (EDPs) were developed to address this issue. An EDP is a practical and stepwise approach for HTA bodies to enhance legitimate health benefit package design based on deliberation between stakeholders to identify, reflect and learn about the meaning and importance of values, and to interpret available evidence on these values. We further developed the conceptual framework and initial 2019 guidance based on academic knowledge exchange, analysing practices of HTA bodies, surveying HTA bodies and experts around the globe, and implementation of EDPs in several countries around the world.

RESULTS

EDPs stem from the general concept of legitimacy, which is translated into four elements - stakeholder involvement ideally operationalised through stakeholder participation with deliberation; evidence-informed evaluation; transparency; and appeal. The 2021 practical guidance distinguishes six practical steps of a HTA process and provides recommendations on how these elements can be implemented in each of these steps.

CONCLUSION

There is an increased attention for legitimacy, deliberative processes for HTA and health benefit package design, but the development of theories and methods for such processes remain behind. The added value of EDPs lies in the operationalisation of the general concept of legitimacy into practical guidance for HTA bodies.

Evidence-Informed Deliberative Processes for Health Benefit Package Design - Part II: A Practical Guide

BACKGROUND

Countries around the world are using health technology assessment (HTA) for health benefit package design. Evidence-informed deliberative processes (EDPs) are a practical and stepwise approach to enhance legitimate health benefit package design based on deliberation between stakeholders to identify, reflect and learn about the meaning and importance of values, informed by evidence on these values. This paper reports on the development of practical guidance on EDPs, while the conceptual framework of EDPs is described in a companion paper.

METHODS

The first guide on EDPs (2019) is further developed based on academic knowledge exchange, surveying 27 HTA bodies and 66 experts around the globe, and the implementation of EDPs in several countries. We present the revised steps of EDPs and how selected HTA bodies (in Australia, Brazil, Canada, France, Germany, Scotland, Thailand and the United Kingdom) organize key issues of legitimacy in their processes. This is based on a review of literature via PubMed and HTA bodies' websites.

RESULTS

HTA bodies around the globe vary considerable in how they address legitimacy (stakeholder involvement ideally through participation with deliberation; evidence-informed evaluation; transparency; and appeal) in their processes. While there is increased attention for improving legitimacy in decision-making processes, we found that the selected HTA bodies are still lacking or just starting to develop activities in this area. We provide recommendations on how HTA bodies can improve on this.

CONCLUSION

The design and implementation of EDPs is in its infancy. We call for a systematic analysis of experiences of a variety of countries, from which general principles on EDPs might subsequently be inferred.

Research priority setting related to older adults: a scoping review to inform the Cochrane-Campbell Global Ageing Partnership work programme

OBJECTIVE

To explore and map the findings of prior research priority-setting initiatives related to improving the health and well-being of older adults.

DESIGN

Scoping review.

DATA SOURCES

Searched MEDLINE, EMBASE, AgeLine, CINAHL and PsycINFO databases from January 2014 to 26 April 2021, and the James Lind Alliance top 10 priorities.

ELIGIBILITY CRITERIA

We included primary studies reporting research priorities gathered from stakeholders that focused on ageing or the health of older adults (≥60 years). There were no restrictions by setting, but language was limited to English and French.

DATA EXTRACTION AND SYNTHESIS

We used a modified Reporting Guideline for Priority Setting of Health Research (REPRISE) guideline to assess the transparency of the reported methods. Population-intervention-control-outcome (PICO) priorities were categorised according to their associated International Classification of Health Interventions (ICHI) and International Classification of Functioning (ICF) outcomes. Broad research topics were categorised thematically.

RESULTS

Sixty-four studies met our inclusion criteria. The studies gathered opinions from various stakeholder groups, including clinicians (n=56 studies) and older adults (n=35), and caregivers (n=24), with 75% of the initiatives involving multiple groups. None of the included priority-setting initiatives reported gathering opinions from stakeholders located in low-income or middle-income countries. Of the priorities extracted, 272 were identified as broad research topics, while 217 were identified as PICO priorities. PICO priorities that involved clinical outcomes (n=165 priorities) and interventions concerning health-related behaviours (n=59) were identified most often. Broad research topics on health services and systems were identified most often (n=60). Across all these included studies, the reporting of six REPRISE elements was deemed to be critically low.

CONCLUSION

Future priority setting initiatives should focus on documenting a more detailed methodology with all initiatives eliciting opinions from caregivers and older adults to ensure priorities reflect the opinions of all key stakeholder groups.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force

Objectives

Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stake-holders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR—The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap.

Methods

The joint Task Force consisted of fifteen members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to two rounds of peer review.

Results

A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation.

Conclusions

The guidance and the checklist contain a series of questions, grouped by six phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Public legitimacy of healthcare resource allocation committees: lessons learned from assessing an Israeli case study

Background

The National Health Insurance Law enacted in 1995 stipulates a list of health services to which all Israeli residents are entitled. For the past 20 years, the list has been updated annually, as a function of a predetermined budget, according to recommendations from the Public National Advisory Committee (PNAC), which evaluates and prioritizes candidate technologies. We assessed the legitimacy of this resource-allocation process as reflected in Israeli public discourse and its congruence with the accountability for reasonableness (A4R) framework.

Methods

A qualitative analysis of public discourse documents (articles in the print media, court rulings and parliamentary debates (N = 119) was conducted to assess the perceived legitimacy by the Israeli public of the PNAC. Further content analysis of these documents and semi-structured interviews with stakeholders (N = 70) revealed the mainstays and threats to its legitimacy. Based on these data sources, on governmental documents specifying PNAC's procedures, and on data from participant observations, we assessed its congruence with A4R’s four conditions: publicity, relevance, revision and appeals, regulation.

Results

The PNAC enjoys ongoing support for its legitimacy in Israeli public discourse, which stem from its perceived professional focus and transparency. These strengths are consistent with the A4R’s emphasis on the publicity and the relevance conditions. The three major threats to PNAC's legitimacy pertain to: (1) the composition of the committee; (2) its operating procedures; (3) its guiding principles. These perceived shortcomings are also consistent with incongruencies between PNAC's work model and A4R. These findings thus further support the empirical validity of the A4R.

Conclusion

The analysis of the fit between the PNAC and A4R points to refinements in all four conditions that could make the A4R a more precise evaluative framework. Concurrently, it highlights areas that the PNAC should improve to increase its legitimacy, such as incorporating cost-effectiveness analyses and including patient representatives in the decision-making process.

Hebrew and Arabic abstracts for this article are available as an additional file.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force

OBJECTIVES

Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap.

METHODS

The joint Task Force consisted of 15 members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to 2 rounds of peer review.

RESULTS

A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation.

CONCLUSIONS

The guidance and the checklist contain a series of questions, grouped by 6 phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making in healthcare systems: a scoping review

OBJECTIVE

To identify the key themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making across the organisation and system domains of healthcare systems.

METHODS

We conducted a scoping review. Seven databases of journal articles were searched from their inception to June 2019. Eligible articles were literature reviews published in English and provided useful information for determining aspects of engaging patients and family caregivers in decision-making to evaluate. We extracted text under three predetermined categories: structure, process and outcomes that were adapted from the Donabedian conceptual framework. These excerpts were then independently open-coded among four researchers. The subsequent themes and their corresponding excerpts were summarised to provide a rich description of each theme.

RESULTS

Of 7747 unique articles identified, 366 were potentially relevant, from which we selected the 42 literature reviews. 18 unique themes were identified across the three predetermined categories. There were six structure themes: engagement plan, level of engagement, time and timing of engagement, format and composition, commitment to support and environment. There were four process themes: objectives, engagement approach, communication and engagement activities. There were eight outcome themes: decision-making process, stakeholder relationship, capacity development, stakeholder experience, shape policy/service/programme, health status, healthcare quality, and cost-effectiveness.

CONCLUSIONS

The 18 themes and their descriptions provide a foundation for identifying constructs and selecting measures to evaluate the quality of initiatives for engaging patients and family caregivers in healthcare system decision-making within the organisation and system domains. The themes can be used to investigate the mechanisms through which relevant initiatives are effective and investigate their effectiveness.

A mapping and synthesis of tools for stakeholder and community engagement in quality improvement initiatives for reproductive, maternal, newborn, child and adolescent health

BACKGROUND

Stakeholder and community engagement promotes collaboration and gives service users an opportunity to actively participate in the care they receive. Recognizing this potential, The Network for Improving Quality of Care for maternal, newborn and child health aimed to identify tools and operational guidance to integrate stakeholder and community engagement into quality improvement (QI) implementation.

METHODS

A mapping, consisting of a literature review and an open call through email and listservers, for implementation tools was conducted. Materials were included if they provided guidance on stakeholder and community engagement aligned to the Network's QI framework comprising seven phases. Screening of tools was done by two reviewers.

RESULTS

The literature search and the call for tools returned 197 documents with 70 tools included after screening. Most included tools (70%) were published after 2010. International organizations were the most frequently cited authors of tools. Only 15 tools covered all seven phases of the QI framework; few tools covered the more 'technical' phase of the QI framework: adapting standards and refining strategies.

CONCLUSION

The quantity of tools and their varied characteristics including types of stakeholder and community engagement processes across the QI framework confirms that engagement cannot be captured in a 'one-size-fits-all' formula. Many tools were designed with a generic focus to allow for adaption and use in different settings and sectors. Country programmes looking to strengthen engagement approaches can take advantage of available tools through an online portal on the WHO website and adapt them to meet their specific needs and context.

PUBLIC INVOLVEMENT

Programme implementers provided tools and resources during data collection.

How to achieve trustworthy artificial intelligence for health

Artificial intelligence holds great promise in terms of beneficial, accurate and effective preventive and curative interventions. At the same time, there is also awareness of potential risks and harm that may be caused by unregulated developments of artificial intelligence. Guiding principles are being developed around the world to foster trustworthy development and application of artificial intelligence systems. These guidelines can support developers and governing authorities when making decisions about the use of artificial intelligence. The High-Level Expert Group on Artificial Intelligence set up by the European Commission launched the report Ethical guidelines for trustworthy artificial intelligence in2019. The report aims to contribute to reflections and the discussion on the ethics of artificial intelligence technologies also beyond the countries of the European Union (EU). In this paper, we use the global health sector as a case and argue that the EU’s guidance leaves too much room for local, contextualized discretion for it to foster trustworthy artificial intelligence globally. We point to the urgency of shared globalized efforts to safeguard against the potential harms of artificial intelligence technologies in health care.

Use of Evidence-Informed Deliberative Processes by Health Technology Assessment Agencies Around the Globe

Background: Evidence-informed deliberative processes (EDPs) were recently introduced to guide health technology assessment (HTA) agencies to improve their processes towards more legitimate decision-making. The EDP framework provides guidance that covers the HTA process, ie, contextual factors, installation of an appraisal committee, selecting health technologies and criteria, assessment, appraisal, and communication and appeal. The aims of this study were to identify the level of use of EDPs by HTA agencies, identify their needs for guidance, and to learn about best practices.

Methods: A questionnaire for an online survey was developed based on the EDP framework, consisting of elements that reflect each part of the framework. The survey was sent to members of the International Network of Agencies for Health Technology Assessment (INAHTA). Two weeks following the invitation, a reminder was sent. The data collection took place between September-December 2018.

Results: Contact persons from 27 member agencies filled out the survey (response rate: 54%), of which 25 completed all questions. We found that contextual factors to support HTA development and the critical elements regarding conducting and reporting on HTA are overall in place. Respondents indicated that guidance was needed for specific elements related to selecting technologies and criteria, appraisal, and communication and appeal. With regard to best practices, the Canadian Agency for Drugs and Technologies and the National Institute for Health and Care Excellence (NICE, UK) were most often mentioned.

Conclusion: This is the first survey among HTA agencies regarding the use of EDPs and provides useful information for further developing a practical guide for HTA agencies around the globe. The results could support HTA agencies in improving their processes towards more legitimate decision-making, as they could serve as a baseline measurement for future monitoring and evaluation.

Applying priority-setting frameworks: A review of public and vulnerable populations' participation in health-system priority setting

BACKGROUND

There is a growing body of literature that describes, applies, and evaluates applications of health-system priority-setting frameworks in different contexts. However, little explicit focus has been given to examining operationalization of the stakeholder participation component of these frameworks. The literature identifies the public as a stakeholder group and recommends their participation when applying the frameworks.

METHODS

We conducted a scoping review to search the PubMed, EMBASE, HealthSTAR, Medline, and PsycINFO databases for cases where priority-setting frameworks were applied (2000-2017). We aimed to synthesize current literature to examine the degree to which the public and vulnerable populations have been engaged through applications of these frameworks FINDINGS: The following stakeholders commonly participated: managers, administrators/coordinators, clinicians/physicians, non-physician health care providers, health economists, academics/researchers, experts, decision-makers, and policy-makers. Few papers reported on public participation, and even fewer identified vulnerable groups that participate. Stakeholders were most commonly reported to participate in identifying areas for prioritization.

CONCLUSIONS

While the frameworks were developed with stakeholder participation in mind, in practice not all stakeholders are participating in priority-setting processes as envisioned by the frameworks. The public and vulnerable groups do not consistently participate, challenging the utility of the participation component of frameworks in guiding stakeholder participation in health-system priority setting. Frameworks can be more explicit about which stakeholders should participate and detailing how their participation should be operationalized.

Reports of rationing from the neglected realm of capital investment: Responses to resource constraint in the English National Health Service

Health systems around the world face financial pressures that can affect sustainability and patient outcomes, and there is a vast literature devoted to the allocation of scarce health care resources. Capital spending - for example on estates, equipment and information technology - is an important but often neglected area of this literature. This study explores the constraints on the allocation of capital budgets in health care, before addressing the question: what is the role of priority setting and rationing in responses to these constraints? The paper presents findings from interviews conducted with senior finance professionals in 30 National Health Service local provider organisations across England. Findings suggest a pervasive sense of impending crisis, with capital restrictions limiting investment in buildings, infrastructure and equipment. The paper applies a conceptual classification scheme from the classic rationing literature (the forms of rationing framework) and identifies widespread practices of 'selection', 'dilution' and 'delay', with 'denial' and 'termination' comparatively rare. Practices of 'deflection' and 'deterrence' are ascribed to national actors as a means of restricting the flow of capital resources to the system. The study suggests that there is little by way of tailored support for priority setting in capital spending, and a perception that decisions are often reactive and short term. It also suggests that wider system features and dynamics can preclude or constrain priority setting at the organisational level. The authors use these findings to suggest future conceptual development of the forms of rationing framework and make recommendations for research and practice in this area.