Using evaluation theory in priority setting and resource allocation

Using a Modified Delphi Approach and Nominal Group Technique for Organisational Priority Setting of Evidence-Based Interventions That Advance Women in Healthcare Leadership

BACKGROUND

Few studies address how to prioritise organisational interventions that advance women in leadership. We report on the relevance, feasibility and importance of evidence-based interventions for a large healthcare organisation. This study supports the first stage of implementation in a large National Health and Medical Research Council funded initiative seeking to advance women in healthcare leadership.

METHODS

An expert multi-disciplinary panel comprised of health professionals and leaders from a large healthcare network in Australia participated. The initial Delphi survey was administered online and results were presented in a Nominal Group Technique workshop. Here, the group made sense of the survey results, then evaluated findings against a framework on implementation criteria. Two further consensus surveys were conducted during the workshop.

RESULTS

Five priority areas were identified. These included: 1. A committed and supportive leadership team; 2. Improved governance structures; 3. Mentoring opportunities; 4. Leadership training and development; and 5. Flexibility in working. We describe the overall priority setting process in the context of our findings.

CONCLUSIONS

With evidence and expert input, we established priorities for advancing women in healthcare leadership with a partnering healthcare organisation. This approach can be adapted in other settings, seeking to advance women in leadership.

A qualitative evaluation of priority-setting by the Health Benefits Package Advisory Panel in Kenya

Kenya's Ministry of Health established the Health Benefits Package Advisory Panel (HBPAP) in 2018 to develop a benefits package for universal health coverage. This study evaluated HBPAP's process for developing the benefits package against the normative procedural (acceptable way of doing things) and outcome (acceptable consequences) conditions of an ideal healthcare priority-setting process as outlined in the study's conceptual framework. We conducted a qualitative case study using in-depth interviews with national-level respondents (n = 20) and document reviews. Data were analysed using a thematic approach. HBPAP's process partially fulfilled the procedural and outcome conditions of the study's evaluative framework. Concerning the procedural conditions, transparency and publicity were partially met and were limited by the lack of publication of HBPAP's report. While HBPAP used explicit and evidence-based priority-setting criteria, challenges included lack of primary data and local cost-effectiveness threshold, weak health information systems, short timelines and political interference. While a wide range of stakeholders were engaged, this was limited by short timelines and inadequate financial resources. Empowerment of non-HBPAP members was limited by their inadequate technical knowledge and experience in priority-setting. Finally, appeals and revisions were limited by short timelines and lack of implementation of the proposed benefits package. Concerning the outcome conditions, stakeholder understanding was limited by the technical nature of the process and short timelines, while stakeholder acceptance and satisfaction were limited by lack of transparency. HBPAP's benefits package was not implemented due to stakeholder interests and opposition. Priority-setting processes for benefits package development in Kenya could be improved by publicizing the outcome of the process, allocating adequate time and financial resources, strengthening health information systems, generating local evidence and enhancing stakeholder awareness and engagement to increase their empowerment, understanding and acceptance of the process. Managing politics and stakeholder interests is key in enhancing the success of priority-setting processes.

Assessing and Improving Performance: A Longitudinal Evaluation of Priority Setting and Resource Allocation in a Canadian Health Region

Background: In order to meet the challenges presented by increasing demand and scarcity of resources, healthcare organizations are faced with difficult decisions related to resource allocation. Tools to facilitate evaluation and improvement of these processes could enable greater transparency and more optimal distribution of resources.

Methods: The Resource Allocation Performance Assessment Tool (RAPAT) was implemented in a healthcare organization in British Columbia, Canada. Recommendations for improvement were delivered, and a follow up evaluation exercise was conducted to assess the trajectory of the organization’s priority setting and resource allocation (PSRA) process 2 years post the original evaluation.

Results: Implementation of RAPAT in the pilot organization identified strengths and weaknesses of the organization’s PSRA process at the time of the original evaluation. Strengths included the use of criteria and evidence, an ability to reallocate resources, and the involvement of frontline staff in the process. Weaknesses included training, communication, and lack of program budgeting. Although the follow up revealed a regression from a more formal PSRA process, a legacy of explicit resource allocation was reported to be providing ongoing benefit for the organization.

Conclusion: While past studies have taken a cross-sectional approach, this paper introduces the first longitudinal evaluation of PSRA in a healthcare organization. By including the strengths, weaknesses, and evolution of one organization’s journey, the authors’ intend that this paper will assist other healthcare leaders in meeting the challenges of allocating scarce resources.

Don't Discount Societal Value in Cost-Effectiveness Comment on "Priority Setting for Universal Health Coverage: We Need Evidence-Informed Deliberative Processes, Not Just More Evidence on Cost-Effectiveness"

As healthcare resources become increasingly scarce due to growing demand and stagnating budgets, the need for effective priority setting and resource allocation will become ever more critical to providing sustainable care to patients. While societal values should certainly play a part in guiding these processes, the methodology used to capture these values need not necessarily be limited to multi-criterion decision analysis (MCDA)-based processes including 'evidence-informed deliberative processes.' However, if decision-makers intend to not only incorporates the values of the public they serve into decisions but have the decisions enacted as well, consideration should be given to more direct involvement of stakeholders. Based on the examples provided by Baltussen et al, MCDA-based processes like 'evidence-informed deliberative processes' could be one way of achieving this laudable goal.

Healthcare resource allocation decisions affecting uninsured services

Purpose Using the example of community access programs (CAPs), the purpose of this paper is to describe resource allocation and policy decisions related to providing health services for the uninsured in the USA and the organizational values affecting these decisions. Design/methodology/approach The study used comparative case study methodology at two geographically diverse sites. Researchers collected data from program documents, meeting observations, and interviews with program stakeholders. Findings Five resource allocation or policy decisions relevant to providing healthcare services were described at each site across three categories: designing the health plan, reacting to funding changes, and revising policies. Organizational values of access to care and stewardship most frequently affected resource allocation and policy decisions, while economic and political pressures affect the relative prioritization of values. Research limitations/implications Small sample size, the potential for social desirability or recall bias, and the exclusion of provider, member or community perspectives beyond those represented among participating board members. Practical implications Program directors or researchers can use this study to assess the extent to which resource allocation and policy decisions align with organizational values and mission statements. Social implications The description of how healthcare decisions are actually made can be matched with literature that describes how healthcare resource decisions ought to be made, in order to provide a normative grounding for future decisions. Originality/value This study addresses a gap in literature regarding how CAPs actually make resource allocation decisions that affect access to healthcare services.

Adapting the nominal group technique for priority setting of evidence-practice gaps in implementation science

BACKGROUND

There are a variety of methods for priority setting in health research but few studies have addressed how to prioritise the gaps that exist between research evidence and clinical practice. This study aimed to build a suite of robust, evidence based techniques and tools for use in implementation science projects. We applied the priority setting methodology in lung cancer care as an example.

METHODS

We reviewed existing techniques and tools for priority setting in health research and the criteria used to prioritise items. An expert interdisciplinary consensus group comprised of health service, cancer and nursing researchers iteratively reviewed and adapted the techniques and tools. We tested these on evidence-practice gaps identified for lung cancer. The tools were pilot tested and finalised. A brief process evaluation was conducted.

RESULTS

We based our priority setting on the Nominal Group Technique (NGT). The adapted tools included a matrix for individuals to privately rate priority gaps; the same matrix was used for group discussion and reaching consensus. An investment exercise was used to validate allocation of priorities across the gaps. We describe the NGT process, criteria and tool adaptations and process evaluation results.

CONCLUSIONS

The modified NGT process, criteria and tools contribute to building a suite of methods that can be applied in prioritising evidence-practice gaps. These methods could be adapted for other health settings within the broader context of implementation science projects.

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation

BACKGROUND

Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices.

METHODS

We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken.

RESULTS

Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values.

CONCLUSION

Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought.