Evaluating priority setting success in healthcare: a pilot study

Priority Setting and Equity in COVID-19 Pandemic Plans: A Comparative Analysis of eighteen African Countries

Priority setting represents an even bigger challenge during public health emergencies than routine times. This is because such emergencies compete with routine programmes for the available health resources, strain health systems and shift health-care attention and resources towards containing the spread of the epidemic and treating those that fall seriously ill. This paper is part of a larger global study, the aim of which is to evaluate the degree to which national COVID-19 preparedness and response plans incorporated priority setting concepts. It provides important insights into what and how priority decisions were made in the context of a pandemic. Specifically, with a focus on a sample of 18 African countries’ pandemic plans, the paper aims to: (1) explore the degree to which the documented priority setting processes adhere to established quality indicators of effective priority setting and (2) examine if there is a relationship between the number of quality indicators present in the pandemic plans and the country’s economic context, health system and prior experiences with disease outbreaks. All the reviewed plans contained some aspects of expected priority setting processes but none of the national plans addressed all quality parameters. Most of the parameters were mentioned by less than 10 of the 18 country plans reviewed, and several plans identified one or two aspects of fair priority setting processes. Very few plans identified equity as a criterion for priority setting. Since the parameters are relevant to the quality of priority setting that is implemented during public health emergencies and most of the countries have pre-existing pandemic plans; it would be advisable that, for the future (if not already happening), countries consider priority setting as a critical part of their routine health emergency and disease outbreak plans. Such an approach would ensure that priority setting is integral to pandemic planning, response and recovery.

Biases distorting priority setting

Modern health care faces an ever widening gap between technological possibilities and available resources. To handle this challenge we have constructed elaborate systems for health policy making and priority setting. Despite such systems many health care systems provide a wide range of documented low-value care while being unable to afford emerging high-value care. Accordingly, this article sets out asking why priority setting in health care has so poor outcomes while relevant systems are well developed and readily available. It starts to identify some rational and structural explanations for the discrepancy between theoretical efforts and practical outcomes in priority setting. However, even if these issues are addressed, practical priority setting may still not obtain its goals. This is because a wide range of irrational effects is hampering priority setting: biases. By using examples from the literature the article identifies and analyses a wide range of biases indicating how they can distort priority setting processes. Overuse, underuse, and overinvestment, as well as hampered disinvestment and undermined priority setting principles are but some of the identified implications. Moreover, while some biases are operating mainly on one level, many are active on the micro, meso and on the macro level. Identifying and analyzing biases affecting priority setting is the first, but crucial, step towards improving health policy making and priority setting in health care.

Priority setting for maternal, newborn and child health in Uganda: a qualitative study evaluating actual practice

Background

Despite continued investment, Maternal, Newborn and Child Health (MNCH) indicators in low and middle income countries have remained relatively poor. This could, in part, be explained by inadequate resources to adequately address these problems, inappropriate allocation of the available resources, or lack of implementation of the most effective interventions. Systematic priority setting and resource allocation could contribute to alleviating these limitations. There is a paucity of literature that follows through MNCH prioritization processes to implementation, making it difficult for policy makers to understand the impact of their decision-making on population health. The overall objective of this paper was to describe and evaluate priority setting for maternal, newborn and child health interventions in Uganda.

Methods

Fifty-four key informant interviews and a review of policies and media reports were used to describe priority setting for MNCH in Uganda. Kapiriri and Martin’s conceptual framework was used to evaluate priority setting for MNCH.

Results

There were three main prioritization exercises for maternal, newborn and child health in Uganda. The processes were participatory and were guided by explicit tools, evidence, and criteria, however, the public and the districts were insufficiently involved in the priority setting process. While there were conducive contextual factors including strong political support, implementation was constrained by the presence of competing actors, with varying priorities, an unequal allocation of resources between child health and maternal health interventions, limited financial and human resources, a weak health system and limited institutional capacity.

Conclusions

Stronger institutional capacity at the Ministry of Health and equitable engagement of key stakeholders in decision-making processes, especially the public, and implementers, would improve understanding, satisfaction and compliance with the priority setting process. Availability of financial and human resources that are appropriately allocated would facilitate the implementation of well-developed policies.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4170-6) contains supplementary material, which is available to authorized users.

Priority setting for disease outbreaks in Uganda: A case study evaluating the process

Priority setting (PS) and resource allocation during health emergencies are key factors influencing an effective response. However, there is limited understanding of how priorities and resource allocation during disease outbreaks occur and the extent to which these processes are successful. This paper, based on 23 in-depth interviews with policy makers and a review of policy and emergency preparedness documents, used a PS evaluation framework to evaluate PS for disease outbreaks in Uganda. With regard to PS for disease outbreaks in Uganda, we identified a conducive socio-political-economical context, credible institutions, formal participatory prioritisation processes, evidence informed the processes, demonstrated implementation capacity, institutional strengthening and positive health outcomes. Factors that compromised the success of PS included limited resources - especially in between disease outbreaks and unfair processes. Investment in sustaining the established prioritisation infrastructure to oversee preparedness activities between the outbreaks would strengthen the prioritisation process. This should be supported with health system strengthening. The framework enabled us to evaluate some aspects of PS during disease outbreaks. The framework's inability to evaluate all aspects, and reported as opposed to actual PS calls for the integration of evaluation throughout the planning and implementation process to ensure validity and continuous implementation of improvement strategies.

Assessing and Improving Performance: A Longitudinal Evaluation of Priority Setting and Resource Allocation in a Canadian Health Region

Background: In order to meet the challenges presented by increasing demand and scarcity of resources, healthcare organizations are faced with difficult decisions related to resource allocation. Tools to facilitate evaluation and improvement of these processes could enable greater transparency and more optimal distribution of resources.

Methods: The Resource Allocation Performance Assessment Tool (RAPAT) was implemented in a healthcare organization in British Columbia, Canada. Recommendations for improvement were delivered, and a follow up evaluation exercise was conducted to assess the trajectory of the organization’s priority setting and resource allocation (PSRA) process 2 years post the original evaluation.

Results: Implementation of RAPAT in the pilot organization identified strengths and weaknesses of the organization’s PSRA process at the time of the original evaluation. Strengths included the use of criteria and evidence, an ability to reallocate resources, and the involvement of frontline staff in the process. Weaknesses included training, communication, and lack of program budgeting. Although the follow up revealed a regression from a more formal PSRA process, a legacy of explicit resource allocation was reported to be providing ongoing benefit for the organization.

Conclusion: While past studies have taken a cross-sectional approach, this paper introduces the first longitudinal evaluation of PSRA in a healthcare organization. By including the strengths, weaknesses, and evolution of one organization’s journey, the authors’ intend that this paper will assist other healthcare leaders in meeting the challenges of allocating scarce resources.

Based Real Time Remote Health Monitoring Systems: A Review on Patients Prioritization and Related "Big Data" Using Body Sensors information and Communication Technology

The growing worldwide population has increased the need for technologies, computerised software algorithms and smart devices that can monitor and assist patients anytime and anywhere and thus enable them to lead independent lives. The real-time remote monitoring of patients is an important issue in telemedicine. In the provision of healthcare services, patient prioritisation poses a significant challenge because of the complex decision-making process it involves when patients are considered 'big data'. To our knowledge, no study has highlighted the link between 'big data' characteristics and real-time remote healthcare monitoring in the patient prioritisation process, as well as the inherent challenges involved. Thus, we present comprehensive insights into the elements of big data characteristics according to the six 'Vs': volume, velocity, variety, veracity, value and variability. Each of these elements is presented and connected to a related part in the study of the connection between patient prioritisation and real-time remote healthcare monitoring systems. Then, we determine the weak points and recommend solutions as potential future work. This study makes the following contributions. (1) The link between big data characteristics and real-time remote healthcare monitoring in the patient prioritisation process is described. (2) The open issues and challenges for big data used in the patient prioritisation process are emphasised. (3) As a recommended solution, decision making using multiple criteria, such as vital signs and chief complaints, is utilised to prioritise the big data of patients with chronic diseases on the basis of the most urgent cases.

How have systematic priority setting approaches influenced policy making? A synthesis of the current literature

BACKGROUND

There is a growing body of literature on systematic approaches to healthcare priority setting from various countries and different levels of decision making. This paper synthesizes the current literature in order to assess the extent to which program budgeting and marginal analysis (PBMA), burden of disease & cost-effectiveness analysis (BOD/CEA), multi-criteria decision analysis (MCDA), and accountability for reasonableness (A4R), are reported to have been institutionalized and influenced policy making and practice.

METHODS

We searched for English language publications on health care priority setting approaches (2000-2017). Our sources of literature included PubMed and Ovid databases (including Embase, Global Health, Medline, PsycINFO, EconLit).

FINDINGS

Of the four approaches PBMA and A4R were commonly applied in high income countries while BOD/CEA was exclusively applied in low income countries. PBMA and BOD/CEA were most commonly reported to have influenced policy making. The explanations for limited adoption of an approach were related to its complexity, poor policy maker understanding and resource requirements.

CONCLUSIONS

While systematic approaches have the potential to improve healthcare priority setting; most have not been adopted in routine policy making. The identified barriers call for sustained knowledge exchange between researchers and policy-makers and development of practical guidelines to ensure that these frameworks are more accessible, applicable and sustainable in informing policy making.

International validation of quality indicators for evaluating priority setting in low income countries: process and key lessons

Background

While there have been efforts to develop frameworks to guide healthcare priority setting; there has been limited focus on evaluation frameworks. Moreover, while the few frameworks identify quality indicators for successful priority setting, they do not provide the users with strategies to verify these indicators. Kapiriri and Martin (Health Care Anal 18:129-147, 2010) developed a framework for evaluating priority setting in low and middle income countries. This framework provides BOTH parameters for successful priority setting and proposes means of their verification. Before its use in real life contexts, this paper presents results from a validation process of the framework.

Methods

The framework validation involved 53 policy makers and priority setting researchers at the global, national and sub-national levels (in Uganda). They were requested to indicate the relative importance of the proposed parameters as well as the feasibility of obtaining the related information. We also pilot tested the proposed means of verification.

Results

Almost all the respondents evaluated all the parameters, including the contextual factors, as ‘very important’. However, some respondents at the global level thought ‘presence of incentives to comply’, ‘reduced disagreements’, ‘increased public understanding,’ ‘improved institutional accountability’ and ‘meeting the ministry of health objectives’, which could be a reflection of their levels of decision making. All the proposed means of verification were assessed as feasible with the exception of meeting observations which would require an insider. These findings results were consistent with those obtained from the pilot testing.

Conclusions

These findings are relevant to policy makers and researchers involved in priority setting in low and middle income countries. To the best of our knowledge, this is one of the few initiatives that has involved potential users of a framework (at the global and in a Low Income Country) in its validation. The favorable validation of all the parameters at the national and sub-national levels implies that the framework has potential usefulness at those levels, as is. The parameters that were disputed at the global level necessitate further discussion when using the framework at that level. The next step is to use the validated framework in evaluating actual priority setting at the different levels.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2360-7) contains supplementary material, which is available to authorized users.

Resource allocation criteria in a hospital

INTRODUCTION

Allocate fixed resources among competing users is a challenge in terms of hospital management in order to obtain the best performance considering strategic objectives. In order to address this need, a system of evaluation in an important research and teaching hospital was designed. This study describes resource allocation criteria in a hospital focusing on the evaluation system and its developed application methodology.

METHODS

The indicator system allows the strategic management to rapidly detect the priorities in the evaluations of the Strategic, Organizational, Managerial, Economic, Research and Qualitative conditions of each unit. The chosen indicators are expressed with three numerical values, (1 indicating critical status, 2 acceptable conditions and 3 a good operational situation).

RESULTS AND DISCUSSION

The adopted evaluation system considered different thematic areas: Strategic, Organizational, Managerial, Economic, Research and Qualitative. In order to define each area, 3 fields of evaluation have been chosen. The indicators have been structured according to a pyramid system allowing creating a single indicator for each area for each unit. Furthermore, a single indicator has been fixed in order to facilitate a first consideration on whether to carry out or not closer examinations of the most critical units. This manuscript describes an attempt to define objective criteria for the allocation of scarce resources in order to achieve the hospital's strategic objectives. The indicators identified allow to obtain an overall score for each unit, which allows the management to prioritize the needs.

Citizens' preferences on healthcare expenditure allocation: evidence from Greece

BACKGROUND OF CONTEXT

Priority setting and resource allocation across various healthcare functions are critical issues in health policy and strategic decision making. As health resources are limited while there are so many health challenges to resolve, consumers and payers have to make difficult decisions about expenditure allocation.

OBJECTIVE

Our research focus on the (dis)agreement between citizens' preferences and actual public health expenditure across broad healthcare functions, on whether this (dis)agreement is persistent, on whether various demographic factors amplify this (dis)agreement and to derive useful implications for public health policies.

SETTING AND PARTICIPANTS

Using survey data of 3029 citizens in Greece for the year 2012 and employing logit estimation techniques, we analysed the effect of demographic and other factors in shaping citizens' (dis)agreement with public health expenditure allocation.

RESULTS

Our results demonstrate the important role of income, family members and residence in shaping citizens' preferences regarding health expenditure priorities in almost all healthcare functions, while other demographic factors such as job, age, gender and marital status do partly associate and play a significant role.

CONCLUSIONS

Government should encourage the citizens' participation in the decision-making process in order to eliminate the unveiled and significant disagreement between citizens' preferences and actual public health expenditure across all healthcare functions.

Implementing priority setting frameworks: Insights from leading researchers

In spite of a substantial literature developing frameworks for policymakers to use in resource allocation decisions in healthcare, there remains limited published work reporting on the implementation or evaluation of such frameworks in practice. This paper presents findings of a targeted survey of 18 leading researchers around the implementation and evaluation of priority-setting exercises. Approximately one third of respondents knew of situations where recommendations of priority-setting exercises had been implemented, one third knew that recommendations had not been implemented and the final third responded that they did not know whether recommendations had been adopted. The lack of evidence linking the implementation of priority-setting recommendations to equity and efficiency outcomes was highlighted by all respondents. Features identified as facilitating successful implementation of priority-setting recommendations included having a climate ready to accept priority-setting, good leadership or a 'champion' for the priority-setting process and having a health economist to guide the process. Successful disinvestment was very uncommon in the experience of the researchers surveyed. Recommendations emerging from Program Budgeting and Marginal Analysis exercises appeared to be more widely implemented than those coming from alternative processes. Identifying if the process was repeated following the initial process was suggested as a means to measure success.

Criteria for Drug Reimbursement Decision-Making: An Emerging Public Health Challenge in Bulgaria

BACKGROUND

During times of fiscal austerity, means of reimbursement decision-making are of particular interest for public health theory and practice. Introduction of advanced health technologies, growing health expenditures and increased public scrutiny over drug reimbursement decisions have pushed governments to consider mechanisms that promote the use of effective health technologies, while constraining costs.

AIMS

The study's aim was to explore the current rationale of the drug reimbursement decision-making framework in Bulgaria. Our pilot research focused on one particular component of this process - the criteria used - because of the critical role that criteria are known to have in setting budgets and priorities in the field of public health. The analysis pursued two objectives: to identify important criteria relevant to drug reimbursement decision-making and to unveil relationships between theory and practice.

STUDY DESIGN

Cross-sectional study.

METHODS

The study was realized through a closed-ended survey on reimbursement criteria among four major public health stakeholders - medical professionals, patients, health authorities, and industry. Empirical outcomes were then cross-compared with the theoretical framework, as defined by current Bulgarian public health legislation. Analysis outlined what is done and what needs to be done in the field of public health reimbursement decision-making.

RESULTS

Bulgarian public health stakeholders agreed on 15 criteria to form a tentative optimal framework for drug reimbursement decision-making. The most apparent gap between the empirically found preferences and the official legislation is the lack of consideration for the strength of evidence in reimbursement decisions.

CONCLUSION

Bulgarian policy makers need to address specific gaps, such as formal consideration for strength of evidence, explicit role of efficiency criteria, and means to effectively empower patient and citizen involvement in public health decision-making. Drug reimbursement criteria have to be integrated into legitimate public health decision support tools that ensure the achievement of national public health objectives. These recommendations could be expanded to all Eastern European countries who share common public health problems.

Communicable Diseases Prioritized According to Their Public Health Relevance, Sweden, 2013

To establish strategic priorities for the Public Health Agency of Sweden we prioritized pathogens according to their public health relevance in Sweden in order to guide resource allocation. We then compared the outcome to ongoing surveillance. We used a modified prioritization method developed at the Robert Koch Institute in Germany. In a Delphi process experts scored pathogens according to ten variables. We ranked the pathogens according to the total score and divided them into four priority groups. We then compared the priority groups to self-reported time spent on surveillance by epidemiologists and ongoing programmes for surveillance through mandatory and/or voluntary notifications and for surveillance of typing results. 106 pathogens were scored. The result of the prioritization process was similar to the outcome of the prioritization in Germany. Common pathogens such as calicivirus and Influenza virus as well as blood-borne pathogens such as human immunodeficiency virus, hepatitis B and C virus, gastro-intestinal infections such as Campylobacter and Salmonella and vector-borne pathogens such as Borrelia were all in the highest priority group. 63% of time spent by epidemiologists on surveillance was spent on pathogens in the highest priority group and all pathogens in the highest priority group, except for Borrelia and varicella-zoster virus, were under surveillance through notifications. Ten pathogens in the highest priority group (Borrelia, calicivirus, Campylobacter, Echinococcus multilocularis, hepatitis C virus, HIV, respiratory syncytial virus, SARS- and MERS coronavirus, tick-borne encephalitis virus and varicella-zoster virus) did not have any surveillance of typing results. We will evaluate the possibilities of surveillance for the pathogens in the highest priority group where we currently do not have any ongoing surveillance and evaluate the need of surveillance for the pathogens from the low priority group where there is ongoing surveillance in order to focus our work on the pathogens with the highest relevance.

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation

BACKGROUND

Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices.

METHODS

We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken.

RESULTS

Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values.

CONCLUSION

Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought.

Investigating client perception and attitude to decentralization of HIV/AIDS treatment services to primary health centres in three Nigerian states

Background

The opinions of consumers in decentralization provide insights into possible levels of improvement in access and uptake of services.

Objectives

The study examined clients' perception and attitude towards decentralization of antiretroviral treatment services from central hospitals to primary health centres (PHCs).

Methodology

A cross‐sectional survey was undertaken in three states in Nigeria. A total of 1265 exit interviews were conducted with HIV/AIDS clients in nine health facilities.

Findings

About a third of all the respondents were not comfortable with receiving ART services in a PHC facility close to where they live. The reasons given by 385 respondents who would not want their treatment centres near were as follows: fear of disclosure, 299 (80.4%); fear of being discriminated against, 278 (74.3%); and satisfaction with care received at current facility, 278 (74.3%). However, more than 90% of respondents in all three states felt that decentralization of ART services to PHCs would be beneficial in controlling HIV/AIDS in Nigeria; the difference in respondents' perception across the three state was found to be statistically significant (P < 0.001).

Conclusion

The findings imply that scaling‐up of treatment services to PHCs would be widely accepted, and probably result in increased uptake. However, this must be accompanied by targeted behaviour change interventions for clients who for the fear of disclosure and stigma would still not access care from proximate facilities.

An evaluation tool for assessing performance in priority setting and resource allocation: multi-site application to identify strengths and weaknesses

Introduction

An evaluation tool should help improve formal priority setting and resource allocation (PSRA) processes in Canada and elsewhere. These are crucial to maximizing value from limited resources.

Methods

On the basis of case studies, balanced scorecard development protocols and use-focused evaluation principles, an evaluation tool was developed based on an existing framework for high PSRA performance and implemented in two health care organizations in British Columbia, Canada.

Results

Implementation of the tool identified areas of strength, improvement and weakness in the pilot organizations’ processes for PSRA including: communication, staff engagement and culture. Refinements were identified and incorporated into the tool for future application.

Conclusion

This is the first documented multi-site application of such an evaluation tool. Broader dissemination should have use both in further refining the basis of the tool and in catalysing improved performance of PSRA practice.

Infectious diseases prioritisation for event-based surveillance at the European Union level for the 2012 Olympic and Paralympic Games

In 2012, London hosted the Olympic and Paralympic Games (the Games), with events occurring throughout the United Kingdom (UK) between 27 July and 9 September 2012. Public health surveillance was performed by the Health Protection Agency (HPA). Collaboration between the HPA and the European Centre for Disease Prevention and Control (ECDC) was established for the detection and assessment of significant infectious disease events (SIDEs) occurring outside the UK during the time of the Games. Additionally, ECDC undertook an internal prioritisation exercise to facilitate ECDC’s decisions on which SIDEs should have preferentially enhanced monitoring through epidemic intelligence activities for detection and reporting in daily surveillance in the European Union (EU). A team of ECDC experts evaluated potential public health risks to the Games, selecting and prioritising SIDEs for event-based surveillance with regard to their potential for importation to the Games, occurrence during the Games or export to the EU/European Economic Area from the Games. The team opted for a multilevel approach including comprehensive disease selection, development and use of a qualitative matrix scoring system and a Delphi method for disease prioritisation. The experts selected 71 infectious diseases to enter the prioritisation exercise of which 27 were considered as priority for epidemic intelligence activities by ECDC for the EU for the Games.

The disease burden of hepatitis B, influenza, measles and salmonellosis in Germany: first results of the Burden of Communicable Diseases in Europe Study

Setting priorities in the field of infectious diseases requires evidence-based and robust baseline estimates of disease burden. Therefore, the European Centre for Disease Prevention and Control initiated the Burden of Communicable Diseases in Europe (BCoDE) project. The project uses an incidence- and pathogen-based approach to measure the impact of both acute illness and sequelae of infectious diseases expressed in disability-adjusted life years (DALYs). This study presents first estimates of disease burden for four pathogens in Germany. The number of reported incident cases adjusted for underestimation served as model input. For the study period 2005–2007, the average disease burden was estimated at 33 116 DALYs/year for influenza virus, 19 115 DALYs/year for Salmonella spp., 8708 DALYs/year for hepatitis B virus and 740 DALYs/year for measles virus. This methodology highlights the importance of sequelae, particularly for hepatitis B and salmonellosis, because if omitted, the burden would have been underestimated by 98% and 56%, respectively.

Health sector priority setting at meso-level in lower and middle income countries: lessons learned, available options and suggested steps

Setting priority for health programming and budget allocation is an important issue, but there is little consensus on related processes. It is particularly relevant in low resource settings and at province- and district- or "meso-level", where contextual influences may be greater, information scarce and capacity lower. Although recent changes in disease epidemiology and health financing suggest even greater need to allocate resources effectively, the literature is relatively silent on evidence-based priority-setting in low and middle income countries (LMICs). We conducted a comprehensive review of the peer-reviewed and grey literature on health resource priority-setting in LMICs, focussing on meso-level and the evidence-based priority-setting processes (PSPs) piloted or suggested there. Our objective was to assess PSPs according to whether they have influenced resource allocation and impacted the outcome indicators prioritised. An exhaustive search of the peer-reviewed and grey literature published in the last decade yielded 57 background articles and 75 reports related to priority-setting at meso-level in LMICs. Although proponents of certain PSPs still advocate their use, other experts instead suggest broader elements to guide priority-setting. We conclude that currently no process can be confidently recommended for such settings. We also assessed the common reasons for failure at all levels of priority-setting and concluded further that local authorities should additionally consider contextual and systems limitations likely to prevent a satisfactory process and outcomes, particularly at meso-level. Recent literature proposes a list of related attributes and warning signs, and facilitated our preparation of a simple decision-tree or roadmap to help determine whether or not health systems issues should be improved in parallel to support for needed priority-setting; what elements of the PSP need improving; monitoring, and evaluation. Health priority-setting at meso-level in LMICs can involve common processes, but will often require additional attention to local health systems.

Decision maker perceptions of resource allocation processes in Canadian health care organizations: a national survey

BACKGROUND

Resource allocation is a key challenge for healthcare decision makers. While several case studies of organizational practice exist, there have been few large-scale cross-organization comparisons.

METHODS

Between January and April 2011, we conducted an on-line survey of senior decision makers within regional health authorities (and closely equivalent organizations) across all Canadian provinces and territories. We received returns from 92 individual managers, from 60 out of 89 organizations in total. The survey inquired about structures, process features, and behaviours related to organization-wide resource allocation decisions. We focus here on three main aspects: type of process, perceived fairness, and overall rating.

RESULTS

About one-half of respondents indicated that their organization used a formal process for resource allocation, while the others reported that political or historical factors were predominant. Seventy percent (70%) of respondents self-reported that their resource allocation process was fair and just over one-half assessed their process as 'good' or 'very good'. This paper explores these findings in greater detail and assesses them in context of the larger literature.

CONCLUSION

Data from this large-scale cross-jurisdictional survey helps to illustrate common challenges and areas of positive performance among Canada's health system leadership teams.

Using evaluation theory in priority setting and resource allocation

PURPOSE

Public sector interest in methods for priority setting and program or policy evaluation has grown considerably over the last several decades, given increased expectations for accountable and efficient use of resources and emphasis on evidence-based decision making as a component of good management practice. While there has been some occasional effort to conduct evaluation of priority setting projects, the literatures around priority setting and evaluation have largely evolved separately. In this paper, the aim is to bring them together.

DESIGN/METHODOLOGY/APPROACH

The contention is that evaluation theory is a means by which evaluators reflect upon what it is they are doing when they do evaluation work. Theories help to organize thinking, sort out relevant from irrelevant information, provide transparent grounds for particular implementation choices, and can help resolve problematic issues which may arise in the conduct of an evaluation project.

FINDINGS

A detailed review of three major branches of evaluation theory--methods, utilization, and valuing--identifies how such theories can guide the development of efforts to evaluate priority setting and resource allocation initiatives. Evaluation theories differ in terms of their guiding question, anticipated setting or context, evaluation foci, perspective from which benefits are calculated, and typical methods endorsed.

ORIGINALITY/VALUE

Choosing a particular theoretical approach will structure the way in which any priority setting process is evaluated. The paper suggests that explicitly considering evaluation theory makes key aspects of the evaluation process more visible to all stakeholders, and can assist in the design of effective evaluation of priority setting processes; this should iteratively serve to improve the understanding of priority setting practices themselves.

Prioritizing strategies for comprehensive liver cancer control in Asia: a conjoint analysis

Background

Liver cancer is a complex and burdensome disease, with Asia accounting for 75% of known cases. Comprehensive cancer control requires the use of multiple strategies, but various stakeholders may have different views as to which strategies should have the highest priority. This study identified priorities across multiple strategies for comprehensive liver cancer control (CLCC) from the perspective of liver cancer clinical, policy, and advocacy stakeholders in China, Japan, South Korea and Taiwan. Concordance of priorities was assessed across the region and across respondent roles.

Methods

Priorities for CLCC were examined as part of a cross-sectional survey of liver cancer experts. Respondents completed several conjoint-analysis choice tasks to prioritize 11 strategies. In each task, respondents judged which of two competing CLCC plans, consisting of mutually exclusive and exhaustive subsets of the strategies, would have the greatest impact. The dependent variable was the chosen plan, which was then regressed on the strategies of different plans. The restricted least squares (RLS) method was utilized to compare aggregate and stratified models, and t-tests and Wald tests were used to test for significance and concordance, respectively.

Results

Eighty respondents (69.6%) were eligible and completed the survey. Their primary interests were hepatitis (26%), hepatocellular carcinoma (HCC) (58%), metastatic liver cancer (10%) and transplantation (6%). The most preferred strategies were monitoring at-risk populations (p<0.001), clinician education (p<0.001), and national guidelines (p<0.001). Most priorities were concordant across sites except for three strategies: transplantation infrastructure (p=0.009) was valued lower in China, measuring social burden (p=0.037) was valued higher in Taiwan, and national guidelines (p=0.025) was valued higher in China. Priorities did not differ across stakeholder groups (p=0.438).

Conclusions

Priorities for CLCC in Asia include monitoring at-risk populations, clinician education, national guidelines, multidisciplinary management, public awareness and centers of excellence. As most priorities are relatively concordant across the region, multilateral approaches to addressing comprehensive liver cancer would be beneficial. However, where priorities are discordant among sites, such as transplantation infrastructure, strategies should be tailored to local needs.

Decision-making in healthcare: a practical application of partial least square path modelling to coverage of newborn screening programmes

Background

Decision-making in healthcare is complex. Research on coverage decision-making has focused on comparative studies for several countries, statistical analyses for single decision-makers, the decision outcome and appraisal criteria. Accounting for decision processes extends the complexity, as they are multidimensional and process elements need to be regarded as latent constructs (composites) that are not observed directly. The objective of this study was to present a practical application of partial least square path modelling (PLS-PM) to evaluate how it offers a method for empirical analysis of decision-making in healthcare.

Methods

Empirical approaches that applied PLS-PM to decision-making in healthcare were identified through a systematic literature search. PLS-PM was used as an estimation technique for a structural equation model that specified hypotheses between the components of decision processes and the reasonableness of decision-making in terms of medical, economic and other ethical criteria. The model was estimated for a sample of 55 coverage decisions on the extension of newborn screening programmes in Europe. Results were evaluated by standard reliability and validity measures for PLS-PM.

Results

After modification by dropping two indicators that showed poor measures in the measurement models’ quality assessment and were not meaningful for newborn screening, the structural equation model estimation produced plausible results. The presence of three influences was supported: the links between both stakeholder participation or transparency and the reasonableness of decision-making; and the effect of transparency on the degree of scientific rigour of assessment. Reliable and valid measurement models were obtained to describe the composites of ‘transparency’, ‘participation’, ‘scientific rigour’ and ‘reasonableness’.

Conclusions

The structural equation model was among the first applications of PLS-PM to coverage decision-making. It allowed testing of hypotheses in situations where there are links between several non-observable constructs. PLS-PM was compatible in accounting for the complexity of coverage decisions to obtain a more realistic perspective for empirical analysis. The model specification can be used for hypothesis testing by using larger sample sizes and for data in the full domain of health technologies.

Facilitating comparative effectiveness research in cancer genomics: evaluating stakeholder perceptions of the engagement process

AIMS

The Center for Comparative Effectiveness Research in Cancer Genomics completed a 2-year stakeholder-guided process for the prioritization of genomic tests for comparative effectiveness research studies. We sought to evaluate the effectiveness of engagement procedures in achieving project goals and to identify opportunities for future improvements.

MATERIALS & METHODS

The evaluation included an online questionnaire, one-on-one telephone interviews and facilitated discussion. Responses to the online questionnaire were tabulated for descriptive purposes, while transcripts from key informant interviews were analyzed using a directed content analysis approach.

RESULTS

A total of 11 out of 13 stakeholders completed both the online questionnaire and interview process, while nine participated in the facilitated discussion. Eighty-nine percent of questionnaire items received overall ratings of agree or strongly agree; 11% of responses were rated as neutral with the exception of a single rating of disagreement with an item regarding the clarity of how stakeholder input was incorporated into project decisions. Recommendations for future improvement included developing standard recruitment practices, role descriptions and processes for improved communication with clinical and comparative effectiveness research investigators.

CONCLUSIONS

Evaluation of the stakeholder engagement process provided constructive feedback for future improvements and should be routinely conducted to ensure maximal effectiveness of stakeholder involvement.

Research priorities in the field of HIV and AIDS in Iran

BACKGROUND

HIV is a multidimensional problem. Therefore, prioritization of research topics in this field is a serious challenge. We decided to prioritize the major areas of research on HIV/AIDS in Iran.

MATERIALS AND METHODS

In a brain-storming session with the main national and provincial stakeholders and experts from different relevant fields, the direct and indirect dimensions of HIV/AIDS and its related research issues were explored. Afterward, using the Delphi method, we sent questionnaires to 20 experts (13 respondents) from different sectors. In this electronic based questioner, we requested experts to evaluate main topics and their subtopics. The ranges of scores were between 0 and 100.

RESULTS

The score of priorities of main themes were preventive activities (43.2), large scale planning (25.4), the estimation of the HIV/AIDS burden (20.9), and basic scientific research (10.5). The most important priority in each main theme was education particularly in high risk groups (52.5), developing the national strategy to address the epidemic (31.8), estimation of the incidence and prevalence among high-risk groups (59.5) and developing new preventive methods (66.7), respectively.

CONCLUSIONS

The most important priorities of researches on HIV/AIDS were preventive activities and developing national strategy. As high risk groups are the most involved people in the epidemic, and they are also the most hard-to-reach sub-populations, a national well designated comprehensive strategy is essential. However, we believe with a very specific and directed scheme, special attention to research in basic sciences is necessary, at least in limited number of institutes.