Selecting Performance Indicators and Targets in Health Care: An International Scoping Review and Standardized Process Framework

Measuring what matters: Context-specific indicators for assessing immunisation performance in Pacific Island Countries and Areas

Increasing countries' access to data can improve immunisation coverage through evidence-based decision-making. However, data collection and reporting is resource-intensive, so needs to be pragmatic, especially in low-and-middle-income countries. We aimed to identify which indicators are most important for measuring, and improving, national immunisation performance in Pacific Island Countries (PICs). We conducted an expert elicitation study, asking 13 experts involved in delivering immunisation programs, decision-makers, health information specialists, and global development partners across PICs to rate 41 indicators based on their knowledge of the feasibility and relevance of each indicator. We also asked experts their preferences for indicators to be retained or removed from a list of indicators for PICs. Experts participated in two rating rounds, with a discussion on the reasons for ratings before the second round. We calculated mean scores for feasibility and relevance, and ranked indicators based on experts' preferences and mean scores. We used framework analysis to identify reasons for selecting indicators. Experts agreed that certain indicators were essential to measure (e.g. data use in program planning and measles vaccination coverage), but preferences varied for most indicators. Preferences to include indicators in a set of indicators for PICs moderately correlated with scores for relevance (r = 0.68) and feasibility (r = 0.56). In discussions, experts highlighted usefulness for decision-making and ease of data collection, reporting and interpretation as the main reasons driving indicator selection. Country-specific factors such as health system factors, roles and influence of various immunisation actors, and macro-level factors (namely population size, distribution and mobility) affected relevance and feasibility, leading us to conclude that a single set of indicators for all PICs is inappropriate. Rather than having a strict set of indicators that all countries must measure and report against, performance indicators should be flexible, country-specific, and selected in consultation with immunisation actors who collect and use the data.

Multiple case study of processes used by hospitals to select performance indicators: do they align with best practices?

Several health policy institutes recommend reducing the number of indicators monitored by hospitals to better focus on indicators most relevant to local contexts. To determine which indicators are the most appropriate to eliminate, one must understand how indicator selection processes are undertaken. This study classifies hospital indicator selection processes and analyzes how they align with practices outlined in the 5-P Indicator Selection Process Framework. This qualitative, multiple case study examined indicator selection processes used by four large acute care hospitals in Ontario, Canada. Data were collected through 13 semistructured interviews and document analysis. A thematic analysis compared processes to the 5-P Indicator Selection Process Framework. Two types of hospital indicator selection processes were identified. Hospitals deployed most elements found within the 5-P Indicator Selection Process Framework including setting clear aims, having governance structures, considering indicators required by health agencies, and categorizing indicators into strategic themes. Framework elements largely absent included: adopting evidence-based selection criteria; incorporating finance and human resources indicators; considering if indicators measure structures, processes, or outcomes; and engaging a broader set of end users in the selection process. Hospitals have difficulty in balancing how to monitor government-mandated indicators with indicators more relevant to local operations. Hospitals often do not involve frontline managers in indicator selection processes. Not engaging frontline managers in selecting indicators may risk hospitals only choosing government-mandated indicators that are not reflective of frontline operations or valued by those managers accountable for improving unit-level performance.

The impact of decentralisation on health systems: a systematic review of reviews

BACKGROUND

Decentralisation is a common mechanism for health system reform; yet, evidence of how it impacts health systems remains fragmented. Despite published findings from primary and secondary research illustrating range of impacts, a comprehensive and clear understanding is currently lacking. This review synthesised the existing evidence to assess how decentralisation (by devolution) impacts each of the six WHO building blocks, and the health system.

METHOD

We systematically searched five electronic databases for reviews exploring impact of decentralisation on health systems, globally. Reviews, both systematic and non-systematic, published in the English language from January 1990 to February 2022 were included. Data were synthesised across each of six building blocks. Quality assessment of the reviews was conducted using Critical Appraisal Skills Program for systematic and Scale for Assessment of Narrative Review Articles for non-systematic reviews.

RESULTS

Nine reviews, each addressing somewhat different questions, contexts and issues, were included. A range of positive and negative impacts of decentralisation on health system building blocks were identified; yet, overall, the impacts were more negative. Although inconclusive, evidence suggested that the impacts on leadership and governance and financing components in particular shape the impact on overall health system. Assessment of how the impact on building blocks translates to the broader impact on health systems is challenged by the dynamic complexities related to contexts, process and the health system itself.

CONCLUSIONS

Decentralisation, even if well intentioned, can have unintended consequences. Despite the difficulty of reaching universally applicable conclusions about the pros and cons of decentralisation, this review highlights some of the common potential issues to consider in advance.

PROSPERO REGISTRATION NUMBER

CRD42022302013.

A Potential Indicator for Assessing Patient Blood Management Standard Implementation

(1) Background: Patient blood management (PBM) program as a multidisciplinary practice and a standard of care for the anemic surgical patient has an increasingly important role in reducing transfusions and optimizing both clinical outcomes and costs. Documented success of PBM implementation is not sufficient for implementation of recommendations and correct use at hospital level. The primary objective of our study was to define a composite patient blood management process safety index-Safety Index in PBM (SIPBM)-that measures the impact of screening and treating anemic patients on the efficiency and effectiveness of the patient care process undergoing elective surgery. (2) Methods: We conducted a retrospective comparative study in a tertiary hospital by collecting data and analyzing the Safety Index in PBM (SIPBM) in patients undergoing major elective surgical procedures. (3) Results: The percentage of patients from the total of 354 patients (178 in 2019 and 176 in 2022) included in the study who benefited from preoperative iron treatment increased in 2022 compared to 2019 from 27.40% to 36.71%. The median value of the SIPBM was 1.00 in both periods analyzed, although there is a significant difference between the two periods (p < 0.005), in favor of 2022. (4) Conclusions: Measuring the effectiveness of PBM implementation and providing ongoing feedback through the Safety Index in PBM (SIPBM) increases the degree to which opportunities to improve the PBM process are identified. The study represents a first step for future actions and baselines to develop tools to measure the safety and impact of the patient blood management process in the surgical field.