Multidimensional performance assessment of public sector organisations using dominance criteria

Exploring Norwegian homecare healthcare professionals' perceptions of risk and the link to high-quality care: a qualitative multiple case study

OBJECTIVES

Homecare is a critical component of the ongoing restructuring of healthcare worldwide, given the shift from institution- to home-based care. The homecare evidence base still contains significant gaps: There is a lack of knowledge regarding quality and safety work and interventions. This study explores how home healthcare professionals perceive and use the concept of risk to guide them in providing high-quality healthcare while maintaining resilience.

DESIGN

The study design is a qualitative multiple case study. The phenomena explored were risk perception, sensemaking and adaptations of care delivered to patients in their homes. Inductive content analysis was conducted.

SETTING

The study was conducted in three Norwegian municipalities. Each municipality was defined as a single case.

PARTICIPANTS

Interviews with healthcare professionals were performed both individually and in focus groups of three to five persons. 19 interviews with 35 informants were conducted: 11 individual semistructured interviews and 8 focus groups.

RESULTS

Four themes were identified: 'professionalism is constantly prioritising and aligning care based on here-and-now observations' 'teamwork feels safe and enhances quality' 'taking responsibility for system risk' and 'reluctantly accepting the extended expectations from society'.

CONCLUSIONS

To make sense of risk when aspiring for high-quality care in everyday work, the healthcare professionals in this sample mainly used their clinical gaze, gut feeling and experience to detect subtle changes in the patients' condition. Assessing risk information, not only individually but also as a team, was reportedly crucial for high-quality care. Healthcare professionals emphasised the well-being, safety and soundness of the patients when acting on risk information. They felt obliged to act on their gut feeling, moral compass and clinical understanding of quality.

An econometric approach to aggregating multiple cardiovascular outcomes in German hospitals

OBJECTIVE

Development of an aggregate quality index to evaluate hospital performance in cardiovascular events treatment.

METHODS

We applied a two-stage regression approach using an accelerated failure time model based on variance weights to estimate hospital quality over four cardiovascular interventions: elective coronary bypass graft, elective cardiac resynchronization therapy, and emergency treatment for acute myocardial infarction. Mortality and readmissions were used as outcomes. For the estimation we used data from a statutory health insurer in Germany from 2005 to 2016.

RESULTS

The precision-based weights calculated in the first stage were higher for mortality than for readmissions. In general, teaching hospitals performed better in our ranking of hospital quality compared to non-teaching hospitals, as did private not-for-profit hospitals compared to hospitals with public or private for-profit ownership.

DISCUSSION

The proposed approach is a new method to aggregate single hospital quality outcomes using objective, precision-based weights. Likelihood-based accelerated failure time models make use of existing data more efficiently compared to widely used models relying on dichotomized data. The main advantage of the variance-based weights approach is that the extent to which an indicator contributes to the aggregate index depends on the amount of its variance.

Factors associated with longer wait times, admission and reattendances in older patients attending emergency departments: an analysis of linked healthcare data

BACKGROUND AND OBJECTIVE

Care for older patients in the ED is an increasingly important issue with the ageing society. To better assess the quality of care in this patient group, we assessed predictors for three outcomes related to ED care: being seen and discharged within 4 hours of ED arrival; being admitted from ED to hospital and reattending the ED within 30 days. We also used these outcomes to identify better-performing EDs.

METHODS

The CUREd Research Database was used for a retrospective observational study of all 1 039 251 attendances by 368 754 patients aged 75+ years in 18 type 1 EDs in the Yorkshire and the Humber region of England between April 2012 and March 2017. We estimated multilevel logit models, accounting for patients' characteristics and contact with emergency services prior to ED arrival, time variables and the ED itself.

RESULTS

Patients in the oldest category (95+ years vs 75-80 years) were more likely to have a long ED wait (OR=1.13 (95% CI=1.10 to 1.15)), hospital admission (OR=1.26 (95% CI=1.23 to 1.29)) and ED reattendance (OR=1.09 (95% CI=1.06 to 1.12)). Those who had previously attended (3+ vs 0 previous attendances) were more likely to have long wait (OR=1.07 (95% CI=1.06 to 1.08)), hospital admission (OR=1.10 (95% CI=1.09 to 1.12)) and ED attendance (OR=3.13 (95% CI=3.09 to 3.17)). Those who attended out of hours (vs not out of hours) were more likely to have a long ED wait (OR=1.33 (95% CI=1.32 to 1.34)), be admitted to hospital (OR=1.19 (95% CI=1.18 to 1.21)) and have ED reattendance (OR=1.07 (95% CI=1.05 to 1.08)). Those living in less deprived decile (vs most deprived decile) were less likely to have any of these three outcomes: OR=0.93 (95% CI=0.92 to 0.95), 0.92 (95% CI=0.90 to 0.94), 0.86 (95% CI=0.84 to 0.88). These characteristics were not strongly associated with long waits for those who arrived by ambulance. Emergency call handler designation was the strongest predictor of long ED waits and hospital admission: compared with those who did not arrive by ambulance; ORs for these outcomes were 1.18 (95% CI=1.16 to 1.20) and 1.85 (95% CI=1.81 to 1.89) for those designated less urgent; 1.37 (95% CI=1.33 to 1.40) and 2.13 (95% CI=2.07 to 2.18) for urgent attendees; 1.26 (95% CI=1.23 to 1.28) and 2.40 (95% CI=2.36 to 2.45) for emergency attendees; and 1.37 (95% CI=1.28 to 1.45) and 2.42 (95% CI=2.26 to 2.59) for those with life-threatening conditions. We identified two EDs whose patients were less likely to have a long ED, hospital admission or ED reattendance than other EDs in the region.

CONCLUSIONS

Age, previous attendance and attending out of hours were all associated with an increased likelihood of exceeding 4 hours in the ED, hospital admission and reattendance among patients over 75 years. These differences were less pronounced among those arriving by ambulance. Emergency call handler designation could be used to identify those at the highest risk of long ED waits, hospital admission and ED reattendance.

Comparing healthcare quality: A common framework for both ordinal and cardinal data with an application to primary care variation in England

The paper proposes a framework for comparing the quality of healthcare providers and assessing the variation in quality between them, which is directly applicable to both ordinal and cardinal quality data on a comparable basis. The resultant measures are sensitive to the full distribution of quality scores for each provider, not just the mean or the proportion meeting some binary quality threshold, thereby making full use of the multicategory response data increasingly available from patient experience surveys. The measures can also be standardized for factors such as age, sex, ethnicity, health and deprivation using a distribution regression model. We illustrate by measuring the quality of primary care services in England in 2019 using three different sources of publicly available, general practice-level information: multicategory response patient experience data, ordinal inspection ratings and cardinal clinical achievement scores. We find considerable variation at both local and regional levels using all three data sources. However, the correlation between the comparative quality indices calculated using the alternative data sources is weak, suggesting that they capture different aspects of general practice quality.

What makes a good quality indicator set? A systematic review of criteria

BACKGROUND

While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking.

OBJECTIVE

Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far.

METHODS

We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached.

RESULTS

Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]).

CONCLUSION

The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

What is the right level of spending needed for health and care in the UK?

The health and care sector plays a valuable role in improving population health and societal wellbeing, protecting people from the financial consequences of illness, reducing health and income inequalities, and supporting economic growth. However, there is much debate regarding the appropriate level of funding for health and care in the UK. In this Health Policy paper, we look at the economic impact of the COVID-19 pandemic and historical spending in the UK and comparable countries, assess the role of private spending, and review spending projections to estimate future needs. Public spending on health has increased by 3·7% a year on average since the National Health Service (NHS) was founded in 1948 and, since then, has continued to assume a larger share of both the economy and government expenditure. In the decade before the ongoing pandemic started, the rate of growth of government spending for the health and care sector slowed. We argue that without average growth in public spending on health of at least 4% per year in real terms, there is a real risk of degradation of the NHS, reductions in coverage of benefits, increased inequalities, and increased reliance on private financing. A similar, if not higher, level of growth in public spending on social care is needed to provide high standards of care and decent terms and conditions for social care staff, alongside an immediate uplift in public spending to implement long-overdue reforms recommended by the Dilnot Commission to improve financial protection. COVID-19 has highlighted major issues in the capacity and resilience of the health and care system. We recommend an independent review to examine the precise amount of additional funds that are required to better equip the UK to withstand further acute shocks and major threats to health.

Investigating the geographic disparity in quality of care: the case of hospital readmission after acute myocardial infarction in Italy

Unwarranted variation in the quality of care challenges the sustainability of healthcare systems. Especially in decentralised healthcare systems, it is crucial to understand the drivers behind regional differences in hospital qualities such as unplanned readmissions. This paper examines the factors that influence the risk of unplanned hospital readmission and the geographic disparity of readmission rate in Italy. We use hospital discharge data from 2010 to 2015 for patients above 65 years old admitted with Acute Myocardial Infarction. Employing hierarchical models, we identified the patient and hospital-level determinants for unplanned readmission. In line with the literature, the risk of readmission increases with age and being male, while hospitals with higher patient volume and capacity tend to have lower unplanned readmission. In particular, we find that after patient risk-adjustments, there are differential effects of hospitalisation length-of-stay on the probability of readmission across the hospitals that are governed by different payment systems. For hospitals under a prospective payment system, the effect of length-of-stay in reducing the probability of readmission is weaker than hospitals under an ex-post global budget, but the overall readmission rates are the lowest. Moreover, there are substantial geographic variations in readmission rate across Local Health Authority and regions, and these variations of unplanned readmission are explained by differences in hospital length-of-stay and surgical procedures used. Our results demonstrate that differential hospital behaviours can be one of the potential mechanisms that drive geographic quality disparities.

Evaluating the effect of healthcare providers on the clinical path of heart failure patients through a semi-Markov, multi-state model

Background

Investigating similarities and differences among healthcare providers, on the basis of patient healthcare experience, is of interest for policy making. Availability of high quality, routine health databases allows a more detailed analysis of performance across multiple outcomes, but requires appropriate statistical methodology.

Methods

Motivated by analysis of a clinical administrative database of 42,871 Heart Failure patients, we develop a semi-Markov, illness-death, multi-state model of repeated admissions to hospital, subsequent discharge and death. Transition times between these health states each have a flexible baseline hazard, with proportional hazards for patient characteristics (case-mix adjustment) and a discrete distribution for frailty terms representing clusters of providers. Models were estimated using an Expectation-Maximization algorithm and the number of clusters was based on the Bayesian Information Criterion.

Results

We are able to identify clusters of providers for each transition, via the inclusion of a nonparametric discrete frailty. Specifically, we detect 5 latent populations (clusters of providers) for the discharge transition, 3 for the in-hospital to death transition and 4 for the readmission transition. Out of hospital death rates are similar across all providers in this dataset. Adjusting for case-mix, we could detect those providers that show extreme behaviour patterns across different transitions (readmission, discharge and death).

Conclusions

The proposed statistical method incorporates both multiple time-to-event outcomes and identification of clusters of providers with extreme behaviour simultaneously. In this way, the whole patient pathway can be considered, which should help healthcare managers to make a more comprehensive assessment of performance.

Hospital quality indicators are not unidimensional: A reanalysis of Lieberthal and Comer

OBJECTIVE

To evaluate the dimensionality of hospital quality indicators treated as unidimensional in a prior publication.

DATA SOURCE/STUDY DESIGN

Pooled cross-sectional 2010-2011 Hospital Compare data (10/1/10 and 10/1/11 archives) and the 2012 American Hospital Association Annual Survey.

DATA EXTRACTION

We used 71 indicators of structure, process, and outcomes of hospital care in a principal component analysis of Ridit scores to evaluate the dimensionality of the indicators. We conducted an exploratory factor analysis using only the indicators in the Centers for Medicare & Medicaid Services' Hospital Value-Based Purchasing.

PRINCIPAL FINDINGS

There were four underlying dimensions of hospital quality: patient experience, mortality, and two clinical process dimensions.

CONCLUSIONS

Hospital quality should be measured using a variety of indicators reflecting different dimensions of quality. Treating hospital quality as unidimensional leads to erroneous conclusions about the performance of different hospitals.