Ranking hospitals on avoidable death rates derived from retrospective case record review: methodological observations and limitations

Disentangling the contribution of hospitals and municipalities for understanding patient level differences in one-year mortality risk after hip-fracture: A cross-classified multilevel analysis in Sweden

Background

One-year mortality after hip-fracture is a widely used outcome measure when comparing hospital care performance. However, traditional analyses do not explicitly consider the referral of patients to municipality care after just a few days of hospitalization. Furthermore, traditional analyses investigates hospital (or municipality) variation in patient outcomes in isolation rather than as a component of the underlying patient variation. We therefore aimed to extend the traditional approach to simultaneously estimate both case-mix adjusted hospital and municipality comparisons in order to disentangle the amount of the total patient variation in clinical outcomes that was attributable to the hospital and municipality level, respectively.

Methods

We determined 1-year mortality risk in patients aged 65 or above with hip fractures registered in Sweden between 2011 and 2014. We performed cross-classified multilevel analysis with 54,999 patients nested within 54 hospitals and 290 municipalities. We adjusted for individual demographic, socioeconomic and clinical characteristics. To quantify the size of the hospital and municipality variation we calculated the variance partition coefficient (VPC) and the area under the receiver operator characteristic curve (AUC).

Results

The overall 1-year mortality rate was 25.1%. The case-mix adjusted rates varied from 21.7% to 26.5% for the 54 hospitals, and from 18.9% to 29.5% for the 290 municipalities. The VPC was just 0.2% for the hospital and just 0.1% for the municipality level. Patient sociodemographic and clinical characteristics were strong predictors of 1-year mortality (AUC = 0.716), but adding the hospital and municipality levels in the cross-classified model had a minor influence (AUC = 0.718).

Conclusions

Overall in Sweden, one-year mortality after hip-fracture is rather high. However, only a minor part of the patient variation is explained by the hospital and municipality levels. Therefore, a possible intervention should be nation-wide rather than directed to specific hospitals or municipalities.

Identifying and quantifying variation between healthcare organisations and geographical regions: using mixed-effects models

When the degree of variation between healthcare organisations or geographical regions is quantified, there is often a failure to account for the role of chance, which can lead to an overestimation of the true variation. Mixed-effects models account for the role of chance and estimate the true/underlying variation between organisations or regions. In this paper, we explore how a random intercept model can be applied to rate or proportion indicators and how to interpret the estimated variance parameter.

Case fatality ratios for serious emergency conditions in the Republic of Ireland: a longitudinal investigation of trends over the period 2002-2014 using joinpoint analysis

BACKGROUND

In the past decade, the Republic of Ireland has undertaken significant reconfiguration programmes to improve emergency services. During this time the public healthcare system experienced a large real decrease in resources. This study assesses national and regional population outcomes over the period 2002-2014, and whether changes coincide with system reconfiguration and the financial restrictions imposed by the 2008 recession.

METHODS

Case fatality ratios (CFRs) were constructed for emergency conditions for 2002-2014. Total emergency conditions and individual condition trends were analysed nationally using joinpoint analysis. National results informed the investigation of trends at a regional and county level using an inverse standard error weighted generalised linear model with a log link to construct funnel plots. County-level CFRs were compared for the first and last 3 years of the period to further investigate the changes to county results over the 13 year period, specifically in comparison to the national-level CFR.

RESULTS

Nationally, there was an annual fall in CFRs (2.1%). The decline was faster from 2002 to 2007 (annual percentage change = - 3.4; 95% CI-4.4, - 2.4), compared to 2007-2014 (annual percentage change = - 1.2; 95% CI -1.9, - 0.5). The South-East had a lower rate of decrease and the West had a higher rate. Cross sectional analysis of two periods (2002-2004 and 2012-2014) showed high consistency in the counties performance relative to the national CFR in both periods.

CONCLUSION

Change in the national trend coincided with the onset of economic stress on the public health system. Attributing the decline in CFR improvement to economic factors is weakened by the uneven nature of the trend change. No distinct pattern of change was identified among regions which underwent substantial reconfiguration of emergency services.

Variation and statistical reliability of publicly reported primary care diagnostic activity indicators for cancer: a cross-sectional ecological study of routine data

OBJECTIVES

Recent public reporting initiatives in England highlight general practice variation in indicators of diagnostic activity related to cancer. We aimed to quantify the size and sources of variation and the reliability of practice-level estimates of such indicators, to better inform how this information is interpreted and used for quality improvement purposes.

DESIGN

Ecological cross-sectional study.

SETTING

English primary care.

PARTICIPANTS

All general practices in England with at least 1000 patients.

MAIN OUTCOME MEASURES

Sixteen diagnostic activity indicators from the Cancer Services Public Health Profiles.

RESULTS

Mixed-effects logistic and Poisson regression showed that substantial proportions of the observed variance in practice scores reflected chance, variably so for different indicators (between 7% and 85%). However, after accounting for the role of chance, there remained substantial variation between practices (typically up to twofold variation between the 75th and 25th centiles of practice scores, and up to fourfold variation between the 90th and 10th centiles). The age and sex profile of practice populations explained some of this variation, by different amounts across indicators. Generally, the reliability of diagnostic process indicators relating to broader populations of patients most of whom do not have cancer (eg, rate of endoscopic investigations, or urgent referrals for suspected cancer (also known as 'two week wait referrals')) was high (≥0.80) or very high (≥0.90). In contrast, the reliability of diagnostic outcome indicators relating to incident cancer cases (eg, per cent of all cancer cases detected after an emergency presentation) ranged from 0.24 to 0.54, which is well below recommended thresholds (≥0.70).

CONCLUSIONS

Use of indicators of diagnostic activity in individual general practices should principally focus on process indicators which have adequate or high reliability and not outcome indicators which are unreliable at practice level.