Funnel plots, performance variation and the Myocardial Infarction National Audit Project 2003-2004

A Quality Improvement Initiative to Reduce Abdominal X-ray use in Pediatric Patients Presenting with Constipation

OBJECTIVE

To decrease the percentage of patients undergoing an abdominal radiograph for evaluation of constipation within 24 hours of their initial gastroenterology visit.

STUDY DESIGN

In January 2015, we implemented a quality improvement, evidence-based guideline (EBG) aimed at standardizing the initial assessment of patients presenting for a new outpatient gastroenterology visit with a primary complaint of constipation. Over the subsequent 5 years, we followed the clinical impact of this guideline initiation with the goal of decreasing unnecessary abdominal radiograph use by 10% within 1 year of EBG launch. Patients older than 6 months and younger than 19 years were included.

RESULTS

In total, 6723 patients completed new patient gastroenterology visits for a primary diagnosis of constipation between 2013 and 2019. Of these, 993 (14.8%) patients had abdominal radiographs taken within 24 hours of their initial visit. Over the 7 years of this project, a mean frequency of abdominal radiograph use decreased from 24% to less than 11%. In addition, a 57% decrease in hospital charges related to decreased radiograph use for constipation was found. No increases in subsequent emergency department visits or hospitalization for constipation within 30 days of patients' initial visits were seen.

CONCLUSIONS

Through local adoption of an EBG, routine use of abdominal radiographs taken during a patient's initial outpatient gastroenterology visit for constipation decreased by more than 50%. This reduction was maintained over a subsequent 5-year period without any detrimental side effects.

Development of an Australia and New Zealand Lung Cancer Clinical Quality Registry: a protocol paper

INTRODUCTION

Lung cancer is the leading cause of cancer mortality, comprising the largest national cancer disease burden in Australia and New Zealand. Regional reports identify substantial evidence-practice gaps, unwarranted variation from best practice, and variation in processes and outcomes of care between treating centres. The Australia and New Zealand Lung Cancer Registry (ANZLCR) will be developed as a Clinical Quality Registry to monitor the safety, quality and effectiveness of lung cancer care in Australia and New Zealand.

METHODS AND ANALYSIS

Patient participants will include all adults >18 years of age with a new diagnosis of non-small-cell lung cancer (NSCLC), SCLC, thymoma or mesothelioma. The ANZLCR will register confirmed diagnoses using opt-out consent. Data will address key patient, disease, management processes and outcomes reported as clinical quality indicators. Electronic data collection facilitated by local data collectors and local, state and federal data linkage will enhance completeness and accuracy. Data will be stored and maintained in a secure web-based data platform overseen by registry management. Central governance with binational representation from consumers, patients and carers, governance, administration, health department, health policy bodies, university research and healthcare workers will provide project oversight.

ETHICS AND DISSEMINATION

The ANZLCR has received national ethics approval under the National Mutual Acceptance scheme. Data will be routinely reported to participating sites describing performance against measures of agreed best practice and nationally to stakeholders including federal, state and territory departments of health. Local, regional and (bi)national benchmarks, augmented with online dashboard indicator reporting will enable local targeting of quality improvement efforts.

Understanding the Differences in COVID-19 Case Fatality Rates Observed Across Alabama Counties

To understand county-level variation in case fatality rates of COVID-19, a statewide analysis of COVID-19 incidence and fatality data was performed, using publicly available incidence and case fatality rate data of COVID-19 for all 67 Alabama counties and mapped with health disparities at the county level. A specific adaptation of the Shewhart p-chart, called a funnel chart, was used to compare case fatality rates. Important differences in case fatality rates across the counties did not appear to be reflective of differences in testing or incidence rates. Instead, a higher prevalence of comorbidities and vulnerabilities was observed in high fatality rate counties, while showing no differences in access to acute care. Funnel charts reliably identify counties with unexpected high and low COVID-19 case fatality rates. Social determinants of health are strongly associated with such differences. These data may assist in public health decisions including vaccination strategies, especially in southern states with similar demographics.

QualDash: Adaptable Generation of Visualisation Dashboards for Healthcare Quality Improvement

Adapting dashboard design to different contexts of use is an open question in visualisation research. Dashboard designers often seek to strike a balance between dashboard adaptability and ease-of-use, and in hospitals challenges arise from the vast diversity of key metrics, data models and users involved at different organizational levels. In this design study, we present QualDash, a dashboard generation engine that allows for the dynamic configuration and deployment of visualisation dashboards for healthcare quality improvement (QI). We present a rigorous task analysis based on interviews with healthcare professionals, a co-design workshop and a series of one-on-one meetings with front line analysts. From these activities we define a metric card metaphor as a unit of visual analysis in healthcare QI, using this concept as a building block for generating highly adaptable dashboards, and leading to the design of a Metric Specification Structure (MSS). Each MSS is a JSON structure which enables dashboard authors to concisely configure unit-specific variants of a metric card, while offloading common patterns that are shared across cards to be preset by the engine. We reflect on deploying and iterating the design of OualDash in cardiology wards and pediatric intensive care units of five NHS hospitals. Finally, we report evaluation results that demonstrate the adaptability, ease-of-use and usefulness of QualDash in a real-world scenario.

A novel method to assess data quality in large medical registries and databases

BACKGROUND

There is no gold standard to assess data quality in large medical registries. Data auditing may be impeded by data protection regulations.

OBJECTIVE

To explore the applicability and usefulness of funnel plots as a novel tool for data quality control in critical care registries.

METHOD

The Swiss ICU-Registry from all 77 certified adult Swiss ICUs (2014 and 2015) was subjected to quality assessment (completeness/accuracy). For the analysis of accuracy, a list of logical rules and cross-checks was developed. Type and number of errors (true coding errors or implausible data) were calculated for each ICU, along with noticeable error rates (>mean + 3 SD in the variable's summary measure, or >99.8% CI in the respective funnel-plot).

RESULTS

We investigated 164 415 patient records with 31 items each (37 items: trauma diagnosis). Data completeness was excellent; trauma was the only incomplete item in 1495 of 9871 records (0.1%, 0.0%-0.6% [median, IQR]). In 15 572 patients records (9.5%), we found 3121 coding errors and 31 265 implausible situations; the latter primarily due to non-specific information on patients' provenance/diagnosis or supposed incoherence between diagnosis and treatments. Together, the error rate was 7.6% (5.9%-11%; median, IQR).

CONCLUSIONS

The Swiss ICU-Registry is almost complete and data quality seems to be adequate. We propose funnel plots as suitable, easy to implement instrument to assist in quality assurance of such a registry. Based on our analysis, specific feedback to ICUs with special-cause variation is possible and may promote such ICUs to improve the quality of their data.

Enhancing feedback on performance measures: the difference in outlier detection using a binary versus continuous outcome funnel plot and implications for quality improvement

Background

Hospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary outcomes, and continuous variables are dichotomised to fit this format. However, information is lost using a binary measure, which is only sensitive to detect differences in higher values (the tail) rather than the entire distribution. This study therefore aims to investigate whether different outlier hospitals are identified when using a funnel plot for a binary vs a continuous outcome. This is relevant for hospitals with suboptimal performance to decide whether performance can be improved by targeting processes for all patients or a subgroup with higher values.

Methods

We examined the door-to-needle time (DNT) of all (6080) patients with acute ischaemic stroke treated with intravenous thrombolysis in 65 hospitals in 2017, registered in the Dutch Acute Stroke Audit. We compared outlier hospitals in two funnel plots: the median DNT versus the proportion of patients with substantially delayed DNT (above the 90th percentile (P90)), whether these were the same or different hospitals. Two sensitivity analyses were performed using the proportion above the median and a continuous P90 funnel plot.

Results

The median DNT was 24 min and P90 was 50 min. In the binary funnel plot for the proportion of patients above P90, 58 hospitals had average performance, whereas in the funnel plot around the median 14 of these hospitals had significantly higher median DNT (24%). These hospitals can likely improve their DNT by focusing on care processes for all patients, not shown by the binary outcome funnel plot. Similar results were shown in sensitivity analyses.

Conclusion

Using funnel plots for continuous versus binary outcomes identify different outlier hospitals, which may enhance hospital feedback to direct more targeted improvement initiatives.

Leveraging new information technology to monitor medicine use in 71 residential aged care facilities: variation in polypharmacy and antipsychotic use

Objective

The aim of this study was to use routinely collected electronic medicines administration (eMAR) data in residential aged care (RAC) to investigate the quality use of medicines.

Design

A cross-sectional analysis of eMAR data.

Setting

71 RAC facilities in New South Wales and the Australian Capital Territory, Australia.

Participants

Permanent residents living in a participating facility on 1 October 2015.

Intervention

None.

Main Outcome Measures

Variation in polypharmacy (≥5 medications), hyper-polypharmacy (≥10 medications) and antipsychotic use across facilities was examined using funnel plot analysis.

Results

The study dataset included 4775 long-term residents. The mean resident age was 85.3 years and 70.6% of residents were female. The median facility size was 60 residents and 74.3% were in metropolitan locations. 84.3% of residents had polypharmacy, 41.2% hyper-polypharmacy and 21.0% were using an antipsychotic. The extent of polypharmacy (69.75-100% of residents), hyper-polypharmacy (38.81-76.19%) and use of antipsychotic medicines (0-75.6%) varied considerably across the 71 facilities.

Conclusions

Using eMAR data we found substantial variation in polypharmacy, hyper-polypharmacy and antipsychotic medicine use across 71 RAC facilities. Further investigation into the policies and practices of facilities performing above or below expected levels is warranted to understand variation and drive quality improvement.

Trends, patterns and predictive factors of infant and child mortality in well-performing and underperforming states of India: a secondary analysis using National Family Health Surveys

OBJECTIVES

This paper analyses the patterns and trends in the mortality rates of infants and children under the age of 5 in India (1992-2016) and quantifies the variation in performance between different geographical states through three rounds of nationally representative household surveys.

DESIGN

Three rounds of cross-sectional survey data.

SETTING

The study is conducted at the national level: India and its selected good-performing states, namely Haryana, Kerala, Maharashtra, Punjab and Tamil Nadu, and selected poor-performing states, namely Bihar, Chhattisgarh, Madhya Pradesh and Uttar Pradesh.

PARTICIPANTS

Adopting a multistage, stratified random sampling, 601 509 households with 699 686 women aged 15-49 years in 2015-2016, 109 041 households with 124 385 women aged 15-49 years in 2005-2006, and 88 562 households with 89 777 ever married women in the age group 13-49 years in 1992-1993 were selected.

RESULTS

Through the use of maps, this paper clearly shows that the overall trend in infant and child mortality is on a decline in India. Computation of relative change shows that majority of the states have witnessed over 50% reduction in both infant and under-5 mortality rates from National Family Health Survey (NFHS)-I to NFHS-4. However, the improvements are not evenly distributed, and there is huge variation in performance between states over time. Funnel plots show that the most populous states like Uttar Pradesh Bihar and Madhya Pradesh have underperformed consistently across the survey period from 1992 to 2016. Regression analysis comparing high-performing and low-performing states revealed that female infants and women with shorter birth intervals had greater risk of infant deaths in poor-performing states.

CONCLUSION

Attempts to reduce infant and child mortality rates in India are heading in the right direction. Even so, there is huge variation in performance between states. This paper recommends a mix of strategies that reduce child and infant mortality among the high-impact states where the biggest improvements can be expected, including the need to address neonatal mortality.

Assessment of Geographic Variations regarding the Transportation of Stroke Patients in Korea

Introduction

Stroke recently became the second cause of death in Korea. Nevertheless, many studies have reported that the delay of emergency medical service (EMS) transportation in specific areas is a major obstacle to reduce the mortality and morbidity of stroke. The objective of this study was to assess geographic variation regarding this factor at the national level in Korea throughout a funnel plot.

Setting

Emergency departments.

Methods

Data were obtained from the “2010 Emergency Medicine Annual Report” published by the National Emergency Medical Centre of Korea. This report describes the general characteristics of eligible Emergency Departments (EDs) across Korea. Therefore, we were able to calculate retrospectively the percentage of stroke patients who reached an ED within 3 hours after the development of symptoms against the volume of total visiting patients according to area. Subsequently, we generated a correlation matrix between ratio of stroke patients who visited EDs within the therapeutic period and ratio of usage of EMS ambulances.

Results

We found that four areas out of 16 administrative divisions in Korea were located beyond the lower control limit and one area was between the lower warning and lower control limits, according to a funnel plot showing the percentage of patients within the therapeutic period of stroke against the total volume of patients according to area. The ratio of stroke patients who visited EDs within the therapeutic period was significantly correlated with the ratio of use of EMS ambulances to a moderately positive degree (r=0.51; p<0.05).

Conclusions

Encouraging the public in Korea to use an EMS ambulance in case of stroke may shorten the pre-hospital delay.

Equity of utilisation of cardiovascular care and mental health services in England: a cohort-based cross-sectional study using small-area estimation

Background

A strong policy emphasis on the need to reduce both health inequalities and unmet need in deprived areas has resulted in the substantial redistribution of English NHS funding towards deprived areas. This raises the question of whether or not socioeconomically disadvantaged people continue to be disadvantaged in their access to and utilisation of health care.

Objectives

To generate estimates of the prevalence of cardiovascular disease (CVD) and common mental health disorders (CMHDs) at a variety of scales, and to make these available for public use via Public Health England (PHE). To compare these estimates with utilisation of NHS services in England to establish whether inequalities of use relative to need at various stages on the health-care pathway are associated with particular sociodemographic or other factors.

Design

Cross-sectional analysis of practice-, primary care trust- and Clinical Commissioning Group-level variations in diagnosis, prescribing and specialist management of CVD and CMHDs relative to the estimated prevalence of those conditions (calculated using small-area estimation).

Results

The utilisation of CVD care appears more equitable than the utilisation of care for CMHDs. In contrast to the reviewed literature, we found little evidence of underutilisation of services by older populations. Indeed, younger populations appear to be less likely to access care for some CVD conditions. Nor did deprivation emerge as a consistent predictor of lower use relative to need for either CVD or CMHDs. Ethnicity is a consistent predictor of variations in use relative to need. Rates of primary management are lower than expected in areas with higher percentages of black populations for diabetes, stroke and CMHDs. Areas with higher Asian populations have higher-than-expected rates of diabetes presentation and prescribing and lower-than-expected rates of secondary care for diabetes. For both sets of conditions, there are pronounced geographical variations in use relative to need. For instance, the North East has relatively high levels of use of cardiac care services and rural (shire) areas have low levels of use relative to need. For CMHDs, there appears to be a pronounced ‘London effect’, with the number of people registered by general practitioners as having depression, or being prescribed antidepressants, being much lower in London than expected. A total of 24 CVD and 41 CMHD prevalence estimates have been provided to PHE and will be publicly available at a range of scales, from lower- and middle-layer super output areas through to Clinical Commissioning Groups and local authorities.

Conclusions

We found little evidence of socioeconomic inequality in use for CVD and CMHDs relative to underlying need, which suggests that the strong targeting of NHS resources to deprived areas may well have addressed longstanding concerns about unmet need. However, ethnicity has emerged as a significant predictor of inequality, and there are large and unexplained geographical variations in use relative to need for both conditions which undermine the principle of equal access to health care for equal needs. The persistence of ethnic variations and the role of systematic factors (such as rurality) in shaping patterns of utilisation deserve further investigation, as does the fact that the models were far better at explaining variation in use of CVD than mental health services.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Comparison of hospital variation in acute myocardial infarction care and outcome between Sweden and United Kingdom: population based cohort study using nationwide clinical registries

OBJECTIVE

To assess the between hospital variation in use of guideline recommended treatments and clinical outcomes for acute myocardial infarction in Sweden and the United Kingdom.

DESIGN

Population based longitudinal cohort study using nationwide clinical registries.

SETTING AND PARTICIPANTS

Nationwide registry data comprising all hospitals providing acute myocardial infarction care in Sweden (SWEDEHEART/RIKS-HIA, n=87; 119,786 patients) and the UK (NICOR/MINAP, n=242; 391,077 patients), 2004-10.

MAIN OUTCOME MEASURES

Between hospital variation in 30 day mortality of patients admitted with acute myocardial infarction.

RESULTS

Case mix standardised 30 day mortality from acute myocardial infarction was lower in Swedish hospitals (8.4%) than in UK hospitals (9.7%), with less variation between hospitals (interquartile range 2.6% v 3.5%). In both countries, hospital level variation and 30 day mortality were inversely associated with provision of guideline recommended care. Compared with the highest quarter, hospitals in the lowest quarter for use of primary percutaneous coronary intervention had higher volume weighted 30 day mortality for ST elevation myocardial infarction (10.7% v 6.6% in Sweden; 12.7% v 5.8% in the UK). The adjusted odds ratio comparing the highest with the lowest quarters for hospitals' use of primary percutaneous coronary intervention was 0.70 (95% confidence interval 0.62 to 0.79) in Sweden and 0.68 (0.60 to 0.76) in the UK. Differences in risk between hospital quarters of treatment for non-ST elevation myocardial infarction and secondary prevention drugs for all discharged acute myocardial infarction patients were smaller than for reperfusion treatment in both countries.

CONCLUSION

Between hospital variation in 30 day mortality for acute myocardial infarction was greater in the UK than in Sweden. This was associated with, and may be partly accounted for by, the higher practice variation in acute myocardial infarction guideline recommended treatment in the UK hospitals. High quality healthcare across all hospitals, especially in the UK, with better use of guideline recommended treatment, may not only reduce unacceptable practice variation but also deliver improved clinical outcomes for patients with acute myocardial infarction. Clinical trials registration Clinical trials NCT01359033.

An evaluation of composite indicators of hospital acute myocardial infarction care: a study of 136,392 patients from the Myocardial Ischaemia National Audit Project

BACKGROUND

Hospital acute myocardial infarction (AMI) care is increasingly evaluated using composite quality scores. We investigated the influence of three aggregation methods for an AMI indicator on mortality and hospital rank.

METHODS AND RESULTS

We studied 136,392 patients discharged alive from 199 hospitals with AMI recorded in the Myocardial Ischaemia National Audit Project, between 01/01/2008 and 31/12/2009. A composite of prescription of aspirin, thienopyridine inhibitor, β-blocker, angiotensin converting enzyme inhibitor, HMG CoA reductase enzyme inhibitor and enrolment in cardiac rehabilitation at discharge was aggregated as opportunity based (OBCS), weighted opportunity-based (WOBCS) and all-or-nothing (ANCS) scores. We quantified adjusted 30-day, 6-month and 1-year mortality rates and hospital performance rank. Median (IQR) scores were OBCS: 95.0% (3.5), WOBCS: 94.7% (0.8) and ANCS: 80.9% (11.8). The three methods affected the proportion of hospitals outside 99.8% credible limits of the national median (OBCS: 52.2%, WOBCS: 64.3% and ANCS: 37.7%) and hospital rank. Each 1% increase in composite score was significantly associated with a 1 to 3% and a 4% reduction in 6-month and 1-year mortality, respectively. However, the ANCS had fewer cases and no significant association with 30-day mortality.

CONCLUSIONS

A hospital composite score, incorporating 6 aspects of AMI care, was significantly inversely associated with mortality. However, composite aggregation method influenced hospital rank, number of cases available for analysis and size of the association with all-cause mortality, with the ANCS performing least well. The use and choice of composite scores in hospital AMI quality improvement requires careful evaluation.

An assessment of composite measures of hospital performance and associated mortality for patients with acute myocardial infarction. Analysis of individual hospital performance and outcome for the National Institute for Cardiovascular Outcomes Research (NICOR)

AIM

To investigate whether a hospital-specific opportunity-based composite score (OBCS) was associated with mortality in 136,392 patients with acute myocardial infarction (AMI) using data from the Myocardial Ischaemia National Audit Project (MINAP) 2008-2009.

METHODS AND RESULTS

For 199 hospitals a multidimensional hospital OBCS was calculated on the number of times that aspirin, thienopyridine, angiotensin-converting enzyme inhibitor (ACEi), statin, β-blocker, and referral for cardiac rehabilitation was given to individual patients, divided by the overall number of opportunities that hospitals had to give that care. OBCS and its six components were compared using funnel plots. Associations between OBCS performance and 30-day and 6-month all-cause mortality were quantified using mixed-effects regression analysis. Median hospital OBCS was 95.3% (range 75.8-100%). By OBCS, 24.1% of hospitals were below funnel plot 99.8% CI, compared to aspirin (11.1%), thienopyridine (15.1%), β-blockers (14.7%), ACEi (19.1%), statins (12.1%), and cardiac rehabilitation (17.6%) on discharge. Mortality (95% CI) decreased with increasing hospital OBCS quartile at 30 days [Q1, 2.25% (2.07-2.43%) vs. Q4, 1.40% (1.25-1.56%)] and 6 months [Q1, 7.93% (7.61-8.25%) vs. Q4, 5.53% (5.22-5.83%)]. Hospital OBCS quartile was inversely associated with adjusted 30-day and 6-month mortality [OR (95% CI), 0.87 (0.80-0.94) and 0.92 (0.88-0.96), respectively] and persisted after adjustment for coronary artery catheterization [0.89 (0.82-0.96) and 0.95 (0.91-0.98), respectively].

CONCLUSIONS

Multidimensional hospital OBCS in AMI survivors are high, discriminate hospital performance more readily than single performance indicators, and significantly inversely predict early and longer-term mortality.

Statistical profiling of hospital performance using acute coronary syndrome mortality

BACKGROUND

In order to improve the quality of care delivered to patients and to enable patient choice, public reports comparing hospital performances are routinely published. Robust systems of hospital 'report cards' on performance monitoring and evaluation are therefore crucial in medical decision-making processes. In particular, such systems should effectively account for and minimise systematic differences with regard to definitions and data quality, care and treatment quality, and 'case mix'.

METHODS

Four methods for assessing hospital performance on mortality outcome measures were considered. The methods included combinations of Bayesian fixed- and random-effects models, and risk-adjusted mortality rate, and rank-based profiling techniques. The methods were empirically compared using 30-day mortality in patients admitted with acute coronary syndrome. Agreement was firstly assessed using median estimates between risk-adjusted mortality rates for a hospital and between ranks associated with a hospital's risk-adjusted mortality rates. Secondly, assessment of agreement was based on a classification of hospitals into low, normal or high performing using risk-adjusted mortality rates and ranks.

RESULTS

There was poor agreement between the point estimates of risk-adjusted mortality rates, but better agreement between ranks. However, for categorised performance, the observed agreement between the methods' classification of the hospital performance ranged from 90 to 98%. In only two of the six possible pair-wise comparisons was agreement reasonable, as reflected by a Kappa statistic; it was 0.71 between the methods of identifying outliers with the fixed-effect model and 0.77 with the hierarchical model. In the remaining four pair-wise comparisons, the agreement was, at best, moderate.

CONCLUSIONS

Even though the inconsistencies among the studied methods raise questions about which hospitals performed better or worse than others, it seems that the choice of the definition of outlying performance is less critical than that of the statistical approach. Therefore there is a need to find robust systems of 'regulation' or 'performance monitoring' that are meaningful to health service practitioners and providers.

Exploring variations in childhood stunting in Nigeria using league table, control chart and spatial analysis

BACKGROUND

Stunting, linear growth retardation is the best measure of child health inequalities as it captures multiple dimensions of children's health, development and environment where they live. The developmental priorities and socially acceptable health norms and practices in various regions and states within Nigeria remains disaggregated and with this, comes the challenge of being able to ascertain which of the regions and states identifies with either high or low childhood stunting to further investigate the risk factors and make recommendations for action oriented policy decisions.

METHODS

We used data from the birth histories included in the 2008 Nigeria Demographic and Health Survey (DHS) to estimate childhood stunting. Stunting was defined as height for age below minus two standard deviations from the median height for age of the standard World Health Organization reference population. We plotted control charts of the proportion of childhood stunting for the 37 states (including federal capital, Abuja) in Nigeria. The Local Indicators of Spatial Association (LISA) were used as a measure of the overall clustering and is assessed by a test of a null hypothesis.

RESULTS

Childhood stunting is high in Nigeria with an average of about 39%. The percentage of children with stunting ranged from 11.5% in Anambra state to as high as 60% in Kebbi State. Ranking of states with respect to childhood stunting is as follows: Anambra and Lagos states had the least numbers with 11.5% and 16.8% respectively while Yobe, Zamfara, Katsina, Plateau and Kebbi had the highest (with more than 50% of their under-fives having stunted growth).

CONCLUSIONS

Childhood stunting is high in Nigeria and varied significantly across the states. The northern states have a higher proportion than the southern states. There is an urgent need for studies to explore factors that may be responsible for these special cause variations in childhood stunting in Nigeria.

30-day in-hospital mortality after acute myocardial infarction in Tuscany (Italy): an observational study using hospital discharge data

Background

Coronary heart disease is the leading cause of mortality in the world. One of the outcome indicators recently used to measure hospital performance is 30-day mortality after acute myocardial infarction (AMI). This indicator has proven to be a valid and reproducible indicator of the appropriateness and effectiveness of the diagnostic and therapeutic process for AMI patients after hospital admission. The aim of this study was to examine the determinants of inter-hospital variability on 30-day in-hospital mortality after AMI in Tuscany. This indicator is a proxy of 30-day mortality that includes only deaths occurred during the index or subsequent hospitalizations.

Methods

The study population was identified from hospital discharge records (HDRs) and included all patients with primary or secondary ICD-9-CM codes of AMI (ICD-9 codes 410.xx) that were discharged between January 1, 2009 and November 30, 2009 from any hospital in Tuscany. The outcome of interest was 30-day all-cause in-hospital mortality, defined as a death occurring for any reason in the hospital within 30 days of the admission date. Because of the hierarchical structure of the data, with patients clustered into hospitals, random-effects (multilevel) logistic regression models were used. The models included patient risk factors and random intercepts for each hospital.

Results

The study included 5,832 patients, 61.90% male, with a mean age of 72.38 years. During the study period, 7.99% of patients died within 30 days of admission. The 30-day in-hospital mortality rate was significantly higher among patients with ST segment elevation myocardial infarction (STEMI) compared with those with non-ST segment elevation myocardial infarction (NSTEMI). The multilevel analysis which included only the hospital variance showed a significant inter-hospital variation in 30-day in-hospital mortality. When patient characteristics were added to the model, the hospital variance decreased. The multilevel analysis was then carried out separately in the two strata of patients with STEMI and NSTEMI. In the STEMI group, after adjusting for patient characteristics, some residual inter-hospital variation was found, and was related to the presence of a cardiac catheterisation laboratory.

Conclusion

We have shown that it is possible to use routinely collected administrative data to predict mortality risk and to highlight inter-hospital differences. The distinction between STEMI and NSTEMI proved to be useful to detect organisational characteristics, which affected only the STEMI subgroup.

Benchmarking inappropriate empirical antibiotic treatment

Inappropriate empirical antibiotic treatment for severe infections is associated with increased mortality. Superfluous treatment is associated with resistance induction. We aimed to define acceptable rates of inappropriate empirical antibiotic treatment. We included all prospective cohort studies published between 1975 and 2009 reporting the proportion of appropriate and inappropriate empirical antibiotic treatment of microbiologically documented infections. Studies were identified in PubMed and in reference lists of included studies. Funnel plots were drawn using the proportion of inappropriate empirical treatment as the effect size. A pooled estimate of inappropriate empirical antibiotic treatment was calculated using a β-binomial model. Control limits were calculated with the overdispersion factor technique and 20% winsorized data. Heterogeneity was assessed through subgroup analysis for categorical moderators and meta-regression for continuous variables. Eighty-seven studies, comprising 92 study groups, with 27 628 patients met inclusion criteria. The pooled rate of inappropriate empirical antibiotic treatment was 28.6% (95% CI 25.4-31.8). Funnel plot analysis yielded a dispersed graph with only 37 (40%) studies falling within the control limits. Using the overdispersion factor technique with 20% winsorizing, 79 (86%) studies fell within the control limits. None of the clinical or methodological factors could explain the large heterogeneity observed. The funnel plot presented can be used to benchmark rates of inappropriate empirical antibiotic treatment. Based on the control limits found, at least 500 patients should be evaluated before establishing a local rate. Lower and higher than expected rates might indicate overly aggressive treatment or poor performance, respectively.

Exploring variations in under-5 mortality in Nigeria using league table, control chart and spatial analysis

BACKGROUND

Under-5 mortality rate (U5MR) is the probability of a child born in a specific year or period dying before reaching the age of 5. Nigeria has a high rate of U5MR and accounts for a significant proportion of the high U5MR in sub-Sahara Africa. There are differences in health and social practice in the different states in Nigeria, coupled with the differences in developmental priorities of each State government. There is therefore the need to identify the states with high and low U5MR, to further explore the risk factors and make recommendations for planning. This study investigates variation in U5MR in Nigeria using Shewhart's theory of variation and control charts.

METHODS

We used data from the birth histories included in the 2008 Nigerian Demographic and Health Survey to estimate U5MR using a synthetic cohort life table. We plotted control charts of the proportion of under-5 mortality for the 37 states (included federal capital Abuja) in Nigeria. The Local Indicators of Spatial Association was used as a measure of the overall clustering and is assessed by a test of a null hypothesis.

RESULTS

On average, more than 1 in every 10 children born in Nigeria (159 per 1000 live births) does not survive to their fifth birthday. Kwara and Osun states had the lowest U5MR (less than 60 per 1000 live births), while Jigawa, Kano, Sokoto, Niger and Adamawa states had the highest U5MR (more than 200 per 1000 live births). There is a wide variation in the U5MR between the 37 states. The funnel plot identifies 27 (73%) states within the 99% control limits indicating common-cause variation. Four states were above the upper control limit (higher than the average) and six states were below the lower control limit (lower than the average), indicating special-cause variation.

CONCLUSIONS

U5MR is high in Nigeria; the rates are in three digits. There is a wide variation in the U5MR in Nigeria, with 27% showing evidence of special-cause variation which merits further investigation to identify possible causes. However, the vast majority of states (73%) are consistent with common-cause variation.

Funnel plots and risk-adjusted count data adverse events. A limitation of indirect standardisation

Reporting of hospital adverse event data is becoming increasingly mandated and this has motivated work on methods for the analysis and display of these data for groups of institutions. Currently, the method preferred by many workers is the funnel plot. Often, indirect standardisation is employed to produce these plots. It appears that, when used to display binary data such as surgical site infection or mortality data, the method is satisfactory. Increasingly, these data are risk-adjusted. However, risk adjustment of these data usually involves individual patients undergoing the same or similar procedures and the method does not appear to mislead. However, when dealing with count data such as bacteraemias it appears that this method can mislead, particularly where methods for risk adjustment of these data are used. Information about the hospitals or units of interest rather than individual patients is employed. For example, one hospital may have plastic and cardiac surgery units in which bacteraemias occur infrequently whereas another may provide treatment for renal failure (including transplantation) and have a large haematology-oncology unit (also including transplantation), each of which would expect higher bacteraemia rates. Moreover, the hospitals and units within them may differ substantially in size. It is well known that indirect standardisation can give biased results when denominators differ substantially. We illustrate this difficulty with risk-adjusted bacteraemia data from the Queensland Health Centre for Healthcare Infection, Surveillance and Prevention (CHRISP) database.

Cumulative funnel plots for the early detection of interoperator variation: retrospective database analysis of observed versus predicted results of percutaneous coronary intervention

OBJECTIVE

To use funnel plots and cumulative funnel plots to compare in-hospital outcome data for operators undertaking percutaneous coronary interventions with predicted results derived from a validated risk score to allow for early detection of variation in performance.

DESIGN

Analysis of prospectively collected data.

SETTING

Tertiary centre NHS hospital in the north east of England.

PARTICIPANTS

Five cardiologists carrying out percutaneous coronary interventions between January 2003 and December 2006.

MAIN OUTCOME MEASURES

In-hospital major adverse cardiovascular and cerebrovascular events (in-hospital death, Q wave myocardial infarction, emergency coronary artery bypass graft surgery, and cerebrovascular accident) analysed against the logistic north west quality improvement programme predicted risk, for each operator. Results are displayed as funnel plots summarising overall performance for each operator and cumulative funnel plots for an individual operator's performance on a case series basis.

RESULTS

The funnel plots for 5198 patients undergoing percutaneous coronary interventions showed an average observed rate for major adverse cardiovascular and cerebrovascular events of 1.96% overall. This was below the predicted risk of 2.06% by the logistic north west quality improvement programme risk score. Rates of in-hospital major adverse cardiovascular and cerebrovascular events for all operators were within the 3sigma upper control limit of 2.75% and 2sigma upper warning limit of 2.49%.

CONCLUSION

The overall in-hospital major adverse cardiovascular and cerebrovascular events rates were under the predicted event rate. In-hospital rates after percutaneous coronary intervention procedure can be monitored successfully using funnel and cumulative funnel plots with 3sigma control limits to display and publish each operator's outcomes. The upper warning limit (2sigma control limit) could be used for internal monitoring. The main advantage of these charts is their transparency, as they show observed and predicted events separately. By this approach individual operators can monitor their own performance, using the predicted risk for their patients but in a way that is compatible with benchmarking to colleagues, encapsulated by the funnel plot. This methodology is applicable regardless of variations in individual operator case volume and case mix.