A primer on using shrinkage to compare in-hospital mortality between centers

Hospital Quality Mediates Impact of Care Fragmentation Following Elective Colectomy

BACKGROUND

Readmission at a non-index hospital, or care fragmentation (CF), has been previously linked to greater morbidity and resource utilization. However, a contemporary evaluation of the impact of CF on readmission outcomes following elective colectomy is lacking. We additionally sought to evaluate the role of hospital quality in mediating the effect of CF.

METHODS

All records for adults undergoing elective colectomy were tabulated from the 2016 to 2020 Nationwide Readmissions Database. Patients readmitted non-electively within 30 days to a non-index center comprised the CF cohort (others: Non-CF). Hierarchical mixed-effects models were constructed to ascertain risk-adjusted rates of major adverse events (MAEs, a composite of in-hospital mortality and any complication) attributable to center-level effects. Hospitals with risk-adjusted MAE rates ≥50th percentile were considered Low-Quality Hospitals (LQHs) (others: High-Quality Hospitals [HQHs]).

RESULTS

Of 68,185 patients readmitted non-electively within 30 days, 8968 (13.2%) were categorized as CF. On average, CF was older, of greater comorbidity burden, and more often underwent colectomy for cancer, relative to Non-CF. Following risk adjustment, CF remained independently associated with greater likelihood of MAE (adjusted odds ratio [AOR] 1.16, 95% Confidence Interval [CI] 1.05-1.27) and per-patient expenditures (β+$2,280, CI +$1080-3490). Further, readmission to non-index LQH was linked with significantly increased odds of MAE, following initial care at HQH (AOR 1.43, CI 1.03-1.99) and LQH (AOR 1.72, CI 1.30-2.28; Reference: Non-CF).

CONCLUSIONS

Care fragmentation was associated with greater morbidity and resource utilization at readmission following elective colectomy. Further, rehospitalization at non-index LQH conferred significantly inferior outcomes. Novel efforts are needed to improve continuity of care.

Neonatologist staffing is related to the inter-hospital variation of risk-adjusted mortality of very low birth weight infants in Korea

This study investigated whether hospital factors, including patient volume, unit level, and neonatologist staffing, were associated with variations in standardized mortality ratios (SMR) adjusted for patient factors in very-low-birth-weight infants (VLBWIs). A total of 15,766 VLBWIs born in 63 hospitals between 2013 and 2020 were analyzed using data from the Korean Neonatal Network cohort. SMRs were evaluated after adjusting for patient factors. High and low SMR groups were defined as hospitals outside the 95% confidence limits on the SMR funnel plot. The mortality rate of VLBWIs was 12.7%. The average case-mix SMR was 1.1; calculated by adjusting for six significant patient factors: antenatal steroid, gestational age, birth weight, sex, 5-min Apgar score, and congenital anomalies. Hospital factors of the low SMR group (N = 10) had higher unit levels, more annual volumes of VLBWIs, more number of neonatologists, and fewer neonatal intensive care beds per neonatologist than the high SMR group (N = 13). Multi-level risk adjustment revealed that only the number of neonatologists showed a significant fixed-effect on mortality besides fixed patient risk effect and a random hospital effect. Adjusting for the number of neonatologists decreased the variance partition coefficient and random-effects variance between hospitals by 11.36%. The number of neonatologists was independently associated with center-to-center differences in VLBWI mortality in Korea after adjustment for patient risks and hospital factors.

Assessing the quality of public services: For-profits, chains, and concentration in the hospital market

We examine variation in US hospital quality across ownership, chain membership, and market concentration. We propose a new measure of quality derived from penalties imposed on hospitals under the flagship Hospital Readmissions Reduction Program, and use regression models to risk-adjust for hospital characteristics and county demographics. While the overall association between for-profit ownership and quality is negative, there is evidence of substantial heterogeneity. The quality of for-profit relative to non-profit hospitals declines with increasing market concentration. Moreover, the quality gap is primarily driven by for-profit chains. While the competition result mirrors earlier findings in the literature, the chain result appears to be new: it suggests that any potential quality gains afforded by chains are mostly realized by not-for-profit hospitals.

Association between surgical quality and long-term survival in lung cancer

OBJECTIVES

There are significant variations in both perioperative and long-term outcomes after lung cancer resection. While perioperative outcomes are often used as comparative measures of quality, they are unreliable, and their association with long-term outcomes remain unclear. In this context, we evaluated whether historical perioperative mortality after lung cancer resection is associated with 5-year survival.

PATIENTS AND METHODS

The National Cancer Database (NCDB) was queried to identify patients diagnosed with non-small cell lung cancer (NSCLC) in 2010-2016 who underwent surgical resection (n = 234200). Hospital-level reliability-adjusted 90-day mortality rate quartiles for 2010-2013 was used as the independent variable to analyze 5-year survival for patients diagnosed in 2014-2016 (n = 85396).

RESULTS

There were 85,396 patients in the 2014-2016 cohort across 1,086 hospitals. Overall observed 90-day mortality rate was 3.2% (SD 17.6%) with 2.6% (SD 16.0%) for the historically best performing quartile vs. 3.9% (SD 19.4%) for the worst performing quartile (p < 0.0001). Patients who underwent resection at hospitals with the best historical mortality rate had significantly better 5-year survival across all stages compared to those treated at hospitals in the worst performing quartile in multivariate Cox regression analysis (all stages - HR 1.21 [95% CI 1.15-1.26]; stage I - HR 1.19 [95% CI 1.12-1.25]; stage II - HR 1.20 [95% CI 1.09-1.32]; stage III - HR 1.36 [95% CI 1.20-1.54]) and Kaplan-Meier survival estimates (all stages - p < 0.0001, stage I - p < 0.0001; stage II - p = 0.0004; stage III - p < 0.0001).

CONCLUSION

With expanded lung cancer screening criteria and likely increase in early-stage detection, profiling performance is paramount to ensuring mortality benefits. We found that episodes surrounding surgical resection may be used to profile long-term outcomes that likely reflect quality across a broader context of care. Evaluating lung cancer care quality using perioperative outcomes may be useful in profiling provider performance and guiding value-based payment policies.

Separating the wheat from the chaff: How to measure hospital quality in routine data?

OBJECTIVE

To measure hospital quality based on routine data available in many health care systems including the United States, Germany, the United Kingdom, Scandinavia, and Switzerland.

DATA SOURCES AND STUDY SETTING

We use the Swiss Medical Statistics of Hospitals, an administrative hospital dataset of all inpatient stays in acute care hospitals in Switzerland for the years 2017-2019.

STUDY DESIGN

We study hospital quality based on quality indicators used by leading agencies in five countries (the United States, the United Kingdom, Germany, Austria, and Switzerland) for two high-volume elective procedures: inguinal hernia repair and hip replacement surgery. We assess how least absolute shrinkage and selection operator (LASSO), a supervised machine learning technique for variable selection, and Mundlak corrections that account for unobserved heterogeneity between hospitals can be used to improve risk adjustment and correct for imbalances in patient risks across hospitals.

DATA COLLECTION/EXTRACTION METHODS

The Swiss Federal Statistical Office collects annual data on all acute care inpatient stays including basic socio-demographic patient attributes and case-level diagnosis and procedure codes.

PRINCIPAL FINDINGS

We find that LASSO-selected and Mundlak-corrected hospital random effects logit models outperform common practice logistic regression models used for risk adjustment. Besides the more favorable statistical properties, they have superior in- and out-of-sample explanatory power. Moreover, we find that Mundlak-corrected logits and the more complex LASSO-selected models identify the same hospitals as high or low-quality offering public health authorities a valuable alternative to standard logistic regression models. Our analysis shows that hospitals vary considerably in the quality they provide to patients.

CONCLUSION

We find that routine hospital data can be used to measure clinically relevant quality indicators that help patients make informed hospital choices.

Evaluations of statistical methods for outlier detection when benchmarking in clinical registries: a systematic review

OBJECTIVES

Benchmarking is common in clinical registries to support the improvement of health outcomes by identifying underperforming clinician or health service providers. Despite the rise in clinical registries and interest in publicly reporting benchmarking results, appropriate methods for benchmarking and outlier detection within clinical registries are not well established, and the current application of methods is inconsistent. The aim of this review was to determine the current statistical methods of outlier detection that have been evaluated in the context of clinical registry benchmarking.

DESIGN

A systematic search for studies evaluating the performance of methods to detect outliers when benchmarking in clinical registries was conducted in five databases: EMBASE, ProQuest, Scopus, Web of Science and Google Scholar. A modified healthcare modelling evaluation tool was used to assess quality; data extracted from each study were summarised and presented in a narrative synthesis.

RESULTS

Nineteen studies evaluating a variety of statistical methods in 20 clinical registries were included. The majority of studies conducted application studies comparing outliers without statistical performance assessment (79%), while only few studies used simulations to conduct more rigorous evaluations (21%). A common comparison was between random effects and fixed effects regression, which provided mixed results. Registry population coverage, provider case volume minimum and missing data handling were all poorly reported.

CONCLUSIONS

The optimal methods for detecting outliers when benchmarking clinical registry data remains unclear, and the use of different models may provide vastly different results. Further research is needed to address the unresolved methodological considerations and evaluate methods across a range of registry conditions.

PROSPERO REGISTRATION NUMBER

CRD42022296520.

Clinics Optimizing MEthadone Take-homes for opioid use disorder (COMET): Protocol for a stepped-wedge randomized trial to facilitate clinic level changes

INTRODUCTION

Regulatory changes made during the COVID-19 public health emergency (PHE) that relaxed criteria for take-home dosing (THD) of methadone offer an opportunity to improve quality of care with a lifesaving treatment. There is a pressing need for research to study the long-term effects of the new PHE THD rules and to test data-driven interventions to promote more effective adoption by opioid treatment programs (OTPs). We propose a two-phase project to develop and test a multidimensional intervention for OTPs that leverages information from large State administrative data.

METHODS AND ANALYSIS

We propose a two-phased project to develop then test a multidimensional OTP intervention to address clinical decision making, regulatory confusion, legal liability concerns, capacity for clinical practice change, and financial barriers to THD. The intervention will include OTP THD specific dashboards drawn from multiple State databases. The approach will be informed by the Health Equity Implementation Framework (HEIF). In phase 1, we will employ an explanatory sequential mixed methods design to combine analysis of large state administrative databases-Medicaid, treatment registry, THD reporting-with qualitative interviews to develop and refine the intervention. In phase 2, we will conduct a stepped-wedge trial over three years with 36 OTPs randomized to 6 cohorts of a six-month clinic-level intervention. The trial will test intervention effects on OTP-level implementation outcomes and patient outcomes (1) THD use; 2) retention in care; and 3) adverse healthcare events). We will specifically examine intervention effects for Black and Latinx clients. A concurrent triangulation mixed methods design will be used: quantitative and qualitative data collection will occur concurrently and results will be integrated after analysis of each. We will employ generalized linear mixed models (GLMMs) in the analysis of stepped-wedge trials. The primary outcome will be weekly or greater THD. The semi-structured interviews will be transcribed and analyzed with Dedoose to identify key facilitators, barriers, and experiences according to HEIF constructs using directed content analysis.

DISCUSSION

This multi-phase, embedded mixed methods project addresses a critical need to support long-term practice changes in methadone treatment for opioid use disorder following systemic changes emerging from the PHE-particularly for Black and Latinx individuals with opioid use disorder. By combining findings from analyses of large administrative data with lessons gleaned from qualitative interviews of OTPs that were flexible with THD and those that were not, we will build and test the intervention to coach clinics to increase flexibility with THD. The findings will inform policy at the local and national level.

Outlier identification and monitoring of institutional or clinician performance: an overview of statistical methods and application to national audit data

BACKGROUND

Institutions or clinicians (units) are often compared according to a performance indicator such as in-hospital mortality. Several approaches have been proposed for the detection of outlying units, whose performance deviates from the overall performance.

METHODS

We provide an overview of three approaches commonly used to monitor institutional performances for outlier detection. These are the common-mean model, the 'Normal-Poisson' random effects model and the 'Logistic' random effects model. For the latter we also propose a visualisation technique. The common-mean model assumes that the underlying true performance of all units is equal and that any observed variation between units is due to chance. Even after applying case-mix adjustment, this assumption is often violated due to overdispersion and a post-hoc correction may need to be applied. The random effects models relax this assumption and explicitly allow the true performance to differ between units, thus offering a more flexible approach. We discuss the strengths and weaknesses of each approach and illustrate their application using audit data from England and Wales on Adult Cardiac Surgery (ACS) and Percutaneous Coronary Intervention (PCI).

RESULTS

In general, the overdispersion-corrected common-mean model and the random effects approaches produced similar p-values for the detection of outliers. For the ACS dataset (41 hospitals) three outliers were identified in total but only one was identified by all methods above. For the PCI dataset (88 hospitals), seven outliers were identified in total but only two were identified by all methods. The common-mean model uncorrected for overdispersion produced several more outliers. The reason for observing similar p-values for all three approaches could be attributed to the fact that the between-hospital variance was relatively small in both datasets, resulting only in a mild violation of the common-mean assumption; in this situation, the overdispersion correction worked well.

CONCLUSION

If the common-mean assumption is likely to hold, all three methods are appropriate to use for outlier detection and their results should be similar. Random effect methods may be the preferred approach when the common-mean assumption is likely to be violated.

Development and validation of risk-adjusted quality indicators for the long-term outcome of acute sepsis care in German hospitals based on health claims data

Background

Methods for assessing long-term outcome quality of acute care for sepsis are lacking. We investigated a method for measuring long-term outcome quality based on health claims data in Germany.

Materials and methods

Analyses were based on data of the largest German health insurer, covering 32% of the population. Cases (aged 15 years and older) with ICD-10-codes for severe sepsis or septic shock according to sepsis-1-definitions hospitalized in 2014 were included. Short-term outcome was assessed by 90-day mortality; long-term outcome was assessed by a composite endpoint defined by 1-year mortality or increased dependency on chronic care. Risk factors were identified by logistic regressions with backward selection. Hierarchical generalized linear models were used to correct for clustering of cases in hospitals. Predictive validity of the models was assessed by internal validation using bootstrap-sampling. Risk-standardized mortality rates (RSMR) were calculated with and without reliability adjustment and their univariate and bivariate distributions were described.

Results

Among 35,552 included patients, 53.2% died within 90 days after admission; 39.8% of 90-day survivors died within the first year or had an increased dependency on chronic care. Both risk-models showed a sufficient predictive validity regarding discrimination [AUC = 0.748 (95% CI: 0.742; 0.752) for 90-day mortality; AUC = 0.675 (95% CI: 0.665; 0.685) for the 1-year composite outcome, respectively], calibration (Brier Score of 0.203 and 0.220; calibration slope of 1.094 and 0.978), and explained variance (R ² = 0.242 and R ² = 0.111). Because of a small case-volume per hospital, applying reliability adjustment to the RSMR led to a great decrease in variability across hospitals [from median (1st quartile, 3rd quartile) 54.2% (44.3%, 65.5%) to 53.2% (50.7%, 55.9%) for 90-day mortality; from 39.2% (27.8%, 51.1%) to 39.9% (39.5%, 40.4%) for the 1-year composite endpoint]. There was no substantial correlation between the two endpoints at hospital level (observed rates: ρ = 0, p = 0.99; RSMR: ρ = 0.017, p = 0.56; reliability-adjusted RSMR: ρ = 0.067; p = 0.026).

Conclusion

Quality assurance and epidemiological surveillance of sepsis care should include indicators of long-term mortality and morbidity. Claims-based risk-adjustment models for quality indicators of acute sepsis care showed satisfactory predictive validity. To increase reliability of measurement, data sources should cover the full population and hospitals need to improve ICD-10-coding of sepsis.

Hospital variation in admissions for low back pain following an emergency department presentation: a retrospective study

Background

One in 6 patients with low back pain (LBP) presenting to emergency departments (EDs) are subsequently admitted to hospital each year, making LBP the ninth most common reason for hospital admission in Australia. No studies have investigated and quantified the extent of clinical variation in hospital admission following an ED presentation for LBP.

Methods

We used routinely collected ED data from public hospitals within the state of New South Wales, Australia, to identify presentations of patients aged between 18 and 111 with a discharge diagnosis of LBP. We fitted a series of random effects multilevel logistic regression models adjusted by case-mix and hospital variables. The main outcome was the hospital-adjusted admission rate (HAAR). Data were presented as funnel plots with 95% and 99.8% confidence limits. Hospitals with a HAAR outside the 95% confidence limit were considered to have a HAAR significantly different to the state average.

Results

We identified 176,729 LBP presentations across 177 public hospital EDs and 44,549 hospital admissions (25.2%). The mean (SD) age was 51.8 (19.5) and 52% were female. Hospital factors explained 10% of the variation (ICC = 0.10), and the median odds ratio (MOR) was 2.03. We identified marked variation across hospitals, with HAAR ranging from 6.9 to 65.9%. After adjusting for hospital variables, there was still marked variation between hospitals with similar characteristics.

Conclusion

We found substantial variation in hospital admissions following a presentation to the ED due to LBP even after controlling by case-mix and hospital characteristics. Given the substantial costs associated with these admissions, our findings indicate the need to investigate sources of variation and to determine instances where the observed variation is warranted or unwarranted.

Supplementary information

The online version contains supplementary material available at 10.1186/s12913-022-08134-8.

The Independent Effects of Procurement Biopsy Findings on Ten-Year Outcomes of Extended Criteria Donor Kidney Transplants

Introduction

Methods

Results

Conclusion

Disease activity and its predictors in early inflammatory arthritis: findings from a national cohort

OBJECTIVES

We set out to characterize patient factors that predict disease activity during the first year of treatment for early inflammatory arthritis (EIA).

METHODS

We used an observational cohort study design, extracting data from a national clinical audit. All NHS organizations providing secondary rheumatology care in England and Wales were eligible to take part, with recruitment from 215/218 (99%) clinical commissioning groups (CCGs)/Health Boards. Participants were >16 years old and newly diagnosed with RA pattern EIA between May 2018 and May 2019. Demographic details collected at baseline included age, gender, ethnicity, work status and postcode, which was converted to an area level measure of socioeconomic position (SEP). Disease activity scores (DAS28) were collected at baseline, three and 12 months follow-up.

RESULTS

A total of 7455 participants were included in analyses. Significant levels of CCG/Health board variation could not be robustly identified from mixed effects modelling. Gender and SEP were predictors of low disease activity at baseline, three and 12 months follow-up. Mapping of margins identified a gradient for SEP, whereby those with higher degrees of deprivation had higher disease activity. Black, Asian and Minority Ethnic patients had lower odds of remission at three months follow-up.

CONCLUSION

Patient factors (gender, SEP, ethnicity) predict disease activity. The rheumatology community should galvanise to improve access to services for all members of society. More data are required to characterize area level variation in disease activity.

Assessment of Hospital Characteristics and Interhospital Transfer Patterns of Adults With Emergency General Surgery Conditions

Importance

Although patients with emergency general surgery (EGS) conditions frequently undergo interhospital transfers, the transfer patterns and associated factors are not well understood.

Objective

To examine whether patients with EGS conditions are consistently directed to hospitals with more resources and better outcomes.

Design, Setting, and Participants

This cohort study performed a network analysis of interhospital transfers among adults with EGS conditions from January 1 to December 31, 2016. The analysis used all-payer claims data from the 2016 Healthcare Cost and Utilization Project state inpatient and emergency department databases in 8 states. A total of 728 hospitals involving 85 415 transfers of 80 307 patients were included. Patients were eligible for inclusion if they were 18 years or older and had an acute care hospital encounter with a diagnosis of an EGS condition as defined by the American Association for the Surgery of Trauma. Data were analyzed from January 1, 2020, to June 17, 2021.

Exposures

Hospital-level measures of size (total bed capacity), resources (intensive care unit [ICU] bed capacity, teaching status, trauma center designation, and presence of trauma and/or surgical critical care fellowships), EGS volume (annual EGS encounters), and EGS outcomes (risk-adjusted failure to rescue and in-hospital mortality).

Main Outcomes and Measures

The main outcome was hospital-level centrality ratio, defined as the normalized number of incoming transfers divided by the number of outgoing transfers. A higher centrality ratio indicated more incoming transfers per outgoing transfer. Multivariable regression analysis was used to test the hypothesis that a higher hospital centrality ratio would be associated with more resources, higher volume, and better outcomes.

Results

Among 80 307 total patients, the median age was 63 years (interquartile range [IQR], 50-75 years); 52.1% of patients were male and 78.8% were White. The median number of outgoing and incoming transfers per hospital were 106 (IQR, 61-157) and 36 (IQR, 8-137), respectively. A higher log-transformed centrality ratio was associated with more resources, such as higher ICU capacity (eg, >25 beds vs 0-10 beds: β = 1.67 [95% CI, 1.16-2.17]; P < .001), and higher EGS volume (eg, quartile 4 [highest] vs quartile 1 [lowest]: β = 0.78 [95% CI, 0-1.57]; P = .01). However, a higher log-transformed centrality ratio was not associated with better outcomes, such as lower in-hospital mortality (eg, quartile 4 [highest] vs quartile 1 [lowest]: β = 0.30 [95% CI, -0.09 to 0.68]; P = .83) and lower failure to rescue (eg, quartile 4 [highest] vs quartile 1 [lowest]: β = -0.50 [95% CI, -1.13 to 0.12]; P = .27).

Conclusions and Relevance

In this study, EGS transfers were directed to high-volume hospitals with more resources but were not necessarily directed to hospitals with better clinical outcomes. Optimizing transfer destination in the interhospital transfer network has the potential to improve EGS outcomes.

The association between nurse staffing and inpatient mortality: A shift-level retrospective longitudinal study

BACKGROUND

Worldwide, hospitals face pressure to reduce costs. Some respond by working with a reduced number of nurses or less qualified nursing staff.

OBJECTIVE

This study aims at examining the relationship between mortality and patient exposure to shifts with low or high nurse staffing.

METHODS

This longitudinal study used routine shift-, unit-, and patient-level data for three years (2015-2017) from one Swiss university hospital. Data from 55 units, 79,893 adult inpatients and 3646 nurses (2670 registered nurses, 438 licensed practical nurses, and 538 unlicensed and administrative personnel) were analyzed. After developing a staffing model to identify high- and low-staffed shifts, we fitted logistic regression models to explore associations between nurse staffing and mortality.

RESULTS

Exposure to shifts with high levels of registered nurses had lower odds of mortality by 8.7% [odds ratio 0.91 95% CI 0.89-0.93]. Conversely, low staffing was associated with higher odds of mortality by 10% [odds ratio 1.10 95% CI 1.07-1.13]. The associations between mortality and staffing by other groups was less clear. For example, both high and low staffing of unlicensed and administrative personnel were associated with higher mortality, respectively 1.03 [95% CI 1.01-1.04] and 1.04 [95% CI 1.03-1.06].

DISCUSSION AND IMPLICATIONS

This patient-level longitudinal study suggests a relationship between registered nurses staffing levels and mortality. Higher levels of registered nurses positively impact patient outcome (i.e. lower odds of mortality) and lower levels negatively (i.e. higher odds of mortality). Contributions of the three other groups to patient safety is unclear from these results. Therefore, substitution of either group for registered nurses is not recommended.

Care Fragmentation and Mortality in Readmission after Surgery for Hepatopancreatobiliary and Gastric Cancer: A Patient-Level and Hospital-Level Analysis of the Healthcare Cost and Utilization Project Administrative Database

BACKGROUND

Hepatopancreatobiliary (HPB) and gastric oncologic operations are frequently performed at referral centers. Postoperatively, many patients experience care fragmentation, including readmission to "outside hospitals" (OSH), which is associated with increased mortality. Little is known about patient-level and hospital-level variables associated with this mortality difference.

STUDY DESIGN

Patients undergoing HPB or gastric oncologic surgery were identified from select states within the Healthcare Cost and Utilization Project database (2006-2014). Follow-up was 90 days after discharge. Analyses used Kruskal-Wallis test, Youden index, and multilevel modeling at the hospital level.

RESULTS

There were 7,536 patients readmitted within 90 days of HPB or gastric oncologic surgery to 636 hospitals; 28% of readmissions (n = 2,123) were to an OSH, where 90-day readmission mortality was significantly higher: 8.0% vs 5.4% (p < 0.01). Patients readmitted to an OSH lived farther from the index surgical hospital (median 24 miles vs 10 miles; p < 0.01) and were readmitted later (median 25 days after discharge vs 12; p < 0.01). These variables were not associated with readmission mortality. Surgical complications managed at an OSH were associated with greater readmission mortality: 8.4% vs 5.7% (p < 0.01). Hospitals with <100 annual HPB and gastric operations for benign or malignant indications had higher readmission mortality (6.4% vs 4.7%, p = 0.01), although this was not significant after risk-adjustment (p = 0.226).

CONCLUSIONS

For readmissions after HPB and gastric oncologic surgery, travel distance and timing are major determinants of care fragmentation. However, these variables are not associated with mortality, nor is annual hospital surgical volume after risk-adjustment. This information could be used to determine safe sites of care for readmissions after HPB and gastric surgery. Further analysis is needed to explore the relationship between complications, the site of care, and readmission mortality.

Assessment of Overuse of Medical Tests and Treatments at US Hospitals Using Medicare Claims

Importance

Overuse of health care services exposes patients to unnecessary risk of harm and costs. Distinguishing patterns of overuse among hospitals requires hospital-level measures across multiple services.

Objective

To describe characteristics of hospitals associated with overuse of health care services in the US.

Design, Setting, and Participants

This retrospective cross-sectional analysis used Medicare fee-for-service claims data for beneficiaries older than 65 years from January 1, 2015, to December 31, 2017, with a lookback of 1 year. Inpatient and outpatient services were included, and services offered at specialty and federal hospitals were excluded. Patients were from hospitals with the capacity (based on a claims filter developed for this study) to perform at least 7 of 12 investigated services. Statistical analyses were performed from July 1, 2020, to December 20, 2020.

Main Outcomes and Measures

Outcomes of interest were a composite overuse score ranging from 0 (no overuse of services) to 1 (relatively high overuse of services) and characteristics of hospitals clustered by overuse rates. Twelve published low-value service algorithms were applied to the data to find overuse rates for each hospital, normalized and aggregated to a composite score and then compared across 6 hospital characteristics using multivariable regression. A k-means cluster analysis was used on normalized overuse rates to identify hospital clusters.

Results

The primary analysis was performed on 2415 cohort A hospitals (ie, hospitals with capacity for 7 or more services), which included 1 263 592 patients (mean [SD] age, 72.4 [14] years; 678 549 women [53.7%]; 101 017 191 White patients [80.5%]). Head imaging for syncope was the highest-volume low-value service (377 745 patients [29.9%]), followed by coronary artery stenting for stable coronary disease (199 579 [15.8%]). The mean (SD) composite overuse score was 0.40 (0.10) points. Southern hospitals had a higher mean score than midwestern (difference in means: 0.06 [95% CI, 0.05-0.07] points; P < .001), northeast (0.08 [95% CI, 0.06-0.09] points; P < .001), and western hospitals (0.08 [95% CI, 0.07-0.10] points; P < .001). Nonprofit hospitals had a lower adjusted mean score than for-profit hospitals (-0.03 [95% CI, -0.04 to -0.02] points; P < .001). Major teaching hospitals had significantly lower adjusted mean overuse scores vs minor teaching hospitals (difference in means, -0.07 [95% CI, -0.08 to -0.06] points; P < .001) and nonteaching hospitals (-0.10 [95% CI, -0.12 to -0.09] points; P < .001). Of the 4 clusters identified, 1 was characterized by its low counts of overuse in all services except for spinal fusion; the majority of major teaching hospitals were in this cluster (164 of 223 major teaching hospitals [73.5%]).

Conclusions and Relevance

This cross-sectional study used a novel measurement of hospital-associated overuse; results showed that the highest scores in this Medicare population were associated with nonteaching and for-profit hospitals, particularly in the South.

Milk Fat Globule Membrane as a Modulator of Infant Metabolism and Gut Microbiota: A Formula Supplement Narrowing the Metabolic Differences between Breastfed and Formula-Fed Infants

SCOPE

Milk fat globule membrane (MFGM) is an important component of milk that has previously been removed in the manufacture of infant formulas, but has recently gained attention owing to its potential to improve immunological, cognitive, and metabolic health. The goal of this study is to determine whether supplementing MFGM in infant formula would drive desirable changes in metabolism and gut microbiota to elicit benefits observed in prior studies.

METHODS AND RESULTS

The serum metabolome and fecal microbiota are analyzed using ¹ H NMR spectroscopy and 16S rRNA gene sequencing respectively in a cohort of Chinese infants given a standard formula or a formula supplemented with an MFGM-enriched whey protein fraction. Supplementing MFGM suppressed protein degradation pathways and the levels of insulinogenic amino acids that are typically enhanced in formula-fed infants while facilitating fatty acid oxidation and ketogenesis, a feature that may favor brain development. MFGM supplementation did not induce significant compositional changes in the fecal microbiota but suppressed microbial diversity and altered microbiota-associated metabolites.

CONCLUSION

Supplementing MFGM in a formula reduced some metabolic gaps between formula-fed and breastfed infants.

Bayesian profiling for cost with zeros to decompose total cost into probability of cost and mean nonzero cost

Cost of health care can vary substantially across hospitals, centers, or providers. Data from electronic health records provide information for studying patterns of cost variation and identifying high or low cost centers. Cost data often include zero values when patients receive no care, and joint two-part models have been developed for clustered cost data with zeros. Standard methods for center comparisons, sometimes called profiling, can use these methods to incorporate zero values into total cost. However, zero costs also provide opportunities to further examine sources of cost variation and outliers. For example, a hospital may have high (or low) cost due to frequency of nonzero cost, amount of nonzero cost, or a combination of those. We give methods for decomposing hospital differences in total cost with zeros into components for probability of use (i.e., of nonzero cost) and for cost of use (mean of nonzero cost). The components multiply to total cost and quantify components on the same easily interpreted multiplicative scales. The methods are based on Bayesian hierarchical models and counterfactual arguments, with Markov chain Monte Carlo estimation. We used simulated data to illustrate use, interpretation, and visualization of the methods in diverse situations, and applied the methods to 30,024 patients at 57 US Veterans Administration hospitals to characterize outlier hospitals in one year cost of inpatient care following a cardiac procedure. Twenty eight percent of patients had zero cost. These methods are useful in providing insight into cost variation and outliers for planning future studies or interventions.

Data quality predicts care quality: findings from a national clinical audit

BACKGROUND

Missing clinical outcome data are a common occurrence in longitudinal studies. Data quality in clinical audit is a particular cause for concern. The relationship between departmental levels of missing clinical outcome data and care quality is not known. We hypothesise that completeness of key outcome data in a national audit predicts departmental performance.

METHODS

The National Clinical Audit for Rheumatoid and Early Inflammatory Arthritis (NCAREIA) collected data on care of patients with suspected rheumatoid arthritis (RA) from early 2014 to late 2015. This observational cohort study collected data on patient demographics, departmental variables, service quality measures including time to treatment, and the key RA clinical outcome measure, disease activity at baseline, and 3 months follow-up. A mixed effects model was conducted to identify departments with high/low proportions of missing baseline disease activity data with the results plotted on a caterpillar graph. A mixed effects model was conducted to assess if missing baseline disease activity predicted prompt treatment.

RESULTS

Six thousand two hundred five patients with complete treatment time data and a diagnosis of RA were recruited from 136 departments. 34.3% had missing disease activity at baseline. Mixed effects modelling identified 13 departments with high levels of missing disease activity, with a cluster observed in the Northwest of England. Missing baseline disease activity was associated with not commencing treatment promptly in an adjusted mix effects model, odds ratio 0.50 (95% CI 0.41 to 0.61, p < 0.0001).

CONCLUSIONS

We have shown that poor engagement in a national audit program correlates with the quality of care provided. Our findings support the use of data completeness as an additional service quality indicator.

Hospital and Surgeon Variation in Patient-reported Functional Outcomes After Lumbar Spine Fusion: A Statewide Evaluation

STUDY DESIGN

Statewide retrospective cohort study using prospectively collected data from the Spine Care and Outcomes Assessment Program, capturing ∼75% of the state's spine fusion procedures.

OBJECTIVE

The aim of this study was to estimate the variation in patient-reported outcomes (PROs) 1 year after elective lumbar fusion surgery across surgeons and hospitals; and to discuss the potential impact of guiding patient selection using a PRO prediction tool.

SUMMARY OF BACKGROUND DATA

Despite an increasing interest in incorporating PROs as part of the move toward value-based payment and to improve quality, limited evidence exists on how PROs vary across hospitals and surgeons, a key aspect of using these metrics for quality profiling.

METHODS

We examined patient-reported functional improvement (≥15-point reduction in the Oswestry Disability Index [ODI]) and minimal disability (reaching ≤22 on the ODI) 1 year after surgery in 17 hospitals and 58 surgeons between 2012 and 2017. Outcomes were risk-adjusted for patient characteristics with multiple logistic regressions and reliability-adjusted using hierarchical models.

RESULTS

Of the 737 patients who underwent lumbar fusion (mean [SD] age, 63 [12] years; 60% female; 84% had stenosis; 70% had spondylolisthesis), 58.7% achieved functional improvement and 42.5% reached minimal disability status at 1 year. After adjusting for patient factors, there was little variation between hospitals and surgeons (maximum interclass correlation was 3.5%), and this variation became statistically insignificant after further reliability adjustment. Avoiding operation on patients with <50% chance of functional improvement may reduce current surgical volume by 63%.

CONCLUSION

Variations in PROs across hospitals and surgeons were mainly driven by differences in patient populations undergoing lumbar fusion, suggesting that PROs may not be useful indicators of hospital or surgeon quality. Careful patient selection using validated prediction tools may decrease differences in outcomes across hospitals and providers and improve overall quality, but would significantly reduce surgical volumes.

LEVEL OF EVIDENCE

3.

Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons

OBJECTIVES

Administrative claims data are commonly used for sepsis surveillance, research, and quality improvement. However, variations in diagnosis, documentation, and coding practices for sepsis and organ dysfunction may confound efforts to estimate sepsis rates, compare outcomes, and perform risk adjustment. We evaluated hospital variation in the sensitivity of claims data relative to clinical data from electronic health records and its impact on outcome comparisons.

DESIGN, SETTING, AND PATIENTS

Retrospective cohort study of 4.3 million adult encounters at 193 U.S. hospitals in 2013-2014.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Sepsis was defined using electronic health record-derived clinical indicators of presumed infection (blood culture draws and antibiotic administrations) and concurrent organ dysfunction (vasopressors, mechanical ventilation, doubling in creatinine, doubling in bilirubin to ≥ 2.0 mg/dL, decrease in platelets to < 100 cells/µL, or lactate ≥ 2.0 mmol/L). We compared claims for sepsis prevalence and mortality rates between both methods. All estimates were reliability adjusted to account for random variation using hierarchical logistic regression modeling. The sensitivity of hospitals' claims data was low and variable: median 30% (range, 5-54%) for sepsis, 66% (range, 26-84%) for acute kidney injury, 39% (range, 16-60%) for thrombocytopenia, 36% (range, 29-44%) for hepatic injury, and 66% (range, 29-84%) for shock. Correlation between claims and clinical data was moderate for sepsis prevalence (Pearson coefficient, 0.64) and mortality (0.61). Among hospitals in the lowest sepsis mortality quartile by claims, 46% shifted to higher mortality quartiles using clinical data. Using implicit sepsis criteria based on infection and organ dysfunction codes also yielded major differences versus clinical data.

CONCLUSIONS

Variation in the accuracy of claims data for identifying sepsis and organ dysfunction limits their use for comparing hospitals' sepsis rates and outcomes. Using objective clinical data may facilitate more meaningful hospital comparisons.

The impact of disease severity adjustment on hospital standardised mortality ratios: Results from a service-wide analysis of ischaemic stroke admissions using linked pre-hospital, admissions and mortality data

Background

Administrative data are used to examine variation in thirty-day mortality across health services in several jurisdictions. Hospital performance measurement may be error-prone as information about disease severity is not typically available in routinely collected data to incorporate into case-mix adjusted analyses. Using ischaemic stroke as a case study, we tested the extent to which accounting for disease severity impacts on hospital performance assessment.

Methods

We linked all recorded ischaemic stroke admissions between July, 2011 and June, 2014 to death registrations and a measure of stroke severity obtained at first point of patient contact with health services, across New South Wales, Australia’s largest health service jurisdiction. Thirty-day hospital standardised mortality ratios were adjusted for either comorbidities, as is typically done, or for both comorbidities and stroke severity. The impact of stroke severity adjustment on mortality ratios was determined using 95% and 99% control limits applied to funnel plots and by calculating the change in rank order of hospital risk adjusted mortality rates.

Results

The performance of the stroke severity adjusted model was superior to incorporating comorbidity burden alone (c-statistic = 0.82 versus 0.75; N = 17,700 patients, 176 hospitals). Concordance in outlier classification was 89% and 97% when applying 95% or 99% control limits to funnel plots, respectively. The sensitivity rates of outlier detection using comorbidity adjustment compared with gold-standard severity and comorbidity adjustment was 74% and 83% with 95% and 99% control limits, respectively. Corresponding positive predictive values were 74% and 91%. Hospital rank order of risk adjusted mortality rates shifted between 0 to 22 places with severity adjustment (Median = 4.0, Inter-quartile Range = 2–7).

Conclusions

Rankings of mortality rates varied widely depending on whether stroke severity was taken into account. Funnel plots yielded largely concordant results irrespective of severity adjustment and may be sufficiently accurate as a screening tool for assessing hospital performance.

Regional variation in healthcare spending and mortality among senior high-cost healthcare users in Ontario, Canada: a retrospective matched cohort study

Background

Senior high cost health care users (HCU) are a priority for many governments. Little research has addressed regional variation of HCU incidence and outcomes, especially among incident HCU. This study describes the regional variation in healthcare costs and mortality across Ontario’s health planning districts [Local Health Integration Networks (LHIN)] among senior incident HCU and non-HCU and explores the relationship between healthcare spending and mortality.

Methods

We conducted a retrospective population-based matched cohort study of incident senior HCU defined as Ontarians aged ≥66 years in the top 5% most costly healthcare users in fiscal year (FY) 2013. We matched HCU to non-HCU (1:3) based on age, sex and LHIN. Primary outcomes were LHIN-based variation in costs (total and 12 cost components) and mortality during FY2013 as measured by variance estimates derived from multi-level models. Outcomes were risk-adjusted for age, sex, ADGs, and low-income status. In a cost-mortality analysis by LHIN, risk-adjusted random effects for total costs and mortality were graphically presented together in a cost-mortality plane to identify low and high performers.

Results

We studied 175,847 incident HCU and 527,541 matched non-HCU. On average, 94 out of 1000 seniors per LHIN were HCU (CV = 4.6%). The mean total costs for HCU in FY2013 were 12 times higher that of non-HCU ($29,779 vs. $2472 respectively), whereas all-cause mortality was 13.6 times greater (103.9 vs. 7.5 per 1000 seniors).

Regional variation in costs and mortality was lower in senior HCU compared with non-HCU. We identified greater variability in accessing the healthcare system, but, once the patient entered the system, variation in costs was low. The traditional drivers of costs and mortality that we adjusted for played little role in driving the observed variation in HCUs’ outcomes. We identified LHINs that had high mortality rates despite elevated healthcare expenditures and those that achieved lower mortality at lower costs. Some LHINs achieved low mortality at excessively high costs.

Conclusions

Risk-adjusted allocation of healthcare resources to seniors in Ontario is overall similar across health districts, more so for HCU than non-HCU. Identified important variation in the cost-mortality relationship across LHINs needs to be further explored.

Electronic supplementary material

The online version of this article (10.1186/s12877-018-0952-7) contains supplementary material, which is available to authorized users.

Extent of Risk-Aligned Surveillance for Cancer Recurrence Among Patients With Early-Stage Bladder Cancer

IMPORTANCE

Cancer care guidelines recommend aligning surveillance frequency with underlying cancer risk, ie, more frequent surveillance for patients at high vs low risk of cancer recurrence.

OBJECTIVE

To assess the extent to which such risk-aligned surveillance is practiced within US Department of Veterans Affairs facilities by classifying surveillance patterns for low- vs high-risk patients with early-stage bladder cancer.

DESIGN SETTING AND PARTICIPANTS

US national retrospective cohort study of a population-based sample of patients diagnosed with low-risk or high-risk early-stage bladder between January 1, 2005, and December 31, 2011, with follow-up through December 31, 2014. Analyses were performed March 2017 to April 2018. The study included all Veterans Affairs facilities (n = 85) where both low-and high-risk patients were treated.

EXPOSURES

Low-risk vs high-risk cancer status, based on definitions from the European Association of Urology risk stratification guidelines and on data extracted from diagnostic pathology reports via validated natural language processing algorithms.

MAIN OUTCOMES AND MEASURES

Adjusted cystoscopy frequency for low-risk and high-risk patients for each facility, estimated using multilevel modeling.

RESULTS

The study included 1278 low-risk and 2115 high-risk patients (median [interquartile range] age, 77 [71-82] years; 99% [3368 of 3393] male). Across facilities, the adjusted frequency of surveillance cystoscopy ranged from 3.7 to 6.2 (mean, 4.8) procedures over 2 years per patient for low-risk patients and from 4.6 to 6.0 (mean, 5.4) procedures over 2 years per patient for high-risk patients. In 70 of 85 facilities, surveillance was performed at a comparable frequency for low- and high-risk patients, differing by less than 1 cystoscopy over 2 years. Surveillance frequency among high-risk patients statistically significantly exceeded surveillance among low-risk patients at only 4 facilities. Across all facilities, surveillance frequencies for low- vs high-risk patients were moderately strongly correlated (r = 0.52; P < .001).

CONCLUSIONS AND RELEVANCE

Patients with early-stage bladder cancer undergo cystoscopic surveillance at comparable frequencies regardless of risk. This finding highlights the need to understand barriers to risk-aligned surveillance with the goal of making it easier for clinicians to deliver it in routine practice.

Investigating Risk Adjustment Methods for Health Care Provider Profiling When Observations are Scarce or Events Rare

Background:

When profiling health care providers, adjustment for case-mix is essential. However, conventional risk adjustment methods may perform poorly, especially when provider volumes are small or events rare. Propensity score (PS) methods, commonly used in observational studies of binary treatments, have been shown to perform well when the amount of observations and/or events are low and can be extended to a multiple provider setting. The objective of this study was to evaluate the performance of different risk adjustment methods when profiling multiple health care providers that perform highly protocolized procedures, such as coronary artery bypass grafting.

Methods:

In a simulation study, provider effects estimated using PS adjustment, PS weighting, PS matching, and multivariable logistic regression were compared in terms of bias, coverage and mean squared error (MSE) when varying the event rate, sample size, provider volumes, and number of providers. An empirical example from the field of cardiac surgery was used to demonstrate the different methods.

Results:

Overall, PS adjustment, PS weighting, and logistic regression resulted in provider effects with low amounts of bias and good coverage. The PS matching and PS weighting with trimming led to biased effects and high MSE across several scenarios. Moreover, PS matching is not practical to implement when the number of providers surpasses three.

Conclusions:

None of the PS methods clearly outperformed logistic regression, except when sample sizes were relatively small. Propensity score matching performed worse than the other PS methods considered.

Outlier classification performance of risk adjustment methods when profiling multiple providers

BACKGROUND

When profiling multiple health care providers, adjustment for case-mix is essential to accurately classify the quality of providers. Unfortunately, misclassification of provider performance is not uncommon and can have grave implications. Propensity score (PS) methods have been proposed as viable alternatives to conventional multivariable regression. The objective was to assess the outlier classification performance of risk adjustment methods when profiling multiple providers.

METHODS

In a simulation study based on empirical data, the classification performance of logistic regression (fixed and random effects), PS adjustment, and three PS weighting methods was evaluated when varying parameters such as the number of providers, the average incidence of the outcome, and the percentage of outliers. Traditional classification accuracy measures were considered, including sensitivity and specificity.

RESULTS

Fixed effects logistic regression consistently had the highest sensitivity and negative predictive value, yet a low specificity and positive predictive value. Of the random effects methods, PS adjustment and random effects logistic regression performed equally well or better than all the remaining PS methods for all classification accuracy measures across the studied scenarios.

CONCLUSIONS

Of the evaluated PS methods, only PS adjustment can be considered a viable alternative to random effects logistic regression when profiling multiple providers in different scenarios.

Variability in invasive mediastinal staging for lung cancer: A multicenter regional study

OBJECTIVE

Prior studies have reported underuse of-but not variability in-invasive mediastinal staging in the pretreatment evaluation of patients with lung cancer. We sought to compare rates of invasive mediastinal staging for lung cancer across hospitals participating in a regional quality improvement and research collaborative.

METHODS

We conducted a retrospective study (2011-2013) of patients undergoing resected lung cancer from the Surgical Clinical Outcomes and Assessment Program in Washington State. Invasive mediastinal staging included mediastinoscopy and/or endobronchial/esophageal ultrasound-guided nodal aspiration. We used a mixed-effects model to mitigate the influence of small sample sizes at any 1 hospital on rates of invasive staging and to adjust for hospital-level differences in the frequency of clinical stage IA disease.

RESULTS

A total of 406 patients (mean age, 68 years; 69% clinical stage IA; and 67% lobectomy) underwent resection at 5 hospitals (4 community and 1 academic). Invasive staging occurred in 66% of patients (95% confidence interval [CI], 61%-71%). CI inspection revealed that 2 hospitals performed invasive staging significantly more often than the overall average (94%, [95% CI, 89%-96%] and 84% [95% CI, 78%-88%]), whereas 2 hospitals performed invasive staging significantly less often than overall average (31% [95% CI, 21%-44%] and 17% [95% CI, 7%-36%]).

CONCLUSIONS

Rates of invasive mediastinal staging varied significantly across hospitals providing surgical care for patients with lung cancer. Future studies that aim to understand the reasons underlying variability in care may inform quality improvement initiatives or lead to the development of novel staging algorithms.

Failure-to-Rescue in Thoracic Surgery

Variability in outcomes not attributable to case mix or chance is an indicator of low-quality care. Failure-to-rescue is an outcome measure defined as death during a hospitalization among patients who experience a complication. Researchers have used this measure to better understand the determinants of an untimely death-preventing complications, rescue, or both. Studies repeatedly find that complication rates vary little, if at all, across hospitals ranked by risk-adjusted mortality rates, suggesting that hospitals are equally capable (or incapable) of preventing complications. In contrast, variation in failure-to-rescue rates seems to explain much of the variation in risk-adjusted hospital-level mortality rates. These findings suggest that system-level interventions that allow for the early detection and treatment of complications (ie, rescue) may reduce variability in hospital-level outcomes and improve the quality of thoracic surgical care.

Assessing Hospital Performance After Percutaneous Coronary Intervention Using Big Data

BACKGROUND

Although risk adjustment remains a cornerstone for comparing outcomes across hospitals, optimal strategies continue to evolve in the presence of many confounders. We compared conventional regression-based model to approaches particularly suited to leveraging big data.

METHODS AND RESULTS

We assessed hospital all-cause 30-day excess mortality risk among 8952 adults undergoing percutaneous coronary intervention between October 1, 2011, and September 30, 2012, in 24 Massachusetts hospitals using clinical registry data linked with billing data. We compared conventional logistic regression models with augmented inverse probability weighted estimators and targeted maximum likelihood estimators to generate more efficient and unbiased estimates of hospital effects. We also compared a clinically informed and a machine-learning approach to confounder selection, using elastic net penalized regression in the latter case. Hospital excess risk estimates range from -1.4% to 2.0% across methods and confounder sets. Some hospitals were consistently classified as low or as high excess mortality outliers; others changed classification depending on the method and confounder set used. Switching from the clinically selected list of 11 confounders to a full set of 225 confounders increased the estimation uncertainty by an average of 62% across methods as measured by confidence interval length. Agreement among methods ranged from fair, with a κ statistic of 0.39 (SE: 0.16), to perfect, with a κ of 1 (SE: 0.0).

CONCLUSIONS

Modern causal inference techniques should be more frequently adopted to leverage big data while minimizing bias in hospital performance assessments.

What is quality, and can we define it in lung cancer?-the case for quality improvement

Decades worth of advances in diagnostics and therapeutics are associated with only marginal improvements in survival among lung cancer patients. An obvious explanation is late stage at presentation, but gaps in the quality of care may be another reason for stifled improvements in survival rates. A framework for quality put forth by Avedis Donabedian consists of measuring structures-of-care, processes, and outcomes. Using this approach to explore for potential quality gaps, there is evidence of inexplicable variability in outcomes across patients and hospitals; variation in outcomes across differing provider types (structures-of-care); and variation in approaches to staging (processes-of-care). However, this research has limitations and incontrovertible evidence of quality gaps is challenging to obtain. Other challenges to defining quality include scientific and clinical uncertainty among providers and the fact that quality is a multi-dimensional construct that cannot be measured by a single metric. Nonetheless, two facts compel us to pursue quality improvement: (I) both empirically and anecdotally, actual care falls short of expected care; and (II) evidence of potential quality gaps is not ignorable primarily on ethical grounds.