Indirect Versus Direct Standardization Methods for Reporting Healthcare-Associated Infections: An Analysis of Central Line-Associated Bloodstream Infections in Maryland

Do Common Risk Adjustment Methods Do Their Job Well if Center Effects are Correlated With the Center-Specific Mean Values of Patient Characteristics?

BACKGROUND

Direct and indirect standardization are well-established approaches to performing risk adjustment when comparing outcomes between healthcare providers. However, it is an open question whether they work well when there is an association between the center effects and the distributions of the patient characteristics in these centers.

OBJECTIVES AND METHODS

We try to shed further light on the impact of such an association. We construct an artificial case study with a single covariate, in which centers can be classified as performing above, on, or below average, and the center effects correlate with center-specific mean values of a patient characteristic, as a consequence of differential quality improvement. Based on this case study, direct standardization and indirect standardization-based on marginal as well as conditional models-are compared with respect to systematic differences between their results.

RESULTS

Systematic differences between the methods were observed. All methods produced results that partially reflect differences in mean age across the centers. This may mask the classification as above, on, or below average. The differences could be explained by an inspection of the parameter estimates in the models fitted.

CONCLUSIONS

In case of correlations of center effects with center-specific mean values of a covariate, different risk adjustment methods can produce systematically differing results. This suggests the routine use of sensitivity analyses. Center effects in a conditional model need not reflect the position of a center above or below average, questioning its use in defining the truth. Further empirical investigations are necessary to judge the practical relevance of these findings.

Turning up the volume to address biases in predicted healthcare-associated infections and enhance U.S. hospital rankings: A data-driven approach

OBJECTIVES

To examine potential biases in standardized infection ratio (SIR) metrics due to static U.S. Centers for Disease Control and Prevention (CDC) parameters and non-linearity of infection outcomes with volume. Correspondingly, to enhance the CDC predictions by incorporating additional information from volume metrics and explore an alternative approach to more fairly rank hospitals to address the SIR=0 problem.

METHODS

This population-based study uses publicly available 2019 healthcare-associated infections (HAI) data from 3096 acute care U.S. hospitals. HAI-specific Poisson generalized additive models illustrate the recalibration of CDC predictions, using volume-based spline functions to adjust for biases. Implied cumulative distribution functions (CDF) were derived, and HAI-facility-specific probabilities were calculated. Hospital rankings implied from these HAI-stratified probabilities were calculated.

RESULTS

Calibration plots demonstrate existing biases associated with CDC infection over-predictions. Volume-based spline functions were significant for all HAIs (P<.0004). CDF-based rankings resulted in larger discrimination across hospitals based on strength of evidence, especially among SIR=0 facilities. National maps depict ranking differences by HAI and state.

CONCLUSION

Adjustment of SIR biases, which differ by facility volume, is needed to produce more accurate and fairer hospital rankings.