Relationship of Hospital Star Ratings to Race, Education, and Community Income

Variation in Postoperative Outcomes Across Federally Designated Hospital Star Ratings

Importance

Despite widespread use to guide patients to hospitals providing the best care, it remains unknown whether Centers for Medicare & Medicaid Services (CMS) hospital star ratings are a reliable measure of hospital surgical quality.

Objective

To examine the CMS hospital star ratings and hospital surgical quality measured by 30-day postoperative mortality, serious complications, and readmission rates for Medicare beneficiaries undergoing colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, and incisional hernia repair.

Design, Setting, and Participants

This cohort study evaluated 100% Medicare administrative claims for nonfederal acute care hospitals with a CMS hospital star rating for calendar years 2014-2018. Data analysis was performed from January 15, 2022, to April 30, 2023. Participants included fee-for-service Medicare beneficiaries aged 66 to 99 years who underwent colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, or incisional hernia repair with continuous Medicare coverage for 3 months before and 6 months after surgery.

Exposure

Centers for Medicare & Medicaid Services hospital star rating.

Main Outcomes and Measures

Risk- and reliability-adjusted hospital rates of 30-day postoperative mortality, serious complications, and 30-day readmissions were measured and compared across hospitals and star ratings.

Results

A total of 1 898 829 patients underwent colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, or incisional hernia repair at 3240 hospitals with a CMS hospital star rating. Mean (SD) age was 74.8 (7.0) years, 50.6% of the patients were male, and 86.5% identified as White. Risk- and reliability-adjusted 30-day mortality rate decreased in a stepwise fashion from 6.80% (95% CI, 6.79%-6.81%) in 1-star hospitals to 4.93% (95% CI, 4.93%-4.94%) in 5-star hospitals (adjusted odds ratio, 1.86; 95% CI, 1.73-2.00). There was wide variation in the rates of hospital mortality (variation, 1.89%; range, 2.4%-16.2%), serious complications (variation, 1.97%; range, 5.5%-45.1%), and readmission (variation, 1.27%; range, 9.1%-22.5%) across all hospitals. After stratifying hospitals by their star rating, similar patterns of variation were observed within star rating groups for 30-day mortality: 1 star (variation, 1.91%; range, 3.6%-12.0%), 2 star (variation, 1.86%; range, 2.8%-16.2%), 3 star (variation, 1.84%; range, 2.9%-12.3%), 4 star (variation, 1.76%; range, 2.9%-11.5%), and 5 star (variation, 1.79%; range, 2.4%-9.1%). Similar patterns were observed for serious complications and readmissions.

Conclusion and Relevance

Although CMS hospital star rating was associated with postoperative mortality, serious complications, and readmissions, there was wide variation in surgical outcomes within each star rating group. These findings highlight the limitations of the CMS hospital star rating system as a measure of surgical quality and should be a call for continued improvement of publicly reported hospital grade measures.

Calculation of Overall Hospital Quality Star Ratings With and Without Inclusion of the Peer Grouping Step

Importance

The Centers for Medicare & Medicaid Services (CMS) Overall Star Rating is widely used by patients and consumers, and there is continued stakeholder curiosity surrounding the inclusion of a peer grouping step, implemented to the 2021 Overall Star Rating methods.

Objective

To calculate hospital star rating scores with and without the peer grouping step, with the former approach stratifying hospitals into 3-, 4-, and 5-measure group peer groups based on the number of measure groups with at least 3 reported measures.

Design, Setting, and Participants

This cross-sectional study used Care Compare website data from January 2023 for 3076 hospitals that received a star rating in 2023. Data were analyzed from April 2023 to December 2023.

Exposure

Peer grouping vs no peer grouping.

Main Outcomes and Measures

The primary outcome was the distribution of star ratings, with 1 star being the lowest-performing hospitals and 5 stars, the highest. Analyses additionally identified the number of hospitals with a higher, lower, or identical star rating with the use of the peer grouping step compared with its nonuse, stratified by certain hospital characteristics.

Results

Among 3076 hospitals that received a star rating in 2023, most were nonspecialty (1994 hospitals [64.8%]), nonteaching (1807 hospitals [58.7%]), non-safety net (2326 hospitals [75.6%]), non-critical access (2826 hospitals [91.9%]) hospitals with fewer than 200 beds (1822 hospitals [59.2%]) and located in an urban geographic designations (1935 hospitals [62.9%]). The presence of the peer grouping step resulted in 585 hospitals (19.0%) being assigned a different star rating than if the peer grouping step was absent, including considerably more hospitals receiving a higher star rating (517 hospitals) rather than a lower (68 hospitals) star rating. Hospital characteristics associated with a higher star rating included urbanicity (351 hospitals [67.9%]), non-safety net status (414 hospitals [80.1%]), and fewer than 200 beds (287 hospitals [55.6%]). Collectively, the presence of the peer grouping step supports a like-to-like comparison among hospitals and supports the ability of patients to assess overall hospital quality.

Conclusions and Relevance

In this cross-sectional study, inclusion of the peer grouping in the CMS star rating method resulted in modest changes in hospital star ratings compared with application of the method without peer grouping. Given improvement in face validity and the close association between the current peer grouping approach and stakeholder needs for peer-comparison, the current CMS Overall Star Rating method allows for durable comparisons in hospital performance.

Missed opportunities in hospital quality measurement during the COVID-19 pandemic: a retrospective investigation of US hospitals' CMS Star Ratings and 30-day mortality during the early pandemic

OBJECTIVES

In the USA and UK, pandemic-era outcome data have been excluded from hospital rankings and pay-for-performance programmes. We assessed the relationship between US hospitals' pre-pandemic Centers for Medicare and Medicaid Services (CMS) Overall Hospital Star ratings and early pandemic 30-day mortality among both patients with COVID and non-COVID to understand whether pre-existing structures, processes and outcomes related to quality enabled greater pandemic resiliency.

DESIGN AND DATA SOURCE

A retrospective, claim-based data study using the 100% Inpatient Standard Analytic File and Medicare Beneficiary Summary File including all US Medicare Fee-for-Service inpatient encounters from 1 April 2020 to 30 November 2020 linked with the CMS Hospital Star Ratings using six-digit CMS provider IDs.

OUTCOME MEASURE

The outcome was risk-adjusted 30-day mortality. We used multivariate logistic regression adjusting for age, sex, Elixhauser mortality index, US Census Region, month, hospital-specific January 2020 CMS Star rating (1-5 stars), COVID diagnosis (U07.1) and COVID diagnosis×CMS Star Rating interaction.

RESULTS

We included 4 473 390 Medicare encounters from 2533 hospitals, with 92 896 (28.2%) mortalities among COVID-19 encounters and 387 029 (9.3%) mortalities among non-COVID encounters. There was significantly greater odds of mortality as CMS Star Ratings decreased, with 18% (95% CI 15% to 22%; p<0.0001), 33% (95% CI 30% to 37%; p<0.0001), 38% (95% CI 34% to 42%; p<0.0001) and 60% (95% CI 55% to 66%; p<0.0001), greater odds of COVID mortality comparing 4-star, 3-star, 2-star and 1-star hospitals (respectively) to 5-star hospitals. Among non-COVID encounters, there were 17% (95% CI 16% to 19%; p<0.0001), 24% (95% CI 23% to 26%; p<0.0001), 32% (95% CI 30% to 33%; p<0.0001) and 40% (95% CI 38% to 42%; p<0.0001) greater odds of mortality at 4-star, 3-star, 2-star and 1-star hospitals (respectively) as compared with 5-star hospitals.

CONCLUSION

Our results support a need to further understand how quality outcomes were maintained during the pandemic. Valuable insights can be gained by including the reporting of risk-adjusted pandemic era hospital quality outcomes for high and low performing hospitals.

The geography of Medicare's hospital value-based purchasing in relation to market demographics

OBJECTIVE

To illustrate the association between the sociodemographic characteristics of hospital markets and the geographic patterns of Medicare hospital value-based purchasing (HVBP) scores.

DATA SOURCES AND STUDY SETTING

This is a secondary analysis of United States hospitals with a HVBP Total Performance Score (TPS) for 2019 in the Centers for Medicare and Medicaid Services (CMS) Hospital Compare database (4/2021 release) and American Community Survey (ACS) data for 2015-2019.

STUDY DESIGN

This is a cross-sectional study using spatial multivariable autoregressive models with HVBP TPS and component domain scores as dependent variables and hospital market demographics as the independent variables.

DATA COLLECTION/EXTRACTION METHODS

We calculated hospital market demographics using ZIP code level data from the ACS, weighted the 2019 CMS inpatient Hospital Service Area file.

PRINCIPAL FINDINGS

Spatial autoregressive models using eight nearest neighbors with diversity index, race and ethnicity distribution, families in poverty, unemployment, and lack of health insurance among residents ages 19-64 years provided the best model fit. Diversity index had the highest statistically significant contribution to lower TPS (ß = -12.79, p < 0.0001), followed by the percent of the population coded to "non-Hispanic, some other race" (ß = -2.59, p < 0.0023), and the percent of families in poverty (ß = -0.26, p < 0.0001). Percent of the population was non-Hispanic American Indian/Alaskan Native (ß = 0.35, p < 0.0001) and percent non-Hispanic Asian (ß = 0.12, p < 0.02071) were associated with higher TPS. Lower predicted TPS was observed in large urban cities throughout the US as well as in states throughout the Southeastern US. Similar geographic patterns were observed for the predicted Patient Safety, Person and Community Engagement, and Efficiency and Cost Reduction domain scores but are not for predicted Clinical Outcomes scores.

CONCLUSIONS

The lower predicted scores seen in cities and in the Southeastern region potentially reflect an inherent-that is, structural-association between market sociodemographics and HVBP scores.

Why Did New York State Hospitals Rank So Poorly?

BACKGROUND

According to the Centers for Medicare and Medicaid Services star ratings, New York State (NYS) hospitals are relatively poor performers, with 33% achieving 1 star compared with 5% of hospitals across the United States.

OBJECTIVES

We compared NYS hospitals to all United States hospitals using Centers for Medicare and Medicaid Services Hospital Value-Based Purchasing (HVBP) and star ratings component measures. We perform risk adjustment for hospital and market characteristics associated poor performance.

RESEARCH DESIGN

This was a cross-sectional observational study.

SUBJECTS

All acute care hospitals in the United States which had HVBP scores for 2019 in April 21, 2021, Hospital Care Compare database.

MEASURES

Analysis of variance was used to compare NYS hospitals to all United States hospitals. Multivariable-based risk adjustment was applied to NYS hospitals with adjustment for hospital characteristics (eg, occupancy, size), hospital fiscal ratios (eg, operating margin), and market characteristics (eg, percent of hospital market that has a high school diploma).

RESULTS

NYS hospitals averaged lower patient satisfaction and higher readmissions. These domains were statistically significantly associated with lower socioeconomic status in the hospital market area. Risk adjustment reduced but did not eliminate these differences. NYS also performed poorly on pressure ulcers and deep vein thrombosis/pulmonary embolism prevention. NYS hospitals were similar to the United States in mortality and hospital-acquired infections.

CONCLUSIONS

Differences in the demographic makeup of hospital markets account for some of the poor performance of NYS hospitals. Some aspects, such as long length of stay, may be associated with wider regional trends.

Assessing Hospital Quality Scores By Proportion Of Patients Dually Eligible For Medicare And Medicaid

The Centers for Medicare and Medicaid Services has been reporting hospital star ratings since 2016. Some stakeholders have criticized the star ratings methodology for not adjusting for social risk factors. We examined the relationship between 2021 star rating scores and hospitals' proportion of Medicare patients dually eligible for Medicaid. We found that, on average, hospitals caring for a greater proportion of dually eligible patients had lower star ratings, but there was significant overlap in performance among hospitals when we stratified them by quintile of dually eligible patients. Hospitals in the highest quintile (those with the greatest proportion of dually eligible patients) had the best mean mortality scores (0.28) but the worst readmission (-0.44) and patient experience (-0.78) scores. We assigned star ratings after stratifying the readmission measure group by proportion of dually eligible patients and found that a total of 142 hospitals gained a star and 161 hospitals lost a star, of which 126 (89 percent) and 1 (<1 percent) were in the highest quintile, respectively. Adjusting public reporting tools such as star ratings for social risk factors is ultimately a policy decision, and views on the appropriateness of accounting for factors such as proportion of dually eligible patients are mixed, depending on the organization and stakeholder.