Concordance of Hospital Ranks and Category Ratings Using the Current Technical Specification of US Hospital Star Ratings and Reasonable Alternative Specifications

Variation in Postoperative Outcomes Across Federally Designated Hospital Star Ratings

Importance

Despite widespread use to guide patients to hospitals providing the best care, it remains unknown whether Centers for Medicare & Medicaid Services (CMS) hospital star ratings are a reliable measure of hospital surgical quality.

Objective

To examine the CMS hospital star ratings and hospital surgical quality measured by 30-day postoperative mortality, serious complications, and readmission rates for Medicare beneficiaries undergoing colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, and incisional hernia repair.

Design, Setting, and Participants

This cohort study evaluated 100% Medicare administrative claims for nonfederal acute care hospitals with a CMS hospital star rating for calendar years 2014-2018. Data analysis was performed from January 15, 2022, to April 30, 2023. Participants included fee-for-service Medicare beneficiaries aged 66 to 99 years who underwent colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, or incisional hernia repair with continuous Medicare coverage for 3 months before and 6 months after surgery.

Exposure

Centers for Medicare & Medicaid Services hospital star rating.

Main Outcomes and Measures

Risk- and reliability-adjusted hospital rates of 30-day postoperative mortality, serious complications, and 30-day readmissions were measured and compared across hospitals and star ratings.

Results

A total of 1 898 829 patients underwent colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, or incisional hernia repair at 3240 hospitals with a CMS hospital star rating. Mean (SD) age was 74.8 (7.0) years, 50.6% of the patients were male, and 86.5% identified as White. Risk- and reliability-adjusted 30-day mortality rate decreased in a stepwise fashion from 6.80% (95% CI, 6.79%-6.81%) in 1-star hospitals to 4.93% (95% CI, 4.93%-4.94%) in 5-star hospitals (adjusted odds ratio, 1.86; 95% CI, 1.73-2.00). There was wide variation in the rates of hospital mortality (variation, 1.89%; range, 2.4%-16.2%), serious complications (variation, 1.97%; range, 5.5%-45.1%), and readmission (variation, 1.27%; range, 9.1%-22.5%) across all hospitals. After stratifying hospitals by their star rating, similar patterns of variation were observed within star rating groups for 30-day mortality: 1 star (variation, 1.91%; range, 3.6%-12.0%), 2 star (variation, 1.86%; range, 2.8%-16.2%), 3 star (variation, 1.84%; range, 2.9%-12.3%), 4 star (variation, 1.76%; range, 2.9%-11.5%), and 5 star (variation, 1.79%; range, 2.4%-9.1%). Similar patterns were observed for serious complications and readmissions.

Conclusion and Relevance

Although CMS hospital star rating was associated with postoperative mortality, serious complications, and readmissions, there was wide variation in surgical outcomes within each star rating group. These findings highlight the limitations of the CMS hospital star rating system as a measure of surgical quality and should be a call for continued improvement of publicly reported hospital grade measures.

No Association Between Medicare Advantage Providers' Network Restrictiveness and Star Rating Between 2013 and 2017: An Observational Study

BACKGROUND

Medicare beneficiaries are increasingly enrolling in Medicare Advantage (MA), which employs a wide range of practices around restriction of the networks of providers that beneficiaries visit. Though Medicare beneficiaries highly value provider choice, it is unknown whether the MA contract quality metrics which beneficiaries use to inform their contract selection capture the restrictiveness of contracts' provider networks.

OBJECTIVE

We evaluated whether there are meaningful associations between provider network restrictiveness (across primary care, psychiatry, and endocrinology providers) and contracts' overall star quality rating, as well as between network restrictiveness and contracts' performance on access to care measures from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey.

PARTICIPANTS

Medicare Advantage contracts with health maintenance organization (HMO), local preferred provider organization (PPO), and point of service (POS) plans with available data.

DESIGN

A cross-sectional analysis using multivariable linear regressions to assess the relationship between provider network restrictiveness and contract quality scores in 2013 through 2017.

MEASURES

Statistical significance in the relationship between network restrictiveness and contract performance on quality measures.

RESULTS

Across all study years, we included 562 unique contracts and 2801 contract-years. We find no evidence of consistent relationships between MA physician network restrictiveness and contract star rating. For primary care, psychiatry, and endocrinology, respectively, a 10 percentage point increase in restrictiveness was associated with a 0.02 (95% confidence interval [CI] -0.01 to 0.04), 0.0008 (95% CI, -0.01 to 0.02), and -0.01 (95% CI, -0.01 to 0.001) difference in star rating (p-value > 0.05 for all). Similarly, we find no evidence of consistent relationships between network restrictiveness and access to care measures.

CONCLUSIONS

Our findings suggest that existing MA contract quality measures are not useful for indicating differences in network restrictiveness. Given the importance of provider choice to beneficiaries, more specific metrics may be needed to facilitate informed decisions about MA coverage.

Impact of the Italian Healthcare Outcomes Program (PNE) on the Care Quality of the Poorest Performing Hospitals

One of the main aims of the Italian National Healthcare Outcomes Program (Programma Nazionale Esiti, PNE) is the identification of the hospitals with the lowest performance, leading them to improve their quality. In order to evaluate PNE impact for a subset of outcome indicators, we evaluated whether the performance of the hospitals with the lowest scores in 2016 had significantly improved after five years. The eight indicators measured the risk-adjusted likelihood of the death of each patient (adjusted relative risk-RR) 30 days after the admission for acute myocardial infarction, congestive heart failure, stroke, chronic obstructive pulmonary disease, chronic kidney disease, femur fracture or lung and colon cancer. In 2016, the PNE identified 288 hospitals with a very low performance in at least one of the selected indicators. Overall, 51.0% (n = 147) of these hospitals showed some degree of improvement in 2021, and 27.4% of them improved so much that the death risk of their patients fell below the national mean value. In 34.7% of the hospitals, however, the patients still carried a mean risk of death >30% higher than the average Italian patient with the same disease. Only 38.5% of the hospitals in Southern Italy improved the scores of the selected indicators, versus 68.0% in Northern and Central Italy. Multivariate analyses, adjusting for the baseline performance in 2016, confirmed univariate results and showed a significantly lower likelihood of improvement with increasing hospital volume. Despite the overall methodological validity of the PNE system, current Italian policies and actions aimed at translating hospital quality scores into effective organizational changes need to be reinforced with a special focus on larger southern regions.

What is a star worth to Medicare beneficiaries? A discrete choice experiment of hospital quality ratings

Hospital quality ratings are widely available to help Medicare beneficiaries make an informed choice about where to receive care. However, how beneficiaries' trade-off between different quality domains (clinical outcomes, patient experience, safety, efficiency) and other considerations (out-of-pocket cost, travel distance) is not well understood. We sought to study how beneficiaries make trade-offs when choosing a hypothetical hospital. We administered an online survey that included a discrete choice experiment to a nationally representative sample of 1025 Medicare beneficiaries. On average, beneficiaries were willing to pay $1698 more for a hospital with a 1-star higher rating on clinical outcomes. This was over twice the value of the patient experience ($691) and safety ($615) domains and nearly 8 times the value of the efficiency domain ($218). We also found that the value of a 1-star improvement depends not only on the quality domain but also the baseline level of performance of the hospital. Generally, it is more valuable for low-performing hospitals to achieve average performance than for average hospitals to achieve excellence.

Impact of possible errors in natural language processing-derived data on downstream epidemiologic analysis

Objective

To assess the impact of potential errors in natural language processing (NLP) on the results of epidemiologic studies.

Materials and Methods

We utilized data from three outcomes research studies where the primary predictor variable was generated using NLP. For each of these studies, Monte Carlo simulations were applied to generate datasets simulating potential errors in NLP-derived variables. We subsequently fit the original regression models to these partially simulated datasets and compared the distribution of coefficient estimates to the original study results.

Results

Among the four models evaluated, the mean change in the point estimate of the relationship between the predictor variable and the outcome ranged from -21.9% to 4.12%. In three of the four models, significance of this relationship was not eliminated in a single of the 500 simulations, and in one model it was eliminated in 12% of simulations. Mean changes in the estimates for confounder variables ranged from 0.27% to 2.27% and significance of the relationship was eliminated between 0% and 9.25% of the time. No variables underwent a shift in the direction of its interpretation.

Discussion

Impact of simulated NLP errors on the results of epidemiologic studies was modest, with only small changes in effect estimates and no changes in the interpretation of the findings (direction and significance of association with the outcome) for either the NLP-generated variables or other variables in the models.

Conclusion

NLP errors are unlikely to affect the results of studies that use NLP as the source of data.

Methods to derive composite indicators used for quality and safety measurement and monitoring in healthcare: a scoping review protocol

INTRODUCTION

Composite indicators of quality and safety in healthcare summarise performance across multiple indicators into a single performance measure. Composite indicators can identify domains and drivers of quality, improve the ability to detect differences, aid prioritisation for quality improvement and facilitate decision making about future healthcare needs. However, the use of composite indicators can be controversial, particularly when used to rank healthcare providers. Many of the concerns around transparency, appropriateness and uncertainty may be addressed by a robust and transparent development and review process.The aim of this scoping review is to describe methodologies used at each of the stages of development of composite indicators of quality and safety in healthcare. This review will provide those tasked with developing or reviewing composite indicators with a valuable consolidated analysis of a substantial and wide-ranging literature.

METHODS AND ANALYSIS

The framework proposed by the Joanna Briggs Institute and enhancements proposed by Peters et al (2015, 2017, 2020) will be used in conducting this scoping review, and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for scoping reviews will guide the reporting. Grey literature and peer-reviewed documents will be in-scope. Electronic databases (PubMed, Embase, CINAHL, ABI/INFORM and SafetyLit) will be searched, and publications will be screened by two reviewers. Discussion, policy and guidance publications will be included if they discuss any aspect of the methods used in the development of a composite indicator of quality or safety in a healthcare setting. The search period ranges from 1 January 2000 to 31 December 2022. Data extraction will capture information on 11 stages of composite indicator development, augmenting a 10-stage framework developed by the European Commission Joint Research Centre.

ETHICS AND DISSEMINATION

Ethical approval is not required. Review findings will be published in a peer-reviewed journal and presented at scientific conferences.

Options for states to constrain pricing power of health care providers

Health care is becoming increasingly unaffordable for both individuals and employers and prices vary in nearly incomprehensible ways that do not correlate with quality. In many areas, consolidation of insurers and providers resulted in market failure that needs policy interventions. With federal gridlock, state policymakers are seeking options for controlling health care costs in markets where competition has failed. In this article, we discuss a spectrum of options that policymakers have to more directly control healthcare prices: (1) establishing a cost-growth benchmark, (2) creating a public option, (3) capping or establishing a default out-of-network payment rate for health care services, (4) creating affordability standards that authorize the insurance commissioner to reject contracts with excessive rate increases, (5) creating global budgets for hospital-based care, (6) capping excessive prices and/or tiering allowed rate updates, and (7) creating a population-based payment model. We provide a roadmap for state policymakers to consider these options, review the experiences with states who have tried these models, and discuss additional design considerations that policymakers should consider with any of these models. In the 1970's and 1980's, during a time of rapid growth in health care prices and spending, states took a decisive leadership role in developing regulatory models to curb the growth in health care costs and improve affordability for their citizens. It is time for states to lead the nation once again in addressing the current health care cost and affordability crisis in the U.S.