Rating Hospital Performance in China: Review of Publicly Available Measures and Development of a Ranking System

BACKGROUND

In China, significant emphasis and investment in health care reform since 2009 has brought with it increasing scrutiny of its public hospitals. Calls for greater accountability in the quality of hospital care have led to increasing attention toward performance measurement and the development of hospital ratings. Despite such interest, there has yet to be a comprehensive analysis of what performance information is publicly available to understand the performance of hospitals in China.

OBJECTIVE

This study aims to review the publicly available performance information about hospitals in China to assess options for ranking hospital performance.

METHODS

A review was undertaken to identify performance measures based on publicly available data. Following several rounds of expert consultation regarding the utility of these measures, we clustered the available options into three key areas: research and development, academic reputation, and quality and safety. Following the identification and clustering of the available performance measures, we set out to translate these into a practical performance ranking system to assess variation in hospital performance.

RESULTS

A new hospital ranking system termed the China Hospital Development Index (CHDI) is thus presented. Furthermore, we used CHDI for ranking well-known tertiary hospitals in China.

CONCLUSIONS

Despite notable limitations, our assessment of available measures and the development of a new ranking system break new ground in understanding hospital performance in China. In doing so, CHDI has the potential to contribute to wider discussions and debates about assessing hospital performance across global health care systems.

Disagreement Between Hospital Rating Systems: Measuring the Correlation of Multiple Benchmarks and Developing a Quality Composite Rank

In the United States, hospital rating system usefulness is limited by heterogeneity and conflicting results. US News Best Hospitals, Vizient Quality and Accountability Study, Centers for Medicare & Medicaid Services (CMS) Star Rating, Leapfrog Hospital Safety Grade, and the Truven Top 100 Hospitals ratings were compared using Spearman correlations. Rank aggregation was used to combine the scores generating a Quality Composite Rank (QCR). The highest correlation between rating systems was shown between the Leapfrog Safety Grade and the CMS Star Rating. In a proportional odds logistic regression, a greater discordance between the CMS Star Rating, Vizient rank, US News, and Leapfrog was associated with a lower overall rank in the QCR. Lack of transparency and understanding about the differences and similarities for these hospital ranking systems complicates use of the measures. By combining the results of these ranking systems into a composite, the measurement of hospital quality can be simplified.

Commercial quality "awards" are not a strong indicator of quality surgical care

BACKGROUND

This study aimed to determine whether publicized hospital rankings can be used to predict surgical outcomes.

METHODS

Patients undergoing one of nine surgical procedures were identified, using the Healthcare Cost and Utilization Project State Inpatient Database for Florida and New York 2011-2013 and merged with hospital data from the American Hospital Association Annual Survey. Nine quality designations were analyzed as possible predictors of inpatient mortality and postoperative complications, using logistic regression, decision trees, and support vector machines.

RESULTS

We identified 229,657 patients within 177 hospitals. Decision trees were the highest performing machine learning algorithm for predicting inpatient mortality and postoperative complications (accuracy 0.83, P<.001). The top 3 variables associated with low surgical mortality (relative impact) were Hospital Compare (42), total procedure volume (16) and, Joint Commission (12). When analyzed separately for each individual procedure, hospital quality awards were not predictors of postoperative complications for 7 of the 9 studied procedures. However, when grouping together procedures with a volume-outcome relationship, hospital ranking becomes a significant predictor of postoperative complications.

CONCLUSION

Hospital quality rankings are not a reliable indicator of quality for all surgical procedures. Hospital and provider quality must be evaluated with an emphasis on creating consistent, reliable, and accurate measures of quality that translate to improved patient outcomes.

Standardized Infection Ratio for Surgical Site Infection after Colon Surgery: Discord in Models Measuring Healthcare Quality

The government publishes 3 different public report surgical site infection (SSI) metrics, all called standardized infection ratios (SIRs), that impact perceived hospital quality. We conducted a non-random cross-sectional observational pilot study of 20 California hospitals that voluntarily submitted colon surgery and SSI data. Discordant SIR values, leading to contradictory conclusions, occurred in 35% of these hospitals. Infect Control Hosp Epidemiol 2016;1-5.

Buyer beware: health choices information broadcast to the public

Purpose

– This paper is based on the author’s 2015 Northwest Patient Safety Conference presentation, consistent with a conference theme of improving doctor-patient communication. The paper aims to discuss these issues.

Design/methodology/approach

– Ongoing systematic accumulation and critical review of research literature regarding design of effective public information websites, conducted from 2008-2015 while the author was supervising the prototyping, refinement and evaluation of healthcare-associated infections public information websites.

Findings

– In 2005, the US Centers for Medicare and Medicaid Services launched its Hospital Compare website, announced as an enormous step forward by providing objective information to inform consumer choices. Subsequently, many other websites and programs emerged to report quality-of-care ratings of hospitals and doctors, and provide other advice intended to help the public inform their choices. When objectively evaluated to a scientific publication-level standard, websites like Hospital Compare show relatively low usage and disappointing impact; individual providers rank so differently across ratings websites that it is difficult to see how trustworthy conclusions could be drawn; and much of the advice offered through popular media is not supported by believable evidence. Further, research shows healthcare professionals and members of the lay public view concepts of evidence and evidence-based decisions quite differently. Badly informed misguided decisions can have negative consequences for providers, patients and public trust.

Originality/value

– Populism and celebrity seem to have trumped science during recent growth of public information resources for health choices. This paper summarizes serious flaws underlying resulting information products, indicating necessary changes to better serve a legitimate need.