Association of Patient-reported Experiences and Surgical Outcomes Among Group Practices: Retrospective Cohort Study

Association Between the Physician Quality Score in the Merit-Based Incentive Payment System and Hospital Performance in Hospital Compare in the First Year of the Program

IMPORTANCE

The scientific validity of the Merit-Based Incentive Payment System (MIPS) quality score as a measure of hospital-level patient outcomes is unknown.

OBJECTIVE

To examine whether better physician performance on the MIPS quality score is associated with better hospital outcomes.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study of 38 830 physicians used data from the Centers for Medicare & Medicaid Services (CMS) Physician Compare (2017) merged with CMS Hospital Compare data. Data analysis was conducted from September to November 2020.

MAIN OUTCOMES AND MEASURES

Linear regression was used to examine the association between physician MIPS quality scores aggregated at the hospital level and hospitalwide measures of (1) postoperative complications, (2) failure to rescue, (3) individual postoperative complications, and (4) readmissions.

RESULTS

The study cohort of 38 830 clinicians (5198 [14.6%] women; 12 103 [31.6%] with 11-20 years in practice) included 6580 (17.2%) general surgeons, 8978 (23.4%) orthopedic surgeons, 1617 (4.2%) vascular surgeons, 582 (1.5%) cardiac surgeons, 904 (2.4%) thoracic surgeons, 18 149 (47.4%) anesthesiologists, and 1520 (4.0%) intensivists at 3055 hospitals. The MIPS quality score was not associated with the hospital composite rate of postoperative complications. MIPS quality scores for vascular surgeons in the 11th to 25th percentile, compared with those in the 51st to 100th percentile, were associated with a 0.55-percentage point higher hospital rate of failure to rescue (95% CI, 0.06-1.04 percentage points; P = .03). MIPS quality scores for cardiac surgeons in the 1st to 10th percentile, compared with those in the 51st to 100th percentile, were associated with a 0.41-percentage point higher hospital coronary artery bypass graft (CABG) mortality rate (95% CI, 0.10-0.71 percentage points; P = .01). MIPS quality scores for cardiac surgeons in the 1st to 10th percentile and 11th to 25th percentile, compared with those in the 51st to 100th percentile, were associated with 0.65-percentage point (95% CI, 0.013-1.16 percentage points; P = .02) and 0.48-percentage point (95% CI, 0.07-0.90 percentage points; P = .02) higher hospital CABG readmission rates, respectively.

CONCLUSIONS AND RELEVANCE

In this study, better performance on the physician MIPS quality score was associated with better hospital surgical outcomes for some physician specialties during the first year of MIPS.

Categorising patient concerns using natural language processing techniques

Objectives

Patient feedback is critical to identify and resolve patient safety and experience issues in healthcare systems. However, large volumes of unstructured text data can pose problems for manual (human) analysis. This study reports the results of using a semiautomated, computational topic-modelling approach to analyse a corpus of patient feedback.

Methods

Patient concerns were received by Alberta Health Services between 2011 and 2018 (n=76 163), regarding 806 care facilities in 163 municipalities, including hospitals, clinics, community care centres and retirement homes, in a province of 4.4 million. Their existing framework requires manual labelling of pre-defined categories. We applied an automated latent Dirichlet allocation (LDA)-based topic modelling algorithm to identify the topics present in these concerns, and thereby produce a framework-free categorisation.

Results

The LDA model produced 40 topics which, following manual interpretation by researchers, were reduced to 28 coherent topics. The most frequent topics identified were communication issues causing delays (frequency: 10.58%), community care for elderly patients (8.82%), interactions with nurses (8.80%) and emergency department care (7.52%). Many patient concerns were categorised into multiple topics. Some were more specific versions of categories from the existing framework (eg, communication issues causing delays), while others were novel (eg, smoking in inappropriate settings).

Discussion

LDA-generated topics were more nuanced than the manually labelled categories. For example, LDA found that concerns with community care were related to concerns about nursing for seniors, providing opportunities for insight and action.

Conclusion

Our findings outline the range of concerns patients share in a large health system and demonstrate the usefulness of using LDA to identify categories of patient concerns.

Relationship Between Perioperative Outcomes Used for Profiling Hospital Noncardiac Surgical Quality

BACKGROUND

Risk-adjusted morbidity and mortality are commonly used by national surgical quality improvement (QI) programs to measure hospital-level surgical quality. However, the degree of hospital-level correlation between mortality, morbidity, and other perioperative outcomes (like reoperation) collected by contemporary surgical QI programs has not been well-characterized.

MATERIALS AND METHODS

Veterans Affairs (VA) Surgical Quality Improvement Program (VASQIP) data (2015-2016) were used to evaluate hospital-level correlation in performance between risk-adjusted 30-d mortality, morbidity, major morbidity, reoperation, and 2 composite outcomes (1- mortality, major morbidity, or reoperation; 2- mortality or major morbidity) after noncardiac surgery. Correlation between outcomes rates was evaluated using Pearson's correlation coefficient. Correlation between hospital risk-adjusted performance rankings was evaluated using Spearman's correlation.

RESULTS

Based on a median of 232 [IQR 95-331] quarterly surgical cases abstracted by VASQIP, statistical power for identifying 30-d mortality outlier hospitals was estimated between 3.3% for an observed-to-expected ratio of 1.1 and 45.7% for 3.0. Among 230,247 Veterans who underwent a noncardiac operation at 137 VA hospitals, there were moderate hospital-level correlations between various risk-adjusted outcome rates (highest r = 0.40, mortality and composite 1; lowest r = 0.32, mortality and morbidity). When hospitals were ranked based on performance, there was low-to-moderate correlation between rankings on the various outcomes (highest ρ = 0.47, mortality and composite 1; lowest ρ = 0.37, mortality and major morbidity).

CONCLUSIONS

Modest hospital-level correlations between perioperative outcomes suggests it may be difficult to identify high (or low) performing hospitals using a single measure. Additionally, while composites of currently measured outcomes may be an efficient way to improve analytic sample size (relative to evaluations based on any individual outcome), further work is needed to understand whether they provide a more robust and accurate picture of hospital quality or whether evaluating performance across a portfolio of individual measures is most effective for driving QI.