Effect of Case-Mix and Random Variation on Breast Cancer Care Quality Indicators and Their Rankability

Process indicators outshine outcome measures: assessing hospital quality of care in breast cancer treatment in China

Reporting the results of quality indicators can narrow the gap in the quality of care between hospitals. While most studies rely on outcome indicators, they may not accurately measure the quality of care. Process indicators are not only strongly associated with treatment outcomes, but are also more sensitive to whether patients are treated accurately, enabling timely intervention. Our study aims to investigate whether process indicators provide a more reasonable assessment of hospital quality of care compared to outcome indicators. Data were sourced from the Specific Disease Medical Service Quality Management and Control System in China. A total of 113,942 patients with breast cancer treated in 298 hospitals between January 2019 and April 2023 were included in this retrospective study. The rankability of 11 process indicators was calculated and used as a weight to create a new composite indicator. The composite indicators and outcome measures were compared using the O/E ratio categories. Finally, in order to determine the impact of different years on the results, a sensitivity analysis was conducted using bootstrap sampling. The rankability ( ρ ) values of the eleven process indicators showed significant differences, with the highest ρ value for preoperative cytological or histological examination before surgery (0.919). The ρ value for the outcome indicator was 0.011. The rankability-weighting method yielded a comprehensive score ( ρ  = 0.883). The comparison with categorical results of the outcome indicator has different performance classifications for 113 hospitals (37.92%) for composite scores and 140 (46.98%) for preoperative cytological or histological examinationbefore surgery. Process indicators are more suitable than outcome indicators for assessing the quality of breast cancer care in hospitals. Healthcare providers can use process indicators to identify specific areas for improvement, thereby driving continuous quality improvement efforts.

Developing quality indicators for cancer hospitals in China: a national modified Delphi process

OBJECTIVE

Although demand and supply of cancer care have been rapidly increasing in recent decades, there is a lack of systemic quality measurement for cancer hospitals in China. This study aimed to develop a set of core indicators for measuring quality of care for cancer hospitals in China.

DESIGN

The development of quality indicators was based on a literature review and a two-round modified Delphi survey. The theoretical framework and initial indicators were identified through the comprehensive literature review, and the selection of quality indicators relied on experts' consensus on the importance and feasibility of indicators by the modified Delphi process. In addition, indicator weight was identified using the analytical hierarchical process method and percentage weight method.

SETTING AND PARTICIPANTS

A panel of leading experts including oncologists, cancer care nurses, quality management experts from various regions of China were invited to participate in the two-round modified Delphi process from October to December 2020. A total of 25 experts completed the two-round modified Delphi process.

RESULTS

The experts reached consensus on a set of 47 indicators, comprising 17 structure indicators, 19 process indicators and 11 outcome indicators. Experts gave much higher weight to outcome indicators (accounting for 53.96% relative weight) than to structure (16.34%) and process (29.70%) indicators. In addition, experts also showed concerns and gave suggestions on data availability of specific outcome indicators.

CONCLUSIONS

Drawing on the comprehensive literature review and the modified Delphi process, this study developed a core set of quality indicators that can be used to evaluate quality performance of cancer hospitals. This is helpful in supporting quality cancer care in China and will provide new insights into the systemic measurement of cancer care internationally.

Quality indicators in surgical oncology: systematic review of measures used to compare quality across hospitals

BACKGROUND

Measurement and reporting of quality indicators at the hospital level has been shown to improve outcomes and support patient choice. Although there are many studies validating individual quality indicators, there has been no systematic approach to understanding what quality indicators exist for surgical oncology and no standardization for their use. The aim of this study was to review quality indicators used to assess variation in quality in surgical oncology care across hospitals or regions. It also sought to describe the aims of these studies and what, if any, feedback was offered to the analysed groups.

METHODS

A literature search was performed to identify studies published between 1 January 2000 and 23 October 2023 that applied surgical quality indicators to detect variation in cancer care at the hospital or regional level.

RESULTS

A total of 89 studies assessed 91 unique quality indicators that fell into the following Donabedian domains: process indicators (58; 64%); outcome indicators (26; 29%); structure indicators (6; 7%); and structure and outcome indicators (1; 1%). Purposes of evaluating variation included: identifying outliers (43; 48%); comparing centres with a benchmark (14; 16%); and supplying evidence of practice variation (29; 33%). Only 23 studies (26%) reported providing the results of their analyses back to those supplying data.

CONCLUSION

Comparisons of quality in surgical oncology within and among hospitals and regions have been undertaken in high-income countries. Quality indicators tended to be process measures and reporting focused on identifying outlying hospitals. Few studies offered feedback to data suppliers.

European quality indicators developed by the European Commission Initiative on Breast Cancer: a first nationwide assessment for the Dutch setting

PURPOSE

This observational study aims to assess the feasibility of calculating indicators developed by the European Commission Initiative on Breast Cancer (ECIBC) for the Dutch breast cancer population.

METHODS

Patients diagnosed with invasive or in situ breast cancer between 2012 and 2018 were selected from the Netherlands Cancer Registry (NCR). Outcomes of the quality indicators (QI) were presented as mean scores and were compared to a stated norm. Variation between hospitals was assessed by standard deviations and funnel plots and trends over time were evaluated. The quality indicator calculator (QIC) was validated by comparing these outcomes with the outcomes of constructed algorithms in Stata.

RESULTS

In total, 133,527 patients were included. Data for 24 out of 26 QIs were available in the NCR. For 67% and 67% of the QIs, a mean score above the norm and low or medium hospital variation was observed, respectively. The proportion of patients undergoing a breast reconstruction or neoadjuvant systemic therapy increased over time. The proportion treated within 4 weeks from diagnosis, having >10 lymph nodes removed or estrogen negative breast cancer who underwent adjuvant chemotherapy decreased. The outcomes of the constructed algorithms in this study and the QIC showed 100% similarity.

CONCLUSION

Data from the NCR could be used for the calculation of more than 92% of the ECIBC indicators. The quality of breast cancer care in the Netherlands is high, as more than half of the QIs already score above the norm and medium hospital variation was observed. The QIC can be easy and reliably applied.

Stability of hospital quality indicators over time: A multi-year observational study of German hospital data

BACKGROUND

Retrospective hospital quality indicators can only be useful if they are trustworthy signals of current or future quality. Despite extensive longitudinal quality indicator data and many hospital quality public reporting initiatives, research on quality indicator stability over time is scarce and skepticism about their usefulness widespread.

OBJECTIVE

Based on aggregated, widely available hospital-level quality indicators, this paper sought to determine whether quality indicators are stable over time. Implications for health policy were drawn and the limited methodological foundation for stability assessments of hospital-level quality indicators enhanced.

METHODS

Two longitudinal datasets (self-reported and routine data), including all hospitals in Germany and covering the period from 2004 to 2017, were analysed. A logistic regression using Generalized Estimating Equations, a time-dependent, graphic quintile representation of risk-adjusted rates and Spearman's rank correlation coefficient were used.

RESULTS

For a total of eight German quality indicators significant stability over time was demonstrated. The probability of remaining in the best quality cluster in the future across all hospitals reached from 46.9% (CI: 42.4-51.6%) for hip replacement reoperations to 80.4% (CI: 76.4-83.8%) for decubitus. Furthermore, graphical descriptive analysis showed that the difference in adverse event rates for the 20% top performing compared to the 20% worst performing hospitals in the two following years is on average between 30% for stroke and AMI and 79% for decubitus. Stability over time has been shown to vary strongly between indicators and treatment areas.

CONCLUSION

Quality indicators were found to have sufficient stability over time for public reporting. Potentially, increasing case volumes per hospital, centralisation of medical services and minimum-quantity regulations may lead to more stable and reliable quality of care indicators. Finally, more robust policy interventions such as outcome-based payment, should only be applied to outcome indicators with a higher level of stability over time. This should be subject to future research.

Population-Based Clinical Cancer Registration in Germany

INTRODUCTION

In 2013, a new federal law obligated all German federal states to collect additional clinical data in population-based cancer registries as an active tool for monitoring and improving the quality of cancer care, increasing transparency and promoting health research. Now, 10 years later, the current status of the expanded cancer registration is presented, including current figures on cancer in Germany.

METHODS

Reporting of cancer is mandatory for physicians, and about 5 to 10 reports from different healthcare providers are expected for each case. A uniform national dataset of about 130 items is used, and reports are usually sent electronically to the registry. We used the most recent data available from cancer registries up to the year of diagnosis in 2019. We calculated incidence rates and 5-year relative survival (5YRS) for common cancers. Data on clinical outcomes and benchmarking based on quality indicators (QIs) from guidelines were provided by the Cancer Registry Schleswig-Holstein (CR SH).

RESULTS

All federal state cancer registries met most of the previously defined national eligibility criteria. Approximately 505,000 cancer cases were registered in 2019, with breast, prostate, colorectal and lung cancer being the most common cancers. The age-standardised cancer incidence has slightly decreased during the last decade. and spatial heterogeneity can be observed within Germany. 5YRS for all cancers was 67% and 63% for women and men, respectively. Therapy data for rectal cancer in 2019-2021 from the CR SH are shown as an example: 69% of the registered patients underwent surgery, mostly with curative intent (84%) and tumour-free resection (91%). Radiotherapy was given to 33% of the patients, and chemotherapy was given to 40%. Three selected QIs showed differences between involved healthcare providers.

DISCUSSION

The implementation of population-based clinical cancer registration can be considered a success. Comprehensive recording of diagnosis, treatment and disease progression and the use of registry data for quality assurance, benchmarking and feedback have been implemented.

Risk-adjusted benchmarking of long-term overall survival in patients with HER2-positive early-stage Breast cancer: A Swedish retrospective cohort study

AIM

The main objective of the current study was to explore the value of risk-adjustment when comparing (i.e. benchmarking) long-term overall survival (OS) in breast cancer (BC) between Swedish regions. We performed risk-adjusted benchmarking of 5- and 10-year OS after HER2-positive early BC diagnosis between Sweden's two largest healthcare regions, constituting approximately a third of the total population in Sweden.

METHODS

All patients diagnosed with HER2-positive early-stage BC between 01-01-2009 and 31-12-2016 in healthcare regions Stockholm-Gotland and Skane were included in the study. Cox proportional hazards model was used for risk-adjustment. Unadjusted (i.e. crude) and adjusted 5- and 10-year OS was benchmarked between the two regions.

RESULTS

The crude 5-year OS was 90.3% in the Stockholm-Gotland region and 87.8% in the Skane region. The crude 10-year OS was 81.7% in the Stockholm-Gotland region and 77.3% in the Skane region. However, when adjusted for age, menopausal status and tumour biology, there was no significant OS disparity between the regions, neither at the 5-year nor 10-year follow-up.

CONCLUSION

This study showed that risk-adjustment is relevant when benchmarking OS in BC, even when comparing regions from the same country that share the same national treatment guidelines. This is, to our knowledge, the first published risk-adjusted benchmarking of OS in HER2-positive BC.

Patient healthcare experiences of cancer hospitals in China: A multilevel modeling analysis based on a national survey

Importance

Patient satisfaction is a crucial indicator for assessing quality of care in healthcare settings. However, patient satisfaction benchmark for cancer hospitals in China is not established.

Objective

To examine patient satisfaction levels in tertiary cancer hospitals in China, and inter-hospital variations after case-mix adjustment.

Design

A nationwide cross-sectional hospital performance survey conducted from January to March 2021.

Settings

At 30 tertiary cancer hospitals in China.

Participants

A total of 4,847 adult inpatients consecutively recruited at 30 tertiary cancer hospitals were included.

Exposures

Patient characteristics included demographic characteristics (sex, age, education, and annual family income), clinical characteristics (cancer type, cancer stage, self-reported health status, and length of stay), and actual respondents of questionnaire.

Main outcomes and measures

Patient satisfaction was measured using 23 items covering five aspects, administrative process, hospital environment, medical care, symptom management, and overall satisfaction. Responses to each item were recorded using a 5-point Likert scale. Patient satisfaction level for each aspect was described at individual and hospital levels. Using multilevel logistic regression, patient characteristics associated with patient satisfaction were examined as case-mix adjusters and inter-hospital variation were determined.

Results

The satisfaction rates for symptom management, administrative process, hospital environment, overall satisfaction, and medical care aspects were 74.56, 81.70, 84.18, 84.26, and 90.86% with a cut-off value of 4, respectively. Significant predictors of patient satisfaction included sex, age, cancer type, cancer stage, self-reported health status, and actual respondent (representative or patient) (all P < 0.05). The ranking of the hospitals' performance in satisfaction was altered after the case-mix adjustment was made. But even after the adjustment, significant variation in satisfaction among hospitals remained.

Conclusions and relevance

This study pointed to symptom management as a special area, to which a keen attention should be paid by policymakers and hospital administrators. Significant variation in satisfaction among hospitals remained, implying that future studies should examine major factors affecting the variation. In review, target interventions are needed in low-performing hospitals.

Associations of hospital volume and hospital competition with short-term, middle-term and long-term patient outcomes after breast cancer surgery: a retrospective population-based study

OBJECTIVES

For oncological care, there is a clear tendency towards centralisation and collaboration aimed at improving patient outcomes. However, in market-based healthcare systems, this trend is related to the potential trade-off between hospital volume and hospital competition. We analyse the association between hospital volume, competition from neighbouring hospitals and outcomes for patients who underwent surgery for invasive breast cancer (IBC).

OUTCOME MEASURES

Surgical margins, 90 days re-excision, overall survival.

DESIGN, SETTING, PARTICIPANTS

In this population-based study, we use data from the Netherlands Cancer Registry. Our study sample consists of 136 958 patients who underwent surgery for IBC between 2004 and 2014 in the Netherlands.

RESULTS

Our findings show that treatment types as well as patient and tumour characteristics explain most of the variation in all outcomes. After adjusting for confounding variables and intrahospital correlation in multivariate logistic regressions, hospital volume and competition from neighbouring hospitals did not show significant associations with surgical margins and re-excision rates. For patients who underwent surgery in hospitals annually performing 250 surgeries or more, multilevel Cox proportional hazard models show that survival was somewhat higher (HR 0.94). Survival in hospitals with four or more (potential) competitors within 30 km was slightly higher (HR 0.97). However, this effect did not hold after changing this proxy for hospital competition.

CONCLUSIONS

Based on the selection of patient outcomes, hospital volume and regional competition appear to play only a limited role in the explanation of variation in IBC outcomes across Dutch hospitals. Further research into hospital variation for high-volume tumours like the one studied here is recommended to (i) use consistently measured quality indicators that better reflect multidisciplinary clinical practice and patient and provider decision-making, (ii) include more sophisticated measures for hospital competition and (iii) assess the entire process of care within the hospital, as well as care provided by other providers in cancer networks.

Quality indicators for cardiac rehabilitation after myocardial infarction in China: a consensus panel and practice test

OBJECTIVES

Cardiac rehabilitation (CR) improves outcomes after myocardial infarction (MI), but it is underused in China. The purpose of this study was to develop a set of quality indicators (QIs) to improve clinical practices and to confirm the measurability and performance of the developed QIs for CR in Chinese patients after MI.

DESIGN AND SETTING

The QIs were developed by a Chinese expert consensus panel during in-person meetings. The five QIs most in need of improvement were selected using a national questionnaire. Finally, the completion rate and feasibility of the QIs were verified in a group of MI survivors at university hospitals in China.

PARTICIPANTS

Seventeen professionals participated in the consensus panel, 89 personnel in the field of CR participated in the national questionnaire and 165 MI survivors participated in the practice test.

RESULTS

A review of 17 eligible articles generated 26 potential QIs, among which 17 were selected by the consensus panel after careful evaluation. The 17 QIs were divided into two domains: (1) improving participation and adherence and (2) CR process standardisation. Nationwide telephone and WeChat surveys identified the five QIs most in need of improvement. A multicenter practice test (n=165) revealed that the mean performance value of the proposed QIs was 43.9% (9.9%-86.1%) according to patients with post-MI.

CONCLUSIONS

The consensus panel identified a comprehensive set of QIs for CR in patients with post-MI. A nationwide questionnaire survey was used to identify the QIs that need immediate attention to improve the quality of CR. Although practice tests confirmed the measurability of the proposed QIs in clinical practice, the implementation of the QIs needs to be improved.

TRIAL REGISTRATION NUMBER

This study is part of a study registered in ClinicalTrials.gov (NCT03528382).

Assessing Nursing Homes Quality Indicators' Between-Provider Variability and Reliability: A Cross-Sectional Study Using ICCs and Rankability

Nursing home quality indicators are often used to publicly report the quality of nursing home care. In Switzerland, six national nursing home quality indicators covering four clinical domains (polypharmacy, pain, use of physical restraints and weight loss) were recently developed. To allow for meaningful comparisons, these indicators must reliably show differences in quality of care levels between nursing homes. This study’s objectives were to assess nursing home quality indicators’ between-provider variability and reliability using intraclass correlations and rankability. This approach has not yet been used in long-term care contexts but presents methodological advantages. This cross-sectional multicenter study uses data of 11,412 residents from a convenience sample of 152 Swiss nursing homes. After calculating intraclass correlation 1 (ICC1) and rankability, we describe between-provider variability for each quality indicator using empirical Bayes estimate-based caterpillar plots. To assess reliability, we used intraclass correlation 2 (ICC2). Overall, ICC1 values were high, ranging from 0.068 (95% confidence interval (CI) 0.047–0.086) for polypharmacy to 0.396 (95% CI 0.297–0.474) for physical restraints, with quality indicator caterpillar plots showing sufficient between-provider variability. However, testing for rankability produced mixed results, with low figures for two indicators (0.144 for polypharmacy; 0.471 for self-reported pain) and moderate to high figures for the four others (from 0.692 for observed pain to 0.976 for physical restraints). High ICC2 figures, ranging from 0.896 (95% CI 0.852–0.917) (self-reported pain) to 0.990 (95% CI 0.985–0.993) (physical restraints), indicated good reliability for all six quality indicators. Intraclass correlations and rankability can be used to assess nursing home quality indicators’ between-provider variability and reliability. The six selected quality indicators reliably distinguish care differences between nursing homes and can be recommended for use, although the variability of two—polypharmacy and self-reported pain—is substantially chance-driven, limiting their utility.