Ranking and rankability of hospital postoperative mortality rates in colorectal cancer surgery

Process indicators outshine outcome measures: assessing hospital quality of care in breast cancer treatment in China

Reporting the results of quality indicators can narrow the gap in the quality of care between hospitals. While most studies rely on outcome indicators, they may not accurately measure the quality of care. Process indicators are not only strongly associated with treatment outcomes, but are also more sensitive to whether patients are treated accurately, enabling timely intervention. Our study aims to investigate whether process indicators provide a more reasonable assessment of hospital quality of care compared to outcome indicators. Data were sourced from the Specific Disease Medical Service Quality Management and Control System in China. A total of 113,942 patients with breast cancer treated in 298 hospitals between January 2019 and April 2023 were included in this retrospective study. The rankability of 11 process indicators was calculated and used as a weight to create a new composite indicator. The composite indicators and outcome measures were compared using the O/E ratio categories. Finally, in order to determine the impact of different years on the results, a sensitivity analysis was conducted using bootstrap sampling. The rankability ( ρ ) values of the eleven process indicators showed significant differences, with the highest ρ value for preoperative cytological or histological examination before surgery (0.919). The ρ value for the outcome indicator was 0.011. The rankability-weighting method yielded a comprehensive score ( ρ  = 0.883). The comparison with categorical results of the outcome indicator has different performance classifications for 113 hospitals (37.92%) for composite scores and 140 (46.98%) for preoperative cytological or histological examinationbefore surgery. Process indicators are more suitable than outcome indicators for assessing the quality of breast cancer care in hospitals. Healthcare providers can use process indicators to identify specific areas for improvement, thereby driving continuous quality improvement efforts.

Measurement that matters: A systematic review and modified Delphi of multidisciplinary colorectal cancer quality indicators

AIM

To develop a priority set of quality indicators (QIs) for use by colorectal cancer (CRC) multidisciplinary teams (MDTs).

METHODS

The review search strategy was executed in four databases from 2009-August 2019. Two reviewers screened abstracts/manuscripts. Candidate QIs and characteristics were extracted using a tailored abstraction tool and assessed for scientific soundness. To prioritize candidate indicators, a modified Delphi consensus process was conducted. Consensus was sought over two rounds; (1) multidisciplinary expert workshops to identify relevance to Australian CRC MDTs, and (2) an online survey to prioritize QIs by clinical importance.

RESULTS

A total of 93 unique QIs were extracted from 118 studies and categorized into domains of care within the CRC patient pathway. Approximately half the QIs involved more than one discipline (52.7%). One-third of QIs related to surgery of primary CRC (31.2%). QIs on supportive care (6%) and neoadjuvant therapy (6%) were limited. In the Delphi Round 1, workshop participants (n = 12) assessed 93 QIs and produced consensus on retaining 49 QIs including six new QIs. In Round 2, survey participants (n = 44) rated QIs and prioritized a final 26 QIs across all domains of care and disciplines with a concordance level > 80%. Participants represented all MDT disciplines, predominantly surgical (32%), radiation (23%) and medical (20%) oncology, and nursing (18%), across six Australian states, with an even spread of experience level.

CONCLUSION

This study identified a large number of existing CRC QIs and prioritized the most clinically relevant QIs for use by Australian MDTs to measure and monitor their performance.

Stability of hospital quality indicators over time: A multi-year observational study of German hospital data

BACKGROUND

Retrospective hospital quality indicators can only be useful if they are trustworthy signals of current or future quality. Despite extensive longitudinal quality indicator data and many hospital quality public reporting initiatives, research on quality indicator stability over time is scarce and skepticism about their usefulness widespread.

OBJECTIVE

Based on aggregated, widely available hospital-level quality indicators, this paper sought to determine whether quality indicators are stable over time. Implications for health policy were drawn and the limited methodological foundation for stability assessments of hospital-level quality indicators enhanced.

METHODS

Two longitudinal datasets (self-reported and routine data), including all hospitals in Germany and covering the period from 2004 to 2017, were analysed. A logistic regression using Generalized Estimating Equations, a time-dependent, graphic quintile representation of risk-adjusted rates and Spearman's rank correlation coefficient were used.

RESULTS

For a total of eight German quality indicators significant stability over time was demonstrated. The probability of remaining in the best quality cluster in the future across all hospitals reached from 46.9% (CI: 42.4-51.6%) for hip replacement reoperations to 80.4% (CI: 76.4-83.8%) for decubitus. Furthermore, graphical descriptive analysis showed that the difference in adverse event rates for the 20% top performing compared to the 20% worst performing hospitals in the two following years is on average between 30% for stroke and AMI and 79% for decubitus. Stability over time has been shown to vary strongly between indicators and treatment areas.

CONCLUSION

Quality indicators were found to have sufficient stability over time for public reporting. Potentially, increasing case volumes per hospital, centralisation of medical services and minimum-quantity regulations may lead to more stable and reliable quality of care indicators. Finally, more robust policy interventions such as outcome-based payment, should only be applied to outcome indicators with a higher level of stability over time. This should be subject to future research.

Data analytics and artificial intelligence in predicting length of stay, readmission, and mortality: a population-based study of surgical management of colorectal cancer

Data analytics and artificial intelligence (AI) have been used to predict patient outcomes after colorectal cancer surgery. A prospectively maintained colorectal cancer database was used, covering 4336 patients who underwent colorectal cancer surgery between 2003 and 2019. The 47 patient parameters included demographics, peri- and post-operative outcomes, surgical approaches, complications, and mortality. Data analytics were used to compare the importance of each variable and AI prediction models were built for length of stay (LOS), readmission, and mortality. Accuracies of at least 80% have been achieved. The significant predictors of LOS were age, ASA grade, operative time, presence or absence of a stoma, robotic or laparoscopic approach to surgery, and complications. The model with support vector regression (SVR) algorithms predicted the LOS with an accuracy of 83% and mean absolute error (MAE) of 9.69 days. The significant predictors of readmission were age, laparoscopic procedure, stoma performed, preoperative nodal (N) stage, operation time, operation mode, previous surgery type, LOS, and the specific procedure. A BI-LSTM model predicted readmission with 87.5% accuracy, 84% sensitivity, and 90% specificity. The significant predictors of mortality were age, ASA grade, BMI, the formation of a stoma, preoperative TNM staging, neoadjuvant chemotherapy, curative resection, and LOS. Classification predictive modelling predicted three different colorectal cancer mortality measures (overall mortality, and 31- and 91-days mortality) with 80-96% accuracy, 84-93% sensitivity, and 75-100% specificity. A model using all variables performed only slightly better than one that used just the most significant ones.

Quality-of-care comparison of stroke: The reliability and robustness of ranking by process or outcome measures

BACKGROUND AND AIM

Discussion on the most rational types of performance measures for care quality comparisons has received increasing attention. The important consideration is to what extent will the measure detect a genuine difference in the underlying quality. In this study, we aimed to compare the ranking of hospitals on the performance of individual indicators, composite scores (CS, that were calculated by the method of opportunity-based score on patient-level), and in-hospital outcome of acute ischemic stroke across hospitals, and determined the reliability and robustness of the three types of ranking.

METHODS

We analyzed data from 15,090 patients diagnosed with acute ischemic stroke who were treated at 184 large tertiary hospitals from January 2014 to May 2017. We ranked the hospital effects of recombinant tissue plasminogen activator (rt-PA) and CS and independence (modified Rankin Scale ≤2) at discharge based on fixed- and random-effects regression models before and after case-mix adjustment. We assessed the time-robustness of the hospital effects and calculated the rankability by relating the uncertainty within the hospital and the total hospital variation "beyond chance."

RESULTS

After case-mix and reliability adjustment, we estimated that 84.03% of the variance in CS between hospitals was due to true quality differences. The uncertainty within hospitals caused a poor (49.51%) rankability in rt-PA and moderate rankability (63.34%) in independence at discharge. The hospital rankings of CS were more robust across years compared with rt-PA and independence.

CONCLUSIONS

Our data indicated that CS is the optimal measure to indicate the quality-of-care variation of acute ischemic stroke between hospitals.

Variability in outcomes and quality-of-care indicators across clinics participating in a large smoking-cessation program

BACKGROUND

The effectiveness of care for substance-related problems varies across providers. Best-known treatments are rarely universally applied, and various process differences can affect participant outcomes. Measuring and understanding this variability can suggest changes that will improve system performance.

METHODS

We measure variability in 7-day cigarette abstinence at a six-month follow-up; return for a second clinical visit; and receipt of combination nicotine replacement therapy across 223 primary care clinics participating in the Smoking Treatment for Ontario Patients program, a large smoking cessation initiative in Ontario, Canada. We include 41,992 enrolments from April 11, 2016 and May 31, 2019. We risk adjust for demographic and clinical case-mix differences and characterize variability using funnel plots and measures based on clinic-level variance explained. The abstinence outcome is missing for 38% of participants. We adjust for missingness using multiple imputation and inverse probability weighting.

RESULTS

Abstinence was achieved by 28.0% (95% CI = 27.5%-28.5%) of participants, 63.2% (62.8%-63.7%) received combination NRT, and 72.9% (72.4%-73.3%) returned for a second clinical visit. Variability was moderate for abstinence (median odds ratio (MOR) = 1.16) and pronounced for return visit (MOR = 1.29) and combination therapy (MOR = 1.89).

CONCLUSION

Outcomes and processes vary significantly across clinics within a program with shared guidelines and standards. Differences across providers may be greater in other contexts. Results underscore the importance of measuring and understanding variability, and of ongoing maintenance and improvement. The existence of high outliers holds out the possibility of identifying practices that might be more widely adopted.

Assessment of Cancer Center Variation in Textbook Oncologic Outcomes Following Colectomy for Adenocarcinoma

BACKGROUND

Traditional metrics may inadequately represent rates of attaining optimal oncologic care. We evaluated a composite "textbook oncologic outcome" (TOO) to assess the incidence of achieving an "optimal" clinical result after colon adenocarcinoma (CA) resection.

METHODS

The National Cancer Database (NCDB) was queried to identify patients undergoing colectomy for non-metastatic CA between 2010 and 2015. TOO was defined as a margin negative resection with an AJCC compliant lymph node evaluation, no prolonged length of stay (LOS) or 30-day readmission/mortality, as well as receipt of stage appropriate adjuvant chemotherapy.

RESULTS

Among 170,120 patients who underwent colectomy at 1315 hospitals, 93,204 (54.8%) achieved TOO with large variations observed among facilities. While certain factors were achieved nearly universally (R0 margin, 95.6%; no 30-day mortality, 97.2%), avoidance of prolonged LOS (77.3%) and appropriate adjuvant chemotherapy (83.0%) were achieved less consistently. On multivariable analysis, Black race/ethnicity (OR 0.82, 95% CI 0.80-0.85), Medicaid insurance (OR 0.64, 0.61-0.68), and low-volume facility (< 50/year) (OR 0.83, 0.77-0.89) were associated with decreased likelihood of TOO. Achievement of TOO was associated with improved long-term survival (HR 0.45; 95% CI 0.44-0.46).

CONCLUSIONS

Roughly one-half of patients undergoing resection of CA achieved an optimal clinical outcome. TOO may be a more useful quality metric to assess patient-centric composite outcomes following surgical procedures.

Nutritional status and its effect on complications in patients with colorectal cancer

BACKGROUND

Nutritional status is an important factor for predicting the risk of developing complications after a surgical procedure. Many nutritional assessments are used in clinical practice, but their role in predicting postoperative outcomes is unknown. Therefore, the aim of this study was to assess the utility of nutritional risk factors at the diagnosis of colorectal cancer (CRC) for predicting early postsurgical complications.

METHODS

This was a prospective observational study including 127 patients with CRC at diagnosis. Their preoperative nutritional status was analyzed by body mass index (BMI), triceps and subscapular skinfolds and two nutritional scales: the Patient-Generated Subjective Global Assessment (PG-SGA) and the Malnutrition Universal Screening Tool (MUST). The outcome variables, including postoperative complications, length of hospital stay and mortality, were analyzed.

RESULTS

Patients identified as malnourished by PG-SGA score had prolonged hospital stays (p = 0.01). The risk of infection was increased in older patients (hazard ratio, HR 1.12; 95% confidence interval, CI 1.04-1.21) but was not associated with nutritional status. Early wound dehiscence was increased in patients with higher BMI (HR 1.15; 95% CI 1.01-1.29), with higher subscapular skinfold thickness and increased age (HR 1.05; 95% CI 1.05-1.10). Postoperative mortality was not significantly associated with nutritional status.

CONCLUSION

Malnourished patients, as identified by the PG-SGA score, stayed longer in hospital than patients who were not malnourished, while increased BMI was recognized as a risk factor for wound dehiscence.

Between-Hospital Variation in Revision Rates After Total Hip and Knee Arthroplasty in the Netherlands: Directing Quality-Improvement Initiatives

BACKGROUND

Variation in 1-year revision rates between Dutch hospitals after primary total hip and knee arthroplasty (THA and TKA) may direct quality-improvement initiatives if this variation accurately reflects true hospital differences. The aim of the present study was to assess the extent of variation, both overall and for specific indications, as well as the statistical reliability of ranking hospitals.

METHODS

All primary THAs and TKAs that were performed between January 2014 and December 2016 were included. Observed/expected (O/E) ratios regarding 1-year revision rates were depicted in a funnel plot with 95% control limits to identify outliers based on 1 or 3 years of data, both overall and by specific indication for revision. The expected number was calculated on the basis of patient mix with use of logistic regression models. The statistical reliability of ranking hospitals (rankability) on these outcomes indicates the percentage of total variation that is explained by "true" hospital differences rather than chance. Rankability was evaluated using fixed and random effects models, for overall revisions and specific indications for revision, including 1 versus 3 years of data.

RESULTS

The present study included 86,468 THAs and 73,077 TKAs from 97 and 98 hospitals, respectively. Thirteen hospitals performing THAs were identified as negative outliers (median O/E ratio, 1.9; interquartile range [IQR], 1.5-2.5), with 5 hospitals as outliers in multiple years. Eight negative outliers were identified for periprosthetic joint infection; 4, for dislocation; and 2, for prosthesis loosening. Seven hospitals performing TKAs were identified as negative outliers (median O/E ratio, 2.3; IQR, 2.2-2.8), with 2 hospitals as outliers in multiple years. Two negative outlier hospitals were identified for periprosthetic joint infection and 1 was identified for technical failures. The rankability for overall revisions was 62% (moderate) for THA and 46% (low) for TKA.

CONCLUSIONS

There was large between-hospital variation in 1-year revision rates after primary THA and TKA. For most outlier hospitals, a specific indication for revision could be identified as contributing to worse performance, particularly for THA; these findings are starting points for quality-improvement initiatives.

Causal variance decompositions for institutional comparisons in healthcare

There is increasing interest in comparing institutions delivering healthcare in terms of disease-specific quality indicators (QIs) that capture processes or outcomes showing variations in the care provided. Such comparisons can be framed in terms of causal models, where adjusting for patient case-mix is analogous to controlling for confounding, and exposure is being treated in a given hospital, for instance. Our goal here is to help identify good QIs rather than comparing hospitals in terms of an already chosen QI, and so we focus on the presence and magnitude of overall variation in care between the hospitals rather than the pairwise differences between any two hospitals. We consider how the observed variation in care received at patient level can be decomposed into that causally explained by the hospital performance adjusting for the case-mix, the case-mix itself, and residual variation. For this purpose, we derive a three-way variance decomposition, with particular attention to its causal interpretation in terms of potential outcome variables. We propose model-based estimators for the decomposition, accommodating different link functions and either fixed or random effect models. We evaluate their performance in a simulation study and demonstrate their use in a real data application.

Reliability of hospital scores for the Cancer Patient Experience Survey: analysis of publicly reported patient survey data

OBJECTIVES

To assess the degree to which variations in publicly reported hospital scores arising from the English Cancer Patient Experience Survey (CPES) are subject to chance.

DESIGN

Secondary analysis of publically reported data.

SETTING

English National Health Service hospitals.

PARTICIPANTS

72 756 patients who were recently treated for cancer in one of 146 hospitals and responded to the 2016 English CPES.

MAIN OUTCOME MEASURES

Spearman-Brown reliability of hospital scores on 51 evaluative questions regarding cancer care.

RESULTS

Hospitals varied in respondent sample size with a median hospital sample size of 419 responses (range 31-1972). There were some hospitals with generally highly reliable scores across most questions, whereas other hospitals had generally unreliable scores (the median reliability of question scores within individual hospitals varied between 0.11 and 0.86). Similarly, there were some questions with generally high reliability across most hospitals, whereas other questions had generally low reliability. Of the 7377 individual hospital scores publically reported (146 hospitals by 51 questions, minus 69 suppressed scores), only 34% reached a reliability of 0.7, the minimum generally considered to be useful. In order for 80% of the individual hospital scores to reach a reliability of 0.7, some hospitals would require a fourfold increase in number of respondents; although in a few other hospitals sample sizes could be reduced.

CONCLUSIONS

The English Patient Experience Survey represents a globally unique source for understanding experience of a patient with cancer; but in its present form, it is not reliable for high stakes comparisons of the performance of different hospitals. Revised sampling strategies and survey questions could help increase the reliability of hospital scores, and thus make the survey fit for use in performance comparisons.

Hospital differences in mortality rates after hip fracture surgery in Denmark

Background

Thirty-day mortality after hip fracture is widely used when ranking hospital performance, but the reliability of such hospital ranking is seldom calculated. We aimed to quantify the variation in 30-day mortality across hospitals and to determine the hospital general contextual effect for understanding patient differences in 30-day mortality risk.

Methods

Patients aged ≥65 years with an incident hip fracture registered in the Danish Multidisciplinary Fracture Registry between 2007 and 2016 were identified (n=60,004). We estimated unadjusted and patient-mix adjusted risk of 30-day mortality in 32 hospitals. We performed a multilevel analysis of individual heterogeneity and discriminatory accuracy with patients nested within hospitals. We expressed the hospital general contextual effect by the median odds ratio (MOR), the area under the receiver operating characteristics curve and the variance partition coefficient (VPC).

Results

The overall 30-day mortality rate was 10%. Patient characteristics including high sociodemographic risk score, underweight, comorbidity, a subtrochanteric fracture, and living at a nursing home were strong predictors of 30-day mortality (area under the curve=0.728). The adjusted differences between hospital averages in 30-day mortality varied from 5% to 9% across the 32 hospitals, which correspond to a MOR of 1.18 (95% CI: 1.12-1.25). However, the hospital general context effect was low, as the VPC was below 1% and adding the hospital level to a single-level model with adjustment for patient-mix increased the area under the receiver operating characteristics curve by only 0.004 units.

Conclusions

Only minor hospital differences were found in 30-day mortality after hip fracture. Mortality after hip fracture needs to be lowered in Denmark but possible interventions should be patient oriented and universal rather than focused on specific hospitals.

Ranking hospital performance based on individual indicators: can we increase reliability by creating composite indicators?

BACKGROUND

Report cards on the health care system increasingly report provider-specific performance on indicators that measure the quality of health care delivered. A natural reaction to the publishing of hospital-specific performance on a given indicator is to create 'league tables' that rank hospitals according to their performance. However, many indicators have been shown to have low to moderate rankability, meaning that they cannot be used to accurately rank hospitals. Our objective was to define conditions for improving the ability to rank hospitals by combining several binary indicators with low to moderate rankability.

METHODS

Monte Carlo simulations to examine the rankability of composite ordinal indicators created by pooling three binary indicators with low to moderate rankability. We considered scenarios in which the prevalences of the three binary indicators were 0.05, 0.10, and 0.25 and the within-hospital correlation between these indicators varied between - 0.25 and 0.90.

RESULTS

Creation of an ordinal indicator with high rankability was possible when the three component binary indicators were strongly correlated with one another (the within-hospital correlation in indicators was at least 0.5). When the binary indicators were independent or weakly correlated with one another (the within-hospital correlation in indicators was less than 0.5), the rankability of the composite ordinal indicator was often less than at least one of its binary components. The rankability of the composite indicator was most affected by the rankability of the most prevalent indicator and the magnitude of the within-hospital correlation between the indicators.

CONCLUSIONS

Pooling highly-correlated binary indicators can result in a composite ordinal indicator with high rankability. Otherwise, the composite ordinal indicator may have lower rankability than some of its constituent components. It is recommended that binary indicators be combined to increase rankability only if they represent the same concept of quality of care.

Ranking hospitals: do we gain reliability by using composite rather than individual indicators?

Background

Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part of variation in performance reflects ‘true’ hospital differences in outcomes versus random noise.

Objective

This study sought to assess whether combining data into composites or including data from multiple years improves the reliability of ranking quality indicators for hospital care.

Methods

Using the Dutch National Medical Registration (2007–2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction and total hiparthroplasty (THA)/ total knee arthroplasty (TKA) in osteoarthritis (OA), we calculated the rankability for in-hospital mortality, 30-day acute readmission and prolonged length of stay (LOS) for single years and 3-year periods and for a dichotomous and ordinal composite measure in which mortality, readmission and prolonged LOS were combined. Rankability, defined as (between-hospital variation/between-hospital+within hospital variation)×100% is classified as low (<50%), moderate (50%–75%) and high (>75%).

Results

Admissions from 555 053 patients treated in 95 hospitals were included. The rankability for mortality was generally low or moderate, varying from less than 1% for patients with OA undergoing THA/TKA in 2011 to 71% for stroke in 2010. Rankability for acute readmission was low, except for acute myocardial infarction in 2009 (51%) and 2012 (62%). Rankability for prolonged LOS was at least moderate. Combining multiple years improved rankability but still remained low in eight cases for both mortality and acute readmission. Combining the individual indicators into the dichotomous composite, all diagnoses had at least moderate rankability (range: 51%–96%). For the ordinal composite, only heart failure had low rankability (46% in 2008) (range: 46%–95%).

Conclusion

Combining multiple years or into multiple indicators results in more reliable ranking of hospitals, particularly compared with mortality and acute readmission in single years, thereby improving the ability to detect true hospital differences. The composite measures provide more information and more reliable rankings than combining multiple years of individual indicators.

Evaluation of hospital outcomes: the relation between length-of-stay, readmission, and mortality in a large international administrative database

Background

Hospital mortality, readmission and length of stay (LOS) are commonly used measures for quality of care. We aimed to disentangle the correlations between these interrelated measures and propose a new way of combining them to evaluate the quality of hospital care.

Methods

We analyzed administrative data from the Global Comparators Project from 26 hospitals on patients discharged between 2007 and 2012. We correlated standardized and risk-adjusted hospital outcomes on mortality, readmission and long LOS. We constructed a composite measure with 5 levels, based on literature review and expert advice, from survival without readmission and normal LOS (best) to mortality (worst outcome). This composite measure was analyzed using ordinal regression, to obtain a standardized outcome measure to compare hospitals.

Results

Overall, we observed a 3.1% mortality rate, 7.8% readmission rate (in survivors) and 20.8% long LOS rate among 4,327,105 admissions. Mortality and LOS were correlated at the patient and the hospital level. A patient in the upper quartile LOS had higher odds of mortality (odds ratio = 1.45, 95% confidence interval 1.43–1.47) than those in the lowest quartile. Hospitals with a high standardized mortality had higher proportions of long LOS (r = 0.79, p < 0.01). Readmission rates did not correlate with either mortality or long LOS rates. The interquartile range of the standardized ordinal composite outcome was 74–117. The composite outcome had similar or better reliability in ranking hospitals than individual outcomes.

Conclusions

Correlations between different outcome measures are complex and differ between hospital- and patient-level. The proposed composite measure combines three outcomes in an ordinal fashion for a more comprehensive and reliable view of hospital performance than its component indicators.

Validity of the CR-POSSUM model in surgery for colorectal cancer in Spain (CCR-CARESS study) and comparison with other models to predict operative mortality

Background

To validate and recalibrate the CR- POSSUM model and compared its discriminatory capacity with other European models such as POSSUM, P-POSSUM, AFC or IRCS to predict operative mortality in surgery for colorectal cancer.

Methods

Prospective multicenter cohort study from 22 hospitals in Spain. We included patients undergoing planned or urgent surgery for primary invasive colorectal cancers between June 2010 and December 2012 (N = 2749). Clinical data were gathered through medical chart review. We validated and recalibrated the predictive models using logistic regression techniques. To calculate the discriminatory power of each model, we estimated the areas under the curve - AUC (95% CI). We also assessed the calibration of the models by applying the Hosmer-Lemeshow test.

Results

In-hospital mortality was 1.5% and 30-day mortality, 1.7%. In the validation process, the discriminatory power of the CR-POSSUM for predicting in-hospital mortality was 73.6%. However, in the recalibration process, the AUCs improved slightly: the CR-POSSUM reached 75.5% (95% CI: 67.3–83.7). The discriminatory power of the CR-POSSUM for predicting 30-day mortality was 74.2% (95% CI: 67.1–81.2) after recalibration; among the other models the POSSUM had the greatest discriminatory power, with an AUC of 77.0% (95% CI: 68.9–85.2). The Hosmer-Lemeshow test showed good fit for all the recalibrated models.

Conclusion

The CR-POSSUM and the other models showed moderate capacity to discriminate the risk of operative mortality in our context, where the actual operative mortality is low. Nevertheless the IRCS might better predict in-hospital mortality, with fewer variables, while the CR-POSSUM could be slightly better for predicting 30-day mortality.

Trail registration

Registered at: ClinicalTrials.gov Identifier: NCT02488161

Electronic supplementary material

The online version of this article (10.1186/s12913-018-2839-x) contains supplementary material, which is available to authorized users.

Variation and statistical reliability of publicly reported primary care diagnostic activity indicators for cancer: a cross-sectional ecological study of routine data

OBJECTIVES

Recent public reporting initiatives in England highlight general practice variation in indicators of diagnostic activity related to cancer. We aimed to quantify the size and sources of variation and the reliability of practice-level estimates of such indicators, to better inform how this information is interpreted and used for quality improvement purposes.

DESIGN

Ecological cross-sectional study.

SETTING

English primary care.

PARTICIPANTS

All general practices in England with at least 1000 patients.

MAIN OUTCOME MEASURES

Sixteen diagnostic activity indicators from the Cancer Services Public Health Profiles.

RESULTS

Mixed-effects logistic and Poisson regression showed that substantial proportions of the observed variance in practice scores reflected chance, variably so for different indicators (between 7% and 85%). However, after accounting for the role of chance, there remained substantial variation between practices (typically up to twofold variation between the 75th and 25th centiles of practice scores, and up to fourfold variation between the 90th and 10th centiles). The age and sex profile of practice populations explained some of this variation, by different amounts across indicators. Generally, the reliability of diagnostic process indicators relating to broader populations of patients most of whom do not have cancer (eg, rate of endoscopic investigations, or urgent referrals for suspected cancer (also known as 'two week wait referrals')) was high (≥0.80) or very high (≥0.90). In contrast, the reliability of diagnostic outcome indicators relating to incident cancer cases (eg, per cent of all cancer cases detected after an emergency presentation) ranged from 0.24 to 0.54, which is well below recommended thresholds (≥0.70).

CONCLUSIONS

Use of indicators of diagnostic activity in individual general practices should principally focus on process indicators which have adequate or high reliability and not outcome indicators which are unreliable at practice level.

Benchmarking quality for renal cancer surgery: Canadian Kidney Cancer information system (CKCis) perspective

INTRODUCTION

There is a lack of validated quality metrics to evaluate the care of patients receiving surgery for renal cell carcinoma (RCC). To address this, the Kidney Cancer Research Network of Canada defined a list of quality indicators (QI) to assess hospital-level performance. We have case-mix adjusted these QIs to benchmark RCC surgical care at Canadian academic centres.

METHODS

The Canadian Kidney Cancer information system (CKCis) was used to measure six QIs: laparoscopic approach proportion (LA), partial nephrectomy proportion (PN), partial nephrectomy in patients with chronic kidney disease (CKDPN), positive margin rate (PMR), partial nephrectomy complication rate (PNCx), and warm ischemia time (WIT). To benchmark performance, indirect standardization (observed-to-expected ratio) methodology was employed using multivariate regression models.

RESULTS

Multivariate models for LA, PN, and CKDPN demonstrated good discrimination and were used for benchmarking. National averages of 74% (70-78%), 73% (70-75%), and 70% (67-74%) for the LA, PN, and CKDPN QIs, respectively, were determined and used to benchmark individual hospital performance. Overall, three (23%), two (15%), and two (15%) hospitals performed below expected for LA, PN, and CKDPN, respectively. Hospital identity was an independent predictor of LA, PN, and CKDPN (p<0.001).

CONCLUSIONS

Significant variability between CKCis hospitals for three RCC surgical QIs exists. Using the CKCis infrastructure may provide a framework for institution-level audit feedback for quality improvement. Greater CKCis capture rates and further data supporting the construct validity of these QIs are required to extend the use of this dataset to real-world quality initiatives.

Surgical Site Infections: Volume-Outcome Relationship and Year-to-Year Stability of Performance Rankings

BACKGROUND

Surgical site infection (SSI) rates are publicly reported as quality metrics and increasingly used to determine financial reimbursement.

OBJECTIVE

To evaluate the volume-outcome relationship as well as the year-to-year stability of performance rankings following coronary artery bypass graft (CABG) surgery and hip arthroplasty.

RESEARCH DESIGN

We performed a retrospective cohort study of Medicare beneficiaries who underwent CABG surgery or hip arthroplasty at US hospitals from 2005 to 2011, with outcomes analyzed through March 2012. Nationally validated claims-based surveillance methods were used to assess for SSI within 90 days of surgery. The relationship between procedure volume and SSI rate was assessed using logistic regression and generalized additive modeling. Year-to-year stability of SSI rates was evaluated using logistic regression to assess hospitals' movement in and out of performance rankings linked to financial penalties.

RESULTS

Case-mix adjusted SSI risk based on claims was highest in hospitals performing <50 CABG/year and <200 hip arthroplasty/year compared with hospitals performing ≥200 procedures/year. At that same time, hospitals in the worst quartile in a given year based on claims had a low probability of remaining in that quartile the following year. This probability increased with volume, and when using 2 years' experience, but the highest probabilities were only 0.59 for CABG (95% confidence interval, 0.52-0.66) and 0.48 for hip arthroplasty (95% confidence interval, 0.42-0.55).

CONCLUSIONS

Aggregate SSI risk is highest in hospitals with low annual procedure volumes, yet these hospitals are currently excluded from quality reporting. Even for higher volume hospitals, year-to-year random variation makes past experience an unreliable estimator of current performance.

Short-term outcome in patients treated with cytoreduction and HIPEC compared to conventional colon cancer surgery

Cytoreductive surgery (CRS) and hyperthermic intraperitoneal chemotherapy (HIPEC) is an extensive procedure with considerable morbidity. Since only few hospitals perform CRS + HIPEC, this might lead to confounded outcomes between hospitals when audited. This study aims to compare outcomes between peritoneally metastasized (PM) colon cancer patients treated with CRS + HIPEC and patients undergoing conventional colon surgery. Furthermore, the impact of CRS + HIPEC on the risk of postoperative complications will be assessed, probably leading to better insight into how to report on postoperative outcomes in this distinct group of patients undergoing extensive colon surgery.All patients with primary colon cancer who underwent segmental colon resection in a tertiary referral hospital between 2011 and 2014 were included in this prospective cohort study. Outcome after surgery was compared between patients who underwent additional CRS + HIPEC treatment or conventional surgery.Consequently, 371 patients underwent surgery, of which 43 (12%) underwent CRS + HIPEC. These patients were younger and healthier than patients undergoing conventional surgery. Tumor characteristics were less favorable and surgery was more extensive in CRS + HIPEC patients. The morbidity rate was also higher in CRS + HIPEC patients (70% vs 41%; P < 0.001). CRS + HIPEC was an independent predictor of postoperative complications (odds ratio 6.4), but was not associated with more severe postoperative complications or higher treatment-related mortality.Although patients with colonic PM undergoing CRS + HIPEC treatment were younger and healthier, the postoperative outcome was worse. This is most probably due to less favorable tumor characteristics and more extensive surgery. Nevertheless, CRS + HIPEC treatment was not associated with severe complications or increased treatment-related mortality. These results stress the need for adequate case-mix correction in colorectal surgery audits.

Performance Measurement to Demonstrate Value

Anesthesiologists are obligated to demonstrate the value of the care they provide. The Centers for Medicare and Medicaid Services has multiple performance-based payment programs to drive high-value care and motivate integrated care for surgical patients and hospitalized patients. These programs rely on diverse arrays of performance measures and complex reporting rules. Among all specialties, anesthesiology has tremendous potential to effect wide-ranging change on diverse measures. Performance measures deserve scrutiny by anesthesiologists as tools to improve care, the means by which payment is determined, and as a means to demonstrate the value of care to surgeons, hospitals, and patients.

Individual and Clustered Rankability of ICUs According to Case-Mix-Adjusted Mortality

OBJECTIVES

The performance of ICUs can be compared by ranking them into a league table according to their risk-adjusted mortality rate. The statistical quality of a league table can be expressed as its rankability, the percentage of variation between ICUs attributable to unexplained differences. We examine whether we can improve the rankability of our league table by using data from a longer period or by grouping ICUs with similar performance constructing a league table of clusters rather than individual ICUs.

DESIGN

We developed a league table for risk-adjusted mortality rate with its rankability. The effect of assessment period was determined using a resampling procedure. Hierarchical clustering was used to obtain clusters of similar ICUs.

PATIENTS

We used data from ICUs participating in the Dutch National Intensive Care Evaluation registry between 2011 and 2013.

MEASUREMENTS AND MAIN RESULTS

We constructed league tables using 157,394 admissions from 78 ICUs with risk-adjusted mortality rate between 5.9% and 13.9% per ICU over the inclusion period. The rankability was 73% for 2013 and 89% for the whole period 2011-2013. Rankability over the year 2013 increased till 98% when clustering ICUs, reaching an optimum at a league table of seven clusters.

CONCLUSIONS

We conclude that, when using data from a single year, the rankability of a league table of Dutch ICUs based on risk-adjusted mortality rate was unacceptably low. We could improve the rankability of this league table by increasing the period of data collection or by grouping similar ICUs into clusters and constructing a league table of clusters of ICUs rather than individual ICUs. Ranking clusters of ICUs could be useful for identifying possible differences in performance between clusters of ICUs.

Outcomes the Ontario Bariatric Network: a cohort study

BACKGROUND

Bariatric surgery centres of excellence are relatively new in Canada and were first started in Ontario in 2009. This study presents short-term outcomes of Canada's largest bariatric collaborative, from Ontario, during its first 3 years.

METHODS

We performed a population-based cohort study that included all patients (age ≥ 18) who received a Roux-en-Y gastric bypass or sleeve gastrectomy for the purpose of weight loss from March 2009 to April 2012 within Ontario. Data were derived from the Canadian Institute for Health Information Discharge Abstract and Hospital Morbidity Databases. Primary outcomes included short-term overall complication rate, reoperation rate, anastomotic leak rate and death. Hierarchical logistic regression was used to identify risk factors for overall complications. A median odds ratio (OR) was used to compare risk-adjusted complication rates across centres of excellence.

RESULTS

A total of 5007 procedures (91.7% Roux-en-Y gastric bypass, 8.3% sleeve gastrectomy) were performed during the 3-year study period, with an overall complication rate of 11.7% (95% confidence interval [CI] 10.8%-12.6%). The leak rate was 0.84% (95% CI 0.61%-1.13%), the reoperation rate was 4.6% (95% CI 4.0%-5.2%) and mortality was 0.16% (95% CI 0.07%-0.31%). Male sex, chronic kidney disease and osteoarthritis were identified as risk factors for overall complications (p value < 0.05). The median ORs across centres of excellence, calculated for both overall complications and reoperation rate, were 1.76 and 1.49, respectively.

INTERPRETATION

Bariatric surgery within Ontario has similar short-term outcomes to those of other major world centres. The variability of outcomes within centres of excellence highlights areas for program quality improvement.

Long-term follow-up and development of diverticulitis in patients diagnosed with diverticulosis of the colon

INTRODUCTION

Diverticulosis of the colon is the most occurring abnormality in the digestive tract. Little is known on the risk of developing diverticulitis.

AIM

The study aims to assess the risk of diverticulitis.

PATIENTS AND METHODS

All patients undergoing colonoscopy in the years 1998, 1999, and 2000 were studied. Patients with cancer, inflammatory bowel disease, anastomoses, and prior diverticulitis were excluded. In the summer of 2015, all hospital records, endoscopy reports, and reports from the department of radiology were studied. Diverticulitis had to be confirmed by the clinical presentation but also via ultrasound or CT scan. In order to obtain enough follow-up years, patients above the age of 75 years were excluded.

RESULTS

After exclusions, a study group of 433 patients remained. There was no difference is gender between patients developing diverticulitis and those who did not. There was no difference in age at time of the index colonoscopy. The sum of follow-up years was 6191. Range of follow-up was 0 to 17 years. The mean follow-up was 14.1 years per patient. Thirty cases of diverticulitis (7 %) could be identified; this is 4.8 cases per 1000 years. The mean time to development of diverticulitis was 5.9 years. Diverticulitis had a mild presentation in 19 patients and a severe presentation needing surgical intervention in 11.

CONCLUSION

The risk of developing diverticulitis is low. This contradicts the belief that diverticulosis has a high rate of progression. These results can help inform patients with diverticulosis about their risk of developing acute diverticulitis.

Thirty days post-operative mortality after surgery for colorectal cancer: a descriptive study

BACKGROUND

The goal of surgery for colorectal cancer is cure. Unfortunately post-operative mortality occurs. This study aims to identify co-morbidity and causes of mortality in the post-operative period in relation to direct technical complications of surgery.

METHODS

All consecutive patients who underwent surgery for colorectal cancer were included. Co-morbidity was determined via the Charlson co-morbidity score. The post-operative course was studied and cause of death within 30 days was determined. Patients were divided in two groups: group 1 died within 30 days after surgery and group 2 survived for longer than 30 days.

RESULTS

Twenty three out of 333 patients (6.9%) with colon cancer and 6 out of 112 (5.3%) with rectal cancer died in the post-operative period. Patients in group 1 were significantly older than patients in group 2 (P<0.001). Patients in group 1 with colon cancer also significantly had more often a higher stage of cancer (P=0.03). The Charlson co-morbidity score for patients with colon cancer in group 1 was mean 5.17 (SD 1.57, range, 1-8), and for rectal cancer mean 4.83 (SD 2.32, range, 2-7). There was no difference in Charlson co-morbidity score when patients from groups 1 and 2 were compared. In group 1, 13 (44%) died as a direct consequence of technical surgical complications. Sixteen patients died due to complications because of pre-existing co-morbidity.

CONCLUSIONS

Post-operative mortality very often is the direct result of pre-existing co-morbidity and not always the direct result of the surgical procedure.

Variation of outcome and charges in operative management for diverticulitis

BACKGROUND

Outcomes after surgery for diverticulitis are of continued interest to improve quality of care. The aim of this study was to assess variations in mortality, length of stay, and patient charges between geographic regions of the United States.

METHODS

A retrospective analysis of the Nationwide Inpatient Sample database was performed. Adults with diverticulitis who underwent laparoscopic or open segmental colectomy were identified using ICD-9 codes. Subset analyses were performed by state and then compared. Outcomes included mortality, length of stay (LOS), and total charges. Results were adjusted for age, race, gender, findings of peritonitis, stoma placement, Charlson comorbidity index, and insurance status on multivariate analysis.

RESULTS

148,874 patients underwent segmental colectomy for diverticulitis from 1998 to 2010. Using California as the comparison state and after adjusting for covariates, in-hospital mortality was significantly higher in the State of New York (OR 1.32; 95 % CI 1.13-1.55; P < 0.05) and Mississippi (OR 2.84; 95 % CI 1.24-6.51, P < 0.02). Wisconsin had a significant lower mortality rate (OR 0.74; 95 % CI 0.59-0.94, P < 0.01). LOS was 1.4 days longer in New York and 0.54 days shorter in Wisconsin than in California (P < 0.01). Patients with age >40 years, findings of peritonitis, and without private insurance had higher in-hospital mortality and longer length of stay. Average hospital charges differed dramatically between the states in the observation period. The highest hospital charges occurred in California, Nebraska, and Nevada while lowest occurred in Maryland, Wisconsin and Utah.

CONCLUSIONS

Patients who undergo surgical treatment for diverticulitis in the United States have high geographic variation in mortality, LOS, and hospital charges despite adjusting for demographic and socioeconomic factors. Further analysis should be performed to identify the causes of outlier regions, with the goal of improving and standardizing best practices.