Creating Individual Surgeon Performance Assessments in a Statewide Hospital Surgical Quality Improvement Collaborative

Surgeon behaviors related to engaging patients in smoking cessation at the time of elective surgery

BACKGROUND

Despite the abundance of evidence supporting smoking cessation before elective surgery, there is wide variation in surgeon adherence to these best practices.

METHODS

This qualitative study used convenience sampling to recruit General Surgery trained surgeons. Surgeons participated in semi-structured interviews based on domains from the Theoretical Domains Framework (TDF). Content analysis was guided by the TDF.

RESULTS

Of the 14 TDF domains, social or professional role/identity, memory, attention and decision processes, environmental context and resources, and beliefs about consequences emerged most frequently. Mapping these domains to the Behavior Change Wheel identified education, enablement, and incentivization as effective intervention functions.

CONCLUSIONS

Using the TDF, this study identified a widespread sense of responsibility among surgeons to engage patients in perioperative smoking cessation despite workplace barriers and lacking resources. These findings provide valuable insight to facilitate surgeon participation in health promotion through targeted, theory-based interventions informed by surgeon identified barriers to perioperative smoking cessation.

Measuring Quality at the Surgeon Level

Patients expect high-quality surgical care and increasingly are looking for ways to assess the quality of the surgeon they are seeing, but quality measurement is often more complicated than one might expect. Measurement of individual surgeon quality in a manner that allows for comparison among surgeons is particularly difficult. While the concept of measuring individual surgeon quality has a long history, technology now allows for new and innovative ways to measure and achieve surgical excellence. However, some recent efforts to make surgeon-level quality data publicly available have highlighted the challenges of this work. Through this chapter, the reader will be introduced to a brief history of surgical quality measurement, learn about the current state of quality measurement, and get a glimpse into what the future holds.

The Measurement Reliability of Complications and Patient Satisfaction After Common Surgical Procedures

OBJECTIVE

To evaluate the reliability of surgeon outcomes.

BACKGROUND

Surgeons' outcomes are now widely used in public reporting and value-based reimbursement, but the reliability of these measures continues to raise concerns.

METHODS

We performed a retrospective study of surgeons performing cholecystectomy, colectomy, and hernia repair on adult patients between January 1, 2017, and December 31, 2020. Outcomes were risk-adjusted rates of complications and high patient satisfaction. We estimated the reliability of each outcome, its relationship with case volume, and the number of surgeons who reached an acceptable level of reliability (≥0.70).

RESULTS

A total of 23,533 patients with a mean age of 56.8 (16.2) years and 10,191 (43.3%) females underwent operations by 333 surgeons. Risk-adjusted complication rate was 2.5% [95% confidence interval (CI): 2.2%-2.8%] and risk-adjusted high satisfaction rate was 79.9% (95% CI: 78.7%-81.0%). The reliability of the complication rate was 0.27 (95% CI: 0.25-0.29) and the reliability of the high satisfaction rate was 0.53 (95% CI: 0.50-0.55). Reliability increased with case volume; however, only 5 (1.5%) surgeons performed enough cases to reach acceptable reliability for their complication rate, while 86 (25.8%) surgeons reached acceptable reliability for their patient satisfaction rate. After adjustment for reliability, the range of complication rates decreased 29-fold from 0% to 14.3% to 2.4% to 2.9%, and the range of patient satisfaction decreased 2.6-fold from 25.3% to 100.0% to 64.9% to 92.4%.

CONCLUSIONS

Among surgeons performing common operations, complications and patient satisfaction had relatively low reliability. Although reliability increased with volume, most surgeons had insufficient case volume to achieve acceptable reliability of their outcomes. As such, these measures likely offer little to no meaningful information to inform decision-making.

Tissue damage force estimation in porcine small intestine from its elasticity

PURPOSE

Post-surgical complications are correlated to the surgeon's technical skill level. Thus, efforts are being put in finding ways to improve the surgeon's technical skills, such as not causing unwanted damage to tissues during surgery. In this study, we aim to investigate the possibility of estimating biological tissue damage, in view of preventing unwanted damage during surgery.

METHODS

A series of tensile tests were performed on porcine small intestinal tissue to determine the elasticity and the tearing force. The tissue was then microscopically observed to investigate the influence of fibrous protein configuration in the tissue's mechanical properties.

RESULTS

The results from the tensile test showed that the fracture energy had a positive and linear correlation with the elasticity to the negative 0.5th power (R² = 0.897), which was also suggested by an existing damage model for polymeric materials (Lake-Thomas model). The results from the microscopic observations also showed a resembling influence of fiber configuration on the elasticity as suggested in polymer mechanics (affine network model).

CONCLUSION

We showed that the fracture energy had a correlation with the elasticity in porcine small intestinal tissues, which was also suggested in polymer mechanics, thus being a promising avenue toward the ability to estimate the maximum applicable force onto a biological tissue without causing damage during surgery. Attention should also be pointed, however, towards investigating the extent at which polymer mechanics and biomechanics overlap.

The Doctors' Effect on Patients' Physical Health Outcomes Beyond the Intervention: A Methodological Review

Background

Previous research suggests that when a treatment is delivered, patients' outcomes may vary systematically by medical practitioner.

Objective

To conduct a methodological review of studies reporting on the effect of doctors on patients' physical health outcomes and to provide recommendations on how this effect could be measured and reported in a consistent and appropriate way.

Methods

The data source was 79 included studies and randomized controlled trials from a systematic review of doctors' effects on patients' physical health. We qualitatively assessed the studies and summarized how the doctors' effect was measured and reported.

Results

The doctors' effects on patients' physical health outcomes were reported as fixed effects, identifying high and low outliers, or random effects, which estimate the variation in patient health outcomes due to the doctor after accounting for all available variables via the intra-class correlation coefficient. Multivariable multilevel regression is commonly used to adjust for patient risk, doctor experience and other demographics, and also to account for the clustering effect of hospitals in estimating both fixed and random effects.

Conclusion

This methodological review identified inconsistencies in how the doctor's effect on patients' physical health outcomes is measured and reported. For grading doctors from worst to best performances and estimating random effects, specific recommendations are given along with the specific data points to report.

Is There a Surgeons’ Effect on Patients’ Physical Health, Beyond the Intervention, That Requires Further Investigation? A Systematic Review

Objective

To find and review published papers researching surgeons’ effects on patients’ physical health. Clinical outcomes of surgery patients with similar prognoses cannot be fully explained by surgeon skill or experience. Just as there are “hospital” and “psychotherapist” effects, there may be “surgeons” effects that persist after controlling for known variables like patient health and operation riskiness.

Methods

Cohort studies and randomized controlled trials (RCTs) of any surgical intervention, which, after multivariate adjustment, either showed proportion of variance in patients’ physical health outcomes due to surgeons (random effects) or graded surgeons from best to worst (fixed effects). Studies with <15 surgeons or only ascribing surgeons’ effects to known variables excluded. Medline, PubMed, Embase, and PsycINFO were used for search until June 2020. Manual search for papers referring/referred by resulting studies. Risk of bias assessed by Cochrane risk-of-bias tool and Newcastle–Ottawa Scale.

Results

Included studies: 52 cohort studies and three RCTs of 52,436+ surgeons covering 102 outcomes (33 unique). Studies either graded surgeons from best to worst or calculated the intra-class correlation coefficient (ICC), the percentage of patients’ variation due to surgeons, in diverse ways. Sixteen studies showed exceptionally good and/or bad performers with confidence intervals wholly above or below the average performance. ICCs ranged from 0 to 47%, median 4.0%. There are no well-established reporting standards; highly heterogeneous reporting, therefore no meta-analysis.

Discussion

Interpretation: There is a surgeons' effect on patients’ physical health for many types of surgeries and outcomes, ranging from small to substantial. Surgeons with exceptional patient outcomes appear regularly even after accounting for all known confounding variables. Many existing cohort studies and RCTs could be reanalyzed for surgeons’ effects especially after methodological reporting guidelines are published.

Conclusion

In terms of patient outcomes, it can matter which surgeon is chosen. Surgeons with exceptional patient outcomes are worth studying further.

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

https://youtu.be/pL-eGyAGhSk

Patient, Surgeon, or Hospital: Explaining Variation in Outcomes after Colectomy

BACKGROUND

Complication rates after colectomy remain high. Previous work has failed to establish the relative contribution of patient comorbidities, surgeon performance, and hospital systems in the development of complications after elective colectomy.

STUDY DESIGN

We identified all patients undergoing elective colectomy between 2012 and 2018 at hospitals participating in the Michigan Surgical Quality Collaborative. The primary outcome was development of a postoperative complication. We used risk- and reliability-adjusted generalized linear mixed models to estimate the degree to which variance in patient-, surgeon-, and hospital-level factors contribute to complications.

RESULTS

A total of 15,755 patients were included in the study. The mean hospital-level complication rate was 15.8% (range, 8.7% to 30.2%). The proportion of variance attributable to the patient level was 35.0%, 2.4% was attributable to the surgeon level, and 1.8% was attributable to the hospital level. The predicted probability of complication for the least comorbid patient was 1.5% (CI 0.7-3.1%) at the highest performing hospital with the highest performing surgeon, and 6.6% (CI 3.2-12.2%) at the lowest performing hospital with the lowest performing surgeon. By contrast, the most comorbid patient in the cohort had a 66.3% (CI 39.5-85.6%) or 89.4% (CI 73.7-96.2%) risk of complication.

CONCLUSIONS

This study demonstrated that variance from measured factors at the patient level contributed more than 8-fold more to the development of complications after colectomy compared with variance at the surgeon and hospital level, highlighting the impact of patient comorbidities on postoperative outcomes. These results underscore the importance of initiatives that optimize patient foundational health to improve surgical care.

How to Establish Benchmarks for Surgical Outcomes?: A Checklist Based on an International Expert Delphi Consensus

OBJECTIVE

To define a standardized methodology for establishing benchmarks for relevant outcomes in surgery.

SUMMARY BACKGROUND DATA

Benchmarking is an established tool to improve quality in industry and economics, and is emerging in assessing outcome values in surgery. Despite a recent 10-step approach to identify such benchmark values, a standardized and more widely agreed-on approach is still lacking.

METHODS

A multinational web-based Delphi survey with a focus on methodological requirements for establishing benchmarks for surgical outcomes was performed. Participants were selected among internationally renowned specialists in abdominal, vascular, and thoracic surgery. Consensus was defined as ≥70% agreement and results were used to develop a checklist to establish benchmarks in surgery.

RESULTS

Forty-one surgical opinion leaders from 19 countries and 5 continents were involved. Experts' response rates were 98% and 80% in rounds 1 and 2, respectively. Upon completion of the final Delphi round, consensus was successfully achieved for 26 of 36 items covering the following areas: center eligibility, validation of databases, patient cohort selection, procedure selection, duration of follow-up, statistical analysis, and publication requirements regarding center-specific outcomes.

CONCLUSIONS

This multinational Delphi survey represents the first expert-led process for developing a standardized approach for establishing benchmarks for relevant outcome measures in surgery. The provided consensual checklist customizes the methodology of outcome reporting in surgery and thus improves reproducibility and comparability of data and should ultimately serve to improve quality of care.

Association Between Surgeon Technical Skills and Patient Outcomes

Importance

Postoperative complications remain common after surgery, but little is known about the extent of variation in operative technical skill and whether variation is associated with patient outcomes.

Objectives

To examine the (1) variation in technical skill scores of practicing surgeons, (2) association between technical skills and patient outcomes, and (3) amount of variation in patient outcomes explained by a surgeon's technical skill.

Design, Setting, and Participants

In this quality improvement study, 17 practicing surgeons submitted a video of a laparoscopic right hemicolectomy that was then rated by at least 10 blinded peer surgeons and 2 expert raters. The association between surgeon technical skill scores and risk-adjusted outcomes was examined using data from the American College of Surgeons National Surgical Quality Improvement Program. The association between technical skill scores and outcomes was examined for colorectal procedures and noncolorectal procedures (ie, assessed on whether technical skills demonstrated during colectomy were associated with patient outcomes across other cases). In addition, the proportion of patient outcomes explained by technical skill scores was examined using robust regression techniques. The study was conducted from September 23, 2016, to February 10, 2018; data analysis was performed from November 2018 to January 2019.

Exposures

Colorectal and noncolorectal procedures.

Main Outcomes and Measures

Any complication, mortality, unplanned hospital readmission, unplanned reoperation related to principal procedure, surgical site infection, and death or serious morbidity.

Results

Of the 17 surgeons included in the study, 13 were men (76%). The participants had a range from 1 to 28 years in surgical practice (median, 11 years). Based on 10 or more reviewers per video and with a maximum quality score of 5, overall technical skill scores ranged from 2.8 to 4.6. From 2014 to 2016, study participants performed a total of 3063 procedures (1120 colectomies). Higher technical skill scores were significantly associated with lower rates of any complication (15.5% vs 20.6%, P = .03; Spearman rank-order correlation coefficient r = -0.54, P = .03), unplanned reoperation (4.7% vs 7.2%, P = .02; r = -0.60, P = .01), and a composite measure of death or serious morbidity (15.9% vs 21.4%, P = .02; r = -0.60, P = .01) following colectomy. Similar associations were found between colectomy technical skill scores and patient outcomes for all types of procedures performed by a surgeon. Overall, technical skill scores appeared to account for 25.8% of the variation in postcolectomy complication rates and 27.5% of the variation when including noncolectomy complication rates.

Conclusions and Relevance

The findings of this study suggest that there is wide variation in technical skill among practicing surgeons, accounting for more than 25% of the variation in patient outcomes. Higher colectomy technical skill scores appear to be associated with lower complication rates for colectomy and for all other procedures performed by a surgeon. Efforts to improve surgeon technical skills may result in better patient outcomes.

The Effect of Feedback on Surgeon Performance: A Narrative Review

Surgeons play a critical role in the healthcare community and provide a service that can tremendously impact patients' livelihood. However, there are relatively few means for monitoring surgeons' performance quality and seeking improvement. Surgeon-level data provide an important metric for quality improvement and future training. A narrative review was conducted to analyze the utility of providing surgeons direct feedback on their individual performance. The articles selected identified means of collecting surgeon-specific data, suggested ways to report this information, identified pertinent gaps in the field, and concluded the results of giving feedback to surgeons. There is a relative sparsity of data pertaining to the effect of providing surgeons with information regarding their individual performance. However, the literature available does suggest that providing surgeons with individualized feedback can help make meaningful improvements in the quality of practice and can be done in a way that is safe for the surgeons' reputation.

Association of Coworker Reports About Unprofessional Behavior by Surgeons With Surgical Complications in Their Patients

Importance

For surgical teams, high reliability and optimal performance depend on effective communication, mutual respect, and continuous situational awareness. Surgeons who model unprofessional behaviors may undermine a culture of safety, threaten teamwork, and thereby increase the risk for medical errors and surgical complications.

Objective

To test the hypothesis that patients of surgeons with higher numbers of reports from coworkers about unprofessional behaviors are at greater risk for postoperative complications than patients whose surgeons generate fewer coworker reports.

Design, Setting, and Participants

This retrospective cohort study assessed data from 2 geographically diverse academic medical centers that participated in the National Surgical Quality Improvement Program (NSQIP) and recorded and acted on electronic reports of safety events from coworkers describing unprofessional behavior by surgeons. Patients included in the NSQIP database who underwent inpatient or outpatient operations at 1 of the 2 participating sites from January 1, 2012, through December 31, 2016, were eligible. Patients were excluded if they were younger than 18 years on the date of the operation or if the attending surgeon had less than 36 months of monitoring for coworker reports preceding the date of the operation. Data were analyzed from August 8, 2018, through April 9, 2019.

Exposures

Coworker reports about unprofessional behavior by the surgeon in the 36 months preceding the date of the operation.

Main Outcomes and Measures

Postoperative surgical or medical complications, as defined by the NSQIP, within 30 days of the operation.

Results

Among 13 653 patients in the cohort (54.0% [7368 ] female; mean [SD] age, 57 [16] years) who underwent operations performed by 202 surgeons (70.8% [143] male), 1583 (11.6%) experienced a complication, including 825 surgical (6.0%) and 1070 medical (7.8%) complications. Patients whose surgeons had more coworker reports were significantly more likely to experience any complication (0 reports, 954 of 8916 [10.7%]; ≥4 reports, 294 of 2087 [14.1%]; P < .001), any surgical complication (0 reports, 516 of 8916 [5.8%]; ≥4 reports, 159 of 2087 [7.6%]; P < .01), or any medical complication (0 reports, 634 of 8916 [7.1%]; ≥4 reports, 196 of 2087 [9.4%]; P < .001). The adjusted complication rate was 14.3% higher for patients whose surgeons had 1 to 3 reports and 11.9% higher for patients whose surgeons had 4 or more reports compared with patients whose surgeons had no coworker reports (P = .05).

Conclusions and Relevance

Patients whose surgeons had higher numbers of coworker reports about unprofessional behavior in the 36 months before the patient's operation appeared to be at increased risk of surgical and medical complications. These findings suggest that organizations interested in ensuring optimal patient outcomes should focus on addressing surgeons whose behavior toward other medical professionals may increase patients' risk for adverse outcomes.