Quality control of resident operative experience: compliance with RRC criteria

The impact of the first wave of the COVID-19 pandemic on the exposure of general surgery trainees to operative procedures

INTRODUCTION

During the COVID-19 pandemic, the redeployment of operating room (OR) staff resulted in a significant ramp-down of elective surgery. To mitigate the negative effects of the pandemic on surgical education, this study was planned to estimate the impact of the first wave of the pandemic on the participation of general surgery residency and fellowship trainees in operative procedures.

METHODS

This study is a retrospective review of all adult general surgery procedures performed at 3 sites of an academic health care network. Cases performed during the first wave of the pandemic (March-June 2020) were compared to the same period of the previous year pre-pandemic (March-June 2019). Trainees were categorized as junior (Post-Graduate-Year [PGY] 1-2), senior (PGY3-5), or fellows (PGY6-7). Operating exposure was defined as (1) the total number of cases attended by at least one trainee and (2) total time spent in the OR by all trainees (hours). The impact of the pandemic was estimated as percentage of baseline (2019).

RESULTS

During the first wave of the pandemic, a total of 914 cases were performed, compared to 1328 in the pre-pandemic period (69%). Junior trainees were more affected than senior trainees with reductions in both case volume (68% versus 78% of baseline attendance) and time (68% versus 77% of baseline operating time). Minimally invasive surgery fellows were most severely affected trainees and colorectal fellows were least affected (14% and 75% of baseline cases, respectively). Participation in emergency surgery cases and surgical oncology cases was relatively preserved (87% and 105% of baseline, respectively).

CONCLUSIONS

The first wave of the COVID-19 pandemic reduced operative exposure for general surgery trainees by approximately 30%. Procedure-specific patterns reflected institutional policies for prioritizing cancer operations and emergency surgeries. These findings may inform the design of remediation activities to mitigate the impact of the pandemic on surgical training.

A thematic review of the use of electronic logbooks for surgical assessment in sub-Saharan Africa

INTRODUCTION

Ensuring that surgical training programmes in low- and middle-income countries (LMICs) provide high quality training, including adequate operative experience, is of crucial importance in meeting the goals set out in the Lancet Global Surgery 2030. Electronic logbooks (eLogbooks) have been adopted to monitor both individual trainee progression and the performance of surgical training programmes.

METHODS

We performed a thematic review of the current evidence base surrounding the use of eLogbooks for the assessment of surgeons in training in sub-Saharan Africa, with a view to identifying the learning to date and areas for future research.

RESULTS

Whilst there are multiple papers highlighting the use of surgical eLogbooks in high-income countries, we identified only three papers which discussed their use in sub-Saharan Africa. Four common themes emerged which related to the use of surgical eLogbooks throughout sub-Saharan Africa: ease of analysis, centralised databases, discrepancies in reporting and technology limitations.

CONCLUSIONS

Robust data to demonstrate trainee progression and the quality of surgical training programmes are of crucial importance in ensuring that surgical training programmes can rapidly scale up to deliver large numbers of well-trained surgical providers to address the unmet patient need in LMICs in the next decade. The limited data on the use of well designed, centralised electronic surgical logbooks indicate that this tool may play an important role in providing key data to underpin these training programmes.

ACGME Case Volume Minimums Decrease the Number of Shoulder and Knee Arthroscopies Performed by Residents

Purpose

The purpose of this study was to examine how the implementation of Accreditation Council of Graduate Medical Education (ACGME) case minimums in 2013 has affected the number of shoulder and knee arthroscopies performed by orthopaedic surgery residents during their clinical training.

Methods

The ACGME orthopaedic surgery case log data from graduation years 2007 to 2019 was used to evaluate the number of shoulder and knee arthroscopies performed. The mean and median number of cases performed per resident were compared for the years prior to implementation of the ACGME case minimum (2007-2012) and after (2013-2019).

Results

The ACGME orthopaedic surgery case minimums resulted in a significant decrease in the mean number of shoulder and knee arthroscopies performed. The mean number of shoulder arthroscopies performed in the years before and after the case minimum requirement were 109.8 and 82.0 (P = .025), respectively. The mean number of knee arthroscopies performed in the years before and after the case minimum requirement were 178.6 and 124.8 (P = .006), respectively. Residents in the tenth percentile of cases performed still met the required ACGME case minimums each year. The mean total of all cases performed in the years before and after the case minimum requirement were 2045.5 and 1699.3 (P = .038), respectively.

Conclusions

The number of shoulder and knee arthroscopies performed by orthopaedic surgery residents significantly decreased after the implementation of ACGME case minimums, which may be due to underreporting of cases.

Clinical Relevance

This study may demonstrate the effect of the implementation of the ACGME case minimums on the number of shoulder and knee arthroscopies performed by orthopaedic surgery residents.

Analysing the Operative Experience of Paediatric Surgical Trainees in Sub-Saharan Africa Using a Web-Based Logbook

BACKGROUND

The expansion of local training programmes is crucial to address the shortages of specialist paediatric surgeons across Sub-Saharan Africa. This study assesses whether the current training programme for paediatric surgery at the College of Surgeons of East, Central and Southern Africa (COSECSA) is exposing trainees to adequate numbers and types of surgical procedures, as defined by local and international guidelines.

METHODS

Using data from the COSECSA web-based logbook, we retrospectively analysed numbers and types of operations carried out by paediatric surgical trainees at each stage of training between 2015 and 2019, comparing results with indicative case numbers from regional (COSECSA) and international (Joint Commission on Surgical Training) guidelines.

RESULTS

A total of 7,616 paediatric surgical operations were recorded by 15 trainees, at different stages of training, working across five countries in Sub-Saharan Africa. Each trainee recorded a median number of 456 operations (range 56-1111), with operative experience increasing between the first and final year of training. The most commonly recorded operation was inguinal hernia (n = 1051, 13.8%). Trainees performed the majority (n = 5607, 73.6%) of operations recorded in the eLogbook themselves, assisting in the remainder. Trainees exceeded both local and international recommended case numbers for general surgical procedures, with little exposure to sub-specialities.

CONCLUSIONS

Trainees obtain a wide experience in common and general paediatric surgical procedures, the number of which increases during training. Post-certification may be required for those who wish to sub-specialise. The data from the logbook are useful in identifying individuals who may require additional experience and centres which should be offering increased levels of supervised surgical exposure.

Objective scoring of an electronic surgical logbook: Analysis of impact and observations within a surgical training body

BACKGROUND

Historically, evaluating operative-volumes has proven difficult due to mass-variability in operative-complexities and participation. This study aimed to introduce a national scoring interface for residents' operative-logs while forming meaningful observations on specialities, training-institutes and technical competency.

METHODS

A weighted-scoring algorithm was applied prospectively to residents' operative volumes since July 8th^, 2013 with daily web-based quantitative feedback. Pre and post intervention analyses were performed with historical volumes. Operative volumes were correlated with work-based and university technical-skills' assessments.

RESULTS

Ninety-five residents completed two-year preliminary training since 2013 recording 79,490 operations. These residents recorded significant (p < 0.050) increases in mean-score (case-load), total, performed and assisted operations of >16,528 (50%), 234 (45%), 115 (66%) and 113 (33%) respectively. The number of resident-performed operations was a significant predictor of performance in work-based and university technical-skills assessments (p < 0.050). There were no associations between these measures and the volume of assisted-operations.

CONCLUSIONS

Open-benchmarking of surgical-volumes stimulates residents to actively pursue operative-opportunities and record those experiences. It provides objective performance data on residents and training-institutes while providing evidence that level of operative participation is significant in technical skills development.

A survey of resident perspectives on surgical case minimums and the impact on milestones, graduation, credentialing, and preparation for practice: AOA critical issues

Residency education continues to evolve. Several major changes have occurred in the past several years, including emphasis on core competencies, duty-hour restrictions, and call. The Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System (NAS) implemented educational milestones in orthopaedic surgery in July 2013. Additionally, the Residency Review Committee for orthopaedic surgery published suggested surgical case minimums in 2012, which overlap with several of the milestones.We conducted a survey to assess the opinions of orthopaedic residents regarding the ACGME-suggested surgical case minimums and the effects that these may have on resident education and potential future privileges in hospitals. The survey was sent via e-mail to all of the residents participating in the American Orthopaedic Association (AOA) Resident Leadership Forum for both 2011 and 2012. Participants in the Resident Leadership Forum are in either postgraduate year 4 or postgraduate year 5, are selected by the program directors as resident leaders, and represent 80% of the orthopaedic residency programs in the United States. The survey was completed by 157 of the 314 participants. Sixty-nine percent of the participants believed that case logs with minimum numbers of surgical procedures were an effective way to monitor the work but were not necessarily the only way to monitor the educational progress of the residents. Thirty-two percent believed that the minimums should not be required. Overwhelmingly, there was agreement that important cases were missing from the currently proposed sixteen core surgical minimums. Specifically, the residents believed that a minimum number of cases are necessary for distal radial fracture fixation and proximal humeral fracture fixation and possibly have a milestone to reflect the progress of the residents for each fixation.Most residents thought that surgical case minimums are an effective tool in monitoring the progress of residents and measuring the effectiveness of residency programs. However, the surgical ability of an individual resident should not be evaluated on case minimums alone. The development of the milestones to assess competency should continue, but, as surgical skill is not a specific core competency, perhaps other methods for assessing surgical proficiency need to be developed rather than case minimums. Surgical skills laboratories and proctoring residents independently performing procedures may help to assess surgical proficiency, in addition to traditional faculty and 360° evaluations. Combining these types of assessments with surgical case logs documenting the residents' educational experience seems to be the best path going forward in assessing the development of young surgeons.

Higher surgical training opportunities in the general hospital setting; getting the balance right

BACKGROUND

The general hospital can play an important role in training of higher surgical trainees (HSTs) in Ireland and abroad. Training opportunities in such a setting have not been closely analysed to date.

AIMS

The aim of this study was to quantify operative exposure for HSTs over a 5-year period in a single institution.

METHODS

Analysis of electronic training logbooks (over a 5-year period, 2007-2012) was performed for general surgery trainees on the higher surgical training programme in Ireland. The most commonly performed adult and paediatric procedures per trainee, per year were analysed.

RESULTS

Standard general surgery operations such as herniae (average 58, range 32-86) and cholecystectomy (average 60, range 49-72) ranked highly in each logbook. The most frequently performed emergency operations were appendicectomy (average 45, range 33-53) and laparotomy for acute abdomen (average 48, range 10-79). Paediatric surgical experience included appendicectomy, circumcision, orchidopexy and hernia/hydrocoele repair. Overall, the procedure most commonly performed in the adult setting was endoscopy, with each trainee recording an average of 116 (range 98-132) oesophagogastroduodenoscopies and 284 (range 227-354) colonoscopies.

CONCLUSIONS

General hospitals continue to play a major role in the training of higher surgical trainees. Analysis of the electronic logbooks over a 5-year period reveals the high volume of procedures available to trainees in a non-specialist centre. Such training opportunities are invaluable in the context of changing work practices and limited resources.

Effect of the new standards for case logging on resident operative volume: doing better cases or better numbers?

OBJECTIVES

The Accreditation Council for Graduate Medical Education (ACGME) modified the designation of major (index) operative cases to include those previously considered "minor." This study assessed the potential effect of these changes on resident operative experience.

METHODS

With Institutional Review Board approval, we analyzed National Surgical Quality Improvement Program participant use files for 2005-2008 for general and vascular surgery cases. Primary CPT case coding was mapped to the ACGME major case category using both the old and new classification schemes. The variables were analyzed using χ(2) analysis in SPSS IBM 19 (IBM, Armonk, New York).

RESULTS

A total of 576,019 cases were reviewed. Major cases as defined by the new classification represented an increasing proportion of the cases each year, rising from 88.3% in 2005 to 95% by 2008 (p < 0.001). Major cases as defined by the old scheme decreased from 71% in 2005 to 62% by 2008 (p < 0.001). The cases covered by a resident dropped from 82% in 2005 to 61% in 2008 (p < 0.001). When comparing the new to the old scheme, 364,366 (63.3%) cases were considered major and 30,587 (5.3%) were minor by both standards; 7089 (1.2%) cases previously classified as major were changed to minor, whereas 173,977 (30.2%) (p < 0.001) previously classified as minor were now major. This latter group showed top procedures to include excision of breast lesion (22,175 [12.7%]), laparoscopic gastric bypass (18,825 [10.8%]), ventral hernia repair (14,732 [8.5%]), and appendectomy (10,190 [5.9%]). Of these newly designated major cases, the proportion not covered by residents increased from 22% in 2005 to 44% in 2007 and 2008 (p < 0.001).

CONCLUSIONS

Although some operative cases newly classified as major are technically advanced procedures (eg, Roux-en-Y gastric bypass), other cases are not (eg, breast lesion excision), which raises the issue as to whether the major case category has been diluted by less demanding case types. The implications of these findings may suggest preservation of case volumes at the expense of case quality.

Local and National Trends in General Surgery Residents’ Operative Experience: Do Work Hour Limitations Negatively Affect Case Volume in Small Community-Based Programs?

The goals of this study were to analyze the impact of work hour restrictions on the operative case volume at a small community-based general surgery residency training program and compare changes with the national level. Annual national resident case log data from Accreditation Council for Graduate Medical Education (ACGME) website and case logs of graduating Easton Hospital residents (years 2002-2009) were used for analysis. Weighted average change in total number of cases in our institution was —1.20 ( P = 0.52) vs 1.78 ( P = 0.07) for the national program average with statistically significant difference on comparison ( P = 0.027). We also found significant difference in case volume changes at the national level compared with our institution for the following ACGME defined subcategories: alimentary tract [8.19 ( P < 0.01) vs -1.08 ( P = 0.54)], abdomen [8.48 ( P < 0.01) vs -6.29 ( P < 0.01)], breast [1.91 ( P = 0.89) vs -3.6 ( P = 0.02)], and vascular [4.03 ( P = 0.02) vs -3.98 ( P = 0.01)]. Comparing the national trend to the community hospital we see that there is total increase in cases at the national level whereas there is a decrease in case volume at the community hospital. These trends can also be followed in ACGME defined subcategories which form the major case load for a general surgical training such as alimentary tract, abdominal, breast, and vascular procedures. We hypothesize that work hour restrictions have been favorable for the larger programs, as these programs were able to better integrate the night float system, restructure their call schedule, and implement institutional modifications which are too resource demanding for smaller training programs.

Analysing the operative experience of basic surgical trainees in Ireland using a web-based logbook

BACKGROUND

There is concern about the adequacy of operative exposure in surgical training programmes, in the context of changing work practices. We aimed to quantify the operative exposure of all trainees on the National Basic Surgical Training (BST) programme in Ireland and compare the results with arbitrary training targets.

METHODS

Retrospective analysis of data obtained from a web-based logbook (http://www.elogbook.org) for all general surgery and orthopaedic training posts between July 2007 and June 2009.

RESULTS

104 trainees recorded 23,918 operations between two 6-month general surgery posts. The most common general surgery operation performed was simple skin excision with trainees performing an average of 19.7 (± 9.9) over the 2-year training programme. Trainees most frequently assisted with cholecystectomy with an average of 16.0 (± 11.0) per trainee. Comparison of trainee operative experience to arbitrary training targets found that 2-38% of trainees achieved the targets for 9 emergency index operations and 24-90% of trainees achieved the targets for 8 index elective operations. 72 trainees also completed a 6-month post in orthopaedics and recorded 7,551 operations. The most common orthopaedic operation that trainees performed was removal of metal, with an average of 2.90 (± 3.27) per trainee. The most common orthopaedic operation that trainees assisted with was total hip replacement, with an average of 10.46 (± 6.21) per trainee.

CONCLUSIONS

A centralised web-based logbook provides valuable data to analyse training programme performance. Analysis of logbooks raises concerns about operative experience at junior trainee level. The provision of adequate operative exposure for trainees should be a key performance indicator for training programmes.

Operative Experience in the Era of Duty Hour Restrictions: Is Broad-Based General Surgery Training Coming to an End?

Since the institution of Accreditation Council for Graduate Medical Education (ACGME) resident work hour restrictions, conflicting evidence exists regarding the impact on resident case volume with most data from single-institution studies. We examined the effect of restrictions on national resident operative experience. After permission from the ACGME, we reviewed the publicly available national resident case log data (1999 through 2008) maintained on the ACGME web site ( www.acgme.org ), including total major cases with subanalysis of the ACGME-specified categories. The mean cases per resident were compared before (1999 to 2003) and after (2003 to 2008) restrictions. After the implementation of duty hour restrictions, the mean number of total cases per resident significantly decreased (949 ± 18 vs 911 ± 14, P = 0.004). Subanalysis showed a significant increase in alimentary tract (217 ± 7 vs 229 ± 3, P = 0.004), skin/soft tissue (31 ± 3 vs 36 ± 1, P = 0.01), and endocrine (26 ± 2 vs 31 ± 2, P = 0.006) cases. However, we observed a significant decrease in head and neck (21 ± 0.3 vs 20 ± 0.3, P = 0.01), vascular (164 ± 29 vs 126 ± 5, P = 0.01), pediatric (41 ± 1 vs 37 ± 2, P = 0.006), genitourinary (10 ± 2 vs 7 ± 1, P = 0.004), gynecologic surgery (5 ± 1 vs 3 ± 0.6, P = 0.002), plastics (16 ± 0.3 vs 15 ± 0.7, P = 0.03), and endoscopy (91 ± 3 vs 82 ± 2, P < 0.001) procedures. Analysis of the ACGME-compiled national data confirms that duty hour restrictions have significantly impacted resident operative experience. Importantly, experience in specialty areas, including vascular and endoscopy, appears to have been sacrificed for consolidation of resources into general surgery services as indicated by the increase in alimentary tract and endocrine cases. These findings raise the following question: Is the era of truly broad-based general surgery training coming to an end?

Use of the Surgical Council on Resident Education (SCORE) curriculum as a template for evaluating and planning a program's clinical curriculum

BACKGROUND

The SCORE curriculum defines surgical operations/procedures that residents are expected to be competent with by the end of the residency.

OBJECTIVE

The purpose of this study was to conduct a gap analysis to determine how well the operative experience in a general surgery residency program approximates the expectations of the SCORE curriculum, especially regarding those procedures considered essential to general surgical training.

SETTING/PARTICIPANTS

Final ACGME resident operative experience reports of recent Northwestern University general surgery program graduates (n = 15) were compared with the specific procedures and procedure levels (ie, Essential-Common, Essential-Uncommon, Complex) defined in the SCORE curriculum. The average numbers of individual SCORE procedures and procedures per SCORE procedure level performed per resident were summarized using descriptive statistics.

RESULTS

During their 5 years of training general surgery residents logged a mean of 1025.7 (SD 152.9) primary procedures per resident. We were able to match 87.1% of these ACGME logged procedures with specific procedures identified in the SCORE curriculum. On average, of the Essential-common procedures, 23 (35%) were performed >10 times and 35 (53%) were performed >five times. Conversely, the number of Essential-uncommon and Complex procedures performed >five times were 3 (5%) and 10 (7%), respectively. Several procedures identified in the SCORE curriculum were performed at very low frequency during residency training.

CONCLUSIONS

This experience suggests that leadership at SCORE and the ACGME need to make the curriculum and logging system compatible and that surgical residents need to be better educated with regards to case logging. Despite these issues, important differences appeared to exist between actual resident operative experiences and expectations set by the SCORE curriculum. Based on these finding we advocate that similar gap analyses be performed at other surgical residency training programs to identify discrepancies between program experience and SCORE curriculum expectations.

Effect of the 80-hour work week on resident operative experience in general surgery

BACKGROUND

The goal of this study was to determine the effect of the 80-hour work week on resident operative experience.

METHODS

General surgery resident operative experience was evaluated during a 4-year period and divided into 2 groups: before (group A [July 1, 2001, to June 30, 2003]) and after (group B July 1, 2003, to June 30, 2005]) implementation of the Accreditation Council for Graduate Medical Education duty hour guidelines.

RESULTS

There was a significant decrease in mean total and primary surgeon cases in group B for postgraduate year (PGY) levels 1, 2, and 4 (P < or = .001). There was a significant decrease in PGY 5 teaching assistant and PGY 1 first assistant experience in group B (P < or = .001). There was no difference in PGY 3 resident operative volume.

CONCLUSIONS

The mandated work-hour guidelines have negatively impacted the operative experience of general surgery residents, especially at the junior level. Despite implementing modifications designed to optimize resident operative experience, surgical training programs may require further adaptations.

Effect of work-hour reforms on operative case volume of surgical residents

PURPOSE

There has been much concern regarding the impact of work-hour reforms on the operative case volume of surgical residents. Operative case volume by PGY year and clinical rotation were examined to determine if changes in work hours affected residents' operative case volume.

METHODS

A careful but aggressive plan of work-hour reduction was devised for the residency of the authors' institution with the goal to decrease work hours while maintaining optimal patient care and resident education, including operative case volume. Changes made included hiring physician extenders (PEs), decreasing call schedules to every fourth night (with the next day free from clinical activities-Q4) or call from home (HC), and night float rotation coverage for services (NF). Case volume before (academic year 2002) and after (academic year 2003) changes were compared by PGY year, for all residents and for specific rotations-private general surgery, which had changes of PE, HC, and NF for PGY5; PE, Q4 and NF for PGY1 and 10% exemption for work hours; Churchill service (a resident-run ward teaching service), which had changes of PE and Q4 for PGY5 and PGY1 and 10% exemption for work hours; and vascular surgery, which had HC and NF for PGY5. Total case volume on these services was likewise compared. Statistical analysis was by student t test. Operative case volume was measured with data from the resident-entered information on the ACGME Surgical Operative Log (SOL). Case volume for PGY4 residents could not be compared over this time period because of lack of access to archived data by PGY year for graduated residents through the ACGME SOL. Work hours before and after rotation changes were measured with an intranet-based monitoring system. This article is a retrospective review of the affects of these work-hour changes on operative case volume.

RESULTS

Total case volume for the general surgical services (both private and Churchill) was unchanged over this period (5905 in 02, 5930 in 03), and likewise for the vascular service (1101 vs 1196). Overall, there was no change in mean operative volume per year for surgical residents in this program (231 cases in 2002, 246 cases in 2003; p = 0.61). For PGY5 residents, the case volume increased; 339 cases 02, 390 in 03, and p = 0.05. Mean case volume for PGY5 residents increased on the private general surgery service (136 in 02, 160 in 03, p = 0.03), but it remained stable on the Churchill service (137 in 02, 158 in 03, p = 0.39) and vascular service (65 in 02, 73 in 03, p = 0.42). For PGY3 residents, case volume remained stable (171 in 02, 187 in 03, p = 0.62), as it did for PGY2 and PGY1 residents (PGY2: 148 in 02, 121 in 03, p = 0.12; PGY1: 265 in 02, 246 in 03, p = 0.23). However, operative case volume for PGY1 residents did decrease on the private general surgery service (mean 52 cases per month 02, 43 cases per month 03, p = 0.07), while remaining stable on the Churchill service (mean 23 cases per month 02, 25 cases per month 03, p = 0.66). Average hours worked per week decreased significantly over the time period. Before work-hour reforms, residents' average work hours were as follows: PGY1 105, PGY2 97, PGY3 78.7, PGY4 111, and PGY5 92. After the changes, average work hours were PGY1 81.5, PGY2 77.7, PGY3 78.7, PGY4 75.5, and PGY5 75.9.

CONCLUSIONS

Work-hour limitation can be devised to maximize resident education, optimize patient care, and maintain resident operative volume. Although some changes (HC, PE, NF) seemed to increase the operative case volume for PGY5 residents, others had no effect (Q4, HC). There does not seem to be a clear relationship between types of changes and case volume. At the PGY1 level, Q4 and PE changes decreased operative experience on 1 rotation but not on another, although the difference in this decrease seems clinically insignificant. Individualization of changes to meet the needs of specific rotations seems more important than specific changes in coverage pattern. Perhaps the most important finding is that changes can be made to bring work hours into compliance without materially effecting operative case volume.