Assessment of progress and competence in tonsillectomy surgery using cumulative sum analysis

Characterizing individual trainee learning curves in surgical training: Challenges and opportunities

The surgical learning curve is an observable and measurable phenomenon. In the era of competency-based approaches to surgical training, monitoring the trajectory of individual trainee competence attainment could represent a meaningful method of formative and summative assessment. While technology can assist this approach, a number of significant barriers to the implementation of such assessment methods remain, including: accurate data collection, standard setting, and reliable assessment. Translating individual learning curve data into quantifiable case minimum targets in training poses further difficulties, and may not be possible for all procedures, particularly those that are less frequently performed and assessed. In spite of these challenges, significant benefits could be realized through an individualized approach to competency assessment using trainee learning curve data. Tracking competence acquisition against criterion-referenced standards could allow for targeted training and remediation, conforming with modern theories of adult education and empowering trainees to take control of their own learning. Learning curve data could also be used to assess the effects of educational interventions such as simulation-based training on subsequent competence acquisition rates. Ultimately, the individual learning curves of trainees could be used to inform personalised decisions regarding entrustment, credentialing, and certification, allowing training programmes to move beyond minimum operative experience targets as a crude proxy measure of competence.

Defining operative experience targets in surgical training: A systematic review

BACKGROUND

The surgical learning curve is an observable and measurable phenomenon. Operative experience targets are well established as a proxy measure for operative competence in surgical training across jurisdictions. The aim of this study was to critique the available evidence regarding the relationship between operative experience in surgical training and trainee competence.

METHODS

A systematic review of the PubMed, Embase, Web of Science, and Cochrane library databases was conducted in accordance with the Preferred Items for Systematic Reviews and Meta-Analyses guidelines. Articles were sought that defined the relationship between procedural volume in surgical training and trainee competence, proficiency, or mastery. The educational impact of included studies was evaluated using a modified Kirkpatrick model.

RESULTS

Of 3,672 records identified on database searching, 30 papers were ultimately included. Fourteen studies defined operative experience thresholds using operative time as a surrogate measure of competence, whereas another 8 used trainer assessments of operative performance (Kirkpatrick level 3). A further 5 studies were able to determine the relationship between trainee case volumes and subsequent patient outcomes (Kirkpatrick level 4b).

CONCLUSION

Many studies have recorded competent trainee performance in key index procedures after reaching experience threshold numbers in excess of currently mandated targets across jurisdictions. The evidence relating current operative experience targets to patient outcomes across a range of surgical subspecialties of surgical subspecialties is lacking. This review supports a move toward criterion-based referencing of operative performance targets in surgical training.

The learning curve of BiZact™ tonsillectomy

OBJECTIVE

To evaluate the learning curve of BiZact™ tonsillectomy amongst trainees and fellows as a further technical skill.

METHODS

Prospective audit of consecutive BiZact™ tonsillectomies undertaken by a trainee and fellow at Starship Children's Hospital. Primary outcomes measured were operative time from gag open to gag closed, readmission and post tonsillectomy bleed rates. Secondary outcomes included need for bipolar rescue dissection, bipolar haemostasis including site and side, tonsil size (Brodsky grade 1-4) and depth (shallow or deep). Operative time was plotted against number of procedures performed and moving averages was used to investigate the learning curve. Generalised linear model was applied to examine the association between operation time and tonsil depth.

RESULTS

Mean operative time for the trainee and fellow was similar (07 min 34 s versus 07min and 37 s). We observed a trend of decreased operative time over number of procedures performed by the trainee but the learning curve was short. The fellow's initial time was lower than the trainee's and remained similar over time. No bipolar rescue dissection was required. Additional bipolar haemostasis was required 95% of the time, but the majority involved the superior pole only (87.5%, p = 0.0001). The overall operative time for shallow tonsils was shorter than for deep tonsils (06 min 31 s versus 08 min 09 s, p = 0.0066). A learning curve was evident for both shallow and deep tonsils for both surgeons combined but was steeper for deep. There were no primary bleeds and the secondary haemorrhage rate was 5.8% (3/51 cases) with no returns to theatre.

CONCLUSION

This is the first study to investigate the BiZact™ tonsillectomy learning curve. BiZact™ tonsillectomy is safe and easy to learn. We observed a clear trend of shallow tonsils being quicker to remove than deep tonsils using this technique. The secondary bleed rate of 5.8% is comparable to the previously published Starship rate of 4.7% for all tonsillectomies performed over a 10 year period. Moving averages is a useful method to assess operative learning curves during training and beyond.

Does length of time between cases affect resident operative time for tonsillectomy and adenoidectomy?

OBJECTIVE

To evaluate the effect of prolonged time intervals between tonsillectomy and adenoidectomy (TA) on resident operative time and complications.

STUDY DESIGN

Retrospective cohort.

SETTING

Tertiary academic hospital.

METHODS

This retrospective study covers a five-year period from 2015 to 2020. Time intervals between isolated pediatric TA cases performed by eight otolaryngology residents were reviewed to assess effect on operative time (defined as prolonged if ≥ 30 min and non-prolonged if < 30 min). Intervals including a procedure involving either a tonsillectomy or adenoidectomy that was a non-isolated TA were excluded.

RESULTS

A total of 309 isolated TAs were identified with 67.3% of procedures performed under 30 min. The mean surgical time interval between procedures was 5.83 ± 10.02 days (range 0.02-69.82). Most TAs were performed on patients aged 7 years or younger. Surgical time interval between TA was not a significant factor in determining prolonged operative time on univariable logistic regression, OR 1.01 (CI: 0.98 to 1.03) (p = 0.63). Patient age at surgery, adenoid grade, tonsil size and total number of TAs performed to date were significant factors in determining prolonged operative time in both univariable and multivariable logistic regression models. Prolonged operative time did not have a significant effect on readmission, reoperation, or post-operative bleeding.

CONCLUSION

Extended time interval (up to 3 months) between routine TA does not affect operative time. Expansion of our methodology to more complex cases would be beneficial in designing resident training curriculum.

Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence

PURPOSE

Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries.

METHOD

Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries.

RESULTS

From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified.

CONCLUSIONS

This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.