Attainment of surgical competence in otolaryngology training

Quality of Office-based Procedure Training During Laryngology Fellowship

OBJECTIVES

The primary objective was to assess the perspectives of recent laryngology fellowship graduates on office-based procedure training, with a secondary objective to compare this with previous research on perspectives of fellowship directors.

METHODS

Recent laryngology fellowship graduates were surveyed via an online survey platform regarding post-fellowship practice and various aspects of office-based procedure training, including perceived competence, mentorship, and barriers.

RESULTS

There were 51 respondents. Seventy-six percent of respondents felt they "definitely" received adequate office procedure training. Number of procedures as primary surgeon was significantly associated with perception of adequate training (OR 1.54, 95% CI: 1.08-2.19, p = 0.018) and high post-fellowship office procedure volume (OR 1.56, 95% CI: 1.02-2.39, p = 0.040). Fellows reported a lower percentage of procedures as primary surgeons compared with program directors (46.8% vs. 61.9%, p = 0.028). Fellows and directors agreed that informal debriefs were more commonly employed than more structured training elements such as checklists and simulators. Of nine office procedures, laryngeal electromyography, KTP laser, and transnasal esophagoscopy had the greatest decreases in practice after training.

CONCLUSION

Although most recent laryngology fellowship graduates endorse adequate office-based procedure training, a range of individual experiences exists, and office procedure volume, both overall and across individual procedures, may decrease after fellowship. Fellows performing office procedures as primary surgeons may be linked to perceived quality of training and post-fellowship volume.

LEVEL OF EVIDENCE

NA Laryngoscope, 134:1802-1806, 2024.

Bias in Surgical Residency Evaluations: A Scoping Review

OBJECTIVE

Given widespread disparities in the surgical workforce and the advent of competency-based training models that rely on objective evaluations of resident performance, this review aims to describe the landscape of bias in the evaluation methods of residents in surgical training programs in the United States.

DESIGN

A scoping review was conducted within PubMed, Embase, Web of Science, and ERIC in May 2022, without a date restriction. Studies were screened and reviewed in duplicate by 3 reviewers. Data were described descriptively.

SETTING/PARTICIPANTS

English-language studies conducted in the United States that assessed bias in the evaluation of surgical residents were included.

RESULTS

The search yielded 1641 studies, of which 53 met inclusion criteria. Of the included studies, 26 (49.1%) were retrospective cohort studies, 25 (47.2%) were cross-sectional studies, and 2 (3.8%) were prospective cohort studies. The majority included general surgery residents (n = 30, 56.6%) and nonstandardized examination modalities (n = 38, 71.7%), such as video-based skills evaluations (n = 5, 13.2%). The most common performance metric evaluated was operative skill (n = 22, 41.5%). Overall, the majority of studies demonstrated bias (n = 38, 73.6%) and most investigated gender bias (n = 46, 86.8%). Most studies reported disadvantages for female trainees regarding standardized examinations (80.0%), self-evaluations (73.7%), and program-level evaluations (71.4%). Four studies (7.6%) assessed racial bias, of which all reported disadvantages for trainees underrepresented in surgery.

CONCLUSIONS

Evaluation methods for surgery residents may be prone to bias, particularly with regard to female trainees. Research is warranted regarding other implicit and explicit biases, such as racial bias, as well as for nongeneral surgery subspecialties.

Do Attending and Trainee Surgeons Agree on What Happens in the Operating Room During Septoplasty?

Background: Surgeons must select cases whose complexity aligns with their skill set. Objectives: To determine how accurately trainees report involvement in procedures, judge case complexity, and assess their own skills. Methods: We recruited attendings and trainees from two otolaryngology departments. After performing septoplasty, they completed identical surveys regarding case complexity, achievement of goals, who performed which steps, and trainee skill using the septoplasty global assessment tool (SGAT) and visual analog scale (VAS). Agreement regarding which steps were performed by the trainee was assessed with Cohen's kappa coefficients (κ). Correlations between trainee and attending responses were measured with Spearman's correlation coefficients (rho). Results: Seven attendings and 42 trainees completed 181 paired surveys. Trainees and attendings sometimes disagreed about which steps were performed by trainees (range of κ = 0.743-0.846). Correlation between attending and trainee responses was low for VAS skill ratings (range of rho = 0.12-0.34), SGAT questions (range of rho = 0.03-0.53), and evaluation of case complexity (range of rho = 0.24-0.48). Conclusion: Trainees sometimes disagree with attendings about which septoplasty steps they perform and are limited in their ability to judge complexity, goals, and their skill.

Assessment of operative competency for thyroidectomy: Comparison of resident self-assessment vs attending surgeon assessment

BACKGROUND

Self-awareness of skill, essential for progression as a surgeon, has direct bearing on postresidency practice; however, studies have supported that residents achieve self-perceived competence later than believed by program directors. This study compares residents' self-perception of operative competency to attending surgeon's evaluation using Computer Enhanced Visual Learning, a validated online hemithyroidectomy-specific assessment tool.

METHODS

Eleven otolaryngology-Head and Neck Surgery (HNS) residents completed a preoperative module and postoperative survey, later reviewed by an attending surgeon. Eighty-three performances were assessed for inter-rater reliabilities of key surgical steps.

RESULTS

Almost perfect agreement (Kappa = 0.81-1.00) was shown in 11 of 18 parameters. Substantial agreement (Kappa = 0.61-0.80) was demonstrated in the remaining seven parameters.

CONCLUSIONS

Otolaryngology-HNS trainees have high self-awareness of their performance at each step in a hemithyroidectomy. Standardized assessment tools can allow for documentation of procedural performance and serve as guides for improvement. This is the only study to examine otolaryngology-HNS trainees' self-perceived skill compared to an attending surgeon's assessment for hemithyroidectomy.

Completion of an Individualized Learning Plan for Otology-Related Milestone Subcompetencies Leads to Improved Otology Section Otolaryngology Training Exam Scores

OBJECTIVE

To examine the relationships among self-assessment of knowledge in otology via an individualized learning plan (ILP), otology milestone achievement rate, and otolaryngology training exam (OTE) otology scores.

STUDY DESIGN

Prospective study.

SETTING

One otolaryngology residency covering a tertiary care facility, trauma and hospital center, outpatient ambulatory surgery center, and outpatient clinics.

PARTICIPANTS

Twenty otolaryngology residents, four from each class.

METHODS

Residents identified four milestones from otology-related sub-competencies to achieve in a 3-month rotation via an ILP. During the same rotation, the residents sat for the OTE, and their overall and otology scores were analyzed.

MAIN OUTCOME MEASURES

Completion of an ILP before and at the end of the rotation, self-reported achievement of otology milestones, and OTE score components including total percent correct, scaled score, group stanine, national stanine, and residency group weighted scores.

RESULTS

Group stanine OTE otology scores were higher for those residents who completed pre- and post-rotation ILPs compared with those who did not, 4.0 (±0.348) versus 2.75 (±0.453), respectively (p = 0.04). Residents who self-reported achieving all four otology milestones had significantly higher otology group stanine scores than the residents who achieved less, 4.1 (±0.348) versus 2.9 ± 0.433, respectively (p = 0.045). Residents who performed well in their PGY program cohort on the otology OTE 1 year were less inclined to complete an ILP for otology in the subsequent year (Pearson correlation -0.528, p = 0.035).

CONCLUSION

In the otology subspecialty, residents who completed ILPs scored better on OTE examinations independent of resident class. Consequently, programs may find ILPs useful in other otolaryngology subspecialties and across residencies.

Assessments of Otolaryngology Resident Operative Experiences Using Mobile Technology: A Pilot Study

Objectives

Surgical education has shifted from the Halstedian model of “see one, do one, teach one” to a competency-based model of training. Otolaryngology residency programs can benefit from a fast and simple system to assess residents’ surgical skills. In this quality initiative, we hypothesize that a novel smartphone application called System for Improving and Measuring Procedural Learning (SIMPL) could be applied in an otolaryngology residency to facilitate the assessment of resident operative experiences.

Methods

The Plan Do Study Act method of quality improvement was used. After researching tools of surgical assessment and trialing SIMPL in a resident-attending pair, we piloted SIMPL across an otolaryngology residency program. Faculty and residents were trained to use SIMPL to rate resident operative performance and autonomy with a previously validated Zwisch Scale.

Results

Residents (n = 23) and faculty (n = 17) were trained to use SIMPL using a standardized curriculum. A total of 833 assessments were completed from December 1, 2017, to June 30, 2018. Attendings completed a median 20 assessments, and residents completed a median 14 self-assessments. All evaluations were resident initiated, and attendings had a 78% median response rate. Evaluations took residents a median 22 seconds to complete; 126 unique procedures were logged, representing all 14 key indicator cases for otolaryngology.

Discussion

This is the first residency-wide application of a mobile platform to track the operative experiences of otolaryngology residents.

Implications for Practice

We adapted and implemented a novel assessment tool in a large otolaryngology program. Future multicenter studies will benchmark resident operative experiences nationwide.

Contemporary analysis of otolaryngic allergy

OBJECTIVES

Investigate the current trends in otolaryngic allergy (OA).

STUDY DESIGN

Cross-sectional survey.

METHODS

Survey of active AAOA membership.

RESULTS

Response rate was 27.3%. Regional response distribution rates were proportional to the AAOA membership distribution within the United States (R2 = 0.95; P < 0.001), with no significant regional response bias demonstrated (P = 0.428). Self-reported duration to OA competency was 5.8 years. Physicians reporting OA competency were more likely to be board-certified (P < 0.001) and have AAOA fellowship status (P < 0.001). The AAOA was reported to be the most valuable educational resource toward achieving OA competency, with residency training being least valuable (P < 0.001). 91.5% of respondents' practices offered OA services. Subcutaneous injections predominated at twice the utilization of sublingual immunotherapy. Allergy immunotherapy tablets were rarely utilized. Home allergy shots were offered by 45.2% of respondents. In-office immunotherapy vial compounding was preferred (95.8%) to third-party vendors. 94% of AAOA respondents identified patient compliance to be an issue within OA. Non-inhalant allergy service integration included food allergy (63.5%), asthma (44.9%), allergic fungal sinusitis (43.8%), penicillin allergy (18%), stinging insect allergy (12.6%), and aspirin desensitization (3.9%).

CONCLUSION

Reported duration to OA competency after residency was surprisingly long. Further investigation of current and future educational/clinical training is warranted given clinical integration reported for OA. Otolaryngology is in the unique position to develop a comprehensive sino-allergy evidence-based strategy integrating the extensive diagnostic and medical treatment arms alongside the surgical expertise of the specialty within a clinical sino-allergy home concept.

LEVEL OF EVIDENCE

5 Laryngoscope, 130:283-289, 2020.

Individualized Learning Plan (ILP) Is an Effective Tool in Assessing Achievement of Otology-related Subcompetency Milestones

OBJECTIVE

To investigate the individualized learning plan (ILP) as a tool in assessment of residents' milestone achievements as they pertain to Otology subcompetencies: Chronic Ear Disease, Pediatric Otitis Media, and Hearing Loss.

STUDY DESIGN

Prospective study.

METHODS

Twenty otolaryngology residents were instructed to use an ILP and identify six milestones from three otology-related subcompetencies to focus on during the course of a 3-month rotation. They were also asked to plan out specific activities which would help them achieve these milestones, to specify whether or not they successfully achieved them, by what instructional or learning methods and to identify any barriers. The completed ILPs were reviewed by a faculty member.

MAIN OUTCOME MEASURES

The effectiveness of the ILP was assessed by response compliance rate, corroboration of self-reported milestone achievement with faculty evaluations and the ability to set attainable milestones.

RESULTS

There was 95% compliance in using an ILP to achieve milestones. Self-reported milestone scores corresponded to the faculty evaluations in a large majority (89.6%) of patients, and tended to be underestimated by the residents. Out of 114 total milestones identified, 44 (38.6%) were not achieved, with particular overestimation in the use of independent study as a learning method.

CONCLUSION

The ILP is an effective tool in measuring residents' achievement of otology-related milestones, and could possibly be used to supplement or replace faculty assessment. The ILP provides valuable information on barriers to achieving milestones and informs trainees on how to set attainable goals as they pertain to patient care and medical knowledge in otology.

Open tracheostomy training: a nationwide survey among Otolaryngology-Head and Neck Surgery residents

The aim of this study was to examine the training methods and needs of Otolaryngology-Head and Neck Surgery (OTO-HNS) residents to independently perform open tracheostomy (OT). An anonymous 26-items questionnaire pertaining to OT teaching aspects was distributed to all 93 Israeli OTO-HNS residents during March-June 2016. Residents were categorized as 'juniors,' if they were in their post-graduate year (PGY)-1 and PGY-2; 'mid-residency' (PGY-3 and PGY-4); or 'seniors' (PGY-5 and PGY-6). Response rate was 74% (n = 69). There were 25 'juniors' (36%), 24 'mid-residency' (35%) and 20 'seniors' (29%). Overall, the responses of the 3 groups were similar. Forty-seven (68%) residents estimated that there are ≥ 50 tracheostomies/year in their hospital, which roughly corresponds to an exposure of ~ 8 tracheostomies/year/resident. There was an inconsistency between the number of teaching hours given and the number of hours requested for OT training (23% received ≥ 5 h, but 82% declared they needed ≥ 5 h). Eighty-two percentage reported that their main training was conducted during surgery with peer residents or senior physicians. Forty-five (65%) feel competent to perform OT, including juniors. Due to the need to perform OT in urgent scenarios, the competency of OTO-HNS resident is crucial. Training for OT in Israeli OTO-HNS residency programs is not well structured. Yet, residents reported they feel confident to perform OT, already in the beginning of their residency. Planned educational programs to improve OT training should be done in the beginning of the residency and may include designated 'hands-on' platforms; objective periodic surgical competence assessments; and specialist's feedback, using structured assessment forms.

Declining Otolaryngology Resident Training Experience in Tracheostomies: Case Log Trends from 2005 to 2015

Objective

To evaluate recent tracheostomy surgical experience among otolaryngology residents and general surgery residents.

Study Design

Retrospective database review.

Setting

Accreditation Council for Graduate Medical Education otolaryngology and general surgery programs.

Subjects and Methods

Accreditation Council for Graduate Medical Education case log data from 2005 to 2015 for resident graduates in otolaryngology and general surgery were used to obtain mean graduate tracheostomy numbers, mean graduate composite case numbers, and number of graduating residents. Market share for each specialty was estimated through the derived metric of nationwide total tracheostomy graduate experience, calculated by multiplying the number of graduating residents by the mean number of graduate tracheostomies. Linear regression analysis was used to calculate trends. Multiple linear regression analysis was used for pairwise comparison of trends.

Results

From 2005 to 2015, mean graduate tracheostomy numbers for otolaryngology residents declined 2.3% per year, while those for general surgery residents increased 1.8% per year. Accounting for changes in number of resident graduates, market share of tracheostomy decreased 1.0% per year for otolaryngology and increased 3.0% per year for general surgery. Mean graduate composite case numbers increased significantly by 1.8% and 1.0% per year for otolaryngology and general surgery residents, respectively.

Conclusion

Tracheostomy case volume in otolaryngology residency has decreased steadily in comparison with general surgery residency. However, current otolaryngology graduates have more experience in tracheostomy when compared with general surgery graduates. While otolaryngology residents have excellent exposure to tracheostomy, otolaryngology programs should be made aware of this declining trend as well as changing procedural trends, which may affect training needs.

Does resident involvement in thyroid surgery lead to increased postoperative complications?

OBJECTIVES/HYPOTHESIS

To evaluate the impact of resident involvement during thyroid surgery on 30-day postoperative complications.

STUDY DESIGN

Retrospective cohort study.

METHODS

All patients who underwent thyroid surgery in 2011 were identified from the American College of Surgeons National Surgical Quality Improvement Program database. Patient demographics, perioperative details, resident involvement in surgery, and 30-day postoperative complications were extracted. Propensity score analysis was used to match resident and nonresident cases. Univariate and multivariate analysis were performed to determine the relationship between resident involvement in thyroid surgery and postoperative outcomes.

RESULTS

One thousand seven hundred forty-seven patients with and 1,747 patients without resident involvement were case-matched for patient demographics, perioperative variables, and surgical case type. There was no significant difference (P = .19) in 30-day postoperative complication rates of surgeries with and without resident involvement, which were 1.4% and 2%, respectively. Operative time was longer in surgeries with residents than those without residents (119 ± 67 minutes vs. 102 ± 55 minutes, P < .001). Cases with resident involvement had an unplanned reoperation rate of 0.9%, which was significantly lower than the 2.3% rate of surgeries without residents (P = .001). Multivariate analysis revealed no significant association between resident involvement and overall complications (odds ratio = 0.70; P = .18).

CONCLUSIONS

Resident participation in thyroid surgery was not associated with an increased 30-day postoperative complication rate. These findings demonstrate that patient safety is not adversely affected by resident participation in thyroid surgery.

LEVEL OF EVIDENCE

2C Laryngoscope, 127:1242-1246, 2017.