Using Intraoperative Recordings to Evaluate Surgical Technique and Performance in Mastoidectomy

Surgeon Skill and Perioperative Outcomes in Robot-Assisted Partial Nephrectomy

Importance

Technical skill in complex surgical procedures may affect clinical outcomes, and there is growing interest in understanding the clinical implications of surgeon proficiency levels.

Objectives

To determine whether surgeon scores representing technical skills of robot-assisted kidney surgery are associated with patient outcomes.

Design, Setting, and Participants

This quality improvement study included 10 urological surgeons participating in a surgical collaborative in Michigan from July 2021 to September 2022. Each surgeon submitted up to 7 videos of themselves performing robot-assisted partial nephrectomy. Videos were segmented into 6 key steps, yielding 127 video clips for analysis. Each video clip was deidentified and distributed to at least 3 of the 24 blinded peer surgeons from the collaborative who also perform robot-assisted partial nephrectomy. Reviewers rated technical skill and provided written feedback. Statistical analysis was performed from May 2023 to January 2024.

Main Outcomes and Measures

Reviewers scored each video clip using a validated instrument to assess technical skill for partial nephrectomy on a scale of 1 to 5 (higher scores indicating greater skill). For all submitting surgeons, outcomes from a clinical registry were assessed for length of stay (LOS) greater than 3 days, estimated blood loss (EBL) greater than 500 mL, warm ischemia time (WIT) greater than 30 minutes, positive surgical margin (PSM), 30-day emergency department (ED) visits, and 30-day readmission.

Results

Among the 27 unique surgeons who participated in this study as reviewers and/or individuals performing the procedures, 3 (11%) were female, and the median age was 47 (IQR, 39-52) years. Risk-adjusted outcomes were associated with scores representing surgeon skills. The overall performance score ranged from 3.5 to 4.7 points with a mean (SD) of 4.1 (0.4) points. Greater skill was correlated with significantly lower rates of LOS greater than 3 days (-6.8% [95% CI, -8.3% to -5.2%]), EBL greater than 500 mL (-2.6% [95% CI, -3.0% to -2.1%]), PSM (-8.2% [95% CI, -9.2% to -7.2%]), ED visits (-3.9% [95% CI, -5.0% to -2.8%]), and readmissions (-5.7% [95% CI, -6.9% to -4.6%]) (P < .001 for all). Higher overall score was also associated with higher partial nephrectomy volume (β coefficient, 11.4 [95% CI, 10.0-12.7]; P < .001).

Conclusions and Relevance

In this quality improvement study on video-based evaluation of robot-assisted partial nephrectomy, higher technical skill was associated with lower rates of adverse clinical outcomes. These findings suggest that video-based evaluation plays a role in assessing surgical skill and can be used in quality improvement initiatives to improve patient care.

Bias in Surgical Residency Evaluations: A Scoping Review

OBJECTIVE

Given widespread disparities in the surgical workforce and the advent of competency-based training models that rely on objective evaluations of resident performance, this review aims to describe the landscape of bias in the evaluation methods of residents in surgical training programs in the United States.

DESIGN

A scoping review was conducted within PubMed, Embase, Web of Science, and ERIC in May 2022, without a date restriction. Studies were screened and reviewed in duplicate by 3 reviewers. Data were described descriptively.

SETTING/PARTICIPANTS

English-language studies conducted in the United States that assessed bias in the evaluation of surgical residents were included.

RESULTS

The search yielded 1641 studies, of which 53 met inclusion criteria. Of the included studies, 26 (49.1%) were retrospective cohort studies, 25 (47.2%) were cross-sectional studies, and 2 (3.8%) were prospective cohort studies. The majority included general surgery residents (n = 30, 56.6%) and nonstandardized examination modalities (n = 38, 71.7%), such as video-based skills evaluations (n = 5, 13.2%). The most common performance metric evaluated was operative skill (n = 22, 41.5%). Overall, the majority of studies demonstrated bias (n = 38, 73.6%) and most investigated gender bias (n = 46, 86.8%). Most studies reported disadvantages for female trainees regarding standardized examinations (80.0%), self-evaluations (73.7%), and program-level evaluations (71.4%). Four studies (7.6%) assessed racial bias, of which all reported disadvantages for trainees underrepresented in surgery.

CONCLUSIONS

Evaluation methods for surgery residents may be prone to bias, particularly with regard to female trainees. Research is warranted regarding other implicit and explicit biases, such as racial bias, as well as for nongeneral surgery subspecialties.

Intraoperative video recording in otolaryngology for surgical education: evolution and considerations

BACKGROUND

Otolaryngology is a surgical speciality well suited for the application of intraoperative video recording as an educational tool considering the number procedures within the speciality that utilize digital technology. Intraoperative recording has been utilized in endoscopic surgeries and in evaluating technique in mastoidectomy, myringotomy and grommet insertion. The impact of intra-operative video recording in otolaryngology education is vast in creating access to surgical videos for preparation outside the operating room to individualized coaching and assessment. The purpose of this project is to highlight the role of intraoperative video recording in otolaryngology training and elucidate the challenges and considerations associated with implementation.

METHODS

Related publications between 1999 to 2022 were reviewed from PubMed and Embase databases utilizing search terms "intraoperative videography," "video recording surgery," "otolaryngology," and "surgical education." 109 articles were screened independently by HB and SK, by title and abstract then full text review. 28 articles from the original search and 6 from the secondary reference review were included.

RESULTS

The application of intraoperative video recording is evident in otolaryngology surgeries including endoscopic sinus surgery, laryngeal surgery, and other endoscopic procedures. There have been significant advancements in recording tools, including devices that can capture the surgeon's perspective. The considerations and challenges identified with utilizing this educational tool were categorized into different themes including ethics/consent, regulation, liability, data, technology, and human resources.

CONCLUSION

Intra-operative video recording has been demonstrated to have significant impact within otolaryngology education. It is critical to elucidate the challenges and considerations involved to utilize this educational tool effectively. Future directives will see video-based performance analytics providing comparative metrics to encourage precise coaching of surgical residents.

Video Analysis of Otologic Instrument Movement During Resident Mastoidectomies

OBJECTIVE

To measure surgical instrument movement during resident mastoidectomies and identify metrics that correlate with experience.

STUDY DESIGN

Retrospective case series.

SETTING

Tertiary care center.

SUBJECTS

Ten postgraduate year (PGY) 2, 6 PGY3, 7 PGY4, and 19 PGY5 recordings of mastoidectomy performed by otolaryngology residents.

INTERVENTIONS

One-minute intraoperative recordings of mastoidectomies performed during cochlear implantation were collected. Drill and suction-irrigator motion were analyzed with sports motion tracking software.

MAIN OUTCOME MEASURES

Mean instrument speed, angle, and angular velocity were calculated. Mann-Whitney U tests compared mean instrument metrics between PGY levels. Change in drill speed for seven residents between their PGY2 to PGY5 years was individually analyzed.

RESULTS

Mean drill speed was significantly greater for PGY5 residents compared with PGY2s (2.9 versus 1.8 cm/s, p = 0.001). Compared with PGY2 residents, suction speed was greater as a PGY5 (1.2 versus 0.9 cm/s; p = 0.201) and significantly greater as a PGY4 (1.5 versus 0.9 cm/s, p = 0.039). Of the seven residents individually analyzed, group mean drill speed increased by 0.4 cm/s, yearly.

CONCLUSIONS

Drill and suction-irrigator movement during the second minute of drilling of a cortical mastoidectomy seems to increase with resident level. Objective video analysis is a potential adjunct for differentiating novices from more experienced surgeons and monitoring surgical skills progress.