Teaching Vaginal Hysterectomy via Simulation: Creation and Validation of the Objective Skills Assessment Tool for Simulated Vaginal Hysterectomy on a Task Trainer and Performance Among Different Levels of Trainees

Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal - a validation study

OBJECTIVE

The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning.

DESIGN

This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor™ virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process.

RESULTS

Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach's alpha = 0.14, p = 0.15) and across the three modules (Cronbach's alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences.

CONCLUSION

Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.

Validation of Transvaginal Hysterectomy Surgical Model - Modification of the Flowerpot Model to Improve Vesicovaginal Plane Simulation

STUDY OBJECTIVE

To establish face and construct validity for a novel variation of American College of Obstetrics and Gynecology "Flowerpot Model" for transvaginal hysterectomy (TVH) surgical simulation with improved vesicovaginal dissection during surgical education simulation.

DESIGN

Cross-sectional face and construct validation study using the "Flowerpot Model." The vesicovaginal dissection plane was modified to include additional felt and balloon materials to simulate the bladder.

SETTING

Single academic center.

PARTICIPANTS

Fourteen residents and fellows, postgraduate year (PGY) 2 to 6, subdivided into junior (n = 8) with ≤10 prior TVH surgeries and senior groups (n = 6) with >10 prior TVH surgeries performed.

INTERVENTIONS

All subjects watched a brief introductory video and then were filmed simulating a TVH.

MEASUREMENTS AND MAIN RESULTS

For face validity, subjects completed an anatomic checklist and pre/post simulation satisfaction survey. For construct validation, 2 independent, blinded expert surgeons (M.A. and J.M.) graded films using the Global Rating Scale of Operative Performance (GRS). Primary outcome was mean GRS between groups. The junior group consisted of PGY 2 to 3 with ≤ 10 prior TVH, median 7.5 (interquartile range [IQR] 6.75) and senior group PGY 3 to 6 with >10 TVH, median 19 (IQR 10) (p <.01). Subjects were "satisfied" or "very satisfied" with bladder and anterior peritoneal fold simulation (92%) and found vesicovaginal dissection "realistic" (100%). GRS score was significantly different between groups (juniors, 19.5 [IQR 5] vs seniors, 28.5 [IQR 8.5]; p = .048). Intergrader correlation was high (ρ = 0.87, p <.01). Surgeon volume of prior TVH was not significantly correlated to average GRS score, ρ = 0.49 (p = .10). The model improved comfort and confidence scores in the junior group more than senior group (p = .04), but senior group still had higher post simulation confidence scores than the junior group (p = .02).

CONCLUSION

Face and construct validity with the modified Flowerpot Model was demonstrated. This low fidelity model is capable of simulation of a TVH with a novel vesicovaginal dissection. Prior surgical experience was not correlated to GRS score or time to procedure completion.

Multi-institutional Validation of a Vaginal Hysterectomy Simulation Model for Resident Training

STUDY OBJECTIVE

The purpose of the research was to both develop a vaginal hysterectomy model with surgically pertinent anatomic landmarks and assess its validity for simulation training.

DESIGN

A low-cost, reproducible vaginal hysterectomy model with relevant anatomic landmarks for key surgical steps.

SETTING

Nine academic and community-based obstetrics and gynecology residency programs.

PARTICIPANTS

One hundred sixty-nine obstetrics and gynecology residents.

INTERVENTIONS

A vaginal hysterectomy model with surgically pertinent anatomic landmarks was developed and tested for construct validity.

MEASUREMENTS AND MAIN RESULTS

Of the 184 available residents, 169 (91%) participated in this study and performed a vaginal hysterectomy procedure on the described model. The validated objective 7-item global rating scale (GRS) and the 13-item task-specific checklist (TSC) were used as tools to assess performance. The median TSC and GRS scores correlated with year of training, prior experience, and trainee confidence. In addition, the TSC scores also correlated with the GRS scores (p <.001) with regard to performance and resident year of training. Receiver Operator Curves for identification of the residents meeting national residency accreditation minimum numbers for vaginal hysterectomy using the GRS and TSC scores had an area under the curve of 0.89 and 0.83, respectively.

CONCLUSION

This reduced-cost vaginal hysterectomy model offers high construct validity and pertinence for simulation.

Needs Assessment for Lower Urinary Tract Injury Curriculum for FPMRS Fellowships

OBJECTIVE

The aim of this study was to determine the level and types of training Accreditation Council for Graduate Medical Education-accredited programs use for female pelvic medicine and reconstructive surgery (FPMRS) fellows' education on lower urinary tract injuries (LUTIs).

METHODS

Two surveys were developed to assess the need for LUTI curriculum from both program director (PD) and fellow vantages through a multistage process, including review by knowledgeable colleagues, cognitive interviews, and pilot testing. Surveys were distributed in an electronic link via e-mail to graduating fellows and program directors from each of the 58 Accreditation Council for Graduate Medical Education-accredited FPMRS programs.

RESULTS

Thirty-four graduating FPMRS fellows (71%) and 39 FPMRS PDs (67%) completed the survey. Both PDs and fellows responded that both the evaluation and management of LUTI were necessary to FPMRS training. The majority of PDs use a combination of didactics and hands-on learning in the operating room (60% and 71%). Only 40% and 30% incorporate simulation into the curriculum to address LUTI. Graduating fellows report low numbers of procedures to evaluate and manage LUTI. Specifically, only 15% of fellows graduate with greater than 2 ureteral reimplantations and 44% graduate with no minimally invasive abdominal vesicovaginal fistula repairs. The majority of graduating fellows reported feeling prepared to evaluate for LUTI, but nearly one third do not feel ready to independently manage LUTI upon graduation.

CONCLUSIONS

FPMRS PDs and fellows agree that the evaluation and management of LUTI are important; however, most programs use only didactics and hands-on learning in the operating room with extremely low case volumes, leading to decreased proficiency.

Establishing Validity for a Vaginal Hysterectomy Simulation Model for Surgical Skills Assessment

OBJECTIVE

To use the Messick validity framework for a simulation-based assessment of vaginal hysterectomy skills.

METHODS

Video recordings of physicians at different levels of training and experience performing vaginal hysterectomy on a high-fidelity vaginal surgery model were objectively assessed using a modified 10-item Vaginal Surgical Skills Index, a one-item global scale of overall performance, and a pass-fail criterion. Participants included obstetrics and gynecology trainees and faculty from five institutions. Video recordings were independently assessed by expert surgeons blinded to the identities of the study participants.

RESULTS

Fifty surgeons (11 faculty, 39 trainees) were assessed. Experience level correlated strongly with both the modified Vaginal Surgical Skills Index and global scale score, with more experienced participants receiving higher scores (Pearson r=0.81, P<.001; Pearson r=0.74, P<.001). Likewise, surgical experience was also moderately correlated with the modified Vaginal Surgical Skills Index and global scale score (Pearson r=0.55, P<.001; Pearson r=0.58, P<.001). The internal consistency of the modified Vaginal Surgical Skills Index was excellent (Cronbach's alpha=0.97). Interrater reliability of the modified Vaginal Surgical Skills Index and global scale score, as measured by the intraclass correlation coefficient, was moderate to good (0.49-0.95; 0.50-0.87). Using the receiver operating characteristic curve and the pass-fail criterion, a modified Vaginal Surgical Skills Index cutoff score of 27 was found to most accurately (area under the curve 0.951, 95% CI 0.917-0.983) differentiate competent from noncompetent surgeons.

CONCLUSION

We demonstrated validity evidence for using a high-fidelity vaginal surgery model with the modified Vaginal Surgical Skills Index or global scale score to assess vaginal hysterectomy skills.

Teaching learners to raise the roof: a vaginal surgery simulator for apical suspension

INTRODUCTION AND HYPOTHESIS

The objective was to discuss the importance of apical suspension following vaginal hysterectomy and demonstrate a surgical model to aide in educating learners on a variety of apical suspension procedures.

METHODS

Rates of pelvic organ prolapse are not insignificant following hysterectomy. Re-support of the vaginal apex should be performed at the time of hysterectomy in those with or without a diagnosis of prolapse. Exposure to vaginal apical support procedures may be limited owing to declining rates of vaginal hysterectomy and limited trainee work hours. Surgical models are increasingly being used to supplement operating room experience. The model we present was originally developed for hysterectomy, although its design allows for teaching a variety of apical support procedures that incorporate the uterosacral ligament (USL) for support. We demonstrate performing a USL suspension, internal McCall suture, and modified McCall suture using the model.

RESULTS

The model is constructed from readily available supplies, is multi-use, and inexpensive. It allows learners to identify relevant anatomy, understand/visualize surgical steps, and practice suturing technique.

CONCLUSION

Pelvic organ prolapse is common in women, although opportunities to teach apical suspension procedures may be limited. The proposed vaginal surgery simulator can be used to supplement the experience of gynecological surgery trainees with apical suspension procedures.

A live porcine model for robotic sacrocolpopexy training

INTRODUCTION AND HYPOTHESIS

Robotic sacrocolpopexy is an effective and durable technique for pelvic organ prolapse repair. However, the learning curve for this procedure has underscored the need for an effective surgical training module. Given the cost, infection risk, poor tissue compliance, and scarcity of human cadavers, the live porcine model represents a realistic, available, and cost-effective alternative. This article describes a live porcine model for teaching robotic sacrocolpopexy to determine whether it teaches key aspects of live human robotic sacrocolpopexy to the learner.

METHODS

This robotic sacrocolpopexy model was created using the Da Vinci Xi or Si robotic system on domestic pigs under general anesthesia. The main steps of the model include: (1) creating the porcine "cervix" and (2) performing robotic sacrocolpopexy. The model was evaluated with a survey given to 18 board-certified surgeons who attended the training course between December 2016 and April 2018.

RESULTS

All of the participants reported improvements in their economy of motion, tissue handling ability, suturing efficiency, and overall performance of robotic sacrocolpopexy. Furthermore, a majority of participants were likely to incorporate aspects of the model into their practice (88.8%) and recommend the model to colleagues (94.2%).

CONCLUSIONS

The porcine model provides a feasible tool for teaching robotic sacrocolpopexy to physicians.