Development of an evidence-based training program for laparoscopic hysterectomy on a virtual reality simulator

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review

INTRODUCTION

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.

MATERIAL AND METHODS

A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis.

RESULTS

A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.

CONCLUSIONS

Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

COVID Recovery Laparoscopic Simulation Program for Gynecological Registrars-Trainee Perceptions of Regional Model

STUDY OBJECTIVE

The acquisition of gynecological operating skills can be challenging for trainees given the conflicting demands of clinical work. Alternative models of surgical skill training such as laparoscopic simulation is, therefore, required. This study demonstrates the development of a regional gynecological surgery laparoscopic simulation program and trainee perceptions of such an approach.

DESIGN

An intervention-based cohort study.

SETTING

A regional model based in West Midlands training region.

PATIENTS/PARTICIPANTS

Responses from 64 trainees in the training region who participated in this regional program were included.

INTERVENTIONS

A 3-stream curriculum was developed to deliver key training outcomes as required by the Royal College of Obstetricians and Gynaecologists (RCOG) core curriculum as a component of a COVID Recovery Program. Courses were held in 7 teaching hospitals. Courses consisted of both theory and practical teaching.

MEASUREMENTS

A structured feedback tool was used to collect trainee perceptions of the program. Trainee satisfaction was measured on the Likert scale of 1 to 3. A qualitative thematic analysis was conducted with rank-order analysis of coded free-text responses.

MAIN RESULTS

Overall, the majority of trainees 92% (n = 58/64) were very satisfied with the course. Rank-order analysis demonstrated hands-on-practice to be the key perceived benefit of laparoscopic simulation among basic and intermediate trainees, while feedback on procedural skills was felt most useful among advanced trainees.

CONCLUSION

A regional approach to laparoscopic simulation training is both achievable and acceptable. Trainee perceptions of usefulness are altered by seniority and experience. This should be accounted for in the development of laparoscopic simulation programs.

A haptic laparoscopic trainer based on affine velocity analysis: engineering and preliminary results

BACKGROUND

There is a general agreement upon the importance of acquiring laparoscopic skills outside the operation room through simulation-based training. However, high-fidelity simulators are cost-prohibitive and elicit a high cognitive load, while low-fidelity simulators lack effective feedback. This paper describes a low-fidelity simulator bridging the existing gaps with affine velocity as a new assessment variable. Primary validation results are also presented.

METHODS

Psycho-motor skills and engineering key features have been considered e.g. haptic feedback and complementary assessment variables. Seventy-seven participants tested the simulator (17 expert surgeons, 12 intermediates, 28 inexperienced interns, and 20 novices). The content validity was tested with a 10-point Likert scale and the discriminative power by comparing the four groups' performance over two sessions.

RESULTS

Participants rated the simulator positively, from 7.25 to 7.72 out of 10 (mean, 7.57). Experts and intermediates performed faster with fewer errors (collisions) than inexperienced interns and novices. The affine velocity brought additional differentiations, especially between interns and novices.

CONCLUSION

This affordable haptic simulator makes it possible to learn and train laparoscopic techniques. Self-assessment of basic skills was easily performed with slight additional cost compared to low-fidelity simulators. It could be a good trade-off among the products currently used for surgeons' training.

Performance Assessment for Total Laparoscopic Hysterectomy in the Operating Room: Validity Evidence of a Procedure-specific Rating Scale

STUDY OBJECTIVE

The technical conduct of total laparoscopic hysterectomy (LH) is critical to surgical outcomes. This study explored the validity evidence of an objective scale specific to the assessment of technical skills (H-OSATS) for 7 tasks of an LH with salpingo-oophorectomy procedure performed in the operating room.

DESIGN

Observational cohort study.

SETTING

Two academic hospitals in Marseille and Montpellier, France.

PATIENTS

Three groups of operators (novice, intermediate, and experienced surgeons) were video recorded during their live performances of LH on a simple case. For each group, a dozen unedited videos were obtained for the following tasks: division of the round ligament, division of the infundibulopelvic ligament, creation of the bladder flap, opening of the posterior peritoneum, division of the uterine vessels, colpotomy, and closure of the vault.

INTERVENTIONS

Two qualified raters blindly assessed each video using the H-OSATS rating scale. Inter-rater reliability and test-retest reliability were calculated as measures of internal structure. In a separate round of evaluations, the raters provided a global competent/noncompetent decision for each performance. As a measure of consequential validity, a pass/fail score was set for each task using the contrasting group method.

MEASUREMENTS AND MAIN RESULTS

Three tasks (creation of the bladder flap, colpotomy, and closure of the vault) displayed sound validity evidence: a meaningful total score difference among the 3 groups of experience as well as between the intermediate and experienced surgeons, reliability outcomes of >0.7, and a pass/fail score with a theoretical false-positive rate of <10%.

CONCLUSION

The validity evidence of the H-OSATS rating scale differed for separate evaluations of the 7 tasks. Three tasks (i.e., creation of the bladder flap, colpotomy, and closure of the vault) revealed sound validity evidence, including at the level of the attending surgeon, whereas other tasks were more consistent with low-stakes formative evaluation standards.

Video-based self-assessment enhances laparoscopic skills on a virtual reality simulator: a randomized controlled trial

BACKGROUND

Hysterectomy rates are decreasing in many countries, and virtual reality (VR) simulators bring new training opportunities for residents. As coaching interventions while training on a simulated complex procedure represents a resource challenge, alternative strategies to improve surgical skills must be investigated. We sought to determine whether self-guided learning using a video-based self-assessment (SA) leads to improved surgical skills in laparoscopic hysterectomy (LH) on a VR simulator.

METHODS

Twenty-four gynecology residents from two university hospitals were randomized into an SA group (n = 12) and a Control group (n = 12). Each participant's baseline performance on a validated VR basic task was assessed. Both groups then performed three virtually simulated LHs during which the participants received no guidance nor feedback. Following each LH, the SA group participants rated the video of their own performance using a generic and a procedure-specific rating scale, while the Control group participants watched an LH video demonstration. The LH videos of both groups' participants were blindly reviewed and rated by expert surgeons, using modified Objective Structured Assessment of Technical Skills scores (OSATS). Objective metrics recorded by the VR simulator were also compared.

RESULTS

There was no difference between the groups' baseline performances on the VR basic task. For the first LH, the OSATS-derived scores did not differ between SA and Control groups (9 [7-13] versus 9 [8-14]; p = 0.728). For the third LH, the OSATS-derived scores were higher for the SA group than for the Control group (17 [15-21] versus 15 [11-17], p = 0.039). Between the two groups, the objective metrics did not differ from the first to the third LH.

CONCLUSIONS

The use of a structured video-based SA leads to improved procedural skills in LH on a VR simulator compared to watching benchmark expert performance, in a population of residents with moderate experience in the operating room.

Virtual reality simulation to enhance laparoscopic salpingectomy skills

BACKGROUND

To assess skill enhancement and maintenance by virtual-reality simulation of laparoscopic salpingectomy in gynecologic surgery fellows. Skill acquisition by virtual-reality surgical simulation is an active field of research and technological development. Salpingectomy is one of the first gynecologic surgery techniques taught to fellows that requires accompanied learning.

METHODS

A single-center prospective study was performed in the University of Lyon, France, including 26 junior fellows (≤ 3 semesters' internship) performing laparoscopic salpingectomy exercises on a LapSim® virtual reality simulator. Salpingectomy was performed and timed on 3 trials in session 1 and 3 trials in session 2, at a 3-month interval. Analysis was based on students' subjective assessments and a senior surgeon's objective assessment of skill. Progress between the 2 sessions was assessed on McNemar test and Wilcoxon test for matched series.

RESULTS

26 junior specialist trainees performed all trials. Most performed anterograde salpingectomy, both in session 1 (69 %) and session 2 (86 %). Mean procedure time was significantly shorter in session 2: 6.10min versus 7.82min (p=0.0003). There was a significant decrease in blood loss between the first trial in session 1 and the last trial in session 2: 167ml versus 70.3ml (p=0.02). Subjective assessment showed a significant decrease in anxiety and significant increase in perceived efficacy, eye-hand coordination and ergonomics. Efficacy, performance quality and speed of execution as assessed by the senior surgeon all improved significantly from trial to trial, while hesitation significantly decreased.

CONCLUSIONS

The study showed that junior trainees improved their surgical skills on a short laparoscopic exercise using a virtual reality simulator. Virtual reality simulation is useful in the early learning curve, accelerating the acquisition of reflexes. Maintaining skill requires simulation sessions at shorter intervals.

Simulator training and residents' first laparoscopic hysterectomy: a randomized controlled trial

BACKGROUND

Hysterectomy rates are decreasing in many countries, and virtual reality simulators bring new opportunities into residents' surgical education. The objective of this study was to evaluate the effect of training in laparoscopic hysterectomy module with virtual reality simulator on surgical outcomes among residents performing their first laparoscopic hysterectomy.

METHODS

This randomized study was carried out at the Department of Obstetrics and Gynecology in Helsinki University Hospital and Hyvinkää Hospital. We recruited twenty residents and randomly signed half of them to train ten times with the laparoscopic hysterectomy module on a virtual reality simulator, while the rest represented the control group. Their first laparoscopic hysterectomy was video recorded and assessed later by using the Objective Structured Assessment of Technical Skills (OSATS) forms and Visual Analog Scale (VAS). The scores and surgical outcomes were compared between the groups.

RESULTS

The mean OSATS score for the Global Rating Scale (GRS) was 17.0 (SD 3.1) in the intervention group and 11.2 (SD 2.4) in the control group (p = 0.002). The mean procedure-specific OSATS score was 20.0 (SD 3.3) and 16.0 (SD 2.8) (p = 0.012), and the mean VAS score was 55.0 (SD 14.8) and 29.9 (SD 14.9) (p = 0.001). Operative time was 144 min in the intervention group and 165 min in the control group, but the difference did not reach statistical significance (p = 0.205). There were no differences between the groups in blood loss or direct complications.

CONCLUSION

Residents training with a virtual reality simulator prior to the first laparoscopic hysterectomy seem to perform better in the actual live operation. Thus, a virtual reality simulator hysterectomy module could be considered as a part of laparoscopic training curriculum.

A scoping review of assessment tools for laparoscopic suturing

BACKGROUND

A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions.

METHODS

Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills.

RESULTS

Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences.

CONCLUSION

Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.

Development of an objective assessment tool for total laparoscopic hysterectomy: A Delphi method among experts and evaluation on a virtual reality simulator

INTRODUCTION

Total Laparoscopic hysterectomy (LH) requires an advanced level of operative skills and training. The aim of this study was to develop an objective scale specific for the assessment of technical skills for LH (H-OSATS) and to demonstrate feasibility of use and validity in a virtual reality setting.

MATERIAL AND METHODS

The scale was developed using a hierarchical task analysis and a panel of international experts. A Delphi method obtained consensus among experts on relevant steps that should be included into the H-OSATS scale for assessment of operative performances. Feasibility of use and validity of the scale were evaluated by reviewing video recordings of LH performed on a virtual reality laparoscopic simulator. Three groups of operators of different levels of experience were assessed in a Marseille teaching hospital (10 novices, 8 intermediates and 8 experienced surgeons). Correlations with scores obtained using a recognised generic global rating tool (OSATS) were calculated.

RESULTS

A total of 76 discrete steps were identified by the hierarchical task analysis. 14 experts completed the two rounds of the Delphi questionnaire. 64 steps reached consensus and were integrated in the scale. During the validation process, median time to rate each video recording was 25 minutes. There was a significant difference between the novice, intermediate and experienced group for total H-OSATS scores (133, 155.9 and 178.25 respectively; p = 0.002). H-OSATS scale demonstrated high inter-rater reliability (intraclass correlation coefficient [ICC] = 0.930; p<0.001) and test retest reliability (ICC = 0.877; p<0.001). High correlations were found between total H-OSATS scores and OSATS scores (rho = 0.928; p<0.001).

CONCLUSION

The H-OSATS scale displayed evidence of validity for assessment of technical performances for LH performed on a virtual reality simulator. The implementation of this scale is expected to facilitate deliberate practice. Next steps should focus on evaluating the validity of the scale in the operating room.