Scoping Evidence Review on Training and Skills Assessment for Open Emergency Surgery

Technologies Used for Telementoring in Open Surgery: A Scoping Review

Background: Telementoring technologies enable a remote mentor to guide a mentee in real-time during surgical procedures. This addresses challenges, such as lack of expertise and limited surgical training/education opportunities in remote locations. This review aims to provide a comprehensive account of these technologies tailored for open surgery. Methods: A comprehensive scoping review of the scientific literature was conducted using PubMed, ScienceDirect, ACM Digital Library, and IEEE Xplore databases. Broad and inclusive searches were done to identify articles reporting telementoring or teleguidance technologies in open surgery. Results: Screening of the search results yielded 43 articles describing surgical telementoring for open approach. The studies were categorized based on the type of open surgery (surgical specialty, surgical procedure, and stage of clinical trial), the telementoring technology used (information transferred between mentor and mentee, devices used for rendering the information), and assessment of the technology (experience level of mentor and mentee, study design, and assessment criteria). Majority of the telementoring technologies focused on trauma-related surgeries and mixed reality headsets were commonly used for rendering information (telestrations, surgical tools, or hand gestures) to the mentee. These technologies were primarily assessed on high-fidelity synthetic phantoms. Conclusions: Despite longer operative time, these telementoring technologies demonstrated clinical viability during open surgeries through improved performance and confidence of the mentee. In general, usage of immersive devices and annotations appears to be promising, although further clinical trials will be required to thoroughly assess its benefits.

Early Evaluation of a New French Surgery Course in the Best Practice of Dealing With Major Incidents and Mass Casualty Events

OBJECTIVE

The main objective of this study is to evaluate the impact of a nationwide 5-month course aimed to prepare surgeons for Major Incidents through the acquisition of key knowledge and competencies. Learners' satisfaction was also measured as a secondary objective.

DESIGN

This course was evaluated thanks to various teaching efficacy metrics, mainly based on Kirkpatrick's hierarchy in medical education. Gain in knowledge of participants was evaluated by multiple-choice tests. Self-reported confidence was measured with 2 detailed pre and post training questionnaires.

SETTING

Creation in 2020 of a nationwide, optional and comprehensive Surgical Training in War and Disaster Situation as part of the French surgery residency program. In 2021, data was gathered regarding the impact of the course on participants' knowledge and competencies.

PARTICIPANTS

The study included 26 students in the 2021 cohort (13 residents and 13 practitioners).

RESULTS

Mean scores were significantly higher in the post-test compared to the pre-test, showing significant increase in participants' knowledge during the course: 73,3% vs. 47,3% respectively (p ≤ 0.001). Average learners' confidence scores to perform technical procedures showed at least a +1-point increase on the Likert scale for 65% of items tested (p ≤ 0.001). 89% of items showed at least a +1-point increase on the Likert scale when it came to average learners' confidence score on dealing with complicated situations (p ≤ 0.001). Our post-training satisfaction survey showed that 92% of all participants have noticed the impact of the course on their daily practice.

CONCLUSION

Our study shows that the third level of Kirkpatrick's hierarchy in medical education was reached. This course therefore appears to be meeting the objectives set by the Ministry of Health. Being only 2 years old, it is on the road to gathering momentum and further development.

A Novel Paradigm for Surgical Skills Training and Assessment of Competency

Importance

Sustainment of comprehensive procedural skills in trauma surgery is a particular problem for surgeons in rural, global, and combat settings. Trauma care often requires open surgical procedures for low-frequency/high-risk injuries at a time when open surgical experience is declining in general and trauma surgery training.

Objective

To determine whether general surgeons participating in a 2-day standardized trauma skills course demonstrate measurable improvement in accuracy and independent performance of specific trauma skills.

Design, Setting, and Participants

General surgeons in active surgical practice were enrolled from a simulation center with anatomic laboratory from October 2019 to October 2020. Differences in pretraining/training and posttraining performance outcomes were examined using (1) pretraining/posttraining surveys, (2) pretraining/posttraining knowledge assessment, and (3) training/posttraining faculty assessment. Analysis took place in November 2020.

Interventions

A 2-day standardized, immersive, cadaver-based skills course, developed with best practices in instructional design, that teaches and assesses 24 trauma surgical procedures was used.

Main Outcomes and Measures

Trauma surgery capability, as measured by confidence, knowledge, abilities, and independent performance of specific trauma surgical procedures; 3-month posttraining skill transfer.

Results

The study cohort included 65 active-duty general surgeons, of which 16 (25%) were women and 49 (75%) were men. The mean (SD) age was 38.5 (4.2) years. Before and during training, 1 of 65 participants (1%) were able to accurately perform all 24 procedures without guidance. After course training, 64 participants (99%) met the benchmark performance requirements for the 24 trauma procedures, and 51 (78%) were able to perform them without guidance. Procedural confidence and knowledge increased significantly from before to after the course. At 3 months after training, 37 participants (56%) reported skill transfer to trauma or other procedures.

Conclusions and Relevance

In this study, direct measurement of procedural performance following standardized training demonstrated significant improvement in skill performance in a broad array of trauma procedures. This model may be useful for assessment of procedural competence in other specialties.

Enhanced Training Benefits of Video Recording Surgery With Automated Hand Motion Analysis

Background

Hand motion analysis by video recording during surgery has potential for evaluation of surgical performance. The aim was to identify how technical skill during open surgery can be measured unobtrusively by video recording during a surgical procedure. We hypothesized that procedural-step timing, hand movements, instrument use and Shannon entropy differ with expertise and training and are concordant with a performance-based validated individual procedure score.

Methods

Surgeon and non-surgeon participants with varying training and levels of expertise were video recorded performing axillary artery exposure and control (AA) on un-preserved cadavers. Color-coded gloves permitted motion-tracking and automated extraction of entropy data from recordings. Timing and instrument-use metrics were obtained through observational video reviews. Shannon entropy measured speed, acceleration and direction by computer-vision algorithms. Findings were compared with individual procedure score for AA performance

Results

Experts had lowest entropy values, idle time, active time and shorter time to divide pectoralis minor, using fewer instruments. Residents improved with training, without reaching expert levels, and showed deterioration 12–18 months later. Individual procedure scores mirrored these results. Non-surgeons differed substantially.

Conclusions

Hand motion entropy and timing metrics discriminate levels of surgical skill and training, and these findings are congruent with individual procedure score evaluations. These measures can be collected using consumer-level cameras and analyzed automatically with free software. Hand motion with video timing data may have widespread application to evaluate resident performance and can contribute to the range of evaluation and testing modalities available to educators, training course designers and surgical quality assurance programs.

Supplementary Information

The online version of this article (10.1007/s00268-020-05916-1) contains supplementary material, which is available to authorized users.