Classification of Intermediate and Novice Surgeons' Skill Assessment Through Performance Metrics

Background. Endoscopic surgeries have become an alternative for open procedures whenever possible. For such types of operations, surgeons are required to gain several skills, whose development needs hands-on practice. Accordingly, gaining these skills today is a challenge for surgical education programs. Despite the development of several technology-enhanced training environments, there are still problems to better integrate these technologies into educational programs. For an appropriate integration, it is critical to assess the skill levels and adapt the training content according to the trainees' requirements. In the literature, there exist several methods for assessing these skill levels. However, there are still problems in practice for objective and repetitive assessment. Methods. The present study aims to estimate the skill levels of participants in surgical training programs in an objective manner by collecting experimental data from residents in an endoscopic surgical simulation environment and gathering performance metrics. Results. It is shown that, by comparing the results of a number of classification algorithms for the best accuracy estimation and feature set, the "novice" and "intermediate" skill levels can be estimated with an accuracy of 86%. Conclusions. The outcomes help surgical educators and instructional system designers to better assess the skill levels of the trainees and guide them accordingly. In addition, objective assessments as highlighted in this study can be beneficial when designing technology-enhanced adaptive learning environments.

Simulation-Based Training Platforms for Arthroscopy: A Randomized Comparison of Virtual Reality Learning to Benchtop Learning

PURPOSE

To determine whether a virtual reality (VR) arthroscopy simulator or benchtop (BT) arthroscopy simulator showed superiority as a training tool.

METHODS

Arthroscopic novices were randomized to a training program on a BT or a VR knee arthroscopy simulator. The VR simulator provided user performance feedback. Individuals performed a diagnostic arthroscopy on both simulators before and after the training program. Performance was assessed using wireless objective motion analysis and a global rating scale.

RESULTS

The groups (8 in the VR group, 9 in the BT group) were well matched at baseline across all parameters (P > .05). Training on each simulator resulted in significant performance improvements across all parameters (P < .05). BT training conferred a significant improvement in all parameters when trainees were reassessed on the VR simulator (P < .05). In contrast, VR training did not confer improvement in performance when trainees were reassessed on the BT simulator (P > .05). BT-trained subjects outperformed VR-trained subjects in all parameters during final assessments on the BT simulator (P < .05). There was no difference in objective performance between VR-trained and BT-trained subjects on final VR simulator wireless objective motion analysis assessment (P > .05).

CONCLUSIONS

Both simulators delivered improvements in arthroscopic skills. BT training led to skills that readily transferred to the VR simulator. Skills acquired after VR training did not transfer as readily to the BT simulator. Despite trainees receiving automated metric feedback from the VR simulator, the results suggest a greater gain in psychomotor skills for BT training. Further work is required to determine if this finding persists in the operating room.

CLINICAL RELEVANCE

This study suggests that there are differences in skills acquired on different simulators and skills learnt on some simulators may be more transferable. Further work in identifying user feedback metrics that enhance learning is also required.