Evaluation of robotic minimally invasive surgical skills using motion studies

Scene-dependent, feedforward eye gaze metrics can differentiate technical skill levels of trainees in laparoscopic surgery

INTRODUCTION

In laparoscopic surgery, looking in the target areas is an indicator of proficiency. However, gaze behaviors revealing feedforward control (i.e., looking ahead) and their importance have been under-investigated in surgery. This study aims to establish the sensitivity and relative importance of different scene-dependent gaze and motion metrics for estimating trainee proficiency levels in surgical skills.

METHODS

Medical students performed the Fundamentals of Laparoscopic Surgery peg transfer task while recording their gaze on the monitor and tool activities inside the trainer box. Using computer vision and fixation algorithms, five scene-dependent gaze metrics and one tool speed metric were computed for 499 practice trials. Cluster analysis on the six metrics was used to group the trials into different clusters/proficiency levels, and ANOVAs were conducted to test differences between proficiency levels. A Random Forest model was trained to study metric importance at predicting proficiency levels.

RESULTS

Three clusters were identified, corresponding to three proficiency levels. The correspondence between the clusters and proficiency levels was confirmed by differences between completion times (F2,488 = 38.94, p < .001). Further, ANOVAs revealed significant differences between the three levels for all six metrics. The Random Forest model predicted proficiency level with 99% out-of-bag accuracy and revealed that scene-dependent gaze metrics reflecting feedforward behaviors were more important for prediction than the ones reflecting feedback behaviors.

CONCLUSION

Scene-dependent gaze metrics revealed skill levels of trainees more precisely than between experts and novices as suggested in the literature. Further, feedforward gaze metrics appeared to be more important than feedback ones at predicting proficiency.

Evaluation of robotic surgery skills using dynamic time warping

BACKGROUND AND OBJECTIVE

accompanied with the wide acceptance of robot assisted minimally invasive surgery (RMIS), the demand for efficient and objective surgical skills evaluation method is increased. Recently, with the development of medical engineering technology, several evaluation methods have been proposed. Among them, kinematic analysis, an unsupervised and data-based method, has been accepted by many researchers. However, this method is still limited by the number of metrics and unconvinced scoring system. This paper aims to propose a new evaluation method to assess surgical skills efficiently and objectively.

METHODS

this research proposed an efficient and effective surgical skills evaluation algorithm which used the trajectories of instrument tip and dynamic time warping (DTW) to provide trainees with real-time and summative feedback. The optimum trajectories based on 'Therbligs' theory was designed as a template. DTW algorithm was used to align actual trajectories to optimum trajectories with an evaluating indicator designed to emphasize the crucial motion features in surgical skills evaluation. The real-time feedback was obtained through a sliding time window to help trainees improve learning efficiency.

RESULTS

experts (n = 2) and novices (n = 8) were invited to complete the peg transfer tasks and 60 instrument tip trajectories were assessed by the proposed algorithm. Significant differences between different groups were observed (experts' right trajectories versus experts' left trajectories, p = 0.0002; experts' right trajectories versus novices' right trajectories, p = 0.0124). In addition, evaluation results of trajectories with operational mistakes were significantly different from those of others.

CONCLUSIONS

the proposed evaluation method showed its advantages in distinguishing and evaluating surgical performance. Given its ability to evaluate the performance based on kinematic information, the proposed evaluation method can be further developed in the future. Furthermore, because the proposed method can provide real-time feedback, it also has the potential to be a monitoring system in operation room.