Validation of Motion Tracking Software for Evaluation of Surgical Performance in Laparoscopic Cholecystectomy

Utility of visualization and quantification of surgical techniques using motion analysis software for thoracoscopic surgery

In this era of endoscopic surgery, feedback from recorded surgical videos is useful and efficient; therefore, effective methods of obtaining this feedback are needed. We analyzed surgical videos using motion analysis software and verified the usefulness of visualizing and objectively evaluating surgical procedures. We measured the grasping and traction angles of the vascular sheath when using forceps and the trajectory of the forceps tip for the upper pulmonary vein during right upper lobectomy during video-assisted thoracoscopic surgery performed by three trainers and trainees. Compared with the trainers, the trainees exhibited insufficient traction of the vascular sheath, performed many slow and unnecessary manipulations, and sometimes performed sudden and fast movements. By visualizing the surgical procedures, the trainee will be better able to identify dangerous or futile movements. It may also make it easier to objectively recognize improvements in one's technique. Motion analysis software could allow for efficient surgical education and self-learning.

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.

Quantifying the Impact of Signal-to-background Ratios on Surgical Discrimination of Fluorescent Lesions

PURPOSE

Surgical fluorescence guidance has gained popularity in various settings, e.g., minimally invasive robot-assisted laparoscopic surgery. In pursuit of novel receptor-targeted tracers, the field of fluorescence-guided surgery is currently moving toward increasingly lower signal intensities. This highlights the importance of understanding the impact of low fluorescence intensities on clinical decision making. This study uses kinematics to investigate the impact of signal-to-background ratios (SBR) on surgical performance.

METHODS

Using a custom grid exercise containing hidden fluorescent targets, a da Vinci Xi robot with Firefly fluorescence endoscope and ProGrasp and Maryland forceps instruments, we studied how the participants' (N = 16) actions were influenced by the fluorescent SBR. To monitor the surgeon's actions, the surgical instrument tip was tracked using a custom video-based tracking framework. The digitized instrument tracks were then subjected to multi-parametric kinematic analysis, allowing for the isolation of various metrics (e.g., velocity, jerkiness, tortuosity). These were incorporated in scores for dexterity (Dx), decision making (DM), overall performance (PS) and proficiency. All were related to the SBR values.

RESULTS

Multi-parametric analysis showed that task completion time, time spent in fluorescence-imaging mode and total pathlength are metrics that are directly related to the SBR. Below SBR 1.5, these values substantially increased, and handling errors became more frequent. The difference in Dx and DM between the targets that gave SBR < 1.50 and SBR > 1.50, indicates that the latter group generally yields a 2.5-fold higher Dx value and a threefold higher DM value. As these values provide the basis for the PS score, proficiency could only be achieved at SBR > 1.55.

CONCLUSION

By tracking the surgical instruments we were able to, for the first time, quantitatively and objectively assess how the instrument positioning is impacted by fluorescent SBR. Our findings suggest that in ideal situations a minimum SBR of 1.5 is required to discriminate fluorescent lesions, a substantially lower value than the SBR 2 often reported in literature.

Video Analysis of Otologic Instrument Movement During Resident Mastoidectomies

OBJECTIVE

To measure surgical instrument movement during resident mastoidectomies and identify metrics that correlate with experience.

STUDY DESIGN

Retrospective case series.

SETTING

Tertiary care center.

SUBJECTS

Ten postgraduate year (PGY) 2, 6 PGY3, 7 PGY4, and 19 PGY5 recordings of mastoidectomy performed by otolaryngology residents.

INTERVENTIONS

One-minute intraoperative recordings of mastoidectomies performed during cochlear implantation were collected. Drill and suction-irrigator motion were analyzed with sports motion tracking software.

MAIN OUTCOME MEASURES

Mean instrument speed, angle, and angular velocity were calculated. Mann-Whitney U tests compared mean instrument metrics between PGY levels. Change in drill speed for seven residents between their PGY2 to PGY5 years was individually analyzed.

RESULTS

Mean drill speed was significantly greater for PGY5 residents compared with PGY2s (2.9 versus 1.8 cm/s, p = 0.001). Compared with PGY2 residents, suction speed was greater as a PGY5 (1.2 versus 0.9 cm/s; p = 0.201) and significantly greater as a PGY4 (1.5 versus 0.9 cm/s, p = 0.039). Of the seven residents individually analyzed, group mean drill speed increased by 0.4 cm/s, yearly.

CONCLUSIONS

Drill and suction-irrigator movement during the second minute of drilling of a cortical mastoidectomy seems to increase with resident level. Objective video analysis is a potential adjunct for differentiating novices from more experienced surgeons and monitoring surgical skills progress.

Value-assessment of computer-assisted navigation strategies during percutaneous needle placement

Purpose

Navigational strategies create a scenario whereby percutaneous needle-based interventions of the liver can be guided using both pre-interventional 3D imaging datasets and dynamic interventional ultrasound (US). To score how such technologies impact the needle placement process, we performed kinematic analysis on different user groups.

Methods

Using a custom biopsy phantom, three consecutive exercises were performed by both novices and experts (n = 26). The exercise came in three options: (1) US-guidance, (2) US-guidance with pre-interventional image-registration (US + Reg) and (3) US-guidance with pre-interventional image-registration and needle-navigation (US + Reg + Nav). The traveled paths of the needle were digitized in 3D. Using custom software algorithms, kinematic metrics were extracted and related to dexterity, decision making indices to obtain overall performance scores (PS).

Results

Kinematic analysis helped quantifying the visual assessment of the needle trajectories. Compared to US-guidance, novices yielded most improvements using Reg (PSavg(US) = 0.43 vs. PSavg(US+Reg) = 0.57 vs. PSavg(US+Reg+Nav) = 0.51). Interestingly, the expert group yielded a reversed trend (PSavg(US) = 0.71 vs PSavg(US+Reg) = 0.58 vs PSavg(US+Reg+Nav) = 0.59).

Conclusion

Digitizing the movement trajectory allowed us to objectively assess the impact of needle-navigation strategies on percutaneous procedures. In particular, our findings suggest that these advanced technologies have a positive impact on the kinematics derived performance of novices.

Application of an Optical Tracking System for Motor Skill Assessment in Laparoscopic Surgery

Objective

Motion analysis of surgical instruments can be used to evaluate laparoscopic surgical skills, and this study assessed the validity of an optical tracking system for the assessment of laparoscopic surgical motor skills.

Methods

Ten experienced surgeons and ten novices were recruited to complete the transferring tasks on a laparoscopic simulator. An optical tracking system, Micron Tracker, was used to capture the marker points on each instrument and to obtain the coordinates of the marker points and the corresponding instrument tip coordinates. The data are processed to create a coordinate system based on the laparoscopic simulator and to calculate the movement parameters of the instruments, such as operating time, path length, speed, acceleration, and smoothness. At the same time, the range of motion of the instrument (insertion depth and pivoting angle) is also calculated.

Results

The position that the tip of the instrument can reach is a small, irregularly shaped spatial area. Significant differences (p < 0.05) were found between the surgeon and novice groups in parameters such as operating time, path length, mean speed, mean acceleration, and mean smoothness. The range of insertion depth of the instruments was approximately 150 mm to 240 mm, and the pivoting angles of the left and right instruments were 30.9° and 46.6° up and down and 28.0° and 35.0° left and right, respectively.

Conclusions

The optical tracking system was effective in subjectively evaluating laparoscopic surgical skills, with significant differences between the surgeon and novice groups in terms of movement parameters, but not in terms of range of motion.

Vision-based Tracking of Surgical Motion during Live Open-Heart Surgery

BACKGROUND

Motion tracking during live surgeries may be used to assess surgeons' intra-operative performance, provide feedback, and predict outcome. Current assessment protocols rely on human observations, controlled laboratory settings, or tracking technologies not suitable for live operating theatres. In this study, a novel method for motion tracking of live open-heart surgery was developed, and evaluated.

MATERIALS AND METHODS

Three-D-printed 'tracking die' with miniature markers were fitted to DeBakey forceps. The surgical field was recorded with a video camera mounted above the operating table. Software was developed for tracking the die from the recordings. The system was tested on five open-heart procedures. Surgeons were asked to report subjective system related concerns during live surgery and assess the weight of the die on blind test. The accuracy of the system was evaluated against ground truth generated by a robot.

RESULTS

The 3D-printed die weighed 6 g and tolerated sterilization with hydrogen peroxide, which added approximately 13% to the mass of the forceps. Surgeons sensed a shift in the balance of the instrument but could on blind test not correctly verify changes in weight. When two or more markers were detected, the 3D position estimate was on average within 2-3 mm, and 1.1-2.6 degrees from ground truth. Computational time was 30-50 ms per frame on a standard laptop.

CONCLUSIONS

The vision-based motion tracking system was applicable for live surgeries with negligible inconvenience to the surgeons. Motion data was extracted with acceptable accuracy and speed at low computational cost.

Using Motion Tracking to Analyze Forceps Paths During Simulated Forceps-Assisted Vaginal Deliveries

OBJECTIVE

The purpose of this study was to evaluate the ability of motion tracking to discern variation in forceps paths during standardized simulated forceps-assisted vaginal deliveries among experienced and inexperienced obstetric providers.

METHODS

This is a pilot study involving 24 obstetrics and gynecology residents and 6 faculty at a single institution. Each participant was filmed performing standardized simulated forceps-assisted vaginal deliveries on a high-fidelity model. Motion tracking software (Kinovea, Medoc, France) was used to track the path of the forceps shank. Data were analyzed for total path length, total x-plane displacement, total y-plane displacement, and final forceps angle. One-way analysis of variance was used to evaluate for statistically significant differences between groups based on education year, with Turkey HSD post hoc test to identify interactions.

RESULTS

Statistically significant differences were noted between groups in the total path length (F = 7.57, P < 0.001) and total y-plane displacement (F = 5.79, P < 0.001). On pairwise comparison, significant differences were noted between faculty and postgraduate year 1 as well as faculty and postgraduate year 2 for total y-plane displacement and total path length. Significant differences were not observed between groups for total x-plane displacement (F = 0.89, P = 0.475) and final forceps angle (F = 2.45, P = 0.052).

CONCLUSIONS

Motion tracking of standardized simulated forceps-assisted vaginal deliveries identifies statistically significant differences between experienced and inexperienced obstetric providers. Our findings suggest that motion tracking can be used to design an educational intervention to improve forceps technique among obstetrics and gynecology residents.