Objective assessment for open surgical suturing training by finger tracking can discriminate novices from experts

It is difficult, time consuming and expensive to assess manual skills in open surgery. The aim of this study is to investigate the construct validity of a low-cost, easily accessible tracking technique for basic open suturing tasks. Medical master students, surgical residents, and surgeons at the Radboud University Medical Center were recruited between September 2020 until September 2021. The participants were divided, according to experience, in a novice group (≤10 sutures performed) and an expert group (>50 sutures performed). For objective tracking, a tablet with SurgTrac software was used, which tracked a blue and a red tag placed on respectively their left and right index finger. The participants executed four basic tasks on a suturing model: 1) knot tying by hand, 2) transcutaneous suturing with an instrument knot, 3) 'Donati' (vertical mattress suture) with an instrument knot and 4) continuous intracutaneous suturing without a knot. In total 76 participants were included: 57 novices and 19 experts. All four tasks showed significant differences between the novice group and expert group for the parameters time (p<0.001), distance (p<0.001 for Task 1, 2 and 3 and p=0.034 for Task 4) and smoothness (p<0.001). Additionally, Task 3 showed a significant difference for the parameter handedness (p=0.006) and Task 4 for speed (p=0.033). Tracking index finger movements using SurgTrac software on a tablet while executing basic open suturing skills on a simulator shows excellent construct validity for time, distance and motion smoothness in all four suturing tasks.

A systematic review on classification and assessment of surgical skill levels for simulation-based training programs

BACKGROUND

Nowadays, advances in medical informatics have made minimally invasive surgery (MIS) procedures the preferred choice. However, there are several problems with the education programs in terms of surgical skill acquisition. For instance, defining and objectively measuring surgical skill levels is a challenging process. Accordingly, the aim of this study is to conduct a literature review for an investigation of the current approaches for classifying the surgical skill levels and for identifying the skill training tools and measurement methods.

MATERIALS AND METHODS

In this research, a search is conducted and a corpus is created. Exclusion and inclusion criteria are applied by limiting the number of articles based on surgical education, training approximations, hand movements, and endoscopic or laparoscopic operations. To satisfy these criteria, 57 articles are included in the corpus of this study.

RESULTS

Currently used surgical skill assessment approaches have been summarized. Results show that various classification approaches for the surgical skill level definitions are being used. Besides, many studies are conducted by omitting particularly important skill levels in between. Additionally, some inconsistencies are also identified across the skill level classification studies.

CONCLUSION

In order to improve the benefits of simulation-based training programs, a standardized interdisciplinary approach should be developed. For this reason, specific to each surgical procedure, the required skills should be identified. Additionally, appropriate measures for assessing these skills, which can be defined in simulation-based MIS training environments, should be refined. Finally, the skill levels gained during the developmental stages of these skills, with their threshold values referencing the identified measures, should be redefined in a standardized manner.

Feasibility of tracking in open surgical simulation

The aim of this study was to develop an adequate tracking method for open surgical training, using tracking of the instrument or hand motions.

An open surgical training model and the SurgTrac application were used to track four separate suturing tasks. These tasks were performed with colour markings of either instruments or fingers, to find the most promising setting for reliable tracking.

Four experiments were used to find the optimal settings for the tracking system. Tracking of instruments was not usable for knot tying by hand. Tracking of fingers seemed to be a more promising method. Tagging the fingers with a coloured balloon-tube, seemed to be a more promising method (1.2–3.0% right hand vs. 9.2–17.9% left hand off-screen) than covering the nails with coloured tape (1.5–3.5% right hand vs. 25.5–55.4% left hand off-screen). However, analysis of the videos showed that redness of the hand was seen as red tagging as well. To prevent misinterpreting of the red tag by redness of the hand, white surgical gloves were worn underneath in the last experiment. The off-screen percentage of the right side decreased from 1.0 to 1.2 without gloves to 0.8 with gloves and the off-screen percentage of the left side decreased from 16.9–17.9 to 6.6–7.2, with an adequate tracking mark on the video images.

This study shows that tagging of the index fingers with a red (right) and blue (left) balloon-tube while wearing surgical gloves is a feasible method for tracking movements during basic open suturing tasks.

Beyond Efficiency: Surface Electromyography Enables Further Insights into the Surgical Movements of Urologists

INTRODUCTION

Surgical skill evaluation while performing minimally invasive surgeries is a highly complex task. It is important to objectively assess an individual's technical skills throughout surgical training to monitor progress and to intervene when skills are not commensurate with the year of training. The miniaturization of wireless wearable platforms integrated with sensor technology has made it possible to non-invasively assess muscle activations and movement variability during performance of minimally invasive surgical tasks. Our objective was to use electromyography to deconstruct the motions of a surgeon during robotic suturing and distinguish quantifiable movements that characterize the skill of an experienced, expert urologic surgeon from trainees.

METHODS

Three skill groups of participants: novice (n=11), intermediate (n=12) and expert (n=3) were enrolled in the study. A total of 12 wireless wearable sensors consisting of surface electromyograms (EMGs) and accelerometers were placed along upper extremity muscles to assess muscle activations and movement variability, respectively. Participants then performed a robotic suturing task.

RESULTS

EMG-based parameters: total time, dominant frequency, cumulative muscular workload (CMW were significantly different across the three skill groups. We also found nonlinear movement variability parameters such as correlation dimension, Lyapunov exponent trended differently across the three skill groups.

CONCLUSIONS

These findings suggest that economy of motion variables and nonlinear movement variabilities are affected by surgical experience level. Wearable sensor signal analysis could make it possible to objectively evaluate surgical skill level periodically throughout the residency training experience.

Vision-based Tracking of Surgical Motion during Live Open-Heart Surgery

BACKGROUND

Motion tracking during live surgeries may be used to assess surgeons' intra-operative performance, provide feedback, and predict outcome. Current assessment protocols rely on human observations, controlled laboratory settings, or tracking technologies not suitable for live operating theatres. In this study, a novel method for motion tracking of live open-heart surgery was developed, and evaluated.

MATERIALS AND METHODS

Three-D-printed 'tracking die' with miniature markers were fitted to DeBakey forceps. The surgical field was recorded with a video camera mounted above the operating table. Software was developed for tracking the die from the recordings. The system was tested on five open-heart procedures. Surgeons were asked to report subjective system related concerns during live surgery and assess the weight of the die on blind test. The accuracy of the system was evaluated against ground truth generated by a robot.

RESULTS

The 3D-printed die weighed 6 g and tolerated sterilization with hydrogen peroxide, which added approximately 13% to the mass of the forceps. Surgeons sensed a shift in the balance of the instrument but could on blind test not correctly verify changes in weight. When two or more markers were detected, the 3D position estimate was on average within 2-3 mm, and 1.1-2.6 degrees from ground truth. Computational time was 30-50 ms per frame on a standard laptop.

CONCLUSIONS

The vision-based motion tracking system was applicable for live surgeries with negligible inconvenience to the surgeons. Motion data was extracted with acceptable accuracy and speed at low computational cost.

Using Haptic Feedback in a Virtual Reality Bone Drilling Simulation to Reduce Plunge Distance

Background Bone drilling is a procedure that demands a high level of dexterity, fine motor skills and spatial awareness from the operating surgeon. An important consideration when drilling bone is minimising soft tissue damage. There are numerous causes of drilling associated soft tissue injury, of which most concerning is drilling into the tissue beyond the far cortex as unseen injury can occur. This is known as plunging. Objectives The objective of this study was to evaluate the impact of haptic feedback in virtual reality (VR) simulation-based training. The acquisition of drilling skill was assessed by changes to their drill plunge depth. Study Design & Methods The participants in the study were medical students, doctors and biomedical scientists. Participants were randomly allocated into two groups. One group had simulation with haptic feedback as part of their VR simulated learning, whereas the second group undertook the same VR simulation but did not receive haptic feedback during the simulation. Following completion of the simulated bone drilling protocol, a bone drilling exercise took place. Each participant was allowed to drill a synthetic tibia bone five times and then the plunge depth was measured. We quantified outcome in the form of plunge depth. Results  There were four participants in each group. The average plunge distance in the group who were able to practice with haptic assisted VR simulation was 46mm (range: 37-56mm), the average plunge distance in the non-haptic group was 79mm (range: 44-136mm). Results showed an average reduction of 33mm in plunge depth from users in the haptic group compared to the non-haptic group. Conclusion  Bone drilling simulation with haptic feedback may be an effective simulator of the motor skills that would be required to perform this action on a live patient. The study results suggest that there could be a reduction in soft tissue damage for users trained in VR simulations with haptic feedback.

Dynamic Thumb Circumduction Measured With a Wearable Motion Sensor: A Prospective Comparison of Patients With Basal Joint Arthritis to Controls

Purpose

The purpose of this study was to compare the active range of motion in patients with thumb carpometacarpal (CMC) arthritis to healthy controls. A secondary objective of this study was to examine the feasibility of using wearable motion sensors in a clinical setting.

Methods

Asymptomatic controls and patients with radiographic and clinical evidence of thumb CMC joint arthritis were recruited. The experimental setup allowed participants to rest their forearm in neutral pronosupination with immobilization of the second through fifth CMC joints. An electromagnetic motion sensor was embedded into a thumb interphalangeal joint immobilizer, and participants were asked to complete continuous thumb circumduction movements. Data were continuously recorded, and circumduction curves were created based on degrees of motion. Peak thumb abduction and extension angles were also extracted from the data.

Results

A total of 29 extremities with thumb CMC arthritis and 18 asymptomatic extremities were analyzed. Bilateral disease was present in 64% of patients. Patient age range was 35–83 years, and the control group age range was 26–83 years. The most affected extremities had Eaton stage 3 disease (38%, N = 11). The average maximum thumb abduction was 53.9° ± 19.6° in affected extremities and 70.8° ± 10.1° for controls. Average maximum thumb extension was 50.0° ± 15.2° in affected extremities and 58.4° ± 9.1° for controls. When comparing patients with Eaton stage 3 and 4 disease to controls, average maximum abduction and extension decreased with increasing disease stage (42.3°, 46.1°, and 70.8° for abduction, respectively, and 58.4°, 43.3°, and 41.3° for extension, respectively).

Conclusions

We observed more severe motion limitations with increasing Eaton stage, and statistically significant differences were seen with stage 3 and 4 disease. A wearable motion sensor using a portable experimental setup was used to obtain measurements in a clinical setting.

Type of study/level of evidence

Diagnostic II.

Towards markerless surgical tool and hand pose estimation

Purpose: 

Tracking of tools and surgical activity is becoming more and more important in the context of computer assisted surgery. In this work, we present a data generation framework, dataset and baseline methods to facilitate further research in the direction of markerless hand and instrument pose estimation in realistic surgical scenarios.

Methods: 

We developed a rendering pipeline to create inexpensive and realistic synthetic data for model pretraining. Subsequently, we propose a pipeline to capture and label real data with hand and object pose ground truth in an experimental setup to gather high-quality real data. We furthermore present three state-of-the-art RGB-based pose estimation baselines.

Results: 

We evaluate three baseline models on the proposed datasets. The best performing baseline achieves an average tool 3D vertex error of 16.7 mm on synthetic data as well as 13.8 mm on real data which is comparable to the state-of-the art in RGB-based hand/object pose estimation.

Conclusion: 

To the best of our knowledge, we propose the first synthetic and real data generation pipelines to generate hand and object pose labels for open surgery. We present three baseline models for RGB based object and object/hand pose estimation based on RGB frames. Our realistic synthetic data generation pipeline may contribute to overcome the data bottleneck in the surgical domain and can easily be transferred to other medical applications.

Supplementary Information

The online version supplementary material available at 10.1007/s11548-021-02369-2.

A Randomized Prospective Blinded Study Evaluating the Effect of Music on Novice Surgical Trainees' Ability to Perform a Simulated Surgical Task

OBJECTIVE

To determine if playing music would affect novice surgical trainees' ability to perform a complex surgical task.

BACKGROUND

The effect of music in the operating room (OR) is controversial. Some studies from the anesthesiology literature suggest that OR music is distracting and should be banned. Other nonblinded studies have indicated that music improves surgeons' efficiency with simple tasks.

DESIGN/METHODS

A prospective, blinded, randomized trial of 19 novice surgical trainees was conducted using an in vitro model. Each trainee performed a baseline vascular anastomosis (VA) without music. Subsequently, they performed one VA with music (song validated to reduce anxiety) and one without, in random order and without prior knowledge of the study's purpose. The primary endpoint was a difference in differences from baseline with and without music with respect to time to completion, acceleration/deceleration (using a previously validated hand-tracking motion device), and video performance scoring (3 blinded experts using a validated scale). The participants completed a poststudy survey to gauge their opinions regarding music during tasks.

RESULTS

Overall, 57 VAs by 19 trainees were evaluated. Average time to completion was 11.6 minutes. When compared to baseline, time to completion improved for both the music group (p = 0.01) and no-music group (p = 0.001). When comparing music to no music, there was no difference in time to completion (p = 0.7), acceleration/deceleration (p = 0.3), or video performance scorings (p = NS). Among participants, 89% responded that they enjoy listening to music while performing tasks.

CONCLUSIONS

Using three outcome measures, relaxing music did not improve the performance of novice surgical trainees performing a complex surgical task, and the music did not make their performance worse. However, nearly all trainees reported enjoying listening to music while performing tasks.

Global versus task-specific postoperative feedback in surgical procedure learning

BACKGROUND

Task-specific checklists and global rating scales are both recommended assessment tools to provide constructive feedback on surgical performance. This study evaluated the most effective feedback tool by comparing the effects of the Observational Clinical Human Reliability Analysis (OCHRA) and the Objective Structured Assessment of Technical Skills (OSATS) on surgical performance in relation to the visual-spatial ability of the learners.

METHODS

In a randomized controlled trial, medical students were allocated to either the OCHRA (n = 25) or OSATS (n = 25) feedback group. Visual-spatial ability was measured by a Mental Rotation Test. Participants performed an open inguinal hernia repair procedure on a simulation model twice. Feedback was provided after the first procedure. Improvement in performance was evaluated blindly using a global rating scale (performance score) and hand-motion analysis (time and path length).

RESULTS

Mean improvement in performance score was not significantly different between the OCHRA and OSATS feedback groups (P = .100). However, mean improvement in time (371.0 ± 223.4 vs 274.6 ± 341.6; P = .027) and path length (53.5 ± 42.4 vs 34.7 ± 39.0; P = .046) was significantly greater in the OCHRA feedback group. When stratified by mental rotation test scores, the greater improvement in time (P = .032) and path length (P = .053) was observed only among individuals with low visual-spatial abilities.

CONCLUSION

A task-specific (OCHRA) feedback is more effective in improving surgical skills in terms of time and path length in novices compared to a global rating scale (OSATS). The effects of a task-specific feedback are present mostly in individuals with lower visual-spatial abilities.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

Surgical skill levels: Classification and analysis using deep neural network model and motion signals

BACKGROUND AND OBJECTIVES

Currently, the assessment of surgical skills relies primarily on the observations of expert surgeons. This may be time-consuming, non-scalable, inconsistent and subjective. Therefore, an automated system that can objectively identify the actual skills level of a junior trainee is highly desirable. This study aims to design an automated surgical skills evaluation system.

METHODS

We propose to use a deep neural network model that can analyze raw surgical motion data with minimal preprocessing. A platform with inertial measurement unit sensors was developed and participants with different levels of surgical experience were recruited to perform core open surgical skills tasks. JIGSAWS a publicly available robot based surgical training dataset was used to evaluate the generalization of our deep network model. 15 participants (4 experts, 4 intermediates and 7 novices) were recruited into the study.

RESULTS

The proposed deep model achieved an accuracy of 98.2%. With comparison to JIGSAWS; our method outperformed some existing approaches with an accuracy of 98.4%, 98.4% and 94.7% for suturing, needle-passing, and knot-tying, respectively. The experimental results demonstrated the applicability of this method in both open surgery and robot-assisted minimally invasive surgery.

CONCLUSIONS

This study demonstrated the potential ability of the proposed deep network model to learn the discriminative features between different surgical skills levels.

Economy of Hand Motion During Cleft Palate Surgery Using a High-Fidelity Cleft Palate Simulator

OBJECTIVE

The objectives of this study were to assess economy of hand motion of residents, fellows, and staff surgeons using a high-fidelity cleft palate simulator to (1) stratify performance for the purpose of simulator validation and (2) to estimate the learning curve.

DESIGN

Two residents, 2 fellows, and 2 staff surgeons performed cleft palate surgery on a high-fidelity cleft palate simulator while their hand motion was tracked using an electromagnetic hand sensor. The time, number of hand movements, and path length of their hands were determined for 10 steps of the procedure. The magnitude of these metrics was compared among the 3 groups of participants and utilized to estimate the learning curve using curve-fitting analysis.

RESULTS

The residents required the most time, number of hand movements, and path length to complete the procedure. Although the number of hand movements was closely matched between the fellows and staff, the overall total path length was shorter for the staff. Inverse curves were fit to the data to represent the learning curve and 25 and 113 simulation sessions are required to reach within 5% and 1% of the expert level, respectively.

CONCLUSION

The simulator successfully stratified performance using economy of hand motion. Path length is better matched to previous level of experience compared to time or number of hand movements.