Quantifying surgeon maneuevers across experience levels through marker-less hand motion kinematics of simulated surgical tasks

Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review

When technical requirements are high, and patient outcomes are critical, opportunities for monitoring and improving surgical skills via objective motion analysis feedback may be particularly beneficial. This narrative review synthesises work on technical and non-technical surgical skills, collaborative task performance, and pose estimation to illustrate new opportunities to advance cardiothoracic surgical performance with innovations from computer vision and artificial intelligence. These technological innovations are critically evaluated in terms of the benefits they could offer the cardiothoracic surgical community, and any barriers to the uptake of the technology are elaborated upon. Like some other specialities, cardiothoracic surgery has relatively few opportunities to benefit from tools with data capture technology embedded within them (as is possible with robotic-assisted laparoscopic surgery, for example). In such cases, pose estimation techniques that allow for movement tracking across a conventional operating field without using specialist equipment or markers offer considerable potential. With video data from either simulated or real surgical procedures, these tools can (1) provide insight into the development of expertise and surgical performance over a surgeon's career, (2) provide feedback to trainee surgeons regarding areas for improvement, (3) provide the opportunity to investigate what aspects of skill may be linked to patient outcomes which can (4) inform the aspects of surgical skill which should be focused on within training or mentoring programmes. Classifier or assessment algorithms that use artificial intelligence to 'learn' what expertise is from expert surgical evaluators could further assist educators in determining if trainees meet competency thresholds. With collaborative efforts between surgical teams, medical institutions, computer scientists and researchers to ensure this technology is developed with usability and ethics in mind, the developed feedback tools could improve cardiothoracic surgical practice in a data-driven way.

Change in economy of ultrasound probe motion among general medicine trainees

OBJECTIVES

To observe change in economy of 9 ultrasound probe movement metrics among internal medicine trainees during a 5-day training course in cardiac point of care ultrasound (POCUS).

METHODS

We used a novel probe tracking device to record nine features of ultrasound probe movement, while trainees and experts optimized ultrasound clips on the same volunteer patients. These features included translational movements, gyroscopic movements (titling, rocking, and rotation), smoothness, total path length, and scanning time. We determined the adjusted difference between each trainee's movements and the mean value of the experts' movements for each patient. We then used a mixed effects model to trend average the adjusted differences between trainees and experts throughout the 5 days of the course.

RESULTS

Fifteen trainees were enrolled. Three echocardiographer technicians and the course director served as experts. Across 16 unique patients, 294 ultrasound clips were acquired. For all 9 movements, the adjusted difference between trainees and experts narrowed day-to-day (p value < 0.05), suggesting ongoing improvement during training. By the last day of the course, there were no statistically significant differences between trainees and experts in translational movement, gyroscopic movement, smoothness, or total path length; yet on average trainees took 28 s (95% CI [14.7-40.3] seconds) more to acquire a clip.

CONCLUSIONS

We detected improved ultrasound probe motion economy among internal medicine trainees during a 5-day training course in cardiac POCUS using an inexpensive probe tracking device. Objectively quantifying probe motion economy may help assess a trainee's level of proficiency in this skill and individualize their POCUS training.

Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery

PURPOSE

Surgical skill assessment has primarily been performed using checklists or rating scales, which are prone to bias and subjectivity. To tackle this shortcoming, assessment of surgical tool motion can be implemented to objectively classify skill levels. Due to the challenges involved in motion tracking of surgical tooltips in minimally invasive surgeries, formerly used assessment approaches may not be feasible for real-world skill assessment. We proposed an assessment approach based on the virtual marker on surgical tooltips to derive the tooltip's 3D position and introduced a novel metric for surgical skill assessment.

METHODS

We obtained the 3D tooltip position based on markers placed on the tool handle. Then, we derived tooltip motion metrics to identify the metrics differentiating the skill levels for objective surgical skill assessment. We proposed a new tooltip motion metric, i.e., motion inconsistency, that can assess the skill level, and also can evaluate the stage of skill learning. In this study, peg transfer, dual transfer, and rubber band translocation tasks were included, and nine novices, five surgical residents and five attending general surgeons participated.

RESULTS

Our analyses showed that tooltip path length (p [Formula: see text] 0.007) and path length along the instrument axis (p [Formula: see text] 0.014) differed across the three skill levels in all the tasks and decreased by skill level. Tooltip motion inconsistency showed significant differences among the three skill levels in the dual transfer (p [Formula: see text] 0.025) and the rubber band translocation tasks (p [Formula: see text] 0.021). Lastly, bimanual dexterity differed across the three skill levels in all the tasks (p [Formula: see text] 0.012) and increased by skill level.

CONCLUSION

Depth perception ability (indicated by shorter tooltip path lengths along the instrument axis), bimanual dexterity, tooltip motion consistency, and economical tooltip movements (shorter tooltip path lengths) are related to surgical skill. Our findings can contribute to objective surgical skill assessment, reducing subjectivity, bias, and associated costs.

Motion Smoothness-Based Assessment of Surgical Expertise: The Importance of Selecting Proper Metrics

The smooth movement of hand/surgical instruments is considered an indicator of skilled, coordinated surgical performance. Jerky surgical instrument movements or hand tremors can cause unwanted damages to the surgical site. Different methods have been used in previous studies for assessing motion smoothness, causing conflicting results regarding the comparison among surgical skill levels. We recruited four attending surgeons, five surgical residents, and nine novices. The participants conducted three simulated laparoscopic tasks, including peg transfer, bimanual peg transfer, and rubber band translocation. Tooltip motion smoothness was computed using the mean tooltip motion jerk, logarithmic dimensionless tooltip motion jerk, and 95% tooltip motion frequency (originally proposed in this study) to evaluate their capability of surgical skill level differentiation. The results revealed that logarithmic dimensionless motion jerk and 95% motion frequency were capable of distinguishing skill levels, indicated by smoother tooltip movements observed in high compared to low skill levels. Contrarily, mean motion jerk was not able to distinguish the skill levels. Additionally, 95% motion frequency was less affected by the measurement noise since it did not require the calculation of motion jerk, and 95% motion frequency and logarithmic dimensionless motion jerk yielded a better motion smoothness assessment outcome in distinguishing skill levels than mean motion jerk.

"Stand-up straight!": human pose estimation to evaluate postural skills during orthopedic surgery simulations

PURPOSE

Surgery simulators can be used to learn technical and non-technical skills and, to analyse posture. Ergonomic skill can be automatically detected with a Human Pose Estimation algorithm to help improve the surgeon's work quality. The objective of this study was to analyse the postural behaviour of surgeons and identify expertise-dependent movements. Our hypothesis was that hesitation and the occurrence of surgical instruments interfering with movement (defined as interfering movements) decrease with expertise.

MATERIAL AND METHODS

Sixty surgeons with three expertise levels (novice, intermediate, and expert) were recruited. During a training session using an arthroscopic simulator, each participant's movements were video-recorded with an RGB camera. A modified OpenPose algorithm was used to detect the surgeon's joints. The detection frequency of each joint in a specific area was visualized with a heatmap-like approach and used to calculate a mobility score.

RESULTS

This analysis allowed quantifying surgical movements. Overall, the mean mobility score was 0.823, 0.816, and 0.820 for novice, intermediate and expert surgeons, respectively. The mobility score alone was not enough to identify postural behaviour differences. A visual analysis of each participants' movements highlighted expertise-dependent interfering movements.

CONCLUSION

Video-recording and analysis of surgeon's movements are a non-invasive approach to obtain quantitative and qualitative ergonomic information in order to provide feedback during training. Our findings suggest that the interfering movements do not decrease with expertise but differ in function of the surgeon's level.

Influence of experience on kinematics of upper limbs during sewing gesture

To teach a skilled motor task, it is crucial to understand the characteristics of expertise. The aim of the present study was to compare the kinematics of the hand sewing task between novices (n = 10), intermediates (n = 10) and experts (n = 10). Compared to novices and intermediates, the proximal joint of expert participants was less involved in the task than their distal joints. The shoulder of experts stayed closer to the trunk, while the ranges of motion of the wrist and fingers were higher. This ability enabled them to avoid lifting the arm, which was resting on the table. We observed a low cycle-to-cycle variability of the movement pattern for experts, while it was more variable in novices. Moreover, experts shared similar joints synergies attesting of an "experts" common gesture. This knowledge gained about the hand sewing kinematics can be used to refine the training process of dressmakers.

Modeling Performance of Open Surgical Cases

INTRODUCTION

Previous efforts used digital video to develop computer-generated assessments of surgical hand motion economy and fluidity of motion. This study tests how well previously trained assessment models match expert ratings of suturing and tying video clips recorded in a new operating room (OR) setting.

METHODS

Enabled through computer vision of the hands, this study tests the applicability of assessments born out of benchtop simulations to in vivo suturing and tying tasks recorded in the OR.

RESULTS

Compared with expert ratings, computer-generated assessments for fluidity of motion (slope = 0.83, intercept = 1.77, R2 = 0.55) performed better than motion economy (slope = 0.73, intercept = 2.04, R2 = 0.49), although 85% of ratings for both models were within ±2 of the expert response. Neither assessment performed as well in the OR as they did on the training data. Assessments were sensitive to changing hand postures, dropped ligatures, and poor tissue contact-features typically missing from training data. Computer-generated assessment of OR tasks was contingent on a clear, consistent view of both surgeon's hands.

CONCLUSIONS

Computer-generated assessment may help provide formative feedback during deliberate practice, albeit with greater variability in the OR compared with benchtop simulations. Future work will benefit from expanded available bimanual video records.

Remembering Ben-Tzion Karsh's scholarship, impact, and legacy

Dr. Ben-Tzion (Bentzi) Karsh was a mentor, collaborator, colleague, and friend who profoundly impacted the fields of human factors and ergonomics (HFE), medical informatics, patient safety, and primary care, among others. In this paper we honor his contributions by reflecting on his scholarship, impact, and legacy in three ways: first, through an updated simplified bibliometric analysis in 2020, highlighting the breadth of his scholarly impact from the perspective of the number and types of communities and collaborators with which and whom he engaged; second, through targeted reflections on the history and impact of Dr. Karsh's most cited works, commenting on the particular ways they impacted our academic community; and lastly, through quotes from collaborators and mentees, illustrating Dr. Karsh's long-lasting impact on his contemporaries and students.