Effects of robotic manipulators on movements of novices and surgeons

Assessment of Surgeons' Stress Levels with Digital Sensors during Robot-Assisted Surgery: An Experimental Study

Robot-Assisted Minimally Invasive Surgery (RAMIS) marks a paradigm shift in surgical procedures, enhancing precision and ergonomics. Concurrently it introduces complex stress dynamics and ergonomic challenges regarding the human-robot interface and interaction. This study explores the stress-related aspects of RAMIS, using the da Vinci XI Surgical System and the Sea Spikes model as a standard skill training phantom to establish a link between technological advancement and human factors in RAMIS environments. By employing different physiological and kinematic sensors for heart rate variability, hand movement tracking, and posture analysis, this research aims to develop a framework for quantifying the stress and ergonomic loads applied to surgeons. Preliminary findings reveal significant correlations between stress levels and several of the skill-related metrics measured by external sensors or the SURG-TLX questionnaire. Furthermore, early analysis of this preliminary dataset suggests the potential benefits of applying machine learning for surgeon skill classification and stress analysis. This paper presents the initial findings, identified correlations, and the lessons learned from the clinical setup, aiming to lay down the cornerstones for wider studies in the fields of clinical situation awareness and attention computing.

Rigid flexible coupling contact mechanism for oral and maxillofacial skin and soft tissues

BACKGROUND AND OBJECTIVES

The existing medical clinical treatment institutions mostly use rigid structures to come into contact with flexible skin. The rigid flexible coupled contact biomechanical model for the skin is the first step that urgently needs to be considered in the process of medical clinical operations. However, there has been currently no effective biomechanical contact model available.

METHODS

Based on the principle of elastic interface deformation, the basic biomechanical characteristics of oral and maxillofacial skin and soft tissues were analyzed to address the unknown mechanism of rigid body and maxillofacial contact in oral imaging operations. A nonlinear characterization method for the mechanical properties of oral and maxillofacial skin soft tissues was proposed by deriving a general contact force model that takes into account energy dissipation. However, the problem of the inability to obtain analytical solutions for the parameters of the dynamic model exists. It is necessary to perform particle swarm parameter identification on different nonlinear contact models and verify the accuracy of the algorithm through numerical simulation. A maxillofacial contact experiment was conducted to verify the operation process of an oral imaging robot.

RESULTS

After experimental analysis, it was found that the comprehensive average error between the model and the actual contact force was 0.13325 N. The absolute error of the maximum deformation displacement was below 0.18 N, which verified the effectiveness and safety of the contact model in the contact process of the oral imaging robot system.

CONCLUSIONS

The results indicate that the output force of the model has been in good agreement with the actual contact force.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery

PURPOSE

Surgical skill assessment has primarily been performed using checklists or rating scales, which are prone to bias and subjectivity. To tackle this shortcoming, assessment of surgical tool motion can be implemented to objectively classify skill levels. Due to the challenges involved in motion tracking of surgical tooltips in minimally invasive surgeries, formerly used assessment approaches may not be feasible for real-world skill assessment. We proposed an assessment approach based on the virtual marker on surgical tooltips to derive the tooltip's 3D position and introduced a novel metric for surgical skill assessment.

METHODS

We obtained the 3D tooltip position based on markers placed on the tool handle. Then, we derived tooltip motion metrics to identify the metrics differentiating the skill levels for objective surgical skill assessment. We proposed a new tooltip motion metric, i.e., motion inconsistency, that can assess the skill level, and also can evaluate the stage of skill learning. In this study, peg transfer, dual transfer, and rubber band translocation tasks were included, and nine novices, five surgical residents and five attending general surgeons participated.

RESULTS

Our analyses showed that tooltip path length (p [Formula: see text] 0.007) and path length along the instrument axis (p [Formula: see text] 0.014) differed across the three skill levels in all the tasks and decreased by skill level. Tooltip motion inconsistency showed significant differences among the three skill levels in the dual transfer (p [Formula: see text] 0.025) and the rubber band translocation tasks (p [Formula: see text] 0.021). Lastly, bimanual dexterity differed across the three skill levels in all the tasks (p [Formula: see text] 0.012) and increased by skill level.

CONCLUSION

Depth perception ability (indicated by shorter tooltip path lengths along the instrument axis), bimanual dexterity, tooltip motion consistency, and economical tooltip movements (shorter tooltip path lengths) are related to surgical skill. Our findings can contribute to objective surgical skill assessment, reducing subjectivity, bias, and associated costs.

Motion Smoothness-Based Assessment of Surgical Expertise: The Importance of Selecting Proper Metrics

The smooth movement of hand/surgical instruments is considered an indicator of skilled, coordinated surgical performance. Jerky surgical instrument movements or hand tremors can cause unwanted damages to the surgical site. Different methods have been used in previous studies for assessing motion smoothness, causing conflicting results regarding the comparison among surgical skill levels. We recruited four attending surgeons, five surgical residents, and nine novices. The participants conducted three simulated laparoscopic tasks, including peg transfer, bimanual peg transfer, and rubber band translocation. Tooltip motion smoothness was computed using the mean tooltip motion jerk, logarithmic dimensionless tooltip motion jerk, and 95% tooltip motion frequency (originally proposed in this study) to evaluate their capability of surgical skill level differentiation. The results revealed that logarithmic dimensionless motion jerk and 95% motion frequency were capable of distinguishing skill levels, indicated by smoother tooltip movements observed in high compared to low skill levels. Contrarily, mean motion jerk was not able to distinguish the skill levels. Additionally, 95% motion frequency was less affected by the measurement noise since it did not require the calculation of motion jerk, and 95% motion frequency and logarithmic dimensionless motion jerk yielded a better motion smoothness assessment outcome in distinguishing skill levels than mean motion jerk.

A distal external focus of attention facilitates compensatory coordination of body parts

Many studies have shown that focusing on an intended movement effect that is farther away from the body (distal external focus) results in performance benefits relative to focusing on an effect that is closer to the body (proximal external focus) or focusing on the body itself (internal focus) (see, Chua, Jimenez-Diaz, Lewthwaite, Kim & Wulf, 2021). Furthermore, the advantages of a distal external focus seem to be particularly pronounced in skilled performers (Singh & Wulf, 2020). The present study examined whether such benefits of more distal attentional focus may be associated with enhanced functional variability. Volleyball players (n = 20) performed 60 overhand volleyball serves to a target. Using a within-participants design, the effects of a distal external focus (bullseye), proximal external focus (ball) and an internal focus (hand) were compared. The distal focus condition resulted in significantly higher accuracy scores than did the proximal and internal focus conditions. In addition, uncontrolled manifold analysis showed that functional variability (as measured by the index of synergy) was greatest in the distal focus condition. These findings suggest that a distal external focus on the task goal may enhance movement outcomes by optimising compensatory coordination of body parts.

Haptic Feedback Based on Movement Smoothness Improves Performance in a Perceptual-Motor Task

In many training scenarios, and in surgery in particular, feedback is provided to the trainee after the task has been performed, and the assessment is often qualitative in nature. In this paper, we demonstrate the effect of real-time objective performance feedback conveyed through a vibrotactile cue. Subjects performed a mirror-tracing task that requires coordination and dexterity similar in nature to that required in endovascular surgery. Movement smoothness, a characteristic associated with skilled and coordinated movement, was measured by spectral arc length, a frequency-domain measure of smoothness. The smoothness-based performance metric was encoded as a vibrotactile cue displayed on the user's arm. Performance on the mirror tracing task with smoothness-based feedback was compared to position-based feedback (where the subject was alerted when they moved outside the path boundary) and to a no vibrotactile feedback control condition. Subjects receiving smoothness-based feedback altered their task completion strategies, resulting in faster task completion times, but their accuracy was slightly worse overall than the other two groups. In procedures such as endovascular surgery, the reduction of procedure time that could be achieved with smoothness-based feedback training may be advantageous, despite the fact that accuracy was inferior to that observed with no feedback or position-based feedback.

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.

Grasping Assisting Algorithm in Tele-Operated Robotic Gripper

The involvement of Robots and automated machines in different industries has increased drastically in recent years. Part of this revolution is accomplishing tasks previously performed by humans with advanced robots, which would replace the entire human workforce in the future. In some industries the workers are required to complete different operations in hazardous or difficult environments. Operations like these could be replaced with the use of tele-operated systems that have the capability of grasping objects in their surroundings, thus abandoning the need for the physical presence of the human operator at the area while still allowing control. In this research our goal is to create an assisting system that would improve the grasping of a human operator using a tele-operated robotic gripper and arm, while advising the operator but not forcing a solution. For a given set of objects we computed the optimal grasp to be achieved by the gripper, based on two grasp quality measures of our choosing (namely power grasp and precision grasp). We then tested the performance of different human subjects who tried to grasp the different objects with the tele-operated system, while comparing their success to unassisted and assisted grasping. Our goal is to create an assisting algorithm that would compute optimal grasps and might be integrated into a complete, state-of-the-art tele-operated system.

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set II: learning curve analysis

PURPOSE

The Johns Hopkins-Intuitive Gesture and Skill Assessment Working Set (JIGSAWS) dataset is used to develop robotic surgery skill assessment tools, but there has been no detailed analysis of this dataset. The aim of this study is to perform a learning curve analysis of the existing JIGSAWS dataset.

METHODS

Five trials were performed in JIGSAWS by eight participants (four novices, two intermediates and two experts) for three exercises (suturing, knot-tying and needle passing). Global Rating Scores and time, path length and movements were analyzed quantitatively and qualitatively by graphical analysis.

RESULTS

There are no significant differences in Global Rating Scale scores over time. Time in the suturing exercise and path length in needle passing had significant differences. Other kinematic parameters were not significantly different. Qualitative analysis shows a learning curve only for suturing. Cumulative sum analysis suggests completion of the learning curve for suturing by trial 4.

CONCLUSIONS

The existing JIGSAWS dataset does not show a quantitative learning curve for Global Rating Scale scores, or most kinematic parameters which may be due in part to the limited size of the dataset. Qualitative analysis shows a learning curve for suturing. Cumulative sum analysis suggests completion of the suturing learning curve by trial 4. An expanded dataset is needed to facilitate subset analyses.

Velocity-Domain Motion Quality Measures for Surgical Performance Evaluation and Feedback

Endovascular navigation proficiency requires a significant amount of manual dexterity from surgeons. Objective performance measures derived from endovascular tool tip kinematics have been shown to correlate with expertise; however, such metrics have not yet been used during training as a basis for real-time performance feedback. This paper evaluates a set of velocity-based performance measures derived from guidewire motion to determine their suitability for online performance evaluation and feedback. We evaluated the endovascular navigation skill of 75 participants using three metrics (spectral arc length, average velocity, and idle time) as they steered tools to anatomical targets using a virtual reality simulator. First, we examined the effect of navigation task and experience level on performance and found that novice performance was significantly different from intermediate and expert performance. Then we computed correlations between measures calculated online and spectral arc length, our “gold standard” metric, calculated offline (at the end of the trial, using data from the entire trial). Our results suggest that average velocity and idle time calculated online are strongly and consistently correlated with spectral arc length computed offline, which was not the case when comparing spectral arc length computed online and offline. Average velocity and idle time, both time-domain based performance measures, are therefore more suitable measures than spectral arc length, a frequency-domain based metric, to use as the basis of online performance feedback. Future work is needed to determine how to best provide real-time performance feedback to endovascular surgery trainees based on these metrics.

Surgeon-Centered Analysis of Robot-Assisted Needle Driving Under Different Force Feedback Conditions

Robotic assisted minimally invasive surgery (RAMIS) systems present many advantages to the surgeon and patient over open and standard laparoscopic surgery. However, haptic feedback, which is crucial for the success of many surgical procedures, is still an open challenge in RAMIS. Understanding the way that haptic feedback affects performance and learning can be useful in the development of haptic feedback algorithms and teleoperation control systems. In this study, we examined the performance and learning of inexperienced participants under different haptic feedback conditions in a task of surgical needle driving via a soft homogeneous deformable object-an artificial tissue. We designed an experimental setup to characterize their movement trajectories and the forces that they applied on the artificial tissue. Participants first performed the task in an open condition, with a standard surgical needle holder, followed by teleoperation in one of three feedback conditions: (1) no haptic feedback, (2) haptic feedback based on position exchange, and (3) haptic feedback based on direct recording from a force sensor, and then again with the open needle holder. To quantify the effect of different force feedback conditions on the quality of needle driving, we developed novel metrics that assess the kinematics of needle driving and the tissue interaction forces, and we combined our novel metrics with classical metrics. We analyzed the final teleoperated performance in each condition, the improvement during teleoperation, and the aftereffect of teleoperation on the performance when using the open needle driver. We found that there is no significant difference in the final performance and in the aftereffect between the 3 conditions. Only the two conditions with force feedback presented statistically significant improvement during teleoperation in several of the metrics, but when we compared directly between the improvements in the three different feedback conditions none of the effects reached statistical significance. We discuss possible explanations for the relative similarity in performance. We conclude that we developed several new metrics for the quality of surgical needle driving, but even with these detailed metrics, the advantage of state of the art force feedback methods to tasks that require interaction with homogeneous soft tissue is questionable.

Self-debriefing Model Based on an Integrated Video-Capture System: An Efficient Solution to Skill Degradation

OBJECTIVE

Video-based teaching is considered highly effective in debriefing, especially in minimally invasive surgeries. In this study, the benefits of using a new integrated video recording system, were investigated and compared to those of the standard basic skills robotic training procedure.

DESIGN

Fifty residents from the 2nd and 3rd year medical faculty without any experience of robot usage or laparoscopy were randomized into 2 groups: group A--a natural self-training group without a trainer, and group B--a self-training group assisted by an integrated video recording system during training. The training was divided into four 2-hour sessions, with a 72-hour delay between each session. Two tasks were selected for testing on the dV-Trainer, a virtual reality based robotic simulator: Match board 2 and Thread the Rings 1. After each session, the practice video recorded by the system of group B was transferred to the residents' smartphones for self-debriefing. At the end of each session, the performance score was evaluated automatically by using the simulator to plot learning curves A and B.

RESULTS

Group A showed a significant drop in performance score due to skill decay caused by the 72-hour delay. Group B exhibited a regular stepwise rising learning curve. At the end of the training, group B showed a significantly higher performance score both in Match board 2 and Thread the Rings 1. The autoanalysis and capture function, which selects only the critical errors and most valuable parts, could facilitate time saving.

CONCLUSIONS

The use of an integrated video recording system makes the self-manipulated protocol with own smartphone feasible to improve training efficiency and overcome the skill decay during robotic surgical training.

Surgical motion analysis using discriminative interpretable patterns

OBJECTIVE

The analysis of surgical motion has received a growing interest with the development of devices allowing their automatic capture. In this context, the use of advanced surgical training systems makes an automated assessment of surgical trainee possible. Automatic and quantitative evaluation of surgical skills is a very important step in improving surgical patient care.

MATERIAL AND METHOD

In this paper, we present an approach for the discovery and ranking of discriminative and interpretable patterns of surgical practice from recordings of surgical motions. A pattern is defined as a series of actions or events in the kinematic data that together are distinctive of a specific gesture or skill level. Our approach is based on the decomposition of continuous kinematic data into a set of overlapping gestures represented by strings (bag of words) for which we compute comparative numerical statistic (tf-idf) enabling the discriminative gesture discovery via its relative occurrence frequency.

RESULTS

We carried out experiments on three surgical motion datasets. The results show that the patterns identified by the proposed method can be used to accurately classify individual gestures, skill levels and surgical interfaces. We also present how the patterns provide a detailed feedback on the trainee skill assessment.

CONCLUSIONS

The proposed approach is an interesting addition to existing learning tools for surgery as it provides a way to obtain a feedback on which parts of an exercise have been used to classify the attempt as correct or incorrect.

An Uncontrolled Manifold Analysis of Arm Joint Variability in Virtual Planar Position and Orientation Telemanipulation

OBJECTIVE

In teleoperated robot-assisted tasks, the user interacts with manipulators to finely control remote tools. Manipulation of robotic devices, characterized by specific kinematic and dynamic proprieties, is a complex task for the human sensorimotor system due to the inherent biomechanical and neuronal redundancies that characterize the human arm and its control. We investigate how master devices with different kinematics structures and how different task constraints influence users capabilities in exploiting arm redundancy.

METHODS

A virtual teleoperation workbench was designed and the arm kinematics of seven users was acquired during the execution of two planar virtual tasks, involving either the control of position only or position-orientation of a tool. Using the uncontrolled manifold analysis of arm joint variability, we estimated the logarithmic ratio between the task irrelevant and the task relevant manifolds ( R_v).

RESULTS

The R_v values obtained in the position-orientation task were higher than in the position only task, while no differences were found between the master devices. A modulation of R_v was found through the execution of the position task and a positive correlation was found between task performance and redundancy exploitation.

CONCLUSION

Users exploited additional portions of arm redundancy when dealing with the tool orientation. The R_v modulation seems influenced by the task constraints and by the users possibility of reconfiguring the arm position.

SIGNIFICANCE

This paper advances the general understanding of the exploitation of arm redundancy in complex tasks, and can improve the development of future robotic devices.

Human-centric predictive model of task difficulty for human-in-the-loop control tasks

Quantitatively measuring the difficulty of a manipulation task in human-in-the-loop control systems is ill-defined. Currently, systems are typically evaluated through task-specific performance measures and post-experiment user surveys; however, these methods do not capture the real-time experience of human users. In this study, we propose to analyze and predict the difficulty of a bivariate pointing task, with a haptic device interface, using human-centric measurement data in terms of cognition, physical effort, and motion kinematics. Noninvasive sensors were used to record the multimodal response of human user for 14 subjects performing the task. A data-driven approach for predicting task difficulty was implemented based on several task-independent metrics. We compare four possible models for predicting task difficulty to evaluated the roles of the various types of metrics, including: (I) a movement time model, (II) a fusion model using both physiological and kinematic metrics, (III) a model only with kinematic metrics, and (IV) a model only with physiological metrics. The results show significant correlation between task difficulty and the user sensorimotor response. The fusion model, integrating user physiology and motion kinematics, provided the best estimate of task difficulty (R2 = 0.927), followed by a model using only kinematic metrics (R2 = 0.921). Both models were better predictors of task difficulty than the movement time model (R2 = 0.847), derived from Fitt's law, a well studied difficulty model for human psychomotor control.

Evaluation of robotic surgery skills using dynamic time warping

BACKGROUND AND OBJECTIVE

accompanied with the wide acceptance of robot assisted minimally invasive surgery (RMIS), the demand for efficient and objective surgical skills evaluation method is increased. Recently, with the development of medical engineering technology, several evaluation methods have been proposed. Among them, kinematic analysis, an unsupervised and data-based method, has been accepted by many researchers. However, this method is still limited by the number of metrics and unconvinced scoring system. This paper aims to propose a new evaluation method to assess surgical skills efficiently and objectively.

METHODS

this research proposed an efficient and effective surgical skills evaluation algorithm which used the trajectories of instrument tip and dynamic time warping (DTW) to provide trainees with real-time and summative feedback. The optimum trajectories based on 'Therbligs' theory was designed as a template. DTW algorithm was used to align actual trajectories to optimum trajectories with an evaluating indicator designed to emphasize the crucial motion features in surgical skills evaluation. The real-time feedback was obtained through a sliding time window to help trainees improve learning efficiency.

RESULTS

experts (n = 2) and novices (n = 8) were invited to complete the peg transfer tasks and 60 instrument tip trajectories were assessed by the proposed algorithm. Significant differences between different groups were observed (experts' right trajectories versus experts' left trajectories, p = 0.0002; experts' right trajectories versus novices' right trajectories, p = 0.0124). In addition, evaluation results of trajectories with operational mistakes were significantly different from those of others.

CONCLUSIONS

the proposed evaluation method showed its advantages in distinguishing and evaluating surgical performance. Given its ability to evaluate the performance based on kinematic information, the proposed evaluation method can be further developed in the future. Furthermore, because the proposed method can provide real-time feedback, it also has the potential to be a monitoring system in operation room.

From dV-Trainer to Real Robotic Console: The Limitations of Robotic Skill Training

OBJECTIVES

To investigate operators' performance quality, mental stress, and ergonomic habits through a training curriculum on robotic simulators.

DESIGN

Forty volunteers without robotic surgery experience were recruited to practice 2 exercises on a dV-Trainer (dVT) for 14 hours. The simulator software (M-score^a) provided an automatic evaluation of the overall score for the surgeons' performance. Each participant provided a subjective difficulty score (validity to be proven) for each exercise. Their ergonomic habits were evaluated based on the workspace range and armrest load-validated criteria for evaluating the proficiency of using the armrest. They then repeated the same tasks on a da Vinci Surgical Skill Simulator for a final-level test. Their final scores were compared with their initial scores and the scores of 5 experts on the da Vinci Surgical Skill Simulator.

RESULTS

A total of 14 hours of training on the dVT significantly improved the surgeons' performance scores to the expert level with a significantly reduced workload, but their ergonomic score was still far from the expert level.

CONCLUSION

Sufficient training on the dVT improves novices' performance, reduces psychological stress, and inculcates better ergonomic habits. Among the evaluated criteria, novices had the most difficulty in achieving expert levels of ergonomic skills. The training benefits of robotic surgery simulators should be determined with quantified variables. The detection of the limitations during robotic training curricula could guide the targeted training and improve the training effect.

Motion control skill assessment based on kinematic analysis of robotic end-effector movements

BACKGROUND

The performance of robotic end-effector movements can reflect the user's operation skill difference in robot-assisted minimally invasive surgery. This study quantified the trade-off of speed-accuracy-stability by kinematic analysis of robotic end-effector movements to assess the motion control skill of users with different levels of experience.

METHODS

Using 'MicroHand S' system, 10 experts, 10 residents and 10 novices performed single-hand test and bimanual coordination test. Eight metrics based on the movements of robotic end-effectors were applied to evaluate the users' performance.

RESULTS

In the single-hand test, experts outperformed other groups except for movement speed; in the bimanual coordination test, experts also performed better except for movement time and movement speed. No statistically significant difference in performance was found between residents and novices.

CONCLUSIONS

The kinematic differences obtained from the movements of robotic end-effectors can be applied to assess the motion control skill of users with different skill levels.

Using Contact Forces and Robot Arm Accelerations to Automatically Rate Surgeon Skill at Peg Transfer

OBJECTIVE

Most trainees begin learning robotic minimally invasive surgery by performing inanimate practice tasks with clinical robots such as the Intuitive Surgical da Vinci. Expert surgeons are commonly asked to evaluate these performances using standardized five-point rating scales, but doing such ratings is time consuming, tedious, and somewhat subjective. This paper presents an automatic skill evaluation system that analyzes only the contact force with the task materials, the broad-bandwidth accelerations of the robotic instruments and camera, and the task completion time.

METHODS

We recruited N = 38 participants of varying skill in robotic surgery to perform three trials of peg transfer with a da Vinci Standard robot instrumented with our Smart Task Board. After calibration, three individuals rated these trials on five domains of the Global Evaluative Assessment of Robotic Skill (GEARS) structured assessment tool, providing ground-truth labels for regression and classification machine learning algorithms that predict GEARS scores based on the recorded force, acceleration, and time signals.

RESULTS

Both machine learning approaches produced scores on the reserved testing sets that were in good to excellent agreement with the human raters, even when the force information was not considered. Furthermore, regression predicted GEARS scores more accurately and efficiently than classification.

CONCLUSION

A surgeon's skill at robotic peg transfer can be reliably rated via regression using features gathered from force, acceleration, and time sensors external to the robot.

SIGNIFICANCE

We expect improved trainee learning as a result of providing these automatic skill ratings during inanimate task practice on a surgical robot.

Design and evaluation of a trilateral shared-control architecture for teleoperated training robots

Multilateral teleoperated robots can be used to train humans to perform complex tasks that require collaborative interaction and expert supervision, such as laparoscopic surgical procedures. In this paper, we explain the design and performance evaluation of a shared-control architecture that can be used in trilateral teleoperated training robots. The architecture includes dominance and observation factors inspired by the determinants of motor learning in humans, including observational practice, focus of attention, feedback and augmented feedback, and self-controlled practice. Toward the validation of such an architecture, we (1) verify the stability of a trilateral system by applying Llewellyn's criterion on a two-port equivalent architecture, and (2) demonstrate that system transparency remains generally invariant across relevant observation factors and movement frequencies. In a preliminary experimental study, a dyad of two human users (one novice, one expert) collaborated on the control of a robot to follow a trajectory. The experiment showed that the framework can be used to modulate the efforts of the users and adjust the source and level of haptic feedback to the novice user.

Determining the bias and variance of a deterministic finger-tracking algorithm

Finger tracking has the potential to expand haptic research and applications, as eye tracking has done in vision research. In research applications, it is desirable to know the bias and variance associated with a finger-tracking method. However, assessing the bias and variance of a deterministic method is not straightforward. Multiple measurements of the same finger position data will not produce different results, implying zero variance. Here, we present a method of assessing deterministic finger-tracking variance and bias through comparison to a non-deterministic measure. A proof-of-concept is presented using a video-based finger-tracking algorithm developed for the specific purpose of tracking participant fingers during a psychological research study. The algorithm uses ridge detection on videos of the participant's hand, and estimates the location of the right index fingertip. The algorithm was evaluated using data from four participants, who explored tactile maps using only their right index finger and all right-hand fingers. The algorithm identified the index fingertip in 99.78 % of one-finger video frames and 97.55 % of five-finger video frames. Although the algorithm produced slightly biased and more dispersed estimates relative to a human coder, these differences (x=0.08 cm, y=0.04 cm) and standard deviations (σ x =0.16 cm, σ y =0.21 cm) were small compared to the size of a fingertip (1.5-2.0 cm). Some example finger-tracking results are provided where corrections are made using the bias and variance estimates.

Conventional Laparoscopic vs Robotic Training: Which is Better for Naive Users? A Randomized Prospective Crossover Study

OBJECTIVE

Robotic training (RT) using the da Vinci skills simulator and conventional training (CT) using a laparoscopic "training box" are both used to augment operative skills in minimally invasive surgery. The current study tests the hypothesis that skill acquisition is more rapid using RT than using CT among naive learners.

DESIGN AND PARTICIPANTS

A total of 40 subjects without laparoscopic or robotic surgical experience were enrolled and randomized to begin with either RT or CT. Then, 2 specific RT tasks were reproduced for CT and repeated 5 times each with RT and CT. Time and quality indicators were measured quantitatively. A crossover technique was used to control for in-study experience bias.

RESULTS

The tasks "pick and place jacks" (PP) and "thread the rings" (TR) were achieved faster with RT than with CT despite crossover (p < 0.0001). An RT-favoring difference was observed in speed for both tasks when changing modality. Percentage improvement with increasing trials was similar for RT and CT: RT completion time averaged 39 seconds and 211 seconds (PP and TR, respectively), compared with 65 seconds and 362 seconds when using CT (p < 0.0001); final improvement averaged 26% and 46% for RT (PP and TR, respectively) vs 31% and 47% for CT (p was 0.76 for PP and 0.20 for TR). Within the PP task, RT times averaged 41 seconds without previous CT experience vs 35 seconds with previous CT experience (p = 0.20); CT times averaged 61 seconds without and 69 seconds with previous RT experience (p = 0.48). Comparable times for the TR task were 212 seconds vs 216 seconds (p = 0.66) and 388 seconds vs 334 seconds (p = 0.17). Both instrument collisions and excessive force occurred more commonly for RT than for CT within the TR task (p < 0.0001).

CONCLUSIONS

Speeds were faster overall with RT than with CT, but the percentage of speed improvement with trials was similar, suggesting similar learning curves, with minimal transfer effect appreciated.

Robot-assisted surgery: an emerging platform for human neuroscience research

Classic studies in human sensorimotor control use simplified tasks to uncover fundamental control strategies employed by the nervous system. Such simple tasks are critical for isolating specific features of motor, sensory, or cognitive processes, and for inferring causality between these features and observed behavioral changes. However, it remains unclear how these theories translate to complex sensorimotor tasks or to natural behaviors. Part of the difficulty in performing such experiments has been the lack of appropriate tools for measuring complex motor skills in real-world contexts. Robot-assisted surgery (RAS) provides an opportunity to overcome these challenges by enabling unobtrusive measurements of user behavior. In addition, a continuum of tasks with varying complexity-from simple tasks such as those in classic studies to highly complex tasks such as a surgical procedure-can be studied using RAS platforms. Finally, RAS includes a diverse participant population of inexperienced users all the way to expert surgeons. In this perspective, we illustrate how the characteristics of RAS systems make them compelling platforms to extend many theories in human neuroscience, as well as, to develop new theories altogether.

Uncontrolled manifold analysis of arm joint angle variability during robotic teleoperation and freehand movement of surgeons and novices

Teleoperated robot-assisted surgery (RAS) is used to perform a wide variety of minimally invasive procedures. However, current understanding of the effect of robotic manipulation on the motor coordination of surgeons is limited. Recent studies in human motor control suggest that we optimize hand movement stability and task performance while minimizing control effort and improving robustness to unpredicted disturbances. To achieve this, the variability of joint angles and muscle activations is structured to reduce task-relevant variability and increase task-irrelevant variability. In this study, we determine whether teleoperation of a da Vinci Si surgical system in a nonclinical task of simple planar movements changes this structure of variability in experienced surgeons and novices. To answer this question, we employ the UnControlled manifold analysis that partitions users' joint angle variability into task-irrelevant and task-relevant manifolds. We show that experienced surgeons coordinate their joint angles to stabilize hand movements more than novices, and that the effect of teleoperation depends on experience--experts increase teleoperated stabilization relative to freehand whereas novices decrease it. We suggest that examining users' exploitation of the task-irrelevant manifold for stabilization of hand movements may be applied to: (1) evaluation and optimization of teleoperator design and control parameters, and (2) skill assessment and optimization of training in RAS.