1
|
Takács K, Lukács E, Levendovics R, Pekli D, Szijártó A, Haidegger T. Assessment of Surgeons' Stress Levels with Digital Sensors during Robot-Assisted Surgery: An Experimental Study. SENSORS (BASEL, SWITZERLAND) 2024; 24:2915. [PMID: 38733021 PMCID: PMC11086209 DOI: 10.3390/s24092915] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Revised: 04/25/2024] [Accepted: 04/30/2024] [Indexed: 05/13/2024]
Abstract
Robot-Assisted Minimally Invasive Surgery (RAMIS) marks a paradigm shift in surgical procedures, enhancing precision and ergonomics. Concurrently it introduces complex stress dynamics and ergonomic challenges regarding the human-robot interface and interaction. This study explores the stress-related aspects of RAMIS, using the da Vinci XI Surgical System and the Sea Spikes model as a standard skill training phantom to establish a link between technological advancement and human factors in RAMIS environments. By employing different physiological and kinematic sensors for heart rate variability, hand movement tracking, and posture analysis, this research aims to develop a framework for quantifying the stress and ergonomic loads applied to surgeons. Preliminary findings reveal significant correlations between stress levels and several of the skill-related metrics measured by external sensors or the SURG-TLX questionnaire. Furthermore, early analysis of this preliminary dataset suggests the potential benefits of applying machine learning for surgeon skill classification and stress analysis. This paper presents the initial findings, identified correlations, and the lessons learned from the clinical setup, aiming to lay down the cornerstones for wider studies in the fields of clinical situation awareness and attention computing.
Collapse
Affiliation(s)
- Kristóf Takács
- Antal Bejczy Center for Intelligent Robotics (IROB), University Research and Innovation Center (EKIK), Óbuda University, 1034 Budapest, Hungary; (E.L.); (R.L.)
| | - Eszter Lukács
- Antal Bejczy Center for Intelligent Robotics (IROB), University Research and Innovation Center (EKIK), Óbuda University, 1034 Budapest, Hungary; (E.L.); (R.L.)
| | - Renáta Levendovics
- Antal Bejczy Center for Intelligent Robotics (IROB), University Research and Innovation Center (EKIK), Óbuda University, 1034 Budapest, Hungary; (E.L.); (R.L.)
- John von Neumann Faculty of Informatics (NIK), Óbuda University, 1034 Budapest, Hungary
- Austrian Center for Medical Innovation and Technology (ACMIT), 2700 Wiener Neustadt, Austria
| | - Damján Pekli
- Department of Surgery, Transplantation and Gastroenterology, Semmelweis University, 1082 Budapest, Hungary; (D.P.); (A.S.)
| | - Attila Szijártó
- Department of Surgery, Transplantation and Gastroenterology, Semmelweis University, 1082 Budapest, Hungary; (D.P.); (A.S.)
| | - Tamás Haidegger
- Antal Bejczy Center for Intelligent Robotics (IROB), University Research and Innovation Center (EKIK), Óbuda University, 1034 Budapest, Hungary; (E.L.); (R.L.)
- Austrian Center for Medical Innovation and Technology (ACMIT), 2700 Wiener Neustadt, Austria
| |
Collapse
|
2
|
Hao X, Cheng Q, Xu J, Sun T, Wang Y, Liu Z. Rigid flexible coupling contact mechanism for oral and maxillofacial skin and soft tissues. J Mech Behav Biomed Mater 2024; 150:106329. [PMID: 38113825 DOI: 10.1016/j.jmbbm.2023.106329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 12/11/2023] [Accepted: 12/12/2023] [Indexed: 12/21/2023]
Abstract
BACKGROUND AND OBJECTIVES The existing medical clinical treatment institutions mostly use rigid structures to come into contact with flexible skin. The rigid flexible coupled contact biomechanical model for the skin is the first step that urgently needs to be considered in the process of medical clinical operations. However, there has been currently no effective biomechanical contact model available. METHODS Based on the principle of elastic interface deformation, the basic biomechanical characteristics of oral and maxillofacial skin and soft tissues were analyzed to address the unknown mechanism of rigid body and maxillofacial contact in oral imaging operations. A nonlinear characterization method for the mechanical properties of oral and maxillofacial skin soft tissues was proposed by deriving a general contact force model that takes into account energy dissipation. However, the problem of the inability to obtain analytical solutions for the parameters of the dynamic model exists. It is necessary to perform particle swarm parameter identification on different nonlinear contact models and verify the accuracy of the algorithm through numerical simulation. A maxillofacial contact experiment was conducted to verify the operation process of an oral imaging robot. RESULTS After experimental analysis, it was found that the comprehensive average error between the model and the actual contact force was 0.13325 N. The absolute error of the maximum deformation displacement was below 0.18 N, which verified the effectiveness and safety of the contact model in the contact process of the oral imaging robot system. CONCLUSIONS The results indicate that the output force of the model has been in good agreement with the actual contact force.
Collapse
Affiliation(s)
- Xiaolong Hao
- Institute of Advanced Manufacturing and Intelligent Technology, Beijing University of Technology, Beijing, 100124, China; Beijing Key Laboratory of Advanced Manufacturing Technology, Beijing University of Technology, Beijing, 100124, China
| | - Qiang Cheng
- Institute of Advanced Manufacturing and Intelligent Technology, Beijing University of Technology, Beijing, 100124, China; Beijing Key Laboratory of Advanced Manufacturing Technology, Beijing University of Technology, Beijing, 100124, China.
| | - Jingjing Xu
- Institute of Advanced Manufacturing and Intelligent Technology, Beijing University of Technology, Beijing, 100124, China; Beijing Key Laboratory of Advanced Manufacturing Technology, Beijing University of Technology, Beijing, 100124, China
| | - Ting Sun
- Institute of Advanced Manufacturing and Intelligent Technology, Beijing University of Technology, Beijing, 100124, China; Beijing Key Laboratory of Advanced Manufacturing Technology, Beijing University of Technology, Beijing, 100124, China
| | - Yi Wang
- Department of Stomatology, First Medical Center, Chinese PLA General Hospital, Beijing, 100141, China.
| | - Zhifeng Liu
- Key Laboratory of CNC Equipment Reliability, Ministry of Education, Jilin University, Jilin, 130015, China
| |
Collapse
|
3
|
Boal MWE, Anastasiou D, Tesfai F, Ghamrawi W, Mazomenos E, Curtis N, Collins JW, Sridhar A, Kelly J, Stoyanov D, Francis NK. Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review. Br J Surg 2024; 111:znad331. [PMID: 37951600 PMCID: PMC10771126 DOI: 10.1093/bjs/znad331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 11/14/2023]
Abstract
BACKGROUND There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula. METHODS A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias. RESULTS Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent. CONCLUSIONS Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.
Collapse
Affiliation(s)
- Matthew W E Boal
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
| | - Dimitrios Anastasiou
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Freweini Tesfai
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
| | - Walaa Ghamrawi
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
| | - Evangelos Mazomenos
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Nathan Curtis
- Department of General Surgey, Dorset County Hospital NHS Foundation Trust, Dorchester, UK
| | - Justin W Collins
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Ashwin Sridhar
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - John Kelly
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Danail Stoyanov
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Computer Science, UCL, London, UK
| | - Nader K Francis
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- Yeovil District Hospital, Somerset Foundation NHS Trust, Yeovil, Somerset, UK
| |
Collapse
|
4
|
Aghazadeh F, Zheng B, Tavakoli M, Rouhani H. Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery. Int J Comput Assist Radiol Surg 2023; 18:2191-2202. [PMID: 37597089 DOI: 10.1007/s11548-023-03007-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Accepted: 07/19/2023] [Indexed: 08/21/2023]
Abstract
PURPOSE Surgical skill assessment has primarily been performed using checklists or rating scales, which are prone to bias and subjectivity. To tackle this shortcoming, assessment of surgical tool motion can be implemented to objectively classify skill levels. Due to the challenges involved in motion tracking of surgical tooltips in minimally invasive surgeries, formerly used assessment approaches may not be feasible for real-world skill assessment. We proposed an assessment approach based on the virtual marker on surgical tooltips to derive the tooltip's 3D position and introduced a novel metric for surgical skill assessment. METHODS We obtained the 3D tooltip position based on markers placed on the tool handle. Then, we derived tooltip motion metrics to identify the metrics differentiating the skill levels for objective surgical skill assessment. We proposed a new tooltip motion metric, i.e., motion inconsistency, that can assess the skill level, and also can evaluate the stage of skill learning. In this study, peg transfer, dual transfer, and rubber band translocation tasks were included, and nine novices, five surgical residents and five attending general surgeons participated. RESULTS Our analyses showed that tooltip path length (p [Formula: see text] 0.007) and path length along the instrument axis (p [Formula: see text] 0.014) differed across the three skill levels in all the tasks and decreased by skill level. Tooltip motion inconsistency showed significant differences among the three skill levels in the dual transfer (p [Formula: see text] 0.025) and the rubber band translocation tasks (p [Formula: see text] 0.021). Lastly, bimanual dexterity differed across the three skill levels in all the tasks (p [Formula: see text] 0.012) and increased by skill level. CONCLUSION Depth perception ability (indicated by shorter tooltip path lengths along the instrument axis), bimanual dexterity, tooltip motion consistency, and economical tooltip movements (shorter tooltip path lengths) are related to surgical skill. Our findings can contribute to objective surgical skill assessment, reducing subjectivity, bias, and associated costs.
Collapse
Affiliation(s)
- Farzad Aghazadeh
- Department of Mechanical Engineering, 10-390 Donadeo Innovation Centre for Engineering, University of Alberta, 9211-116 Street NW, Edmonton, AB, T6G 1H9, Canada
| | - Bin Zheng
- Department of Surgery, University of Alberta, Edmonton, AB, Canada
| | - Mahdi Tavakoli
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
| | - Hossein Rouhani
- Department of Mechanical Engineering, 10-390 Donadeo Innovation Centre for Engineering, University of Alberta, 9211-116 Street NW, Edmonton, AB, T6G 1H9, Canada.
| |
Collapse
|
5
|
Aghazadeh F, Zheng B, Tavakoli M, Rouhani H. Motion Smoothness-Based Assessment of Surgical Expertise: The Importance of Selecting Proper Metrics. SENSORS (BASEL, SWITZERLAND) 2023; 23:3146. [PMID: 36991855 PMCID: PMC10057623 DOI: 10.3390/s23063146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 03/07/2023] [Accepted: 03/12/2023] [Indexed: 06/19/2023]
Abstract
The smooth movement of hand/surgical instruments is considered an indicator of skilled, coordinated surgical performance. Jerky surgical instrument movements or hand tremors can cause unwanted damages to the surgical site. Different methods have been used in previous studies for assessing motion smoothness, causing conflicting results regarding the comparison among surgical skill levels. We recruited four attending surgeons, five surgical residents, and nine novices. The participants conducted three simulated laparoscopic tasks, including peg transfer, bimanual peg transfer, and rubber band translocation. Tooltip motion smoothness was computed using the mean tooltip motion jerk, logarithmic dimensionless tooltip motion jerk, and 95% tooltip motion frequency (originally proposed in this study) to evaluate their capability of surgical skill level differentiation. The results revealed that logarithmic dimensionless motion jerk and 95% motion frequency were capable of distinguishing skill levels, indicated by smoother tooltip movements observed in high compared to low skill levels. Contrarily, mean motion jerk was not able to distinguish the skill levels. Additionally, 95% motion frequency was less affected by the measurement noise since it did not require the calculation of motion jerk, and 95% motion frequency and logarithmic dimensionless motion jerk yielded a better motion smoothness assessment outcome in distinguishing skill levels than mean motion jerk.
Collapse
Affiliation(s)
- Farzad Aghazadeh
- Department of Mechanical Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada;
| | - Bin Zheng
- Department of Surgery, University of Alberta, Edmonton, AB T6G 2B7, Canada
| | - Mahdi Tavakoli
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 2R3, Canada
| | - Hossein Rouhani
- Department of Mechanical Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada;
| |
Collapse
|
6
|
Singh H, Shih HT, Kal E, Bennett T, Wulf G. A distal external focus of attention facilitates compensatory coordination of body parts. J Sports Sci 2022; 40:2282-2291. [PMID: 36418176 DOI: 10.1080/02640414.2022.2150419] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
Many studies have shown that focusing on an intended movement effect that is farther away from the body (distal external focus) results in performance benefits relative to focusing on an effect that is closer to the body (proximal external focus) or focusing on the body itself (internal focus) (see, Chua, Jimenez-Diaz, Lewthwaite, Kim & Wulf, 2021). Furthermore, the advantages of a distal external focus seem to be particularly pronounced in skilled performers (Singh & Wulf, 2020). The present study examined whether such benefits of more distal attentional focus may be associated with enhanced functional variability. Volleyball players (n = 20) performed 60 overhand volleyball serves to a target. Using a within-participants design, the effects of a distal external focus (bullseye), proximal external focus (ball) and an internal focus (hand) were compared. The distal focus condition resulted in significantly higher accuracy scores than did the proximal and internal focus conditions. In addition, uncontrolled manifold analysis showed that functional variability (as measured by the index of synergy) was greatest in the distal focus condition. These findings suggest that a distal external focus on the task goal may enhance movement outcomes by optimising compensatory coordination of body parts.
Collapse
Affiliation(s)
- Harjiv Singh
- Department of Kinesiology and Nutrition Sciences, University of Nevada, Las Vegas, NV, USA.,Orlando Magic Basketball Club, Orlando, FL, USA
| | - Hui-Ting Shih
- Department of Kinesiology and Nutrition Sciences, University of Nevada, Las Vegas, NV, USA
| | - Elmar Kal
- Centre for Cognitive Neuroscience, Brunel University London, London, UK
| | - Tim Bennett
- Carnegie School of Sport, Leeds Beckett University, Leeds, UK
| | - Gabriele Wulf
- Department of Kinesiology and Nutrition Sciences, University of Nevada, Las Vegas, NV, USA
| |
Collapse
|
7
|
Sullivan JL, Pandey S, Byrne MD, O'Malley MK. Haptic Feedback Based on Movement Smoothness Improves Performance in a Perceptual-Motor Task. IEEE TRANSACTIONS ON HAPTICS 2022; 15:382-391. [PMID: 34752403 DOI: 10.1109/toh.2021.3126017] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
In many training scenarios, and in surgery in particular, feedback is provided to the trainee after the task has been performed, and the assessment is often qualitative in nature. In this paper, we demonstrate the effect of real-time objective performance feedback conveyed through a vibrotactile cue. Subjects performed a mirror-tracing task that requires coordination and dexterity similar in nature to that required in endovascular surgery. Movement smoothness, a characteristic associated with skilled and coordinated movement, was measured by spectral arc length, a frequency-domain measure of smoothness. The smoothness-based performance metric was encoded as a vibrotactile cue displayed on the user's arm. Performance on the mirror tracing task with smoothness-based feedback was compared to position-based feedback (where the subject was alerted when they moved outside the path boundary) and to a no vibrotactile feedback control condition. Subjects receiving smoothness-based feedback altered their task completion strategies, resulting in faster task completion times, but their accuracy was slightly worse overall than the other two groups. In procedures such as endovascular surgery, the reduction of procedure time that could be achieved with smoothness-based feedback training may be advantageous, despite the fact that accuracy was inferior to that observed with no feedback or position-based feedback.
Collapse
|
8
|
Vandevoorde K, Vollenkemper L, Schwan C, Kohlhase M, Schenck W. Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks. SENSORS 2022; 22:s22072481. [PMID: 35408094 PMCID: PMC9002555 DOI: 10.3390/s22072481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Revised: 03/18/2022] [Accepted: 03/20/2022] [Indexed: 11/03/2022]
Abstract
Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.
Collapse
|
9
|
Zruya O, Sharon Y, Kossowsky H, Forni F, Geftler A, Nisky I. A New Power Law Linking the Speed to the Geometry of Tool-Tip Orientation in Teleoperation of a Robot-Assisted Surgical System. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3193485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- Or Zruya
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Yarden Sharon
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Hanna Kossowsky
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Fulvio Forni
- Department of Engineering, University of Cambridge, Cambridge, U.K
| | - Alex Geftler
- Department of Orthopedic Surgery, Soroka Medical Center, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Ilana Nisky
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| |
Collapse
|
10
|
Abstract
The involvement of Robots and automated machines in different industries has increased drastically in recent years. Part of this revolution is accomplishing tasks previously performed by humans with advanced robots, which would replace the entire human workforce in the future. In some industries the workers are required to complete different operations in hazardous or difficult environments. Operations like these could be replaced with the use of tele-operated systems that have the capability of grasping objects in their surroundings, thus abandoning the need for the physical presence of the human operator at the area while still allowing control. In this research our goal is to create an assisting system that would improve the grasping of a human operator using a tele-operated robotic gripper and arm, while advising the operator but not forcing a solution. For a given set of objects we computed the optimal grasp to be achieved by the gripper, based on two grasp quality measures of our choosing (namely power grasp and precision grasp). We then tested the performance of different human subjects who tried to grasp the different objects with the tele-operated system, while comparing their success to unassisted and assisted grasping. Our goal is to create an assisting algorithm that would compute optimal grasps and might be integrated into a complete, state-of-the-art tele-operated system.
Collapse
|
11
|
Lefor AK, Harada K, Dosis A, Mitsuishi M. Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set II: learning curve analysis. Int J Comput Assist Radiol Surg 2021; 16:589-595. [PMID: 33723706 DOI: 10.1007/s11548-021-02339-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2020] [Accepted: 02/25/2021] [Indexed: 01/12/2023]
Abstract
PURPOSE The Johns Hopkins-Intuitive Gesture and Skill Assessment Working Set (JIGSAWS) dataset is used to develop robotic surgery skill assessment tools, but there has been no detailed analysis of this dataset. The aim of this study is to perform a learning curve analysis of the existing JIGSAWS dataset. METHODS Five trials were performed in JIGSAWS by eight participants (four novices, two intermediates and two experts) for three exercises (suturing, knot-tying and needle passing). Global Rating Scores and time, path length and movements were analyzed quantitatively and qualitatively by graphical analysis. RESULTS There are no significant differences in Global Rating Scale scores over time. Time in the suturing exercise and path length in needle passing had significant differences. Other kinematic parameters were not significantly different. Qualitative analysis shows a learning curve only for suturing. Cumulative sum analysis suggests completion of the learning curve for suturing by trial 4. CONCLUSIONS The existing JIGSAWS dataset does not show a quantitative learning curve for Global Rating Scale scores, or most kinematic parameters which may be due in part to the limited size of the dataset. Qualitative analysis shows a learning curve for suturing. Cumulative sum analysis suggests completion of the suturing learning curve by trial 4. An expanded dataset is needed to facilitate subset analyses.
Collapse
Affiliation(s)
- Alan Kawarai Lefor
- Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan.
| | - Kanako Harada
- Mechanical Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
- Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| | | | - Mamoru Mitsuishi
- Mechanical Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
- Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
12
|
Murali B, Belvroy VM, Pandey S, Bismuth J, Byrne MD, O'Malley MK. Velocity-Domain Motion Quality Measures for Surgical Performance Evaluation and Feedback. J Med Device 2021. [DOI: 10.1115/1.4049310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Abstract
Abstract
Endovascular navigation proficiency requires a significant amount of manual dexterity from surgeons. Objective performance measures derived from endovascular tool tip kinematics have been shown to correlate with expertise; however, such metrics have not yet been used during training as a basis for real-time performance feedback. This paper evaluates a set of velocity-based performance measures derived from guidewire motion to determine their suitability for online performance evaluation and feedback. We evaluated the endovascular navigation skill of 75 participants using three metrics (spectral arc length, average velocity, and idle time) as they steered tools to anatomical targets using a virtual reality simulator. First, we examined the effect of navigation task and experience level on performance and found that novice performance was significantly different from intermediate and expert performance. Then we computed correlations between measures calculated online and spectral arc length, our “gold standard” metric, calculated offline (at the end of the trial, using data from the entire trial). Our results suggest that average velocity and idle time calculated online are strongly and consistently correlated with spectral arc length computed offline, which was not the case when comparing spectral arc length computed online and offline. Average velocity and idle time, both time-domain based performance measures, are therefore more suitable measures than spectral arc length, a frequency-domain based metric, to use as the basis of online performance feedback. Future work is needed to determine how to best provide real-time performance feedback to endovascular surgery trainees based on these metrics.
Collapse
Affiliation(s)
- Barathwaj Murali
- Department of Mechanical Engineering, Rice University, Houston, TX 77005
| | - Viony M. Belvroy
- DeBakey Heart & Vascular Center, Houston Methodist Hospital, Houston, TX 77030
| | - Shivam Pandey
- Department of Psychological Sciences, Rice University, Houston, TX 77005
| | - Jean Bismuth
- DeBakey Heart & Vascular Center, Houston Methodist Hospital, Houston, TX 77030
| | - Michael D. Byrne
- Department of Psychological Sciences, Rice University, Houston, TX 77005
| | - Marcia K. O'Malley
- Department of Mechanical Engineering, Rice University, Houston, TX 77005
| |
Collapse
|
13
|
Mariani A, Colaci G, Da Col T, Sanna N, Vendrame E, Menciassi A, De Momi E. An Experimental Comparison Towards Autonomous Camera Navigation to Optimize Training in Robot Assisted Surgery. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2965067] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
14
|
Bahar L, Sharon Y, Nisky I. Surgeon-Centered Analysis of Robot-Assisted Needle Driving Under Different Force Feedback Conditions. Front Neurorobot 2020; 13:108. [PMID: 32038218 PMCID: PMC6993204 DOI: 10.3389/fnbot.2019.00108] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Accepted: 12/06/2019] [Indexed: 11/24/2022] Open
Abstract
Robotic assisted minimally invasive surgery (RAMIS) systems present many advantages to the surgeon and patient over open and standard laparoscopic surgery. However, haptic feedback, which is crucial for the success of many surgical procedures, is still an open challenge in RAMIS. Understanding the way that haptic feedback affects performance and learning can be useful in the development of haptic feedback algorithms and teleoperation control systems. In this study, we examined the performance and learning of inexperienced participants under different haptic feedback conditions in a task of surgical needle driving via a soft homogeneous deformable object-an artificial tissue. We designed an experimental setup to characterize their movement trajectories and the forces that they applied on the artificial tissue. Participants first performed the task in an open condition, with a standard surgical needle holder, followed by teleoperation in one of three feedback conditions: (1) no haptic feedback, (2) haptic feedback based on position exchange, and (3) haptic feedback based on direct recording from a force sensor, and then again with the open needle holder. To quantify the effect of different force feedback conditions on the quality of needle driving, we developed novel metrics that assess the kinematics of needle driving and the tissue interaction forces, and we combined our novel metrics with classical metrics. We analyzed the final teleoperated performance in each condition, the improvement during teleoperation, and the aftereffect of teleoperation on the performance when using the open needle driver. We found that there is no significant difference in the final performance and in the aftereffect between the 3 conditions. Only the two conditions with force feedback presented statistically significant improvement during teleoperation in several of the metrics, but when we compared directly between the improvements in the three different feedback conditions none of the effects reached statistical significance. We discuss possible explanations for the relative similarity in performance. We conclude that we developed several new metrics for the quality of surgical needle driving, but even with these detailed metrics, the advantage of state of the art force feedback methods to tasks that require interaction with homogeneous soft tissue is questionable.
Collapse
Affiliation(s)
| | | | - Ilana Nisky
- Department of Biomedical Engineering, Zlotowski Center of Neuroscience, Ben-Gurion University of the Negev, Be'er Sheva, Israel
| |
Collapse
|
15
|
Mishima T, Yoshida K, Takayasu K, Watanabe M, Kinoshita H, Matsuda T. Analysis of Gripping Force on a Master Controller During Simulated Robotic Surgery. J Endourol 2019; 33:802-808. [DOI: 10.1089/end.2019.0240] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Affiliation(s)
- Takao Mishima
- Department of Urology and Andrology, Kansai Medical University, Osaka, Japan
| | - Kenji Yoshida
- Department of Urology and Andrology, Kansai Medical University, Osaka, Japan
| | - Kenta Takayasu
- Department of Urology and Andrology, Kansai Medical University, Osaka, Japan
| | - Masato Watanabe
- Department of Urology and Andrology, Kansai Medical University, Osaka, Japan
| | - Hidefumi Kinoshita
- Department of Urology and Andrology, Kansai Medical University, Osaka, Japan
| | - Tadashi Matsuda
- Department of Urology and Andrology, Kansai Medical University, Osaka, Japan
| |
Collapse
|
16
|
Abdelaal AE, Sakr M, Avinash A, Mohammed SK, Bajwa AK, Sahni M, Hor S, Fels S, Salcudean SE. Play Me Back: A Unified Training Platform for Robotic and Laparoscopic Surgery. IEEE Robot Autom Lett 2019. [DOI: 10.1109/lra.2018.2890209] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
17
|
Kun Y, Hubert J, Bin L, Huan WX. Self-debriefing Model Based on an Integrated Video-Capture System: An Efficient Solution to Skill Degradation. JOURNAL OF SURGICAL EDUCATION 2019; 76:362-369. [PMID: 30292454 DOI: 10.1016/j.jsurg.2018.08.017] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2018] [Revised: 07/09/2018] [Accepted: 08/19/2018] [Indexed: 05/10/2023]
Abstract
OBJECTIVE Video-based teaching is considered highly effective in debriefing, especially in minimally invasive surgeries. In this study, the benefits of using a new integrated video recording system, were investigated and compared to those of the standard basic skills robotic training procedure. DESIGN Fifty residents from the 2nd and 3rd year medical faculty without any experience of robot usage or laparoscopy were randomized into 2 groups: group A--a natural self-training group without a trainer, and group B--a self-training group assisted by an integrated video recording system during training. The training was divided into four 2-hour sessions, with a 72-hour delay between each session. Two tasks were selected for testing on the dV-Trainer, a virtual reality based robotic simulator: Match board 2 and Thread the Rings 1. After each session, the practice video recorded by the system of group B was transferred to the residents' smartphones for self-debriefing. At the end of each session, the performance score was evaluated automatically by using the simulator to plot learning curves A and B. RESULTS Group A showed a significant drop in performance score due to skill decay caused by the 72-hour delay. Group B exhibited a regular stepwise rising learning curve. At the end of the training, group B showed a significantly higher performance score both in Match board 2 and Thread the Rings 1. The autoanalysis and capture function, which selects only the critical errors and most valuable parts, could facilitate time saving. CONCLUSIONS The use of an integrated video recording system makes the self-manipulated protocol with own smartphone feasible to improve training efficiency and overcome the skill decay during robotic surgical training.
Collapse
Affiliation(s)
- Yang Kun
- Department of Urology, ZhongNan hospital, Wuhan University, China; Medicine - remote mapping associated laboratory, Wuhan University, China
| | - Jacques Hubert
- Department of Urology, CHU Nancy, Nancy, France; IADI/Inserm U947, Lorraine University, Nancy, France
| | - Luo Bin
- Medicine - remote mapping associated laboratory, Wuhan University, China; State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China
| | - Wang Xing Huan
- Department of Urology, ZhongNan hospital, Wuhan University, China; Medicine - remote mapping associated laboratory, Wuhan University, China.
| |
Collapse
|
18
|
Forestier G, Petitjean F, Senin P, Despinoy F, Huaulmé A, Fawaz HI, Weber J, Idoumghar L, Muller PA, Jannin P. Surgical motion analysis using discriminative interpretable patterns. Artif Intell Med 2018; 91:3-11. [PMID: 30172445 DOI: 10.1016/j.artmed.2018.08.002] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2017] [Revised: 07/06/2018] [Accepted: 08/13/2018] [Indexed: 11/29/2022]
Abstract
OBJECTIVE The analysis of surgical motion has received a growing interest with the development of devices allowing their automatic capture. In this context, the use of advanced surgical training systems makes an automated assessment of surgical trainee possible. Automatic and quantitative evaluation of surgical skills is a very important step in improving surgical patient care. MATERIAL AND METHOD In this paper, we present an approach for the discovery and ranking of discriminative and interpretable patterns of surgical practice from recordings of surgical motions. A pattern is defined as a series of actions or events in the kinematic data that together are distinctive of a specific gesture or skill level. Our approach is based on the decomposition of continuous kinematic data into a set of overlapping gestures represented by strings (bag of words) for which we compute comparative numerical statistic (tf-idf) enabling the discriminative gesture discovery via its relative occurrence frequency. RESULTS We carried out experiments on three surgical motion datasets. The results show that the patterns identified by the proposed method can be used to accurately classify individual gestures, skill levels and surgical interfaces. We also present how the patterns provide a detailed feedback on the trainee skill assessment. CONCLUSIONS The proposed approach is an interesting addition to existing learning tools for surgery as it provides a way to obtain a feedback on which parts of an exercise have been used to classify the attempt as correct or incorrect.
Collapse
Affiliation(s)
- Germain Forestier
- IRIMAS, Université de Haute-Alsace, Mulhouse, France; Faculty of Information Technology, Monash University, Melbourne, Australia.
| | - François Petitjean
- Faculty of Information Technology, Monash University, Melbourne, Australia.
| | - Pavel Senin
- Los Alamos National Laboratory, University Of Hawai'i at Mānoa, United States.
| | - Fabien Despinoy
- Univ Rennes, Inserm, LTSI - UMR_S 1099, F35000 Rennes, France.
| | - Arnaud Huaulmé
- Univ Rennes, Inserm, LTSI - UMR_S 1099, F35000 Rennes, France.
| | | | | | | | | | - Pierre Jannin
- Univ Rennes, Inserm, LTSI - UMR_S 1099, F35000 Rennes, France.
| |
Collapse
|
19
|
Buzzi J, De Momi E, Nisky I. An Uncontrolled Manifold Analysis of Arm Joint Variability in Virtual Planar Position and Orientation Telemanipulation. IEEE Trans Biomed Eng 2018; 66:391-402. [PMID: 29993394 DOI: 10.1109/tbme.2018.2842458] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
OBJECTIVE In teleoperated robot-assisted tasks, the user interacts with manipulators to finely control remote tools. Manipulation of robotic devices, characterized by specific kinematic and dynamic proprieties, is a complex task for the human sensorimotor system due to the inherent biomechanical and neuronal redundancies that characterize the human arm and its control. We investigate how master devices with different kinematics structures and how different task constraints influence users capabilities in exploiting arm redundancy. METHODS A virtual teleoperation workbench was designed and the arm kinematics of seven users was acquired during the execution of two planar virtual tasks, involving either the control of position only or position-orientation of a tool. Using the uncontrolled manifold analysis of arm joint variability, we estimated the logarithmic ratio between the task irrelevant and the task relevant manifolds ( Rv). RESULTS The Rv values obtained in the position-orientation task were higher than in the position only task, while no differences were found between the master devices. A modulation of Rv was found through the execution of the position task and a positive correlation was found between task performance and redundancy exploitation. CONCLUSION Users exploited additional portions of arm redundancy when dealing with the tool orientation. The Rv modulation seems influenced by the task constraints and by the users possibility of reconfiguring the arm position. SIGNIFICANCE This paper advances the general understanding of the exploitation of arm redundancy in complex tasks, and can improve the development of future robotic devices.
Collapse
|
20
|
Wang Z, Majewicz Fey A. Human-centric predictive model of task difficulty for human-in-the-loop control tasks. PLoS One 2018; 13:e0195053. [PMID: 29621301 PMCID: PMC5886487 DOI: 10.1371/journal.pone.0195053] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2017] [Accepted: 03/07/2018] [Indexed: 11/18/2022] Open
Abstract
Quantitatively measuring the difficulty of a manipulation task in human-in-the-loop control systems is ill-defined. Currently, systems are typically evaluated through task-specific performance measures and post-experiment user surveys; however, these methods do not capture the real-time experience of human users. In this study, we propose to analyze and predict the difficulty of a bivariate pointing task, with a haptic device interface, using human-centric measurement data in terms of cognition, physical effort, and motion kinematics. Noninvasive sensors were used to record the multimodal response of human user for 14 subjects performing the task. A data-driven approach for predicting task difficulty was implemented based on several task-independent metrics. We compare four possible models for predicting task difficulty to evaluated the roles of the various types of metrics, including: (I) a movement time model, (II) a fusion model using both physiological and kinematic metrics, (III) a model only with kinematic metrics, and (IV) a model only with physiological metrics. The results show significant correlation between task difficulty and the user sensorimotor response. The fusion model, integrating user physiology and motion kinematics, provided the best estimate of task difficulty (R2 = 0.927), followed by a model using only kinematic metrics (R2 = 0.921). Both models were better predictors of task difficulty than the movement time model (R2 = 0.847), derived from Fitt's law, a well studied difficulty model for human psychomotor control.
Collapse
Affiliation(s)
- Ziheng Wang
- Department of Mechanical Engineering, The University of Texas at Dallas, Richardson, TX 75080, United States of America
| | - Ann Majewicz Fey
- Department of Mechanical Engineering, The University of Texas at Dallas, Richardson, TX 75080, United States of America
- Department of Surgery, UT Southwestern Medical Center, Dallas, TX 75390, United States of America
| |
Collapse
|
21
|
Jiang J, Xing Y, Wang S, Liang K. Evaluation of robotic surgery skills using dynamic time warping. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2017; 152:71-83. [PMID: 29054262 DOI: 10.1016/j.cmpb.2017.09.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/15/2016] [Revised: 06/21/2017] [Accepted: 09/11/2017] [Indexed: 06/07/2023]
Abstract
BACKGROUND AND OBJECTIVE accompanied with the wide acceptance of robot assisted minimally invasive surgery (RMIS), the demand for efficient and objective surgical skills evaluation method is increased. Recently, with the development of medical engineering technology, several evaluation methods have been proposed. Among them, kinematic analysis, an unsupervised and data-based method, has been accepted by many researchers. However, this method is still limited by the number of metrics and unconvinced scoring system. This paper aims to propose a new evaluation method to assess surgical skills efficiently and objectively. METHODS this research proposed an efficient and effective surgical skills evaluation algorithm which used the trajectories of instrument tip and dynamic time warping (DTW) to provide trainees with real-time and summative feedback. The optimum trajectories based on 'Therbligs' theory was designed as a template. DTW algorithm was used to align actual trajectories to optimum trajectories with an evaluating indicator designed to emphasize the crucial motion features in surgical skills evaluation. The real-time feedback was obtained through a sliding time window to help trainees improve learning efficiency. RESULTS experts (n = 2) and novices (n = 8) were invited to complete the peg transfer tasks and 60 instrument tip trajectories were assessed by the proposed algorithm. Significant differences between different groups were observed (experts' right trajectories versus experts' left trajectories, p = 0.0002; experts' right trajectories versus novices' right trajectories, p = 0.0124). In addition, evaluation results of trajectories with operational mistakes were significantly different from those of others. CONCLUSIONS the proposed evaluation method showed its advantages in distinguishing and evaluating surgical performance. Given its ability to evaluate the performance based on kinematic information, the proposed evaluation method can be further developed in the future. Furthermore, because the proposed method can provide real-time feedback, it also has the potential to be a monitoring system in operation room.
Collapse
Affiliation(s)
- Jingyu Jiang
- Key Lab for Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300000, China
| | - Yuan Xing
- Key Lab for Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300000, China.
| | - Shuxin Wang
- Key Lab for Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300000, China
| | - Ke Liang
- Key Lab for Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300000, China
| |
Collapse
|
22
|
Yang K, Zhen H, Hubert N, Perez M, Wang XH, Hubert J. From dV-Trainer to Real Robotic Console: The Limitations of Robotic Skill Training. JOURNAL OF SURGICAL EDUCATION 2017; 74:1074-1080. [PMID: 28462814 DOI: 10.1016/j.jsurg.2017.03.006] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/2016] [Revised: 02/05/2017] [Accepted: 03/21/2017] [Indexed: 06/07/2023]
Abstract
OBJECTIVES To investigate operators' performance quality, mental stress, and ergonomic habits through a training curriculum on robotic simulators. DESIGN Forty volunteers without robotic surgery experience were recruited to practice 2 exercises on a dV-Trainer (dVT) for 14 hours. The simulator software (M-scorea) provided an automatic evaluation of the overall score for the surgeons' performance. Each participant provided a subjective difficulty score (validity to be proven) for each exercise. Their ergonomic habits were evaluated based on the workspace range and armrest load-validated criteria for evaluating the proficiency of using the armrest. They then repeated the same tasks on a da Vinci Surgical Skill Simulator for a final-level test. Their final scores were compared with their initial scores and the scores of 5 experts on the da Vinci Surgical Skill Simulator. RESULTS A total of 14 hours of training on the dVT significantly improved the surgeons' performance scores to the expert level with a significantly reduced workload, but their ergonomic score was still far from the expert level. CONCLUSION Sufficient training on the dVT improves novices' performance, reduces psychological stress, and inculcates better ergonomic habits. Among the evaluated criteria, novices had the most difficulty in achieving expert levels of ergonomic skills. The training benefits of robotic surgery simulators should be determined with quantified variables. The detection of the limitations during robotic training curricula could guide the targeted training and improve the training effect.
Collapse
Affiliation(s)
- Kun Yang
- Department of Urology, Zhongnan Hospital, Wuhan University, Wuhan, China; IADI/Inserm U947, Lorraine University, Nancy, France; Department of Emergency and General Surgery, CHU Nancy, Nancy, France
| | - Hang Zhen
- Department of Urology, Zhongnan Hospital, Wuhan University, Wuhan, China
| | - Nicolas Hubert
- IADI/Inserm U947, Lorraine University, Nancy, France; Department of Urology, CHU Nancy, Nancy, France
| | - Manuela Perez
- IADI/Inserm U947, Lorraine University, Nancy, France; Department of Emergency and General Surgery, CHU Nancy, Nancy, France
| | | | - Jacques Hubert
- IADI/Inserm U947, Lorraine University, Nancy, France; Department of Emergency and General Surgery, CHU Nancy, Nancy, France.
| |
Collapse
|
23
|
|
24
|
Buzzi J, Gatti C, Ferrigno G, De Momi E. Analysis of Joint and Hand Impedance During Teleoperation and Free-Hand Task Execution. IEEE Robot Autom Lett 2017. [DOI: 10.1109/lra.2017.2678546] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|
25
|
Liang K, Xing Y, Li J, Wang S, Li A, Li J. Motion control skill assessment based on kinematic analysis of robotic end-effector movements. Int J Med Robot 2017; 14. [PMID: 28660644 DOI: 10.1002/rcs.1845] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2016] [Revised: 05/08/2017] [Accepted: 05/08/2017] [Indexed: 11/05/2022]
Abstract
BACKGROUND The performance of robotic end-effector movements can reflect the user's operation skill difference in robot-assisted minimally invasive surgery. This study quantified the trade-off of speed-accuracy-stability by kinematic analysis of robotic end-effector movements to assess the motion control skill of users with different levels of experience. METHODS Using 'MicroHand S' system, 10 experts, 10 residents and 10 novices performed single-hand test and bimanual coordination test. Eight metrics based on the movements of robotic end-effectors were applied to evaluate the users' performance. RESULTS In the single-hand test, experts outperformed other groups except for movement speed; in the bimanual coordination test, experts also performed better except for movement time and movement speed. No statistically significant difference in performance was found between residents and novices. CONCLUSIONS The kinematic differences obtained from the movements of robotic end-effectors can be applied to assess the motion control skill of users with different skill levels.
Collapse
Affiliation(s)
- Ke Liang
- Department of Mechanical Engineering, Tianjin University, Tianjin, China
| | - Yuan Xing
- School of Mechanical Engineering, Tianjin University, Tianjin, China
| | - Jianmin Li
- School of Mechanical Engineering, Tianjin University, Tianjin, China
| | - Shuxin Wang
- School of Mechanical Engineering, Tianjin University, Tianjin, China
| | - Aimin Li
- PLA Rocket Forces General Hospital, Beijing, China
| | - Jinhua Li
- School of Mechanical Engineering, Tianjin University, Tianjin, China
| |
Collapse
|
26
|
Brown JD, O Brien CE, Leung SC, Dumon KR, Lee DI, Kuchenbecker KJ. Using Contact Forces and Robot Arm Accelerations to Automatically Rate Surgeon Skill at Peg Transfer. IEEE Trans Biomed Eng 2016; 64:2263-2275. [PMID: 28113295 DOI: 10.1109/tbme.2016.2634861] [Citation(s) in RCA: 35] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
OBJECTIVE Most trainees begin learning robotic minimally invasive surgery by performing inanimate practice tasks with clinical robots such as the Intuitive Surgical da Vinci. Expert surgeons are commonly asked to evaluate these performances using standardized five-point rating scales, but doing such ratings is time consuming, tedious, and somewhat subjective. This paper presents an automatic skill evaluation system that analyzes only the contact force with the task materials, the broad-bandwidth accelerations of the robotic instruments and camera, and the task completion time. METHODS We recruited N = 38 participants of varying skill in robotic surgery to perform three trials of peg transfer with a da Vinci Standard robot instrumented with our Smart Task Board. After calibration, three individuals rated these trials on five domains of the Global Evaluative Assessment of Robotic Skill (GEARS) structured assessment tool, providing ground-truth labels for regression and classification machine learning algorithms that predict GEARS scores based on the recorded force, acceleration, and time signals. RESULTS Both machine learning approaches produced scores on the reserved testing sets that were in good to excellent agreement with the human raters, even when the force information was not considered. Furthermore, regression predicted GEARS scores more accurately and efficiently than classification. CONCLUSION A surgeon's skill at robotic peg transfer can be reliably rated via regression using features gathered from force, acceleration, and time sensors external to the robot. SIGNIFICANCE We expect improved trainee learning as a result of providing these automatic skill ratings during inanimate task practice on a surgical robot.
Collapse
|
27
|
Yang K, Perez M, Hossu G, Hubert N, Perrenot C, Hubert J. “Alarm-corrected” ergonomic armrest use could improve learning curves of novices on robotic simulator. Surg Endosc 2016; 31:100-106. [DOI: 10.1007/s00464-016-4934-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2015] [Accepted: 04/09/2016] [Indexed: 11/24/2022]
|
28
|
Petersen JG, Bowyer SA, Baena FRY. Mass and Friction Optimization for Natural Motion in Hands-On Robotic Surgery. IEEE T ROBOT 2016. [DOI: 10.1109/tro.2015.2507163] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
29
|
Yang K, Perez M, Perrenot C, Hubert N, Felblinger J, Hubert J. A new system for evaluation of armrest use in robotic surgery and validation of a new ergonomic concept - armrest load. Int J Med Robot 2016; 12:604-612. [DOI: 10.1002/rcs.1723] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2015] [Revised: 11/23/2015] [Accepted: 11/23/2015] [Indexed: 12/28/2022]
Affiliation(s)
- K. Yang
- IADI/Inserm U947; Lorraine University; Nancy France
- Wuhan University; Department of Urology, Zhongnan Hospital; Wuhan People's Republic of China
- Department of Urology; CHU Nancy; France
| | - M. Perez
- IADI/Inserm U947; Lorraine University; Nancy France
- Department of Emergency and General Surgery; CHU Nancy; France
| | - C. Perrenot
- IADI/Inserm U947; Lorraine University; Nancy France
- Department of Emergency and General Surgery; CHU Nancy; France
| | - N. Hubert
- IADI/Inserm U947; Lorraine University; Nancy France
- Department of Urology; CHU Nancy; France
| | | | - J. Hubert
- IADI/Inserm U947; Lorraine University; Nancy France
- Department of Urology; CHU Nancy; France
| |
Collapse
|
30
|
Shamaei K, Kim LH, Okamura AM. Design and evaluation of a trilateral shared-control architecture for teleoperated training robots. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2016; 2015:4887-93. [PMID: 26737388 DOI: 10.1109/embc.2015.7319488] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Multilateral teleoperated robots can be used to train humans to perform complex tasks that require collaborative interaction and expert supervision, such as laparoscopic surgical procedures. In this paper, we explain the design and performance evaluation of a shared-control architecture that can be used in trilateral teleoperated training robots. The architecture includes dominance and observation factors inspired by the determinants of motor learning in humans, including observational practice, focus of attention, feedback and augmented feedback, and self-controlled practice. Toward the validation of such an architecture, we (1) verify the stability of a trilateral system by applying Llewellyn's criterion on a two-port equivalent architecture, and (2) demonstrate that system transparency remains generally invariant across relevant observation factors and movement frequencies. In a preliminary experimental study, a dyad of two human users (one novice, one expert) collaborated on the control of a robot to follow a trajectory. The experiment showed that the framework can be used to modulate the efforts of the users and adjust the source and level of haptic feedback to the novice user.
Collapse
|
31
|
Abstract
Finger tracking has the potential to expand haptic research and applications, as eye tracking has done in vision research. In research applications, it is desirable to know the bias and variance associated with a finger-tracking method. However, assessing the bias and variance of a deterministic method is not straightforward. Multiple measurements of the same finger position data will not produce different results, implying zero variance. Here, we present a method of assessing deterministic finger-tracking variance and bias through comparison to a non-deterministic measure. A proof-of-concept is presented using a video-based finger-tracking algorithm developed for the specific purpose of tracking participant fingers during a psychological research study. The algorithm uses ridge detection on videos of the participant's hand, and estimates the location of the right index fingertip. The algorithm was evaluated using data from four participants, who explored tactile maps using only their right index finger and all right-hand fingers. The algorithm identified the index fingertip in 99.78 % of one-finger video frames and 97.55 % of five-finger video frames. Although the algorithm produced slightly biased and more dispersed estimates relative to a human coder, these differences (x=0.08 cm, y=0.04 cm) and standard deviations (σ x =0.16 cm, σ y =0.21 cm) were small compared to the size of a fingertip (1.5-2.0 cm). Some example finger-tracking results are provided where corrections are made using the bias and variance estimates.
Collapse
|
32
|
Hassan SO, Dudhia J, Syed LH, Patel K, Farshidpour M, Cunningham SC, Kowdley GC. Conventional Laparoscopic vs Robotic Training: Which is Better for Naive Users? A Randomized Prospective Crossover Study. JOURNAL OF SURGICAL EDUCATION 2015; 72:592-599. [PMID: 25687957 DOI: 10.1016/j.jsurg.2014.12.008] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2014] [Revised: 11/18/2014] [Accepted: 12/16/2014] [Indexed: 06/04/2023]
Abstract
OBJECTIVE Robotic training (RT) using the da Vinci skills simulator and conventional training (CT) using a laparoscopic "training box" are both used to augment operative skills in minimally invasive surgery. The current study tests the hypothesis that skill acquisition is more rapid using RT than using CT among naive learners. DESIGN AND PARTICIPANTS A total of 40 subjects without laparoscopic or robotic surgical experience were enrolled and randomized to begin with either RT or CT. Then, 2 specific RT tasks were reproduced for CT and repeated 5 times each with RT and CT. Time and quality indicators were measured quantitatively. A crossover technique was used to control for in-study experience bias. RESULTS The tasks "pick and place jacks" (PP) and "thread the rings" (TR) were achieved faster with RT than with CT despite crossover (p < 0.0001). An RT-favoring difference was observed in speed for both tasks when changing modality. Percentage improvement with increasing trials was similar for RT and CT: RT completion time averaged 39 seconds and 211 seconds (PP and TR, respectively), compared with 65 seconds and 362 seconds when using CT (p < 0.0001); final improvement averaged 26% and 46% for RT (PP and TR, respectively) vs 31% and 47% for CT (p was 0.76 for PP and 0.20 for TR). Within the PP task, RT times averaged 41 seconds without previous CT experience vs 35 seconds with previous CT experience (p = 0.20); CT times averaged 61 seconds without and 69 seconds with previous RT experience (p = 0.48). Comparable times for the TR task were 212 seconds vs 216 seconds (p = 0.66) and 388 seconds vs 334 seconds (p = 0.17). Both instrument collisions and excessive force occurred more commonly for RT than for CT within the TR task (p < 0.0001). CONCLUSIONS Speeds were faster overall with RT than with CT, but the percentage of speed improvement with trials was similar, suggesting similar learning curves, with minimal transfer effect appreciated.
Collapse
Affiliation(s)
- Syed Omar Hassan
- Department of Surgery, Saint Agnes Hospital, Baltimore, Maryland
| | - Jaimin Dudhia
- Department of Surgery, Saint Agnes Hospital, Baltimore, Maryland
| | - Labiq H Syed
- Department of Surgery, Saint Agnes Hospital, Baltimore, Maryland
| | - Kalpesh Patel
- Department of Oncology, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | | | | | - Gopal C Kowdley
- Department of Surgery, Saint Agnes Hospital, Baltimore, Maryland.
| |
Collapse
|
33
|
Jarc AM, Nisky I. Robot-assisted surgery: an emerging platform for human neuroscience research. Front Hum Neurosci 2015; 9:315. [PMID: 26089785 PMCID: PMC4455232 DOI: 10.3389/fnhum.2015.00315] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2015] [Accepted: 05/18/2015] [Indexed: 12/26/2022] Open
Abstract
Classic studies in human sensorimotor control use simplified tasks to uncover fundamental control strategies employed by the nervous system. Such simple tasks are critical for isolating specific features of motor, sensory, or cognitive processes, and for inferring causality between these features and observed behavioral changes. However, it remains unclear how these theories translate to complex sensorimotor tasks or to natural behaviors. Part of the difficulty in performing such experiments has been the lack of appropriate tools for measuring complex motor skills in real-world contexts. Robot-assisted surgery (RAS) provides an opportunity to overcome these challenges by enabling unobtrusive measurements of user behavior. In addition, a continuum of tasks with varying complexity-from simple tasks such as those in classic studies to highly complex tasks such as a surgical procedure-can be studied using RAS platforms. Finally, RAS includes a diverse participant population of inexperienced users all the way to expert surgeons. In this perspective, we illustrate how the characteristics of RAS systems make them compelling platforms to extend many theories in human neuroscience, as well as, to develop new theories altogether.
Collapse
Affiliation(s)
- Anthony M Jarc
- Medical Research, Intuitive Surgical, Inc. Sunnyvale, CA, USA
| | - Ilana Nisky
- Biomedical Engineering, Ben-Gurion University of the Negev Beer Sheva, Israel
| |
Collapse
|
34
|
Nisky I, Hsieh MH, Okamura AM. Uncontrolled manifold analysis of arm joint angle variability during robotic teleoperation and freehand movement of surgeons and novices. IEEE Trans Biomed Eng 2014; 61:2869-81. [PMID: 24967980 PMCID: PMC8085739 DOI: 10.1109/tbme.2014.2332359] [Citation(s) in RCA: 43] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Teleoperated robot-assisted surgery (RAS) is used to perform a wide variety of minimally invasive procedures. However, current understanding of the effect of robotic manipulation on the motor coordination of surgeons is limited. Recent studies in human motor control suggest that we optimize hand movement stability and task performance while minimizing control effort and improving robustness to unpredicted disturbances. To achieve this, the variability of joint angles and muscle activations is structured to reduce task-relevant variability and increase task-irrelevant variability. In this study, we determine whether teleoperation of a da Vinci Si surgical system in a nonclinical task of simple planar movements changes this structure of variability in experienced surgeons and novices. To answer this question, we employ the UnControlled manifold analysis that partitions users' joint angle variability into task-irrelevant and task-relevant manifolds. We show that experienced surgeons coordinate their joint angles to stabilize hand movements more than novices, and that the effect of teleoperation depends on experience--experts increase teleoperated stabilization relative to freehand whereas novices decrease it. We suggest that examining users' exploitation of the task-irrelevant manifold for stabilization of hand movements may be applied to: (1) evaluation and optimization of teleoperator design and control parameters, and (2) skill assessment and optimization of training in RAS.
Collapse
|