1
|
Kim YG, Shim JW, Gimm G, Kang S, Rhee W, Lee JH, Kim BS, Yoon D, Kim M, Cho M, Kim S. Speech-mediated manipulation of da Vinci surgical system for continuous surgical flow. Biomed Eng Lett 2025; 15:117-129. [PMID: 39781059 PMCID: PMC11704117 DOI: 10.1007/s13534-024-00429-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2024] [Revised: 09/05/2024] [Accepted: 09/17/2024] [Indexed: 01/12/2025] Open
Abstract
With the advent of robot-assisted surgery, user-friendly technologies have been applied to the da Vinci surgical system (dVSS), and their efficacy has been validated in worldwide surgical fields. However, further improvements are required to the traditional manipulation methods, which cannot control an endoscope and surgical instruments simultaneously. This study proposes a speech recognition control interface (SRCI) for controlling the endoscope via speech commands while manipulating surgical instruments to replace the traditional method. The usability-focused comparisons of the newly proposed SRCI-based and the traditional manipulation method were conducted based on ISO 9241-11. 20 surgeons and 18 novices evaluated both manipulation methods through the line tracking task (LTT) and sea spike pod task (SSPT). After the tasks, they responded to the globally reliable questionnaires: after-scenario questionnaire (ASQ), system usability scale (SUS), and NASA task load index (TLX). The completion times in the LTT and SSPT using the proposed method were 44.72% and 26.59% respectively less than the traditional method, which shows statistically significant differences (p < 0.001). The overall results of ASQ, SUS, and NASA TLX were positive for the proposed method, especially substantial reductions in the workloads such as physical demands and efforts (p < 0.05). The proposed speech-mediated method can be a candidate suitable for the simultaneous manipulation of an endoscope and surgical instruments in dVSS-used robotic surgery. Therefore, it can replace the traditional method when controlling the endoscope while manipulating the surgical instruments, which contributes to enabling the continuous surgical flow in operations consequentially. Supplementary Information The online version contains supplementary material available at 10.1007/s13534-024-00429-5.
Collapse
Affiliation(s)
- Young Gyun Kim
- Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| | - Jae Woo Shim
- Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| | - Geunwu Gimm
- Department of Biomedical Engineering, Seoul National University College of Medicine, 103 Daehak-ro, Jongno- gu, Seoul, 03080 Republic of Korea
| | - Seongjoon Kang
- Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| | - Wounsuk Rhee
- Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080 Republic of Korea
| | - Jong Hyeon Lee
- Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| | - Byeong Soo Kim
- Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| | - Dan Yoon
- Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| | - Myungjoon Kim
- MedInTech Inc., 60 Daehak-ro, Jongno-gu, Seoul, 03100 Republic of Korea
| | - Minwoo Cho
- Department of Transdisciplinary Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080 Republic of Korea
- Department of Medicine, Seoul National University College of Medicine, 103 Daehak-ro, Jongno-gu, Seoul, 03080 Republic of Korea
| | - Sungwan Kim
- Department of Biomedical Engineering, Seoul National University College of Medicine, 103 Daehak-ro, Jongno- gu, Seoul, 03080 Republic of Korea
- Artificial Intelligence Institute, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826 Republic of Korea
| |
Collapse
|
2
|
Bourdillon AT, Garg A, Wang H, Woo YJ, Pavone M, Boyd J. Integration of Reinforcement Learning in a Virtual Robotic Surgical Simulation. Surg Innov 2023; 30:94-102. [PMID: 35503302 DOI: 10.1177/15533506221095298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Background. The revolutions in AI hold tremendous capacity to augment human achievements in surgery, but robust integration of deep learning algorithms with high-fidelity surgical simulation remains a challenge. We present a novel application of reinforcement learning (RL) for automating surgical maneuvers in a graphical simulation.Methods. In the Unity3D game engine, the Machine Learning-Agents package was integrated with the NVIDIA FleX particle simulator for developing autonomously behaving RL-trained scissors. Proximal Policy Optimization (PPO) was used to reward movements and desired behavior such as movement along desired trajectory and optimized cutting maneuvers along the deformable tissue-like object. Constant and proportional reward functions were tested, and TensorFlow analytics was used to informed hyperparameter tuning and evaluate performance.Results. RL-trained scissors reliably manipulated the rendered tissue that was simulated with soft-tissue properties. A desirable trajectory of the autonomously behaving scissors was achieved along 1 axis. Proportional rewards performed better compared to constant rewards. Cumulative reward and PPO metrics did not consistently improve across RL-trained scissors in the setting for movement across 2 axes (horizontal and depth).Conclusion. Game engines hold promising potential for the design and implementation of RL-based solutions to simulated surgical subtasks. Task completion was sufficiently achieved in one-dimensional movement in simulations with and without tissue-rendering. Further work is needed to optimize network architecture and parameter tuning for increasing complexity.
Collapse
Affiliation(s)
| | - Animesh Garg
- Vector Institute and Department of Computer Science, University of Toronto, Toronto, ON, Canada
| | - Hanjay Wang
- Department of Cardiothoracic Surgery, 198869Stanford University, Stanford, CA, USA
| | - Y Joseph Woo
- Department of Cardiothoracic Surgery, 198869Stanford University, Stanford, CA, USA.,Department of Bioengineering, 198869Stanford University, Stanford, CA, USA
| | - Marco Pavone
- Department of Aeronautics and Astronautics, 198869Stanford University, Stanford, CA, USA
| | - Jack Boyd
- Department of Cardiothoracic Surgery, 198869Stanford University, Stanford, CA, USA
| |
Collapse
|
3
|
Lin Y, Zhang C, Liu C, Ma X, Yang Q, Guan B, Liu Z. Imaging-Navigated Surgery. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2023; 1199:87-106. [PMID: 37460728 DOI: 10.1007/978-981-32-9902-3_5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2023]
Abstract
It is vitally important to guide or navigate therapeutic proceedings with a direct and visual approach in order to carefully undertake precision medical manipulations and efficiently evaluate the treatments. Imaging-navigated surgery is one of the common and prevailing technologies to realize this target, and more importantly it merges visualized medicine into next-generation theranostic paradigms in modern medicine. Endoscopes, surgical robots, and nanorobots are three major domains in terms of imaging-navigated surgery. The history of endoscopy has seen upgraded developments since the early 1800s. In contrast, surgical robots have been widely used and investigated in recent years, and they came into clinical uses only in the past decades. Nanorobots which closely depend on innovated and multifunctional biomaterials are still in their infancy. All these imaging-navigated technologies show similar and apparent advantages such as minimal invasiveness, minimized pain, positive prognosis, and relatively expected recovery, which have greatly improved surgery efficiency and patients' life quality. Therefore, the imaging-navigated surgery will be discussed in this chapter, and advanced clinical and preclinical medical applications will also be demonstrated for a diverse readers and comprehensive understanding.
Collapse
Affiliation(s)
- Yandai Lin
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Chen Zhang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Chenxi Liu
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Xinyong Ma
- Division of Academic & Cultural Activities, Academic Divisions of the Chinese Academy of Sciences, Beijing, China
| | - Qiang Yang
- Tianjin Hospital, Tianjin University, Tianjin, China
| | - Binggang Guan
- Tianjin Hospital, Tianjin University, Tianjin, China
| | - Zhe Liu
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China.
| |
Collapse
|
4
|
Zhou D, Aoyama Y, Takeyama H, Tadano K, Haraguchi D. Automation of Intraoperative Tool Changing for Robot-Assisted Laparoscopic Surgery. JOURNAL OF ROBOTICS AND MECHATRONICS 2022. [DOI: 10.20965/jrm.2022.p1216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Intraoperative tool change is a time-consuming and labor-intensive task for robot-assisted laparoscopic surgery. Serial multi-DOF manipulators are potential devices for realizing automatic intraoperative tool changes because of the layout flexibility and motion range, and multi-DOF makes it feasible for the manipulator to access and fetch the surgical tools by itself. However, the direction of the trocar may change because of the soft abdomen, and the lack of a fixed RCM makes it difficult for manipulators to reinsert a new surgical tool through the trocar. This study proposes a system prototype using a 7-DOF manipulator to automatically conduct the intraoperative tool-changing task. The newly designed surgical tool docking station facilitates surgical tool coupling/decoupling by rotating the manipulator’s end effector once. The proposed trocar recognition method with position error compensation is reliable for aligning a new surgical tool to the trocar port, even when the direction of the trocar is changed. The experimental results confirms that the manipulator can accomplish an intraoperative tool changing task without additional assistance or correction from the human.
Collapse
|
5
|
Li W, Yin Ng W, Zhang X, Huang Y, Li Y, Song C, Chiu PWY, Li Z. A Kinematic Modeling and Control Scheme for Different Robotic Endoscopes: A Rudimentary Research Prototype. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3186758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- Weibing Li
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
| | - Wing Yin Ng
- Department of Surgery and the Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Hong Kong, China
| | - Xue Zhang
- Department of Surgery and the Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Hong Kong, China
| | - Yisen Huang
- Department of Surgery and the Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Hong Kong, China
| | - Yehui Li
- Department of Surgery and the Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Hong Kong, China
| | - Chengzhi Song
- Shenzhen Cornerstone Technology Co., Ltd., Shenzhen, China
| | - Philip Wai-Yan Chiu
- Department of Surgery and the Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Hong Kong, China
| | - Zheng Li
- Department of Surgery, Chow Yuk Ho Technology Centre for Innovative Medicine, Li Ka Shing Institute of Health Science, and Multi-Scale Medical Robotics Centre, The Chinese University of Hong Kong, Hong Kong, China
| |
Collapse
|
6
|
A Reliable Algorithm for Obtaining All-Inclusive Inverse Kinematics’ Solutions and Redundancy Resolution of Continuum Robots. ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING 2022. [DOI: 10.1007/s13369-022-07065-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
7
|
Yang S, Wang Y, Zhao H, Cheng H, Ding H. Autonomous Laparoscope Control for Minimally Invasive Surgery With Intuition and RCM Constraints. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3186507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Affiliation(s)
- Sihang Yang
- State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China
| | - Yiwei Wang
- State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China
| | - Huan Zhao
- State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China
| | - Haoyuan Cheng
- State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China
| | - Han Ding
- State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China
| |
Collapse
|
8
|
Nazari AA, Zareinia K, Janabi-Sharifi F. Visual servoing of continuum robots: Methods, challenges, and prospects. Int J Med Robot 2022; 18:e2384. [PMID: 35199451 DOI: 10.1002/rcs.2384] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Revised: 02/16/2022] [Accepted: 02/19/2022] [Indexed: 01/05/2023]
Abstract
BACKGROUND Recent advancements in continuum robotics have accentuated developing efficient and stable controllers to handle shape deformation and compliance. The control of continuum robots (CRs) using physical sensors attached to the robot, particularly in confined spaces, is difficult due to their limited accuracy in three-dimensional deflections and challenging localisation. Therefore, using non-contact imaging sensors finds noticeable importance, particularly in medical scenarios. Accordingly, given the need for direct control of the robot tip and notable uncertainties in the kinematics and dynamics of CRs, many papers have focussed on the visual servoing (VS) of CRs in recent years. METHODS The significance of this research towards safe human-robot interaction has fuelled our survey on the previous methods, current challenges, and future opportunities. RESULTS Beginning with actuation modalities and modelling approaches, the paper investigates VS methods in medical and non-medical scenarios. CONCLUSIONS Finally, challenges and prospects of VS for CRs are discussed, followed by concluding remarks.
Collapse
Affiliation(s)
- Ali A Nazari
- Department of Mechanical and Industrial Engineering, Ryerson University, Toronto, Ontario, Canada
| | - Kourosh Zareinia
- Department of Mechanical and Industrial Engineering, Ryerson University, Toronto, Ontario, Canada
| | - Farrokh Janabi-Sharifi
- Department of Mechanical and Industrial Engineering, Ryerson University, Toronto, Ontario, Canada
| |
Collapse
|
9
|
Coupling Effect Suppressed Compact Surgical Robot with 7-Axis Multi-Joint Using Wire-Driven Method. MATHEMATICS 2022. [DOI: 10.3390/math10101698] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Currently, the most prevalent surgical treatment method is laparoscopic surgery. Robotic surgery has many advantages over laparoscopic surgery. Therefore, robotic surgery technology is currently constantly evolving. The advantages of robotic surgery are that it can minimize incision, bleeding, and sequelae. Other advantages of robotic surgery are that it can reduce hospitalization, recovery period, and side effects. The appeal of robotic surgery is that it requires fewer surgical personnel compared to laparoscopic surgery. This paper proposes an ultra-compact 7-axis vertical multi-joint robot that employs the wire-driven method for minimally invasive surgery. The proposed robot analyzes the degree of freedom and motion coupling for control. The robot joint is composed of a total of seven joints, and among them, the 7-axis joint operates the forceps. At this time, the forceps joint (#7 axis) can only operate open and close functions, while the link is bent and rotatable, regardless of position change. This phenomenon can be analyzed by Forward Kinematics. Also, when the DOF rotates, the passing wires become twisted, and the wire is generated through length change and coupling phenomenon. The maximum rotation angle of DOF is 90° and the rotating passing wire is wound by the rotation of the wire pulley. If the DOF is rotated to the full range of 120°, the second DOF will be rotated to 90°, and at this time, the coupling phenomenon caused by the first DOF rotation can be eliminated. The length change and the robot joint angle change related to the motor drive, based on the surgical robot control using the wire-driven method, are correlated, and the values for the position and direction of the end effector of the robot can be obtained through a forward kinematic analysis. The coupling problem occurring in the wire connecting the robot driving part can be solved through a kinematic analysis. Therefore, it was possible to test the position of the slave robot and the performance of the surgical forceps movement using the master system.
Collapse
|
10
|
Zuo S, Chen T, Chen X, Chen B. A Wearable Hands-Free Human-Robot Interface for Robotized Flexible Endoscope. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3149303] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
11
|
Gao C, Phalen H, Sefati S, Ma J, Taylor R, Unberath M, Armand M. Fluoroscopic Navigation for a Surgical Robotic System Including a Continuum Manipulator. IEEE Trans Biomed Eng 2022; 69:453-464. [PMID: 34270412 PMCID: PMC8817231 DOI: 10.1109/tbme.2021.3097631] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
We present an image-based navigation solution for a surgical robotic system with a Continuum Manipulator (CM). Our navigation system uses only fluoroscopic images from a mobile C-arm to estimate the CM shape and pose with respect to the bone anatomy. The CM pose and shape estimation is achieved using image intensity-based 2D/3D registration. A learning-based framework is used to automatically detect the CM in X-ray images, identifying landmark features that are used to initialize and regularize image registration. We also propose a modified hand-eye calibration method that numerically optimizes the hand-eye matrix during image registration. The proposed navigation system for CM positioning was tested in simulation and cadaveric studies. In simulation, the proposed registration achieved a mean error of 1.10±0.72 mm between the CM tip and a target entry point on the femur. In cadaveric experiments, the mean CM tip position error was 2.86±0.80 mm after registration and repositioning of the CM. The results suggest that our proposed fluoroscopic navigation is feasible to guide the CM in orthopedic applications.
Collapse
|
12
|
Huber M, Mitchell JB, Henry R, Ourselin S, Vercauteren T, Bergeles C. Homography-based Visual Servoing with Remote Center of Motion for Semi-autonomous Robotic Endoscope Manipulation. ... INTERNATIONAL SYMPOSIUM ON MEDICAL ROBOTICS. INTERNATIONAL SYMPOSIUM ON MEDICAL ROBOTICS 2021; 220:1-7. [PMID: 39351396 PMCID: PMC7616652 DOI: 10.1109/ismr48346.2021.9661563] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 10/04/2024]
Abstract
The dominant visual servoing approaches in Minimally Invasive Surgery (MIS) follow single points or adapt the endoscope's field of view based on the surgical tools' distance. These methods rely on point positions with respect to the camera frame to infer a control policy. Deviating from the dominant methods, we formulate a robotic controller that allows for image-based visual servoing that requires neither explicit tool and camera positions nor any explicit image depth information. The proposed method relies on homography-based image registration, which changes the automation paradigm from point-centric towards surgical-scene-centric approach. It simultaneously respects a programmable Remote Center of Motion (RCM). Our approach allows a surgeon to build a graph of desired views, from which, once built, views can be manually selected and automatically servoed to irrespective of robot-patient frame transformation changes. We evaluate our method on an abdominal phantom and provide an open source ROS Moveit integration for use with any serial manipulator. A video is provided.
Collapse
Affiliation(s)
- Martin Huber
- School of Biomedical Engineering & Image Sciences, Faculty of Life Sciences & Medicine, King's College London, London, United Kingdom
| | - John Bason Mitchell
- School of Biomedical Engineering & Image Sciences, Faculty of Life Sciences & Medicine, King's College London, London, United Kingdom
- Department of Medical Physics and Biomedical Engineering, Faculty of Engineering Sciences, University College London, London, United Kingdom
| | - Ross Henry
- School of Biomedical Engineering & Image Sciences, Faculty of Life Sciences & Medicine, King's College London, London, United Kingdom
| | - Sébastien Ourselin
- School of Biomedical Engineering & Image Sciences, Faculty of Life Sciences & Medicine, King's College London, London, United Kingdom
| | - Tom Vercauteren
- School of Biomedical Engineering & Image Sciences, Faculty of Life Sciences & Medicine, King's College London, London, United Kingdom
| | - Christos Bergeles
- School of Biomedical Engineering & Image Sciences, Faculty of Life Sciences & Medicine, King's College London, London, United Kingdom
| |
Collapse
|
13
|
Abstract
Compliant continuum robots (CCRs) have slender and elastic bodies. Compared with a traditional serial robot, they have more degrees of freedom and can deform their flexible bodies to go through a constrained environment. In this paper, we classify CCRs according to basic transmission units. The merits, materials and potential drawbacks of each type of CCR are described. Drive systems depend on the basic transmission units significantly, and their advantages and disadvantages are reviewed and summarized. Variable stiffness and intrinsic sensing are desired characteristics of CCRs, and the methods of obtaining the two characteristics are discussed. Finally, we discuss the friction, buckling, singularity and twisting problems of CCRs, and emphasise the ways to reduce their effects, followed by several proposing perspectives, such as the collaborative CCRs.
Collapse
|
14
|
Development of a mechanical decoupling surgical scissors for robot-assisted minimally invasive surgery. ROBOTICA 2021. [DOI: 10.1017/s0263574721000564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
AbstractIn minimally invasive surgery, surgical instruments with a wrist joint have better flexibility. However, the bending motion of the wrist joint causes a coupling motion between the end-effector and wrist joint, affecting the accuracy of the movement of the surgical instrument. Aiming at this problem, a new gear train decoupling method is proposed in the paper, which can automatically compensate for the coupled motion in real-time. Based on the performance tests of the instrument prototype, a series of decoupling effects tests are carried out. The test results show that the surgical instrument has excellent decoupling ability and stable performance.
Collapse
|
15
|
Design and Evaluation of a Foot-Controlled Robotic System for Endoscopic Surgery. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3062009] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
16
|
Sefati S, Hegeman R, Alambeigi F, Iordachita I, Kazanzides P, Khanuja H, Taylor RH, Armand M. A Surgical Robotic System for Treatment of Pelvic Osteolysis Using an FBG-Equipped Continuum Manipulator and Flexible Instruments. IEEE/ASME TRANSACTIONS ON MECHATRONICS : A JOINT PUBLICATION OF THE IEEE INDUSTRIAL ELECTRONICS SOCIETY AND THE ASME DYNAMIC SYSTEMS AND CONTROL DIVISION 2021; 26:369-380. [PMID: 34025108 PMCID: PMC8132934 DOI: 10.1109/tmech.2020.3020504] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
This paper presents the development and experimental evaluation of a redundant robotic system for the less-invasive treatment of osteolysis (bone degradation) behind the acetabular implant during total hip replacement revision surgery. The system comprises a rigid-link positioning robot and a Continuum Dexterous Manipulator (CDM) equipped with highly flexible debriding tools and a Fiber Bragg Grating (FBG)-based sensor. The robot and the continuum manipulator are controlled concurrently via an optimization-based framework using the Tip Position Estimation (TPE) from the FBG sensor as feedback. Performance of the system is evaluated on a setup that consists of an acetabular cup and saw-bone phantom simulating the bone behind the cup. Experiments consist of performing the surgical procedure on the simulated phantom setup. CDM TPE using FBGs, target location placement, cutting performance, and the concurrent control algorithm capability in achieving the desired tasks are evaluated. Mean and standard deviation of the CDM TPE from the FBG sensor and the robotic system are 0.50 mm, and 0.18 mm, respectively. Using the developed surgical system, accurate positioning and successful cutting of desired straight-line and curvilinear paths on saw-bone phantoms behind the cup with different densities are demonstrated. Compared to the conventional rigid tools, the workspace reach behind the acetabular cup is 2.47 times greater when using the developed robotic system.
Collapse
Affiliation(s)
- Shahriar Sefati
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA, 21218
| | - Rachel Hegeman
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA, 21218; Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA
| | - Farshid Alambeigi
- Johns Hopkins University during the completion of this work and he is currently with the Department of Mechanical Engineering, University of Texas at Austin, Austin, TX, USA, 78712
| | - Iulian Iordachita
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA, 21218
| | - Peter Kazanzides
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA, 21218
| | - Harpal Khanuja
- Department of Orthopedic Surgery, The Johns Hopkins Medical School, Baltimore, MD, USA, 21205
| | - Russell H Taylor
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA, 21218
| | - Mehran Armand
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA, 21218; Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA; Department of Orthopedic Surgery, The Johns Hopkins Medical School, Baltimore, MD, USA, 21205
| |
Collapse
|
17
|
Li W, Chiu PWY, Li Z. An Accelerated Finite-Time Convergent Neural Network for Visual Servoing of a Flexible Surgical Endoscope With Physical and RCM Constraints. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:5272-5284. [PMID: 32011270 DOI: 10.1109/tnnls.2020.2965553] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
This article designs and analyzes a recurrent neural network (RNN) for the visual servoing of a flexible surgical endoscope. The flexible surgical endoscope is based on a commercially available UR5 robot with a flexible endoscope attached as an end-effector. Most of the existing visual servo control frameworks of the robotic endoscopes or robot arms have not considered either the physical limits of the robot or the remote center of motion (RCM) constraints (i.e., the fulcrum effect). To tackle this issue, this article first conducts the kinematic modeling of the flexible robotic endoscope to achieve automation by visual servo control. The kinematic modeling results in a quadratic programming (QP) framework with physical limits and RCM constraints involved, making the UR5 robot applicable to surgical field. To solve the QP problem and accomplish the visual task, an RNN activated by a sign-bi-power activation function (AF) is proposed. The motivation of using the sign-bi-power AF is to enable the RNN to exhibit an accelerated finite-time convergence, which is more preferred in time-critical applications. Theoretically, the finite-time convergence of the RNN is rigorously proved using the Lyapunov theory. Compared with the previous AFs applied to the RNN, theoretical analysis shows that the RNN activated by the sign-bi-power AF delivers an accelerated convergence speed. Comparative validations are performed, showing that the proposed finite-time convergent neural network is effective to achieve visual servoing of the flexible endoscope with physical limits and RCM constraints handled simultaneously.
Collapse
|
18
|
Ma X, Song C, Chiu PW, Li Z. Visual Servo of a 6-DOF Robotic Stereo Flexible Endoscope Based on da Vinci Research Kit (dVRK) System. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2965863] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
19
|
Zhang X, Li W, Chiu PWY, Li Z. A Novel Flexible Robotic Endoscope With Constrained Tendon-Driven Continuum Mechanism. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2967737] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
20
|
A Clamping Force Estimation Method Based on a Joint Torque Disturbance Observer Using PSO-BPNN for Cable-Driven Surgical Robot End-Effectors. SENSORS 2019; 19:s19235291. [PMID: 31805636 PMCID: PMC6929025 DOI: 10.3390/s19235291] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/28/2019] [Revised: 11/29/2019] [Accepted: 11/29/2019] [Indexed: 01/17/2023]
Abstract
The ability to sense external force is an important technique for force feedback, haptics and safe interaction control in minimally-invasive surgical robots (MISRs). Moreover, this ability plays a significant role in the restricting refined surgical operations. The wrist joints of surgical robot end-effectors are usually actuated by several long-distance wire cables. Its two forceps are each actuated by two cables. The scope of force sensing includes multidimensional external force and one-dimensional clamping force. This paper focuses on one-dimensional clamping force sensing method that do not require any internal force sensor integrated in the end-effector's forceps. A new clamping force estimation method is proposed based on a joint torque disturbance observer (JTDO) for a cable-driven surgical robot end-effector. The JTDO essentially considers the variations in cable tension between the actual cable tension and the estimated cable tension using a Particle Swarm Optimization Back Propagation Neural Network (PSO-BPNN) under free motion. Furthermore, a clamping force estimator is proposed based on the forceps' JTDO and their mechanical relations. According to comparative analyses in experimental studies, the detection resolutions of collision force and clamping force were 0.11 N. The experimental results verify the feasibility and effectiveness of the proposed clamping force sensing method.
Collapse
|
21
|
Song C, Ma X, Xia X, Chiu PWY, Chong CCN, Li Z. A robotic flexible endoscope with shared autonomy: a study of mockup cholecystectomy. Surg Endosc 2019; 34:2730-2741. [PMID: 31722046 DOI: 10.1007/s00464-019-07241-8] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2019] [Accepted: 10/28/2019] [Indexed: 11/25/2022]
Abstract
BACKGROUND Endoscope is the eye of surgeon in minimally invasive surgery (MIS). Prevailing handheld endoscopes are manually steered, which can cause endoscope-instrument fencing. Robotic endoscopes can reduce the fatigue but could not reduce collisions. Handheld endoscopes with a flexible bending tip can reduce the shaft pivoting and collisions. However, its steering is challenging. In this paper, we present a robotic flexible endoscope with auto-tracking function and compare it with the conventional rigid endoscopes. METHODS A robotic flexible endoscope (RFE) with shared autonomy is developed. The RFE could either track the instruments automatically or be controlled by a foot pedal. A mockup cholecystectomy was designed to evaluate the performance. Five surgeons were invited to perform the mockup cholecystectomy in an abdominal cavity phantom with a manual rigid endoscope (MRE), a robotic rigid endoscope (RRE), and the RFE. Space occupation, time consumption, and questionnaires based on the NASA task load index were adopted to evaluate the performances and compare the three endoscope systems. An ex vivo experiment was conducted to demonstrate the feasibility of using the RFE in a biological tissue environment. RESULTS All surgeons completed the mockup cholecystectomy with the RFE independently. Failure occurred in the cases involving the RRE and the MRE. Inside the body cavity, the space occupied when using the RFE is 17.28% and 23.95% (p < 0.05) of that when using the MRE and the RRE, respectively. Outside the body cavity, the space occupied when using the RFE is 14.60% and 15.53% (p < 0.05) of that by using MRE and RRE. Time consumed in the operations with MRE, RRE, and RFE are 28.3 s, 93.2 s and 34.8 s, respectively. Questionnaires reveal that the performance of the RFE is the best among the three endoscope systems. CONCLUSIONS The RFE provides a wider field of view (FOV) and occupies less space than rigid endoscopes.
Collapse
Affiliation(s)
- Chengzhi Song
- Department of Surgery, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR
| | - Xin Ma
- Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR
| | - Xianfeng Xia
- Department of Surgery, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR
| | - Philip Wai Yan Chiu
- Department of Surgery, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR
- Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR
| | | | - Zheng Li
- Department of Surgery, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR.
- Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR.
| |
Collapse
|
22
|
Cheng T, Li W, Ng CSH, Chiu PWY, Li Z. Visual Servo Control of a Novel Magnetic Actuated Endoscope for Uniportal Video-Assisted Thoracic Surgery. IEEE Robot Autom Lett 2019. [DOI: 10.1109/lra.2019.2924838] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|