1
|
Karstensen L, Ritter J, Hatzl J, Ernst F, Langejürgen J, Uhl C, Mathis-Ullrich F. Recurrent neural networks for generalization towards the vessel geometry in autonomous endovascular guidewire navigation in the aortic arch. Int J Comput Assist Radiol Surg 2023; 18:1735-1744. [PMID: 37245181 PMCID: PMC10491528 DOI: 10.1007/s11548-023-02938-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 04/24/2023] [Indexed: 05/29/2023]
Abstract
PURPOSE Endovascular intervention is the state-of-the-art treatment for common cardiovascular diseases, such as heart attack and stroke. Automation of the procedure may improve the working conditions of physicians and provide high-quality care to patients in remote areas, posing a major impact on overall treatment quality. However, this requires the adaption to individual patient anatomies, which currently poses an unsolved challenge. METHODS This work investigates an endovascular guidewire controller architecture based on recurrent neural networks. The controller is evaluated in-silico on its ability to adapt to new vessel geometries when navigating through the aortic arch. The controller's generalization capabilities are examined by reducing the number of variations seen during training. For this purpose, an endovascular simulation environment is introduced, which allows guidewire navigation in a parametrizable aortic arch. RESULTS The recurrent controller achieves a higher navigation success rate of 75.0% after 29,200 interventions compared to 71.6% after 156,800 interventions for a feedforward controller. Furthermore, the recurrent controller generalizes to previously unseen aortic arches and is robust towards size changes of the aortic arch. Being trained on 2048 aortic arch geometries gives the same results as being trained with full variation when evaluated on 1000 different geometries. For interpolation a gap of 30% of the scaling range and for extrapolation additional 10% of the scaling range can be navigated successfully. CONCLUSION Adaption to new vessel geometries is essential in the navigation of endovascular instruments. Therefore, the intrinsic generalization to new vessel geometries poses an essential step towards autonomous endovascular robotics.
Collapse
Affiliation(s)
- Lennart Karstensen
- Fraunhofer IPA, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany
- Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander University Erlangen-Nürnberg, Werner-von-Siemens-Straße 61, 91052 Erlangen, Germany
| | | | - Johannes Hatzl
- Department of Vascular and Endovascular Surgery, University Hospital Heidelberg, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany
| | - Floris Ernst
- Institute for Robotics and Cognitive Systems, University of Lübeck, 23562 Lübeck, Germany
| | - Jens Langejürgen
- Fraunhofer IPA, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany
| | - Christian Uhl
- Department of Vascular and Endovascular Surgery, University Hospital Heidelberg, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany
| | - Franziska Mathis-Ullrich
- Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander University Erlangen-Nürnberg, Werner-von-Siemens-Straße 61, 91052 Erlangen, Germany
| |
Collapse
|
2
|
Robertshaw H, Karstensen L, Jackson B, Sadati H, Rhode K, Ourselin S, Granados A, Booth TC. Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review. Front Hum Neurosci 2023; 17:1239374. [PMID: 37600553 PMCID: PMC10438983 DOI: 10.3389/fnhum.2023.1239374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 07/20/2023] [Indexed: 08/22/2023] Open
Abstract
Background Autonomous navigation of catheters and guidewires in endovascular interventional surgery can decrease operation times, improve decision-making during surgery, and reduce operator radiation exposure while increasing access to treatment. Objective To determine from recent literature, through a systematic review, the impact, challenges, and opportunities artificial intelligence (AI) has for the autonomous navigation of catheters and guidewires for endovascular interventions. Methods PubMed and IEEEXplore databases were searched to identify reports of AI applied to autonomous navigation methods in endovascular interventional surgery. Eligibility criteria included studies investigating the use of AI in enabling the autonomous navigation of catheters/guidewires in endovascular interventions. Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), articles were assessed using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). PROSPERO: CRD42023392259. Results Four hundred and sixty-two studies fulfilled the search criteria, of which 14 studies were included for analysis. Reinforcement learning (RL) (9/14, 64%) and learning from expert demonstration (7/14, 50%) were used as data-driven models for autonomous navigation. These studies evaluated models on physical phantoms (10/14, 71%) and in-silico (4/14, 29%) models. Experiments within or around the blood vessels of the heart were reported by the majority of studies (10/14, 71%), while non-anatomical vessel platforms "idealized" for simple navigation were used in three studies (3/14, 21%), and the porcine liver venous system in one study. We observed that risk of bias and poor generalizability were present across studies. No procedures were performed on patients in any of the studies reviewed. Moreover, all studies were limited due to the lack of patient selection criteria, reference standards, and reproducibility, which resulted in a low level of evidence for clinical translation. Conclusion Despite the potential benefits of AI applied to autonomous navigation of endovascular interventions, the field is in an experimental proof-of-concept stage, with a technology readiness level of 3. We highlight that reference standards with well-identified performance metrics are crucial to allow for comparisons of data-driven algorithms proposed in the years to come. Systematic review registration identifier: CRD42023392259.
Collapse
Affiliation(s)
- Harry Robertshaw
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | - Lennart Karstensen
- Fraunhofer IPA, Mannheim, Germany
- AIBE, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
| | - Benjamin Jackson
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | - Hadi Sadati
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | - Kawal Rhode
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | - Sebastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | - Alejandro Granados
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | - Thomas C. Booth
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
- Department of Neuroradiology, Kings College Hospital, London, United Kingdom
| |
Collapse
|
3
|
Shi P, Guo S, Jin X, Hirata H, Tamiya T, Kawanishi M. A novel catheter interaction simulating method for virtual reality interventional training systems. Med Biol Eng Comput 2023; 61:685-697. [PMID: 36585560 DOI: 10.1007/s11517-022-02730-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 12/09/2022] [Indexed: 12/31/2022]
Abstract
Endovascular robotic systems have been applied in robot-assisted interventional surgery to improve surgical safety and reduce radiation to surgeons. However, this surgery requires surgeons to be highly skilled at operating vascular interventional surgical robot. Virtual reality (VR) interventional training systems for robot-assisted interventional surgical training have many advantages over traditional training methods. For virtual interventional radiology, simulation of the behaviors of surgical tools (here mainly refers to catheter and guidewire) is a challenging work. In this paper, we developed a novel virtual reality interventional training system. This system is an extension of the endovascular robotic system. Because the master side of this system can be used for both the endovascular robotic system and the VR interventional training system, the proposed system improves training and reduces the cost of education. Moreover, we proposed a novel method to solve catheterization modeling during the interventional simulation. Our method discretizes the catheter by the collision points. The catheter between two adjacent collision points is treated as thin torsion-free elastic rods. The deformation of the rod is mainly affected by the force applied at the collision points. Meanwhile, the virtual contact force is determined by the collision points. This simplification makes the model more stable and reduces the computational complexity, and the behavior of the surgical tools can be approximated. Therefore, we realized the catheter interaction simulation and virtual force feedback for the proposed VR interventional training system. The performance of our method is experimentally validated.
Collapse
Affiliation(s)
- Peng Shi
- School of Medical Technology and Engineering, Henan University of Science and Technology, Luoyang, 471023, China.,Faculty of Engineering and Design, Kagawa University, 2217-20 Hayashi-Cho, Takamatsu, 760-8521, Japan
| | - Shuxiang Guo
- Key Laboratory of Convergence Medical Engineering System and Healthcare Technology, the Ministry of Industry and Information Technology, Beijing Institute of Technology, No. 5, Zhongguancun South Street, Haidian District, Beijing, 100081, China. .,Faculty of Engineering and Design, Kagawa University, 2217-20 Hayashi-Cho, Takamatsu, 760-8521, Japan.
| | - Xiaoliang Jin
- Faculty of Engineering and Design, Kagawa University, 2217-20 Hayashi-Cho, Takamatsu, 760-8521, Japan.,State Key Laboratory of Bioelectronics and the Jiangsu Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China
| | - Hideyuki Hirata
- Faculty of Engineering and Design, Kagawa University, 2217-20 Hayashi-Cho, Takamatsu, 760-8521, Japan
| | - Takashi Tamiya
- Department of Neurological Surgery, Faculty of Medicine, Kagawa University, Takamatsu, 761-0793, Japan
| | - Masahiko Kawanishi
- Department of Neurological Surgery, Faculty of Medicine, Kagawa University, Takamatsu, 761-0793, Japan
| |
Collapse
|
4
|
Design and evaluation of vascular interventional robot system for complex coronary artery lesions. Med Biol Eng Comput 2023; 61:1365-1380. [PMID: 36705768 DOI: 10.1007/s11517-023-02775-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 01/05/2023] [Indexed: 01/28/2023]
Abstract
At present, most vascular intervention robots cannot cope with the more common coronary complex lesions in the clinic. Moreover, the lack of effective force feedback increases the risk of surgery. In this paper, a vascular interventional robot that can collaboratively deliver multiple interventional instruments has been developed to assist doctors in the operation of complex lesions. Based on the doctor's skills and the delivery principle of interventional instruments, the main and slave manipulators of the robot system are designed. Haptic force feedback is generated through resistance measuring mechanism and active drag system. In addition, a force feedback control strategy based on force-velocity mapping is proposed to realize the continuous change of force and avoid vibration. The proposed robot system was evaluated through a series of experiments. The experimental results show that the system can accurately measure the delivery resistance of interventional instruments, and provide haptic force feedback to doctors. The capability of the system to collaboratively deliver multiple interventional instruments is effective. Therefore, it can be considered that the robot system is feasible and effective.
Collapse
|
5
|
Discrete soft actor-critic with auto-encoder on vascular robotic system. ROBOTICA 2022. [DOI: 10.1017/s0263574722001527] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Abstract
Instrument delivery is critical part in vascular intervention surgery. Due to the soft-body structure of instruments, the relationship between manipulation commands and instrument motion is non-linear, making instrument delivery challenging and time-consuming. Reinforcement learning has the potential to learn manipulation skills and automate instrument delivery with enhanced success rates and reduced workload of physicians. However, due to the sample inefficiency when using high-dimensional images, existing reinforcement learning algorithms are limited on realistic vascular robotic systems. To alleviate this problem, this paper proposes discrete soft actor-critic with auto-encoder (DSAC-AE) that augments SAC-discrete with an auxiliary reconstruction task. The algorithm is applied with distributed sample collection and parameter update in a robot-assisted preclinical environment. Experimental results indicate that guidewire delivery can be automatically implemented after 50k sampling steps in less than 15 h, demonstrating the proposed algorithm has the great potential to learn manipulation skill for vascular robotic systems.
Collapse
|
6
|
Innovation, disruptive Technologien und Transformation in der Gefäßchirurgie. GEFÄSSCHIRURGIE 2022. [DOI: 10.1007/s00772-022-00943-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
7
|
Zhao Y, Wang Y, Zhang J, Liu X, Li Y, Guo S, Yang X, Hong S. Surgical GAN: Towards real-time path planning for passive flexible tools in endovascular surgeries. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.05.044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
8
|
Karstensen L, Ritter J, Hatzl J, Pätz T, Langejürgen J, Uhl C, Mathis-Ullrich F. Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver. Int J Comput Assist Radiol Surg 2022; 17:2033-2040. [PMID: 35604490 PMCID: PMC9515141 DOI: 10.1007/s11548-022-02646-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 04/15/2022] [Indexed: 11/22/2022]
Abstract
Purpose The navigation of endovascular guidewires is a dexterous task where physicians and patients can benefit from automation. Machine learning-based controllers are promising to help master this task. However, human-generated training data are scarce and resource-intensive to generate. We investigate if a neural network-based controller trained without human-generated data can learn human-like behaviors. Methods We trained and evaluated a neural network-based controller via deep reinforcement learning in a finite element simulation to navigate the venous system of a porcine liver without human-generated data. The behavior is compared to manual expert navigation, and real-world transferability is evaluated. Results The controller achieves a success rate of 100% in simulation. The controller applies a wiggling behavior, where the guidewire tip is continuously rotated alternately clockwise and counterclockwise like the human expert applies. In the ex vivo porcine liver, the success rate drops to 30%, because either the wrong branch is probed, or the guidewire becomes entangled. Conclusion In this work, we prove that a learning-based controller is capable of learning human-like guidewire navigation behavior without human-generated data, therefore, mitigating the requirement to produce resource-intensive human-generated training data. Limitations are the restriction to one vessel geometry, the neglected safeness of navigation, and the reduced transferability to the real world. Supplementary Information The online version contains supplementary material available at 10.1007/s11548-022-02646-8.
Collapse
Affiliation(s)
- Lennart Karstensen
- Fraunhofer IPA, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany. .,Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Engler-Bunte-Ring 8, 76131, Karlsruhe, Germany.
| | | | - Johannes Hatzl
- Department of Vascular and Endovascular Surgery, University Hospital Heidelberg, Im Neuenheimer Feld 420, 69120, Heidelberg, Germany
| | - Torben Pätz
- Fraunhofer MEVIS, Max-von-Laue-Str. 2, 28359, Bremen, Germany
| | - Jens Langejürgen
- Fraunhofer IPA, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Christian Uhl
- Department of Vascular and Endovascular Surgery, University Hospital Heidelberg, Im Neuenheimer Feld 420, 69120, Heidelberg, Germany
| | - Franziska Mathis-Ullrich
- Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Engler-Bunte-Ring 8, 76131, Karlsruhe, Germany
| |
Collapse
|
9
|
Zhao Y, Mei Z, Luo X, Mao J, Zhao Q, Liu G, Wu D. Remote vascular interventional surgery robotics: a literature review. Quant Imaging Med Surg 2022; 12:2552-2574. [PMID: 35371939 PMCID: PMC8923856 DOI: 10.21037/qims-21-792] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Accepted: 12/22/2021] [Indexed: 07/25/2023]
Abstract
Vascular interventional doctors are exposed to radiation hazards during surgery and endure high work intensity. Remote vascular interventional surgery robotics is a hot research field, in which researchers aim to not only protect the health of interventional doctors, but to also improve surgical accuracy and efficiency. However, the current vascular interventional robots have numerous shortcomings, such as poor haptic feedback, few compatible surgeries and instruments, and cumbersome maintenance and operational procedures. Nevertheless, vascular interventional surgery combined with robotics provides more cutting-edge directions, such as Internet remote surgery combined with 5G network technology and the application of artificial intelligence in surgical procedures. To summarize the developmental status and key technical points of intravascular interventional surgical robotics research, we performed a systematic literature search to retrieve original articles related to remote vascular interventional surgery robotics published up to December 2020. This review, which includes 113 articles published in English, introduces the mechanical and structural characteristics of various aspects of vascular interventional surgical robotics, discusses the current key features of vascular interventional surgical robotics in force sensing, haptic feedback, and control methods, and summarizes current frontiers in autonomous surgery, long-distance robotic telesurgery, and magnetic resonance imaging (MRI)-compatible structures. On the basis of summarizing the current research status of remote vascular interventional surgery robotics, we aim to propose a variety of prospects for future robotic systems.
Collapse
Affiliation(s)
- Yang Zhao
- Department of Mechanical & Electrical Engineering, Xiamen University, Xiamen, China
| | - Ziyang Mei
- Department of Mechanical & Electrical Engineering, Xiamen University, Xiamen, China
| | - Xiaoxiao Luo
- Department of Mechanical & Electrical Engineering, Xiamen University, Xiamen, China
| | - Jingsong Mao
- Department of Radiology, Xiang’an Hospital of Xiamen University, Xiamen, China
| | - Qingliang Zhao
- State Key Laboratory of Molecular Vaccinology and Molecular Diagnostics Center for Molecular Imaging and Translational Medicine, School of Public Health, Xiamen University, Xiamen, China
| | - Gang Liu
- State Key Laboratory of Molecular Vaccinology and Molecular Diagnostics Center for Molecular Imaging and Translational Medicine, School of Public Health, Xiamen University, Xiamen, China
| | - Dezhi Wu
- Department of Mechanical & Electrical Engineering, Xiamen University, Xiamen, China
| |
Collapse
|
10
|
Zhou W, Guo S, Guo J, Meng F, Chen Z. ADRC-Based Control Method for the Vascular Intervention Master-Slave Surgical Robotic System. MICROMACHINES 2021; 12:mi12121439. [PMID: 34945289 PMCID: PMC8707856 DOI: 10.3390/mi12121439] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Revised: 11/20/2021] [Accepted: 11/23/2021] [Indexed: 12/25/2022]
Abstract
In vascular interventional surgery, surgeons operate guidewires and catheters to diagnose and treat patients with the assistance of the digital subtraction angiography (DSA). Therefore, the surgeon will be exposed to X-rays for extended periods. To protect the surgeon, the development of a robot-assisted surgical system is of great significance. The displacement tracking accuracy is the most important issue to be considered in the development of the system. In this study, the active disturbance rejection control (ADRC) method is applied to guarantee displacement tracking accuracy. First, the core contents of the proportional–integral–derivative (PID) and ADRC methods are analyzed. Second, comparative evaluation experiments for incremental PID and ADRC methods are presented. The results show that the ADRC method has better performance of than that of the incremental PID method. Finally, the calibration experiments for the ADRC control method are implemented using the master–slave robotic system. These experiments demonstrate that the maximum tracking error is 0.87 mm using the ADRC method, effectively guaranteeing surgical safety.
Collapse
Affiliation(s)
- Wei Zhou
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China; (W.Z.); (F.M.); (Z.C.)
| | - Shuxiang Guo
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China; (W.Z.); (F.M.); (Z.C.)
- Key Laboratory of Convergence Medical Engineering System and Healthcare Technology, Ministry of Industry and Information Technology, School of Life Science, Beijing Institute of Technology, Beijing 100081, China
- Faculty of Engineering, Kagawa University, 2217-20 Hayashi-cho, Takamatsu 760-8521, Japan
- Correspondence: (S.G.); (J.G.); Tel.: +86-186-0020-0326 (S.G.)
| | - Jin Guo
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China; (W.Z.); (F.M.); (Z.C.)
- Correspondence: (S.G.); (J.G.); Tel.: +86-186-0020-0326 (S.G.)
| | - Fanxu Meng
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China; (W.Z.); (F.M.); (Z.C.)
| | - Zhengyang Chen
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China; (W.Z.); (F.M.); (Z.C.)
| |
Collapse
|
11
|
Tariciotti L, Palmisciano P, Giordano M, Remoli G, Lacorte E, Bertani G, Locatelli M, Dimeco F, Caccavella VM, Prada F. Artificial intelligence-enhanced intraoperative neurosurgical workflow: state of the art and future perspectives. J Neurosurg Sci 2021; 66:139-150. [PMID: 34545735 DOI: 10.23736/s0390-5616.21.05483-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
BACKGROUND Artificial Intelligence (AI) and Machine Learning (ML) augment decision-making processes and productivity by supporting surgeons over a range of clinical activities: from diagnosis and preoperative planning to intraoperative surgical assistance. We reviewed the literature to identify current AI platforms applied to neurosurgical perioperative and intraoperative settings and describe their role in multiple subspecialties. METHODS A systematic review of the literature was conducted following the PRISMA guidelines. PubMed, EMBASE, and Scopus databases were searched from inception to December 31, 2020. Original articles were included if they: presented AI platforms implemented in perioperative, intraoperative settings and reported ML models' performance metrics. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed appropriate. The risk of bias and applicability of predicted outcomes were assessed using the PROBAST tool. RESULTS 41 articles were included. All studies evaluated a supervised learning algorithm. A total of 10 ML models were described; the most frequent were neural networks (n = 15) and tree-based models (n = 13). Overall, the risk of bias was medium-high, but applicability was considered positive for all studies. Articles were grouped into 4 categories according to the subspecialty of interest: neuro-oncology, spine, functional and other. For each category, different prediction tasks were identified. CONCLUSIONS In this review, we summarize the state-of-art applications of AI for the intraoperative augmentation of neurosurgical workflows across multiple subspecialties. ML models may boost surgical team performances by reducing human errors and providing patient-tailored surgical plans, but further and higher-quality studies need to be conducted.
Collapse
Affiliation(s)
- Leonardo Tariciotti
- Unit of Neurosurgery, Fondazione IRCCS Cà Granda Ospedale Maggiore Policlinico, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,NEVRALIS, Milan, Italy
| | - Paolo Palmisciano
- NEVRALIS, Milan, Italy.,Department of Neurosurgery, Trauma, Gamma Knife Center Cannizzaro Hospital, Catania, Italy
| | - Martina Giordano
- NEVRALIS, Milan, Italy.,Department of Neurosurgery, Fondazione Policlinico Universitario A Gemelli IRCCS, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Giulia Remoli
- NEVRALIS, Milan, Italy.,National Center for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy
| | - Eleonora Lacorte
- National Center for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy
| | - Giulio Bertani
- Unit of Neurosurgery, Fondazione IRCCS Cà Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Marco Locatelli
- Unit of Neurosurgery, Fondazione IRCCS Cà Granda Ospedale Maggiore Policlinico, Milan, Italy.,Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy.,Aldo Ravelli Research Center for Neurotechnology and Experimental Brain Therapeutics, University of Milan, Milan, Italy
| | - Francesco Dimeco
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico C. Besta, Milan, Italy
| | - Valerio M Caccavella
- NEVRALIS, Milan, Italy - .,Department of Neurosurgery, Fondazione Policlinico Universitario A Gemelli IRCCS, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Francesco Prada
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico C. Besta, Milan, Italy.,Department of Neurological Surgery, University of Virginia Health Science Center, Charlottesville, VA, USA
| |
Collapse
|
12
|
Gumbs AA, Frigerio I, Spolverato G, Croner R, Illanes A, Chouillard E, Elyan E. Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery? SENSORS (BASEL, SWITZERLAND) 2021; 21:5526. [PMID: 34450976 PMCID: PMC8400539 DOI: 10.3390/s21165526] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 08/03/2021] [Accepted: 08/11/2021] [Indexed: 12/30/2022]
Abstract
Most surgeons are skeptical as to the feasibility of autonomous actions in surgery. Interestingly, many examples of autonomous actions already exist and have been around for years. Since the beginning of this millennium, the field of artificial intelligence (AI) has grown exponentially with the development of machine learning (ML), deep learning (DL), computer vision (CV) and natural language processing (NLP). All of these facets of AI will be fundamental to the development of more autonomous actions in surgery, unfortunately, only a limited number of surgeons have or seek expertise in this rapidly evolving field. As opposed to AI in medicine, AI surgery (AIS) involves autonomous movements. Fortuitously, as the field of robotics in surgery has improved, more surgeons are becoming interested in technology and the potential of autonomous actions in procedures such as interventional radiology, endoscopy and surgery. The lack of haptics, or the sensation of touch, has hindered the wider adoption of robotics by many surgeons; however, now that the true potential of robotics can be comprehended, the embracing of AI by the surgical community is more important than ever before. Although current complete surgical systems are mainly only examples of tele-manipulation, for surgeons to get to more autonomously functioning robots, haptics is perhaps not the most important aspect. If the goal is for robots to ultimately become more and more independent, perhaps research should not focus on the concept of haptics as it is perceived by humans, and the focus should be on haptics as it is perceived by robots/computers. This article will discuss aspects of ML, DL, CV and NLP as they pertain to the modern practice of surgery, with a focus on current AI issues and advances that will enable us to get to more autonomous actions in surgery. Ultimately, there may be a paradigm shift that needs to occur in the surgical community as more surgeons with expertise in AI may be needed to fully unlock the potential of AIS in a safe, efficacious and timely manner.
Collapse
Affiliation(s)
- Andrew A. Gumbs
- Centre Hospitalier Intercommunal de POISSY/SAINT-GERMAIN-EN-LAYE 10, Rue Champ de Gaillard, 78300 Poissy, France;
| | - Isabella Frigerio
- Department of Hepato-Pancreato-Biliary Surgery, Pederzoli Hospital, 37019 Peschiera del Garda, Italy;
| | - Gaya Spolverato
- Department of Surgical, Oncological and Gastroenterological Sciences, University of Padova, 35122 Padova, Italy;
| | - Roland Croner
- Department of General-, Visceral-, Vascular- and Transplantation Surgery, University of Magdeburg, Haus 60a, Leipziger Str. 44, 39120 Magdeburg, Germany;
| | - Alfredo Illanes
- INKA–Innovation Laboratory for Image Guided Therapy, Medical Faculty, Otto-von-Guericke University Magdeburg, 39120 Magdeburg, Germany;
| | - Elie Chouillard
- Centre Hospitalier Intercommunal de POISSY/SAINT-GERMAIN-EN-LAYE 10, Rue Champ de Gaillard, 78300 Poissy, France;
| | - Eyad Elyan
- School of Computing, Robert Gordon University, Aberdeen AB10 7JG, UK;
| |
Collapse
|
13
|
Kumar R, Khan FU, Sharma A, Aziz IB, Poddar NK. Recent Applications of Artificial Intelligence in detection of Gastrointestinal, Hepatic and Pancreatic Diseases. Curr Med Chem 2021; 29:66-85. [PMID: 33820515 DOI: 10.2174/0929867328666210405114938] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2020] [Revised: 02/25/2021] [Accepted: 03/06/2021] [Indexed: 11/22/2022]
Abstract
There is substantial progress in artificial intelligence (AI) algorithms and their medical sciences applications in the last two decades. AI-assisted programs have already been established for remotely health monitoring using sensors and smartphones. A variety of AI-based prediction models available for the gastrointestinal inflammatory, non-malignant diseases, and bowel bleeding using wireless capsule endoscopy, electronic medical records for hepatitis-associated fibrosis, pancreatic carcinoma using endoscopic ultrasounds. AI-based models may be of immense help for healthcare professionals in the identification, analysis, and decision support using endoscopic images to establish prognosis and risk assessment of patient's treatment using multiple factors. Although enough randomized clinical trials are warranted to establish the efficacy of AI-algorithms assisted and non-AI based treatments before approval of such techniques from medical regulatory authorities. In this article, available AI approaches and AI-based prediction models for detecting gastrointestinal, hepatic, and pancreatic diseases are reviewed. The limitation of AI techniques in such disease prognosis, risk assessment, and decision support are discussed.
Collapse
Affiliation(s)
- Rajnish Kumar
- Amity Institute of Biotechnology, Amity University Uttar Pradesh Lucknow Campus, Uttar Pradesh. India
| | - Farhat Ullah Khan
- Computer and Information Sciences Department, Universiti Teknologi Petronas, 32610, Seri Iskander, Perak. Malaysia
| | - Anju Sharma
- Department of Applied Science, Indian Institute of Information Technology, Allahabad, Uttar Pradesh. India
| | - Izzatdin Ba Aziz
- Computer and Information Sciences Department, Universiti Teknologi Petronas, 32610, Seri Iskander, Perak. Malaysia
| | | |
Collapse
|
14
|
Xu Z, Tan Y, Jiang Z, Huang S. WITHDRAWN: Detection of Atrial Fibrillation Patients and Analysis of Their Nerve and Infection Problems by CNN and Related Detection Images. Neurosci Lett 2020:135194. [PMID: 32599317 DOI: 10.1016/j.neulet.2020.135194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2020] [Revised: 06/14/2020] [Accepted: 06/19/2020] [Indexed: 11/28/2022]
Abstract
This article has been withdrawn at the request of the Editor-in-Chief. The Publisher apologizes for any inconvenience this may cause. The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/our-business/policies/article-withdrawal.
Collapse
Affiliation(s)
- Zhiwei Xu
- Department of Cardiac Surgery, Huai'an First People's Hospital, The AffiliatedHuaianNo. 1, People's Hospital of Nanjing Medical University, Huai'an, 223000, China.
| | - Yan Tan
- Department of intensive care unit, Shanghai Pudong Hospital, Fudan University Pudong Medical Center, Shanghai, China
| | - Zhaolei Jiang
- Department of Cardiothoracic Surgery, Xinhua Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, China
| | - Su Huang
- Department of Cardiac Surgery, Huai'an First People's Hospital, The AffiliatedHuaianNo. 1, People's Hospital of Nanjing Medical University, Huai'an, 223000, China
| |
Collapse
|
15
|
Hashemi S, Veisi H, Jafarzadehpur E, Rahmani R, Heshmati Z. Multi-view deep learning for rigid gas permeable lens base curve fitting based on Pentacam images. Med Biol Eng Comput 2020; 58:1467-1482. [PMID: 32363555 DOI: 10.1007/s11517-020-02154-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2019] [Accepted: 03/05/2020] [Indexed: 02/06/2023]
Abstract
Many studies in the rigid gas permeable (RGP) lens fitting field have focused on providing the best fit for patients with irregular astigmatism, a challenging issue. Despite the ease and accuracy of fitting in the current fitting methods, no studies have provided a high-pace solution with the final best fit to assist experts. This work presents a deep learning solution for identifying features in Pentacam four refractive maps and RGP base curve identification. An authentic dataset of 247 samples of Pentacam four refractive maps was gathered, providing a multi-view image of the corneal structure. Scratch-based convolutional neural network (CNN) architectures and well-known CNN architectures such as AlexNet, GoogLeNet, and ResNet have been used to extract features and transfer learning. Features are aggregated through a fusion technique. Based on a comparison of means square error (MSE) of normalized labels, the multi-view scratch-based CNN provided R-squared of 0.849, 0.846, 0.835, and 0.834 followed by GoogLeNet, comparable with current methods. Transfer learning outperforms various scratch-based CNN models, through which proper specifications some scratch-based models were able to increase coefficient of determinations. CNNs on multi-view Pentacam images have enabled fast detection of the RGP lens base curve, higher patient satisfaction, and reduced chair time. Graphical abstract The Pentacam four refractive maps is learned by the proposed scratch-based and transfer learning-based CNN methodology. The deep network-based solutions enable identification of rigid gas permeable lens for patients with irregular astigmatism.
Collapse
Affiliation(s)
- Sara Hashemi
- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran
| | - Hadi Veisi
- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran.
| | - Ebrahim Jafarzadehpur
- Department of Optometry, School of Rehabilitation Science, Iran University of Medical Sciences, Tehran, Iran
| | | | | |
Collapse
|
16
|
A novel noncontact detection method of surgeon's operation for a master-slave endovascular surgery robot. Med Biol Eng Comput 2020; 58:871-885. [PMID: 32077011 DOI: 10.1007/s11517-020-02143-7] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2019] [Accepted: 02/11/2020] [Indexed: 12/20/2022]
Abstract
Master-slave endovascular interventional surgery (EIS) robots have brought revolutionary advantages to traditional EIS, such as avoiding X-ray radiation to the surgeon and improving surgical precision and safety. However, the master controllers of most of the current EIS robots always lead to bad human-machine interaction, because of the difference in nature between the rigid operating handle and the flexible medical catheter used in EIS. In this paper, a noncontact detection method is proposed, and a novel master controller is developed to realize real-time detection of surgeon's operation without interference to the surgeon. A medical catheter is used as the operating handle. It is enabled by using FAST corner detection algorithm and optical flow algorithm to track the corner points of the continuous markers on a designed sensing pipe. A mathematical model is established to calculate the axial and rotational motion of the sensing pipe according to the moving distance of the corner points in image coordinates. A master-slave EIS robot system is constructed by integrating the proposed master controller and a developed slave robot. Surgical task performance evaluation in an endovascular evaluator (EVE) is conducted, and the results indicate that the proposed detection method breaks through the axial measuring range limitation of the previous marker-based detection method. In addition, the rotational detection error is reduced by 92.5% compared with the previous laser-based detection method. The results also demonstrate the capability and efficiency of the proposed master controller to control the slave robot for surgical task implementation. Graphical abstract A novel master controller is developed to realize real-time noncontact detection of surgeon's operation without interference to the surgeon. The master controller is used to remotely control the slave robot to implement certain surgical tasks.
Collapse
|
17
|
Artificial intelligence, machine learning, vascular surgery, automatic image processing. Implications for clinical practice. ANGIOLOGIA 2020. [DOI: 10.20960/angiologia.00177] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
18
|
Andras I, Mazzone E, van Leeuwen FWB, De Naeyer G, van Oosterom MN, Beato S, Buckle T, O'Sullivan S, van Leeuwen PJ, Beulens A, Crisan N, D'Hondt F, Schatteman P, van Der Poel H, Dell'Oglio P, Mottrie A. Artificial intelligence and robotics: a combination that is changing the operating room. World J Urol 2019; 38:2359-2366. [PMID: 31776737 DOI: 10.1007/s00345-019-03037-6] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Accepted: 11/21/2019] [Indexed: 12/12/2022] Open
Abstract
PURPOSE The aim of the current narrative review was to summarize the available evidence in the literature on artificial intelligence (AI) methods that have been applied during robotic surgery. METHODS A narrative review of the literature was performed on MEDLINE/Pubmed and Scopus database on the topics of artificial intelligence, autonomous surgery, machine learning, robotic surgery, and surgical navigation, focusing on articles published between January 2015 and June 2019. All available evidences were analyzed and summarized herein after an interactive peer-review process of the panel. LITERATURE REVIEW The preliminary results of the implementation of AI in clinical setting are encouraging. By providing a readout of the full telemetry and a sophisticated viewing console, robot-assisted surgery can be used to study and refine the application of AI in surgical practice. Machine learning approaches strengthen the feedback regarding surgical skills acquisition, efficiency of the surgical process, surgical guidance and prediction of postoperative outcomes. Tension-sensors on the robotic arms and the integration of augmented reality methods can help enhance the surgical experience and monitor organ movements. CONCLUSIONS The use of AI in robotic surgery is expected to have a significant impact on future surgical training as well as enhance the surgical experience during a procedure. Both aim to realize precision surgery and thus to increase the quality of the surgical care. Implementation of AI in master-slave robotic surgery may allow for the careful, step-by-step consideration of autonomous robotic surgery.
Collapse
Affiliation(s)
- Iulia Andras
- ORSI Academy, Melle, Belgium
- Department of Urology, Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca, Romania
| | - Elio Mazzone
- ORSI Academy, Melle, Belgium
- Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
- Department of Urology and Division of Experimental Oncology, URI, Urological Research Institute, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Fijs W B van Leeuwen
- ORSI Academy, Melle, Belgium
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Centre, Leiden, The Netherlands
- Department of Urology, Antoni Van Leeuwenhoek Hospital, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Geert De Naeyer
- ORSI Academy, Melle, Belgium
- Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
| | - Matthias N van Oosterom
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Centre, Leiden, The Netherlands
- Department of Urology, Antoni Van Leeuwenhoek Hospital, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Tessa Buckle
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Shane O'Sullivan
- Department of Pathology, Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brazil
| | - Pim J van Leeuwen
- Department of Urology, Antoni Van Leeuwenhoek Hospital, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Alexander Beulens
- Department of Urology, Catharina Hospital, Eindhoven, The Netherlands
- Netherlands Institute for Health Services (NIVEL), Utrecht, The Netherlands
| | - Nicolae Crisan
- Department of Urology, Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca, Romania
| | - Frederiek D'Hondt
- ORSI Academy, Melle, Belgium
- Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
| | - Peter Schatteman
- ORSI Academy, Melle, Belgium
- Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
| | - Henk van Der Poel
- Department of Urology, Antoni Van Leeuwenhoek Hospital, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Paolo Dell'Oglio
- ORSI Academy, Melle, Belgium.
- Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium.
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Centre, Leiden, The Netherlands.
- Department of Urology, Antoni Van Leeuwenhoek Hospital, The Netherlands Cancer Institute, Amsterdam, The Netherlands.
| | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium
- Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
| |
Collapse
|