1
|
Lin Y, Guo J, Yang X, Xu W, Li Z. Online advance respiration prediction model for percutaneous puncture robotics. Int J Comput Assist Radiol Surg 2024; 19:383-394. [PMID: 38070074 DOI: 10.1007/s11548-023-03041-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 11/20/2023] [Indexed: 02/22/2024]
Abstract
PURPOSE Surgical robots have significant research value and clinical significance in the field of percutaneous punctures. There have been numerous studies on ultrasound-guided percutaneous surgical robots; however, addressing the respiratory compensation problem of deep punctures remains a significant obstacle. Herein we propose a robotic system for percutaneous puncture with respiratory compensation. METHODS We proposed an online advance respiratory prediction model based on Bidirectional Gate Recurrent Unit (Bi-GRU) for the respiratory prediction requirements of surgical robot systems. By analyzing the main factors governing the accuracy of the respiratory motion prediction models, various parameters of the online advance prediction model were optimized. Subsequently, we integrated and developed ultrasound-guided percutaneous puncture robot software and a hardware platform to implement respiratory compensation, thus verifying the effectiveness and reliability of various key technologies in the system. RESULTS The proposed respiratory prediction model has a significantly reduced update time, with an average root mean square error (RMSE) of less than 0.4 mm. This represents a reduction of ~ 20% compared to the online training long short-term memory(LSTM). By conducting puncture experiments based on a respiratory phantom, the average puncture error was 2.71 ± 0.65 mm and the average single-round puncture time was 65.00 ± 6.67 s. CONCLUSION Herein we proposed and optimized an online training respiratory prediction network model based on Bi-GRU. The stability and reliability of this system are verified by conducting puncture experiments on a respiratory phantom.
Collapse
Affiliation(s)
- Yanping Lin
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China.
| | - Jin Guo
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xu Yang
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Wangjie Xu
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Zhaojun Li
- Department of Ultrasound, Shanghai General Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, People's Republic of China
- Department of Ultrasound, Shanghai General Hospital Jiading Branch, Shanghai Jiaotong University School of Medicine, Shanghai, People's Republic of China
| |
Collapse
|
2
|
Seitz PK, Karger CP, Bendl R, Schwahofer A. Strategy for automatic ultrasound (US) probe positioning in robot-assisted ultrasound guided radiation therapy. Phys Med Biol 2023; 68. [PMID: 36584398 DOI: 10.1088/1361-6560/acaf46] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Accepted: 12/30/2022] [Indexed: 12/31/2022]
Abstract
Objective. As part of image-guided radiotherapy, ultrasound-guided radiotherapy is currently already in use and under investigation for robot assisted systems Ipsen 2021. It promises a real-time tumor localization during irradiation (intrafractional) without extra dose. The ultrasound probe is held and guided by a robot. However, there is a lack of basic safety mechanisms and interaction strategies to enable a safe clinical procedure. In this study we investigate potential positioning strategies with safety mechanisms for a safe robot-human-interaction.Approach. A compact setup of ultrasound device, lightweight robot, tracking camera, force sensor and control computer were integrated in a software application to represent a potential USgRT setup. For the realization of a clinical procedure, positioning strategies for the ultrasound head with the help of the robot were developed, implemented, and tested. In addition, basic safety mechanisms for the robot have been implemented, using the integrated force sensor, and have been tested by intentional collisions.Main results. Various positioning methods from manual guidance to completely automated procedures were tested. Robot-guided methods achieved higher positioning accuracy and were faster in execution compared to conventional hand-guided methods. The developed safety mechanisms worked as intended and the detected collision force were below 20 N.Significance. The study demonstrates the feasibility of a new approach for safe robotic ultrasound imaging, with a focus on abdominal usage (liver, prostate, kidney). The safety measures applied here can be extended to other human-robot interactions and present the basic for further studies in medical applications.
Collapse
Affiliation(s)
- Peter Karl Seitz
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.,National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany.,University of Heidelberg, Faculty of Medicine Heidelberg, Heidelberg, Germany.,Medical Informatics, Heilbronn University, Heilbronn, Germany
| | - Christian P Karger
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.,National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany
| | - Rolf Bendl
- Medical Informatics, Heilbronn University, Heilbronn, Germany
| | - Andrea Schwahofer
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.,National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany.,Therapanacea, Paris, France
| |
Collapse
|
3
|
Wulff D, Hagenah J, Ernst F. Landmark tracking in 4D ultrasound using generalized representation learning. Int J Comput Assist Radiol Surg 2023; 18:493-500. [PMID: 36242701 PMCID: PMC9939499 DOI: 10.1007/s11548-022-02768-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Accepted: 09/26/2022] [Indexed: 11/25/2022]
Abstract
PURPOSE In this study, we present and validate a novel concept for target tracking in 4D ultrasound. The key idea is to replace image patch similarity metrics by distances in a latent representation. For this, 3D ultrasound patches are mapped into a representation space using sliced-Wasserstein autoencoders. METHODS A novel target tracking method for 4D ultrasound is presented that performs tracking in a representation space instead of in images space. Sliced-Wasserstein autoencoders are trained in an unsupervised manner which are used to map 3D ultrasound patches into a representation space. The tracking procedure is based on a greedy algorithm approach and measuring distances between representation vectors to relocate the target . The proposed algorithm is validated on an in vivo data set of liver images. Furthermore, three different concepts for training the autoencoder are presented to provide cross-patient generalizability, aiming at minimal training time on data of the individual patient. RESULTS Eight annotated 4D ultrasound sequences are used to test the tracking method. Tracking could be performed in all sequences using all autoencoder training approaches. A mean tracking error of 3.23 mm could be achieved using generalized fine-tuned autoencoders. It is shown that using generalized autoencoders and fine-tuning them achieves better tracking results than training subject individual autoencoders. CONCLUSION It could be shown that distances between encoded image patches in a representation space can serve as a meaningful measure of the image patch similarity, even under realistic deformations of the anatomical structure. Based on that, we could validate the proposed tracking algorithm in an in vivo setting. Furthermore, our results indicate that using generalized autoencoders, fine-tuning on only a small number of patches from the individual patient provides promising results.
Collapse
Affiliation(s)
- Daniel Wulff
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, Lübeck, 23562 Schleswig-Holstein Germany
| | - Jannis Hagenah
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, ParksRoad, Oxford, OX1 3PJ UK
| | - Floris Ernst
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, Lübeck, 23562 Schleswig-Holstein Germany
| |
Collapse
|
4
|
Sprenger J, Bengs M, Gerlach S, Neidhardt M, Schlaefer A. Systematic analysis of volumetric ultrasound parameters for markerless 4D motion tracking. Int J Comput Assist Radiol Surg 2022; 17:2131-2139. [PMID: 35597846 PMCID: PMC9515030 DOI: 10.1007/s11548-022-02665-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 04/27/2022] [Indexed: 11/05/2022]
Abstract
OBJECTIVES Motion compensation is an interesting approach to improve treatments of moving structures. For example, target motion can substantially affect dose delivery in radiation therapy, where methods to detect and mitigate the motion are widely used. Recent advances in fast, volumetric ultrasound have rekindled the interest in ultrasound for motion tracking. We present a setup to evaluate ultrasound based motion tracking and we study the effect of imaging rate and motion artifacts on its performance. METHODS We describe an experimental setup to acquire markerless 4D ultrasound data with precise ground truth from a robot and evaluate different real-world trajectories and system settings toward accurate motion estimation. We analyze motion artifacts in continuously acquired data by comparing to data recorded in a step-and-shoot fashion. Furthermore, we investigate the trade-off between the imaging frequency and resolution. RESULTS The mean tracking errors show that continuously acquired data leads to similar results as data acquired in a step-and-shoot fashion. We report mean tracking errors up to 2.01 mm and 1.36 mm on the continuous data for the lower and higher resolution, respectively, while step-and-shoot data leads to mean tracking errors of 2.52 mm and 0.98 mm. CONCLUSIONS We perform a quantitative analysis of different system settings for motion tracking with 4D ultrasound. We can show that precise tracking is feasible and additional motion in continuously acquired data does not impair the tracking. Moreover, the analysis of the frequency resolution trade-off shows that a high imaging resolution is beneficial in ultrasound tracking.
Collapse
Affiliation(s)
- Johanna Sprenger
- Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany.
| | - Marcel Bengs
- Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany
| | - Stefan Gerlach
- Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany
| | - Maximilian Neidhardt
- Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany
| | - Alexander Schlaefer
- Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany
| |
Collapse
|
5
|
Gherman B, Hajjar NA, Tucan P, Radu C, Vaida C, Mois E, Burz A, Pisla D. Risk Assessment-Oriented Design of a Needle Insertion Robotic System for Non-Resectable Liver Tumors. Healthcare (Basel) 2022; 10:healthcare10020389. [PMID: 35207006 PMCID: PMC8872014 DOI: 10.3390/healthcare10020389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Revised: 01/19/2022] [Accepted: 02/15/2022] [Indexed: 11/16/2022] Open
Abstract
Medical robotics is a highly challenging and rewarding field of research, especially in the development of minimally invasive solutions for the treatment of the worldwide leading cause of death, cancer. The aim of the paper is to provide a design methodology for the development of a safe and efficient medical robotic system for the minimally invasive, percutaneous, targeted treatment of hepatocellular carcinoma, which can be extended with minimal modification for other types of abdominal cancers. Using as input a set of general medical requirements to comply with currently applicable standards, and a set of identified hazards and failure modes, specific methods, such as the Analytical Hierarchy Prioritization, Risk Analysis and fuzzy logic Failure Modes and Effect Analysis have been used within a stepwise approach to help in the development of a medical device targeting the insertion of multiple needles in brachytherapy procedures. The developed medical device, which is visually guided using CT scanning, has been tested for validation in a medical environment using a human-size ballistic gel liver, with promising results. These prove that the robotic system can be used for the proposed medical task, while the modular approach increases the chances of acceptance.
Collapse
Affiliation(s)
- Bogdan Gherman
- CESTER—Research Center for Industrial Robots Simulation and Testing, Technical University of Cluj-Napoca, Bulevardul Muncii Street, No. 103-105, 400641 Cluj-Napoca, Romania; (B.G.); (P.T.); (C.V.); (A.B.)
| | - Nadim Al Hajjar
- “Prof. Dr. Octavian Fodor” Regional Institute of Gastroenterology and Hepatology Cluj-Napoca, Croitorilor Street, No. 19-21, 400162 Cluj-Napoca, Romania; (N.A.H.); (C.R.); (E.M.)
| | - Paul Tucan
- CESTER—Research Center for Industrial Robots Simulation and Testing, Technical University of Cluj-Napoca, Bulevardul Muncii Street, No. 103-105, 400641 Cluj-Napoca, Romania; (B.G.); (P.T.); (C.V.); (A.B.)
| | - Corina Radu
- “Prof. Dr. Octavian Fodor” Regional Institute of Gastroenterology and Hepatology Cluj-Napoca, Croitorilor Street, No. 19-21, 400162 Cluj-Napoca, Romania; (N.A.H.); (C.R.); (E.M.)
| | - Calin Vaida
- CESTER—Research Center for Industrial Robots Simulation and Testing, Technical University of Cluj-Napoca, Bulevardul Muncii Street, No. 103-105, 400641 Cluj-Napoca, Romania; (B.G.); (P.T.); (C.V.); (A.B.)
| | - Emil Mois
- “Prof. Dr. Octavian Fodor” Regional Institute of Gastroenterology and Hepatology Cluj-Napoca, Croitorilor Street, No. 19-21, 400162 Cluj-Napoca, Romania; (N.A.H.); (C.R.); (E.M.)
| | - Alin Burz
- CESTER—Research Center for Industrial Robots Simulation and Testing, Technical University of Cluj-Napoca, Bulevardul Muncii Street, No. 103-105, 400641 Cluj-Napoca, Romania; (B.G.); (P.T.); (C.V.); (A.B.)
| | - Doina Pisla
- CESTER—Research Center for Industrial Robots Simulation and Testing, Technical University of Cluj-Napoca, Bulevardul Muncii Street, No. 103-105, 400641 Cluj-Napoca, Romania; (B.G.); (P.T.); (C.V.); (A.B.)
- Correspondence:
| |
Collapse
|
6
|
Event-Triggered Adaptive Hybrid Position-Force Control for Robot-Assisted Ultrasonic Examination System. J INTELL ROBOT SYST 2021. [DOI: 10.1007/s10846-021-01428-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
7
|
Unger M, Berger J, Melzer A. Robot-Assisted Image-Guided Interventions. Front Robot AI 2021; 8:664622. [PMID: 34322519 PMCID: PMC8312560 DOI: 10.3389/frobt.2021.664622] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 07/01/2021] [Indexed: 12/23/2022] Open
Abstract
Image guidance is a common methodology of minimally invasive procedures. Depending on the type of intervention, various imaging modalities are available. Common imaging modalities are computed tomography, magnetic resonance tomography, and ultrasound. Robotic systems have been developed to enable and improve the procedures using these imaging techniques. Spatial and technological constraints limit the development of versatile robotic systems. This paper offers a brief overview of the developments of robotic systems for image-guided interventions since 2015 and includes samples of our current research in this field.
Collapse
Affiliation(s)
- Michael Unger
- Innovation Center Computer Assisted Surgery, Leipzig, Germany
| | - Johann Berger
- Innovation Center Computer Assisted Surgery, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery, Leipzig, Germany.,Institute for Medical Science and Technology, IMSaT, University Dundee, Dundee, United Kingdom
| |
Collapse
|
8
|
Chen S, Li Z, Lin Y, Wang F, Cao Q. Automatic ultrasound scanning robotic system with optical waveguide-based force measurement. Int J Comput Assist Radiol Surg 2021; 16:1015-1025. [PMID: 33939078 DOI: 10.1007/s11548-021-02385-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2020] [Accepted: 04/19/2021] [Indexed: 12/18/2022]
Abstract
PURPOSE The three-dimensional (3D) ultrasound (US) imaging realized by continuous scanning of a region is of great value for medical diagnosis and robot-assisted needle insertion. During scanning, the contact force and posture between the probe and skin of the patient are crucial factors that determine the quality of US imaging. We propose a robotic system for automatic scanning of curved surfaces with a stable contact force and vertical contact posture (the probe is parallel to the normal of the surface at the contact point). METHODS A 6-DOF robotic arm is used to hold and drive a two-dimensional (2D) US probe to complete automatic scanning. Further, a path-planning strategy is proposed to generate the scan path covering the selected area automatically. We also developed a novel force-measuring device based on optical waveguides to measure the distributed contact force and contact posture. Based on the measured force and posture, the robotic arm automatically adjusts the position and orientation of the probe and maintains a stable contact force and vertical contact posture at each scan point. RESULTS The novel force-measuring device is easy to fabricate, integrates with the probe and has the capacity of measuring the force distributed on the contact surface and estimating the contact posture. The experimental results of automatic scanning of a US phantom and parts of the human body demonstrate that the proposed system performs well in automatically scanning curved surfaces, maintaining a stable contact force and vertical contact posture and producing a good quality 3D US volume. CONCLUSION An automatic US scanning robotic system with an optical waveguide-based force-measuring device was developed and tested successfully. Experimental results demonstrated the feasibility of the proposed system to scan the human body.
Collapse
Affiliation(s)
- Shihang Chen
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Zhaojun Li
- Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Yanping Lin
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China. .,Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China.
| | - Fang Wang
- Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Qixin Cao
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China.,Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
9
|
Ipsen S, Wulff D, Kuhlemann I, Schweikard A, Ernst F. Towards automated ultrasound imaging-robotic image acquisition in liver and prostate for long-term motion monitoring. Phys Med Biol 2021; 66. [PMID: 33770768 DOI: 10.1088/1361-6560/abf277] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 03/26/2021] [Indexed: 11/12/2022]
Abstract
Real-time volumetric (4D) ultrasound has shown high potential for diagnostic and therapy guidance tasks. One of the main drawbacks of ultrasound imaging to date is the reliance on manual probe positioning and the resulting user dependence. Robotic assistance could help overcome this issue and facilitate the acquisition of long-term image data to observe dynamic processesin vivoover time. The aim of this study is to assess the feasibility of robotic probe manipulation and organ motion quantification during extended imaging sessions. The system consists of a collaborative robot and a 4D ultrasound system providing real-time data access. Five healthy volunteers received liver and prostate scans during free breathing over 30 min. Initial probe placement was performed with real-time remote control with a predefined contact force of 10 N. During scan acquisition, the probe position was continuously adjusted to the body surface motion using impedance control. Ultrasound volumes, the pose of the end-effector and the estimated contact forces were recorded. For motion analysis, one anatomical landmark was manually annotated in a subset of ultrasound frames for each experiment. Probe contact was uninterrupted over the entire scan duration in all ten sessions. Organ drift and imaging artefacts were successfully compensated using remote control. The median contact force along the probe's longitudinal axis was 10.0 N with maximum values of 13.2 and 21.3 N for liver and prostate, respectively. Forces exceeding 11 N only occurred in 0.3% of the time. Probe and landmark motion were more pronounced in the liver, with median interquartile ranges of 1.5 and 9.6 mm, compared to 0.6 and 2.7 mm in the prostate. The results show that robotic ultrasound imaging with dynamic force control can be used for stable, long-term imaging of anatomical regions affected by motion. The system facilitates the acquisition of 4D image datain vivoover extended scanning periods for the first time and holds the potential to be used for motion monitoring for therapy guidance as well as diagnostic tasks.
Collapse
Affiliation(s)
- Svenja Ipsen
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany.,Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering IMTE, Luebeck, Germany
| | - Daniel Wulff
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Ivo Kuhlemann
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Achim Schweikard
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Floris Ernst
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| |
Collapse
|
10
|
von Haxthausen F, Böttger S, Wulff D, Hagenah J, García-Vázquez V, Ipsen S. Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends. ACTA ACUST UNITED AC 2021; 2:55-71. [PMID: 34977593 PMCID: PMC7898497 DOI: 10.1007/s43154-020-00037-y] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/21/2020] [Indexed: 12/17/2022]
Abstract
Abstract
Purpose of Review
This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).
Recent Findings
Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.
Summary
A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.
Collapse
Affiliation(s)
- Felix von Haxthausen
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Sven Böttger
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Daniel Wulff
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Jannis Hagenah
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Verónica García-Vázquez
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Svenja Ipsen
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| |
Collapse
|
11
|
Reinertsen I, Collins DL, Drouin S. The Essential Role of Open Data and Software for the Future of Ultrasound-Based Neuronavigation. Front Oncol 2021; 10:619274. [PMID: 33604299 PMCID: PMC7884817 DOI: 10.3389/fonc.2020.619274] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Accepted: 12/11/2020] [Indexed: 01/17/2023] Open
Abstract
With the recent developments in machine learning and modern graphics processing units (GPUs), there is a marked shift in the way intra-operative ultrasound (iUS) images can be processed and presented during surgery. Real-time processing of images to highlight important anatomical structures combined with in-situ display, has the potential to greatly facilitate the acquisition and interpretation of iUS images when guiding an operation. In order to take full advantage of the recent advances in machine learning, large amounts of high-quality annotated training data are necessary to develop and validate the algorithms. To ensure efficient collection of a sufficient number of patient images and external validity of the models, training data should be collected at several centers by different neurosurgeons, and stored in a standard format directly compatible with the most commonly used machine learning toolkits and libraries. In this paper, we argue that such effort to collect and organize large-scale multi-center datasets should be based on common open source software and databases. We first describe the development of existing open-source ultrasound based neuronavigation systems and how these systems have contributed to enhanced neurosurgical guidance over the last 15 years. We review the impact of the large number of projects worldwide that have benefited from the publicly available datasets “Brain Images of Tumors for Evaluation” (BITE) and “Retrospective evaluation of Cerebral Tumors” (RESECT) that include MR and US data from brain tumor cases. We also describe the need for continuous data collection and how this effort can be organized through the use of a well-adapted and user-friendly open-source software platform that integrates both continually improved guidance and automated data collection functionalities.
Collapse
Affiliation(s)
- Ingerid Reinertsen
- Department of Health Research, SINTEF Digital, Trondheim, Norway.,Department of Circulation and Medical Imaging, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
| | - D Louis Collins
- NIST Laboratory, McConnell Brain Imaging Center, Montreal Neurological Institute and Hospital, McGill University, Montréal, QC, Canada
| | - Simon Drouin
- Laboratoire Multimédia, École de Technologie Supérieure, Montréal, QC, Canada
| |
Collapse
|