1
|
Song L, Li Q, Sai Z, Sun K, Chen W, Zhang H, Wang Y, Jiao Z, Ni X. A Multimodal Point Cloud-Based Method for Tumor Localization in Robotic Ultrasound-Guided Radiotherapy. Technol Cancer Res Treat 2024; 23:15330338241273149. [PMID: 39155658 DOI: 10.1177/15330338241273149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/20/2024] Open
Abstract
Objectives: Part of the tumor localization methods in radiotherapy have poor real-time performance and may generate additional radiation. We propose a multimodal point cloud-based method for tumor localization in robotic ultrasound-guided radiotherapy, which only irradiates computed tomography (CT) during radiotherapy planning to avoid additional radiation. Methods: The tumor position was determined using the CT point cloud, and the red green blue depth (RGBD) point cloud was used to determine body surface scanning location corresponding to the tumor location. The relationship between the CT point cloud and RGBD point cloud was established through multi-modal point cloud registration. The point cloud was then used for robot tumor localization through coordinate transformation between camera and robot. Results: The maximum mean absolute error of the tumor location in the X, Y, and Z directions of the robot coordinate system were 0.781, 1.334, and 1.490 mm, respectively. The average point-to-point translation mean absolute error between the actual and predicted positions of the localization points was 1.847 mm. The maximum error in the random positioning experiment was 1.77 mm. Conclusion: The proposed method is radiation free and has real-time performance, with tumor localization accuracy that meets the requirements of radiotherapy. The proposed method, which potentially reduces the risks associated with radiation exposure while ensuring efficient and accurate tumor localization, represents a promising advancement in the field of radiotherapy.
Collapse
Affiliation(s)
- Lintao Song
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Qixuan Li
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou, China
| | - Zhang Sai
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Kangkang Sun
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou, China
| | - Wei Chen
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou, China
| | - Heng Zhang
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Yibo Wang
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Zhuqing Jiao
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou, China
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou, China
| | - Xinye Ni
- Department of Radiotherapy, The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| |
Collapse
|
2
|
Mezheritsky T, Romaguera LV, Le W, Kadoury S. Population-based 3D respiratory motion modelling from convolutional autoencoders for 2D ultrasound-guided radiotherapy. Med Image Anal 2021; 75:102260. [PMID: 34670149 DOI: 10.1016/j.media.2021.102260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Revised: 09/29/2021] [Accepted: 10/01/2021] [Indexed: 10/20/2022]
Abstract
Radiotherapy is a widely used treatment modality for various types of cancers. A challenge for precise delivery of radiation to the treatment site is the management of internal motion caused by the patient's breathing, especially around abdominal organs such as the liver. Current image-guided radiation therapy (IGRT) solutions rely on ionising imaging modalities such as X-ray or CBCT, which do not allow real-time target tracking. Ultrasound imaging (US) on the other hand is relatively inexpensive, portable and non-ionising. Although 2D US can be acquired at a sufficient temporal frequency, it doesn't allow for target tracking in multiple planes, while 3D US acquisitions are not adapted for real-time. In this work, a novel deep learning-based motion modelling framework is presented for ultrasound IGRT. Our solution includes an image similarity-based rigid alignment module combined with a deep deformable motion model. Leveraging the representational capabilities of convolutional autoencoders, our deformable motion model associates complex 3D deformations with 2D surrogate US images through a common learned low dimensional representation. The model is trained on a variety of deformations and anatomies which enables it to generate the 3D motion experienced by the liver of a previously unseen subject. During inference, our framework only requires two pre-treatment 3D volumes of the liver at extreme breathing phases and a live 2D surrogate image representing the current state of the organ. In this study, the presented model is evaluated on a 3D+t US data set of 20 volunteers based on image similarity as well as anatomical target tracking performance. We report results that surpass comparable methodologies in both metric categories with a mean tracking error of 3.5±2.4 mm, demonstrating the potential of this technique for IGRT.
Collapse
Affiliation(s)
- Tal Mezheritsky
- MedICAL Laboratory, École Polytechnique de Montréal, Montréal, Canada.
| | | | | | - Samuel Kadoury
- MedICAL Laboratory, École Polytechnique de Montréal, Montréal, Canada; CHUM Research Center, Montréal, Canada
| |
Collapse
|
3
|
Ipsen S, Wulff D, Kuhlemann I, Schweikard A, Ernst F. Towards automated ultrasound imaging-robotic image acquisition in liver and prostate for long-term motion monitoring. Phys Med Biol 2021; 66. [PMID: 33770768 DOI: 10.1088/1361-6560/abf277] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 03/26/2021] [Indexed: 11/12/2022]
Abstract
Real-time volumetric (4D) ultrasound has shown high potential for diagnostic and therapy guidance tasks. One of the main drawbacks of ultrasound imaging to date is the reliance on manual probe positioning and the resulting user dependence. Robotic assistance could help overcome this issue and facilitate the acquisition of long-term image data to observe dynamic processesin vivoover time. The aim of this study is to assess the feasibility of robotic probe manipulation and organ motion quantification during extended imaging sessions. The system consists of a collaborative robot and a 4D ultrasound system providing real-time data access. Five healthy volunteers received liver and prostate scans during free breathing over 30 min. Initial probe placement was performed with real-time remote control with a predefined contact force of 10 N. During scan acquisition, the probe position was continuously adjusted to the body surface motion using impedance control. Ultrasound volumes, the pose of the end-effector and the estimated contact forces were recorded. For motion analysis, one anatomical landmark was manually annotated in a subset of ultrasound frames for each experiment. Probe contact was uninterrupted over the entire scan duration in all ten sessions. Organ drift and imaging artefacts were successfully compensated using remote control. The median contact force along the probe's longitudinal axis was 10.0 N with maximum values of 13.2 and 21.3 N for liver and prostate, respectively. Forces exceeding 11 N only occurred in 0.3% of the time. Probe and landmark motion were more pronounced in the liver, with median interquartile ranges of 1.5 and 9.6 mm, compared to 0.6 and 2.7 mm in the prostate. The results show that robotic ultrasound imaging with dynamic force control can be used for stable, long-term imaging of anatomical regions affected by motion. The system facilitates the acquisition of 4D image datain vivoover extended scanning periods for the first time and holds the potential to be used for motion monitoring for therapy guidance as well as diagnostic tasks.
Collapse
Affiliation(s)
- Svenja Ipsen
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany.,Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering IMTE, Luebeck, Germany
| | - Daniel Wulff
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Ivo Kuhlemann
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Achim Schweikard
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Floris Ernst
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| |
Collapse
|
4
|
von Haxthausen F, Böttger S, Wulff D, Hagenah J, García-Vázquez V, Ipsen S. Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends. ACTA ACUST UNITED AC 2021; 2:55-71. [PMID: 34977593 PMCID: PMC7898497 DOI: 10.1007/s43154-020-00037-y] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/21/2020] [Indexed: 12/17/2022]
Abstract
Abstract
Purpose of Review
This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).
Recent Findings
Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.
Summary
A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.
Collapse
Affiliation(s)
- Felix von Haxthausen
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Sven Böttger
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Daniel Wulff
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Jannis Hagenah
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Verónica García-Vázquez
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Svenja Ipsen
- Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| |
Collapse
|
5
|
Czerska K, Emert F, Kopec R, Langen K, McClelland JR, Meijers A, Miyamoto N, Riboldi M, Shimizu S, Terunuma T, Zou W, Knopf A, Rucinski A. Clinical practice vs. state-of-the-art research and future visions: Report on the 4D treatment planning workshop for particle therapy - Edition 2018 and 2019. Phys Med 2021; 82:54-63. [PMID: 33588228 DOI: 10.1016/j.ejmp.2020.12.013] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Revised: 12/09/2020] [Accepted: 12/16/2020] [Indexed: 12/18/2022] Open
Abstract
The 4D Treatment Planning Workshop for Particle Therapy, a workshop dedicated to the treatment of moving targets with scanned particle beams, started in 2009 and since then has been organized annually. The mission of the workshop is to create an informal ground for clinical medical physicists, medical physics researchers and medical doctors interested in the development of the 4D technology, protocols and their translation into clinical practice. The 10th and 11th editions of the workshop took place in Sapporo, Japan in 2018 and Krakow, Poland in 2019, respectively. This review report from the Sapporo and Krakow workshops is structured in two parts, according to the workshop programs. The first part comprises clinicians and physicists review of the status of 4D clinical implementations. Corresponding talks were given by speakers from five centers around the world: Maastro Clinic (The Netherlands), University Medical Center Groningen (The Netherlands), MD Anderson Cancer Center (United States), University of Pennsylvania (United States) and The Proton Beam Therapy Center of Hokkaido University Hospital (Japan). The second part is dedicated to novelties in 4D research, i.e. motion modelling, artificial intelligence and new technologies which are currently being investigated in the radiotherapy field.
Collapse
Affiliation(s)
- Katarzyna Czerska
- Institute of Nuclear Physics Polish Academy of Sciences, PL-31342 Krakow, Poland.
| | - Frank Emert
- Center for Proton Therapy, Paul Scherrer Institute, Switzerland
| | - Renata Kopec
- Institute of Nuclear Physics Polish Academy of Sciences, PL-31342 Krakow, Poland
| | - Katja Langen
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Jamie R McClelland
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - Arturs Meijers
- Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Naoki Miyamoto
- Department of Medical Physics, Hokkaido University Hospital, Sapporo, Hokkaido, Japan; Faculty of Engineering, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Marco Riboldi
- Department of Medical Physics, Ludwig-Maximilians-Universität München, Germany
| | - Shinichi Shimizu
- Department of Medical Physics, Hokkaido University Hospital, Sapporo, Hokkaido, Japan; Department of Radiation Medical Science and Engineering, Faculty of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Toshiyuki Terunuma
- Faculty of Medicine, University of Tsukuba, Japan; Proton Medical Research Center, University of Tsukuba Hospital, Japan
| | - Wei Zou
- Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, USA
| | - Antje Knopf
- Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Antoni Rucinski
- Institute of Nuclear Physics Polish Academy of Sciences, PL-31342 Krakow, Poland
| |
Collapse
|