1
|
Huang L, Kurz C, Freislederer P, Manapov F, Corradini S, Niyazi M, Belka C, Landry G, Riboldi M. Simultaneous object detection and segmentation for patient-specific markerless lung tumor tracking in simulated radiographs with deep learning. Med Phys 2024; 51:1957-1973. [PMID: 37683107 DOI: 10.1002/mp.16705] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2022] [Revised: 04/23/2023] [Accepted: 05/12/2023] [Indexed: 09/10/2023] Open
Abstract
BACKGROUND Real-time tumor tracking is one motion management method to address motion-induced uncertainty. To date, fiducial markers are often required to reliably track lung tumors with X-ray imaging, which carries risks of complications and leads to prolonged treatment time. A markerless tracking approach is thus desirable. Deep learning-based approaches have shown promise for markerless tracking, but systematic evaluation and procedures to investigate applicability in individual cases are missing. Moreover, few efforts have been made to provide bounding box prediction and mask segmentation simultaneously, which could allow either rigid or deformable multi-leaf collimator tracking. PURPOSE The purpose of this study was to implement a deep learning-based markerless lung tumor tracking model exploiting patient-specific training which outputs both a bounding box and a mask segmentation simultaneously. We also aimed to compare the two kinds of predictions and to implement a specific procedure to understand the feasibility of markerless tracking on individual cases. METHODS We first trained a Retina U-Net baseline model on digitally reconstructed radiographs (DRRs) generated from a public dataset containing 875 CT scans and corresponding lung nodule annotations. Afterwards, we used an independent cohort of 97 lung patients to develop a patient-specific refinement procedure. In order to determine the optimal hyperparameters for automatic patient-specific training, we selected 13 patients for validation where the baseline model predicted a bounding box on planning CT (PCT)-DRR with intersection over union (IoU) with the ground-truth higher than 0.7. The final test set contained the remaining 84 patients with varying PCT-DRR IoU. For each testing patient, the baseline model was refined on the PCT-DRR to generate a patient-specific model, which was then tested on a separate 10-phase 4DCT-DRR to mimic the intrafraction motion during treatment. A template matching algorithm served as benchmark model. The testing results were evaluated by four metrics: the center of mass (COM) error and the Dice similarity coefficient (DSC) for segmentation masks, and the center of box (COB) error and the DSC for bounding box detections. Performance was compared to the benchmark model including statistical testing for significance. RESULTS A PCT-DRR IoU value of 0.2 was shown to be the threshold dividing inconsistent (68%) and consistent (100%) success (defined as mean bounding box DSC > 0.6) of PS models on 4DCT-DRRs. Thirty-seven out of the eighty-four testing cases had a PCT-DRR IoU above 0.2. For these 37 cases, the mean COM error was 2.6 mm, the mean segmentation DSC was 0.78, the mean COB error was 2.7 mm, and the mean box DSC was 0.83. Including the validation cases, the model was applicable to 50 out of 97 patients when using the PCT-DRR IoU threshold of 0.2. The inference time per frame was 170 ms. The model outperformed the benchmark model on all metrics, and the comparison was significant (p < 0.001) over the 37 PCT-DRR IoU > 0.2 cases, but not over the undifferentiated 84 testing cases. CONCLUSIONS The implemented patient-specific refinement approach based on a pre-trained baseline model was shown to be applicable to markerless tumor tracking in simulated radiographs for lung cases.
Collapse
Affiliation(s)
- Lili Huang
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
- Department of Medical Physics, Faculty of Physics, Ludwig-Maximilians-Universität München, München, Germany
| | - Christopher Kurz
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Philipp Freislederer
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Farkhad Manapov
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Stefanie Corradini
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Maximilian Niyazi
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Claus Belka
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
- German Cancer Consortium (DKTK), partner site Munich, a partnership between DKFZ and LMU University Hospital Munich, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Guillaume Landry
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Marco Riboldi
- Department of Medical Physics, Faculty of Physics, Ludwig-Maximilians-Universität München, München, Germany
| |
Collapse
|
2
|
Lin Z, Lei C, Yang L. Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization. SENSORS (BASEL, SWITZERLAND) 2023; 23:9872. [PMID: 38139718 PMCID: PMC10748263 DOI: 10.3390/s23249872] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 11/15/2023] [Accepted: 12/13/2023] [Indexed: 12/24/2023]
Abstract
Medical image analysis forms the basis of image-guided surgery (IGS) and many of its fundamental tasks. Driven by the growing number of medical imaging modalities, the research community of medical imaging has developed methods and achieved functionality breakthroughs. However, with the overwhelming pool of information in the literature, it has become increasingly challenging for researchers to extract context-relevant information for specific applications, especially when many widely used methods exist in a variety of versions optimized for their respective application domains. By being further equipped with sophisticated three-dimensional (3D) medical image visualization and digital reality technology, medical experts could enhance their performance capabilities in IGS by multiple folds. The goal of this narrative review is to organize the key components of IGS in the aspects of medical image processing and visualization with a new perspective and insights. The literature search was conducted using mainstream academic search engines with a combination of keywords relevant to the field up until mid-2022. This survey systemically summarizes the basic, mainstream, and state-of-the-art medical image processing methods as well as how visualization technology like augmented/mixed/virtual reality (AR/MR/VR) are enhancing performance in IGS. Further, we hope that this survey will shed some light on the future of IGS in the face of challenges and opportunities for the research directions of medical image processing and visualization.
Collapse
Affiliation(s)
- Zhefan Lin
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310030, China;
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China;
| | - Chen Lei
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China;
| | - Liangjing Yang
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310030, China;
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China;
| |
Collapse
|
3
|
Terunuma T, Sakae T, Hu Y, Takei H, Moriya S, Okumura T, Sakurai H. Explainability and controllability of patient-specific deep learning with attention-based augmentation for markerless image-guided radiotherapy. Med Phys 2023; 50:480-494. [PMID: 36354286 PMCID: PMC10100026 DOI: 10.1002/mp.16095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 10/27/2022] [Accepted: 10/27/2022] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND We reported the concept of patient-specific deep learning (DL) for real-time markerless tumor segmentation in image-guided radiotherapy (IGRT). The method was aimed to control the attention of convolutional neural networks (CNNs) by artificial differences in co-occurrence probability (CoOCP) in training datasets, that is, focusing CNN attention on soft tissues while ignoring bones. However, the effectiveness of this attention-based data augmentation has not been confirmed by explainable techniques. Furthermore, compared to reasonable ground truths, the feasibility of tumor segmentation in clinical kilovolt (kV) X-ray fluoroscopic (XF) images has not been confirmed. PURPOSE The first aim of this paper was to present evidence that the proposed method provides an explanation and control of DL behavior. The second purpose was to validate the real-time lung tumor segmentation in clinical kV XF images for IGRT. METHODS This retrospective study included 10 patients with lung cancer. Patient-specific and XF angle-specific image pairs comprising digitally reconstructed radiographs (DRRs) and projected-clinical-target-volume (pCTV) images were calculated from four-dimensional computer tomographic data and treatment planning information. The training datasets were primarily augmented by random overlay (RO) and noise injection (NI): RO aims to differentiate positional CoOCP in soft tissues and bones, and NI aims to make a difference in the frequency of occurrence of local and global image features. The CNNs for each patient-and-angle were automatically optimized in the DL training stage to transform the training DRRs into pCTV images. In the inference stage, the trained CNNs transformed the test XF images into pCTV images, thus identifying target positions and shapes. RESULTS The visual analysis of DL attention heatmaps for a test image demonstrated that our method focused CNN attention on soft tissue and global image features rather than bones and local features. The processing time for each patient-and-angle-specific dataset in the training stage was ∼30 min, whereas that in the inference stage was 8 ms/frame. The estimated three-dimensional 95 percentile tracking error, Jaccard index, and Hausdorff distance for 10 patients were 1.3-3.9 mm, 0.85-0.94, and 0.6-4.9 mm, respectively. CONCLUSIONS The proposed attention-based data augmentation with both RO and NI made the CNN behavior more explainable and more controllable. The results obtained demonstrated the feasibility of real-time markerless lung tumor segmentation in kV XF images for IGRT.
Collapse
Affiliation(s)
- Toshiyuki Terunuma
- Faculty of Medicine, University of Tsukuba, Tsukuba, Japan.,Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan
| | - Takeji Sakae
- Faculty of Medicine, University of Tsukuba, Tsukuba, Japan.,Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan
| | - Yachao Hu
- Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan.,Center Hospital and Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shenzhen, China
| | - Hideyuki Takei
- Faculty of Medicine, University of Tsukuba, Tsukuba, Japan.,Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan
| | - Shunsuke Moriya
- Faculty of Medicine, University of Tsukuba, Tsukuba, Japan.,Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan
| | - Toshiyuki Okumura
- Faculty of Medicine, University of Tsukuba, Tsukuba, Japan.,Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan
| | - Hideyuki Sakurai
- Faculty of Medicine, University of Tsukuba, Tsukuba, Japan.,Proton Medical Research Center, University of Tsukuba Hospital, Tsukuba, Japan
| |
Collapse
|
4
|
Nakao M, Nakamura M, Matsuda T. Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3747-3761. [PMID: 35901001 DOI: 10.1109/tmi.2022.3194517] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.
Collapse
|
5
|
de Bruin K, Dahele M, Mostafavi H, Slotman BJ, Verbakel WF. Markerless Real-Time 3-Dimensional kV Tracking of Lung Tumors During Free Breathing Stereotactic Radiation Therapy. Adv Radiat Oncol 2021; 6:100705. [PMID: 34113742 PMCID: PMC8170355 DOI: 10.1016/j.adro.2021.100705] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2020] [Revised: 03/04/2021] [Accepted: 03/30/2021] [Indexed: 11/30/2022] Open
Abstract
Purpose Accurate verification of tumor position during irradiation could reduce the probability of target miss. We investigated whether a commercial gantry-mounted 2-dimensional (2D) kilo-voltage (kV) imaging system could be used for real-time 3D tumor tracking during volumetric modulated arc therapy (VMAT) lung stereotactic body radiation therapy (SBRT). Markerless tumor tracking on kV fluoroscopic images was validated using a life-like moving thorax phantom and subsequently performed on kV images continuously acquired before and during free-breathing VMAT lung SBRT. Methods and Materials The 3D-printed/molded phantom containing 3 lung tumors was moved in 3D in TrueBeam developer mode, using simulated regular/irregular breathing patterns. Planar kV images were acquired at 7 frames/s during 11 Gy/fraction 10 MV flattening filter free VMAT. 2D reference templates were created for each gantry angle using the planning 4D computed tomography inspiration phase. kV images and templates were matched using normalized cross correlation to determine 2D tumor position, and triangulation of 2D matched projections determined the third dimension. 3D target tracking performed on cone beam computed tomography projection data from 18 patients (20 tumors) and real-time online tracking data from 2 of the 18 patients who underwent free-breathing VMAT lung SBRT are presented. Results For target 1 and 2 of the phantom (upper lung and middle/medial lung, mean density –130 Hounsfield units), 3D results within 2 mm of the known position were present in 92% and 96% of the kV projections, respectively. For target 3 (inferior lung, mean density –478 Hounsfield units) this dropped to 80%. Benchmarking against the respiratory signal, 13/20 (65%) tumors (10.5 ± 11.1 cm3) were considered successfully tracked on the cone beam computed tomography data. Tracking was less successful (≤50% of the time) in 7/20 (1.2 ± 1.5 cm3). Successful online tracking during lung SBRT was demonstrated. Conclusions 3D markerless tumor tracking on a standard linear accelerator using template matching and triangulation of free-breathing kV fluoroscopic images was possible in 65% of small lung tumors. The smallest tumors were most challenging.
Collapse
Affiliation(s)
- Kimmie de Bruin
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Cancer Center Amsterdam, Amsterdam, the Netherlands
| | - Max Dahele
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Cancer Center Amsterdam, Amsterdam, the Netherlands
| | | | - Berend J. Slotman
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Cancer Center Amsterdam, Amsterdam, the Netherlands
| | - Wilko F.A.R. Verbakel
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Cancer Center Amsterdam, Amsterdam, the Netherlands
- Corresponding author: Wilko F.A.R. Verbakel, PhD, PDEng
| |
Collapse
|
6
|
Dellmann MFW, Jerg KI, Stratemeier J, Heiman R, Hesser JW, Aschenbrenner KP, Blessing M. Noise-robust breathing-phase estimation on marker-free, ultra low dose X-ray projections for real-time tumor localization via surrogate structures. Z Med Phys 2021; 31:355-364. [PMID: 34088565 DOI: 10.1016/j.zemedi.2021.04.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2020] [Revised: 11/11/2020] [Accepted: 04/08/2021] [Indexed: 11/25/2022]
Abstract
PURPOSE This paper presents a novel strategy for feature-based breathing-phase estimation on ultra low-dose X-ray projections for tumor motion control in radiation therapy. METHODS Coarse-scaled Curvelet coefficients are identified as motion sensitive but noise-robust features for this purpose. For feature-based breathing-phase estimation, an ensemble strategy with two classifiers is used. This consensus-based estimation substantially increases tracking reliability by rejection of false positives. The algorithm is evaluated on both synthetic and measured phantom data: Monte Carlo simulated ultra low dose projections for a C-arm X-ray and on the basis of 4D-chest-CTs of eight patients on one hand side and real measurements based on a motion phantom. RESULTS To achieve an accuracy of breathing-phase estimation of more than 95% a fluence between 20 and 400 photons per pixel (open field) is required depending on the patient. Furthermore, the algorithm is evaluated on real ultra low dose projections from an XVI R5.0 system (Elekta AB, Stockholm, Sweden) using an additional lead filter to reduce fluence. The classifiers-consensus-based-gating method estimated the correct position of the test projections in all test cases at a fluence of ∼180 photons per pixel and 92% at a fluence of ∼40 photons per pixel. The deposited dose to patient per image is in the range of nGy. CONCLUSIONS A novel method is presented for estimation of breathing-phases for real-time tumor localization at ultra low dose both on a simulation and a phantom basis. Its accuracy is comparable to state of the art X-ray based algorithms while the released dose to patients is reduced by two to three orders of magnitude compared to conventional template-based approaches. This allows for continuous motion control during irradiation without the need of external markers.
Collapse
Affiliation(s)
- Max F W Dellmann
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany.
| | - Katharina I Jerg
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany
| | - Johanna Stratemeier
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany
| | - Ron Heiman
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany
| | - Jürgen W Hesser
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany; Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Im Neuenheimer Feld 205, 69120 Heidelberg, Germany; Central Institute for Computer Engineering (ZITI), Heidelberg University, Im Neuenheimer Feld 368, 69120 Heidelberg, Germany
| | - Katharina P Aschenbrenner
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany; Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Im Neuenheimer Feld 205, 69120 Heidelberg, Germany
| | - Manuel Blessing
- Department of Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167 Mannheim, Germany
| |
Collapse
|
7
|
Statistical deformation reconstruction using multi-organ shape features for pancreatic cancer localization. Med Image Anal 2021; 67:101829. [DOI: 10.1016/j.media.2020.101829] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2020] [Revised: 08/12/2020] [Accepted: 09/12/2020] [Indexed: 11/20/2022]
|
8
|
Dhont J, Verellen D, Mollaert I, Vanreusel V, Vandemeulebroucke J. RealDRR - Rendering of realistic digitally reconstructed radiographs using locally trained image-to-image translation. Radiother Oncol 2020; 153:213-219. [PMID: 33039426 DOI: 10.1016/j.radonc.2020.10.004] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Revised: 09/30/2020] [Accepted: 10/01/2020] [Indexed: 12/25/2022]
Abstract
INTRODUCTION Digitally reconstructed radiographs (DRRs) represent valuable patient-specific pre-treatment training data for tumor tracking algorithms. However, using current rendering methods, the similarity of the DRRs to real X-ray images is limited, requires time-consuming measurements and/or are computationally expensive. In this study we present RealDRR, a novel framework for highly realistic and computationally efficient DRR rendering. MATERIALS AND METHODS RealDRR consists of two components applied sequentially to render a DRR. First, a raytracer is applied for forward projection from 3D CT data to a 2D image. Second, a conditional Generative Adverserial Network (cGAN) is applied to translate the 2D forward projection to a realistic 2D DRR. The planning CT and CBCT projections from a CIRS thorax phantom and 6 radiotherapy patients (3 prostate, 3 brain) were split in training and test sets for evaluating the intra-patient, inter-patient and inter-anatomical region generalization performance of the trained framework. Several image similarity metrics, as well as a verification based on template matching, were used between the rendered DRRs and respective CBCT projections in the test sets, and results were compared to those of a current state-of-the-art DRR rendering method. RESULTS When trained on 800 CBCT projection images from two patients and tested on a third unseen patient from either anatomical region, RealDRR outperformed the current state-of-the-art with statistical significance on all metrics (two-sample t-test, p < 0.05). Once trained, the framework is able to render 100 highly realistic DRRs in under two minutes. CONCLUSION A novel framework for realistic and efficient DRR rendering was proposed. As the framework requires a reasonable amount of computational resources, the internal parameters can be tailored to imaging systems and protocols through on-site training on retrospective imaging data.
Collapse
Affiliation(s)
- Jennifer Dhont
- Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel, Brussels, Belgium; Imec, Leuven, Belgium; Faculty of Medicine and Pharmaceutical Sciences, Vrije Universiteit Brussel, Brussels, Belgium.
| | - Dirk Verellen
- Iridium Kankernetwerk, Antwerp, Belgium; University of Antwerp, Faculty of Medicine and Health Sciences, Antwerp, Belgium
| | | | | | - Jef Vandemeulebroucke
- Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel, Brussels, Belgium; Imec, Leuven, Belgium
| |
Collapse
|
9
|
Mann P, Witte M, Mercea P, Nill S, Lang C, Karger CP. Feasibility of markerless fluoroscopic real-time tumor detection for adaptive radiotherapy: development and end-to-end testing. Phys Med Biol 2020; 65:115002. [PMID: 32235075 DOI: 10.1088/1361-6560/ab8578] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
Respiratory-gated radiotherapy treatments of lung tumors reduce the irradiated normal tissue volume and potentially lower the risk of side effects. However, in clinical routine, the gating signal is usually derived from external markers or other surrogate signals and may not always correlate well with the actual tumor position. This study uses the kV-imaging system of a LINAC in combination with a multiple template matching algorithm for markerless real-time detection of the tumor position in a dynamic anthropomorphic porcine lung phantom. The tumor was realized by a small container filled with polymer dosimetry gel, the so-called gel tumor. A full end-to-end test for a gated treatment was performed and the geometric and dosimetric accuracy was validated. The accuracy of the tumor detection algorithm in SI- direction was found to be [Formula: see text] mm and the gel tumor was automatically detected in 98 out of 100 images. The measured 3D dose distribution showed a uniform coverage of the gel tumor and comparison with the treatment plan revealed a high 3D [Formula: see text]-passing rate of [Formula: see text] ([Formula: see text]). The simulated treatment confirmed the employed margin sizes for residual motion within the gating window and serves as an end-to-end test for a gated treatment based on a markerless fluoroscopic real-time tumor detection.
Collapse
Affiliation(s)
- P Mann
- Department of Medical Physics in Radiation Therapy, German Cancer Research Center, Im Neuenheimer Feld 280, Heidelberg, Germany. National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Im Neuenheimer Feld 280, Heidelberg, Germany
| | | | | | | | | | | |
Collapse
|
10
|
Takahashi W, Oshikawa S, Mori S. Real-time markerless tumour tracking with patient-specific deep learning using a personalised data generation strategy: proof of concept by phantom study. Br J Radiol 2020; 93:20190420. [PMID: 32101456 PMCID: PMC7217583 DOI: 10.1259/bjr.20190420] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Revised: 01/20/2020] [Accepted: 02/07/2020] [Indexed: 11/05/2022] Open
Abstract
OBJECTIVE For real-time markerless tumour tracking in stereotactic lung radiotherapy, we propose a different approach which uses patient-specific deep learning (DL) using a personalised data generation strategy, avoiding the need for collection of a large patient data set. We validated our strategy with digital phantom simulation and epoxy phantom studies. METHODS We developed lung tumour tracking for radiotherapy using a convolutional neural network trained for each phantom's lesion by using multiple digitally reconstructed radiographs (DRRs) generated from each phantom's treatment planning four-dimensional CT. We trained tumour-bone differentiation using large numbers of training DRRs generated with various projection geometries to simulate tumour motion. We solved the problem of using DRRs for training and X-ray images for tracking using the training DRRs with random contrast transformation and random noise addition. RESULTS We defined adequate tracking accuracy as the percentage frames satisfying <1 mm tracking error of the isocentre. In the simulation study, we achieved 100% tracking accuracy in 3 cm spherical and 1.5×2.25×3 cm ovoid masses. In the phantom study, we achieved 100 and 94.7% tracking accuracy in 3 cm and 2 cm spherical masses, respectively. This required 32.5 ms/frame (30.8 fps) real-time processing. CONCLUSIONS We proved the potential feasibility of a real-time markerless tumour tracking framework for stereotactic lung radiotherapy based on patient-specific DL with personalised data generation with digital phantom and epoxy phantom studies. ADVANCES IN KNOWLEDGE Using DL with personalised data generation is an efficient strategy for real-time lung tumour tracking.
Collapse
Affiliation(s)
- Wataru Takahashi
- Technology Research Laboratory, Shimadzu Corporation, Kyoto, 619-0237, Japan
| | - Shota Oshikawa
- Technology Research Laboratory, Shimadzu Corporation, Kyoto, 619-0237, Japan
| | - Shinichiro Mori
- Research Center for Charged Particle Therapy, National Institute of Radiological Sciences, Chiba, 263-8555, Japan
| |
Collapse
|
11
|
Mueller M, Zolfaghari R, Briggs A, Furtado H, Booth J, Keall P, Nguyen D, O'Brien R, Shieh CC. The first prospective implementation of markerless lung target tracking in an experimental quality assurance procedure on a standard linear accelerator. Phys Med Biol 2020; 65:025008. [PMID: 31783395 DOI: 10.1088/1361-6560/ab5d8b] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
The ability to track tumour motion without implanted markers on a standard linear accelerator (linac) could enable wide access to real-time adaptive radiotherapy for cancer patients. We previously have retrospectively validated a method for 3D markerless target tracking using intra-fractional kilovoltage (kV) projections acquired on a standard linac. This paper presents the first prospective implementation of markerless lung target tracking on a standard linac and its quality assurance (QA) procedure. The workflow and the algorithm developed to track the 3D target position during volumetric modulated arc therapy treatment delivery were optimised. The linac was operated in clinical QA mode, while kV projections were streamed to a dedicated computer using a frame-grabber software. The markerless target tracking accuracy and precision were measured in a lung phantom experiment under the following conditions: static localisation of seven distinct positions, dynamic localisation of five patient-measured motion traces, and dynamic localisation with treatment interruption. The QA guidelines were developed following the AAPM Task Group 147 report with the requirement that the tracking margin components, the margins required to account for tracking errors, did not exceed 5 mm in any direction. The mean tracking error ranged from 0.0 to 0.9 mm (left-right), -0.6 to -0.1 mm (superior-inferior) and -0.7 to 0.1 mm (anterior-posterior) over the three tests. Larger errors were found in cases with large left-right or anterior-posterior and small superior-inferior motion. The tracking margin components did not exceed 5 mm in any direction and ranged from 0.4 to 3.2 mm (left-right), 0.7 to 1.6 mm (superior-inferior) and 0.8 to 1.5 mm (anterior-posterior). This study presents the first prospective implementation of markerless lung target tracking on a standard linac and provides a QA procedure for its safe clinical implementation, potentially enabling real-time adaptive radiotherapy for a large population of lung cancer patients.
Collapse
Affiliation(s)
- Marco Mueller
- ACRF Image X Institute, The University of Sydney, Sydney, NSW, Australia. Author to whom any correspondence should be addressed
| | | | | | | | | | | | | | | | | |
Collapse
|
12
|
Jerg KI, Lyatskaya Y, Stratemeier J, Hesser JW, Aschenbrenner KP. Conditional random fields for phase-based lung feature tracking with ultra-low-dose x-rays. Med Phys 2019; 46:2337-2346. [PMID: 30779358 DOI: 10.1002/mp.13447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2018] [Revised: 02/07/2019] [Accepted: 02/07/2019] [Indexed: 11/06/2022] Open
Abstract
PURPOSE During radiation therapy, a continuous internal tumor monitoring without additional imaging dose is desirable. In this study, a sequential feature-based position estimation with ultra-low-dose (ULD) kV x rays using linear-chain conditional random fields (CRFs) is performed. METHODS Four-dimensional computed tomography (4D-CTs) of eight patients serve as a-priori information from which ULD projections are simulated using a Monte Carlo method. CRFs are trained with Local Energy-based Shape Histogram features extracted from the ULD images to estimate one out of ten breathing phases from the 4D-CT associated with the tumor position. RESULTS Compared to a mean accuracy for ±1 breathing phase of 0.867 using a support vector machine (SVM), a mean accuracy of 0.958 results for the CRF with ten incident photons per pixel. This corresponds to a position estimation with a discretization error of 2.4-5.3 mm assuming a linear displacement relation between the breathing phases and a systematic error of 2.0-4.4 mm due to motion underestimation of the 4D-CT. CONCLUSIONS The tumor position estimation is comparable to state-of-the-art methods despite its low imaging dose. Training CRFs further allows a prediction of the following phase and offers a precise post-treatment evaluation tool when decoding the full image sequence.
Collapse
Affiliation(s)
- Katharina I Jerg
- Department of Experimental Radiation Oncology, Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Yulia Lyatskaya
- Department of Radiation Oncology, Brigham and Women's Hospital, 75 Francis Street, Boston, MA, 02115, USA.,Harvard Medical School, 25 Shattuck St, Boston, MA, 02115, USA
| | - Johanna Stratemeier
- Department of Experimental Radiation Oncology, Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Jürgen W Hesser
- Department of Experimental Radiation Oncology, Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany.,IWR, Heidelberg University, Im Neuenheimer Feld 205, 69120, Heidelberg, Germany
| | - Katharina P Aschenbrenner
- Department of Experimental Radiation Oncology, Medical Faculty of Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany.,IWR, Heidelberg University, Im Neuenheimer Feld 205, 69120, Heidelberg, Germany
| |
Collapse
|
13
|
Bhusal Chhatkuli R, Demachi K, Uesaka M, Nakagawa K, Haga A. Development of a markerless tumor-tracking algorithm using prior four-dimensional cone-beam computed tomography. JOURNAL OF RADIATION RESEARCH 2019; 60:109-115. [PMID: 30407560 PMCID: PMC6373695 DOI: 10.1093/jrr/rry085] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2018] [Revised: 05/14/2018] [Indexed: 06/08/2023]
Abstract
Respiratory motion management is a huge challenge in radiation therapy. Respiratory motion induces temporal anatomic changes that distort the tumor volume and its position. In this study, a markerless tumor-tracking algorithm was investigated by performing phase recognition during stereotactic body radiation therapy (SBRT) using four-dimensional cone-beam computer tomography (4D-CBCT) obtained at patient registration, and in-treatment cone-beam projection images. The data for 20 treatment sessions (five lung cancer patients) were selected for this study. Three of the patients were treated with conventional flattening filter (FF) beams, and the other two were treated with flattening filter-free (FFF) beams. Prior to treatment, 4D-CBCT was acquired to create the template projection images for 10 phases. In-treatment images were obtained at near real time during treatment. Template-based phase recognition was performed for 4D-CBCT re-projected templates using prior 4D-CBCT based phase recognition algorithm and was compared with the results generated by the Amsterdam Shroud (AS) technique. Visual verification technique was used for the verification of the phase recognition and AS technique at certain tumor-visible angles. Offline template matching analysis using the cross-correlation indicated that phase recognition performed using the prior 4D-CBCT and visual verification matched up to 97.5% in the case of FFF, and 95% in the case of FF, whereas the AS technique matched 83.5% with visual verification for FFF and 93% for FF. Markerless tumor tracking based on phase recognition using prior 4D-CBCT has been developed successfully. This is the first study that reports on the use of prior 4D-CBCT based on normalized cross-correlation technique for phase recognition.
Collapse
Affiliation(s)
- Ritu Bhusal Chhatkuli
- Department of Nuclear Engineering and Management, School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Kazuyuki Demachi
- Department of Nuclear Engineering and Management, School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Mitsuru Uesaka
- Department of Nuclear Engineering and Management, School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Keiichi Nakagawa
- Department of Radiology, The University of Tokyo hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Akihiro Haga
- Department of Radiology, The University of Tokyo hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
- Graduate School of Biomedical Sciences, Tokushima University, 3-18-15, Kuramoto-cho, Tokushima, Japan
| |
Collapse
|
14
|
Ichiji K, Yoshida Y, Homma N, Zhang X, Bukovsky I, Takai Y, Yoshizawa M. A key-point based real-time tracking of lung tumor in x-ray image sequence by using difference of Gaussians filtering and optical flow. ACTA ACUST UNITED AC 2018; 63:185007. [DOI: 10.1088/1361-6560/aada71] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
15
|
Hazelaar C, Dahele M, Mostafavi H, van der Weide L, Slotman B, Verbakel W. Markerless positional verification using template matching and triangulation of kV images acquired during irradiation for lung tumors treated in breath-hold. ACTA ACUST UNITED AC 2018; 63:115005. [DOI: 10.1088/1361-6560/aac1a9] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
16
|
Feasibility of using single photon counting X-ray for lung tumor position estimation based on 4D-CT. Z Med Phys 2017; 27:243-254. [PMID: 28595774 DOI: 10.1016/j.zemedi.2017.05.001] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2016] [Revised: 01/04/2017] [Accepted: 05/12/2017] [Indexed: 11/24/2022]
Abstract
PURPOSE In stereotactic body radiation therapy of lung tumors, reliable position estimation of the tumor is necessary in order to minimize normal tissue complication rate. While kV X-ray imaging is frequently used, continuous application during radiotherapy sessions is often not possible due to concerns about the additional dose. Thus, ultra low-dose (ULD) kV X-ray imaging based on a single photon counting detector is suggested. This paper addresses the lower limit of photons to locate the tumor reliably with an accuracy in the range of state-of-the-art methods, i.e. a few millimeters. METHOD 18 patient cases with four dimensional CT (4D-CT), which serves as a-priori information, are included in the study. ULD cone beam projections are simulated from the 4D-CTs including Poisson noise. The projections from the breathing phases which correspond to different tumor positions are compared to the ULD projection by means of Poisson log-likelihood (PML) and correlation coefficient (CC), and template matching under these metrics. RESULTS The results indicate that in full thorax imaging five photons per pixel suffice for a standard deviation in tumor positions of less than half a breathing phase. Around 50 photons per pixel are needed to achieve this accuracy with the field of view restricted to the tumor region. Compared to CC, PML tends to perform better for low photon counts and shifts in patient setup. Template matching only improves the position estimation in high photon counts. The quality of the reconstruction is independent of the projection angle. CONCLUSIONS The accuracy of the proposed ULD single photon counting system is in the range of a few millimeters and therefore comparable to state-of-the-art tumor tracking methods. At the same time, a reduction in photons per pixel by three to four orders of magnitude relative to commercial systems with flatpanel detectors can be achieved. This enables continuous kV image-based position estimation during all fractions since the additional dose to the patient is negligible.
Collapse
|
17
|
Shieh CC, Caillet V, Dunbar M, Keall PJ, Booth JT, Hardcastle N, Haddad C, Eade T, Feain I. A Bayesian approach for three-dimensional markerless tumor tracking using kV imaging during lung radiotherapy. Phys Med Biol 2017; 62:3065-3080. [PMID: 28323642 DOI: 10.1088/1361-6560/aa6393] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
The ability to monitor tumor motion without implanted markers can potentially enable broad access to more accurate and precise lung radiotherapy. A major challenge is that kilovoltage (kV) imaging based methods are rarely able to continuously track the tumor due to the inferior tumor visibility on 2D kV images. Another challenge is the estimation of 3D tumor position based on only 2D imaging information. The aim of this work is to address both challenges by proposing a Bayesian approach for markerless tumor tracking for the first time. The proposed approach adopts the framework of the extended Kalman filter, which combines a prediction and measurement steps to make the optimal tumor position update. For each imaging frame, the tumor position is first predicted by a respiratory-correlated model. The 2D tumor position on the kV image is then measured by template matching. Finally, the prediction and 2D measurement are combined based on the 3D distribution of tumor positions in the past 10 s and the estimated uncertainty of template matching. To investigate the clinical feasibility of the proposed method, a total of 13 lung cancer patient datasets were used for retrospective validation, including 11 cone-beam CT scan pairs and two stereotactic ablative body radiotherapy cases. The ground truths for tumor motion were generated from the the 3D trajectories of implanted markers or beacons. The mean, standard deviation, and 95th percentile of the 3D tracking error were found to range from 1.6-2.9 mm, 0.6-1.5 mm, and 2.6-5.8 mm, respectively. Markerless tumor tracking always resulted in smaller errors compared to the standard of care. The improvement was the most pronounced in the superior-inferior (SI) direction, with up to 9.5 mm reduction in the 95th-percentile SI error for patients with >10 mm 5th-to-95th percentile SI tumor motion. The percentage of errors with 3D magnitude <5 mm was 96.5% for markerless tumor tracking and 84.1% for the standard of care. The feasibility of 3D markerless tumor tracking has been demonstrated on realistic clinical scenarios for the first time. The clinical implementation of the proposed method will enable more accurate and precise lung radiotherapy using existing hardware and workflow. Future work is focused on the clinical and real-time implementation of this method.
Collapse
Affiliation(s)
- Chun-Chien Shieh
- Sydney Medical School, The University of Sydney, NSW 2006, Australia
| | | | | | | | | | | | | | | | | |
Collapse
|
18
|
Nasehi Tehrani J, McEwan A, Wang J. Lung surface deformation prediction from spirometry measurement and chest wall surface motion. Med Phys 2016; 43:5493. [PMID: 27782714 PMCID: PMC5035308 DOI: 10.1118/1.4962479] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2016] [Revised: 08/26/2016] [Accepted: 08/29/2016] [Indexed: 11/07/2022] Open
Abstract
PURPOSE The authors have developed and evaluated a method to predict lung surface motion based on spirometry measurements, and chest and abdomen motion at selected locations. METHODS A patient-specific 3D triangular surface mesh of the lung region was obtained at the end expiratory phase by the threshold-based segmentation method. Lung flow volume changes were recorded with a spirometer for each patient. A total of 192 selected points at a regular spacing of 2 × 2 cm matrix points were used to detect chest wall motion over a total area of 32 × 24 cm covering the chest and abdomen surfaces. QR factorization with column pivoting was employed to remove redundant observations of the chest and abdominal areas. To create a statistical model between the lung surface and the corresponding surrogate signals, the authors developed a predictive model based on canonical ridge regression. Two unique weighting vectors were selected for each vertex on the lung surface; they were optimized during the training process using all other 4D-CT phases except for the test inspiration phase. These parameters were employed to predict the vertex locations of a testing data set. RESULTS The position of each lung surface mesh vertex was estimated from the motion at selected positions within the chest wall surface and from spirometry measurements in ten lung cancer patients. The average estimation of the 98th error percentile for the end inspiration phase was less than 1 mm (AP = 0.9 mm, RL = 0.6 mm, and SI = 0.8 mm). The vertices located at the lower region of the lung had a larger estimation error as compared with those within the upper region of the lung. The average landmark motion errors, derived from the biomechanical modeling using real surface deformation vector fields (SDVFs), and the predicted SDVFs were 3.0 and 3.1 mm, respectively. CONCLUSIONS Our newly developed predictive model provides a noninvasive approach to derive lung boundary conditions. The proposed system can be used with personalized biomechanical respiration modeling to derive lung tumor motion during radiation therapy from noninvasive measurements.
Collapse
Affiliation(s)
- Joubin Nasehi Tehrani
- Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, Texas 75235-8808
| | - Alistair McEwan
- School of Electrical and Information Engineering, University of Sydney, New South Wales 2006, Australia
| | - Jing Wang
- Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, Texas 75235-8808
| |
Collapse
|
19
|
Shieh CC, Keall PJ, Kuncic Z, Huang CY, Feain I. Markerless tumor tracking using short kilovoltage imaging arcs for lung image-guided radiotherapy. Phys Med Biol 2015; 60:9437-54. [PMID: 26583772 DOI: 10.1088/0031-9155/60/24/9437] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
The ability to monitor tumor motion without implanted markers is clinically advantageous for lung image-guided radiotherapy (IGRT). Existing markerless tracking methods often suffer from overlapping structures and low visibility of tumors on kV projection images. We introduce the short arc tumor tracking (SATT) method to overcome these issues. The proposed method utilizes multiple kV projection images selected from a nine-degree imaging arc to improve tumor localization, and respiratory-correlated 4D cone-beam CT (CBCT) prior knowledge to minimize the effects of overlapping anatomies. The 3D tumor position is solved as an optimization problem with prior knowledge incorporated via regularization. We retrospectively validated SATT on 11 clinical scans from four patients with central tumors. These patients represent challenging scenarios for markerless tumor tracking due to the inferior adjacent contrast. The 3D trajectories of implanted fiducial markers were used as the ground truth for tracking accuracy evaluation. In all cases, the tumors were successfully tracked at all gantry angles. Compared to standard pre-treatment CBCT guidance alone, trajectory errors were significantly smaller with tracking in all cases, and the improvements were the most prominent in the superior-inferior direction. The mean 3D tracking error ranged from 2.2-9.9 mm, which was 0.4-2.6 mm smaller compared to pre-treatment CBCT. In conclusion, we were able to directly track tumors with inferior visibility on kV projection images using SATT. Tumor localization accuracies are significantly better with tracking compared to the current standard of care of lung IGRT. Future work involves the prospective evaluation and clinical implementation of SATT.
Collapse
Affiliation(s)
- Chun-Chien Shieh
- Radiation Physics Laboratory, Sydney Medical School, The University of Sydney, NSW 2006, Australia. Institute of Medical Physics, School of Physics, The University of Sydney, NSW 2006, Australia
| | | | | | | | | |
Collapse
|