Feasibility of markerless 3D position monitoring of the central airways using kilovoltage projection images: Managing the risks of central lung stereotactic radiotherapy

Deep learning-based target decomposition for markerless lung tumor tracking in radiotherapy

BACKGROUND

In radiotherapy, real-time tumor tracking can verify tumor position during beam delivery, guide the radiation beam to target the tumor, and reduce the chance of a geometric miss. Markerless kV x-ray image-based tumor tracking is challenging due to the low tumor visibility caused by tumor-obscuring structures. Developing a new method to enhance tumor visibility for real-time tumor tracking is essential.

PURPOSE

To introduce a novel method for markerless kV image-based tracking of lung tumors via deep learning-based target decomposition.

METHODS

We utilized a conditional Generative Adversarial Network (cGAN), known as Pix2Pix, to build a patient-specific model and generate the synthetic decomposed target image (sDTI) to enhance tumor visibility on the real-time kV projection images acquired by the onboard kV imager equipped on modern linear accelerators. We used 4DCT simulation images to generate the digitally reconstructed radiograph (DRR) and DTI image pairs for model training. We augmented the training dataset by randomly shifting the 4DCT in the superior-inferior, anterior-posterior, and left-right directions during the DRR and DTI generation process. We performed real-time 2D tumor tracking via template matching between the DTI generated from the CT simulation and the sDTI generated from the real-time kV projection images. We validated the proposed method using nine patients' datasets with implanted beacons near the tumor.

RESULTS

The sDTI can effectively improve the image contrast around the lung tumors on the kV projection images for the nine patients. With the beacon motion as ground truth, the tracking errors were on average 0.8 ± 0.7 mm in the superior-inferior (SI) direction and 0.9 ± 0.8 mm in the in-plane left-right (IPLR) direction. The percentage of successful tracking, defined as a tracking error less than 2 mm in the SI direction, is 92.2% on the 4312 tested images. The patient-specific model took approximately 12 h to train. During testing, it took approximately 35 ms to generate one sDTI, and 13 ms to perform the tumor tracking using template matching.

CONCLUSIONS

Our method offers the potential solution for nearly real-time markerless lung tumor tracking. It achieved a high level of accuracy and an impressive tracking rate. Further development of 3D lung tumor tracking is warranted.

RT-SRTS: Angle-agnostic real-time simultaneous 3D reconstruction and tumor segmentation from single X-ray projection

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

Enhancing the target visibility with synthetic target specific digitally reconstructed radiograph for intrafraction motion monitoring: A proof-of-concept study

BACKGROUND

Intrafraction motion monitoring in External Beam Radiation Therapy (EBRT) is usually accomplished by establishing a correlation between the tumor and the surrogates such as an external infrared reflector, implanted fiducial markers, or patient skin surface. These techniques either have unstable surrogate-tumor correlation or are invasive. Markerless real-time onboard imaging is a noninvasive alternative that directly images the target motion. However, the low target visibility due to overlapping tissues along the X-ray projection path makes tumor tracking challenging.

PURPOSE

To enhance the target visibility in projection images, a patient-specific model was trained to synthesize the Target Specific Digitally Reconstructed Radiograph (TS-DRR).

METHODS

Patient-specific models were built using a conditional Generative Adversarial Network (cGAN) to map the onboard projection images to TS-DRR. The standard Pix2Pix network was adopted as our cGAN model. We synthesized the TS-DRR based on the onboard projection images using phantom and patient studies for spine tumors and lung tumors. Using previously acquired CT images, we generated DRR and its corresponding TS-DRR to train the network. For data augmentation, random translations were applied to the CT volume when generating the training images. For the spine, separate models were trained for an anthropomorphic phantom and a patient treated with paraspinal stereotactic body radiation therapy (SBRT). For lung, separate models were trained for a phantom with a spherical tumor insert and a patient treated with free-breathing SBRT. The models were tested using Intrafraction Review Images (IMR) for the spine and CBCT projection images for the lung. The performance of the models was validated using phantom studies with known couch shifts for the spine and known tumor deformation for the lung.

RESULTS

Both the patient and phantom studies showed that the proposed method can effectively enhance the target visibility of the projection images by mapping them into synthetic TS-DRR (sTS-DRR). For the spine phantom with known shifts of 1 mm, 2 mm, 3 mm, and 4 mm, the absolute mean errors for tumor tracking were 0.11 ± 0.05 mm in the x direction and 0.25 ± 0.08 mm in the y direction. For the lung phantom with known tumor motion of 1.8 mm, 5.8 mm, and 9 mm superiorly, the absolute mean errors for the registration between the sTS-DRR and ground truth are 0.1 ± 0.3 mm in both the x and y directions. Compared to the projection images, the sTS-DRR has increased the image correlation with the ground truth by around 83% and increased the structural similarity index measure with the ground truth by around 75% for the lung phantom.

CONCLUSIONS

The sTS-DRR can greatly enhance the target visibility in the onboard projection images for both the spine and lung tumors. The proposed method could be used to improve the markerless tumor tracking accuracy for EBRT.

The markerless lung target tracking AAPM Grand Challenge (MATCH) results

PURPOSE

Lung stereotactic ablative body radiotherapy (SABR) is a radiation therapy success story with level 1 evidence demonstrating its efficacy. To provide real-time respiratory motion management for lung SABR, several commercial and preclinical markerless lung target tracking (MLTT) approaches have been developed. However, these approaches have yet to be benchmarked using a common measurement methodology. This knowledge gap motivated the MArkerless lung target Tracking CHallenge (MATCH). The aim was to localize lung targets accurately and precisely in a retrospective in silico study and a prospective experimental study.

METHODS

MATCH was an American Association of Physicists in Medicine sponsored Grand Challenge. Common materials for the in silico and experimental studies were the experiment setup including an anthropomorphic thorax phantom with two targets within the lungs, and a lung SABR planning protocol. The phantom was moved rigidly with patient-measured lung target motion traces, which also acted as ground truth motion. In the retrospective in silico study a volumetric modulated arc therapy treatment was simulated and a dataset consisting of treatment planning data and intra-treatment kilovoltage (kV) and megavoltage (MV) images for four blinded lung motion traces was provided to the participants. The participants used their MLTT approach to localize the moving target based on the dataset. In the experimental study, the participants received the phantom experiment setup and five patient-measured lung motion traces. The participants used their MLTT approach to localize the moving target during an experimental SABR phantom treatment. The challenge was open to any participant, and participants could complete either one or both parts of the challenge. For both the in silico and experimental studies the MLTT results were analyzed and ranked using the prospectively defined metric of the percentage of the tracked target position being within 2 mm of the ground truth.

RESULTS

A total of 30 institutions registered and 15 result submissions were received, four for the in silico study and 11 for the experimental study. The participating MLTT approaches were: Accuray CyberKnife (2), Accuray Radixact (2), BrainLab Vero, C-RAD, and preclinical MLTT (5) on a conventional linear accelerator (Varian TrueBeam). For the in silico study the percentage of the 3D tracking error within 2 mm ranged from 50% to 92%. For the experimental study, the percentage of the 3D tracking error within 2 mm ranged from 39% to 96%.

CONCLUSIONS

A common methodology for measuring the accuracy of MLTT approaches has been developed and used to benchmark preclinical and commercial approaches retrospectively and prospectively. Several MLTT approaches were able to track the target with sub-millimeter accuracy and precision. The study outcome paves the way for broader clinical implementation of MLTT. MATCH is live, with datasets and analysis software being available online at https://www.aapm.org/GrandChallenge/MATCH/ to support future research.

Markerless 3D tumor tracking during single-fraction free-breathing 10MV flattening-filter-free stereotactic lung radiotherapy

BACKGROUND AND PURPOSE

Positional verification during single fraction lung SBRT could increase confidence and reduce the chance of geographic miss. As planar 2DkV imaging during VMAT irradiation is already available on current linear accelerators, markerless tracking based on these images could offer widely available and low-cost verification. We evaluated treatment delivery data and template matching and triangulation for 3D-positional verification during free-breathing, single fraction (34 Gy), 10 MV flattening-filter-free VMAT lung SBRT.

METHODS AND MATERIALS

Tumor tracking based on kV imaging at 7 frames/second was performed during irradiation in 6 consecutive patients (7 lesions). Tumor characteristics, tracking ability, comparison of tracking displacements with CBCT-based shifts, tumor position relative to the PTV margin, and treatment times are reported.

RESULTS

For all 7 lesions combined, 3D tumor position could be determined for, on average, 71% (51-84%) of the total irradiation time. Visually estimated tracked and automated match +/- manually-corrected CBCT-derived displacements generally agreed within 1 mm. During the tracked period, the longitudinal, lateral and vertical position of the tumor was within a 5 mm/3 mm PTV margin 95.5/85.3% of the time. The PTV was derived from the ITV including all tumor motion. The total time from first set-up imaging to end of the last arc was 18.3-31.4 min (mean = 23.4, SD = 4.1).

CONCLUSION

3D positional verification during irradiation of small lung targets with limited motion, was feasible. However, tumor position could not be determined for on average 29% of the time. Improvements are needed. Margin reduction may be feasible. Imaging and delivery of a single 34 Gy fraction was fast.

Markerless Real-Time 3-Dimensional kV Tracking of Lung Tumors During Free Breathing Stereotactic Radiation Therapy

Purpose

Accurate verification of tumor position during irradiation could reduce the probability of target miss. We investigated whether a commercial gantry-mounted 2-dimensional (2D) kilo-voltage (kV) imaging system could be used for real-time 3D tumor tracking during volumetric modulated arc therapy (VMAT) lung stereotactic body radiation therapy (SBRT). Markerless tumor tracking on kV fluoroscopic images was validated using a life-like moving thorax phantom and subsequently performed on kV images continuously acquired before and during free-breathing VMAT lung SBRT.

Methods and Materials

The 3D-printed/molded phantom containing 3 lung tumors was moved in 3D in TrueBeam developer mode, using simulated regular/irregular breathing patterns. Planar kV images were acquired at 7 frames/s during 11 Gy/fraction 10 MV flattening filter free VMAT. 2D reference templates were created for each gantry angle using the planning 4D computed tomography inspiration phase. kV images and templates were matched using normalized cross correlation to determine 2D tumor position, and triangulation of 2D matched projections determined the third dimension. 3D target tracking performed on cone beam computed tomography projection data from 18 patients (20 tumors) and real-time online tracking data from 2 of the 18 patients who underwent free-breathing VMAT lung SBRT are presented.

Results

For target 1 and 2 of the phantom (upper lung and middle/medial lung, mean density –130 Hounsfield units), 3D results within 2 mm of the known position were present in 92% and 96% of the kV projections, respectively. For target 3 (inferior lung, mean density –478 Hounsfield units) this dropped to 80%. Benchmarking against the respiratory signal, 13/20 (65%) tumors (10.5 ± 11.1 cm³) were considered successfully tracked on the cone beam computed tomography data. Tracking was less successful (≤50% of the time) in 7/20 (1.2 ± 1.5 cm³). Successful online tracking during lung SBRT was demonstrated.

Conclusions

3D markerless tumor tracking on a standard linear accelerator using template matching and triangulation of free-breathing kV fluoroscopic images was possible in 65% of small lung tumors. The smallest tumors were most challenging.

Technical Note: 3D localization of lung tumors on cone beam CT projections via a convolutional recurrent neural network

PURPOSE

To design a convolutional recurrent neural network (CRNN) that calculates three-dimensional (3D) positions of lung tumors from continuously acquired cone beam computed tomography (CBCT) projections, and facilitates the sorting and reconstruction of 4D-CBCT images.

METHOD

Under an IRB-approved clinical lung protocol, kilovoltage (kV) projections of the setup CBCT were collected in free-breathing. Concurrently, an electromagnetic signal-guided system recorded motion traces of three transponders implanted in or near the tumor. Convolutional recurrent neural network was designed to utilize a convolutional neural network (CNN) for extracting relevant features of the kV projections around the tumor, followed by a recurrent neural network for analyzing the temporal patterns of the moving features. Convolutional recurrent neural network was trained on the simultaneously collected kV projections and motion traces, subsequently utilized to calculate motion traces solely based on the continuous feed of kV projections. To enhance performance, CRNN was also facilitated by frequent calibrations (e.g., at 10° gantry rotation intervals) derived from cross-correlation-based registrations between kV projections and templates created from the planning 4DCT. Convolutional recurrent neural network was validated on a leave-one-out strategy using data from 11 lung patients, including 5500 kV images. The root-mean-square error between the CRNN and motion traces was calculated to evaluate the localization accuracy.

RESULT

Three-dimensional displacement around the simulation position shown in the Calypso traces was 3.4 ± 1.7 mm. Using motion traces as ground truth, the 3D localization error of CRNN with calibrations was 1.3 ± 1.4 mm. CRNN had a success rate of 86 ± 8% in determining whether the motion was within a 3D displacement window of 2 mm. The latency was 20 ms when CRNN ran on a high-performance computer cluster.

CONCLUSIONS

CRNN is able to provide accurate localization of lung tumors with aid from frequent recalibrations using the conventional cross-correlation-based registration approach, and has the potential to remove reliance on the implanted fiducials.

Real-time intrafraction motion monitoring in external beam radiotherapy

Radiotherapy (RT) aims to deliver a spatially conformal dose of radiation to tumours while maximizing the dose sparing to healthy tissues. However, the internal patient anatomy is constantly moving due to respiratory, cardiac, gastrointestinal and urinary activity. The long term goal of the RT community to 'see what we treat, as we treat' and to act on this information instantaneously has resulted in rapid technological innovation. Specialized treatment machines, such as robotic or gimbal-steered linear accelerators (linac) with in-room imaging suites, have been developed specifically for real-time treatment adaptation. Additional equipment, such as stereoscopic kilovoltage (kV) imaging, ultrasound transducers and electromagnetic transponders, has been developed for intrafraction motion monitoring on conventional linacs. Magnetic resonance imaging (MRI) has been integrated with cobalt treatment units and more recently with linacs. In addition to hardware innovation, software development has played a substantial role in the development of motion monitoring methods based on respiratory motion surrogates and planar kV or Megavoltage (MV) imaging that is available on standard equipped linacs. In this paper, we review and compare the different intrafraction motion monitoring methods proposed in the literature and demonstrated in real-time on clinical data as well as their possible future developments. We then discuss general considerations on validation and quality assurance for clinical implementation. Besides photon RT, particle therapy is increasingly used to treat moving targets. However, transferring motion monitoring technologies from linacs to particle beam lines presents substantial challenges. Lessons learned from the implementation of real-time intrafraction monitoring for photon RT will be used as a basis to discuss the implementation of these methods for particle RT.

Incorporating imaging information from deep neural network layers into image guided radiation therapy (IGRT)

BACKGROUND AND PURPOSE

To investigate a novel markerless prostate localization strategy using a pre-trained deep learning model to interpret routine projection kilovoltage (kV) X-ray images in image-guided radiation therapy (IGRT).

MATERIALS AND METHODS

We developed a personalized region-based convolutional neural network to localize the prostate treatment target without implanted fiducials. To train the deep neural network (DNN), we used the patient's planning computed tomography (pCT) images with pre-delineated prostate target to generate a large amount of synthetic kV projection X-ray images in the geometry of onboard imager (OBI) system. The DNN model was evaluated by retrospectively studying 10 patients who underwent prostate IGRT. Three out of the ten patients who had implanted fiducials and the fiducials' positions in the OBI images acquired for treatment setup were examined to show the potential of the proposed method for prostate IGRT. Statistical analysis using Lin's concordance correlation coefficient was calculated to assess the results along with the difference between the digitally reconstructed radiographs (DRR) derived and DNN predicted locations of the prostate.

RESULTS

Differences between the predicted target positions using DNN and their actual positions are (mean ± standard deviation) 1.58 ± 0.43 mm, 1.64 ± 0.43 mm, and 1.67 ± 0.36 mm in anterior-posterior, lateral, and oblique directions, respectively. Prostate position identified on the OBI kV images is also found to be consistent with that derived from the implanted fiducials.

CONCLUSIONS

Highly accurate, markerless prostate localization based on deep learning is achievable. The proposed method is useful for daily patient positioning and real-time target tracking during prostate radiotherapy.