Markerless Real-Time 3-Dimensional kV Tracking of Lung Tumors During Free Breathing Stereotactic Radiation Therapy

Deep learning-based target decomposition for markerless lung tumor tracking in radiotherapy

BACKGROUND

In radiotherapy, real-time tumor tracking can verify tumor position during beam delivery, guide the radiation beam to target the tumor, and reduce the chance of a geometric miss. Markerless kV x-ray image-based tumor tracking is challenging due to the low tumor visibility caused by tumor-obscuring structures. Developing a new method to enhance tumor visibility for real-time tumor tracking is essential.

PURPOSE

To introduce a novel method for markerless kV image-based tracking of lung tumors via deep learning-based target decomposition.

METHODS

We utilized a conditional Generative Adversarial Network (cGAN), known as Pix2Pix, to build a patient-specific model and generate the synthetic decomposed target image (sDTI) to enhance tumor visibility on the real-time kV projection images acquired by the onboard kV imager equipped on modern linear accelerators. We used 4DCT simulation images to generate the digitally reconstructed radiograph (DRR) and DTI image pairs for model training. We augmented the training dataset by randomly shifting the 4DCT in the superior-inferior, anterior-posterior, and left-right directions during the DRR and DTI generation process. We performed real-time 2D tumor tracking via template matching between the DTI generated from the CT simulation and the sDTI generated from the real-time kV projection images. We validated the proposed method using nine patients' datasets with implanted beacons near the tumor.

RESULTS

The sDTI can effectively improve the image contrast around the lung tumors on the kV projection images for the nine patients. With the beacon motion as ground truth, the tracking errors were on average 0.8 ± 0.7 mm in the superior-inferior (SI) direction and 0.9 ± 0.8 mm in the in-plane left-right (IPLR) direction. The percentage of successful tracking, defined as a tracking error less than 2 mm in the SI direction, is 92.2% on the 4312 tested images. The patient-specific model took approximately 12 h to train. During testing, it took approximately 35 ms to generate one sDTI, and 13 ms to perform the tumor tracking using template matching.

CONCLUSIONS

Our method offers the potential solution for nearly real-time markerless lung tumor tracking. It achieved a high level of accuracy and an impressive tracking rate. Further development of 3D lung tumor tracking is warranted.

RT-SRTS: Angle-agnostic real-time simultaneous 3D reconstruction and tumor segmentation from single X-ray projection

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

Simultaneous object detection and segmentation for patient-specific markerless lung tumor tracking in simulated radiographs with deep learning

BACKGROUND

Real-time tumor tracking is one motion management method to address motion-induced uncertainty. To date, fiducial markers are often required to reliably track lung tumors with X-ray imaging, which carries risks of complications and leads to prolonged treatment time. A markerless tracking approach is thus desirable. Deep learning-based approaches have shown promise for markerless tracking, but systematic evaluation and procedures to investigate applicability in individual cases are missing. Moreover, few efforts have been made to provide bounding box prediction and mask segmentation simultaneously, which could allow either rigid or deformable multi-leaf collimator tracking.

PURPOSE

The purpose of this study was to implement a deep learning-based markerless lung tumor tracking model exploiting patient-specific training which outputs both a bounding box and a mask segmentation simultaneously. We also aimed to compare the two kinds of predictions and to implement a specific procedure to understand the feasibility of markerless tracking on individual cases.

METHODS

We first trained a Retina U-Net baseline model on digitally reconstructed radiographs (DRRs) generated from a public dataset containing 875 CT scans and corresponding lung nodule annotations. Afterwards, we used an independent cohort of 97 lung patients to develop a patient-specific refinement procedure. In order to determine the optimal hyperparameters for automatic patient-specific training, we selected 13 patients for validation where the baseline model predicted a bounding box on planning CT (PCT)-DRR with intersection over union (IoU) with the ground-truth higher than 0.7. The final test set contained the remaining 84 patients with varying PCT-DRR IoU. For each testing patient, the baseline model was refined on the PCT-DRR to generate a patient-specific model, which was then tested on a separate 10-phase 4DCT-DRR to mimic the intrafraction motion during treatment. A template matching algorithm served as benchmark model. The testing results were evaluated by four metrics: the center of mass (COM) error and the Dice similarity coefficient (DSC) for segmentation masks, and the center of box (COB) error and the DSC for bounding box detections. Performance was compared to the benchmark model including statistical testing for significance.

RESULTS

A PCT-DRR IoU value of 0.2 was shown to be the threshold dividing inconsistent (68%) and consistent (100%) success (defined as mean bounding box DSC > 0.6) of PS models on 4DCT-DRRs. Thirty-seven out of the eighty-four testing cases had a PCT-DRR IoU above 0.2. For these 37 cases, the mean COM error was 2.6 mm, the mean segmentation DSC was 0.78, the mean COB error was 2.7 mm, and the mean box DSC was 0.83. Including the validation cases, the model was applicable to 50 out of 97 patients when using the PCT-DRR IoU threshold of 0.2. The inference time per frame was 170 ms. The model outperformed the benchmark model on all metrics, and the comparison was significant (p < 0.001) over the 37 PCT-DRR IoU > 0.2 cases, but not over the undifferentiated 84 testing cases.

CONCLUSIONS

The implemented patient-specific refinement approach based on a pre-trained baseline model was shown to be applicable to markerless tumor tracking in simulated radiographs for lung cases.

Deep learning-based markerless lung tumor tracking in stereotactic radiotherapy using Siamese networks

BACKGROUND

Radiotherapy (RT) is involved in about 50% of all cancer patients, making it a very important treatment modality. The most common type of RT is external beam RT, which consists of delivering the radiation to the tumor from outside the body. One novel treatment delivery method is volumetric modulated arc therapy (VMAT), where the gantry continuously rotates around the patient during the radiation delivery.

PURPOSE

Accurate tumor position monitoring during stereotactic body radiotherapy (SBRT) for lung tumors can help to ensure that the tumor is only irradiated when it is inside the planning target volume. This can maximize tumor control and reduce uncertainty margins, lowering organ-at-risk dose. Conventional tracking methods are prone to errors, or have a low tracking rate, especially for small tumors that are in close vicinity to bony structures.

METHODS

We investigated patient-specific deep Siamese networks for real-time tumor tracking, during VMAT. Due to lack of ground truth tumor locations in the kilovoltage (kV) images, each patient-specific model was trained on synthetic data (DRRs), generated from the 4D planning CT scans, and evaluated on clinical data (x-rays). Since there are no annotated datasets with kV images, we evaluated the model on a 3D printed anthropomorphic phantom but also on six patients by computing the correlation coefficient with the breathing-related vertical displacement of the surface-mounted marker (RPM). For each patient/phantom, we used 80% of DRRs for training and 20% for validation.

RESULTS

The proposed Siamese model outperformed the conventional benchmark template matching-based method (RTR): (1) when evaluating both methods on the 3D phantom, the Siamese model obtained a 0.57-0.79-mm mean absolute distance to the ground truth tumor locations, compared to 1.04-1.56 mm obtained by RTR; (2) on patient data, the Siamese-determined longitudinal tumor position had a correlation coefficient of 0.71-0.98 with the RPM, compared to 0.07-0.85 for RTR; (3) the Siamese model had a 100% tracking rate, compared to 62%-82% for RTR.

CONCLUSIONS

Based on these results, we argue that Siamese-based real-time 2D markerless tumor tracking during radiation delivery is possible. Further investigation and development of 3D tracking is warranted.

Markerless motion tracking with simultaneous MV and kV imaging in spine SBRT treatment-a feasibility study

Objective. Motion tracking with simultaneous MV-kV imaging has distinct advantages over single kV systems. This research is a feasibility study of utilizing this technique for spine stereotactic body radiotherapy (SBRT) through phantom and patient studies.Approach. A clinical spine SBRT plan was developed using 6xFFF beams and nine sliding-window IMRT fields. The plan was delivered to a chest phantom on a linear accelerator. Simultaneous MV-kV image pairs were acquired during beam delivery. KV images were triggered at predefined intervals, and synthetic MV images showing enlarged MLC apertures were created by combining multiple raw MV frames with corrections for scattering and intensity variation. Digitally reconstructed radiograph (DRR) templates were generated using high-resolution CBCT reconstructions (isotropic voxel size (0.243 mm)3) as the reference for 2D-2D matching. 3D shifts were calculated from triangulation of kV-to-DRR and MV-to-DRR registrations. To evaluate tracking accuracy, detected shifts were compared to known phantom shifts as introduced before treatment. The patient study included a T-spine patient and an L-spine patient. Patient datasets were retrospectively analyzed to demonstrate the performance in clinical settings.Main results. The treatment plan was delivered to the phantom in five scenarios: no shift, 2 mm shift in one of the longitudinal, lateral and vertical directions, and 2 mm shift in all the three directions. The calculated 3D shifts agreed well with the actual couch shifts, and overall, the uncertainty of 3D detection is estimated to be 0.3 mm. The patient study revealed that with clinical patient image quality, the calculated 3D motion agreed with the post-treatment cone beam CT. It is feasible to automate both kV-to-DRR and MV-to-DRR registrations using a mutual information-based method, and the difference from manual registration is generally less than 0.3 mm.Significance. The MV-kV imaging-based markerless motion tracking technique was validated through a feasibility study. It is a step forward toward effective motion tracking and accurate delivery for spinal SBRT.

Explainability and controllability of patient-specific deep learning with attention-based augmentation for markerless image-guided radiotherapy

BACKGROUND

We reported the concept of patient-specific deep learning (DL) for real-time markerless tumor segmentation in image-guided radiotherapy (IGRT). The method was aimed to control the attention of convolutional neural networks (CNNs) by artificial differences in co-occurrence probability (CoOCP) in training datasets, that is, focusing CNN attention on soft tissues while ignoring bones. However, the effectiveness of this attention-based data augmentation has not been confirmed by explainable techniques. Furthermore, compared to reasonable ground truths, the feasibility of tumor segmentation in clinical kilovolt (kV) X-ray fluoroscopic (XF) images has not been confirmed.

PURPOSE

The first aim of this paper was to present evidence that the proposed method provides an explanation and control of DL behavior. The second purpose was to validate the real-time lung tumor segmentation in clinical kV XF images for IGRT.

METHODS

This retrospective study included 10 patients with lung cancer. Patient-specific and XF angle-specific image pairs comprising digitally reconstructed radiographs (DRRs) and projected-clinical-target-volume (pCTV) images were calculated from four-dimensional computer tomographic data and treatment planning information. The training datasets were primarily augmented by random overlay (RO) and noise injection (NI): RO aims to differentiate positional CoOCP in soft tissues and bones, and NI aims to make a difference in the frequency of occurrence of local and global image features. The CNNs for each patient-and-angle were automatically optimized in the DL training stage to transform the training DRRs into pCTV images. In the inference stage, the trained CNNs transformed the test XF images into pCTV images, thus identifying target positions and shapes.

RESULTS

The visual analysis of DL attention heatmaps for a test image demonstrated that our method focused CNN attention on soft tissue and global image features rather than bones and local features. The processing time for each patient-and-angle-specific dataset in the training stage was ∼30 min, whereas that in the inference stage was 8 ms/frame. The estimated three-dimensional 95 percentile tracking error, Jaccard index, and Hausdorff distance for 10 patients were 1.3-3.9 mm, 0.85-0.94, and 0.6-4.9 mm, respectively.

CONCLUSIONS

The proposed attention-based data augmentation with both RO and NI made the CNN behavior more explainable and more controllable. The results obtained demonstrated the feasibility of real-time markerless lung tumor segmentation in kV XF images for IGRT.

Feasibility of 4D VMAT-CT

Objective.Feasibility of three-dimensional (3D) tracking of volumetric modulated arc therapy (VMAT) based on VMAT-computed tomography (VMAT-CT) has been shown previously by our group. However, 3D VMAT-CT is not suitable for treatments that involve significant target movement due to patient breathing. The goal of this study was to reconstruct four-dimensional (4D) VMAT-CT and evaluate the feasibility of tracking based on 4D VMAT-CT.Approach.Synchronized portal images of phantoms and linac log were both sorted into four phases, and VMAT-CT+ was generated in each phase by fusing reconstructed VMAT-CT and planning CT using rigid or deformable registration. Dose was calculated in each phase and was registered to the mean position planning CT for 4D dose reconstruction. Trackings based on 4D VMAT-CT+ and 4D cone beam CT (CBCT) were compared. Potential uncertainties were also evaluated.Main results.Tracking based on 4D VMAT-CT+ was accurate, could detect phantom deformation and/or change of breathing pattern, and was superior to that based on 4D CBCT. The impact of uncertainties on tracking was minimal.Significance.Our study shows it is feasible to accurately track position and dose based on 4D VMAT-CT for patients whose VMAT treatments are subject to respiratory motion. It will significantly increase the confidence of VMAT and is a clinically viable solution to daily patient positioning,in vivodosimetry and treatment monitoring.

Improved Tumor Image Estimation in X-Ray Fluoroscopic Images by Augmenting 4DCT Data for Radiotherapy

Measurement of tumor position is important for the radiotherapy of lung tumors with respiratory motion. Although tumors can be observed using X-ray fluoroscopy during radiotherapy, it is often difficult to measure tumor position from X-ray image sequences accurately because of overlapping organs. To measure tumor position accurately, a method for extracting tumor intensities from X-ray image sequences using a hidden Markov model (HMM) has been proposed. However, the performance of tumor intensity extraction depends on limited knowledge regarding the tumor motion observed in the four-dimensional computed tomography (4DCT) data used to construct the HMM. In this study, we attempted to improve the performance of tumor intensity extraction by augmenting 4DCT data. The proposed method was tested using simulated datasets of X-ray image sequences. The experimental results indicated that the HMM using the augmentation method could improve tumor-tracking performance when the range of tumor movement during treatment differed from that in the 4DCT data.

MArkerless image Guidance using Intrafraction Kilovoltage x-ray imaging (MAGIK): study protocol for a phase I interventional study for lung cancer radiotherapy

INTRODUCTION

In radiotherapy, tumour tracking leads the radiation beam to accurately target the tumour while it moves in a complex and unpredictable way due to respiration. Several tumour tracking techniques require the implantation of fiducial markers around the tumour, a procedure that involves unnecessary risks and costs. Markerless tumour tracking (MTT) negates the need for implanted markers, potentially enabling accurate and optimal radiotherapy in a non-invasive way.

METHODS AND ANALYSIS

We will perform a phase I interventional trial called MArkerless image Guidance using Intrafraction Kilovoltage x-ray imaging (MAGIK) to investigate the technical feasibility of the MTT technology developed at the University of Sydney (sponsor). 30 participants will undergo the current standard of care lung stereotactic ablative radiation therapy, with the exception that kilovoltage X-ray images will be acquired continuously during treatment delivery to enable MTT. If MTT indicates that the mean lung tumour position has shifted >3 mm, a warning message will be displayed to indicate the need for a treatment intervention. The radiation therapist will then pause the treatment, shift the treatment couch to account for the shift in tumour position and resume the treatment. Participants will be implanted with fiducial markers, which act as the ground truth for evaluating the accuracy of MTT. MTT is considered feasible if the tracking accuracy is <3 mm in each dimension for >80% of the treatment time.

ETHICS AND DISSEMINATION

The MAGIK trial has received ethical approval from The Alfred Human Research Ethics Committee and has been registered with ClinicalTrials.gov with the Identifier: NCT04086082. Estimated time of first recruitment is early 2022. The study recruitment and data analysis phases will be performed concurrently. Treatment for all 30 participants is expected to be completed within 2 years and participant follow-up within a total duration of 7 years. Findings will be disseminated through peer-reviewed publications and conference presentations.

TRIAL REGISTRATION NUMBER

NCT04086082; Pre-result.

Risks and Benefits of Fiducial Marker Placement in Tumor Lesions for Robotic Radiosurgery: Technical Outcomes of 357 Implantations

Simple Summary

Robotic radiosurgery (RRS) allows for the accurate treatment of primary tumors or metastases with high single doses. However, organ motion during or between fractions can lead to imprecise irradiation. We sought to evaluate the risks and advantages of fiducial marker (FM) implantation regarding clinical complications, marker migration, and motion amplitude. Complications were most common in Synchrony^®-tracked lesions affected by respiratory motion, particularly lung lesions. Pneumothoraces and pulmonary bleeding were the most common complications. An increased complication rate was associated with concomitant biopsy sampling and FM implantation. Most FM migration observed in this study occurred after CT-guided placements and clinical FM insertions. The largest motion amplitudes were observed in hepatic and lower lung lobe lesions. This study highlights the benefits of marker implantation, especially in lesions with a large motion amplitude, including hepatic lesions and lesions of the lower lobe of the lung located >100.0 mm from the spine.

Abstract

Fiducial markers (FM) inserted into tumors increase the precision of irradiation during robotic radiosurgery (RRS). This retrospective study evaluated the clinical complications, marker migration, and motion amplitude of FM implantations by analyzing 288 cancer patients (58% men; 63.1 ± 13.0 years) who underwent 357 FM implantations prior to RRS with CyberKnife, between 2011 and 2019. Complications were classified according to the Society of Interventional Radiology (SIR) guidelines. The radial motion amplitude was calculated for tumors that moved with respiration. A total of 725 gold FM was inserted. SIR-rated complications occurred in 17.9% of all procedures. Most complications (32.0%, 62/194 implantations) were observed in Synchrony^®-tracked lesions affected by respiratory motion, particularly in pulmonary lesions (46.9% 52/111 implantations). Concurrent biopsy sampling was associated with a higher complication rate (p = 0.001). FM migration occurred in 3.6% after CT-guided and clinical FM implantations. The largest motion amplitudes were observed in hepatic (20.5 ± 11.0 mm) and lower lung lobe (15.4 ± 10.5 mm) lesions. This study increases the awareness of the risks of FM placement, especially in thoracic lesions affected by respiratory motion. Considering the maximum motion amplitude, FM placement remains essential in hepatic and lower lung lobe lesions located >100.0 mm from the spine.

Markerless 3D tumor tracking during single-fraction free-breathing 10MV flattening-filter-free stereotactic lung radiotherapy

BACKGROUND AND PURPOSE

Positional verification during single fraction lung SBRT could increase confidence and reduce the chance of geographic miss. As planar 2DkV imaging during VMAT irradiation is already available on current linear accelerators, markerless tracking based on these images could offer widely available and low-cost verification. We evaluated treatment delivery data and template matching and triangulation for 3D-positional verification during free-breathing, single fraction (34 Gy), 10 MV flattening-filter-free VMAT lung SBRT.

METHODS AND MATERIALS

Tumor tracking based on kV imaging at 7 frames/second was performed during irradiation in 6 consecutive patients (7 lesions). Tumor characteristics, tracking ability, comparison of tracking displacements with CBCT-based shifts, tumor position relative to the PTV margin, and treatment times are reported.

RESULTS

For all 7 lesions combined, 3D tumor position could be determined for, on average, 71% (51-84%) of the total irradiation time. Visually estimated tracked and automated match +/- manually-corrected CBCT-derived displacements generally agreed within 1 mm. During the tracked period, the longitudinal, lateral and vertical position of the tumor was within a 5 mm/3 mm PTV margin 95.5/85.3% of the time. The PTV was derived from the ITV including all tumor motion. The total time from first set-up imaging to end of the last arc was 18.3-31.4 min (mean = 23.4, SD = 4.1).

CONCLUSION

3D positional verification during irradiation of small lung targets with limited motion, was feasible. However, tumor position could not be determined for on average 29% of the time. Improvements are needed. Margin reduction may be feasible. Imaging and delivery of a single 34 Gy fraction was fast.