Reducing the tracking drift of an uncontoured tumor for a portal-image-based dynamically adapted conformal radiotherapy treatment

Landmark tracking in liver US images using cascade convolutional neural networks with long short-term memory

Accurate tracking of anatomic landmarks is critical for motion management in liver radiation therapy. Ultrasound (US) is a safe, low-cost technology that is broadly available and offer real-time imaging capability. This study proposed a deep learning-based tracking method for the US image-guided radiation therapy. The proposed cascade deep learning model is composed of an attention network, a mask region-based convolutional neural network (mask R-CNN), and a long short-term memory (LSTM) network. The attention network learns a mapping from an US image to a suspected area of landmark motion in order to reduce the search region. The mask R-CNN then produces multiple region-of-interest proposals in the reduced region and identifies the proposed landmark via three network heads: bounding box regression, proposal classification, and landmark segmentation. The LSTM network models the temporal relationship among the successive image frames for bounding box regression and proposal classification. To consolidate the final proposal, a selection method is designed according to the similarities between sequential frames. The proposed method was tested on the liver US tracking datasets used in the medical image computing and computer assisted interventions 2015 challenges, where the landmarks were annotated by three experienced observers to obtain their mean positions. Five-fold cross validation on the 24 given US sequences with ground truths shows that the mean tracking error for all landmarks is 0.65 ± 0.56 mm, and the errors of all landmarks are within 2 mm. We further tested the proposed model on 69 landmarks from the testing dataset that have the similar image pattern with the training pattern, resulting in a mean tracking error of 0.94 ± 0.83 mm. The proposed deep-learning model was implemented on a graphics processing unit (GPU), tracking 47-81 frames s-1. Our experimental results have demonstrated the feasibility and accuracy of our proposed method in tracking liver anatomic landmarks using US images, providing a potential solution for real-time liver tracking for active motion management during radiation therapy.

Development of AI-driven prediction models to realize real-time tumor tracking during radiotherapy

BACKGROUND

In infrared reflective (IR) marker-based hybrid real-time tumor tracking (RTTT), the internal target position is predicted with the positions of IR markers attached on the patient's body surface using a prediction model. In this work, we developed two artificial intelligence (AI)-driven prediction models to improve RTTT radiotherapy, namely, a convolutional neural network (CNN) and an adaptive neuro-fuzzy inference system (ANFIS) model. The models aim to improve the accuracy in predicting three-dimensional tumor motion.

METHODS

From patients whose respiration-induced motion of the tumor, indicated by the fiducial markers, exceeded 8 mm, 1079 logfiles of IR marker-based hybrid RTTT (IR Tracking) with the gimbal-head radiotherapy system were acquired and randomly divided into two datasets. All the included patients were breathing freely with more than four external IR markers. The historical dataset for the CNN model contained 1003 logfiles, while the remaining 76 logfiles complemented the evaluation dataset. The logfiles recorded the external IR marker positions at a frequency of 60 Hz and fiducial markers as surrogates for the detected target positions every 80-640 ms for 20-40 s. For each logfile in the evaluation dataset, the prediction models were trained based on the data in the first three quarters of the recording period. In the last quarter, the performance of the patient-specific prediction models was tested and evaluated. The overall performance of the AI-driven prediction models was ranked by the percentage of predicted target position within 2 mm of the detected target position. Moreover, the performance of the AI-driven models was compared to a regression prediction model currently implemented in gimbal-head radiotherapy systems.

RESULTS

The percentage of the predicted target position within 2 mm of the detected target position was 95.1%, 92.6% and 85.6% for the CNN, ANFIS, and regression model, respectively. In the evaluation dataset, the CNN, ANFIS, and regression model performed best in 43, 28 and 5 logfiles, respectively.

CONCLUSIONS

The proposed AI-driven prediction models outperformed the regression prediction model, and the overall performance of the CNN model was slightly better than that of the ANFIS model on the evaluation dataset.

Fusion Siamese network with drift correction for target tracking in ultrasound sequences

Motion tracking techniques can revise the bias arising from respiration-caused motion in radiation therapy. Tracking key structures accurately and at a real-time speed is necessary for effective motion tracking. In this work, we propose a fusion Siamese network with drift correction for target tracking in ultrasound sequences. Specifically, the network fuses four response maps generated by the cross-correlation between convolution layers at different resolutions to reduce up-sampling error. A correction strategy combining local structural similarity and target trajectory is proposed to revise the target drift predicted by the network. Moreover, a coarse-to-fine strategy is proposed to train the network with a limited number of annotated images, in which an augmented dataset is generated by corner points to learn network features with high generalizability. The proposed method is evaluated on the basis of the public dataset of the MICCAI 2015 Challenge on Liver UltraSound Tracking (CLUST) and our ultrasound image dataset, which is provided by the Chinese People’s Liberation Army General Hospital (CPLAGH). A tracking error of 0.80 ± 1.16 mm is observed for 85 targets across 39 ultrasound sequences in the CLUST dataset. A tracking error of 0.61 ± 0.36 mm is observed for 20 targets across 10 ultrasound sequences in the CPLAGH dataset. The effectiveness of the proposed fusion and correction strategies is verified via two ablation experiments. Overall, the experimental results demonstrate the effectiveness of the proposed fusion Siamese network with drift correction and reveal its potential in clinical practice.

A feasibility study on the development and use of a deep learning model to automate real-time monitoring of tumor position and assessment of interfraction fiducial marker migration in prostate radiotherapy patients

PurposeThe aim of this study was to assess the feasibility of the development and training of a deep learning object detection model for automating the assessment of fiducial marker migration and tracking of the prostate in radiotherapy patients.Methods and MaterialsA fiducial marker detection model was trained on the YOLO v2 detection framework using approximately 20,000 pelvis kV projection images with fiducial markers labelled. The ability of the trained model to detect marker positions was validated by tracking the motion of markers in a respiratory phantom and comparing detection data with the expected displacement from a reference position. Marker migration was then assessed in 14 prostate radiotherapy patients using the detector for comparison with previously conducted studies. This was done by determining variations in intermarker distance between the first and subsequent fractions in each patient.ResultsOn completion of training, a detection model was developed that operated at a 96% detection efficacy and with a root mean square error of 0.3 pixels. By determining the displacement from a reference position in a respiratory phantom, experimentally and with the detector it was found that the detector was able to compute displacements with a mean accuracy of 97.8% when compared to the actual values. Interfraction marker migration was measured in 14 patients and the average and maximum ± standard deviation marker migration were found to be 2.0±0.9 mm and 2.3±0.9 mm, respectively.ConclusionThis study demonstrates the benefits of pairing deep learning object detection, and image-guided radiotherapy and how a workflow to automate the assessment of organ motion and seed migration during prostate radiotherapy can be developed. The high detection efficacy and low error make the advantages of using a pre-trained model to automate the assessment of the target volume positional variation and the migration of fiducial markers between fractions.

Dense feature-based motion estimation in MV fluoroscopy during dynamic tumor tracking treatment: preliminary study on reduced aperture and partial occlusion handling

Quality assurance solutions to complement available motion compensation technologies are central for their safe routine implementation and success of treatment. This work presents a dense feature-based method for soft-tissue tumor motion estimation in megavoltage (MV) beam's-eye-view (BEV) projections for potential intra-treatment monitoring during dynamic tumor tracking (DTT). Dense sampling and matching principles were employed to track a gridded set of features landmarks (FLs) in MV-BEV projections and estimate tumor motion, capable to overcome reduced field aperture and partial occlusion challenges. The algorithm's performance was evaluated by retrospectively applying it to fluoroscopic sequences acquired at ∼2 frames s^-1 (fps) for a dynamic phantom and two lung stereotactic body radiation therapy (SBRT) patients treated with DTT on the Vero SBRT system. First, a field-specific train image is initialized by sampling the tumor region at, S, pixel intervals on a grid using a representative frame from a stream of query frames. Sampled FLs are locally characterized in the form of descriptor vectors and geometric attributes representing the target. For motion tracking, subsequent query frames are likewise sampled, corresponding feature descriptors determined, and then patch-wise matched to the training set based on their descriptors and geometric relationships. FLs with high correspondence are pruned and used to estimate tumor displacement. In scenarios of partial occlusions, position is estimated from the set of correctly (visible) FLs on past observations. Reconstructed trajectories were benchmarked against ground-truth manual tracking using the root-mean-square (RMS) as a metric of positional accuracy. A total of 19 fluoroscopy sequences were analyzed. This included scenarios of field aperture obstruction during three-dimensional conformal, as well as step-and-shoot intensity modulated radiotherapy (IMRT) delivery assisted with DTT. The algorithm resolved target motion satisfactorily. The RMS was <1.2 mm and <1.8 mm for the phantom and the clinical dataset, respectively. Dense tracking showed promising results to overcome localization challenges at the field penumbra and partial obstruction by multi-leaf collimator (MLC). Motion retrieval was possible in ∼66% of the control points studied. In addition to MLC obstruction, changes in the external/internal breathing dynamics and baseline drifts were a major source of estimation bias. Dense feature-based tracking is a viable alternative. The algorithm is rotation-/scale-invariant and robust to photometric changes. Tracking multiple features may help overcome partial occlusion challenges by the MLC. This in turn opens up new possibilities for motion detection and intra-treatment monitoring during IMRT and potentially VMAT.

Evaluating a potential technique with local optical flow vectors for automatic organ-at-risk (OAR) intrusion detection and avoidance during radiotherapy

Various techniques of deep inspiration breath hold (DIBH) have been used to mitigate the likelihood and risk of exposing the heart, an organ-at-risk (OAR) for unintended radiation during left breast radiotherapy. However, issues of reproducibility of these techniques warrant further investigation into the feasibility of detecting the intrusion of an OAR into the treatment field during intra-fractional treatment delivery. The increase of high-dose, low-fraction radiotherapy treatments makes it important to immediately adapt treatment once an OAR is detected in the treatment field. This proof-of-concept implementation includes an algorithm that detects and tracks the motion at the edges of a treatment field and a control algorithm that adapts the treatment aperture according to the motion detected. In accordance to the AAPM Task-Group (TG-132) report, image registration techniques should be verified with virtual and physical phantoms prior to clinical application. Since most OARs move as a result of respiration-induced motion, we have used a lung phantom to generate images of a generic OAR intruding into a treatment field with known velocity. The phantom was programmed to move with sinusoidal and lung patient tumor motion patterns and the accuracy of intrusion tracking and MLC adaptation were benchmarked with the ground truth-programmed motion of the OAR. The motions were recorded with an electronic portal imaging device (EPID). An optimal cluster size of 9  ×  9 motion vectors was found to provide the smallest average absolute position error of 0.3 mm. A strong linear correlation between the adapted MLC leaves and the actual OAR position was observed. The algorithm had a mean position tracking error of  -0.4  ±  0.3 mm and a precision of 1.1 mm. It is possible to adapt MLC leaves based on the motion detected at the edges of the irradiated field, and it would be feasible to shield an unplanned intrusion of an OAR into the treatment field.