Patient specific deep learning based segmentation for magnetic resonance guided prostate radiotherapy

Incorporating patient-specific information for the development of rectal tumor auto-segmentation models for online adaptive magnetic resonance Image-guided radiotherapy

Background and purpose

In online adaptive magnetic resonance image (MRI)-guided radiotherapy (MRIgRT), manual contouring of rectal tumors on daily images is labor-intensive and time-consuming. Automation of this task is complex due to substantial variation in tumor shape and location between patients. The aim of this work was to investigate different approaches of propagating patient-specific prior information to the online adaptive treatment fractions to improve deep-learning based auto-segmentation of rectal tumors.

Materials and methods

243 T2-weighted MRI scans of 49 rectal cancer patients treated on the 1.5T MR-Linear accelerator (MR-Linac) were utilized to train models to segment rectal tumors. As benchmark, an MRI_only auto-segmentation model was trained. Three approaches of including a patient-specific prior were studied: 1. include the segmentations of fraction 1 as extra input channel for the auto-segmentation of subsequent fractions, 2. fine-tuning of the MRI_only model to fraction 1 (PSF_1) and 3. fine-tuning of the MRI_only model on all earlier fractions (PSF_cumulative). Auto-segmentations were compared to the manual segmentation using geometric similarity metrics. Clinical impact was assessed by evaluating post-treatment target coverage.

Results

All patient-specific methods outperformed the MRI_only segmentation approach. Median 95th percentile Hausdorff (95HD) were 22.0 (range: 6.1-76.6) mm for MRI_only segmentation, 9.9 (range: 2.5-38.2) mm for MRI+prior segmentation, 6.4 (range: 2.4-17.8) mm for PSF_1 and 4.8 (range: 1.7-26.9) mm for PSF_cumulative. PSF_cumulative was found to be superior to PSF_1 from fraction 4 onward (p = 0.014).

Conclusion

Patient-specific fine-tuning of automatically segmented rectal tumors, using images and segmentations from all previous fractions, yields superior quality compared to other auto-segmentation approaches.

Artificial intelligence for treatment delivery: image-guided radiotherapy

Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.

A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy

BACKGROUND AND PURPOSE

Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap.

METHODS AND MATERIALS

A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended.

RESULTS

The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated.

CONCLUSION

A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

Automatic AI-based contouring of prostate MRI for online adaptive radiotherapy

BACKGROUND AND PURPOSE

MR-guided radiotherapy (MRgRT) online plan adaptation accounts for tumor volume changes, interfraction motion and thus allows daily sparing of relevant organs at risk. Due to the high interfraction variability of bladder and rectum, patients with tumors in the pelvic region may strongly benefit from adaptive MRgRT. Currently, fast automatic annotation of anatomical structures is not available within the online MRgRT workflow. Therefore, the aim of this study was to train and validate a fast, accurate deep learning model for automatic MRI segmentation at the MR-Linac for future implementation in a clinical MRgRT workflow.

MATERIALS AND METHODS

For a total of 47 patients, T2w MRI data were acquired on a 1.5 T MR-Linac (Unity, Elekta) on five different days. Prostate, seminal vesicles, rectum, anal canal, bladder, penile bulb, body and bony structures were manually annotated. These training data consisting of 232 data sets in total was used for the generation of a deep learning based autocontouring model and validated on 20 unseen T2w-MRIs. For quantitative evaluation the validation set was contoured by a radiation oncologist as gold standard contours (GSC) and compared in MATLAB to the automatic contours (AIC). For the evaluation, dice similarity coefficients (DSC), and 95% Hausdorff distances (95% HD), added path length (APL) and surface DSC (sDSC) were calculated in a caudal-cranial window of ± 4 cm with respect to the prostate ends. For qualitative evaluation, five radiation oncologists scored the AIC on the possible usage within an online adaptive workflow as follows: (1) no modifications needed, (2) minor adjustments needed, (3) major adjustments/ multiple minor adjustments needed, (4) not usable.

RESULTS

The quantitative evaluation revealed a maximum median 95% HD of 6.9 mm for the rectum and minimum median 95% HD of 2.7 mm for the bladder. Maximal and minimal median DSC were detected for bladder with 0.97 and for penile bulb with 0.73, respectively. Using a tolerance level of 3 mm, the highest and lowest sDSC were determined for rectum (0.94) and anal canal (0.68), respectively. Qualitative evaluation resulted in a mean score of 1.2 for AICs over all organs and patients across all expert ratings. For the different autocontoured structures, the highest mean score of 1.0 was observed for anal canal, sacrum, femur left and right, and pelvis left, whereas for prostate the lowest mean score of 2.0 was detected. In total, 80% of the contours were rated be clinically acceptable, 16% to require minor and 4% major adjustments for online adaptive MRgRT.

CONCLUSION

In this study, an AI-based autocontouring was successfully trained for online adaptive MR-guided radiotherapy on the 1.5 T MR-Linac system. The developed model can automatically generate contours accepted by physicians (80%) or only with the need of minor corrections (16%) for the irradiation of primary prostate on the clinically employed sequences.

Intra-fractional geometric and dose/volume metric variations of magnetic resonance imaging-guided stereotactic radiotherapy of prostate bed after radical prostatectomy

Background and purpose

Magnetic Resonance Imaging (MRI)-guided Stereotactic body radiotherapy (SBRT) treatment to prostate bed after radical prostatectomy has garnered growing interests. The aim of this study is to evaluate intra-fractional anatomic and dose/volume metric variations for patients receiving this treatment.

Materials and methods

Nineteen patients who received 30-34 Gy in 5 fractions on a 0.35T MR-Linac were included. Pre- and post-treatment MRIs were acquired for each fraction (total of 75 fractions). The Clinical Target Volume (CTV), bladder, rectum, and rectal wall were contoured on all images. Volumetric changes, Hausdorff distance, Mean Distance to Agreement (MDA), and Dice similarity coefficient (DSC) for each structure were calculated. Median value and Interquartile range (IQR) were recorded. Changes in target coverage and Organ at Risk (OAR) constraints were compared and evaluated using Wilcoxon rank sum tests at a significant level of 0.05.

Results

Bladder had the largest volumetric changes, with a median volume increase of 48.9 % (IQR 28.9-76.8 %) and a median MDA of 5.1 mm (IQR 3.4-7.1 mm). Intra-fractional CTV volume remained stable with a median volume change of 1.2 % (0.0-4.8 %). DSC was 0.97 (IQR 0.94-0.99). For the dose/volume metrics, there were no statistically significant changes observed except for an increase in bladder hotspot and a decrease of bladder V32.5 Gy and mean dose. The CTV V95% changed from 99.9 % (IQR 98.8-100 %) to 99.6 % (IQR 93.9-100 %).

Conclusion

Despite intra-fractional variations of OARs, CTV coverage remained stable during MRI-guided SBRT treatments for the prostate bed.

Full daily re-optimization improves plan quality during online adaptive radiotherapy

Background and purpose

Daily online treatment plan adaptation requires a fast workflow and planning process. Current online planning consists of adaptation of a predefined reference plan, which might be suboptimal in cases of large anatomic changes. The aim of this study was to investigate plan quality differences between the current online re-planning approach and a complete re-optimization.

Material and methods

Magnetic resonance linear accelerator reference plans for ten prostate cancer patients were automatically generated using particle swarm optimization (PSO). Adapted plans were created for each fraction using (1) the current re-planning approach and (2) full PSO re-optimization and evaluated overall compliance with institutional dose-volume criteria compared to (3) clinically delivered fractions. Relative volume differences between reference and daily anatomy were assessed for planning target volumes (PTV60, PTV57.6), rectum and bladder and correlated with dose-volume results.

Results

The PSO approach showed significantly higher adherence to dose-volume criteria than the reference approach and clinical fractions (p < 0.001). In 74 % of PSO plans at most one criterion failed compared to 56 % in the reference approach and 41 % in clinical plans. A fair correlation between PTV60 D98% and relative bladder volume change was observed for the reference approach. Bladder volume reductions larger than 50 % compared to the reference plan recurrently decreased PTV60 D98% below 56 Gy.

Conclusion

Complete re-optimization maintained target coverage and organs at risk sparing even after large anatomic variations. Re-planning based on daily magnetic resonance imaging was sufficient for small variations, while large variations led to decreasing target coverage and organ-at-risk sparing.

Comparing multi-image and image augmentation strategies for deep learning-based prostate segmentation

During MR-Linac-based adaptive radiotherapy, multiple images are acquired per patient. These can be applied in training deep learning networks to reduce annotation efforts. This study examined the advantage of using multiple versus single images for prostate treatment segmentation. Findings indicate minimal improvement in DICE and Hausdorff 95% metrics with multiple images. Maximum difference was seen for the rectum in the low data regime, training with images from five patients. Utilizing a 2D U-net resulted in DICE values of 0.80/0.83 when including 1/5 images per patient, respectively. Including more patients in training reduced the difference. Standard augmentation methods remained more effective.

The role of artificial intelligence in radiotherapy clinical practice

This review article visits the current state of artificial intelligence (AI) in radiotherapy clinical practice. We will discuss how AI has a place in the modern radiotherapy workflow at the level of automatic segmentation and planning, two applications which have seen real-work implementation. A special emphasis will be placed on the role AI can play in online adaptive radiotherapy, such as performed at MR-linacs, where online plan adaptation is a procedure which could benefit from automation to reduce on-couch time for patients. Pseudo-CT generation and AI for motion tracking will be introduced in the scope of online adaptive radiotherapy as well. We further discuss the use of AI for decision-making and response assessment, for example for personalized prescription and treatment selection, risk stratification for outcomes and toxicities, and AI for quantitative imaging and response assessment. Finally, the challenges of generalizability and ethical aspects will be covered. With this, we provide a comprehensive overview of the current and future applications of AI in radiotherapy.

Prior knowledge based deep learning auto-segmentation in magnetic resonance imaging-guided radiotherapy of prostate cancer

Background and purpose

Automation is desirable for organ segmentation in radiotherapy. This study compared deep learning methods for auto-segmentation of organs-at-risk (OARs) and clinical target volume (CTV) in prostate cancer patients undergoing fractionated magnetic resonance (MR)-guided adaptive radiation therapy. Models predicting dense displacement fields (DDFMs) between planning and fraction images were compared to patient-specific (PSM) and baseline (BM) segmentation models.

Materials and methods

A dataset of 92 patients with planning and fraction MR images (MRIs) from two institutions were used. DDFMs were trained to predict dense displacement fields (DDFs) between the planning and fraction images, which were subsequently used to propagate the planning contours of the bladder, rectum, and CTV to the daily MRI. The training was performed either with true planning-fraction image pairs or with planning images and their counterparts deformed by known DDFs. The BMs were trained on 53 planning images, while to generate PSMs, the BMs were fine-tuned using the planning image of a given single patient. The evaluation included Dice similarity coefficient (DSC), the average (HDavg) and the 95th percentile (HD95) Hausdorff distance (HD).

Results

The DDFMs with DSCs for bladder/rectum of 0.76/0.76 performed worse than PSMs (0.91/0.90) and BMs (0.89/0.88). The same trend was observed for HDs. For CTV, DDFM and PSM performed similarly yielding DSCs of 0.87 and 0.84, respectively.

Conclusions

DDFMs were found suitable for CTV delineation after rigid alignment. However, for OARs they were outperformed by PSMs, as they predicted only limited deformations even in the presence of substantial anatomical changes.

A network score-based metric to optimize the quality assurance of automatic radiotherapy target segmentations

Background and purpose

Existing methods for quality assurance of the radiotherapy auto-segmentations focus on the correlation between the average model entropy and the Dice Similarity Coefficient (DSC) only. We identified a metric directly derived from the output of the network and correlated it with clinically relevant metrics for contour accuracy.

Materials and Methods

Magnetic Resonance Imaging auto-segmentations were available for the gross tumor volume for cervical cancer brachytherapy (106 segmentations) and for the clinical target volume for rectal cancer external-beam radiotherapy (77 segmentations). The nnU-Net's output before binarization was taken as a score map. We defined a metric as the mean of the voxels in the score map above a threshold (λ). Comparisons were made with the mean and standard deviation over the score map and with the mean over the entropy map. The DSC, the 95th Hausdorff distance, the mean surface distance (MSD) and the surface DSC were computed for segmentation quality. Correlations between the studied metrics and model quality were assessed with the Pearson correlation coefficient (r). The area under the curve (AUC) was determined for detecting segmentations that require reviewing.

Results

For both tasks, our metric (λ = 0.30) correlated more strongly with the segmentation quality than the mean over the entropy map (for surface DSC, r > 0.65 vs. r < 0.60). The AUC was above 0.84 for detecting MSD values above 2 mm.

Conclusions

Our metric correlated strongly with clinically relevant segmentation metrics and detected segmentations that required reviewing, indicating its potential for automatic quality assurance of radiotherapy target auto-segmentations.

Deep learning based automatic segmentation of organs-at-risk for 0.35 T MRgRT of lung tumors

BACKGROUND AND PURPOSE

Magnetic resonance imaging guided radiotherapy (MRgRT) offers treatment plan adaptation to the anatomy of the day. In the current MRgRT workflow, this requires the time consuming and repetitive task of manual delineation of organs-at-risk (OARs), which is also prone to inter- and intra-observer variability. Therefore, deep learning autosegmentation (DLAS) is becoming increasingly attractive. No investigation of its application to OARs in thoracic magnetic resonance images (MRIs) from MRgRT has been done so far. This study aimed to fill this gap.

MATERIALS AND METHODS

122 planning MRIs from patients treated at a 0.35 T MR-Linac were retrospectively collected. Using an 80/19/23 (training/validation/test) split, individual 3D U-Nets for segmentation of the left lung, right lung, heart, aorta, spinal canal and esophagus were trained. These were compared to the clinically used contours based on Dice similarity coefficient (DSC) and Hausdorff distance (HD). They were also graded on their clinical usability by a radiation oncologist.

RESULTS

Median DSC was 0.96, 0.96, 0.94, 0.90, 0.88 and 0.78 for left lung, right lung, heart, aorta, spinal canal and esophagus, respectively. Median 95th percentile values of the HD were 3.9, 5.3, 5.8, 3.0, 2.6 and 3.5 mm, respectively. The physician preferred the network generated contours over the clinical contours, deeming 85 out of 129 to not require any correction, 25 immediately usable for treatment planning, 15 requiring minor and 4 requiring major corrections.

CONCLUSIONS

We trained 3D U-Nets on clinical MRI planning data which produced accurate delineations in the thoracic region. DLAS contours were preferred over the clinical contours.

Deep Learning in MRI-guided Radiation Therapy: A Systematic Review

MRI-guided radiation therapy (MRgRT) offers a precise and adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed. MRI-guided radiation therapy offers a precise, adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed with emphasis placed on underlying methods. Studies are further categorized into the areas of segmentation, synthesis, radiomics, and real time MRI. Finally, clinical implications, current challenges, and future directions are discussed.

Image masking using convolutional networks improves performance classification of radiation pneumonitis for non-small cell lung cancer

Radiation pneumonitis (RP) is a serious side effect of radiotherapy in patients with locally advanced non-small-cell lung cancer (NSCLC). Image cropping reduces training noise and may improve classification accuracy. This study proposes a prediction model for RP grade ≥ 2 using a convolutional neural network (CNN) model with image cropping. The 3D computed tomography (CT) images cropped in the whole-body, normal lung (nLung), and nLung regions overlapping the region over 20 Gy (nLung∩20 Gy) used in treatment planning were used as the input data. The output classifies patients as RP grade < 2 or RP grade ≥ 2. The sensitivity, specificity, accuracy, and area under the curve (AUC) were evaluated using the receiver operating characteristic curve (ROC). The accuracy, specificity, sensitivity, and AUC were 53.9%, 80.0%, 25.5%, and 0.58, respectively, for the whole-body method, and 60.0%, 81.7%, 36.4%, and 0.64, respectively, for the nLung method. For the nLung∩20 Gy method, the accuracy, specificity, sensitivity, and AUC improved to 75.7%, 80.0%, 70.9%, and 0.84, respectively. The CNN model, in which the input image is segmented in the normal lung considering the dose distribution, can help predict an RP grade ≥ 2 for NSCLC patients after definitive radiotherapy.