Vision 20/20: perspectives on automated image segmentation for radiotherapy

Applications of artificial intelligence in computed tomography imaging for phenotyping pulmonary hypertension

PURPOSE OF REVIEW

Pulmonary hypertension is a heterogeneous condition with significant morbidity and mortality. Computer tomography (CT) plays a central role in determining the phenotype of pulmonary hypertension, informing treatment strategies. Many artificial intelligence tools have been developed in this modality for the assessment of pulmonary hypertension. This article reviews the latest CT artificial intelligence applications in pulmonary hypertension and related diseases.

RECENT FINDINGS

Multistructure segmentation tools have been developed in both pulmonary hypertension and nonpulmonary hypertension cohorts using state-of-the-art UNet architecture. These segmentations correspond well with those of trained radiologists, giving clinically valuable metrics in significantly less time. Artificial intelligence lung parenchymal assessment accurately identifies and quantifies lung disease patterns by integrating multiple radiomic techniques such as texture analysis and classification. This gives valuable information on disease burden and prognosis. There are many accurate artificial intelligence tools to detect acute pulmonary embolism. Detection of chronic pulmonary embolism proves more challenging with further research required.

SUMMARY

There are numerous artificial intelligence tools being developed to identify and quantify many clinically relevant parameters in both pulmonary hypertension and related disease cohorts. These potentially provide accurate and efficient clinical information, impacting clinical decision-making.

An operator-independent quality assurance system for automatically generated structure sets

Objective. This study describes geometry-based and intensity-based tools for quality assurance (QA) of automatically generated structures for online adaptive radiotherapy, and designs an operator-independent traffic light system that identifies erroneous structure sets.Approach.A cohort of eight head and neck (HN) patients with daily CBCTs was selected for test development. Radiotherapy contours were propagated from planning computed tomography (CT) to daily cone beam CT (CBCT) using deformable image registration. These propagated structures were visually verified for acceptability. For each CBCT, several error scenarios were used to generate what were judged unacceptable structures. Ten additional HN patients with daily CBCTs and different error scenarios were selected for validation. A suite of tests based on image intensity, intensity gradient, and structure geometry was developed using acceptable and unacceptable HN planning structures. Combinations of one test applied to one structure, referred to as structure-test combinations, were selected for inclusion in the QA system based on their discriminatory power. A traffic light system was used to aggregate the structure-test combinations, and the system was evaluated on all fractions of the ten validation HN patients.Results.The QA system distinguished between acceptable and unacceptable fractions with high accuracy, labeling 294/324 acceptable fractions as green or yellow and 19/20 unacceptable fractions as yellow or red.Significance.This study demonstrates a system to supplement manual review of radiotherapy planning structures. Automated QA is performed by aggregating results from multiple intensity- and geometry-based tests.

Artificial intelligence for treatment delivery: image-guided radiotherapy

Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.

Radiomics as a measure superior to common similarity metrics for tumor segmentation performance evaluation

PURPOSE

To propose radiomics features as a superior measure for evaluating the segmentation ability of physicians and auto-segmentation tools and to compare its performance with the most commonly used metrics: Dice similarity coefficient (DSC), surface Dice similarity coefficient (sDSC), and Hausdorff distance (HD).

MATERIALS/METHODS

The data of 10 lung cancer patients' CT images with nine tumor segmentations per tumor were downloaded from the RIDER (Reference Database to Evaluate Response) database. Radiomics features of 90 segmented tumors were extracted using the PyRadiomics program. The intraclass correlation coefficient (ICC) of radiomics features were used to evaluate the segmentation similarity and compare their performance with DSC, sDSC, and HD. We calculated one ICC per radiomics feature and per tumor for nine segmentations and 36 ICCs per radiomics feature for 36 pairs of nine segmentations. Meanwhile, there were 360 DSC, sDSC, and HD values calculated for 36 pairs for 10 tumors.

RESULTS

The ICC of radiomics features exhibited greater sensitivity to segmentation changes than DSC and sDSC. The ICCs of the wavelet-LLL first order Maximum, wavelet-LLL glcm MCC, wavelet-LLL glcm Cluster Shade features ranged from 0.130 to 0.997, 0.033 to 0.978, and 0.160 to 0.998, respectively. On the other hand, all DSC and sDSC were larger than 0.778 and 0.700, respectively, while HD varied from 0 to 1.9 mm. The results indicated that the radiomics features could capture subtle variations in tumor segmentation characteristics, which could not be easily detected by DSC and sDSC.

CONCLUSIONS

This study demonstrates the superiority of radiomics features with ICC as a measure for evaluating a physician's tumor segmentation ability and the performance of auto-segmentation tools. Radiomics features offer a more sensitive and comprehensive evaluation, providing valuable insights into tumor characteristics. Therefore, the new metrics can be used to evaluate new auto-segmentation methods and enhance trainees' segmentation skills in medical training and education.

Landmark-based auto-contouring of clinical target volumes for radiotherapy of nasopharyngeal cancer

BACKGROUND

The delineation of clinical target volumes (CTVs) for radiotherapy for nasopharyngeal cancer is complex and varies based on the location and extent of disease.

PURPOSE

The current study aimed to develop an auto-contouring solution following one protocol guidelines (NRG-HN001) that can be adjusted to meet other guidelines, such as RTOG-0225 and the 2018 International guidelines.

METHODS

The study used 2-channel 3-dimensional U-Net and nnU-Net framework to auto-contour 27 normal structures in the head and neck (H&N) region that are used to define CTVs in the protocol. To define the CTV-Expansion (CTV1 and CTV2) and CTV-Overall (the outer envelope of all the CTV contours), we used adjustable morphological geometric landmarks and mimicked physician interpretation of the protocol rules by partially or fully including select anatomic structures. The results were evaluated quantitatively using the dice similarity coefficient (DSC) and mean surface distance (MSD) and qualitatively by independent reviews by two H&N radiation oncologists.

RESULTS

The auto-contouring tool showed high accuracy for nasopharyngeal CTVs. Comparison between auto-contours and clinical contours for 19 patients with cancers of various stages showed a DSC of 0.94 ± 0.02 and MSD of 0.4 ± 0.4 mm for CTV-Expansion and a DSC of 0.83 ± 0.02 and MSD of 2.4 ± 0.5 mm for CTV-Overall. Upon independent review, two H&N physicians found the auto-contours to be usable without edits in 85% and 75% of cases. In 15% of cases, minor edits were required by both physicians. Thus, one physician rated 100% of the auto-contours as usable (use as is, or after minor edits), while the other physician rated 90% as usable. The second physician required major edits in 10% of cases.

CONCLUSIONS

The study demonstrates the ability of an auto-contouring tool to reliably delineate nasopharyngeal CTVs based on protocol guidelines. The tool was found to be clinically acceptable by two H&N radiation oncology physicians in at least 90% of the cases.

Deep Learning Auto-Segmentation Network for Pediatric Computed Tomography Data Sets: Can We Extrapolate From Adults?

PURPOSE

Artificial intelligence (AI)-based auto-segmentation models hold promise for enhanced efficiency and consistency in organ contouring for adaptive radiation therapy and radiation therapy planning. However, their performance on pediatric computed tomography (CT) data and cross-scanner compatibility remain unclear. This study aimed to evaluate the performance of AI-based auto-segmentation models trained on adult CT data when applied to pediatric data sets and explore the improvement in performance gained by including pediatric training data. It also examined their ability to accurately segment CT data acquired from different scanners.

METHODS AND MATERIALS

Using the nnU-Net framework, segmentation models were trained on data sets of adult, pediatric, and combined CT scans for 7 pelvic/thoracic organs. Each model was trained on 290 to 300 cases per category and organ. Training data sets included a combination of clinical data and several open repositories. The study incorporated a database of 459 pediatric (0-16 years) CT scans and 950 adults (>18 years), ensuring all scans had human expert ground-truth contours of the selected organs. Performance was evaluated based on Dice similarity coefficients (DSC) of the model-generated contours.

RESULTS

AI models trained exclusively on adult data underperformed on pediatric data, especially for the 0 to 2 age group: mean DSC was below 0.5 for the bladder and spleen. The addition of pediatric training data demonstrated significant improvement for all age groups, achieving a mean DSC of above 0.85 for all organs in every age group. Larger organs like the liver and kidneys maintained consistent performance for all models across age groups. No significant difference emerged in the cross-scanner performance evaluation, suggesting robust cross-scanner generalization.

CONCLUSIONS

For optimal segmentation across age groups, it is important to include pediatric data in the training of segmentation models. The successful cross-scanner generalization also supports the real-world clinical applicability of these AI models. This study emphasizes the significance of data set diversity in training robust AI systems for medical image interpretation tasks.

Deep learning in pulmonary nodule detection and segmentation: a systematic review

OBJECTIVES

The accurate detection and precise segmentation of lung nodules on computed tomography are key prerequisites for early diagnosis and appropriate treatment of lung cancer. This study was designed to compare detection and segmentation methods for pulmonary nodules using deep-learning techniques to fill methodological gaps and biases in the existing literature.

METHODS

This study utilized a systematic review with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, searching PubMed, Embase, Web of Science Core Collection, and the Cochrane Library databases up to May 10, 2023. The Quality Assessment of Diagnostic Accuracy Studies 2 criteria was used to assess the risk of bias and was adjusted with the Checklist for Artificial Intelligence in Medical Imaging. The study analyzed and extracted model performance, data sources, and task-focus information.

RESULTS

After screening, we included nine studies meeting our inclusion criteria. These studies were published between 2019 and 2023 and predominantly used public datasets, with the Lung Image Database Consortium Image Collection and Image Database Resource Initiative and Lung Nodule Analysis 2016 being the most common. The studies focused on detection, segmentation, and other tasks, primarily utilizing Convolutional Neural Networks for model development. Performance evaluation covered multiple metrics, including sensitivity and the Dice coefficient.

CONCLUSIONS

This study highlights the potential power of deep learning in lung nodule detection and segmentation. It underscores the importance of standardized data processing, code and data sharing, the value of external test datasets, and the need to balance model complexity and efficiency in future research.

CLINICAL RELEVANCE STATEMENT

Deep learning demonstrates significant promise in autonomously detecting and segmenting pulmonary nodules. Future research should address methodological shortcomings and variability to enhance its clinical utility.

KEY POINTS

Deep learning shows potential in the detection and segmentation of pulmonary nodules. There are methodological gaps and biases present in the existing literature. Factors such as external validation and transparency affect the clinical application.

Multiple organ segmentation framework for brain metastasis radiotherapy

Radiotherapy is a preferred treatment for brain metastases, which kills cancer cells via high doses of radiation meanwhile hardly avoiding damage to surrounding healthy cells. Therefore, the delineation of organs-at-risk (OARs) is vital in treatment planning to minimize radiation-induced toxicity. However, the following aspects make OAR delineation a challenging task: extremely imbalanced organ sizes, ambiguous boundaries, and complex anatomical structures. To alleviate these challenges, we imitate how specialized clinicians delineate OARs and present a novel cascaded multi-OAR segmentation framework, called OAR-SegNet. OAR-SegNet comprises two distinct levels of segmentation networks: an Anatomical-Prior-Guided network (APG-Net) and a Point-Cloud-Guided network (PCG-Net). Specifically, APG-Net handles segmentation for all organs, where multi-view segmentation modules and a deep prior loss are designed under the guidance of prior knowledge. After APG-Net, PCG-Net refines small organs through the mini-segmentation and the point-cloud alignment heads. The mini-segmentation head is further equipped with the deep prior feature. Extensive experiments were conducted to demonstrate the superior performance of the proposed method compared to other state-of-the-art medical segmentation methods.

A deep learning-based 3D Prompt-nnUnet model for automatic segmentation in brachytherapy of postoperative endometrial carcinoma

PURPOSE

To create and evaluate a three-dimensional (3D) Prompt-nnUnet module that utilizes the prompts-based model combined with 3D nnUnet for producing the rapid and consistent autosegmentation of high-risk clinical target volume (HR CTV) and organ at risk (OAR) in high-dose-rate brachytherapy (HDR BT) for patients with postoperative endometrial carcinoma (EC).

METHODS AND MATERIALS

On two experimental batches, a total of 321 computed tomography (CT) scans were obtained for HR CTV segmentation from 321 patients with EC, and 125 CT scans for OARs segmentation from 125 patients. The numbers of training/validation/test were 257/32/32 and 87/13/25 for HR CTV and OARs respectively. A novel comparison of the deep learning neural network 3D Prompt-nnUnet and 3D nnUnet was applied for HR CTV and OARs segmentation. Three-fold cross validation and several quantitative metrics were employed, including Dice similarity coefficient (DSC), Hausdorff distance (HD), 95th percentile of Hausdorff distance (HD95%), and intersection over union (IoU).

RESULTS

The Prompt-nnUnet included two forms of parameters Predict-Prompt (PP) and Label-Prompt (LP), with the LP performing most similarly to the experienced radiation oncologist and outperforming the less experienced ones. During the testing phase, the mean DSC values for the LP were 0.96 ± 0.02, 0.91 ± 0.02, and 0.83 ± 0.07 for HR CTV, rectum and urethra, respectively. The mean HD values (mm) were 2.73 ± 0.95, 8.18 ± 4.84, and 2.11 ± 0.50, respectively. The mean HD95% values (mm) were 1.66 ± 1.11, 3.07 ± 0.94, and 1.35 ± 0.55, respectively. The mean IoUs were 0.92 ± 0.04, 0.84 ± 0.03, and 0.71 ± 0.09, respectively. A delineation time < 2.35 s per structure in the new model was observed, which was available to save clinician time.

CONCLUSION

The Prompt-nnUnet architecture, particularly the LP, was highly consistent with ground truth (GT) in HR CTV or OAR autosegmentation, reducing interobserver variability and shortening treatment time.

Enhancing semantic segmentation in chest X-ray images through image preprocessing: ps-KDE for pixel-wise substitution by kernel density estimation

BACKGROUND

In medical imaging, the integration of deep-learning-based semantic segmentation algorithms with preprocessing techniques can reduce the need for human annotation and advance disease classification. Among established preprocessing techniques, Contrast Limited Adaptive Histogram Equalization (CLAHE) has demonstrated efficacy in improving segmentation algorithms across various modalities, such as X-rays and CT. However, there remains a demand for improved contrast enhancement methods considering the heterogeneity of datasets and the various contrasts across different anatomic structures.

METHOD

This study proposes a novel preprocessing technique, ps-KDE, to investigate its impact on deep learning algorithms to segment major organs in posterior-anterior chest X-rays. Ps-KDE augments image contrast by substituting pixel values based on their normalized frequency across all images. We evaluate our approach on a U-Net architecture with ResNet34 backbone pre-trained on ImageNet. Five separate models are trained to segment the heart, left lung, right lung, left clavicle, and right clavicle.

RESULTS

The model trained to segment the left lung using ps-KDE achieved a Dice score of 0.780 (SD = 0.13), while that of trained on CLAHE achieved a Dice score of 0.717 (SD = 0.19), p<0.01. ps-KDE also appears to be more robust as CLAHE-based models misclassified right lungs in select test images for the left lung model. The algorithm for performing ps-KDE is available at https://github.com/wyc79/ps-KDE.

DISCUSSION

Our results suggest that ps-KDE offers advantages over current preprocessing techniques when segmenting certain lung regions. This could be beneficial in subsequent analyses such as disease classification and risk stratification.

Evaluation of Deep Learning Clinical Target Volumes Auto-Contouring for Magnetic Resonance Imaging-Guided Online Adaptive Treatment of Rectal Cancer

Purpose

Segmentation of clinical target volumes (CTV) on medical images can be time-consuming and is prone to interobserver variation (IOV). This is a problem for online adaptive radiation therapy, where CTV segmentation must be performed every treatment fraction, leading to longer treatment times and logistic challenges. Deep learning (DL)-based auto-contouring has the potential to speed up CTV contouring, but its current clinical use is limited. One reason for this is that it can be time-consuming to verify the accuracy of CTV contours produced using auto-contouring, and there is a risk of bias being introduced. To be accepted by clinicians, auto-contouring must be trustworthy. Therefore, there is a need for a comprehensive commissioning framework when introducing DL-based auto-contouring in clinical practice. We present such a framework and apply it to an in-house developed DL model for auto-contouring of the CTV in rectal cancer patients treated with MRI-guided online adaptive radiation therapy.

Methods and Materials

The framework for evaluating DL-based auto-contouring consisted of 3 steps: (1) Quantitative evaluation of the model's performance and comparison with IOV; (2) Expert observations and corrections; and (3) Evaluation of the impact on expected volumetric target coverage. These steps were performed on independent data sets. The framework was applied to an in-house trained nnU-Net model, using the data of 44 rectal cancer patients treated at our institution.

Results

The framework established that the model's performance after expert corrections was comparable to IOV, and although the model introduced a bias, this had no relevant impact on clinical practice. Additionally, we found a substantial time gain without reducing quality as determined by volumetric target coverage.

Conclusions

Our framework provides a comprehensive evaluation of the performance and clinical usability of target auto-contouring models. Based on the results, we conclude that the model is eligible for clinical use.

Comparison of commercial atlas-based automatic segmentation software for prostate radiotherapy treatment planning

This study aims to assess the accuracy of automatic atlas-based contours for various key anatomical structures in prostate radiotherapy treatment planning. The evaluated structures include the bladder, rectum, prostate, seminal vesicles, femoral heads and penile bulb. CT images from 20 patients who underwent intensity-modulated radiotherapy were randomly chosen to create an atlas library. Atlas contours of the seven anatomical structures were generated using four software packages: ABAS, Eclipse, MIM, and RayStation. These contours were then compared to manual delineations performed by oncologists, which served as the ground truth. Evaluation metrics such as dice similarity coefficient (DSC), mean distance to agreement (MDA), and volume ratio (VR) were calculated to assess the accuracy of the contours. Additionally, the time taken by each software to generate the atlas contour was recorded. The mean DSC values for the bladder exhibited strong agreement (>0.8) with manual delineations for all software except for Eclipse and RayStation. Similarly, the femoral heads showed significant similarity between the atlas contours and ground truth across all software, with mean DSC values exceeding 0.9 and MDA values close to zero. On the other hand, the penile bulb displayed only moderate agreement with the ground truth, with mean DSC values ranging from 0.5 to 0.7 for all software. A similar trend was observed in the prostate atlas contours, except for MIM, which achieved a mean DSC of over 0.8. For the rectum, both ABAS and MIM atlases demonstrated strong agreement with the ground truth, resulting in mean DSC values of more than 0.8. Overall, MIM and ABAS outperformed Eclipse and RayStation in both DSC and MDA. These results indicate that the atlas-based segmentation employed in this study produces acceptable contours for the anatomical structures of interest in prostate radiotherapy treatment planning.

Large-scale dose evaluation of deep learning organ contours in head-and-neck radiotherapy by leveraging existing plans

Background and purpose

Retrospective dose evaluation for organ-at-risk auto-contours has previously used small cohorts due to additional manual effort required for treatment planning on auto-contours. We aimed to do this at large scale, by a) proposing and assessing an automated plan optimization workflow that used existing clinical plan parameters and b) using it for head-and-neck auto-contour dose evaluation.

Materials and methods

Our automated workflow emulated our clinic's treatment planning protocol and reused existing clinical plan optimization parameters. This workflow recreated the original clinical plan (P OG ) with manual contours (P MC ) and evaluated the dose effect (P OG - P MC ) on 70 photon and 30 proton plans of head-and-neck patients. As a use-case, the same workflow (and parameters) created a plan using auto-contours (P AC ) of eight head-and-neck organs-at-risk from a commercial tool and evaluated their dose effect (P MC - P AC ).

Results

For plan recreation (P OG - P MC ), our workflow had a median impact of 1.0% and 1.5% across dose metrics of auto-contours, for photon and proton respectively. Computer time of automated planning was 25% (photon) and 42% (proton) of manual planning time. For auto-contour evaluation (P MC - P AC ), we noticed an impact of 2.0% and 2.6% for photon and proton radiotherapy. All evaluations had a median Δ NTCP (Normal Tissue Complication Probability) less than 0.3%.

Conclusions

The plan replication capability of our automated program provides a blueprint for other clinics to perform auto-contour dose evaluation with large patient cohorts. Finally, despite geometric differences, auto-contours had a minimal median dose impact, hence inspiring confidence in their utility and facilitating their clinical adoption.

Deep learning and atlas-based models to streamline the segmentation workflow of total marrow and lymphoid irradiation

PURPOSE

To improve the workflow of total marrow and lymphoid irradiation (TMLI) by enhancing the delineation of organs at risk (OARs) and clinical target volume (CTV) using deep learning (DL) and atlas-based (AB) segmentation models.

MATERIALS AND METHODS

Ninety-five TMLI plans optimized in our institute were analyzed. Two commercial DL software were tested for segmenting 18 OARs. An AB model for lymph node CTV (CTV_LN) delineation was built using 20 TMLI patients. The AB model was evaluated on 20 independent patients, and a semiautomatic approach was tested by correcting the automatic contours. The generated OARs and CTV_LN contours were compared to manual contours in terms of topological agreement, dose statistics, and time workload. A clinical decision tree was developed to define a specific contouring strategy for each OAR.

RESULTS

The two DL models achieved a median [interquartile range] dice similarity coefficient (DSC) of 0.84 [0.71;0.93] and 0.85 [0.70;0.93] across the OARs. The absolute median Dmean difference between manual and the two DL models was 2.0 [0.7;6.6]% and 2.4 [0.9;7.1]%. The AB model achieved a median DSC of 0.70 [0.66;0.74] for CTV_LN delineation, increasing to 0.94 [0.94;0.95] after manual revision, with minimal Dmean differences. Since September 2022, our institution has implemented DL and AB models for all TMLI patients, reducing from 5 to 2 h the time required to complete the entire segmentation process.

CONCLUSION

DL models can streamline the TMLI contouring process of OARs. Manual revision is still necessary for lymph node delineation using AB models.

Enhancing medical image segmentation with a multi-transformer U-Net

Various segmentation networks based on Swin Transformer have shown promise in medical segmentation tasks. Nonetheless, challenges such as lower accuracy and slower training convergence have persisted. To tackle these issues, we introduce a novel approach that combines the Swin Transformer and Deformable Transformer to enhance overall model performance. We leverage the Swin Transformer's window attention mechanism to capture local feature information and employ the Deformable Transformer to adjust sampling positions dynamically, accelerating model convergence and aligning it more closely with object shapes and sizes. By amalgamating both Transformer modules and incorporating additional skip connections to minimize information loss, our proposed model excels at rapidly and accurately segmenting CT or X-ray lung images. Experimental results demonstrate the remarkable, showcasing the significant prowess of our model. It surpasses the performance of the standalone Swin Transformer's Swin Unet and converges more rapidly under identical conditions, yielding accuracy improvements of 0.7% (resulting in 88.18%) and 2.7% (resulting in 98.01%) on the COVID-19 CT scan lesion segmentation dataset and Chest X-ray Masks and Labels dataset, respectively. This advancement has the potential to aid medical practitioners in early diagnosis and treatment decision-making.

Reference standard for the evaluation of automatic segmentation algorithms: Quantification of inter observer variability of manual delineation of prostate contour on MRI

PURPOSE

The purpose of this study was to investigate the relationship between inter-reader variability in manual prostate contour segmentation on magnetic resonance imaging (MRI) examinations and determine the optimal number of readers required to establish a reliable reference standard.

MATERIALS AND METHODS

Seven radiologists with various experiences independently performed manual segmentation of the prostate contour (whole-gland [WG] and transition zone [TZ]) on 40 prostate MRI examinations obtained in 40 patients. Inter-reader variability in prostate contour delineations was estimated using standard metrics (Dice similarity coefficient [DSC], Hausdorff distance and volume-based metrics). The impact of the number of readers (from two to seven) on segmentation variability was assessed using pairwise metrics (consistency) and metrics with respect to a reference segmentation (conformity), obtained either with majority voting or simultaneous truth and performance level estimation (STAPLE) algorithm.

RESULTS

The average segmentation DSC for two readers in pairwise comparison was 0.919 for WG and 0.876 for TZ. Variability decreased with the number of readers: the interquartile ranges of the DSC were 0.076 (WG) / 0.021 (TZ) for configurations with two readers, 0.005 (WG) / 0.012 (TZ) for configurations with three readers, and 0.002 (WG) / 0.0037 (TZ) for configurations with six readers. The interquartile range decreased slightly faster between two and three readers than between three and six readers. When using consensus methods, variability often reached its minimum with three readers (with STAPLE, DSC = 0.96 [range: 0.945-0.971] for WG and DSC = 0.94 [range: 0.912-0.957] for TZ, and interquartile range was minimal for configurations with three readers.

CONCLUSION

The number of readers affects the inter-reader variability, in terms of inter-reader consistency and conformity to a reference. Variability is minimal for three readers, or three readers represent a tipping point in the variability evolution, with both pairwise-based metrics or metrics with respect to a reference. Accordingly, three readers may represent an optimal number to determine references for artificial intelligence applications.

Creating patient-specific digital phantoms with a longitudinal atlas for evaluating deformable CT-CBCT registration in adaptive lung radiotherapy

BACKGROUND

Quality assurance of deformable image registration (DIR) is challenging because the ground truth is often unavailable. In addition, current approaches that rely on artificial transformations do not adequately resemble clinical scenarios encountered in adaptive radiotherapy.

PURPOSE

We developed an atlas-based method to create a variety of patient-specific serial digital phantoms with CBCT-like image quality to assess the DIR performance for longitudinal CBCT imaging data in adaptive lung radiotherapy.

METHODS

A library of deformations was created by extracting the longitudinal changes observed between a planning CT and weekly CBCT from an atlas of lung radiotherapy patients. The planning CT of an inquiry patient was first deformed by mapping the deformation pattern from a matched atlas patient, and subsequently appended with CBCT artifacts to imitate a weekly CBCT. Finally, a group of digital phantoms around an inquiry patient was produced to simulate a series of possible evolutions of tumor and adjacent normal structures. We validated the generated deformation vector fields (DVFs) to ensure numerically and physiologically realistic transformations. The proposed framework was applied to evaluate the performance of the DIR algorithm implemented in the commercial Eclipse treatment planning system in a retrospective study of eight inquiry patients.

RESULTS

The generated DVFs were inverse consistent within less than 3 mm and did not exhibit unrealistic folding. The deformation patterns adequately mimicked the observed longitudinal anatomical changes of the matched atlas patients. Worse Eclipse DVF accuracy was observed in regions of low image contrast or artifacts. The structure volumes exhibiting a DVF error magnitude of equal or more than 2 mm ranged from 24.5% (spinal cord) to 69.2% (heart) and the maximum DVF error exceeded 5 mm for all structures except the spinal cord. Contour-based evaluations showed a high degree of alignment with dice similarity coefficients above 0.8 in all cases, which underestimated the overall DVF accuracy within the structures.

CONCLUSIONS

It is feasible to create and augment digital phantoms based on a particular patient of interest using multiple series of deformation patterns from matched patients in an atlas. This can provide a semi-automated procedure to complement the quality assurance of CT-CBCT DIR and facilitate the clinical implementation of image-guided and adaptive radiotherapy that involve longitudinal CBCT imaging studies.

CT radiomics based on different machine learning models for classifying gross tumor volume and normal liver tissue in hepatocellular carcinoma

BACKGROUND & AIMS

The present study utilized extracted computed tomography radiomics features to classify the gross tumor volume and normal liver tissue in hepatocellular carcinoma by mainstream machine learning methods, aiming to establish an automatic classification model.

METHODS

We recruited 104 pathologically confirmed hepatocellular carcinoma patients for this study. GTV and normal liver tissue samples were manually segmented into regions of interest and randomly divided into five-fold cross-validation groups. Dimensionality reduction using LASSO regression. Radiomics models were constructed via logistic regression, support vector machine (SVM), random forest, Xgboost, and Adaboost algorithms. The diagnostic efficacy, discrimination, and calibration of algorithms were verified using area under the receiver operating characteristic curve (AUC) analyses and calibration plot comparison.

RESULTS

Seven screened radiomics features excelled at distinguishing the gross tumor area. The Xgboost machine learning algorithm had the best discrimination and comprehensive diagnostic performance with an AUC of 0.9975 [95% confidence interval (CI): 0.9973-0.9978] and mean MCC of 0.9369. SVM had the second best discrimination and diagnostic performance with an AUC of 0.9846 (95% CI: 0.9835- 0.9857), mean Matthews correlation coefficient (MCC)of 0.9105, and a better calibration. All other algorithms showed an excellent ability to distinguish between gross tumor area and normal liver tissue (mean AUC 0.9825, 0.9861,0.9727,0.9644 for Adaboost, random forest, logistic regression, naivem Bayes algorithm respectively).

CONCLUSION

CT radiomics based on machine learning algorithms can accurately classify GTV and normal liver tissue, while the Xgboost and SVM algorithms served as the best complementary algorithms.

The Use of Magnetic Resonance Imaging in Radiation Therapy Treatment Simulation and Planning

Ever since its introduction as a diagnostic imaging tool the potential of magnetic resonance imaging (MRI) in radiation therapy (RT) treatment simulation and planning has been recognized. Recent technical advances have addressed many of the impediments to use of this technology and as a result have resulted in rapid and growing adoption of MRI in RT. The purpose of this article is to provide a broad review of the multiple uses of MR in the RT treatment simulation and planning process, identify several of the most used clinical scenarios in which MR is integral to the simulation and planning process, highlight existing limitations and provide multiple unmet needs thereby highlighting opportunities for the diagnostic MR imaging community to contribute and collaborate with our oncology colleagues. EVIDENCE LEVEL: 5 TECHNICAL EFFICACY: Stage 5.

Auto-segmentation of neck nodal metastases using self-distilled masked image transformer on longitudinal MR images

Objectives

Auto-segmentation promises greater speed and lower inter-reader variability than manual segmentations in radiation oncology clinical practice. This study aims to implement and evaluate the accuracy of the auto-segmentation algorithm, "Masked Image modeling using the vision Transformers (SMIT)," for neck nodal metastases on longitudinal T2-weighted (T2w) MR images in oropharyngeal squamous cell carcinoma (OPSCC) patients.

Methods

This prospective clinical trial study included 123 human papillomaviruses (HPV-positive [+]) related OSPCC patients who received concurrent chemoradiotherapy. T2w MR images were acquired on 3 T at pre-treatment (Tx), week 0, and intra-Tx weeks (1-3). Manual delineations of metastatic neck nodes from 123 OPSCC patients were used for the SMIT auto-segmentation, and total tumor volumes were calculated. Standard statistical analyses compared contour volumes from SMIT vs manual segmentation (Wilcoxon signed-rank test [WSRT]), and Spearman's rank correlation coefficients (ρ) were computed. Segmentation accuracy was evaluated on the test data set using the dice similarity coefficient (DSC) metric value. P-values <0.05 were considered significant.

Results

No significant difference in manual and SMIT delineated tumor volume at pre-Tx (8.68 ± 7.15 vs 8.38 ± 7.01 cm3, P = 0.26 [WSRT]), and the Bland-Altman method established the limits of agreement as -1.71 to 2.31 cm3, with a mean difference of 0.30 cm3. SMIT model and manually delineated tumor volume estimates were highly correlated (ρ = 0.84-0.96, P < 0.001). The mean DSC metric values were 0.86, 0.85, 0.77, and 0.79 at the pre-Tx and intra-Tx weeks (1-3), respectively.

Conclusions

The SMIT algorithm provides sufficient segmentation accuracy for oncological applications in HPV+ OPSCC.

Advances in knowledge

First evaluation of auto-segmentation with SMIT using longitudinal T2w MRI in HPV+ OPSCC.

Lesion segmentation on 18F-fluciclovine PET/CT images using deep learning

Background and purpose

A novel radiotracer, 18F-fluciclovine (anti-3-18F-FACBC), has been demonstrated to be associated with significantly improved survival when it is used in PET/CT imaging to guide postprostatectomy salvage radiotherapy for prostate cancer. We aimed to investigate the feasibility of using a deep learning method to automatically detect and segment lesions on 18F-fluciclovine PET/CT images.

Materials and methods

We retrospectively identified 84 patients who are enrolled in Arm B of the Emory Molecular Prostate Imaging for Radiotherapy Enhancement (EMPIRE-1) trial. All 84 patients had prostate adenocarcinoma and underwent prostatectomy and 18F-fluciclovine PET/CT imaging with lesions identified and delineated by physicians. Three different neural networks with increasing levels of complexity (U-net, Cascaded U-net, and a cascaded detection segmentation network) were trained and tested on the 84 patients with a fivefold cross-validation strategy and a hold-out test, using manual contours as the ground truth. We also investigated using both PET and CT or using PET only as input to the neural network. Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), center-of-mass distance (CMD), and volume difference (VD) were used to quantify the quality of segmentation results against ground truth contours provided by physicians.

Results

All three deep learning methods were able to detect 144/155 lesions and 153/155 lesions successfully when PET+CT and PET only, respectively, served as input. Quantitative results demonstrated that the neural network with the best performance was able to segment lesions with an average DSC of 0.68 ± 0.15 and HD95 of 4 ± 2 mm. The center of mass of the segmented contours deviated from physician contours by approximately 2 mm on average, and the volume difference was less than 1 cc. The novel network proposed by us achieves the best performance compared to current networks. The addition of CT as input to the neural network contributed to more cases of failure (DSC = 0), and among those cases of DSC > 0, it was shown to produce no statistically significant difference with the use of only PET as input for our proposed method.

Conclusion

Quantitative results demonstrated the feasibility of the deep learning methods in automatically segmenting lesions on 18F-fluciclovine PET/CT images. This indicates the great potential of 18F-fluciclovine PET/CT combined with deep learning for providing a second check in identifying lesions as well as saving time and effort for physicians in contouring.

Clinical Use of a Commercial Artificial Intelligence-Based Software for Autocontouring in Radiation Therapy: Geometric Performance and Dosimetric Impact

PURPOSE

When autocontouring based on artificial intelligence (AI) is used in the radiotherapy (RT) workflow, the contours are reviewed and eventually adjusted by a radiation oncologist before an RT treatment plan is generated, with the purpose of improving dosimetry and reducing both interobserver variability and time for contouring. The purpose of this study was to evaluate the results of application of a commercial AI-based autocontouring for RT, assessing both geometric accuracies and the influence on optimized dose from automatically generated contours after review by human operator.

MATERIALS AND METHODS

A commercial autocontouring system was applied to a retrospective database of 40 patients, of which 20 were treated with radiotherapy for prostate cancer (PCa) and 20 for head and neck cancer (HNC). Contours resulting from AI were compared against AI contours reviewed by human operator and human-only contours using Dice similarity coefficient (DSC), Hausdorff distance (HD), and relative volume difference (RVD). Dosimetric indices such as Dmean, D0.03cc, and normalized plan quality metrics were used to compare dose distributions from RT plans generated from structure sets contoured by humans assisted by AI against plans from manual contours. The reduction in contouring time obtained by using automated tools was also assessed. A Wilcoxon rank sum test was computed to assess the significance of differences. Interobserver variability of the comparison of manual vs. AI-assisted contours was also assessed among two radiation oncologists for PCa.

RESULTS

For PCa, AI-assisted segmentation showed good agreement with expert radiation oncologist structures with average DSC among patients ≥ 0.7 for all structures, and minimal radiation oncology adjustment of structures (DSC of adjusted versus AI structures ≥ 0.91). For HNC, results of comparison between manual and AI contouring varied considerably e.g., 0.77 for oral cavity and 0.11-0.13 for brachial plexus, but again, adjustment was generally minimal (DSC of adjusted against AI contours 0.97 for oral cavity, 0.92-0.93 for brachial plexus). The difference in dose for the target and organs at risk were not statistically significant between human and AI-assisted, with the only exceptions of D0.03cc to the anal canal and Dmean to the brachial plexus. The observed average differences in plan quality for PCa and HNC cases were 8% and 6.7%, respectively. The dose parameter changes due to interobserver variability in PCa were small, with the exception of the anal canal, where large dose variations were observed. The reduction in time required for contouring was 72% for PCa and 84% for HNC.

CONCLUSIONS

When an autocontouring system is used in combination with human review, the time of the RT workflow is significantly reduced without affecting dose distribution and plan quality.

Clinical evaluation of deep learning and atlas-based auto-segmentation for organs at risk delineation

Manual delineation of organs at risk and clinical target volumes is essential in radiotherapy planning. Atlas-based auto-segmentation (ABAS) algorithms have become available and been shown to provide accurate contouring for various anatomical sites. Recently, deep learning auto-segmentation (DL-AS) algorithms have emerged as the state-of-the-art in medical image segmentation. This study aimed to evaluate the effect of auto-segmentation on the clinical workflow for contouring different anatomical sites of cancer, such as head and neck (H&N), breast, abdominal region, and prostate. Patients with H&N, breast, abdominal, and prostate cancer (n = 30 each) were enrolled in the study. Twenty-seven different organs at four sites were evaluated. RayStation was used to apply the ABAS. Siemens AI-Rad Companion Organs RT was used to apply the DL-AS. Evaluations were performed with similarity indices using geometric methods, time-evaluation, and qualitative scoring visual evaluations by radiation oncologists. The DL-AS algorithm was more accurate than ABAS algorithm on geometric indices for half of the structures. The qualitative scoring results of the two algorithms were significantly different, and DL-AS was more accurate on many contours. DL-AS had 41%, 29%, 86%, and 15% shorter edit times in the HnN, breast, abdomen, and prostate groups, respectively, than ABAS. There were no correlations between the geometric indices and visual assessments. The time required to edit the contours was considerably shorter for DL-AS than for ABAS. Auto-segmentation with deep learning could be the first step for clinical workflow optimization in radiotherapy.

A real-time contouring feedback tool for consensus-based contour training

Purpose

Variability in contouring structures of interest for radiotherapy continues to be challenging. Although training can reduce such variability, having radiation oncologists provide feedback can be impractical. We developed a contour training tool to provide real-time feedback to trainees, thereby reducing variability in contouring.

Methods

We developed a novel metric termed localized signed square distance (LSSD) to provide feedback to the trainee on how their contour compares with a reference contour, which is generated real-time by combining trainee contour and multiple expert radiation oncologist contours. Nine trainees performed contour training by using six randomly assigned training cases that included one test case of the heart and left ventricle (LV). The test case was repeated 30 days later to assess retention. The distribution of LSSD maps of the initial contour for the training cases was combined and compared with the distribution of LSSD maps of the final contours for all training cases. The difference in standard deviations from the initial to final LSSD maps, ΔLSSD, was computed both on a per-case basis and for the entire group.

Results

For every training case, statistically significant ΔLSSD were observed for both the heart and LV. When all initial and final LSSD maps were aggregated for the training cases, before training, the mean LSSD ([range], standard deviation) was -0.8 mm ([-37.9, 34.9], 4.2) and 0.3 mm ([-25.1, 32.7], 4.8) for heart and LV, respectively. These were reduced to -0.1 mm ([-16.2, 7.3], 0.8) and 0.1 mm ([-6.6, 8.3], 0.7) for the final LSSD maps during the contour training sessions. For the retention case, the initial and final LSSD maps of the retention case were aggregated and were -1.5 mm ([-22.9, 19.9], 3.4) and -0.2 mm ([-4.5, 1.5], 0.7) for the heart and 1.8 mm ([-16.7, 34.5], 5.1) and 0.2 mm ([-3.9, 1.6],0.7) for the LV.

Conclusions

A tool that uses real-time contouring feedback was developed and successfully used for contour training of nine trainees. In all cases, the utility was able to guide the trainee and ultimately reduce the variability of the trainee's contouring.

IPEM topical report: guidance for the clinical implementation of online treatment monitoring solutions for IMRT/VMAT

This report provides guidance for the implementation of online treatment monitoring (OTM) solutions in radiotherapy (RT), with a focus on modulated treatments. Support is provided covering the implementation process, from identification of an OTM solution to local implementation strategy. Guidance has been developed by a RT special interest group (RTSIG) working party (WP) on behalf of the Institute of Physics and Engineering in Medicine (IPEM). Recommendations within the report are derived from the experience of the WP members (in consultation with manufacturers, vendors and user groups), existing guidance or legislation and a UK survey conducted in 2020 (Stevenset al2021). OTM is an inclusive term representing any system capable of providing a direct or inferred measurement of the delivered dose to a RT patient. Information on each type of OTM is provided but, commensurate with UK demand, guidance is largely influenced byin vivodosimetry methods utilising the electronic portal imager device (EPID). Sections are included on the choice of OTM solutions, acceptance and commissioning methods with recommendations on routine quality control, analytical methods and tolerance setting, clinical introduction and staffing/resource requirements. The guidance aims to give a practical solution to sensitivity and specificity testing. Functionality is provided for the user to introduce known errors into treatment plans for local testing. Receiver operating characteristic analysis is discussed as a tool to performance assess OTM systems. OTM solutions can help verify the correct delivery of radiotherapy treatment. Furthermore, modern systems are increasingly capable of providing clinical decision-making information which can impact the course of a patient's treatment. However, technical limitations persist. It is not within the scope of this guidance to critique each available solution, but the user is encouraged to carefully consider workflow and engage with manufacturers in resolving compatibility issues.

Optimized Atlas-Based Auto-Segmentation of Bony Structures from Whole-Body Computed Tomography

PURPOSE

To develop and test a method for fully automated segmentation of bony structures from whole-body computed tomography (CT) and evaluate its performance compared with manual segmentation.

METHODS AND MATERIALS

We developed a workflow for automatic whole-body bone segmentation using atlas-based segmentation (ABS) method with a postprocessing module (ABSPP) in MIM MAESTRO software. Fifty-two CT scans comprised the training set to build the atlas library, and 29 CT scans comprised the test set. To validate the workflow, we compared Dice similarity coefficient (DSC), mean distance to agreement, and relative volume errors between ABSPP and ABS with no postprocessing (ABSNPP) with manual segmentation as the reference (gold standard).

RESULTS

The ABSPP method resulted in significantly improved segmentation accuracy (DSC range, 0.85-0.98) compared with the ABSNPP method (DSC range, 0.55-0.87; P < .001). Mean distance to agreement results also indicated high agreement between ABSPP and manual reference delineations (range, 0.11-1.56 mm), which was significantly improved compared with ABSNPP (range, 1.00-2.34 mm) for the majority of tested bony structures. Relative volume errors were also significantly lower for ABSPP compared with ABSNPP for most bony structures.

CONCLUSIONS

We developed a fully automated MIM workflow for bony structure segmentation from whole-body CT, which exhibited high accuracy compared with manual delineation. The integrated postprocessing module significantly improved workflow performance.

Clinical evaluation of atlas-based auto-segmentation in breast and nodal radiotherapy

OBJECTIVES

Accurate contouring of anatomical structures allows for high-precision radiotherapy planning, targeting the dose at treatment volumes and avoiding organs at risk. Manual contouring is time-consuming with significant user variability, whereas auto-segmentation (AS) has proven efficiency benefits but requires editing before treatment planning. This study investigated whether atlas-based AS (ABAS) accuracy improves with template atlas group size and character-specific atlas and test case selection.

METHODS AND MATERIALS

One clinician retrospectively contoured the breast, nodes, lung, heart, and brachial plexus on 100 CT scans, adhering to peer-reviewed guidelines. Atlases were clustered in group sizes, treatment positions, chest wall separations, and ASs created with Mirada software. The similarity of ASs compared to reference contours was described by the Jaccard similarity coefficient (JSC) and centroid distance variance (CDV).

RESULTS

Across group sizes, for all structures combined, the mean JSC was 0.6 (SD 0.3, p = .999). Across atlas-specific groups, 0.6 (SD 0.3, p = 1.000). The correlation between JSC and structure volume was weak in both scenarios (adjusted R2-0.007 and 0.185).Mean CDV was similar across groups but varied up to 1.2 cm for specific structures.

CONCLUSIONS

Character-specific atlas groups and test case selection did not improve accuracy outcomes. High-quality ASs were obtained from groups containing as few as ten atlases, subsequently simplifying the application of ABAS. CDV measures indicating auto-segmentation variations on the x, y, and z axes can be utilised to decide on the clinical relevance of variations and reduce AS editing.

ADVANCES IN KNOWLEDGE

High-quality ABASs can be obtained from as few as ten template atlases.Atlas and test case selection do not improve AS accuracy.Unlike well-known quantitative similarity indices, volume displacement metrics provide information on the location of segmentation variations, helping assessment of the clinical relevance of variations and reducing clinician editing. Volume displacement metrics combined with the qualitative measure of clinician assessment could reduce user variability.

ASSET: Auto-Segmentation of the Seventeen SEgments for Ventricular Tachycardia Ablation in Radiation Therapy

There has been a recent effort to treat high-risk ventricular tachycardia (VT) patients through radio-ablation. However, manual segmentation of the VT target is complex and time-consuming. This work introduces ASSET, or Auto-segmentation of the Seventeen SEgments for Tachycardia ablation, to aid in radiation therapy (RT) planning. ASSET was retrospectively applied to CTs for 26 thoracic RT patients (13 undergoing VT ablation). The physician-defined parasternal long-axis of the left ventricle (LV) and the axes generated from principal component analysis (PCA) were compared using mean distance to agreement (MDA) and angle of separation. The manually selected right ventricle insertion point and LVs were used to apply the ASSET model to automatically generate the 17 segments of the LV myocardium (LVM). Physician-defined parasternal long-axis differed from PCA by 1.2 ± 0.3 mm MDA and 6.9 ± 0.7 degrees. Segments differed by 0.69 ± 0.29 mm MDA and 0.89 ± 0.03 Dice similarity coefficient. Running ASSET takes <5 min where manual segmentation took >2 h/patient. Agreement between ASSET and expert contours was comparable to inter-observer variability. Qualitative scoring conducted by three experts revealed automatically generated segmentations were clinically useable as-is. ASSET offers efficient and reliable automatic segmentations for the 17 segments of the LVM for target generation in RT planning.

A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy

Purpose/objectives

Auto-segmentation with artificial intelligence (AI) offers an opportunity to reduce inter- and intra-observer variability in contouring, to improve the quality of contours, as well as to reduce the time taken to conduct this manual task. In this work we benchmark the AI auto-segmentation contours produced by five commercial vendors against a common dataset.

Methods and materials

The organ at risk (OAR) contours generated by five commercial AI auto-segmentation solutions (Mirada (Mir), MVision (MV), Radformation (Rad), RayStation (Ray) and TheraPanacea (Ther)) were compared to manually-drawn expert contours from 20 breast, 20 head and neck, 20 lung and 20 prostate patients. Comparisons were made using geometric similarity metrics including volumetric and surface Dice similarity coefficient (vDSC and sDSC), Hausdorff distance (HD) and Added Path Length (APL). To assess the time saved, the time taken to manually draw the expert contours, as well as the time to correct the AI contours, were recorded.

Results

There are differences in the number of CT contours offered by each AI auto-segmentation solution at the time of the study (Mir 99; MV 143; Rad 83; Ray 67; Ther 86), with all offering contours of some lymph node levels as well as OARs. Averaged across all structures, the median vDSCs were good for all systems and compared favorably with existing literature: Mir 0.82; MV 0.88; Rad 0.86; Ray 0.87; Ther 0.88. All systems offer substantial time savings, ranging between: breast 14-20 mins; head and neck 74-93 mins; lung 20-26 mins; prostate 35-42 mins. The time saved, averaged across all structures, was similar for all systems: Mir 39.8 mins; MV 43.6 mins; Rad 36.6 min; Ray 43.2 mins; Ther 45.2 mins.

Conclusions

All five commercial AI auto-segmentation solutions evaluated in this work offer high quality contours in significantly reduced time compared to manual contouring, and could be used to render the radiotherapy workflow more efficient and standardized.

Deep learning-based dominant index lesion segmentation for MR-guided radiation therapy of prostate cancer

BACKGROUND

Dose escalation radiotherapy enables increased control of prostate cancer (PCa) but requires segmentation of dominant index lesions (DIL). This motivates the development of automated methods for fast, accurate, and consistent segmentation of PCa DIL.

PURPOSE

To construct and validate a model for deep-learning-based automatic segmentation of PCa DIL defined by Gleason score (GS) ≥3+4 from MR images applied to MR-guided radiation therapy. Validate generalizability of constructed models across scanner and acquisition differences.

METHODS

Five deep-learning networks were evaluated on apparent diffusion coefficient (ADC) MRI from 500 lesions in 365 patients arising from internal training Dataset 1 (156 lesions in 125 patients, 1.5Tesla GE MR with endorectal coil), testing using Dataset 1 (35 lesions in 26 patients), external ProstateX Dataset 2 (299 lesions in 204 patients, 3Tesla Siemens MR), and internal inter-rater Dataset 3 (10 lesions in 10 patients, 3Tesla Philips MR). The five networks include: multiple resolution residually connected network (MRRN) and MRRN regularized in training with deep supervision implemented into the last convolutional block (MRRN-DS), Unet, Unet++, ResUnet, and fast panoptic segmentation (FPSnet) as well as fast panoptic segmentation with smoothed labels (FPSnet-SL). Models were evaluated by volumetric DIL segmentation accuracy using Dice similarity coefficient (DSC) and the balanced F1 measure of detection accuracy, as a function of lesion aggressiveness and size (Dataset 1 and 2), and accuracy with respect to two-raters (on Dataset 3). Upon acceptance for publication segmentation models will be made available in an open-source GitHub repository.

RESULTS

In general, MRRN-DS more accurately segmented tumors than other methods on the testing datasets. MRRN-DS significantly outperformed ResUnet in Dataset2 (DSC of 0.54 vs. 0.44, p < 0.001) and the Unet++ in Dataset3 (DSC of 0.45 vs. p = 0.04). FPSnet-SL was similarly accurate as MRRN-DS in Dataset2 (p = 0.30), but MRRN-DS significantly outperformed FPSnet and FPSnet-SL in both Dataset1 (0.60 vs. 0.51 [p = 0.01] and 0.54 [p = 0.049] respectively) and Dataset3 (0.45 vs. 0.06 [p = 0.002] and 0.24 [p = 0.004] respectively). Finally, MRRN-DS produced slightly higher agreement with experienced radiologist than two radiologists in Dataset 3 (DSC of 0.45 vs. 0.41).

CONCLUSIONS

MRRN-DS was generalizable to different MR testing datasets acquired using different scanners. It produced slightly higher agreement with an experienced radiologist than that between two radiologists. Finally, MRRN-DS more accurately segmented aggressive lesions, which are generally candidates for radiative dose ablation.

Deep learning auto-segmentation on multi-sequence magnetic resonance images for upper abdominal organs

Introduction

Multi-sequence multi-parameter MRIs are often used to define targets and/or organs at risk (OAR) in radiation therapy (RT) planning. Deep learning has so far focused on developing auto-segmentation models based on a single MRI sequence. The purpose of this work is to develop a multi-sequence deep learning based auto-segmentation (mS-DLAS) based on multi-sequence abdominal MRIs.

Materials and methods

Using a previously developed 3DResUnet network, a mS-DLAS model using 4 T1 and T2 weighted MRI acquired during routine RT simulation for 71 cases with abdominal tumors was trained and tested. Strategies including data pre-processing, Z-normalization approach, and data augmentation were employed. Additional 2 sequence specific T1 weighted (T1-M) and T2 weighted (T2-M) models were trained to evaluate performance of sequence-specific DLAS. Performance of all models was quantitatively evaluated using 6 surface and volumetric accuracy metrics.

Results

The developed DLAS models were able to generate reasonable contours of 12 upper abdomen organs within 21 seconds for each testing case. The 3D average values of dice similarity coefficient (DSC), mean distance to agreement (MDA mm), 95 percentile Hausdorff distance (HD95% mm), percent volume difference (PVD), surface DSC (sDSC), and relative added path length (rAPL mm/cc) over all organs were 0.87, 1.79, 7.43, -8.95, 0.82, and 12.25, respectively, for mS-DLAS model. Collectively, 71% of the auto-segmented contours by the three models had relatively high quality. Additionally, the obtained mS-DLAS successfully segmented 9 out of 16 MRI sequences that were not used in the model training.

Conclusion

We have developed an MRI-based mS-DLAS model for auto-segmenting of upper abdominal organs on MRI. Multi-sequence segmentation is desirable in routine clinical practice of RT for accurate organ and target delineation, particularly for abdominal tumors. Our work will act as a stepping stone for acquiring fast and accurate segmentation on multi-contrast MRI and make way for MR only guided radiation therapy.

Contour-aware network with class-wise convolutions for 3D abdominal multi-organ segmentation

Accurate delineation of multiple organs is a critical process for various medical procedures, which could be operator-dependent and time-consuming. Existing organ segmentation methods, which were mainly inspired by natural image analysis techniques, might not fully exploit the traits of the multi-organ segmentation task and could not accurately segment the organs with various shapes and sizes simultaneously. In this work, the characteristics of multi-organ segmentation are considered: the global count, position and scale of organs are generally predictable, while their local shape and appearance are volatile. Thus, we supplement the region segmentation backbone with a contour localization task to increase the certainty along delicate boundaries. Meantime, each organ has exclusive anatomical traits, which motivates us to deal with class variability with class-wise convolutions to highlight organ-specific features and suppress irrelevant responses at different field-of-views. To validate our method with adequate amounts of patients and organs, we constructed a multi-center dataset, which contains 110 3D CT scans with 24,528 axial slices, and provided voxel-level manual segmentations of 14 abdominal organs, which adds up to 1,532 3D structures in total. Extensive ablation and visualization studies on it validate the effectiveness of the proposed method. Quantitative analysis shows that we achieve state-of-the-art performance for most abdominal organs, and obtain 3.63 mm 95% Hausdorff Distance and 83.32% Dice Similarity Coefficient on an average.

Practical and technical key challenges in head and neck adaptive radiotherapy: The GORTEC point of view

Anatomical variations occur during head and neck (H&N) radiotherapy (RT) treatment. These variations may result in underdosage to the target volume or overdosage to the organ at risk. Replanning during the treatment course can be triggered to overcome this issue. Due to technological, methodological and clinical evolutions, tools for adaptive RT (ART) are becoming increasingly sophisticated. The aim of this paper is to give an overview of the key steps of an H&N ART workflow and tools from the point of view of a group of French-speaking medical physicists and physicians (from GORTEC). Focuses are made on image registration, segmentation, estimation of the delivered dose of the day, workflow and quality assurance for an implementation of H&N offline and online ART. Practical recommendations are given to assist physicians and medical physicists in a clinical workflow.

Automated detection, delineation and quantification of whole-body bone metastasis using FDG-PET/CT images

Non-small cell lung cancer (NSCLC) patients with the metastatic spread of disease to the bone have high morbidity and mortality. Stereotactic ablative body radiotherapy increases the progression free survival and overall survival of these patients with oligometastases. FDG-PET/CT, a functional imaging technique combining positron emission tomography (PET) with 18 F-fluorodeoxyglucose (FDG) and computer tomography (CT) provides improved staging and identification of treatment response. It is also associated with reduction in size of the radiotherapy tumour volume delineation compared with CT based contouring in radiotherapy, thus allowing for dose escalation to the target volume with lower doses to the surrounding organs at risk. FDG-PET/CT is increasingly being used for the clinical management of NSCLC patients undergoing radiotherapy and has shown high sensitivity and specificity for the detection of bone metastases in these patients. Here, we present a software tool for detection, delineation and quantification of bone metastases using FDG-PET/CT images. The tool extracts standardised uptake values (SUV) from FDG-PET images for auto-segmentation of bone lesions and calculates volume of each lesion and associated mean and maximum SUV. The tool also allows automatic statistical validation of the auto-segmented bone lesions against the manual contours of a radiation oncologist. A retrospective review of FDG-PET/CT scans of more than 30 candidate NSCLC patients was performed and nine patients with one or more metastatic bone lesions were selected for the present study. The SUV threshold prediction model was designed by splitting the cohort of patients into a subset of 'development' and 'validation' cohorts. The development cohort yielded an optimum SUV threshold of 3.0 for automatic detection of bone metastases using FDG-PET/CT images. The validity of the derived optimum SUV threshold on the validation cohort demonstrated that auto-segmented and manually contoured bone lesions showed strong concordance for volume of bone lesion (r = 0.993) and number of detected lesions (r = 0.996). The tool has various applications in radiotherapy, including but not limited to studies determining optimum SUV threshold for accurate and standardised delineation of bone lesions and in scientific studies utilising large patient populations for instance for investigation of the number of metastatic lesions that can be treated safety with an ablative dose of radiotherapy without exceeding the normal tissue toxicity.

Assessment of Optimizers and their Performance in Autosegmenting Lung Tumors

Purpose

Optimizers are widely utilized across various domains to enhance desired outcomes by either maximizing or minimizing objective functions. In the context of deep learning, they help to minimize the loss function and improve model's performance. This study aims to evaluate the accuracy of different optimizers employed for autosegmentation of non-small cell lung cancer (NSCLC) target volumes on thoracic computed tomography images utilized in oncology.

Materials and Methods

The study utilized 112 patients, comprising 92 patients from "The Cancer Imaging Archive" (TCIA) and 20 of our local clinical patients, to evaluate the efficacy of various optimizers. The gross tumor volume was selected as the foreground mask for training and testing the models. Of the 92 TCIA patients, 57 were used for training and validation, and the remaining 35 for testing using nnU-Net. The performance of the final model was further evaluated on the 20 local clinical patient datasets. Six different optimizers, namely AdaDelta, AdaGrad, Adam, NAdam, RMSprop, and stochastic gradient descent (SGD), were investigated. To assess the agreement between the predicted volume and the ground truth, several metrics including Dice similarity coefficient (DSC), Jaccard index, sensitivity, precision, Hausdorff distance (HD), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASSD) were utilized.

Results

The DSC values for AdaDelta, AdaGrad, Adam, NAdam, RMSprop, and SGD were 0.75, 0.84, 0.85, 0.84, 0.83, and 0.81, respectively, for the TCIA test data. However, when the model trained on TCIA datasets was applied to the clinical datasets, the DSC, HD, HD95, and ASSD metrics showed a statistically significant decrease in performance compared to the TCIA test datasets, indicating the presence of image and/or mask heterogeneity between the data sources.

Conclusion

The choice of optimizer in deep learning is a critical factor that can significantly impact the performance of autosegmentation models. However, it is worth noting that the behavior of optimizers may vary when applied to new clinical datasets, which can lead to changes in models' performance. Therefore, selecting the appropriate optimizer for a specific task is essential to ensure optimal performance and generalizability of the model to different datasets.

Educative Impact of Automatic Delineation Applied to Head and Neck Cancer Patients on Radiation Oncology Residents

To evaluate the educational impact on radiation oncology residents in training when introducing an automatic segmentation software in head and neck cancer patients regarding organs at risk (OARs) and prophylactic cervical lymph node level (LNL) volumes. Two cases treated by exclusive intensity-modulated radiotherapy were delineated by an expert radiation oncologist and were considered as reference. Then, these cases were delineated by residents divided into two groups: group 1 (control group), experienced residents delineating manually, group 2 (experimental group), young residents on their first rotation trained with automatic delineation, delineating manually first (M -) and then after using the automatic system (M +). The delineation accuracy was assessed using the Overlap Volume (OV). Regarding the OARs, mean OV was 0.62 (SD = 0.05) for group 1, 0.56 (SD = 0.04) for group 2 M - , and 0.61 (SD = 0.03) for group 2 M + . Mean OV was higher in group 1 compared to group 2 M - (p = 0.01). There was no OV difference between group 1 and group 2 M + (p = 0.67). Mean OV was higher in the group 2 M + compared to group 2 M - (p < 0.003). Regarding LNL, mean OV was 0.53 (SD = 0.06) in group 1, 0.54 (SD = 0.03) in group 2 M - , and 0.58 (SD = 0.04) in group 2 M + . Mean OV was higher in group 2 M + for 11 of the 12 analysed structures compared to group 2 M - (p = 0.016). Prior use of the automatic delineation software reduced the average contouring time per case by 34 to 40%. Prior use of atlas-based automatic segmentation reduces the delineation duration, and provides reliable OARs and LNL delineations.

Analytic calculations and synthetic shapes for validation of quantitative contour comparison software

A high level of variability in reported values was observed in a recent survey of contour similarity measures (CSMs) calculation tools. Such variations in the output measurements prevent meaningful comparison between studies. The purpose of this study was to develop a dataset with analytically calculated gold standard values to facilitate standardization and ensure accuracy of CSM implementations. The dataset was generated in the Digital Imaging and Communications in Medicine (DICOM) format. Both the dataset and the software used for its generation are made publicly available to encourage robust testing of CSM implementations for accuracy, improving consistency between different implementations.

Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.

Segmentation of multiple Organs-at-Risk associated with brain tumors based on coarse-to-fine stratified networks

BACKGROUND

Delineation of Organs-at-Risks (OARs) is an important step in radiotherapy treatment planning. As manual delineation is time-consuming, labor-intensive and affected by inter- and intra-observer variability, a robust and efficient automatic segmentation algorithm is highly desirable for improving the efficiency and repeatability of OAR delineation.

PURPOSE

Automatic segmentation of OARs in medical images is challenged by low contrast, various shapes and imbalanced sizes of different organs. We aim to overcome these challenges and develop a high-performance method for automatic segmentation of 10 OARs required in radiotherapy planning for brain tumors.

METHODS

A novel two-stage segmentation framework is proposed, where a coarse and simultaneous localization of all the target organs is obtained in the first stage, and a fine segmentation is achieved for each organ, respectively, in the second stage. To deal with organs with various sizes and shapes, a stratified segmentation strategy is proposed, where a High- and Low-Resolution Residual Network (HLRNet) that consists of a multiresolution branch and a high-resolution branch is introduced to segment medium-sized organs, and a High-Resolution Residual Network (HRRNet) is used to segment small organs. In addition, a label fusion strategy is proposed to better deal with symmetric pairs of organs like the left and right cochleas and lacrimal glands.

RESULTS

Our method was validated on the dataset of MICCAI ABCs 2020 challenge for OAR segmentation. It obtained an average Dice of 75.8% for 10 OARs, and significantly outperformed several state-of-the-art models including nnU-Net (71.6%) and FocusNet (72.4%). Our proposed HLRNet and HRRNet improved the segmentation accuracy for medium-sized and small organs, respectively. The label fusion strategy led to higher accuracy for symmetric pairs of organs.

CONCLUSIONS

Our proposed method is effective for the segmentation of OARs of brain tumors, with a better performance than existing methods, especially on medium-sized and small organs. It has a potential for improving the efficiency of radiotherapy planning with high segmentation accuracy.

Automatic segmentation of prostate MRI based on 3D pyramid pooling Unet

PURPOSE

Automatic segmentation of prostate magnetic resonance (MR) images is crucial for the diagnosis, evaluation, and prognosis of prostate diseases (including prostate cancer). In recent years, the mainstream segmentation method for the prostate has been converted to convolutional neural networks. However, owing to the complexity of the tissue structure in MR images and the limitations of existing methods in spatial context modeling, the segmentation performance should be improved further.

METHODS

In this study, we proposed a novel 3D pyramid pool Unet that benefits from the pyramid pooling structure embedded in the skip connection (SC) and the deep supervision (DS) in the up-sampling of the 3D Unet. The parallel SC of the conventional 3D Unet network causes low-resolution information to be sent to the feature map repeatedly, resulting in blurred image features. To overcome the shortcomings of the conventional 3D Unet, we merge each decoder layer with the feature map of the same scale as the encoder and the smaller scale feature map of the pyramid pooling encoder. This SC combines the low-level details and high-level semantics at two different levels of feature maps. In addition, pyramid pooling performs multifaceted feature extraction on each image behind the convolutional layer, and DS learns hierarchical representations from comprehensive aggregated feature maps, which can improve the accuracy of the task.

RESULTS

Experiments on 3D prostate MR images of 78 patients demonstrated that our results were highly correlated with expert manual segmentation. The average relative volume difference and Dice similarity coefficient of the prostate volume area were 2.32% and 91.03%, respectively.

CONCLUSION

Quantitative experiments demonstrate that, compared with other methods, the results of our method are highly consistent with the expert manual segmentation.

Improved accuracy of auto-segmentation of organs at risk in radiotherapy planning for nasopharyngeal carcinoma based on fully convolutional neural network deep learning

OBJECTIVE

We examined a modified encoder-decoder architecture-based fully convolutional neural network, OrganNet, for simultaneous auto-segmentation of 24 organs at risk (OARs) in the head and neck, followed by validation tests and evaluation of clinical application.

MATERIALS AND METHODS

Computed tomography (CT) images from 310 radiotherapy plans were used as the experimental data set, of which 260 and 50 were used as the training and test sets, respectively. An improved U-Net architecture was established by introducing a batch normalization layer, residual squeeze-and-excitation layer, and unique organ-specific loss function for deep learning training. The performance of the trained network model was evaluated by comparing the manual-delineation and the STAPLE contour of 10 physicians from different centers.

RESULTS

Our model achieved good segmentation in all 24 OARs in nasopharyngeal cancer radiotherapy plan CT images, with an average Dice similarity coefficient of 83.75%. Specifically, the mean Dice coefficients in large-volume organs (brainstem, spinal cord, left/right parotid glands, left/right temporal lobes, and left/right mandibles) were 84.97% - 95.00%, and in small-volume organs (pituitary, lens, optic nerve, and optic chiasma) were 55.46% - 91.56%. respectively. Using the STAPLE contours as standard contour, the OrganNet achieved comparable or better DICE in organ segmentation then that of the manual-delineation as well.

CONCLUSION

The established OrganNet enables simultaneous automatic segmentation of multiple targets on CT images of the head and neck radiotherapy plans, effectively improves the accuracy of U-Net based segmentation for OARs, especially for small-volume organs.

Stress-testing pelvic autosegmentation algorithms using anatomical edge cases

Commercial autosegmentation has entered clinical use, however real-world performance may suffer in certain cases. We aimed to assess the influence of anatomic variants on performance. We identified 112 prostate cancer patients with anatomic variations (edge cases). Pelvic anatomy was autosegmented using three commercial tools. To evaluate performance, Dice similarity coefficients, and mean surface and 95% Hausdorff distances were calculated versus clinician-delineated references. Deep learning autosegmentation outperformed atlas-based and model-based methods. However, edge case performance was lower versus the normal cohort (0.12 mean DSC reduction). Anatomic variation presents challenges to commercial autosegmentation.

An uncertainty-aware deep learning architecture with outlier mitigation for prostate gland segmentation in radiotherapy treatment planning

PURPOSE

Task automation is essential for efficient and consistent image segmentation in radiation oncology. We report on a deep learning architecture, comprising a U-Net and a variational autoencoder (VAE) for automatic contouring of the prostate gland incorporating interobserver variation for radiotherapy treatment planning. The U-Net/VAE generates an ensemble set of segmentations for each image CT slice. A novel outlier mitigation (OM) technique was implemented to enhance the model segmentation accuracy.

METHODS

The primary source dataset (source_prim) consisted of 19 200 CT slices (from 300 patient planning CT image datasets) with manually contoured prostate glands. A smaller secondary source dataset (source_sec) comprised 640 CT slices (from 10 patient CT datasets), where prostate glands were segmented by 5 independent physicians on each dataset to account for interobserver variability. Data augmentation via random rotation (<5 degrees), cropping, and horizontal flipping was applied to each dataset to increase sample size by a factor of 100. A probabilistic hierarchical U-Net with VAE was implemented and pretrained using the augmented source_prim dataset for 30 epochs. Model parameters of the U-Net/VAE were fine-tuned using the augmented source_sec dataset for 100 epochs. After the first round of training, outlier contours in the training dataset were automatically detected and replaced by the most accurate contours (based on Dice similarity coefficient, DSC) generated by the model. The U-Net/OM-VAE was retrained using the revised training dataset. Metrics for comparison included DSC, Hausdorff distance (HD, mm), normalized cross-correlation (NCC) coefficient, and center-of-mass (COM) distance (mm).

RESULTS

Results for U-Net/OM-VAE with outliers replaced in the training dataset versus U-Net/VAE without OM were as follows: DSC = 0.82 ± 0.01 versus 0.80 ± 0.02 (p = 0.019), HD = 9.18 ± 1.22 versus 10.18 ± 1.35 mm (p = 0.043), NCC = 0.59 ± 0.07 versus 0.62 ± 0.06, and COM = 3.36 ± 0.81 versus 4.77 ± 0.96 mm over the average of 15 contours. For the average of 15 highest accuracy contours, values were as follows: DSC = 0.90 ± 0.02 versus 0.85 ± 0.02, HD = 5.47 ± 0.02 versus 7.54 ± 1.36 mm, and COM = 1.03 ± 0.58 versus 1.46 ± 0.68 mm (p < 0.03 for all metrics). Results for the U-Net/OM-VAE with outliers removed were as follows: DSC = 0.78 ± 0.01, HD = 10.65 ± 1.95 mm, NCC = 0.46 ± 0.10, COM = 4.17 ± 0.79 mm for the average of 15 contours, and DSC = 0.88 ± 0.02, HD = 7.00 ± 1.17 mm, COM = 1.58 ± 0.63 mm for the average of 15 highest accuracy contours. All metrics for U-Net/VAE trained on the source_prim and source_sec datasets via pretraining, followed by fine-tuning, show statistically significant improvement over that trained on the source_sec dataset only. Finally, all metrics for U-Net/VAE with or without OM showed statistically significant improvement over those for the standard U-Net.

CONCLUSIONS

A VAE combined with a hierarchical U-Net and an OM strategy (U-Net/OM-VAE) demonstrates promise toward capturing interobserver variability and produces accurate prostate auto-contours for radiotherapy planning. The availability of multiple contours for each CT slice enables clinicians to determine trade-offs in selecting the "best fitting" contour on each CT slice. Mitigation of outlier contours in the training dataset improves prediction accuracy, but one must be wary of reduction in variability in the training dataset.

Research on automatic classification technology of kidney tumor and normal kidney tissue based on computed tomography radiomics

Purpose

By using a radiomics-based approach, multiple radiomics features can be extracted from regions of interest in computed tomography (CT) images, which may be applied to automatically classify kidney tumors and normal kidney tissues. The study proposes a method based on CT radiomics and aims to use extracted radiomics features to automatically classify of kidney tumors and normal kidney tissues and to establish an automatic classification model.

Methods

CT data were retrieved from the 2019 Kidney and Kidney Tumor Segmentation Challenge (KiTS19) in The Cancer Imaging Archive (TCIA) open access database. Arterial phase-enhanced CT images from 210 cases were used to establish an automatic classification model. These CT images of patients were randomly divided into training (168 cases) and test (42 cases) sets. Furthermore, the radiomics features of gross tumor volume (GTV) and normal kidney tissues in the training set were extracted and screened, and a binary logistic regression model was established. For the test set, the radiomic features and cutoff value of P were consistent with the training set.

Results

Three radiomics features were selected to establish the binary logistic regression model. The accuracy (ACC), sensitivity (SENS), specificity (SPEC), area under the curve (AUC), and Youden index of the training and test sets based on the CT radiomics classification model were all higher than 0.85.

Conclusion

The automatic classification model of kidney tumors and normal kidney tissues based on CT radiomics exhibited good classification ability. Kidney tumors could be distinguished from normal kidney tissues. This study may complement automated tumor delineation techniques and warrants further research.

The future of MRI in radiation therapy: Challenges and opportunities for the MR community

Radiation therapy is a major component of cancer treatment pathways worldwide. The main aim of this treatment is to achieve tumor control through the delivery of ionizing radiation while preserving healthy tissues for minimal radiation toxicity. Because radiation therapy relies on accurate localization of the target and surrounding tissues, imaging plays a crucial role throughout the treatment chain. In the treatment planning phase, radiological images are essential for defining target volumes and organs-at-risk, as well as providing elemental composition (e.g., electron density) information for radiation dose calculations. At treatment, onboard imaging informs patient setup and could be used to guide radiation dose placement for sites affected by motion. Imaging is also an important tool for treatment response assessment and treatment plan adaptation. MRI, with its excellent soft tissue contrast and capacity to probe functional tissue properties, holds great untapped potential for transforming treatment paradigms in radiation therapy. The MR in Radiation Therapy ISMRM Study Group was established to provide a forum within the MR community to discuss the unmet needs and fuel opportunities for further advancement of MRI for radiation therapy applications. During the summer of 2021, the study group organized its first virtual workshop, attended by a diverse international group of clinicians, scientists, and clinical physicists, to explore our predictions for the future of MRI in radiation therapy for the next 25 years. This article reviews the main findings from the event and considers the opportunities and challenges of reaching our vision for the future in this expanding field.

Artificial intelligence for prediction of response to cancer immunotherapy

Artificial intelligence (AI) indicates the application of machines to imitate intelligent behaviors for solving complex tasks with minimal human intervention, including machine learning and deep learning. The use of AI in medicine improves health-care systems in multiple areas such as diagnostic confirmation, risk stratification, analysis, prognosis prediction, treatment surveillance, and virtual health support, which has considerable potential to revolutionize and reshape medicine. In terms of immunotherapy, AI has been applied to unlock underlying immune signatures to associate with responses to immunotherapy indirectly as well as predict responses to immunotherapy responses directly. The AI-based analysis of high-throughput sequences and medical images can provide useful information for management of cancer immunotherapy considering the excellent abilities in selecting appropriate subjects, improving therapeutic regimens, and predicting individualized prognosis. In present review, we aim to evaluate a broad framework about AI-based computational approaches for prediction of response to cancer immunotherapy on both indirect and direct manners. Furthermore, we summarize our perspectives about challenges and opportunities of further AI applications on cancer immunotherapy relating to clinical practicability.

Comparison of atlas-based and deep learning methods for organs at risk delineation on head-and-neck CT images using an automated treatment planning system

BACKGROUND AND PURPOSE

To investigate the performance of head-and-neck (HN) organs-at-risk (OAR) automatic segmentation (AS) using four atlas-based (ABAS) and two deep learning (DL) solutions.

MATERIAL AND METHODS

All patients underwent iodine contrast-enhanced planning CT. Fourteen OAR were manually delineated. DL.1 and DL.2 solutions were trained with 63 mono-centric patients and > 1000 multi-centric patients, respectively. Ten and 15 patients with varied anatomies were selected for the atlas library and for testing, respectively. The evaluation was based on geometric indices (DICE coefficient and 95th percentile-Hausdorff Distance (HD_95%)), time needed for manual corrections and clinical dosimetric endpoints obtained using automated treatment planning.

RESULTS

Both DICE and HD_95% results indicated that DL algorithms generally performed better compared with ABAS algorithms for automatic segmentation of HN OAR. However, the hybrid-ABAS (ABAS.3) algorithm sometimes provided the highest agreement to the reference contours compared with the 2 DL. Compared with DL.2 and ABAS.3, DL.1 contours were the fastest to correct. For the 3 solutions, the differences in dose distributions obtained using AS contours and AS + manually corrected contours were not statistically significant. High dose differences could be observed when OAR contours were at short distances to the targets. However, this was not always interrelated.

CONCLUSION

DL methods generally showed higher delineation accuracy compared with ABAS methods for AS segmentation of HN OAR. Most ABAS contours had high conformity to the reference but were more time consuming than DL algorithms, especially when considering the computing time and the time spent on manual corrections.

Towards automated organs at risk and target volumes contouring: Defining precision radiation therapy in the modern era

Precision radiotherapy is a critical and indispensable cancer treatment means in the modern clinical workflow with the goal of achieving "quality-up and cost-down" in patient care. The challenge of this therapy lies in developing computerized clinical-assistant solutions with precision, automation, and reproducibility built-in to deliver it at scale. In this work, we provide a comprehensive yet ongoing, incomplete survey of and discussions on the recent progress of utilizing advanced deep learning, semantic organ parsing, multimodal imaging fusion, neural architecture search and medical image analytical techniques to address four corner-stone problems or sub-problems required by all precision radiotherapy workflows, namely, organs at risk (OARs) segmentation, gross tumor volume (GTV) segmentation, metastasized lymph node (LN) detection, and clinical tumor volume (CTV) segmentation. Without loss of generality, we mainly focus on using esophageal and head-and-neck cancers as examples, but the methods can be extrapolated to other types of cancers. High-precision, automated and highly reproducible OAR/GTV/LN/CTV auto-delineation techniques have demonstrated their effectiveness in reducing the inter-practitioner variabilities and the time cost to permit rapid treatment planning and adaptive replanning for the benefit of patients. Through the presentation of the achievements and limitations of these techniques in this review, we hope to encourage more collective multidisciplinary precision radiotherapy workflows to transpire.

A deep image-to-image network organ segmentation algorithm for radiation treatment planning: principles and evaluation

Background

We describe and evaluate a deep network algorithm which automatically contours organs at risk in the thorax and pelvis on computed tomography (CT) images for radiation treatment planning.

Methods

The algorithm identifies the region of interest (ROI) automatically by detecting anatomical landmarks around the specific organs using a deep reinforcement learning technique. The segmentation is restricted to this ROI and performed by a deep image-to-image network (DI2IN) based on a convolutional encoder-decoder architecture combined with multi-level feature concatenation. The algorithm is commercially available in the medical products “syngo.via RT Image Suite VB50” and “AI-Rad Companion Organs RT VA20” (Siemens Healthineers). For evaluation, thoracic CT images of 237 patients and pelvic CT images of 102 patients were manually contoured following the Radiation Therapy Oncology Group (RTOG) guidelines and compared to the DI2IN results using metrics for volume, overlap and distance, e.g., Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD₉₅). The contours were also compared visually slice by slice.

Results

We observed high correlations between automatic and manual contours. The best results were obtained for the lungs (DSC 0.97, HD₉₅ 2.7 mm/2.9 mm for left/right lung), followed by heart (DSC 0.92, HD₉₅ 4.4 mm), bladder (DSC 0.88, HD₉₅ 6.7 mm) and rectum (DSC 0.79, HD₉₅ 10.8 mm). Visual inspection showed excellent agreements with some exceptions for heart and rectum.

Conclusions

The DI2IN algorithm automatically generated contours for organs at risk close to those by a human expert, making the contouring step in radiation treatment planning simpler and faster. Few cases still required manual corrections, mainly for heart and rectum.

Clinical validation of a semi‐automated segmentation algorithm for target volume definition on planning CT and CBCT in stereotactic body radiotherapy ( SBRT ) for peripheral lung lesions

INTRODUCTION

Stereotactic body radiotherapy (SBRT) is an ablative method for lung malignancies. Here, the definition of the gross target volume (GTV) is subject to interobserver variation. In this study, we aimed to evaluate the interobserver variability during SBRT and its dosimetric impact, as well as to introduce a semi-automated delineation tool for both planning computer tomography (P-CT) and cone beam CT (CBCT) to help to standardise GTV delineation and adaptive volume-change registration.

METHODS

The interobserver variation of GTV manual contours from five physicians was analysed in 15 patients after lung SBRT on free breathing (FB) P-CT (n = 15) and CBCT (n = 90) before and after each fraction. The dosimetric impact from interobserver variations of GTV based on the original treatment plan was analysed. Next, the accuracy of an in-house easy-to-use semi-automated-segmentation algorithm for pulmonary lesions was compared with gold standard contours in FB P-CT and CBCT, as well as 4D P-CT of additional 10 patients.

RESULTS

The interobserver variability in manual contours resulted in violations of dose coverage of the planning target volume (PTV), which, in turn, resulted in compromised tumour control probability in contours from four physicians. The validation of the semi-automated delineation algorithm using thorax phantom led to a highly reliable accuracy in defining GTVs. Comparing the unsupervised auto-contours with the gold standard delineation revealed high equal high concordance for FB P-CT, 4D P-CT and CBCT, with a DSC of 0.83, 0.76 and 0.8, respectively. The supervised use of the semi-automated delineation tool improved its accuracy, with DSCs of 0.86, 0.86 and 0.8 for FB P-CT, 4D P-CT and CBCT, respectively. The use of the algorithm was associated with a significantly shorter working time. The semi-automated delineation tool can accurately register volume changes in CBCTs.

CONCLUSION

The segmentation algorithm provides a reliable, standardised and time-saving alternative for manual delineation in lung SBRT in P-CT and CBCT.