Machine Learning for Auto-Segmentation in Radiotherapy Planning

Artificial intelligence for treatment delivery: image-guided radiotherapy

Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.

A joint learning framework for multisite CBCT-to-CT translation using a hybrid CNN-transformer synthesizer and a registration network

Background

Cone-beam computed tomography (CBCT) is a convenient method for adaptive radiation therapy (ART), but its application is often hindered by its image quality. We aim to develop a unified deep learning model that can consistently enhance the quality of CBCT images across various anatomical sites by generating synthetic CT (sCT) images.

Methods

A dataset of paired CBCT and planning CT images from 135 cancer patients, including head and neck, chest and abdominal tumors, was collected. This dataset, with its rich anatomical diversity and scanning parameters, was carefully selected to ensure comprehensive model training. Due to the imperfect registration, the inherent challenge of local structural misalignment of paired dataset may lead to suboptimal model performance. To address this limitation, we propose SynREG, a supervised learning framework. SynREG integrates a hybrid CNN-transformer architecture designed for generating high-fidelity sCT images and a registration network designed to correct local structural misalignment dynamically during training. An independent test set of 23 additional patients was used to evaluate the image quality, and the results were compared with those of several benchmark models (pix2pix, cycleGAN and SwinIR). Furthermore, the performance of an autosegmentation application was also assessed.

Results

The proposed model disentangled sCT generation from anatomical correction, leading to a more rational optimization process. As a result, the model effectively suppressed noise and artifacts in multisite applications, significantly enhancing CBCT image quality. Specifically, the mean absolute error (MAE) of SynREG was reduced to 16.81 ± 8.42 HU, whereas the structural similarity index (SSIM) increased to 94.34 ± 2.85%, representing improvements over the raw CBCT data, which had the MAE of 26.74 ± 10.11 HU and the SSIM of 89.73 ± 3.46%. The enhanced image quality was particularly beneficial for organs with low contrast resolution, significantly increasing the accuracy of automatic segmentation in these regions. Notably, for the brainstem, the mean Dice similarity coefficient (DSC) increased from 0.61 to 0.89, and the MDA decreased from 3.72 mm to 0.98 mm, indicating a substantial improvement in segmentation accuracy and precision.

Conclusions

SynREG can effectively alleviate the differences in residual anatomy between paired datasets and enhance the quality of CBCT images.

Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives

The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.

Radiomics as a measure superior to common similarity metrics for tumor segmentation performance evaluation

PURPOSE

To propose radiomics features as a superior measure for evaluating the segmentation ability of physicians and auto-segmentation tools and to compare its performance with the most commonly used metrics: Dice similarity coefficient (DSC), surface Dice similarity coefficient (sDSC), and Hausdorff distance (HD).

MATERIALS/METHODS

The data of 10 lung cancer patients' CT images with nine tumor segmentations per tumor were downloaded from the RIDER (Reference Database to Evaluate Response) database. Radiomics features of 90 segmented tumors were extracted using the PyRadiomics program. The intraclass correlation coefficient (ICC) of radiomics features were used to evaluate the segmentation similarity and compare their performance with DSC, sDSC, and HD. We calculated one ICC per radiomics feature and per tumor for nine segmentations and 36 ICCs per radiomics feature for 36 pairs of nine segmentations. Meanwhile, there were 360 DSC, sDSC, and HD values calculated for 36 pairs for 10 tumors.

RESULTS

The ICC of radiomics features exhibited greater sensitivity to segmentation changes than DSC and sDSC. The ICCs of the wavelet-LLL first order Maximum, wavelet-LLL glcm MCC, wavelet-LLL glcm Cluster Shade features ranged from 0.130 to 0.997, 0.033 to 0.978, and 0.160 to 0.998, respectively. On the other hand, all DSC and sDSC were larger than 0.778 and 0.700, respectively, while HD varied from 0 to 1.9 mm. The results indicated that the radiomics features could capture subtle variations in tumor segmentation characteristics, which could not be easily detected by DSC and sDSC.

CONCLUSIONS

This study demonstrates the superiority of radiomics features with ICC as a measure for evaluating a physician's tumor segmentation ability and the performance of auto-segmentation tools. Radiomics features offer a more sensitive and comprehensive evaluation, providing valuable insights into tumor characteristics. Therefore, the new metrics can be used to evaluate new auto-segmentation methods and enhance trainees' segmentation skills in medical training and education.

Optimization of treatment workflow for 0.35T MR-Linac system

PURPOSE

This study presents a novel and comprehensive framework for evaluating magnetic resonance guided radiotherapy (MRgRT) workflow by integrating the Failure Modes and Effects Analysis (FMEA) approach with Time-Driven Activity-Based Costing (TDABC). We assess the workflow for safety, quality, and economic implications, providing a holistic understanding of the MRgRT implementation. The aim is to offer valuable insights to healthcare practitioners and administrators, facilitating informed decision-making regarding the 0.35T MRIdian MR-Linac system's clinical workflow.

METHODS

For FMEA, a multidisciplinary team followed the TG-100 methodology to assess the MRgRT workflow's potential failure modes. Following the mitigation of primary failure modes and workflow optimization, a treatment process was established for TDABC analysis. The TDABC was applied to both MRgRT and computed tomography guided RT (CTgRT) for typical five-fraction stereotactic body RT (SBRT) treatments, assessing total workflow and costs associated between the two treatment workflows.

RESULTS

A total of 279 failure modes were identified, with 31 categorized as high-risk, 55 as medium-risk, and the rest as low-risk. The top 20% risk priority numbers (RPN) were determined for each radiation oncology care team member. Total MRgRT and CTgRT costs were assessed. Implementing technological advancements, such as real-time multi leaf collimator (MLC) tracking with volumetric modulated arc therapy (VMAT), auto-segmentation, and increasing the Linac dose rate, led to significant cost savings for MRgRT.

CONCLUSION

In this study, we integrated FMEA with TDABC to comprehensively evaluate the workflow and the associated costs of MRgRT compared to conventional CTgRT for five-fraction SBRT treatments. FMEA analysis identified critical failure modes, offering insights to enhance patient safety. TDABC analysis revealed that while MRgRT provides unique advantages, it may involve higher costs. Our findings underscore the importance of exploring cost-effective strategies and key technological advancements to ensure the widespread adoption and financial sustainability of MRgRT in clinical practice.

Radiomics for clinical decision support in radiation oncology

Radiomics is a promising tool for the development of quantitative biomarkers to support clinical decision-making. It has been shown to improve the prediction of response to treatment and outcome in different settings, particularly in the field of radiation oncology by optimising the dose delivery solutions and reducing the rate of radiation-induced side effects, leading to a fully personalised approach. Despite the promising results offered by radiomics at each of these stages, standardised methodologies, reproducibility and interpretability of results are still lacking, limiting the potential clinical impact of these tools. In this review, we briefly describe the principles of radiomics and the most relevant applications of radiomics at each stage of cancer management in the framework of radiation oncology. Furthermore, the integration of radiomics into clinical decision support systems is analysed, defining the challenges and offering possible solutions for translating radiomics into a clinically applicable tool.

Applications of artificial intelligence for machine- and patient-specific quality assurance in radiation therapy: current status and future directions

Machine- and patient-specific quality assurance (QA) is essential to ensure the safety and accuracy of radiotherapy. QA methods have become complex, especially in high-precision radiotherapy such as intensity-modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT), and various recommendations have been reported by AAPM Task Groups. With the widespread use of IMRT and VMAT, there is an emerging demand for increased operational efficiency. Artificial intelligence (AI) technology is quickly growing in various fields owing to advancements in computers and technology. In the radiotherapy treatment process, AI has led to the development of various techniques for automated segmentation and planning, thereby significantly enhancing treatment efficiency. Many new applications using AI have been reported for machine- and patient-specific QA, such as predicting machine beam data or gamma passing rates for IMRT or VMAT plans. Additionally, these applied technologies are being developed for multicenter studies. In the current review article, AI application techniques in machine- and patient-specific QA have been organized and future directions are discussed. This review presents the learning process and the latest knowledge on machine- and patient-specific QA. Moreover, it contributes to the understanding of the current status and discusses the future directions of machine- and patient-specific QA.

The Evolving Role of Artificial Intelligence in Radiotherapy Treatment Planning-A Literature Review

This paper examines the integration of artificial intelligence (AI) in radiotherapy for cancer treatment. The importance of radiotherapy in cancer management and its time-intensive planning process make AI adoption appealing especially with the escalating demand for radiotherapy. This review highlights the efficacy of AI across medical domains, where it surpasses human capabilities in areas such as cardiology and dermatology. Focusing on radiotherapy, the paper details AI's applications in target segmentation, dose optimization, and outcome prediction. It discusses adaptive radiotherapy's benefits and AI's potential to enhance patient outcomes with much improved treatment accuracy. The paper explores ethical concerns, including data privacy and bias, stressing the need for robust guidelines. Educating healthcare professionals and patients about AI's role is crucial as it acknowledges potential job-role changes and concerns about patients' trust in the use of AI. Overall, the integration of AI in radiotherapy holds transformative potential in streamlining processes, improving outcomes, and reducing costs. AI's potential to reduce healthcare costs underscores its significance with impactful change globally. However, successful implementation hinges on addressing ethical and logistical challenges and fostering collaboration among healthcare professionals and patient population data sets for its optimal utilization. Rigorous education, collaborative efforts, and global data sharing will be the compass guiding its' success in radiotherapy and healthcare.

Gross failure rates and failure modes for a commercial AI-based auto-segmentation algorithm in head and neck cancer patients

PURPOSE

Artificial intelligence (AI) based commercial software can be used to automatically delineate organs at risk (OAR), with potential for efficiency savings in the radiotherapy treatment planning pathway, and reduction of inter- and intra-observer variability. There has been little research investigating gross failure rates and failure modes of such systems.

METHOD

50 head and neck (H&N) patient data sets with "gold standard" contours were compared to AI-generated contours to produce expected mean and standard deviation values for the Dice Similarity Coefficient (DSC), for four common H&N OARs (brainstem, mandible, left and right parotid). An AI-based commercial system was applied to 500 H&N patients. AI-generated contours were compared to manual contours, outlined by an expert human, and a gross failure was set at three standard deviations below the expected mean DSC. Failures were inspected to assess reason for failure of the AI-based system with failures relating to suboptimal manual contouring censored. True failures were classified into 4 sub-types (setup position, anatomy, image artefacts and unknown).

RESULTS

There were 24 true failures of the AI-based commercial software, a gross failure rate of 1.2%. Fifteen failures were due to patient anatomy, four were due to dental image artefacts, three were due to patient position and two were unknown. True failure rates by OAR were 0.4% (brainstem), 2.2% (mandible), 1.4% (left parotid) and 0.8% (right parotid).

CONCLUSION

True failures of the AI-based system were predominantly associated with a non-standard element within the CT scan. It is likely that these non-standard elements were the reason for the gross failure, and suggests that patient datasets used to train the AI model did not contain sufficient heterogeneity of data. Regardless of the reasons for failure, the true failure rate for the AI-based system in the H&N region for the OARs investigated was low (∼1%).

Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation

PURPOSE

The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors.

METHODS

Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure.

RESULTS

Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations.

CONCLUSION

Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Deep Learning and Multimodal Artificial Intelligence in Orthopaedic Surgery

This review article focuses on the applications of deep learning with neural networks and multimodal neural networks in the orthopaedic domain. By providing practical examples of how artificial intelligence (AI) is being applied successfully in orthopaedic surgery, particularly in the realm of imaging data sets and the integration of clinical data, this study aims to provide orthopaedic surgeons with the necessary tools to not only evaluate existing literature but also to consider AI's potential in their own clinical or research pursuits. We first review standard deep neural networks which can analyze numerical clinical variables, then describe convolutional neural networks which can analyze image data, and then introduce multimodal AI models which analyze various types of different data. Then, we contrast these deep learning techniques with related but more limited techniques such as radiomics, describe how to interpret deep learning studies, and how to initiate such studies at your institution. Ultimately, by empowering orthopaedic surgeons with the knowledge and know-how of deep learning, this review aspires to facilitate the translation of research into clinical practice, thereby enhancing the efficacy and precision of real-world orthopaedic care for patients.

A novel LVPA-UNet network for target volume automatic delineation: An MRI case study of nasopharyngeal carcinoma

Accurate delineation of Gross Tumor Volume (GTV) is crucial for radiotherapy. Deep learning-driven GTV segmentation technologies excel in rapidly and accurately delineating GTV, providing a basis for radiologists in formulating radiation plans. The existing 2D and 3D segmentation models of GTV based on deep learning are limited by the loss of spatial features and anisotropy respectively, and are both affected by the variability of tumor characteristics, blurred boundaries, and background interference. All these factors seriously affect the segmentation performance. To address the above issues, a Layer-Volume Parallel Attention (LVPA)-UNet model based on 2D-3D architecture has been proposed in this study, in which three strategies are introduced. Firstly, 2D and 3D workflows are introduced in the LVPA-UNet. They work in parallel and can guide each other. Both the fine features of each slice of 2D MRI and the 3D anatomical structure and spatial features of the tumor can be extracted by them. Secondly, parallel multi-branch depth-wise strip convolutions adapt the model to tumors of varying shapes and sizes within slices and volumetric spaces, and achieve refined processing of blurred boundaries. Lastly, a Layer-Channel Attention mechanism is proposed to adaptively adjust the weights of slices and channels according to their different tumor information, and then to highlight slices and channels with tumor. The experiments by LVPA-UNet on 1010 nasopharyngeal carcinoma (NPC) MRI datasets from three centers show a DSC of 0.7907, precision of 0.7929, recall of 0.8025, and HD95 of 1.8702 mm, outperforming eight typical models. Compared to the baseline model, it improves DSC by 2.14 %, precision by 2.96 %, and recall by 1.01 %, while reducing HD95 by 0.5434 mm. Consequently, while ensuring the efficiency of segmentation through deep learning, LVPA-UNet is able to provide superior GTV delineation results for radiotherapy and offer technical support for precision medicine.

Cardiac substructure delineation in radiation therapy - A state-of-the-art review

Delineation of cardiac substructures is crucial for a better understanding of radiation-related cardiotoxicities and to facilitate accurate and precise cardiac dose calculation for developing and applying risk models. This review examines recent advancements in cardiac substructure delineation in the radiation therapy (RT) context, aiming to provide a comprehensive overview of the current level of knowledge, challenges and future directions in this evolving field. Imaging used for RT planning presents challenges in reliably visualising cardiac anatomy. Although cardiac atlases and contouring guidelines aid in standardisation and reduction of variability, significant uncertainties remain in defining cardiac anatomy. Coupled with the inherent complexity of the heart, this necessitates auto-contouring for consistent large-scale data analysis and improved efficiency in prospective applications. Auto-contouring models, developed primarily for breast and lung cancer RT, have demonstrated performance comparable to manual contouring, marking a significant milestone in the evolution of cardiac delineation practices. Nevertheless, several key concerns require further investigation. There is an unmet need for expanding cardiac auto-contouring models to encompass a broader range of cancer sites. A shift in focus is needed from ensuring accuracy to enhancing the robustness and accessibility of auto-contouring models. Addressing these challenges is paramount for the integration of cardiac substructure delineation and associated risk models into routine clinical practice, thereby improving the safety of RT for future cancer patients.

NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions

Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.

Dosimetric impact of contour editing on CT and MRI deep-learning autosegmentation for brain OARs

PURPOSE

To establish the clinical applicability of deep-learning organ-at-risk autocontouring models (DL-AC) for brain radiotherapy. The dosimetric impact of contour editing, prior to model training, on performance was evaluated for both CT and MRI-based models. The correlation between geometric and dosimetric measures was also investigated to establish whether dosimetric assessment is required for clinical validation.

METHOD

CT and MRI-based deep learning autosegmentation models were trained using edited and unedited clinical contours. Autosegmentations were dosimetrically compared to gold standard contours for a test cohort. D1%, D5%, D50%, and maximum dose were used as clinically relevant dosimetric measures. The statistical significance of dosimetric differences between the gold standard and autocontours was established using paired Student's t-tests. Clinically significant cases were identified via dosimetric headroom to the OAR tolerance. Pearson's Correlations were used to investigate the relationship between geometric measures and absolute percentage dose changes for each autosegmentation model.

RESULTS

Except for the right orbit, when delineated using MRI models, the dosimetric statistical analysis revealed no superior model in terms of the dosimetric accuracy between the CT DL-AC models or between the MRI DL-AC for any investigated brain OARs. The number of patients where the clinical significance threshold was exceeded was higher for the optic chiasm D1% than other OARs, for all autosegmentation models. A weak correlation was consistently observed between the outcomes of dosimetric and geometric evaluations.

CONCLUSIONS

Editing contours before training the DL-AC model had no significant impact on dosimetry. The geometric test metrics were inadequate to estimate the impact of contour inaccuracies on dose. Accordingly, dosimetric analysis is needed to evaluate the clinical applicability of DL-AC models in the brain.

Artificial intelligence-based MRI radiomics and radiogenomics in glioma

The specific genetic subtypes that gliomas exhibit result in variable clinical courses and the need to involve multidisciplinary teams of neurologists, epileptologists, neurooncologists and neurosurgeons. Currently, the diagnosis of gliomas pivots mainly around the preliminary radiological findings and the subsequent definitive surgical diagnosis (via surgical sampling). Radiomics and radiogenomics present a potential to precisely diagnose and predict survival and treatment responses, via morphological, textural, and functional features derived from MRI data, as well as genomic data. In spite of their advantages, it is still lacking standardized processes of feature extraction and analysis methodology among different research groups, which have made external validations infeasible. Radiomics and radiogenomics can be used to better understand the genomic basis of gliomas, such as tumor spatial heterogeneity, treatment response, molecular classifications and tumor microenvironment immune infiltration. These novel techniques have also been used to predict histological features, grade or even overall survival in gliomas. In this review, workflows of radiomics and radiogenomics are elucidated, with recent research on machine learning or artificial intelligence in glioma.

Edge roughness quantifies impact of physician variation on training and performance of deep learning auto-segmentation models for the esophagus

Manual segmentation of tumors and organs-at-risk (OAR) in 3D imaging for radiation-therapy planning is time-consuming and subject to variation between different observers. Artificial intelligence (AI) can assist with segmentation, but challenges exist in ensuring high-quality segmentation, especially for small, variable structures, such as the esophagus. We investigated the effect of variation in segmentation quality and style of physicians for training deep-learning models for esophagus segmentation and proposed a new metric, edge roughness, for evaluating/quantifying slice-to-slice inconsistency. This study includes a real-world cohort of 394 patients who each received radiation therapy (mainly for lung cancer). Segmentation of the esophagus was performed by 8 physicians as part of routine clinical care. We evaluated manual segmentation by comparing the length and edge roughness of segmentations among physicians to analyze inconsistencies. We trained eight multiple- and individual-physician segmentation models in total, based on U-Net architectures and residual backbones. We used the volumetric Dice coefficient to measure the performance for each model. We proposed a metric, edge roughness, to quantify the shift of segmentation among adjacent slices by calculating the curvature of edges of the 2D sagittal- and coronal-view projections. The auto-segmentation model trained on multiple physicians (MD1-7) achieved the highest mean Dice of 73.7 ± 14.8%. The individual-physician model (MD7) with the highest edge roughness (mean ± SD: 0.106 ± 0.016) demonstrated significantly lower volumetric Dice for test cases compared with other individual models (MD7: 58.5 ± 15.8%, MD6: 67.1 ± 16.8%, p < 0.001). A multiple-physician model trained after removing the MD7 data resulted in fewer outliers (e.g., Dice ≤ 40%: 4 cases for MD1-6, 7 cases for MD1-7, Ntotal = 394). While we initially detected this pattern in a single clinician, we validated the edge roughness metric across the entire dataset. The model trained with the lowest-quantile edge roughness (MDER-Q1, Ntrain = 62) achieved significantly higher Dice (Ntest = 270) than the model trained with the highest-quantile ones (MDER-Q4, Ntrain = 62) (MDER-Q1: 67.8 ± 14.8%, MDER-Q4: 62.8 ± 15.7%, p < 0.001). This study demonstrates that there is significant variation in style and quality in manual segmentations in clinical care, and that training AI auto-segmentation algorithms from real-world, clinical datasets may result in unexpectedly under-performing algorithms with the inclusion of outliers. Importantly, this study provides a novel evaluation metric, edge roughness, to quantify physician variation in segmentation which will allow developers to filter clinical training data to optimize model performance.

Auto-segmentation of neck nodal metastases using self-distilled masked image transformer on longitudinal MR images

Objectives

Auto-segmentation promises greater speed and lower inter-reader variability than manual segmentations in radiation oncology clinical practice. This study aims to implement and evaluate the accuracy of the auto-segmentation algorithm, "Masked Image modeling using the vision Transformers (SMIT)," for neck nodal metastases on longitudinal T2-weighted (T2w) MR images in oropharyngeal squamous cell carcinoma (OPSCC) patients.

Methods

This prospective clinical trial study included 123 human papillomaviruses (HPV-positive [+]) related OSPCC patients who received concurrent chemoradiotherapy. T2w MR images were acquired on 3 T at pre-treatment (Tx), week 0, and intra-Tx weeks (1-3). Manual delineations of metastatic neck nodes from 123 OPSCC patients were used for the SMIT auto-segmentation, and total tumor volumes were calculated. Standard statistical analyses compared contour volumes from SMIT vs manual segmentation (Wilcoxon signed-rank test [WSRT]), and Spearman's rank correlation coefficients (ρ) were computed. Segmentation accuracy was evaluated on the test data set using the dice similarity coefficient (DSC) metric value. P-values <0.05 were considered significant.

Results

No significant difference in manual and SMIT delineated tumor volume at pre-Tx (8.68 ± 7.15 vs 8.38 ± 7.01 cm3, P = 0.26 [WSRT]), and the Bland-Altman method established the limits of agreement as -1.71 to 2.31 cm3, with a mean difference of 0.30 cm3. SMIT model and manually delineated tumor volume estimates were highly correlated (ρ = 0.84-0.96, P < 0.001). The mean DSC metric values were 0.86, 0.85, 0.77, and 0.79 at the pre-Tx and intra-Tx weeks (1-3), respectively.

Conclusions

The SMIT algorithm provides sufficient segmentation accuracy for oncological applications in HPV+ OPSCC.

Advances in knowledge

First evaluation of auto-segmentation with SMIT using longitudinal T2w MRI in HPV+ OPSCC.

Organ-contour-driven auto-matching algorithm in image-guided radiotherapy

PURPOSE

This study aimed to demonstrate the potential clinical applicability of an organ-contour-driven auto-matching algorithm in image-guided radiotherapy.

METHODS

This study included eleven consecutive patients with cervical cancer who underwent radiotherapy in 23 or 25 fractions. Daily and reference magnetic resonance images were converted into mesh models. A weight-based algorithm was implemented to optimize the distance between the mesh model vertices and surface of the reference model during the positioning process. Within the cost function, weight parameters were employed to prioritize specific organs for positioning. In this study, three scenarios with different weight parameters were prepared. The optimal translation and rotation values for the cervix and uterus were determined based on the calculated translations alone or in combination with rotations, with a rotation limit of ±3°. Subsequently, the coverage probabilities of the following two planning target volumes (PTV), an isotropic 5 mm and anisotropic margins derived from a previous study, were evaluated.

RESULTS

The percentage of translations exceeding 10 mm varied from 9% to 18% depending on the scenario. For small PTV sizes, more than 80% of all fractions had a coverage of 80% or higher. In contrast, for large PTV sizes, more than 90% of all fractions had a coverage of 95% or higher. The difference between the median coverage with translational positioning alone and that with both translational and rotational positioning was 1% or less.

CONCLUSION

This algorithm facilitates quantitative positioning by utilizing a cost function that prioritizes organs for positioning. Consequently, consistent displacement values were algorithmically generated. This study also revealed that the impact of rotational corrections, limited to ±3°, on PTV coverage was minimal.

Selective image segmentation driven by region, edge and saliency functions

Present active contour methods often struggle with the segmentation of regions displaying variations in texture, color, or intensity a phenomenon referred to as inhomogeneities. These limitation impairs their ability to precisely distinguish and outline diverse components within an image. Further some of these methods employ intricate mathematical formulations for energy minimization. Such complexity introduces computational sluggishness, making these methods unsuitable for tasks requiring real-time processing or rapid segmentation. Moreover, these methods are susceptible to being trapped in energy configurations corresponding to local minimum points. Consequently, the segmentation process fails to converge to the desired outcome. Additionally, the efficacy of these methods diminishes when confronted with regions exhibiting weak or subtle boundaries. To address these limitations comprehensively, our proposed approach introduces a fresh paradigm for image segmentation through the synchronization of region-based, edge-based, and saliency-based segmentation techniques. Initially, we adapt an intensity edge term based on the zero crossing feature detector (ZCD), which is used to highlight significant edges of an image. Secondly, a saliency function is formulated to detect salient regions from an image. We have also included a globally tuned region based SPF (signed pressure force) term to move contour away and capture homogeneous regions. ZCD, saliency and global SPF are jointly incorporated with some scaled value for the level set evolution to develop an effective image segmentation model. In addition, proposed method is capable to perform selective object segmentation, which enables us to choose any single or multiple objects inside an image. Saliency function and ZCD detector are considered feature enhancement tools, which are used to get important features of an image, so this method has a solid capacity to segment nature images (homogeneous or inhomogeneous) precisely. Finally, the adaption of the Gaussian kernel removes the need of any penalization term for level set reinitialization. Experimental results will exhibit the efficiency of the proposed method.

Review and recommendations on deformable image registration uncertainties for radiotherapy applications

Deformable image registration (DIR) is a versatile tool used in many applications in radiotherapy (RT). DIR algorithms have been implemented in many commercial treatment planning systems providing accessible and easy-to-use solutions. However, the geometric uncertainty of DIR can be large and difficult to quantify, resulting in barriers to clinical practice. Currently, there is no agreement in the RT community on how to quantify these uncertainties and determine thresholds that distinguish a good DIR result from a poor one. This review summarises the current literature on sources of DIR uncertainties and their impact on RT applications. Recommendations are provided on how to handle these uncertainties for patient-specific use, commissioning, and research. Recommendations are also provided for developers and vendors to help users to understand DIR uncertainties and make the application of DIR in RT safer and more reliable.

Clinical evaluation of deep learning and atlas-based auto-segmentation for organs at risk delineation

Manual delineation of organs at risk and clinical target volumes is essential in radiotherapy planning. Atlas-based auto-segmentation (ABAS) algorithms have become available and been shown to provide accurate contouring for various anatomical sites. Recently, deep learning auto-segmentation (DL-AS) algorithms have emerged as the state-of-the-art in medical image segmentation. This study aimed to evaluate the effect of auto-segmentation on the clinical workflow for contouring different anatomical sites of cancer, such as head and neck (H&N), breast, abdominal region, and prostate. Patients with H&N, breast, abdominal, and prostate cancer (n = 30 each) were enrolled in the study. Twenty-seven different organs at four sites were evaluated. RayStation was used to apply the ABAS. Siemens AI-Rad Companion Organs RT was used to apply the DL-AS. Evaluations were performed with similarity indices using geometric methods, time-evaluation, and qualitative scoring visual evaluations by radiation oncologists. The DL-AS algorithm was more accurate than ABAS algorithm on geometric indices for half of the structures. The qualitative scoring results of the two algorithms were significantly different, and DL-AS was more accurate on many contours. DL-AS had 41%, 29%, 86%, and 15% shorter edit times in the HnN, breast, abdomen, and prostate groups, respectively, than ABAS. There were no correlations between the geometric indices and visual assessments. The time required to edit the contours was considerably shorter for DL-AS than for ABAS. Auto-segmentation with deep learning could be the first step for clinical workflow optimization in radiotherapy.

Automatic Head and Neck Tumor segmentation and outcome prediction relying on FDG-PET/CT images: Findings from the second edition of the HECKTOR challenge

By focusing on metabolic and morphological tissue properties respectively, FluoroDeoxyGlucose (FDG)-Positron Emission Tomography (PET) and Computed Tomography (CT) modalities include complementary and synergistic information for cancerous lesion delineation and characterization (e.g. for outcome prediction), in addition to usual clinical variables. This is especially true in Head and Neck Cancer (HNC). The goal of the HEad and neCK TumOR segmentation and outcome prediction (HECKTOR) challenge was to develop and compare modern image analysis methods to best extract and leverage this information automatically. We present here the post-analysis of HECKTOR 2nd edition, at the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2021. The scope of the challenge was substantially expanded compared to the first edition, by providing a larger population (adding patients from a new clinical center) and proposing an additional task to the challengers, namely the prediction of Progression-Free Survival (PFS). To this end, the participants were given access to a training set of 224 cases from 5 different centers, each with a pre-treatment FDG-PET/CT scan and clinical variables. Their methods were subsequently evaluated on a held-out test set of 101 cases from two centers. For the segmentation task (Task 1), the ranking was based on a Borda counting of their ranks according to two metrics: mean Dice Similarity Coefficient (DSC) and median Hausdorff Distance at 95th percentile (HD95). For the PFS prediction task, challengers could use the tumor contours provided by experts (Task 3) or rely on their own (Task 2). The ranking was obtained according to the Concordance index (C-index) calculated on the predicted risk scores. A total of 103 teams registered for the challenge, for a total of 448 submissions and 29 papers. The best method in the segmentation task obtained an average DSC of 0.759, and the best predictions of PFS obtained a C-index of 0.717 (without relying on the provided contours) and 0.698 (using the expert contours). An interesting finding was that best PFS predictions were reached by relying on DL approaches (with or without explicit tumor segmentation, 4 out of the 5 best ranked) compared to standard radiomics methods using handcrafted features extracted from delineated tumors, and by exploiting alternative tumor contours (automated and/or larger volumes encompassing surrounding tissues) rather than relying on the expert contours. This second edition of the challenge confirmed the promising performance of fully automated primary tumor delineation in PET/CT images of HNC patients, although there is still a margin for improvement in some difficult cases. For the first time, the prediction of outcome was also addressed and the best methods reached relatively good performance (C-index above 0.7). Both results constitute another step forward toward large-scale outcome prediction studies in HNC.

The role of artificial intelligence in radiotherapy clinical practice

This review article visits the current state of artificial intelligence (AI) in radiotherapy clinical practice. We will discuss how AI has a place in the modern radiotherapy workflow at the level of automatic segmentation and planning, two applications which have seen real-work implementation. A special emphasis will be placed on the role AI can play in online adaptive radiotherapy, such as performed at MR-linacs, where online plan adaptation is a procedure which could benefit from automation to reduce on-couch time for patients. Pseudo-CT generation and AI for motion tracking will be introduced in the scope of online adaptive radiotherapy as well. We further discuss the use of AI for decision-making and response assessment, for example for personalized prescription and treatment selection, risk stratification for outcomes and toxicities, and AI for quantitative imaging and response assessment. Finally, the challenges of generalizability and ethical aspects will be covered. With this, we provide a comprehensive overview of the current and future applications of AI in radiotherapy.

Quantification of tissue property and perfusion uncertainties in hyperthermia treatment planning: Multianalysis using polynomial chaos expansion

INTRODUCTION

Hyperthermia treatment planning (HTP) tools can guide treatment delivery, particularly with locoregional radiative phased array systems. Uncertainties in tissue and perfusion property values presently lead to quantitative inaccuracy of HTP, leading to sub-optimal treatment. Assessment of these uncertainties would allow for better judgement of the reliability of treatment plans and improve their value for treatment guidance. However, systematically investigating the impact of all uncertainties on treatment plans is a complex, high-dimensional problem and too computationally expensive for traditional Monte Carlo approaches. This study aims to systematically quantify the treatment-plan impact of tissue property uncertainties by investigating their individual contribution to, and combined impact on predicted temperature distributions.

METHODS

A novel Polynomial Chaos Expansion (PCE)-based HTP uncertainty quantification was developed and applied for locoregional hyperthermia of modelled tumours in the pancreatic head, prostate, rectum, and cervix. Patient models were based on the Duke and Ella digital human models. Using Plan2Heat, treatment plans were created to optimise tumour temperature (represented by T90) for treatment using the Alba4D system. For all 25-34 modelled tissues, the impact of tissue property uncertainties was analysed individually i.e., electrical and thermal conductivity, permittivity, density, specific heat capacity and perfusion. Next, combined analyses were performed on the top 30 uncertainties with the largest impact.

RESULTS

Uncertainties in thermal conductivity and heat capacity were found to have negligible impact on the predicted temperature ( < 1 × 10-10 °C), density and permittivity uncertainties had a small impact (< 0.3 °C). Uncertainties in electrical conductivity and perfusion can lead to large variations in predicted temperature. However, variations in muscle properties result in the largest impact at locations that could limit treatment quality, with a standard deviation up to almost 6 °C (pancreas) and 3.5 °C (prostate) for perfusion and electrical conductivity, respectively. The combined influence of all significant uncertainties leads to large variations with a standard deviation up to 9.0, 3.6, 3.7 and 4.1 °C for the pancreatic, prostate, rectal and cervical cases, respectively.

CONCLUSION

Uncertainties in tissue and perfusion property values can have a large impact on predicted temperatures from hyperthermia treatment planning. PCE-based analysis helps to identify all major uncertainties, their impact and judge the reliability of treatment plans.

Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy

Background and purpose

Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation.

Materials and Methods

One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction.

Results

The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant.

Conclusion

A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.

Simulation as a tool to model potential workflow enhancements in radiotherapy treatment pathways - A systematic review

This systematic review aimed to synthesize and summarize the use of simulation of radiotherapy pathways. The objective was to establish the suitability of those simulations in modeling the potential introduction of processes and technologies to speed up radiotherapy pathways. A systematic literature search was carried out using PubMed and Scopus databases to evaluate the use of simulation in radiotherapy pathways. Full journal articles and conference proceedings were considered, and the search was limited to the English language only. To be eligible for inclusion, articles had to model multiple sequential processes in the radiotherapy pathway concurrently to demonstrate the suitability of simulation modeling in typical pathways. Papers solely modeling scheduling, capacity, or queuing strategies were excluded. In total, 151 potential studies were identified and screened to find 18 relevant studies in October 2022. Studies showed that various pathways could be modeled, including the entire pathway from referral to end of treatment or the constituent phases such as pre-treatment, treatment, or other subcomponents. The data required to generate models varied from study to study, but at least 3 months of data were needed. This review demonstrates that modeling and simulation of radiotherapy pathways are feasible and that model output matches real-world systems. Validated models give researchers confidence to modify models with potential workflow enhancements to assess their potential effect on real-world systems. It is recommended that researchers follow best practice guidelines when building models to ensure that they are fit for purpose and to enable decision makers to have confidence in their results.

Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation

BACKGROUND

Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance.

METHODS

Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure.

RESULTS

After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals.

CONCLUSION

Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.

Active bone marrow segmentation based on computed tomography imaging in anal cancer patients: A machine-learning-based proof of concept

PURPOSE

Different methods are available to identify haematopoietically active bone marrow (ActBM). However, their use can be challenging for radiotherapy routine treatments, since they require specific equipment and dedicated time. A machine learning (ML) approach, based on radiomic features as inputs to three different classifiers, was applied to computed tomography (CT) images to identify haematopoietically active bone marrow in anal cancer patients.

METHODS

A total of 40 patients was assigned to the construction set (training set + test set). Fluorine-18-Fluorodeoxyglucose Positron Emission Tomography (18FDG-PET) images were used to detect the active part of the pelvic bone marrow (ActPBM) and stored as ground-truth for three subregions: iliac, lower pelvis and lumbosacral bone marrow (ActIBM, ActLPBM, ActLSBM). Three parameters were used for the correspondence analyses between 18FDG-PET and ML classifiers: DICE index, Precision and Recall.

RESULTS

For the 40-patient cohort, median values [min; max] of the Dice index were 0.69 [0.20; 0.84], 0.76 [0.25; 0.89], and 0.36 [0.15; 0.67] for ActIBM, ActLSBM, and ActLPBM, respectively. The Precision/Recall (P/R) ratio median value for the ActLPBM structure was 0.59 [0.20; 1.84] (over segmentation), while for the other two subregions the P/R ratio median has values of 1.249 [0.43; 4.15] for ActIBM and 1.093 [0.24; 1.91] for ActLSBM (under segmentation).

CONCLUSION

A satisfactory degree of overlap compared to 18FDG-PET was found for 2 out of the 3 subregions within pelvic bones. Further optimization and generalization of the process is required before clinical implementation.

Deep learning auto-segmentation on multi-sequence magnetic resonance images for upper abdominal organs

Introduction

Multi-sequence multi-parameter MRIs are often used to define targets and/or organs at risk (OAR) in radiation therapy (RT) planning. Deep learning has so far focused on developing auto-segmentation models based on a single MRI sequence. The purpose of this work is to develop a multi-sequence deep learning based auto-segmentation (mS-DLAS) based on multi-sequence abdominal MRIs.

Materials and methods

Using a previously developed 3DResUnet network, a mS-DLAS model using 4 T1 and T2 weighted MRI acquired during routine RT simulation for 71 cases with abdominal tumors was trained and tested. Strategies including data pre-processing, Z-normalization approach, and data augmentation were employed. Additional 2 sequence specific T1 weighted (T1-M) and T2 weighted (T2-M) models were trained to evaluate performance of sequence-specific DLAS. Performance of all models was quantitatively evaluated using 6 surface and volumetric accuracy metrics.

Results

The developed DLAS models were able to generate reasonable contours of 12 upper abdomen organs within 21 seconds for each testing case. The 3D average values of dice similarity coefficient (DSC), mean distance to agreement (MDA mm), 95 percentile Hausdorff distance (HD95% mm), percent volume difference (PVD), surface DSC (sDSC), and relative added path length (rAPL mm/cc) over all organs were 0.87, 1.79, 7.43, -8.95, 0.82, and 12.25, respectively, for mS-DLAS model. Collectively, 71% of the auto-segmented contours by the three models had relatively high quality. Additionally, the obtained mS-DLAS successfully segmented 9 out of 16 MRI sequences that were not used in the model training.

Conclusion

We have developed an MRI-based mS-DLAS model for auto-segmenting of upper abdominal organs on MRI. Multi-sequence segmentation is desirable in routine clinical practice of RT for accurate organ and target delineation, particularly for abdominal tumors. Our work will act as a stepping stone for acquiring fast and accurate segmentation on multi-contrast MRI and make way for MR only guided radiation therapy.

Addressing the Global Expertise Gap in Radiation Oncology: The Radiation Planning Assistant

PURPOSE

Automation, including the use of artificial intelligence, has been identified as a possible opportunity to help reduce the gap in access and quality for radiotherapy and other aspects of cancer care. The Radiation Planning Assistant (RPA) project was conceived in 2015 (and funded in 2016) to use automated contouring and treatment planning algorithms to support the efforts of oncologists in low- and middle-income countries, allowing them to scale their efforts and treat more patients safely and efficiently (to increase access).

DESIGN

In this review, we discuss the development of the RPA, with a particular focus on clinical acceptability and safety/risk across jurisdictions as these are important indicators for the successful future deployment of the RPA to increase radiotherapy availability and ameliorate global disparities in access to radiation oncology.

RESULTS

RPA tools will be offered through a webpage, where users can upload computed tomography data sets and download automatically generated contours and treatment plans. All interfaces have been designed to maximize ease of use and minimize risk. The current version of the RPA includes automated contouring and planning for head and neck cancer, cervical cancer, breast cancer, and metastases to the brain.

CONCLUSION

The RPA has been designed to bring high-quality treatment planning to more patients across the world, and it may encourage greater investment in treatment devices and other aspects of cancer treatment.

A benchmark study of convolutional neural networks in fully automatic segmentation of aortic root

Recent clinical studies have suggested that introducing 3D patient-specific aortic root models into the pre-operative assessment procedure of transcatheter aortic valve replacement (TAVR) would reduce the incident rate of peri-operative complications. Tradition manual segmentation is labor-intensive and low-efficient, which cannot meet the clinical demands of processing large data volumes. Recent developments in machine learning provided a viable way for accurate and efficient medical image segmentation for 3D patient-specific models automatically. This study quantitively evaluated the auto segmentation quality and efficiency of the four popular segmentation-dedicated three-dimensional (3D) convolutional neural network (CNN) architectures, including 3D UNet, VNet, 3D Res-UNet and SegResNet. All the CNNs were implemented in PyTorch platform, and low-dose CTA image sets of 98 anonymized patients were retrospectively selected from the database for training and testing of the CNNs. The results showed that despite all four 3D CNNs having similar recall, Dice similarity coefficient (DSC), and Jaccard index on the segmentation of the aortic root, the Hausdorff distance (HD) of the segmentation results from 3D Res-UNet is 8.56 ± 2.28, which is only 9.8% higher than that of VNet, but 25.5% and 86.4% lower than that of 3D UNet and SegResNet, respectively. In addition, 3D Res-UNet and VNet also performed better in the 3D deviation location of interest analysis focusing on the aortic valve and the bottom of the aortic root. Although 3D Res-UNet and VNet are evenly matched in the aspect of classical segmentation quality evaluation metrics and 3D deviation location of interest analysis, 3D Res-UNet is the most efficient CNN architecture with an average segmentation time of 0.10 ± 0.04 s, which is 91.2%, 95.3% and 64.3% faster than 3D UNet, VNet and SegResNet, respectively. The results from this study suggested that 3D Res-UNet is a suitable candidate for accurate and fast automatic aortic root segmentation for pre-operative assessment of TAVR.

Comparison of the output of a deep learning segmentation model for locoregional breast cancer radiotherapy trained on 2 different datasets

Introduction

The development of deep learning (DL) models for auto-segmentation is increasing and more models become commercially available. Mostly, commercial models are trained on external data. To study the effect of using a model trained on external data, compared to the same model trained on in-house collected data, the performance of these two DL models was evaluated.

Methods

The evaluation was performed using in-house collected data of 30 breast cancer patients. Quantitative analysis was performed using Dice similarity coefficient (DSC), surface DSC (sDSC) and 95th percentile of Hausdorff Distance (95% HD). These values were compared with previously reported inter-observer variations (IOV).

Results

For a number of structures, statistically significant differences were found between the two models. For organs at risk, mean values for DSC ranged from 0.63 to 0.98 and 0.71 to 0.96 for the in-house and external model, respectively. For target volumes, mean DSC values of 0.57 to 0.94 and 0.33 to 0.92 were found. The difference of 95% HD values ranged 0.08 to 3.23 mm between the two models, except for CTVn4 with 9.95 mm. For the external model, both DSC and 95% HD are outside the range of IOV for CTVn4, whereas this is the case for the DSC found for the thyroid of the in-house model.

Conclusions

Statistically significant differences were found between both models, which were mostly within published inter-observer variations, showing clinical usefulness of both models. Our findings could encourage discussion and revision of existing guidelines, to further decrease inter-observer, but also inter-institute variability.

Artificial Intelligence for Radiotherapy Auto-Contouring: Current Use, Perceptions of and Barriers to Implementation

AIMS

Artificial intelligence has the potential to transform the radiotherapy workflow, resulting in improved quality, safety, accuracy and timeliness of radiotherapy delivery. Several commercially available artificial intelligence-based auto-contouring tools have emerged in recent years. Their clinical deployment raises important considerations for clinical oncologists, including quality assurance and validation, education, training and job planning. Despite this, there is little in the literature capturing the views of clinical oncologists with respect to these factors.

MATERIALS AND METHODS

The Royal College of Radiologists realises the transformational impact artificial intelligence is set to have on our specialty and has appointed the Artificial Intelligence for Clinical Oncology working group. The aim of this work was to survey clinical oncologists with regards to perceptions, current use of and barriers to using artificial intelligence-based auto-contouring for radiotherapy. Here we share our findings with the wider clinical and radiation oncology communities. We hope to use these insights in developing support, guidance and educational resources for the deployment of auto-contouring for clinical use, to help develop the case for wider access to artificial intelligence-based auto-contouring across the UK and to share practice from early-adopters.

RESULTS

In total, 78% of clinical oncologists surveyed felt that artificial intelligence would have a positive impact on radiotherapy. Attitudes to risk were more varied, but 49% felt that artificial intelligence will decrease risk for patients. There is a marked appetite for urgent guidance, education and training on the safe use of such tools in clinical practice. Furthermore, there is a concern that the adoption and implementation of such tools is not equitable, which risks exacerbating existing inequalities across the country.

CONCLUSION

Careful coordination is required to ensure that all radiotherapy departments, and the patients they serve, may enjoy the benefits of artificial intelligence in radiotherapy. Professional organisations, such as the Royal College of Radiologists, have a key role to play in delivering this.

Excitement and Concerns of Young Radiation Oncologists over Automatic Segmentation: A French Perspective

Introduction: Segmentation of organs at risk (OARs) and target volumes need time and precision but are highly repetitive tasks. Radiation oncology has known tremendous technological advances in recent years, the latest being brought by artificial intelligence (AI). Despite the advantages brought by AI for segmentation, some concerns were raised by academics regarding the impact on young radiation oncologists’ training. A survey was thus conducted on young french radiation oncologists (ROs) by the SFjRO (Société Française des jeunes Radiothérapeutes Oncologues). Methodology: The SFjRO organizes regular webinars focusing on anatomical localization, discussing either segmentation or dosimetry. Completion of the survey was mandatory for registration to a dosimetry webinar dedicated to head and neck (H & N) cancers. The survey was generated in accordance with the CHERRIES guidelines. Quantitative data (e.g., time savings and correction needs) were not measured but determined among the propositions. Results: 117 young ROs from 35 different and mostly academic centers participated. Most centers were either already equipped with such solutions or planning to be equipped in the next two years. AI segmentation software was mostly useful for H & N cases. While for the definition of OARs, participants experienced a significant time gain using AI-proposed delineations, with almost 35% of the participants saving between 50–100% of the segmentation time, time gained for target volumes was significantly lower, with only 8.6% experiencing a 50–100% gain. Contours still needed to be thoroughly checked, especially target volumes for some, and edited. The majority of participants suggested that these tools should be integrated into the training so that future radiation oncologists do not neglect the importance of radioanatomy. Fully aware of this risk, up to one-third of them even suggested that AI tools should be reserved for senior physicians only. Conclusions: We believe this survey on automatic segmentation to be the first to focus on the perception of young radiation oncologists. Software developers should focus on enhancing the quality of proposed segmentations, while young radiation oncologists should become more acquainted with these tools.

Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.

A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.

Deep Learning-Based Segmentation of Head and Neck Organs-at-Risk with Clinical Partially Labeled Data

Radiotherapy is one of the main treatments for localized head and neck (HN) cancer. To design a personalized treatment with reduced radio-induced toxicity, accurate delineation of organs at risk (OAR) is a crucial step. Manual delineation is time- and labor-consuming, as well as observer-dependent. Deep learning (DL) based segmentation has proven to overcome some of these limitations, but requires large databases of homogeneously contoured image sets for robust training. However, these are not easily obtained from the standard clinical protocols as the OARs delineated may vary depending on the patient's tumor site and specific treatment plan. This results in incomplete or partially labeled data. This paper presents a solution to train a robust DL-based automated segmentation tool exploiting a clinical partially labeled dataset. We propose a two-step workflow for OAR segmentation: first, we developed longitudinal OAR-specific 3D segmentation models for pseudo-contour generation, completing the missing contours for some patients; with all OAR available, we trained a multi-class 3D convolutional neural network (nnU-Net) for final OAR segmentation. Results obtained in 44 independent datasets showed superior performance of the proposed methodology for the segmentation of fifteen OARs, with an average Dice score coefficient and surface Dice similarity coefficient of 80.59% and 88.74%. We demonstrated that the model can be straightforwardly integrated into the clinical workflow for standard and adaptive radiotherapy.

Dosimetric comparison of automatically propagated prostate contours with manually drawn contours in MRI-guided radiotherapy: A step towards a contouring free workflow?

Background

The prostate demonstrates inter- and intra- fractional changes and thus adaptive radiotherapy would be required to ensure optimal coverage. Daily adaptive radiotherapy for MRI-guided radiotherapy can be both time and resource intensive when structure delineation is completed manually. Contours can be auto-generated on the MR-Linac via a deformable image registration (DIR) based mapping process from the reference image. This study evaluates the performance of automatically generated target structure contours against manually delineated contours by radiation oncologists for prostate radiotherapy on the Elekta Unity MR-Linac.

Methods

Plans were generated from prostate contours propagated by DIR and rigid image registration (RIR) for forty fractions from ten patients. A two-dose level SIB (simultaneous integrated boost) IMRT plan is used to treat localised prostate cancer; 6000 cGy to the prostate and 4860 cGy to the seminal vesicles. The dose coverage of the PTV 6000 and PTV 4860 created from the manually drawn target structures was evaluated with each plan. If the dose objectives were met, the plan was considered successful in covering the gold standard (clinician-delineated) volume.

Results

The mandatory PTV 6000 dose objective (D98% > 5580 cGy) was met in 81 % of DIR plans and 45 % of RIR plans. The SV were mapped by DIR only and for all the plans, the PTV 4860 dose objective met the optimal target (D98% > 4617 cGy). The plans created by RIR led to under-coverage of the clinician-delineated prostate, predominantly at the apex or the bladder-prostate interface.

Conclusion

Plans created from DIR propagation of prostate contours outperform those created from RIR propagation. In approximately 1 in 5 DIR plans, dosimetric coverage of the gold standard PTV was not clinically acceptable. Thus, at our institution, we use a combination of DIR propagation of contours alongside manual editing of contours where deemed necessary for online treatments.

Auto-segmentation for total marrow irradiation

Purpose

To evaluate the accuracy and efficiency of Artificial-Intelligence (AI) segmentation in Total Marrow Irradiation (TMI) including contours throughout the head and neck (H&N), thorax, abdomen, and pelvis.

Methods

An AI segmentation software was clinically introduced for total body contouring in TMI including 27 organs at risk (OARs) and 4 planning target volumes (PTVs). This work compares the clinically utilized contours to the AI-TMI contours for 21 patients. Structure and image dicom data was used to generate comparisons including volumetric, spatial, and dosimetric variations between the AI- and human-edited contour sets. Conventional volume and surface measures including the Sørensen-Dice coefficient (Dice) and the 95th% Hausdorff Distance (HD95) were used, and novel efficiency metrics were introduced. The clinical efficiency gains were estimated by the percentage of the AI-contour-surface within 1mm of the clinical contour surface. An unedited AI-contour has an efficiency gain=100%, an AI-contour with 70% of its surface<1mm from a clinical contour has an efficiency gain of 70%. The dosimetric deviations were estimated from the clinical dose distribution to compute the dose volume histogram (DVH) for all structures.

Results

A total of 467 contours were compared in the 21 patients. In PTVs, contour surfaces deviated by >1mm in 38.6% ± 23.1% of structures, an average efficiency gain of 61.4%. Deviations >5mm were detected in 12.0% ± 21.3% of the PTV contours. In OARs, deviations >1mm were detected in 24.4% ± 27.1% of the structure surfaces and >5mm in 7.2% ± 18.0%; an average clinical efficiency gain of 75.6%. In H&N OARs, efficiency gains ranged from 42% in optic chiasm to 100% in eyes (unedited in all cases). In thorax, average efficiency gains were >80% in spinal cord, heart, and both lungs. Efficiency gains ranged from 60-70% in spleen, stomach, rectum, and bowel and 75-84% in liver, kidney, and bladder. DVH differences exceeded 0.05 in 109/467 curves at any dose level. The most common 5%-DVH variations were in esophagus (86%), rectum (48%), and PTVs (22%).

Conclusions

AI auto-segmentation software offers a powerful solution for enhanced efficiency in TMI treatment planning. Whole body segmentation including PTVs and normal organs was successful based on spatial and dosimetric comparison.

[What do we need to deliver "online" adapted radiotherapy treatment plans?]

During the joint SFRO/SFPM session of the 2019 congress, a state of the art of adaptive radiotherapy announced a strong impact in our clinical practice, in particular with the availability of treatment devices coupled to an MRI system. Three years later, it seems relevant to take stock of adaptive radiotherapy in practice, and especially the "online" strategy because it is indeed more and more accessible with recent hardware and software developments, such as coupled accelerators to a three-dimensional imaging device and algorithms based on artificial intelligence. However, the deployment of this promising strategy is complex because it contracts the usual time scale and upsets the usual organizations. So what do we need to deliver adapted treatment plans with an "online" strategy?

Artificial intelligence in radiotherapy

Radiotherapy is a discipline closely integrated with computer science. Artificial intelligence (AI) has developed rapidly over the past few years. With the explosive growth of medical big data, AI promises to revolutionize the field of radiotherapy through highly automated workflow, enhanced quality assurance, improved regional balances of expert experiences, and individualized treatment guided by multi-omics. In addition to independent researchers, the increasing number of large databases, biobanks, and open challenges significantly facilitated AI studies on radiation oncology. This article reviews the latest research, clinical applications, and challenges of AI in each part of radiotherapy including image processing, contouring, planning, quality assurance, motion management, and outcome prediction. By summarizing cutting-edge findings and challenges, we aim to inspire researchers to explore more future possibilities and accelerate the arrival of AI radiotherapy.

Integrating Structure Propagation Uncertainties in the Optimization of Online Adaptive Proton Therapy Plans

Currently, adaptive strategies require time- and resource-intensive manual structure corrections. This study compares different strategies: optimization without manual structure correction, adaptation with physician-drawn structures, and no adaptation. Strategies were compared for 16 patients with pancreas, liver, and head and neck (HN) cancer with 1-5 repeated images during treatment: 'reference adaptation', with structures drawn by a physician; 'single-DIR adaptation', using a single set of deformably propagated structures; 'multi-DIR adaptation', using robust planning with multiple deformed structure sets; 'conservative adaptation', using the intersection and union of all deformed structures; 'probabilistic adaptation', using the probability of a voxel belonging to the structure in the optimization weight; and 'no adaptation'. Plans were evaluated using reference structures and compared using a scoring system. The reference adaptation with physician-drawn structures performed best, and no adaptation performed the worst. For pancreas and liver patients, adaptation with a single DIR improved the plan quality over no adaptation. For HN patients, integrating structure uncertainties brought an additional benefit. If resources for manual structure corrections would prevent online adaptation, manual correction could be replaced by a fast 'plausibility check', and plans could be adapted with correction-free adaptation strategies. Including structure uncertainties in the optimization has the potential to make online adaptation more automatable.