Clinical assessment of a novel machine-learning automated contouring tool for radiotherapy planning

Contouring has become an increasingly important aspect of radiotherapy due to inverse planning. Several studies have suggested that the clinical implementation of automated contouring tools can reduce inter-observer variation while increasing contouring efficiency, thereby improving the quality of radiotherapy treatment and reducing the time between simulation and treatment. In this study, a novel, commercial automated contouring tool based on machine learning, the AI-Rad Companion Organs RT™ (AI-Rad) software (Version VA31) (Siemens Healthineers, Munich, Germany), was assessed against both manually delineated contours and another commercially available automated contouring software, Varian Smart Segmentation™ (SS) (Version 16.0) (Varian, Palo Alto, CA, United States). The quality of contours generated by AI-Rad in Head and Neck (H&N), Thorax, Breast, Male Pelvis (Pelvis_M), and Female Pelvis (Pevis_F) anatomical areas was evaluated both quantitatively and qualitatively using several metrics. A timing analysis was subsequently performed to explore potential time savings achieved by AI-Rad. Results showed that most automated contours generated by AI-Rad were not only clinically acceptable and required minimal editing, but also superior in quality to contours generated by SS in multiple structures. In addition, timing analysis favored AI-Rad over manual contouring, indicating the largest time saving (753s per patient) in the Thorax area. AI-Rad was concluded to be a promising automated contouring solution that generated clinically acceptable contours and achieved time savings, thereby greatly benefiting the radiotherapy process.

Automating the treatment planning process for 3D-conformal pediatric craniospinal irradiation therapy

PURPOSE

Pediatric patients with medulloblastoma in low- and middle-income countries (LMICs) are most treated with 3D-conformal photon craniospinal irradiation (CSI), a time-consuming, complex treatment to plan, especially in resource-constrained settings. Therefore, we developed and tested a 3D-conformal CSI autoplanning tool for varying patient lengths.

METHODS AND MATERIALS

Autocontours were generated with a deep learning model trained:tested (80:20 ratio) on 143 pediatric medulloblastoma CT scans (patient ages: 2-19 years, median = 7 years). Using the verified autocontours, the autoplanning tool generated two lateral brain fields matched to a single spine field, an extended single spine field, or two matched spine fields. Additional spine subfields were added to optimize the corresponding dose distribution. Feathering was implemented (yielding nine to 12 fields) to give a composite plan. Each planning approach was tested on six patients (ages 3-10 years). A pediatric radiation oncologist assessed clinical acceptability of each autoplan.

RESULTS

The autocontoured structures' average Dice similarity coefficient ranged from .65 to .98. The average V95 for the brain/spinal canal for single, extended, and multi-field spine configurations was 99.9% ± 0.06%/99.9% ± 0.10%, 99.9% ± 0.07%/99.4% ± 0.30%, and 99.9% ± 0.06%/99.4% ± 0.40%, respectively. The average maximum dose across all field configurations to the brainstem, eyes (L/R), lenses (L/R), and spinal cord were 23.7 ± 0.08, 24.1 ± 0.28, 13.3 ± 5.27, and 25.5 ± 0.34 Gy, respectively (prescription = 23.4 Gy/13 fractions). Of the 18 plans tested, all were scored as clinically acceptable as-is or clinically acceptable with minor, time-efficient edits preferred or required. No plans were scored as clinically unacceptable.

CONCLUSION

The autoplanning tool successfully generated pediatric CSI plans for varying patient lengths in 3.50 ± 0.4 minutes on average, indicating potential for an efficient planning aid in a resource-constrained settings.

A Systematic Quality Scoring Analysis to Assess Automated Cardiovascular Magnetic Resonance Segmentation Algorithms

Background

The quantitative measures used to assess the performance of automated methods often do not reflect the clinical acceptability of contouring. A quality-based assessment of automated cardiac magnetic resonance (CMR) segmentation more relevant to clinical practice is therefore needed.

Objective

We propose a new method for assessing the quality of machine learning (ML) outputs. We evaluate the clinical utility of the proposed method as it is employed to systematically analyse the quality of an automated contouring algorithm.

Methods

A dataset of short-axis (SAX) cine CMR images from a clinically heterogeneous population (n = 217) were manually contoured by a team of experienced investigators. On the same images we derived automated contours using a ML algorithm. A contour quality scoring application randomly presented manual and automated contours to four blinded clinicians, who were asked to assign a quality score from a predefined rubric. Firstly, we analyzed the distribution of quality scores between the two contouring methods across all clinicians. Secondly, we analyzed the interobserver reliability between the raters. Finally, we examined whether there was a variation in scores based on the type of contour, SAX slice level, and underlying disease.

Results

The overall distribution of scores between the two methods was significantly different, with automated contours scoring better than the manual (OR (95% CI) = 1.17 (1.07–1.28), p = 0.001; n = 9401). There was substantial scoring agreement between raters for each contouring method independently, albeit it was significantly better for automated segmentation (automated: AC2 = 0.940, 95% CI, 0.937–0.943 vs manual: AC2 = 0.934, 95% CI, 0.931–0.937; p = 0.006). Next, the analysis of quality scores based on different factors was performed. Our approach helped identify trends patterns of lower segmentation quality as observed for left ventricle epicardial and basal contours with both methods. Similarly, significant differences in quality between the two methods were also found in dilated cardiomyopathy and hypertension.

Conclusions

Our results confirm the ability of our systematic scoring analysis to determine the clinical acceptability of automated contours. This approach focused on the contours' clinical utility could ultimately improve clinicians' confidence in artificial intelligence and its acceptability in the clinical workflow.

Synthetic CT-aided multiorgan segmentation for CBCT-guided adaptive pancreatic radiotherapy

PURPOSE

The delineation of organs at risk (OARs) is fundamental to cone-beam CT (CBCT)-based adaptive radiotherapy treatment planning, but is time consuming, labor intensive, and subject to interoperator variability. We investigated a deep learning-based rapid multiorgan delineation method for use in CBCT-guided adaptive pancreatic radiotherapy.

METHODS

To improve the accuracy of OAR delineation, two innovative solutions have been proposed in this study. First, instead of directly segmenting organs on CBCT images, a pretrained cycle-consistent generative adversarial network (cycleGAN) was applied to generating synthetic CT images given CBCT images. Second, an advanced deep learning model called mask-scoring regional convolutional neural network (MS R-CNN) was applied on those synthetic CT to detect the positions and shapes of multiple organs simultaneously for final segmentation. The OAR contours delineated by the proposed method were validated and compared with expert-drawn contours for geometric agreement using the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square distance (RMS).

RESULTS

Across eight abdominal OARs including duodenum, large bowel, small bowel, left and right kidneys, liver, spinal cord, and stomach, the geometric comparisons between automated and expert contours are as follows: 0.92 (0.89-0.97) mean DSC, 2.90 mm (1.63-4.19 mm) mean HD95, 0.89 mm (0.61-1.36 mm) mean MSD, and 1.43 mm (0.90-2.10 mm) mean RMS. Compared to the competing methods, our proposed method had significant improvements (p < 0.05) in all the metrics for all the eight organs. Once the model was trained, the contours of eight OARs can be obtained on the order of seconds.

CONCLUSIONS

We demonstrated the feasibility of a synthetic CT-aided deep learning framework for automated delineation of multiple OARs on CBCT. The proposed method could be implemented in the setting of pancreatic adaptive radiotherapy to rapidly contour OARs with high accuracy.