Segmentation editing improves efficiency while reducing inter-expert variation and maintaining accuracy for normal brain tissues in the presence of space-occupying lesions

Potential of E-Learning Interventions and Artificial Intelligence-Assisted Contouring Skills in Radiotherapy: The ELAISA Study

PURPOSE

Most research on artificial intelligence-based auto-contouring as template (AI-assisted contouring) for organs-at-risk (OARs) stem from high-income countries. The effect and safety are, however, likely to depend on local factors. This study aimed to investigate the effects of AI-assisted contouring and teaching on contouring time and contour quality among radiation oncologists (ROs) working in low- and middle-income countries (LMICs).

MATERIALS AND METHODS

Ninety-seven ROs were randomly assigned to either manual or AI-assisted contouring of eight OARs for two head-and-neck cancer cases with an in-between teaching session on contouring guidelines. Thereby, the effect of teaching (yes/no) and AI-assisted contouring (yes/no) was quantified. Second, ROs completed short-term and long-term follow-up cases all using AI assistance. Contour quality was quantified with Dice Similarity Coefficient (DSC) between ROs' contours and expert consensus contours. Groups were compared using absolute differences in medians with 95% CIs.

RESULTS

AI-assisted contouring without previous teaching increased absolute DSC for optic nerve (by 0.05 [0.01; 0.10]), oral cavity (0.10 [0.06; 0.13]), parotid (0.07 [0.05; 0.12]), spinal cord (0.04 [0.01; 0.06]), and mandible (0.02 [0.01; 0.03]). Contouring time decreased for brain stem (-1.41 [-2.44; -0.25]), mandible (-6.60 [-8.09; -3.35]), optic nerve (-0.19 [-0.47; -0.02]), parotid (-1.80 [-2.66; -0.32]), and thyroid (-1.03 [-2.18; -0.05]). Without AI-assisted contouring, teaching increased DSC for oral cavity (0.05 [0.01; 0.09]) and thyroid (0.04 [0.02; 0.07]), and contouring time increased for mandible (2.36 [-0.51; 5.14]), oral cavity (1.42 [-0.08; 4.14]), and thyroid (1.60 [-0.04; 2.22]).

CONCLUSION

The study suggested that AI-assisted contouring is safe and beneficial to ROs working in LMICs. Prospective clinical trials on AI-assisted contouring should, however, be conducted upon clinical implementation to confirm the effects.

The effect of editing clinical contours on deep-learning segmentation accuracy of the gross tumor volume in glioblastoma

Background and purpose

Deep-learning (DL) models for segmentation of the gross tumor volume (GTV) in radiotherapy are generally based on clinical delineations which suffer from inter-observer variability. The aim of this study was to compare performance of a DL-model based on clinical glioblastoma GTVs to a model based on a single-observer edited version of the same GTVs.

Materials and methods

The dataset included imaging data (Computed Tomography (CT), T1, contrast-T1 (T1C), and fluid-attenuated-inversion-recovery (FLAIR)) of 259 glioblastoma patients treated with post-operative radiotherapy between 2012 and 2019 at a single institute. The clinical GTVs were edited using all imaging data. The dataset was split into 207 cases for training/validation and 52 for testing.GTV segmentation models (nnUNet) were trained on clinical and edited GTVs separately and compared using Surface Dice with 1 mm tolerance (sDSC1mm). We also evaluated model performance with respect to extent of resection (EOR), and different imaging combinations (T1C/T1/FLAIR/CT, T1C/FLAIR/CT, T1C/FLAIR, T1C/CT, T1C/T1, T1C). A Wilcoxon test was used for significance testing.

Results

The median (range) sDSC1mm of the clinical-GTV-model and edited-GTV-model both evaluated with the edited contours, was 0.76 (0.43-0.94) vs. 0.92 (0.60-0.98) respectively (p < 0.001). sDSC1mm was not significantly different between patients with a biopsy, partial, and complete resection. T1C as single input performed as good as use of imaging combinations.

Conclusions

High segmentation accuracy was obtained by the DL-models. Editing of the clinical GTVs significantly increased DL performance with a relevant effect size. DL performance was robust for EOR and highly accurate using only T1C.

A radiomics-incorporated deep ensemble learning model for multi-parametric MRI-based glioma segmentation

Objective.To develop a deep ensemble learning (DEL) model with radiomics spatial encoding execution for improved glioma segmentation accuracy using multi-parametric magnetic resonance imaging (mp-MRI).Approach.This model was developed using 369 glioma patients with a four-modality mp-MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. In each modality volume, a 3D sliding kernel was implemented across the brain to capture image heterogeneity: 56 radiomic features were extracted within the kernel, resulting in a fourth-order tensor. Each radiomic feature can then be encoded as a 3D image volume, namely a radiomic feature map (RFM). For each patient, all RFMs extracted from all four modalities were processed using principal component analysis for dimension reduction, and the first four principal components (PCs) were selected. Next, a DEL model comprised of four U-Net sub-models was trained for the segmentation of a region-of-interest: each sub-model utilizes the mp-MRI and one of the four PCs as a five-channel input for 2D execution. Last, four softmax probability results given by the DEL model were superimposed and binarized using Otsu's method as the segmentation results. Three DEL models were trained to segment the enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The segmentation results given by the proposed ensemble were compared to the mp-MRI-only U-Net results.Main Results.All three radiomics-incorporated DEL models were successfully implemented: compared to the mp-MRI-only U-net results, the dice coefficients of ET (0.777 → 0.817), TC (0.742 → 0.757), and WT (0.823 → 0.854) demonstrated improvement. The accuracy, sensitivity, and specificity results demonstrated similar patterns.Significance.The adopted radiomics spatial encoding execution enriches the image heterogeneity information that leads to the successful demonstration of the proposed DEL model, which offers a new tool for mp-MRI-based medical image segmentation.

Automated Contouring and Planning in Radiation Therapy: What Is 'Clinically Acceptable'?

Developers and users of artificial-intelligence-based tools for automatic contouring and treatment planning in radiotherapy are expected to assess clinical acceptability of these tools. However, what is 'clinical acceptability'? Quantitative and qualitative approaches have been used to assess this ill-defined concept, all of which have advantages and disadvantages or limitations. The approach chosen may depend on the goal of the study as well as on available resources. In this paper, we discuss various aspects of 'clinical acceptability' and how they can move us toward a standard for defining clinical acceptability of new autocontouring and planning tools.

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

Transparency in Machine Learning (ML), often also referred to as interpretability or explainability, attempts to reveal the working mechanisms of complex models. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i.e., a relationship between algorithm and users. Thus, prototyping and user evaluations are critical to attaining solutions that afford transparency. Following human-centered design principles in highly specialized and high stakes domains, such as medical image analysis, is challenging due to the limited access to end users and the knowledge imbalance between those users and ML designers. To investigate the state of transparent ML in medical image analysis, we conducted a systematic review of the literature from 2012 to 2021 in PubMed, EMBASE, and Compendex databases. We identified 2508 records and 68 articles met the inclusion criteria. Current techniques in transparent ML are dominated by computational feasibility and barely consider end users, e.g. clinical stakeholders. Despite the different roles and knowledge of ML developers and end users, no study reported formative user research to inform the design and development of transparent ML models. Only a few studies validated transparency claims through empirical user evaluations. These shortcomings put contemporary research on transparent ML at risk of being incomprehensible to users, and thus, clinically irrelevant. To alleviate these shortcomings in forthcoming research, we introduce the INTRPRT guideline, a design directive for transparent ML systems in medical image analysis. The INTRPRT guideline suggests human-centered design principles, recommending formative user research as the first step to understand user needs and domain requirements. Following these guidelines increases the likelihood that the algorithms afford transparency and enable stakeholders to capitalize on the benefits of transparent ML.

A comparative study of auto-contouring softwares in delineation of organs at risk in lung cancer and rectal cancer

Radiotherapy requires the target area and the organs at risk to be contoured on the CT image of the patient. During the process of organs-at-Risk (OAR) of the chest and abdomen, the doctor needs to contour at each CT image. The delineations of large and varied shapes are time-consuming and laborious. This study aims to evaluate the results of two automatic contouring softwares on OARs definition of CT images of lung cancer and rectal cancer patients. The CT images of 15 patients with rectal cancer and 15 patients with lung cancer were selected separately, and the organs at risk were manually contoured by experienced physicians as reference structures. And then the same datasets were automatically contoured based on AiContour (version 3.1.8.0, Manufactured by Linking MED, Beijing, China) and Raystation (version 4.7.5.4, Manufactured by Raysearch, Stockholm, Sweden) respectively. Deep learning auto-segmentations and Atlas were respectively performed with AiContour and Raystation. Overlap index (OI), Dice similarity index (DSC) and Volume difference (Dv) were evaluated based on the auto-contours, and independent-sample t-test analysis is applied to the results. The results of deep learning auto-segmentations on OI and DSC were better than that of Atlas with statistical difference. There was no significant difference in Dv between the results of two software. With deep learning auto-segmentations, auto-contouring results of most organs in the chest and abdomen are good, and with slight modification, it can meet the clinical requirements for planning. With Atlas, auto-contouring results in most OAR is not as good as deep learning auto-segmentations, and only the auto-contouring results of some organs can be used clinically after modification.

RootPainter3D: Interactive-machine-learning enables rapid and accurate contouring for radiotherapy

PURPOSE

Organ-at-risk contouring is still a bottleneck in radiotherapy, with many deep learning methods falling short of promised results when evaluated on clinical data. We investigate the accuracy and time-savings resulting from the use of an interactive-machine-learning method for an organ-at-risk contouring task.

METHODS

We implement an open-source interactive-machine-learning software application that facilitates corrective-annotation for deep-learning generated contours on X-ray CT images. A trained-physician contoured 933 hearts using our software by delineating the first image, starting model training, and then correcting the model predictions for all subsequent images. These corrections were added into the training data, which was used for continuously training the assisting model. From the 933 hearts, the same physician also contoured the first 10 and last 10 in Eclipse (Varian) to enable comparison in terms of accuracy and duration.

RESULTS

We find strong agreement with manual delineations, with a dice score of 0.95. The annotations created using corrective-annotation also take less time to create as more images are annotated, resulting in substantial time savings compared to manual methods. After 923 images had been delineated, hearts took 2 min and 2 s to delineate on average, which includes time to evaluate the initial model prediction and assign the needed corrections, compared to 7 min and 1 s when delineating manually.

CONCLUSIONS

Our experiment demonstrates that interactive-machine-learning with corrective-annotation provides a fast and accessible way for non computer-scientists to train deep-learning models to segment their own structures of interest as part of routine clinical workflows.

A national study on the inter-observer variability in the delineation of organs at risk in the brain

BACKGROUND

The Danish Neuro Oncology Group (DNOG) has established national consensus guidelines for the delineation of organs at risk (OAR) structures based on published literature. This study was conducted to finalise these guidelines and evaluate the inter-observer variability of the delineated OAR structures by expert observers.

MATERIAL AND METHODS

The DNOG delineation guidelines were formed by participants from all Danish centres that treat brain tumours with radiotherapy. In a two-day workshop, guidelines were discussed and finalised based on a pilot study. Following this, the ten participants contoured the following OARs on T1-weighted gadolinium enhanced MRI from 13 patients with brain tumours: optic tracts, optic nerves, chiasm, spinal cord, brainstem, pituitary gland and hippocampus. The metrics used for comparison were the Dice similarity coefficient (Dice), mean surface distance (MSD) and others.

RESULTS

A total of 968 contours were delineated across the 13 patients. On average eight (range six to nine) individual contour sets were made per patient. Good agreement was found across all structures with a median MSD below 1 mm for most structures, with the chiasm performing the best with a median MSD of 0.45 mm. The Dice was as expected highly volume dependent, the brainstem (the largest structure) had the highest Dice value with a median of 0.89 whereas smaller volumes such as the chiasm had a Dice of 0.71.

CONCLUSION

Except for the caudal definition of the spinal cord, the variances observed in the contours of OARs in the brain were generally low and consistent. Surface mapping revealed sub-regions of higher variance for some organs. The data set is being prepared as a validation data set for auto-segmentation algorithms for use within the Danish Comprehensive Cancer Centre - Radiotherapy and potential collaborators.

Automatic Intracranial Segmentation: Is the Clinician Still Needed?

INTRODUCTION

Stereotactic hypofractionated radiotherapy is an effective treatment for brain metastases in oligometastatic patients. Its planning is however time-consuming because of the number of organs at risk to be manually segmented. This study evaluates 2 automated segmentation commercial software.

METHODS

Patients were scanned in the treatment position. The computed tomography scan was registered on a magnetic resonance imaging and volumes were manually segmented by a clinician. Then 2 automated segmentations were performed (with iPlan and Smart Segmentation). RT STRUCT files were compared with Aquilab's Artistruct segment comparison module. We selected common segmented volume ratio as the main judging criterion. Secondary criteria were Dice-Sørensen coefficients, overlap ratio, and additional segmented volume.

RESULTS

Twenty consecutive patients were included. Agreement between manual and automated contouring was poor. Common segmented volumes ranged from 7.71% to 82.54%, Dice-Sørensen coefficient ranged from 0.0745 to 0.8398, overlap ratio ranged from 0.0414 to 0.7275, and additional segmented volume ranged from 9.80% to 92.25%. Each software outperformed the other on some organs while performing worse on others.

CONCLUSION

No software seemed clearly better than the other. Common segmented volumes were much too low for routine use in stereotactic hypofractionated brain radiotherapy. Manual editing is still needed.

Multi-modal glioblastoma segmentation: man versus machine

BACKGROUND AND PURPOSE

Reproducible segmentation of brain tumors on magnetic resonance images is an important clinical need. This study was designed to evaluate the reliability of a novel fully automated segmentation tool for brain tumor image analysis in comparison to manually defined tumor segmentations.

METHODS

We prospectively evaluated preoperative MR Images from 25 glioblastoma patients. Two independent expert raters performed manual segmentations. Automatic segmentations were performed using the Brain Tumor Image Analysis software (BraTumIA). In order to study the different tumor compartments, the complete tumor volume TV (enhancing part plus non-enhancing part plus necrotic core of the tumor), the TV+ (TV plus edema) and the contrast enhancing tumor volume CETV were identified. We quantified the overlap between manual and automated segmentation by calculation of diameter measurements as well as the Dice coefficients, the positive predictive values, sensitivity, relative volume error and absolute volume error.

RESULTS

Comparison of automated versus manual extraction of 2-dimensional diameter measurements showed no significant difference (p = 0.29). Comparison of automated versus manual segmentation of volumetric segmentations showed significant differences for TV+ and TV (p<0.05) but no significant differences for CETV (p>0.05) with regard to the Dice overlap coefficients. Spearman's rank correlation coefficients (ρ) of TV+, TV and CETV showed highly significant correlations between automatic and manual segmentations. Tumor localization did not influence the accuracy of segmentation.

CONCLUSIONS

In summary, we demonstrated that BraTumIA supports radiologists and clinicians by providing accurate measures of cross-sectional diameter-based tumor extensions. The automated volume measurements were comparable to manual tumor delineation for CETV tumor volumes, and outperformed inter-rater variability for overlap and sensitivity.