A deep image-to-image network organ segmentation algorithm for radiation treatment planning: principles and evaluation

Safety and efficiency of a fully automatic workflow for auto-segmentation in radiotherapy using three commercially available deep learning-based applications

Advancements in radiotherapy auto-segmentation necessitate reliable and efficient workflows. Therefore, a standardized fully automatic workflow was developed for three commercially available deep learning-based auto-segmentation applications and compared to a manual workflow for safety and efficiency. The workflow underwent safety evaluation with failure mode and effects analysis. Notably, eight failure modes were reduced, including seven with severity factors ≥7, indicating the effect on patients, and two with Risk Priority Number value >125, which assesses relative risk level. Efficiency, measured by mouse clicks, showed zero clicks with the automatic workflow. This automation illustrated improvement in both safety and efficiency of workflow.

Deep Learning-Based Approach for Identifying and Measuring Focal Liver Lesions on Contrast-Enhanced MRI

BACKGROUND

The number of focal liver lesions (FLLs) detected by imaging has increased worldwide, highlighting the need to develop a robust, objective system for automatically detecting FLLs.

PURPOSE

To assess the performance of the deep learning-based artificial intelligence (AI) software in identifying and measuring lesions on contrast-enhanced magnetic resonance imaging (MRI) images in patients with FLLs.

STUDY TYPE

Retrospective.

SUBJECTS

395 patients with 1149 FLLs.

FIELD STRENGTH/SEQUENCE

The 1.5 T and 3 T scanners, including T1-, T2-, diffusion-weighted imaging, in/out-phase imaging, and dynamic contrast-enhanced imaging.

ASSESSMENT

The diagnostic performance of AI, radiologist, and their combination was compared. Using 20 mm as the cut-off value, the lesions were divided into two groups, and then divided into four subgroups: <10, 10-20, 20-40, and ≥40 mm, to evaluate the sensitivity of radiologists and AI in the detection of lesions of different sizes. We compared the pathologic sizes of 122 surgically resected lesions with measurements obtained using AI and those made by radiologists.

STATISTICAL TESTS

McNemar test, Bland-Altman analyses, Friedman test, Pearson's chi-squared test, Fisher's exact test, Dice coefficient, and intraclass correlation coefficients. A P-value <0.05 was considered statistically significant.

RESULTS

The average Dice coefficient of AI in segmentation of liver lesions was 0.62. The combination of AI and radiologist outperformed the radiologist alone, with a significantly higher detection rate (0.894 vs. 0.825) and sensitivity (0.883 vs. 0.806). The AI showed significantly sensitivity than radiologists in detecting all lesions <20 mm (0.848 vs. 0.788). Both AI and radiologists achieved excellent detection performance for lesions ≥20 mm (0.867 vs. 0.881, P = 0.671). A remarkable agreement existed in the average tumor sizes among the three measurements (P = 0.174).

DATA CONCLUSION

AI software based on deep learning exhibited practical value in automatically identifying and measuring liver lesions.

LEVEL OF EVIDENCE: 4

TECHNICAL EFFICACY

Stage 2.

Anatomically aware dual-hop learning for pulmonary embolism detection in CT pulmonary angiograms

Pulmonary Embolisms (PE) represent a leading cause of cardiovascular death. While medical imaging, through computed tomographic pulmonary angiography (CTPA), represents the gold standard for PE diagnosis, it is still susceptible to misdiagnosis or significant diagnosis delays, which may be fatal for critical cases. Despite the recently demonstrated power of deep learning to bring a significant boost in performance in a wide range of medical imaging tasks, there are still very few published researches on automatic pulmonary embolism detection. Herein we introduce a deep learning based approach, which efficiently combines computer vision and deep neural networks for pulmonary embolism detection in CTPA. Our method brings novel contributions along three orthogonal axes: (1) automatic detection of anatomical structures; (2) anatomical aware pretraining, and (3) a dual-hop deep neural net for PE detection. We obtain state-of-the-art results on the publicly available multicenter large-scale RSNA dataset.

Evaluation of a deep image-to-image network (DI2IN) auto-segmentation algorithm across a network of cancer centers

PURPOSE/OBJECTIVE S

Due to manual OAR contouring challenges, various automatic contouring solutions have been introduced. Historically, common clinical auto-segmentation algorithms used were atlas-based, which required maintaining a library of self-made contours. Searching the collection was computationally intensive and could take several minutes to complete. Deep learning approaches have shown significant benefits compared to atlas-based methods in improving segmentation accuracy and efficiency in auto-segmentation algorithms. This work represents the first multi-institutional study to describe and evaluate an AI algorithm for the auto-segmentation of organs at risk (OARs) based on a deep image-to-image network (DI2IN).

MATERIALS/METHODS

The AI-Rad Companion Organs RT (AIRC) algorithm (Siemens Healthineers, Erlangen, Germany) uses a two-step approach for segmentation. In the first step, the target organ region in the optimal input image is extracted using a trained deep reinforcement learning network (DRL), which is then used as input to create the contours in the second step based on DI2IN. The study was initially designed as a prospective single-center evaluation. The automated contours generated by AIRC were evaluated by three experienced board-certified radiation oncologists using a four-point scale where 4 is clinically usable and 1 requires re-contouring. After seeing favorable results in a single-center pilot study, we decided to expand the study to six additional institutions, encompassing eight additional evaluators for a total of 11 physician evaluators across seven institutions.

RESULTS

One hundred and fifty-six patients and 1366 contours were prospectively evaluated. The five most commonly contoured organs were the lung (136 contours, average rating = 4.0), spinal cord (106 contours, average rating = 3.1), eye globe (80 contours, average rating = 3.9), lens (77 contours, average rating = 3.9), and optic nerve (75 contours, average rating = 4.0). The average rating per evaluator per contour was 3.6. On average, 124 contours were evaluated by each evaluator. 65% of the contours were rated as 4, and 31% were rated as 3. Only 4% of contours were rated as 1 or 2. Thirty-three organs were evaluated in the study, with 19 structures having a 3.5 or above average rating (ribs, abdominopelvic cavity, skeleton, larynx, lung, aorta, brachial plexus, lens, eye globe, glottis, heart, parotid glands, bladder, kidneys, supraglottic larynx, submandibular glands, esophagus, optic nerve, oral cavity) and the remaining organs having a rating of 3.0 or greater (female breast, proximal femur, seminal vesicles, rectum, sternum, brainstem, prostate, brain, lips, mandible, liver, optic chiasm, spinal cord, spleen). No organ had an average rating below 3.

CONCLUSION

AIRC performed well with greater than 95% of contours accepted by treating physicians with no or minor edits. It supported a fully automated workflow with the potential for time savings and increased standardization with the use of AI-powered algorithms for high-quality OAR contouring.

Performance analysis and knowledge-based quality assurance of critical organ auto-segmentation for pediatric craniospinal irradiation

Craniospinal irradiation (CSI) is a vital therapeutic approach utilized for young patients suffering from central nervous system disorders such as medulloblastoma. The task of accurately outlining the treatment area is particularly time-consuming due to the presence of several sensitive organs at risk (OAR) that can be affected by radiation. This study aimed to assess two different methods for automating the segmentation process: an atlas technique and a deep learning neural network approach. Additionally, a novel method was devised to prospectively evaluate the accuracy of automated segmentation as a knowledge-based quality assurance (QA) tool. Involving a patient cohort of 100, ranging in ages from 2 to 25 years with a median age of 8, this study employed quantitative metrics centered around overlap and distance calculations to determine the most effective approach for practical clinical application. The contours generated by two distinct methods of atlas and neural network were compared to ground truth contours approved by a radiation oncologist, utilizing 13 distinct metrics. Furthermore, an innovative QA tool was conceptualized, designed for forthcoming cases based on the baseline dataset of 100 patient cases. The calculated metrics indicated that, in the majority of cases (60.58%), the neural network method demonstrated a notably higher alignment with the ground truth. Instances where no difference was observed accounted for 31.25%, while utilization of the atlas method represented 8.17%. The QA tool results showed that the two approaches achieved 100% agreement in 39.4% of instances for the atlas method and in 50.6% of instances for the neural network auto-segmentation. The results indicate that the neural network approach showcases superior performance, and its significantly closer physical alignment to ground truth contours in the majority of cases. The metrics derived from overlap and distance measurements have enabled clinicians to discern the optimal choice for practical clinical application.

Multicenter Study of Pelvic Nodal Autosegmentation Algorithm of Siemens Healthineers: Comparison of Male Versus Female Pelvis

Purpose

The autosegmentation algorithm of Siemens Healthineers version VA 30 (AASH) (Siemens Healthineers, Erlangen, Germany) was trained and developed in the male pelvis, with no published data on its usability in the female pelvis. This is the first multi-institutional study to describe and evaluate an artificial intelligence algorithm for autosegmentation of the pelvic nodal region by gender.

Methods and Materials

We retrospectively evaluated AASH pelvic nodal autosegmentation in both male and female patients treated at our network of institutions. The automated pelvic nodal contours generated by AASH were evaluated by 1 board-certified radiation oncologist. A 4-point scale was used for each nodal region contour: a score of 4 is clinically usable with minimal edits; a score of 3 requires minor edits (missing nodal contour region, cutting through vessels, or including bowel loops) in 3 or fewer computed tomography slices; a score of 2 requires major edits, as previously defined but in 4 or more computed tomography slices; and a score of 1 requires complete recontouring of the region. Pelvic nodal regions included the right and left side of the common iliac, external iliac, internal iliac, obturator, and midline presacral nodes. In addition, patients were graded based on their lowest nodal contour score. Statistical analysis was performed using Fisher exact tests and Yates-corrected χ2 tests.

Results

Fifty-two female and 51 male patients were included in the study, representing a total of 468 and 447 pelvic nodal regions, respectively. Ninety-six percent and 99% of contours required minor edits at most (score of 3 or 4) for female and male patients, respectively (P = .004 using Fisher exact test; P = .007 using Yates correction). No nodal regions had a statistically significant difference in scores between female and male patients. The percentage of patients requiring no more than minor edits was 87% (45 patients) and 92% (47 patients) for female and male patients, respectively (P = .53 using Fisher exact test; P = .55 using Yates correction).

Conclusions

AASH pelvic nodal autosegmentation performed very well in both male and female pelvic nodal regions, although with better male pelvic nodal autosegmentation. As autosegmentation becomes more widespread, it may be important to have equal representation from all sexes in training and validation of autosegmentation algorithms.

New Directions in Diagnostics for Aortic Aneurysms: Biomarkers and Machine Learning

This review article presents an appraisal of pioneering technologies poised to revolutionize the diagnosis and management of aortic aneurysm disease, with a primary focus on the thoracic aorta while encompassing insights into abdominal manifestations. Our comprehensive analysis is rooted in an exhaustive survey of contemporary and historical research, delving into the realms of machine learning (ML) and computer-assisted diagnostics. This overview draws heavily upon relevant studies, including Siemens' published field report and many peer-reviewed publications. At the core of our survey lies an in-depth examination of ML-driven diagnostic advancements, dissecting an array of algorithmic suites to unveil the foundational concepts anchoring computer-assisted diagnostics and medical image processing. Our review extends to a discussion of circulating biomarkers, synthesizing insights gleaned from our prior research endeavors alongside contemporary studies gathered from the PubMed Central database. We elucidate the prevalent challenges and envisage the potential fusion of AI-guided aortic measurements and sophisticated ML frameworks with the computational analyses of pertinent biomarkers. By framing current scientific insights, we contemplate the transformative prospect of translating fundamental research into practical diagnostic tools. This narrative not only illuminates present strides, but also forecasts promising trajectories in the clinical evaluation and therapeutic management of aortic aneurysm disease.

"Under the hood": artificial intelligence in personalized radiotherapy

This review presents and discusses the ways in which artificial intelligence (AI) tools currently intervene, or could potentially intervene in the future, to enhance the diverse tasks involved in the radiotherapy workflow. The radiotherapy framework is presented on 2 different levels for the personalization of the treatment, distinct in tasks and methodologies. The first level is the clinically well-established anatomy-based workflow, known as adaptive radiation therapy. The second level is referred to as biology-driven workflow, explored in the research literature and recently appearing in some preliminary clinical trials for personalized radiation treatments. A 2-fold role for AI is defined according to these 2 different levels. In the anatomy-based workflow, the role of AI is to streamline and improve the tasks in terms of time and variability reductions compared to conventional methodologies. The biology-driven workflow instead fully relies on AI, which introduces decision-making tools opening uncharted frontiers that were in the past deemed challenging to explore. These methodologies are referred to as radiomics and dosiomics, handling imaging and dosimetric information, or multiomics, when complemented by clinical and biological parameters (ie, biomarkers). The review explicitly highlights the methodologies that are currently incorporated into clinical practice or still in research, with the aim of presenting the AI's growing role in personalized radiotherapy.

Feasibility evaluation of novel AI-based deep-learning contouring algorithm for radiotherapy

PURPOSE

To evaluate the clinical feasibility of the Siemens Healthineers AI-Rad Companion Organs RT VA30A (Organs-RT) auto-contouring algorithm for organs at risk (OARs) of the pelvis, thorax, and head and neck (H&N).

METHODS

Computed tomography (CT) datasets from 30 patients (10 pelvis, 10 thorax, and 10 H&N) were collected. Four sets of OARs were generated on each scan, one set by Organs-RT and the others by three experienced users independently. A physician (expert) then evaluated each contour by assigning a score from the following scale: 1-Must Redo, 2-Major Edits, 3-Minor Edits, 4-Clinically usable. Using the highest-scored OAR from the human users as a reference, the contours generated by Organs-RT were evaluated via Dice Similarity Coefficient (DSC), Hausdorff Distance (HDD), Mean Distance to Agreement (mDTA), Volume comparison, and visual inspection. Additionally, each human user recorded the time to delineate each structure set and time-saving efficiency was measured.

RESULTS

The average DSC obtained for the pelvic OARs ranged between (0.81 ± 0.06)Rectum and (0.94 ± 0.03)Bladder . (0.75 ± 0.09)Esophagus to( 0.96 ± 0.02 ) Rt . Lung ${( {0.96 \pm 0.02} )}_{{\mathrm{Rt}}.{\mathrm{\ Lung}}}$ for the thoracic OARs and (0.66 ± 0.07)Lips to (0.83 ± 0.04)Brainstem for the H&N. The average HDD in cm for the pelvis cohort ranged between (0.95 ± 0.35)Bladder to (3.62 ± 2.50)Rectum , (0.42 ± 0.06)SpinalCord to (2.09 ± 2.00)Esophagus for the thoracic set and( 0.53 ± 0.22 ) Cerv _ SpinalCord ${( {0.53 \pm 0.22} )}_{{\mathrm{Cerv}}\_{\mathrm{SpinalCord}}}$ to (1.50 ± 0.50)Mandible for the H&N region. The time-saving efficiency was 67% for H&N, 83% for pelvis, and 84% for thorax. 72.5%, 82%, and 50% of the pelvis, thorax, and H&N OARs were scored as clinically usable by the expert, respectively.

CONCLUSIONS

The highest agreement registered between OARs generated by Organs-RT and their respective references was for the bladder, heart, lungs, and femoral heads, with an overall DSC≥0.92. The poorest agreement was for the rectum, esophagus, and lips, with an overall DSC⩽0.81. Nonetheless, Organs-RT serves as a reliable auto-contouring tool by minimizing overall contouring time and increasing time-saving efficiency in radiotherapy treatment planning.

Clinical Validation of Siemens' Syngo.via Automatic Contouring System

Purpose

The manual delineation of organs at risk is a process that requires a great deal of time both for the technician and for the physician. Availability of validated software tools assisted by artificial intelligence would be of great benefit, as it would significantly improve the radiation therapy workflow, reducing the time required for segmentation. The purpose of this article is to validate the deep learning-based autocontouring solution integrated in syngo.via RT Image Suite VB40 (Siemens Healthineers, Forchheim, Germany).

Methods and Materials

For this purpose, we have used our own specific qualitative classification system, RANK, to evaluate more than 600 contours corresponding to 18 different automatically delineated organs at risk. Computed tomography data sets of 95 different patients were included: 30 patients with lung, 30 patients with breast, and 35 male patients with pelvic cancer. The automatically generated structures were reviewed in the Eclipse Contouring module independently by 3 observers: an expert physician, an expert technician, and a junior physician.

Results

There is a statistically significant difference between the Dice coefficient associated with RANK 4 compared with the coefficient associated with RANKs 2 and 3 (P < .001). In total, 64% of the evaluated structures received the maximum score, 4. Only 1% of the structures were classified with the lowest score, 1. The time savings for breast, thorax, and pelvis were 87.6%, 93.5%, and 82.2%, respectively.

Conclusions

Siemens' syngo.via RT Image Suite offers good autocontouring results and significant time savings.