Implementation of deep learning-based auto-segmentation for radiotherapy planning structures: a workflow study at two cancer centers

Fully Automatic Quantitative Measurement of Equilibrium Radionuclide Angiocardiography Using a Convolutional Neural Network

PURPOSE

The aim of this study was to generate deep learning-based regions of interest (ROIs) from equilibrium radionuclide angiography datasets for left ventricular ejection fraction (LVEF) measurement.

PATIENTS AND METHODS

Manually drawn ROIs (mROIs) on end-systolic and end-diastolic images were extracted from reports in a Picture Archiving and Communications System. To reduce observer variability, preprocessed ROIs (pROIs) were delineated using a 41% threshold of the maximal pixel counts of the extracted mROIs and were labeled as ground-truth. Background ROIs were automatically created using an algorithm to identify areas with minimum counts within specified probability areas around the end-systolic ROI. A 2-dimensional U-Net convolutional neural network architecture was trained to generate deep learning-based ROIs (dlROIs) from pROIs. The model's performance was evaluated using Lin's concordance correlation coefficient (CCC). Bland-Altman plots were used to assess bias and 95% limits of agreement.

RESULTS

A total of 41,462 scans (19,309 patients) were included. Strong concordance was found between LVEF measurements from dlROIs and pROIs (CCC = 85.6%; 95% confidence interval, 85.4%-85.9%), and between LVEF measurements from dlROIs and mROIs (CCC = 86.1%; 95% confidence interval, 85.8%-86.3%). In the Bland-Altman analysis, the mean differences and 95% limits of agreement of the LVEF measurements were -0.6% and -6.6% to 5.3%, respectively, for dlROIs and pROIs, and -0.4% and -6.3% to 5.4% for dlROIs and mROIs, respectively. In 37,537 scans (91%), the absolute LVEF difference between dlROIs and mROIs was <5%.

CONCLUSIONS

Our 2-dimensional U-Net convolutional neural network architecture showed excellent performance in generating LV ROIs from equilibrium radionuclide angiography scans. It may enhance the convenience and reproducibility of LVEF measurements.

Clinical evaluation of the efficacy of limbus artificial intelligence software to augment contouring for prostate and nodes radiotherapy

OBJECTIVES

To determine if Limbus, an artificial intelligence (AI) auto-contouring software, can offer meaningful time savings for prostate radiotherapy treatment planning.

METHODS

Three clinical oncologists recorded the time taken to contour prostate and seminal vesicles, lymph nodes, bladder, rectum, bowel, and femoral heads on CT scans for 30 prostate patients (15 prostate, 15 prostate and nodes). Limbus 1.6.0 was used to generate these contours on the 30 CT scans. The time taken by the oncologists to modify individual Limbus contours was noted and compared with manual contouring times. The geometric similarity of Limbus and expert contours was assessed using the Dice Similarity Coefficient (DSC), and the dosimetric impact of using un-edited Limbus organs at risk contours was studied.

RESULTS

Limbus reduced the time to produce clinically acceptable contours by 26 minutes for prostate and nodes patients and by 13 minutes for the prostate only patients. DSC values of greater than 0.7 were calculated for all contours, demonstrating good initial agreement. A dosimetric study showed that 5 of the 20 plans optimized using unmodified AI structures required unnecessary compromise of PTV coverage, highlighting the importance of expert review.

CONCLUSIONS

Limbus offers significant time saving and has become an essential part of our clinical practice.

ADVANCES IN KNOWLEDGE

This article is the first to include bowel and lymph nodes when assessing potential time savings using Limbus software. It demonstrates that Limbus can be used as an aid for prostate and node radiotherapy treatment planning.

A multi-institutional comparison of retrospective deformable dose accumulation for online adaptive magnetic resonance-guided radiotherapy

Background and Purpose

Application of different deformable dose accumulation (DDA) solutions makes institutional comparisons after online-adaptive magnetic resonance-guided radiotherapy (OA-MRgRT) challenging. The aim of this multi-institutional study was to analyze accuracy and agreement of DDA-implementations in OA-MRgRT.

Material and Methods

One gold standard (GS) case deformed with a biomechanical-model and five clinical cases consisting of prostate (2x), cervix, liver, and lymph node cancer, treated with OA-MRgRT, were analyzed. Six centers conducted DDA using institutional implementations. Deformable image registration (DIR) and DDA results were compared using the contour metrics Dice Similarity Coefficient (DSC), surface-DSC, Hausdorff-distance (HD95%), and accumulated dose-volume histograms (DVHs) analyzed via intraclass correlation coefficient (ICC) and clinical dosimetric criteria (CDC).

Results

For the GS, median DDA errors ranged from 0.0 to 2.8 Gy across contours and implementations. DIR of clinical cases resulted in DSC > 0.8 for up to 81.3% of contours and a variability of surface-DSC values depending on the implementation. Maximum HD95%=73.3 mm was found for duodenum in the liver case. Although DVH ICC > 0.90 was found after DDA for all but two contours, relevant absolute CDC differences were observed in clinical cases: Prostate I/II showed maximum differences in bladder V28Gy (10.2/7.6%), while for cervix, liver, and lymph node the highest differences were found for rectum D2cm3 (2.8 Gy), duodenum Dmax (7.1 Gy), and rectum D0.5cm3 (4.6 Gy).

Conclusion

Overall, high agreement was found between the different DIR and DDA implementations. Case- and algorithm-dependent differences were observed, leading to potentially clinically relevant results. Larger studies are needed to define future DDA-guidelines.

Framework for Radiation Oncology Department-wide Evaluation and Implementation of Commercial Artificial Intelligence Autocontouring

PURPOSE

Artificial intelligence (AI)-based autocontouring in radiation oncology has potential benefits such as standardization and time savings. However, commercial AI solutions require careful evaluation before clinical integration. We developed a multidimensional evaluation method to test pretrained AI-based automated contouring solutions across a network of clinics.

METHODS AND MATERIALS

Curated data included 121 patient planning computed tomography (CT) scans with a total of 859 clinically approved contours used for treatment from 4 clinics. Regions of interest (ROIs) were generated with 3 commercial AI-based automated contouring software solutions (AI1, AI2, AI3) spanning the following disease sites: brain, head and neck (H&N), thorax, abdomen, and pelvis. Quantitative agreement between AI-generated and clinical contours was measured by Dice similarity coefficient (DSC) and Hausdorff distance (HD). Qualitative assessment was performed by multiple experts scoring blinded AI-contours using a Likert scale. Workflow and usability surveying was also conducted.

RESULTS

AI1, AI2, and AI3 contours had high quantitative agreement in 27.8%, 32.8%, and 34.1% of cases (DSC >0.9), performing well in pelvis (median DSC = 0.86/0.88/0.91) and thorax (median DSC = 0.91/0.89/0.91). All 3 solutions had low quantitative agreement in 7.4%, 8.8%, and 6.1% of cases (DSC <0.5), performing worse in brain (median DSC = 0.65/0.78/0.75) and H&N (median DSC = 0.76/0.80/0.81). Qualitatively, AI1 and AI2 contours were acceptable (rated 1-2) with at most minor edits in 70.7% and 74.6% of ROIs (2906 ratings), higher for abdomen (AI1: 79.2%) and thorax (AI2: 90.2%), and lower for H&N (29.0/35.6%). An end-user survey showed strong user preference for full automation and mixed preferences for accuracy versus total number of structures generated.

CONCLUSIONS

Our evaluation method provided a comprehensive analysis of both quantitative and qualitative measures of commercially available pretrained AI autocontouring algorithms. The evaluation framework served as a roadmap for clinical integration that aligned with user workflow preference.

Knowledge-based quality assurance of a comprehensive set of organ at risk contours for head and neck radiotherapy

Introduction

Manual review of organ at risk (OAR) contours is crucial for creating safe radiotherapy plans but can be time-consuming and error prone. Statistical and deep learning models show the potential to automatically detect improper contours by identifying outliers using large sets of acceptable data (knowledge-based outlier detection) and may be able to assist human reviewers during review of OAR contours.

Methods

This study developed an automated knowledge-based outlier detection method and assessed its ability to detect erroneous contours for all common head and neck (HN) OAR types used clinically at our institution. We utilized 490 accurate CT-based HN structure sets from unique patients, each with forty-two HN OAR contours when anatomically present. The structure sets were distributed as 80% for training, 10% for validation, and 10% for testing. In addition, 190 and 37 simulated contours containing errors were added to the validation and test sets, respectively. Single-contour features, including location, shape, orientation, volume, and CT number, were used to train three single-contour feature models (z-score, Mahalanobis distance [MD], and autoencoder [AE]). Additionally, a novel contour-to-contour relationship (CCR) model was trained using the minimum distance and volumetric overlap between pairs of OAR contours to quantify overlap and separation. Inferences from single-contour feature models were combined with the CCR model inferences and inferences evaluating the number of disconnected parts in a single contour and then compared.

Results

In the test dataset, before combination with the CCR model, the area under the curve values were 0.922/0.939/0.939 for the z-score, MD, and AE models respectively for all contours. After combination with CCR model inferences, the z-score, MD, and AE had sensitivities of 0.838/0.892/0.865, specificities of 0.922/0.907/0.887, and balanced accuracies (BA) of 0.880/0.900/0.876 respectively. In the validation dataset, with similar overall performance and no signs of overfitting, model performance for individual OAR types was assessed. The combined AE model demonstrated minimum, median, and maximum BAs of 0.729, 0.908, and 0.980 across OAR types.

Discussion

Our novel knowledge-based method combines models utilizing single-contour and CCR features to effectively detect erroneous OAR contours across a comprehensive set of 42 clinically used OAR types for HN radiotherapy.

Multicenter Study of Pelvic Nodal Autosegmentation Algorithm of Siemens Healthineers: Comparison of Male Versus Female Pelvis

Purpose

The autosegmentation algorithm of Siemens Healthineers version VA 30 (AASH) (Siemens Healthineers, Erlangen, Germany) was trained and developed in the male pelvis, with no published data on its usability in the female pelvis. This is the first multi-institutional study to describe and evaluate an artificial intelligence algorithm for autosegmentation of the pelvic nodal region by gender.

Methods and Materials

We retrospectively evaluated AASH pelvic nodal autosegmentation in both male and female patients treated at our network of institutions. The automated pelvic nodal contours generated by AASH were evaluated by 1 board-certified radiation oncologist. A 4-point scale was used for each nodal region contour: a score of 4 is clinically usable with minimal edits; a score of 3 requires minor edits (missing nodal contour region, cutting through vessels, or including bowel loops) in 3 or fewer computed tomography slices; a score of 2 requires major edits, as previously defined but in 4 or more computed tomography slices; and a score of 1 requires complete recontouring of the region. Pelvic nodal regions included the right and left side of the common iliac, external iliac, internal iliac, obturator, and midline presacral nodes. In addition, patients were graded based on their lowest nodal contour score. Statistical analysis was performed using Fisher exact tests and Yates-corrected χ2 tests.

Results

Fifty-two female and 51 male patients were included in the study, representing a total of 468 and 447 pelvic nodal regions, respectively. Ninety-six percent and 99% of contours required minor edits at most (score of 3 or 4) for female and male patients, respectively (P = .004 using Fisher exact test; P = .007 using Yates correction). No nodal regions had a statistically significant difference in scores between female and male patients. The percentage of patients requiring no more than minor edits was 87% (45 patients) and 92% (47 patients) for female and male patients, respectively (P = .53 using Fisher exact test; P = .55 using Yates correction).

Conclusions

AASH pelvic nodal autosegmentation performed very well in both male and female pelvic nodal regions, although with better male pelvic nodal autosegmentation. As autosegmentation becomes more widespread, it may be important to have equal representation from all sexes in training and validation of autosegmentation algorithms.

Validation of a deep-learning segmentation model for adult and pediatric head and neck radiotherapy in different patient positions

Background and purpose

Autocontouring for radiotherapy has the potential to significantly save time and reduce interobserver variability. We aimed to assess the performance of a commercial autocontouring model for head and neck (H&N) patients in eight orientations relevant to particle therapy with fixed beam lines, focusing on validation and implementation for routine clinical use.

Materials and methods

Autocontouring was performed on sixteen organs at risk (OARs) for 98 adult and pediatric patients with 137 H&N CT scans in eight orientations. A geometric comparison of the autocontours and manual segmentations was performed using the Hausdorff Distance 95th percentile, Dice Similarity Coefficient (DSC) and surface DSC and compared to interobserver variability where available. Additional qualitative scoring and dose-volume-histogram (DVH) parameters analyses were performed for twenty patients in two positions, consisting of scoring on a 0-3 scale based on clinical usability and comparing the mean (Dmean) and near-maximum (D2%) dose, respectively.

Results

For the geometric analysis, the model performance in head-first-supine straight and hyperextended orientations was in the same range as the interobserver variability. HD95, DSC and surface DSC was heterogeneous in other orientations. No significant geometric differences were found between pediatric and adult autocontours. The qualitative scoring yielded a median score of ≥ 2 for 13/16 OARs while 7/32 DVH parameters were significantly different.

Conclusions

For head-first-supine straight and hyperextended scans, we found that 13/16 OAR autocontours were suited for use in daily clinical practice and subsequently implemented. Further development is needed for other patient orientations before implementation.

Clinical Use of a Commercial Artificial Intelligence-Based Software for Autocontouring in Radiation Therapy: Geometric Performance and Dosimetric Impact

PURPOSE

When autocontouring based on artificial intelligence (AI) is used in the radiotherapy (RT) workflow, the contours are reviewed and eventually adjusted by a radiation oncologist before an RT treatment plan is generated, with the purpose of improving dosimetry and reducing both interobserver variability and time for contouring. The purpose of this study was to evaluate the results of application of a commercial AI-based autocontouring for RT, assessing both geometric accuracies and the influence on optimized dose from automatically generated contours after review by human operator.

MATERIALS AND METHODS

A commercial autocontouring system was applied to a retrospective database of 40 patients, of which 20 were treated with radiotherapy for prostate cancer (PCa) and 20 for head and neck cancer (HNC). Contours resulting from AI were compared against AI contours reviewed by human operator and human-only contours using Dice similarity coefficient (DSC), Hausdorff distance (HD), and relative volume difference (RVD). Dosimetric indices such as Dmean, D0.03cc, and normalized plan quality metrics were used to compare dose distributions from RT plans generated from structure sets contoured by humans assisted by AI against plans from manual contours. The reduction in contouring time obtained by using automated tools was also assessed. A Wilcoxon rank sum test was computed to assess the significance of differences. Interobserver variability of the comparison of manual vs. AI-assisted contours was also assessed among two radiation oncologists for PCa.

RESULTS

For PCa, AI-assisted segmentation showed good agreement with expert radiation oncologist structures with average DSC among patients ≥ 0.7 for all structures, and minimal radiation oncology adjustment of structures (DSC of adjusted versus AI structures ≥ 0.91). For HNC, results of comparison between manual and AI contouring varied considerably e.g., 0.77 for oral cavity and 0.11-0.13 for brachial plexus, but again, adjustment was generally minimal (DSC of adjusted against AI contours 0.97 for oral cavity, 0.92-0.93 for brachial plexus). The difference in dose for the target and organs at risk were not statistically significant between human and AI-assisted, with the only exceptions of D0.03cc to the anal canal and Dmean to the brachial plexus. The observed average differences in plan quality for PCa and HNC cases were 8% and 6.7%, respectively. The dose parameter changes due to interobserver variability in PCa were small, with the exception of the anal canal, where large dose variations were observed. The reduction in time required for contouring was 72% for PCa and 84% for HNC.

CONCLUSIONS

When an autocontouring system is used in combination with human review, the time of the RT workflow is significantly reduced without affecting dose distribution and plan quality.

Refining critical structure contouring in STereotactic Arrhythmia Radioablation (STAR): Benchmark results and consensus guidelines from the STOPSTORM.eu consortium

BACKGROUND AND PURPOSE

In patients with recurrent ventricular tachycardia (VT), STereotactic Arrhythmia Radioablation (STAR) shows promising results. The STOPSTORM.eu consortium was established to investigate and harmonise STAR treatment in Europe. The primary goals of this benchmark study were to standardise contouring of organs at risk (OAR) for STAR, including detailed substructures of the heart, and accredit each participating centre.

MATERIALS AND METHODS

Centres within the STOPSTORM.eu consortium were asked to delineate 31 OAR in three STAR cases. Delineation was reviewed by the consortium expert panel and after a dedicated workshop feedback and accreditation was provided to all participants. Further quantitative analysis was performed by calculating DICE similarity coefficients (DSC), median distance to agreement (MDA), and 95th percentile distance to agreement (HD95).

RESULTS

Twenty centres participated in this study. Based on DSC, MDA and HD95, the delineations of well-known OAR in radiotherapy were similar, such as lungs (median DSC = 0.96, median MDA = 0.1 mm and median HD95 = 1.1 mm) and aorta (median DSC = 0.90, median MDA = 0.1 mm and median HD95 = 1.5 mm). Some centres did not include the gastro-oesophageal junction, leading to differences in stomach and oesophagus delineations. For cardiac substructures, such as chambers (median DSC = 0.83, median MDA = 0.2 mm and median HD95 = 0.5 mm), valves (median DSC = 0.16, median MDA = 4.6 mm and median HD95 = 16.0 mm), coronary arteries (median DSC = 0.4, median MDA = 0.7 mm and median HD95 = 8.3 mm) and the sinoatrial and atrioventricular nodes (median DSC = 0.29, median MDA = 4.4 mm and median HD95 = 11.4 mm), deviations between centres occurred more frequently. After the dedicated workshop all centres were accredited and contouring consensus guidelines for STAR were established.

CONCLUSION

This STOPSTORM multi-centre critical structure contouring benchmark study showed high agreement for standard radiotherapy OAR. However, for cardiac substructures larger disagreement in contouring occurred, which may have significant impact on STAR treatment planning and dosimetry evaluation. To standardize OAR contouring, consensus guidelines for critical structure contouring in STAR were established.

A geometry and dose-volume based performance monitoring of artificial intelligence models in radiotherapy treatment planning for prostate cancer

Background and Purpose

Clinical Artificial Intelligence (AI) implementations lack ground-truth when applied on real-world data. This study investigated how combined geometrical and dose-volume metrics can be used as performance monitoring tools to detect clinically relevant candidates for model retraining.

Materials and Methods

Fifty patients were analyzed for both AI-segmentation and planning. For AI-segmentation, geometrical (Standard Surface Dice 3 mm and Local Surface Dice 3 mm) and dose-volume based parameters were calculated for two organs (bladder and anorectum) to compare AI output against the clinically corrected structure. A Local Surface Dice was introduced to detect geometrical changes in the vicinity of the target volumes, while an Absolute Dose Difference (ADD) evaluation increased focus on dose-volume related changes. AI-planning performance was evaluated using clinical goal analysis in combination with volume and target overlap metrics.

Results

The Local Surface Dice reported equal or lower values compared to the Standard Surface Dice (anorectum: (0.93 ± 0.11) vs (0.98 ± 0.04); bladder: (0.97 ± 0.06) vs (0.98 ± 0.04)). The ADD metric showed a difference of (0.9 ± 0.8)Gy for the anorectum D 1 cm 3 . The bladder D 5cm 3 reported a difference of (0.7 ± 1.5)Gy. Mandatory clinical goals were fulfilled in 90 % of the DLP plans.

Conclusions

Combining dose-volume and geometrical metrics allowed detection of clinically relevant changes, applied to both auto-segmentation and auto-planning output and the Local Surface Dice was more sensitive to local changes compared to the Standard Surface Dice. This monitoring is able to evaluate AI behavior in clinical practice and allows candidate selection for active learning.

Real-world validation of Artificial Intelligence-based Computed Tomography auto-contouring for prostate cancer radiotherapy planning

Background and purpose

Artificial Intelligence (AI)-based auto-contouring for treatment planning in radiotherapy needs extensive clinical validation, including the impact of editing after automatic segmentation. The aims of this study were to assess the performance of a commercial system for Clinical Target Volumes (CTVs) (prostate/seminal vesicles) and selected Organs at Risk (OARs) (rectum/bladder/femoral heads + femurs), evaluating also inter-observer variability (manual vs automatic + editing) and the reduction of contouring time.

Materials and methods

Two expert observers contoured CTVs/OARs of 20 patients in our Treatment Planning System (TPS). Computed Tomography (CT) images were sent to the automatic contouring workstation: automatic contours were generated and sent back to TPS, where observers could edit them if necessary. Inter- and intra-observer consistency was estimated using Dice Similarity Coefficients (DSC). Radiation oncologists were also asked to score the quality of automatic contours, ranging from 1 (complete re-contouring) to 5 (no editing). Contouring times (manual vs automatic + edit) were compared.

Results

DSCs (manual vs automatic only) were consistent with inter-observer variability (between 0.65 for seminal vesicles and 0.94 for bladder); editing further improved performances (range: 0.76-0.94). The median clinical score was 4 (little editing) and it was <4 in 3/2 patients for the two observers respectively. Inter-observer variability of automatic + editing contours improved significantly, being lower than manual contouring (e.g.: seminal vesicles: 0.83vs0.73; prostate: 0.86vs0.83; rectum: 0.96vs0.81). Oncologist contouring time reduced from 17 to 24 min of manual contouring time to 3-7 min of editing time for the two observers (p < 0.01).

Conclusion

Automatic contouring with a commercial AI-based system followed by editing can replace manual contouring, resulting in significantly reduced time for segmentation and better consistency between operators.

An investigation into the risk of population bias in deep learning autocontouring

BACKGROUND AND PURPOSE

To date, data used in the development of Deep Learning-based automatic contouring (DLC) algorithms have been largely sourced from single geographic populations. This study aimed to evaluate the risk of population-based bias by determining whether the performance of an autocontouring system is impacted by geographic population.

MATERIALS AND METHODS

80 Head Neck CT deidentified scans were collected from four clinics in Europe (n = 2) and Asia (n = 2). A single observer manually delineated 16 organs-at-risk in each. Subsequently, the data was contoured using a DLC solution, and trained using single institution (European) data. Autocontours were compared to manual delineations using quantitative measures. A Kruskal-Wallis test was used to test for any difference between populations. Clinical acceptability of automatic and manual contours to observers from each participating institution was assessed using a blinded subjective evaluation.

RESULTS

Seven organs showed a significant difference in volume between groups. Four organs showed statistical differences in quantitative similarity measures. The qualitative test showed greater variation in acceptance of contouring between observers than between data from different origins, with greater acceptance by the South Korean observers.

CONCLUSION

Much of the statistical difference in quantitative performance could be explained by the difference in organ volume impacting the contour similarity measures and the small sample size. However, the qualitative assessment suggests that observer perception bias has a greater impact on the apparent clinical acceptability than quantitatively observed differences. This investigation of potential geographic bias should extend to more patients, populations, and anatomical regions in the future.

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

AIMS AND OBJECTIVES

To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings.

METHOD

PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy. Reported effects on user experience, decision-making, care delivery and outcomes were summarised.

RESULTS

AI technologies are being applied in a wide variety of clinical areas. Most contemporary systems utilise deep learning, use routinely collected data, support diagnosis and triage, are assistive (requiring users to confirm or approve AI provided information or decisions), and are used by doctors in acute care settings in high-income nations. AI systems are integrated and used within existing clinical information systems including electronic medical records. There is limited support for One Health goals. Evaluation is largely based on quantitative methods measuring effects on decision-making.

CONCLUSION

AI systems are being implemented and evaluated in many clinical areas. There remain many opportunities to understand patterns of routine use and evaluate effects on decision-making, care delivery and patient outcomes using mixed-methods. Support for One Health including integrating data about environmental factors and social determinants needs further exploration.

Enhancing Radiotherapy Workflow for Head and Neck Cancer with Artificial Intelligence: A Systematic Review

BACKGROUND

Head and neck cancer (HNC) is characterized by complex-shaped tumors and numerous organs at risk (OARs), inducing challenging radiotherapy (RT) planning, optimization, and delivery. In this review, we provided a thorough description of the applications of artificial intelligence (AI) tools in the HNC RT process.

METHODS

The PubMed database was queried, and a total of 168 articles (2016-2022) were screened by a group of experts in radiation oncology. The group selected 62 articles, which were subdivided into three categories, representing the whole RT workflow: (i) target and OAR contouring, (ii) planning, and (iii) delivery.

RESULTS

The majority of the selected studies focused on the OARs segmentation process. Overall, the performance of AI models was evaluated using standard metrics, while limited research was found on how the introduction of AI could impact clinical outcomes. Additionally, papers usually lacked information about the confidence level associated with the predictions made by the AI models.

CONCLUSIONS

AI represents a promising tool to automate the RT workflow for the complex field of HNC treatment. To ensure that the development of AI technologies in RT is effectively aligned with clinical needs, we suggest conducting future studies within interdisciplinary groups, including clinicians and computer scientists.

Machine learning in non-small cell lung cancer radiotherapy: A bibliometric analysis

Background

Machine learning is now well-developed in non-small cell lung cancer (NSCLC) radiotherapy. But the research trend and hotspots are still unclear. To investigate the progress in machine learning in radiotherapy NSCLC, we performed a bibliometric analysis of associated research and discuss the current research hotspots and potential hot areas in the future.

Methods

The involved researches were obtained from the Web of Science Core Collection database (WoSCC). We used R-studio software, the Bibliometrix package and VOSviewer (Version 1.6.18) software to perform bibliometric analysis.

Results

We found 197 publications about machine learning in radiotherapy for NSCLC in the WoSCC, and the journal Medical Physics contributed the most articles. The University of Texas MD Anderson Cancer Center was the most frequent publishing institution, and the United States contributed most of the publications. In our bibliometric analysis, “radiomics” was the most frequent keyword, and we found that machine learning is mainly applied to analyze medical images in the radiotherapy of NSCLC.

Results

The research we identified about machine learning in NSCLC radiotherapy was mainly related to the radiotherapy planning of NSCLC and the prediction of treatment effects and adverse events in NSCLC patients who were under radiotherapy. Our research has added new insights into machine learning in NSCLC radiotherapy and could help researchers better identify hot research areas in the future.

Leveraging intelligent optimization for automated, cardiac-sparing accelerated partial breast treatment planning

Background

Accelerated partial breast irradiation (APBI) yields similar rates of recurrence and cosmetic outcomes as compared to whole breast radiation therapy (RT) when patients and treatment techniques are appropriately selected. APBI combined with stereotactic body radiation therapy (SBRT) is a promising technique for precisely delivering high levels of radiation while avoiding uninvolved breast tissue. Here we investigate the feasibility of automatically generating high quality APBI plans in the Ethos adaptive workspace with a specific emphasis on sparing the heart.

Methods

Nine patients (10 target volumes) were utilized to iteratively tune an Ethos APBI planning template for automatic plan generation. Twenty patients previously treated on a TrueBeam Edge accelerator were then automatically replanned using this template without manual intervention or reoptimization. The unbiased validation cohort Ethos plans were benchmarked via adherence to planning objectives, a comparison of DVH and quality indices against the clinical Edge plans, and qualitative reviews by two board-certified radiation oncologists.

Results

85% (17/20) of automated validation cohort plans met all planning objectives; three plans did not achieve the contralateral lung V1.5Gy objective, but all other objectives were achieved. Compared to the Eclipse generated plans, the proposed Ethos template generated plans with greater evaluation planning target volume (PTV_Eval) V100% coverage (p = 0.01), significantly decreased heart V1.5Gy (p< 0.001), and increased contralateral breast V5Gy, skin D0.01cc, and RTOG conformity index (p = 0.03, p = 0.03, and p = 0.01, respectively). However, only the reduction in heart dose was significant after correcting for multiple testing. Physicist-selected plans were deemed clinically acceptable without modification for 75% and 90% of plans by physicians A and B, respectively. Physicians A and B scored at least one automatically generated plan as clinically acceptable for 100% and 95% of planning intents, respectively.

Conclusions

Standard left- and right-sided planning templates automatically generated APBI plans of comparable quality to manually generated plans treated on a stereotactic linear accelerator, with a significant reduction in heart dose compared to Eclipse generated plans. The methods presented in this work elucidate an approach for generating automated, cardiac-sparing APBI treatment plans for daily adaptive RT with high efficiency.

Automated Contouring and Planning in Radiation Therapy: What Is 'Clinically Acceptable'?

Developers and users of artificial-intelligence-based tools for automatic contouring and treatment planning in radiotherapy are expected to assess clinical acceptability of these tools. However, what is 'clinical acceptability'? Quantitative and qualitative approaches have been used to assess this ill-defined concept, all of which have advantages and disadvantages or limitations. The approach chosen may depend on the goal of the study as well as on available resources. In this paper, we discuss various aspects of 'clinical acceptability' and how they can move us toward a standard for defining clinical acceptability of new autocontouring and planning tools.

A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.

How smart is artificial intelligence in organs delineation? Testing a CE and FDA-approved Deep-Learning tool using multiple expert contours delineated on planning CT images

Background

A CE- and FDA-approved cloud-based Deep learning (DL)-tool for automatic organs at risk (OARs) and clinical target volumes segmentation on computer tomography images is available. Before its implementation in the clinical practice, an independent external validation was conducted.

Methods

At least a senior and two in training Radiation Oncologists (ROs) manually contoured the volumes of interest (VOIs) for 6 tumoral sites. The auto-segmented contours were retrieved from the DL-tool and, if needed, manually corrected by ROs. The level of ROs satisfaction and the duration of contouring were registered. Relative volume differences, similarity indices, satisfactory grades, and time saved were analyzed using a semi-automatic tool.

Results

Seven thousand seven hundred sixty-five VOIs were delineated on the CT images of 111 representative patients. The median (range) time for manual VOIs delineation, DL-based segmentation, and subsequent manual corrections were 25.0 (8.0-115.0), 2.3 (1.2-8) and 10.0 minutes (0.3-46.3), respectively. The overall time for VOIs retrieving and modification was statistically significantly lower than for manual contouring (p<0.001). The DL-tool was generally appreciated by ROs, with 44% of vote 4 (well done) and 43% of vote 5 (very well done), correlated with the saved time (p<0.001). The relative volume differences and similarity indexes suggested a better inter-agreement of manually adjusted DL-based VOIs than manually segmented ones.

Conclusions

The application of the DL-tool resulted satisfactory, especially in complex delineation cases, improving the ROs inter-agreement of delineated VOIs and saving time.

Stress-testing pelvic autosegmentation algorithms using anatomical edge cases

Commercial autosegmentation has entered clinical use, however real-world performance may suffer in certain cases. We aimed to assess the influence of anatomic variants on performance. We identified 112 prostate cancer patients with anatomic variations (edge cases). Pelvic anatomy was autosegmented using three commercial tools. To evaluate performance, Dice similarity coefficients, and mean surface and 95% Hausdorff distances were calculated versus clinician-delineated references. Deep learning autosegmentation outperformed atlas-based and model-based methods. However, edge case performance was lower versus the normal cohort (0.12 mean DSC reduction). Anatomic variation presents challenges to commercial autosegmentation.

Implementation of a Commercial Deep Learning-Based Auto Segmentation Software in Radiotherapy: Evaluation of Effectiveness and Impact on Workflow

Proper delineation of both target volumes and organs at risk is a crucial step in the radiation therapy workflow. This process is normally carried out manually by medical doctors, hence demanding timewise. To improve efficiency, auto-contouring methods have been proposed. We assessed a specific commercial software to investigate its impact on the radiotherapy workflow on four specific disease sites: head and neck, prostate, breast, and rectum. For the present study, we used a commercial deep learning-based auto-segmentation software, namely Limbus Contour (LC), Version 1.5.0 (Limbus AI Inc., Regina, SK, Canada). The software uses deep convolutional neural network models based on a U-net architecture, specific for each structure. Manual and automatic segmentation were compared on disease-specific organs at risk. Contouring time, geometrical performance (volume variation, Dice Similarity Coefficient-DSC, and center of mass shift), and dosimetric impact (DVH differences) were evaluated. With respect to time savings, the maximum advantage was seen in the setting of head and neck cancer with a 65%-time reduction. The average DSC was 0.72. The best agreement was found for lungs. Good results were highlighted for bladder, heart, and femoral heads. The most relevant dosimetric difference was in the rectal cancer case, where the mean volume covered by the 45 Gy isodose was 10.4 cm³ for manual contouring and 289.4 cm³ for automatic segmentation. Automatic contouring was able to significantly reduce the time required in the procedure, simplifying the workflow, and reducing interobserver variability. Its implementation was able to improve the radiation therapy workflow in our department.

Artificial intelligence 101 for veterinary diagnostic imaging

The prevalence and pervasiveness of artificial intelligence (AI) with medical images in veterinary and human medicine is rapidly increasing. This article provides essential definitions of AI with medical images with a focus on veterinary radiology. Machine learning methods common in medical image analysis are compared, and a detailed description of convolutional neural networks commonly used in deep learning classification and regression models is provided. A brief introduction to natural language processing (NLP) and its utility in machine learning is also provided. NLP can economize the creation of "truth-data" needed when training AI systems for both diagnostic radiology and radiation oncology applications. The goal of this publication is to provide veterinarians, veterinary radiologists, and radiation oncologists the necessary background needed to understand and comprehend AI-focused research projects and publications.

The role of artificial intelligence in veterinary radiation oncology

Veterinary radiation oncology regularly deploys sophisticated contouring, image registration, and treatment planning optimization software for patient care. Over the past decade, advances in computing power and the rapid development of neural networks, open-source software packages, and data science have been realized and resulted in new research and clinical applications of artificial intelligent (AI) systems in radiation oncology. These technologies differ from conventional software in their level of complexity and ability to learn from representative and local data. We provide clinical and research application examples of AI in human radiation oncology and their potential applications in veterinary medicine throughout the patient's care-path: from treatment simulation, deformable registration, auto-segmentation, automated treatment planning and plan selection, quality assurance, adaptive radiotherapy, and outcomes modeling. These technologies have the potential to offer significant time and cost savings in the veterinary setting; however, since the range of usefulness of these technologies have not been well studied nor understood, care must be taken if adopting AI technologies in clinical practice. Over the next several years, some practical and realizable applications of AI in veterinary radiation oncology include automated segmentation of normal tissues and tumor volumes, deformable registration, multi-criteria plan optimization, and adaptive radiotherapy. Keys in achieving success in adopting AI in veterinary radiation oncology include: establishing "truth-data"; data harmonization; multi-institutional data and collaborations; standardized dose reporting and taxonomy; adopting an open access philosophy, data collection and curation; open-source algorithm development; and transparent and platform-independent code development.

A national survey of the radiotherapy dosimetrist workforce in the UK

OBJECTIVES

To undertake a national survey of the Radiotherapy Dosimetrist workforce within the UK; examining different attributes and experiences, comparing results with published evidence within the literature.

METHODS

A national, anonymised survey was undertaken between Dec 2020 and end of Feb 2021; employing a mixed-methods approach and blend of closed, open-ended answer choices and free-text comments. Questions included range of training routes and job titles; registration status; job tasks and engagement with Continuing Professional Development (CPD).

RESULTS

A total of 223 individuals responded. Nearly half were trained via therapeutic radiography; approximately, a fifth through a clinical technologist/physics routes. Most (70%) had Dosimetrist in their job title. Nearly 70% were statutorily registered, and almost a fifth were in the voluntary register of Clinical Technologists. Most job tasks were in treatment planning - with 57% spending over 70% of their time there. Most notably, 29% were not involved in any CPD scheme. No published evidence showed the same aspects identified here.

CONCLUSIONS

Our survey showed a unique profile of the Radiotherapy Dosimetrist workforce in the UK, with a variety of training routes and statutory registration status. Nearly, a third were not engaged in a CPD scheme - adding to the current discussion that perhaps all Dosimetrists should be statutorily registered, for ensuring safe and effective clinical practice.

ADVANCES IN KNOWLEDGE

A novel and unique national survey of Dosimetrists working in Radiotherapy in the UK is presented, leading to new insights into current training routes, registration status, job tasks and CPD engagement and needs.

Auto-segmentation for total marrow irradiation

Purpose

To evaluate the accuracy and efficiency of Artificial-Intelligence (AI) segmentation in Total Marrow Irradiation (TMI) including contours throughout the head and neck (H&N), thorax, abdomen, and pelvis.

Methods

An AI segmentation software was clinically introduced for total body contouring in TMI including 27 organs at risk (OARs) and 4 planning target volumes (PTVs). This work compares the clinically utilized contours to the AI-TMI contours for 21 patients. Structure and image dicom data was used to generate comparisons including volumetric, spatial, and dosimetric variations between the AI- and human-edited contour sets. Conventional volume and surface measures including the Sørensen-Dice coefficient (Dice) and the 95th% Hausdorff Distance (HD95) were used, and novel efficiency metrics were introduced. The clinical efficiency gains were estimated by the percentage of the AI-contour-surface within 1mm of the clinical contour surface. An unedited AI-contour has an efficiency gain=100%, an AI-contour with 70% of its surface<1mm from a clinical contour has an efficiency gain of 70%. The dosimetric deviations were estimated from the clinical dose distribution to compute the dose volume histogram (DVH) for all structures.

Results

A total of 467 contours were compared in the 21 patients. In PTVs, contour surfaces deviated by >1mm in 38.6% ± 23.1% of structures, an average efficiency gain of 61.4%. Deviations >5mm were detected in 12.0% ± 21.3% of the PTV contours. In OARs, deviations >1mm were detected in 24.4% ± 27.1% of the structure surfaces and >5mm in 7.2% ± 18.0%; an average clinical efficiency gain of 75.6%. In H&N OARs, efficiency gains ranged from 42% in optic chiasm to 100% in eyes (unedited in all cases). In thorax, average efficiency gains were >80% in spinal cord, heart, and both lungs. Efficiency gains ranged from 60-70% in spleen, stomach, rectum, and bowel and 75-84% in liver, kidney, and bladder. DVH differences exceeded 0.05 in 109/467 curves at any dose level. The most common 5%-DVH variations were in esophagus (86%), rectum (48%), and PTVs (22%).

Conclusions

AI auto-segmentation software offers a powerful solution for enhanced efficiency in TMI treatment planning. Whole body segmentation including PTVs and normal organs was successful based on spatial and dosimetric comparison.

Clinical Validation of a Deep-Learning Segmentation Software in Head and Neck: An Early Analysis in a Developing Radiation Oncology Center

Background: Organs at risk (OARs) delineation is a crucial step of radiotherapy (RT) treatment planning workflow. Time-consuming and inter-observer variability are main issues in manual OAR delineation, mainly in the head and neck (H & N) district. Deep-learning based auto-segmentation is a promising strategy to improve OARs contouring in radiotherapy departments. A comparison of deep-learning-generated auto-contours (AC) with manual contours (MC) was performed by three expert radiation oncologists from a single center. Methods: Planning computed tomography (CT) scans of patients undergoing RT treatments for H&N cancers were considered. CT scans were processed by Limbus Contour auto-segmentation software, a commercial deep-learning auto-segmentation based software to generate AC. H&N protocol was used to perform AC, with the structure set consisting of bilateral brachial plexus, brain, brainstem, bilateral cochlea, pharyngeal constrictors, eye globes, bilateral lens, mandible, optic chiasm, bilateral optic nerves, oral cavity, bilateral parotids, spinal cord, bilateral submandibular glands, lips and thyroid. Manual revision of OARs was performed according to international consensus guidelines. The AC and MC were compared using the Dice similarity coefficient (DSC) and 95% Hausdorff distance transform (DT). Results: A total of 274 contours obtained by processing CT scans were included in the analysis. The highest values of DSC were obtained for the brain (DSC 1.00), left and right eye globes and the mandible (DSC 0.98). The structures with greater MC editing were optic chiasm, optic nerves and cochleae. Conclusions: In this preliminary analysis, deep-learning auto-segmentation seems to provide acceptable H&N OAR delineations. For less accurate organs, AC could be considered a starting point for review and manual adjustment. Our results suggest that AC could become a useful time-saving tool to optimize workload and resources in RT departments.

Impact of Using Unedited CT-Based DIR-Propagated Autocontours on Online ART for Pancreatic SBRT

Purpose

To determine the dosimetric impact of using unedited autocontours in daily plan adaptation of patients with locally advanced pancreatic cancer (LAPC) treated with stereotactic body radiotherapy using tumor tracking.

Materials and Methods

The study included 98 daily CT scans of 35 LAPC patients. All scans were manually contoured (MAN), and included the PTV and main organs-at-risk (OAR): stomach, duodenum and bowel. Precision and MIM deformable image registration (DIR) methods followed by contour propagation were used to generate autocontour sets on the daily CT scans. Autocontours remained unedited, and were compared to MAN on the whole organs and at 3, 1 and 0.5 cm from the PTV. Manual and autocontoured OAR were used to generate daily plans using the VOLO™ optimizer, and were compared to non-adapted plans. Resulting planned doses were compared based on PTV coverage and OAR dose-constraints.

Results

Overall, both algorithms reported a high agreement between unclipped MAN and autocontours, but showed worse results when being evaluated on the clipped structures at 1 cm and 0.5 cm from the PTV. Replanning with unedited autocontours resulted in better OAR sparing than non-adapted plans for 95% and 84% plans optimized using Precision and MIM autocontours, respectively, and obeyed OAR constraints in 64% and 56% of replans.

Conclusion

For the majority of fractions, manual correction of autocontours could be avoided or be limited to the region closest to the PTV. This practice could further reduce the overall timings of adaptive radiotherapy workflows for patients with LAPC.

Artificial Intelligence Applications in Health Care Practice: A Scoping Review (Preprint)

Background

Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood.

Objective

The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible?

Methods

A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized.

Results

Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare.

Conclusions

Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.

Real-world analysis of manual editing of deep learning contouring in the thorax region

•

Deep-learning contouring for radiotherapy is evaluated in the thorax region.

•

Specific regions-of-adjustment and variability in editing per organ were found.

•

The consistency and accuracy of training data remains crucial to model performance.

•

Separate models for specific indications or acquisition protocols may be necessary.

•

Subsampling and post-processing make the clinical workflow more efficient.

Background and purpose

User-adjustments after deep-learning (DL) contouring in radiotherapy were evaluated to get insight in real-world editing during clinical practice. This study assessed the amount, type and spatial regions of editing of auto-contouring for organs-at-risk (OARs) in routine clinical workflow for patients in the thorax region.

Materials and methods

A total of 350 lung cancer and 362 breast cancer patients, contoured between March 2020 and March 2021 using a commercial DL-contouring method followed by manual adjustments were retrospectively analyzed. Subsampling was performed for some OARs, using an inter-slice gap of 1–3 slices. Commonly-used whole-organ contouring assessment measures were calculated, and all cases were registered to a common reference shape per OAR to identify regions of manual adjustment. Results were expressed as the median, 10th-90th percentile of adjustment and visualized using 3D renderings.

Results

Per OAR, the median amount of editing was below 1 mm. However, large adjustments were found in some locations for most OARs. In general, enlarging of the auto-contours was needed. Subsampling DL-contours showed less adjustments were made in the interpolated slices compared to simulated no-subsampling for these OARs.

Conclusion

The real-world performance of automatic DL-contouring software was evaluated and proven useful in clinical practice. Specific regions-of-adjustment were identified per OAR in the thorax region, and separate models were found to be necessary for specific clinical indications different from training data. This analysis showed the need to perform routine clinical analysis especially when procedures or acquisition protocols change to have the best configuration of the workflow.

General and custom deep learning autosegmentation models for organs in head and neck, abdomen, and male pelvis

PURPOSE

To reduce workload and inconsistencies in organ segmentation for radiation treatment planning, we developed and evaluated general and custom autosegmentation models on computed tomography (CT) for three major tumor sites using a well-established deep convolutional neural network (DCNN).

METHODS

Five CT-based autosegmentation models for 42 organs at risk (OARs) in head and neck (HN), abdomen (ABD), and male pelvis (MP) were developed using a full three-dimensional (3D) DCNN architecture. Two types of deep learning (DL) models were separately trained using either general diversified multi-institutional datasets or custom well-controlled single-institution datasets. To improve segmentation accuracy, an adaptive spatial resolution approach for small and/or narrow OARs and a pseudo scan extension approach, when CT scan length is too short to cover entire organs, were implemented. The performance of the obtained models was evaluated based on accuracy and clinical applicability of the autosegmented contours using qualitative visual inspection and quantitative calculation of dice similarity coefficient (DSC), mean distance to agreement (MDA), and time efficiency.

RESULTS

The five DL autosegmentation models developed for the three anatomical sites were found to have high accuracy (DSC ranging from 0.8 to 0.98) for 74% OARs and marginally acceptable for 26% OARs. The custom models performed slightly better than the general models, even with smaller custom datasets used for the custom model training. The organ-based approaches improved autosegmentation accuracy for small or complex organs (e.g., eye lens, optic nerves, inner ears, and bowels). Compared with traditional manual contouring times, the autosegmentation times, including subsequent manual editing, if necessary, were substantially reduced by 88% for MP, 80% for HN, and 65% for ABD models.

CONCLUSIONS

The obtained autosegmentation models, incorporating organ-based approaches, were found to be effective and accurate for most OARs in the male pelvis, head and neck, and abdomen. We have demonstrated that our multianatomical DL autosegmentation models are clinically useful for radiation treatment planning.

Deep Learning-Based Segmentation of Various Brain Lesions for Radiosurgery

Semantic segmentation of medical images with deep learning models is rapidly being developed. In this study, we benchmarked state-of-the-art deep learning segmentation algorithms on our clinical stereotactic radiosurgery dataset. The dataset consists of 1688 patients with various brain lesions (pituitary tumors, meningioma, schwannoma, brain metastases, arteriovenous malformation, and trigeminal neuralgia), and we divided the dataset into a training set (1557 patients) and test set (131 patients). This study demonstrates the strengths and weaknesses of deep-learning algorithms in a fairly practical scenario. We compared the model performances concerning their sampling method, model architecture, and the choice of loss functions, identifying suitable settings for their applications and shedding light on the possible improvements. Evidence from this study led us to conclude that deep learning could be promising in assisting the segmentation of brain lesions even if the training dataset was of high heterogeneity in lesion types and sizes.