The Brain Tumor Segmentation (BraTS) Challenge 2023: Focus on Pediatrics (CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs)

Pediatric tumors of the central nervous system are the most common cause of cancer-related death in children. The five-year survival rate for high-grade gliomas in children is less than 20\%. Due to their rarity, the diagnosis of these entities is often delayed, their treatment is mainly based on historic treatment concepts, and clinical trials require multi-institutional collaborations. The MICCAI Brain Tumor Segmentation (BraTS) Challenge is a landmark community benchmark event with a successful history of 12 years of resource creation for the segmentation and analysis of adult glioma. Here we present the CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs 2023 challenge, which represents the first BraTS challenge focused on pediatric brain tumors with data acquired across multiple international consortia dedicated to pediatric neuro-oncology and clinical trials. The BraTS-PEDs 2023 challenge focuses on benchmarking the development of volumentric segmentation algorithms for pediatric brain glioma through standardized quantitative performance evaluation metrics utilized across the BraTS 2023 cluster of challenges. Models gaining knowledge from the BraTS-PEDs multi-parametric structural MRI (mpMRI) training data will be evaluated on separate validation and unseen test mpMRI dataof high-grade pediatric glioma. The CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs 2023 challenge brings together clinicians and AI/imaging scientists to lead to faster development of automated segmentation techniques that could benefit clinical trials, and ultimately the care of children with brain tumors.

A large open access dataset of brain metastasis 3D segmentations on MRI with clinical and imaging information

Resection and whole brain radiotherapy (WBRT) are standard treatments for brain metastases (BM) but are associated with cognitive side effects. Stereotactic radiosurgery (SRS) uses a targeted approach with less side effects than WBRT. SRS requires precise identification and delineation of BM. While artificial intelligence (AI) algorithms have been developed for this, their clinical adoption is limited due to poor model performance in the clinical setting. The limitations of algorithms are often due to the quality of datasets used for training the AI network. The purpose of this study was to create a large, heterogenous, annotated BM dataset for training and validation of AI models. We present a BM dataset of 200 patients with pretreatment T1, T1 post-contrast, T2, and FLAIR MR images. The dataset includes contrast-enhancing and necrotic 3D segmentations on T1 post-contrast and peritumoral edema 3D segmentations on FLAIR. Our dataset contains 975 contrast-enhancing lesions, many of which are sub centimeter, along with clinical and imaging information. We used a streamlined approach to database-building through a PACS-integrated segmentation workflow.

A Large Open Access Dataset of Brain Metastasis 3D Segmentations with Clinical and Imaging Feature Information

Resection and whole brain radiotherapy (WBRT) are the standards of care for the treatment of patients with brain metastases (BM) but are often associated with cognitive side effects. Stereotactic radiosurgery (SRS) involves a more targeted treatment approach and has been shown to avoid the side effects associated with WBRT. However, SRS requires precise identification and delineation of BM. While many AI algorithms have been developed for this purpose, their clinical adoption has been limited due to poor model performance in the clinical setting. Major reasons for non-generalizable algorithms are the limitations in the datasets used for training the AI network. The purpose of this study was to create a large, heterogenous, annotated BM dataset for training and validation of AI models to improve generalizability. We present a BM dataset of 200 patients with pretreatment T1, T1 post-contrast, T2, and FLAIR MR images. The dataset includes contrast-enhancing and necrotic 3D segmentations on T1 post-contrast and whole tumor (including peritumoral edema) 3D segmentations on FLAIR. Our dataset contains 975 contrast-enhancing lesions, many of which are sub centimeter, along with clinical and imaging feature information. We used a streamlined approach to database-building leveraging a PACS-integrated segmentation workflow.

Deep learning based automated epidermal growth factor receptor and anaplastic lymphoma kinase status prediction of brain metastasis in non-small cell lung cancer

Aim

The aim of this study was to investigate the feasibility of developing a deep learning (DL) algorithm for classifying brain metastases from non-small cell lung cancer (NSCLC) into epidermal growth factor receptor (EGFR) mutation and anaplastic lymphoma kinase (ALK) rearrangement groups and to compare the accuracy with classification based on semantic features on imaging.

Methods

Data set of 117 patients was analysed from 2014 to 2018 out of which 33 patients were EGFR positive, 43 patients were ALK positive and 41 patients were negative for either mutation. Convolutional neural network (CNN) architecture efficient net was used to study the accuracy of classification using T1 weighted (T1W) magnetic resonance imaging (MRI) sequence, T2 weighted (T2W) MRI sequence, T1W post contrast (T1post) MRI sequence, fluid attenuated inversion recovery (FLAIR) MRI sequences. The dataset was divided into 80% training and 20% testing. The associations between mutation status and semantic features, specifically sex, smoking history, EGFR mutation and ALK rearrangement status, extracranial metastasis, performance status and imaging variables of brain metastasis were analysed using descriptive analysis [chi-square test (χ2)], univariate and multivariate logistic regression analysis assuming 95% confidence interval (CI).

Results

In this study of 117 patients, the analysis by semantic method showed 79.2% of the patients belonged to ALK positive were non-smokers as compared to double negative groups (P = 0.03). There was a 10-fold increase in ALK positivity as compared to EGFR positivity in ring enhancing lesions patients (P = 0.015) and there was also a 6.4-fold increase in ALK positivity as compared to double negative groups in meningeal involvement patients (P = 0.004). Using CNN Efficient Net DL model, the study achieved 76% accuracy in classifying ALK rearrangement and EGFR mutations without manual segmentation of metastatic lesions. Analysis of the manually segmented dataset resulted in improved accuracy of 89% through this model.

Conclusions

Both semantic features and DL model showed comparable accuracy in classifying EGFR mutation and ALK rearrangement. Both methods can be clinically used to predict mutation status while biopsy or genetic testing is undertaken.

Generative adversarial networks based skin lesion segmentation

Skin cancer is a serious condition that requires accurate diagnosis and treatment. One way to assist clinicians in this task is using computer-aided diagnosis tools that automatically segment skin lesions from dermoscopic images. We propose a novel adversarial learning-based framework called Efficient-GAN (EGAN) that uses an unsupervised generative network to generate accurate lesion masks. It consists of a generator module with a top-down squeeze excitation-based compound scaled path, an asymmetric lateral connection-based bottom-up path, and a discriminator module that distinguishes between original and synthetic masks. A morphology-based smoothing loss is also implemented to encourage the network to create smooth semantic boundaries of lesions. The framework is evaluated on the International Skin Imaging Collaboration Lesion Dataset. It outperforms the current state-of-the-art skin lesion segmentation approaches with a Dice coefficient, Jaccard similarity, and accuracy of 90.1%, 83.6%, and 94.5%, respectively. We also design a lightweight segmentation framework called Mobile-GAN (MGAN) that achieves comparable performance as EGAN but with an order of magnitude lower number of training parameters, thus resulting in faster inference times for low compute resource settings.

The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn)

Automated brain tumor segmentation methods have become well-established and reached performance levels offering clear clinical utility. These methods typically rely on four input magnetic resonance imaging (MRI) modalities: T1-weighted images with and without contrast enhancement, T2-weighted images, and FLAIR images. However, some sequences are often missing in clinical practice due to time constraints or image artifacts, such as patient motion. Consequently, the ability to substitute missing modalities and gain segmentation performance is highly desirable and necessary for the broader adoption of these algorithms in the clinical routine. In this work, we present the establishment of the Brain MR Image Synthesis Benchmark (BraSyn) in conjunction with the Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2023. The primary objective of this challenge is to evaluate image synthesis methods that can realistically generate missing MRI modalities when multiple available images are provided. The ultimate aim is to facilitate automated brain tumor segmentation pipelines. The image dataset used in the benchmark is diverse and multi-modal, created through collaboration with various hospitals and research institutions.

The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI

Clinical monitoring of metastatic disease to the brain can be a laborious and timeconsuming process, especially in cases involving multiple metastases when the assessment is performed manually. The Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) guideline, which utilizes the unidimensional longest diameter, is commonly used in clinical and research settings to evaluate response to therapy in patients with brain metastases. However, accurate volumetric assessment of the lesion and surrounding peri-lesional edema holds significant importance in clinical decision-making and can greatly enhance outcome prediction. The unique challenge in performing segmentations of brain metastases lies in their common occurrence as small lesions. Detection and segmentation of lesions that are smaller than 10 mm in size has not demonstrated high accuracy in prior publications. The brain metastases challenge sets itself apart from previously conducted MICCAI challenges on glioma segmentation due to the significant variability in lesion size. Unlike gliomas, which tend to be larger on presentation scans, brain metastases exhibit a wide range of sizes and tend to include small lesions. We hope that the BraTS-METS dataset and challenge will advance the field of automated brain metastasis detection and segmentation.

The Brain Tumor Segmentation (BraTS) Challenge 2023: Glioma Segmentation in Sub-Saharan Africa Patient Population (BraTS-Africa)

Gliomas are the most common type of primary brain tumors. Although gliomas are relatively rare, they are among the deadliest types of cancer, with a survival rate of less than 2 years after diagnosis. Gliomas are challenging to diagnose, hard to treat and inherently resistant to conventional therapy. Years of extensive research to improve diagnosis and treatment of gliomas have decreased mortality rates across the Global North, while chances of survival among individuals in low- and middle-income countries (LMICs) remain unchanged and are significantly worse in Sub-Saharan Africa (SSA) populations. Long-term survival with glioma is associated with the identification of appropriate pathological features on brain MRI and confirmation by histopathology. Since 2012, the Brain Tumor Segmentation (BraTS) Challenge have evaluated state-of-the-art machine learning methods to detect, characterize, and classify gliomas. However, it is unclear if the state-of-the-art methods can be widely implemented in SSA given the extensive use of lower-quality MRI technology, which produces poor image contrast and resolution and more importantly, the propensity for late presentation of disease at advanced stages as well as the unique characteristics of gliomas in SSA (i.e., suspected higher rates of gliomatosis cerebri). Thus, the BraTS-Africa Challenge provides a unique opportunity to include brain MRI glioma cases from SSA in global efforts through the BraTS Challenge to develop and evaluate computer-aided-diagnostic (CAD) methods for the detection and characterization of glioma in resource-limited settings, where the potential for CAD tools to transform healthcare are more likely.

The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningioma

Meningiomas are the most common primary intracranial tumor in adults and can be associated with significant morbidity and mortality. Radiologists, neurosurgeons, neuro-oncologists, and radiation oncologists rely on multiparametric MRI (mpMRI) for diagnosis, treatment planning, and longitudinal treatment monitoring; yet automated, objective, and quantitative tools for non-invasive assessment of meningiomas on mpMRI are lacking. The BraTS meningioma 2023 challenge will provide a community standard and benchmark for state-of-the-art automated intracranial meningioma segmentation models based on the largest expert annotated multilabel meningioma mpMRI dataset to date. Challenge competitors will develop automated segmentation models to predict three distinct meningioma sub-regions on MRI including enhancing tumor, non-enhancing tumor core, and surrounding nonenhancing T2/FLAIR hyperintensity. Models will be evaluated on separate validation and held-out test datasets using standardized metrics utilized across the BraTS 2023 series of challenges including the Dice similarity coefficient and Hausdorff distance. The models developed during the course of this challenge will aid in incorporation of automated meningioma MRI segmentation into clinical practice, which will ultimately improve care of patients with meningioma.

Federated learning enables big data for rare cancer boundary detection

Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.

NIMG-67. MULTI-PARAMETRIC MRI-BASED MACHINE LEARNING ANALYSIS FOR PREDICTION OF NEOPLASTIC INFILTRATION AND RECURRENCE IN PATIENTS WITH GLIOBLASTOMA: UPDATES FROM THE MULTI-INSTITUTIONAL RESPOND CONSORTIUM

PURPOSE

Glioblastoma is extremely infiltrative with malignant cells extending beyond the enhancing rim where recurrence inevitably occurs, despite aggressive multimodal therapy. We hypothesize that important characteristics of peritumoral tissue heterogeneity captured and analyzed by multi-parametric MRI and artificial intelligence (AI) methods are generalizable in the updated multi-institutional ReSPOND (Radiomics Signatures for PrecisiON Diagnostics) consortium and predictive of neoplastic infiltration and future recurrence.

METHODS

We used the most recent update of the ReSPOND consortium to evaluate and further refine generalizability of our methods with different scanners and acquisition settings. 179 de novo glioblastoma patients with available T1, T1Gd, T2, T2-FLAIR, and ADC sequences at pre-resection baseline and after complete resection with subsequent pathology-confirmed recurrence were included. To establish generalizability of the predictive models, training and testing of the refined AI model was performed through Leave-One-Institution-Out-Cross-Validation schema. The multi-institutional cohort consisted of the Hospital of the University of Pennsylvania (UPenn, 124), Case Western Reserve University/University Hospitals (CWRU/UH, 27), New York University (NYU, 13), Ohio State University (OSU, 13), and University Hospital Río Hortega (RH, 2). Features extracted from pre-resection MRI were used to build the model predicting the spatial pattern of subsequent tumor recurrence. These predictions were evaluated against regions of pathology-confirmed post-resection recurrence.

RESULTS

Our model predicted the locations that later harbored tumor recurrence with overall odds ratio (99% CI)/AUC (99% CI), 12.0(11.8-12.2)/0.80(0.76-0.85), and per institute, CWRU/UH, 11.0(10.7-11.3)/0.80 (0.64-0.97); NYU, 7.0(6.7-7.3)/0.78(0.56-1.00); OSU, 18.3(17.5-19.1)/0.83(0.54-1.00); RH, 40.0(35.3-45.5)/0.93(0.00-1.00); UPenn, 8.00(7.7-8.3)/0.80(0.75-0.84).

CONCLUSION

This study provides extensive multi-institutional validated evidence that machine learning tools can identify peritumoral neoplastic infiltration and predict location of future recurrence, by decrypting the MRI signal heterogeneity in peritumoral tissue. Our analyses leveraged the unique dataset of the ReSPOND consortium, which aims to develop and validate AI-based biomarkers for individualized prediction and prognostication and establish generalizability in a multi-institutional setting.

NIMG-33. PROGNOSTIC STRATIFICATION OF DE NOVO GLIOBLASTOMA PATIENTS ACROSS 22 GEOGRAPHICALLY DISTINCT INSTITUTIONS: UPDATES FROM THE RESPOND CONSORTIUM

PURPOSE

Glioblastoma, IDH-wildtype, is the most common primary malignant adult brain tumor with median overall survival (OS) of ~14 months, with little improvement over the last 20 years. We hypothesize that AI-based integration of quantitative tumor characteristics, independent of acquisition protocol and equipment, can reveal accurate generalizable prognostic stratification. We seek an AI-based OS predictor using routine clinically acquired MRI sequences, quantitatively evaluated across institutions of the ReSPOND (Radiomics Signatures for PrecisiON Diagnostics) consortium.

METHODS

We identified a retrospective cohort of 2,293 diffuse glioma (IDH-wildtype/-NOS/-NEC) patients from 22 geographically distinct institutions across 3 continents, with preoperative structural MRI scans. The entire tumor burden was automatically segmented into 3 sub-compartments, i.e., enhancing, necrotic, peritumoral T2-FLAIR abnormality. We developed our AI predictor by multivariate integration of i)patient age, ii)tumor sub-compartment volume normalized to brain volume, iii)spatial distribution characteristics (tumor location, distance to the ventricles, and laterality), and iv)morphologic descriptors (major axes’ length, axes’ ratio, extent, and number of tumors). The AI predictor returns a continuous value between 0-1, defining short-, intermediate-, and long-survivors based on thresholds on the 25th and 75th percentiles. Leave-One-Site-Out-Cross-Validation was used to assess the generalizability of our stratification. Kaplan-Meier survival curves were computed for OS analysis and evaluated by a Cox proportional hazards model for statistical significance and hazard ratios.

RESULTS

Survival analysis yielded a hazard ratio of 2.07 (95%CI, 2.06-2.08, p-value= 4.8e-102) for patient stratification into short-, intermediate-, and long-survivors. Pearson correlation between the predicted and actual OS yielded an R= 0.49.

CONCLUSION

Multivariate integration of visually quantified tumor characteristics, agnostic to acquisition protocol/equipment, yields an accurate OS surrogate index. Validation of our AI model in the largest centralized glioblastoma imaging dataset, from the ReSPOND consortium, supports its generalizability across diverse patient populations and acquisition settings, potentially contributing to equitable improvements of personalized patient care.

The University of California San Francisco Preoperative Diffuse Glioma MRI Dataset

Supplemental material is available for this article. Keywords: Informatics, MR Diffusion Tensor Imaging, MR Perfusion, MR Imaging, Neuro-Oncology, CNS, Brain/Brain Stem, Oncology, Radiogenomics, Radiology-Pathology Integration © RSNA, 2022.

The federated tumor segmentation (FeTS) tool: an open-source solution to further solid tumor research

Objective.De-centralized data analysis becomes an increasingly preferred option in the healthcare domain, as it alleviates the need for sharing primary patient data across collaborating institutions. This highlights the need for consistent harmonized data curation, pre-processing, and identification of regions of interest based on uniform criteria.Approach.Towards this end, this manuscript describes theFederatedTumorSegmentation (FeTS) tool, in terms of software architecture and functionality.Main results.The primary aim of the FeTS tool is to facilitate this harmonized processing and the generation of gold standard reference labels for tumor sub-compartments on brain magnetic resonance imaging, and further enable federated training of a tumor sub-compartment delineation model across numerous sites distributed across the globe, without the need to share patient data.Significance.Building upon existing open-source tools such as the Insight Toolkit and Qt, the FeTS tool is designed to enable training deep learning models targeting tumor delineation in either centralized or federated settings. The target audience of the FeTS tool is primarily the computational researcher interested in developing federated learning models, and interested in joining a global federation towards this effort. The tool is open sourced athttps://github.com/FETS-AI/Front-End.

ADAM Challenge: Detecting Age-Related Macular Degeneration From Fundus Images

Age-related macular degeneration (AMD) is the leading cause of visual impairment among elderly in the world. Early detection of AMD is of great importance, as the vision loss caused by this disease is irreversible and permanent. Color fundus photography is the most cost-effective imaging modality to screen for retinal disorders. Cutting edge deep learning based algorithms have been recently developed for automatically detecting AMD from fundus images. However, there are still lack of a comprehensive annotated dataset and standard evaluation benchmarks. To deal with this issue, we set up the Automatic Detection challenge on Age-related Macular degeneration (ADAM), which was held as a satellite event of the ISBI 2020 conference. The ADAM challenge consisted of four tasks which cover the main aspects of detecting and characterizing AMD from fundus images, including detection of AMD, detection and segmentation of optic disc, localization of fovea, and detection and segmentation of lesions. As part of the ADAM challenge, we have released a comprehensive dataset of 1200 fundus images with AMD diagnostic labels, pixel-wise segmentation masks for both optic disc and AMD-related lesions (drusen, exudates, hemorrhages and scars, among others), as well as the coordinates corresponding to the location of the macular fovea. A uniform evaluation framework has been built to make a fair comparison of different models using this dataset. During the ADAM challenge, 610 results were submitted for online evaluation, with 11 teams finally participating in the onsite challenge. This paper introduces the challenge, the dataset and the evaluation methods, as well as summarizes the participating methods and analyzes their results for each task. In particular, we observed that the ensembling strategy and the incorporation of clinical domain knowledge were the key to improve the performance of the deep learning models.

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results

Deep learning (DL) models have provided state-of-the-art performance in various medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder translating DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties could enable clinical review of the most uncertain regions, thereby building trust and paving the way toward clinical translation. Several uncertainty estimation methods have recently been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentage of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, highlighting the need for uncertainty quantification in medical image analyses. Finally, in favor of transparency and reproducibility, our evaluation code is made publicly available at https://github.com/RagMeh11/QU-BraTS.

The University of Pennsylvania glioblastoma (UPenn-GBM) cohort: advanced MRI, clinical, genomics, & radiomics

Glioblastoma is the most common aggressive adult brain tumor. Numerous studies have reported results from either private institutional data or publicly available datasets. However, current public datasets are limited in terms of: a) number of subjects, b) lack of consistent acquisition protocol, c) data quality, or d) accompanying clinical, demographic, and molecular information. Toward alleviating these limitations, we contribute the "University of Pennsylvania Glioblastoma Imaging, Genomics, and Radiomics" (UPenn-GBM) dataset, which describes the currently largest publicly available comprehensive collection of 630 patients diagnosed with de novo glioblastoma. The UPenn-GBM dataset includes (a) advanced multi-parametric magnetic resonance imaging scans acquired during routine clinical practice, at the University of Pennsylvania Health System, (b) accompanying clinical, demographic, and molecular information, (d) perfusion and diffusion derivative volumes, (e) computationally-derived and manually-revised expert annotations of tumor sub-regions, as well as (f) quantitative imaging (also known as radiomic) features corresponding to each of these regions. This collection describes our contribution towards repeatable, reproducible, and comparative quantitative studies leading to new predictive, prognostic, and diagnostic assessments.

Deep Residual Separable Convolutional Neural Network for lung tumor segmentation

Lung cancer is one of the deadliest types of cancers. Computed Tomography (CT) is a widely used technique to detect tumors present inside the lungs. Delineation of such tumors is particularly essential for analysis and treatment purposes. With the advancement in hardware technologies, Machine Learning and Deep Learning methods are outperforming the traditional methods in the field of medical imaging. In order to delineate lung cancer tumors, we have proposed a deep learning-based methodology which includes a maximum intensity projection based pre-processing method, two novel deep learning networks and an ensemble strategy. The two proposed networks named Deep Residual Separable Convolutional Neural Network 1 and 2 (DRS-CNN1 and DRS-CNN2) achieved better performance over the state-of-the-art U-net network and other segmentation networks. For fair comparison, we have evaluated the performances of all networks on Medical Segmentation Decathlon (MSD) and StructSeg 2019 datasets. The DRS-CNN2 achieved a mean Dice Similarity Coefficient (DSC) of 0.649, mean 95 Hausdorff Distance (HD95) of 18.26, mean Sensitivity 0.737 and a mean Precision of 0.765 on independent test sets.

MoNuSAC2020: A Multi-Organ Nuclei Segmentation and Classification Challenge

Detecting various types of cells in and around the tumor matrix holds a special significance in characterizing the tumor micro-environment for cancer prognostication and research. Automating the tasks of detecting, segmenting, and classifying nuclei can free up the pathologists' time for higher value tasks and reduce errors due to fatigue and subjectivity. To encourage the computer vision research community to develop and test algorithms for these tasks, we prepared a large and diverse dataset of nucleus boundary annotations and class labels. The dataset has over 46,000 nuclei from 37 hospitals, 71 patients, four organs, and four nucleus types. We also organized a challenge around this dataset as a satellite event at the International Symposium on Biomedical Imaging (ISBI) in April 2020. The challenge saw a wide participation from across the world, and the top methods were able to match inter-human concordance for the challenge metric. In this paper, we summarize the dataset and the key findings of the challenge, including the commonalities and differences between the methods developed by various participants. We have released the MoNuSAC2020 dataset to the public.

NIMG-55. AUGMENTED INTELLIGENCE IS SUPERIOR TO ARTIFICIAL INTELLIGENCE! HUMAN-COMPUTER SYNERGY FOR GENERATING HIGH QUALITY GLIOBLASTOMA SUB-REGION SEGMENTATIONS

PURPOSE

Artificial intelligence (AI) is poised to improve diagnostic methods in neuro-oncologic imaging and contribute to patient management by analyzing pre-operative MRI scans. AI results are better interpreted by compartmentalizing glioblastoma into distinct sub-regions, i.e., necrotic core, enhancing tumor, peritumoral T2/FLAIR signal abnormality (ED). Manual delineation of these sub-regions by expert neuroradiologists is impractical, requiring hours for intricate cases. Computer-aided segmentation (CAS) can mitigate this issue but is limited in the quality of the produced segmentations. We hypothesize that CAS followed by expert refinements is more practical/time-efficient.

METHODS

CAS was used on a total of 359 glioblastoma patients with four MRI sequences (T1, T1Gd, T2, T2-FLAIR) from each patient. All segmentations were sent to expert neuroradiologist annotators for manual refinements. Once refined, our team including two senior attending neuroradiologists with ≥13 years of experience each, reviewed and either approved or returned the segmentations to individual annotators for further refinements. Total time required to refine and review the finalized segmentations was measured.

RESULTS

Following one round of refinements by expert annotators, 244/359 (68%) segmentations were approved by our team while 115/359 (32%) segmentations contained a variety of errors that required a second round of refinements. The most common observed errors were 1) missed ED in the anterior/inferior temporal lobes and corpus callosum (37/115 cases, 32%) and 2) erroneous segmentation of normal choroid plexus and blood vessels (14/115 cases, 12%). The expert annotators required 120 hours to refine all 359 segmentations, and our team required 26 additional hours to review them, resulting in 24 minutes/segmentation following CAS.

CONCLUSION

Our findings support the value of a well-communicated annotation protocol to coordinate CAS and expert annotators. With CAS, our team and expert annotators rapidly finalized segmentations for 359 glioblastoma patients, demonstrating the value of a synergistic approach to creating high quality tumor sub-region segmentations.

NIMG-32. THE FEDERATED TUMOR SEGMENTATION (FETS) INITIATIVE: THE FIRST REAL-WORLD LARGE-SCALE DATA-PRIVATE COLLABORATION FOCUSING ON NEURO-ONCOLOGY

PURPOSE

Robustness and generalizability of artificial intelligent (AI) methods is reliant on the training data size and diversity, which are currently hindered in multi-institutional healthcare collaborations by data ownership and legal concerns. To address these, we introduce the Federated Tumor Segmentation (FeTS) Initiative, as an international consortium using federated learning (FL) for data-private multi-institutional collaborations, where AI models leverage data at participating institutions, without sharing data between them. The initial FeTS use-case focused on detecting brain tumor boundaries in MRI.

METHODS

The FeTS tool incorporates: 1) MRI pre-processing, including image registration and brain extraction; 2) automatic delineation of tumor sub-regions, by label fusion of pretrained top-performing BraTS methods; 3) tools for manual delineation refinements; 4) model training. 55 international institutions identified local retrospective cohorts of glioblastoma patients. Ground truth was generated using the first 3 FeTS functionality modes as mentioned earlier. Finally, the FL training mode comprises of i) an AI model trained on local data, ii) local model updates shared with an aggregator, which iii) combines updates from all collaborators to generate a consensus model, and iv) circulates the consensus model back to all collaborators for iterative performance improvements.

RESULTS

The first FeTS consensus model, from 23 institutions with data of 2,200 patients, showed an average improvement of 11.1% in the performance of the model on each collaborator’s validation data, when compared to a model trained on the publicly available BraTS data (n=231).

CONCLUSION

Our findings support that data increase alone would lead to AI performance improvements without any algorithmic development, hence indicating that the model performance would improve further when trained with all 55 collaborating institutions. FL enables AI model training with knowledge from data of geographically-distinct collaborators, without ever having to share any data, hence overcoming hurdles relating to legal, ownership, and technical concerns of data sharing.

NIMG-39. RADIOMIC ANALYSIS FOR NON-INVASIVE IN VIVO PROGNOSTIC STRATIFICATION OF DE NOVO GLIOBLASTOMA PATIENTS: A MULTI-INSTITUTIONAL EVALUATION FOR GENERALIZABILITY IN THE RESPOND CONSORTIUM

PURPOSE

Multi-parametric MRI based radiomic signatures have highlighted the promise of artificial intelligence (AI) in neuro-oncology. However, inter-institution heterogeneity hinders generalization to data from unseen clinical institutions. To this end, we formulated the ReSPOND (Radiomics Signatures for PrecisiON Diagnostics) consortium for glioblastoma. Here, we seek non-invasive generalizable radiomic signatures from routine clinically-acquired MRI for prognostic stratification of glioblastoma patients.

METHODS

We identified a retrospective cohort of 606 patients with near/gross total tumor resection ( >90%), from 13 geographically-diverse institutions. All pre-operative structural MRI scans (T1,T1-Gd,T2,T2-FLAIR) were aligned to a common anatomical atlas. An automatic algorithm segmented the whole tumors (WTs) into 3 sub-compartments, i.e., enhancing (ET), necrotic core (NC), and peritumoral T2-FLAIR signal abnormality (ED). The combination of ET+NC defines the tumor core (TC). Quantitative radiomic features were extracted to generate our AI model to stratify patients into short- (< 14mts) and long-survivors ( >14mts). The model trained on 276 patients from a single institution was independently validated on 330 unseen patients from 12 left-out institutions, using the area-under-the-receiver-operating-characteristic-curve (AUC).

RESULTS

Each feature individually offered certain (limited but reproducible) value for identifying short-survivors: 1) TC closer to lateral ventricles (AUC=0.62); 2) larger ET/brain (AUC=0.61); 3) larger TC/brain (AUC=0.59); 4) larger WT/brain (AUC=0.55); 5) larger ET/WT (AUC=0.59); 6) smaller ED/WT (AUC=0.57); 7) larger ventricle deformations (AUC=0.6). Integrating all features and age, through a multivariate AI model, resulted in higher accuracy (AUC=0.7; 95% C.I.,0.64-0.77).

CONCLUSION

Prognostic stratification using basic radiomic features is highly reproducible across diverse institutions and patient populations. Multivariate integration yields relatively more accurate and generalizable radiomic signatures, across institutions. Our results offer promise for generalizable non-invasive in vivo signatures of survival prediction in patients with glioblastoma. Extracted features from clinically-acquired imaging, renders these signatures easier for clinical translation. Large-scale evaluation could contribute to improving patient management and treatment planning.

*Indicates equal authorship.

NIMG-22. PREDICTION OF GLIOBLASTOMA CELLULAR INFILTRATION AND RECURRENCE USING MACHINE LEARNING AND MULTI-PARAMETRIC MRI ANALYSIS: RESULTS FROM THE MULTI-INSTITUTIONAL RESPOND CONSORTIUM

PURPOSE

Multi-parametric MRI and artificial intelligence (AI) methods were previously used to predict peritumoral neoplastic cell infiltration and risk of future recurrence in glioblastoma, in single-institution studies. We hypothesize that important characteristics of peritumoral tissue heterogeneity captured, engineered/selected, and quantified by these methods relate to predictions generalizable in the multi-institutional ReSPOND (Radiomics Signatures for PrecisiON Diagnostics) consortium.

METHODS

To support further development, generalization, and clinical translation of our proposed method, we trained the AI model on a retrospective cohort of 29 de novo glioblastoma patients from the Hospital of the University of Pennsylvania (UPenn) (Male/Female:20/9, age:22-78 years) followed by evaluation on a prospective multi-institutional cohort of 84 glioblastoma patients (Male/Female:51/33, age:34-89 years) from Case Western Reserve University/University Hospitals (CWRU/UH, 25), New York University (NYU, 13), Ohio State University (OSU, 13), University Hospital Río Hortega (RH, 2), and UPenn (31). Features extracted from pre-resection MRI (T1, T1-Gd, T2, T2-FLAIR, ADC) were used to build our model predicting the spatial pattern of subsequent tumor recurrence. These predictions were evaluated against regions of pathology-confirmed post-resection recurrence.

RESULTS

Our model predicted the locations that later harbored tumor recurrence with sensitivity 83%, AUC 0.83 (99% CI, 0.73-0.93), and odds ratio 7.23 (99% CI, 7.09-7.37) in the prospective cohort. Odds ratio (99% CI)/AUC(99% CI) per institute were: CWRU/UH, 7.8(7.6-8.1)/0.82(0.75-0.89); NYU, 3.5(3.3-3.6)/0.84(074-0.93); OSU, 7.9(7.6-8.3)/0.8(0.67-0.94); RH, 22.7(20-25.1)/0.94(0.27-1); UPenn, 7.1(6.8-7.3)/0.83(0.76-0.91).

CONCLUSION

This is the first study that provides relatively extensive multi-institutional validated evidence that AI can provide good predictions of peritumoral neoplastic cell infiltration and future recurrence, by dissecting the MRI signal heterogeneity in peritumoral tissue. Our analyses leveraged the unique dataset of the ReSPOND consortium, which aims to develop and evaluate AI-based biomarkers for individualized prediction and prognostication, by moving from single-institution studies to generalizable, well-validated multi-institutional predictive biomarkers.

Cascaded Dilated Deep Residual Network for Volumetric Liver Segmentation From CT Image

Volumetric liver segmentation is a prerequisite for liver transplantation and radiation therapy planning. In this paper, dilated deep residual network (DDRN) has been proposed for automatic segmentation of liver from CT images. The combination of three parallel DDRN is cascaded with fourth DDRN in order to get final result. The volumetric CT data of 40 subjects belongs to “Combined Healthy Abdominal Organ Segmentation” (CHAOS) challenge 2019 is utilized to evaluate the proposed method. Input image converted into three images using windowing ranges and fed to three DDRN. The output of three DDRN along with original image fed to the fourth DDRN as an input. The output of cascaded network is compared with the three parallel DDRN individually. Obtained results were quantitatively evaluated with various evaluation parameters. The results were submitted to online evaluation system, and achieved average dice coefficient is 0.93±0.02; average symmetric surface distance (ASSD) is 4.89±0.91. In conclusion, obtained results are prominent and consistent.

Overall Survival Prediction in Glioblastoma With Radiomic Features Using Machine Learning

Glioblastoma is a WHO grade IV brain tumor, which leads to poor overall survival (OS) of patients. For precise surgical and treatment planning, OS prediction of glioblastoma (GBM) patients is highly desired by clinicians and oncologists. Radiomic research attempts at predicting disease prognosis, thus providing beneficial information for personalized treatment from a variety of imaging features extracted from multiple MR images. In this study, first-order, intensity-based volume and shape-based and textural radiomic features are extracted from fluid-attenuated inversion recovery (FLAIR) and T1ce MRI data. The region of interest is further decomposed with stationary wavelet transform with low-pass and high-pass filtering. Further, radiomic features are extracted on these decomposed images, which helped in acquiring the directional information. The efficiency of the proposed algorithm is evaluated on Brain Tumor Segmentation (BraTS) challenge training, validation, and test datasets. The proposed approach achieved 0.695, 0.571, and 0.558 on BraTS training, validation, and test datasets. The proposed approach secured the third position in BraTS 2018 challenge for the OS prediction task.

A Novel Approach for Fully Automatic Intra-Tumor Segmentation With 3D U-Net Architecture for Gliomas

Purpose: Gliomas are the most common primary brain malignancies, with varying degrees of aggressiveness and prognosis. Understanding of tumor biology and intra-tumor heterogeneity is necessary for planning personalized therapy and predicting response to therapy. Accurate tumoral and intra-tumoral segmentation on MRI is the first step toward understanding the tumor biology through computational methods. The purpose of this study was to design a segmentation algorithm and evaluate its performance on pre-treatment brain MRIs obtained from patients with gliomas.

Materials and Methods: In this study, we have designed a novel 3D U-Net architecture that segments various radiologically identifiable sub-regions like edema, enhancing tumor, and necrosis. Weighted patch extraction scheme from the tumor border regions is proposed to address the problem of class imbalance between tumor and non-tumorous patches. The architecture consists of a contracting path to capture context and the symmetric expanding path that enables precise localization. The Deep Convolutional Neural Network (DCNN) based architecture is trained on 285 patients, validated on 66 patients and tested on 191 patients with Glioma from Brain Tumor Segmentation (BraTS) 2018 challenge dataset. Three dimensional patches are extracted from multi-channel BraTS training dataset to train 3D U-Net architecture. The efficacy of the proposed approach is also tested on an independent dataset of 40 patients with High Grade Glioma from our tertiary cancer center. Segmentation results are assessed in terms of Dice Score, Sensitivity, Specificity, and Hausdorff 95 distance (ITCN intra-tumoral classification network).

Result: Our proposed architecture achieved Dice scores of 0.88, 0.83, and 0.75 for the whole tumor, tumor core and enhancing tumor, respectively, on BraTS validation dataset and 0.85, 0.77, 0.67 on test dataset. The results were similar on the independent patients' dataset from our hospital, achieving Dice scores of 0.92, 0.90, and 0.81 for the whole tumor, tumor core and enhancing tumor, respectively.

Conclusion: The results of this study show the potential of patch-based 3D U-Net for the accurate intra-tumor segmentation. From experiments, it is observed that the weighted patch-based segmentation approach gives comparable performance with the pixel-based approach when there is a thin boundary between tumor subparts.