2.5D and 3D segmentation of brain metastases with deep learning on multinational MRI data

Multicenter privacy-preserving model training for deep learning brain metastases autosegmentation

OBJECTIVES

This work aims to explore the impact of multicenter data heterogeneity on deep learning brain metastases (BM) autosegmentation performance, and assess the efficacy of an incremental transfer learning technique, namely learning without forgetting (LWF), to improve model generalizability without sharing raw data.

MATERIALS AND METHODS

A total of six BM datasets from University Hospital Erlangen (UKER), University Hospital Zurich (USZ), Stanford, UCSF, New York University (NYU), and BraTS Challenge 2023 were used. First, the performance of the DeepMedic network for BM autosegmentation was established for exclusive single-center training and mixed multicenter training, respectively. Subsequently privacy-preserving bilateral collaboration was evaluated, where a pretrained model is shared to another center for further training using transfer learning (TL) either with or without LWF.

RESULTS

For single-center training, average F1 scores of BM detection range from 0.625 (NYU) to 0.876 (UKER) on respective single-center test data. Mixed multicenter training notably improves F1 scores at Stanford and NYU, with negligible improvement at other centers. When the UKER pretrained model is applied to USZ, LWF achieves a higher average F1 score (0.839) than naive TL (0.570) and single-center training (0.688) on combined UKER and USZ test data. Naive TL improves sensitivity and contouring accuracy, but compromises precision. Conversely, LWF demonstrates commendable sensitivity, precision and contouring accuracy. When applied to Stanford, similar performance was observed.

CONCLUSION

Data heterogeneity (e.g., variations in metastases density, spatial distribution, and image spatial resolution across centers) results in varying performance in BM autosegmentation, posing challenges to model generalizability. LWF is a promising approach to peer-to-peer privacy-preserving model training.

The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.

Deep Learning for Perfusion Cerebral Blood Flow (CBF) and Volume (CBV) Predictions and Diagnostics

Dynamic susceptibility contrast magnetic resonance perfusion (DSC-MRP) is a non-invasive imaging technique for hemodynamic measurements. Various perfusion parameters, such as cerebral blood volume (CBV) and cerebral blood flow (CBF), can be derived from DSC-MRP, hence this non-invasive imaging protocol is widely used clinically for the diagnosis and assessment of intracranial pathologies. Currently, most institutions use commercially available software to compute the perfusion parametric maps. However, these conventional methods often have limitations, such as being time-consuming and sensitive to user input, which can lead to inconsistent results; this highlights the need for a more robust and efficient approach like deep learning. Using the relative cerebral blood volume (rCBV) and relative cerebral blood flow (rCBF) perfusion maps generated by FDA-approved software, we trained a multistage deep learning model. The model, featuring a combination of a 1D convolutional neural network (CNN) and a 2D U-Net encoder-decoder network, processes each 4D MRP dataset by integrating temporal and spatial features of the brain for voxel-wise perfusion parameters prediction. An auxiliary model, with similar architecture, but trained with truncated datasets that had fewer time-points, was designed to explore the contribution of temporal features. Both qualitatively and quantitatively evaluated, deep learning-generated rCBV and rCBF maps showcased effective integration of temporal and spatial data, producing comprehensive predictions for the entire brain volume. Our deep learning model provides a robust and efficient approach for calculating perfusion parameters, demonstrating comparable performance to FDA-approved commercial software, and potentially mitigating the challenges inherent to traditional techniques.

Where Does Auto-Segmentation for Brain Metastases Radiosurgery Stand Today?

Detection and segmentation of brain metastases (BMs) play a pivotal role in diagnosis, treatment planning, and follow-up evaluations for effective BM management. Given the rising prevalence of BM cases and its predominantly multiple onsets, automated segmentation is becoming necessary in stereotactic radiosurgery. It not only alleviates the clinician's manual workload and improves clinical workflow efficiency but also ensures treatment safety, ultimately improving patient care. Recent strides in machine learning, particularly in deep learning (DL), have revolutionized medical image segmentation, achieving state-of-the-art results. This review aims to analyze auto-segmentation strategies, characterize the utilized data, and assess the performance of cutting-edge BM segmentation methodologies. Additionally, we delve into the challenges confronting BM segmentation and share insights gleaned from our algorithmic and clinical implementation experiences.

Development of RLK-Unet: a clinically favorable deep learning algorithm for brain metastasis detection and treatment response assessment

Purpose/objectives

Previous deep learning (DL) algorithms for brain metastasis (BM) detection and segmentation have not been commonly used in clinics because they produce false-positive findings, require multiple sequences, and do not reflect physiological properties such as necrosis. The aim of this study was to develop a more clinically favorable DL algorithm (RLK-Unet) using a single sequence reflecting necrosis and apply it to automated treatment response assessment.

Methods and materials

A total of 128 patients with 1339 BMs, who underwent BM magnetic resonance imaging using the contrast-enhanced 3D T1 weighted (T1WI) turbo spin-echo black blood sequence, were included in the development of the DL algorithm. Fifty-eight patients with 629 BMs were assessed for treatment response. The detection sensitivity, precision, Dice similarity coefficient (DSC), and agreement of treatment response assessments between neuroradiologists and RLK-Unet were assessed.

Results

RLK-Unet demonstrated a sensitivity of 86.9% and a precision of 79.6% for BMs and had a DSC of 0.663. Segmentation performance was better in the subgroup with larger BMs (DSC, 0.843). The agreement in the response assessment for BMs between the radiologists and RLK-Unet was excellent (intraclass correlation, 0.84).

Conclusion

RLK-Unet yielded accurate detection and segmentation of BM and could assist clinicians in treatment response assessment.

Brain metastasis tumor segmentation and detection using deep learning algorithms: A systematic review and meta-analysis

BACKGROUND

Manual detection of brain metastases is both laborious and inconsistent, driving the need for more efficient solutions. Accordingly, our systematic review and meta-analysis assessed the efficacy of deep learning algorithms in detecting and segmenting brain metastases from various primary origins in MRI images.

METHODS

We conducted a comprehensive search of PubMed, Embase, and Web of Science up to May 24, 2023, which yielded 42 relevant studies for our analysis. We assessed the quality of these studies using the QUADAS-2 and CLAIM tools. Using a random-effect model, we calculated the pooled lesion-wise dice score as well as patient-wise and lesion-wise sensitivity. We performed subgroup analyses to investigate the influence of factors such as publication year, study design, training center of the model, validation methods, slice thickness, model input dimensions, MRI sequences fed to the model, and the specific deep learning algorithms employed. Additionally, meta-regression analyses were carried out considering the number of patients in the studies, count of MRI manufacturers, count of MRI models, training sample size, and lesion number.

RESULTS

Our analysis highlighted that deep learning models, particularly the U-Net and its variants, demonstrated superior segmentation accuracy. Enhanced detection sensitivity was observed with an increased diversity in MRI hardware, both in terms of manufacturer and model variety. Furthermore, slice thickness was identified as a significant factor influencing lesion-wise detection sensitivity. Overall, the pooled results indicated a lesion-wise dice score of 79%, with patient-wise and lesion-wise sensitivities at 86% and 87%, respectively.

CONCLUSIONS

The study underscores the potential of deep learning in improving brain metastasis diagnostics and treatment planning. Still, more extensive cohorts and larger meta-analysis are needed for more practical and generalizable algorithms. Future research should prioritize these areas to advance the field. This study was funded by the Gen. & Mrs. M.C. Peng Fellowship and registered under PROSPERO (CRD42023427776).

Comparison of 2D, 2.5D, and 3D segmentation networks for maxillary sinuses and lesions in CBCT images

BACKGROUND

The purpose of this study was to compare the segmentation performances of the 2D, 2.5D, and 3D networks for maxillary sinuses (MSs) and lesions inside the maxillary sinus (MSL) with variations in sizes, shapes, and locations in cone beam CT (CBCT) images under the same constraint of memory capacity.

METHODS

The 2D, 2.5D, and 3D networks were compared comprehensively for the segmentation of the MS and MSL in CBCT images under the same constraint of memory capacity. MSLs were obtained by subtracting the prediction of the air region of the maxillary sinus (MSA) from that of the MS.

RESULTS

The 2.5D network showed the highest segmentation performances for the MS and MSA compared to the 2D and 3D networks. The performances of the Jaccard coefficient, Dice similarity coefficient, precision, and recall by the 2.5D network of U-net +  + reached 0.947, 0.973, 0.974, and 0.971 for the MS, respectively, and 0.787, 0.875, 0.897, and 0.858 for the MSL, respectively.

CONCLUSIONS

The 2.5D segmentation network demonstrated superior segmentation performance for various MSLs with an ensemble learning approach of combining the predictions from three orthogonal planes.

Identification of Origin for Spinal Metastases from MR Images: Comparison Between Radiomics and Deep Learning Methods

OBJECTIVE

To determine whether spinal metastatic lesions originated from lung cancer or from other cancers based on spinal contrast-enhanced T1 (CET1) magnetic resonance (MR) images analyzed using radiomics (RAD) and deep learning (DL) methods.

METHODS

We recruited and retrospectively reviewed 173 patients diagnosed with spinal metastases at two different centers between July 2018 and June 2021. Of these, 68 involved lung cancer and 105 were other types of cancer. They were assigned to an internal cohort of 149 patients, randomly divided into a training set and a validation set, and to an external cohort of 24 patients. All patients underwent CET1-MR imaging before surgery or biopsy. We developed two predictive algorithms: a DL model and a RAD model. We compared performance between models, and against human radiological assessment, via accuracy (ACC) and receiver operating characteristic (ROC) analyses. Furthermore, we analyzed the correlation between RAD and DL features.

RESULTS

The DL model outperformed RAD model across the board, with ACC/ area under the receiver operating characteristic curve (AUC) values of 0.93/0.94 (DL) versus 0.84/0.93 (RAD) when applied to the training set from the internal cohort, 0.74/0.76 versus 0.72/0.75 when applied to the validation set, and 0.72/0.76 versus 0.69/0.72 when applied to the external test cohort. For the validation set, it also outperformed expert radiological assessment (ACC: 0.65, AUC: 0.68). We only found weak correlations between DL and RAD features.

CONCLUSION

The DL algorithm successfully identified the origin of spinal metastases from pre-operative CET1-MR images, outperforming both RAD models and expert assessment by trained radiologists.