Brain metastases detection on MR by means of three-dimensional tumor-appearance template matching

Deep learning ensembles for detecting brain metastases in longitudinal multi-modal MRI studies

Metastatic brain cancer is a condition characterized by the migration of cancer cells to the brain from extracranial sites. Notably, metastatic brain tumors surpass primary brain tumors in prevalence by a significant factor, they exhibit an aggressive growth potential and have the capacity to spread across diverse cerebral locations simultaneously. Magnetic resonance imaging (MRI) scans of individuals afflicted with metastatic brain tumors unveil a wide spectrum of characteristics. These lesions vary in size and quantity, spanning from tiny nodules to substantial masses captured within MRI. Patients may present with a limited number of lesions or an extensive burden of hundreds of them. Moreover, longitudinal studies may depict surgical resection cavities, as well as areas of necrosis or edema. Thus, the manual analysis of such MRI scans is difficult, user-dependent and cost-inefficient, and - importantly - it lacks reproducibility. We address these challenges and propose a pipeline for detecting and analyzing brain metastases in longitudinal studies, which benefits from an ensemble of various deep learning architectures originally designed for different downstream tasks (detection and segmentation). The experiments, performed over 275 multi-modal MRI scans of 87 patients acquired in 53 sites, coupled with rigorously validated manual annotations, revealed that our pipeline, built upon open-source tools to ensure its reproducibility, offers high-quality detection, and allows for precisely tracking the disease progression. To objectively quantify the generalizability of models, we introduce a new data stratification approach that accommodates the heterogeneity of the dataset and is used to elaborate training-test splits in a data-robust manner, alongside a new set of quality metrics to objectively assess algorithms. Our system provides a fully automatic and quantitative approach that may support physicians in a laborious process of disease progression tracking and evaluation of treatment efficacy.

Prediction of treatment response after stereotactic radiosurgery of brain metastasis using deep learning and radiomics on longitudinal MRI data

We developed artificial intelligence models to predict the brain metastasis (BM) treatment response after stereotactic radiosurgery (SRS) using longitudinal magnetic resonance imaging (MRI) data and evaluated prediction accuracy changes according to the number of sequential MRI scans. We included four sequential MRI scans for 194 patients with BM and 369 target lesions for the Developmental dataset. The data were randomly split (8:2 ratio) for training and testing. For external validation, 172 MRI scans from 43 patients with BM and 62 target lesions were additionally enrolled. The maximum axial diameter (Dmax), radiomics, and deep learning (DL) models were generated for comparison. We evaluated the simple convolutional neural network (CNN) model and a gated recurrent unit (Conv-GRU)-based CNN model in the DL arm. The Conv-GRU model performed superior to the simple CNN models. For both datasets, the area under the curve (AUC) was significantly higher for the two-dimensional (2D) Conv-GRU model than for the 3D Conv-GRU, Dmax, and radiomics models. The accuracy of the 2D Conv-GRU model increased with the number of follow-up studies. In conclusion, using longitudinal MRI data, the 2D Conv-GRU model outperformed all other models in predicting the treatment response after SRS of BM.

Incidence and imaging characteristics of difficult to detect retrospectively identified brain metastases in patients receiving repeat courses of stereotactic radiosurgery

PURPOSE

During stereotactic radiosurgery (SRS) planning for brain metastases (BM), brain MRIs are reviewed to select appropriate targets based on radiographic characteristics. Some BM are difficult to detect and/or definitively identify and may go untreated initially, only to become apparent on future imaging. We hypothesized that in patients receiving multiple courses of SRS, reviewing the initial planning MRI would reveal early evidence of lesions that developed into metastases requiring SRS.

METHODS

Patients undergoing two or more courses of SRS to BM within 6 months between 2016 and 2018 were included in this single-institution, retrospective study. Brain MRIs from the initial course were reviewed for lesions at the same location as subsequently treated metastases; if present, this lesion was classified as a "retrospectively identified metastasis" or RIM. RIMs were subcategorized as meeting or not meeting diagnostic imaging criteria for BM (+ DC or -DC, respectively).

RESULTS

Among 683 patients undergoing 923 SRS courses, 98 patients met inclusion criteria. There were 115 repeat courses of SRS, with 345 treated metastases in the subsequent course, 128 of which were associated with RIMs found in a prior MRI. 58% of RIMs were + DC. 17 (15%) of subsequent courses consisted solely of metastases associated with + DC RIMs.

CONCLUSION

Radiographic evidence of brain metastases requiring future treatment was occasionally present on brain MRIs from prior SRS treatments. Most RIMs were + DC, and some subsequent SRS courses treated only + DC RIMs. These findings suggest enhanced BM detection might enable earlier treatment and reduce the need for additional SRS.

Construction and evaluation of a gated high-resolution neural network for automatic brain metastasis detection and segmentation

OBJECTIVES

To construct and evaluate a gated high-resolution convolutional neural network for detecting and segmenting brain metastasis (BM).

METHODS

This retrospective study included craniocerebral MRI scans of 1392 patients with 14,542 BMs and 200 patients with no BM between January 2012 and April 2022. A primary dataset including 1000 cases with 11,686 BMs was employed to construct the model, while an independent dataset including 100 cases with 1069 BMs from other hospitals was used to examine the generalizability. The potential of the model for clinical use was also evaluated by comparing its performance in BM detection and segmentation to that of radiologists, and comparing radiologists' lesion detecting performances with and without model assistance.

RESULTS

Our model yielded a recall of 0.88, a dice similarity coefficient (DSC) of 0.90, a positive predictive value (PPV) of 0.93 and a false positives per patient (FP) of 1.01 in the test set, and a recall of 0.85, a DSC of 0.89, a PPV of 0.93, and a FP of 1.07 in dataset from other hospitals. With the model's assistance, the BM detection rates of 4 radiologists improved significantly, ranging from 5.2 to 15.1% (all p < 0.001), and also for detecting small BMs with diameter ≤ 5 mm (ranging from 7.2 to 27.0%, all p < 0.001).

CONCLUSIONS

The proposed model enables accurate BM detection and segmentation with higher sensitivity and less time consumption, showing the potential to augment radiologists' performance in detecting BM.

CLINICAL RELEVANCE STATEMENT

This study offers a promising computer-aided tool to assist the brain metastasis detection and segmentation in routine clinical practice for cancer patients.

KEY POINTS

• The GHR-CNN could accurately detect and segment BM on contrast-enhanced 3D-T1W images. • The GHR-CNN improved the BM detection rate of radiologists, including the detection of small lesions. • The GHR-CNN enabled automated segmentation of BM in a very short time.

False-negative and false-positive outcomes of computer-aided detection on brain metastasis: Secondary analysis of a multicenter, multireader study

BACKGROUND

Errors have seldom been evaluated in computer-aided detection on brain metastases. This study aimed to analyze false negatives (FNs) and false positives (FPs) generated by a brain metastasis detection system (BMDS) and by readers.

METHODS

A deep learning-based BMDS was developed and prospectively validated in a multicenter, multireader study. Ad hoc secondary analysis was restricted to the prospective participants (148 with 1,066 brain metastases and 152 normal controls). Three trainees and 3 experienced radiologists read the MRI images without and with the BMDS. The number of FNs and FPs per patient, jackknife alternative free-response receiver operating characteristic figure of merit (FOM), and lesion features associated with FNs were analyzed for the BMDS and readers using binary logistic regression.

RESULTS

The FNs, FPs, and the FOM of the stand-alone BMDS were 0.49, 0.38, and 0.97, respectively. Compared with independent reading, BMDS-assisted reading generated 79% fewer FNs (1.98 vs 0.42, P < .001); 41% more FPs (0.17 vs 0.24, P < .001) but 125% more FPs for trainees (P < .001); and higher FOM (0.87 vs 0.98, P < .001). Lesions with small size, greater number, irregular shape, lower signal intensity, and located on nonbrain surface were associated with FNs for readers. Small, irregular, and necrotic lesions were more frequently found in FNs for BMDS. The FPs mainly resulted from small blood vessels for the BMDS and the readers.

CONCLUSIONS

Despite the improvement in detection performance, attention should be paid to FPs and small lesions with lower enhancement for radiologists, especially for less-experienced radiologists.

A Deep Learning-Based Computer Aided Detection (CAD) System for Difficult-to-Detect Brain Metastases

PURPOSE

We sought to develop a computer-aided detection (CAD) system that optimally augments human performance, excelling especially at identifying small inconspicuous brain metastases (BMs), by training a convolutional neural network on a unique magnetic resonance imaging (MRI) data set containing subtle BMs that were not detected prospectively during routine clinical care.

METHODS AND MATERIALS

Patients receiving stereotactic radiosurgery (SRS) for BMs at our institution from 2016 to 2018 without prior brain-directed therapy or small cell histology were eligible. For patients who underwent 2 consecutive courses of SRS, treatment planning MRIs from their initial course were reviewed for radiographic evidence of an emerging metastasis at the same location as metastases treated in their second SRS course. If present, these previously unidentified lesions were contoured and categorized as retrospectively identified metastases (RIMs). RIMs were further subcategorized according to whether they did (+DC) or did not (-DC) meet diagnostic imaging-based criteria to definitively classify them as metastases based upon their appearance in the initial MRI alone. Prospectively identified metastases (PIMs) from these patients, and from patients who only underwent a single course of SRS, were also included. An open-source convolutional neural network architecture was adapted and trained to detect both RIMs and PIMs on thin-slice, contrast-enhanced, spoiled gradient echo MRIs. Patients were randomized into 5 groups: 4 for training/cross-validation and 1 for testing.

RESULTS

One hundred thirty-five patients with 563 metastases, including 72 RIMS, met criteria. For the test group, CAD sensitivity was 94% for PIMs, 80% for +DC RIMs, and 79% for PIMs and +DC RIMs with diameter <3 mm, with a median of 2 false positives per patient and a Dice coefficient of 0.79.

CONCLUSIONS

Our CAD model, trained on a novel data set and using a single common MR sequence, demonstrated high sensitivity and specificity overall, outperforming published CAD results for small metastases and RIMs - the lesion types most in need of human performance augmentation.

Application of artificial intelligence to stereotactic radiosurgery for intracranial lesions: detection, segmentation, and outcome prediction

BACKGROUND

Rapid evolution of artificial intelligence (AI) prompted its wide application in healthcare systems. Stereotactic radiosurgery served as a good candidate for AI model development and achieved encouraging result in recent years. This article aimed at demonstrating current AI application in radiosurgery.

METHODS

Literatures published in PubMed during 2010-2022, discussing AI application in stereotactic radiosurgery were reviewed.

RESULTS

AI algorithms, especially machine learning/deep learning models, have been administered to different aspect of stereotactic radiosurgery. Spontaneous tumor detection and automated lesion delineation or segmentation were two of the promising application, which could be further extended to longitudinal treatment follow-up. Outcome prediction utilized machine learning algorithms with radiomic-based analysis was another well-established application.

CONCLUSIONS

Stereotactic radiosurgery has taken a lead role in AI development. Current achievement, limitation, and further investigation was summarized in this article.

Deep learning for brain metastasis detection and segmentation in longitudinal MRI data

PURPOSE

Brain metastases occur frequently in patients with metastatic cancer. Early and accurate detection of brain metastases is essential for treatment planning and prognosis in radiation therapy. Due to their tiny sizes and relatively low contrast, small brain metastases are very difficult to detect manually. With the recent development of deep learning technologies, several researchers have reported promising results in automated brain metastasis detection. However, the detection sensitivity is still not high enough for tiny brain metastases, and integration into clinical practice in regard to differentiating true metastases from false positives is challenging.

METHODS

The DeepMedic network with the binary cross-entropy (BCE) loss is used as our baseline method. To improve brain metastasis detection performance, a custom detection loss called volume-level sensitivity-specificity (VSS) is proposed, which rates metastasis detection sensitivity and specificity at a (sub-)volume level. As sensitivity and precision are always a trade-off, either a high sensitivity or a high precision can be achieved for brain metastasis detection by adjusting the weights in the VSS loss without decline in dice score coefficient for segmented metastases. To reduce metastasis-like structures being detected as false positive metastases, a temporal prior volume is proposed as an additional input of DeepMedic. The modified network is called DeepMedic+ for distinction. Combining a high sensitivity VSS loss and a high specificity loss for DeepMedic+, the majority of true positive metastases are confirmed with high specificity, while additional metastases candidates in each patient are marked with high sensitivity for detailed expert evaluation.

RESULTS

Our proposed VSS loss improves the sensitivity of brain metastasis detection, increasing the sensitivity from 85.3% for DeepMedic with BCE to 97.5% for DeepMedic with VSS. Alternatively, the precision is improved from 69.1% for DeepMedic with BCE to 98.7% for DeepMedic with VSS. Comparing DeepMedic+ with DeepMedic with the same VSS loss, 44.4% of the false positive metastases are reduced in the high sensitivity model and the precision reaches 99.6% for the high specificity model. The mean dice coefficient for all metastases is about 0.81. With the ensemble of the high sensitivity and high specificity models, on average only 1.5 false positive metastases per patient need further check, while the majority of true positive metastases are confirmed.

CONCLUSIONS

Our proposed VSS loss and temporal prior improve brain metastasis detection sensitivity and precision. The ensemble learning is able to distinguish high confidence true positive metastases from metastases candidates that require special expert review or further follow-up, being particularly well-fit to the requirements of expert support in real clinical practice. This facilitates metastasis detection and segmentation for neuroradiologists in diagnostic and radiation oncologists in therapeutic clinical applications. This article is protected by copyright. All rights reserved.

Deep Learning-Based Computer-Aided Detection System for Automated Treatment Response Assessment of Brain Metastases on 3D MRI

BACKGROUND

Although accurate treatment response assessment for brain metastases (BMs) is crucial, it is highly labor intensive. This retrospective study aimed to develop a computer-aided detection (CAD) system for automated BM detection and treatment response evaluation using deep learning.

METHODS

We included 214 consecutive MRI examinations of 147 patients with BM obtained between January 2015 and August 2016. These were divided into the training (174 MR images from 127 patients) and test datasets according to temporal separation (temporal test set #1; 40 MR images from 20 patients). For external validation, 24 patients with BM and 11 patients without BM from other institutions were included (geographic test set). In addition, we included 12 MRIs from BM patients obtained between August 2017 and March 2020 (temporal test set #2). Detection sensitivity, dice similarity coefficient (DSC) for segmentation, and agreements in one-dimensional and volumetric Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) criteria between CAD and radiologists were assessed.

RESULTS

In the temporal test set #1, the sensitivity was 75.1% (95% confidence interval [CI]: 69.6%, 79.9%), mean DSC was 0.69 ± 0.22, and false-positive (FP) rate per scan was 0.8 for BM ≥ 5 mm. Agreements in the RANO-BM criteria were moderate (κ, 0.52) and substantial (κ, 0.68) for one-dimensional and volumetric, respectively. In the geographic test set, sensitivity was 87.7% (95% CI: 77.2%, 94.5%), mean DSC was 0.68 ± 0.20, and FP rate per scan was 1.9 for BM ≥ 5 mm. In the temporal test set #2, sensitivity was 94.7% (95% CI: 74.0%, 99.9%), mean DSC was 0.82 ± 0.20, and FP per scan was 0.5 (6/12) for BM ≥ 5 mm.

CONCLUSIONS

Our CAD showed potential for automated treatment response assessment of BM ≥ 5 mm.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

Automatic segmentation of brain metastases using T1 magnetic resonance and computed tomography images

An increasing number of patients with multiple brain metastases are being treated with stereotactic radiosurgery (SRS). Manually identifying and contouring all metastatic lesions is difficult and time-consuming, and a potential source of variability. Hence, we developed a 3D deep learning approach for segmenting brain metastases on MR and CT images. Five-hundred eleven patients treated with SRS were retrospectively identified for this study. Prior to radiotherapy, the patients were imaged with 3D T1 spoiled-gradient MR post-Gd (T1 + C) and contrast-enhanced CT (CECT), which were co-registered by a treatment planner. The gross tumor volume contours, authored by the attending radiation oncologist, were taken as the ground truth. There were 3 ± 4 metastases per patient, with volume up to 57 ml. We produced a multi-stage model that automatically performs brain extraction, followed by detection and segmentation of brain metastases using co-registered T1 + C and CECT. Augmented data from 80% of these patients were used to train modified 3D V-Net convolutional neural networks for this task. We combined a normalized boundary loss function with soft Dice loss to improve the model optimization, and employed gradient accumulation to stabilize the training. The average Dice similarity coefficient (DSC) for brain extraction was 0.975 ± 0.002 (95% CI). The detection sensitivity per metastasis was 90% (329/367), with moderate dependence on metastasis size. Averaged across 102 test patients, our approach had metastasis detection sensitivity 95 ± 3%, 2.4 ± 0.5 false positives, DSC of 0.76 ± 0.03, and 95th-percentile Hausdorff distance of 2.5 ± 0.3 mm (95% CIs). The volumes of automatic and manual segmentations were strongly correlated for metastases of volume up to 20 ml (r=0.97,p<0.001). This work expounds a fully 3D deep learning approach capable of automatically detecting and segmenting brain metastases using co-registered T1 + C and CECT.

Contrast-Enhanced Black Blood MRI Sequence Is Superior to Conventional T1 Sequence in Automated Detection of Brain Metastases by Convolutional Neural Networks

Background: in magnetic resonance imaging (MRI), automated detection of brain metastases with convolutional neural networks (CNN) represents an extraordinary challenge due to small lesions sometimes posing as brain vessels as well as other confounders. Literature reporting high false positive rates when using conventional contrast enhanced (CE) T1 sequences questions their usefulness in clinical routine. CE black blood (BB) sequences may overcome these limitations by suppressing contrast-enhanced structures, thus facilitating lesion detection. This study compared CNN performance in conventional CE T1 and BB sequences and tested for objective improvement of brain lesion detection. Methods: we included a subgroup of 127 consecutive patients, receiving both CE T1 and BB sequences, referred for MRI concerning metastatic spread to the brain. A pretrained CNN was retrained with a customized monolayer classifier using either T1 or BB scans of brain lesions. Results: CE T1 imaging-based training resulted in an internal validation accuracy of 85.5% vs. 92.3% in BB imaging (p < 0.01). In holdout validation analysis, T1 image-based prediction presented poor specificity and sensitivity with an AUC of 0.53 compared to 0.87 in BB-imaging-based prediction. Conclusions: detection of brain lesions with CNN, BB-MRI imaging represents a highly effective input type when compared to conventional CE T1-MRI imaging. Use of BB-MRI can overcome the current limitations for automated brain lesion detection and the objectively excellent performance of our CNN suggests routine usage of BB sequences for radiological analysis.

Three-dimensional U-Net Convolutional Neural Network for Detection and Segmentation of Intracranial Metastases

PURPOSE

To develop and validate a neural network for automated detection and segmentation of intracranial metastases on brain MRI studies obtained for stereotactic radiosurgery treatment planning.

MATERIALS AND METHODS

In this retrospective study, 413 patients (average age, 61 years ± 12 [standard deviation]; 238 women) with a total of 5202 intracranial metastases (median volume, 0.05 cm³; interquartile range, 0.02-0.18 cm³) undergoing stereotactic radiosurgery at one institution were included (January 2017 to February 2020). A total of 563 MRI examinations were performed among the patients, and studies were split into training (n = 413), validation (n = 50), and test (n = 100) datasets. A three-dimensional (3D) U-Net convolutional network was trained and validated on 413 T1 postcontrast or subtraction scans, and several loss functions were evaluated. After model validation, 100 discrete test patients, who underwent imaging after the training and validation patients, were used for final model evaluation. Performance for detection and segmentation of metastases was evaluated using Dice scores, false discovery rates, and false-negative rates, and a comparison with neuroradiologist interrater reliability was performed.

RESULTS

The median Dice score for segmenting enhancing metastases in the test set was 0.75 (interquartile range, 0.63-0.84). There were strong correlations between manually segmented and predicted metastasis volumes (r = 0.98, P < .001) and between the number of manually segmented and predicted metastases (R = 0.95, P < .001). Higher Dice scores were strongly correlated with larger metastasis volumes on a logarithmically transformed scale (r = 0.71). Sensitivity across the whole test sample was 70.0% overall and 96.4% for metastases larger than 6 mm. There was an average of 0.46 false-positive results per scan, with the positive predictive value being 91.5%. In comparison, the median Dice score between two neuroradiologists was 0.85 (interquartile range, 0.80-0.89), with sensitivity across the test sample being 87.9% overall and 98.4% for metastases larger than 6 mm.

CONCLUSION

A 3D U-Net-based convolutional neural network was able to segment brain metastases with high accuracy and perform detection at the level of human interrater reliability for metastases larger than 6 mm.Keywords: Adults, Brain/Brain Stem, CNS, Feature detection, MR-Imaging, Neural Networks, Neuro-Oncology, Quantification, Segmentation© RSNA, 2021.

Deep learning-based detection and segmentation-assisted management of brain metastases

BACKGROUND

Three-dimensional T1 magnetization prepared rapid acquisition gradient echo (3D-T1-MPRAGE) is preferred in detecting brain metastases (BM) among MRI. We developed an automatic deep learning-based detection and segmentation method for BM (named BMDS net) on 3D-T1-MPRAGE images and evaluated its performance.

METHODS

The BMDS net is a cascaded 3D fully convolution network (FCN) to automatically detect and segment BM. In total, 1652 patients with 3D-T1-MPRAGE images from 3 hospitals (n = 1201, 231, and 220, respectively) were retrospectively included. Manual segmentations were obtained by a neuroradiologist and a radiation oncologist in a consensus reading in 3D-T1-MPRAGE images. Sensitivity, specificity, and dice ratio of the segmentation were evaluated. Specificity and sensitivity measure the fractions of relevant segmented voxels. Dice ratio was used to quantitatively measure the overlap between automatic and manual segmentation results. Paired samples t-tests and analysis of variance were employed for statistical analysis.

RESULTS

The BMDS net can detect all BM, providing a detection result with an accuracy of 100%. Automatic segmentations correlated strongly with manual segmentations through 4-fold cross-validation of the dataset with 1201 patients: the sensitivity was 0.96 ± 0.03 (range, 0.84-0.99), the specificity was 0.99 ± 0.0002 (range, 0.99-1.00), and the dice ratio was 0.85 ± 0.08 (range, 0.62-0.95) for total tumor volume. Similar performances on the other 2 datasets also demonstrate the robustness of BMDS net in correctly detecting and segmenting BM in various settings.

CONCLUSIONS

The BMDS net yields accurate detection and segmentation of BM automatically and could assist stereotactic radiotherapy management for diagnosis, therapy planning, and follow-up.

Brain metastasis detection using machine learning: a systematic review and meta-analysis

BACKGROUND

Accurate detection of brain metastasis (BM) is important for cancer patients. We aimed to systematically review the performance and quality of machine-learning-based BM detection on MRI in the relevant literature.

METHODS

A systematic literature search was performed for relevant studies reported before April 27, 2020. We assessed the quality of the studies using modified tailored questionnaires of the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) criteria and the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Pooled detectability was calculated using an inverse-variance weighting model.

RESULTS

A total of 12 studies were included, which showed a clear transition from classical machine learning (cML) to deep learning (DL) after 2018. The studies on DL used a larger sample size than those on cML. The cML and DL groups also differed in the composition of the dataset, and technical details such as data augmentation. The pooled proportions of detectability of BM were 88.7% (95% CI, 84-93%) and 90.1% (95% CI, 84-95%) in the cML and DL groups, respectively. The false-positive rate per person was lower in the DL group than the cML group (10 vs 135, P < 0.001). In the patient selection domain of QUADAS-2, three studies (25%) were designated as high risk due to non-consecutive enrollment and arbitrary exclusion of nodules.

CONCLUSION

A comparable detectability of BM with a low false-positive rate per person was found in the DL group compared with the cML group. Improvements are required in terms of quality and study design.

Radiomics and Deep Learning from Research to Clinical Workflow: Neuro-Oncologic Imaging

Imaging plays a key role in the management of brain tumors, including the diagnosis, prognosis, and treatment response assessment. Radiomics and deep learning approaches, along with various advanced physiologic imaging parameters, hold great potential for aiding radiological assessments in neuro-oncology. The ongoing development of new technology needs to be validated in clinical trials and incorporated into the clinical workflow. However, none of the potential neuro-oncological applications for radiomics and deep learning has yet been realized in clinical practice. In this review, we summarize the current applications of radiomics and deep learning in neuro-oncology and discuss challenges in relation to evidence-based medicine and reporting guidelines, as well as potential applications in clinical workflows and routine clinical practice.

[Current Applications and Future Perspectives of Brain Tumor Imaging]

뇌종양의 진단 및 치료 반응 평가의 기본이 되는 영상기법은 해부학적 영상이다. 현재 임상에서 사용 가능한 영상기법들 중 확산 강조 영상 및 관류 영상이 추가적인 정보를 제공하고 있다. 최근에는 종양의 유전체 변이와 이질성 평가가 중요해지면서 라디오믹스와 딥러닝을 이용한 영상분석기법의 임상 응용이 기대되고 있다. 본 종설에서는 뇌종양 영상 임상 적용에서 여전히 중요한 해부학적 영상을 중심으로 한 자기공명영상 촬영 권고안, 최신 영상기법 중 확산 강조 영상 및 관류 영상의 기본 원리, 병태생리학적 배경 및 임상응용, 마지막으로 최근 컴퓨터 기술의 발전으로 많이 연구되고 있는 라디오믹스와 딥러닝의 뇌종양에서의 향후 활용가치에 대해 기술하고자 한다.

Deep-Learning Detection of Cancer Metastases to the Brain on MRI

BACKGROUND

Approximately one-fourth of all cancer metastases are found in the brain. MRI is the primary technique for detection of brain metastasis, planning of radiotherapy, and the monitoring of treatment response. Progress in tumor treatment now requires detection of new or growing metastases at the small subcentimeter size, when these therapies are most effective.

PURPOSE

To develop a deep-learning-based approach for finding brain metastasis on MRI.

STUDY TYPE

Retrospective.

SEQUENCE

Axial postcontrast 3D T₁ -weighted imaging.

FIELD STRENGTH

1.5T and 3T.

POPULATION

A total of 361 scans of 121 patients were used to train and test the Faster region-based convolutional neural network (Faster R-CNN): 1565 lesions in 270 scans of 73 patients for training; 488 lesions in 91 scans of 48 patients for testing. From the 48 outputs of Faster R-CNN, 212 lesions in 46 scans of 18 patients were used for training the RUSBoost algorithm (MatLab) and 276 lesions in 45 scans of 30 patients for testing.

ASSESSMENT

Two radiologists diagnosed and supervised annotation of metastases on brain MRI as ground truth. This data were used to produce a 2-step pipeline consisting of a Faster R-CNN for detecting abnormal hyperintensity that may represent brain metastasis and a RUSBoost classifier to reduce the number of false-positive foci detected.

STATISTICAL TESTS

The performance of the algorithm was evaluated by using sensitivity, false-positive rate, and receiver's operating characteristic (ROC) curves. The detection performance was assessed both per-metastases and per-slice.

RESULTS

Testing on held-out brain MRI data demonstrated 96% sensitivity and 20 false-positive metastases per scan. The results showed an 87.1% sensitivity and 0.24 false-positive metastases per slice. The area under the ROC curve was 0.79.

CONCLUSION

Our results showed that deep-learning-based computer-aided detection (CAD) had the potential of detecting brain metastases with high sensitivity and reasonable specificity.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY STAGE: 2 J. Magn. Reson. Imaging 2020;52:1227-1236.

Machine Learning in Neuro-Oncology: Can Data Analysis From 5346 Patients Change Decision-Making Paradigms?

BACKGROUND

Machine learning (ML) is an application of artificial intelligence (AI) that gives computer systems the ability to learn data, without being explicitly programmed. Currently, ML has been successfully used for optical character recognition, spam filtering, and face recognition. The aim of the present study was to review the current applications of ML in the field of neuro-oncology.

METHODS

We conducted a systematic literature review using the PubMed and Cochrane databases using a keyword search for January 30, 2000 to March 31, 2018. The data were clustered for neuro-oncology scope of ML into 3 categories: patient outcome predictors, imaging analysis, and gene expression.

RESULTS

Data from 5346 patients in 29 studies were used to develop ML-based algorithms (MLBAs) in neuro-oncology. MLBAs were used to predict the outcomes for 2483 patients, with a sensitivity range of 78%-98% and specificity range of 76%-95%. In all studies, the MLBAs had greater accuracy than the conventional ones. MLBAs for image analysis showed accuracy in diagnosing low-grade versus high-grade gliomas, ranging from 80% to 93% and 90% for diagnosing high-grade glioma versus lymphoma. Seven studies used MLBAs to analyze gene expression in neuro-oncology.

CONCLUSIONS

MLBAs in neuro-oncology have been shown to predict patients' outcomes more accurately than conventional parameters in a retrospective analysis. If their high diagnostic accuracy in imaging analysis and detection of somatic mutations are corroborated in prospective studies, the use of tissue diagnosis or liquid biopsy might be curtailed. Finally, MLBAs are promising to help guide targeted therapy, can lead to personalized medicine, and open areas of study in the cancer cellular signaling system, not otherwise known.

Emerging Applications of Artificial Intelligence in Neuro-Oncology

Due to the exponential growth of computational algorithms, artificial intelligence (AI) methods are poised to improve the precision of diagnostic and therapeutic methods in medicine. The field of radiomics in neuro-oncology has been and will likely continue to be at the forefront of this revolution. A variety of AI methods applied to conventional and advanced neuro-oncology MRI data can already delineate infiltrating margins of diffuse gliomas, differentiate pseudoprogression from true progression, and predict recurrence and survival better than methods used in daily clinical practice. Radiogenomics will also advance our understanding of cancer biology, allowing noninvasive sampling of the molecular environment with high spatial resolution and providing a systems-level understanding of underlying heterogeneous cellular and molecular processes. By providing in vivo markers of spatial and molecular heterogeneity, these AI-based radiomic and radiogenomic tools have the potential to stratify patients into more precise initial diagnostic and therapeutic pathways and enable better dynamic treatment monitoring in this era of personalized medicine. Although substantial challenges remain, radiologic practice is set to change considerably as AI technology is further developed and validated for clinical use.

Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network

Stereotactic treatments are today the reference techniques for the irradiation of brain metastases in radiotherapy. The dose per fraction is very high, and delivered in small volumes (diameter <1 cm). As part of these treatments, effective detection and precise segmentation of lesions are imperative. Many methods based on deep-learning approaches have been developed for the automatic segmentation of gliomas, but very little for that of brain metastases. We adapted an existing 3D convolutional neural network (DeepMedic) to detect and segment brain metastases on MRI. At first, we sought to adapt the network parameters to brain metastases. We then explored the single or combined use of different MRI modalities, by evaluating network performance in terms of detection and segmentation. We also studied the interest of increasing the database with virtual patients or of using an additional database in which the active parts of the metastases are separated from the necrotic parts. Our results indicated that a deep network approach is promising for the detection and the segmentation of brain metastases on multimodal MRI.

Detection of Volume-Changing Metastatic Brain Tumors on Longitudinal MRI Using a Semiautomated Algorithm Based on the Jacobian Operator Field

BACKGROUND AND PURPOSE

Accurate follow-up of metastatic brain tumors has important implications for patient prognosis and management. The aim of this study was to develop and evaluate the accuracy of a semiautomated algorithm in detecting growing or shrinking metastatic brain tumors on longitudinal brain MRIs.

MATERIALS AND METHODS

We used 50 pairs of successive MR imaging datasets, 30 on 1.5T and 20 on 3T, containing contrast-enhanced 3D T1-weighted sequences. These yielded 150 growing or shrinking metastatic brain tumors. To detect them, we completed 2 major steps: 1) spatial normalization and calculation of the Jacobian operator field to quantify changes between scans, and 2) metastatic brain tumor candidate segmentation and detection of volume-changing metastatic brain tumors with the Jacobian operator field. Receiver operating characteristic analysis was used to assess the detection accuracy of the algorithm, and it was verified with jackknife resampling. The reference standard was based on detections by a neuroradiologist.

RESULTS

The areas under the receiver operating characteristic curves were 0.925 for 1.5T and 0.965 for 3T. Furthermore, at its optimal performance, the algorithm achieved a sensitivity of 85.1% and 92.1% and specificity of 86.7% and 91.3% for 1.5T and 3T, respectively. Vessels were responsible for most false-positives. Newly developed or resolved metastatic brain tumors were a major source of false-negatives.

CONCLUSIONS

The proposed algorithm could detect volume-changing metastatic brain tumors on longitudinal brain MRIs with statistically high accuracy, demonstrating its potential as a computer-aided change-detection tool for complementing the performance of radiologists, decreasing inter- and intraobserver variability, and improving efficacy.

Computer-aided detection of brain metastasis on 3D MR imaging: Observer performance study

PURPOSE

To assess the effect of computer-aided detection (CAD) of brain metastasis (BM) on radiologists' diagnostic performance in interpreting three-dimensional brain magnetic resonance (MR) imaging using follow-up imaging and consensus as the reference standard.

MATERIALS AND METHODS

The institutional review board approved this retrospective study. The study cohort consisted of 110 consecutive patients with BM and 30 patients without BM. The training data set included MR images of 80 patients with 450 BM nodules. The test set included MR images of 30 patients with 134 BM nodules and 30 patients without BM. We developed a CAD system for BM detection using template-matching and K-means clustering algorithms for candidate detection and an artificial neural network for false-positive reduction. Four reviewers (two neuroradiologists and two radiology residents) interpreted the test set images before and after the use of CAD in a sequential manner. The sensitivity, false positive (FP) per case, and reading time were analyzed. A jackknife free-response receiver operating characteristic (JAFROC) method was used to determine the improvement in the diagnostic accuracy.

RESULTS

The sensitivity of CAD was 87.3% with an FP per case of 302.4. CAD significantly improved the diagnostic performance of the four reviewers with a figure-of-merit (FOM) of 0.874 (without CAD) vs. 0.898 (with CAD) according to JAFROC analysis (p < 0.01). Statistically significant improvement was noted only for less-experienced reviewers (FOM without vs. with CAD, 0.834 vs. 0.877, p < 0.01). The additional time required to review the CAD results was approximately 72 sec (40% of the total review time).

CONCLUSION

CAD as a second reader helps radiologists improve their diagnostic performance in the detection of BM on MR imaging, particularly for less-experienced reviewers.