Brain metastasis detection using machine learning: a systematic review and meta-analysis

A systematic review on artificial intelligence evaluating PSMA PET scan for intraprostatic cancer

OBJECTIVES

To assess artificial intelligence (AI) ability to evaluate intraprostatic prostate cancer (PCa) on prostate-specific membrane antigen positron emission tomography (PSMA PET) scans prior to active treatment (radiotherapy or prostatectomy).

MATERIALS AND METHODS

This systematic review was registered on the International Prospective Register of Systematic Reviews (PROSPERO identifier: CRD42023438706). A search was performed on Medline, Embase, Web of Science, and Engineering Village with the following terms: 'artificial intelligence', 'prostate cancer', and 'PSMA PET'. All articles published up to February 2024 were considered. Studies were included if patients underwent PSMA PET scan to evaluate intraprostatic lesions prior to active treatment. The two authors independently evaluated titles, abstracts, and full text. The Prediction model Risk Of Bias Assessment Tool (PROBAST) was used.

RESULTS

Our search yield 948 articles, of which 14 were eligible for inclusion. Eight studies met the primary endpoint of differentiating high-grade PCa. Differentiating between International Society of Urological Pathology (ISUP) Grade Group (GG) ≥3 PCa had an accuracy between 0.671 to 0.992, sensitivity of 0.91, specificity of 0.35. Differentiating ISUP GG ≥4 PCa had an accuracy between 0.83 and 0.88, sensitivity was 0.89, specificity was 0.87. AI could identify non-PSMA-avid lesions with an accuracy of 0.87, specificity of 0.85, and specificity of 0.89. Three studies demonstrated ability of AI to detect extraprostatic extensions with an area under curve between 0.70 and 0.77. Lastly, AI can automate segmentation of intraprostatic lesion and measurement of gross tumour volume.

CONCLUSION

Although the current state of AI differentiating high-grade PCa is promising, it remains experimental and not ready for routine clinical application. Benefits of using AI to assess intraprostatic lesions on PSMA PET scans include: local staging, identifying otherwise radiologically occult lesions, standardisation and expedite reporting of PSMA PET scans. Larger, prospective, multicentre studies are needed.

Deep learning ensembles for detecting brain metastases in longitudinal multi-modal MRI studies

Metastatic brain cancer is a condition characterized by the migration of cancer cells to the brain from extracranial sites. Notably, metastatic brain tumors surpass primary brain tumors in prevalence by a significant factor, they exhibit an aggressive growth potential and have the capacity to spread across diverse cerebral locations simultaneously. Magnetic resonance imaging (MRI) scans of individuals afflicted with metastatic brain tumors unveil a wide spectrum of characteristics. These lesions vary in size and quantity, spanning from tiny nodules to substantial masses captured within MRI. Patients may present with a limited number of lesions or an extensive burden of hundreds of them. Moreover, longitudinal studies may depict surgical resection cavities, as well as areas of necrosis or edema. Thus, the manual analysis of such MRI scans is difficult, user-dependent and cost-inefficient, and - importantly - it lacks reproducibility. We address these challenges and propose a pipeline for detecting and analyzing brain metastases in longitudinal studies, which benefits from an ensemble of various deep learning architectures originally designed for different downstream tasks (detection and segmentation). The experiments, performed over 275 multi-modal MRI scans of 87 patients acquired in 53 sites, coupled with rigorously validated manual annotations, revealed that our pipeline, built upon open-source tools to ensure its reproducibility, offers high-quality detection, and allows for precisely tracking the disease progression. To objectively quantify the generalizability of models, we introduce a new data stratification approach that accommodates the heterogeneity of the dataset and is used to elaborate training-test splits in a data-robust manner, alongside a new set of quality metrics to objectively assess algorithms. Our system provides a fully automatic and quantitative approach that may support physicians in a laborious process of disease progression tracking and evaluation of treatment efficacy.

A systematic review of deep learning-based spinal bone lesion detection in medical images

Spinal bone lesions encompass a wide array of pathologies, spanning from benign abnormalities to aggressive malignancies, such as diffusely localized metastases. Early detection and accurate differentiation of the underlying diseases is crucial for every patient's clinical treatment and outcome, with radiological imaging being a core element in the diagnostic pathway. Across numerous pathologies and imaging techniques, deep learning (DL) models are progressively considered a valuable resource in the clinical setting. This review describes not only the diagnostic performance of these models and the differing approaches in the field of spinal bone malignancy recognition, but also the lack of standardized methodology and reporting that we believe is currently hampering this newly founded area of research. In line with their established and reliable role in lesion detection, this publication focuses on both computed tomography and magnetic resonance imaging, as well as various derivative modalities (i.e. SPECT). After conducting a systematic literature search and subsequent analysis for applicability and quality using a modified QUADAS-2 scoring system, we confirmed that most of the 14 identified studies were plagued by major limitations, such as insufficient reporting of model statistics and data acquisition, a lacking external validation dataset, and potentially biased annotation. Although we experienced these limitations, we nonetheless conclude that the potential of these methods shines through in the presented results. These findings underline the need for more stringent quality controls in DL studies, as well as model development to afford increased insight and progress in this promising novel field.

A 3D Convolutional Neural Network Based on Non-enhanced Brain CT to Identify Patients with Brain Metastases

Dedicated brain imaging for cancer patients is seldom recommended in the absence of symptoms. There is increasing availability of non-enhanced CT (NE-CT) of the brain, mainly owing to a wider utilization of Positron Emission Tomography-CT (PET-CT) in cancer staging. Brain metastases (BM) are often hard to diagnose on NE-CT. This work aims to develop a 3D Convolutional Neural Network (3D-CNN) based on brain NE-CT to distinguish patients with and without BM. We retrospectively included NE-CT scans for 100 patients with single or multiple BM and 100 patients without brain imaging abnormalities. Patients whose largest lesion was < 5 mm were excluded. The largest tumor was manually segmented on a matched contrast-enhanced T1 weighted Magnetic Resonance Imaging (MRI), and shape radiomics were extracted to determine the size and volume of the lesion. The brain was automatically segmented, and masked images were normalized and resampled. The dataset was split into training (70%) and validation (30%) sets. Multiple versions of a 3D-CNN were developed, and the best model was selected based on accuracy (ACC) on the validation set. The median largest tumor Maximum-3D-Diameter was 2.29 cm, and its median volume was 2.81 cc. Solitary BM were found in 27% of the patients, while 49% had > 5 BMs. The best model consisted of 4 convolutional layers with 3D average pooling layers, dropout layers of 50%, and a sigmoid activation function. Mean validation ACC was 0.983 (SD: 0.020) and mean area under receiver-operating characteristic curve was 0.983 (SD: 0.023). Sensitivity was 0.983 (SD: 0.020). We developed an accurate 3D-CNN based on brain NE-CT to differentiate between patients with and without BM. The model merits further external validation.

Imaging Recommendations for Diagnosis, Staging, and Management of Central Nervous System Neoplasms in Adults: CNS Metastases

Brain metastases (BMs) are the most common central nervous system (CNS) neoplasms, with an increasing incidence that is due in part to an overall increase in primary cancers, improved neuroimaging modalities leading to increased detection, better systemic therapies, and longer patient survival.

OBJECTIVE

To identify cancer patients at a higher risk of developing CNS metastases and to evaluate associated prognostic factors.

METHODS

Review of imaging referral guidelines, response criteria, interval imaging assessment, modality of choice, as well as the association of clinical, serological, and imaging findings as per various cancer societies.

RESULTS

Quantitative response assessment of target and non-target brain metastases as well as an interval imaging protocol set up based on primary histological diagnosis and therapy status are discussed as per various cancer societies and imaging programs.

CONCLUSION

Predictive factors in the primary tumor as well as independent variables of brain metastases like size, number, and response to therapy are necessary in management. The location of CNS metastases, symptomatic disease, as well as follow up imaging findings form a skeletal plan to prognosticate the disease, keeping in mind all the available new advanced therapy options of surgery, radiation, and immunotherapy that improve patient outcome significantly.

Role of artificial intelligence in brain tumour imaging

Artificial intelligence (AI) is a rapidly evolving field with many neuro-oncology applications. In this review, we discuss how AI can assist in brain tumour imaging, focusing on machine learning (ML) and deep learning (DL) techniques. We describe how AI can help in lesion detection, differential diagnosis, anatomic segmentation, molecular marker identification, prognostication, and pseudo-progression evaluation. We also cover AI applications in non-glioma brain tumours, such as brain metastasis, posterior fossa, and pituitary tumours. We highlight the challenges and limitations of AI implementation in radiology, such as data quality, standardization, and integration. Based on the findings in the aforementioned areas, we conclude that AI can potentially improve the diagnosis and treatment of brain tumours and provide a path towards personalized medicine and better patient outcomes.

The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.

Prediction of treatment response after stereotactic radiosurgery of brain metastasis using deep learning and radiomics on longitudinal MRI data

We developed artificial intelligence models to predict the brain metastasis (BM) treatment response after stereotactic radiosurgery (SRS) using longitudinal magnetic resonance imaging (MRI) data and evaluated prediction accuracy changes according to the number of sequential MRI scans. We included four sequential MRI scans for 194 patients with BM and 369 target lesions for the Developmental dataset. The data were randomly split (8:2 ratio) for training and testing. For external validation, 172 MRI scans from 43 patients with BM and 62 target lesions were additionally enrolled. The maximum axial diameter (Dmax), radiomics, and deep learning (DL) models were generated for comparison. We evaluated the simple convolutional neural network (CNN) model and a gated recurrent unit (Conv-GRU)-based CNN model in the DL arm. The Conv-GRU model performed superior to the simple CNN models. For both datasets, the area under the curve (AUC) was significantly higher for the two-dimensional (2D) Conv-GRU model than for the 3D Conv-GRU, Dmax, and radiomics models. The accuracy of the 2D Conv-GRU model increased with the number of follow-up studies. In conclusion, using longitudinal MRI data, the 2D Conv-GRU model outperformed all other models in predicting the treatment response after SRS of BM.

Advances in determining the gross tumor target volume for radiotherapy of brain metastases

Brain metastases (BMs) are the most prevalent intracranial malignant tumors in adults and are the leading cause of mortality attributed to malignant brain diseases. Radiotherapy (RT) plays a critical role in the treatment of BMs, with local RT techniques such as stereotactic radiosurgery (SRS)/stereotactic body radiotherapy (SBRT) showing remarkable therapeutic effectiveness. The precise determination of gross tumor target volume (GTV) is crucial for ensuring the effectiveness of SRS/SBRT. Multimodal imaging techniques such as CT, MRI, and PET are extensively used for the diagnosis of BMs and GTV determination. With the development of functional imaging and artificial intelligence (AI) technology, there are more innovative ways to determine GTV for BMs, which significantly improve the accuracy and efficiency of the determination. This article provides an overview of the progress in GTV determination for RT in BMs.

Where Does Auto-Segmentation for Brain Metastases Radiosurgery Stand Today?

Detection and segmentation of brain metastases (BMs) play a pivotal role in diagnosis, treatment planning, and follow-up evaluations for effective BM management. Given the rising prevalence of BM cases and its predominantly multiple onsets, automated segmentation is becoming necessary in stereotactic radiosurgery. It not only alleviates the clinician's manual workload and improves clinical workflow efficiency but also ensures treatment safety, ultimately improving patient care. Recent strides in machine learning, particularly in deep learning (DL), have revolutionized medical image segmentation, achieving state-of-the-art results. This review aims to analyze auto-segmentation strategies, characterize the utilized data, and assess the performance of cutting-edge BM segmentation methodologies. Additionally, we delve into the challenges confronting BM segmentation and share insights gleaned from our algorithmic and clinical implementation experiences.

Reducing false positives in deep learning-based brain metastasis detection by using both gradient-echo and spin-echo contrast-enhanced MRI: validation in a multi-center diagnostic cohort

OBJECTIVES

To develop a deep learning (DL) for detection of brain metastasis (BM) that incorporates both gradient- and turbo spin-echo contrast-enhanced MRI (dual-enhanced DL) and evaluate it in a clinical cohort in comparison with human readers and DL using gradient-echo-based imaging only (GRE DL).

MATERIALS AND METHODS

DL detection was developed using data from 200 patients with BM (training set) and tested in 62 (internal) and 48 (external) consecutive patients who underwent stereotactic radiosurgery and diagnostic dual-enhanced imaging (dual-enhanced DL) and later guide GRE imaging (GRE DL). The detection sensitivity and positive predictive value (PPV) were compared between two DLs. Two neuroradiologists independently analyzed BM and reference standards for BM were separately drawn by another neuroradiologist. The relative differences (RDs) from the reference standard BM numbers were compared between the DLs and neuroradiologists.

RESULTS

Sensitivity was similar between GRE DL (93%, 95% confidence interval [CI]: 90-96%) and dual-enhanced DL (92% [89-94%]). The PPV of the dual-enhanced DL was higher (89% [86-92%], p < .001) than that of GRE DL (76%, [72-80%]). GRE DL significantly overestimated the number of metastases (false positives; RD: 0.05, 95% CI: 0.00-0.58) compared with neuroradiologists (RD: 0.00, 95% CI: - 0.28, 0.15, p < .001), whereas dual-enhanced DL (RD: 0.00, 95% CI: 0.00-0.15) did not show a statistically significant difference from neuroradiologists (RD: 0.00, 95% CI: - 0.20-0.10, p = .913).

CONCLUSION

The dual-enhanced DL showed improved detection of BM and reduced overestimation compared with GRE DL, achieving similar performance to neuroradiologists.

CLINICAL RELEVANCE STATEMENT

The use of deep learning-based brain metastasis detection with turbo spin-echo imaging reduces false positive detections, aiding in the guidance of stereotactic radiosurgery when gradient-echo imaging alone is employed.

KEY POINTS

•Deep learning for brain metastasis detection improved by using both gradient- and turbo spin-echo contrast-enhanced MRI (dual-enhanced deep learning). •Dual-enhanced deep learning increased true positive detections and reduced overestimation. •Dual-enhanced deep learning achieved similar performance to neuroradiologists for brain metastasis counts.

Deep learning-based metastasis detection in patients with lung cancer to enhance reproducibility and reduce workload in brain metastasis screening with MRI: a multi-center study

OBJECTIVES

To assess whether a deep learning-based system (DLS) with black-blood imaging for brain metastasis (BM) improves the diagnostic workflow in a multi-center setting.

MATERIALS AND METHODS

In this retrospective study, a DLS was developed in 101 patients and validated on 264 consecutive patients (with lung cancer) having newly developed BM from two tertiary university hospitals, which performed black-blood imaging between January 2020 and April 2021. Four neuroradiologists independently evaluated BM either with segmented masks and BM counts provided (with DLS) or not provided (without DLS) on a clinical trial imaging management system (CTIMS). To assess reading reproducibility, BM count agreement between the readers and the reference standard were calculated using limits of agreement (LoA). Readers' workload was assessed with reading time, which was automatically measured on CTIMS, and were compared between with and without DLS using linear mixed models considering the imaging center.

RESULTS

In the validation cohort, the detection sensitivity and positive predictive value of the DLS were 90.2% (95% confidence interval [CI]: 88.1-92.2) and 88.2% (95% CI: 85.7-90.4), respectively. The difference between the readers and the reference counts was larger without DLS (LoA: -0.281, 95% CI: -2.888, 2.325) than with DLS (LoA: -0.163, 95% CI: -2.692, 2.367). The reading time was reduced from mean 66.9 s (interquartile range: 43.2-90.6) to 57.3 s (interquartile range: 33.6-81.0) (P <.001) in the with DLS group, regardless of the imaging center.

CONCLUSION

Deep learning-based BM detection and counting with black-blood imaging improved reproducibility and reduced reading time, on multi-center validation.

A large open access dataset of brain metastasis 3D segmentations on MRI with clinical and imaging information

Resection and whole brain radiotherapy (WBRT) are standard treatments for brain metastases (BM) but are associated with cognitive side effects. Stereotactic radiosurgery (SRS) uses a targeted approach with less side effects than WBRT. SRS requires precise identification and delineation of BM. While artificial intelligence (AI) algorithms have been developed for this, their clinical adoption is limited due to poor model performance in the clinical setting. The limitations of algorithms are often due to the quality of datasets used for training the AI network. The purpose of this study was to create a large, heterogenous, annotated BM dataset for training and validation of AI models. We present a BM dataset of 200 patients with pretreatment T1, T1 post-contrast, T2, and FLAIR MR images. The dataset includes contrast-enhancing and necrotic 3D segmentations on T1 post-contrast and peritumoral edema 3D segmentations on FLAIR. Our dataset contains 975 contrast-enhancing lesions, many of which are sub centimeter, along with clinical and imaging information. We used a streamlined approach to database-building through a PACS-integrated segmentation workflow.

Predicting EGFR Mutation Status in Non-Small Cell Lung Cancer Using Artificial Intelligence: A Systematic Review and Meta-Analysis

RATIONALE AND OBJECTIVES

Recent advancements in artificial intelligence (AI) render a substantial promise for epidermal growth factor receptor (EGFR) mutation status prediction in non-small cell lung cancer (NSCLC). We aimed to evaluate the performance and quality of AI algorithms that use radiomics features in predicting EGFR mutation status in patient with NSCLC.

MATERIALS AND METHODS

We searched PubMed (Medline), EMBASE, Web of Science, and IEEExplore for studies published up to February 28, 2022. Studies utilizing an AI algorithm (either conventional machine learning [cML] and deep learning [DL]) for predicting EGFR mutations in patients with NSLCL were included. We extracted binary diagnostic accuracy data and constructed a bivariate random-effects model to obtain pooled sensitivity, specificity, and 95% confidence interval. This study is registered with PROSPERO, CRD42021278738.

RESULTS

Our search identified 460 studies, of which 42 were included. Thirty-five studies were included in the meta-analysis. The AI algorithms exhibited an overall area under the curve (AUC) value of 0.789 and pooled sensitivity and specificity levels of 72.2% and 73.3%, respectively. The DL algorithms outperformed cML in terms of AUC (0.822 vs. 0.775) and sensitivity (80.1% vs. 71.1%), but had lower specificity (70.0% vs. 73.8%, p-value < 0.001) compared to cML. Subgroup analysis revealed that the use of positron-emission tomography/computed tomography, additional clinical information, deep feature extraction, and manual segmentation can improve diagnostic performance.

CONCLUSION

DL algorithms can serve as a novel method for increasing predictive accuracy and thus have considerable potential for use in predicting EGFR mutation status in patient with NSCLC. We also suggest that guidelines on using AI algorithms in medical image analysis should be developed with a focus on oncologic radiomics.

Brain metastasis tumor segmentation and detection using deep learning algorithms: A systematic review and meta-analysis

BACKGROUND

Manual detection of brain metastases is both laborious and inconsistent, driving the need for more efficient solutions. Accordingly, our systematic review and meta-analysis assessed the efficacy of deep learning algorithms in detecting and segmenting brain metastases from various primary origins in MRI images.

METHODS

We conducted a comprehensive search of PubMed, Embase, and Web of Science up to May 24, 2023, which yielded 42 relevant studies for our analysis. We assessed the quality of these studies using the QUADAS-2 and CLAIM tools. Using a random-effect model, we calculated the pooled lesion-wise dice score as well as patient-wise and lesion-wise sensitivity. We performed subgroup analyses to investigate the influence of factors such as publication year, study design, training center of the model, validation methods, slice thickness, model input dimensions, MRI sequences fed to the model, and the specific deep learning algorithms employed. Additionally, meta-regression analyses were carried out considering the number of patients in the studies, count of MRI manufacturers, count of MRI models, training sample size, and lesion number.

RESULTS

Our analysis highlighted that deep learning models, particularly the U-Net and its variants, demonstrated superior segmentation accuracy. Enhanced detection sensitivity was observed with an increased diversity in MRI hardware, both in terms of manufacturer and model variety. Furthermore, slice thickness was identified as a significant factor influencing lesion-wise detection sensitivity. Overall, the pooled results indicated a lesion-wise dice score of 79%, with patient-wise and lesion-wise sensitivities at 86% and 87%, respectively.

CONCLUSIONS

The study underscores the potential of deep learning in improving brain metastasis diagnostics and treatment planning. Still, more extensive cohorts and larger meta-analysis are needed for more practical and generalizable algorithms. Future research should prioritize these areas to advance the field. This study was funded by the Gen. & Mrs. M.C. Peng Fellowship and registered under PROSPERO (CRD42023427776).

Systematic reviews of machine learning in healthcare: a literature review

INTRODUCTION

The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery.

METHODS

A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted.

RESULTS

In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively).

EXPERT OPINION

The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.

The value of machine learning in preoperative identification of lymph node metastasis status in endometrial cancer: a systematic review and meta-analysis

Background

The early identification of lymph node metastasis status in endometrial cancer (EC) is a serious challenge in clinical practice. Some investigators have introduced machine learning into the early identification of lymph node metastasis in EC patients. However, the predictive value of machine learning is controversial due to the diversity of models and modeling variables. To this end, we carried out this systematic review and meta-analysis to systematically discuss the value of machine learning for the early identification of lymph node metastasis in EC patients.

Methods

A systematic search was conducted in Pubmed, Cochrane, Embase, and Web of Science until March 12, 2023. PROBAST was used to assess the risk of bias in the included studies. In the process of meta-analysis, subgroup analysis was performed according to modeling variables (clinical features, radiomic features, and radiomic features combined with clinical features) and different types of models in various variables.

Results

This systematic review included 50 primary studies with a total of 103,752 EC patients, 12,579 of whom had positive lymph node metastasis. Meta-analysis showed that among the machine learning models constructed by the three categories of modeling variables, the best model was constructed by combining radiomic features with clinical features, with a pooled c-index of 0.907 (95%CI: 0.886-0.928) in the training set and 0.823 (95%CI: 0.757-0.890) in the validation set, and good sensitivity and specificity. The c-index of the machine learning model constructed based on clinical features alone was not inferior to that based on radiomic features only. In addition, logistic regression was found to be the main modeling method and has ideal predictive performance with different categories of modeling variables.

Conclusion

Although the model based on radiomic features combined with clinical features has the best predictive efficiency, there is no recognized specification for the application of radiomics at present. In addition, the logistic regression constructed by clinical features shows good sensitivity and specificity. In this context, large-sample studies covering different races are warranted to develop predictive nomograms based on clinical features, which can be widely applied in clinical practice.

Systematic review registration

https://www.crd.york.ac.uk/PROSPERO, identifier CRD42023420774.

Development and Evaluation of Machine Learning in Whole-Body Magnetic Resonance Imaging for Detecting Metastases in Patients With Lung or Colon Cancer: A Diagnostic Test Accuracy Study

OBJECTIVES

Whole-body magnetic resonance imaging (WB-MRI) has been demonstrated to be efficient and cost-effective for cancer staging. The study aim was to develop a machine learning (ML) algorithm to improve radiologists' sensitivity and specificity for metastasis detection and reduce reading times.

MATERIALS AND METHODS

A retrospective analysis of 438 prospectively collected WB-MRI scans from multicenter Streamline studies (February 2013-September 2016) was undertaken. Disease sites were manually labeled using Streamline reference standard. Whole-body MRI scans were randomly allocated to training and testing sets. A model for malignant lesion detection was developed based on convolutional neural networks and a 2-stage training strategy. The final algorithm generated lesion probability heat maps. Using a concurrent reader paradigm, 25 radiologists (18 experienced, 7 inexperienced in WB-/MRI) were randomly allocated WB-MRI scans with or without ML support to detect malignant lesions over 2 or 3 reading rounds. Reads were undertaken in the setting of a diagnostic radiology reading room between November 2019 and March 2020. Reading times were recorded by a scribe. Prespecified analysis included sensitivity, specificity, interobserver agreement, and reading time of radiology readers to detect metastases with or without ML support. Reader performance for detection of the primary tumor was also evaluated.

RESULTS

Four hundred thirty-three evaluable WB-MRI scans were allocated to algorithm training (245) or radiology testing (50 patients with metastases, from primary 117 colon [n = 117] or lung [n = 71] cancer). Among a total 562 reads by experienced radiologists over 2 reading rounds, per-patient specificity was 86.2% (ML) and 87.7% (non-ML) (-1.5% difference; 95% confidence interval [CI], -6.4%, 3.5%; P = 0.39). Sensitivity was 66.0% (ML) and 70.0% (non-ML) (-4.0% difference; 95% CI, -13.5%, 5.5%; P = 0.344). Among 161 reads by inexperienced readers, per-patient specificity in both groups was 76.3% (0% difference; 95% CI, -15.0%, 15.0%; P = 0.613), with sensitivity of 73.3% (ML) and 60.0% (non-ML) (13.3% difference; 95% CI, -7.9%, 34.5%; P = 0.313). Per-site specificity was high (>90%) for all metastatic sites and experience levels. There was high sensitivity for the detection of primary tumors (lung cancer detection rate of 98.6% with and without ML [0.0% difference; 95% CI, -2.0%, 2.0%; P = 1.00], colon cancer detection rate of 89.0% with and 90.6% without ML [-1.7% difference; 95% CI, -5.6%, 2.2%; P = 0.65]). When combining all reads from rounds 1 and 2, reading times fell by 6.2% (95% CI, -22.8%, 10.0%) when using ML. Round 2 read-times fell by 32% (95% CI, 20.8%, 42.8%) compared with round 1. Within round 2, there was a significant decrease in read-time when using ML support, estimated as 286 seconds (or 11%) quicker ( P = 0.0281), using regression analysis to account for reader experience, read round, and tumor type. Interobserver variance suggests moderate agreement, Cohen κ = 0.64; 95% CI, 0.47, 0.81 (with ML), and Cohen κ = 0.66; 95% CI, 0.47, 0.81 (without ML).

CONCLUSIONS

There was no evidence of a significant difference in per-patient sensitivity and specificity for detecting metastases or the primary tumor using concurrent ML compared with standard WB-MRI. Radiology read-times with or without ML support fell for round 2 reads compared with round 1, suggesting that readers familiarized themselves with the study reading method. During the second reading round, there was a significant reduction in reading time when using ML support.

Can machine learning models improve early detection of brain metastases using diffusion weighted imaging-based radiomics?

Background

Metastatic complications are a major cause of cancer-related morbidity, with up to 40% of cancer patients experiencing at least one brain metastasis. Earlier detection may significantly improve patient outcomes and overall survival. We investigated machine learning (ML) models for early detection of brain metastases based on diffusion weighted imaging (DWI) radiomics.

Methods

Longitudinal diffusion imaging from 116 patients previously treated with stereotactic radiosurgery (SRS) for brain metastases were retrospectively analyzed. Clinical contours from 600 metastases were extracted from radiosurgery planning computed tomography, and rigidly registered to corresponding contrast enhanced-T1 and apparent diffusion coefficient (ADC) maps. Contralateral contours located in healthy brain tissue were used as control. The dataset consisted of (I) radiomic features using ADC maps, (II) radiomic feature change calculated using timepoints before the metastasis manifested on contrast enhanced-T1, (III) primary cancer, and (IV) anatomical location. The dataset was divided into training and internal validation sets using an 80/20 split with stratification. Four classification algorithms [Linear Support Vector Machine (SVM), Random Forest (RF), AdaBoost, and XGBoost] underwent supervised classification training, with contours labeled either 'control' or 'metastasis'. Hyperparameters were optimized towards balanced accuracy. Various model metrics (receiver operating characteristic curve area scores, accuracy, recall, and precision) were calculated to gauge performance.

Results

The radiomic and clinical data set, feature engineering, and ML models developed were able to identify metastases with an accuracy of up to 87.7% on the training set, and 85.8% on an unseen test set. XGBoost and RF showed superior accuracy (XGBoost: 0.877±0.021 and 0.833±0.47, RF: 0.823±0.024 and 0.858±0.045) for training and validation sets, respectively. XGBoost and RF also showed strong area under the receiver operating characteristic curve (AUC) performance on the validation set (0.910±0.037 and 0.922±0.034, respectively). AdaBoost performed slightly lower in all metrics. SVM model generalized poorly with the internal validation set. Important features involved changes in radiomics months before manifesting on contrast enhanced-T1.

Conclusions

The proposed models using diffusion-based radiomics showed encouraging results in differentiating healthy brain tissue from metastases using clinical imaging data. These findings suggest that longitudinal diffusion imaging and ML may help improve patient care through earlier diagnosis and increased patient monitoring/follow-up. Future work aims to improve model classification metrics, robustness, user-interface, and clinical applicability.

Feature pyramid network-based computer-aided detection and monitoring treatment response of brain metastases on contrast-enhanced MRI

AIM

To investigate the value of feature pyramid network (FPN)-based computer-aided detection (CAD) of brain metastases (BMs) before and after non-surgical treatment, and to evaluate its performance in monitoring treatment response of BM on contrast-enhanced (CE) magnetic resonance imaging (MRI).

MATERIAL AND METHODS

Eighty-five cancer patients newly diagnosed with BM who had undergone initial and follow-up three-dimensional (3D) CE MRI at Liaocheng People's Hospital were included retrospectively in this study. Manual detection (MD) was performed by reviewer 1. Computer-aided detection (CAD) was performed by reviewer 2 using uAI Discover-BMs software. The treatment response was assessed by the two reviewers for each patient separately. A paired chi-square test was used to compare the differences in the detection of BM between MD and CAD. Agreement between MD and CAD in monitoring treatment response was assessed by kappa test.

RESULTS

The sensitivities of MD and CAD on initial 3D CE MRI were 78.65% and 99.13%, respectively. The sensitivities of MD and CAD on follow-up 3D CE MRI were 76.32% and 98.24%, respectively. There was a very good agreement between Reviewer 1 and Reviewer 2 in evaluating the treatment response of BM.

CONCLUSION

FPN-based CAD has a higher sensitivity of close to 100% and lower false negatives (FNs) for BM detection, compared to MD. Although CAD had a few shortcomings in reflecting changes of BMs after treatment, it had high performance in monitoring treatment response of BM on CE MRI.

Construction and evaluation of a gated high-resolution neural network for automatic brain metastasis detection and segmentation

OBJECTIVES

To construct and evaluate a gated high-resolution convolutional neural network for detecting and segmenting brain metastasis (BM).

METHODS

This retrospective study included craniocerebral MRI scans of 1392 patients with 14,542 BMs and 200 patients with no BM between January 2012 and April 2022. A primary dataset including 1000 cases with 11,686 BMs was employed to construct the model, while an independent dataset including 100 cases with 1069 BMs from other hospitals was used to examine the generalizability. The potential of the model for clinical use was also evaluated by comparing its performance in BM detection and segmentation to that of radiologists, and comparing radiologists' lesion detecting performances with and without model assistance.

RESULTS

Our model yielded a recall of 0.88, a dice similarity coefficient (DSC) of 0.90, a positive predictive value (PPV) of 0.93 and a false positives per patient (FP) of 1.01 in the test set, and a recall of 0.85, a DSC of 0.89, a PPV of 0.93, and a FP of 1.07 in dataset from other hospitals. With the model's assistance, the BM detection rates of 4 radiologists improved significantly, ranging from 5.2 to 15.1% (all p < 0.001), and also for detecting small BMs with diameter ≤ 5 mm (ranging from 7.2 to 27.0%, all p < 0.001).

CONCLUSIONS

The proposed model enables accurate BM detection and segmentation with higher sensitivity and less time consumption, showing the potential to augment radiologists' performance in detecting BM.

CLINICAL RELEVANCE STATEMENT

This study offers a promising computer-aided tool to assist the brain metastasis detection and segmentation in routine clinical practice for cancer patients.

KEY POINTS

• The GHR-CNN could accurately detect and segment BM on contrast-enhanced 3D-T1W images. • The GHR-CNN improved the BM detection rate of radiologists, including the detection of small lesions. • The GHR-CNN enabled automated segmentation of BM in a very short time.

Application of an artificial intelligence-based tool in [18F]FDG PET/CT for the assessment of bone marrow involvement in multiple myeloma

PURPOSE

[18F]FDG PET/CT is an imaging modality of high performance in multiple myeloma (MM). Nevertheless, the inter-observer reproducibility in PET/CT scan interpretation may be hampered by the different patterns of bone marrow (BM) infiltration in the disease. Although many approaches have been recently developed to address the issue of standardization, none can yet be considered a standard method in the interpretation of PET/CT. We herein aim to validate a novel three-dimensional deep learning-based tool on PET/CT images for automated assessment of the intensity of BM metabolism in MM patients.

MATERIALS AND METHODS

Whole-body [18F]FDG PET/CT scans of 35 consecutive, previously untreated MM patients were studied. All patients were investigated in the context of an open-label, multicenter, randomized, active-controlled, phase 3 trial (GMMG-HD7). Qualitative (visual) analysis classified the PET/CT scans into three groups based on the presence and number of focal [18F]FDG-avid lesions as well as the degree of diffuse [18F]FDG uptake in the BM. The proposed automated method for BM metabolism assessment is based on an initial CT-based segmentation of the skeleton, its transfer to the SUV PET images, the subsequent application of different SUV thresholds, and refinement of the resulting regions using postprocessing. In the present analysis, six different SUV thresholds (Approaches 1-6) were applied for the definition of pathological tracer uptake in the skeleton [Approach 1: liver SUVmedian × 1.1 (axial skeleton), gluteal muscles SUVmedian × 4 (extremities). Approach 2: liver SUVmedian × 1.5 (axial skeleton), gluteal muscles SUVmedian × 4 (extremities). Approach 3: liver SUVmedian × 2 (axial skeleton), gluteal muscles SUVmedian × 4 (extremities). Approach 4: ≥ 2.5. Approach 5: ≥ 2.5 (axial skeleton), ≥ 2.0 (extremities). Approach 6: SUVmax liver]. Using the resulting masks, subsequent calculations of the whole-body metabolic tumor volume (MTV) and total lesion glycolysis (TLG) in each patient were performed. A correlation analysis was performed between the automated PET values and the results of the visual PET/CT analysis as well as the histopathological, cytogenetical, and clinical data of the patients.

RESULTS

BM segmentation and calculation of MTV and TLG after the application of the deep learning tool were feasible in all patients. A significant positive correlation (p < 0.05) was observed between the results of the visual analysis of the PET/CT scans for the three patient groups and the MTV and TLG values after the employment of all six [18F]FDG uptake thresholds. In addition, there were significant differences between the three patient groups with regard to their MTV and TLG values for all applied thresholds of pathological tracer uptake. Furthermore, we could demonstrate a significant, moderate, positive correlation of BM plasma cell infiltration and plasma levels of β2-microglobulin with the automated quantitative PET/CT parameters MTV and TLG after utilization of Approaches 1, 2, 4, and 5.

CONCLUSIONS

The automated, volumetric, whole-body PET/CT assessment of the BM metabolic activity in MM is feasible with the herein applied method and correlates with clinically relevant parameters in the disease. This methodology offers a potentially reliable tool in the direction of optimization and standardization of PET/CT interpretation in MM. Based on the present promising findings, the deep learning-based approach will be further evaluated in future prospective studies with larger patient cohorts.

A Large Open Access Dataset of Brain Metastasis 3D Segmentations with Clinical and Imaging Feature Information

Resection and whole brain radiotherapy (WBRT) are the standards of care for the treatment of patients with brain metastases (BM) but are often associated with cognitive side effects. Stereotactic radiosurgery (SRS) involves a more targeted treatment approach and has been shown to avoid the side effects associated with WBRT. However, SRS requires precise identification and delineation of BM. While many AI algorithms have been developed for this purpose, their clinical adoption has been limited due to poor model performance in the clinical setting. Major reasons for non-generalizable algorithms are the limitations in the datasets used for training the AI network. The purpose of this study was to create a large, heterogenous, annotated BM dataset for training and validation of AI models to improve generalizability. We present a BM dataset of 200 patients with pretreatment T1, T1 post-contrast, T2, and FLAIR MR images. The dataset includes contrast-enhancing and necrotic 3D segmentations on T1 post-contrast and whole tumor (including peritumoral edema) 3D segmentations on FLAIR. Our dataset contains 975 contrast-enhancing lesions, many of which are sub centimeter, along with clinical and imaging feature information. We used a streamlined approach to database-building leveraging a PACS-integrated segmentation workflow.

Artificial Intelligence in Neuroradiology: A Review of Current Topics and Competition Challenges

There is an expanding body of literature that describes the application of deep learning and other machine learning and artificial intelligence methods with potential relevance to neuroradiology practice. In this article, we performed a literature review to identify recent developments on the topics of artificial intelligence in neuroradiology, with particular emphasis on large datasets and large-scale algorithm assessments, such as those used in imaging AI competition challenges. Numerous applications relevant to ischemic stroke, intracranial hemorrhage, brain tumors, demyelinating disease, and neurodegenerative/neurocognitive disorders were discussed. The potential applications of these methods to spinal fractures, scoliosis grading, head and neck oncology, and vascular imaging were also reviewed. The AI applications examined perform a variety of tasks, including localization, segmentation, longitudinal monitoring, diagnostic classification, and prognostication. While research on this topic is ongoing, several applications have been cleared for clinical use and have the potential to augment the accuracy or efficiency of neuroradiologists.

Predicting leptomeningeal disease spread after resection of brain metastases using machine learning

OBJECTIVE

The incidence of leptomeningeal disease (LMD) has increased as treatments for brain metastases (BMs) have improved and patients with metastatic disease are living longer. Sample sizes of individual studies investigating LMD after surgery for BMs and its risk factors have been limited, ranging from 200 to 400 patients at risk for LMD, which only allows the use of conventional biostatistics. Here, the authors used machine learning techniques to enhance LMD prediction in a cohort of surgically treated BMs.

METHODS

A conditional survival forest, a Cox proportional hazards model, an extreme gradient boosting (XGBoost) classifier, an extra trees classifier, and logistic regression were trained. A synthetic minority oversampling technique (SMOTE) was used to train the models and handle the inherent class imbalance. Patients were divided into an 80:20 training and test set. Fivefold cross-validation was used on the training set for hyperparameter optimization. Patients eligible for study inclusion were adults who had consecutively undergone neurosurgical BM treatment, had been admitted to Brigham and Women's Hospital from January 2007 through December 2019, and had a minimum of 1 month of follow-up after neurosurgical treatment.

RESULTS

A total of 1054 surgically treated BM patients were included in this analysis. LMD occurred in 168 patients (15.9%) at a median of 7.05 months after BM diagnosis. The discrimination of LMD occurrence was optimal using an XGboost algorithm (area under the curve = 0.83), and the time to LMD was prognosticated evenly by the random forest algorithm and the Cox proportional hazards model (C-index = 0.76). The most important feature for both LMD classification and regression was the BM proximity to the CSF space, followed by a cerebellar BM location. Lymph node metastasis of the primary tumor at BM diagnosis and a cerebellar BM location were the strongest risk factors for both LMD occurrence and time to LMD.

CONCLUSIONS

The outcomes of LMD patients in the BM population are predictable using SMOTE and machine learning. Lymph node metastasis of the primary tumor at BM diagnosis and a cerebellar BM location were the strongest LMD risk factors.

An online survival predictor in glioma patients using machine learning based on WHO CNS5 data

Background

The World Health Organization (WHO) CNS5 classification system highlights the significance of molecular biomarkers in providing meaningful prognostic and therapeutic information for gliomas. However, predicting individual patient survival remains challenging due to the lack of integrated quantitative assessment tools. In this study, we aimed to design a WHO CNS5-related risk signature to predict the overall survival (OS) rate of glioma patients using machine learning algorithms.

Methods

We extracted data from patients who underwent an operation for histopathologically confirmed glioma from our hospital database (2011-2022) and split them into a training and hold-out test set in a 7/3 ratio. We used biological markers related to WHO CNS5, clinical data (age, sex, and WHO grade), and prognosis follow-up information to identify prognostic factors and construct a predictive dynamic nomograph to predict the survival rate of glioma patients using 4 kinds machine learning algorithms (RF, SVM, XGB, and GLM).

Results

A total of 198 patients with complete WHO5 molecular data and follow-up information were included in the study. The median OS time of all patients was 29.77 [95% confidence interval (CI): 21.19-38.34] months. Age, FGFR2, IDH1, CDK4, CDK6, KIT, and CDKN2A were considered vital indicators related to the prognosis and OS time of glioma. To better predict the prognosis of glioma patients, we constructed a WHO5-related risk signature and nomogram. The AUC values of the ROC curves of the nomogram for predicting the 1, 3, and 5-year OS were 0.849, 0.835, and 0.821 in training set, and, 0.844, 0.943, and 0.959 in validation set. The calibration plot confirmed the reliability of the nomogram, and the c-index was 0.742 in training set and 0.775 in validation set. Additionally, our nomogram showed a superior net benefit across a broader scale of threshold probabilities in decision curve analysis. Therefore, we selected it as the backend for the online survival prediction tool (Glioma Survival Calculator, https://who5pumch.shinyapps.io/DynNomapp/), which can calculate the survival probability for a specific time of the patients.

Conclusion

An online prognosis predictor based on WHO5-related biomarkers was constructed. This therapeutically promising tool may increase the precision of forecast therapy outcomes and assess prognosis.

A comprehensive dataset of annotated brain metastasis MR images with clinical and radiomic data

Brain metastasis (BM) is one of the main complications of many cancers, and the most frequent malignancy of the central nervous system. Imaging studies of BMs are routinely used for diagnosis of disease, treatment planning and follow-up. Artificial Intelligence (AI) has great potential to provide automated tools to assist in the management of disease. However, AI methods require large datasets for training and validation, and to date there have been just one publicly available imaging dataset of 156 BMs. This paper publishes 637 high-resolution imaging studies of 75 patients harboring 260 BM lesions, and their respective clinical data. It also includes semi-automatic segmentations of 593 BMs, including pre- and post-treatment T1-weighted cases, and a set of morphological and radiomic features for the cases segmented. This data-sharing initiative is expected to enable research into and performance evaluation of automatic BM detection, lesion segmentation, disease status evaluation and treatment planning methods for BMs, as well as the development and validation of predictive and prognostic tools with clinical applicability.

Prediction of model generalizability for unseen data: Methodology and case study in brain metastases detection in T1-Weighted contrast-enhanced 3D MRI

BACKGROUND AND PURPOSE

A medical AI system's generalizability describes the continuity of its performance acquired from varying geographic, historical, and methodologic settings. Previous literature on this topic has mostly focused on "how" to achieve high generalizability (e.g., via larger datasets, transfer learning, data augmentation, model regularization schemes), with limited success. Instead, we aim to understand "when" the generalizability is achieved: Our study presents a medical AI system that could estimate its generalizability status for unseen data on-the-fly.

MATERIALS AND METHODS

We introduce a latent space mapping (LSM) approach utilizing Fréchet distance loss to force the underlying training data distribution into a multivariate normal distribution. During the deployment, a given test data's LSM distribution is processed to detect its deviation from the forced distribution; hence, the AI system could predict its generalizability status for any previously unseen data set. If low model generalizability is detected, then the user is informed by a warning message integrated into a sample deployment workflow. While the approach is applicable for most classification deep neural networks (DNNs), we demonstrate its application to a brain metastases (BM) detector for T1-weighted contrast-enhanced (T1c) 3D MRI. The BM detection model was trained using 175 T1c studies acquired internally (from the authors' institution) and tested using (1) 42 internally acquired exams and (2) 72 externally acquired exams from the publicly distributed Brain Mets dataset provided by the Stanford University School of Medicine. Generalizability scores, false positive (FP) rates, and sensitivities of the BM detector were computed for the test datasets.

RESULTS AND CONCLUSION

The model predicted its generalizability to be low for 31% of the testing data (i.e., two of the internally and 33 of the externally acquired exams), where it produced (1) ∼13.5 false positives (FPs) at 76.1% BM detection sensitivity for the low and (2) ∼10.5 FPs at 89.2% BM detection sensitivity for the high generalizability groups respectively. These results suggest that the proposed formulation enables a model to predict its generalizability for unseen data.

Machine learning in metastatic cancer research: Potentials, possibilities, and prospects

Cancer has received extensive recognition for its high mortality rate, with metastatic cancer being the top cause of cancer-related deaths. Metastatic cancer involves the spread of the primary tumor to other body organs. As much as the early detection of cancer is essential, the timely detection of metastasis, the identification of biomarkers, and treatment choice are valuable for improving the quality of life for metastatic cancer patients. This study reviews the existing studies on classical machine learning (ML) and deep learning (DL) in metastatic cancer research. Since the majority of metastatic cancer research data are collected in the formats of PET/CT and MRI image data, deep learning techniques are heavily involved. However, its black-box nature and expensive computational cost are notable concerns. Furthermore, existing models could be overestimated for their generality due to the non-diverse population in clinical trial datasets. Therefore, research gaps are itemized; follow-up studies should be carried out on metastatic cancer using machine learning and deep learning tools with data in a symmetric manner.

Automatic detection of punctate white matter lesions in infants using deep learning of composite images from two cases

Punctate white matter lesions (PWMLs) in infants may be related to neurodevelopmental outcomes based on the location or number of lesions. This study aimed to assess the automatic detectability of PWMLs in infants on deep learning using composite images created from several cases. To create the initial composite images, magnetic resonance (MR) images of two infants with the most PWMLs were used; their PWMLs were extracted and pasted onto MR images of infants without abnormality, creating many composite PWML images. Deep learning models based on a convolutional neural network, You Only Look Once v3 (YOLOv3), were constructed using the training set of 600, 1200, 2400, and 3600 composite images. As a result, a threshold of detection probability of 20% and 30% for all deep learning model sets yielded a relatively high sensitivity for automatic PWML detection (0.908-0.957). Although relatively high false-positive detections occurred with the lower threshold of detection probability, primarily, in the partial volume of the cerebral cortex (≥ 85.8%), those can be easily distinguished from the white matter lesions. Relatively highly sensitive automatic detection of PWMLs was achieved by creating composite images from two cases using deep learning.

Diagnostic performance of magnetic resonance imaging-based machine learning in Alzheimer's disease detection: a meta-analysis

PURPOSE

Advanced machine learning (ML) algorithms can assist rapid medical image recognition and realize automatic, efficient, noninvasive, and convenient diagnosis. We aim to further evaluate the diagnostic performance of ML to distinguish patients with probable Alzheimer's disease (AD) from normal older adults based on structural magnetic resonance imaging (MRI).

METHODS

The Medline, Embase, and Cochrane Library databases were searched for relevant literature published up until July 2021. We used the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool and Checklist for Artificial Intelligence in Medical Imaging (CLAIM) to evaluate all included studies' quality and potential bias. Random-effects models were used to calculate pooled sensitivity and specificity, and the Deeks' test was used to assess publication bias.

RESULTS

We included 24 models based on different brain features extracted by ML algorithms in 19 papers. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, and area under the summary receiver operating characteristic curve for ML in detecting AD were 0.85 (95%CI 0.81-0.89), 0.88 (95%CI 0.84-0.91), 7.15 (95%CI 5.40-9.47), 0.17 (95%CI 0.12-0.22), 43.34 (95%CI 26.89-69.84), and 0.93 (95%CI 0.91-0.95).

CONCLUSION

ML using structural MRI data performed well in diagnosing probable AD patients and normal elderly. However, more high-quality, large-scale prospective studies are needed to further enhance the reliability and generalizability of ML for clinical applications before it can be introduced into clinical practice.

A Deep Learning-Based Computer Aided Detection (CAD) System for Difficult-to-Detect Brain Metastases

PURPOSE

We sought to develop a computer-aided detection (CAD) system that optimally augments human performance, excelling especially at identifying small inconspicuous brain metastases (BMs), by training a convolutional neural network on a unique magnetic resonance imaging (MRI) data set containing subtle BMs that were not detected prospectively during routine clinical care.

METHODS AND MATERIALS

Patients receiving stereotactic radiosurgery (SRS) for BMs at our institution from 2016 to 2018 without prior brain-directed therapy or small cell histology were eligible. For patients who underwent 2 consecutive courses of SRS, treatment planning MRIs from their initial course were reviewed for radiographic evidence of an emerging metastasis at the same location as metastases treated in their second SRS course. If present, these previously unidentified lesions were contoured and categorized as retrospectively identified metastases (RIMs). RIMs were further subcategorized according to whether they did (+DC) or did not (-DC) meet diagnostic imaging-based criteria to definitively classify them as metastases based upon their appearance in the initial MRI alone. Prospectively identified metastases (PIMs) from these patients, and from patients who only underwent a single course of SRS, were also included. An open-source convolutional neural network architecture was adapted and trained to detect both RIMs and PIMs on thin-slice, contrast-enhanced, spoiled gradient echo MRIs. Patients were randomized into 5 groups: 4 for training/cross-validation and 1 for testing.

RESULTS

One hundred thirty-five patients with 563 metastases, including 72 RIMS, met criteria. For the test group, CAD sensitivity was 94% for PIMs, 80% for +DC RIMs, and 79% for PIMs and +DC RIMs with diameter <3 mm, with a median of 2 false positives per patient and a Dice coefficient of 0.79.

CONCLUSIONS

Our CAD model, trained on a novel data set and using a single common MR sequence, demonstrated high sensitivity and specificity overall, outperforming published CAD results for small metastases and RIMs - the lesion types most in need of human performance augmentation.

Application of artificial intelligence to stereotactic radiosurgery for intracranial lesions: detection, segmentation, and outcome prediction

BACKGROUND

Rapid evolution of artificial intelligence (AI) prompted its wide application in healthcare systems. Stereotactic radiosurgery served as a good candidate for AI model development and achieved encouraging result in recent years. This article aimed at demonstrating current AI application in radiosurgery.

METHODS

Literatures published in PubMed during 2010-2022, discussing AI application in stereotactic radiosurgery were reviewed.

RESULTS

AI algorithms, especially machine learning/deep learning models, have been administered to different aspect of stereotactic radiosurgery. Spontaneous tumor detection and automated lesion delineation or segmentation were two of the promising application, which could be further extended to longitudinal treatment follow-up. Outcome prediction utilized machine learning algorithms with radiomic-based analysis was another well-established application.

CONCLUSIONS

Stereotactic radiosurgery has taken a lead role in AI development. Current achievement, limitation, and further investigation was summarized in this article.

Deep Learning for Detecting Brain Metastases on MRI: A Systematic Review and Meta-Analysis

Since manual detection of brain metastases (BMs) is time consuming, studies have been conducted to automate this process using deep learning. The purpose of this study was to conduct a systematic review and meta-analysis of the performance of deep learning models that use magnetic resonance imaging (MRI) to detect BMs in cancer patients. A systematic search of MEDLINE, EMBASE, and Web of Science was conducted until 30 September 2022. Inclusion criteria were: patients with BMs; deep learning using MRI images was applied to detect the BMs; sufficient data were present in terms of detective performance; original research articles. Exclusion criteria were: reviews, letters, guidelines, editorials, or errata; case reports or series with less than 20 patients; studies with overlapping cohorts; insufficient data in terms of detective performance; machine learning was used to detect BMs; articles not written in English. Quality Assessment of Diagnostic Accuracy Studies-2 and Checklist for Artificial Intelligence in Medical Imaging was used to assess the quality. Finally, 24 eligible studies were identified for the quantitative analysis. The pooled proportion of patient-wise and lesion-wise detectability was 89%. Articles should adhere to the checklists more strictly. Deep learning algorithms effectively detect BMs. Pooled analysis of false positive rates could not be estimated due to reporting differences.

Automated Detection of Brain Metastases on T1-Weighted MRI Using a Convolutional Neural Network: Impact of Volume Aware Loss and Sampling Strategy

BACKGROUND

Detection of brain metastases (BM) and segmentation for treatment planning could be optimized with machine learning methods. Convolutional neural networks (CNNs) are promising, but their trade-offs between sensitivity and precision frequently lead to missing small lesions.

HYPOTHESIS

Combining volume aware (VA) loss function and sampling strategy could improve BM detection sensitivity.

STUDY TYPE

Retrospective.

POPULATION

A total of 530 radiation oncology patients (55% women) were split into a training/validation set (433 patients/1460 BM) and an independent test set (97 patients/296 BM).

FIELD STRENGTH/SEQUENCE

1.5 T and 3 T, contrast-enhanced three-dimensional (3D) T1-weighted fast gradient echo sequences.

ASSESSMENT

Ground truth masks were based on radiotherapy treatment planning contours reviewed by experts. A U-Net inspired model was trained. Three loss functions (Dice, Dice + boundary, and VA) and two sampling methods (label and VA) were compared. Results were reported with Dice scores, volumetric error, lesion detection sensitivity, and precision. A detected voxel within the ground truth constituted a true positive.

STATISTICAL TESTS

McNemar's exact test to compare detected lesions between models. Pearson's correlation coefficient and Bland-Altman analysis to compare volume agreement between predicted and ground truth volumes. Statistical significance was set at P ≤ 0.05.

RESULTS

Combining VA loss and VA sampling performed best with an overall sensitivity of 91% and precision of 81%. For BM in the 2.5-6 mm estimated sphere diameter range, VA loss reduced false negatives by 58% and VA sampling reduced it further by 30%. In the same range, the boundary loss achieved the highest precision at 81%, but a low sensitivity (24%) and a 31% Dice loss.

DATA CONCLUSION

Considering BM size in the loss and sampling function of CNN may increase the detection sensitivity regarding small BM. Our pipeline relying on a single contrast-enhanced T1-weighted MRI sequence could reach a detection sensitivity of 91%, with an average of only 0.66 false positives per scan.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

Machine Learning for Diagnosis of Systemic Lupus Erythematosus: A Systematic Review and Meta-Analysis

Background

Application of machine learning (ML) for identification of systemic lupus erythematosus (SLE) has been recently drawing increasing attention, while there is still lack of evidence-based support.

Methods

Systematic review and meta-analysis are conducted to evaluate its diagnostic accuracy and application prospect. PubMed, Embase, Cochrane Library, and Web of Science libraries are searched, in combination with manual searching and literature retrospection, for studies regarding machine learning for identifying SLE and neuropsychiatric systemic lupus erythematosus (NPSLE). Quality Assessment of Diagnostic Accuracy Studies (QUADA-2) is applied to assess the quality of included studies. Diagnostic accuracy of the SLE model and NPSLE model is assessed using the bivariate fixed-effect model, and the data are pooled. Summary receiver operator characteristic curve (SROC) is plotted, and area under the curve (AUC) is calculated.

Results

Eighteen (18) studies are included, in which ten (10) focused on SLE and eight (8) on NPSLE. The AUC of SLE identification is 0.95, the sensitivity is 0.90, the specificity is 0.89, the PLR is 8.4, the NLR is 0.12, and the DOR is 73. AUC of NPSLE identification is 0.89, the sensitivity is 0.83, the specificity is 0.83, the PLR is 5.0, the NLR is 0.20, and the DOR is 25.

Conclusion

Machine learning presented remarkable performance in identification of SLE and NPSLE. Based on the convenience for inclusion factor collection and non-invasiveness of detection, machine learning is expected to be widely applied in clinical practice to assist medical decision making.

A Self-Attention-Guided 3D Deep Residual Network With Big Transfer to Predict Local Failure in Brain Metastasis After Radiotherapy Using Multi-Channel MRI

A noticeable proportion of larger brain metastases (BMs) are not locally controlled after stereotactic radiotherapy, and it may take months before local progression is apparent on standard follow-up imaging. This work proposes and investigates new explainable deep-learning models to predict the radiotherapy outcome for BM. A novel self-attention-guided 3D residual network is introduced for predicting the outcome of local failure (LF) after radiotherapy using the baseline treatment-planning MRI. The 3D self-attention modules facilitate capturing long-range intra/inter slice dependencies which are often overlooked by convolution layers. The proposed model was compared to a vanilla 3D residual network and 3D residual network with CBAM attention in terms of performance in outcome prediction. A training recipe was adapted for the outcome prediction models during pretraining and training the down-stream task based on the recently proposed big transfer principles. A novel 3D visualization module was coupled with the model to demonstrate the impact of various intra/peri-lesion regions on volumetric multi-channel MRI upon the network's prediction. The proposed self-attention-guided 3D residual network outperforms the vanilla residual network and the residual network with CBAM attention in accuracy, F1-score, and AUC. The visualization results show the importance of peri-lesional characteristics on treatment-planning MRI in predicting local outcome after radiotherapy. This study demonstrates the potential of self-attention-guided deep-learning features derived from volumetric MRI in radiotherapy outcome prediction for BM. The insights obtained via the developed visualization module for individual lesions can possibly be applied during radiotherapy planning to decrease the chance of LF.

Predicting the outcome of radiotherapy in brain metastasis by integrating the clinical and MRI-based deep learning features

BACKGROUND

A considerable proportion of metastatic brain tumors progress locally despite stereotactic radiation treatment, and it can take months before such local progression is evident on follow-up imaging. Prediction of radiotherapy outcome in terms of tumor local failure is crucial for these patients and can facilitate treatment adjustments or allow for early salvage therapies.

PURPOSE

In this work, a novel deep learning architecture is introduced to predict the outcome of local control/failure in brain metastasis treated with stereotactic radiation therapy using treatment-planning magnetic resonance imaging (MRI) and standard clinical attributes.

METHODS

At the core of the proposed architecture is an InceptionResentV2 network to extract distinct features from each MRI slice for local outcome prediction. A recurrent or transformer network is integrated into the architecture to incorporate spatial dependencies between MRI slices into the predictive modeling. A visualization method based on prediction difference analysis is coupled with the deep learning model to illustrate how different regions of each lesion on MRI contribute to the model's prediction. The model was trained and optimized using the data acquired from 99 patients (116 lesions) and evaluated on an independent test set of 25 patients (40 lesions).

RESULTS

The results demonstrate the promising potential of the MRI deep learning features for outcome prediction, outperforming standard clinical variables. The prediction model with only clinical variables demonstrated an area under the receiver operating characteristic curve (AUC) of 0.68. The MRI deep learning models resulted in AUCs in the range of 0.72 to 0.83 depending on the mechanism to integrate information from MRI slices of each lesion. The best prediction performance (AUC = 0.86) was associated with the model that combined the MRI deep learning features with clinical variables and incorporated the inter-slice dependencies using a long short-term memory recurrent network. The visualization results highlighted the importance of tumor/lesion margins in local outcome prediction for brain metastasis.

CONCLUSIONS

The promising results of this study show the possibility of early prediction of radiotherapy outcome for brain metastasis via deep learning of MRI and clinical attributes at pre-treatment and encourage future studies on larger groups of patients treated with other radiotherapy modalities.

Principles of Lung Cancer Metastasis to Brain

Lung cancer is a disease associated with significant morbidity and mortality on a global setting. This form of cancer commonly gives raise to metastatic lesions the brain, which can further worsen outcomes. In this focused review, we discuss an overview of lung cancers that metastasize to the brain: known risk factors; means of detection and diagnosis; and options for treatment including a comparison between surgical resection, stereotactic radiosurgery, and whole-brain radiation therapy. These interventions are still being assessed by clinical trials and continue to be modified through evidence-based practice.

Small-cell lung cancer brain metastasis: From molecular mechanisms to diagnosis and treatment

Lung cancer is the most malignant human cancer worldwide, also with the highest incidence rate. However, small-cell lung cancer (SCLC) accounts for 14 % of all lung cancer cases. Approximately 10 % of patients with SCLC have brain metastasis at the time of diagnosis, which is the leading cause of death of patients with SCLC worldwide. The median overall survival is only 4.9 months, and a long-tern cure exists for patients with SCLC brain metastasis due to limited common therapeutic options. Recent studies have enhanced our understanding of the molecular mechanisms leading to meningeal metastasis, and multimodality treatments have brought new hopes for a better cure for the disease. This review aimed to offer an insight into the cellular processes of different metastatic stages of SCLC revealed by the established animal models, and into the major diagnostic methods of SCLC. Additionally, it provided in-depth information on the recent advances in SCLC treatments, and highlighted several new models and biomarkers with promises to improve the prognosis of SCLC.

Advancing Brain Metastases Detection in T1-Weighted Contrast-Enhanced 3D MRI Using Noisy Student-Based Training

The detection of brain metastases (BM) in their early stages could have a positive impact on the outcome of cancer patients. The authors previously developed a framework for detecting small BM (with diameters of <15 mm) in T1-weighted contrast-enhanced 3D magnetic resonance images (T1c). This study aimed to advance the framework with a noisy-student-based self-training strategy to use a large corpus of unlabeled T1c data. Accordingly, a sensitivity-based noisy-student learning approach was formulated to provide high BM detection sensitivity with a reduced count of false positives. This paper (1) proposes student/teacher convolutional neural network architectures, (2) presents data and model noising mechanisms, and (3) introduces a novel pseudo-labeling strategy factoring in the sensitivity constraint. The evaluation was performed using 217 labeled and 1247 unlabeled exams via two-fold cross-validation. The framework utilizing only the labeled exams produced 9.23 false positives for 90% BM detection sensitivity, whereas the one using the introduced learning strategy led to ~9% reduction in false detections (i.e., 8.44). Significant reductions in false positives (>10%) were also observed in reduced labeled data scenarios (using 50% and 75% of labeled data). The results suggest that the introduced strategy could be utilized in existing medical detection applications with access to unlabeled datasets to elevate their performances.

Deep learning for brain metastasis detection and segmentation in longitudinal MRI data

PURPOSE

Brain metastases occur frequently in patients with metastatic cancer. Early and accurate detection of brain metastases is essential for treatment planning and prognosis in radiation therapy. Due to their tiny sizes and relatively low contrast, small brain metastases are very difficult to detect manually. With the recent development of deep learning technologies, several researchers have reported promising results in automated brain metastasis detection. However, the detection sensitivity is still not high enough for tiny brain metastases, and integration into clinical practice in regard to differentiating true metastases from false positives is challenging.

METHODS

The DeepMedic network with the binary cross-entropy (BCE) loss is used as our baseline method. To improve brain metastasis detection performance, a custom detection loss called volume-level sensitivity-specificity (VSS) is proposed, which rates metastasis detection sensitivity and specificity at a (sub-)volume level. As sensitivity and precision are always a trade-off, either a high sensitivity or a high precision can be achieved for brain metastasis detection by adjusting the weights in the VSS loss without decline in dice score coefficient for segmented metastases. To reduce metastasis-like structures being detected as false positive metastases, a temporal prior volume is proposed as an additional input of DeepMedic. The modified network is called DeepMedic+ for distinction. Combining a high sensitivity VSS loss and a high specificity loss for DeepMedic+, the majority of true positive metastases are confirmed with high specificity, while additional metastases candidates in each patient are marked with high sensitivity for detailed expert evaluation.

RESULTS

Our proposed VSS loss improves the sensitivity of brain metastasis detection, increasing the sensitivity from 85.3% for DeepMedic with BCE to 97.5% for DeepMedic with VSS. Alternatively, the precision is improved from 69.1% for DeepMedic with BCE to 98.7% for DeepMedic with VSS. Comparing DeepMedic+ with DeepMedic with the same VSS loss, 44.4% of the false positive metastases are reduced in the high sensitivity model and the precision reaches 99.6% for the high specificity model. The mean dice coefficient for all metastases is about 0.81. With the ensemble of the high sensitivity and high specificity models, on average only 1.5 false positive metastases per patient need further check, while the majority of true positive metastases are confirmed.

CONCLUSIONS

Our proposed VSS loss and temporal prior improve brain metastasis detection sensitivity and precision. The ensemble learning is able to distinguish high confidence true positive metastases from metastases candidates that require special expert review or further follow-up, being particularly well-fit to the requirements of expert support in real clinical practice. This facilitates metastasis detection and segmentation for neuroradiologists in diagnostic and radiation oncologists in therapeutic clinical applications. This article is protected by copyright. All rights reserved.

Artificial intelligence for radiological paediatric fracture assessment: a systematic review

BACKGROUND

Majority of research and commercial efforts have focussed on use of artificial intelligence (AI) for fracture detection in adults, despite the greater long-term clinical and medicolegal implications of missed fractures in children. The objective of this study was to assess the available literature regarding diagnostic performance of AI tools for paediatric fracture assessment on imaging, and where available, how this compares with the performance of human readers.

MATERIALS AND METHODS

MEDLINE, Embase and Cochrane Library databases were queried for studies published between 1 January 2011 and 2021 using terms related to 'fracture', 'artificial intelligence', 'imaging' and 'children'. Risk of bias was assessed using a modified QUADAS-2 tool. Descriptive statistics for diagnostic accuracies were collated.

RESULTS

Nine eligible articles from 362 publications were included, with most (8/9) evaluating fracture detection on radiographs, with the elbow being the most common body part. Nearly all articles used data derived from a single institution, and used deep learning methodology with only a few (2/9) performing external validation. Accuracy rates generated by AI ranged from 88.8 to 97.9%. In two of the three articles where AI performance was compared to human readers, sensitivity rates for AI were marginally higher, but this was not statistically significant.

CONCLUSIONS

Wide heterogeneity in the literature with limited information on algorithm performance on external datasets makes it difficult to understand how such tools may generalise to a wider paediatric population. Further research using a multicentric dataset with real-world evaluation would help to better understand the impact of these tools.

Automated brain tumour identification using magnetic resonance imaging: a systematic review and meta-analysis

Background

Automated brain tumor identification facilitates diagnosis and treatment planning. We evaluate the performance of traditional machine learning (TML) and deep learning (DL) in brain tumor detection and segmentation, using MRI.

Methods

A systematic literature search from January 2000 to May 8, 2021 was conducted. Study quality was assessed using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Detection meta-analysis was performed using a unified hierarchical model. Segmentation studies were evaluated using a random effects model. Sensitivity analysis was performed for externally validated studies.

Results

Of 224 studies included in the systematic review, 46 segmentation and 38 detection studies were eligible for meta-analysis. In detection, DL achieved a lower false positive rate compared to TML; 0.018 (95% CI, 0.011 to 0.028) and 0.048 (0.032 to 0.072) (P < .001), respectively. In segmentation, DL had a higher dice similarity coefficient (DSC), particularly for tumor core (TC); 0.80 (0.77 to 0.83) and 0.63 (0.56 to 0.71) (P < .001), persisting on sensitivity analysis. Both manual and automated whole tumor (WT) segmentation had “good” (DSC ≥ 0.70) performance. Manual TC segmentation was superior to automated; 0.78 (0.69 to 0.86) and 0.64 (0.53 to 0.74) (P = .014), respectively. Only 30% of studies reported external validation.

Conclusions

The comparable performance of automated to manual WT segmentation supports its integration into clinical practice. However, manual outperformance for sub-compartmental segmentation highlights the need for further development of automated methods in this area. Compared to TML, DL provided superior performance for detection and sub-compartmental segmentation. Improvements in the quality and design of studies, including external validation, are required for the interpretability and generalizability of automated models.

Machine Learning for the Detection and Segmentation of Benign Tumors of the Central Nervous System: A Systematic Review

Simple Summary

Machine learning in radiology of the central nervous system has seen many interesting publications in the past few years. Since the focus has largely been on malignant tumors such as brain metastases and high-grade gliomas, we conducted a systematic review on benign tumors to summarize what has been published and where there might be gaps in the research. We found several studies that report good results, but the descriptions of methodologies could be improved to enable better comparisons and assessment of biases.

Abstract

Objectives: To summarize the available literature on using machine learning (ML) for the detection and segmentation of benign tumors of the central nervous system (CNS) and to assess the adherence of published ML/diagnostic accuracy studies to best practice. Methods: The MEDLINE database was searched for the use of ML in patients with any benign tumor of the CNS, and the records were screened according to PRISMA guidelines. Results: Eleven retrospective studies focusing on meningioma (n = 4), vestibular schwannoma (n = 4), pituitary adenoma (n = 2) and spinal schwannoma (n = 1) were included. The majority of studies attempted segmentation. Links to repositories containing code were provided in two manuscripts, and no manuscripts shared imaging data. Only one study used an external test set, which raises the question as to whether some of the good performances that have been reported were caused by overfitting and may not generalize to data from other institutions. Conclusions: Using ML for detecting and segmenting benign brain tumors is still in its infancy. Stronger adherence to ML best practices could facilitate easier comparisons between studies and contribute to the development of models that are more likely to one day be used in clinical practice.

A Web-Based Prediction Model for Overall Survival of Elderly Patients With Malignant Bone Tumors: A Population-Based Study

Background: Malignant bone tumors (MBT) are one of the causes of death in elderly patients. The purpose of our study is to establish a nomogram to predict the overall survival (OS) of elderly patients with MBT.

Methods: The clinicopathological data of all elderly patients with MBT from 2004 to 2018 were downloaded from the SEER database. They were randomly assigned to the training set (70%) and validation set (30%). Univariate and multivariate Cox regression analysis was used to identify independent risk factors for elderly patients with MBT. A nomogram was built based on these risk factors to predict the 1-, 3-, and 5-year OS of elderly patients with MBT. Then, used the consistency index (C-index), calibration curve, and the area under the receiver operating curve (AUC) to evaluate the accuracy and discrimination of the prediction model was. Decision curve analysis (DCA) was used to assess the clinical potential application value of the nomogram. Based on the scores on the nomogram, patients were divided into high- and low-risk groups. The Kaplan-Meier (K-M) curve was used to test the difference in survival between the two patients.

Results: A total of 1,641 patients were included, and they were randomly assigned to the training set (N = 1,156) and the validation set (N = 485). The univariate and multivariate analysis of the training set suggested that age, sex, race, primary site, histologic type, grade, stage, M stage, surgery, and tumor size were independent risk factors for elderly patients with MBT. The C-index of the training set and the validation set were 0.779 [0.759–0.799] and 0.801 [0.772–0.830], respectively. The AUC of the training and validation sets also showed similar results. The calibration curves of the training and validation sets indicated that the observed and predicted values were highly consistent. DCA suggested that the nomogram had potential clinical value compared with traditional TNM staging.

Conclusion: We had established a new nomogram to predict the 1-, 3-, 5-year OS of elderly patients with MBT. This predictive model can help doctors and patients develop treatment plans and follow-up strategies.

An artificial intelligence framework and its bias for brain tumor segmentation: A narrative review

BACKGROUND

Artificial intelligence (AI) has become a prominent technique for medical diagnosis and represents an essential role in detecting brain tumors. Although AI-based models are widely used in brain lesion segmentation (BLS), understanding their effectiveness is challenging due to their complexity and diversity. Several reviews on brain tumor segmentation are available, but none of them describe a link between the threats due to risk-of-bias (RoB) in AI and its architectures. In our review, we focused on linking RoB and different AI-based architectural Cluster in popular DL framework. Further, due to variance in these designs and input data types in medical imaging, it is necessary to present a narrative review considering all facets of BLS.

APPROACH

The proposed study uses a PRISMA strategy based on 75 relevant studies found by searching PubMed, Scopus, and Google Scholar. Based on the architectural evolution, DL studies were subsequently categorized into four classes: convolutional neural network (CNN)-based, encoder-decoder (ED)-based, transfer learning (TL)-based, and hybrid DL (HDL)-based architectures. These studies were then analyzed considering 32 AI attributes, with clusters including AI architecture, imaging modalities, hyper-parameters, performance evaluation metrics, and clinical evaluation. Then, after these studies were scored for all attributes, a composite score was computed, normalized, and ranked. Thereafter, a bias cutoff (AP(ai)Bias 1.0, AtheroPoint, Roseville, CA, USA) was established to detect low-, moderate- and high-bias studies.

CONCLUSION

The four classes of architectures, from best-to worst-performing, are TL > ED > CNN > HDL. ED-based models had the lowest AI bias for BLS. This study presents a set of three primary and six secondary recommendations for lowering the RoB.

Diagnostic Accuracy of Artificial Intelligence Based on Imaging Data for Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma: A Systematic Review and Meta-Analysis

Background

The presence of microvascular invasion (MVI) is considered an independent prognostic factor associated with early recurrence and poor survival in hepatocellular carcinoma (HCC) patients after resection. Artificial intelligence (AI), mainly consisting of non-deep learning algorithms (NDLAs) and deep learning algorithms (DLAs), has been widely used for MVI prediction in medical imaging.

Aim

To assess the diagnostic accuracy of AI algorithms for non-invasive, preoperative prediction of MVI based on imaging data.

Methods

Original studies reporting AI algorithms for non-invasive, preoperative prediction of MVI based on quantitative imaging data were identified in the databases PubMed, Embase, and Web of Science. The quality of the included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) scale. The pooled sensitivity, specificity, positive likelihood ratio (PLR), and negative likelihood ratio (NLR) were calculated using a random-effects model with 95% CIs. A summary receiver operating characteristic curve and the area under the curve (AUC) were generated to assess the diagnostic accuracy of the deep learning and non-deep learning models. In the non-deep learning group, we further performed meta-regression and subgroup analyses to identify the source of heterogeneity.

Results

Data from 16 included studies with 4,759 cases were available for meta-analysis. Four studies on deep learning models, 12 studies on non-deep learning models, and two studies compared the efficiency of the two types. For predictive performance of deep learning models, the pooled sensitivity, specificity, PLR, NLR, and AUC values were 0.84 [0.75–0.90], 0.84 [0.77–0.89], 5.14 [3.53–7.48], 0.2 [0.12–0.31], and 0.90 [0.87–0.93]; and for non-deep learning models, they were 0.77 [0.71–0.82], 0.77 [0.73–0.80], 3.30 [2.83–3.84], 0.30 [0.24–0.38], and 0.82 [0.79–0.85], respectively. Subgroup analyses showed a significant difference between the single tumor subgroup and the multiple tumor subgroup in the pooled sensitivity, NLR, and AUC.

Conclusion

This meta-analysis demonstrates the high diagnostic accuracy of non-deep learning and deep learning methods for MVI status prediction and their promising potential for clinical decision-making. Deep learning models perform better than non-deep learning models in terms of the accuracy of MVI prediction, methodology, and cost-effectiveness.

Systematic Review Registration

https://www.crd.york.ac.uk/PROSPERO/display_record.php? RecordID=260891, ID:CRD42021260891.

Machine Learning Applications for Differentiation of Glioma from Brain Metastasis-A Systematic Review

Simple Summary

We present a systematic review of published reports on machine learning (ML) applications for the differentiation of gliomas from brain metastases by summarizing study characteristics, strengths, and pitfalls. Based on these findings, we present recommendations for future research in this field.

Abstract

Glioma and brain metastasis can be difficult to distinguish on conventional magnetic resonance imaging (MRI) due to the similarity of imaging features in specific clinical circumstances. Multiple studies have investigated the use of machine learning (ML) models for non-invasive differentiation of glioma from brain metastasis. Many of the studies report promising classification results, however, to date, none have been implemented into clinical practice. After a screening of 12,470 studies, we included 29 eligible studies in our systematic review. From each study, we aggregated data on model design, development, and best classifiers, as well as quality of reporting according to the TRIPOD statement. In a subset of eligible studies, we conducted a meta-analysis of the reported AUC. It was found that data predominantly originated from single-center institutions (n = 25/29) and only two studies performed external validation. The median TRIPOD adherence was 0.48, indicating insufficient quality of reporting among surveyed studies. Our findings illustrate that despite promising classification results, reliable model assessment is limited by poor reporting of study design and lack of algorithm validation and generalizability. Therefore, adherence to quality guidelines and validation on outside datasets is critical for the clinical translation of ML for the differentiation of glioma and brain metastasis.