Deep-learning and radiomics ensemble classifier for false positive reduction in brain metastases segmentation

Two-Stage Deep Learning Model for Diagnosis of Lumbar Spondylolisthesis Based on Lateral X-Ray Images

BACKGROUND

Diagnosing early lumbar spondylolisthesis is challenging for many doctors because of the lack of obvious symptoms. Using deep learning (DL) models to improve the accuracy of X-ray diagnoses can effectively reduce missed and misdiagnoses in clinical practice. This study aimed to use a two-stage deep learning model, the Res-SE-Net model with the YOLOv8 algorithm, to facilitate efficient and reliable diagnosis of early lumbar spondylolisthesis based on lateral X-ray image identification.

METHODS

A total of 2424 lumbar lateral radiographs of patients treated in the Beijing Tongren Hospital between January 2021 and September 2023 were obtained. The data were labeled and mutually identified by 3 orthopedic surgeons after reshuffling in a random order and divided into a training set, validation set, and test set in a ratio of 7:2:1. We trained 2 models for automatic detection of spondylolisthesis. YOLOv8 model was used to detect the position of lumbar spondylolisthesis, and the Res-SE-Net classification method was designed to classify the clipped area and determine whether it was lumbar spondylolisthesis. The model performance was evaluated using a test set and an external dataset from Beijing Haidian Hospital. Finally, we compared model validation results with professional clinicians' evaluation.

RESULTS

The model achieved promising results, with a high diagnostic accuracy of 92.3%, precision of 93.5%, and recall of 93.1% for spondylolisthesis detection on the test set, the area under the curve (AUC) value was 0.934.

CONCLUSIONS

Our two-stage deep learning model provides doctors with a reference basis for the better diagnosis and treatment of early lumbar spondylolisthesis.

Where Does Auto-Segmentation for Brain Metastases Radiosurgery Stand Today?

Detection and segmentation of brain metastases (BMs) play a pivotal role in diagnosis, treatment planning, and follow-up evaluations for effective BM management. Given the rising prevalence of BM cases and its predominantly multiple onsets, automated segmentation is becoming necessary in stereotactic radiosurgery. It not only alleviates the clinician's manual workload and improves clinical workflow efficiency but also ensures treatment safety, ultimately improving patient care. Recent strides in machine learning, particularly in deep learning (DL), have revolutionized medical image segmentation, achieving state-of-the-art results. This review aims to analyze auto-segmentation strategies, characterize the utilized data, and assess the performance of cutting-edge BM segmentation methodologies. Additionally, we delve into the challenges confronting BM segmentation and share insights gleaned from our algorithmic and clinical implementation experiences.

Robust deep learning from incomplete annotation for accurate lung nodule detection

Deep learning plays a significant role in the detection of pulmonary nodules in low-dose computed tomography (LDCT) scans, contributing to the diagnosis and treatment of lung cancer. Nevertheless, its effectiveness often relies on the availability of extensive, meticulously annotated dataset. In this paper, we explore the utilization of an incompletely annotated dataset for pulmonary nodules detection and introduce the FULFIL (Forecasting Uncompleted Labels For Inexpensive Lung nodule detection) algorithm as an innovative approach. By instructing annotators to label only the nodules they are most confident about, without requiring complete coverage, we can substantially reduce annotation costs. Nevertheless, this approach results in an incompletely annotated dataset, which presents challenges when training deep learning models. Within the FULFIL algorithm, we employ Graph Convolution Network (GCN) to discover the relationships between annotated and unannotated nodules for self-adaptively completing the annotation. Meanwhile, a teacher-student framework is employed for self-adaptive learning using the completed annotation dataset. Furthermore, we have designed a Dual-Views loss to leverage different data perspectives, aiding the model in acquiring robust features and enhancing generalization. We carried out experiments using the LUng Nodule Analysis (LUNA) dataset, achieving a sensitivity of 0.574 at a False positives per scan (FPs/scan) of 0.125 with only 10% instance-level annotations for nodules. This performance outperformed comparative methods by 7.00%. Experimental comparisons were conducted to evaluate the performance of our model and human experts on test dataset. The results demonstrate that our model can achieve a comparable level of performance to that of human experts. The comprehensive experimental results demonstrate that FULFIL can effectively leverage an incomplete pulmonary nodule dataset to develop a robust deep learning model, making it a promising tool for assisting in lung nodule detection.

Sacroiliitis diagnosis based on interpretable features and multi-task learning

Objective.Sacroiliitis is an early pathological manifestation of ankylosing spondylitis (AS), and a positive sacroiliitis test on imaging may help clinical practitioners diagnose AS early. Deep learning based automatic diagnosis algorithms can deliver grading findings for sacroiliitis, however, it requires a large amount of data with precise labels to train the model and lacks grading features visualization. In this paper, we aimed to propose a radiomics and deep learning based deep feature visualization positive diagnosis algorithm for sacroiliitis on CT scans. Visualization of grading features can enhance clinical interpretability with visual grading features, which assist doctors in diagnosis and treatment more effectively.Approach.The region of interest (ROI) is identified by segmenting the sacroiliac joint (SIJ) 3D CT images using a combination of the U-net model and certain statistical approaches. Then, in addition to extracting spatial and frequency domain features from ROI according to the radiographic manifestations of sacroiliitis, the radiomics features have also been integrated into the proposed encoder module to obtain a powerful encoder and extract features effectively. Finally, a multi-task learning technique and five-class labels are utilized to help with performing positive tests to reduce discrepancies in the evaluation of several radiologists.Main results.On our private dataset, proposed methods have obtained an accuracy rate of 87.3%, which is 9.8% higher than the baseline and consistent with assessments made by qualified medical professionals.Significance.The results of the ablation experiment and interpreting analysis demonstrated that the proposed methods are applied in automatic CT scan sacroiliitis diagnosis due to their excellently interpretable and portable advantages.

Deep learning in MRI-guided radiation therapy: A systematic review

Recent advances in MRI-guided radiation therapy (MRgRT) and deep learning techniques encourage fully adaptive radiation therapy (ART), real-time MRI monitoring, and the MRI-only treatment planning workflow. Given the rapid growth and emergence of new state-of-the-art methods in these fields, we systematically review 197 studies written on or before December 31, 2022, and categorize the studies into the areas of image segmentation, image synthesis, radiomics, and real time MRI. Building from the underlying deep learning methods, we discuss their clinical importance and current challenges in facilitating small tumor segmentation, accurate x-ray attenuation information from MRI, tumor characterization and prognosis, and tumor motion tracking. In particular, we highlight the recent trends in deep learning such as the emergence of multi-modal, visual transformer, and diffusion models.

Stratified assessment of an FDA-cleared deep learning algorithm for automated detection and contouring of metastatic brain tumors in stereotactic radiosurgery

PURPOSE

Artificial intelligence-based tools can be leveraged to improve detection and segmentation of brain metastases for stereotactic radiosurgery (SRS). VBrain by Vysioneer Inc. is a deep learning algorithm with recent FDA clearance to assist in brain tumor contouring. We aimed to assess the performance of this tool by various demographic and clinical characteristics among patients with brain metastases treated with SRS.

MATERIALS AND METHODS

We randomly selected 100 patients with brain metastases who underwent initial SRS on the CyberKnife from 2017 to 2020 at a single institution. Cases with resection cavities were excluded from the analysis. Computed tomography (CT) and axial T1-weighted post-contrast magnetic resonance (MR) image data were extracted for each patient and uploaded to VBrain. A brain metastasis was considered "detected" when the VBrain- "predicted" contours overlapped with the corresponding physician contours ("ground-truth" contours). We evaluated performance of VBrain against ground-truth contours using the following metrics: lesion-wise Dice similarity coefficient (DSC), lesion-wise average Hausdorff distance (AVD), false positive count (FP), and lesion-wise sensitivity (%). Kruskal-Wallis tests were performed to assess the relationships between patient characteristics including sex, race, primary histology, age, and size and number of brain metastases, and performance metrics such as DSC, AVD, FP, and sensitivity.

RESULTS

We analyzed 100 patients with 435 intact brain metastases treated with SRS. Our cohort consisted of patients with a median number of 2 brain metastases (range: 1 to 52), median age of 69 (range: 19 to 91), and 50% male and 50% female patients. The primary site breakdown was 56% lung, 10% melanoma, 9% breast, 8% gynecological, 5% renal, 4% gastrointestinal, 2% sarcoma, and 6% other, while the race breakdown was 60% White, 18% Asian, 3% Black/African American, 2% Native Hawaiian or other Pacific Islander, and 17% other/unknown/not reported. The median tumor size was 0.112 c.c. (range: 0.010-26.475 c.c.). We found mean lesion-wise DSC to be 0.723, mean lesion-wise AVD to be 7.34% of lesion size (0.704 mm), mean FP count to be 0.72 tumors per case, and lesion-wise sensitivity to be 89.30% for all lesions. Moreover, mean sensitivity was found to be 99.07%, 97.59%, and 96.23% for lesions with diameter equal to and greater than 10 mm, 7.5 mm, and 5 mm, respectively. No other significant differences in performance metrics were observed across demographic or clinical characteristic groups.

CONCLUSION

In this study, a commercial deep learning algorithm showed promising results in segmenting brain metastases, with 96.23% sensitivity for metastases with diameters of 5 mm or higher. As the software is an assistive AI, future work of VBrain integration into the clinical workflow can provide further clinical and research insights.

Deep Learning in MRI-guided Radiation Therapy: A Systematic Review

MRI-guided radiation therapy (MRgRT) offers a precise and adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed. MRI-guided radiation therapy offers a precise, adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed with emphasis placed on underlying methods. Studies are further categorized into the areas of segmentation, synthesis, radiomics, and real time MRI. Finally, clinical implications, current challenges, and future directions are discussed.

Correlating volumetric and linear measurements of brain metastases on MRI scans using intelligent automation software: a preliminary study

PURPOSE

The Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) working group proposed a guide for treatment responses for BMs by utilizing the longest diameter; however, despite recognizing that many patients with BMs have sub-centimeter lesions, the group referred to these lesions as unmeasurable due to issues with repeatability and interpretation. In light of RANO-BM recommendations, we aimed to correlate linear and volumetric measurements in sub-centimeter BMs on contrast-enhanced MRI using intelligent automation software.

METHODS

In this retrospective study, patients with BMs scanned with MRI between January 1, 2018, and December 31, 2021, were screened. Inclusion criteria were: (1) at least one sub-centimeter BM with an integer millimeter-longest diameter was noted in the MRI report; (2) patients were a minimum of 18 years of age; (3) patients with available pre-treatment three-dimensional T1-weighted spoiled gradient-echo MRI scan. The screening was terminated when there were 20 lesions in each group. Lesion volumes were measured with the help of intelligent automation software Jazz (AI Medical, Zollikon, Switzerland) by two readers. The Kruskal-Wallis test was used to compare volumetric differences.

RESULTS

Our study included 180 patients. The agreement for volumetric measurements was excellent between the two readers. The volumes of the following groups were not significantly different: 1-2 mm, 1-3 mm, 1-4 mm, 2-3 mm, 2-4 mm, 3-4 mm, 3-5 mm, 4-5 mm, 5-6 mm, 5-7 mm, 6-7 mm, 6-8 mm, 6-9 mm, 7-8 mm, 7-9 mm, 8-9 mm.

CONCLUSION

Our findings indicate that the largest diameter of a lesion may not accurately represent its volume. Additional research is required to determine which method is superior for measuring radiologic response to therapy and which parameter correlates best with clinical improvement or deterioration.

DeSeg: auto detector-based segmentation for brain metastases

Delineation of brain metastases (BMs) is a paramount step in stereotactic radiosurgery treatment. Clinical practice has specific expectation on BM auto-delineation that the method is supposed to avoid missing of small lesions and yield accurate contours for large lesions. In this study, we propose a novel coarse-to-fine framework, named detector-based segmentation (DeSeg), to incorporate object-level detection into pixel-wise segmentation so as to meet the clinical demand. DeSeg consists of three components: a center-point-guided single-shot detector to localize the potential lesion regions, a multi-head U-Net segmentation model to refine contours, and a data cascade unit to connect both tasks smoothly. Performance on tiny lesions is measured by the object-based sensitivity and positive predictive value (PPV), while that on large lesions is quantified by dice similarity coefficient (DSC), average symmetric surface distance (ASSD) and 95% Hausdorff distance (HD95). Besides, computational complexity is also considered to study the potential of method in real-time processing. This study retrospectively collected 240 BM patients with Gadolinium injected contrast-enhanced T1-weighted magnetic resonance imaging (T1c-MRI), which were randomly split into training, validating and testing datasets (192, 24 and 24 scans, respectively). The lesions in the testing dataset were further divided into two groups based on the volume size (smallS: ≤1.5 cc,N= 88; largeL: > 1.5 cc,N= 15). On average, DeSeg yielded a sensitivity of 0.91 and a PPV of 0.77 on S group, and a DSC of 0.86, an ASSD 0f 0.76 mm and a HD95 of 2.31 mm onLgroup. The results indicated that DeSeg achieved leading sensitivity and PPV for tiny lesions as well as segmentation metrics for large ones. After our clinical validation, DeSeg showed competitive segmentation performance while kept faster processing speed comparing with existing 3D models.

Ensemble learning for glioma patients overall survival prediction using pre-operative MRIs

Objective: Gliomas are the most common primary brain tumors. Approximately 70% of the glioma patients diagnosed with glioblastoma have an averaged overall survival (OS) of only ∼16 months. Early survival prediction is essential for treatment decision-making in glioma patients. Here we proposed an ensemble learning approach to predict the post-operative OS of glioma patients using only pre-operative MRIs.Approach: Our dataset was from the Medical Image Computing and Computer Assisted Intervention Brain Tumor Segmentation challenge 2020, which consists of multimodal pre-operative MRI scans of 235 glioma patients with survival days recorded. The backbone of our approach was a Siamese network consisting of twinned ResNet-based feature extractors followed by a 3-layer classifier. During training, the feature extractors explored traits of intra and inter-class by minimizing contrastive loss of randomly paired 2D pre-operative MRIs, and the classifier utilized the extracted features to generate labels with cost defined by cross-entropy loss. During testing, the extracted features were also utilized to define distance between the test sample and the reference composed of training data, to generate an additional predictor via K-NN classification. The final label was the ensemble classification from both the Siamese model and the K-NN model.Main results: Our approach classifies the glioma patients into 3 OS classes: long-survivors (>15 months), mid-survivors (between 10 and 15 months) and short-survivors (<10 months). The performance is assessed by the accuracy (ACC) and the area under the curve (AUC) of 3-class classification. The final result achieved an ACC of 65.22% and AUC of 0.81.Significance: Our Siamese network based ensemble learning approach demonstrated promising ability in mining discriminative features with minimal manual processing and generalization requirement. This prediction strategy can be potentially applied to assist timely clinical decision-making.

Tumor Microenvironment, Radiology, and Artificial Intelligence: Should We Consider Tumor Periphery?

OBJECTIVES

The tumor microenvironment (TME) consists of cellular and noncellular components which enable the tumor to interact with its surroundings and plays an important role in the tumor progression and how the immune system reacts to the malignancy. In the present study, we investigate the diagnostic potential of the TME in differentiating benign and malignant lesions using image quantification and machine learning.

METHODS

A total of 229 breast lesions and 220 cervical lymph nodes were included in the study. A group of expert radiologists first performed medical imaging and segmented the lesions, after which a rectangular mask was drawn, encompassing all of the contouring. The mask was extended in each axis up to 50%, and 29 radiomics features were extracted from each mask. Radiomics features that showed a significant difference in each contour were used to develop a support vector machine (SVM) classifier for benign and malignant lesions in breast and lymph node images separately.

RESULTS

Single radiomics features extracted from extended contours outperformed radiologists' contours in both breast and lymph node lesions. Furthermore, when fed into the SVM model, the extended models also outperformed the radiologist's contour, achieving an area under the receiver operating characteristic curve of 0.887 and 0.970 in differentiating breast and lymph node lesions, respectively.

CONCLUSIONS

Our results provide convincing evidence regarding the importance of the tumor periphery and TME in medical imaging diagnosis. We propose that the immediate tumor periphery should be considered for differentiating benign and malignant lesions in image quantification studies.