Comparing 3D, 2.5D, and 2D Approaches to Brain Image Auto-Segmentation

Automated segmentation of deep brain structures from Inversion-Recovery MRI

Methods for the automated segmentation of brain structures are a major subject of medical research. The small structures of the deep brain have received scant attention, notably for lack of manual delineations by medical experts. In this study, we assessed an automated segmentation of a novel clinical dataset containing White Matter Attenuated Inversion-Recovery (WAIR) MRI images and five manually segmented structures (substantia nigra (SN), subthalamic nucleus (STN), red nucleus (RN), mammillary body (MB) and mammillothalamic fascicle (MT-fa)) in 53 patients with severe Parkinson's disease. T1 and DTI images were additionally used. We also assessed the reorientation of DTI diffusion vectors with reference to the ACPC line. A state-of-the-art nnU-Net method was trained and tested on subsets of 38 and 15 image datasets respectively. We used Dice similarity coefficient (DSC), 95% Hausdorff distance (95HD), and volumetric similarity (VS) as metrics to evaluate network efficiency in reproducing manual contouring. Random-effects models statistically compared values according to structures, accounting for between- and within-participant variability. Results show that WAIR significantly outperformed T1 for DSC (0.739 ± 0.073), 95HD (1.739 ± 0.398), and VS (0.892 ± 0.044). The DSC values for automated segmentation of MB, RN, SN, STN, and MT-fa decreased in that order, in line with the increasing complexity observed in manual segmentation. Based on training results, the reorientation of DTI vectors improved the automated segmentation.

Deep learning-based segmentation of acute ischemic stroke MRI lesions and recurrence prediction within 1 year after discharge: A multicenter study

OBJECTIVE

To explore the performance of deep learning-based segmentation of infarcted lesions in the brain magnetic resonance imaging (MRI) of patients with acute ischemic stroke (AIS) and the recurrence prediction value of radiomics within 1 year after discharge as well as to develop a model incorporating radiomics features and clinical factors to accurately predict AIS recurrence.

MATERIALS AND METHODS

To generate a segmentation model of MRI lesions in AIS, the deep learning algorithm multiscale residual attention UNet (MRA-UNet) was employed. Furthermore, the risk factors for AIS recurrence within 1 year were explored using logistic regression (LR) analysis. In addition, to develop the prediction model for AIS recurrence within 1 year after discharge, four machine learning algorithms, namely, LR, RandomForest (RF), CatBoost, and XGBoost, were employed based on radiomics data, clinical data, and their combined data.

RESULTS

In the validation set, the Mean Dice (MDice) and Mean IOU (MIou) of the MRA-UNet segmentation model were 0.816 and 0.801, respectively. In multivariate LR analysis, age, renal insufficiency, C-reactive protein, triglyceride glucose index, prognostic nutritional index, and infarct volume were identified as the independent risk factors for AIS recurrence. Furthermore, in the validation set, combining radiomics data and clinical data, the AUC was 0.835 (95%CI:0.738, 0.932), 0.834 (95%CI:0.740, 0.928), 0.858 (95%CI:0.770, 0.946), and 0.842 (95%CI:0.752, 0.932) for the LR, RF, CatBoost, and XGBoost models, respectively.

CONCLUSION

The MRA-UNet model can effectively improve the segmentation accuracy of MRI. The model, which was established by combining radiomics features and clinical factors, held some value for predicting AIS recurrence within 1 year.

Machine learning-based analysis of microfluidic device immobilized C. elegans for automated developmental toxicity testing

Developmental toxicity (DevTox) tests evaluate the adverse effects of chemical exposures on an organism's development. Although current testing primarily relies on large mammalian models, the emergence of new approach methodologies (NAMs) is encouraging industries and regulatory agencies to evaluate novel assays. C. elegans have emerged as NAMs for rapid toxicity testing because of its biological relevance and suitability to high throughput studies. However, current low-resolution and labor-intensive methodologies prohibit its application for sub-lethal DevTox studies at high throughputs. With the recent advent of the large-scale microfluidic device, vivoChip, we can now rapidly collect 3D high-resolution images of ~ 1000 C. elegans from 24 different populations. While data collection is rapid, analyzing thousands of images remains time-consuming. To address this challenge, we developed a machine-learning (ML)-based image analysis platform using a 2.5D U-Net architecture (vivoBodySeg) that accurately segments C. elegans in images obtained from vivoChip devices, achieving a Dice score of 97.80%. vivoBodySeg processes 36 GB data per device, phenotyping multiple body parameters within 35 min on a desktop PC. This analysis is ~ 140 × faster than the manual analysis. This ML approach delivers highly reproducible DevTox parameters (4-8% CV) to assess the toxicity of chemicals with high statistical power.

Detection of focal cortical dysplasia: Development and multicentric evaluation of artificial intelligence models

OBJECTIVE

Focal cortical dysplasia (FCD) is a common cause of drug-resistant focal epilepsy but can be challenging to detect visually on magnetic resonance imaging. Three artificial intelligence models for automated FCD detection are publicly available (MAP18, deepFCD, MELD) but have only been compared on single-center data. Our first objective is to compare them on independent multicenter test data. Additionally, we train and compare three new models and make them publicly available.

METHODS

We retrospectively collected FCD cases from four epilepsy centers. We chose three novel models that take two-dimensional (2D) slices (2D-nnUNet), 2.5D slices (FastSurferCNN), and large 3D patches (3D-nnUNet) as inputs and trained them on a subset of Bonn data. As core evaluation metrics, we used voxel-level Dice similarity coefficient (DSC), cluster-level F1 score, subject-level detection rate, and specificity.

RESULTS

We collected 329 subjects, 244 diagnosed with FCD (27.7 ± 14.4 years old, 54% male) and 85 healthy controls (7.1 ± 2.4 years old, 51% female). We used 118 subjects for model training and kept the remaining subjects as an independent test set. 3D-nnUNet achieved the highest F1 score of .58, the highest DSC of .36 (95% confidence interval [CI] = .30-.41), a detection rate of 55%, and a specificity of 86%. deepFCD showed the highest detection rate (82%) but had the lowest specificity (0%) and cluster-level precision (.03, 95% CI = .03-.04, F1 score = .07). MELD showed the least performance variation across centers, with detection rates between 46% and 54%.

SIGNIFICANCE

This study shows the variance in performance for FCD detection models in a multicenter dataset. The two models with 3D input data showed the highest sensitivity. The 2D models performed worse than all other models, suggesting that FCD detection requires 3D data. The greatly improved precision of 3D-nnUNet may make it a sensible choice to aid FCD detection.

Deep superpixel generation and clustering for weakly supervised segmentation of brain tumors in MR images

PURPOSE

Training machine learning models to segment tumors and other anomalies in medical images is an important step for developing diagnostic tools but generally requires manually annotated ground truth segmentations, which necessitates significant time and resources. We aim to develop a pipeline that can be trained using readily accessible binary image-level classification labels, to effectively segment regions of interest without requiring ground truth annotations.

METHODS

This work proposes the use of a deep superpixel generation model and a deep superpixel clustering model trained simultaneously to output weakly supervised brain tumor segmentations. The superpixel generation model's output is selected and clustered together by the superpixel clustering model. Additionally, we train a classifier using binary image-level labels (i.e., labels indicating whether an image contains a tumor), which is used to guide the training by localizing undersegmented seeds as a loss term. The proposed simultaneous use of superpixel generation and clustering models, and the guided localization approach allow for the output weakly supervised tumor segmentations to capture contextual information that is propagated to both models during training, resulting in superpixels that specifically contour the tumors. We evaluate the performance of the pipeline using Dice coefficient and 95% Hausdorff distance (HD95) and compare the performance to state-of-the-art baselines. These baselines include the state-of-the-art weakly supervised segmentation method using both seeds and superpixels (CAM-S), and the Segment Anything Model (SAM).

RESULTS

We used 2D slices of magnetic resonance brain scans from the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 dataset and labels indicating the presence of tumors to train and evaluate the pipeline. On an external test cohort from the BraTS 2023 dataset, our method achieved a mean Dice coefficient of 0.745 and a mean HD95 of 20.8, outperforming all baselines, including CAM-S and SAM, which resulted in mean Dice coefficients of 0.646 and 0.641, and mean HD95 of 21.2 and 27.3, respectively.

CONCLUSION

The proposed combination of deep superpixel generation, deep superpixel clustering, and the incorporation of undersegmented seeds as a loss term improves weakly supervised segmentation.

Robust brain MRI image classification with SIBOW-SVM

Primary Central Nervous System tumors in the brain are among the most aggressive diseases affecting humans. Early detection and classification of brain tumor types, whether benign or malignant, glial or non-glial, is critical for cancer prevention and treatment, ultimately improving human life expectancy. Magnetic Resonance Imaging (MRI) is the most effective technique for brain tumor detection, generating comprehensive brain scans. However, human examination can be error-prone and inefficient due to the complexity, size, and location variability of brain tumors. Recently, automated classification techniques using machine learning methods, such as Convolutional Neural Networks (CNNs), have demonstrated significantly higher accuracy than manual screening. However, deep learning-based image classification methods, including CNNs, face challenges in estimating class probabilities without proper model calibration (Guo et al., 2017; Minderer et al., 2021). In this paper, we propose a novel brain tumor image classification method called SIBOW-SVM, which integrates the Bag-of-Features model with SIFT feature extraction and weighted Support Vector Machines. This new approach can effectively extract hidden image features, enabling differentiation of various tumor types, provide accurate label predictions, and estimate probabilities of images belonging to each class, offering high-confidence classification decisions. We have also developed scalable and parallelable algorithms to facilitate the practical implementation of SIBOW-SVM for massive image datasets. To benchmark our method, we apply SIBOW-SVM to a public dataset of brain tumor MRI images containing four classes: glioma, meningioma, pituitary, and normal. Our results demonstrate that the new method outperforms state-of-the-art techniques, including CNNs, in terms of uncertainty quantification, classification accuracy, computational efficiency, and data robustness.

Less is More: Selective reduction of CT data for self-supervised pre-training of deep learning models with contrastive learning improves downstream classification performance

BACKGROUND

Self-supervised pre-training of deep learning models with contrastive learning is a widely used technique in image analysis. Current findings indicate a strong potential for contrastive pre-training on medical images. However, further research is necessary to incorporate the particular characteristics of these images.

METHOD

We hypothesize that the similarity of medical images hinders the success of contrastive learning in the medical imaging domain. To this end, we investigate different strategies based on deep embedding, information theory, and hashing in order to identify and reduce redundancy in medical pre-training datasets. The effect of these different reduction strategies on contrastive learning is evaluated on two pre-training datasets and several downstream classification tasks.

RESULTS

In all of our experiments, dataset reduction leads to a considerable performance gain in downstream tasks, e.g., an AUC score improvement from 0.78 to 0.83 for the COVID CT Classification Grand Challenge, 0.97 to 0.98 for the OrganSMNIST Classification Challenge and 0.73 to 0.83 for a brain hemorrhage classification task. Furthermore, pre-training is up to nine times faster due to the dataset reduction.

CONCLUSIONS

In conclusion, the proposed approach highlights the importance of dataset quality and provides a transferable approach to improve contrastive pre-training for classification downstream tasks on medical images.

2.5D deep learning based on multi-parameter MRI to differentiate primary lung cancer pathological subtypes in patients with brain metastases

BACKGROUND

Brain metastases (BMs) represents a severe neurological complication stemming from cancers originating from various sources. It is a highly challenging clinical task to accurately distinguish the pathological subtypes of brain metastatic tumors from lung cancer (LC).The utility of 2.5-dimensional (2.5D) deep learning (DL) in distinguishing pathological subtypes of LC with BMs is yet to be determined.

METHODS

A total of 250 patients were included in this retrospective study, divided in a 7:3 ratio into training set (N=175) and testing set (N=75). We devised a method to assemble a series of two-dimensional (2D) images by extracting adjacent slices from a central slice in both superior-inferior and anterior-posterior directions to form a 2.5D dataset. Multi-Instance learning (MIL) is a weakly supervised learning method that organizes training instances into "bags" and provides labels for entire bags, with the purpose of learning a classifier based on the labeled positive and negative bags to predict the corresponding class for an unknown bag. Therefore, we employed MIL to construct a comprehensive 2.5D feature set. Then we used the single-slice as input for constructing the 2D model. DL features were extracted from these slices using the pre-trained ResNet101. All feature sets were inputted into the support vector machine (SVM) for evaluation. The diagnostic performance of the classification models were evaluated using five-fold cross-validation, with accuracy and area under the curve (AUC) metrics calculated for analysis.

RESULTS

The optimal performance was obtained using the 2.5D DL model, which achieved the micro-AUC of 0.868 (95% confidence interval [CI], 0.817-0.919) and accuracy of 0.836 in the test cohort. The 2D model achieved the micro-AUC of 0.836 (95 % CI, 0.778-0.894) and accuracy of 0.827 in the test cohort.

CONCLUSIONS

The proposed 2.5D DL model is feasible and effective in identifying pathological subtypes of BMs from lung cancer.

OpenMAP-T1: A Rapid Deep-Learning Approach to Parcellate 280 Anatomical Regions to Cover the Whole Brain

This study introduces OpenMAP-T1, a deep-learning-based method for rapid and accurate whole-brain parcellation in T1- weighted brain MRI, which aims to overcome the limitations of conventional normalization-to-atlas-based approaches and multi-atlas label-fusion (MALF) techniques. Brain image parcellation is a fundamental process in neuroscientific and clinical research, enabling a detailed analysis of specific cerebral regions. Normalization-to-atlas-based methods have been employed for this task, but they face limitations due to variations in brain morphology, especially in pathological conditions. The MALF techniques improved the accuracy of the image parcellation and robustness to variations in brain morphology, but at the cost of high computational demand that requires a lengthy processing time. OpenMAP-T1 integrates several convolutional neural network models across six phases: preprocessing; cropping; skull-stripping; parcellation; hemisphere segmentation; and final merging. This process involves standardizing MRI images, isolating the brain tissue, and parcellating it into 280 anatomical structures that cover the whole brain, including detailed gray and white matter structures, while simplifying the parcellation processes and incorporating robust training to handle various scan types and conditions. The OpenMAP-T1 was validated on the Johns Hopkins University atlas library and eight available open resources, including real-world clinical images, and the demonstration of robustness across different datasets with variations in scanner types, magnetic field strengths, and image processing techniques, such as defacing. Compared with existing methods, OpenMAP-T1 significantly reduced the processing time per image from several hours to less than 90 s without compromising accuracy. It was particularly effective in handling images with intensity inhomogeneity and varying head positions, conditions commonly seen in clinical settings. The adaptability of OpenMAP-T1 to a wide range of MRI datasets and its robustness to various scan conditions highlight its potential as a versatile tool in neuroimaging.

Combining 2.5D deep learning and conventional features in a joint model for the early detection of sICH expansion

The study aims to investigate the potential of training efficient deep learning models by using 2.5D (2.5-Dimension) masks of sICH. Furthermore, it intends to evaluate and compare the predictive performance of a joint model incorporating four types of features with standalone 2.5D deep learning, radiomics, radiology, and clinical models for early expansion in sICH. A total of 254 sICH patients were enrolled retrospectively and divided into two groups according to whether the hematoma was enlarged or not. The 2.5D mask of sICH is constructed with the maximum axial, coronal and sagittal planes of the hematoma, which is used to train the deep learning model and extract deep learning features. Predictive models were built on clinic, radiology, radiomics and deep learning features separately and four type features jointly. The diagnostic performance of each model was measured using the receiver operating characteristic curve (AUC), Accuracy, Recall, F1 and decision curve analysis (DCA). The AUCs of the clinic model, radiology model, radiomics model, deep learning model, joint model, and nomogram model on the train set (training and Cross-validation) were 0.639, 0.682, 0.859, 0.807, 0.939, and 0.942, respectively, while the AUCs on the test set (external validation) were 0.680, 0.758, 0.802, 0.857, 0.929, and 0.926. Decision curve analysis showed that the joint model was superior to the other models and demonstrated good consistency between the predicted probability of early hematoma expansion and the actual occurrence probability. Our study demonstrates that the joint model is a more efficient and robust prediction model, as verified by multicenter data. This finding highlights the potential clinical utility of a multifactorial prediction model that integrates various data sources for prognostication in patients with intracerebral hemorrhage. The Critical Relevance Statement: Combining 2.5D deep learning features with clinic features, radiology markers, and radiomics signatures to establish a joint model enabling physicians to conduct better-individualized assessments the risk of early expansion of sICH.

Deep learning-based segmentation for high-dose-rate brachytherapy in cervical cancer using 3D Prompt-ResUNet

Objective.To develop and evaluate a 3D Prompt-ResUNet module that utilized the prompt-based model combined with 3D nnUNet for rapid and consistent autosegmentation of high-risk clinical target volume (HRCTV) and organ at risk (OAR) in high-dose-rate brachytherapy for cervical cancer patients.Approach.We used 73 computed tomography scans and 62 magnetic resonance imaging scans from 135 (103 for training, 16 for validation, and 16 for testing) cervical cancer patients across two hospitals for HRCTV and OAR segmentation. A novel comparison of the deep learning neural networks 3D Prompt-ResUNet, nnUNet, and segment anything model-Med3D was applied for the segmentation. Evaluation was conducted in two parts: geometric and clinical assessments. Quantitative metrics included the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95%), Jaccard index (JI), and Matthews correlation coefficient (MCC). Clinical evaluation involved interobserver comparison, 4-grade expert scoring, and a double-blinded Turing test.Main results.The Prompt-ResUNet model performed most similarly to experienced radiation oncologists, outperforming less experienced ones. During testing, the DSC, HD95% (mm), JI, and MCC value (mean ± SD) for HRCTV were 0.92 ± 0.03, 2.91 ± 0.69, 0.85 ± 0.04, and 0.92 ± 0.02, respectively. For the bladder, these values were 0.93 ± 0.05, 3.07 ± 1.05, 0.87 ± 0.08, and 0.93 ± 0.05, respectively. For the rectum, they were 0.87 ± 0.03, 3.54 ± 1.46, 0.78 ± 0.05, and 0.87 ± 0.03, respectively. For the sigmoid, they were 0.76 ± 0.11, 7.54 ± 5.54, 0.63 ± 0.14, and 0.78 ± 0.09, respectively. The Prompt-ResUNet achieved a clinical viability score of at least 2 in all evaluation cases (100%) for both HRCTV and bladder and exceeded the 30% positive rate benchmark for all evaluated structures in the Turing test.Significance.The Prompt-ResUNet architecture demonstrated high consistency with ground truth in autosegmentation of HRCTV and OARs, reducing interobserver variability and shortening treatment times.

Non-contrasted computed tomography (NCCT) based chronic thromboembolic pulmonary hypertension (CTEPH) automatic diagnosis using cascaded network with multiple instance learning

Objective.The diagnosis of chronic thromboembolic pulmonary hypertension (CTEPH) is challenging due to nonspecific early symptoms, complex diagnostic processes, and small lesion sizes. This study aims to develop an automatic diagnosis method for CTEPH using non-contrasted computed tomography (NCCT) scans, enabling automated diagnosis without precise lesion annotation.Approach.A novel cascade network (CN) with multiple instance learning (CNMIL) framework was developed to improve the diagnosis of CTEPH. This method uses a CN architecture combining two Resnet-18 CNN networks to progressively distinguish between normal and CTEPH cases. Multiple instance learning (MIL) is employed to treat each 3D CT case as a 'bag' of image slices, using attention scoring to identify the most important slices. An attention module helps the model focus on diagnostically relevant regions within each slice. The dataset comprised NCCT scans from 300 subjects, including 117 males and 183 females, with an average age of 52.5 ± 20.9 years, consisting of 132 normal cases and 168 cases of lung diseases, including 88 cases of CTEPH. The CNMIL framework was evaluated using sensitivity, specificity, and the area under the curve (AUC) metrics, and compared with common 3D supervised classification networks and existing CTEPH automatic diagnosis networks.Main results. The CNMIL framework demonstrated high diagnostic performance, achieving an AUC of 0.807, accuracy of 0.833, sensitivity of 0.795, and specificity of 0.849 in distinguishing CTEPH cases. Ablation studies revealed that integrating MIL and the CN significantly enhanced performance, with the model achieving an AUC of 0.978 and perfect sensitivity (1.000) in normal classification. Comparisons with other 3D network architectures confirmed that the integrated model outperformed others, achieving the highest AUC of 0.8419.Significance. The CNMIL network requires no additional scans or annotations, relying solely on NCCT. This approach can improve timely and accurate CTEPH detection, resulting in better patient outcomes.

vivoBodySeg: Machine learning-based analysis of C. elegans immobilized in vivoChip for automated developmental toxicity testing

Developmental toxicity (DevTox) tests evaluate the adverse effects of chemical exposures on an organism's development. While large animal tests are currently heavily relied on, the development of new approach methodologies (NAMs) is encouraging industries and regulatory agencies to evaluate these novel assays. Several practical advantages have made C. elegansa useful model for rapid toxicity testing and studying developmental biology. Although the potential to study DevTox is promising, current low-resolution and labor-intensive methodologies prohibit the use of C. elegans for sub-lethal DevTox studies at high throughputs. With the recent availability of a large-scale microfluidic device, vivoChip, we can now rapidly collect 3D high-resolution images of ~ 1,000 C. elegans from 24 different populations. In this paper, we demonstrate DevTox studies using a 2.5D U-Net architecture (vivoBodySeg) that can precisely segment C. elegans in images obtained from vivoChip devices, achieving an average Dice score of 97.80. The fully automated platform can analyze 36 GB data from each device to phenotype multiple body parameters within 35 min on a desktop PC at speeds ~ 140x faster than the manual analysis. Highly reproducible DevTox parameters (4-8% CV) and additional autofluorescence-based phenotypes allow us to assess the toxicity of chemicals with high statistical power.

A 2.5D Self-Training Strategy for Carotid Artery Segmentation in T1-Weighted Brain Magnetic Resonance Images

Precise annotations for large medical image datasets can be time-consuming. Additionally, when dealing with volumetric regions of interest, it is typical to apply segmentation techniques on 2D slices, compromising important information for accurately segmenting 3D structures. This study presents a deep learning pipeline that simultaneously tackles both challenges. Firstly, to streamline the annotation process, we employ a semi-automatic segmentation approach using bounding boxes as masks, which is less time-consuming than pixel-level delineation. Subsequently, recursive self-training is utilized to enhance annotation quality. Finally, a 2.5D segmentation technique is adopted, wherein a slice of a volumetric image is segmented using a pseudo-RGB image. The pipeline was applied to segment the carotid artery tree in T1-weighted brain magnetic resonance images. Utilizing 42 volumetric non-contrast T1-weighted brain scans from four datasets, we delineated bounding boxes around the carotid arteries in the axial slices. Pseudo-RGB images were generated from these slices, and recursive segmentation was conducted using a Res-Unet-based neural network architecture. The model's performance was tested on a separate dataset, with ground truth annotations provided by a radiologist. After recursive training, we achieved an Intersection over Union (IoU) score of (0.68 ± 0.08) on the unseen dataset, demonstrating commendable qualitative results.

A deep learning-based 3D Prompt-nnUnet model for automatic segmentation in brachytherapy of postoperative endometrial carcinoma

PURPOSE

To create and evaluate a three-dimensional (3D) Prompt-nnUnet module that utilizes the prompts-based model combined with 3D nnUnet for producing the rapid and consistent autosegmentation of high-risk clinical target volume (HR CTV) and organ at risk (OAR) in high-dose-rate brachytherapy (HDR BT) for patients with postoperative endometrial carcinoma (EC).

METHODS AND MATERIALS

On two experimental batches, a total of 321 computed tomography (CT) scans were obtained for HR CTV segmentation from 321 patients with EC, and 125 CT scans for OARs segmentation from 125 patients. The numbers of training/validation/test were 257/32/32 and 87/13/25 for HR CTV and OARs respectively. A novel comparison of the deep learning neural network 3D Prompt-nnUnet and 3D nnUnet was applied for HR CTV and OARs segmentation. Three-fold cross validation and several quantitative metrics were employed, including Dice similarity coefficient (DSC), Hausdorff distance (HD), 95th percentile of Hausdorff distance (HD95%), and intersection over union (IoU).

RESULTS

The Prompt-nnUnet included two forms of parameters Predict-Prompt (PP) and Label-Prompt (LP), with the LP performing most similarly to the experienced radiation oncologist and outperforming the less experienced ones. During the testing phase, the mean DSC values for the LP were 0.96 ± 0.02, 0.91 ± 0.02, and 0.83 ± 0.07 for HR CTV, rectum and urethra, respectively. The mean HD values (mm) were 2.73 ± 0.95, 8.18 ± 4.84, and 2.11 ± 0.50, respectively. The mean HD95% values (mm) were 1.66 ± 1.11, 3.07 ± 0.94, and 1.35 ± 0.55, respectively. The mean IoUs were 0.92 ± 0.04, 0.84 ± 0.03, and 0.71 ± 0.09, respectively. A delineation time < 2.35 s per structure in the new model was observed, which was available to save clinician time.

CONCLUSION

The Prompt-nnUnet architecture, particularly the LP, was highly consistent with ground truth (GT) in HR CTV or OAR autosegmentation, reducing interobserver variability and shortening treatment time.

An N-Shaped Lightweight Network with a Feature Pyramid and Hybrid Attention for Brain Tumor Segmentation

Brain tumor segmentation using neural networks presents challenges in accurately capturing diverse tumor shapes and sizes while maintaining real-time performance. Additionally, addressing class imbalance is crucial for achieving accurate clinical results. To tackle these issues, this study proposes a novel N-shaped lightweight network that combines multiple feature pyramid paths and U-Net architectures. Furthermore, we ingeniously integrate hybrid attention mechanisms into various locations of depth-wise separable convolution module to improve efficiency, with channel attention found to be the most effective for skip connections in the proposed network. Moreover, we introduce a combination loss function that incorporates a newly designed weighted cross-entropy loss and dice loss to effectively tackle the issue of class imbalance. Extensive experiments are conducted on four publicly available datasets, i.e., UCSF-PDGM, BraTS 2021, BraTS 2019, and MSD Task 01 to evaluate the performance of different methods. The results demonstrate that the proposed network achieves superior segmentation accuracy compared to state-of-the-art methods. The proposed network not only improves the overall segmentation performance but also provides a favorable computational efficiency, making it a promising approach for clinical applications.

Self-supervised pre-training with contrastive and masked autoencoder methods for dealing with small datasets in deep learning for medical imaging

Deep learning in medical imaging has the potential to minimize the risk of diagnostic errors, reduce radiologist workload, and accelerate diagnosis. Training such deep learning models requires large and accurate datasets, with annotations for all training samples. However, in the medical imaging domain, annotated datasets for specific tasks are often small due to the high complexity of annotations, limited access, or the rarity of diseases. To address this challenge, deep learning models can be pre-trained on large image datasets without annotations using methods from the field of self-supervised learning. After pre-training, small annotated datasets are sufficient to fine-tune the models for a specific task. The most popular self-supervised pre-training approaches in medical imaging are based on contrastive learning. However, recent studies in natural image processing indicate a strong potential for masked autoencoder approaches. Our work compares state-of-the-art contrastive learning methods with the recently introduced masked autoencoder approach "SparK" for convolutional neural networks (CNNs) on medical images. Therefore, we pre-train on a large unannotated CT image dataset and fine-tune on several CT classification tasks. Due to the challenge of obtaining sufficient annotated training data in medical imaging, it is of particular interest to evaluate how the self-supervised pre-training methods perform when fine-tuning on small datasets. By experimenting with gradually reducing the training dataset size for fine-tuning, we find that the reduction has different effects depending on the type of pre-training chosen. The SparK pre-training method is more robust to the training dataset size than the contrastive methods. Based on our results, we propose the SparK pre-training for medical imaging tasks with only small annotated datasets.

Comparison of 2D, 2.5D, and 3D segmentation networks for maxillary sinuses and lesions in CBCT images

BACKGROUND

The purpose of this study was to compare the segmentation performances of the 2D, 2.5D, and 3D networks for maxillary sinuses (MSs) and lesions inside the maxillary sinus (MSL) with variations in sizes, shapes, and locations in cone beam CT (CBCT) images under the same constraint of memory capacity.

METHODS

The 2D, 2.5D, and 3D networks were compared comprehensively for the segmentation of the MS and MSL in CBCT images under the same constraint of memory capacity. MSLs were obtained by subtracting the prediction of the air region of the maxillary sinus (MSA) from that of the MS.

RESULTS

The 2.5D network showed the highest segmentation performances for the MS and MSA compared to the 2D and 3D networks. The performances of the Jaccard coefficient, Dice similarity coefficient, precision, and recall by the 2.5D network of U-net +  + reached 0.947, 0.973, 0.974, and 0.971 for the MS, respectively, and 0.787, 0.875, 0.897, and 0.858 for the MSL, respectively.

CONCLUSIONS

The 2.5D segmentation network demonstrated superior segmentation performance for various MSLs with an ensemble learning approach of combining the predictions from three orthogonal planes.

A survey and taxonomy of 2.5D approaches for lung segmentation and nodule detection in CT images

CAD systems for lung cancer diagnosis and detection can significantly offer unbiased, infatiguable diagnostics with minimal variance, decreasing the mortality rate and the five-year survival rate. Lung segmentation and lung nodule detection are critical steps in the lung cancer CAD system pipeline. Literature on lung segmentation and lung nodule detection mostly comprises techniques that process 3-D volumes or 2-D slices and surveys. However, surveys that highlight 2.5D techniques for lung segmentation and lung nodule detection still need to be included. This paper presents a background and discussion on 2.5D methods to fill this gap. Further, this paper also gives a taxonomy of 2.5D approaches and a detailed description of the 2.5D approaches. Based on the taxonomy, various 2.5D techniques for lung segmentation and lung nodule detection are clustered into these 2.5D approaches, which is followed by possible future work in this direction.

Identification of Origin for Spinal Metastases from MR Images: Comparison Between Radiomics and Deep Learning Methods

OBJECTIVE

To determine whether spinal metastatic lesions originated from lung cancer or from other cancers based on spinal contrast-enhanced T1 (CET1) magnetic resonance (MR) images analyzed using radiomics (RAD) and deep learning (DL) methods.

METHODS

We recruited and retrospectively reviewed 173 patients diagnosed with spinal metastases at two different centers between July 2018 and June 2021. Of these, 68 involved lung cancer and 105 were other types of cancer. They were assigned to an internal cohort of 149 patients, randomly divided into a training set and a validation set, and to an external cohort of 24 patients. All patients underwent CET1-MR imaging before surgery or biopsy. We developed two predictive algorithms: a DL model and a RAD model. We compared performance between models, and against human radiological assessment, via accuracy (ACC) and receiver operating characteristic (ROC) analyses. Furthermore, we analyzed the correlation between RAD and DL features.

RESULTS

The DL model outperformed RAD model across the board, with ACC/ area under the receiver operating characteristic curve (AUC) values of 0.93/0.94 (DL) versus 0.84/0.93 (RAD) when applied to the training set from the internal cohort, 0.74/0.76 versus 0.72/0.75 when applied to the validation set, and 0.72/0.76 versus 0.69/0.72 when applied to the external test cohort. For the validation set, it also outperformed expert radiological assessment (ACC: 0.65, AUC: 0.68). We only found weak correlations between DL and RAD features.

CONCLUSION

The DL algorithm successfully identified the origin of spinal metastases from pre-operative CET1-MR images, outperforming both RAD models and expert assessment by trained radiologists.

3D Capsule Networks for Brain Image Segmentation

BACKGROUND AND PURPOSE

Current autosegmentation models such as UNets and nnUNets have limitations, including the inability to segment images that are not represented during training and lack of computational efficiency. 3D capsule networks have the potential to address these limitations.

MATERIALS AND METHODS

We used 3430 brain MRIs, acquired in a multi-institutional study, to train and validate our models. We compared our capsule network with standard alternatives, UNets and nnUNets, on the basis of segmentation efficacy (Dice scores), segmentation performance when the image is not well-represented in the training data, performance when the training data are limited, and computational efficiency including required memory and computational speed.

RESULTS

The capsule network segmented the third ventricle, thalamus, and hippocampus with Dice scores of 95%, 94%, and 92%, respectively, which were within 1% of the Dice scores of UNets and nnUNets. The capsule network significantly outperformed UNets in segmenting images that were not well-represented in the training data, with Dice scores 30% higher. The computational memory required for the capsule network is less than one-tenth of the memory required for UNets or nnUNets. The capsule network is also >25% faster to train compared with UNet and nnUNet.

CONCLUSIONS

We developed and validated a capsule network that is effective in segmenting brain images, can segment images that are not well-represented in the training data, and is computationally efficient compared with alternatives.

Comparing Detection Schemes for Adversarial Images against Deep Learning Models for Cancer Imaging

Deep learning (DL) models have demonstrated state-of-the-art performance in the classification of diagnostic imaging in oncology. However, DL models for medical images can be compromised by adversarial images, where pixel values of input images are manipulated to deceive the DL model. To address this limitation, our study investigates the detectability of adversarial images in oncology using multiple detection schemes. Experiments were conducted on thoracic computed tomography (CT) scans, mammography, and brain magnetic resonance imaging (MRI). For each dataset we trained a convolutional neural network to classify the presence or absence of malignancy. We trained five DL and machine learning (ML)-based detection models and tested their performance in detecting adversarial images. Adversarial images generated using projected gradient descent (PGD) with a perturbation size of 0.004 were detected by the ResNet detection model with an accuracy of 100% for CT, 100% for mammogram, and 90.0% for MRI. Overall, adversarial images were detected with high accuracy in settings where adversarial perturbation was above set thresholds. Adversarial detection should be considered alongside adversarial training as a defense technique to protect DL models for cancer imaging classification from the threat of adversarial images.