Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation

Age-specific optimization of the T2-weighted MRI contrast in infant and toddler brain

PURPOSE

In 0-2-year-old brains, the T2-weighted (T2w) contrast between white matter (WM) and gray matter (GM) is weaker compared with that in adult brains and rapidly changes with age. This study aims to design variable-flip-angle (VFA) trains in 3D fast spin-echo sequence that adapt to the dynamically changing relaxation times to improve the contrast in the T2w images of the developing brains.

METHODS

T1 and T2 relaxation times in 0-2-year-old brains were measured, and several age groups were defined according to the age-dependent pattern of T2 values. Based on the static pseudo-steady-state theory and the extended phase graph algorithm, VFA trains were designed for each age group to maximize WM/GM contrast, constrained by the maximum specific absorption rate and overall signal intensity. The optimized VFA trains were compared with the default one used for adult brains based on the relative contrast between WM and GM. Dice coefficient was used to demonstrate the advantage of contrast-improved images as inputs for automatic tissue segmentation in infant brains.

RESULTS

The 0-2-year-old pool was divided into groups of 0-8 months, 8-12 months, and 12-24 months. The optimal VFA trains were tested in each age group in comparison with the default sequence. Quantitative analyses demonstrated improved relative contrasts in infant and toddler brains by 1.5-2.3-fold at different ages. The Dice coefficient for contrast-optimized images was improved compared with default images (p < 0.001).

CONCLUSION

An effective strategy was proposed to improve the 3D T2w contrast in 0-2-year-old brains.

Convolutional neural network (CNN) configuration using a learning automaton model for neonatal brain image segmentation

CNN is considered an efficient tool in brain image segmentation. However, neonatal brain images require specific methods due to their nature and structural differences from adult brain images. Hence, it is necessary to determine the optimal structure and parameters for these models to achieve the desired results. In this article, an adaptive method for CNN automatic configuration for neonatal brain image segmentation is presented based on the encoder-decoder structure, in which the hyperparameters of this network, i.e., size, length, and width of the filter in each layer along with the type of pooling functions with a reinforcement learning approach and an LA model are determined. These LA models determine the optimal configuration for the CNN model by using DICE and ASD segmentation quality evaluation criteria, so that the segmentation quality can be maximized based on the goal criteria. The effectiveness of the proposed method has been evaluated using a database of infant MRI images and the results have been compared with previous methods. The results show that by using the proposed method, it is possible to segment NBI with higher quality and accuracy.

BIBSNet: A Deep Learning Baby Image Brain Segmentation Network for MRI Scans

Objectives

Brain segmentation of infant magnetic resonance (MR) images is vitally important for studying typical and atypical brain development. The infant brain undergoes many changes throughout the first years of postnatal life, making tissue segmentation difficult for most existing algorithms. Here we introduce a deep neural network BIBSNet ( B aby and I nfant B rain S egmentation Neural Net work), an open-source, community-driven model for robust and generalizable brain segmentation leveraging data augmentation and a large sample size of manually annotated images.

Experimental Design

Included in model training and testing were MR brain images from 90 participants with an age range of 0-8 months (median age 4.6 months). Using the BOBs repository of manually annotated real images along with synthetic segmentation images produced using SynthSeg, the model was trained using a 10-fold procedure. Model performance of segmentations was assessed by comparing BIBSNet, joint label fusion (JLF) inferred segmentation to ground truth segmentations using Dice Similarity Coefficient (DSC). Additionally, MR data along with the FreeSurfer compatible segmentations were processed with the DCAN labs infant-ABCD-BIDS processing pipeline from ground truth, JLF, and BIBSNet to further assess model performance on derivative data, including cortical thickness, resting state connectivity and brain region volumes.

Principal Observations

BIBSNet segmentations outperforms JLF across all regions based on DSC comparisons. Additionally, with processed derived metrics, BIBSNet segmentations outperforms JLF segmentations across nearly all metrics.

Conclusions

BIBSNet segmentation shows marked improvement over JLF across all age groups analyzed. The BIBSNet model is 600x faster compared to JLF, produces FreeSurfer-compatible segmentation labels, and can be easily included in other processing pipelines. BIBSNet provides a viable alternative for segmenting the brain in the earliest stages of development.

PatchCL-AE: Anomaly detection for medical images using patch-wise contrastive learning-based auto-encoder

Anomaly detection is an important yet challenging task in medical image analysis. Most anomaly detection methods are based on reconstruction, but the performance of reconstruction-based methods is limited due to over-reliance on pixel-level losses. To address the limitation, we propose a patch-wise contrastive learning-based auto-encoder for medical anomaly detection. The key contribution is the patch-wise contrastive learning loss that provides supervision on local semantics to enforce semantic consistency between corresponding input-output patches. Contrastive learning pulls corresponding patch pairs closer while pushing non-corresponding ones apart between input and output, enabling the model to learn local normal features better and improve discriminability on anomalous regions. Additionally, we design an anomaly score based on local semantic discrepancies to pinpoint abnormalities by comparing feature difference rather than pixel variations. Extensive experiments on three public datasets (i.e., brain MRI, retinal OCT, and chest X-ray) achieve state-of-the-art performance, with our method achieving over 99% AUC on retinal and brain images. Both the contrastive patch-wise supervision and patch-discrepancy score provide targeted advancements to overcome the weaknesses in existing approaches.

VINNA for neonates: Orientation independence through latent augmentations

A robust, fast, and accurate segmentation of neonatal brain images is highly desired to better understand and detect changes during development and disease, specifically considering the rise in imaging studies for this cohort. Yet, the limited availability of ground truth datasets, lack of standardized acquisition protocols, and wide variations of head positioning in the scanner pose challenges for method development. A few automated image analysis pipelines exist for newborn brain Magnetic Resonance Image (MRI) segmentation, but they often rely on time-consuming non-linear spatial registration procedures and require resampling to a common resolution, subject to loss of information due to interpolation and down-sampling. Without registration and image resampling, variations with respect to head positions and voxel resolutions have to be addressed differently. In deep learning, external augmentations such as rotation, translation, and scaling are traditionally used to artificially expand the representation of spatial variability, which subsequently increases both the training dataset size and robustness. However, these transformations in the image space still require resampling, reducing accuracy specifically in the context of label interpolation. We recently introduced the concept of resolution-independence with the Voxel-size Independent Neural Network framework, VINN. Here, we extend this concept by additionally shifting all rigid-transforms into the network architecture with a four degree of freedom (4-DOF) transform module, enabling resolution-aware internal augmentations (VINNA) for deep learning. In this work, we show that VINNA (i) significantly outperforms state-of-the-art external augmentation approaches, (ii) effectively addresses the head variations present specifically in newborn datasets, and (iii) retains high segmentation accuracy across a range of resolutions (0.5-1.0 mm). Furthermore, the 4-DOF transform module together with internal augmentations is a powerful, general approach to implement spatial augmentation without requiring image or label interpolation. The specific network application to newborns will be made publicly available as VINNA4neonates.

Glioma segmentation based on dense contrastive learning and multimodal features recalibration

Accurate segmentation of different regions of gliomas from multimodal magnetic resonance (MR) images is crucial for glioma grading and precise diagnosis, but many existing segmentation methods are difficult to effectively utilize multimodal MR image information to recognize accurately the lesion regions with small size, low contrast and irregular shape. To address this issue, this work proposes a novel 3D glioma segmentation model DCL-MANet. DCL-MANet has an architecture of multiple encoders and one single decoder. Each encoder is used to extract MR image features of a given modality. To overcome the entangle problems of multimodal semantic features, a dense contrastive learning (DCL) strategy is presented to extract the modality-specific and common features. Following that, feature recalibration block (RFB) based on modality-wise attention is used to recalibrate the semantic features of each modality, enabling the model to focus on the features that are beneficial for glioma segmentation. These recalibrated features are input into the decoder to obtain the segmentation results. To verify the superiority of the proposed method, we compare it with several state-of-the-art (SOTA) methods in terms of Dice, average symmetric surface distance (ASSD), HD95 and volumetric similarity (Vs). The comparison results show that the average Dice, ASSD, HD95 and Vs of DCL-MANet on all tumor regions are improved at least by 0.66%, 3.47%, 8.94% and 1.07% respectively. For small enhance tumor (ET) region, the corresponding improvement can be up to 0.37%, 7.83%, 11.32%, and 1.35%, respectively. In addition, the ablation results demonstrate the effectiveness of the proposed DCL and RFB, and combining them can significantly increase Dice (1.59%) and Vs (1.54%) while decreasing ASSD (40.51%) and HD95 (45.16%) on ET region. The proposed DCL-MANet could disentangle multimodal features and enhance the semantics of modality-dependent features, providing a potential means to accurately segment small lesion regions in gliomas.

Deep learning techniques for isointense infant brain tissue segmentation: a systematic literature review

Introduction

To improve comprehension of initial brain growth in wellness along with sickness, it is essential to precisely segment child brain magnetic resonance imaging (MRI) into white matter (WM) and gray matter (GM), along with cerebrospinal fluid (CSF). Nonetheless, in the isointense phase (6-8 months of age), the inborn myelination and development activities, WM along with GM display alike stages of intensity in both T1-weighted and T2-weighted MRI, making tissue segmentation extremely difficult.

Methods

The comprehensive review of studies related to isointense brain MRI segmentation approaches is highlighted in this publication. The main aim and contribution of this study is to aid researchers by providing a thorough review to make their search for isointense brain MRI segmentation easier. The systematic literature review is performed from four points of reference: (1) review of studies concerning isointense brain MRI segmentation; (2) research contribution and future works and limitations; (3) frequently applied evaluation metrics and datasets; (4) findings of this studies.

Results and discussion

The systemic review is performed on studies that were published in the period of 2012 to 2022. A total of 19 primary studies of isointense brain MRI segmentation were selected to report the research question stated in this review.

Deep Ensembles Are Robust to Occasional Catastrophic Failures of Individual DNNs for Organs Segmentations in CT Images

Deep neural networks (DNNs) have recently showed remarkable performance in various computer vision tasks, including classification and segmentation of medical images. Deep ensembles (an aggregated prediction of multiple DNNs) were shown to improve a DNN's performance in various classification tasks. Here we explore how deep ensembles perform in the image segmentation task, in particular, organ segmentations in CT (Computed Tomography) images. Ensembles of V-Nets were trained to segment multiple organs using several in-house and publicly available clinical studies. The ensembles segmentations were tested on images from a different set of studies, and the effects of ensemble size as well as other ensemble parameters were explored for various organs. Compared to single models, Deep Ensembles significantly improved the average segmentation accuracy, especially for those organs where the accuracy was lower. More importantly, Deep Ensembles strongly reduced occasional "catastrophic" segmentation failures characteristic of single models and variability of the segmentation accuracy from image to image. To quantify this we defined the "high risk images": images for which at least one model produced an outlier metric (performed in the lower 5% percentile). These images comprised about 12% of the test images across all organs. Ensembles performed without outliers for 68%-100% of the "high risk images" depending on the performance metric used.

An automatic and accurate deep learning-based neuroimaging pipeline for the neonatal brain

BACKGROUND

Accurate segmentation of neonatal brain tissues and structures is crucial for studying normal development and diagnosing early neurodevelopmental disorders. However, there is a lack of an end-to-end pipeline for automated segmentation and imaging analysis of the normal and abnormal neonatal brain.

OBJECTIVE

To develop and validate a deep learning-based pipeline for neonatal brain segmentation and analysis of structural magnetic resonance images (MRI).

MATERIALS AND METHODS

Two cohorts were enrolled in the study, including cohort 1 (582 neonates from the developing Human Connectome Project) and cohort 2 (37 neonates imaged using a 3.0-tesla MRI scanner in our hospital).We developed a deep leaning-based architecture capable of brain segmentation into 9 tissues and 87 structures. Then, extensive validations were performed for accuracy, effectiveness, robustness and generality of the pipeline. Furthermore, regional volume and cortical surface estimation were measured through in-house bash script implemented in FSL (Oxford Centre for Functional MRI of the Brain Software Library) to ensure reliability of the pipeline. Dice similarity score (DSC), the 95th percentile Hausdorff distance (H95) and intraclass correlation coefficient (ICC) were calculated to assess the quality of our pipeline. Finally, we finetuned and validated our pipeline on 2-dimensional thick-slice MRI in cohorts 1 and 2.

RESULTS

The deep learning-based model showed excellent performance for neonatal brain tissue and structural segmentation, with the best DSC and the 95th percentile Hausdorff distance (H95) of 0.96 and 0.99 mm, respectively. In terms of regional volume and cortical surface analysis, our model showed good agreement with ground truth. The ICC values for the regional volume were all above 0.80. Considering the thick-slice image pipeline, the same trend was observed for brain segmentation and analysis. The best DSC and H95 were 0.92 and 3.00 mm, respectively. The regional volumes and surface curvature had ICC values just below 0.80.

CONCLUSIONS

We propose an automatic, accurate, stable and reliable pipeline for neonatal brain segmentation and analysis from thin and thick structural MRI. The external validation showed very good reproducibility of the pipeline.

An efficient memory reserving-and-fading strategy for vector quantization based 3D brain segmentation and tumor extraction using an unsupervised deep learning network

Deep learning networks are state-of-the-art approaches for 3D brain image segmentation, and the radiological characteristics extracted from tumors are of great significance for clinical diagnosis, treatment planning, and treatment outcome evaluation. However, two problems have been the hindering factors in brain image segmentation techniques. One is that deep learning networks require large amounts of manually annotated data. Another issue is the computational efficiency of 3D deep learning networks. In this study, we propose a vector quantization (VQ)-based 3D segmentation method that employs a novel unsupervised 3D deep embedding clustering (3D-DEC) network and an efficiency memory reserving-and-fading strategy. The VQ-based 3D-DEC network is trained on volume data in an unsupervised manner to avoid manual data annotation. The memory reserving-and-fading strategy beefs up model efficiency greatly. The designed methodology makes deep learning-based model feasible for biomedical image segmentation. The experiment is divided into two parts. First, we extensively evaluate the effectiveness and robustness of the proposed model on two authoritative MRI brain tumor databases (i.e., IBSR and BrainWeb). Second, we validate the model using real 3D brain tumor data collected from our institute for clinical practice significance. Results show that our method (without data manual annotation) has superior accuracy (0.74 ± 0.04 Tanimoto coefficient on IBSR, 97.5% TP and 97.7% TN on BrainWeb, and 91% Dice, 88% sensitivity and 87% specificity on real brain data) and remarkable efficiency (speedup ratio is 18.72 on IBSR, 31.16 on BrainWeb, 31.00 on real brain data) compared to the state-of-the-art methods. The results show that our proposed model can address the lacks of manual annotations, and greatly increase computation speedup with competitive segmentation accuracy compared to other state-of-the-art 3D CNN models. Moreover, the proposed model can be used for tumor treatment follow-ups every 6 months, providing critical details for surgical and postoperative treatment by correctly extracting numerical radiomic features of tumors.

An Ensemble of Deep Learning Object Detection Models for Anatomical and Pathological Regions in Brain MRI

This paper proposes ensemble strategies for the deep learning object detection models carried out by combining the variants of a model and different models to enhance the anatomical and pathological object detection performance in brain MRI. In this study, with the help of the novel Gazi Brains 2020 dataset, five different anatomical parts and one pathological part that can be observed in brain MRI were identified, such as the region of interest, eye, optic nerves, lateral ventricles, third ventricle, and a whole tumor. Firstly, comprehensive benchmarking of the nine state-of-the-art object detection models was carried out to determine the capabilities of the models in detecting the anatomical and pathological parts. Then, four different ensemble strategies for nine object detectors were applied to boost the detection performance using the bounding box fusion technique. The ensemble of individual model variants increased the anatomical and pathological object detection performance by up to 10% in terms of the mean average precision (mAP). In addition, considering the class-based average precision (AP) value of the anatomical parts, an up to 18% AP improvement was achieved. Similarly, the ensemble strategy of the best different models outperformed the best individual model by 3.3% mAP. Additionally, while an up to 7% better FAUC, which is the area under the TPR vs. FPPI curve, was achieved on the Gazi Brains 2020 dataset, a 2% better FAUC score was obtained on the BraTS 2020 dataset. The proposed ensemble strategies were found to be much more efficient in finding the anatomical and pathological parts with a small number of anatomic objects, such as the optic nerve and third ventricle, and producing higher TPR values, especially at low FPPI values, compared to the best individual methods.

BBox-Guided Segmentor: Leveraging expert knowledge for accurate stroke lesion segmentation using weakly supervised bounding box prior

Stroke is one of the leading causes of death and disability in the world. Despite intensive research on automatic stroke lesion segmentation from non-invasive imaging modalities including diffusion-weighted imaging (DWI), challenges remain such as a lack of sufficient labeled data for training deep learning models and failure in detecting small lesions. In this paper, we propose BBox-Guided Segmentor, a method that significantly improves the accuracy of stroke lesion segmentation by leveraging expert knowledge. Specifically, our model uses a very coarse bounding box label provided by the expert and then performs accurate segmentation automatically. The small overhead of having the expert provide a rough bounding box leads to large performance improvement in segmentation, which is paramount to accurate stroke diagnosis. To train our model, we employ a weakly-supervised approach that uses a large number of weakly-labeled images with only bounding boxes and a small number of fully labeled images. The scarce fully labeled images are used to train a generator segmentation network, while adversarial training is used to leverage the large number of weakly-labeled images to provide additional learning signals. We evaluate our method extensively using a unique clinical dataset of 99 fully labeled cases (i.e., with full segmentation map labels) and 831 weakly labeled cases (i.e., with only bounding box labels), and the results demonstrate the superior performance of our approach over state-of-the-art stroke lesion segmentation models. We also achieve competitive performance as a SOTA fully supervised method using less than one-tenth of the complete labels. Our proposed approach has the potential to improve stroke diagnosis and treatment planning, which may lead to better patient outcomes.

3D-MASNet: 3D mixed-scale asymmetric convolutional segmentation network for 6-month-old infant brain MR images

Precise segmentation of infant brain magnetic resonance (MR) images into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) are essential for studying neuroanatomical hallmarks of early brain development. However, for 6-month-old infants, the extremely low-intensity contrast caused by inherent myelination hinders accurate tissue segmentation. Existing convolutional neural networks (CNNs) based segmentation models for this task generally employ single-scale symmetric convolutions, which are inefficient for encoding the isointense tissue boundaries in baby brain images. Here, we propose a 3D mixed-scale asymmetric convolutional segmentation network (3D-MASNet) framework for brain MR images of 6-month-old infants. We replaced the traditional convolutional layer of an existing to-be-trained network with a 3D mixed-scale convolution block consisting of asymmetric kernels (MixACB) during the training phase and then equivalently converted it into the original network. Five canonical CNN segmentation models were evaluated using both T1- and T2-weighted images of 23 6-month-old infants from iSeg-2019 datasets, which contained manual labels as ground truth. MixACB significantly enhanced the average accuracy of all five models and obtained the most considerable improvement in the fully convolutional network model (CC-3D-FCN) and the highest performance in the Dense U-Net model. This approach further obtained Dice coefficient accuracies of 0.931, 0.912, and 0.961 in GM, WM, and CSF, respectively, ranking first among 30 teams on the validation dataset of the iSeg-2019 Grand Challenge. Thus, the proposed 3D-MASNet can improve the accuracy of existing CNNs-based segmentation models as a plug-and-play solution that offers a promising technique for future infant brain MRI studies.

A Bibliometric Review: Brain Tumor Magnetic Resonance Imagings Using Different Convolutional Neural Network Architectures

BACKGROUND

Numerous scientists and researchers have been developing advanced procedures and methods for diagnosing the kind and phase of a human tumor. Brain tumors, which are neoplastic and abnormal developments of brain cells, are one of the most prominent causes of death. Brain tumors, also known as lesions or neoplasia, may be roughly classified as either primary or metastatic. Primary brain tumors arise from brain tissue and its surrounding environment. The recognition of brain tumors using magnetic resonance images via a deep learning technique such as convolutional neural network (CNN) has garnered significant academic interest over the last few decades.

METHODS

In this study, a detailed evaluation based on bibliometrics is considered in order to synthesize and organize the available academic literature and to identify current research trends and hotspots. We used bibliometric methodologies and a literature review for the CNN-based brain tumor to synthesize and evaluate prior studies.

RESULTS

For this bibliometric analysis, we applied the Visualization of Similarity Viewer program to classify the major publications, notable journals, financial sponsors, and affiliations.

CONCLUSIONS

In conclusion, we suggest that one of the next paths of study will be the incorporation of other databases to advance CNN-based brain tumor identification from magnetic resonance images. No drug dosages are applied in this work.

FMTM-feature-map-based transform model for brain image segmentation in tumor detection

The segmentation of brain images is a leading quantitative measure for detecting physiological changes and for analysing structural functions. Based on trends and dimensions of brain, the images indicate heterogeneity. Accurate brain tumour segmentation remains a critical challenge despite the persistent efforts of researchers were owing to a variety of obstacles. This impacts the outcome of tumour detection, causing errors. For addressing this issue, a Feature-Map based Transform Model (FMTM) is introduced to focus on heterogeneous features of input picture to map differences and intensity based on transition Fourier. Unchecked machine learning is used for reliable characteristic map recognition in this mapping process. For the determination of severity and variability, the method of identification depends on symmetry and texture. Learning instances are taught to improve precision using predefined data sets, regardless of loss of labels. The process is recurring until the maximum precision of tumour detection is achieved in low convergence. In this research, FMTM has been applied to brain tumour segmentation to automatically extract feature representations and produce accurate and steady performance because of promising performance made by powerful transition Fourier methods. The suggested model's performance is shown by the metrics processing time, precision, accuracy, and F1-Score.

Segmentation with mixed supervision: Confidence maximization helps knowledge distillation

Despite achieving promising results in a breadth of medical image segmentation tasks, deep neural networks (DNNs) require large training datasets with pixel-wise annotations. Obtaining these curated datasets is a cumbersome process which limits the applicability of DNNs in scenarios where annotated images are scarce. Mixed supervision is an appealing alternative for mitigating this obstacle. In this setting, only a small fraction of the data contains complete pixel-wise annotations and other images have a weaker form of supervision, e.g., only a handful of pixels are labeled. In this work, we propose a dual-branch architecture, where the upper branch (teacher) receives strong annotations, while the bottom one (student) is driven by limited supervision and guided by the upper branch. Combined with a standard cross-entropy loss over the labeled pixels, our novel formulation integrates two important terms: (i) a Shannon entropy loss defined over the less-supervised images, which encourages confident student predictions in the bottom branch; and (ii) a Kullback-Leibler (KL) divergence term, which transfers the knowledge (i.e., predictions) of the strongly supervised branch to the less-supervised branch and guides the entropy (student-confidence) term to avoid trivial solutions. We show that the synergy between the entropy and KL divergence yields substantial improvements in performance. We also discuss an interesting link between Shannon-entropy minimization and standard pseudo-mask generation, and argue that the former should be preferred over the latter for leveraging information from unlabeled pixels. We evaluate the effectiveness of the proposed formulation through a series of quantitative and qualitative experiments using two publicly available datasets. Results demonstrate that our method significantly outperforms other strategies for semantic segmentation within a mixed-supervision framework, as well as recent semi-supervised approaches. Moreover, in line with recent observations in classification, we show that the branch trained with reduced supervision and guided by the top branch largely outperforms the latter. Our code is publicly available: https://github.com/by-liu/ConfKD.

PTNet3D: A 3D High-Resolution Longitudinal Infant Brain MRI Synthesizer Based on Transformers

An increased interest in longitudinal neurodevelopment during the first few years after birth has emerged in recent years. Noninvasive magnetic resonance imaging (MRI) can provide crucial information about the development of brain structures in the early months of life. Despite the success of MRI collections and analysis for adults, it remains a challenge for researchers to collect high-quality multimodal MRIs from developing infant brains because of their irregular sleep pattern, limited attention, inability to follow instructions to stay still during scanning. In addition, there are limited analytic approaches available. These challenges often lead to a significant reduction of usable MRI scans and pose a problem for modeling neurodevelopmental trajectories. Researchers have explored solving this problem by synthesizing realistic MRIs to replace corrupted ones. Among synthesis methods, the convolutional neural network-based (CNN-based) generative adversarial networks (GANs) have demonstrated promising performance. In this study, we introduced a novel 3D MRI synthesis framework- pyramid transformer network (PTNet3D)- which relies on attention mechanisms through transformer and performer layers. We conducted extensive experiments on high-resolution Developing Human Connectome Project (dHCP) and longitudinal Baby Connectome Project (BCP) datasets. Compared with CNN-based GANs, PTNet3D consistently shows superior synthesis accuracy and superior generalization on two independent, large-scale infant brain MRI datasets. Notably, we demonstrate that PTNet3D synthesized more realistic scans than CNN-based models when the input is from multi-age subjects. Potential applications of PTNet3D include synthesizing corrupted or missing images. By replacing corrupted scans with synthesized ones, we observed significant improvement in infant whole brain segmentation.

Effectiveness of Federated Learning and CNN Ensemble Architectures for Identifying Brain Tumors Using MRI Images

Medical institutions often revoke data access due to the privacy concern of patients. Federated Learning (FL) is a collaborative learning paradigm that can generate an unbiased global model based on collecting updates from local models trained by client's data while keeping the local data private. This study aims to address the centralized data collection issue through the application of FL on brain tumor identification from MRI images. At first, several CNN models were trained using the MRI data and the best three performing CNN models were selected to form different variants of ensemble classifiers. Afterward, the FL model was constructed using the ensemble architecture. It was trained using model weights from the local model without sharing the client's data (MRI images) using the FL approach. Experimental results show only a slight decline in the performance of the FL approach as it achieved 91.05% accuracy compared to the 96.68% accuracy of the base ensemble model. Additionally, same approach was taken for another slightly larger dataset to prove the scalability of the method. This study shows that the FL approach can achieve privacy-protected tumor classification from MRI images without compromising much accuracy compared to the traditional deep learning approach.

Cardiac MRI segmentation with sparse annotations: Ensembling deep learning uncertainty and shape priors

The performance of deep learning for cardiac magnetic resonance imaging (MRI) segmentation is oftentimes degraded when using small datasets and sparse annotations for training or adapting a pre-trained model to previously unseen datasets. Here, we developed and evaluated an approach to addressing some of these issues to facilitate broader use of deep learning for short-axis cardiac MRI segmentation. We developed a globally optimal label fusion (GOLF) algorithm that enforced spatial smoothness to generate consensus segmentation from segmentation predictions provided by a deep learning ensemble algorithm. The GOLF consensus was entered into an uncertainty-guided coupled continuous kernel cut (ugCCKC) algorithm that employed normalized cut, image-grid continuous regularization, and "nesting" and circular shape priors of the left ventricular myocardium (LVM) and cavity (LVC). In addition, the uncertainty measurements derived from the segmentation predictions were used to constrain the similarity of GOLF and final segmentation. We optimized ugCCKC through upper bound relaxation, for which we developed an efficient coupled continuous max-flow algorithm implemented in an iterative manner. We showed that GOLF yielded average symmetric surface distance (ASSD) 0.2-0.8 mm lower than an averaging method with higher or similar Dice similarity coefficient (DSC). We also demonstrated that ugCCKC incorporating the shape priors improved DSC by 0.01-0.05 and reduced ASSD by 0.1-0.9 mm. In addition, we integrated GOLF and ugCCKC into a deep learning ensemble algorithm by refining the segmentation of an unannotated dataset and using the refined segmentation to update the trained models. With the proposed framework, we demonstrated the capability of using relatively small datasets (5-10 subjects) with sparse (5-25% slices labeled) annotations to train a deep learning algorithm, while achieving DSC of 0.871-0.893 for LVM and 0.933-0.959 for LVC on the LVQuan dataset, and these were 0.844-0.871 for LVM and 0.923-0.931 for LVC on the ACDC dataset. Furthermore, we showed that the proposed approach can be adapted to substantially alleviate the domain shift issue. Moreover, we calculated a number of commonly used LV function measurements using the derived segmentation and observed strong correlations (Pearson r=0.77-1.00, p<0.001) between algorithm and manual LV function analyses. These results suggest that the developed approaches can be used to facilitate broader application of deep learning in research and clinical cardiac MR imaging workflow.

FetalGAN: Automated Segmentation of Fetal Functional Brain MRI Using Deep Generative Adversarial Learning and Multi-Scale 3D U-Net

An important step in the preprocessing of resting state functional magnetic resonance images (rs-fMRI) is the separation of brain from non-brain voxels. Widely used imaging tools such as FSL's BET2 and AFNI's 3dSkullStrip accomplish this task effectively in children and adults. In fetal functional brain imaging, however, the presence of maternal tissue around the brain coupled with the non-standard position of the fetal head limit the usefulness of these tools. Accurate brain masks are thus generated manually, a time-consuming and tedious process that slows down preprocessing of fetal rs-fMRI. Recently, deep learning-based segmentation models such as convolutional neural networks (CNNs) have been increasingly used for automated segmentation of medical images, including the fetal brain. Here, we propose a computationally efficient end-to-end generative adversarial neural network (GAN) for segmenting the fetal brain. This method, which we call FetalGAN, yielded whole brain masks that closely approximated the manually labeled ground truth. FetalGAN performed better than 3D U-Net model and BET2: FetalGAN, Dice score = 0.973 ± 0.013, precision = 0.977 ± 0.015; 3D U-Net, Dice score = 0.954 ± 0.054, precision = 0.967 ± 0.037; BET2, Dice score = 0.856 ± 0.084, precision = 0.758 ± 0.113. FetalGAN was also faster than 3D U-Net and the manual method (7.35 s vs. 10.25 s vs. ∼5 min/volume). To the best of our knowledge, this is the first successful implementation of 3D CNN with GAN on fetal fMRI brain images and represents a significant advance in fully automating processing of rs-MRI images.

Prediction and Risk Assessment Models for Subarachnoid Hemorrhage: A Systematic Review on Case Studies

Subarachnoid hemorrhage (SAH) is one of the major health issues known to society and has a higher mortality rate. The clinical factors with computed tomography (CT), magnetic resonance image (MRI), and electroencephalography (EEG) data were used to evaluate the performance of the developed method. In this paper, various methods such as statistical analysis, logistic regression, machine learning, and deep learning methods were used in the prediction and detection of SAH which are reviewed. The advantages and limitations of SAH prediction and risk assessment methods are also being reviewed. Most of the existing methods were evaluated on the collected dataset for the SAH prediction. In some researches, deep learning methods were applied, which resulted in higher performance in the prediction process. EEG data were applied in the existing methods for the prediction process, and these methods demonstrated higher performance. However, the existing methods have the limitations of overfitting problems, imbalance data problems, and lower efficiency in feature analysis. The artificial neural network (ANN) and support vector machine (SVM) methods have been applied for the prediction process, and considerably higher performance is achieved by using this method.

Inverted bell-curve-based ensemble of deep learning models for detection of COVID-19 from chest X-rays

Novel Coronavirus 2019 disease or COVID-19 is a viral disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The use of chest X-rays (CXRs) has become an important practice to assist in the diagnosis of COVID-19 as they can be used to detect the abnormalities developed in the infected patients' lungs. With the fast spread of the disease, many researchers across the world are striving to use several deep learning-based systems to identify the COVID-19 from such CXR images. To this end, we propose an inverted bell-curve-based ensemble of deep learning models for the detection of COVID-19 from CXR images. We first use a selection of models pretrained on ImageNet dataset and use the concept of transfer learning to retrain them with CXR datasets. Then the trained models are combined with the proposed inverted bell curve weighted ensemble method, where the output of each classifier is assigned a weight, and the final prediction is done by performing a weighted average of those outputs. We evaluate the proposed method on two publicly available datasets: the COVID-19 Radiography Database and the IEEE COVID Chest X-ray Dataset. The accuracy, F1 score and the AUC ROC achieved by the proposed method are 99.66%, 99.75% and 99.99%, respectively, in the first dataset, and, 99.84%, 99.81% and 99.99%, respectively, in the other dataset. Experimental results ensure that the use of transfer learning-based models and their combination using the proposed ensemble method result in improved predictions of COVID-19 in CXRs.

Deep learning methods for automatic segmentation of lower leg muscles and bones from MRI scans of children with and without cerebral palsy

Cerebral palsy is a neurological condition that is known to affect muscle growth. Detailed investigations of muscle growth require segmentation of muscles from MRI scans, which is typically done manually. In this study, we evaluated the performance of 2D, 3D, and hybrid deep learning models for automatic segmentation of 11 lower leg muscles and two bones from MRI scans of children with and without cerebral palsy. All six models were trained and evaluated on manually segmented T₁ -weighted MRI scans of the lower legs of 20 children, six of whom had cerebral palsy. The segmentation results were assessed using the median Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and volume error (VError) of all 13 labels of every scan. The best performance was achieved by H-DenseUNet, a hybrid model (DSC 0.90, ASSD 0.5 mm, and VError 2.6 cm³ ). The performance was equivalent to the inter-rater performance of manual segmentation (DSC 0.89, ASSD 0.6 mm, and VError 3.3 cm³ ). Models trained with the Dice loss function outperformed models trained with the cross-entropy loss function. Near-optimal performance could be attained using only 11 scans for training. Segmentation performance was similar for scans of typically developing children (DSC 0.90, ASSD 0.5 mm, and VError 2.8 cm³ ) and children with cerebral palsy (DSC 0.85, ASSD 0.6 mm, and VError 2.4 cm³ ). These findings demonstrate the feasibility of fully automatic segmentation of individual muscles and bones from MRI scans of children with and without cerebral palsy.

Automated tibiofemoral joint segmentation based on deeply supervised 2D-3D ensemble U-Net: Data from the Osteoarthritis Initiative

Improving longevity is one of the greatest achievements in humanity. Because of this, the population is growing older, and the ubiquity of knee osteoarthritis (OA) is on the rise. Nonetheless, the understanding and ability to investigate potential precursors of knee OA have been impeded by time-consuming and laborious manual delineation processes which are prone to poor reproducibility. A method for automatic segmentation of the tibiofemoral joint using magnetic resonance imaging (MRI) is presented in this work. The proposed method utilizes a deeply supervised 2D-3D ensemble U-Net, which consists of foreground class oversampling, deep supervision loss branches, and Gaussian weighted softmax score aggregation. It was designed, optimized, and tested on 507 3D double echo steady-state (DESS) MR volumes using a two-fold cross-validation approach. A state-of-the-art segmentation accuracy measured as Dice similarity coefficient (DSC) for the femur bone (98.6 ± 0.27%), tibia bone (98.8 ± 0.31%), femoral cartilage (90.3 ± 2.89%), and tibial cartilage (86.7 ± 4.07%) is achieved. Notably, the proposed method yields sub-voxel accuracy for an average symmetric surface distance (ASD) less than 0.36 mm. The model performance is not affected by the severity of radiographic osteoarthritis (rOA) grades or the presence of pathophysiological changes. The proposed method offers an accurate segmentation with high time efficiency (~62 s) per 3D volume, which is well suited for efficient processing and analysis of the large prospective cohorts of the Osteoarthritis Initiative (OAI).

Computer-Aided Diagnosis of Children with Cerebral Palsy under Deep Learning Convolutional Neural Network Image Segmentation Model Combined with Three-Dimensional Cranial Magnetic Resonance Imaging

In this paper, we analyzed the application value and effect of deep learn-based image segmentation model of convolutional neural network (CNN) algorithm combined with 3D brain magnetic resonance imaging (MRI) in diagnosis of cerebral palsy in children. 3D brain model was segmented based on CNN algorithm to obtain the segmented MRI images of brain tissue, and the validity was verified. Then, 70 children with cerebral palsy were rolled into the observation group (n = 35), which received MRI for diagnosis after segmentation of brain tissue, and control group (n = 35), which were diagnosed by computed tomography (CT). The diagnosis results of the two groups were compared. The validity experiment verified that the image segmentation method based on CNN algorithm can obtain effective style graphics. In clinical trials, the diagnostic accuracy of 88.6% in the observation group was evidently superior to that of 80% in the control group (P < 0.05). In the observation group, one patient was diagnosed as normal, four patients had white matter lesions, 17 patients had corpus callosum lesions, and five patients had basal ganglia softening foci. In the control group, two patients were diagnosed as normal, two patients had white matter lesions, 19 patients had corpus callosum lesions, and four patients had basal ganglia softening foci. No notable difference was found between the two groups (P > 0.05). According to the research results, in the diagnosis of cerebral palsy in children, the image segmentation of brain 3D model based on CNN to obtain the MRI image of segmented brain tissue can effectively improve the detection accuracy. Moreover, the specific symptoms can be diagnosed clearly. It can provide the corresponding diagnostic basis for clinical diagnosis and treatment and was worthy of clinical promotion.

Deep learning for neuroimaging-based diagnosis and rehabilitation of Autism Spectrum Disorder: A review

Accurate diagnosis of Autism Spectrum Disorder (ASD) followed by effective rehabilitation is essential for the management of this disorder. Artificial intelligence (AI) techniques can aid physicians to apply automatic diagnosis and rehabilitation procedures. AI techniques comprise traditional machine learning (ML) approaches and deep learning (DL) techniques. Conventional ML methods employ various feature extraction and classification techniques, but in DL, the process of feature extraction and classification is accomplished intelligently and integrally. DL methods for diagnosis of ASD have been focused on neuroimaging-based approaches. Neuroimaging techniques are non-invasive disease markers potentially useful for ASD diagnosis. Structural and functional neuroimaging techniques provide physicians substantial information about the structure (anatomy and structural connectivity) and function (activity and functional connectivity) of the brain. Due to the intricate structure and function of the brain, proposing optimum procedures for ASD diagnosis with neuroimaging data without exploiting powerful AI techniques like DL may be challenging. In this paper, studies conducted with the aid of DL networks to distinguish ASD are investigated. Rehabilitation tools provided for supporting ASD patients utilizing DL networks are also assessed. Finally, we will present important challenges in the automated detection and rehabilitation of ASD and propose some future works.

A novel M-SegNet with global attention CNN architecture for automatic segmentation of brain MRI

In this paper, we propose a novel M-SegNet architecture with global attention for the segmentation of brain magnetic resonance imaging (MRI). The proposed architecture consists of a multiscale deep network at the encoder side, deep supervision at the decoder side, a global attention mechanism, different sizes of convolutional kernels, and combined-connections with skip connections and pooling indices. The multiscale side input layers were used to support deep layers for extracting the discriminative information and the upsampling layer at the decoder side provided deep supervision, which reduced the gradient problem. The global attention mechanism is utilized to capture rich contextual information in the decoder stage by integrating local features with their respective global dependencies. In addition, multiscale convolutional kernels of different sizes were used to extract abundant semantic features from brain MRI scans in the encoder and decoder modules. Moreover, combined-connections were used to pass features from the encoder to the decoder path to recover the spatial information lost during downsampling and makes the model converge faster. Furthermore, we adopted uniform non-overlapping input patches to focus on fine details for the segmentation of brain MRI. We verified the proposed architecture on publicly accessible datasets for the task of segmentation of brain MRI. The experimental results show that the proposed model outperforms conventional methods by achieving an average Dice similarity coefficient score of 0.96.

Deep Learning-Based Studies on Pediatric Brain Tumors Imaging: Narrative Review of Techniques and Challenges

Brain tumors diagnosis in children is a scientific concern due to rapid anatomical, metabolic, and functional changes arising in the brain and non-specific or conflicting imaging results. Pediatric brain tumors diagnosis is typically centralized in clinical practice on the basis of diagnostic clues such as, child age, tumor location and incidence, clinical history, and imaging (Magnetic resonance imaging MRI / computed tomography CT) findings. The implementation of deep learning has rapidly propagated in almost every field in recent years, particularly in the medical images’ evaluation. This review would only address critical deep learning issues specific to pediatric brain tumor imaging research in view of the vast spectrum of other applications of deep learning. The purpose of this review paper is to include a detailed summary by first providing a succinct guide to the types of pediatric brain tumors and pediatric brain tumor imaging techniques. Then, we will present the research carried out by summarizing the scientific contributions to the field of pediatric brain tumor imaging processing and analysis. Finally, to establish open research issues and guidance for potential study in this emerging area, the medical and technical limitations of the deep learning-based approach were included.

Learning U-Net Based Multi-Scale Features in Encoding-Decoding for MR Image Brain Tissue Segmentation

Accurate brain tissue segmentation of MRI is vital to diagnosis aiding, treatment planning, and neurologic condition monitoring. As an excellent convolutional neural network (CNN), U-Net is widely used in MR image segmentation as it usually generates high-precision features. However, the performance of U-Net is considerably restricted due to the variable shapes of the segmented targets in MRI and the information loss of down-sampling and up-sampling operations. Therefore, we propose a novel network by introducing spatial and channel dimensions-based multi-scale feature information extractors into its encoding-decoding framework, which is helpful in extracting rich multi-scale features while highlighting the details of higher-level features in the encoding part, and recovering the corresponding localization to a higher resolution layer in the decoding part. Concretely, we propose two information extractors, multi-branch pooling, called MP, in the encoding part, and multi-branch dense prediction, called MDP, in the decoding part, to extract multi-scale features. Additionally, we designed a new multi-branch output structure with MDP in the decoding part to form more accurate edge-preserving predicting maps by integrating the dense adjacent prediction features at different scales. Finally, the proposed method is tested on datasets MRbrainS13, IBSR18, and ISeg2017. We find that the proposed network performs higher accuracy in segmenting MRI brain tissues and it is better than the leading method of 2018 at the segmentation of GM and CSF. Therefore, it can be a useful tool for diagnostic applications, such as brain MRI segmentation and diagnosing.

Analysis of the nonperfused volume ratio of adenomyosis from MRI images based on fewshot learning

The nonperfused volume (NPV) ratio is the key to the success of high intensity focused ultrasound (HIFU) ablation treatment of adenomyosis. However, there are no qualitative interpretation standards for predicting the NPV ratio of adenomyosis using magnetic resonance imaging (MRI) before HIFU ablation treatment, which leading to inter-reader variability. Convolutional neural networks (CNNs) have achieved state-of-the-art performance in the automatic disease diagnosis of MRI. Since the use of HIFU to treat adenomyosis is a novel treatment, there is not enough MRI data to support CNNs. We proposed a novel few-shot learning framework that extends CNNs to predict NPV ratio of HIFU ablation treatment for adenomyosis. We collected a dataset from 208 patients with adenomyosis who underwent MRI examination before and after HIFU treatment. Our proposed method was trained and evaluated by fourfold cross validation. This framework obtained sensitivity of 85.6%, 89.6% and 92.8% at 0.799, 0.980 and 1.180 FPs per patient. In the receiver operating characteristics analysis for NPV ratio of adenomyosis, our proposed method received the area under the curve of 0.8233, 0.8289, 0.8412, 0.8319, 0.7010, 0.7637, 0.8375, 0.8219, 0.8207, 0.9812 for the classifications of NPV ratio interval [0%-10%), [10%-20%), …, [90%-100%], respectively. The present study demonstrated that few-shot learning on NPV ratio prediction of HIFU ablation treatment for adenomyosis may contribute to the selection of eligible patients and the pre-judgment of clinical efficacy.

Automatic brain extraction from 3D fetal MR image with deep learning-based multi-step framework

Brain extraction is a fundamental prerequisite step in neuroimage analysis for fetus. Due to surrounding maternal tissues and unpredictable movement, brain extraction from fetal Magnetic Resonance (MR) images is a challenging task. In this paper, we propose a novel deep learning-based multi-step framework for brain extraction from 3D fetal MR images. In the first step, a global localization network is applied to estimate probability maps for brain candidates. Connected-component labeling algorithm is applied to eliminate small erroneous components and accurately locate the candidate brain area. In the second step, a local refinement network is implemented in the brain candidate area to obtain fine-grained probability maps. Final extraction results are derived by a fusion network with the two cascaded probability maps obtained from previous two steps. Experimental results demonstrate that our proposed method has superior performance compared with existing deep learning-based methods.

MRI image synthesis with dual discriminator adversarial learning and difficulty-aware attention mechanism for hippocampal subfields segmentation

BACKGROUND AND OBJECTIVE

Hippocampal subfields (HS) segmentation accuracy on high resolution (HR) MRI images is higher than that on low resolution (LR) MRI images. However, HR MRI data collection is more expensive and time-consuming. Thus, we intend to generate HR MRI images from the corresponding LR MRI images for HS segmentation.

METHODS AND RESULTS

To generate high-quality HR MRI hippocampus region images, we use a dual discriminator adversarial learning model with difficulty-aware attention mechanism in hippocampus regions (da-GAN). A local discriminator is applied in da-GAN to evaluate the visual quality of hippocampus region voxels of the synthetic images. And the difficulty-aware attention mechanism based on the local discriminator can better model the generation of hard-to-synthesis voxels in hippocampus regions. Additionally, we design a SemiDenseNet model with 3D Dense CRF postprocessing and an Unet-based model to perform HS segmentation. The experiments are implemented on Kulaga-Yoskovitz dataset. Compared with conditional generative adversarial network (c-GAN), the PSNR of generated HR T2w images acquired by our da-GAN achieves 0.406 and 0.347 improvement in left and right hippocampus regions. When using two segmentation models to segment HS, the DSC values achieved on the generated HR T1w and T2w images are both improved than that on LR T1w images.

CONCLUSION

Experimental results show that da-GAN model can generate higher-quality MRI images, especially in hippocampus regions, and the generated MRI images can improve HS segmentation accuracy.

Optimization of Deep Learning Network Parameters Using Uniform Experimental Design for Breast Cancer Histopathological Image Classification

Breast cancer, a common cancer type, is a major health concern in women. Recently, researchers used convolutional neural networks (CNNs) for medical image analysis and demonstrated classification performance for breast cancer diagnosis from within histopathological image datasets. However, the parameter settings of a CNN model are complicated, and using Breast Cancer Histopathological Database data for the classification is time-consuming. To overcome these problems, this study used a uniform experimental design (UED) and optimized the CNN parameters of breast cancer histopathological image classification. In UED, regression analysis was used to optimize the parameters. The experimental results indicated that the proposed method with UED parameter optimization provided 84.41% classification accuracy rate. In conclusion, the proposed method can improve the classification accuracy effectively, with results superior to those of other similar methods.

AssemblyNet: A large ensemble of CNNs for 3D whole brain MRI segmentation

Whole brain segmentation of fine-grained structures using deep learning (DL) is a very challenging task since the number of anatomical labels is very high compared to the number of available training images. To address this problem, previous DL methods proposed to use a single convolution neural network (CNN) or few independent CNNs. In this paper, we present a novel ensemble method based on a large number of CNNs processing different overlapping brain areas. Inspired by parliamentary decision-making systems, we propose a framework called AssemblyNet, made of two "assemblies" of U-Nets. Such a parliamentary system is capable of dealing with complex decisions, unseen problem and reaching a relevant consensus. AssemblyNet introduces sharing of knowledge among neighboring U-Nets, an "amendment" procedure made by the second assembly at higher-resolution to refine the decision taken by the first one, and a final decision obtained by majority voting. During our validation, AssemblyNet showed competitive performance compared to state-of-the-art methods such as U-Net, Joint label fusion and SLANT. Moreover, we investigated the scan-rescan consistency and the robustness to disease effects of our method. These experiences demonstrated the reliability of AssemblyNet. Finally, we showed the interest of using semi-supervised learning to improve the performance of our method.

Using 2D CNN with Taguchi Parametric Optimization for Lung Cancer Recognition from CT Images

Lung cancer is one of the common causes of cancer deaths. Early detection and treatment of lung cancer is essential. However, the detection of lung cancer in patients produces many false positives. Therefore, increasing the accuracy of the classification of diagnosis or true detection by computed tomography (CT) is a difficult task. Solving this problem using intelligent and automated methods has become a hot research topic in recent years. Hence, we propose a 2D convolutional neural network (2D CNN) with Taguchi parametric optimization for automatically recognizing lung cancer from CT images. In the Taguchi method, 36 experiments and 8 control factors of mixed levels were selected to determine the optimum parameters of the 2D CNN architecture and improve the classification accuracy of lung cancer. The experimental results show that the average classification accuracy of the 2D CNN with Taguchi parameter optimization and the original 2D CNN in lung cancer recognition are 91.97% and 98.83% on the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset, and 94.68% and 99.97% on the International Society for Optics and Photonics with the support of the American Association of Physicists in Medicine (SPIE-AAPM) dataset, respectively. The proposed method is 6.86% and 5.29% more accurate than the original 2D CNN on the two datasets, respectively, proving the superiority of proposed model.

A multilevel features selection framework for skin lesion classification

Melanoma is considered to be one of the deadliest skin cancer types, whose occurring frequency elevated in the last few years; its earlier diagnosis, however, significantly increases the chances of patients’ survival. In the quest for the same, a few computer based methods, capable of diagnosing the skin lesion at initial stages, have been recently proposed. Despite some success, however, margin exists, due to which the machine learning community still considers this an outstanding research challenge. In this work, we come up with a novel framework for skin lesion classification, which integrates deep features information to generate most discriminant feature vector, with an advantage of preserving the original feature space. We utilize recent deep models for feature extraction, and by taking advantage of transfer learning. Initially, the dermoscopic images are segmented, and the lesion region is extracted, which is later subjected to retrain the selected deep models to generate fused feature vectors. In the second phase, a framework for most discriminant feature selection and dimensionality reduction is proposed, entropy-controlled neighborhood component analysis (ECNCA). This hierarchical framework optimizes fused features by selecting the principle components and extricating the redundant and irrelevant data. The effectiveness of our design is validated on four benchmark dermoscopic datasets; PH2, ISIC MSK, ISIC UDA, and ISBI-2017. To authenticate the proposed method, a fair comparison with the existing techniques is also provided. The simulation results clearly show that the proposed design is accurate enough to categorize the skin lesion with 98.8%, 99.2% and 97.1% and 95.9% accuracy with the selected classifiers on all four datasets, and by utilizing less than 3% features.

Using Deep Convolutional Neural Networks for Neonatal Brain Image Segmentation

INTRODUCTION

Deep learning neural networks are especially potent at dealing with structured data, such as images and volumes. Both modified LiviaNET and HyperDense-Net performed well at a prior competition segmenting 6-month-old infant magnetic resonance images, but neonatal cerebral tissue type identification is challenging given its uniquely inverted tissue contrasts. The current study aims to evaluate the two architectures to segment neonatal brain tissue types at term equivalent age.

METHODS

Both networks were retrained over 24 pairs of neonatal T1 and T2 data from the Developing Human Connectome Project public data set and validated on another eight pairs against ground truth. We then reported the best-performing model from training and its performance by computing the Dice similarity coefficient (DSC) for each tissue type against eight test subjects.

RESULTS

During the testing phase, among the segmentation approaches tested, the dual-modality HyperDense-Net achieved the best statistically significantly test mean DSC values, obtaining 0.94/0.95/0.92 for the tissue types and took 80 h to train and 10 min to segment, including preprocessing. The single-modality LiviaNET was better at processing T2-weighted images than processing T1-weighted images across all tissue types, achieving mean DSC values of 0.90/0.90/0.88 for gray matter, white matter, and cerebrospinal fluid, respectively, while requiring 30 h to train and 8 min to segment each brain, including preprocessing.

DISCUSSION

Our evaluation demonstrates that both neural networks can segment neonatal brains, achieving previously reported performance. Both networks will be continuously retrained over an increasingly larger repertoire of neonatal brain data and be made available through the Canadian Neonatal Brain Platform to better serve the neonatal brain imaging research community.

Automatic extraction of the intracranial volume in fetal and neonatal MR scans using convolutional neural networks

•

Automatic intracranial volume segmentation.

•

Fetal and neonatal MRI.

•

Deep learning.

MR images of infants and fetuses allow non-invasive analysis of the brain. Quantitative analysis of brain development requires automatic brain tissue segmentation that is typically preceded by segmentation of the intracranial volume (ICV). Fast changes in the size and morphology of the developing brain, motion artifacts, and large variation in the field of view make ICV segmentation a challenging task.

We propose an automatic method for segmentation of the ICV in fetal and neonatal MRI scans. The method was developed and tested with a diverse set of scans regarding image acquisition parameters (i.e. field strength, image acquisition plane, image resolution), infant age (23–45 weeks post menstrual age), and pathology (posthaemorrhagic ventricular dilatation, stroke, asphyxia, and Down syndrome). The results demonstrate that the method achieves accurate segmentation with a Dice coefficient (DC) ranging from 0.98 to 0.99 in neonatal and fetal scans regardless of image acquisition parameters or patient characteristics. Hence, the algorithm provides a generic tool for segmentation of the ICV that may be used as a preprocessing step for brain tissue segmentation in fetal and neonatal brain MR scans.

Accurate and robust segmentation of neuroanatomy in T1-weighted MRI by combining spatial priors with deep convolutional neural networks

Neuroanatomical segmentation in magnetic resonance imaging (MRI) of the brain is a prerequisite for quantitative volume, thickness, and shape measurements, as well as an important intermediate step in many preprocessing pipelines. This work introduces a new highly accurate and versatile method based on 3D convolutional neural networks for the automatic segmentation of neuroanatomy in T1‐weighted MRI. In combination with a deep 3D fully convolutional architecture, efficient linear registration‐derived spatial priors are used to incorporate additional spatial context into the network. An aggressive data augmentation scheme using random elastic deformations is also used to regularize the networks, allowing for excellent performance even in cases where only limited labeled training data are available. Applied to hippocampus segmentation in an elderly population (mean Dice coefficient = 92.1%) and subcortical segmentation in a healthy adult population (mean Dice coefficient = 89.5%), we demonstrate new state‐of‐the‐art accuracies and a high robustness to outliers. Further validation on a multistructure segmentation task in a scan–rescan dataset demonstrates accuracy (mean Dice coefficient = 86.6%) similar to the scan–rescan reliability of expert manual segmentations (mean Dice coefficient = 86.9%), and improved reliability compared to both expert manual segmentations and automated segmentations using FIRST. Furthermore, our method maintains a highly competitive runtime performance (e.g., requiring only 10 s for left/right hippocampal segmentation in 1 × 1 × 1 mm³ MNI stereotaxic space), orders of magnitude faster than conventional multiatlas segmentation methods.