1
|
Zhu H, Cheng W, Hu K, He G. DSnet: a new dual-branch network for hippocampus subfield segmentation. Sci Rep 2024; 14:15317. [PMID: 38961218 PMCID: PMC11222372 DOI: 10.1038/s41598-024-66415-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Accepted: 07/01/2024] [Indexed: 07/05/2024] Open
Abstract
The hippocampus is a critical component of the brain and is associated with many neurological disorders. It can be further subdivided into several subfields, and accurate segmentation of these subfields is of great significance for diagnosis and research. However, the structures of hippocampal subfields are irregular and have complex boundaries, and their voxel values are close to surrounding brain tissues, making the segmentation task highly challenging. Currently, many automatic segmentation tools exist for hippocampal subfield segmentation, but they suffer from high time costs and low segmentation accuracy. In this paper, we propose a new dual-branch segmentation network structure (DSnet) based on deep learning for hippocampal subfield segmentation. While traditional convolutional neural network-based methods are effective in capturing hierarchical structures, they struggle to establish long-term dependencies. The DSnet integrates the Transformer architecture and a hybrid attention mechanism, enhancing the network's global perceptual capabilities. Moreover, the dual-branch structure of DSnet leverages the segmentation results of the hippocampal region to facilitate the segmentation of its subfields. We validate the efficacy of our algorithm on the public Kulaga-Yoskovitz dataset. Experimental results indicate that our method is more effective in segmenting hippocampal subfields than conventional single-branch network structures. Compared to the classic 3D U-Net, our proposed DSnet improves the average Dice accuracy of hippocampal subfield segmentation by 0.57%.
Collapse
Affiliation(s)
- Hancan Zhu
- School of Mathematics, Physics and Information, Shaoxing University, 900 ChengNan Rd, Shaoxing, 312000, Zhejiang, China
- Institute of Artificial Intelligence, Shaoxing University, Shaoxing, 312000, Zhejiang, China
| | - Wangang Cheng
- School of Mathematics, Physics and Information, Shaoxing University, 900 ChengNan Rd, Shaoxing, 312000, Zhejiang, China
| | - Keli Hu
- Institute of Artificial Intelligence, Shaoxing University, Shaoxing, 312000, Zhejiang, China
| | - Guanghua He
- School of Mathematics, Physics and Information, Shaoxing University, 900 ChengNan Rd, Shaoxing, 312000, Zhejiang, China.
| |
Collapse
|
2
|
Poiret C, Bouyeure A, Patil S, Boniteau C, Duchesnay E, Grigis A, Lemaitre F, Noulhiane M. Attention-gated 3D CapsNet for robust hippocampal segmentation. J Med Imaging (Bellingham) 2024; 11:014003. [PMID: 38173654 PMCID: PMC10760147 DOI: 10.1117/1.jmi.11.1.014003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 11/18/2023] [Accepted: 12/04/2023] [Indexed: 01/05/2024] Open
Abstract
Purpose The hippocampus is organized in subfields (HSF) involved in learning and memory processes and widely implicated in pathologies at different ages of life, from neonatal hypoxia to temporal lobe epilepsy or Alzheimer's disease. Getting a highly accurate and robust delineation of sub-millimetric regions such as HSF to investigate anatomo-functional hypotheses is a challenge. One of the main difficulties encountered by those methodologies is related to the small size and anatomical variability of HSF, resulting in the scarcity of manual data labeling. Recently introduced, capsule networks solve analogous problems in medical imaging, providing deep learning architectures with rotational equivariance. Nonetheless, capsule networks are still two-dimensional and unassessed for the segmentation of HSF. Approach We released a public 3D Capsule Network (3D-AGSCaps, https://github.com/clementpoiret/3D-AGSCaps) and compared it to equivalent architectures using classical convolutions on the automatic segmentation of HSF on small and atypical datasets (incomplete hippocampal inversion, IHI). We tested 3D-AGSCaps on three datasets with manually labeled hippocampi. Results Our main results were: (1) 3D-AGSCaps produced segmentations with a better Dice Coefficient compared to CNNs on rotated hippocampi (p = 0.004 , cohen's d = 0.179 ); (2) on typical subjects, 3D-AGSCaps produced segmentations with a Dice coefficient similar to CNNs while having 15 times fewer parameters (2.285M versus 35.069M). This may greatly facilitate the study of atypical subjects, including healthy and pathological cases like those presenting an IHI. Conclusion We expect our newly introduced 3D-AGSCaps to allow a more accurate and fully automated segmentation on atypical populations, small datasets, as well as on and large cohorts where manual segmentations are nearly intractable.
Collapse
Affiliation(s)
- Clement Poiret
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
- Université Paris Cité, InDEV team, U1141 NeuroDiderot, Inserm, Paris, France
| | - Antoine Bouyeure
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
- Université Paris Cité, InDEV team, U1141 NeuroDiderot, Inserm, Paris, France
| | - Sandesh Patil
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
- Université Paris Cité, InDEV team, U1141 NeuroDiderot, Inserm, Paris, France
| | - Cécile Boniteau
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
- Université Paris Cité, InDEV team, U1141 NeuroDiderot, Inserm, Paris, France
| | - Edouard Duchesnay
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
| | - Antoine Grigis
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
| | - Frederic Lemaitre
- Université de Rouen, CETAPS EA 3832, Rouen, France
- CRIOBE, UAR 3278, CNRS-EPHE-UPVD, Mooréa, Polynésie Française
| | - Marion Noulhiane
- UNIACT, NeuroSpin, Institut Joliot, CEA Paris-Saclay, Gif-sur-Yvette, France
- Université Paris Cité, InDEV team, U1141 NeuroDiderot, Inserm, Paris, France
| |
Collapse
|
3
|
Li X, Wang L, Liu H, Ma B, Chu L, Dong X, Zeng D, Che T, Jiang X, Wang W, Hu J, Li S. Syn_SegNet: A Joint Deep Neural Network for Ultrahigh-Field 7T MRI Synthesis and Hippocampal Subfield Segmentation in Routine 3T MRI. IEEE J Biomed Health Inform 2023; 27:4866-4877. [PMID: 37581964 DOI: 10.1109/jbhi.2023.3305377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/17/2023]
Abstract
Precise delineation of hippocampus subfields is crucial for the identification and management of various neurological and psychiatric disorders. However, segmenting these subfields automatically in routine 3T MRI is challenging due to their complex morphology and small size, as well as the limited signal contrast and resolution of the 3T images. This research proposes Syn_SegNet, an end-to-end, multitask joint deep neural network that leverages ultrahigh-field 7T MRI synthesis to improve hippocampal subfield segmentation in 3T MRI. Our approach involves two key components. First, we employ a modified Pix2PixGAN as the synthesis model, incorporating self-attention modules, image and feature matching loss, and ROI loss to generate high-quality 7T-like MRI around the hippocampal region. Second, we utilize a variant of 3D-U-Net with multiscale deep supervision as the segmentation subnetwork, incorporating an anatomic weighted cross-entropy loss that capitalizes on prior anatomical knowledge. We evaluate our method on hippocampal subfield segmentation in paired 3T MRI and 7T MRI with seven different anatomical structures. The experimental findings demonstrate that Syn_SegNet's segmentation performance benefits from integrating synthetic 7T data in an online manner and is superior to competing methods. Furthermore, we assess the generalizability of the proposed approach using a publicly accessible 3T MRI dataset. The developed method would be an efficient tool for segmenting hippocampal subfields in routine clinical 3T MRI.
Collapse
|
4
|
Zhang G, Zhou J, He G, Zhu H. Deep fusion of multi-modal features for brain tumor image segmentation. Heliyon 2023; 9:e19266. [PMID: 37664757 PMCID: PMC10468380 DOI: 10.1016/j.heliyon.2023.e19266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 08/09/2023] [Accepted: 08/17/2023] [Indexed: 09/05/2023] Open
Abstract
Accurate segmentation of pathological regions in brain magnetic resonance images (MRI) is essential for the diagnosis and treatment of brain tumors. Multi-modality MRIs, which offer diverse feature information, are commonly utilized in brain tumor image segmentation. Deep neural networks have become prevalent in this field; however, many approaches simply concatenate different modalities and input them directly into the neural network for segmentation, disregarding the unique characteristics and complementarity of each modality. In this study, we propose a brain tumor image segmentation method that leverages deep residual learning with multi-modality image feature fusion. Our approach involves extracting and fusing distinct and complementary features from various modalities, fully exploiting the multi-modality information within a deep convolutional neural network to enhance the performance of brain tumor image segmentation. We evaluate the effectiveness of our proposed method using the BraTS2021 dataset and demonstrate that deep residual learning with multi-modality image feature fusion significantly improves segmentation accuracy. Our method achieves competitive segmentation results, with Dice values of 83.3, 89.07, and 91.44 for enhanced tumor, tumor core, and whole tumor, respectively. These findings highlight the potential of our method in improving brain tumor diagnosis and treatment through accurate segmentation of pathological regions in brain MRIs.
Collapse
Affiliation(s)
- Guying Zhang
- School of Mathematics, Physics and Information, Shaoxing University, Shaoxing, Zhejiang, 312000, China
| | - Jia Zhou
- Cancer Center, Gamma Knife Treatment Center, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, 310014, China
| | - Guanghua He
- School of Mathematics, Physics and Information, Shaoxing University, Shaoxing, Zhejiang, 312000, China
- Institute of Artificial Intelligence, Shaoxing University, Shaoxing, Zhejiang, 312000, China
| | - Hancan Zhu
- School of Mathematics, Physics and Information, Shaoxing University, Shaoxing, Zhejiang, 312000, China
- Institute of Artificial Intelligence, Shaoxing University, Shaoxing, Zhejiang, 312000, China
| |
Collapse
|
5
|
Poiret C, Bouyeure A, Patil S, Grigis A, Duchesnay E, Faillot M, Bottlaender M, Lemaitre F, Noulhiane M. A fast and robust hippocampal subfields segmentation: HSF revealing lifespan volumetric dynamics. Front Neuroinform 2023; 17:1130845. [PMID: 37396459 PMCID: PMC10308024 DOI: 10.3389/fninf.2023.1130845] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 05/22/2023] [Indexed: 07/04/2023] Open
Abstract
The hippocampal subfields, pivotal to episodic memory, are distinct both in terms of cyto- and myeloarchitectony. Studying the structure of hippocampal subfields in vivo is crucial to understand volumetric trajectories across the lifespan, from the emergence of episodic memory during early childhood to memory impairments found in older adults. However, segmenting hippocampal subfields on conventional MRI sequences is challenging because of their small size. Furthermore, there is to date no unified segmentation protocol for the hippocampal subfields, which limits comparisons between studies. Therefore, we introduced a novel segmentation tool called HSF short for hippocampal segmentation factory, which leverages an end-to-end deep learning pipeline. First, we validated HSF against currently used tools (ASHS, HIPS, and HippUnfold). Then, we used HSF on 3,750 subjects from the HCP development, young adults, and aging datasets to study the effect of age and sex on hippocampal subfields volumes. Firstly, we showed HSF to be closer to manual segmentation than other currently used tools (p < 0.001), regarding the Dice Coefficient, Hausdorff Distance, and Volumetric Similarity. Then, we showed differential maturation and aging across subfields, with the dentate gyrus being the most affected by age. We also found faster growth and decay in men than in women for most hippocampal subfields. Thus, while we introduced a new, fast and robust end-to-end segmentation tool, our neuroanatomical results concerning the lifespan trajectories of the hippocampal subfields reconcile previous conflicting results.
Collapse
Affiliation(s)
- Clement Poiret
- UNIACT, NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- InDEV, NeuroDiderot, Université Paris Cité, Inserm, Paris, France
| | - Antoine Bouyeure
- UNIACT, NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- InDEV, NeuroDiderot, Université Paris Cité, Inserm, Paris, France
| | - Sandesh Patil
- UNIACT, NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- InDEV, NeuroDiderot, Université Paris Cité, Inserm, Paris, France
| | - Antoine Grigis
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- InDEV, NeuroDiderot, Université Paris Cité, Inserm, Paris, France
| | - Edouard Duchesnay
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- InDEV, NeuroDiderot, Université Paris Cité, Inserm, Paris, France
| | - Matthieu Faillot
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- BioMaps, Service Hospitalier Frédéric Joliot, CNRS, Inserm, Université Paris-Saclay, Orsay, France
| | - Michel Bottlaender
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- BioMaps, Service Hospitalier Frédéric Joliot, CNRS, Inserm, Université Paris-Saclay, Orsay, France
| | - Frederic Lemaitre
- CETAPS EA 3832, Université de Rouen, Rouen, France
- CRIOBE, UAR 3278, CNRS-EPHE-UPVD, Mooréa, France
| | - Marion Noulhiane
- UNIACT, NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- NeuroSpin, CEA Paris-Saclay, Frederic Joliot Institute, Gif-sur-Yvette, France
- InDEV, NeuroDiderot, Université Paris Cité, Inserm, Paris, France
| |
Collapse
|
6
|
Islam Sumon R, Bhattacharjee S, Hwang YB, Rahman H, Kim HC, Ryu WS, Kim DM, Cho NH, Choi HK. Densely Convolutional Spatial Attention Network for nuclei segmentation of histological images for computational pathology. Front Oncol 2023; 13:1009681. [PMID: 37305563 PMCID: PMC10248729 DOI: 10.3389/fonc.2023.1009681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Accepted: 05/05/2023] [Indexed: 06/13/2023] Open
Abstract
Introduction Automatic nuclear segmentation in digital microscopic tissue images can aid pathologists to extract high-quality features for nuclear morphometrics and other analyses. However, image segmentation is a challenging task in medical image processing and analysis. This study aimed to develop a deep learning-based method for nuclei segmentation of histological images for computational pathology. Methods The original U-Net model sometime has a caveat in exploring significant features. Herein, we present the Densely Convolutional Spatial Attention Network (DCSA-Net) model based on U-Net to perform the segmentation task. Furthermore, the developed model was tested on external multi-tissue dataset - MoNuSeg. To develop deep learning algorithms for well-segmenting nuclei, a large quantity of data are mandatory, which is expensive and less feasible. We collected hematoxylin and eosin-stained image data sets from two hospitals to train the model with a variety of nuclear appearances. Because of the limited number of annotated pathology images, we introduced a small publicly accessible data set of prostate cancer (PCa) with more than 16,000 labeled nuclei. Nevertheless, to construct our proposed model, we developed the DCSA module, an attention mechanism for capturing useful information from raw images. We also used several other artificial intelligence-based segmentation methods and tools to compare their results to our proposed technique. Results To prioritize the performance of nuclei segmentation, we evaluated the model's outputs based on the Accuracy, Dice coefficient (DC), and Jaccard coefficient (JC) scores. The proposed technique outperformed the other methods and achieved superior nuclei segmentation with accuracy, DC, and JC of 96.4% (95% confidence interval [CI]: 96.2 - 96.6), 81.8 (95% CI: 80.8 - 83.0), and 69.3 (95% CI: 68.2 - 70.0), respectively, on the internal test data set. Conclusion Our proposed method demonstrates superior performance in segmenting cell nuclei of histological images from internal and external datasets, and outperforms many standard segmentation algorithms used for comparative analysis.
Collapse
Affiliation(s)
- Rashadul Islam Sumon
- Department of Digital Anti-Aging Healthcare, Ubiquitous-Anti-aging-Healthcare Research Center (u-AHRC), Inje University, Gimhae, Republic of Korea
| | - Subrata Bhattacharjee
- Department of Computer Engineering, Ubiquitous-Anti-aging-Healthcare Research Center (u-AHRC), Inje University, Gimhae, Republic of Korea
| | - Yeong-Byn Hwang
- Department of Digital Anti-Aging Healthcare, Ubiquitous-Anti-aging-Healthcare Research Center (u-AHRC), Inje University, Gimhae, Republic of Korea
| | - Hafizur Rahman
- Department of Digital Anti-Aging Healthcare, Ubiquitous-Anti-aging-Healthcare Research Center (u-AHRC), Inje University, Gimhae, Republic of Korea
| | - Hee-Cheol Kim
- Department of Digital Anti-Aging Healthcare, Ubiquitous-Anti-aging-Healthcare Research Center (u-AHRC), Inje University, Gimhae, Republic of Korea
| | - Wi-Sun Ryu
- Artificial Intelligence R&D Center, JLK Inc., Seoul, Republic of Korea
| | - Dong Min Kim
- Artificial Intelligence R&D Center, JLK Inc., Seoul, Republic of Korea
| | - Nam-Hoon Cho
- Department of Pathology, Yonsei University Hospital, Seoul, Republic of Korea
| | - Heung-Kook Choi
- Department of Computer Engineering, Ubiquitous-Anti-aging-Healthcare Research Center (u-AHRC), Inje University, Gimhae, Republic of Korea
- Artificial Intelligence R&D Center, JLK Inc., Seoul, Republic of Korea
| |
Collapse
|
7
|
Fel JT, Ellis CT, Turk-Browne NB. Automated and manual segmentation of the hippocampus in human infants. Dev Cogn Neurosci 2023; 60:101203. [PMID: 36791555 PMCID: PMC9957787 DOI: 10.1016/j.dcn.2023.101203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Revised: 01/25/2023] [Accepted: 01/25/2023] [Indexed: 01/30/2023] Open
Abstract
The hippocampus, critical for learning and memory, undergoes substantial changes early in life. Investigating the developmental trajectory of hippocampal structure and function requires an accurate method for segmenting this region from anatomical MRI scans. Although manual segmentation is regarded as the "gold standard" approach, it is laborious and subjective. This has fueled the pursuit of automated segmentation methods in adults. However, little is known about the reliability of these automated protocols in infants, particularly when anatomical scan quality is degraded by head motion or the use of shorter and quieter infant-friendly sequences. During a task-based fMRI protocol, we collected quiet T1-weighted anatomical scans from 42 sessions with awake infants aged 4-23 months. Two expert tracers first segmented the hippocampus in both hemispheres manually. The resulting inter-rater reliability (IRR) was only moderate, reflecting the difficulty of infant segmentation. We then used four protocols to predict these manual segmentations: average adult template, average infant template, FreeSurfer software, and Automated Segmentation of Hippocampal Subfields (ASHS) software. ASHS generated the most reliable hippocampal segmentations in infants, exceeding the manual IRR of experts. Automated methods thus provide robust hippocampal segmentations of noisy T1-weighted infant scans, opening new possibilities for interrogating early hippocampal development.
Collapse
Affiliation(s)
- J T Fel
- Department of Psychology, Yale University, New Haven, CT 06511, USA
| | - C T Ellis
- Department of Psychology, Stanford University, Stanford, CA 94305, USA
| | - N B Turk-Browne
- Department of Psychology, Yale University, New Haven, CT 06511, USA; Wu Tsai Institute, Yale University, New Haven, CT 06511, USA.
| |
Collapse
|
8
|
Sharrock MF, Mould WA, Hildreth M, Ryu EP, Walborn N, Awad IA, Hanley DF, Muschelli J. Bayesian deep learning outperforms clinical trial estimators of intracerebral and intraventricular hemorrhage volume. J Neuroimaging 2022; 32:968-976. [PMID: 35434846 PMCID: PMC9474710 DOI: 10.1111/jon.12997] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2021] [Revised: 03/10/2022] [Accepted: 03/21/2022] [Indexed: 11/27/2022] Open
Abstract
BACKGROUND AND PURPOSE Intracerebral hemorrhage (ICH) and intraventricular hemorrhage (IVH) clinical trials rely on manual linear and semi-quantitative (LSQ) estimators like the ABC/2, modified Graeb and IVH scores for timely volumetric estimation from CT. Deep learning (DL) volumetrics of ICH have recently approached the accuracy of gold-standard planimetry. However, DL and LSQ strategies have been limited by unquantified uncertainty, in particular when ICH and IVH estimates intersect. Bayesian deep learning methods can be used to approximate uncertainty, presenting an opportunity to improve quality assurance in clinical trials. METHODS A DL model was trained to simultaneously segment ICH and IVH using diagnostic CT data from the Minimally Invasive Surgery Plus Alteplase for ICH Evacuation (MISTIE) III and Clot Lysis: Evaluating Accelerated Resolution of IVH (CLEAR) III clinical trials. Bayesian uncertainty approximation was performed using Monte-Carlo dropout. We compared the performance of our model with estimators used in the CLEAR IVH and MISTIE II trials. The reliability of planimetry, DL, and LSQ volumetrics in the setting of high ICH and IVH intersection is quantified using consensus estimates. RESULTS Our DL model produced volume correlations and median Dice scores of .994 and .946 for ICH in MISTIE II, and .980 and .863 for IVH in CLEAR IVH, respectively, outperforming LSQ estimates from the clinical trials. We found significant linear relationships between ICH uncertainty, Dice scores (r = -.849), and relative volume difference (r = .735). CONCLUSION In our validation clinical trial dataset, DL models with Bayesian uncertainty approximation provided superior volumetric estimates to LSQ methods with real-time estimates of model uncertainty.
Collapse
Affiliation(s)
- Matthew F. Sharrock
- Division of Neurocritical Care, Department of Neurology, University of North Carolina at Chapel Hill, NC, USA
| | - W. Andrew Mould
- Division of Brain Injury Outcomes, Department of Neurology, Johns Hopkins University, Baltimore, MD, USA
| | - Meghan Hildreth
- Division of Brain Injury Outcomes, Department of Neurology, Johns Hopkins University, Baltimore, MD, USA
| | - E. Paul Ryu
- Division of Brain Injury Outcomes, Department of Neurology, Johns Hopkins University, Baltimore, MD, USA
| | - Nathan Walborn
- Division of Brain Injury Outcomes, Department of Neurology, Johns Hopkins University, Baltimore, MD, USA
| | - Issam A. Awad
- Neurovascular Surgery Program, Section of Neurosurgery, Department of Surgery, University of Chicago Medicine and Biological Sciences, Chicago, IL, USA
| | - Daniel F. Hanley
- Division of Brain Injury Outcomes, Department of Neurology, Johns Hopkins University, Baltimore, MD, USA
| | - John Muschelli
- Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
9
|
Li Y, Wu Y, Huang M, Zhang Y, Bai Z. Automatic prostate and peri-prostatic fat segmentation based on pyramid mechanism fusion network for T2-weighted MRI. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 223:106918. [PMID: 35779461 DOI: 10.1016/j.cmpb.2022.106918] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/01/2022] [Revised: 05/10/2022] [Accepted: 05/25/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic and accurate segmentation of prostate and peri-prostatic fat in male pelvic MRI images is a critical step in the diagnosis and prognosis of prostate cancer. The boundary of prostate tissue is not clear, which makes the task of automatic segmentation very challenging. The main issues, especially for the peri-prostatic fat, which is being offered for the first time, are hazy boundaries and a large form variation. METHODS We propose a pyramid mechanism fusion network (PMF-Net) to learn global features and more comprehensive context information. In the proposed PMF-Net, we devised two pyramid techniques in particular. A pyramid mechanism module made of dilated convolutions of varying rates is inserted before each down sample of the fundamental network architecture encoder. The module is intended to address the issue of information loss during the feature coding process, particularly in the case of segmentation object boundary information. In the transition stage from encoder to decoder, pyramid fusion module is designed to extract global features. The features of the decoder not only integrate the features of the previous stage after up sampling and the output features of pyramid mechanism, but also include the features of skipping connection transmission under the same scale of the encoder. RESULTS The segmentation results of prostate and peri-prostatic fat on numerous diverse male pelvic MRI datasets show that our proposed PMF-Net has higher performance than existing methods. The average surface distance (ASD) and Dice similarity coefficient (DSC) of prostate segmentation results reached 10.06 and 90.21%, respectively. The ASD and DSC of the peri-prostatic fat segmentation results reached 50.96 and 82.41%. CONCLUSIONS The results of our segmentation are substantially connected and consistent with those of expert manual segmentation. Furthermore, peri-prostatic fat segmentation is a new issue, and good automatic segmentation has substantial therapeutic implications.
Collapse
Affiliation(s)
- Yuchun Li
- State Key Laboratory of Marine Resource Utilization in South China Sea, College of Information Science and Technology, Hainan University, Haikou 570288, China
| | - Yuanyuan Wu
- State Key Laboratory of Marine Resource Utilization in South China Sea, College of Information Science and Technology, Hainan University, Haikou 570288, China
| | - Mengxing Huang
- State Key Laboratory of Marine Resource Utilization in South China Sea, College of Information Science and Technology, Hainan University, Haikou 570288, China.
| | - Yu Zhang
- School of Computer science and Technology, Hainan University, Haikou 570288, China
| | - Zhiming Bai
- Haikou Municipal People's Hospital and Central South University Xiangya Medical College Affiliated Hospital, Haikou 570288, China
| |
Collapse
|
10
|
Li G, Chen MH, Li G, Wu D, Lian C, Sun Q, Rushmore RJ, Wang L. Volumetric Analysis of Amygdala and Hippocampal Subfields for Infants with Autism. J Autism Dev Disord 2022; 53:2475-2489. [PMID: 35389185 DOI: 10.1007/s10803-022-05535-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/15/2022] [Indexed: 10/18/2022]
Abstract
Previous studies have demonstrated abnormal brain overgrowth in children with autism spectrum disorder (ASD), but the development of specific brain regions, such as the amygdala and hippocampal subfields in infants, is incompletely documented. To address this issue, we performed the first MRI study of amygdala and hippocampal subfields in infants from 6 to 24 months of age using a longitudinal dataset. A novel deep learning approach, Dilated-Dense U-Net, was proposed to address the challenge of low tissue contrast and small structural size of these subfields. We performed a volume-based analysis on the segmentation results. Our results show that infants who were later diagnosed with ASD had larger left and right volumes of amygdala and hippocampal subfields than typically developing controls.
Collapse
Affiliation(s)
- Guannan Li
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China.,Department of Radiology and Biomedical Research Imaging Center, Bioinformatics Building, University of North Carolina at Chapel Hill, 130 Mason Farm Rd, Chapel Hill, NC, 27599, USA
| | - Meng-Hsiang Chen
- Department of Diagnostic Radiology, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan
| | - Gang Li
- Department of Radiology and Biomedical Research Imaging Center, Bioinformatics Building, University of North Carolina at Chapel Hill, 130 Mason Farm Rd, Chapel Hill, NC, 27599, USA
| | - Di Wu
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Chunfeng Lian
- Department of Radiology and Biomedical Research Imaging Center, Bioinformatics Building, University of North Carolina at Chapel Hill, 130 Mason Farm Rd, Chapel Hill, NC, 27599, USA
| | - Quansen Sun
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China
| | - R Jarrett Rushmore
- Department of Anatomy and Neurobiology, Boston University School of Medicine, Boston, MA, USA.,Department of Psychiatry, Brigham and Women's Hospital, Boston, MA, USA.,Center for Morphometric Analysis, Massachusetts General Hospital, 149 Thirteenth Street, Charlestown, MA, 02129, USA
| | - Li Wang
- Department of Radiology and Biomedical Research Imaging Center, Bioinformatics Building, University of North Carolina at Chapel Hill, 130 Mason Farm Rd, Chapel Hill, NC, 27599, USA.
| |
Collapse
|
11
|
Du G, Zhan Y, Zhang Y, Guo J, Chen X, Liang J, Zhao H. Automated segmentation of the gastrocnemius and soleus in shank ultrasound images through deep residual neural network. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
12
|
Manjón JV, Romero JE, Coupe P. A novel deep learning based hippocampus subfield segmentation method. Sci Rep 2022; 12:1333. [PMID: 35079061 PMCID: PMC8789929 DOI: 10.1038/s41598-022-05287-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Accepted: 01/04/2022] [Indexed: 12/02/2022] Open
Abstract
The automatic assessment of hippocampus volume is an important tool in the study of several neurodegenerative diseases such as Alzheimer's disease. Specifically, the measurement of hippocampus subfields properties is of great interest since it can show earlier pathological changes in the brain. However, segmentation of these subfields is very difficult due to their complex structure and for the need of high-resolution magnetic resonance images manually labeled. In this work, we present a novel pipeline for automatic hippocampus subfield segmentation based on a deeply supervised convolutional neural network. Results of the proposed method are shown for two available hippocampus subfield delineation protocols. The method has been compared to other state-of-the-art methods showing improved results in terms of accuracy and execution time.
Collapse
Affiliation(s)
- José V Manjón
- Instituto de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València, Camino de Vera s/n, 46022, Valencia, Spain.
| | - José E Romero
- Instituto de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València, Camino de Vera s/n, 46022, Valencia, Spain
| | - Pierrick Coupe
- Univ. Bordeaux, LaBRI, UMR 5800, PICTURA, 33400, Talence, France.,CNRS, LaBRI, UMR 5800, PICTURA, 33400, Talence, France
| |
Collapse
|
13
|
Wu J, Zhou S, Zuo S, Chen Y, Sun W, Luo J, Duan J, Wang H, Wang D. U-Net combined with multi-scale attention mechanism for liver segmentation in CT images. BMC Med Inform Decis Mak 2021; 21:283. [PMID: 34654419 PMCID: PMC8520298 DOI: 10.1186/s12911-021-01649-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2021] [Accepted: 10/04/2021] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND The liver is an important organ that undertakes the metabolic function of the human body. Liver cancer has become one of the cancers with the highest mortality. In clinic, it is an important work to extract the liver region accurately before the diagnosis and treatment of liver lesions. However, manual liver segmentation is a time-consuming and boring process. Not only that, but the segmentation results usually varies from person to person due to different work experience. In order to assist in clinical automatic liver segmentation, this paper proposes a U-shaped network with multi-scale attention mechanism for liver organ segmentation in CT images, which is called MSA-UNet. Our method makes a new design of U-Net encoder, decoder, skip connection, and context transition structure. These structures greatly enhance the feature extraction ability of encoder and the efficiency of decoder to recover spatial location information. We have designed many experiments on publicly available datasets to show the effectiveness of MSA-UNet. Compared with some other advanced segmentation methods, MSA-UNet finally achieved the best segmentation effect, reaching 98.00% dice similarity coefficient (DSC) and 96.08% intersection over union (IOU).
Collapse
Affiliation(s)
- Jiawei Wu
- School of Medical Imaging, Xuzhou Medical University, Xuzhou, China
| | - Shengqiang Zhou
- School of Economics and Finance, Xi’an Jiaotong University, Xi’an, China
- Jiangsu Union Technical Institute, Xuzhou, China
| | - Songlin Zuo
- School of the First Clinical Medical, Xuzhou Medical University, Xuzhou, China
| | - Yiyin Chen
- School of the First Clinical Medical, Xuzhou Medical University, Xuzhou, China
| | - Weiqin Sun
- School of Medical Imaging, Xuzhou Medical University, Xuzhou, China
| | - Jiang Luo
- School of Medical Imaging, Xuzhou Medical University, Xuzhou, China
| | - Jiantuan Duan
- School of Economics and Finance, Xi’an Jiaotong University, Xi’an, China
| | - Hui Wang
- School of Medical Imaging, Xuzhou Medical University, Xuzhou, China
| | - Deguang Wang
- School of Medical Imaging, Xuzhou Medical University, Xuzhou, China
| |
Collapse
|
14
|
Qiu Q, Yang Z, Wu S, Qian D, Wei J, Gong G, Wang L, Yin Y. Automatic segmentation of hippocampus in hippocampal sparing whole brain radiotherapy: A multitask edge-aware learning. Med Phys 2021; 48:1771-1780. [PMID: 33555048 DOI: 10.1002/mp.14760] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Revised: 01/08/2021] [Accepted: 01/29/2021] [Indexed: 02/03/2023] Open
Abstract
PURPOSE This study aimed to improve the accuracy of the hippocampus segmentation through multitask edge-aware learning. METHOD We developed a multitask framework for computerized hippocampus segmentation. We used three-dimensional (3D) U-net as our backbone model with two training objectives: (a) to minimize the difference between the targeted binary mask and the model prediction; and (b) to optimize an auxiliary edge-prediction task which is designed to guide the model detection of the weak boundary of the hippocampus in model optimization. To balance the multiple task objectives, we proposed an improved gradient normalization by adaptively adjusting the weight of losses from different tasks. A total of 247 T1-weighted MRIs including 131 without contrast and 116 with contrast were collected from 247 patients to train and validate the proposed method. Segmentation was quantitatively evaluated with the dice coefficient (Dice), Hausdorff distance (HD), and average Hausdorff distance (AVD). The 3D U-net was used for baseline comparison. We used a Wilcoxon signed-rank test to compare repeated measurements (Dice, HD, and AVD) by different segmentations. RESULTS Through fivefold cross-validation, our multitask edge-aware learning achieved Dice of 0.8483 ± 0.0036, HD of 7.5706 ± 1.2330 mm, and AVD of 0.1522 ± 0.0165 mm, respectively. Conversely, the baseline results were 0.8340 ± 0.0072, 10.4631 ± 2.3736 mm, and 0.1884 ± 0.0286 mm, respectively. With a Wilcoxon signed-rank test, we found that the differences between our method and the baseline were statistically significant (P < 0.05). CONCLUSION Our results demonstrated the efficiency of multitask edge-aware learning in hippocampus segmentation for hippocampal sparing whole-brain radiotherapy. The proposed framework may also be useful for other low-contrast small organ segmentations on medical imaging modalities.
Collapse
Affiliation(s)
- Qingtao Qiu
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, P.R. China
| | - Ziduo Yang
- Perception Vision Medical Technologies Co. Ltd., Guangzhou, Guangdong, P.R. China
| | - Shuyu Wu
- Perception Vision Medical Technologies Co. Ltd., Guangzhou, Guangdong, P.R. China
| | - Dongdong Qian
- Perception Vision Medical Technologies Co. Ltd., Guangzhou, Guangdong, P.R. China
| | - Jun Wei
- Perception Vision Medical Technologies Co. Ltd., Guangzhou, Guangdong, P.R. China
| | - Guanzhong Gong
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, P.R. China
| | - Lizhen Wang
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, P.R. China
| | - Yong Yin
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, P.R. China
| |
Collapse
|
15
|
Ma B, Zhao Y, Yang Y, Zhang X, Dong X, Zeng D, Ma S, Li S. MRI image synthesis with dual discriminator adversarial learning and difficulty-aware attention mechanism for hippocampal subfields segmentation. Comput Med Imaging Graph 2020; 86:101800. [PMID: 33130416 DOI: 10.1016/j.compmedimag.2020.101800] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2020] [Revised: 07/27/2020] [Accepted: 09/24/2020] [Indexed: 12/01/2022]
Abstract
BACKGROUND AND OBJECTIVE Hippocampal subfields (HS) segmentation accuracy on high resolution (HR) MRI images is higher than that on low resolution (LR) MRI images. However, HR MRI data collection is more expensive and time-consuming. Thus, we intend to generate HR MRI images from the corresponding LR MRI images for HS segmentation. METHODS AND RESULTS To generate high-quality HR MRI hippocampus region images, we use a dual discriminator adversarial learning model with difficulty-aware attention mechanism in hippocampus regions (da-GAN). A local discriminator is applied in da-GAN to evaluate the visual quality of hippocampus region voxels of the synthetic images. And the difficulty-aware attention mechanism based on the local discriminator can better model the generation of hard-to-synthesis voxels in hippocampus regions. Additionally, we design a SemiDenseNet model with 3D Dense CRF postprocessing and an Unet-based model to perform HS segmentation. The experiments are implemented on Kulaga-Yoskovitz dataset. Compared with conditional generative adversarial network (c-GAN), the PSNR of generated HR T2w images acquired by our da-GAN achieves 0.406 and 0.347 improvement in left and right hippocampus regions. When using two segmentation models to segment HS, the DSC values achieved on the generated HR T1w and T2w images are both improved than that on LR T1w images. CONCLUSION Experimental results show that da-GAN model can generate higher-quality MRI images, especially in hippocampus regions, and the generated MRI images can improve HS segmentation accuracy.
Collapse
Affiliation(s)
- Baoqiang Ma
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Yan Zhao
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Yujing Yang
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Xiaohui Zhang
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Xiaoxi Dong
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Debin Zeng
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Siyu Ma
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China
| | - Shuyu Li
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, China.
| |
Collapse
|
16
|
Kong Y, Genchev GZ, Wang X, Zhao H, Lu H. Nuclear Segmentation in Histopathological Images Using Two-Stage Stacked U-Nets With Attention Mechanism. Front Bioeng Biotechnol 2020; 8:573866. [PMID: 33195135 PMCID: PMC7649338 DOI: 10.3389/fbioe.2020.573866] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Accepted: 10/05/2020] [Indexed: 12/05/2022] Open
Abstract
Nuclei segmentation is a fundamental but challenging task in histopathological image analysis. One of the main problems is the existence of overlapping regions which increases the difficulty of independent nuclei separation. In this study, to solve the segmentation of nuclei and overlapping regions, we introduce a nuclei segmentation method based on two-stage learning framework consisting of two connected Stacked U-Nets (SUNets). The proposed SUNets consists of four parallel backbone nets, which are merged by the attention generation model. In the first stage, a Stacked U-Net is utilized to predict pixel-wise segmentation of nuclei. The output binary map together with RGB values of the original images are concatenated as the input of the second stage of SUNets. Due to the sizable imbalance of overlapping and background regions, the first network is trained with cross-entropy loss, while the second network is trained with focal loss. We applied the method on two publicly available datasets and achieved state-of-the-art performance for nuclei segmentation-mean Aggregated Jaccard Index (AJI) results were 0.5965 and 0.6210, and F1 scores were 0.8247 and 0.8060, respectively; our method also segmented the overlapping regions between nuclei, with average AJI = 0.3254. The proposed two-stage learning framework outperforms many current segmentation methods, and the consistent good segmentation performance on images from different organs indicates the generalized adaptability of our approach.
Collapse
Affiliation(s)
- Yan Kong
- SJTU-Yale Joint Center for Biostatistics and Data Science, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China
| | - Georgi Z. Genchev
- SJTU-Yale Joint Center for Biostatistics and Data Science, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China
- Center for Biomedical Informatics, Shanghai Engineering Research Center for Big Data in Pediatric Precision Medicine, Shanghai Children’s Hospital, Shanghai, China
- Bulgarian Institute for Genomics and Precision Medicine, Sofia, Bulgaria
| | - Xiaolei Wang
- SJTU-Yale Joint Center for Biostatistics and Data Science, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China
| | - Hongyu Zhao
- Department of Biostatistics, Yale University, New Haven, CT, United States
| | - Hui Lu
- SJTU-Yale Joint Center for Biostatistics and Data Science, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China
- Center for Biomedical Informatics, Shanghai Engineering Research Center for Big Data in Pediatric Precision Medicine, Shanghai Children’s Hospital, Shanghai, China
| |
Collapse
|
17
|
Majdi MS, Keerthivasan MB, Rutt BK, Zahr NM, Rodriguez JJ, Saranathan M. Automated thalamic nuclei segmentation using multi-planar cascaded convolutional neural networks. Magn Reson Imaging 2020; 73:45-54. [PMID: 32828985 DOI: 10.1016/j.mri.2020.08.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2020] [Revised: 07/25/2020] [Accepted: 08/17/2020] [Indexed: 12/15/2022]
Abstract
PURPOSE To develop a fast and accurate convolutional neural network based method for segmentation of thalamic nuclei. METHODS A cascaded multi-planar scheme with a modified residual U-Net architecture was used to segment thalamic nuclei on conventional and white-matter-nulled (WMn) magnetization prepared rapid gradient echo (MPRAGE) data. A single network was optimized to work with images from healthy controls and patients with multiple sclerosis (MS) and essential tremor (ET), acquired at both 3 T and 7 T field strengths. WMn-MPRAGE images were manually delineated by a trained neuroradiologist using the Morel histological atlas as a guide to generate reference ground truth labels. Dice similarity coefficient and volume similarity index (VSI) were used to evaluate performance. Clinical utility was demonstrated by applying this method to study the effect of MS on thalamic nuclei atrophy. RESULTS Segmentation of each thalamus into twelve nuclei was achieved in under a minute. For 7 T WMn-MPRAGE, the proposed method outperforms current state-of-the-art on patients with ET with statistically significant improvements in Dice for five nuclei (increase in the range of 0.05-0.18) and VSI for four nuclei (increase in the range of 0.05-0.19), while performing comparably for healthy and MS subjects. Dice and VSI achieved using 7 T WMn-MPRAGE data are comparable to those using 3 T WMn-MPRAGE data. For conventional MPRAGE, the proposed method shows a statistically significant Dice improvement in the range of 0.14-0.63 over FreeSurfer for all nuclei and disease types. Effect of noise on network performance shows robustness to images with SNR as low as half the baseline SNR. Atrophy of four thalamic nuclei and whole thalamus was observed for MS patients compared to healthy control subjects, after controlling for the effect of parallel imaging, intracranial volume, gender, and age (p < 0.004). CONCLUSION The proposed segmentation method is fast, accurate, performs well across disease types and field strengths, and shows great potential for improving our understanding of thalamic nuclei involvement in neurological diseases.
Collapse
Affiliation(s)
- Mohammad S Majdi
- Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, United States of America
| | - Mahesh B Keerthivasan
- Department of Medical Imaging, University of Arizona, Tucson, AZ, United States of America; Siemens Healthcare, Tucson, AZ, USA
| | - Brian K Rutt
- Department of Radiology, Stanford University, Stanford, CA, United States of America
| | - Natalie M Zahr
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, United States of America
| | - Jeffrey J Rodriguez
- Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, United States of America
| | - Manojkumar Saranathan
- Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, United States of America; Department of Medical Imaging, University of Arizona, Tucson, AZ, United States of America.
| |
Collapse
|
18
|
Yang Z, Zhuang X, Mishra V, Sreenivasan K, Cordes D. CAST: A multi-scale convolutional neural network based automated hippocampal subfield segmentation toolbox. Neuroimage 2020; 218:116947. [PMID: 32474081 DOI: 10.1016/j.neuroimage.2020.116947] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2020] [Revised: 05/12/2020] [Accepted: 05/13/2020] [Indexed: 11/29/2022] Open
Abstract
In this study, we developed a multi-scale Convolutional neural network based Automated hippocampal subfield Segmentation Toolbox (CAST) for automated segmentation of hippocampal subfields. Although training CAST required approximately three days on a single workstation with a high-quality GPU card, CAST can segment a new subject in less than 1 min even with GPU acceleration disabled, thus this method is more time efficient than current automated methods and manual segmentation. This toolbox is highly flexible with either a single modality or multiple modalities and can be easily set up to be trained with a researcher's unique data. A 3D multi-scale deep convolutional neural network is the key algorithm used in the toolbox. The main merit of multi-scale images is the capability to capture more global structural information from down-sampled images without dramatically increasing memory and computational burden. The original images capture more local information to refine the boundary between subfields. Residual learning is applied to alleviate the vanishing gradient problem and improve the performance with a deeper network. We applied CAST with the same settings on two datasets, one 7T dataset (the UMC dataset) with only the T2 image and one 3T dataset (the MNI dataset) with both T1 and T2 images available. The segmentation accuracy of both CAST and the state-of-the-art automated method ASHS, in terms of the dice similarity coefficient (DSC), were comparable. CAST significantly improved the reliability of segmenting small subfields, such as CA2, CA3, and the entorhinal cortex (ERC), in terms of the intraclass correlation coefficient (ICC). Both ASHS and manual segmentation process some subfields (e.g. CA2 and ERC) with high DSC values but low ICC values, consequently increasing the difficulty of judging segmentation quality. CAST produces very consistent DSC and ICC values, with a maximal discrepancy of 0.01 (DSC-ICC) across all subfields. The pre-trained model, source code, and settings for the CAST toolbox are publicly available.
Collapse
Affiliation(s)
- Zhengshi Yang
- Cleveland Clinic Lou Ruvo Center for Brain Health, Las Vegas, NV, 89106, USA
| | - Xiaowei Zhuang
- Cleveland Clinic Lou Ruvo Center for Brain Health, Las Vegas, NV, 89106, USA
| | - Virendra Mishra
- Cleveland Clinic Lou Ruvo Center for Brain Health, Las Vegas, NV, 89106, USA
| | - Karthik Sreenivasan
- Cleveland Clinic Lou Ruvo Center for Brain Health, Las Vegas, NV, 89106, USA
| | - Dietmar Cordes
- Cleveland Clinic Lou Ruvo Center for Brain Health, Las Vegas, NV, 89106, USA; Department of Psychology and Neuroscience, University of Colorado, Boulder, CO, 80309, USA.
| |
Collapse
|
19
|
Kalmet PHS, Sanduleanu S, Primakov S, Wu G, Jochems A, Refaee T, Ibrahim A, Hulst LV, Lambin P, Poeze M. Deep learning in fracture detection: a narrative review. Acta Orthop 2020; 91:215-220. [PMID: 31928116 PMCID: PMC7144272 DOI: 10.1080/17453674.2019.1711323] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Artificial intelligence (AI) is a general term that implies the use of a computer to model intelligent behavior with minimal human intervention. AI, particularly deep learning, has recently made substantial strides in perception tasks allowing machines to better represent and interpret complex data. Deep learning is a subset of AI represented by the combination of artificial neuron layers. In the last years, deep learning has gained great momentum. In the field of orthopaedics and traumatology, some studies have been done using deep learning to detect fractures in radiographs. Deep learning studies to detect and classify fractures on computed tomography (CT) scans are even more limited. In this narrative review, we provide a brief overview of deep learning technology: we (1) describe the ways in which deep learning until now has been applied to fracture detection on radiographs and CT examinations; (2) discuss what value deep learning offers to this field; and finally (3) comment on future directions of this technology.
Collapse
Affiliation(s)
| | - Sebastian Sanduleanu
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Sergey Primakov
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Guangyao Wu
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Arthur Jochems
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Turkey Refaee
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Abdalla Ibrahim
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Luca v. Hulst
- Maastricht University Medical Center+, Department of Trauma Surgery, Maastricht
| | - Philippe Lambin
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Martijn Poeze
- Maastricht University Medical Center+, Department of Trauma Surgery, Maastricht
- Nutrim School for Nutrition, Toxicology and Metabolism, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
20
|
Chung YS, Lee U, Heo S, Silva RR, Na CI, Kim Y. Image-Based Machine Learning Characterizes Root Nodule in Soybean Exposed to Silicon. FRONTIERS IN PLANT SCIENCE 2020; 11:520161. [PMID: 33193467 PMCID: PMC7655541 DOI: 10.3389/fpls.2020.520161] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2019] [Accepted: 10/06/2020] [Indexed: 05/07/2023]
Abstract
Silicon promotes nodule formation in legume roots which is crucial for nitrogen fixation. However, it is very time-consuming and laborious to count the number of nodules and to measure nodule size manually, which led nodule characterization not to be study as much as other agronomical characters. Thus, the current study incorporated various techniques including machine learning to determine the number and size of root nodules and identify various root phenotypes from root images that may be associated with nodule formation with and without silicon treatment. Among those techniques, the machine learning for characterizing nodule is the first attempt, which enabled us to find high correlations among root phenotypes including root length, number of forks, and average link angles, and nodule characters such as number of nodules and nodule size with silicon treatments. The methods here could greatly accelerate further investigation such as delineating the optimal concentration of silicon for nodule formation.
Collapse
Affiliation(s)
- Yong Suk Chung
- Department of Plant Resources and Environment, Jeju National University, Jeju-si, South Korea
| | - Unseok Lee
- Smart Farm Research Center, Korea Institute of Science and Technology, Gangneung-si, South Korea
| | - Seong Heo
- Department of Horticulture, Kongju National University, Yesan, South Korea
| | | | - Chae-In Na
- Department of Agronomy, Gyeongsang National University, Jinju-si, South Korea
| | - Yoonha Kim
- School of Applied Life Science, Kyungpook National University, Daegu, South Korea
- *Correspondence: Yoonha Kim, ;
| |
Collapse
|
21
|
Zhu H, Tang Z, Cheng H, Wu Y, Fan Y. Multi-atlas label fusion with random local binary pattern features: Application to hippocampus segmentation. Sci Rep 2019; 9:16839. [PMID: 31727982 PMCID: PMC6856174 DOI: 10.1038/s41598-019-53387-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2018] [Accepted: 10/30/2019] [Indexed: 01/15/2023] Open
Abstract
Automatic and reliable segmentation of the hippocampus from magnetic resonance (MR) brain images is extremely important in a variety of neuroimage studies. To improve the hippocampus segmentation performance, a local binary pattern based feature extraction method is developed for machine learning based multi-atlas hippocampus segmentation. Under the framework of multi-atlas image segmentation (MAIS), a set of selected atlases are registered to images to be segmented using a non-linear image registration algorithm. The registered atlases are then used as training data to build linear regression models for segmenting the images based on the image features, referred to as random local binary pattern (RLBP), extracted using a novel image feature extraction method. The RLBP based MAIS algorithm has been validated for segmenting hippocampus based on a data set of 135 T1 MR images which are from the Alzheimer’s Disease Neuroimaging Initiative database (adni.loni.usc.edu). By using manual segmentation labels produced by experienced tracers as the standard of truth, six segmentation evaluation metrics were used to evaluate the image segmentation results by comparing automatic segmentation results with the manual segmentation labels. We further computed Cohen’s d effect size to investigate the sensitivity of each segmenting method in detecting volumetric differences of the hippocampus between different groups of subjects. The evaluation results showed that our method was competitive to state-of-the-art label fusion methods in terms of accuracy. Hippocampal volumetric analysis showed that the proposed RLBP method performed well in detecting the volumetric differences of the hippocampus between groups of Alzheimer’s disease patients, mild cognitive impairment subjects, and normal controls. These results have demonstrated that the RLBP based multi-atlas image segmentation method could facilitate efficient and accurate extraction of the hippocampus and may help predict Alzheimer’s disease. The codes of the proposed method is available (https://www.nitrc.org/frs/?group_id=1242).
Collapse
Affiliation(s)
- Hancan Zhu
- School of Mathematics Physics and Information, Shaoxing University, Shaoxing, Zhejiang, 312000, China
| | - Zhenyu Tang
- Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, 100191, China
| | - Hewei Cheng
- Department of Biomedical Engineering, School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China
| | - Yihong Wu
- National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
| | - Yong Fan
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA.
| |
Collapse
|
22
|
Li H, Fang J, Liu S, Liang X, Yang X, Mai Z, Van MT, Wang T, Chen Z, Ni D. CR-Unet: A Composite Network for Ovary and Follicle Segmentation in Ultrasound Images. IEEE J Biomed Health Inform 2019; 24:974-983. [PMID: 31603808 DOI: 10.1109/jbhi.2019.2946092] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Transvaginal ultrasound (TVUS) is widely used in infertility treatment. The size and shape of the ovary and follicles must be measured manually for assessing their physiological status by sonographers. However, this process is extremely time-consuming and operator-dependent. In this study, we propose a novel composite network, namely CR-Unet, to simultaneously segment the ovary and follicles in TVUS. The CR-Unet incorporates the spatial recurrent neural network (RNN) into a plain U-Net. It can effectively learn multi-scale and long-range spatial contexts to combat the challenges of this task, such as the poor image quality, low contrast, boundary ambiguity, and complex anatomy shapes. We further adopt deep supervision strategy to make model training more effective and efficient. In addition, self-supervision is employed to iteratively refine the segmentation results. Experiments on 3204 TVUS images from 219 patients demonstrate the proposed method achieved the best segmentation performance compared to other state-of-the-art methods for both the ovary and follicles, with a Dice Similarity Coefficient (DSC) of 0.912 and 0.858, respectively.
Collapse
|
23
|
Hashempour N, Tuulari JJ, Merisaari H, Lidauer K, Luukkonen I, Saunavaara J, Parkkola R, Lähdesmäki T, Lehtola SJ, Keskinen M, Lewis JD, Scheinin NM, Karlsson L, Karlsson H. A Novel Approach for Manual Segmentation of the Amygdala and Hippocampus in Neonate MRI. Front Neurosci 2019; 13:1025. [PMID: 31616245 PMCID: PMC6768976 DOI: 10.3389/fnins.2019.01025] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2018] [Accepted: 09/09/2019] [Indexed: 12/16/2022] Open
Abstract
The gross anatomy of the infant brain at term is fairly similar to that of the adult brain, but structures are immature, and the brain undergoes rapid growth during the first 2 years of life. Neonate magnetic resonance (MR) images have different contrasts compared to adult images, and automated segmentation of brain magnetic resonance imaging (MRI) can thus be considered challenging as less software options are available. Despite this, most anatomical regions are identifiable and thus amenable to manual segmentation. In the current study, we developed a protocol for segmenting the amygdala and hippocampus in T2-weighted neonatal MR images. The participants were 31 healthy infants between 2 and 5 weeks of age. Intra-rater reliability was measured in 12 randomly selected MR images, where 6 MR images were segmented at 1-month intervals between the delineations, and another 6 MR images at 6-month intervals. The protocol was also tested by two independent raters in 20 randomly selected T2-weighted images, and finally with T1 images. Intraclass correlation coefficient (ICC) and Dice similarity coefficient (DSC) for intra-rater, inter-rater, and T1 vs. T2 comparisons were computed. Moreover, manual segmentations were compared to automated segmentations performed by iBEAT toolbox in 10 T2-weighted MR images. The intra-rater reliability was high ICC ≥ 0.91, DSC ≥ 0.89, the inter-rater reliabilities were satisfactory ICC ≥ 0.90, DSC ≥ 0.75 for hippocampus and DSC ≥ 0.52 for amygdalae. Segmentations for T1 vs. T2-weighted images showed high consistency ICC ≥ 0.90, DSC ≥ 0.74. The manual and iBEAT segmentations showed no agreement, DSC ≥ 0.39. In conclusion, there is a clear need to improve and develop the procedures for automated segmentation of infant brain MR images.
Collapse
Affiliation(s)
- Niloofar Hashempour
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland.,Department of Psychiatry, Turku University Hospital, University of Turku, Turku, Finland
| | - Jetro J Tuulari
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland.,Department of Psychiatry, Turku University Hospital, University of Turku, Turku, Finland.,Turku Collegium for Science and Medicine, University of Turku, Turku, Finland
| | - Harri Merisaari
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Kristian Lidauer
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland
| | - Iiris Luukkonen
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland
| | - Jani Saunavaara
- Department of Medical Physics, Turku University Hospital, Turku, Finland
| | - Riitta Parkkola
- Department of Radiology, Turku University Hospital, University of Turku, Turku, Finland
| | - Tuire Lähdesmäki
- Department of Pediatric Neurology, Turku University Hospital, University of Turku, Turku, Finland
| | - Satu J Lehtola
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland
| | - Maria Keskinen
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland
| | - John D Lewis
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Noora M Scheinin
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland.,Department of Psychiatry, Turku University Hospital, University of Turku, Turku, Finland.,Turku PET Centre, University of Turku, Turku, Finland
| | - Linnea Karlsson
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland.,Department of Child Psychiatry, Turku University Hospital, University of Turku, Turku, Finland
| | - Hasse Karlsson
- FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Institute of Clinical Medicine, University of Turku, Turku, Finland.,Department of Psychiatry, Turku University Hospital, University of Turku, Turku, Finland
| |
Collapse
|