1
|
Ebner M, Wang G, Li W, Aertsen M, Patel PA, Aughwane R, Melbourne A, Doel T, Dymarkowski S, De Coppi P, David AL, Deprest J, Ourselin S, Vercauteren T. An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI. Neuroimage 2020; 206:116324. [PMID: 31704293 PMCID: PMC7103783 DOI: 10.1016/j.neuroimage.2019.116324] [Citation(s) in RCA: 95] [Impact Index Per Article: 23.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Revised: 09/26/2019] [Accepted: 10/29/2019] [Indexed: 12/17/2022] Open
Abstract
High-resolution volume reconstruction from multiple motion-corrupted stacks of 2D slices plays an increasing role for fetal brain Magnetic Resonance Imaging (MRI) studies. Currently existing reconstruction methods are time-consuming and often require user interactions to localize and extract the brain from several stacks of 2D slices. We propose a fully automatic framework for fetal brain reconstruction that consists of four stages: 1) fetal brain localization based on a coarse segmentation by a Convolutional Neural Network (CNN), 2) fine segmentation by another CNN trained with a multi-scale loss function, 3) novel, single-parameter outlier-robust super-resolution reconstruction, and 4) fast and automatic high-resolution visualization in standard anatomical space suitable for pathological brains. We validated our framework with images from fetuses with normal brains and with variable degrees of ventriculomegaly associated with open spina bifida, a congenital malformation affecting also the brain. Experiments show that each step of our proposed pipeline outperforms state-of-the-art methods in both segmentation and reconstruction comparisons including expert-reader quality assessments. The reconstruction results of our proposed method compare favorably with those obtained by manual, labor-intensive brain segmentation, which unlocks the potential use of automatic fetal brain reconstruction studies in clinical practice.
Collapse
Affiliation(s)
- Michael Ebner
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.
| | - Guotai Wang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China; Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.
| | - Wenqi Li
- Nvidia, Cambridge, UK; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| | - Michael Aertsen
- Department of Radiology, University Hospitals KU Leuven, Leuven, Belgium
| | - Premal A Patel
- Department of Radiology, Great Ormond Street Hospital for Children, London, UK; Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Rosalind Aughwane
- Institute for Women's Health, University College London, London, UK; Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Andrew Melbourne
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK; Medical Physics and Biomedical Engineering, University College London, London, UK
| | - Tom Doel
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Steven Dymarkowski
- Department of Radiology, University Hospitals KU Leuven, Leuven, Belgium
| | - Paolo De Coppi
- Institute of Child Health, University College London, London, UK
| | - Anna L David
- Institute for Women's Health, University College London, London, UK; Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK; Department of Obstetrics and Gynaecology, University Hospitals KU Leuven, Leuven, Belgium
| | - Jan Deprest
- Department of Obstetrics and Gynaecology, University Hospitals KU Leuven, Leuven, Belgium; Institute for Women's Health, University College London, London, UK; Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Sébastien Ourselin
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK; Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK; Department of Obstetrics and Gynaecology, University Hospitals KU Leuven, Leuven, Belgium
| |
Collapse
|
2
|
Khalili N, Turk E, Benders MJNL, Moeskops P, Claessens NHP, de Heus R, Franx A, Wagenaar N, Breur JMPJ, Viergever MA, Išgum I. Automatic extraction of the intracranial volume in fetal and neonatal MR scans using convolutional neural networks. NEUROIMAGE-CLINICAL 2019; 24:102061. [PMID: 31835284 PMCID: PMC6909142 DOI: 10.1016/j.nicl.2019.102061] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/07/2019] [Revised: 10/24/2019] [Accepted: 10/26/2019] [Indexed: 01/21/2023]
Abstract
Automatic intracranial volume segmentation. Fetal and neonatal MRI. Deep learning.
MR images of infants and fetuses allow non-invasive analysis of the brain. Quantitative analysis of brain development requires automatic brain tissue segmentation that is typically preceded by segmentation of the intracranial volume (ICV). Fast changes in the size and morphology of the developing brain, motion artifacts, and large variation in the field of view make ICV segmentation a challenging task. We propose an automatic method for segmentation of the ICV in fetal and neonatal MRI scans. The method was developed and tested with a diverse set of scans regarding image acquisition parameters (i.e. field strength, image acquisition plane, image resolution), infant age (23–45 weeks post menstrual age), and pathology (posthaemorrhagic ventricular dilatation, stroke, asphyxia, and Down syndrome). The results demonstrate that the method achieves accurate segmentation with a Dice coefficient (DC) ranging from 0.98 to 0.99 in neonatal and fetal scans regardless of image acquisition parameters or patient characteristics. Hence, the algorithm provides a generic tool for segmentation of the ICV that may be used as a preprocessing step for brain tissue segmentation in fetal and neonatal brain MR scans.
Collapse
Affiliation(s)
- Nadieh Khalili
- Image Sciences Institute, Utrecht University and University Medical Center Utrecht, Utrecht, the Netherlands.
| | - E Turk
- Department of Neonatology, Wilhelmina Childrens Hospital, University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| | - M J N L Benders
- Department of Neonatology, Wilhelmina Childrens Hospital, University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| | - P Moeskops
- Medical Image Analysis, Department of Biomedical Engineering, Eindhoven University of Technology, the Netherlands
| | - N H P Claessens
- Department of Neonatology, Wilhelmina Childrens Hospital, University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| | - R de Heus
- Department of Obstetrics, University Medical Center Utrecht, the Netherlands
| | - A Franx
- Department of Obstetrics, University Medical Center Utrecht, the Netherlands
| | - N Wagenaar
- Department of Neonatology, Wilhelmina Childrens Hospital, University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| | - J M P J Breur
- Department of Neonatology, Wilhelmina Childrens Hospital, University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| | - M A Viergever
- Image Sciences Institute, Utrecht University and University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| | - I Išgum
- Image Sciences Institute, Utrecht University and University Medical Center Utrecht, Utrecht, the Netherlands; Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands
| |
Collapse
|
3
|
Pishghadam M, Kazemi K, Nekooei S, Seilanian-Toosi F, Hoseini-Ghahfarokhi M, Zabizadeh M, Fatemi A. A new approach to automatic fetal brain extraction from MRI using a variational level set method. Med Phys 2019; 46:4983-4991. [PMID: 31419312 DOI: 10.1002/mp.13766] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2019] [Revised: 07/30/2019] [Accepted: 07/31/2019] [Indexed: 11/11/2022] Open
Abstract
BACKGROUND AND PURPOSE Appropriate images extracted from the MRI of mothers' wombs can be of great help in the medical diagnosis of fetal abnormalities. As maternal tissue may appear in such images, affecting visualization of myelination of the fetal brain, it is not possible to use methods routinely used for extraction of adult brains for fetal brains. The aim of the present study was to use a variational level set approach to extract fetal brain from T2-weighted MR images of the womb. METHODS Coronal T2-weighted images were acquired using fast MRI protocols (to avoid artifacts). The database includes 105 MR images from eight subjects. After correcting the inhomogeneity of the images, the fetal eyes were located, and from that information, the location of the fetus brain was automatically determined. Then, the variational level set was used for fetus brain extraction. The results were analyzed by a clinical specialist (radiologist) and the similarity (Dice and Jaccard coefficients), sensitivity and specificity were calculated. RESULTS AND CONCLUSIONS The means of the statistical analysis for the Dice and Jaccard coefficients, sensitivity and specificity, were 99.56%, 96.89%, 95.71%, and 97.96%, respectively. Thus, extraction of fetal brain from MR images was confirmed, both statistically and visually through cross-validation.
Collapse
Affiliation(s)
- Morteza Pishghadam
- Faculty of Medicine, North Khorasan University of Medical Sciences, Bojnurd, Iran
| | - Kamran Kazemi
- Department of Electrical and Electronics Engineering, Shiraz University of Technology, Shiraz, Iran
| | - Sirous Nekooei
- Department of Radiology, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
| | - Farrokh Seilanian-Toosi
- Department of Radiology, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
| | - Mojtaba Hoseini-Ghahfarokhi
- Department of Medical Physics, Faculty of Medicine, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran
| | - Mansour Zabizadeh
- Department of Radiology and Nuclear Medicine, School of Para Medical Sciences, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Ali Fatemi
- Department of Radiation Oncology and Radiology, University of Mississippi Medical Center (UMMC), Jackson, MS, USA
| |
Collapse
|
4
|
Zhang Z, Coyle JL, Sejdić E. Automatic hyoid bone detection in fluoroscopic images using deep learning. Sci Rep 2018; 8:12310. [PMID: 30120314 PMCID: PMC6097989 DOI: 10.1038/s41598-018-30182-6] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2018] [Accepted: 07/25/2018] [Indexed: 12/19/2022] Open
Abstract
The displacement of the hyoid bone is one of the key components evaluated in the swallow study, as its motion during swallowing is related to overall swallowing integrity. In daily research settings, experts visually detect the hyoid bone in the video frames and manually plot hyoid bone position frame by frame. This study aims to develop an automatic method to localize the location of the hyoid bone in the video sequence. To automatically detect the location of the hyoid bone in a frame, we proposed a single shot multibox detector, a deep convolutional neural network, which is employed to detect and classify the location of the hyoid bone. We also evaluated the performance of two other state-of-art detection methods for comparison. The experimental results clearly showed that the single shot multibox detector can detect the hyoid bone with an average precision of 89.14% and outperform other auto-detection algorithms. We conclude that this automatic hyoid bone tracking system is accurate enough to be widely applied as a pre-processing step for image processing in dysphagia research, as well as a promising development that may be useful in the diagnosis of dysphagia.
Collapse
Affiliation(s)
- Zhenwei Zhang
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA, 15261, USA
| | - James L Coyle
- Department of Communication Science and Disorders, University of Pittsburgh, Pittsburgh, PA, 15260, USA
| | - Ervin Sejdić
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA, 15261, USA.
| |
Collapse
|
5
|
|
6
|
Tourbier S, Velasco-Annis C, Taimouri V, Hagmann P, Meuli R, Warfield SK, Bach Cuadra M, Gholipour A. Automated template-based brain localization and extraction for fetal brain MRI reconstruction. Neuroimage 2017; 155:460-472. [PMID: 28408290 DOI: 10.1016/j.neuroimage.2017.04.004] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2016] [Revised: 03/30/2017] [Accepted: 04/01/2017] [Indexed: 12/22/2022] Open
Abstract
Most fetal brain MRI reconstruction algorithms rely only on brain tissue-relevant voxels of low-resolution (LR) images to enhance the quality of inter-slice motion correction and image reconstruction. Consequently the fetal brain needs to be localized and extracted as a first step, which is usually a laborious and time consuming manual or semi-automatic task. We have proposed in this work to use age-matched template images as prior knowledge to automatize brain localization and extraction. This has been achieved through a novel automatic brain localization and extraction method based on robust template-to-slice block matching and deformable slice-to-template registration. Our template-based approach has also enabled the reconstruction of fetal brain images in standard radiological anatomical planes in a common coordinate space. We have integrated this approach into our new reconstruction pipeline that involves intensity normalization, inter-slice motion correction, and super-resolution (SR) reconstruction. To this end we have adopted a novel approach based on projection of every slice of the LR brain masks into the template space using a fusion strategy. This has enabled the refinement of brain masks in the LR images at each motion correction iteration. The overall brain localization and extraction algorithm has shown to produce brain masks that are very close to manually drawn brain masks, showing an average Dice overlap measure of 94.5%. We have also demonstrated that adopting a slice-to-template registration and propagation of the brain mask slice-by-slice leads to a significant improvement in brain extraction performance compared to global rigid brain extraction and consequently in the quality of the final reconstructed images. Ratings performed by two expert observers show that the proposed pipeline can achieve similar reconstruction quality to reference reconstruction based on manual slice-by-slice brain extraction. The proposed brain mask refinement and reconstruction method has shown to provide promising results in automatic fetal brain MRI segmentation and volumetry in 26 fetuses with gestational age range of 23 to 38 weeks.
Collapse
Affiliation(s)
- Sébastien Tourbier
- Computational Radiology Laboratory (CRL), Department of Radiology, Boston Children's Hospital, and Harvard Medical School, USA; Medical Image Analysis Laboratory (MIAL), Centre d'Imagerie BioMédicale (CIBM), Switzerland; Radiology Department, Lausanne University Hospital Center (CHUV) and University of Lausanne (UNIL), Switzerland
| | - Clemente Velasco-Annis
- Computational Radiology Laboratory (CRL), Department of Radiology, Boston Children's Hospital, and Harvard Medical School, USA
| | - Vahid Taimouri
- Computational Radiology Laboratory (CRL), Department of Radiology, Boston Children's Hospital, and Harvard Medical School, USA
| | - Patric Hagmann
- Radiology Department, Lausanne University Hospital Center (CHUV) and University of Lausanne (UNIL), Switzerland
| | - Reto Meuli
- Radiology Department, Lausanne University Hospital Center (CHUV) and University of Lausanne (UNIL), Switzerland
| | - Simon K Warfield
- Computational Radiology Laboratory (CRL), Department of Radiology, Boston Children's Hospital, and Harvard Medical School, USA
| | - Meritxell Bach Cuadra
- Medical Image Analysis Laboratory (MIAL), Centre d'Imagerie BioMédicale (CIBM), Switzerland; Radiology Department, Lausanne University Hospital Center (CHUV) and University of Lausanne (UNIL), Switzerland; Signal Processing Laboratory (LTS5), Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland.
| | - Ali Gholipour
- Computational Radiology Laboratory (CRL), Department of Radiology, Boston Children's Hospital, and Harvard Medical School, USA
| |
Collapse
|
7
|
Histograms of Oriented 3D Gradients for Fully Automated Fetal Brain Localization and Robust Motion Correction in 3 T Magnetic Resonance Images. BIOMED RESEARCH INTERNATIONAL 2017; 2017:3956363. [PMID: 28251155 PMCID: PMC5304316 DOI: 10.1155/2017/3956363] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/22/2016] [Revised: 12/13/2016] [Accepted: 12/26/2016] [Indexed: 01/05/2023]
Abstract
Fetal brain magnetic resonance imaging (MRI) is a rapidly emerging diagnostic imaging tool. However, automated fetal brain localization is one of the biggest obstacles in expediting and fully automating large-scale fetal MRI processing. We propose a method for automatic localization of fetal brain in 3 T MRI when the images are acquired as a stack of 2D slices that are misaligned due to fetal motion. First, the Histogram of Oriented Gradients (HOG) feature descriptor is extended from 2D to 3D images. Then, a sliding window is used to assign a score to all possible windows in an image, depending on the likelihood of it containing a brain, and the window with the highest score is selected. In our evaluation experiments using a leave-one-out cross-validation strategy, we achieved 96% of complete brain localization using a database of 104 MRI scans at gestational ages between 34 and 38 weeks. We carried out comparisons against template matching and random forest based regression methods and the proposed method showed superior performance. We also showed the application of the proposed method in the optimization of fetal motion correction and how it is essential for the reconstruction process. The method is robust and does not rely on any prior knowledge of fetal brain development.
Collapse
|
8
|
Levman J, Takahashi E. Multivariate Analyses Applied to Healthy Neurodevelopment in Fetal, Neonatal, and Pediatric MRI. Front Neuroanat 2016; 9:163. [PMID: 26834576 PMCID: PMC4720794 DOI: 10.3389/fnana.2015.00163] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2015] [Accepted: 12/04/2015] [Indexed: 11/13/2022] Open
Abstract
Multivariate analysis (MVA) is a class of statistical and pattern recognition techniques that involve the processing of data that contains multiple measurements per sample. MVA can be used to address a wide variety of neurological medical imaging related challenges including the evaluation of healthy brain development, the automated analysis of brain tissues and structures through image segmentation, evaluating the effects of genetic and environmental factors on brain development, evaluating sensory stimulation's relationship with functional brain activity and much more. Compared to adult imaging, pediatric, neonatal and fetal imaging have attracted less attention from MVA researchers, however, recent years have seen remarkable MVA research growth in pre-adult populations. This paper presents the results of a systematic review of the literature focusing on MVA applied to healthy subjects in fetal, neonatal and pediatric magnetic resonance imaging (MRI) of the brain. While the results of this review demonstrate considerable interest from the scientific community in applications of MVA technologies in brain MRI, the field is still young and significant research growth will continue into the future.
Collapse
Affiliation(s)
- Jacob Levman
- Division of Newborn Medicine, Department of Medicine, Boston Children's Hospital, Harvard Medical SchoolBoston, MA, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General HospitalCharlestown, MA, USA
| | - Emi Takahashi
- Division of Newborn Medicine, Department of Medicine, Boston Children's Hospital, Harvard Medical SchoolBoston, MA, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General HospitalCharlestown, MA, USA
| |
Collapse
|
9
|
|
10
|
Automated Localization of Fetal Organs in MRI Using Random Forests with Steerable Features. ACTA ACUST UNITED AC 2015. [DOI: 10.1007/978-3-319-24574-4_74] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/21/2023]
|
11
|
Motion corrected 3D reconstruction of the fetal thorax from prenatal MRI. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2014; 17:284-91. [PMID: 25485390 DOI: 10.1007/978-3-319-10470-6_36] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/08/2022]
Abstract
In this paper we present a semi-automatic method for analysis of the fetal thorax in genuine three-dimensional volumes. After one initial click we localize the spine and accurately determine the volume of the fetal lung from high resolution volumetric images reconstructed from motion corrupted prenatal Magnetic Resonance Imaging (MRI). We compare the current state-of-the-art method of segmenting the lung in a slice-by-slice manner with the most recent multi-scan reconstruction methods. We use fast rotation invariant spherical harmonics image descriptors with Classification Forest ensemble learning methods to extract the spinal cord and show an efficient way to generate a segmentation prior for the fetal lung from this information for two different MRI field strengths. The spinal cord can be segmented with a DICE coefficient of 0.89 and the automatic lung segmentation has been evaluated with a DICE coefficient of 0.87. We evaluate our method on 29 fetuses with a gestational age (GA) between 20 and 38 weeks and show that our computed segmentations and the manual ground truth correlate well with the recorded values in literature.
Collapse
|