1
|
Huang L, Ruan S, Xing Y, Feng M. A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods. Med Image Anal 2024; 97:103223. [PMID: 38861770 DOI: 10.1016/j.media.2024.103223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Revised: 03/16/2024] [Accepted: 05/27/2024] [Indexed: 06/13/2024]
Abstract
The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the results. In this review, we offer a comprehensive overview of the prevailing methods proposed to quantify the uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models.
Collapse
Affiliation(s)
- Ling Huang
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Su Ruan
- Quantif, LITIS, University of Rouen Normandy, France.
| | - Yucheng Xing
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Mengling Feng
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore; Institute of Data Science, National University of Singapore, Singapore
| |
Collapse
|
2
|
Zhang J, Fu T, Xiao D, Fan J, Song H, Ai D, Yang J. Bi-Fusion of Structure and Deformation at Multi-Scale for Joint Segmentation and Registration. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2024; 33:3676-3691. [PMID: 38837936 DOI: 10.1109/tip.2024.3407657] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2024]
Abstract
Medical image segmentation and registration are two fundamental and highly related tasks. However, current works focus on the mutual promotion between the two at the loss function level, ignoring the feature information generated by the encoder-decoder network during the task-specific feature mapping process and the potential inter-task feature relationship. This paper proposes a unified multi-task joint learning framework based on bi-fusion of structure and deformation at multi-scale, called BFM-Net, which simultaneously achieves the segmentation results and deformation field in a single-step estimation. BFM-Net consists of a segmentation subnetwork (SegNet), a registration subnetwork (RegNet), and the multi-task connection module (MTC). The MTC module is used to transfer the latent feature representation between segmentation and registration at multi-scale and link different tasks at the network architecture level, including the spatial attention fusion module (SAF), the multi-scale spatial attention fusion module (MSAF) and the velocity field fusion module (VFF). Extensive experiments on MR, CT and ultrasound images demonstrate the effectiveness of our approach. The MTC module can increase the Dice scores of segmentation and registration by 3.2%, 1.6%, 2.2%, and 6.2%, 4.5%, 3.0%, respectively. Compared with six state-of-the-art algorithms for segmentation and registration, BFM-Net can achieve superior performance in various modal images, fully demonstrating its effectiveness and generalization.
Collapse
|
3
|
Zigiotto L, Amorosino G, Saviola F, Jovicich J, Annicchiarico L, Rozzanigo U, Olivetti E, Avesani P, Sarubbo S. Spontaneous unilateral spatial neglect recovery after brain tumour resection: A multimodal diffusion and rs-fMRI case report. J Neuropsychol 2024; 18 Suppl 1:91-114. [PMID: 37431064 DOI: 10.1111/jnp.12339] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Accepted: 05/25/2023] [Indexed: 07/12/2023]
Abstract
Patients with unilateral spatial neglect (USN) are unable to explore or to report stimuli presented in the left personal and extra-personal space. USN is usually caused by lesion of the right parietal lobe: nowadays, it is also clear the key role of structural connections (the second and the third branch of the right Superior Longitudinal Fasciculus, respectively, SLF II and III) and functional networks (Dorsal and Ventral Attention Network, respectively, DAN and VAN) in USN. In this multimodal case report, we have merged those structural and functional information derived from a patient with a right parietal lobe tumour and USN before surgery. Functional, structural and neuropsychological data were also collected 6 months after surgery, when the USN was spontaneously recovered. Diffusion metrics and Functional Connectivity (FC) of the right SLF and DAN, before and after surgery, were compared with the same data of a patient with a tumour in a similar location, but without USN, and with a control sample. Results indicate an impairment in the right SLF III and a reduction of FC of the right DAN in patients with USN before surgery compared to controls; after surgery, when USN was recovered, patient's diffusion metrics and FC showed no differences compared to the controls. This single case and its multimodal approach reinforce the crucial role of the right SLF III and DAN in the development and recovery of egocentric and allocentric extra-personal USN, highlighting the need to preserve these structural and functional areas during brain surgery.
Collapse
Affiliation(s)
- Luca Zigiotto
- Department of Neurosurgery, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
- Structural and Functional Connectivity Lab Project, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
- Department of Psychology, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
| | - Gabriele Amorosino
- Neuroinformatics Laboratory (NILab), Bruno Kessler Foundation (FBK), Trento, Italy
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Francesca Saviola
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Jorge Jovicich
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Luciano Annicchiarico
- Department of Neurosurgery, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
- Structural and Functional Connectivity Lab Project, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
| | - Umberto Rozzanigo
- Department of Neuroradiology, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
| | - Emanuele Olivetti
- Neuroinformatics Laboratory (NILab), Bruno Kessler Foundation (FBK), Trento, Italy
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Paolo Avesani
- Neuroinformatics Laboratory (NILab), Bruno Kessler Foundation (FBK), Trento, Italy
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Silvio Sarubbo
- Department of Neurosurgery, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
- Structural and Functional Connectivity Lab Project, 'S. Chiara' Hospital, Azienda Provinciale per i Servizi Sanitari, Trento, Italy
| |
Collapse
|
4
|
Popat M, Patel S. Research perspective and review towards brain tumour segmentation and classification using different image modalities. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2022. [DOI: 10.1080/21681163.2022.2124546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Affiliation(s)
- Mayuri Popat
- U & P.U. Patel Department of Computer Engineering, Chandubhai S Patel Institute of Technology (CSPIT), Charotar University of Science and Technology (CHARUSAT), Gujarat, India
| | - Sanskruti Patel
- Smt. Chandaben Mohanbhai Patel Institute of Computer Applications (CMPICA), Charotar University of Science and Technology (CHARUSAT), Gujarat, India
| |
Collapse
|
5
|
Longitudinal diffusion MRI analysis using Segis-Net: A single-step deep-learning framework for simultaneous segmentation and registration. Neuroimage 2021; 235:118004. [PMID: 33794359 DOI: 10.1016/j.neuroimage.2021.118004] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2020] [Revised: 03/12/2021] [Accepted: 03/19/2021] [Indexed: 01/02/2023] Open
Abstract
This work presents a single-step deep-learning framework for longitudinal image analysis, coined Segis-Net. To optimally exploit information available in longitudinal data, this method concurrently learns a multi-class segmentation and nonlinear registration. Segmentation and registration are modeled using a convolutional neural network and optimized simultaneously for their mutual benefit. An objective function that optimizes spatial correspondence for the segmented structures across time-points is proposed. We applied Segis-Net to the analysis of white matter tracts from N=8045 longitudinal brain MRI datasets of 3249 elderly individuals. Segis-Net approach showed a significant increase in registration accuracy, spatio-temporal segmentation consistency, and reproducibility compared with two multistage pipelines. This also led to a significant reduction in the sample-size that would be required to achieve the same statistical power in analyzing tract-specific measures. Thus, we expect that Segis-Net can serve as a new reliable tool to support longitudinal imaging studies to investigate macro- and microstructural brain changes over time.
Collapse
|
6
|
Gao Y, Wu Y, Zhang N, Yuan H, Wang F, Xu H, Yu J, Ma J, Hou S, Cao X. IDH1 gene mutation activates Smad signaling molecules to regulate the expression levels of cell cycle and biological rhythm genes in human glioma U87‑MG cells. Mol Med Rep 2021; 23:354. [PMID: 33760141 PMCID: PMC7974315 DOI: 10.3892/mmr.2021.11993] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 02/09/2021] [Indexed: 02/07/2023] Open
Abstract
Isocitrate dehydrogenase1 (IDH1) mutation is the most important genetic change in glioma. The most common IDH1 mutation results in the amino acid substitution of arginine 132 (Arg/R132), which is located at the active site of the enzyme. IDH1 Arg132His (R132H) mutation can reduce the proliferative rate of glioma cells. Numerous diseases follow circadian rhythms, and there is growing evidence that circadian disruption may be a risk factor for cancer in humans. Dysregulation of the circadian clock serves an important role in the development of malignant tumors, including glioma. Brain-Muscle Arnt-Like protein 1 (BMAL1) and Circadian Locomotor Output Cycles Kaput (CLOCK) are the main biological rhythm genes. The present study aimed to further study whether there is an association between IDH1 R132H mutation and biological rhythm in glioma, and whether this affects the occurrence of glioma. The Cancer Genome Atlas (TCGA) database was used to detect the expression levels of the biological rhythm genes BMAL1 and CLOCK in various types of tumor. Additionally, U87-MG cells were infected with wild-type and mutant IDH1 lentiviruses. Colony formation experiments were used to detect cell proliferation in each group, cell cycle distribution was detected by flow cytometry and western blotting was used to detect the expression levels of wild-type and mutant IDH1, cyclins, biological rhythm genes and Smad signaling pathway-associated genes in U87-MG cells. TCGA database results suggested that BMAL1 and CLOCK were abnormally expressed in glioma. Cells were successfully infected with wild-type and mutant IDH1 lentiviruses. Colony formation assay revealed decreased cell proliferation in the IDH1 R132H mutant group. The cell cycle distribution detected by flow cytometry indicated that IDH1 gene mutation increased the G1 phase ratio and decreased the S phase ratio in U87-MG cells. The western blotting results demonstrated that IDH1 R132H mutation decreased the expression levels of the S phase-associated proteins Cyclin A and CDK2, and increased the expression levels of the G1 phase-associated proteins Cyclin D3 and CDK4, but did not significantly change the expression levels of the G2/M phase-associated protein Cyclin B1. The expression levels of the positive and negative rhythm regulation genes BMAL1, CLOCK, period (PER s (PER1, 2 and 3) and cryptochrom (CRY)s (CRY1 and 2) were significantly decreased, those of the Smad signaling pathway-associated genes Smad2, Smad3 and Smad2-3 were decreased, and those of phosphorylated (p)-Smad2, p-Smad3 and Smad4 were increased. Therefore, the present results suggested that the IDH1 R132H mutation may alter the cell cycle and biological rhythm genes in U87-MG cells through the TGF-β/Smad signaling pathway.
Collapse
Affiliation(s)
- Yongying Gao
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Yanwei Wu
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Ningmei Zhang
- Department of Pathology, Tumor Hospital, General Hospital of Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Hongmei Yuan
- Functional Department, Ningxia Hui Autonomous Region People's Hospital, Yinchuan, Ningxia 750021, P.R. China
| | - Fei Wang
- Department of Pathology, The First People's Hospital of Yinchuan, Yinchuan, Ningxia 750001, P.R. China
| | - Hui Xu
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Jiaxiang Yu
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Jie Ma
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Shaozhang Hou
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| | - Xiangmei Cao
- Department of Pathology, School of Basic Medicine, Ningxia Medical University, Yinchuan, Ningxia 750004, P.R. China
| |
Collapse
|
7
|
Gryska E, Schneiderman J, Björkman-Burtscher I, Heckemann RA. Automatic brain lesion segmentation on standard magnetic resonance images: a scoping review. BMJ Open 2021; 11:e042660. [PMID: 33514580 PMCID: PMC7849889 DOI: 10.1136/bmjopen-2020-042660] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 01/09/2021] [Accepted: 01/12/2021] [Indexed: 12/11/2022] Open
Abstract
OBJECTIVES Medical image analysis practices face challenges that can potentially be addressed with algorithm-based segmentation tools. In this study, we map the field of automatic MR brain lesion segmentation to understand the clinical applicability of prevalent methods and study designs, as well as challenges and limitations in the field. DESIGN Scoping review. SETTING Three databases (PubMed, IEEE Xplore and Scopus) were searched with tailored queries. Studies were included based on predefined criteria. Emerging themes during consecutive title, abstract, methods and whole-text screening were identified. The full-text analysis focused on materials, preprocessing, performance evaluation and comparison. RESULTS Out of 2990 unique articles identified through the search, 441 articles met the eligibility criteria, with an estimated growth rate of 10% per year. We present a general overview and trends in the field with regard to publication sources, segmentation principles used and types of lesions. Algorithms are predominantly evaluated by measuring the agreement of segmentation results with a trusted reference. Few articles describe measures of clinical validity. CONCLUSIONS The observed reporting practices leave room for improvement with a view to studying replication, method comparison and clinical applicability. To promote this improvement, we propose a list of recommendations for future studies in the field.
Collapse
Affiliation(s)
- Emilia Gryska
- Medical Radiation Sciences, Goteborgs universitet Institutionen for kliniska vetenskaper, Goteborg, Sweden
| | - Justin Schneiderman
- Sektionen för klinisk neurovetenskap, Goteborgs Universitet Institutionen for Neurovetenskap och fysiologi, Goteborg, Sweden
| | | | - Rolf A Heckemann
- Medical Radiation Sciences, Goteborgs universitet Institutionen for kliniska vetenskaper, Goteborg, Sweden
| |
Collapse
|
8
|
Mang A, Bakas S, Subramanian S, Davatzikos C, Biros G. Integrated Biophysical Modeling and Image Analysis: Application to Neuro-Oncology. Annu Rev Biomed Eng 2020; 22:309-341. [PMID: 32501772 PMCID: PMC7520881 DOI: 10.1146/annurev-bioeng-062117-121105] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Central nervous system (CNS) tumors come with vastly heterogeneous histologic, molecular, and radiographic landscapes, rendering their precise characterization challenging. The rapidly growing fields of biophysical modeling and radiomics have shown promise in better characterizing the molecular, spatial, and temporal heterogeneity of tumors. Integrative analysis of CNS tumors, including clinically acquired multi-parametric magnetic resonance imaging (mpMRI) and the inverse problem of calibrating biophysical models to mpMRI data, assists in identifying macroscopic quantifiable tumor patterns of invasion and proliferation, potentially leading to improved (a) detection/segmentation of tumor subregions and (b) computer-aided diagnostic/prognostic/predictive modeling. This article presents a summary of (a) biophysical growth modeling and simulation,(b) inverse problems for model calibration, (c) these models' integration with imaging workflows, and (d) their application to clinically relevant studies. We anticipate that such quantitative integrative analysis may even be beneficial in a future revision of the World Health Organization (WHO) classification for CNS tumors, ultimately improving patient survival prospects.
Collapse
Affiliation(s)
- Andreas Mang
- Department of Mathematics, University of Houston, Houston, Texas 77204, USA;
| | - Spyridon Bakas
- Department of Mathematics, University of Houston, Houston, Texas 77204, USA;
| | - Shashank Subramanian
- Oden Institute of Computational Engineering and Sciences, The University of Texas at Austin, Austin, Texas 78712, USA; ,
| | - Christos Davatzikos
- Center for Biomedical Image Computing and Analytics (CBICA); Department of Radiology; and Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA; ,
| | - George Biros
- Oden Institute of Computational Engineering and Sciences, The University of Texas at Austin, Austin, Texas 78712, USA; ,
| |
Collapse
|
9
|
Mlynarski P, Delingette H, Alghamdi H, Bondiau PY, Ayache N. Anatomically consistent CNN-based segmentation of organs-at-risk in cranial radiotherapy. J Med Imaging (Bellingham) 2020; 7:014502. [PMID: 32064300 PMCID: PMC7016364 DOI: 10.1117/1.jmi.7.1.014502] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Accepted: 01/17/2020] [Indexed: 11/14/2022] Open
Abstract
Planning of radiotherapy involves accurate segmentation of a large number of organs at risk (OAR), i.e., organs for which irradiation doses should be minimized to avoid important side effects of the therapy. We propose a deep learning method for segmentation of OAR inside the head, from magnetic resonance images (MRIs). Our system performs segmentation of eight structures: eye, lens, optic nerve, optic chiasm, pituitary gland, hippocampus, brainstem, and brain. We propose an efficient algorithm to train neural networks for an end-to-end segmentation of multiple and nonexclusive classes, addressing problems related to computational costs and missing ground truth segmentations for a subset of classes. We enforce anatomical consistency of the result in a postprocessing step. In particular, we introduce a graph-based algorithm for segmentation of the optic nerves, enforcing the connectivity between the eyes and the optic chiasm. We report cross-validated quantitative results on a database of 44 contrast-enhanced T1-weighted MRIs with provided segmentations of the considered OAR, which were originally used for radiotherapy planning. In addition, the segmentations produced by our model on an independent test set of 50 MRIs were evaluated by an experienced radiotherapist in order to qualitatively assess their accuracy. The mean distances between produced segmentations and the ground truth ranged from 0.1 to 0.7 mm across different organs. A vast majority (96%) of the produced segmentations were found acceptable for radiotherapy planning.
Collapse
Affiliation(s)
- Pawel Mlynarski
- Université Côte d’Azur, Inria, Epione Research Team, Nice, France
| | - Hervé Delingette
- Université Côte d’Azur, Inria, Epione Research Team, Nice, France
| | - Hamza Alghamdi
- Université Côte d’Azur, Centre Antoine Lacassagne, Nice, France
| | | | - Nicholas Ayache
- Université Côte d’Azur, Inria, Epione Research Team, Nice, France
| |
Collapse
|
10
|
Wang G, Li W, Aertsen M, Deprest J, Ourselin S, Vercauteren T. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing 2019; 335:34-45. [PMID: 31595105 PMCID: PMC6783308 DOI: 10.1016/j.neucom.2019.01.103] [Citation(s) in RCA: 151] [Impact Index Per Article: 30.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
Abstract
Despite the state-of-the-art performance for medical image segmentation, deep convolutional neural networks (CNNs) have rarely provided uncertainty estimations regarding their segmentation outputs, e.g., model (epistemic) and image-based (aleatoric) uncertainties. In this work, we analyze these different types of uncertainties for CNN-based 2D and 3D medical image segmentation tasks at both pixel level and structure level. We additionally propose a test-time augmentation-based aleatoric uncertainty to analyze the effect of different transformations of the input image on the segmentation output. Test-time augmentation has been previously used to improve segmentation accuracy, yet not been formulated in a consistent mathematical framework. Hence, we also propose a theoretical formulation of test-time augmentation, where a distribution of the prediction is estimated by Monte Carlo simulation with prior distributions of parameters in an image acquisition model that involves image transformations and noise. We compare and combine our proposed aleatoric uncertainty with model uncertainty. Experiments with segmentation of fetal brains and brain tumors from 2D and 3D Magnetic Resonance Images (MRI) showed that 1) the test-time augmentation-based aleatoric uncertainty provides a better uncertainty estimation than calculating the test-time dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions, and 2) our test-time augmentation outperforms a single-prediction baseline and dropout-based multiple predictions.
Collapse
Affiliation(s)
- Guotai Wang
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Wenqi Li
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
| | - Michael Aertsen
- Department of Radiology, University Hospitals Leuven, Leuven, Belgium
| | - Jan Deprest
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Department of Radiology, University Hospitals Leuven, Leuven, Belgium
- Institute for Women’s Health, University College London, London, UK
- Department of Obstetrics and Gynaecology, University Hospitals Leuven, Leuven, Belgium
| | - Sébastien Ourselin
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
| | - Tom Vercauteren
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- Department of Obstetrics and Gynaecology, University Hospitals Leuven, Leuven, Belgium
| |
Collapse
|
11
|
Scheufele K, Mang A, Gholami A, Davatzikos C, Biros G, Mehl M. Coupling brain-tumor biophysical models and diffeomorphic image registration. COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING 2019; 347:533-567. [PMID: 31857736 PMCID: PMC6922029 DOI: 10.1016/j.cma.2018.12.008] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
We present SIBIA (Scalable Integrated Biophysics-based Image Analysis), a framework for joint image registration and biophysical inversion and we apply it to analyze MR images of glioblastomas (primary brain tumors). We have two applications in mind. The first one is normal-to-abnormal image registration in the presence of tumor-induced topology differences. The second one is biophysical inversion based on single-time patient data. The underlying optimization problem is highly non-linear and non-convex and has not been solved before with a gradient-based approach. Given the segmentation of a normal brain MRI and the segmentation of a cancer patient MRI, we determine tumor growth parameters and a registration map so that if we "grow a tumor" (using our tumor model) in the normal brain and then register it to the patient image, then the registration mismatch is as small as possible. This "coupled problem" two-way couples the biophysical inversion and the registration problem. In the image registration step we solve a large-deformation diffeomorphic registration problem parameterized by an Eulerian velocity field. In the biophysical inversion step we estimate parameters in a reaction-diffusion tumor growth model that is formulated as a partial differential equation (PDE). In SIBIA, we couple these two sub-components in an iterative manner. We first presented the components of SIBIA in "Gholami et al., Framework for Scalable Biophysics-based Image Analysis, IEEE/ACM Proceedings of the SC2017", in which we derived parallel distributed memory algorithms and software modules for the decoupled registration and biophysical inverse problems. In this paper, our contributions are the introduction of a PDE-constrained optimization formulation of the coupled problem, and the derivation of a Picard iterative solution scheme. We perform extensive tests to experimentally assess the performance of our method on synthetic and clinical datasets. We demonstrate the convergence of the SIBIA optimization solver in different usage scenarios. We demonstrate that using SIBIA, we can accurately solve the coupled problem in three dimensions (2563 resolution) in a few minutes using 11 dual-x86 nodes.
Collapse
Affiliation(s)
- Klaudius Scheufele
- University of Stuttgart, IPVS, Universitätstraße 38, 70569 Stuttgart, Germany
| | - Andreas Mang
- University of Houston, Department of Mathematics, 3551 Cullen Blvd., Houston, TX 77204-3008, USA
| | - Amir Gholami
- University of California Berkeley, EECS, Berkeley, CA 94720-1776, USA
| | - Christos Davatzikos
- Department of Radiology, University of Pennsylvania School of Medicine, 3700 Hamilton Walk, Philadelphia, PA 19104, USA
| | - George Biros
- University of Texas, ICES, 201 East 24th St, Austin, TX 78712-1229, USA
| | - Miriam Mehl
- University of Stuttgart, IPVS, Universitätstraße 38, 70569 Stuttgart, Germany
| |
Collapse
|
12
|
Narayanan A, Rajasekaran MP, Zhang Y, Govindaraj V, Thiyagarajan A. Multi-channeled MR brain image segmentation: A novel double optimization approach combined with clustering technique for tumor identification and tissue segmentation. Biocybern Biomed Eng 2019. [DOI: 10.1016/j.bbe.2018.12.003] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
13
|
A novel fully automatic multilevel thresholding technique based on optimized intuitionistic fuzzy sets and tsallis entropy for MR brain tumor image segmentation. AUSTRALASIAN PHYSICAL & ENGINEERING SCIENCES IN MEDICINE 2017; 41:41-58. [PMID: 29238919 DOI: 10.1007/s13246-017-0609-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/14/2016] [Accepted: 12/06/2017] [Indexed: 10/18/2022]
Abstract
In the present paper, a hybrid multilevel thresholding technique that combines intuitionistic fuzzy sets and tsallis entropy has been proposed for the automatic delineation of the tumor from magnetic resonance images having vague boundaries and poor contrast. This novel technique takes into account both the image histogram and the uncertainty information for the computation of multiple thresholds. The benefit of the methodology is that it provides fast and improved segmentation for the complex tumorous images with imprecise gray levels. To further boost the computational speed, the mutation based particle swarm optimization is used that selects the most optimal threshold combination. The accuracy of the proposed segmentation approach has been validated on simulated, real low-grade glioma tumor volumes taken from MICCAI brain tumor segmentation (BRATS) challenge 2012 dataset and the clinical tumor images, so as to corroborate its generality and novelty. The designed technique achieves an average Dice overlap equal to 0.82010, 0.78610 and 0.94170 for three datasets. Further, a comparative analysis has also been made between the eight existing multilevel thresholding implementations so as to show the superiority of the designed technique. In comparison, the results indicate a mean improvement in Dice by an amount equal to 4.00% (p < 0.005), 9.60% (p < 0.005) and 3.58% (p < 0.005), respectively in contrast to the fuzzy tsallis approach.
Collapse
|
14
|
Vishnuvarthanan A, Rajasekaran MP, Govindaraj V, Zhang Y, Thiyagarajan A. An automated hybrid approach using clustering and nature inspired optimization technique for improved tumor and tissue segmentation in magnetic resonance brain images. Appl Soft Comput 2017. [DOI: 10.1016/j.asoc.2017.04.023] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
15
|
Le Folgoc L, Delingette H, Criminisi A, Ayache N. Sparse Bayesian registration of medical images for self-tuning of parameters and spatially adaptive parametrization of displacements. Med Image Anal 2017; 36:79-97. [DOI: 10.1016/j.media.2016.09.008] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2015] [Revised: 09/14/2016] [Accepted: 09/20/2016] [Indexed: 10/20/2022]
|
16
|
Kaur T, Saini BS, Gupta S. A joint intensity and edge magnitude-based multilevel thresholding algorithm for the automatic segmentation of pathological MR brain images. Neural Comput Appl 2016. [DOI: 10.1007/s00521-016-2751-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
17
|
Wang B, Prastawa M, Irimia A, Saha A, Liu W, Goh SM, Vespa PM, Van Horn JD, Gerig G. Modeling 4D Pathological Changes by Leveraging Normative Models. COMPUTER VISION AND IMAGE UNDERSTANDING : CVIU 2016; 151:3-13. [PMID: 27818606 PMCID: PMC5094466 DOI: 10.1016/j.cviu.2016.01.007] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
With the increasing use of efficient multimodal 3D imaging, clinicians are able to access longitudinal imaging to stage pathological diseases, to monitor the efficacy of therapeutic interventions, or to assess and quantify rehabilitation efforts. Analysis of such four-dimensional (4D) image data presenting pathologies, including disappearing and newly appearing lesions, represents a significant challenge due to the presence of complex spatio-temporal changes. Image analysis methods for such 4D image data have to include not only a concept for joint segmentation of 3D datasets to account for inherent correlations of subject-specific repeated scans but also a mechanism to account for large deformations and the destruction and formation of lesions (e.g., edema, bleeding) due to underlying physiological processes associated with damage, intervention, and recovery. In this paper, we propose a novel framework that provides a joint segmentation-registration framework to tackle the inherent problem of image registration in the presence of objects not present in all images of the time series. Our methodology models 4D changes in pathological anatomy across time and and also provides an explicit mapping of a healthy normative template to a subject's image data with pathologies. Since atlas-moderated segmentation methods cannot explain appearance and locality pathological structures that are not represented in the template atlas, the new framework provides different options for initialization via a supervised learning approach, iterative semisupervised active learning, and also transfer learning, which results in a fully automatic 4D segmentation method. We demonstrate the effectiveness of our novel approach with synthetic experiments and a 4D multimodal MRI dataset of severe traumatic brain injury (TBI), including validation via comparison to expert segmentations. However, the proposed methodology is generic in regard to different clinical applications requiring quantitative analysis of 4D imaging representing spatio-temporal changes of pathologies.
Collapse
Affiliation(s)
- Bo Wang
- Scientific Computing and Imaging Institute, University of Utah, 72 Central Campus Drive, Salt Lake City, UT 84112 USA
- School of Computing, University of Utah, 50 S., Central Campus Drive, Salt Lake City, UT 84112 USA
| | - Marcel Prastawa
- Icahn School of Medicine at Mount Sinai, 1468 Madison Avenue, New York, NY 10029 USA
| | - Andrei Irimia
- The Institute for Neuroimaging and Informatics, Keck School of Medicine, University of Southern California, 2001 North Soto Street, Los Angeles CA 90089 USA
| | - Avishek Saha
- Yahoo Labs, 701 1st Ave, Sunnyvale, CA 94089 USA
| | - Wei Liu
- Scientific Computing and Imaging Institute, University of Utah, 72 Central Campus Drive, Salt Lake City, UT 84112 USA
- School of Computing, University of Utah, 50 S., Central Campus Drive, Salt Lake City, UT 84112 USA
| | - S.Y. Matthew Goh
- The Institute for Neuroimaging and Informatics, Keck School of Medicine, University of Southern California, 2001 North Soto Street, Los Angeles CA 90089 USA
| | - Paul M. Vespa
- Brain Injury Research Center, Department of Neurosurgery, David Geffen School of Medicine, University of California, Los Angeles, CA 90095 USA
| | - John D. Van Horn
- The Institute for Neuroimaging and Informatics, Keck School of Medicine, University of Southern California, 2001 North Soto Street, Los Angeles CA 90089 USA
| | - Guido Gerig
- Tandon School of Engineering, Department of Computer Science and Engineering, NYU, USA
| |
Collapse
|
18
|
Paragios N, Ferrante E, Glocker B, Komodakis N, Parisot S, Zacharaki EI. (Hyper)-graphical models in biomedical image analysis. Med Image Anal 2016; 33:102-106. [DOI: 10.1016/j.media.2016.06.028] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2016] [Revised: 06/16/2016] [Accepted: 06/16/2016] [Indexed: 11/29/2022]
|
19
|
Parisot S, Darlix A, Baumann C, Zouaoui S, Yordanova Y, Blonski M, Rigau V, Chemouny S, Taillandier L, Bauchet L, Duffau H, Paragios N. A Probabilistic Atlas of Diffuse WHO Grade II Glioma Locations in the Brain. PLoS One 2016; 11:e0144200. [PMID: 26751577 PMCID: PMC4709135 DOI: 10.1371/journal.pone.0144200] [Citation(s) in RCA: 45] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2015] [Accepted: 11/13/2015] [Indexed: 11/18/2022] Open
Abstract
Diffuse WHO grade II gliomas are diffusively infiltrative brain tumors characterized by an unavoidable anaplastic transformation. Their management is strongly dependent on their location in the brain due to interactions with functional regions and potential differences in molecular biology. In this paper, we present the construction of a probabilistic atlas mapping the preferential locations of diffuse WHO grade II gliomas in the brain. This is carried out through a sparse graph whose nodes correspond to clusters of tumors clustered together based on their spatial proximity. The interest of such an atlas is illustrated via two applications. The first one correlates tumor location with the patient's age via a statistical analysis, highlighting the interest of the atlas for studying the origins and behavior of the tumors. The second exploits the fact that the tumors have preferential locations for automatic segmentation. Through a coupled decomposed Markov Random Field model, the atlas guides the segmentation process, and characterizes which preferential location the tumor belongs to and consequently which behavior it could be associated to. Leave-one-out cross validation experiments on a large database highlight the robustness of the graph, and yield promising segmentation results.
Collapse
Affiliation(s)
- Sarah Parisot
- Center for Visual Computing, Ecole Centrale Paris, Chatenay Malabry, France
- INRIA, Galen Team, Saclay-Ile-de-France Center, Chatenay Malabry, France
- Intrasense SAS, Montpellier, France
| | - Amélie Darlix
- INSERM U1051, Montpellier Neurosciences Institute, University Hospital, Montpellier, France
- Department of Medical Oncology, Institut Régional du Cancer de Montpellier - Val d’Aurelle, Montpellier, France
| | - Cédric Baumann
- Department of Clinical Epidemiology and Evaluation, University Hospital, Nancy, France
| | - Sonia Zouaoui
- Department of Epidemiology, Groupe de Neuro-Oncologie du Languedoc-Roussillon, Registre des Tumeurs de l’Hérault, Institut Régional du Cancer de Montpellier - Val d’Aurelle, Montpellier, France
| | | | - Marie Blonski
- Neuro-oncology Unit, Department of Neurology, University Hospital, Hôpital Central, Nancy, France
| | - Valérie Rigau
- Department of Pathology, University Hospital, Hôpital Gui de Chauliac, Montpellier, France
| | | | - Luc Taillandier
- Neuro-oncology Unit, Department of Neurology, University Hospital, Hôpital Central, Nancy, France
| | - Luc Bauchet
- Department of Medical Oncology, Institut Régional du Cancer de Montpellier - Val d’Aurelle, Montpellier, France
- Department of Neurosurgery, University Hospital, Hôpital Gui de Chauliac, Montpellier, France
| | - Hugues Duffau
- Department of Medical Oncology, Institut Régional du Cancer de Montpellier - Val d’Aurelle, Montpellier, France
- Department of Neurosurgery, University Hospital, Hôpital Gui de Chauliac, Montpellier, France
| | - Nikos Paragios
- Center for Visual Computing, Ecole Centrale Paris, Chatenay Malabry, France
- INRIA, Galen Team, Saclay-Ile-de-France Center, Chatenay Malabry, France
| |
Collapse
|
20
|
Goetz M, Weber C, Binczyk F, Polanska J, Tarnawski R, Bobek-Billewicz B, Koethe U, Kleesiek J, Stieltjes B, Maier-Hein KH. DALSA: Domain Adaptation for Supervised Learning From Sparsely Annotated MR Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2016; 35:184-196. [PMID: 26259241 DOI: 10.1109/tmi.2015.2463078] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
We propose a new method that employs transfer learning techniques to effectively correct sampling selection errors introduced by sparse annotations during supervised learning for automated tumor segmentation. The practicality of current learning-based automated tissue classification approaches is severely impeded by their dependency on manually segmented training databases that need to be recreated for each scenario of application, site, or acquisition setup. The comprehensive annotation of reference datasets can be highly labor-intensive, complex, and error-prone. The proposed method derives high-quality classifiers for the different tissue classes from sparse and unambiguous annotations and employs domain adaptation techniques for effectively correcting sampling selection errors introduced by the sparse sampling. The new approach is validated on labeled, multi-modal MR images of 19 patients with malignant gliomas and by comparative analysis on the BraTS 2013 challenge data sets. Compared to training on fully labeled data, we reduced the time for labeling and training by a factor greater than 70 and 180 respectively without sacrificing accuracy. This dramatically eases the establishment and constant extension of large annotated databases in various scenarios and imaging setups and thus represents an important step towards practical applicability of learning-based approaches in tissue classification.
Collapse
|
21
|
Heinrich MP, Simpson IJ, Papież BW, Brady SM, Schnabel JA. Deformable image registration by combining uncertainty estimates from supervoxel belief propagation. Med Image Anal 2016; 27:57-71. [DOI: 10.1016/j.media.2015.09.005] [Citation(s) in RCA: 53] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2014] [Revised: 09/20/2015] [Accepted: 09/22/2015] [Indexed: 11/26/2022]
|
22
|
Pagnozzi AM, Gal Y, Boyd RN, Fiori S, Fripp J, Rose S, Dowson N. The need for improved brain lesion segmentation techniques for children with cerebral palsy: A review. Int J Dev Neurosci 2015; 47:229-46. [DOI: 10.1016/j.ijdevneu.2015.08.004] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2015] [Revised: 08/24/2015] [Accepted: 08/24/2015] [Indexed: 01/18/2023] Open
Affiliation(s)
- Alex M. Pagnozzi
- CSIRO Digital Productivity and Services FlagshipThe Australian e‐Health Research CentreBrisbaneAustralia
- The University of QueenslandSchool of MedicineSt. LuciaBrisbaneAustralia
| | - Yaniv Gal
- The University of QueenslandCentre for Medical Diagnostic Technologies in QueenslandSt. LuciaBrisbaneAustralia
| | - Roslyn N. Boyd
- The University of QueenslandQueensland Cerebral Palsy and Rehabilitation Research CentreSchool of MedicineBrisbaneAustralia
| | - Simona Fiori
- Department of Developmental NeuroscienceStella Maris Scientific InstitutePisaItaly
| | - Jurgen Fripp
- CSIRO Digital Productivity and Services FlagshipThe Australian e‐Health Research CentreBrisbaneAustralia
| | - Stephen Rose
- CSIRO Digital Productivity and Services FlagshipThe Australian e‐Health Research CentreBrisbaneAustralia
| | - Nicholas Dowson
- CSIRO Digital Productivity and Services FlagshipThe Australian e‐Health Research CentreBrisbaneAustralia
| |
Collapse
|
23
|
Ou Y, Akbari H, Bilello M, Da X, Davatzikos C. Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights. IEEE TRANSACTIONS ON MEDICAL IMAGING 2014; 33:2039-65. [PMID: 24951685 PMCID: PMC4371548 DOI: 10.1109/tmi.2014.2330355] [Citation(s) in RCA: 98] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
Evaluating various algorithms for the inter-subject registration of brain magnetic resonance images (MRI) is a necessary topic receiving growing attention. Existing studies evaluated image registration algorithms in specific tasks or using specific databases (e.g., only for skull-stripped images, only for single-site images, etc.). Consequently, the choice of registration algorithms seems task- and usage/parameter-dependent. Nevertheless, recent large-scale, often multi-institutional imaging-related studies create the need and raise the question whether some registration algorithms can 1) generally apply to various tasks/databases posing various challenges; 2) perform consistently well, and while doing so, 3) require minimal or ideally no parameter tuning. In seeking answers to this question, we evaluated 12 general-purpose registration algorithms, for their generality, accuracy and robustness. We fixed their parameters at values suggested by algorithm developers as reported in the literature. We tested them in 7 databases/tasks, which present one or more of 4 commonly-encountered challenges: 1) inter-subject anatomical variability in skull-stripped images; 2) intensity homogeneity, noise and large structural differences in raw images; 3) imaging protocol and field-of-view (FOV) differences in multi-site data; and 4) missing correspondences in pathology-bearing images. Totally 7,562 registrations were performed. Registration accuracies were measured by (multi-)expert-annotated landmarks or regions of interest (ROIs). To ensure reproducibility, we used public software tools, public databases (whenever possible), and we fully disclose the parameter settings. We show evaluation results, and discuss the performances in light of algorithms' similarity metrics, transformation models and optimization strategies. We also discuss future directions for the algorithm development and evaluations.
Collapse
|
24
|
Kwon D, Shinohara RT, Akbari H, Davatzikos C. Combining generative models for multifocal glioma segmentation and registration. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2014; 17:763-70. [PMID: 25333188 DOI: 10.1007/978-3-319-10404-1_95] [Citation(s) in RCA: 53] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
In this paper, we propose a new method for simultaneously segmenting brain scans of glioma patients and registering these scans to a normal atlas. Performing joint segmentation and registration for brain tumors is very challenging when tumors include multifocal masses and have complex shapes with heterogeneous textures. Our approach grows tumors for each mass from multiple seed points using a tumor growth model and modifies a normal atlas into one with tumors and edema using the combined results of grown tumors. We also generate a tumor shape prior via the random walk with restart, utilizing multiple tumor seeds as initial foreground information. We then incorporate this shape prior into an EM framework which estimates the mapping between the modified atlas and the scans, posteriors for each tissue labels, and the tumor growth model parameters. We apply our method to the BRATS 2013 leaderboard dataset to evaluate segmentation performance. Our method shows the best performance among all participants.
Collapse
|