1
|
Zhang L, Ning G, Liang H, Han B, Liao H. One-shot neuroanatomy segmentation through online data augmentation and confidence aware pseudo label. Med Image Anal 2024; 95:103182. [PMID: 38688039 DOI: 10.1016/j.media.2024.103182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 11/26/2023] [Accepted: 04/18/2024] [Indexed: 05/02/2024]
Abstract
Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg.
Collapse
Affiliation(s)
- Liutong Zhang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Guochen Ning
- School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Hanying Liang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Boxuan Han
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China; School of Biomedical Engineering, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
2
|
Li X, Bellotti R, Meier G, Bachtiary B, Weber D, Lomax A, Buhmann J, Zhang Y. Uncertainty-aware MR-based CT synthesis for robust proton therapy planning of brain tumour. Radiother Oncol 2024; 191:110056. [PMID: 38104781 DOI: 10.1016/j.radonc.2023.110056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 12/06/2023] [Accepted: 12/08/2023] [Indexed: 12/19/2023]
Abstract
BACKGROUND AND PURPOSE Deep learning techniques excel in MR-based CT synthesis, but missing uncertainty prediction limits its clinical use in proton therapy. We developed an uncertainty-aware framework and evaluated its efficiency in robust proton planning. MATERIALS AND METHODS A conditional generative-adversarial network was trained on 64 brain tumour patients with paired MR-CT images to generate synthetic CTs (sCT) from combined T1-T2 MRs of three orthogonal planes. A Bayesian neural network predicts Laplacian distributions for all voxels with parameters (μ, b). A robust proton plan was optimized using three sCTs of μ and μ±b. The dosimetric differences between the plan from sCT (sPlan) and the recalculated plan (rPlan) on planning CT (pCT) were quantified for each patient. The uncertainty-aware robust plan was compared to conventional robust (global ± 3 %) and non-robust plans. RESULTS In 8-fold cross-validation, sCT-pCT image differences (Mean-Absolute-Error) were 80.84 ± 9.84HU (body), 35.78 ± 6.07HU (soft tissues) and 221.88 ± 31.69HU (bones), with Dice scores of 90.33 ± 2.43 %, 95.13 ± 0.80 %, and 85.53 ± 4.16 %, respectively. The uncertainty distribution positively correlated with absolute prediction error (Correlation Coefficient: 0.62 ± 0.01). The uncertainty-conditioned robust optimisation improved the rPlan-sPlan agreement, e.g., D95 absolute difference (CTV) was 1.10 ± 1.24 % compared to conventional (1.64 ± 2.71 %) and non-robust (2.08 ± 2.96 %) optimisation. This trend was consistent across all target and organs-at-risk indexes. CONCLUSION The enhanced framework incorporates 3D uncertainty prediction and generates high-quality sCTs from MR images. The framework also facilitates conditioned robust optimisation, bolstering proton plan robustness against network prediction errors. The innovative feature of uncertainty visualisation and robust analyses contribute to evaluating sCT clinical utility for individual patients.
Collapse
Affiliation(s)
- Xia Li
- Center for Proton Therapy, Paul Scherrer Institut, Switzerland; Department of Computer Science, ETH Zurich, Switzerland
| | - Renato Bellotti
- Center for Proton Therapy, Paul Scherrer Institut, Switzerland; Department of Physics, ETH Zurich, Switzerland
| | - Gabriel Meier
- Center for Proton Therapy, Paul Scherrer Institut, Switzerland
| | | | - Damien Weber
- Center for Proton Therapy, Paul Scherrer Institut, Switzerland; Department of Radiation Oncology, University Hospital of Zurich, Switzerland; Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Switzerland
| | - Antony Lomax
- Center for Proton Therapy, Paul Scherrer Institut, Switzerland; Department of Physics, ETH Zurich, Switzerland
| | | | - Ye Zhang
- Center for Proton Therapy, Paul Scherrer Institut, Switzerland.
| |
Collapse
|
3
|
Achar S, Hwang D, Finkenstaedt T, Malis V, Bae WC. Deep-Learning-Aided Evaluation of Spondylolysis Imaged with Ultrashort Echo Time Magnetic Resonance Imaging. SENSORS (BASEL, SWITZERLAND) 2023; 23:8001. [PMID: 37766055 PMCID: PMC10538057 DOI: 10.3390/s23188001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 08/31/2023] [Accepted: 09/15/2023] [Indexed: 09/29/2023]
Abstract
Isthmic spondylolysis results in fracture of pars interarticularis of the lumbar spine, found in as many as half of adolescent athletes with persistent low back pain. While computed tomography (CT) is the gold standard for the diagnosis of spondylolysis, the use of ionizing radiation near reproductive organs in young subjects is undesirable. While magnetic resonance imaging (MRI) is preferable, it has lowered sensitivity for detecting the condition. Recently, it has been shown that ultrashort echo time (UTE) MRI can provide markedly improved bone contrast compared to conventional MRI. To take UTE MRI further, we developed supervised deep learning tools to generate (1) CT-like images and (2) saliency maps of fracture probability from UTE MRI, using ex vivo preparation of cadaveric spines. We further compared quantitative metrics of the contrast-to-noise ratio (CNR), mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) between UTE MRI (inverted to make the appearance similar to CT) and CT and between CT-like images and CT. Qualitative results demonstrated the feasibility of successfully generating CT-like images from UTE MRI to provide easier interpretability for bone fractures thanks to improved image contrast and CNR. Quantitatively, the mean CNR of bone against defect-filled tissue was 35, 97, and 146 for UTE MRI, CT-like, and CT images, respectively, being significantly higher for CT-like than UTE MRI images. For the image similarity metrics using the CT image as the reference, CT-like images provided a significantly lower mean MSE (0.038 vs. 0.0528), higher mean PSNR (28.6 vs. 16.5), and higher SSIM (0.73 vs. 0.68) compared to UTE MRI images. Additionally, the saliency maps enabled quick detection of the location with probable pars fracture by providing visual cues to the reader. This proof-of-concept study is limited to the data from ex vivo samples, and additional work in human subjects with spondylolysis would be necessary to refine the models for clinical use. Nonetheless, this study shows that the utilization of UTE MRI and deep learning tools could be highly useful for the evaluation of isthmic spondylolysis.
Collapse
Affiliation(s)
- Suraj Achar
- Department of Family Medicine, University of California-San Diego, La Jolla, CA 92093, USA
| | - Dosik Hwang
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
- Center for Healthcare Robotics, Korea Institute of Science and Technology, Seoul 02792, Republic of Korea
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul 03722, Republic of Korea
- Department of Oral and Maxillofacial Radiology, Yonsei University College of Dentistry, Seoul 03722, Republic of Korea
| | - Tim Finkenstaedt
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, University Zurich, 8091 Zurich, Switzerland
| | - Vadim Malis
- Department of Radiology, University of California-San Diego, La Jolla, CA 92093, USA
| | - Won C. Bae
- Department of Radiology, University of California-San Diego, La Jolla, CA 92093, USA
- Department of Radiology, VA San Diego Healthcare System, San Diego, CA 92161, USA
| |
Collapse
|
4
|
Walluscheck S, Canalini L, Strohm H, Diekmann S, Klein J, Heldmann S. MR-CT multi-atlas registration guided by fully automated brain structure segmentation with CNNs. Int J Comput Assist Radiol Surg 2023; 18:483-491. [PMID: 36334164 PMCID: PMC9939492 DOI: 10.1007/s11548-022-02786-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Accepted: 10/25/2022] [Indexed: 11/08/2022]
Abstract
PURPOSE Computed tomography (CT) is widely used to identify anomalies in brain tissues because their localization is important for diagnosis and therapy planning. Due to the insufficient soft tissue contrast of CT, the division of the brain into anatomical meaningful regions is challenging and is commonly done with magnetic resonance imaging (MRI). METHODS We propose a multi-atlas registration approach to propagate anatomical information from a standard MRI brain atlas to CT scans. This translation will enable a detailed automated reporting of brain CT exams. We utilize masks of the lateral ventricles and the brain volume of CT images as adjuvant input to guide the registration process. Besides using manual annotations to test the registration in a first step, we then verify that convolutional neural networks (CNNs) are a reliable solution for automatically segmenting structures to enhance the registration process. RESULTS The registration method obtains mean Dice values of 0.92 and 0.99 in brain ventricles and parenchyma on 22 healthy test cases when using manually segmented structures as guidance. When guiding with automatically segmented structures, the mean Dice values are 0.87 and 0.98, respectively. CONCLUSION Our registration approach is a fully automated solution to register MRI atlas images to CT scans and thus obtain detailed anatomical information. The proposed CNN segmentation method can be used to obtain masks of ventricles and brain volume which guide the registration.
Collapse
Affiliation(s)
- Sina Walluscheck
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
| | - Luca Canalini
- grid.428590.20000 0004 0496 8246Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Hannah Strohm
- grid.428590.20000 0004 0496 8246Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Susanne Diekmann
- grid.428590.20000 0004 0496 8246Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Jan Klein
- grid.428590.20000 0004 0496 8246Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Stefan Heldmann
- grid.428590.20000 0004 0496 8246Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| |
Collapse
|
5
|
Sreeja S, Muhammad Noorul Mubarak D. Pseudo computed tomography image generation from brain magnetic resonance image using integration of PCA & DCNN-UNET: A comparative analysis. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-213367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
MRI-Only Radiation (RT) now avoids some of the issues associated with employing Computed Tomography(CT) in RT chains, such as MRI registration to a separate CT, excess dosage administration, and the cost of recurrent imaging. The fact that MRI signal intensities are unrelated to the biological tissue’s attenuation coefficient poses a problem. This raises workloads, creates uncertainty as a result of the required inter-modality image registrations, and exposes patients to needless radiation. While using only MRI would be preferable, a method for estimating a pseudo-CT (pCT)or synthetic-CT(sCT) for producing electron density maps and patient positioning reference images is required. As Deep Learning(DL) is revolutionized in so many fields these days, an effective and accurate model is required for generating pCT from MRI. So, this paper depicts an efficient DL model in which the following are the stages: a) Data Acquisition where CT and MRI images are collected b) preprocessing these to avoid the anomalies and noises using techniques like outlier elimination, data smoothening and data normalizing c) feature extraction and selection using Principal Component Analysis (PCA) & regression method d) generating pCT from MRI using Deep Convolutional Neural Network and UNET (DCNN-UNET). We here compare both feature extraction (PCA) and classification model (DCNN-UNET) with other methods such as Discrete Wavelet Tranform(DWT), Independent Component Analysis(ICA), Fourier Transform and VGG16, ResNet, AlexNet, DenseNet, CNN (Convolutional Neural Network)respectively. The performance measures used to evaluate these models are Dice Coefficient(DC), Structured Similarity Index Measure(SSIM), Mean Absolute Error(MAE), Mean Squared Error(MSE), Accuracy, Computation Time in which our proposed system outperforms better with 0.94±0.02 over other state-of-art models.
Collapse
Affiliation(s)
- S Sreeja
- Department of Computer Science, University of Kerala, Karyavattom Campus, Trivandrum, Kerala, India
| | | |
Collapse
|
6
|
Burgos N, Cardoso MJ, Samper-González J, Habert MO, Durrleman S, Ourselin S, Colliot O. Anomaly detection for the individual analysis of brain PET images. J Med Imaging (Bellingham) 2021; 8:024003. [PMID: 33842668 PMCID: PMC8021015 DOI: 10.1117/1.jmi.8.2.024003] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 03/12/2021] [Indexed: 11/14/2022] Open
Abstract
Purpose: In clinical practice, positron emission tomography (PET) images are mostly analyzed visually, but the sensitivity and specificity of this approach greatly depend on the observer's experience. Quantitative analysis of PET images would alleviate this problem by helping define an objective limit between normal and pathological findings. We present an anomaly detection framework for the individual analysis of PET images. Approach: We created subject-specific abnormality maps that summarize the pathology's topographical distribution in the brain by comparing the subject's PET image to a model of healthy PET appearance that is specific to the subject under investigation. This model was generated from demographically and morphologically matched PET scans from a control dataset. Results: We generated abnormality maps for healthy controls, patients at different stages of Alzheimer's disease and with different frontotemporal dementia syndromes. We showed that no anomalies were detected for the healthy controls and that the anomalies detected from the patients with dementia coincided with the regions where abnormal uptake was expected. We also validated the proposed framework using the abnormality maps as inputs of a classifier and obtained higher classification accuracies than when using the PET images themselves as inputs. Conclusions: The proposed method was able to automatically locate and characterize the areas characteristic of dementia from PET images. The abnormality maps are expected to (i) help clinicians in their diagnosis by highlighting, in a data-driven fashion, the pathological areas, and (ii) improve the interpretability of subsequent analyses, such as computer-aided diagnosis or spatiotemporal modeling.
Collapse
Affiliation(s)
- Ninon Burgos
- Paris Brain Institute, Hôpital Pitié-Salpêtrière, Paris, France
- INSERM, U 1127, Hôpital Pitié-Salpêtrière, Paris, France
- CNRS, UMR 7225, Hôpital Pitié-Salpêtrière, Paris, France
- Sorbonne Université, Hôpital Pitié-Salpêtrière, Paris, France
- Inria, Aramis Project-Team, Hôpital Pitié-Salpêtrière, Paris, France
| | - M. Jorge Cardoso
- King’s College London, Department of Imaging and Biomedical Engineering, London, United Kingdom
| | - Jorge Samper-González
- Paris Brain Institute, Hôpital Pitié-Salpêtrière, Paris, France
- INSERM, U 1127, Hôpital Pitié-Salpêtrière, Paris, France
- CNRS, UMR 7225, Hôpital Pitié-Salpêtrière, Paris, France
- Sorbonne Université, Hôpital Pitié-Salpêtrière, Paris, France
- Inria, Aramis Project-Team, Hôpital Pitié-Salpêtrière, Paris, France
| | - Marie-Odile Habert
- AP-HP, Hôpital Pitié-Salpêtrière, Department of Nuclear Medicine, Paris, France
- Laboratoire d’Imagerie Biomédicale, Sorbonne Université, Inserm U 1146, CNRS UMR 7371, Hôpital Pitié-Salpêtrière, Paris, France
- Centre Acquisition et Traitement des Images, Hôpital Pitié-Salpêtrière, Paris, France
| | - Stanley Durrleman
- Paris Brain Institute, Hôpital Pitié-Salpêtrière, Paris, France
- INSERM, U 1127, Hôpital Pitié-Salpêtrière, Paris, France
- CNRS, UMR 7225, Hôpital Pitié-Salpêtrière, Paris, France
- Sorbonne Université, Hôpital Pitié-Salpêtrière, Paris, France
- Inria, Aramis Project-Team, Hôpital Pitié-Salpêtrière, Paris, France
| | - Sébastien Ourselin
- King’s College London, Department of Imaging and Biomedical Engineering, London, United Kingdom
| | - Olivier Colliot
- Paris Brain Institute, Hôpital Pitié-Salpêtrière, Paris, France
- INSERM, U 1127, Hôpital Pitié-Salpêtrière, Paris, France
- CNRS, UMR 7225, Hôpital Pitié-Salpêtrière, Paris, France
- Sorbonne Université, Hôpital Pitié-Salpêtrière, Paris, France
- Inria, Aramis Project-Team, Hôpital Pitié-Salpêtrière, Paris, France
| | | | | |
Collapse
|
7
|
Generation of Pseudo-CT using High-Degree Polynomial Regression on Dual-Contrast Pelvic MRI Data. Sci Rep 2020; 10:8118. [PMID: 32415138 PMCID: PMC7229007 DOI: 10.1038/s41598-020-64842-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Accepted: 04/20/2020] [Indexed: 01/06/2023] Open
Abstract
Increasing interests in using magnetic resonance imaging only in radiation therapy require methods for predicting the computed tomography numbers from MRI data. Here we propose a simple voxel method to generate the pseudo-CT (pCT) image using dual-contrast pelvic MRI data. The method is first trained with the CT data and dual-contrast MRI data (two sets of MRI with different sequences) of multiple patients, where the anatomical structures in the images after deformable image registration are segmented into several regions, and after MRI intensity normalizations a regression analysis is used to determine a two-variable polynomial function for each region to relate a voxel’s two MRI intensity values to its CT number. We first evaluate the accuracy via the Hounsfield unit (HU) difference between the pseudo-CT and reference-CT (rCT) images and obtain the average mean absolute error as 40.3 ± 2.9 HU from leave-one-out-cross-validation (LOOCV) across all six patients, which is better than most previous results and comparable to another study using the more complicated atlas-based method. We also perform a dosimetric evaluation of the treatment plans based on pCT and rCT images and find the average passing rate within 2% dose difference to be 95.4% in point-to-point dose comparisons. Therefore, our method shows encouraging results in predicting the CT numbers. This polynomial method needs less computer storage than the interpolation method and can be readily extended to the case of more than two MRI sequences.
Collapse
|
8
|
Tie X, Lam SK, Zhang Y, Lee KH, Au KH, Cai J. Pseudo-CT generation from multi-parametric MRI using a novel multi-channel multi-path conditional generative adversarial network for nasopharyngeal carcinoma patients. Med Phys 2020; 47:1750-1762. [PMID: 32012292 DOI: 10.1002/mp.14062] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2020] [Revised: 01/08/2020] [Accepted: 01/27/2020] [Indexed: 12/17/2022] Open
Abstract
PURPOSE To develop and evaluate a novel method for pseudo-CT generation from multi-parametric MR images using multi-channel multi-path generative adversarial network (MCMP-GAN). METHODS Pre- and post-contrast T1-weighted (T1-w), T2-weighted (T2-w) MRI, and treatment planning CT images of 32 nasopharyngeal carcinoma (NPC) patients were employed to train a pixel-to-pixel MCMP-GAN. The network was developed based on a 5-level Residual U-Net (ResU-Net) with the channel-based independent feature extraction network to generate pseudo-CT images from multi-parametric MR images. The discriminator with five convolutional layers was added to distinguish between the real CT and pseudo-CT images, improving the nonlinearity and prediction accuracy of the model. Eightfold cross validation was implemented to validate the proposed MCMP-GAN. The pseudo-CT images were evaluated against the corresponding planning CT images based on mean absolute error (MAE), peak signal-to-noise ratio (PSNR), Dice similarity coefficient (DSC), and Structural similarity index (SSIM). Similar comparisons were also performed against the multi-channel single-path GAN (MCSP-GAN), the single-channel single-path GAN (SCSP-GAN). RESULTS It took approximately 20 h to train the MCMP-GAN model on a Quadro P6000, and less than 10 s to generate all pseudo-CT images for the subjects in the test set. The average head MAE between pseudo-CT and planning CT was 75.7 ± 14.6 Hounsfield Units (HU) for MCMP-GAN, significantly (P-values < 0.05) lower than that for MCSP-GAN (79.2 ± 13.0 HU) and SCSP-GAN (85.8 ± 14.3 HU). For bone only, the MCMP-GAN yielded a smaller mean MAE (194.6 ± 38.9 HU) than MCSP-GAN (203.7 ± 33.1 HU), SCSP-GAN (227.0 ± 36.7 HU). The average PSNR of MCMP-GAN (29.1 ± 1.6) was found to be higher than that of MCSP-GAN (28.8 ± 1.2) and SCSP-GAN (28.2 ± 1.3). In terms of metrics for image similarity, MCMP-GAN achieved the highest SSIM (0.92 ± 0.02) but did not show significantly improved bone DSC results in comparison with MCSP-GAN. CONCLUSIONS We developed a novel multi-channel GAN approach for generating pseudo-CT from multi-parametric MR images. Our preliminary results in NPC patients showed that the MCMP-GAN method performed apparently superior to the U-Net-GAN and SCSP-GAN, and slightly better than MCSP-GAN.
Collapse
Affiliation(s)
- Xin Tie
- The Hong Kong Polytechnic University, Hong Kong SAR, China.,Nanjing University, Nanjing, China
| | - Sai-Kit Lam
- The Hong Kong Polytechnic University, Hong Kong SAR, China
| | | | - Kar-Ho Lee
- Queen Elizabeth Hospital, Hong Kong SAR, China
| | | | - Jing Cai
- The Hong Kong Polytechnic University, Hong Kong SAR, China
| |
Collapse
|
9
|
Liu Y, Lei Y, Wang Y, Shafai-Erfani G, Wang T, Tian S, Patel P, Jani AB, McDonald M, Curran WJ, Liu T, Zhou J, Yang X. Evaluation of a deep learning-based pelvic synthetic CT generation technique for MRI-based prostate proton treatment planning. Phys Med Biol 2019; 64:205022. [PMID: 31487698 DOI: 10.1088/1361-6560/ab41af] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
The purpose of this work is to validate the application of a deep learning-based method for pelvic synthetic CT (sCT) generation that can be used for prostate proton beam therapy treatment planning. We propose to integrate dense block minimization into 3D cycle-consistent generative adversarial networks (cycleGAN) framework to effectively learn the nonlinear mapping between MRI and CT pairs. A cohort of 17 patients with co-registered CT and MR pairs were used to test the deep learning-based sCT generation method by leave-one-out cross-validation. Image quality between the sCT and CT images, gamma analysis passing rate, dose-volume metrics, distal range displacement, and the individual pencil beam Bragg peak shift between sCT- and CT-based proton plans were evaluated. The average mean absolute error (MAE) was 51.32 ± 16.91 HU. The relative differences of the statistics of the PTV dose-volume histogram (DVH) metrics in between sCT and CT were generally less than 1%. Mean values of dose difference, absolute dose difference (in percent of the prescribed dose) were -0.07% ± 0.07% and 0.23% ± 0.08%. Mean gamma analysis pass rate of 1 mm/1%, 2 mm/2%, 3 mm/3% criteria with 10% dose threshold were 92.39% ± 5.97%, 97.95% ± 2.95% and 98.97% ± 1.62% respectively. The median, mean and standard deviation of absolute maximum range differences were 0.09 cm and 0.23 ± 0.25 cm. The median and mean Bragg peak shifts among the 17 patients were 0.09 cm and 0.18 ± 0.07 cm. The image similarity, dosimetric and distal range agreement between sCT and original CT suggests the feasibility of further development of an MRI-only workflow for prostate proton radiotherapy.
Collapse
Affiliation(s)
- Yingzi Liu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA 30322, United States of America
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
10
|
Zhong L, Chen Y, Zhang X, Liu S, Wu Y, Liu Y, Lin L, Feng Q, Chen W, Yang W. Flexible Prediction of CT Images From MRI Data Through Improved Neighborhood Anchored Regression for PET Attenuation Correction. IEEE J Biomed Health Inform 2019; 24:1114-1124. [PMID: 31295129 DOI: 10.1109/jbhi.2019.2927368] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Given the complicated relationship between the magnetic resonance imaging (MRI) signals and the attenuation values, the attenuation correction in hybrid positron emission tomography (PET)/MRI systems remains a challenging task. Currently, existing methods are either time-consuming or require sufficient samples to train the models. In this paper, an efficient approach for predicting pseudo computed tomography (CT) images from T1- and T2-weighted MRI data with limited data is proposed. The proposed approach uses improved neighborhood anchored regression (INAR) as a baseline method to pre-calculate projected matrices to flexibly predict the pseudo CT patches. Techniques, including the augmentation of the MR/CT dataset, learning of the nonlinear descriptors of MR images, hierarchical search for nearest neighbors, data-driven optimization, and multi-regressor ensemble, are adopted to improve the effectiveness of the proposed approach. In total, 22 healthy subjects were enrolled in the study. The pseudo CT images obtained using INAR with multi-regressor ensemble yielded mean absolute error (MAE) of 92.73 ± 14.86 HU, peak signal-to-noise ratio of 29.77 ± 1.63 dB, Pearson linear correlation coefficient of 0.82 ± 0.05, dice similarity coefficient of 0.81 ± 0.03, and the relative mean absolute error (rMAE) in PET attenuation correction of 1.30 ± 0.20% compared with true CT images. Moreover, our proposed INAR method, without any refinement strategies, can achieve considerable results with only seven subjects (MAE 106.89 ± 14.43 HU, rMAE 1.51 ± 0.21%). The experiments prove the superior performance of the proposed method over the six innovative methods. Moreover, the proposed method can rapidly generate the pseudo CT images that are suitable for PET attenuation correction.
Collapse
|
11
|
Dinkla AM, Florkow MC, Maspero M, Savenije MHF, Zijlstra F, Doornaert PAH, Stralen M, Philippens MEP, Berg CAT, Seevinck PR. Dosimetric evaluation of synthetic CT for head and neck radiotherapy generated by a patch‐based three‐dimensional convolutional neural network. Med Phys 2019; 46:4095-4104. [DOI: 10.1002/mp.13663] [Citation(s) in RCA: 45] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2019] [Revised: 05/15/2019] [Accepted: 06/10/2019] [Indexed: 12/21/2022] Open
Affiliation(s)
- Anna M. Dinkla
- Department of Radiotherapy, Division of Imaging & Oncology University Medical Center Utrecht Utrecht The Netherlands
- Computational Imaging Group for MR Diagnostics & Therapy, Center for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
| | - Mateusz C. Florkow
- Centre for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
| | - Matteo Maspero
- Department of Radiotherapy, Division of Imaging & Oncology University Medical Center Utrecht Utrecht The Netherlands
- Computational Imaging Group for MR Diagnostics & Therapy, Center for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
| | - Mark H. F. Savenije
- Department of Radiotherapy, Division of Imaging & Oncology University Medical Center Utrecht Utrecht The Netherlands
- Computational Imaging Group for MR Diagnostics & Therapy, Center for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
| | - Frank Zijlstra
- Centre for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
| | - Patricia A. H. Doornaert
- Department of Radiotherapy, Division of Imaging & Oncology University Medical Center Utrecht Utrecht The Netherlands
| | - Marijn Stralen
- Centre for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
- MRIguidance B.V Utrecht The Netherlands
| | - Marielle E. P. Philippens
- Department of Radiotherapy, Division of Imaging & Oncology University Medical Center Utrecht Utrecht The Netherlands
| | - Cornelis A. T. Berg
- Department of Radiotherapy, Division of Imaging & Oncology University Medical Center Utrecht Utrecht The Netherlands
- Computational Imaging Group for MR Diagnostics & Therapy, Center for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
| | - Peter R. Seevinck
- Centre for Image Sciences University Medical Center Utrecht Utrecht The Netherlands
- MRIguidance B.V Utrecht The Netherlands
| |
Collapse
|
12
|
Automatic Labeling of MR Brain Images Through the Hashing Retrieval Based Atlas Forest. J Med Syst 2019; 43:241. [PMID: 31227923 DOI: 10.1007/s10916-019-1385-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2019] [Accepted: 06/10/2019] [Indexed: 10/26/2022]
Abstract
The multi-atlas method is one of the efficient and common automatic labeling method, which uses the prior information provided by expert-labeled images to guide the labeling of the target. However, most multi-atlas-based methods depend on the registration that may not give the correct information during the label propagation. To address the issue, we designed a new automatic labeling method through the hashing retrieval based atlas forest. The proposed method propagates labels without registration to reduce the errors, and constructs a target-oriented learning model to integrate information among the atlases. This method innovates a coarse classification strategy to preprocess the dataset, which retains the integrity of dataset and reduces computing time. Furthermore, the method considers each voxel in the atlas as a sample and encodes these samples with hashing for the fast sample retrieval. In the stage of labeling, the method selects suitable samples through hashing learning and trains atlas forests by integrating the information from the dataset. Then, the trained model is used to predict the labels of the target. Experimental results on two datasets illustrated that the proposed method is promising in the automatic labeling of MR brain images.
Collapse
|
13
|
Bray TJ, Karsa A, Bainbridge A, Sakai N, Punwani S, Hall‐Craggs MA, Shmueli K. Association of bone mineral density and fat fraction with magnetic susceptibility in inflamed trabecular bone. Magn Reson Med 2019; 81:3094-3107. [PMID: 30615213 PMCID: PMC6492090 DOI: 10.1002/mrm.27634] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2018] [Revised: 11/19/2018] [Accepted: 11/20/2018] [Indexed: 12/17/2022]
Abstract
PURPOSE To evaluate the relationship between bone mineral density (BMD) and magnetic susceptibility, and between proton density fat fraction and susceptibility, in inflamed trabecular bone. METHODS Two different phantoms modeling the fat fraction (FF) and BMD values of healthy bone marrow and disease states were scanned using a multiecho gradient echo acquisition at 3T. After correction for fat-water chemical shift, susceptibility mapping was performed, and susceptibility measurements were compared with BMD and FF values using linear regression. Patients with spondyloarthritis were scanned using the same protocol, and susceptibility values were calculated in areas of inflamed bone (edema) and fat metaplasia, both before and after accounting for the contribution of fat to the total susceptibility. RESULTS Susceptibility values in the phantoms were accurately described by a 2D linear function, with a negative correlation between BMD and susceptibility and a positive correlation between FF and susceptibility (adjusted R2 = 0.77; P = 3·10-5 ). In patients, significant differences in susceptibility were observed between fat metaplasia and normal marrow, but these differences were eliminated by removing the fat contribution to the total susceptibility. CONCLUSIONS BMD and proton density fat fraction both influence the total susceptibility of bone marrow and failure to account for the fat contribution could lead to errors in BMD quantification. We propose a method for removing the fat contribution from the total susceptibility, based on the observed linear relationship between susceptibility and FF. In inflamed bone, the overall increase in susceptibility in areas of fat metaplasia is at least partly due to increased fat content.
Collapse
Affiliation(s)
- Timothy J.P. Bray
- Centre for Medical ImagingUniversity College LondonUnited Kingdom
- Arthritis Research UK Centre for Adolescent RheumatologyUniversity College LondonUnited Kingdom
| | - Anita Karsa
- Department of Medical Physics and Biomedical EngineeringUniversity College LondonUnited Kingdom
| | - Alan Bainbridge
- Department of Medical PhysicsUniversity College London HospitalsUnited Kingdom
| | - Naomi Sakai
- Centre for Medical ImagingUniversity College LondonUnited Kingdom
| | - Shonit Punwani
- Centre for Medical ImagingUniversity College LondonUnited Kingdom
| | | | - Karin Shmueli
- Department of Medical Physics and Biomedical EngineeringUniversity College LondonUnited Kingdom
| |
Collapse
|
14
|
Advanced Multimodal Methods for Cranial Pseudo-CT Generation Validated by IMRT and VMAT Radiation Therapy Plans. Int J Radiat Oncol Biol Phys 2018; 102:792-800. [DOI: 10.1016/j.ijrobp.2018.06.024] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2017] [Revised: 06/05/2018] [Accepted: 06/14/2018] [Indexed: 12/27/2022]
|
15
|
Arabi H, Dowling JA, Burgos N, Han X, Greer PB, Koutsouvelis N, Zaidi H. Comparative study of algorithms for synthetic CT generation from MRI: Consequences for MRI-guided radiation planning in the pelvic region. Med Phys 2018; 45:5218-5233. [PMID: 30216462 DOI: 10.1002/mp.13187] [Citation(s) in RCA: 73] [Impact Index Per Article: 12.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2018] [Revised: 07/29/2018] [Accepted: 09/06/2018] [Indexed: 11/10/2022] Open
Abstract
PURPOSE Magnetic resonance imaging (MRI)-guided radiation therapy (RT) treatment planning is limited by the fact that the electron density distribution required for dose calculation is not readily provided by MR imaging. We compare a selection of novel synthetic CT generation algorithms recently reported in the literature, including segmentation-based, atlas-based and machine learning techniques, using the same cohort of patients and quantitative evaluation metrics. METHODS Six MRI-guided synthetic CT generation algorithms were evaluated: one segmentation technique into a single tissue class (water-only), four atlas-based techniques, namely, median value of atlas images (ALMedian), atlas-based local weighted voting (ALWV), bone enhanced atlas-based local weighted voting (ALWV-Bone), iterative atlas-based local weighted voting (ALWV-Iter), and a machine learning technique using deep convolution neural network (DCNN). RESULTS Organ auto-contouring from MR images was evaluated for bladder, rectum, bones, and body boundary. Overall, DCNN exhibited higher segmentation accuracy resulting in Dice indices (DSC) of 0.93 ± 0.17, 0.90 ± 0.04, and 0.93 ± 0.02 for bladder, rectum, and bones, respectively. On the other hand, ALMedian showed the lowest accuracy with DSC of 0.82 ± 0.20, 0.81 ± 0.08, and 0.88 ± 0.04, respectively. DCNN reached the best performance in terms of accurate derivation of synthetic CT values within each organ, with a mean absolute error within the body contour of 32.7 ± 7.9 HU, followed by the advanced atlas-based methods (ALWV: 40.5 ± 8.2 HU, ALWV-Iter: 42.4 ± 8.1 HU, ALWV-Bone: 44.0 ± 8.9 HU). ALMedian led to the highest error (52.1 ± 11.1 HU). Considering the dosimetric evaluation results, ALWV-Iter, ALWV, DCNN and ALWV-Bone led to similar mean dose estimation within each organ at risk and target volume with less than 1% dose discrepancy. However, the two-dimensional gamma analysis demonstrated higher pass rates for ALWV-Bone, DCNN, ALMedian and ALWV-Iter at 1%/1 mm criterion with 94.99 ± 5.15%, 94.59 ± 5.65%, 93.68 ± 5.53% and 93.10 ± 5.99% success, respectively, while ALWV and water-only resulted in 86.91 ± 13.50% and 80.77 ± 12.10%, respectively. CONCLUSIONS Overall, machine learning and advanced atlas-based methods exhibited promising performance by achieving reliable organ segmentation and synthetic CT generation. DCNN appears to have slightly better performance by achieving accurate automated organ segmentation and relatively small dosimetric errors (followed closely by advanced atlas-based methods, which in some cases achieved similar performance). However, the DCNN approach showed higher vulnerability to anatomical variation, where a greater number of outliers was observed with this method. Considering the dosimetric results obtained from the evaluated methods, the challenge of electron density estimation from MR images can be resolved with a clinically tolerable error.
Collapse
Affiliation(s)
- Hossein Arabi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, Geneva, CH-1211, Switzerland
| | - Jason A Dowling
- CSIRO Australian e-Health Research Centre, Herston, QLD, Australia
| | - Ninon Burgos
- Inria Paris, Aramis Project-Team, Institut du Cerveau et de la Moelle épinière, ICM, Inserm U 1127, CNRS, UMR 7225, Sorbonne Université, Paris, F-75013, France
| | - Xiao Han
- Elekta Inc., Maryland Heights, MO, 63043, USA
| | - Peter B Greer
- Calvary Mater Newcastle Hospital, Waratah, NSW, Australia.,University of Newcastle, Callaghan, NSW, Australia
| | - Nikolaos Koutsouvelis
- Division of Radiation Oncology, Geneva University Hospital, Geneva, CH-1211, Switzerland
| | - Habib Zaidi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, Geneva, CH-1211, Switzerland.,Geneva University Neurocenter, University of Geneva, Geneva, 1205, Switzerland.,Department of Nuclear Medicine and Molecular Imaging, University of Groningen, Groningen, the Netherlands.,Department of Nuclear Medicine, University of Southern Denmark, Odense, DK-500, Denmark
| |
Collapse
|
16
|
Huo J, Wu J, Cao J, Wang G. Supervoxel based method for multi-atlas segmentation of brain MR images. Neuroimage 2018; 175:201-214. [PMID: 29625235 DOI: 10.1016/j.neuroimage.2018.04.001] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2017] [Revised: 03/30/2018] [Accepted: 04/01/2018] [Indexed: 01/01/2023] Open
Abstract
Multi-atlas segmentation has been widely applied to the analysis of brain MR images. However, the state-of-the-art techniques in multi-atlas segmentation, including both patch-based and learning-based methods, are strongly dependent on the pairwise registration or exhibit huge spatial inconsistency. The paper proposes a new segmentation framework based on supervoxels to solve the existing challenges of previous methods. The supervoxel is an aggregation of voxels with similar attributes, which can be used to replace the voxel grid. By formulating the segmentation as a tissue labeling problem associated with a maximum-a-posteriori inference in Markov random field, the problem is solved via a graphical model with supervoxels being considered as the nodes. In addition, a dense labeling scheme is developed to refine the supervoxel labeling results, and the spatial consistency is incorporated in the proposed method. The proposed approach is robust to the pairwise registration errors and of high computational efficiency. Extensive experimental evaluations on three publically available brain MR datasets demonstrate the effectiveness and superior performance of the proposed approach.
Collapse
Affiliation(s)
- Jie Huo
- Department of ECE, University of Windsor, Windsor N9B 3P4, Canada.
| | - Jonathan Wu
- Department of ECE, University of Windsor, Windsor N9B 3P4, Canada; Institute of Information and Control, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Jiuwen Cao
- Institute of Information and Control, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Guanghui Wang
- Department of EECS, University of Kansas, Lawrence, KS 66045, USA.
| |
Collapse
|
17
|
Pathmanathan AU, van As NJ, Kerkmeijer LGW, Christodouleas J, Lawton CAF, Vesprini D, van der Heide UA, Frank SJ, Nill S, Oelfke U, van Herk M, Li XA, Mittauer K, Ritter M, Choudhury A, Tree AC. Magnetic Resonance Imaging-Guided Adaptive Radiation Therapy: A "Game Changer" for Prostate Treatment? Int J Radiat Oncol Biol Phys 2018; 100:361-373. [PMID: 29353654 DOI: 10.1016/j.ijrobp.2017.10.020] [Citation(s) in RCA: 118] [Impact Index Per Article: 19.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2017] [Revised: 10/09/2017] [Accepted: 10/12/2017] [Indexed: 01/25/2023]
Abstract
Radiation therapy to the prostate involves increasingly sophisticated delivery techniques and changing fractionation schedules. With a low estimated α/β ratio, a larger dose per fraction would be beneficial, with moderate fractionation schedules rapidly becoming a standard of care. The integration of a magnetic resonance imaging (MRI) scanner and linear accelerator allows for accurate soft tissue tracking with the capacity to replan for the anatomy of the day. Extreme hypofractionation schedules become a possibility using the potentially automated steps of autosegmentation, MRI-only workflow, and real-time adaptive planning. The present report reviews the steps involved in hypofractionated adaptive MRI-guided prostate radiation therapy and addresses the challenges for implementation.
Collapse
Affiliation(s)
- Angela U Pathmanathan
- The Institute of Cancer Research, London, United Kingdom; The Royal Marsden National Health Service Foundation Trust, London, United Kingdom
| | - Nicholas J van As
- The Institute of Cancer Research, London, United Kingdom; The Royal Marsden National Health Service Foundation Trust, London, United Kingdom
| | | | | | | | - Danny Vesprini
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Uulke A van der Heide
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Steven J Frank
- The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Simeon Nill
- The Institute of Cancer Research, London, United Kingdom; The Royal Marsden National Health Service Foundation Trust, London, United Kingdom
| | - Uwe Oelfke
- The Institute of Cancer Research, London, United Kingdom; The Royal Marsden National Health Service Foundation Trust, London, United Kingdom
| | - Marcel van Herk
- Manchester Cancer Research Centre, University of Manchester, Manchester Academic Health Science Centre, The Christie National Health Service Foundation Trust, Manchester, United Kingdom; National Institute of Health Research, Manchester Biomedical Research Centre, Central Manchester University Hospitals National Health Service Foundation Trust, Manchester Academic Health Science Centre, Manchester, United Kingdom
| | - X Allen Li
- Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Kathryn Mittauer
- University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Mark Ritter
- University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Ananya Choudhury
- Manchester Cancer Research Centre, University of Manchester, Manchester Academic Health Science Centre, The Christie National Health Service Foundation Trust, Manchester, United Kingdom; National Institute of Health Research, Manchester Biomedical Research Centre, Central Manchester University Hospitals National Health Service Foundation Trust, Manchester Academic Health Science Centre, Manchester, United Kingdom.
| | - Alison C Tree
- The Institute of Cancer Research, London, United Kingdom; The Royal Marsden National Health Service Foundation Trust, London, United Kingdom
| |
Collapse
|
18
|
Uncertainty in Multitask Learning: Joint Representations for Probabilistic MR-only Radiotherapy Planning. MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION – MICCAI 2018 2018. [DOI: 10.1007/978-3-030-00937-3_1] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
19
|
Partridge M, Oelfke U. Recent progress in applications of computing to radiotherapy (ICCR 2016). Phys Med Biol 2017. [DOI: 10.1088/1361-6560/aa6b3c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|