1
|
Bambach S, Ho ML. Deep Learning for Synthetic CT from Bone MRI in the Head and Neck. AJNR Am J Neuroradiol 2022; 43:1172-1179. [PMID: 36920777 PMCID: PMC9575432 DOI: 10.3174/ajnr.a7588] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Accepted: 06/13/2022] [Indexed: 11/07/2022]
Abstract
BACKGROUND AND PURPOSE Bone MR imaging techniques enable visualization of cortical bone without the need for ionizing radiation. Automated conversion of bone MR imaging to synthetic CT is highly desirable for downstream image processing and eventual clinical adoption. Given the complex anatomy and pathology of the head and neck, deep learning models are ideally suited for learning such mapping. MATERIALS AND METHODS This was a retrospective study of 39 pediatric and adult patients with bone MR imaging and CT examinations of the head and neck. For each patient, MR imaging and CT data sets were spatially coregistered using multiple-point affine transformation. Paired MR imaging and CT slices were generated for model training, using 4-fold cross-validation. We trained 3 different encoder-decoder models: Light_U-Net (2 million parameters) and VGG-16 U-Net (29 million parameters) without and with transfer learning. Loss functions included mean absolute error, mean squared error, and a weighted average. Performance metrics included Pearson R, mean absolute error, mean squared error, bone precision, and bone recall. We investigated model generalizability by training and validating across different conditions. RESULTS The Light_U-Net architecture quantitatively outperformed VGG-16 models. Mean absolute error loss resulted in higher bone precision, while mean squared error yielded higher bone recall. Performance metrics decreased when using training data captured only in a different environment but increased when local training data were augmented with those from different hospitals, vendors, or MR imaging techniques. CONCLUSIONS We have optimized a robust deep learning model for conversion of bone MR imaging to synthetic CT, which shows good performance and generalizability when trained on different hospitals, vendors, and MR imaging techniques. This approach shows promise for facilitating downstream image processing and adoption into clinical practice.
Collapse
Affiliation(s)
- S Bambach
- From the Abigail Wexner Research Institute at Nationwide Children's Hospital (S.B.), Columbus, Ohio
| | - M-L Ho
- Department of Radiology (M.-L.H.), Nationwide Children's Hospital, Columbus, Ohio.
| |
Collapse
|
2
|
Qian P, Zheng J, Zheng Q, Liu Y, Wang T, Al Helo R, Baydoun A, Avril N, Ellis RJ, Friel H, Traughber MS, Devaraj A, Traughber B, Muzic RF. Transforming UTE-mDixon MR Abdomen-Pelvis Images Into CT by Jointly Leveraging Prior Knowledge and Partial Supervision. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2021; 18:70-82. [PMID: 32175868 PMCID: PMC7932030 DOI: 10.1109/tcbb.2020.2979841] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
Computed tomography (CT) provides information for diagnosis, PET attenuation correction (AC), and radiation treatment planning (RTP). Disadvantages of CT include poor soft tissue contrast and exposure to ionizing radiation. While MRI can overcome these disadvantages, it lacks the photon absorption information needed for PET AC and RTP. Thus, an intelligent transformation from MR to CT, i.e., the MR-based synthetic CT generation, is of great interest as it would support PET/MR AC and MR-only RTP. Using an MR pulse sequence that combines ultra-short echo time (UTE) and modified Dixon (mDixon), we propose a novel method for synthetic CT generation jointly leveraging prior knowledge as well as partial supervision (SCT-PK-PS for short) on large-field-of-view images that span abdomen and pelvis. Two key machine learning techniques, i.e., the knowledge-leveraged transfer fuzzy c-means (KL-TFCM) and the Laplacian support vector machine (LapSVM), are used in SCT-PK-PS. The significance of our effort is threefold: 1) Using the prior knowledge-referenced KL-TFCM clustering, SCT-PK-PS is able to group the feature data of MR images into five initial clusters of fat, soft tissue, air, bone, and bone marrow. Via these initial partitions, clusters needing to be refined are observed and for each of them a few additionally labeled examples are given as the partial supervision for the subsequent semi-supervised classification using LapSVM; 2) Partial supervision is usually insufficient for conventional algorithms to learn the insightful classifier. Instead, exploiting not only the given supervision but also the manifold structure embedded primarily in numerous unlabeled data, LapSVM is capable of training multiple desired tissue-recognizers; 3) Benefiting from the joint use of KL-TFCM and LapSVM, and assisted by the edge detector filter based feature extraction, the proposed SCT-PK-PS method features good recognition accuracy of tissue types, which ultimately facilitates the good transformation from MR images to CT images of the abdomen-pelvis. Applying the method on twenty subjects' feature data of UTE-mDixon MR images, the average score of the mean absolute prediction deviation (MAPD) of all subjects is 140.72 ± 30.60 HU which is statistically significantly better than the 241.36 ± 21.79 HU obtained using the all-water method, the 262.77 ± 42.22 HU obtained using the four-cluster-partitioning (FCP, i.e., external-air, internal-air, fat, and soft tissue) method, and the 197.05 ± 76.53 HU obtained via the conventional SVM method. These results demonstrate the effectiveness of our method for the intelligent transformation from MR to CT on the body section of abdomen-pelvis.
Collapse
|
3
|
Kim S, Jang H, Jang J, Lee YH, Hwang D. Deep‐learned short tau inversion recovery imaging using multi‐contrast MR images. Magn Reson Med 2020; 84:2994-3008. [DOI: 10.1002/mrm.28327] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2020] [Revised: 04/27/2020] [Accepted: 04/27/2020] [Indexed: 01/22/2023]
Affiliation(s)
- Sewon Kim
- School of Electrical and Electronic Engineering Yonsei University Seoul Korea
| | - Hanbyol Jang
- School of Electrical and Electronic Engineering Yonsei University Seoul Korea
| | - Jinseong Jang
- School of Electrical and Electronic Engineering Yonsei University Seoul Korea
| | - Young Han Lee
- Department of Radiology and Center for Clinical Imaging Data Science (CCIDS) Yonsei University College of Medicine Seoul Korea
| | - Dosik Hwang
- School of Electrical and Electronic Engineering Yonsei University Seoul Korea
| |
Collapse
|
4
|
Generation of Pseudo-CT using High-Degree Polynomial Regression on Dual-Contrast Pelvic MRI Data. Sci Rep 2020; 10:8118. [PMID: 32415138 PMCID: PMC7229007 DOI: 10.1038/s41598-020-64842-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Accepted: 04/20/2020] [Indexed: 01/06/2023] Open
Abstract
Increasing interests in using magnetic resonance imaging only in radiation therapy require methods for predicting the computed tomography numbers from MRI data. Here we propose a simple voxel method to generate the pseudo-CT (pCT) image using dual-contrast pelvic MRI data. The method is first trained with the CT data and dual-contrast MRI data (two sets of MRI with different sequences) of multiple patients, where the anatomical structures in the images after deformable image registration are segmented into several regions, and after MRI intensity normalizations a regression analysis is used to determine a two-variable polynomial function for each region to relate a voxel’s two MRI intensity values to its CT number. We first evaluate the accuracy via the Hounsfield unit (HU) difference between the pseudo-CT and reference-CT (rCT) images and obtain the average mean absolute error as 40.3 ± 2.9 HU from leave-one-out-cross-validation (LOOCV) across all six patients, which is better than most previous results and comparable to another study using the more complicated atlas-based method. We also perform a dosimetric evaluation of the treatment plans based on pCT and rCT images and find the average passing rate within 2% dose difference to be 95.4% in point-to-point dose comparisons. Therefore, our method shows encouraging results in predicting the CT numbers. This polynomial method needs less computer storage than the interpolation method and can be readily extended to the case of more than two MRI sequences.
Collapse
|
5
|
Schilling KG, Blaber J, Huo Y, Newton A, Hansen C, Nath V, Shafer AT, Williams O, Resnick SM, Rogers B, Anderson AW, Landman BA. Synthesized b0 for diffusion distortion correction (Synb0-DisCo). Magn Reson Imaging 2019; 64:62-70. [PMID: 31075422 DOI: 10.1016/j.mri.2019.05.008] [Citation(s) in RCA: 86] [Impact Index Per Article: 17.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2018] [Revised: 04/02/2019] [Accepted: 05/04/2019] [Indexed: 02/07/2023]
Abstract
Diffusion magnetic resonance images typically suffer from spatial distortions due to susceptibility induced off-resonance fields, which may affect the geometric fidelity of the reconstructed volume and cause mismatches with anatomical images. State-of-the art susceptibility correction (for example, FSL's TOPUP algorithm) typically requires data acquired twice with reverse phase encoding directions, referred to as blip-up blip-down acquisitions, in order to estimate an undistorted volume. Unfortunately, not all imaging protocols include a blip-up blip-down acquisition, and cannot take advantage of the state-of-the art susceptibility and motion correction capabilities. In this study, we aim to enable TOPUP-like processing with historical and/or limited diffusion imaging data that include only a structural image and single blip diffusion image. We utilize deep learning to synthesize an undistorted non-diffusion weighted image from the structural image, and use the non-distorted synthetic image as an anatomical target for distortion correction. We evaluate the efficacy of this approach (named Synb0-DisCo) and show that our distortion correction process results in better matching of the geometry of undistorted anatomical images, reduces variation in diffusion modeling, and is practically equivalent to having both blip-up and blip-down non-diffusion weighted images.
Collapse
Affiliation(s)
- Kurt G Schilling
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, United States of America.
| | - Justin Blaber
- Electrical Engineering & Computer Science, Vanderbilt University, Nashville, TN, United States of America
| | - Yuankai Huo
- Department of Electrical Engineering, Vanderbilt University, Nashville, TN, United States of America
| | - Allen Newton
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, United States of America; Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, United States of America
| | - Colin Hansen
- Department of Electrical Engineering, Vanderbilt University, Nashville, TN, United States of America
| | - Vishwesh Nath
- Electrical Engineering & Computer Science, Vanderbilt University, Nashville, TN, United States of America
| | - Andrea T Shafer
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, United States of America
| | - Owen Williams
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, United States of America
| | - Susan M Resnick
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, United States of America
| | - Baxter Rogers
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, United States of America
| | - Adam W Anderson
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, United States of America
| | - Bennett A Landman
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, United States of America; Department of Electrical Engineering, Vanderbilt University, Nashville, TN, United States of America; Electrical Engineering & Computer Science, Vanderbilt University, Nashville, TN, United States of America
| |
Collapse
|
6
|
Reinhold JC, Dewey BE, Carass A, Prince JL. Evaluating the Impact of Intensity Normalization on MR Image Synthesis. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10949. [PMID: 31551645 DOI: 10.1117/12.2513089] [Citation(s) in RCA: 54] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Image synthesis learns a transformation from the intensity features of an input image to yield a different tissue contrast of the output image. This process has been shown to have application in many medical image analysis tasks including imputation, registration, and segmentation. To carry out synthesis, the intensities of the input images are typically scaled-i.e., normalized-both in training to learn the transformation and in testing when applying the transformation, but it is not presently known what type of input scaling is optimal. In this paper, we consider seven different intensity normalization algorithms and three different synthesis methods to evaluate the impact of normalization. Our experiments demonstrate that intensity normalization as a preprocessing step improves the synthesis results across all investigated synthesis algorithms. Furthermore, we show evidence that suggests intensity normalization is vital for successful deep learning-based MR image synthesis.
Collapse
Affiliation(s)
- Jacob C Reinhold
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA 21218
| | - Blake E Dewey
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA 21218.,F.M. Kirby Center for Functional Brain Imaging, Kennedy Krieger Institute, Baltimore, MD, USA, 21205
| | - Aaron Carass
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA 21218.,Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA 21218
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA 21218.,Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA 21218
| |
Collapse
|
7
|
Zhao C, Carass A, Lee J, Jog A, Prince JL. A supervoxel based random forest synthesis framework for bidirectional MR/CT synthesis. SIMULATION AND SYNTHESIS IN MEDICAL IMAGING : ... INTERNATIONAL WORKSHOP, SASHIMI ..., HELD IN CONJUNCTION WITH MICCAI ..., PROCEEDINGS. SASHIMI (WORKSHOP) 2017; 10557:33-40. [PMID: 30221260 DOI: 10.1007/978-3-319-68127-6_4] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
Synthesizing magnetic resonance (MR) and computed tomography (CT) images (from each other) has important implications for clinical neuroimaging. The MR to CT direction is critical for MRI-based radiotherapy planning and dose computation, whereas the CT to MR direction can provide an economic alternative to real MRI for image processing tasks. Additionally, synthesis in both directions can enhance MR/CT multi-modal image registration. Existing approaches have focused on synthesizing CT from MR. In this paper, we propose a multi-atlas based hybrid method to synthesize T1-weighted MR images from CT and CT images from T1-weighted MR images using a common framework. The task is carried out by: (a) computing a label field based on supervoxels for the subject image using joint label fusion; (b) correcting this result using a random forest classifier (RF-C); (c) spatial smoothing using a Markov random field; (d) synthesizing intensities using a set of RF regressors, one trained for each label. The algorithm is evaluated using a set of six registered CT and MR image pairs of the whole head.
Collapse
Affiliation(s)
- Can Zhao
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218
| | - Aaron Carass
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218
| | - Junghoon Lee
- Dept. of Radiation Oncology, The Johns Hopkins School of Medicine, Baltimore, MD 21287
| | - Amod Jog
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218
| | - Jerry L Prince
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218
| |
Collapse
|