1
|
Su S, Zhao Y, Ding Y, Lau V, Xiao L, Leung GKK, Lau GKK, Huang F, Vardhanabhuti V, Leong ATL, Wu EX. Ultra-low-field magnetization transfer imaging at 0.055T with low specific absorption rate. Magn Reson Med 2024; 92:2420-2432. [PMID: 39044654 DOI: 10.1002/mrm.30231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Revised: 06/14/2024] [Accepted: 07/09/2024] [Indexed: 07/25/2024]
Abstract
PURPOSE To demonstrate magnetization transfer (MT) effects with low specific absorption rate (SAR) on ultra-low-field (ULF) MRI. METHODS MT imaging was implemented by using sinc-modulated RF pulse train (SPT) modules to provide bilateral off-resonance irradiation. They were incorporated into 3D gradient echo (GRE) and fast spin echo (FSE) protocols on a shielding-free 0.055T head scanner. MT effects were first verified using phantoms. Brain MT imaging was conducted in both healthy subjects and patients. RESULTS MT effects were clearly observed in phantoms using six SPT modules with total flip angle 3600° at central primary saturation bands of approximate offset ±786 Hz, even in the presence of large relative B0 inhomogeneity. For brain, strong MT effects were observed in gray matter, white matter, and muscle in 3D GRE and FSE imaging using six and sixteen SPT modules with total flip angle 3600° and 9600°, respectively. Fat, cerebrospinal fluid, and blood exhibited relatively weak MT effects. MT preparation enhanced tissue contrasts in T2-weighted and FLAIR-like images, and improved brain lesion delineation. The estimated MT SAR was 0.0024 and 0.0008 W/kg for two protocols, respectively, which is far below the US Food and Drug Administration (FDA) limit of 3.0 W/kg. CONCLUSION Robust MT effects can be readily obtained at ULF with extremely low SAR, despite poor relative B0 homogeneity in ppm. This unique advantage enables flexible MT pulse design and implementation on low-cost ULF MRI platforms to achieve strong MT effects in brain and beyond, potentially augmenting their clinical utility in the future.
Collapse
Affiliation(s)
- Shi Su
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Yujiao Zhao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Ye Ding
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Vick Lau
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Linfang Xiao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Gilberto K K Leung
- Department of Surgery, The University of Hong Kong, Hong Kong SAR, China
| | - Gary K K Lau
- Department of Medicine, The University of Hong Kong, Hong Kong SAR, China
| | - Fan Huang
- Department of Diagnostic Radiology, The University of Hong Kong, Hong Kong SAR, China
| | - Vince Vardhanabhuti
- Department of Diagnostic Radiology, The University of Hong Kong, Hong Kong SAR, China
| | - Alex T L Leong
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Ed X Wu
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| |
Collapse
|
2
|
Javadi M, Sharma R, Tsiamyrtzis P, Webb AG, Leiss E, Tsekos NV. Let UNet Play an Adversarial Game: Investigating the Effect of Adversarial Training in Enhancing Low-Resolution MRI. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01205-8. [PMID: 39085718 DOI: 10.1007/s10278-024-01205-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Revised: 07/08/2024] [Accepted: 07/12/2024] [Indexed: 08/02/2024]
Abstract
Adversarial training has attracted much attention in enhancing the visual realism of images, but its efficacy in clinical imaging has not yet been explored. This work investigated adversarial training in a clinical context, by training 206 networks on the OASIS-1 dataset for improving low-resolution and low signal-to-noise ratio (SNR) magnetic resonance images. Each network corresponded to a different combination of perceptual and adversarial loss weights and distinct learning rate values. For each perceptual loss weighting, we identified its corresponding adversarial loss weighting that minimized structural disparity. Each optimally weighted adversarial loss yielded an average SSIM reduction of 1.5%. We further introduced a set of new metrics to assess other clinically relevant image features: Gradient Error (GE) to measure structural disparities; Sharpness to compute edge clarity; and Edge-Contrast Error (ECE) to quantify any distortion of the pixel distribution around edges. Including adversarial loss increased structural enhancement in visual inspection, which correlated with statistically consistent GE reductions (p-value << 0.05). This also resulted in increased Sharpness; however, the level of statistical significance was dependent on the perceptual loss weighting. Additionally, adversarial loss yielded ECE reductions for smaller perceptual loss weightings, while showing non-significant increases (p-value >> 0.05) when these weightings were higher, demonstrating that the increased Sharpness does not adversely distort the pixel distribution around the edges in the image. These studies clearly suggest that adversarial training significantly improves the performance of an MRI enhancement pipeline, and highlights the need for systematic studies of hyperparameter optimization and investigation of alternative image quality metrics.
Collapse
Affiliation(s)
- Mohammad Javadi
- Medical Robotics and Imaging Lab, Department of Computer Science, University of Houston, 501, Philip G. Hoffman Hall, 4800 Calhoun Road, Houston, TX, 77204, USA
| | - Rishabh Sharma
- Medical Robotics and Imaging Lab, Department of Computer Science, University of Houston, 501, Philip G. Hoffman Hall, 4800 Calhoun Road, Houston, TX, 77204, USA
| | - Panagiotis Tsiamyrtzis
- Department of Mechanical Engineering, Politecnico di Milano, Milan, Italy
- Department of Statistics, Athens University of Economics and Business, Athens, Greece
| | - Andrew G Webb
- C.J. Gorter Center for High Field MRI, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Ernst Leiss
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Nikolaos V Tsekos
- Medical Robotics and Imaging Lab, Department of Computer Science, University of Houston, 501, Philip G. Hoffman Hall, 4800 Calhoun Road, Houston, TX, 77204, USA.
| |
Collapse
|
3
|
Heckel R, Jacob M, Chaudhari A, Perlman O, Shimron E. Deep learning for accelerated and robust MRI reconstruction. MAGMA (NEW YORK, N.Y.) 2024; 37:335-368. [PMID: 39042206 DOI: 10.1007/s10334-024-01173-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Revised: 05/24/2024] [Accepted: 05/28/2024] [Indexed: 07/24/2024]
Abstract
Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.
Collapse
Affiliation(s)
- Reinhard Heckel
- Department of computer engineering, Technical University of Munich, Munich, Germany
| | - Mathews Jacob
- Department of Electrical and Computer Engineering, University of Iowa, Iowa, 52242, IA, USA
| | - Akshay Chaudhari
- Department of Radiology, Stanford University, Stanford, 94305, CA, USA
- Department of Biomedical Data Science, Stanford University, Stanford, 94305, CA, USA
| | - Or Perlman
- Department of Biomedical Engineering, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Efrat Shimron
- Department of Electrical and Computer Engineering, Technion-Israel Institute of Technology, Haifa, 3200004, Israel.
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, 3200004, Israel.
| |
Collapse
|
4
|
Zhao Y, Xiao L, Hu J, Wu EX. Robust EMI elimination for RF shielding-free MRI through deep learning direct MR signal prediction. Magn Reson Med 2024; 92:112-127. [PMID: 38376455 DOI: 10.1002/mrm.30046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 01/23/2024] [Accepted: 01/23/2024] [Indexed: 02/21/2024]
Abstract
PURPOSE To develop a new electromagnetic interference (EMI) elimination strategy for RF shielding-free MRI via active EMI sensing and deep learning direct MR signal prediction (Deep-DSP). METHODS Deep-DSP is proposed to directly predict EMI-free MR signals. During scanning, MRI receive coil and EMI sensing coils simultaneously sample data within two windows (i.e., for MR data and EMI characterization data acquisition, respectively). Afterward, a residual U-Net model is trained using synthetic MRI receive coil data and EMI sensing coil data acquired during EMI signal characterization window, to predict EMI-free MR signals from signals acquired by MRI receive and EMI sensing coils. The trained model is then used to directly predict EMI-free MR signals from data acquired by MRI receive and sensing coils during the MR signal-acquisition window. This strategy was evaluated on an ultralow-field 0.055T brain MRI scanner without any RF shielding and a 1.5T whole-body scanner with incomplete RF shielding. RESULTS Deep-DSP accurately predicted EMI-free MR signals in presence of strong EMI. It outperformed recently developed EDITER and convolutional neural network methods, yielding better EMI elimination and enabling use of few EMI sensing coils. Furthermore, it could work well without dedicated EMI characterization data. CONCLUSION Deep-DSP presents an effective EMI elimination strategy that outperforms existing methods, advancing toward truly portable and patient-friendly MRI. It exploits electromagnetic coupling between MRI receive and EMI sensing coils as well as typical MR signal characteristics. Despite its deep learning nature, Deep-DSP framework is computationally simple and efficient.
Collapse
Affiliation(s)
- Yujiao Zhao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong, People's Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, People's Republic of China
| | - Linfang Xiao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong, People's Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, People's Republic of China
| | - Jiahao Hu
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong, People's Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, People's Republic of China
| | - Ed X Wu
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong, People's Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, People's Republic of China
| |
Collapse
|
5
|
Samardzija A, Selvaganesan K, Zhang HZ, Sun H, Sun C, Ha Y, Galiana G, Constable RT. Low-Field, Low-Cost, Point-of-Care Magnetic Resonance Imaging. Annu Rev Biomed Eng 2024; 26:67-91. [PMID: 38211326 DOI: 10.1146/annurev-bioeng-110122-022903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2024]
Abstract
Low-field magnetic resonance imaging (MRI) has recently experienced a renaissance that is largely attributable to the numerous technological advancements made in MRI, including optimized pulse sequences, parallel receive and compressed sensing, improved calibrations and reconstruction algorithms, and the adoption of machine learning for image postprocessing. This new attention on low-field MRI originates from a lack of accessibility to traditional MRI and the need for affordable imaging. Low-field MRI provides a viable option due to its lack of reliance on radio-frequency shielding rooms, expensive liquid helium, and cryogen quench pipes. Moreover, its relatively small size and weight allow for easy and affordable installation in most settings. Rather than replacing conventional MRI, low-field MRI will provide new opportunities for imaging both in developing and developed countries. This article discusses the history of low-field MRI, low-field MRI hardware and software, current devices on the market, advantages and disadvantages, and low-field MRI's global potential.
Collapse
Affiliation(s)
- Anja Samardzija
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut, USA;
| | - Kartiga Selvaganesan
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut, USA;
| | - Horace Z Zhang
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut, USA;
| | - Heng Sun
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut, USA;
| | - Chenhao Sun
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut, USA
| | - Yonghyun Ha
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut, USA
| | - Gigi Galiana
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut, USA;
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut, USA
| | - R Todd Constable
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut, USA;
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut, USA
| |
Collapse
|
6
|
Suwannasak A, Angkurawaranon S, Sangpin P, Chatnuntawech I, Wantanajittikul K, Yarach U. Deep learning-based super-resolution of structural brain MRI at 1.5 T: application to quantitative volume measurement. MAGMA (NEW YORK, N.Y.) 2024; 37:465-475. [PMID: 38758489 DOI: 10.1007/s10334-024-01165-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 04/27/2024] [Accepted: 04/30/2024] [Indexed: 05/18/2024]
Abstract
OBJECTIVE This study investigated the feasibility of using deep learning-based super-resolution (DL-SR) technique on low-resolution (LR) images to generate high-resolution (HR) MR images with the aim of scan time reduction. The efficacy of DL-SR was also assessed through the application of brain volume measurement (BVM). MATERIALS AND METHODS In vivo brain images acquired with 3D-T1W from various MRI scanners were utilized. For model training, LR images were generated by downsampling the original 1 mm-2 mm isotropic resolution images. Pairs of LR and HR images were used for training 3D residual dense net (RDN). For model testing, actual scanned 2 mm isotropic resolution 3D-T1W images with one-minute scan time were used. Normalized root-mean-square error (NRMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) were used for model evaluation. The evaluation also included brain volume measurement, with assessments of subcortical brain regions. RESULTS The results showed that DL-SR model improved the quality of LR images compared with cubic interpolation, as indicated by NRMSE (24.22% vs 30.13%), PSNR (26.19 vs 24.65), and SSIM (0.96 vs 0.95). For volumetric assessments, there were no significant differences between DL-SR and actual HR images (p > 0.05, Pearson's correlation > 0.90) at seven subcortical regions. DISCUSSION The combination of LR MRI and DL-SR enables addressing prolonged scan time in 3D MRI scans while providing sufficient image quality without affecting brain volume measurement.
Collapse
Affiliation(s)
- Atita Suwannasak
- Department of Radiologic Technology, Faculty of Associated Medical Sciences, Chiang Mai University, 110 Intavaroros Road, Muang, Chiang Mai, 50200, Thailand
| | - Salita Angkurawaranon
- Department of Radiology, Faculty of Medicine, Chiang Mai University, Intavaroros Road, Muang, Chiang Mai, Thailand
| | - Prapatsorn Sangpin
- Philips (Thailand) Ltd, New Petchburi Road, Bangkapi, Huaykwang, Bangkok, Thailand
| | - Itthi Chatnuntawech
- National Nanotechnology Center (NANOTEC), Phahon Yothin Road, Khlong Nueng, Khlong Luang, Pathum Thani, Thailand
| | - Kittichai Wantanajittikul
- Department of Radiologic Technology, Faculty of Associated Medical Sciences, Chiang Mai University, 110 Intavaroros Road, Muang, Chiang Mai, 50200, Thailand
| | - Uten Yarach
- Department of Radiologic Technology, Faculty of Associated Medical Sciences, Chiang Mai University, 110 Intavaroros Road, Muang, Chiang Mai, 50200, Thailand.
| |
Collapse
|
7
|
Seghier ML. 7 T and beyond: toward a synergy between fMRI-based presurgical mapping at ultrahigh magnetic fields, AI, and robotic neurosurgery. Eur Radiol Exp 2024; 8:73. [PMID: 38945979 PMCID: PMC11214939 DOI: 10.1186/s41747-024-00472-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2024] [Accepted: 04/22/2024] [Indexed: 07/02/2024] Open
Abstract
Presurgical evaluation with functional magnetic resonance imaging (fMRI) can reduce postsurgical morbidity. Here, we discuss presurgical fMRI mapping at ultra-high magnetic fields (UHF), i.e., ≥ 7 T, in the light of the current growing interest in artificial intelligence (AI) and robot-assisted neurosurgery. The potential of submillimetre fMRI mapping can help better appreciate uncertainty on resection margins, though geometric distortions at UHF might lessen the accuracy of fMRI maps. A useful trade-off for UHF fMRI is to collect data with 1-mm isotropic resolution to ensure high sensitivity and subsequently a low risk of false negatives. Scanning at UHF might yield a revival interest in slow event-related fMRI, thereby offering a richer depiction of the dynamics of fMRI responses. The potential applications of AI concern denoising and artefact removal, generation of super-resolution fMRI maps, and accurate fusion or coregistration between anatomical and fMRI maps. The latter can benefit from the use of T1-weighted echo-planar imaging for better visualization of brain activations. Such AI-augmented fMRI maps would provide high-quality input data to robotic surgery systems, thereby improving the accuracy and reliability of robot-assisted neurosurgery. Ultimately, the advancement in fMRI at UHF would promote clinically useful synergies between fMRI, AI, and robotic neurosurgery.Relevance statement This review highlights the potential synergies between fMRI at UHF, AI, and robotic neurosurgery in improving the accuracy and reliability of fMRI-based presurgical mapping.Key points• Presurgical fMRI mapping at UHF improves spatial resolution and sensitivity.• Slow event-related designs offer a richer depiction of fMRI responses dynamics.• AI can support denoising, artefact removal, and generation of super-resolution fMRI maps.• AI-augmented fMRI maps can provide high-quality input data to robotic surgery systems.
Collapse
Affiliation(s)
- Mohamed L Seghier
- Department of Biomedical Engineering and Biotechnology, Khalifa University of Science and Technology, Abu Dhabi, UAE.
- Healtcare Engineering Innovation Center (HEIC), Khalifa University of Science and Technology, Abu Dhabi, UAE.
| |
Collapse
|
8
|
Donnay C, Okar SV, Tsagkas C, Gaitán MI, Poorman M, Reich DS, Nair G. Super resolution using sparse sampling at portable ultra-low field MR. Front Neurol 2024; 15:1330203. [PMID: 38854960 PMCID: PMC11157107 DOI: 10.3389/fneur.2024.1330203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 05/03/2024] [Indexed: 06/11/2024] Open
Abstract
Ultra-low field (ULF) magnetic resonance imaging (MRI) holds the potential to make MRI more accessible, given its cost-effectiveness, reduced power requirements, and portability. However, signal-to-noise ratio (SNR) drops with field strength, necessitating imaging with lower resolution and longer scan times. This study introduces a novel Fourier-based Super Resolution (FouSR) approach, designed to enhance the resolution of ULF MRI images with minimal increase in total scan time. FouSR combines spatial frequencies from two orthogonal ULF images of anisotropic resolution to create an isotropic T2-weighted fluid-attenuated inversion recovery (FLAIR) image. We hypothesized that FouSR could effectively recover information from under-sampled slice directions, thereby improving the delineation of multiple sclerosis (MS) lesions and other significant anatomical features. Importantly, the FouSR algorithm can be implemented on the scanner with changes to the k-space trajectory. Paired ULF (Hyperfine SWOOP, 0.064 tesla) and high field (Siemens, Skyra, 3 Tesla) FLAIR scans were collected on the same day from a phantom and a cohort of 10 participants with MS or suspected MS (6 female; mean ± SD age: 44.1 ± 4.1). ULF scans were acquired along both coronal and axial planes, featuring an in-plane resolution of 1.7 mm × 1.7 mm with a slice thickness of 5 mm. FouSR was evaluated against registered ULF coronal and axial scans, their average (ULF average) and a gold standard SR (ANTs SR). FouSR exhibited higher SNR (47.96 ± 12.6) compared to ULF coronal (36.7 ± 12.2) and higher lesion conspicuity (0.12 ± 0.06) compared to ULF axial (0.13 ± 0.07) but did not exhibit any significant differences contrast-to-noise-ratio (CNR) compared to other methods in patient scans. However, FouSR demonstrated superior image sharpness (0.025 ± 0.0040) compared to all other techniques (ULF coronal 0.021 ± 0.0037, q = 5.9, p-adj. = 0.011; ULF axial 0.018 ± 0.0026, q = 11.1, p-adj. = 0.0001; ULF average 0.019 ± 0.0034, q = 24.2, p-adj. < 0.0001) and higher lesion sharpness (-0.97 ± 0.31) when compared to the ULF average (-1.02 ± 0.37, t(543) = -10.174, p = <0.0001). Average blinded qualitative assessment by three experienced MS neurologists showed no significant difference in WML and sulci or gyri visualization between FouSR and other methods. FouSR can, in principle, be implemented on the scanner to produce clinically useful FLAIR images at higher resolution on the fly, providing a valuable tool for visualizing lesions and other anatomical structures in MS.
Collapse
Affiliation(s)
- Corinne Donnay
- Translational Neuroradiology Section, National Institutes of Health, National Institute of Neurological Disorders and Stroke, Bethesda, MD, United States
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Serhat V. Okar
- Translational Neuroradiology Section, National Institutes of Health, National Institute of Neurological Disorders and Stroke, Bethesda, MD, United States
| | - Charidimos Tsagkas
- Translational Neuroradiology Section, National Institutes of Health, National Institute of Neurological Disorders and Stroke, Bethesda, MD, United States
| | - María I. Gaitán
- Translational Neuroradiology Section, National Institutes of Health, National Institute of Neurological Disorders and Stroke, Bethesda, MD, United States
| | | | - Daniel S. Reich
- Translational Neuroradiology Section, National Institutes of Health, National Institute of Neurological Disorders and Stroke, Bethesda, MD, United States
| | - Govind Nair
- Quantitative MRI Core, National Institutes of Health, National Institute of Neurological Disorders and Stroke, Bethesda, MD, United States
| |
Collapse
|
9
|
Shin M, Seo M, Lee K, Yoon K. Super-resolution techniques for biomedical applications and challenges. Biomed Eng Lett 2024; 14:465-496. [PMID: 38645589 PMCID: PMC11026337 DOI: 10.1007/s13534-024-00365-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 02/12/2024] [Accepted: 02/18/2024] [Indexed: 04/23/2024] Open
Abstract
Super-resolution (SR) techniques have revolutionized the field of biomedical applications by detailing the structures at resolutions beyond the limits of imaging or measuring tools. These techniques have been applied in various biomedical applications, including microscopy, magnetic resonance imaging (MRI), computed tomography (CT), X-ray, electroencephalogram (EEG), ultrasound, etc. SR methods are categorized into two main types: traditional non-learning-based methods and modern learning-based approaches. In both applications, SR methodologies have been effectively utilized on biomedical images, enhancing the visualization of complex biological structures. Additionally, these methods have been employed on biomedical data, leading to improvements in computational precision and efficiency for biomedical simulations. The use of SR techniques has resulted in more detailed and accurate analyses in diagnostics and research, essential for early disease detection and treatment planning. However, challenges such as computational demands, data interpretation complexities, and the lack of unified high-quality data persist. The article emphasizes these issues, underscoring the need for ongoing development in SR technologies to further improve biomedical research and patient care outcomes.
Collapse
Affiliation(s)
- Minwoo Shin
- School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, 50 Yonsei-Ro, Seodaemun-Gu, Seoul, 03722 Republic of Korea
| | - Minjee Seo
- School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, 50 Yonsei-Ro, Seodaemun-Gu, Seoul, 03722 Republic of Korea
| | - Kyunghyun Lee
- School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, 50 Yonsei-Ro, Seodaemun-Gu, Seoul, 03722 Republic of Korea
| | - Kyungho Yoon
- School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, 50 Yonsei-Ro, Seodaemun-Gu, Seoul, 03722 Republic of Korea
| |
Collapse
|
10
|
Wang L, Guo T, Wang L, Yang W, Wang J, Nie J, Cui J, Jiang P, Li J, Zhang H. Improving radiomic modeling for the identification of symptomatic carotid atherosclerotic plaques using deep learning-based 3D super-resolution CT angiography. Heliyon 2024; 10:e29331. [PMID: 38644848 PMCID: PMC11033096 DOI: 10.1016/j.heliyon.2024.e29331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2024] [Revised: 04/04/2024] [Accepted: 04/05/2024] [Indexed: 04/23/2024] Open
Abstract
Rationale and objectives Radiomic models based on normal-resolution (NR) computed tomography angiography (CTA) images can fail to distinguish between symptomatic and asymptomatic carotid atherosclerotic plaques. This study aimed to explore the effectiveness of a deep learning-based three-dimensional super-resolution (SR) CTA radiomic model for improved identification of symptomatic carotid atherosclerotic plaques. Materials and methods A total of 193 patients with carotid atherosclerotic plaques were retrospectively enrolled and allocated into either a symptomatic (n = 123) or an asymptomatic (n = 70) groups. SR CTA images were derived from NR CTA images using deep learning-based three-dimensional SR technology. Handcrafted radiomic features were extracted from both the SR and NR CTA images and three risk models were developed based on manually measured quantitative CTA characteristics and NR and SR radiomic features. Model performances were assessed via receiver operating characteristic, calibration, and decision curve analyses. Results The SR model exhibited the optimal performance (area under the curve [AUC] 0.820, accuracy 0.802, sensitivity 0.854, F1 score 0.847) in the testing cohort, outperforming the other two models. The calibration curve analyses and Hosmer-Lemeshow test demonstrated that the SR model exhibited the best goodness of fit, and decision curve analysis revealed that SR model had the highest clinical value and potential patient benefits. Conclusions Deep learning-based three-dimensional SR technology could improve the CTA-based radiomic models in identifying symptomatic carotid plaques, potentially providing more accurate and valuable information to guide clinical decision-making to reduce the risk of ischemic stroke.
Collapse
Affiliation(s)
- Lingjie Wang
- Department of Medical Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi Province, 030001, China
| | - Tiedan Guo
- Department of Medical Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi Province, 030001, China
| | - Li Wang
- Department of Medical Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi Province, 030001, China
| | - Wentao Yang
- Basic Medical College, Shanxi Medical University, Taiyuan, Shanxi Province, 030001, China
| | - Jingying Wang
- Department of Endemic Disease Prevention and Control, Shanxi Province Disease Prevention and Control Center, Shanxi Province, 030001, China
| | - Jianlong Nie
- Shanghai United Imaging Intelligence, Co., Ltd., Shanghai City, 200030, China
| | - Jingjing Cui
- Shanghai United Imaging Intelligence, Co., Ltd., Shanghai City, 200030, China
| | - Pengbo Jiang
- Shanghai United Imaging Intelligence, Co., Ltd., Shanghai City, 200030, China
| | - Junlin Li
- Department of Imaging Medicine, Inner Mongolia Autonomous Region People's Hospital, Hohhot, 010017, China
| | - Hua Zhang
- Department of Medical Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi Province, 030001, China
| |
Collapse
|
11
|
Grover J, Liu P, Dong B, Shan S, Whelan B, Keall P, Waddington DEJ. Super-resolution neural networks improve the spatiotemporal resolution of adaptive MRI-guided radiation therapy. COMMUNICATIONS MEDICINE 2024; 4:64. [PMID: 38575723 PMCID: PMC10994938 DOI: 10.1038/s43856-024-00489-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 03/22/2024] [Indexed: 04/06/2024] Open
Abstract
BACKGROUND Magnetic resonance imaging (MRI) offers superb non-invasive, soft tissue imaging of the human body. However, extensive data sampling requirements severely restrict the spatiotemporal resolution achievable with MRI. This limits the modality's utility in real-time guidance applications, particularly for the rapidly growing MRI-guided radiation therapy approach to cancer treatment. Recent advances in artificial intelligence (AI) could reduce the trade-off between the spatial and the temporal resolution of MRI, thus increasing the clinical utility of the imaging modality. METHODS We trained deep learning-based super-resolution neural networks to increase the spatial resolution of real-time MRI. We developed a framework to integrate neural networks directly onto a 1.0 T MRI-linac enabling real-time super-resolution imaging. We integrated this framework with the targeting system of the MRI-linac to demonstrate real-time beam adaptation with super-resolution-based imaging. We tested the integrated system using large publicly available datasets, healthy volunteer imaging, phantom imaging, and beam tracking experiments using bicubic interpolation as a baseline comparison. RESULTS Deep learning-based super-resolution increases the spatial resolution of real-time MRI across a variety of experiments, offering measured performance benefits compared to bicubic interpolation. The temporal resolution is not compromised as measured by a real-time adaptation latency experiment. These two effects, an increase in the spatial resolution with a negligible decrease in the temporal resolution, leads to a net increase in the spatiotemporal resolution. CONCLUSIONS Deployed super-resolution neural networks can increase the spatiotemporal resolution of real-time MRI. This has applications to domains such as MRI-guided radiation therapy and interventional procedures.
Collapse
Affiliation(s)
- James Grover
- Image X Institute, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia.
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia.
| | - Paul Liu
- Image X Institute, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia
| | - Bin Dong
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia
| | - Shanshan Shan
- Image X Institute, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia
- State Key Laboratory of Radiation Medicine and Protection, School for Radiological and Interdisciplinary Sciences (RAD-X), Collaborative Innovation Center of Radiation Medicine of Jiangsu Higher Education Institutions, Soochow University, Suzhou, Jiangsu, China
| | - Brendan Whelan
- Image X Institute, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia
| | - Paul Keall
- Image X Institute, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia
| | - David E J Waddington
- Image X Institute, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
- Department of Medical Physics, Ingham Institute for Applied Medical Research, Sydney, NSW, Australia
| |
Collapse
|
12
|
Bao Q, Liu X, Xu J, Xia L, Otikovs M, Xie H, Liu K, Zhang Z, Zhou X, Liu C. Unsupervised deep learning model for correcting Nyquist ghosts of single-shot spatiotemporal encoding. Magn Reson Med 2024; 91:1368-1383. [PMID: 38073072 DOI: 10.1002/mrm.29925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 10/20/2023] [Accepted: 10/25/2023] [Indexed: 02/03/2024]
Abstract
PURPOSE To design an unsupervised deep learning (DL) model for correcting Nyquist ghosts of single-shot spatiotemporal encoding (SPEN) and evaluate the model for real MRI applications. METHODS The proposed method consists of three main components: (1) an unsupervised network that combines Residual Encoder and Restricted Subspace Mapping (RERSM-net) and is trained to generate a phase-difference map based on the even and odd SPEN images; (2) a spin physical forward model to obtain the corrected image with the learned phase difference map; and (3) cycle-consistency loss that is explored for training the RERSM-net. RESULTS The proposed RERSM-net could effectively generate smooth phase difference maps and correct Nyquist ghosts of single-shot SPEN. Both simulation and real in vivo MRI experiments demonstrated that our method outperforms the state-of-the-art SPEN Nyquist ghost correction method. Furthermore, the ablation experiments of generating phase-difference maps show the advantages of the proposed unsupervised model. CONCLUSION The proposed method can effectively correct Nyquist ghosts for the single-shot SPEN sequence.
Collapse
Affiliation(s)
- Qingjia Bao
- Key Laboratory of Magnetic Resonance in Biological Systems, Innovation Academy for Precision Measurement Science and Technology, Wuhan, China
| | - Xinjie Liu
- Key Laboratory of Magnetic Resonance in Biological Systems, Innovation Academy for Precision Measurement Science and Technology, Wuhan, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Jingyun Xu
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Liyang Xia
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | | | - Han Xie
- Key Laboratory of Magnetic Resonance in Biological Systems, Innovation Academy for Precision Measurement Science and Technology, Wuhan, China
| | - Kewen Liu
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Zhi Zhang
- Key Laboratory of Magnetic Resonance in Biological Systems, Innovation Academy for Precision Measurement Science and Technology, Wuhan, China
| | - Xin Zhou
- Key Laboratory of Magnetic Resonance in Biological Systems, Innovation Academy for Precision Measurement Science and Technology, Wuhan, China
- University of Chinese Academy of Sciences, Beijing, China
- Optics Valley Laboratory, Wuhan, China
| | - Chaoyang Liu
- Key Laboratory of Magnetic Resonance in Biological Systems, Innovation Academy for Precision Measurement Science and Technology, Wuhan, China
- University of Chinese Academy of Sciences, Beijing, China
- Optics Valley Laboratory, Wuhan, China
| |
Collapse
|
13
|
Lucas A, Campbell Arnold T, Okar SV, Vadali C, Kawatra KD, Ren Z, Cao Q, Shinohara RT, Schindler MK, Davis KA, Litt B, Reich DS, Stein JM. Multi-contrast high-field quality image synthesis for portable low-field MRI using generative adversarial networks and paired data. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.12.28.23300409. [PMID: 38234785 PMCID: PMC10793526 DOI: 10.1101/2023.12.28.23300409] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2024]
Abstract
Introduction Portable low-field strength (64mT) MRI scanners promise to increase access to neuroimaging for clinical and research purposes, however these devices produce lower quality images compared to high-field scanners. In this study, we developed and evaluated a deep learning architecture to generate high-field quality brain images from low-field inputs using a paired dataset of multiple sclerosis (MS) patients scanned at 64mT and 3T. Methods A total of 49 MS patients were scanned on portable 64mT and standard 3T scanners at Penn (n=25) or the National Institutes of Health (NIH, n=24) with T1-weighted, T2-weighted and FLAIR acquisitions. Using this paired data, we developed a generative adversarial network (GAN) architecture for low- to high-field image translation (LowGAN). We then evaluated synthesized images with respect to image quality, brain morphometry, and white matter lesions. Results Synthetic high-field images demonstrated visually superior quality compared to low-field inputs and significantly higher normalized cross-correlation (NCC) to actual high-field images for T1 (p=0.001) and FLAIR (p<0.001) contrasts. LowGAN generally outperformed the current state-of-the-art for low-field volumetrics. For example, thalamic, lateral ventricle, and total cortical volumes in LowGAN outputs did not differ significantly from 3T measurements. Synthetic outputs preserved MS lesions and captured a known inverse relationship between total lesion volume and thalamic volume. Conclusions LowGAN generates synthetic high-field images with comparable visual and quantitative quality to actual high-field scans. Enhancing portable MRI image quality could add value and boost clinician confidence, enabling wider adoption of this technology.
Collapse
Affiliation(s)
- Alfredo Lucas
- Perelman School of Medicine, University of Pennsylvania
- Center for Neuroengineering and Therapeutics, Departments of Bioengineering and Neurology, University of Pennsylvania
| | - T Campbell Arnold
- Center for Neuroengineering and Therapeutics, Departments of Bioengineering and Neurology, University of Pennsylvania
| | - Serhat V Okar
- National Institute of Neurological Disorders and Stroke, National Institutes of Health
| | - Chetan Vadali
- Center for Neuroengineering and Therapeutics, Departments of Bioengineering and Neurology, University of Pennsylvania
- Department of Radiology, University of Pennsylvania
| | - Karan D Kawatra
- National Institute of Neurological Disorders and Stroke, National Institutes of Health
| | - Zheng Ren
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
| | - Quy Cao
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
| | - Russell T Shinohara
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania
| | - Matthew K Schindler
- Perelman School of Medicine, University of Pennsylvania
- Department of Neurology, University of Pennsylvania
| | - Kathryn A Davis
- Perelman School of Medicine, University of Pennsylvania
- Center for Neuroengineering and Therapeutics, Departments of Bioengineering and Neurology, University of Pennsylvania
- Department of Neurology, University of Pennsylvania
| | - Brian Litt
- Perelman School of Medicine, University of Pennsylvania
- Center for Neuroengineering and Therapeutics, Departments of Bioengineering and Neurology, University of Pennsylvania
- Department of Neurology, University of Pennsylvania
| | - Daniel S Reich
- National Institute of Neurological Disorders and Stroke, National Institutes of Health
| | - Joel M Stein
- Perelman School of Medicine, University of Pennsylvania
- Center for Neuroengineering and Therapeutics, Departments of Bioengineering and Neurology, University of Pennsylvania
- Department of Radiology, University of Pennsylvania
| |
Collapse
|
14
|
Islam KT, Zhong S, Zakavi P, Chen Z, Kavnoudias H, Farquharson S, Durbridge G, Barth M, McMahon KL, Parizel PM, Dwyer A, Egan GF, Law M, Chen Z. Improving portable low-field MRI image quality through image-to-image translation using paired low- and high-field images. Sci Rep 2023; 13:21183. [PMID: 38040835 PMCID: PMC10692211 DOI: 10.1038/s41598-023-48438-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 11/27/2023] [Indexed: 12/03/2023] Open
Abstract
Low-field portable magnetic resonance imaging (MRI) scanners are more accessible, cost-effective, sustainable with lower carbon emissions than superconducting high-field MRI scanners. However, the images produced have relatively poor image quality, lower signal-to-noise ratio, and limited spatial resolution. This study develops and investigates an image-to-image translation deep learning model, LoHiResGAN, to enhance the quality of low-field (64mT) MRI scans and generate synthetic high-field (3T) MRI scans. We employed a paired dataset comprising T1- and T2-weighted MRI sequences from the 64mT and 3T and compared the performance of the LoHiResGAN model with other state-of-the-art models, including GANs, CycleGAN, U-Net, and cGAN. Our proposed method demonstrates superior performance in terms of image quality metrics, such as normalized root-mean-squared error, structural similarity index measure, peak signal-to-noise ratio, and perception-based image quality evaluator. Additionally, we evaluated the accuracy of brain morphometry measurements for 33 brain regions across the original 3T, 64mT, and synthetic 3T images. The results indicate that the synthetic 3T images created using our proposed LoHiResGAN model significantly improve the image quality of low-field MRI data compared to other methods (GANs, CycleGAN, U-Net, cGAN) and provide more consistent brain morphometry measurements across various brain regions in reference to 3T. Synthetic images generated by our method demonstrated high quality both quantitatively and qualitatively. However, additional research, involving diverse datasets and clinical validation, is necessary to fully understand its applicability for clinical diagnostics, especially in settings where high-field MRI scanners are less accessible.
Collapse
Affiliation(s)
- Kh Tohidul Islam
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, Australia.
| | - Shenjun Zhong
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, Australia
- Australian National Imaging Facility, Brisbane, QLD, Australia
| | - Parisa Zakavi
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, Australia
| | - Zhifeng Chen
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, Australia
- Department of Data Science and AI, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia
| | - Helen Kavnoudias
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
- Department of Radiology, Alfred Hospital, Melbourne, VIC, Australia
| | | | - Gail Durbridge
- Herston Imaging Research Facility, University of Queensland, Brisbane, QLD, Australia
| | - Markus Barth
- School of Information Technology and Electrical Engineering and Centre for Advanced Imaging, University of Queensland, Brisbane, QLD, Australia
| | - Katie L McMahon
- School of Clinical Science, Herston Imaging Research Facility, Queensland University of Technology, Brisbane, QLD, Australia
| | - Paul M Parizel
- David Hartley Chair of Radiology, Department of Radiology, Royal Perth Hospital, Perth, WA, Australia
- Medical School, University of Western Australia, Perth, WA, Australia
| | - Andrew Dwyer
- South Australian Health and Medical Research Institute, Adelaide, SA, Australia
| | - Gary F Egan
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, Australia
| | - Meng Law
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
- Department of Radiology, Alfred Hospital, Melbourne, VIC, Australia
| | - Zhaolin Chen
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, Australia
- Department of Data Science and AI, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
15
|
Eddy CZ, Naylor A, Cunningham CT, Sun B. Facilitating cell segmentation with the projection-enhancement network. Phys Biol 2023; 20:10.1088/1478-3975/acfe53. [PMID: 37769666 PMCID: PMC10586931 DOI: 10.1088/1478-3975/acfe53] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 09/28/2023] [Indexed: 10/03/2023]
Abstract
Contemporary approaches to instance segmentation in cell science use 2D or 3D convolutional networks depending on the experiment and data structures. However, limitations in microscopy systems or efforts to prevent phototoxicity commonly require recording sub-optimally sampled data that greatly reduces the utility of such 3D data, especially in crowded sample space with significant axial overlap between objects. In such regimes, 2D segmentations are both more reliable for cell morphology and easier to annotate. In this work, we propose the projection enhancement network (PEN), a novel convolutional module which processes the sub-sampled 3D data and produces a 2D RGB semantic compression, and is trained in conjunction with an instance segmentation network of choice to produce 2D segmentations. Our approach combines augmentation to increase cell density using a low-density cell image dataset to train PEN, and curated datasets to evaluate PEN. We show that with PEN, the learned semantic representation in CellPose encodes depth and greatly improves segmentation performance in comparison to maximum intensity projection images as input, but does not similarly aid segmentation in region-based networks like Mask-RCNN. Finally, we dissect the segmentation strength against cell density of PEN with CellPose on disseminated cells from side-by-side spheroids. We present PEN as a data-driven solution to form compressed representations of 3D data that improve 2D segmentations from instance segmentation networks.
Collapse
Affiliation(s)
| | - Austin Naylor
- Oregon State University, Department of Physics, Corvallis, 97331, USA
| | | | - Bo Sun
- Oregon State University, Department of Physics, Corvallis, 97331, USA
| |
Collapse
|
16
|
Campbell-Washburn AE, Keenan KE, Hu P, Mugler JP, Nayak KS, Webb AG, Obungoloch J, Sheth KN, Hennig J, Rosen MS, Salameh N, Sodickson DK, Stein JM, Marques JP, Simonetti OP. Low-field MRI: A report on the 2022 ISMRM workshop. Magn Reson Med 2023; 90:1682-1694. [PMID: 37345725 PMCID: PMC10683532 DOI: 10.1002/mrm.29743] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 04/21/2023] [Accepted: 05/17/2023] [Indexed: 06/23/2023]
Abstract
In March 2022, the first ISMRM Workshop on Low-Field MRI was held virtually. The goals of this workshop were to discuss recent low field MRI technology including hardware and software developments, novel methodology, new contrast mechanisms, as well as the clinical translation and dissemination of these systems. The virtual Workshop was attended by 368 registrants from 24 countries, and included 34 invited talks, 100 abstract presentations, 2 panel discussions, and 2 live scanner demonstrations. Here, we report on the scientific content of the Workshop and identify the key themes that emerged. The subject matter of the Workshop reflected the ongoing developments of low-field MRI as an accessible imaging modality that may expand the usage of MRI through cost reduction, portability, and ease of installation. Many talks in this Workshop addressed the use of computational power, efficient acquisitions, and contemporary hardware to overcome the SNR limitations associated with low field strength. Participants discussed the selection of appropriate clinical applications that leverage the unique capabilities of low-field MRI within traditional radiology practices, other point-of-care settings, and the broader community. The notion of "image quality" versus "information content" was also discussed, as images from low-field portable systems that are purpose-built for clinical decision-making may not replicate the current standard of clinical imaging. Speakers also described technical challenges and infrastructure challenges related to portability and widespread dissemination, and speculated about future directions for the field to improve the technology and establish clinical value.
Collapse
Affiliation(s)
- Adrienne E Campbell-Washburn
- Cardiovascular Branch, Division of Intramural Research, National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland, USA
| | - Kathryn E Keenan
- Physical Measurement Laboratory, National Institute of Standards and Technology, Boulder, Colorado, USA
| | - Peng Hu
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - John P Mugler
- Department of Radiology & Medical Imaging, Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Krishna S Nayak
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, California, USA
- Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, California, USA
| | - Andrew G Webb
- Department of Radiology, Leiden University Medical Center, Leiden, Netherlands
| | | | - Kevin N Sheth
- Division of Neurocritical Care and Emergency Neurology, Departments of Neurology and Neurosurgery, and the Yale Center for Brain and Mind Health, Yale School of Medicine, New Haven, Connecticut, USA
| | - Jürgen Hennig
- Dept.of Radiology, Medical Physics, University Medical Center Freiburg, Freiburg, Germany
- Center for Basics in NeuroModulation (NeuroModulBasics), Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Matthew S Rosen
- Massachusetts General Hospital, A. A. Martinos Center for Biomedical Imaging, Boston, Massachusetts, USA
- Department of Physics, Harvard University, Cambridge, Massachusetts, USA
| | - Najat Salameh
- Center for Adaptable MRI Technology (AMT Center), Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland
| | - Daniel K Sodickson
- Department of Radiology, NYU Langone Health, New York, New York, USA
- Center for Advanced Imaging Innovation and Research, NYU Langone Health, New York, New York, USA
| | - Joel M Stein
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - José P Marques
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands
| | - Orlando P Simonetti
- Division of Cardiovascular Medicine, Department of Internal Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA
- Department of Radiology, The Ohio State University, Columbus, Ohio, USA
| |
Collapse
|
17
|
Man C, Lau V, Su S, Zhao Y, Xiao L, Ding Y, Leung GK, Leong AT, Wu EX. Deep learning enabled fast 3D brain MRI at 0.055 tesla. SCIENCE ADVANCES 2023; 9:eadi9327. [PMID: 37738341 PMCID: PMC10516503 DOI: 10.1126/sciadv.adi9327] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Accepted: 08/21/2023] [Indexed: 09/24/2023]
Abstract
In recent years, there has been an intensive development of portable ultralow-field magnetic resonance imaging (MRI) for low-cost, shielding-free, and point-of-care applications. However, its quality is poor and scan time is long. We propose a fast acquisition and deep learning reconstruction framework to accelerate brain MRI at 0.055 tesla. The acquisition consists of a single average three-dimensional (3D) encoding with 2D partial Fourier sampling, reducing the scan time of T1- and T2-weighted imaging protocols to 2.5 and 3.2 minutes, respectively. The 3D deep learning leverages the homogeneous brain anatomy available in high-field human brain data to enhance image quality, reduce artifacts and noise, and improve spatial resolution to synthetic 1.5-mm isotropic resolution. Our method successfully overcomes low-signal barrier, reconstructing fine anatomical structures that are reproducible within subjects and consistent across two protocols. It enables fast and quality whole-brain MRI at 0.055 tesla, with potential for widespread biomedical applications.
Collapse
Affiliation(s)
- Christopher Man
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Vick Lau
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Shi Su
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Yujiao Zhao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Linfang Xiao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Ye Ding
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Gilberto K. K. Leung
- Department of Surgery, LKS Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Alex T. L. Leong
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| | - Ed X. Wu
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People’s Republic of China
| |
Collapse
|
18
|
Wu Z, Chen X, Xie S, Shen J, Zeng Y. Super-resolution of brain MRI images based on denoising diffusion probabilistic model. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/03/2023]
|
19
|
Lee DB, Yoon HG, Park SM, Choi JW, Chen G, Kwon HY, Won C. Super-resolution of magnetic systems using deep learning. Sci Rep 2023; 13:11526. [PMID: 37460591 DOI: 10.1038/s41598-023-38335-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Accepted: 07/06/2023] [Indexed: 07/20/2023] Open
Abstract
We construct a deep neural network to enhance the resolution of spin structure images formed by spontaneous symmetry breaking in the magnetic systems. Through the deep neural network, an image is expanded to a super-resolution image and reduced to the original image size to be fitted with the input feed image. The network does not require ground truth images in the training process. Therefore, it can be applied when low-resolution images are provided as training datasets, while high-resolution images are not obtainable due to the intrinsic limitation of microscope techniques. To show the usefulness of the network, we train the network with two types of simulated magnetic structure images; one is from self-organized maze patterns made of chiral magnetic structures, and the other is from magnetic domains separated by walls that are topological defects of the system. The network successfully generates high-resolution images highly correlated with the exact solutions in both cases. To investigate the effectiveness and the differences between datasets, we study the network's noise tolerance and compare the networks' reliabilities. The network is applied with experimental data obtained by magneto-optical Kerr effect microscopy and spin-polarized low-energy electron microscopy.
Collapse
Affiliation(s)
- D B Lee
- Department of Physics, Kyung Hee University, Seoul, 02447, South Korea
- Department of Battery-Smart Factory, Korea University, Seoul, 02841, South Korea
| | - H G Yoon
- Department of Physics, Kyung Hee University, Seoul, 02447, South Korea
| | - S M Park
- Department of Physics, Kyung Hee University, Seoul, 02447, South Korea
| | - J W Choi
- Center for Spintronics, Korea Institute of Science and Technology, Seoul, 02792, South Korea
| | - G Chen
- National Laboratory of Solid State Microstructures and Department of Physics, Nanjing University, Nanjing, 210093, China
- Collaborative Innovation Center of Advanced Microstructures, Nanjing, 210093, China
| | - H Y Kwon
- Center for Spintronics, Korea Institute of Science and Technology, Seoul, 02792, South Korea.
| | - C Won
- Department of Physics, Kyung Hee University, Seoul, 02447, South Korea.
| |
Collapse
|
20
|
Eisenmenger LB, Peret A, Roberts GS, Spahic A, Tang C, Kuner AD, Grayev AM, Field AS, Rowley HA, Kennedy TA. Focused Abbreviated Survey MRI Protocols for Brain and Spine Imaging. Radiographics 2023; 43:e220147. [PMID: 37167089 PMCID: PMC10262597 DOI: 10.1148/rg.220147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 10/12/2022] [Accepted: 10/18/2022] [Indexed: 05/13/2023]
Abstract
There has been extensive growth in both the technical development and the clinical applications of MRI, establishing this modality as one of the most powerful diagnostic imaging tools. However, long examination and image interpretation times still limit the application of MRI, especially in emergent clinical settings. Rapid and abbreviated MRI protocols have been developed as alternatives to standard MRI, with reduced imaging times, and in some cases limited numbers of sequences, to more efficiently answer specific clinical questions. A group of rapid MRI protocols used at the authors' institution, referred to as FAST (focused abbreviated survey techniques), are designed to include or exclude emergent or urgent conditions or screen for specific entities. These FAST protocols provide adequate diagnostic image quality with use of accelerated approaches to produce imaging studies faster than traditional methods. FAST protocols have become critical diagnostic screening tools at the authors' institution, allowing confident and efficient confirmation or exclusion of actionable findings. The techniques commonly used to reduce imaging times, the imaging protocols used at the authors' institution, and future directions in FAST imaging are reviewed to provide a practical and comprehensive overview of FAST MRI for practicing neuroradiologists. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.
Collapse
Affiliation(s)
| | | | - Grant S. Roberts
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Alma Spahic
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Chenwei Tang
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Anthony D. Kuner
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Allison M. Grayev
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Aaron S. Field
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Howard A. Rowley
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| | - Tabassum A. Kennedy
- From the Departments of Radiology (L.B.E., A.P., A.D.K., A.M.G.,
A.S.F., H.A.R., T.A.K.) and Medical Physics (G.S.R., A.S., C.T.), University of
Wisconsin School of Medicine and Public Health, 600 Highland Ave, Madison, WI
53792-3252
| |
Collapse
|
21
|
Liang Z, Zhang J. Mouse brain MR super-resolution using a deep learning network trained with optical imaging data. FRONTIERS IN RADIOLOGY 2023; 3:1155866. [PMID: 37492378 PMCID: PMC10365285 DOI: 10.3389/fradi.2023.1155866] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 04/28/2023] [Indexed: 07/27/2023]
Abstract
Introduction The resolution of magnetic resonance imaging is often limited at the millimeter level due to its inherent signal-to-noise disadvantage compared to other imaging modalities. Super-resolution (SR) of MRI data aims to enhance its resolution and diagnostic value. While deep learning-based SR has shown potential, its applications in MRI remain limited, especially for preclinical MRI, where large high-resolution MRI datasets for training are often lacking. Methods In this study, we first used high-resolution mouse brain auto-fluorescence (AF) data acquired using serial two-photon tomography (STPT) to examine the performance of deep learning-based SR for mouse brain images. Results We found that the best SR performance was obtained when the resolutions of training and target data were matched. We then applied the network trained using AF data to MRI data of the mouse brain, and found that the performance of the SR network depended on the tissue contrast presented in the MRI data. Using transfer learning and a limited set of high-resolution mouse brain MRI data, we were able to fine-tune the initial network trained using AF to enhance the resolution of MRI data. Discussion Our results suggest that deep learning SR networks trained using high-resolution data of a different modality can be applied to MRI data after transfer learning.
Collapse
Affiliation(s)
| | - Jiangyang Zhang
- Department of Radiology, Center for Biomedical Imaging, New York University, New York, NY, United States
| |
Collapse
|
22
|
Anazodo UC, Adewole M, Dako F. AI for Population and Global Health in Radiology. Radiol Artif Intell 2022; 4:e220107. [PMID: 35923372 PMCID: PMC9344206 DOI: 10.1148/ryai.220107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Revised: 06/05/2022] [Accepted: 06/08/2022] [Indexed: 06/15/2023]
|
23
|
Clinical evaluation of super-resolution for brain MRI images based on generative adversarial networks. INFORMATICS IN MEDICINE UNLOCKED 2022. [DOI: 10.1016/j.imu.2022.101030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
|