1
|
Li J, Tuckute G, Fedorenko E, Edlow BL, Dalca AV, Fischl B. JOSA: Joint surface-based registration and atlas construction of brain geometry and function. Med Image Anal 2024; 98:103292. [PMID: 39173411 DOI: 10.1016/j.media.2024.103292] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 06/21/2024] [Accepted: 07/30/2024] [Indexed: 08/24/2024]
Abstract
Surface-based cortical registration is an important topic in medical image analysis and facilitates many downstream applications. Current approaches for cortical registration are mainly driven by geometric features, such as sulcal depth and curvature, and often assume that registration of folding patterns leads to alignment of brain function. However, functional variability of anatomically corresponding areas across subjects has been widely reported, particularly in higher-order cognitive areas. In this work, we present JOSA, a novel cortical registration framework that jointly models the mismatch between geometry and function while simultaneously learning an unbiased population-specific atlas. Using a semi-supervised training strategy, JOSA achieves superior registration performance in both geometry and function to the state-of-the-art methods but without requiring functional data at inference. This learning framework can be extended to any auxiliary data to guide spherical registration that is available during training but is difficult or impossible to obtain during inference, such as parcellations, architectonic identity, transcriptomic information, and molecular profiles. By recognizing the mismatch between geometry and function, JOSA provides new insights into the future development of registration methods using joint analysis of brain structure and function.
Collapse
Affiliation(s)
- Jian Li
- A. A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, United States of America; Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, United States of America.
| | - Greta Tuckute
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, United States of America; McGovern Institute for Brain Research, Massachusetts Institute of Technology, United States of America
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, United States of America; McGovern Institute for Brain Research, Massachusetts Institute of Technology, United States of America; Program in Speech Hearing Bioscience and Technology, Harvard University, United States of America
| | - Brian L Edlow
- A. A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, United States of America; Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, United States of America
| | - Adrian V Dalca
- A. A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, United States of America; Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, United States of America
| | - Bruce Fischl
- A. A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, United States of America; Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, United States of America
| |
Collapse
|
2
|
Perry JL, Gilbert IR, Xing F, Jin R, Kuehn DP, Shosted RK, Woo J, Liang ZP, Sutton BP. Preliminary Development of an MRI Atlas for Application to Cleft Care: Findings and Future Recommendations. Cleft Palate Craniofac J 2024; 61:1912-1918. [PMID: 37335134 DOI: 10.1177/10556656231183385] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/21/2023] Open
Abstract
OBJECTIVE To introduce a highly innovative imaging method to study the complex velopharyngeal (VP) system and introduce the potential future clinical applications of a VP atlas in cleft care. DESIGN Four healthy adults participated in a 20-min dynamic magnetic resonance imaging scan that included a high-resolution T2-weighted turbo-spin-echo 3D structural scan and five custom dynamic speech imaging scans. Subjects repeated a variety of phrases when in the scanner as real-time audio was captured. SETTING Multisite institution and clinical setting. PARTICIPANTS Four adult subjects with normal anatomy were recruited for this study. MAIN OUTCOME Establishment of 4-D atlas constructed from dynamic VP MRI data. RESULTS Three-dimensional dynamic magnetic resonance imaging was successfully used to obtain high quality dynamic speech scans in an adult population. Scans were able to be re-sliced in various imaging planes. Subject-specific MR data were then reconstructed and time-aligned to create a velopharyngeal atlas representing the averaged physiological movements across the four subjects. CONCLUSIONS The current preliminary study examined the feasibility of developing a VP atlas for potential clinical applications in cleft care. Our results indicate excellent potential for the development and use of a VP atlas for assessing VP physiology during speech.
Collapse
Affiliation(s)
- Jamie L Perry
- Department of Communication Sciences and Disorders, East Carolina University, Greenville, NC, USA
| | - Imani R Gilbert
- Department of Communication Sciences and Disorders, East Carolina University, Greenville, NC, USA
| | - Fangxu Xing
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital/Harvard Medical School, Boston, MA, USA
| | - Riwei Jin
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - David P Kuehn
- Department of Speech and Hearing Science, University of Illinois Urbana-Champaign, Urbana, IL, USA
| | - Ryan K Shosted
- Department of Linguistics, University of Illinois at Urbana-Champaign, Urbana, IL USA
| | - Jonghye Woo
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital/Harvard Medical School, Boston, MA, USA
| | - Zhi-Pei Liang
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Bradley P Sutton
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| |
Collapse
|
3
|
Yerly J, Roy CW, Milani B, Eyre K, Raifee MJ, Stuber M. High on sparsity: Interbin compensation of cardiac motion for improved assessment of left-ventricular function using 5D whole-heart MRI. Magn Reson Med 2024. [PMID: 39385350 DOI: 10.1002/mrm.30323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2024] [Revised: 08/21/2024] [Accepted: 09/12/2024] [Indexed: 10/12/2024]
Abstract
PURPOSE Cardiac magnetic resonance is the gold standard for evaluating left-ventricular ejection fraction (LVEF). Standard protocols, however, can be inefficient, facing challenges due to significant operator and patient involvement. Although the free-running framework (FRF) addresses these challenges, the potential of the extensive data it collects remains underutilized. Therefore, we propose to leverage the large amount of data collected by incorporating interbin cardiac motion compensation into FRF (FRF-MC) to improve both image quality and LVEF measurement accuracy, while reducing the sensitivity to user-defined regularization parameters. METHODS FRF-MC consists of several steps: data acquisition, self-gating signal extraction, deformation field estimations, and motion-resolved reconstruction with interbin cardiac motion compensation. FRF-MC was compared with the original 5D-FRF method using LVEF and several image-quality metrics. The cardiac regularization weight (λ c $$ {\lambda}_c $$ ) was optimized for both methods by maximizing image quality without compromising LVEF measurement accuracy. Evaluations were performed in numerical simulations and in 9 healthy participants. In vivo images were assessed by blinded expert reviewers and compared with reference standard 2D-cine images. RESULTS Both in silico and in vivo results revealed that FRF-MC outperformed FRF in terms of image quality and LVEF accuracy. FRF-MC reduced temporal blurring, preserving detailed anatomy even at higher cardiac regularization weights, and led to more accurate LVEF measurements. Optimizedλ c $$ {\lambda}_c $$ produced accurate LVEF for both methods compared with the 2D-cine reference (FRF-MC: 0.59% [-7.2%, 6.0%], p = 0.47; FRF: 0.86% [-8.5%, 6.7%], p = 0.36), but FRF-MC resulted in superior image quality (FRF-MC: 2.89 ± 0.58, FRF: 2.11 ± 0.47; p < 10-3). CONCLUSION Incorporating interbin cardiac motion compensation significantly improved image quality, supported higher cardiac regularization weights without compromising LVEF measurement accuracy, and reduced sensitivity to user-defined regularization parameters.
Collapse
Affiliation(s)
- Jérôme Yerly
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Vaud, Switzerland
- Center for Biomedical Imaging (CIBM), Lausanne, Vaud, Switzerland
| | - Christopher W Roy
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Vaud, Switzerland
| | - Bastien Milani
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Vaud, Switzerland
| | - Katerina Eyre
- Research Institute, McGill University Health Center, Montréal, Québec, Canada
| | - Mozedin Javad Raifee
- Research Institute, McGill University Health Center, Montréal, Québec, Canada
- Department of Medicine and Radiology, McGill University Health Centre, Montréal, Québec, Canada
| | - Matthias Stuber
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Vaud, Switzerland
- Center for Biomedical Imaging (CIBM), Lausanne, Vaud, Switzerland
| |
Collapse
|
4
|
Jafari R, Verma R, Aggarwal V, Gupta RK, Singh A. Deep learning-based segmentation of left ventricular myocardium on dynamic contrast-enhanced MRI: a comprehensive evaluation across temporal frames. Int J Comput Assist Radiol Surg 2024; 19:2055-2062. [PMID: 38965165 DOI: 10.1007/s11548-024-03221-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 06/24/2024] [Indexed: 07/06/2024]
Abstract
PURPOSE Cardiac perfusion MRI is vital for disease diagnosis, treatment planning, and risk stratification, with anomalies serving as markers of underlying ischemic pathologies. AI-assisted methods and tools enable accurate and efficient left ventricular (LV) myocardium segmentation on all DCE-MRI timeframes, offering a solution to the challenges posed by the multidimensional nature of the data. This study aims to develop and assess an automated method for LV myocardial segmentation on DCE-MRI data of a local hospital. METHODS The study consists of retrospective DCE-MRI data from 55 subjects acquired at the local hospital using a 1.5 T MRI scanner. The dataset included subjects with and without cardiac abnormalities. The timepoint for the reference frame (post-contrast LV myocardium) was identified using standard deviation across the temporal sequences. Iterative image registration of other temporal images with respect to this reference image was performed using Maxwell's demons algorithm. The registered stack was fed to the model built using the U-Net framework for predicting the LV myocardium at all timeframes of DCE-MRI. RESULTS The mean and standard deviation of the dice similarity coefficient (DSC) for myocardial segmentation using pre-trained network Net_cine is 0.78 ± 0.04, and for the fine-tuned network Net_dyn which predicts mask on all timeframes individually, it is 0.78 ± 0.03. The DSC for Net_dyn ranged from 0.71 to 0.93. The average DSC achieved for the reference frame is 0.82 ± 0.06. CONCLUSION The study proposed a fast and fully automated AI-assisted method to segment LV myocardium on all timeframes of DCE-MRI data. The method is robust, and its performance is independent of the intra-temporal sequence registration and can easily accommodate timeframes with potential registration errors.
Collapse
Affiliation(s)
- Raufiya Jafari
- Centre for Biomedical Engineering, Indian Institute of Technology Delhi, New Delhi, Delhi, 110016, India
| | - Radhakrishan Verma
- Department of Radiology, Fortis Memorial Research Institute, Gurugram, India
| | - Vinayak Aggarwal
- Department of Cardiology, Fortis Memorial Research Institute, Gurugram, India
| | - Rakesh Kumar Gupta
- Department of Radiology, Fortis Memorial Research Institute, Gurugram, India
| | - Anup Singh
- Centre for Biomedical Engineering, Indian Institute of Technology Delhi, New Delhi, Delhi, 110016, India.
- Department of Biomedical Engineering, All India Institute of Medical Sciences, New Delhi, Delhi, India.
- Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi, New Delhi, Delhi, India.
| |
Collapse
|
5
|
Soni ND, Swain A, Juul H, Cao Q, Haris M, Wolk DA, Lee VM, Detre JA, Nanga RPR, Reddy R. Detection of sex-specific glutamate changes in subregions of hippocampus in an early-stage Alzheimer's disease mouse model using GluCEST MRI. Alzheimers Dement 2024; 20:7124-7137. [PMID: 39262197 PMCID: PMC11485308 DOI: 10.1002/alz.14190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2024] [Revised: 07/05/2024] [Accepted: 07/24/2024] [Indexed: 09/13/2024]
Abstract
INTRODUCTION Regional glucose hypometabolism resulting in glutamate loss has been shown as one of the characteristics of Alzheimer's disease (AD). Because the impact of AD varies between the sexes, we utilized glutamate-weighted chemical exchange saturation transfer (GluCEST) magnetic resonance imaging (MRI) for high-resolution spatial mapping of cerebral glutamate and investigated subregional changes in a sex-specific manner. METHODS Eight-month-old male and female AD mice harboring mutant amyloid precursor protein (APPNL-F/NL-F: n = 36) and wild-type (WT: n = 39) mice underwent GluCEST MRI, followed by proton magnetic resonance spectroscopy (1H-MRS) in hippocampus and thalamus/hypothalamus using 9.4T preclinical MR scanner. RESULTS GluCEST measurements revealed significant (p ≤ 0.02) glutamate loss in the entorhinal cortex (% change ± standard error: 8.73 ± 2.12%), hippocampus (11.29 ± 2.41%), and hippocampal fimbriae (19.15 ± 2.95%) of male AD mice. A similar loss of hippocampal glutamate in male AD mice (11.22 ± 2.33%; p = 0.01) was also observed in 1H-MRS. DISCUSSIONS GluCEST MRI detected glutamate reductions in the fimbria and entorhinal cortex of male AD mice, which was not reported previously. Resilience in female AD mice against these changes indicates an intact status of cerebral energy metabolism. HIGHLIGHTS Glutamate levels were monitored in different brain regions of early-stage Alzheimer's disease (AD) and wild-type male and female mice using glutamate-weighted chemical exchange saturation transfer (GluCEST) magnetic resonance imaging (MRI). Male AD mice exhibited significant glutamate loss in the hippocampus, entorhinal cortex, and the fimbriae of the hippocampus. Interestingly, female AD mice did not have any glutamate loss in any brain region and should be investigated further to find the probable cause. These findings demonstrate previously unreported sex-specific glutamate changes in hippocampal sub-regions using high-resolution GluCEST MRI.
Collapse
Affiliation(s)
- Narayan Datt Soni
- Department of Radiology, Perelman School of MedicineCenter for Advanced Metabolic Imaging in Precision MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Anshuman Swain
- Department of Radiology, Perelman School of MedicineCenter for Advanced Metabolic Imaging in Precision MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of Bioengineering, School of Engineering and Applied SciencesUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Halvor Juul
- Department of Radiology, Perelman School of MedicineCenter for Advanced Metabolic Imaging in Precision MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Quy Cao
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Mohammad Haris
- Department of Radiology, Perelman School of MedicineCenter for Advanced Metabolic Imaging in Precision MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - David A. Wolk
- Center for Cognitive NeuroscienceUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Alzheimer's Disease Research CenterPerelman School of MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Virginia M.‐Y. Lee
- Alzheimer's Disease Research CenterPerelman School of MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Center for Neurodegenerative Disease ResearchPerelman School of MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - John A. Detre
- Department of Neurology, Perelman School of MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Ravi Prakash Reddy Nanga
- Department of Radiology, Perelman School of MedicineCenter for Advanced Metabolic Imaging in Precision MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Ravinder Reddy
- Department of Radiology, Perelman School of MedicineCenter for Advanced Metabolic Imaging in Precision MedicineUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| |
Collapse
|
6
|
Friedman SF, Moran GE, Rakic M, Phillipakis A. Genetic Architectures of Medical Images Revealed by Registration of Multiple Modalities. Bioinform Biol Insights 2024; 18:11779322241282489. [PMID: 39372505 PMCID: PMC11450573 DOI: 10.1177/11779322241282489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Accepted: 08/16/2024] [Indexed: 10/08/2024] Open
Abstract
The advent of biobanks with vast quantities of medical imaging and paired genetic measurements creates huge opportunities for a new generation of genotype-phenotype association studies. However, disentangling biological signals from the many sources of bias and artifacts remains difficult. Using diverse medical images and time-series (ie, magnetic resonance imagings [MRIs], electrocardiograms [ECGs], and dual-energy X-ray absorptiometries [DXAs]), we show how registration, both spatial and temporal, guided by domain knowledge or learned de novo, helps uncover biological information. A multimodal autoencoder comparison framework quantifies and characterizes how registration affects the representations that unsupervised and self-supervised encoders learn. In this study we (1) train autoencoders before and after registration with nine diverse types of medical image, (2) demonstrate how neural network-based methods (VoxelMorph, DeepCycle, and DropFuse) can effectively learn registrations allowing for more flexible and efficient processing than is possible with hand-crafted registration techniques, and (3) conduct exhaustive phenotypic screening, comprised of millions of statistical tests, to quantify how registration affects the generalizability of learned representations. Genome- and phenome-wide association studies (GWAS and PheWAS) uncover significantly more associations with registered modality representations than with equivalently trained and sized representations learned from native coordinate spaces. Specifically, registered PheWAS yielded 61 more disease associations for ECGs, 53 more disease associations for cardiac MRIs, and 10 more disease associations for brain MRIs. Registration also yields significant increases in the coefficient of determination when regressing continuous phenotypes (eg, 0.36 ± 0.01 with ECGs and 0.11 ± 0.02 for DXA scans). Our findings reveal the crucial role registration plays in enhancing the characterization of physiological states across a broad range of medical imaging data types. Importantly, this finding extends to more flexible types of registration, such as the cross-modal and the circular mapping methods presented here.
Collapse
Affiliation(s)
| | | | - Marianne Rakic
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - Anthony Phillipakis
- Data Sciences Platform, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| |
Collapse
|
7
|
Bai X, Wang H, Qin Y, Han J, Yu N. SparseMorph: A weakly-supervised lightweight sparse transformer for mono- and multi-modal deformable image registration. Comput Biol Med 2024; 182:109205. [PMID: 39332116 DOI: 10.1016/j.compbiomed.2024.109205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 05/14/2024] [Accepted: 09/22/2024] [Indexed: 09/29/2024]
Abstract
PURPOSE Deformable image registration (DIR) is crucial for improving the precision of clinical diagnosis. Recent Transformer-based DIR methods have shown promising performance by capturing long-range dependencies. Nevertheless, these methods still grapple with high computational complexity. This work aims to enhance the performance of DIR in both computational efficiency and registration accuracy. METHODS We proposed a weakly-supervised lightweight Transformer model, named SparseMorph. To reduce computational complexity without compromising the representative feature capture ability, we designed a sparse multi-head self-attention (SMHA) mechanism. To accumulate representative features while preserving high computational efficiency, we constructed a multi-branch multi-layer perception (MMLP) module. Additionally, we developed an anatomically-constrained weakly-supervised strategy to guide the alignment of regions-of-interest in mono- and multi-modal images. RESULTS We assessed SparseMorph in terms of registration accuracy and computational complexity. Within the mono-modal brain datasets IXI and OASIS, our SparseMorph outperforms the state-of-the-art method TransMatch with improvements of 3.2 % and 2.9 % in DSC scores for MRI-to-CT registration tasks, respectively. Moreover, in the multi-modal cardiac dataset MMWHS, our SparseMorph shows DSC score improvements of 9.7 % and 11.4 % compared to TransMatch in MRI-to-CT and CT-to-MRI registration tasks, respectively. Notably, SparseMorph attains these performance advantages while utilizing 33.33 % of the parameters of TransMatch. CONCLUSIONS The proposed weakly-supervised deformable image registration model, SparseMorph, demonstrates efficiency in both mono- and multi-modal registration tasks, exhibiting superior performance compared to state-of-the-art algorithms, and establishing an effective DIR method for clinical applications.
Collapse
Affiliation(s)
- Xinhao Bai
- College of Artificial Intelligence, Nankai University, Tianjin, 300350, China; Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, Tianjin, 300350, China; Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, 518083, China
| | - Hongpeng Wang
- College of Artificial Intelligence, Nankai University, Tianjin, 300350, China; Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, Tianjin, 300350, China; Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, 518083, China
| | - Yanding Qin
- College of Artificial Intelligence, Nankai University, Tianjin, 300350, China; Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, Tianjin, 300350, China; Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, 518083, China
| | - Jianda Han
- College of Artificial Intelligence, Nankai University, Tianjin, 300350, China; Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, Tianjin, 300350, China; Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, 518083, China
| | - Ningbo Yu
- College of Artificial Intelligence, Nankai University, Tianjin, 300350, China; Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, Tianjin, 300350, China; Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, 518083, China.
| |
Collapse
|
8
|
Deng L, Lan Q, Yang X, Wang J, Huang S. DELR-Net: a network for 3D multimodal medical image registration in more lightweight application scenarios. Abdom Radiol (NY) 2024:10.1007/s00261-024-04602-3. [PMID: 39400589 DOI: 10.1007/s00261-024-04602-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2024] [Revised: 09/14/2024] [Accepted: 09/17/2024] [Indexed: 10/15/2024]
Abstract
PURPOSE 3D multimodal medical image deformable registration plays a significant role in medical image analysis and diagnosis. However, due to the substantial differences between images of different modalities, registration is challenging and requires high computational costs. Deep learning-based registration methods face these challenges. The primary aim of this paper is to design a 3D multimodal registration network that ensures high-quality registration results while reducing the number of parameters. METHODS This study designed a Dual-Encoder More Lightweight Registration Network (DELR-Net). DELR-Net is a low-complexity network that integrates Mamba and ConvNet. The State Space Sequence Module and the Dynamic Large Kernel block are used as the main components of the dual encoders, while the Dynamic Feature Fusion block is used as the main component of the decoder. RESULTS This study conducted experiments on 3D brain MR images and abdominal MR and CT images. Compared to existing registration methods, DELR-Net achieved better registration results while maintaining a lower number of parameters. Additionally, generalization experiments on other modalities showed that DELR-Net has superior generalization capabilities. CONCLUSION DELR-Net significantly improves the limitations of 3D multimodal medical image deformable registration, achieving better registration performance with fewer parameters.
Collapse
Affiliation(s)
- Liwei Deng
- Harbin University of Science and Technology, Harbin, China
| | - Qi Lan
- Harbin University of Science and Technology, Harbin, China
| | - Xin Yang
- Sun Yat-sen University Cancer Center, Guangzhou, China.
| | - Jing Wang
- South China Normal University, Guangzhou, China
| | - Sijuan Huang
- Sun Yat-sen University Cancer Center, Guangzhou, China.
| |
Collapse
|
9
|
Wang X, Alkaabi F, Cornett A, Choi M, Scheven UM, Di Natale MR, Furness JB, Liu Z. Magnetic Resonance Imaging of Gastric Motility in Conscious Rats. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.09.09.612090. [PMID: 39314428 PMCID: PMC11419018 DOI: 10.1101/2024.09.09.612090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/25/2024]
Abstract
Introduction Gastrointestinal (GI) magnetic resonance imaging (MRI) can simultaneously capture gastric peristalsis, emptying, and intestinal filling and transit. Performing GI MRI with animals requires anesthesia, which complicates physiology and confounds interpretation and translation from animals to humans. This study aims to enable MRI in conscious rats, and for the first time, characterize GI motor functions in awake versus anesthetized conditions. Methods We acclimated rats to remain awake, still, and minimally stressed during MRI. We scanned 14 Sprague-Dawley rats in both awake and anesthetized conditions after voluntarily consuming a contrast-enhanced test meal. Results Awake rats remained physiologically stable during MRI, showed gastric emptying of 23.7±1.4% after 48 minutes, and exhibited strong peristaltic contractions propagating through the antrum with a velocity of 0.72±0.04 mm/s, a relative amplitude of 40.7±2.3%, and a frequency of 5.1±0.1 cycles per minute. In the anesthetized condition, gastric emptying was about half of that in the awake condition, likely due to the effect of anesthesia in halving the amplitudes of peristaltic contractions rather than their frequency (not significantly changed) or velocity. In awake rats, the intestine filled more quickly and propulsive contractions were more occlusive. Conclusion We demonstrated the effective acquisition and analysis of GI MRI in awake rats. Awake rats show faster gastric emptying, stronger gastric contraction with a faster propagation speed, and more effective intestinal filling and transit, compared to anesthetized rats. Our protocol is expected to benefit future preclinical studies of GI physiology and pathophysiology.
Collapse
|
10
|
Wang Y, Chang W, Huang C, Kong D. Multiscale unsupervised network for deformable image registration. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024:XST240159. [PMID: 39240617 DOI: 10.3233/xst-240159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/07/2024]
Abstract
BACKGROUND Deformable image registration (DIR) plays an important part in many clinical tasks, and deep learning has made significant progress in DIR over the past few years. OBJECTIVE To propose a fast multiscale unsupervised deformable image registration (referred to as FMIRNet) method for monomodal image registration. METHODS We designed a multiscale fusion module to estimate the large displacement field by combining and refining the deformation fields of three scales. The spatial attention mechanism was employed in our fusion module to weight the displacement field pixel by pixel. Except mean square error (MSE), we additionally added structural similarity (ssim) measure during the training phase to enhance the structural consistency between the deformed images and the fixed images. RESULTS Our registration method was evaluated on EchoNet, CHAOS and SLIVER, and had indeed performance improvement in terms of SSIM, NCC and NMI scores. Furthermore, we integrated the FMIRNet into the segmentation network (FCN, UNet) to boost the segmentation task on a dataset with few manual annotations in our joint leaning frameworks. The experimental results indicated that the joint segmentation methods had performance improvement in terms of Dice, HD and ASSD scores. CONCLUSIONS Our proposed FMIRNet is effective for large deformation estimation, and its registration capability is generalizable and robust in joint registration and segmentation frameworks to generate reliable labels for training segmentation tasks.
Collapse
Affiliation(s)
- Yun Wang
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Wanru Chang
- College of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou, China
| | | | - Dexing Kong
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
- Zhejiang Qiushi Institute for Mathematical Medicine, Hangzhou, China
| |
Collapse
|
11
|
Voskrebenzev A, Gutberlet M, Klimeš F, Kaireit TF, Shin HO, Kauczor HU, Welte T, Wacker F, Vogel-Claussen J. A synthetic lung model (ASYLUM) for validation of functional lung imaging methods shows significant differences between signal-based and deformation-field-based ventilation measurements. Front Med (Lausanne) 2024; 11:1418052. [PMID: 39296894 PMCID: PMC11409849 DOI: 10.3389/fmed.2024.1418052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 07/30/2024] [Indexed: 09/21/2024] Open
Abstract
Introduction Validation of functional free-breathing MRI involves a comparison to more established or more direct measurements. This procedure is cost-intensive, as it requires access to patient cohorts, lengthy protocols, expenses for consumables, and binds working time. Therefore, the purpose of this study is to introduce a synthetic lung model (ASYLUM), which mimics dynamic MRI acquisition and includes predefined lung abnormalities for an alternative validation approach. The model is evaluated with different registration and quantification methods and compared with real data. Methods A combination of trigonometric functions, deformation fields, and signal combinations were used to create 20 synthetic image time series. Lung voxels were assigned either to normal or one of six abnormality classes. The images were registered with three registration algorithms. The registered images were further analyzed with three quantification methods: deformation-based or signal-based regional ventilation (JVent/RVent) analysis and perfusion amplitude (QA). The registration results were compared with predefined deformations. Quantification methods were evaluated regarding predefined amplitudes and with respect to sensitivity, specificity, and spatial overlap of defects. In addition, 36 patients with chronic obstructive pulmonary disease were included for verification of model interpretations using CT as the gold standard. Results One registration method showed considerably lower quality results (76% correlation vs. 92/97%, p ≤ 0.0001). Most ventilation defects were correctly detected with RVent and QA (e.g., one registration variant with sensitivity ≥78%, specificity ≥88). Contrary to this, JVent showed very low sensitivity for lower lung quadrants (0-16%) and also very low specificity (1-29%) for upper lung quadrants. Similar patterns of defect detection differences between RVent and JVent were also observable in patient data: Firstly, RVent was more aligned with CT than JVent for all quadrants (p ≤ 0.01) except for one registration variant in the lower left region. Secondly, stronger differences in overlap were observed for the upper quadrants, suggesting a defect bias in the JVent measurements in the upper lung regions. Conclusion The feasibility of a validation framework for free-breathing functional lung imaging using synthetic time series was demonstrated. Evaluating different ventilation measurements, important differences were detected in synthetic and real data, with signal-based regional ventilation assessment being a more reliable method in the investigated setting.
Collapse
Affiliation(s)
- Andreas Voskrebenzev
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| | - Marcel Gutberlet
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| | - Filip Klimeš
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| | - Till F Kaireit
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| | - Hoen-Oh Shin
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| | - Hans-Ulrich Kauczor
- Department of Diagnostic and Interventional Radiology, University Hospital of Heidelberg, Heidelberg, Germany
- Translational Lung Research Center Heidelberg (TLRC), Member of the German Lung Research Center (DZL), Heidelberg, Germany
| | - Tobias Welte
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
- Clinic of Pneumology, Hannover Medical School, Hannover, Germany
| | - Frank Wacker
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| | - Jens Vogel-Claussen
- Institute for Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
- Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research, Hannover, Germany
| |
Collapse
|
12
|
Zhou Z, Yin P, Liu Y, Hu J, Qian X, Chen G, Hu C, Dai Y. Uncertain prediction of deformable image registration on lung CT using multi-category features and supervised learning. Med Biol Eng Comput 2024; 62:2669-2686. [PMID: 38658497 DOI: 10.1007/s11517-024-03092-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 04/08/2024] [Indexed: 04/26/2024]
Abstract
The assessment of deformable registration uncertainty is an important task for the safety and reliability of registration methods in clinical applications. However, it is typically done by a manual and time-consuming procedure. We propose a novel automatic method to predict registration uncertainty based on multi-category features and supervised learning. Three types of features, including deformation field statistical features, deformation field physiologically realistic features, and image similarity features, are introduced and calculated to train the random forest regressor for local registration uncertain prediction. Deformation field statistical features represent the numerical stability of registration optimization, which are correlated to the uncertainty of deformation fields; deformation field physiologically realistic features represent the biomechanical properties of organ motions, which mathematically reflect the physiological reality of deformation; image similarity features reflect the similarity between the warped image and fixed image. The multi-category features comprehensively reflect the registration uncertainty. The strategy of spatial adaptive random perturbations is also introduced to accurately simulate spatial distribution of registration uncertainty, which makes deformation field statistical features more discriminative to the uncertainty of deformation fields. Experiments were conducted on three publicly available thoracic CT image datasets. Seventeen randomly selected image pairs are used to train the random forest model, and 9 image pairs are used to evaluate the prediction model. The quantitative experiments on lung CT images show that the proposed method outperforms the baseline method for uncertain prediction of classical iterative optimization-based registration and deep learning-based registration with different registration qualities. The proposed method achieves good performance for registration uncertain prediction, which has great potential in improving the accuracy of registration uncertain prediction.
Collapse
Affiliation(s)
- Zhiyong Zhou
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Pengfei Yin
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Yuhang Liu
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China
| | - Jisu Hu
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Xusheng Qian
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Guangqiang Chen
- The Second Affiliated Hospital of Soochow University, Suzhou, 215163, China
| | - Chunhong Hu
- The First Affiliated Hospital of Soochow University, Suzhou, 215163, China.
| | - Yakang Dai
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, 215163, China.
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China.
| |
Collapse
|
13
|
Kim KM, Suh M, Selvam HSMS, Tan TH, Cheon GJ, Kang KW, Lee JS. Enhancing voxel-based dosimetry accuracy with an unsupervised deep learning approach for hybrid medical image registration. Med Phys 2024; 51:6432-6444. [PMID: 38772037 DOI: 10.1002/mp.17129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 03/27/2024] [Accepted: 05/04/2024] [Indexed: 05/23/2024] Open
Abstract
BACKGROUND Deformable registration is required to generate a time-integrated activity (TIA) map which is essential for voxel-based dosimetry. The conventional iterative registration algorithm using anatomical images (e.g., computed tomography (CT)) could result in registration errors in functional images (e.g., single photon emission computed tomography (SPECT) or positron emission tomography (PET)). Various deep learning-based registration tools have been proposed, but studies specifically focused on the registration of serial hybrid images were not found. PURPOSE In this study, we introduce CoRX-NET, a novel unsupervised deep learning network designed for deformable registration of hybrid medical images. The CoRX-NET structure is based on the Swin-transformer (ST), allowing for the representation of complex spatial connections in images. Its self-attention mechanism aids in the effective exchange and integration of information across diverse image regions. To augment the amalgamation of SPECT and CT features, cross-stitch layers have been integrated into the network. METHODS Two different 177 Lu DOTATATE SPECT/CT datasets were acquired at different medical centers. 22 sets from Seoul National University and 14 sets from Sunway Medical Centre are used for training/internal validation and external validation respectively. The CoRX-NET architecture builds upon the ST, enabling the modeling of intricate spatial relationships within images. To further enhance the fusion of SPECT and CT features, cross-stitch layers have been incorporated within the network. The network takes a pair of SPECT/CT images (e.g., fixed and moving images) and generates a deformed SPECT/CT image. The performance of the network was compared with Elastix and TransMorph using L1 loss and structural similarity index measure (SSIM) of CT, SSIM of normalized SPECT, and local normalized cross correlation (LNCC) of SPECT as metrics. The voxel-wise root mean square errors (RMSE) of TIA were compared among the different methods. RESULTS The ablation study revealed that cross-stitch layers improved SPECT/CT registration performance. The cross-stitch layers notably enhance SSIM (internal validation: 0.9614 vs. 0.9653, external validation: 0.9159 vs. 0.9189) and LNCC of normalized SPECT images (internal validation: 0.7512 vs. 0.7670, external validation: 0.8027 vs. 0.8027). CoRX-NET with the cross-stitch layer achieved superior performance metrics compared to Elastix and TransMorph, except for CT SSIM in the external dataset. When qualitatively analyzed for both internal and external validation cases, CoRX-NET consistently demonstrated superior SPECT registration results. In addition, CoRX-NET accomplished SPECT/CT image registration in less than 6 s, whereas Elastix required approximately 50 s using the same PC's CPU. When employing CoRX-NET, it was observed that the voxel-wise RMSE values for TIA were approximately 27% lower for the kidney and 33% lower for the tumor, compared to when Elastix was used. CONCLUSION This study represents a major advancement in achieving precise SPECT/CT registration using an unsupervised deep learning network. It outperforms conventional methods like Elastix and TransMorph, reducing uncertainties in TIA maps for more accurate dose assessments.
Collapse
Affiliation(s)
- Keon Min Kim
- Interdisciplinary Program in Bioengineering, Seoul National University Graduate School, Seoul, Republic of Korea
- Integrated Major in Innovative Medical Science, Seoul National University Graduate School, Seoul, Republic of Korea
| | - Minseok Suh
- Department of Nuclear Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Institute of Radiation Medicine, Medical Research Center, Seoul National University, Seoul, Republic of Korea
| | | | - Teik Hin Tan
- Nuclear Medicine Centre, Sunway Medical Centre, Subang Jaya, Selangor, Malaysia
| | - Gi Jeong Cheon
- Department of Nuclear Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Department of Molecular Medicine and Biopharmaceutical Sciences, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, Republic of Korea
- Cancer Research Institute & Institute on Aging, Seoul National University, Seoul, Republic of Korea
| | - Keon Wook Kang
- Department of Nuclear Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Institute of Radiation Medicine, Medical Research Center, Seoul National University, Seoul, Republic of Korea
- Bio-MAX Institute, Seoul National University, Seoul, Republic of Korea
| | - Jae Sung Lee
- Interdisciplinary Program in Bioengineering, Seoul National University Graduate School, Seoul, Republic of Korea
- Integrated Major in Innovative Medical Science, Seoul National University Graduate School, Seoul, Republic of Korea
- Department of Nuclear Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Institute of Radiation Medicine, Medical Research Center, Seoul National University, Seoul, Republic of Korea
| |
Collapse
|
14
|
Zhu Z, Li Q, Wei Y, Song R. Hierarchical multi-level dynamic hyperparameter deformable image registration with convolutional neural network. Phys Med Biol 2024; 69:175007. [PMID: 39053510 DOI: 10.1088/1361-6560/ad67a6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 07/25/2024] [Indexed: 07/27/2024]
Abstract
Objective. To enable the registration network to be trained only once, achieving fast regularization hyperparameter selection during the inference phase, and to improve registration accuracy and deformation field regularity.Approach. Hyperparameter tuning is an essential process for deep learning deformable image registration (DLDIR). Most DLDIR methods usually perform a large number of independent experiments to select the appropriate regularization hyperparameters, which are time-consuming and resource-consuming. To address this issue, we propose a novel dynamic hyperparameter block, which comprises a distributed mapping network, dynamic convolution, attention feature extraction layer, and instance normalization layer. The dynamic hyperparameter block encodes the input feature vectors and regularization hyperparameters into learnable feature variables and dynamic convolution parameters which changes the feature statistics of the high-dimensional features layer feature variables, respectively. In addition, the proposed method replaced the single-level structure residual blocks in LapIRN with a hierarchical multi-level architecture for the dynamic hyperparameter block in order to improve registration performance.Main results. On the OASIS dataset, the proposed method reduced the percentage of|Jϕ|⩽0by 28.01%, 9.78%and improved Dice similarity coefficient by 1.17%, 1.17%, compared with LapIRN and CIR, respectively. On the DIR-Lab dataset, the proposed method reduced the percentage of|Jϕ|⩽0by 10.00%, 5.70%and reduced target registration error by 10.84%, 10.05%, compared with LapIRN and CIR, respectively.Significance. The proposed method can fast achieve the corresponding registration deformation field for arbitrary hyperparameter value during the inference phase. Extensive experiments demonstrate that the proposed method reduces training time compared to DLDIR with fixed regularization hyperparameters while outperforming the state-of-the-art registration methods concerning registration accuracy and deformation smoothness on brain dataset OASIS and lung dataset DIR-Lab.
Collapse
Affiliation(s)
- Zhenyu Zhu
- School of Control Science and Engineering, Shandong University, Jinan, People's Republic of China
| | - Qianqian Li
- School and Hospital of Stomatology, Cheeloo College of Medicine, Shandong University, Jinan, People's Republic of China
| | - Ying Wei
- School of Control Science and Engineering, Shandong University, Jinan, People's Republic of China
- Shandong Research Institute of Industrial Technology, Jinan, People's Republic of China
| | - Rui Song
- School of Control Science and Engineering, Shandong University, Jinan, People's Republic of China
- Shandong Research Institute of Industrial Technology, Jinan, People's Republic of China
| |
Collapse
|
15
|
Yuan W, Cheng J, Gong Y, He L, Zhang J. MACG-Net: Multi-axis cross gating network for deformable medical image registration. Comput Biol Med 2024; 178:108673. [PMID: 38905891 DOI: 10.1016/j.compbiomed.2024.108673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Revised: 04/18/2024] [Accepted: 05/26/2024] [Indexed: 06/23/2024]
Abstract
Deformable Image registration is a fundamental yet vital task for preoperative planning, intraoperative information fusion, disease diagnosis and follow-ups. It solves the non-rigid deformation field to align an image pair. Latest approaches such as VoxelMorph and TransMorph compute features from a simple concatenation of moving and fixed images. However, this often leads to weak alignment. Moreover, the convolutional neural network (CNN) or the hybrid CNN-Transformer based backbones are constrained to have limited sizes of receptive field and cannot capture long range relations while full Transformer based approaches are computational expensive. In this paper, we propose a novel multi-axis cross grating network (MACG-Net) for deformable medical image registration, which combats these limitations. MACG-Net uses a dual stream multi-axis feature fusion module to capture both long-range and local context relationships from the moving and fixed images. Cross gate blocks are integrated with the dual stream backbone to consider both independent feature extractions in the moving-fixed image pair and the relationship between features from the image pair. We benchmark our method on several different datasets including 3D atlas-based brain MRI, inter-patient brain MRI and 2D cardiac MRI. The results demonstrate that the proposed method has achieved state-of-the-art performance. The source code has been released at https://github.com/Valeyards/MACG.
Collapse
Affiliation(s)
- Wei Yuan
- College of Biomedical Engineering, Sichuan University, Chengdu 610065, China
| | - Jun Cheng
- Institute for Infocomm Research, Agency for Science, Technology and Research, 138632, Singapore
| | - Yuhang Gong
- College of Biomedical Engineering, Sichuan University, Chengdu 610065, China
| | - Ling He
- College of Biomedical Engineering, Sichuan University, Chengdu 610065, China.
| | - Jing Zhang
- College of Biomedical Engineering, Sichuan University, Chengdu 610065, China
| |
Collapse
|
16
|
Hernandez M, Ramon Julvez U. Insights into traditional Large Deformation Diffeomorphic Metric Mapping and unsupervised deep-learning for diffeomorphic registration and their evaluation. Comput Biol Med 2024; 178:108761. [PMID: 38908357 DOI: 10.1016/j.compbiomed.2024.108761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Revised: 06/04/2024] [Accepted: 06/13/2024] [Indexed: 06/24/2024]
Abstract
This paper explores the connections between traditional Large Deformation Diffeomorphic Metric Mapping methods and unsupervised deep-learning approaches for non-rigid registration, particularly emphasizing diffeomorphic registration. The study provides useful insights and establishes connections between the methods, thereby facilitating a profound understanding of the methodological landscape. The methods considered in our study are extensively evaluated in T1w MRI images using traditional NIREP and Learn2Reg OASIS evaluation protocols with a focus on fairness, to establish equitable benchmarks and facilitate informed comparisons. Through a comprehensive analysis of the results, we address key questions, including the intricate relationship between accuracy and transformation quality in performance, the disentanglement of the influence of registration ingredients on performance, and the determination of benchmark methods and baselines. We offer valuable insights into the strengths and limitations of both traditional and deep-learning methods, shedding light on their comparative performance and guiding future advancements in the field.
Collapse
Affiliation(s)
- Monica Hernandez
- Computer Science Department, University of Zaragoza, Spain; Aragon Institute on Engineering Research, Spain.
| | - Ubaldo Ramon Julvez
- Computer Science Department, University of Zaragoza, Spain; Aragon Institute on Engineering Research, Spain
| |
Collapse
|
17
|
Pan J, Huang W, Rueckert D, Kustner T, Hammernik K. Motion-Compensated MR CINE Reconstruction With Reconstruction-Driven Motion Estimation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2420-2433. [PMID: 38354077 DOI: 10.1109/tmi.2024.3364504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/16/2024]
Abstract
In cardiac CINE, motion-compensated MR reconstruction (MCMR) is an effective approach to address highly undersampled acquisitions by incorporating motion information between frames. In this work, we propose a novel perspective for addressing the MCMR problem and a more integrated and efficient solution to the MCMR field. Contrary to state-of-the-art (SOTA) MCMR methods which break the original problem into two sub-optimization problems, i.e. motion estimation and reconstruction, we formulate this problem as a single entity with one single optimization. Our approach is unique in that the motion estimation is directly driven by the ultimate goal, reconstruction, but not by the canonical motion-warping loss (similarity measurement between motion-warped images and target images). We align the objectives of motion estimation and reconstruction, eliminating the drawbacks of artifacts-affected motion estimation and therefore error-propagated reconstruction. Further, we can deliver high-quality reconstruction and realistic motion without applying any regularization/smoothness loss terms, circumventing the non-trivial weighting factor tuning. We evaluate our method on two datasets: 1) an in-house acquired 2D CINE dataset for the retrospective study and 2) the public OCMR cardiac dataset for the prospective study. The conducted experiments indicate that the proposed MCMR framework can deliver artifact-free motion estimation and high-quality MR images even for imaging accelerations up to 20x, outperforming SOTA non-MCMR and MCMR methods in both qualitative and quantitative evaluation across all experiments. The code is available at https://github.com/JZPeterPan/MCMR-Recon-Driven-Motion.
Collapse
|
18
|
Mehdi RR, Kadivar N, Mukherjee T, Mendiola EA, Shah DJ, Karniadakis G, Avazmohammadi R. Multi-Modality Deep Infarct: Non-invasive identification of infarcted myocardium using composite in-silico-human data learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.31.596513. [PMID: 38895325 PMCID: PMC11185550 DOI: 10.1101/2024.05.31.596513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
Myocardial infarction (MI) continues to be a leading cause of death worldwide. The precise quantification of infarcted tissue is crucial to diagnosis, therapeutic management, and post-MI care. Late gadolinium enhancement-cardiac magnetic resonance (LGE-CMR) is regarded as the gold standard for precise infarct tissue localization in MI patients. A fundamental limitation of LGE-CMR is the invasive intravenous introduction of gadolinium-based contrast agents that present potential high-risk toxicity, particularly for individuals with underlying chronic kidney diseases. Herein, we develop a completely non-invasive methodology that identifies the location and extent of an infarct region in the left ventricle via a machine learning (ML) model using only cardiac strains as inputs. In this transformative approach, we demonstrate the remarkable performance of a multi-fidelity ML model that combines rodent-based in-silico-generated training data (low-fidelity) with very limited patient-specific human data (high-fidelity) in predicting LGE ground truth. Our results offer a new paradigm for developing feasible prognostic tools by augmenting synthetic simulation-based data with very small amounts of in-vivo human data. More broadly, the proposed approach can significantly assist with addressing biomedical challenges in healthcare where human data are limited.
Collapse
Affiliation(s)
- Rana Raza Mehdi
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Nikhil Kadivar
- School of Engineering, Brown University, Providence, RI 02912, USA
| | - Tanmay Mukherjee
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Emilio A. Mendiola
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Dipan J. Shah
- Houston Methodist DeBakey Heart & Vascular Center, Houston, TX 77030, USA
| | - George Karniadakis
- School of Engineering, Brown University, Providence, RI 02912, USA
- Division of Applied Mathematics, Brown University, Providence, RI 02912, USA
| | - Reza Avazmohammadi
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
- School of Engineering Medicine, Texas A&M University, Houston, TX 77030, USA
- J. Mike Walker ‘66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA
- Department of Cardiovascular Sciences, Houston Methodist Research Institute, Houston, TX 77030, USA
| |
Collapse
|
19
|
Mukherjee T, Keshavarzian M, Fugate EM, Naeini V, Darwish A, Ohayon J, Myers KJ, Shah DJ, Lindquist D, Sadayappan S, Pettigrew RI, Avazmohammadi R. Complete spatiotemporal quantification of cardiac motion in mice through enhanced acquisition and super-resolution reconstruction. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.31.596322. [PMID: 38895261 PMCID: PMC11185553 DOI: 10.1101/2024.05.31.596322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
The quantification of cardiac motion using cardiac magnetic resonance imaging (CMR) has shown promise as an early-stage marker for cardiovascular diseases. Despite the growing popularity of CMR-based myocardial strain calculations, measures of complete spatiotemporal strains (i.e., three-dimensional strains over the cardiac cycle) remain elusive. Complete spatiotemporal strain calculations are primarily hampered by poor spatial resolution, with the rapid motion of the cardiac wall also challenging the reproducibility of such strains. We hypothesize that a super-resolution reconstruction (SRR) framework that leverages combined image acquisitions at multiple orientations will enhance the reproducibility of complete spatiotemporal strain estimation. Two sets of CMR acquisitions were obtained for five wild-type mice, combining short-axis scans with radial and orthogonal long-axis scans. Super-resolution reconstruction, integrated with tissue classification, was performed to generate full four-dimensional (4D) images. The resulting enhanced and full 4D images enabled complete quantification of the motion in terms of 4D myocardial strains. Additionally, the effects of SRR in improving accurate strain measurements were evaluated using an in-silico heart phantom. The SRR framework revealed near isotropic spatial resolution, high structural similarity, and minimal loss of contrast, which led to overall improvements in strain accuracy. In essence, a comprehensive methodology was generated to quantify complete and reproducible myocardial deformation, aiding in the much-needed standardization of complete spatiotemporal strain calculations.
Collapse
Affiliation(s)
- Tanmay Mukherjee
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Maziyar Keshavarzian
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Elizabeth M. Fugate
- Department of Radiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229, USA
| | - Vahid Naeini
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Amr Darwish
- Houston Methodist DeBakey Heart & Vascular Center, Houston, TX 77030, USA
| | - Jacques Ohayon
- Savoie Mont-Blanc University, Polytech Annecy-Chambéry, Le Bourget du Lac, France
- Laboratory TIMC-CNRS, UMR 5525, Grenoble-Alpes University, Grenoble, France
| | - Kyle J. Myers
- Hagler Institute for Advanced Study, Texas A&M University, College Station, TX 77843, USA
| | - Dipan J. Shah
- Houston Methodist DeBakey Heart & Vascular Center, Houston, TX 77030, USA
| | - Diana Lindquist
- Department of Radiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229, USA
| | - Sakthivel Sadayappan
- Department of Internal Medicine, Division of Cardiovascular Health and Disease, University of Cincinnati College of Medicine, Cincinnati, OH 45267, USA
| | - Roderic I. Pettigrew
- School of Engineering Medicine, Texas AM University, Houston, TX 77030, USA
- Department of Cardiovascular Sciences, Houston Methodist Academic Institute, Houston, TX 77030, USA
| | - Reza Avazmohammadi
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
- Department of Cardiovascular Sciences, Houston Methodist Academic Institute, Houston, TX 77030, USA
- J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA
| |
Collapse
|
20
|
Lisi C, Moser LJ, Mergen V, Flohr T, Eberhard M, Alkadhi H. Increasing the rate of datasets amenable to CT FFR and quantitative plaque analysis: Value of software for reducing stair-step artifacts demonstrated in photon-counting detector CT. Eur J Radiol Open 2024; 12:100574. [PMID: 38882632 PMCID: PMC11179571 DOI: 10.1016/j.ejro.2024.100574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2024] [Revised: 05/25/2024] [Accepted: 05/28/2024] [Indexed: 06/18/2024] Open
Abstract
Purpose To determine the value of an algorithm for reducing stair-step artifacts for advanced coronary analyses in sequential mode coronary CT angiography (CCTA). Methods Forty patients undergoing sequential mode photon-counting detector CCTA with at least one stair-step artifact were included. Twenty patients (14 males; mean age 57±17years) with 45 segments showing stair-step artifacts and without atherosclerosis were included for CTFFR analysis. Twenty patients (20 males; mean age 74±13years) with 22 segments showing stair-step artifacts crossing an atherosclerotic plaque were included for quantitative plaque analysis. Artifacts were graded, and CTFFR and quantitative coronary plaque analyses were performed in standard reconstructions and in those reconstructed with a software (entitled ZeeFree) for artifact reduction. Results Stair-step artifacts were significantly reduced in ZeeFree compared to standard reconstructions (p<0.05). In standard reconstructions, CTFFR was not feasible in 3/45 (7 %) segments but was feasible in all ZeeFree reconstructions. In 9/45 (20 %) segments without atherosclerosis, the ZeeFree algorithm led to a change of CTFFR values from pathologic in standard to physiologic values in ZeeFree reconstructions. In one segment (1/22, 5 %), quantitative plaque analysis was not feasible in standard but only in ZeeFree reconstruction. The mean overall plaque volume (111±60 mm3), the calcific (77±47 mm3), fibrotic (31±28 mm3), and lipidic (4±3 mm3) plaque components were higher in standard than in ZeeFree reconstructions (overall 75±50 mm3, p<0.001; calcific 51±42 mm3, p<0.001; fibrotic 22±19 mm3, p<0.05; lipidic 3±3 mm3, p=0.055). Conclusion Despite the lack of reference standard modalities for CTFFR and coronary plaque analysis, initial evidence indicates that an algorithm for reducing stair-step artifacts in sequential mode CCTA increases the rate and quality of datasets amenable to advanced coronary artery analysis, hereby potentially improving patient management.
Collapse
Affiliation(s)
- Costanza Lisi
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
- Department of Biomedical Sciences, Humanitas University, via Rita Levi Montalcini 4, 20090 Pieve Emanuele, Milan, Italy
| | - Lukas J Moser
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Victor Mergen
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Thomas Flohr
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Matthias Eberhard
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Hatem Alkadhi
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| |
Collapse
|
21
|
Sweeney A, Xavierselvan M, Langley A, Solomon P, Arora A, Mallidi S. Vascular regional analysis unveils differential responses to anti-angiogenic therapy in pancreatic xenografts through macroscopic photoacoustic imaging. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.27.595784. [PMID: 38854042 PMCID: PMC11160648 DOI: 10.1101/2024.05.27.595784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2024]
Abstract
Pancreatic cancer (PC) is a highly lethal malignancy and the third leading cause of cancer deaths in the U.S. Despite major innovations in imaging technologies, there are limited surrogate radiographic indicators to aid in therapy planning and monitoring. Amongst the various imaging techniques Ultrasound-guided photoacoustic imaging (US-PAI) is a promising modality based on endogenous blood (hemoglobin) and blood oxygen saturation (StO 2 ) contrast to monitor response to anti-angiogenic therapies. Adaptation of US-PAI to the clinical realm requires macroscopic configurations for adequate depth visualization, illuminating the need for surrogate radiographic markers, including the tumoral microvessel density (MVD). In this work, subcutaneous xenografts with PC cell lines AsPC-1 and MIA-PaCa-2 were used to investigate the effects of receptor tyrosine kinase inhibitor (sunitinib) treatment on MVD and StO 2 . Through histological correlation, we have shown that regions of high and low vascular density (HVD and LVD) can be identified through frequency domain filtering of macroscopic PA images which could not be garnered from purely global analysis. We utilized vascular regional analysis (VRA) of treatment-induced StO 2 and total hemoglobin (HbT) changes. VRA as a tool to monitor treatment response allowed us to identify potential timepoints of vascular remodeling, highlighting its ability to provide insights into the TME not only for sunitinib treatment but also other anti-angiogenic therapies.
Collapse
|
22
|
Wang H, Ni D, Wang Y. Recursive Deformable Pyramid Network for Unsupervised Medical Image Registration. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2229-2240. [PMID: 38319758 DOI: 10.1109/tmi.2024.3362968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/08/2024]
Abstract
Complicated deformation problems are frequently encountered in medical image registration tasks. Although various advanced registration models have been proposed, accurate and efficient deformable registration remains challenging, especially for handling the large volumetric deformations. To this end, we propose a novel recursive deformable pyramid (RDP) network for unsupervised non-rigid registration. Our network is a pure convolutional pyramid, which fully utilizes the advantages of the pyramid structure itself, but does not rely on any high-weight attentions or transformers. In particular, our network leverages a step-by-step recursion strategy with the integration of high-level semantics to predict the deformation field from coarse to fine, while ensuring the rationality of the deformation field. Meanwhile, due to the recursive pyramid strategy, our network can effectively attain deformable registration without separate affine pre-alignment. We compare the RDP network with several existing registration methods on three public brain magnetic resonance imaging (MRI) datasets, including LPBA, Mindboggle and IXI. Experimental results demonstrate our network consistently outcompetes state of the art with respect to the metrics of Dice score, average symmetric surface distance, Hausdorff distance, and Jacobian. Even for the data without the affine pre-alignment, our network maintains satisfactory performance on compensating for the large deformation. The code is publicly available at https://github.com/ZAX130/RDP.
Collapse
|
23
|
Cutrì E, Morel-Corlu E, Rolland Y, Saint-Jalmes H, Eliat PA, Garin E, Bezy-Wendling J. A microscopic model of the dose distribution in hepatocellular carcinoma after selective internal radiation therapy. Phys Med 2024; 122:103384. [PMID: 38824827 DOI: 10.1016/j.ejmp.2024.103384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 04/03/2024] [Accepted: 05/21/2024] [Indexed: 06/04/2024] Open
Abstract
The dosimetry evaluation for the selective internal radiation therapy is currently performed assuming a uniform activity distribution, which is in contrast with literature findings. A 2D microscopic model of the perfused liver was developed to evaluate the effect of two different 90Y microspheres distributions: i) homogeneous partitioning with the microspheres equally distributed in the perfused liver, and ii) tumor-clustered partitioning where the microspheres distribution is inferred from the patient specific images. METHODS Two subjects diagnosed with liver cancer were included in this study. For each subject, abdominal CT scans acquired prior to the SIRT and post-treatment 90Y positron emission tomography were considered. Two microspheres partitionings were simulated namely homogeneous and tumor-clustered partitioning. The homogeneous and tumor-clustered partitionings were derived starting from CT images. The microspheres radiation is simulated by means of Russell's law. RESULTS In homogenous simulations, the dose delivery is uniform in the whole liver while in the tumor-clustered simulations a heterogeneous distribution of the delivered dose is visible with higher values in the tumor regions. In addition, in the tumor-clustered simulation, the delivered dose is higher in the viable tumor than in the necrotic tumor, for all patients. In the tumor-clustered case, the dose delivered in the non-tumoral tissue (NTT) was considerably lower than in the perfused liver. CONCLUSIONS The model proposed here represents a proof-of-concept for personalized dosimetry assessment based on preoperative CT images.
Collapse
Affiliation(s)
- Elena Cutrì
- Univ Rennes, CLCC Eugène Marquis, Inserm, LTSI - UMR 1099, F-35000, Rennes, France; Université de technologie de Compiègne, CNRS, Biomechanics and Bioengineering, 60203 Compiègne Cedex, France; Inria, Saclay Ile-de-France, Palaiseau, 91120, France.
| | - Ewan Morel-Corlu
- Univ Rennes, CLCC Eugène Marquis, Inserm, LTSI - UMR 1099, F-35000, Rennes, France
| | - Yan Rolland
- Univ Rennes, CLCC Eugène Marquis, Inserm, LTSI - UMR 1099, F-35000, Rennes, France
| | - Hervé Saint-Jalmes
- Univ Rennes, CLCC Eugène Marquis, Inserm, LTSI - UMR 1099, F-35000, Rennes, France
| | - Pierre-Antoine Eliat
- INRAE, INSERM, Univ Rennes, Nutrition Metabolisms and Cancer, NuMeCan, St Gilles, Rennes, France; CNRS, INSERM, Biosit UAR 3480 US_S 018, PRISM, Univ Rennes, Rennes, France
| | - Etienne Garin
- INRAE, INSERM, Univ Rennes, Nutrition Metabolisms and Cancer, NuMeCan, St Gilles, Rennes, France; Department of Nuclear Medicine, Centre Eugène Marquis, Rennes, France
| | | |
Collapse
|
24
|
Peggs ZJT, Brooke JP, Bolton CE, Hall IP, Francis ST, Gowland PA. Free-Breathing Functional Pulmonary Proton MRI: A Novel Approach Using Voxel-Wise Lung Ventilation (VOLVE) Assessment in Healthy Volunteers and Patients With Chronic Obstructive Pulmonary Disease. J Magn Reson Imaging 2024. [PMID: 38819593 DOI: 10.1002/jmri.29444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 04/27/2024] [Accepted: 04/30/2024] [Indexed: 06/01/2024] Open
Abstract
BACKGROUND In respiratory medicine, there is a need for sensitive measures of regional lung function that can be performed using standard imaging technology, without the need for inhaled or intravenous contrast agents. PURPOSE To describe VOxel-wise Lung VEntilation (VOLVE), a new method for quantifying regional lung ventilation (V) and perfusion (Q) using free-breathing proton MRI, and to evaluate VOLVE in healthy never-smokers, healthy people with smoking history, and people with chronic obstructive pulmonary disease (COPD). STUDY TYPE Prospective pilot. POPULATION Twelve healthy never-smoker participants (age 30.3 ± 12.5 years, five male), four healthy participants with smoking history (>10 pack-years) (age 42.5 ± 18.3 years, one male), and 12 participants with COPD (age 62.8 ± 11.1 years, seven male). FIELD STRENGTH/SEQUENCE Single-slice free-breathing two-dimensional fast field echo sequence at 3 T. ASSESSMENT A novel postprocessing was developed to evaluate the MR signal changes in the lung parenchyma using a linear regression-based approach, which makes use of all the data in the time series for maximum sensitivity. V/Q-weighted maps were produced by computing the cross-correlation, lag and gradient between the respiratory/cardiac phase time course and lung parenchyma signal time courses. A comparison of histogram median and skewness values and spirometry was performed. STATISTICAL TESTS Kruskal-Wallis tests with Dunn's multiple comparison tests to compare VOLVE metrics between groups; Spearman correlation to assess the correlation between MRI and spirometry-derived parameters; and Bland-Altman analysis and coefficient of variation to evaluate repeatability were used. A P-value <0.05 was considered significant. RESULTS Significant differences between the groups were found for ventilation between healthy never-smoker and COPD groups (median XCCV, LagV, and GradV) and perfusion (median XCCQ, LagQ, and GradQ). Minimal bias and no significant differences between intravisit scans were found (P range = 0.12-0.97). DATA CONCLUSION This preliminary study showed that VOLVE has potential to provide metrics of function quantification. LEVEL OF EVIDENCE 2 TECHNICAL EFFICACY: Stage 1.
Collapse
Affiliation(s)
- Zachary J T Peggs
- Sir Peter Mansfield Imaging Centre, School of Physics and Astronomy, University of Nottingham, Nottingham, UK
- Centre for Respiratory Research, NIHR Nottingham Biomedical Research Centre, Nottingham, UK
- Centre for Respiratory Research, Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
| | - Jonathan P Brooke
- Centre for Respiratory Research, NIHR Nottingham Biomedical Research Centre, Nottingham, UK
- Centre for Respiratory Research, Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Respiratory Medicine, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Charlotte E Bolton
- Centre for Respiratory Research, NIHR Nottingham Biomedical Research Centre, Nottingham, UK
- Centre for Respiratory Research, Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Respiratory Medicine, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Ian P Hall
- Centre for Respiratory Research, NIHR Nottingham Biomedical Research Centre, Nottingham, UK
- Centre for Respiratory Research, Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Respiratory Medicine, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Susan T Francis
- Sir Peter Mansfield Imaging Centre, School of Physics and Astronomy, University of Nottingham, Nottingham, UK
- Centre for Respiratory Research, NIHR Nottingham Biomedical Research Centre, Nottingham, UK
| | - Penny A Gowland
- Sir Peter Mansfield Imaging Centre, School of Physics and Astronomy, University of Nottingham, Nottingham, UK
- Centre for Respiratory Research, NIHR Nottingham Biomedical Research Centre, Nottingham, UK
| |
Collapse
|
25
|
Osman AFI, Al-Mugren KS, Tamam NM, Shahine B. Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology. Radiat Oncol 2024; 19:61. [PMID: 38773620 PMCID: PMC11110381 DOI: 10.1186/s13014-024-02452-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 05/13/2024] [Indexed: 05/24/2024] Open
Abstract
PURPOSE Accurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma. METHODS This study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model's performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms. RESULTS The ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU. CONCLUSIONS The developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.
Collapse
Affiliation(s)
- Alexander F I Osman
- Department of Medical Physics, Al-Neelain University, Khartoum, 11121, Sudan.
| | - Kholoud S Al-Mugren
- Department of Physics, College of Science, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Nissren M Tamam
- Department of Physics, College of Science, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Bilal Shahine
- Department of Radiation Oncology, American University of Beirut Medical Center, Beirut, Lebanon
| |
Collapse
|
26
|
Aganj I, Mora J, Fischl B, Augustinack JC. Automatic geometry-based estimation of the locus coeruleus region on T 1-weighted magnetic resonance images. Front Neurosci 2024; 18:1375530. [PMID: 38774790 PMCID: PMC11106368 DOI: 10.3389/fnins.2024.1375530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 04/09/2024] [Indexed: 05/24/2024] Open
Abstract
The locus coeruleus (LC) is a key brain structure implicated in cognitive function and neurodegenerative disease. Automatic segmentation of the LC is a crucial step in quantitative non-invasive analysis of the LC in large MRI cohorts. Most publicly available imaging databases for training automatic LC segmentation models take advantage of specialized contrast-enhancing (e.g., neuromelanin-sensitive) MRI. Segmentation models developed with such image contrasts, however, are not readily applicable to existing datasets with conventional MRI sequences. In this work, we evaluate the feasibility of using non-contrast neuroanatomical information to geometrically approximate the LC region from standard 3-Tesla T1-weighted images of 20 subjects from the Human Connectome Project (HCP). We employ this dataset to train and internally/externally evaluate two automatic localization methods, the Expected Label Value and the U-Net. For out-of-sample segmentation, we compare the results with atlas-based segmentation, as well as test the hypothesis that using the phase image as input can improve the robustness. We then apply our trained models to a larger subset of HCP, while exploratorily correlating LC imaging variables and structural connectivity with demographic and clinical data. This report provides an evaluation of computational methods estimating neural structure.
Collapse
Affiliation(s)
- Iman Aganj
- Radiology Department, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, United States
- Radiology Department, Harvard Medical School, Boston, MA, United States
| | - Jocelyn Mora
- Radiology Department, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, United States
| | - Bruce Fischl
- Radiology Department, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, United States
- Radiology Department, Harvard Medical School, Boston, MA, United States
| | - Jean C. Augustinack
- Radiology Department, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, United States
- Radiology Department, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
27
|
Huber MT, Flint KM, McNally PJ, Ellestad SC, Trahey GE. Human Observer Sensitivity to Temporal Noise During B-Mode Ultrasound Scanning: Characterization and Imaging Implications. ULTRASONIC IMAGING 2024; 46:151-163. [PMID: 38497455 DOI: 10.1177/01617346241236160] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
This work measures temporal signal-to-noise ratio (SNR) thresholds that indicate when random noise during ultrasound scanning becomes imperceptible to expert human observers. Visible noise compromises image quality and can potentially lead to non-diagnostic scans. Noise can arise from both stable acoustic sources (clutter) or randomly varying electronic sources (temporal noise). Extensive engineering effort has focused on decreasing noise in both of these categories. In this work, an observer study with five practicing sonographers was performed to assess sonographer sensitivity to temporal noise in ultrasound cine clips. Understanding the conditions where temporal noise is no longer visible during ultrasound imaging can inform engineering efforts seeking to minimize the impact this noise has on image quality. The sonographers were presented with paired temporal noise-free and noise-added simulated speckle cine clips and asked to select the noise-added clips. The degree of motion in the imaging target was found to have a significant effect on the SNR levels where noise was perceived, while changing imaging frequency had little impact. At realistic in vivo motion levels, temporal noise was not perceived in cine clips at and above 28 dB SNR. In a case study presented here, the potential of adaptive intensity adjustment based on this noise perception threshold is validated in a fetal imaging scenario. This study demonstrates how noise perception thresholds can be applied to help design or tune ultrasound systems for different imaging tasks and noise conditions.
Collapse
Affiliation(s)
- Matthew T Huber
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
| | - Katelyn M Flint
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
| | - Patricia J McNally
- Department of Women's and Children's Services, Duke University Hospital, Durham, NC, USA
| | - Sarah C Ellestad
- Division of Maternal-Fetal Medicine, Duke University School of Medicine, Durham, NC, USA
| | - Gregg E Trahey
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
- Department of Radiology, Duke University School of Medicine, Durham, NC, USA
| |
Collapse
|
28
|
Jalili-Mallak N, Tu Y, Lu ZL, Wang Y. ENHANCING 3T RETINOTOPIC MAPS USING DIFFEOMORPHIC REGISTRATION. PROCEEDINGS. IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING 2024; 2024:10.1109/isbi56570.2024.10635556. [PMID: 39421192 PMCID: PMC11486508 DOI: 10.1109/isbi56570.2024.10635556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2024]
Abstract
Retinotopic mapping aims to uncover the relationship between visual stimuli on the retina and neural responses on the visual cortical surface. This study advances retinotopic mapping by applying diffeomorphic registration to the 3T NYU retinotopy dataset, encompassing analyze-PRF and mrVista data. Diffeomorphic Registration for Retinotopic Maps (DRRM) quantifies the diffeomorphic condition, ensuring accurate alignment of retinotopic maps without topological violations. Leveraging the Beltrami coefficient and topological condition, DRRM significantly enhances retinotopic map accuracy. Evaluation against existing methods demonstrates DRRM's superiority on various datasets, including 3T and 7T retinotopy data. The application of diffeomorphic registration improves the interpretability of low-quality retinotopic maps, holding promise for clinical applications.
Collapse
Affiliation(s)
- Negar Jalili-Mallak
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA
| | - Yanshuai Tu
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA
| | - Zhong-Lin Lu
- Division of Arts and Sciences, NYU Shanghai, Shanghai, China
- Center for Neural Science and Department of Psychology, New York University, New York, USA
- NYU-ECNU Institute of Brain and Cognitive Science, NYU Shanghai, Shanghai, China
| | - Yalin Wang
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA
| |
Collapse
|
29
|
Criscuolo ER, Fu Y, Hao Y, Zhang Z, Yang D. A comprehensive lung CT landmark pair dataset for evaluating deformable image registration algorithms. Med Phys 2024; 51:3806-3817. [PMID: 38478966 PMCID: PMC11302745 DOI: 10.1002/mp.17026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 01/29/2024] [Accepted: 03/03/2024] [Indexed: 05/08/2024] Open
Abstract
PURPOSE Deformable image registration (DIR) is a key enabling technology in many diagnostic and therapeutic tasks, but often does not meet the required robustness and accuracy for supporting clinical tasks. This is in large part due to a lack of high-quality benchmark datasets by which new DIR algorithms can be evaluated. Our team was supported by the National Institute of Biomedical Imaging and Bioengineering to develop DIR benchmark dataset libraries for multiple anatomical sites, comprising of large numbers of highly accurate landmark pairs on matching blood vessel bifurcations. Here we introduce our lung CT DIR benchmark dataset library, which was developed to improve upon the number and distribution of landmark pairs in current public lung CT benchmark datasets. ACQUISITION AND VALIDATION METHODS Thirty CT image pairs were acquired from several publicly available repositories as well as authors' institution with IRB approval. The data processing workflow included multiple steps: (1) The images were denoised. (2) Lungs, airways, and blood vessels were automatically segmented. (3) Bifurcations were directly detected on the skeleton of the segmented vessel tree. (4) Falsely identified bifurcations were filtered out using manually defined rules. (5) A DIR was used to project landmarks detected on the first image onto the second image of the image pair to form landmark pairs. (6) Landmark pairs were manually verified. This workflow resulted in an average of 1262 landmark pairs per image pair. Estimates of the landmark pair target registration error (TRE) using digital phantoms were 0.4 mm ± 0.3 mm. DATA FORMAT AND USAGE NOTES The data is published in Zenodo at https://doi.org/10.5281/zenodo.8200423. Instructions for use can be found at https://github.com/deshanyang/Lung-DIR-QA. POTENTIAL APPLICATIONS The dataset library generated in this work is the largest of its kind to date and will provide researchers with a new and improved set of ground truth benchmarks for quantitatively validating DIR algorithms within the lung.
Collapse
Affiliation(s)
| | - Yabo Fu
- Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA
| | - Yao Hao
- Washington University School of Medicine, St. Louis, MO, 63110, USA
| | - Zhendong Zhang
- Department of Radiation Oncology, Duke University, Durham, NC, 27701, USA
| | - Deshan Yang
- Department of Radiation Oncology, Duke University, Durham, NC, 27701, USA
| |
Collapse
|
30
|
Stouffer KM, Trouvé A, Younes L, Kunst M, Ng L, Zeng H, Anant M, Fan J, Kim Y, Chen X, Rue M, Miller MI. Cross-modality mapping using image varifolds to align tissue-scale atlases to molecular-scale measures with application to 2D brain sections. Nat Commun 2024; 15:3530. [PMID: 38664422 PMCID: PMC11045777 DOI: 10.1038/s41467-024-47883-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 04/10/2024] [Indexed: 04/28/2024] Open
Abstract
This paper explicates a solution to building correspondences between molecular-scale transcriptomics and tissue-scale atlases. This problem arises in atlas construction and cross-specimen/technology alignment where specimens per emerging technology remain sparse and conventional image representations cannot efficiently model the high dimensions from subcellular detection of thousands of genes. We address these challenges by representing spatial transcriptomics data as generalized functions encoding position and high-dimensional feature (gene, cell type) identity. We map onto low-dimensional atlas ontologies by modeling regions as homogeneous random fields with unknown transcriptomic feature distribution. We solve simultaneously for the minimizing geodesic diffeomorphism of coordinates through LDDMM and for these latent feature densities. We map tissue-scale mouse brain atlases to gene-based and cell-based transcriptomics data from MERFISH and BARseq technologies and to histopathology and cross-species atlases to illustrate integration of diverse molecular and cellular datasets into a single coordinate system as a means of comparison and further atlas construction.
Collapse
Affiliation(s)
- Kaitlin M Stouffer
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA.
- Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, USA.
- Centre Borelli, ENS Paris-Saclay, Gif-sur-yvette, France.
| | - Alain Trouvé
- Centre Borelli, ENS Paris-Saclay, Gif-sur-yvette, France
| | - Laurent Younes
- Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
| | | | - Lydia Ng
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Hongkui Zeng
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Manjari Anant
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Jean Fan
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Yongsoo Kim
- Department of Neural and Behavioral Sciences, Penn State University, College of Medicine, State College, PA, USA
| | - Xiaoyin Chen
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Mara Rue
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Michael I Miller
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA.
- Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, USA.
| |
Collapse
|
31
|
Hooijmans MT, Lockard CA, Zhou X, Coolbaugh C, Pineda Guzman R, Kersh ME, Damon BM. A registration strategy to characterize DTI-observed changes in skeletal muscle architecture due to passive shortening. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.11.589123. [PMID: 38645028 PMCID: PMC11030449 DOI: 10.1101/2024.04.11.589123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/23/2024]
Abstract
Skeletal muscle architecture is a key determinant of muscle function. Architectural properties such as fascicle length, pennation angle, and curvature can be characterized using Diffusion Tensor Imaging (DTI), but acquiring these data during a contraction is not currently feasible. However, an image registration-based strategy may be able to convert muscle architectural properties observed at rest to their contracted state. As an initial step toward this long-term objective, the aim of this study was to determine if an image registration strategy could be used to convert the whole-muscle average architectural properties observed in the extended joint position to those of a flexed position, following passive rotation. DTI and high-resolution fat/water scans were acquired in the lower leg of seven healthy participants on a 3T MR system in +20° (plantarflexion) and -10° (dorsiflexion) foot positions. The diffusion and anatomical images from the two positions were used to propagate DTI fiber-tracts from seed points along a mesh representation of the aponeurosis of fiber insertion. The -10° and +20° anatomical images were registered and the displacement fields were used to transform the mesh and fiber-tracts from the +20° to the -10° position. Student's paired t-tests were used to compare the mean architectural parameters between the original and transformed fiber-tracts. The whole-muscle average fiber-tract length, pennation angle, curvature, and physiological cross-sectional areas estimates did not differ significantly. DTI fiber-tracts in plantarflexion can be transformed to dorsiflexion position without significantly affecting the average architectural characteristics of the fiber-tracts. In the future, a similar approach could be used to evaluate muscle architecture in a contracted state.
Collapse
Affiliation(s)
- Melissa T. Hooijmans
- Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN, United States of America
- Carle Clinical Imaging Research Program, Stephens Family Clinical Research Institute, Carle Health, Urbana, IL, United States of America
- Department of Radiology and Nuclear Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
| | - Carly A. Lockard
- Carle Clinical Imaging Research Program, Stephens Family Clinical Research Institute, Carle Health, Urbana, IL, United States of America
| | - Xingyu Zhou
- Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN, United States of America
- Carle Clinical Imaging Research Program, Stephens Family Clinical Research Institute, Carle Health, Urbana, IL, United States of America
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, United States of America
| | - Crystal Coolbaugh
- Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN, United States of America
| | - Roberto Pineda Guzman
- Carle Clinical Imaging Research Program, Stephens Family Clinical Research Institute, Carle Health, Urbana, IL, United States of America
| | - Mariana E. Kersh
- Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States of America
- Department of Biomedical and Translational Sciences, Carle-Illinois College of Medicine, University of Illinois at Urbana-Champaign, Urbana, IL, United States of America
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, United States of America
| | - Bruce M. Damon
- Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN, United States of America
- Carle Clinical Imaging Research Program, Stephens Family Clinical Research Institute, Carle Health, Urbana, IL, United States of America
- Department of Biomedical and Translational Sciences, Carle-Illinois College of Medicine, University of Illinois at Urbana-Champaign, Urbana, IL, United States of America
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, United States of America
- Department of Radiology and Radiological Sciences, Vanderbilt University, Nashville, TN, United States of America
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States of America
| |
Collapse
|
32
|
Li A, Ying Y, Gao T, Zhang L, Zhao X, Zhao Y, Song G, Zhang H. MF-Net: multi-scale feature extraction-integration network for unsupervised deformable registration. Front Neurosci 2024; 18:1364409. [PMID: 38680447 PMCID: PMC11045908 DOI: 10.3389/fnins.2024.1364409] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Accepted: 03/20/2024] [Indexed: 05/01/2024] Open
Abstract
Deformable registration plays a fundamental and crucial role in scenarios such as surgical navigation and image-assisted analysis. While deformable registration methods based on unsupervised learning have shown remarkable success in predicting displacement fields with high accuracy, many existing registration networks are limited by the lack of multi-scale analysis, restricting comprehensive utilization of global and local features in the images. To address this limitation, we propose a novel registration network called multi-scale feature extraction-integration network (MF-Net). First, we propose a multiscale analysis strategy that enables the model to capture global and local semantic information in the image, thus facilitating accurate texture and detail registration. Additionally, we introduce grouped gated inception block (GI-Block) as the basic unit of the feature extractor, enabling the feature extractor to selectively extract quantitative features from images at various resolutions. Comparative experiments demonstrate the superior accuracy of our approach over existing methods.
Collapse
Affiliation(s)
- Andi Li
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Yuhan Ying
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Tian Gao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- School of Automation and Electrical Engineering, Shenyang Ligong University, Shenyang, China
| | - Lei Zhang
- Spine Surgery Unit, Shengjing Hospital of China Medical University, Shenyang, China
| | - Xingang Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Yiwen Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Guoli Song
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - He Zhang
- Orthopedic Department, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| |
Collapse
|
33
|
Le Bao V, Haworth A, Dowling J, Walker A, Arumugam S, Jameson M, Chlap P, Wiltshire K, Keats S, Cloak K, Sidhom M, Kneebone A, Holloway L. Evaluating the relationship between contouring variability and modelled treatment outcome for prostate bed radiotherapy. Phys Med Biol 2024; 69:085008. [PMID: 38471173 DOI: 10.1088/1361-6560/ad3325] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 03/12/2024] [Indexed: 03/14/2024]
Abstract
Objectives.Contouring similarity metrics are often used in studies of inter-observer variation and automatic segmentation but do not provide an assessment of clinical impact. This study focused on post-prostatectomy radiotherapy and aimed to (1) identify if there is a relationship between variations in commonly used contouring similarity metrics and resulting dosimetry and (2) identify the variation in clinical target volume (CTV) contouring that significantly impacts dosimetry.Approach.The study retrospectively analysed CT scans of 10 patients from the TROG 08.03 RAVES trial. The CTV, rectum, and bladder were contoured independently by three experienced observers. Using these contours reference simultaneous truth and performance level estimation (STAPLE) volumes were established. Additional CTVs were generated using an atlas algorithm based on a single benchmark case with 42 manual contours. Volumetric-modulated arc therapy (VMAT) treatment plans were generated for the observer, atlas, and reference volumes. The dosimetry was evaluated using radiobiological metrics. Correlations between contouring similarity and dosimetry metrics were calculated using Spearman coefficient (Γ). To access impact of variations in planning target volume (PTV) margin, the STAPLE PTV was uniformly contracted and expanded, with plans created for each PTV volume. STAPLE dose-volume histograms (DVHs) were exported for plans generated based on the contracted/expanded volumes, and dose-volume metrics assessed.Mainresults. The study found no strong correlations between the considered similarity metrics and modelled outcomes. Moderate correlations (0.5 <Γ< 0.7) were observed for Dice similarity coefficient, Jaccard, and mean distance to agreement metrics and rectum toxicities. The observations of this study indicate a tendency for variations in CTV contraction/expansion below 5 mm to result in minor dosimetric impacts.Significance. Contouring similarity metrics must be used with caution when interpreting them as indicators of treatment plan variation. For post-prostatectomy VMAT patients, this work showed variations in contours with an expansion/contraction of less than 5 mm did not lead to notable dosimetric differences, this should be explored in a larger dataset to assess generalisability.
Collapse
Affiliation(s)
- Viet Le Bao
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
| | - Annette Haworth
- Institute of Medical Physics, School of Physics, University of Sydney, Australia
| | - Jason Dowling
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
| | - Amy Walker
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
- Centre for Medical Radiation Physics, University of Wollongong, Wollongong, Australia
| | - Sankar Arumugam
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | - Michael Jameson
- St Vincent's Clinical School, University of New South Wales, Sydney, Australia
- GenesisCare, Sydney, NSW, Australia
| | - Phillip Chlap
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | - Kirsty Wiltshire
- Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
- Sir Peter MacCallum Department of Oncology, The University of Melbourne, Victoria, Australia
| | - Sarah Keats
- Ingham Institute for Applied Medical Research, Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | - Kirrily Cloak
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | - Mark Sidhom
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | | | - Lois Holloway
- South Western Clinical School, University of New South Wales, Sydney, Australia
- Ingham Institute for Applied Medical Research, Sydney, Australia
- Institute of Medical Physics, School of Physics, University of Sydney, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
- Centre for Medical Radiation Physics, University of Wollongong, Wollongong, Australia
| |
Collapse
|
34
|
Cheng C, Wu B, Zhang L, Wan Q, Peng H, Liu X, Zheng H, Zhang H, Zou C. Automatic segmentation of the interscapular brown adipose tissue in rats based on deep learning using the dynamic magnetic resonance fat fraction images. MAGMA (NEW YORK, N.Y.) 2024; 37:215-226. [PMID: 38019377 DOI: 10.1007/s10334-023-01133-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 10/26/2023] [Accepted: 10/27/2023] [Indexed: 11/30/2023]
Abstract
OBJECTIVE The study aims to propose an accurate labelling method of interscapular BAT (iBAT) in rats using dynamic MR fat fraction (FF) images with noradrenaline (NE) stimulation and then develop an automatic iBAT segmentation method using a U-Net model. MATERIALS AND METHODS Thirty-four rats fed different diets or housed at different temperatures underwent successive MR scans before and after NE injection. The iBAT were labelled automatically by identifying the regions with obvious FF change in response to the NE stimulation. Further, these FF images along with the recognized iBAT mask images were used to develop a deep learning network to accomplish the robust segmentation of iBAT in various rat models, even without NE stimulation. The trained model was then validated in rats fed with high-fat diet (HFD) in comparison with normal diet (ND). RESULT A total of 6510 FF images were collected using a clinical 3.0 T MR scanner. The dice similarity coefficient (DSC) between the automatic and manual labelled results was 0.895 ± 0.022. For the network training, the DSC, precision rate, and recall rate were found to be 0.897 ± 0.061, 0.901 ± 0.068 and 0.899 ± 0.086, respectively. The volumes and FF values of iBAT in HFD rats were higher than ND rats, while the FF decrease was larger in ND rats after NE injection. CONCLUSION An automatic iBAT segmentation method for rats was successfully developed using the dynamic labelled FF images of activated BAT and deep learning network.
Collapse
Affiliation(s)
- Chuanli Cheng
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, 518055, China
- Imaging Research Institute of Innovative Medical Equipment, Shenzhen, China
| | - Bingxia Wu
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Lei Zhang
- Radiology Department, Bethune First Hospital of Jilin University, Changchun, China
| | - Qian Wan
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, 518055, China
| | - Hao Peng
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, 518055, China
| | - Xin Liu
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, 518055, China
| | - Hairong Zheng
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, 518055, China
| | - Huimao Zhang
- Radiology Department, Bethune First Hospital of Jilin University, Changchun, China
| | - Chao Zou
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, 518055, China.
- Imaging Research Institute of Innovative Medical Equipment, Shenzhen, China.
| |
Collapse
|
35
|
Roy CW, Milani B, Yerly J, Si-Mohamed S, Romanin L, Bustin A, Tenisch E, Rutz T, Prsa M, Stuber M. Intra-bin correction and inter-bin compensation of respiratory motion in free-running five-dimensional whole-heart magnetic resonance imaging. J Cardiovasc Magn Reson 2024; 26:101037. [PMID: 38499269 PMCID: PMC10987330 DOI: 10.1016/j.jocmr.2024.101037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 03/09/2024] [Accepted: 03/11/2024] [Indexed: 03/20/2024] Open
Abstract
BACKGROUND Free-running cardiac and respiratory motion-resolved whole-heart five-dimensional (5D) cardiovascular magnetic resonance (CMR) can reduce scan planning and provide a means of evaluating respiratory-driven changes in clinical parameters of interest. However, respiratory-resolved imaging can be limited by user-defined parameters which create trade-offs between residual artifact and motion blur. In this work, we develop and validate strategies for both correction of intra-bin and compensation of inter-bin respiratory motion to improve the quality of 5D CMR. METHODS Each component of the reconstruction framework was systematically validated and compared to the previously established 5D approach using simulated free-running data (N = 50) and a cohort of 32 patients with congenital heart disease. The impact of intra-bin respiratory motion correction was evaluated in terms of image sharpness while inter-bin respiratory motion compensation was evaluated in terms of reconstruction error, compression of respiratory motion, and image sharpness. The full reconstruction framework (intra-acquisition correction and inter-acquisition compensation of respiratory motion [IIMC] 5D) was evaluated in terms of image sharpness and scoring of image quality by expert reviewers. RESULTS Intra-bin motion correction provides significantly (p < 0.001) sharper images for both simulated and patient data. Inter-bin motion compensation results in significant (p < 0.001) lower reconstruction error, lower motion compression, and higher sharpness in both simulated (10/11) and patient (9/11) data. The combined framework resulted in significantly (p < 0.001) sharper IIMC 5D reconstructions (End-expiration (End-Exp): 0.45 ± 0.09, End-inspiration (End-Ins): 0.46 ± 0.10) relative to the previously established 5D implementation (End-Exp: 0.43 ± 0.08, End-Ins: 0.39 ± 0.09). Similarly, image scoring by three expert reviewers was significantly (p < 0.001) higher using IIMC 5D (End-Exp: 3.39 ± 0.44, End-Ins: 3.32 ± 0.45) relative to 5D images (End-Exp: 3.02 ± 0.54, End-Ins: 2.45 ± 0.52). CONCLUSION The proposed IIMC reconstruction significantly improves the quality of 5D whole-heart MRI. This may be exploited for higher resolution or abbreviated scanning. Further investigation of the diagnostic impact of this framework and comparison to gold standards is needed to understand its full clinical utility, including exploration of respiratory-driven changes in physiological measurements of interest.
Collapse
Affiliation(s)
- Christopher W Roy
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
| | - Bastien Milani
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Jérôme Yerly
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; Center for Biomedical Imaging (CIBM), Lausanne, Switzerland
| | - Salim Si-Mohamed
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; University Lyon, INSA-Lyon, University Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1206, F-69621, 7 Avenue Jean Capelle O, 69100 Villeurbanne, France; Department of Radiology, Louis Pradel Hospital, Hospices Civils de Lyon, 59 Boulevard Pinel, 69500 Bron, France
| | - Ludovica Romanin
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; Advanced Clinical Imaging Technology, Siemens Healthineers International AG, Lausanne, Switzerland
| | - Aurélien Bustin
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; IHU LIRYC, Electrophysiology and Heart Modeling Institute, Université de Bordeaux - INSERM U1045, Avenue du Haut Lévêque, 33604 Pessac, France; Department of Cardiovascular Imaging, Hôpital Cardiologique du Haut-Lévêque, CHU de Bordeaux, Avenue de Magellan, 33604 Pessac, France
| | - Estelle Tenisch
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Tobias Rutz
- Service of Cardiology, Heart and Vessel Department, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Milan Prsa
- Division of Pediatric Cardiology, Woman-Mother-Child Department, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Matthias Stuber
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; Center for Biomedical Imaging (CIBM), Lausanne, Switzerland
| |
Collapse
|
36
|
Löffler MT, Wu PH, Pirmoazen AM, Joseph GB, Stewart JM, Saeed I, Liu J, Schafer AL, Schwartz AV, Link TM, Kazakia GJ. Microvascular disease not type 2 diabetes is associated with increased cortical porosity: A study of cortical bone microstructure and intracortical vessel characteristics. Bone Rep 2024; 20:101745. [PMID: 38444830 PMCID: PMC10912053 DOI: 10.1016/j.bonr.2024.101745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 12/05/2023] [Accepted: 02/22/2024] [Indexed: 03/07/2024] Open
Abstract
Introduction Fracture risk is elevated in type 2 diabetes (T2D) despite normal or even high bone mineral density (BMD). Microvascular disease (MVD) is a diabetic complication, but also associated with other diseases, for example chronic kidney disease. We hypothesize that increased fracture risk in T2D could be due to increased cortical porosity (Ct.Po) driven by expansion of the vascular network in MVD. The purpose of this study was to investigate associations of T2D and MVD with cortical microstructure and intracortical vessel parameters. Methods The study group consisted of 75 participants (38 with T2D and 37 without T2D). High-resolution peripheral quantitative CT (HR-pQCT) and dynamic contrast-enhanced MRI (DCE-MRI) of the ultra-distal tibia were performed to assess cortical bone and intracortical vessels (outcomes). MVD was defined as ≥1 manifestation including neuropathy, nephropathy, or retinopathy based on clinical exams in all participants. Adjusted means of outcomes were compared between groups with/without T2D or between participants with/without MVD in both groups using linear regression models adjusting for age, sex, BMI, and T2D as applicable. Results MVD was found in 21 (55 %) participants with T2D and in 9 (24 %) participants without T2D. In T2D, cortical pore diameter (Ct.Po.Dm) and diameter distribution (Ct.Po.Dm.SD) were significantly higher by 14.6 μm (3.6 %, 95 % confidence interval [CI]: 2.70, 26.5 μm, p = 0.017) and by 8.73 μm (4.8 %, CI: 0.79, 16.7 μm, p = 0.032), respectively. In MVD, but not in T2D, cortical porosity was significantly higher by 2.25 % (relative increase = 12.9 %, CI: 0.53, 3.97 %, p = 0.011) and cortical BMD (Ct.BMD) was significantly lower by -43.6 mg/cm3 (2.6 %, CI: -77.4, -9.81 mg/cm3, p = 0.012). In T2D, vessel volume and vessel diameter were significantly higher by 0.02 mm3 (13.3 %, CI: 0.004, 0.04 mm3, p = 0.017) and 15.4 μm (2.9 %, CI: 0.42, 30.4 μm, p = 0.044), respectively. In MVD, vessel density was significantly higher by 0.11 mm-3 (17.8 %, CI: 0.01, 0.21 mm-3, p = 0.033) and vessel volume and diameter were significantly lower by -0.02 mm3 (13.7 %, CI: -0.04, -0.004 mm3, p = 0.015) and - 14.6 μm (2.8 %, CI: -29.1, -0.11 μm, p = 0.048), respectively. Conclusions The presence of MVD, rather than T2D, was associated with increased cortical porosity. Increased porosity in MVD was coupled with a larger number of smaller vessels, which could indicate upregulation of neovascularization triggered by ischemia. It is unclear why higher variability and average diameters of pores in T2D were accompanied by larger vessels.
Collapse
Affiliation(s)
- Maximilian T. Löffler
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
- Department of Diagnostic and Interventional Radiology, University Medical Center Freiburg, Freiburg im Breisgau, Germany
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany
| | - Po-hung Wu
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| | - Amir M. Pirmoazen
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| | - Gabby B. Joseph
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| | - Jay M. Stewart
- Department of Ophthalmology, University of California, San Francisco, CA, USA
| | - Isra Saeed
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| | - Jing Liu
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| | - Anne L. Schafer
- Department of Medicine, University of California, San Francisco, CA, USA
- Department of Epidemiology and Biostatistics, University of California, San Francisco, CA, USA
| | - Ann V. Schwartz
- Department of Epidemiology and Biostatistics, University of California, San Francisco, CA, USA
| | - Thomas M. Link
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| | - Galateia J. Kazakia
- Department of Radiology and Biomedical Imaging, University of California, 185 Berry St, Suite 350, San Francisco, CA 94107, USA
| |
Collapse
|
37
|
Wagner MG, Whitehead JF, Periyasamy S, Laeseke PF, Speidel MA. Spatiotemporal frequency domain analysis for blood velocity measurement during embolization procedures. Med Phys 2024; 51:1726-1737. [PMID: 37665770 PMCID: PMC10909916 DOI: 10.1002/mp.16715] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 08/22/2023] [Accepted: 08/23/2023] [Indexed: 09/06/2023] Open
Abstract
BACKGROUND Currently, determining procedural endpoints and treatment efficacy of vascular interventions is largely qualitative and relies on subjective visual assessment of digital subtraction angiography (DSA) images leading to large interobserver variabilities and poor reproducibility. Quantitative metrics such as the residual blood velocity in embolized vessel branches could help establish objective and reproducible endpoints. Recently, velocity quantification techniques based on a contrast enhanced X-ray sequence such as qDSA and 4D DSA have been proposed. These techniques must be robust, and, to avoid radiation dose concerns, they should be compatible with low dose per frame image acquisition. PURPOSE To develop and evaluate a technique for robust blood velocity quantification from low dose contrast enhanced X-ray image sequences that leverages the oscillating signal created by pulsatile blood flow. METHODS The proposed spatiotemporal frequency domain (STF) approach quantifies velocities from time attenuation maps (TAMs) representing the oscillating signal over time for all points along a vessel centerline. Due to the time it takes a contrast bolus to travel along the vessel centerline, the resulting TAM resembles a sheared sine wave. The shear angle is related to the velocity and can be determined in the spatiotemporal frequency domain after applying the 2D Fourier transform to the TAM. The approach was evaluated in a straight tube phantom using three different radiation dose levels and compared to ultrasound transit-time-based measurements. The STF velocity results were also compared to previously published approaches for the measurement of blood velocity from contrast enhanced X-ray sequences including shifted least squared (SLS) and phase shift (PHS). Additionally, an in vivo porcine study (n = 8) was performed where increasing amounts of embolic particles were injected into a hepatic or splenic artery with intermittent velocity measurements after each injection to monitor the resulting reduction in velocity. RESULTS At the lowest evaluated dose level (average air kerma rate 1.3 mGy/s at the interventional reference point), the Pearson correlation between ultrasound and STF velocity measurements was99 % $99\%$ . This was significantly higher (p < 0.0001 $p < 0.0001$ ) than corresponding correlation results between ultrasound and the previously published SLS and PHS approaches (91 $\hskip.001pt 91$ and93 % $93\%$ , respectively). In the in vivo study, a reduction in velocity was observed in85.7 % $85.7\%$ of cases after injection of 1 mL,96.4 % $96.4\%$ after 3 mL, and100.0 % $100.0\%$ after 4 mL of embolic particles. CONCLUSIONS The results show good agreement of the spatiotemporal frequency domain approach with ultrasound even in low dose per frame image sequences. Additionally, the in vivo study demonstrates the ability to monitor the physiological changes due to embolization. This could provide quantitative metrics during vascular procedures to establish objective and reproducible endpoints.
Collapse
Affiliation(s)
- Martin G Wagner
- Department of Radiology, University of Wisconsin, Madison, Wisconsin, USA
- Department of Medical Physics, University of Wisconsin, Madison, Wisconsin, USA
| | - Joseph F Whitehead
- Department of Medical Physics, University of Wisconsin, Madison, Wisconsin, USA
| | - Sarvesh Periyasamy
- Department of Radiology, University of Wisconsin, Madison, Wisconsin, USA
| | - Paul F Laeseke
- Department of Radiology, University of Wisconsin, Madison, Wisconsin, USA
| | - Michael A Speidel
- Department of Medical Physics, University of Wisconsin, Madison, Wisconsin, USA
- Department of Medicine, University of Wisconsin, Madison, Wisconsin, USA
| |
Collapse
|
38
|
Liu L, Fan X, Liu H, Zhang C, Kong W, Dai J, Jiang Y, Xie Y, Liang X. QUIZ: An arbitrary volumetric point matching method for medical image registration. Comput Med Imaging Graph 2024; 112:102336. [PMID: 38244280 DOI: 10.1016/j.compmedimag.2024.102336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Revised: 12/02/2023] [Accepted: 01/09/2024] [Indexed: 01/22/2024]
Abstract
Rigid pre-registration involving local-global matching or other large deformation scenarios is crucial. Current popular methods rely on unsupervised learning based on grayscale similarity, but under circumstances where different poses lead to varying tissue structures, or where image quality is poor, these methods tend to exhibit instability and inaccuracies. In this study, we propose a novel method for medical image registration based on arbitrary voxel point of interest matching, called query point quizzer (QUIZ). QUIZ focuses on the correspondence between local-global matching points, specifically employing CNN for feature extraction and utilizing the Transformer architecture for global point matching queries, followed by applying average displacement for local image rigid transformation.We have validated this approach on a large deformation dataset of cervical cancer patients, with results indicating substantially smaller deviations compared to state-of-the-art methods. Remarkably, even for cross-modality subjects, it achieves results surpassing the current state-of-the-art.
Collapse
Affiliation(s)
- Lin Liu
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Xinxin Fan
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Haoyang Liu
- Guangdong Medical University, Dongguan, 523808, China.
| | - Chulong Zhang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China.
| | - Weibin Kong
- Guangdong Medical University, Dongguan, 523808, China.
| | - Jingjing Dai
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China.
| | - Yuming Jiang
- Department of Radiation Oncology, Wake Forest University School of Medicine, Winston Salem, 27587, USA.
| | - Yaoqin Xie
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China.
| | - Xiaokun Liang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China.
| |
Collapse
|
39
|
Liu H, Nie D, Yang J, Wang J, Tang Z. A New Multi-Atlas Based Deep Learning Segmentation Framework With Differentiable Atlas Feature Warping. IEEE J Biomed Health Inform 2024; 28:1484-1493. [PMID: 38113158 DOI: 10.1109/jbhi.2023.3344646] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2023]
Abstract
Deep learning based multi-atlas segmentation (DL-MA) has achieved the state-of-the-art performance in many medical image segmentation tasks, e.g., brain parcellation. In DL-MA methods, atlas-target correspondence is the key for accurate segmentation. In most existing DL-MA methods, such correspondence is usually established using traditional or deep learning based registration methods at image level with no further feature level adaption. This could cause possible atlas-target feature inconsistency. As a result, the information from atlases often has limited positive and even counteractive impact on the final segmentation results. To tackle this issue, in this paper, we propose a new DL-MA framework, where a novel differentiable atlas feature warping module with a new smooth regularization term is presented to establish feature level atlas-target correspondence. Comparing with the existing DL-MA methods, in our framework, atlas features containing anatomical prior knowledge are more relevant to the target image feature, leading the final segmentation results to a high accuracy level. We evaluate our framework in the context of brain parcellation using two public MR brain image datasets: LPBA40 and NIREP-NA0. The experimental results demonstrate that our framework outperforms both traditional multi-atlas segmentation (MAS) and state-of-the-art DL-MA methods with statistical significance. Further ablation studies confirm the effectiveness of the proposed differentiable atlas feature warping module.
Collapse
|
40
|
Pionteck A, Abderezaei J, Fillingham P, Chuang YC, Sakai Y, Belani P, Rigney B, De Leacy R, Fifi JT, Chien A, Colby GP, Jahan R, Duckwiler G, Sayre J, Holdsworth SJ, Mossa-Basha M, Levitt MR, Mocco J, Kurt M, Nael K. Intracranial aneurysm wall displacement depicted by amplified Flow predicts growth. J Neurointerv Surg 2024:jnis-2023-021227. [PMID: 38320850 PMCID: PMC11300705 DOI: 10.1136/jnis-2023-021227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 01/21/2024] [Indexed: 03/01/2024]
Abstract
BACKGROUND Abnormal intracranial aneurysm (IA) wall motion has been associated with IA growth and rupture. Recently, a new image processing algorithm called amplified Flow (aFlow) has been used to successfully track IA wall motion by combining the amplification of cine and four-dimensional (4D) Flow MRI. We sought to apply aFlow to assess wall motion as a potential marker of IA growth in a paired-wise analysis of patients with growing versus stable aneurysms. METHODS In this retrospective case-control study, 10 patients with growing IAs and a matched cohort of 10 patients with stable IAs who had baseline 4D Flow MRI were included. The aFlow was used to amplify and extract IA wall displacements from 4D Flow MRI. The associations of aFlow parameters with commonly used risk factors and morphometric features were assessed using paired-wise univariate and multivariate analyses. RESULTS aFlow quantitative results showed significantly (P=0.035) higher wall motion displacement depicted by mean±SD 90th% values of 2.34±0.72 in growing IAs versus 1.39±0.58 in stable IAs with an area under the curve of 0.85. There was also significantly (P<0.05) higher variability of wall deformation across IA geometry in growing versus stable IAs depicted by the dispersion variables including 121-150% larger standard deviation ([Formula: see text]) and 128-161% wider interquartile range [Formula: see text]. CONCLUSIONS aFlow-derived quantitative assessment of IA wall motion showed greater wall motion and higher variability of wall deformation in growing versus stable IAs.
Collapse
Affiliation(s)
- Aymeric Pionteck
- Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Javid Abderezaei
- Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Patrick Fillingham
- Neurological Surgery, University of Washington School of Medicine, Seattle, Washington, USA
| | - Ya-Chen Chuang
- Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Yu Sakai
- Diagnostic, Molecular and Interventional Radiology, Mount Sinai Health System, New York, New York, USA
| | - Puneet Belani
- Diagnostic, Molecular and Interventional Radiology, Mount Sinai Health System, New York, New York, USA
| | - Brian Rigney
- Diagnostic, Molecular and Interventional Radiology, Mount Sinai Health System, New York, New York, USA
| | - Reade De Leacy
- Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Johanna T Fifi
- Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Aichi Chien
- Radiological Sciences, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - Geoffrey P Colby
- Neurosurgery, University of California Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - Reza Jahan
- Radiological Sciences, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - Gary Duckwiler
- Radiological Sciences, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - James Sayre
- Radiological Sciences, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | | | - Mahmud Mossa-Basha
- Radiology, University of Washington School of Medicine, Seattle, Washington, USA
| | - Michael R Levitt
- Neurological Surgery, University of Washington School of Medicine, Seattle, Washington, USA
| | - J Mocco
- Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Mehmet Kurt
- Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Kambiz Nael
- Diagnostic, Molecular and Interventional Radiology, Mount Sinai Health System, New York, New York, USA
- Radiological Sciences, University of California, Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| |
Collapse
|
41
|
Kim B, Mathai TS, Summers RM. Unsupervised Multi-parametric MRI Registration Using Neural Optimal Transport. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12927:129270U. [PMID: 39371588 PMCID: PMC11450653 DOI: 10.1117/12.3006289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/08/2024]
Abstract
Precise deformable image registration of multi-parametric MRI sequences is necessary for radiologists in order to identify abnormalities and diagnose diseases, such as prostate cancer and lymphoma. Despite recent advances in unsupervised learning-based registration, volumetric medical image registration that requires considering the variety of data distributions is still challenging. To address the problem of multi-parametric MRI sequence data registration, we propose an unsupervised domain-transported registration method, called OTMorph by employing neural optimal transport that learns an optimal transport plan to map different data distributions. We have designed a novel framework composed of a transport module and a registration module: the former transports data distribution from the moving source domain to the fixed target domain, and the latter takes the transported data and provides the deformed moving volume that is aligned with the fixed volume. Through end-to-end learning, our proposed method can effectively learn deformable registration for the volumes in different distributions. Experimental results with abdominal multi-parametric MRI sequence data show that our method has superior performance over around 67-85% in deforming the MRI volumes compared to the existing learning-based methods. Our method is generic in nature and can be used to register inter-/intra-modality images by mapping the different data distributions in network training.
Collapse
Affiliation(s)
- Boah Kim
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, MD, United States
| | - Tejas Sudharshan Mathai
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, MD, United States
| | - Ronald M Summers
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, MD, United States
| |
Collapse
|
42
|
Merton R, Bosshardt D, Strijkers GJ, Nederveen AJ, Schrauben EM, van Ooij P. Reproducibility of 3D thoracic aortic displacement from 3D cine balanced SSFP at 3 T without contrast enhancement. Magn Reson Med 2024; 91:466-480. [PMID: 37831612 DOI: 10.1002/mrm.29856] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 08/02/2023] [Accepted: 08/16/2023] [Indexed: 10/15/2023]
Abstract
PURPOSE Aortic motion has direct impact on the mechanical stresses acting on the aorta. In aortic disease, increased stiffness of the aorta may lead to decreased aortic motion over time, which could be a predictor for aortic dissection or rupture. This study investigates the reproducibility of obtaining 3D displacement and diameter maps quantified using accelerated 3D cine MRI at 3 T. METHODS A noncontrast-enhanced, free-breathing 3D cine sequence based on balanced SSFP and pseudo-spiral undersampling with high spatial isotropic resolution was developed (spatial/temporal resolution [1.6 mm]3 /67 ms). The thoracic aorta of 14 healthy volunteers was prospectively scanned three times at 3 T: twice on the same day and a third time 2 weeks later. Aortic displacement was calculated using iterative closest point nonrigid registration of manual segmentations of the 3D aorta at end-systole and mid-diastole. Interexamination and interobserver regional analysis of mean displacement for five regions of interest was performed using Bland-Altman analysis. Additionally, a complementary voxel-by-voxel analysis was done, allowing a more local inspection of the method. RESULTS No significant differences were found in mean and maximum displacement for any of the regions of interest for the interexamination and interobserver analysis. The maximum displacement measured in the lower half of the ascending aorta was 11.0 ± 3.4 mm (range: 3.0-17.5 mm) for the first scan. The smallest detectable change in mean displacement in the lower half of the ascending aorta was 3 mm. CONCLUSION Detailed 3D cine balanced SSFP at 3 T allows for reproducible quantification of systolic-diastolic mean aortic displacement within acceptable limits.
Collapse
Affiliation(s)
- Renske Merton
- Radiology and Nuclear Medicine, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
| | - Daan Bosshardt
- Radiology and Nuclear Medicine, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
| | - Gustav J Strijkers
- Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
- Biomedical Physics and Engineering, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Movement Sciences, Amsterdam, the Netherlands
| | - Aart J Nederveen
- Radiology and Nuclear Medicine, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
- Amsterdam Movement Sciences, Amsterdam, the Netherlands
| | - Eric M Schrauben
- Radiology and Nuclear Medicine, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
| | - Pim van Ooij
- Radiology and Nuclear Medicine, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
- Amsterdam Movement Sciences, Amsterdam, the Netherlands
| |
Collapse
|
43
|
Yuan PHS, Athwal A, Shalaby M, Mehnert A, Yu DY, Preti RC, Sarunic M, Navajas EV. Retinal capillary perfusion heterogeneity in diabetic retinopathy detected by optical coherence tomography angiography. Int J Retina Vitreous 2024; 10:12. [PMID: 38273321 PMCID: PMC10809479 DOI: 10.1186/s40942-024-00528-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 01/06/2024] [Indexed: 01/27/2024] Open
Abstract
BACKGROUND Diabetic retinopathy (DR) is a leading cause of blindness and involves retinal capillary damage, microaneurysms, and altered blood flow regulation. Optical coherence tomography angiography (OCTA) is a non-invasive way of visualizing retinal vasculature but has not been used extensively to study blood flow heterogeneity. The purpose of this study is to detect and quantify blood flow heterogeneity utilizing en-face swept source OCTA in patients with DR. METHODS This is a prospective clinical study which examined patients with either type 1 or 2 diabetes mellitus. Each included eye was graded clinically as no DR, mild DR, or moderate-severe DR. Ten consecutive en face 6 × 6 mm foveal SS-OCTA images were obtained from each eye using a PLEX Elite 9000 (Zeiss Meditec, Dublin, CA). Built-in fixation-tracking, follow-up functions were utilized to reduce motion artifacts and ensure same location imaging in sequential frames. Images of the superficial and deep vascular complexes (SVC and DVC) were arranged in temporal stacks of 10 and registered to a reference frame for segmentation using a deep neural network. The vessel segmentation was then masked onto each stack to calculate the pixel intensity coefficient of variance (PICoV) and map the spatiotemporal perfusion heterogeneity of each stack. RESULTS Twenty-nine eyes were included: 7 controls, 7 diabetics with no DR, 8 mild DR, and 7 moderate-severe DR. The PICoV correlated significantly and positively with DR severity. In patients with DR, the perfusion heterogeneity was higher in the temporal half of the macula, particularly in areas of capillary dropout. PICoV also correlates as expected with the established OCTA metrics of perfusion density and vessel density. CONCLUSION PICoV is a novel way to analyze OCTA imaging and quantify perfusion heterogeneity. Retinal capillary perfusion heterogeneity in both the SVC and DVC increased with DR severity. This may be related to the loss of retinal capillary perfusion autoregulation in diabetic retinopathy.
Collapse
Affiliation(s)
- Po Hsiang Shawn Yuan
- Department of Ophthalmology and Visual Sciences, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Arman Athwal
- School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada
| | - Mena Shalaby
- School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada
| | - Andrew Mehnert
- Centre for Ophthalmology and Visual Science, University of Western Australia, Perth, Australia
- Lions Eye Institute, Nedlands, WA, Australia
| | - Dao-Yi Yu
- Centre for Ophthalmology and Visual Science, University of Western Australia, Perth, Australia
- Lions Eye Institute, Nedlands, WA, Australia
| | - Rony C Preti
- Department of Ophthalmology, University of Sao Paulo, Sau Paulo, Brazil
| | - Marinko Sarunic
- School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada
- Institute of Ophthalmology, University College London, London, UK
- Department of Medical Physics and Biomedical Engineering, University College London, London, UK
| | - Eduardo V Navajas
- Department of Ophthalmology and Visual Sciences, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada.
- Eye Care Centre at Vancouver General Hospital, 2550 Willow Street, Vancouver, BC, V5Z 0A6, Canada.
| |
Collapse
|
44
|
Chai Y, Qi K, Wu Y, Li D, Tan G, Guo Y, Chu J, Mu Y, Shen C, Wen Q. All-optical interrogation of brain-wide activity in freely swimming larval zebrafish. iScience 2024; 27:108385. [PMID: 38205255 PMCID: PMC10776927 DOI: 10.1016/j.isci.2023.108385] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 09/22/2023] [Accepted: 10/30/2023] [Indexed: 01/12/2024] Open
Abstract
We introduce an all-optical technique that enables volumetric imaging of brain-wide calcium activity and targeted optogenetic stimulation of specific brain regions in unrestrained larval zebrafish. The system consists of three main components: a 3D tracking module, a dual-color fluorescence imaging module, and a real-time activity manipulation module. Our approach uses a sensitive genetically encoded calcium indicator in combination with a long Stokes shift red fluorescence protein as a reference channel, allowing the extraction of Ca2+ activity from signals contaminated by motion artifacts. The method also incorporates rapid 3D image reconstruction and registration, facilitating real-time selective optogenetic stimulation of different regions of the brain. By demonstrating that selective light activation of the midbrain regions in larval zebrafish could reliably trigger biased turning behavior and changes of brain-wide neural activity, we present a valuable tool for investigating the causal relationship between distributed neural circuit dynamics and naturalistic behavior.
Collapse
Affiliation(s)
- Yuming Chai
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| | - Kexin Qi
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| | - Yubin Wu
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| | - Daguang Li
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| | - Guodong Tan
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| | - Yuqi Guo
- Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology and Center for Biomedical Optics and Molecular Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Jun Chu
- Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology and Center for Biomedical Optics and Molecular Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yu Mu
- Institute of Neuroscience, State Key Laboratory of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
| | - Chen Shen
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| | - Quan Wen
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- Hefei National Research Center for Physical Sciences at the Microscale, Center for Integrative Imaging, University of Science and Technology of China, Hefei, China
| |
Collapse
|
45
|
Whitehead JF, Periyasamy S, Laeseke PF, Speidel MA, Wagner MG. Motion-compensation approach for quantitative digital subtraction angiography and its effect on in-vivo blood velocity measurement. J Med Imaging (Bellingham) 2024; 11:013501. [PMID: 38188936 PMCID: PMC10765039 DOI: 10.1117/1.jmi.11.1.013501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 11/13/2023] [Accepted: 12/18/2023] [Indexed: 01/09/2024] Open
Abstract
Purpose Quantitative monitoring of flow-altering interventions has been proposed using algorithms that quantify blood velocity from time-resolved two-dimensional angiograms. These algorithms track the movement of contrast oscillations along a vessel centerline. Vessel motion may occur relative to a statically defined vessel centerline, corrupting the blood velocity measurement. We provide a method for motion-compensated blood velocity quantification. Approach The motion-compensation approach utilizes a vessel segmentation algorithm to perform frame-by-frame vessel registration and creates a dynamic vessel centerline that moves with the vasculature. Performance was evaluated in-vivo through comparison with manually annotated centerlines. The method was also compared to a previous uncompensated method using best- and worst-case static centerlines chosen to minimize and maximize centerline placement accuracy. Blood velocities determined through quantitative DSA (qDSA) analysis for each centerline type were compared through linear regression analysis. Results Centerline distance errors were 0.3 ± 0.1 mm relative to gold standard manual annotations. For the uncompensated approach, the best- and worst-case static centerlines had distance errors of 1.1 ± 0.6 and 2.9 ± 1.2 mm , respectively. Linear regression analysis found a high R -squared between qDSA-derived blood velocities using gold standard centerlines and motion-compensated centerlines (R 2 = 0.97 ) with a slope of 1.15 and a small offset of - 0.6 cm / s . The use of static centerlines resulted in low coefficients of determination for the best case (R 2 = 0.35 ) and worst-case (R 2 = 0.20 ) scenarios, with slopes close to zero. Conclusions In-vivo validation of motion-compensated qDSA analysis demonstrated improved velocity quantification accuracy in vessels with motion, addressing an important clinical limitation of the current qDSA algorithm.
Collapse
Affiliation(s)
- Joseph F. Whitehead
- University of Wisconsin – Madison, Department of Medical Physics, Madison, Wisconsin, United States
| | - Sarvesh Periyasamy
- University of Wisconsin – Madison, Department of Radiology, Madison, Wisconsin, United States
| | - Paul F. Laeseke
- University of Wisconsin – Madison, Department of Radiology, Madison, Wisconsin, United States
| | - Michael A. Speidel
- University of Wisconsin – Madison, Department of Medical Physics, Madison, Wisconsin, United States
- University of Wisconsin – Madison, Department of Medicine, Madison, Wisconsin, United States
| | - Martin G. Wagner
- University of Wisconsin – Madison, Department of Medical Physics, Madison, Wisconsin, United States
- University of Wisconsin – Madison, Department of Radiology, Madison, Wisconsin, United States
| |
Collapse
|
46
|
Gao X, Zheng G. SMILE: Siamese Multi-scale Interactive-representation LEarning for Hierarchical Diffeomorphic Deformable image registration. Comput Med Imaging Graph 2024; 111:102322. [PMID: 38157671 DOI: 10.1016/j.compmedimag.2023.102322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 11/23/2023] [Accepted: 12/14/2023] [Indexed: 01/03/2024]
Abstract
Deformable medical image registration plays an important role in many clinical applications. It aims to find a dense deformation field to establish point-wise correspondences between a pair of fixed and moving images. Recently, unsupervised deep learning-based registration methods have drawn more and more attention because of fast inference at testing stage. Despite remarkable progress, existing deep learning-based methods suffer from several limitations including: (a) they often overlook the explicit modeling of feature correspondences due to limited receptive fields; (b) the performance on image pairs with large spatial displacements is still limited since the dense deformation field is regressed from features learned by local convolutions; and (c) desirable properties, including topology-preservation and the invertibility of transformation, are often ignored. To address above limitations, we propose a novel Convolutional Neural Network (CNN) consisting of a Siamese Multi-scale Interactive-representation LEarning (SMILE) encoder and a Hierarchical Diffeomorphic Deformation (HDD) decoder. Specifically, the SMILE encoder aims for effective feature representation learning and spatial correspondence establishing while the HDD decoder seeks to regress the dense deformation field in a coarse-to-fine manner. We additionally propose a novel Local Invertible Loss (LIL) to encourage topology-preservation and local invertibility of the regressed transformation while keeping high registration accuracy. Extensive experiments conducted on two publicly available brain image datasets demonstrate the superiority of our method over the state-of-the-art (SOTA) approaches. Specifically, on the Neurite-OASIS dataset, our method achieved an average DSC of 0.815 and an average ASSD of 0.633 mm.
Collapse
Affiliation(s)
- Xiaoru Gao
- Institute of Medical Robotics, School of Biomedical Engineering, 800 DongChuan Road, Shanghai Jiao Tong University, Shanghai, 200240, China
| | - Guoyan Zheng
- Institute of Medical Robotics, School of Biomedical Engineering, 800 DongChuan Road, Shanghai Jiao Tong University, Shanghai, 200240, China.
| |
Collapse
|
47
|
Zheng JQ, Wang Z, Huang B, Lim NH, Papież BW. Residual Aligner-based Network (RAN): Motion-separable structure for coarse-to-fine discontinuous deformable registration. Med Image Anal 2024; 91:103038. [PMID: 38000258 DOI: 10.1016/j.media.2023.103038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 11/09/2023] [Accepted: 11/15/2023] [Indexed: 11/26/2023]
Abstract
Deformable image registration, the estimation of the spatial transformation between different images, is an important task in medical imaging. Deep learning techniques have been shown to perform 3D image registration efficiently. However, current registration strategies often only focus on the deformation smoothness, which leads to the ignorance of complicated motion patterns (e.g., separate or sliding motions), especially for the intersection of organs. Thus, the performance when dealing with the discontinuous motions of multiple nearby objects is limited, causing undesired predictive outcomes in clinical usage, such as misidentification and mislocalization of lesions or other abnormalities. Consequently, we proposed a novel registration method to address this issue: a new Motion Separable backbone is exploited to capture the separate motion, with a theoretical analysis of the upper bound of the motions' discontinuity provided. In addition, a novel Residual Aligner module was used to disentangle and refine the predicted motions across the multiple neighboring objects/organs. We evaluate our method, Residual Aligner-based Network (RAN), on abdominal Computed Tomography (CT) scans and it has shown to achieve one of the most accurate unsupervised inter-subject registration for the 9 organs, with the highest-ranked registration of the veins (Dice Similarity Coefficient (%)/Average surface distance (mm): 62%/4.9mm for the vena cava and 34%/7.9mm for the portal and splenic vein), with a smaller model structure and less computation compared to state-of-the-art methods. Furthermore, when applied to lung CT, the RAN achieves comparable results to the best-ranked networks (94%/3.0mm), also with fewer parameters and less computation.
Collapse
Affiliation(s)
- Jian-Qing Zheng
- The Kennedy Institute of Rheumatology, University of Oxford, UK.
| | - Ziyang Wang
- Department of Computer Science, University of Oxford, Oxford, UK
| | - Baoru Huang
- The Hamlyn Centre for Robotic Surgery, Imperial College, London, UK
| | - Ngee Han Lim
- The Kennedy Institute of Rheumatology, University of Oxford, UK
| | | |
Collapse
|
48
|
Yao Y, Zhong J, Zhang L, Khan S, Chen W. CartiMorph: A framework for automated knee articular cartilage morphometrics. Med Image Anal 2024; 91:103035. [PMID: 37992496 DOI: 10.1016/j.media.2023.103035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 08/25/2023] [Accepted: 11/13/2023] [Indexed: 11/24/2023]
Abstract
We introduce CartiMorph, a framework for automated knee articular cartilage morphometrics. It takes an image as input and generates quantitative metrics for cartilage subregions, including the percentage of full-thickness cartilage loss (FCL), mean thickness, surface area, and volume. CartiMorph leverages the power of deep learning models for hierarchical image feature representation. Deep learning models were trained and validated for tissue segmentation, template construction, and template-to-image registration. We established methods for surface-normal-based cartilage thickness mapping, FCL estimation, and rule-based cartilage parcellation. Our cartilage thickness map showed less error in thin and peripheral regions. We evaluated the effectiveness of the adopted segmentation model by comparing the quantitative metrics obtained from model segmentation and those from manual segmentation. The root-mean-squared deviation of the FCL measurements was less than 8%, and strong correlations were observed for the mean thickness (Pearson's correlation coefficient ρ∈[0.82,0.97]), surface area (ρ∈[0.82,0.98]) and volume (ρ∈[0.89,0.98]) measurements. We compared our FCL measurements with those from a previous study and found that our measurements deviated less from the ground truths. We observed superior performance of the proposed rule-based cartilage parcellation method compared with the atlas-based approach. CartiMorph has the potential to promote imaging biomarkers discovery for knee osteoarthritis.
Collapse
Affiliation(s)
- Yongcheng Yao
- CU Lab of AI in Radiology (CLAIR), Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China.
| | - Junru Zhong
- CU Lab of AI in Radiology (CLAIR), Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China
| | - Liping Zhang
- CU Lab of AI in Radiology (CLAIR), Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China
| | - Sheheryar Khan
- School of Professional Education and Executive Development, The Hong Kong Polytechnic University, Hong Kong, China
| | - Weitian Chen
- CU Lab of AI in Radiology (CLAIR), Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China.
| |
Collapse
|
49
|
Wang Y, Zhang Y, Ma C, Wang R, Guo Z, Shen Y, Wang M, Meng H. Neonatal White Matter Damage Analysis Using DTI Super-Resolution and Multi-Modality Image Registration. Int J Neural Syst 2024; 34:2450001. [PMID: 37982259 DOI: 10.1142/s0129065724500011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2023]
Abstract
Punctate White Matter Damage (PWMD) is a common neonatal brain disease, which can easily cause neurological disorder and strongly affect life quality in terms of neuromotor and cognitive performance. Especially, at the neonatal stage, the best cure time can be easily missed because PWMD is not conducive to the diagnosis based on current existing methods. The lesion of PWMD is relatively straightforward on T1-weighted Magnetic Resonance Imaging (T1 MRI), showing semi-oval, cluster or linear high signals. Diffusion Tensor Magnetic Resonance Image (DT-MRI, referred to as DTI) is a noninvasive technique that can be used to study brain microstructures in vivo, and provide information on movement and cognition-related nerve fiber tracts. Therefore, a new method was proposed to use T1 MRI combined with DTI for better neonatal PWMD analysis based on DTI super-resolution and multi-modality image registration. First, after preprocessing, neonatal DTI super-resolution was performed with the three times B-spline interpolation algorithm based on the Log-Euclidean space to improve DTIs' resolution to fit the T1 MRIs and facilitate nerve fiber tractography. Second, the symmetric diffeomorphic registration algorithm and inverse b0 image were selected for multi-modality image registration of DTI and T1 MRI. Finally, the 3D lesion models were combined with fiber tractography results to analyze and predict the degree of PWMD lesions affecting fiber tracts. Extensive experiments demonstrated the effectiveness and super performance of our proposed method. This streamlined technique can play an essential auxiliary role in diagnosing and treating neonatal PWMD.
Collapse
Affiliation(s)
- Yi Wang
- School of Electronics and Information, Northwestern Polytechnical University, 1 Dongxiang Road, Chang'an District, Xi'an, Shaanxi 710129, P. R. China
| | - Yuan Zhang
- School of Electronics and Information, Northwestern Polytechnical University, 1 Dongxiang Road, Chang'an District, Xi'an, Shaanxi 710129, P. R. China
| | - Chi Ma
- School of Electronics and Information, Northwestern Polytechnical University, 1 Dongxiang Road, Chang'an District, Xi'an, Shaanxi 710129, P. R. China
| | - Rui Wang
- School of Electronics and Information, Northwestern Polytechnical University, 1 Dongxiang Road, Chang'an District, Xi'an, Shaanxi 710129, P. R. China
| | - Zhe Guo
- School of Electronics and Information, Northwestern Polytechnical University, 1 Dongxiang Road, Chang'an District, Xi'an, Shaanxi 710129, P. R. China
| | - Yu Shen
- Henan Provincial People's Hospital, Henan Province No. 7 Weiwu, Henan 450000, P. R. China
| | - Miaomiao Wang
- The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi 710000, P. R. China
| | - Hongying Meng
- College of Engineering, Brunel University, Kingston Lane, Uxbridge, Middlesex, London, UB8 3PH, UK
| |
Collapse
|
50
|
Liu Y, Li X, Li R, Huang S, Yang X. A multi-view assisted registration network for MRI registration pre- and post-therapy. Med Biol Eng Comput 2023; 61:3181-3191. [PMID: 38093154 DOI: 10.1007/s11517-023-02949-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 10/11/2023] [Indexed: 12/24/2023]
Abstract
Image registration of magnetic resonance imaging (MRI) pre- and post-therapy is an important part of evaluating the effect of therapy in tumor patients. The accuracy of evaluation results heavily relies on the alignment of the MRI image after registration. Although recent advancements have been made in medical image registration, applying these methods to MRI registration pre- and post-therapy remains challenging. Existing methods typically utilize single-view data for registration. However, when applied to MRI data where some slices are clear while others are blurred, these methods can be misled by erroneous spatial information in the blurred regions, leading to poor registration outcomes. To mitigate the interference caused by erroneous spatial information in single-view data, this paper proposes a multi-stream fusion-assisted registration network that incorporates different-view MRIs of the same patient at the same site. Additionally, a cross-attention guided fusion module is designed within the network to effectively utilize accurate spatial information from multi-view data. The proposed approach was evaluated on clinical data, and the experimental results demonstrated that incorporating multiple view data as auxiliary information significantly enhances the accuracy of MRI image registration before and after radiotherapy.
Collapse
Affiliation(s)
- Yanxia Liu
- School of Software Engineering, South China University of Technology, Guangzhou, Guangdong, 510006, China
| | - Xiaozhen Li
- School of Software Engineering, South China University of Technology, Guangzhou, Guangdong, 510006, China
| | - Rui Li
- School of Software Engineering, South China University of Technology, Guangzhou, Guangdong, 510006, China
| | - SiJuan Huang
- Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, 510060, China.
- State Key Laboratory of Oncology in South China, Guangzhou, Guangdong, 510060, China.
- Collaborative Innovation Center for Cancer Medicine, Guangzhou, Guangdong, 510060, China.
- Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, Guangdong, 510060, China.
| | - Xin Yang
- Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, 510060, China.
- State Key Laboratory of Oncology in South China, Guangzhou, Guangdong, 510060, China.
- Collaborative Innovation Center for Cancer Medicine, Guangzhou, Guangdong, 510060, China.
- Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, Guangdong, 510060, China.
| |
Collapse
|