1
|
Ta K, Ahn SS, Thorn SL, Stendahl JC, Zhang X, Langdon J, Staib LH, Sinusas AJ, Duncan JS. Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2010-2020. [PMID: 38231820 DOI: 10.1109/tmi.2024.3355383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2024]
Abstract
Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.
Collapse
|
2
|
Kumari S, Singh P. Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives. Comput Biol Med 2024; 170:107912. [PMID: 38219643 DOI: 10.1016/j.compbiomed.2023.107912] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Revised: 11/02/2023] [Accepted: 12/24/2023] [Indexed: 01/16/2024]
Abstract
Deep learning has demonstrated remarkable performance across various tasks in medical imaging. However, these approaches primarily focus on supervised learning, assuming that the training and testing data are drawn from the same distribution. Unfortunately, this assumption may not always hold true in practice. To address these issues, unsupervised domain adaptation (UDA) techniques have been developed to transfer knowledge from a labeled domain to a related but unlabeled domain. In recent years, significant advancements have been made in UDA, resulting in a wide range of methodologies, including feature alignment, image translation, self-supervision, and disentangled representation methods, among others. In this paper, we provide a comprehensive literature review of recent deep UDA approaches in medical imaging from a technical perspective. Specifically, we categorize current UDA research in medical imaging into six groups and further divide them into finer subcategories based on the different tasks they perform. We also discuss the respective datasets used in the studies to assess the divergence between the different domains. Finally, we discuss emerging areas and provide insights and discussions on future research directions to conclude this survey.
Collapse
Affiliation(s)
- Suruchi Kumari
- Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, India.
| | - Pravendra Singh
- Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, India.
| |
Collapse
|
3
|
Li C, Fan X, Aronson JP, Hong J, Khan T, Paulsen KD. Model-based image updating in deep brain stimulation with assimilation of deep brain sparse data. Med Phys 2023; 50:7904-7920. [PMID: 37418478 DOI: 10.1002/mp.16578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 04/06/2023] [Accepted: 05/01/2023] [Indexed: 07/09/2023] Open
Abstract
BACKGROUND Accuracy of electrode placement for deep brain stimulation (DBS) is critical to achieving desired surgical outcomes and impacts the efficacy of treating neurodegenerative diseases. Intraoperative brain shift degrades the accuracy of surgical navigation based on preoperative images. PURPOSE We extended a model-based image updating scheme to address intraoperative brain shift in DBS surgery and improved its accuracy in deep brain. METHODS We evaluated 10 patients, retrospectively, who underwent bilateral DBS surgery and classified them into groups of large and small deformation based on a 2 mm subsurface movement threshold and brain shift index of 5%. In each case, sparse brain deformation data were used to estimate whole brain displacements and deform preoperative CT (preCT) to generate updated CT (uCT). Accuracy of uCT was assessed using target registration errors (TREs) at the Anterior Commissure (AC), Posterior Commissure (PC), and four calcification points in the sub-ventricular area by comparing their locations in uCT with their ground truth counterparts in postoperative CT (postCT). RESULTS In the large deformation group, TREs were reduced from 2.5 mm in preCT to 1.2 mm in uCT (53% compensation); in the small deformation group, errors were reduced from 1.25 to 0.74 mm (41%). Average reduction of TREs at AC, PC and pineal gland were significant, statistically (p ⩽ 0.01). CONCLUSIONS With more rigorous validation of model results, this study confirms the feasibility of improving the accuracy of model-based image updating in compensating for intraoperative brain shift during DBS procedures by assimilating deep brain sparse data.
Collapse
Affiliation(s)
- Chen Li
- Thayer School of Engineering, Dartmouth College, Hanover, New Hampshire, USA
| | - Xiaoyao Fan
- Thayer School of Engineering, Dartmouth College, Hanover, New Hampshire, USA
| | - Joshua P Aronson
- Geisel School of Medicine, Dartmouth College, Hanover, New Hampshire, USA
- Section of Neurosurgery, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire, USA
| | - Jennifer Hong
- Geisel School of Medicine, Dartmouth College, Hanover, New Hampshire, USA
- Section of Neurosurgery, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire, USA
| | - Tahsin Khan
- Thayer School of Engineering, Dartmouth College, Hanover, New Hampshire, USA
| | - Keith D Paulsen
- Thayer School of Engineering, Dartmouth College, Hanover, New Hampshire, USA
- Geisel School of Medicine, Dartmouth College, Hanover, New Hampshire, USA
- Norris Cotton Cancer Center, Lebanon, New Hampshire, USA
| |
Collapse
|
4
|
Li L, Ding W, Huang L, Zhuang X, Grau V. Multi-modality cardiac image computing: A survey. Med Image Anal 2023; 88:102869. [PMID: 37384950 DOI: 10.1016/j.media.2023.102869] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 05/01/2023] [Accepted: 06/12/2023] [Indexed: 07/01/2023]
Abstract
Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.
Collapse
Affiliation(s)
- Lei Li
- Department of Engineering Science, University of Oxford, Oxford, UK.
| | - Wangbin Ding
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
| | - Liqin Huang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
| | - Xiahai Zhuang
- School of Data Science, Fudan University, Shanghai, China
| | - Vicente Grau
- Department of Engineering Science, University of Oxford, Oxford, UK
| |
Collapse
|
5
|
Li H, Bhatt M, Qu Z, Zhang S, Hartel MC, Khademhosseini A, Cloutier G. Deep learning in ultrasound elastography imaging: A review. Med Phys 2022; 49:5993-6018. [PMID: 35842833 DOI: 10.1002/mp.15856] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2021] [Revised: 02/04/2022] [Accepted: 07/06/2022] [Indexed: 11/11/2022] Open
Abstract
It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Hongliang Li
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada.,Institute of Biomedical Engineering, University of Montreal, Montréal, Québec, Canada
| | - Manish Bhatt
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada
| | - Zhen Qu
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada
| | - Shiming Zhang
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Martin C Hartel
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Ali Khademhosseini
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Guy Cloutier
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada.,Institute of Biomedical Engineering, University of Montreal, Montréal, Québec, Canada.,Department of Radiology, Radio-Oncology and Nuclear Medicine, University of Montreal, Montréal, Québec, Canada
| |
Collapse
|
6
|
Feher A, Baldassarre LA, Sinusas AJ. Novel Cardiac Computed Tomography Methods for the Assessment of Anthracycline Induced Cardiotoxicity. Front Cardiovasc Med 2022; 9:875150. [PMID: 35571206 PMCID: PMC9094702 DOI: 10.3389/fcvm.2022.875150] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Accepted: 03/25/2022] [Indexed: 12/12/2022] Open
Abstract
Anthracyclines are among the most frequently utilized anti-cancer therapies; however, their use is frequently associated with off-target cardiotoxic effects. Cardiac computed tomography (CCT) is a validated and rapidly evolving technology for the evaluation of cardiac structures, coronary anatomy and plaque, cardiac function and preprocedural planning. However, with emerging new techniques, CCT is rapidly evolving to offer information beyond the evaluation of cardiac structure and epicardial coronary arteries to provide details on myocardial deformation, extracellular volume, and coronary vasoreactivity. The potential for molecular imaging in CCT is also growing. In the current manuscript we review these emerging computed tomography techniques and their potential role in the evaluation of anthracycline-induced cardiotoxicity.
Collapse
Affiliation(s)
- Attila Feher
- Department of Internal Medicine, Section of Cardiovascular Medicine, Yale University School of Medicine, New Haven, CT, United States
- *Correspondence: Attila Feher,
| | - Lauren A. Baldassarre
- Department of Internal Medicine, Section of Cardiovascular Medicine, Yale University School of Medicine, New Haven, CT, United States
- Department of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, CT, United States
| | - Albert J. Sinusas
- Department of Internal Medicine, Section of Cardiovascular Medicine, Yale University School of Medicine, New Haven, CT, United States
- Department of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, CT, United States
- Department of Biomedical Engineering, Yale University, New Haven, CT, United States
| |
Collapse
|
7
|
Mukaddim RA, Meshram NH, Weichmann AM, Mitchell CC, Varghese T. Spatiotemporal Bayesian Regularization for Cardiac Strain Imaging: Simulation and In Vivo Results. IEEE OPEN JOURNAL OF ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 1:21-36. [PMID: 35174360 PMCID: PMC8846604 DOI: 10.1109/ojuffc.2021.3130021] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Cardiac strain imaging (CSI) plays a critical role in the detection of myocardial motion abnormalities. Displacement estimation is an important processing step to ensure the accuracy and precision of derived strain tensors. In this paper, we propose and implement Spatiotemporal Bayesian regularization (STBR) algorithms for two-dimensional (2-D) normalized cross-correlation (NCC) based multi-level block matching along with incorporation into a Lagrangian cardiac strain estimation framework. Assuming smooth temporal variation over a short span of time, the proposed STBR algorithm performs displacement estimation using at least four consecutive ultrasound radio-frequency (RF) frames by iteratively regularizing 2-D NCC matrices using information from a local spatiotemporal neighborhood in a Bayesian sense. Two STBR schemes are proposed to construct Bayesian likelihood functions termed as Spatial then Temporal Bayesian (STBR-1) and simultaneous Spatiotemporal Bayesian (STBR-2). Radial and longitudinal strain estimated from a finite-element-analysis (FEA) model of realistic canine myocardial deformation were utilized to quantify strain bias, normalized strain error and total temporal relative error (TTR). Statistical analysis with one-way analysis of variance (ANOVA) showed that all Bayesian regularization methods significantly outperform NCC with lower bias and errors (p < 0.001). However, there was no significant difference among Bayesian methods. For example, mean longitudinal TTR for NCC, SBR, STBR-1 and STBR-2 were 25.41%, 9.27%, 10.38% and 10.13% respectively An in vivo feasibility study using RF data from ten healthy mice hearts were used to compare the elastographic signal-to-noise ratio (SNRe) calculated using stochastic analysis. STBR-2 had the highest expected SNRe both for radial and longitudinal strain. The mean expected SNRe values for accumulated radial strain for NCC, SBR, STBR-1 and STBR-2 were 5.03, 9.43, 9.42 and 10.58, respectively. Overall results suggest that STBR improves CSI in vivo.
Collapse
Affiliation(s)
- Rashid Al Mukaddim
- Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI 53706 USA.,Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706 USA
| | - Nirvedh H Meshram
- Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI 53706 USA.,Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706 USA
| | - Ashley M Weichmann
- Small Animal Imaging and Radiotherapy Facility, UW Carbone Cancer Center, Madison, WI 53705 USA
| | - Carol C Mitchell
- Department of Medicine/Division of Cardiovascular Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WI 53792 USA
| | - Tomy Varghese
- Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI 53706 USA.,Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706 USA
| |
Collapse
|