Deep Learning for Cardiac Image Segmentation: A Review

Post-hoc out-of-distribution detection for cardiac MRI segmentation

In real-world scenarios, medical image segmentation models encounter input images that may deviate from the training images in various ways. These differences can arise from changes in image scanners and acquisition protocols, or even the images can come from a different modality or domain. When the model encounters these out-of-distribution (OOD) images, it can behave unpredictably. Therefore, it is important to develop a system that handles such out-of-distribution images to ensure the safe usage of the models in clinical practice. In this paper, we propose a post-hoc out-of-distribution (OOD) detection method that can be used with any pre-trained segmentation model. Our method utilizes multi-scale representations extracted from the encoder blocks of the segmentation model and employs Mahalanobis distance as a metric to measure the similarity between the input image and the in-distribution images. The segmentation model is pre-trained on a publicly available cardiac short-axis cine MRI dataset. The detection performance of the proposed method is evaluated on 13 different OOD datasets, which can be categorized as near, mild, and far OOD datasets based on their similarity to the in-distribution dataset. The results show that our method outperforms state-of-the-art feature space-based and uncertainty-based OOD detection methods across the various OOD datasets. Our method successfully detects near, mild, and far OOD images with high detection accuracy, showcasing the advantage of using the multi-scale and semantically rich representations of the encoder. In addition to the feature-based approach, we also propose a Dice coefficient-based OOD detection method, which demonstrates superior performance for adversarial OOD detection and shows a high correlation with segmentation quality. For the uncertainty-based method, despite having a strong correlation with the quality of the segmentation results in the near OOD datasets, they failed to detect mild and far OOD images, indicating the weakness of these methods when the images are more dissimilar. Future work will explore combining Mahalanobis distance and uncertainty scores for improved detection of challenging OOD images that are difficult to segment.

DeSPPNet: A Multiscale Deep Learning Model for Cardiac Segmentation

BACKGROUND

Cardiac magnetic resonance imaging (MRI) plays a crucial role in monitoring disease progression and evaluating the effectiveness of treatment interventions. Cardiac MRI allows medical practitioners to assess cardiac function accurately by providing comprehensive and quantitative information about the structure and function, hence making it an indispensable tool for monitoring the disease and treatment response. Deep learning-based segmentation enables the precise delineation of cardiac structures including the myocardium, right ventricle, and left ventricle. The accurate segmentation of these structures helps in the diagnosis of heart failure, cardiac functional response to therapies, and understanding the state of the heart functions after treatment.

OBJECTIVES

The objective of this study is to develop a multiscale deep learning model to segment cardiac organs based on MRI imaging data. Good segmentation performance is difficult to achieve due to the complex nature of the cardiac structure, which includes a variety of chambers, arteries, and tissues. Furthermore, the human heart is also constantly beating, leading to motion artifacts that reduce image clarity and consistency. As a result, a multiscale method is explored to overcome various challenges in segmenting cardiac MRI images.

METHODS

This paper proposes DeSPPNet, a multiscale-based deep learning network. Its foundation follows encoder-decoder pair architecture that utilizes the Spatial Pyramid Pooling (SPP) layer to improve the performance of cardiac semantic segmentation. The SPP layer is designed to pool features from densely convolutional layers at different scales or sizes, which will be combined to maintain a set of spatial information. By processing features at different spatial resolutions, the multiscale densely connected layer in the form of the Pyramid Pooling Dense Module (PPDM) helps the network to capture both local and global context, preserving finer details of the cardiac structure while also capturing the broader context required to accurately segment larger cardiac structures. The PPDM is incorporated into the deeper layer of the encoder section of the deep learning network to allow it to recognize complex semantic features.

RESULTS

An analysis of multiple PPDM placement scenarios and structural variations revealed that the 3-path PPDM, positioned at the encoder layer 5, yielded optimal segmentation performance, achieving dice, intersection over union (IoU), and accuracy scores of 0.859, 0.800, and 0.993, respectively.

CONCLUSIONS

Different PPDM configurations produce a different effect on the network; as such, a shallower layer placement, like encoder layer 4, retains more spatial data that need more parallel paths to gather the optimal set of multiscale features. In contrast, deeper layers contain more informative features but at a lower spatial resolution, which reduces the number of parallel paths required to provide optimal multiscale context.

CMR-BENet: A confidence map refinement boundary enhancement network for left ventricular myocardium segmentation

BACKGROUND AND OBJECTIVE

Left ventricular myocardium segmentation is of great significance for clinical diagnosis, treatment, and prognosis. However, myocardium segmentation is challenging as the medical image quality is disturbed by various factors such as motion, artifacts, and noise. Its accuracy largely depends on the accurate identification of edges and structures. Most existing encoder-decoder based segmentation methods capture limited contextual information and ignore the awareness of myocardial shape and structure, often producing unsatisfactory boundary segmentation results in noisy scenes. Moreover, these methods fail to assess the reliability of the predictions, which is crucial for clinical decisions and applications in medical tasks. Therefore, this study explores how to effectively combine contextual information with myocardial edge structure and confidence maps to improve segmentation performance in an end-to-end network.

METHODS

In this paper, we propose an end-to-end confidence map refinement boundary enhancement network (CMR-BENet) for left ventricular myocardium segmentation. CMR-BENet has three components: a layer semantic-aware module (LSA), an edge information enhancement module (EIE), and a confidence map-based refinement module (CMR). Specifically, LSA first adaptively fuses high- and low-level semantic information across hierarchical layers to mitigate the bias of single-layer features affected by noise. EIE then improves the edge and structure recognition by designing the edge and mask guidance module (EMG) and the edge structure-aware module (ESA). Finally, CMR provides a simple and efficient way to estimate confidence maps and effectively combines the encoder features to refine the segmentation results.

RESULTS

Experiments on two echocardiography datasets and one cardiac MRI dataset show that the proposed CMR-BENet outperforms its rivals in the left ventricular myocardium segmentation task with Dice (DI) of 87.71%, 79.33%, and 89.11%, respectively.

CONCLUSION

This paper utilizes edge information to characterize the shape and structure of the myocardium and introduces learnable confidence maps to evaluate and refine the segmentation results. Our findings provide strong support and reference for physicians in diagnosis and treatment.

WFUMB Commentary Paper on Artificial intelligence in Medical Ultrasound Imaging

Artificial intelligence (AI) is defined as the theory and development of computer systems able to perform tasks normally associated with human intelligence. At present, AI has been widely used in a variety of ultrasound tasks, including in point-of-care ultrasound, echocardiography, and various diseases of different organs. However, the characteristics of ultrasound, compared to other imaging modalities, such as computed tomography (CT) and magnetic resonance imaging (MRI), poses significant additional challenges to AI. Application of AI can not only reduce variability during ultrasound image acquisition, but can standardize these interpretations and identify patterns that escape the human eye and brain. These advances have enabled greater innovations in ultrasound AI applications that can be applied to a variety of clinical settings and disease states. Therefore, The World Federation of Ultrasound in Medicine and Biology (WFUMB) is addressing the topic with a brief and practical overview of current and potential future AI applications in medical ultrasound, as well as discuss some current limitations and future challenges to AI implementation.

Deep learning for cardiac imaging: focus on myocardial diseases, a narrative review

The integration of computational technologies into cardiology has significantly advanced the diagnosis and management of cardiovascular diseases. Computational cardiology, particularly, through cardiovascular imaging and informatics, enables a precise diagnosis of myocardial diseases utilizing techniques such as echocardiography, cardiac magnetic resonance imaging, and computed tomography. Early-stage disease classification, especially in asymptomatic patients, benefits from these advancements, potentially altering disease progression and improving patient outcomes. Automatic segmentation of myocardial tissue using deep learning (DL) algorithms improves efficiency and consistency in analyzing large patient populations. Radiomic analysis can reveal subtle disease characteristics from medical images and can enhance disease detection, enable patient stratification, and facilitate monitoring of disease progression and treatment response. Radiomic biomarkers have already demonstrated high diagnostic accuracy in distinguishing myocardial pathologies and promise treatment individualization in cardiology, earlier disease detection, and disease monitoring. In this context, this narrative review explores the current state of the art in DL applications in medical imaging (CT, CMR, echocardiography, and SPECT), focusing on automatic segmentation, radiomic feature phenotyping, and prediction of myocardial diseases, while also discussing challenges in integration of DL models in clinical practice.

Automatic Segmentation of the Left Ventricle in Apical Four-Chamber View on Transesophageal Echocardiography Based on UNeXt Deep Neural Network

Background/Objectives: The automatic left ventricle segmentation in transesophageal echocardiography (TEE) is of significant importance. In this paper, we constructed a large-scale TEE apical four-chamber view (A4CV) image dataset and proposed an automatic left ventricular segmentation method for the TEE A4CV based on the UNeXt deep neural network. Methods: UNeXt, a variant of U-Net integrating a multilayer perceptron, was employed for left ventricle segmentation in the TEE A4CV because it could yield promising segmentation performance while reducing both the number of network parameters and computational complexity. We also compared the proposed method with U-Net, TransUNet, and Attention U-Net models. Standard TEE A4CV videos were collected from 60 patients undergoing cardiac surgery, from the onset of anesthesia to the conclusion of the procedure. After preprocessing, a dataset comprising 3000 TEE images and their corresponding labels was generated. The dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio on the patient level. The training and validation sets were used to train the UNeXt, U-Net, TransUNet, and Attention U-Net models for left ventricular segmentation. The dice similarity coefficient (DSC) and Intersection over Union (IoU) were used to evaluate the segmentation performance of each model, and the Kruskal-Wallis test was employed to analyze the significance of DSC differences. Results: On the test set, the UNeXt model achieved an average DSC of 88.60%, outperforming U-Net (87.76%), TransUNet (85.75%, p < 0.05), and Attention U-Net (79.98%; p < 0.05). Additionally, the UNeXt model had a smaller number of parameters (1.47 million) and floating point operations (2.28 giga) as well as a shorter average inference time per image (141.73 ms), compared to U-Net (185.12 ms), TransUNet (209.08 ms), and Attention U-Net (201.13 ms). The average IoU of UNeXt (77.60%) was also higher than that of U-Net (76.61%), TransUNet (77.35%), and Attention U-Net (68.86%). Conclusions: This study pioneered the construction of a large-scale TEE A4CV dataset and the application of UNeXt to left ventricle segmentation in the TEE A4CV. The proposed method may be used for automatic segmentation of the left ventricle in the TEE A4CV.

Artificial Intelligence Algorithms in Cardiovascular Medicine: An Attainable Promise to Improve Patient Outcomes or an Inaccessible Investment?

PURPOSE OF REVIEW

This opinion paper highlights the advancements in artificial intelligence (AI) technology for cardiovascular disease (CVD), presents best practices and transformative impacts, and addresses current concerns that must be resolved for broader adoption.

RECENT FINDINGS

With the evolution of digitization in data collection, large amounts of data have become available, surpassing the human capacity for processing and analysis, thus enabling the application of AI. These models can learn complex spatial and temporal patterns from large amounts of data, providing patient-specific outputs. These advantages have resulted, at the moment, in more than 900 AI-based devices being approved, today, by regulatory entities, for clinical use, with similar to improved performance and efficiency compared to traditional technologies. However, issues such as model generalization, bias, transparency, interpretability, accountability, and data privacy remain significant barriers for broad adoption of these technologies. AI shows great promise in enhancing CVD care through more accurate and efficient approaches. Yet, widespread adoption is hindered by unresolved concerns of interested stakeholders. Addressing these challenges is crucial for fully integrating AI into clinical practice and shaping the future of CVD prevention, diagnosis and treatment.

Cardiac substructure delineation in radiation therapy - A state-of-the-art review

Delineation of cardiac substructures is crucial for a better understanding of radiation-related cardiotoxicities and to facilitate accurate and precise cardiac dose calculation for developing and applying risk models. This review examines recent advancements in cardiac substructure delineation in the radiation therapy (RT) context, aiming to provide a comprehensive overview of the current level of knowledge, challenges and future directions in this evolving field. Imaging used for RT planning presents challenges in reliably visualising cardiac anatomy. Although cardiac atlases and contouring guidelines aid in standardisation and reduction of variability, significant uncertainties remain in defining cardiac anatomy. Coupled with the inherent complexity of the heart, this necessitates auto-contouring for consistent large-scale data analysis and improved efficiency in prospective applications. Auto-contouring models, developed primarily for breast and lung cancer RT, have demonstrated performance comparable to manual contouring, marking a significant milestone in the evolution of cardiac delineation practices. Nevertheless, several key concerns require further investigation. There is an unmet need for expanding cardiac auto-contouring models to encompass a broader range of cancer sites. A shift in focus is needed from ensuring accuracy to enhancing the robustness and accessibility of auto-contouring models. Addressing these challenges is paramount for the integration of cardiac substructure delineation and associated risk models into routine clinical practice, thereby improving the safety of RT for future cancer patients.

Virtual Reality for Preprocedure Planning of Covered Stent Correction of Superior Sinus Venosus Atrial Septal Defects

BACKGROUND

Covered stent correction (CSC) of a superior sinus venosus atrial septal defect is an alternative to surgery in selected patients, but anatomic variation means that assessment for CSC requires a 3-dimensional anatomic understanding. Heart VR is a virtual reality (VR) system that rapidly displays and renders multimodality imaging without prior image segmentation. The aim of this study was to evaluate the performance of the Heart VR system to assess patient suitability for CSC.

METHODS

In a blinded fashion, 2 interventionalists reviewed preprocedural computed tomography scans using Heart VR to assess the feasibility of CSC, including the potential need for pulmonary vein protection. The total review time using VR was recorded.

RESULTS

Using conventional imaging, 15 patients were deemed suitable for CSC, but at catheterization, 3 cases were unsuitable. Using VR, when both interventionalists agreed that a case was suitable for CSC (n=12), all proved technically feasible. In the 3 cases that were unsuitable for CSC, the interventionalists using VR were either uncertain (n=1) or did not agree on suitability (n=2). The strategy for pulmonary vein protection was correctly identified by interventionalist 1 and 2 in 9/12 and 8/12 cases, respectively. In cases where pulmonary vein protection was required intraprocedurally (n=5), this was correctly identified using Heart VR. Using VR, in 3 cases it was determined that pulmonary vein protection would be required, but this was not the case on balloon interrogation. VR data loading and review times were 82 seconds and 7 minutes, respectively. Verbal feedback indicated that Heart VR assisted in the assessment of case suitability.

CONCLUSIONS

Heart VR is a rapid and effective tool for predicting suitability for CSC in patients with a superior sinus venosus atrial septal defect and could be a feasible alternative to segmented virtual or physical 3-dimensional models.

Automatic pipeline for segmentation of LV myocardium on quantitative MR T1 maps using deep learning model and computation of radial T1 and ECV values

Native T1 mapping is a non-invasive technique used for early detection of diffused myocardial abnormalities, and it provides baseline tissue characterization. Post-contrast T1 mapping enhances tissue differentiation, enables extracellular volume (ECV) calculation, and improves myocardial viability assessment. Accurate and precise segmenting of the left ventricular (LV) myocardium on T1 maps is crucial for assessing myocardial tissue characteristics and diagnosing cardiovascular diseases (CVD). This study presents a deep learning (DL)-based pipeline for automatically segmenting LV myocardium on T1 maps and automatic computation of radial T1 and ECV values. The study employs a multicentric dataset consisting of retrospective multiparametric MRI data of 332 subjects to develop and assess the performance of the proposed method. The study compared DL architectures U-Net and Deep Res U-Net for LV myocardium segmentation, which achieved a dice similarity coefficient of 0.84 ± 0.43 and 0.85 ± 0.03, respectively. The dice similarity coefficients computed for radial sub-segmentation of the LV myocardium on basal, mid-cavity, and apical slices were 0.77 ± 0.21, 0.81 ± 0.17, and 0.61 ± 0.14, respectively. The t-test performed between ground truth vs. predicted values of native T1, post-contrast T1, and ECV showed no statistically significant difference (p > 0.05) for any of the radial sub-segments. The proposed DL method leverages the use of quantitative T1 maps for automatic LV myocardium segmentation and accurately computing radial T1 and ECV values, highlighting its potential for assisting radiologists in objective cardiac assessment and, hence, in CVD diagnostics.

Deep learning for 3D vascular segmentation in hierarchical phase contrast tomography: a case study on kidney

Automated blood vessel segmentation is critical for biomedical image analysis, as vessel morphology changes are associated with numerous pathologies. Still, precise segmentation is difficult due to the complexity of vascular structures, anatomical variations across patients, the scarcity of annotated public datasets, and the quality of images. Our goal is to provide a foundation on the topic and identify a robust baseline model for application to vascular segmentation using a new imaging modality, Hierarchical Phase-Contrast Tomography (HiP-CT). We begin with an extensive review of current machine-learning approaches for vascular segmentation across various organs. Our work introduces a meticulously curated training dataset, verified by double annotators, consisting of vascular data from three kidneys imaged using HiP-CT as part of the Human Organ Atlas Project. HiP-CT pioneered at the European Synchrotron Radiation Facility in 2020, revolutionizes 3D organ imaging by offering a resolution of around 20 μm/voxel and enabling highly detailed localised zooms up to 1-2 μm/voxel without physical sectioning. We leverage the nnU-Net framework to evaluate model performance on this high-resolution dataset, using both known and novel samples, and implementing metrics tailored for vascular structures. Our comprehensive review and empirical analysis on HiP-CT data sets a new standard for evaluating machine learning models in high-resolution organ imaging. Our three experiments yielded Dice similarity coefficient (DSC) scores of 0.9523, 0.9410, and 0.8585, respectively. Nevertheless, DSC primarily assesses voxel-to-voxel concordance, overlooking several crucial characteristics of the vessels and should not be the sole metric for deciding the performance of vascular segmentation. Our results show that while segmentations yielded reasonably high scores-such as centerline DSC ranging from 0.82 to 0.88, certain errors persisted. Specifically, large vessels that collapsed due to the lack of hydrostatic pressure (HiP-CT is an ex vivo technique) were segmented poorly. Moreover, decreased connectivity in finer vessels and higher segmentation errors at vessel boundaries were observed. Such errors, particularly in significant vessels, obstruct the understanding of the structures by interrupting vascular tree connectivity. Our study establishes the benchmark across various evaluation metrics, for vascular segmentation of HiP-CT imaging data, an imaging technology that has the potential to substantively shift our understanding of human vascular networks.

Enhancing Precision in Cardiac Segmentation for Magnetic Resonance-Guided Radiation Therapy Through Deep Learning

PURPOSE

Cardiac substructure dose metrics are more strongly linked to late cardiac morbidities than to whole-heart metrics. Magnetic resonance (MR)-guided radiation therapy (MRgRT) enables substructure visualization during daily localization, allowing potential for enhanced cardiac sparing. We extend a publicly available state-of-the-art deep learning framework, "No New" U-Net, to incorporate self-distillation (nnU-Net.wSD) for substructure segmentation for MRgRT.

METHODS AND MATERIALS

Eighteen (institute A) patients who underwent thoracic or abdominal radiation therapy on a 0.35 T MR-guided linear accelerator were retrospectively evaluated. On each image, 1 of 2 radiation oncologists delineated reference contours of 12 cardiac substructures (chambers, great vessels, and coronary arteries) used to train (n = 10), validate (n = 3), and test (n = 5) nnU-Net.wSD by leveraging a teacher-student network and comparing it to standard 3-dimensional U-Net. The impact of using simulation data or including 3 to 4 daily images for augmentation during training was evaluated for nnU-Net.wSD. Geometric metrics (Dice similarity coefficient, mean distance to agreement, and 95% Hausdorff distance), visual inspection, and clinical dose-volume histograms were evaluated. To determine generalizability, institute A's model was tested on an unlabeled data set from institute B (n = 22) and evaluated via consensus scoring and volume comparisons.

RESULTS

nnU-Net.wSD yielded a Dice similarity coefficient (reported mean ± SD) of 0.65 ± 0.25 across the 12 substructures (chambers, 0.85 ± 0.05; great vessels, 0.67 ± 0.19; and coronary arteries, 0.33 ± 0.16; mean distance to agreement, <3 mm; mean 95% Hausdorff distance, <9 mm) while outperforming the 3-dimensional U-Net (0.583 ± 0.28; P <.01). Leveraging fractionated data for augmentation improved over a single MR simulation time point (0.579 ± 0.29; P <.01). Predicted contours yielded dose-volume histograms that closely matched those of the clinical treatment plans where mean and maximum (ie, dose to 0.03 cc) doses deviated by 0.32 ± 0.5 Gy and 1.42 ± 2.6 Gy, respectively. There were no statistically significant differences between institute A and B volumes (P >.05) for 11 of 12 substructures, with larger volumes requiring minor changes and coronary arteries exhibiting more variability.

CONCLUSIONS

This work is a critical step toward rapid and reliable cardiac substructure segmentation to improve cardiac sparing in low-field MRgRT.

Artificial Intelligence-Driven Advances in Coronary Calcium Scoring: Expanding Preventive Cardiology

Coronary artery disease (CAD) remains a leading global cause of morbidity and mortality, underscoring the need for effective cardiovascular risk stratification and preventive strategies. Coronary artery calcium (CAC) scoring, traditionally performed using electrocardiogram (ECG)-gated cardiac computed tomography (CT) scans, has been widely validated as a robust tool for assessing cardiovascular risk. However, its application has been largely limited to high-risk populations due to the costs, technical requirements, and limited accessibility of cardiac CT scans. Recent advancements in artificial intelligence (AI) have introduced transformative opportunities to extend CAC detection to noncardiac CT scans, such as those performed for lung cancer screening, enabling broader and more accessible cardiovascular screening. This review provides a comprehensive analysis of AI-driven CAC detection, examining various types of AI models for CAC detection, like convolutional neural networks (CNNs) and U-Net architectures, and exploring the clinical, operational, and ethical implications of incorporating these technologies into routine practice. Technical challenges, including imaging variability, data privacy, and model bias, are discussed alongside essential areas for further research, such as standardization and validation across diverse populations. By leveraging widely available imaging data, AI-enabled CAC detection has the potential to advance preventive cardiology, supporting earlier risk identification, optimizing healthcare resources, and improving patient outcomes.

Dynamic-Guided Spatiotemporal Attention for Echocardiography Video Segmentation

Left ventricle (LV) endocardium segmentation in echocardiography video has received much attention as an important step in quantifying LV ejection fraction. Most existing methods are dedicated to exploiting temporal information on top of 2D convolutional networks. In addition to single appearance semantic learning, some research attempted to introduce motion cues through the optical flow estimation (OFE) task to enhance temporal consistency modeling. However, OFE in these methods is tightly coupled to LV endocardium segmentation, resulting in noisy inter-frame flow prediction, and post-optimization based on these flows accumulates errors. To address these drawbacks, we propose dynamic-guided spatiotemporal attention (DSA) for semi-supervised echocardiography video segmentation. We first fine-tune the off-the-shelf OFE network RAFT on echocardiography data to provide dynamic information. Taking inter-frame flows as additional input, we use a dual-encoder structure to extract motion and appearance features separately. Based on the connection between dynamic continuity and semantic consistency, we propose a bilateral feature calibration module to enhance both features. For temporal consistency modeling, the DSA is proposed to aggregate neighboring frame context using deformable attention that is realized by offsets grid attention. Dynamic information is introduced into DSA through a bilateral offset estimation module to effectively combine with appearance semantics and predict attention offsets, thereby guiding semantic-based spatiotemporal attention. We evaluated our method on two popular echocardiography datasets, CAMUS and EchoNet-Dynamic, and achieved state-of-the-art.

Using artificial intelligence to study atherosclerosis from computed tomography imaging: A state-of-the-art review of the current literature

With the enormous progress in the field of cardiovascular imaging in recent years, computed tomography (CT) has become readily available to phenotype atherosclerotic coronary artery disease. New analytical methods using artificial intelligence (AI) enable the analysis of complex phenotypic information of atherosclerotic plaques. In particular, deep learning-based approaches using convolutional neural networks (CNNs) facilitate tasks such as lesion detection, segmentation, and classification. New radiotranscriptomic techniques even capture underlying bio-histochemical processes through higher-order structural analysis of voxels on CT images. In the near future, the international large-scale Oxford Risk Factors And Non-invasive Imaging (ORFAN) study will provide a powerful platform for testing and validating prognostic AI-based models. The goal is the transition of these new approaches from research settings into a clinical workflow. In this review, we present an overview of existing AI-based techniques with focus on imaging biomarkers to determine the degree of coronary inflammation, coronary plaques, and the associated risk. Further, current limitations using AI-based approaches as well as the priorities to address these challenges will be discussed. This will pave the way for an AI-enabled risk assessment tool to detect vulnerable atherosclerotic plaques and to guide treatment strategies for patients.

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

A Deep Learning-Based Method for Rapid 3D Whole-Heart Modeling in Congenital Heart Disease

INTRODUCTION

This study aimed to develop a deep learning-based method for generating three-dimensional heart mesh models for patients with congenital heart disease by integrating medical imaging and clinical diagnostic information.

METHODS

A deep learning model was trained using CT and cardiac MRI, along with clinical data from 110 patients. The Web-based platform automatically outputs STL files for 3D printing and Unity 3D OBJ files for virtual reality (VR) applications upon uploading the medical images and diagnostic information. The models were tested on three congenital heart disease cases, with corresponding 3D-printed and VR heart models generated.

RESULTS

The 3D-printed and VR heart models received high praise from professional doctors for their anatomical accuracy and clarity. Evaluations indicated that the proposed method effectively and rapidly reconstructs complex congenital heart disease structures, proving useful for preoperative planning and diagnostic support.

CONCLUSION

The 3D modeling approach has the potential to enhance the precision of surgical planning and diagnosis for congenital heart disease. Future studies should explore larger datasets and training models for different types of congenital heart disease to validate the model's broad applicability.

Emerging Analytical Approaches for Personalized Medicine Using Machine Learning In Pediatric and Congenital Heart Disease

Precision and personalized medicine, the process by which patient management is tailored to individual circumstances, are now terms that are familiar to cardiologists, despite it still being an emerging field. Although precision medicine relies most often on the underlying biology and pathophysiology of a patient's condition, personalized medicine relies on digital biomarkers generated through algorithms. Given the complexity of the underlying data, these digital biomarkers are most often generated through machine-learning algorithms. There are a number of analytic considerations regarding the creation of digital biomarkers that are discussed in this review, including data preprocessing, time dependency and gating, dimensionality reduction, and novel methods, both in the realm of supervised and unsupervised machine learning. Some of these considerations, such as sample size requirements and measurements of model performance, are particularly challenging in small and heterogeneous populations with rare outcomes such as children with congenital heart disease. Finally, we review analytic considerations for the deployment of digital biomarkers in clinical settings, including the emerging field of clinical artificial intelligence (AI) operations, computational needs for deployment, efforts to increase the explainability of AI, algorithmic drift, and the needs for distributed surveillance and federated learning. We conclude this review by discussing a recent simulation study that shows that, despite these analytic challenges and complications, the use of digital biomarkers in managing clinical care might have substantial benefits regarding individual patient outcomes.

Dehazing Ultrasound Using Diffusion Models

Echocardiography has been a prominent tool for the diagnosis of cardiac disease. However, these diagnoses can be heavily impeded by poor image quality. Acoustic clutter emerges due to multipath reflections imposed by layers of skin, subcutaneous fat, and intercostal muscle between the transducer and heart. As a result, haze and other noise artifacts pose a real challenge to cardiac ultrasound imaging. In many cases, especially with difficult-to-image patients such as patients with obesity, a diagnosis from B-Mode ultrasound imaging is effectively rendered unusable, forcing sonographers to resort to contrast-enhanced ultrasound examinations or refer patients to other imaging modalities. Tissue harmonic imaging has been a popular approach to combat haze, but in severe cases is still heavily impacted by haze. Alternatively, denoising algorithms are typically unable to remove highly structured and correlated noise, such as haze. It remains a challenge to accurately describe the statistical properties of structured haze, and develop an inference method to subsequently remove it. Diffusion models have emerged as powerful generative models and have shown their effectiveness in a variety of inverse problems. In this work, we present a joint posterior sampling framework that combines two separate diffusion models to model the distribution of both clean ultrasound and haze in an unsupervised manner. Furthermore, we demonstrate techniques for effectively training diffusion models on radio-frequency ultrasound data and highlight the advantages over image data. Experiments on both in-vitro and in-vivo cardiac datasets show that the proposed dehazing method effectively removes haze while preserving signals from weakly reflected tissue.

Challenges for augmenting intelligence in cardiac imaging

Artificial Intelligence (AI), through deep learning, has brought automation and predictive capabilities to cardiac imaging. However, despite considerable investment, tangible health-care cost reductions remain unproven. Although AI holds promise, there has been insufficient time for both methodological development and prospective clinical trials to establish its advantage over human interpretations in terms of its effect on patient outcomes. Challenges such as data scarcity, privacy issues, and ethical concerns impede optimal AI training. Furthermore, the absence of a unified model for the complex structure and function of the heart and evolving domain knowledge can introduce heuristic biases and influence underlying assumptions in model development. Integrating AI into diverse institutional picture archiving and communication systems and devices also presents a clinical hurdle. This hurdle is further compounded by an absence of high-quality labelled data, difficulty sharing data between institutions, and non-uniform and inadequate gold standards for external validations and comparisons of model performance in real-world settings. Nevertheless, there is a strong push in industry and academia for AI solutions in medical imaging. This Series paper reviews key studies and identifies challenges that require a pragmatic change in the approach for using AI for cardiac imaging, whereby AI is viewed as augmented intelligence to complement, not replace, human judgement. The focus should shift from isolated measurements to integrating non-linear and complex data towards identifying disease phenotypes-emphasising pattern recognition where AI excels. Algorithms should enhance imaging reports, enriching patients' understanding, communication between patients and clinicians, and shared decision making. The emergence of professional standards and guidelines is essential to address these developments and ensure the safe and effective integration of AI in cardiac imaging.

Deep-Learning-Based Disease Classification in Patients Undergoing Cine Cardiac MRI

BACKGROUND

Automated approaches may allow for fast, reproducible clinical assessment of cardiovascular diseases from MRI.

PURPOSE

To develop an MRI-based deep learning (DL) disease classification algorithm to distinguish among normal subjects (NORM), patients with dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy (HCM), and ischemic heart disease (IHD).

STUDY TYPE

Retrospective.

POPULATION

A total of 1337 subjects (55% female), comprising normal subjects (N = 568), and patients with DCM (N = 151), HCM (N = 177), and IHD (N = 441).

FIELD STRENGTH/SEQUENCE

Balanced steady-state free precession cine sequence at 1.5/3.0 T.

ASSESSMENT

Bi-ventricular morphological and functional features and global and segmental left ventricular strain features were automatically extracted from short- and long-axis cine images. Variational autoencoder models were trained on the extracted features and compared against consensus disease label provided by two expert readers (13 and 14 years of experience). Adding unlabeled, normal data to the training was explored to increase specificity of NORM class.

STATISTICAL TESTS

Tenfold cross-validation for model development; mean, standard deviation (SD) for measurements; classification metrics: area under the curve (AUC), confusion matrix, accuracy, specificity, precision, recall; 95% confidence intervals; Mann-Whitney U test for significance.

RESULTS

AUCs of 0.952 for NORM, 0.881 for DCM, 0.908 for HCM, and 0.856 for IHD and overall accuracy of 0.778 were obtained, with specificity of 0.908 for the NORM class using both SAX and LAX features. Longitudinal strain features slightly improved classification metrics by 0.001 to 0.03 points, except for HCM-AUC. Differences in accuracy, metrics for NORM class and HCM-AUC were statistically significant. Cotraining using unlabeled data increased the specificity for the NORM class to 0.961.

DATA CONCLUSION

Cardiac function features automatically extracted from cine MRI have potential to be used for disease classification, especially for normal-abnormal classification. Feature analyses showed that strain features were important for disease labeling. Cotraining using unlabeled data may help to increase specificity for normal-abnormal classification.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 1.

Automated biventricular quantification in patients with repaired tetralogy of Fallot using a three-dimensional deep learning segmentation model

BACKGROUND

Deep learning is the state-of-the-art approach for automated segmentation of the left ventricle (LV) and right ventricle (RV) in cardiovascular magnetic resonance (CMR) images. However, these models have been mostly trained and validated using CMR datasets of structurally normal hearts or cases with acquired cardiac disease, and are therefore not well-suited to handle cases with congenital cardiac disease such as tetralogy of Fallot (TOF). We aimed to develop and validate a dedicated model with improved performance for LV and RV cavity and myocardium quantification in patients with repaired TOF.

METHODS

We trained a three-dimensional (3D) convolutional neural network (CNN) with 5-fold cross-validation using manually delineated end-diastolic (ED) and end-systolic (ES) short-axis image stacks obtained from either a public dataset containing patients with no or acquired cardiac pathology (n = 100), an institutional dataset of TOF patients (n = 96), or both datasets mixed. Our method allows for missing labels in the training images to accommodate for different ED and ES phases for LV and RV as is commonly the case in TOF. The best performing model was applied to all frames of a separate test set of TOF cases (n = 36) and ED and ES phases were automatically determined for LV and RV separately. The model was evaluated against the performance of a commercial software (suiteHEART®, NeoSoft, Pewaukee, Wisconsin, US).

RESULTS

Training on the mixture of both datasets yielded the best agreement with the manual ground truth for the TOF cases, achieving a median Dice similarity coefficient of (93.8%, 89.8%) for LV cavity and of (92.9%, 90.9%) for RV cavity at (ED, ES) respectively, and of 80.9% and 61.8% for LV and RV myocardium at ED. The offset in automated ED and ES frame selection was 0.56 and 0.89 frames on average for LV and RV respectively. No statistically significant differences were found between our model and the commercial software for LV quantification (two-sided Wilcoxon signed rank test, p<5%), while RV quantification was significantly improved with our model achieving a mean absolute error of 12 ml for RV cavity compared to 36 ml for the commercial software.

CONCLUSION

We developed and validated a fully automatic segmentation and quantification approach for LV and RV, including RV mass, in patients with repaired TOF. Compared to a commercial software, our approach is superior for RV quantification indicating its potential in clinical practice.

Vascular liver segmentation: a narrative review on methods and new insights brought by artificial intelligence

Liver vessel segmentation from routinely performed medical imaging is a useful tool for diagnosis, treatment planning and delivery, and prognosis evaluation for many diseases, particularly liver cancer. A precise representation of liver anatomy is crucial to define the extent of the disease and, when suitable, the consequent resective or ablative procedure, in order to guarantee a radical treatment without sacrificing an excessive volume of healthy liver. Once mainly performed manually, with notable cost in terms of time and human energies, vessel segmentation is currently realized through the application of artificial intelligence (AI), which has gained increased interest and development of the field. Many different AI-driven models adopted for this aim have been described and can be grouped into different categories: thresholding methods, edge- and region-based methods, model-based methods, and machine learning models. The latter includes neural network and deep learning models that now represent the principal algorithms exploited for vessel segmentation. The present narrative review describes how liver vessel segmentation can be realized through AI models, with a summary of model results in terms of accuracy, and an overview on the future progress of this topic.

Evaluation of a deep learning-enabled automated computational heart modelling workflow for personalized assessment of ventricular arrhythmias

Personalized, image-based computational heart modelling is a powerful technology that can be used to improve patient-specific arrhythmia risk stratification and ventricular tachycardia (VT) ablation targeting. However, most state-of-the-art methods still require manual interactions by expert users. The goal of this study is to evaluate the feasibility of an automated, deep learning-based workflow for reconstructing personalized computational electrophysiological heart models to guide patient-specific treatment of VT. Contrast-enhanced computed tomography (CE-CT) images with expert ventricular myocardium segmentations were acquired from 111 patients across five cohorts from three different institutions. A deep convolutional neural network (CNN) for segmenting left ventricular myocardium from CE-CT was developed, trained and evaluated. From both CNN-based and expert segmentations in a subset of patients, personalized electrophysiological heart models were reconstructed and rapid pacing was used to induce VTs. CNN-based and expert segmentations were more concordant in the middle myocardium than in the heart's base or apex. Wavefront propagation during pacing was similar between CNN-based and original heart models. Between most sets of heart models, VT inducibility was the same, the number of induced VTs was strongly correlated, and VT circuits co-localized. Our results demonstrate that personalized computational heart models reconstructed from deep learning-based segmentations even with a small training set size can predict similar VT inducibility and circuit locations as those from expertly-derived heart models. Hence, a user-independent, automated framework for simulating arrhythmias in personalized heart models could feasibly be used in clinical settings to aid VT risk stratification and guide VT ablation therapy. KEY POINTS: Personalized electrophysiological heart modelling can aid in patient-specific ventricular tachycardia (VT) risk stratification and VT ablation targeting. Current state-of-the-art, image-based heart models for VT prediction require expert-dependent, manual interactions that may not be accessible across clinical settings. In this study, we develop an automated, deep learning-based workflow for reconstructing personalized heart models capable of simulating arrhythmias and compare its predictions with that of expert-generated heart models. The number and location of VTs was similar between heart models generated from the deep learning-based workflow and expert-generated heart models. These results demonstrate the feasibility of using an automated computational heart modelling workflow to aid in VT therapeutics and has implications for generalizing personalized computational heart technology to a broad range of clinical centres.

A Semi-supervised Four-Chamber Echocardiographic Video Segmentation Algorithm Based on Multilevel Edge Perception and Calibration Fusion

OBJECTIVE

Echocardiographic videos are commonly used for automatic semantic segmentation of endocardium, which is crucial in evaluating cardiac function and assisting doctors to make accurate diagnoses of heart disease. However, this task faces two distinct challenges: one is the edge blurring, which is caused by the presence of speckle noise or excessive de-noising operation, and the other is the lack of an effective feature fusion approach for multilevel features for obtaining accurate endocardium.

METHODS

In this study, a deep learning model, based on multilevel edge perception and calibration fusion is proposed to improve the segmentation performance. First, a multilevel edge perception module is proposed to comprehensively extract edge features through both a detail branch and a semantic branch to alleviate the adverse impact of noise. Second, a calibration fusion module is proposed that calibrates and integrates various features, including semantic and detailed information, to maximize segmentation performance. Furthermore, the features obtained from the calibration fusion module are stored by using a memory architecture to achieve semi-supervised segmentation through both labeled and unlabeled data.

RESULTS

Our method is evaluated on two public echocardiography video data sets, achieving average Dice coefficients of 93.05% and 93.93%, respectively. Additionally, we validated our method on a local hospital clinical data set, achieving a Pearson correlation of 0.765 for predicting left ventricular ejection fraction.

CONCLUSION

The proposed model effectively solves the challenges encountered in echocardiography by using semi-supervised networks, thereby improving the segmentation accuracy of the ventricles. This indicates that the proposed model can assist cardiologists in obtaining accurate and effective research and diagnostic results.

Random effects during training: Implications for deep learning-based medical image segmentation

BACKGROUND

A single learning algorithm can produce deep learning-based image segmentation models that vary in performance purely due to random effects during training. This study assessed the effect of these random performance fluctuations on the reliability of standard methods of comparing segmentation models.

METHODS

The influence of random effects during training was assessed by running a single learning algorithm (nnU-Net) with 50 different random seeds for three multiclass 3D medical image segmentation problems, including brain tumour, hippocampus, and cardiac segmentation. Recent literature was sampled to find the most common methods for estimating and comparing the performance of deep learning segmentation models. Based on this, segmentation performance was assessed using both hold-out validation and 5-fold cross-validation and the statistical significance of performance differences was measured using the Paired t-test and the Wilcoxon signed rank test on Dice scores.

RESULTS

For the different segmentation problems, the seed producing the highest mean Dice score statistically significantly outperformed between 0 % and 76 % of the remaining seeds when estimating performance using hold-out validation, and between 10 % and 38 % when estimating performance using 5-fold cross-validation.

CONCLUSION

Random effects during training can cause high rates of statistically-significant performance differences between segmentation models from the same learning algorithm. Whilst statistical testing is widely used in contemporary literature, our results indicate that a statistically-significant difference in segmentation performance is a weak and unreliable indicator of a true performance difference between two learning algorithms.

The role of artificial intelligence in cardiovascular magnetic resonance imaging

Cardiovascular magnetic resonance (CMR) imaging is the gold standard test for myocardial tissue characterization and chamber volumetric and functional evaluation. However, manual CMR analysis can be time-consuming and is subject to intra- and inter-observer variability. Artificial intelligence (AI) is a field that permits automated task performance through the identification of high-level and complex data relationships. In this review, we review the rapidly growing role of AI in CMR, including image acquisition, sequence prescription, artifact detection, reconstruction, segmentation, and data reporting and analysis including quantification of volumes, function, myocardial infarction (MI) and scar detection, and prediction of outcomes. We conclude with a discussion of the emerging challenges to widespread adoption and solutions that will allow for successful, broader uptake of this powerful technology.

PFMNet: Prototype-based feature mapping network for few-shot domain adaptation in medical image segmentation

Lack of data is one of the biggest hurdles for rare disease research using deep learning. Due to the lack of rare-disease images and annotations, training a robust network for automatic rare-disease image segmentation is very challenging. To address this challenge, few-shot domain adaptation (FSDA) has emerged as a practical research direction, aiming to leverage a limited number of annotated images from a target domain to facilitate adaptation of models trained on other large datasets in a source domain. In this paper, we present a novel prototype-based feature mapping network (PFMNet) designed for FSDA in medical image segmentation. PFMNet adopts an encoder-decoder structure for segmentation, with the prototype-based feature mapping (PFM) module positioned at the bottom of the encoder-decoder structure. The PFM module transforms high-level features from the target domain into the source domain-like features that are more easily comprehensible by the decoder. By leveraging these source domain-like features, the decoder can effectively perform few-shot segmentation in the target domain and generate accurate segmentation masks. We evaluate the performance of PFMNet through experiments on three typical yet challenging few-shot medical image segmentation tasks: cross-center optic disc/cup segmentation, cross-center polyp segmentation, and cross-modality cardiac structure segmentation. We consider four different settings: 5-shot, 10-shot, 15-shot, and 20-shot. The experimental results substantiate the efficacy of our proposed approach for few-shot domain adaptation in medical image segmentation.

Assessing aortic motion with automated 3D cine balanced steady state free precession cardiovascular magnetic resonance segmentation

PURPOSE

To apply a free-running three-dimensional (3D) cine balanced steady state free precession (bSSFP) cardiovascular magnetic resonance (CMR) framework in combination with artificial intelligence (AI) segmentations to quantify time-resolved aortic displacement, diameter and diameter change.

METHODS

In this prospective study, we implemented a free-running 3D cine bSSFP sequence with scan time of approximately 4 min facilitated by pseudo-spiral Cartesian undersampling and compressed-sensing reconstruction. Automated segmentation of the aorta in all cardiac timeframes was applied through the use of nnU-Net. Dynamic 3D motion maps were created for three repeated scans per volunteer, leading to the detailed quantification of aortic motion, as well as the measurement and change in diameter of the ascending aorta.

RESULTS

A total of 14 adult healthy volunteers (median age, 28 years (interquartile range [IQR]: 26.0-31.3), 6 females) were included. Automated segmentation compared to manual segmentation of the aorta test set showed a Dice score of 0.93 ± 0.02. The median (IQR) over all volunteers for the largest maximum and mean ascending aorta (AAo) displacement in the first scan was 13.0 (4.4) mm and 5.6 (2.4) mm, respectively. Peak mean diameter in the AAo was 25.9 (2.2) mm and peak mean diameter change was 1.4 (0.5) mm. The maximum individual variability over the three repeated scans of maximum and mean AAo displacement was 3.9 (1.6) mm and 2.2 (0.8) mm, respectively. The maximum individual variability of mean diameter and diameter change were 1.2 (0.5) mm and 0.9 (0.4) mm.

CONCLUSION

A free-running 3D cine bSSFP CMR scan with a scan time of four minutes combined with an automated nnU-net segmentation consistently captured the aorta's cardiac motion-related 4D displacement, diameter, and diameter change.

A Shape-Consistent Deep-Learning Segmentation Architecture for Low-Quality and High-Interference Myocardial Contrast Echocardiography

OBJECTIVE

Myocardial contrast echocardiography (MCE) plays a crucial role in diagnosing ischemia, infarction, masses and other cardiac conditions. In the realm of MCE image analysis, accurate and consistent myocardial segmentation results are essential for enabling automated analysis of various heart diseases. However, current manual diagnostic methods in MCE suffer from poor repeatability and limited clinical applicability. MCE images often exhibit low quality and high noise due to the instability of ultrasound signals, while interference structures can further disrupt segmentation consistency.

METHODS

To overcome these challenges, we proposed a deep-learning network for the segmentation of MCE. This architecture leverages dilated convolutions to capture high-scale information without sacrificing positional accuracy and modifies multi-head self-attention to enhance global context and ensure consistency, effectively overcoming issues related to low image quality and interference. Furthermore, we also adapted the cascade application of transformers with convolutional neural networks for improved segmentation in MCE.

RESULTS

In our experiments, our architecture achieved the best Dice score of 84.35% for standard MCE views compared with that of several state-of-the-art segmentation models. For non-standard views and frames with interfering structures (mass), our models also attained the best Dice scores of 83.33% and 83.97%, respectively.

CONCLUSION

These studies proved that our architecture is of excellent shape consistency and robustness, which allows it to deal with segmentation of various types of MCE. Our relatively precise and consistent myocardial segmentation results provide fundamental conditions for the automated analysis of various heart diseases, with the potential to discover underlying pathological features and reduce healthcare costs.

Application of Artificial Intelligence in Cardiology: A Bibliometric Analysis

Recent advancements in artificial intelligence (AI) applications in medicine have been significant over the past 30 years. To monitor current research developments, it is crucial to examine the latest trends in AI adoption across various medical fields. This bibliometric analysis focuses on AI applications in cardiology. Unlike existing literature reviews, this study specifically examines journal articles published in the last decade, sourced from both Scopus and Web of Science databases, to illustrate the recent trends in AI within cardiology. The bibliometric analysis involves a statistical and quantitative evaluation of the literature on AI application in cardiovascular medicine over a defined period. A comprehensive global literature review is conducted to identify key research areas, authors, and their interrelationships through published works. The leading institutions and most influential authors in research on the role of AI in cardiology were located in the United States, the United Kingdom, and China. This study also provides researchers with an overview of the evolution of research in AI and cardiology. The main contribution of this study is to highlight the prominent authors, countries, journals, institutions, keywords, and trends in the development of AI in cardiology.

Heart and great vessels segmentation in congenital heart disease via CNN and conditioned energy function postprocessing

PURPOSE

The segmentation of the heart and great vessels in CT images of congenital heart disease (CHD) is critical for the clinical assessment of cardiac anomalies and the diagnosis of CHD. However, the diverse types and abnormalities inherent in CHD present significant challenges to comprehensive heart segmentation.

METHODS

We proposed a novel two-stage segmentation approach, integrating a Convolutional Neural Network (CNN) with a postprocessing method with conditioned energy function for pulmonary and aorta. The initial stage employs a CNN enhanced by a gated self-attention mechanism for the segmentation of five primary heart structures and two major vessels. Subsequently, the second stage utilizes a conditioned energy function specifically tailored to refine the segmentation of the pulmonary artery and aorta, ensuring vascular continuity.

RESULTS

Our method was evaluated on a public dataset including 110 3D CT volumes, encompassing 16 CHD variants. Compared to prevailing segmentation techniques (U-Net, V-Net, Unetr, dynUnet), our approach demonstrated improvements of 1.02, 1.04, and 1.41% in Dice Coefficient (DSC), Intersection over Union (IOU), and the 95th percentile Hausdorff Distance (HD95), respectively, for heart structure segmentation. For the two great vessels, the enhancements were 1.05, 1.07, and 1.42% in these metrics.

CONCLUSION

The outcomes on the public dataset affirm the efficacy of our proposed segmentation method. Precise segmentation of the entire heart and great vessels can significantly aid in the diagnosis and treatment of CHD, underscoring the clinical relevance of our findings.

Unenhanced computed tomography as a diagnostic tool in suspected pulmonary hypertension: a retrospective cross-sectional pilot study

Background

Computed tomography pulmonary angiography (CTPA) has been proposed to be diagnostic for pulmonary hypertension (PH) in multiple studies. However, the utility of the unenhanced CT measurements diagnosing PH has not been fully assessed. This study aimed to assess the diagnostic utility and reproducibility of cardiac and great vessel parameters on unenhanced computed tomography (CT) in suspected pulmonary hypertension (PH).

Methods

In total, 42 patients with suspected PH who underwent unenhanced CT thorax and right heart catheterization (RHC) were included in the study. Three observers (a consultant radiologist, a specialist registrar in radiology, and a medical student) measured the parameters by using unenhanced CT. Diagnostic accuracy of the parameters was assessed by area under the receiver operating characteristic curve (AUC). Inter-observer variability between the consultant radiologist (primary observer) and the two secondary observers was determined by intra-class correlation analysis (ICC).

Results

Overall, 35 patients were diagnosed with PH by RHC while 7 patients were not. Main pulmonary arterial (MPA) diameter was the strongest (AUC 0.79 to 0.87) and the most reproducible great vessel parameter. ICC comparing the MPA diameter measurement of the consultant radiologist to the specialist registrar's and the medical student's were 0.96 and 0.92, respectively. Right atrial area was the cardiac measurement with highest accuracy and reproducibility (AUC 0.76 to 0.79; ICC 0.980, 0.950) followed by tricuspid annulus diameter (AUC 0.76 to 0.79; ICC 0.790, 0.800).

Conclusions

MPA diameter and right atrial areas showed high reproducibility. Diagnostic accuracies of these were within the range of acceptable to excellent, and might have clinical value. Tricuspid annular diameter was less reliable and less diagnostic and was therefore not a recommended diagnostic measurement.

Novel Domain Knowledge-Encoding Algorithm Enables Label-Efficient Deep Learning for Cardiac CT Segmentation to Guide Atrial Fibrillation Treatment in a Pilot Dataset

Background: Segmenting computed tomography (CT) is crucial in various clinical applications, such as tailoring personalized cardiac ablation for managing cardiac arrhythmias. Automating segmentation through machine learning (ML) is hindered by the necessity for large, labeled training data, which can be challenging to obtain. This article proposes a novel approach for automated, robust labeling using domain knowledge to achieve high-performance segmentation by ML from a small training set. The approach, the domain knowledge-encoding (DOKEN) algorithm, reduces the reliance on large training datasets by encoding cardiac geometry while automatically labeling the training set. The method was validated in a hold-out dataset of CT results from an atrial fibrillation (AF) ablation study. Methods: The DOKEN algorithm parses left atrial (LA) structures, extracts "anatomical knowledge" by leveraging digital LA models (available publicly), and then applies this knowledge to achieve high ML segmentation performance with a small number of training samples. The DOKEN-labeled training set was used to train a nnU-Net deep neural network (DNN) model for segmenting cardiac CT in N = 20 patients. Subsequently, the method was tested in a hold-out set with N = 100 patients (five times larger than training set) who underwent AF ablation. Results: The DOKEN algorithm integrated with the nn-Unet model achieved high segmentation performance with few training samples, with a training to test ratio of 1:5. The Dice score of the DOKEN-enhanced model was 96.7% (IQR: 95.3% to 97.7%), with a median error in surface distance of boundaries of 1.51 mm (IQR: 0.72 to 3.12) and a mean centroid-boundary distance of 1.16 mm (95% CI: -4.57 to 6.89), similar to expert results (r = 0.99; p < 0.001). In digital hearts, the novel DOKEN approach segmented the LA structures with a mean difference for the centroid-boundary distances of -0.27 mm (95% CI: -3.87 to 3.33; r = 0.99; p < 0.0001). Conclusions: The proposed novel domain knowledge-encoding algorithm was able to perform the segmentation of six substructures of the LA, reducing the need for large training data sets. The combination of domain knowledge encoding and a machine learning approach could reduce the dependence of ML on large training datasets and could potentially be applied to AF ablation procedures and extended in the future to other imaging, 3D printing, and data science applications.

Deep Learning for 3D Vascular Segmentation in Phase Contrast Tomography

Automated blood vessel segmentation is critical for biomedical image analysis, as vessel morphology changes are associated with numerous pathologies. Still, precise segmentation is difficult due to the complexity of vascular structures, anatomical variations across patients, the scarcity of annotated public datasets, and the quality of images. Our goal is to provide a foundation on the topic and identify a robust baseline model for application to vascular segmentation using a new imaging modality, Hierarchical Phase-Contrast Tomography (HiP-CT). We begin with an extensive review of current machine learning approaches for vascular segmentation across various organs. Our work introduces a meticulously curated training dataset, verified by double annotators, consisting of vascular data from three kidneys imaged using Hierarchical Phase-Contrast Tomography (HiP-CT) as part of the Human Organ Atlas Project. HiP-CT, pioneered at the European Synchrotron Radiation Facility in 2020, revolutionizes 3D organ imaging by offering resolution around 20μm/voxel, and enabling highly detailed localized zooms up to 1μm/voxel without physical sectioning. We leverage the nnU-Net framework to evaluate model performance on this high-resolution dataset, using both known and novel samples, and implementing metrics tailored for vascular structures. Our comprehensive review and empirical analysis on HiP-CT data sets a new standard for evaluating machine learning models in high-resolution organ imaging. Our three experiments yielded Dice scores of 0.9523 and 0.9410, and 0.8585, respectively. Nevertheless, DSC primarily assesses voxel-to-voxel concordance, overlooking several crucial characteristics of the vessels and should not be the sole metric for deciding the performance of vascular segmentation. Our results show that while segmentations yielded reasonably high scores-such as centerline Dice values ranging from 0.82 to 0.88, certain errors persisted. Specifically, large vessels that collapsed due to the lack of hydro-static pressure (HiP-CT is an ex vivo technique) were segmented poorly. Moreover, decreased connectivity in finer vessels and higher segmentation errors at vessel boundaries were observed. Such errors, particularly in significant vessels, obstruct the understanding of the structures by interrupting vascular tree connectivity. Through our review and outputs, we aim to set a benchmark for subsequent model evaluations using various modalities, especially with the HiP-CT imaging database.

CapNet: An Automatic Attention-Based with Mixer Model for Cardiovascular Magnetic Resonance Image Segmentation

Deep neural networks have shown excellent performance in medical image segmentation, especially for cardiac images. Transformer-based models, though having advantages over convolutional neural networks due to the ability of long-range dependence learning, still have shortcomings such as having a large number of parameters and and high computational cost. Additionally, for better results, they are often pretrained on a larger data, thus requiring large memory size and increasing resource expenses. In this study, we propose a new lightweight but efficient model, namely CapNet, based on convolutions and mixing modules for cardiac segmentation from magnetic resonance images (MRI) that can be trained from scratch with a small amount of parameters. To handle varying sizes and shapes which often occur in cardiac systolic and diastolic phases, we propose attention modules for pooling, spatial, and channel information. We also propose a novel loss called the Tversky Shape Power Distance function based on the shape dissimilarity between labels and predictions that shows promising performances compared to other losses. Experiments on three public datasets including ACDC benchmark, Sunnybrook data, and MS-CMR challenge are conducted and compared with other state of the arts (SOTA). For binary segmentation, the proposed CapNet obtained the Dice similarity coefficient (DSC) of 94% and 95.93% for respectively the Endocardium and Epicardium regions with Sunnybrook dataset, 94.49% for Endocardium, and 96.82% for Epicardium with the ACDC data. Regarding the multiclass case, the average DSC by CapNet is 93.05% for the ACDC data; and the DSC scores for the MS-CMR are 94.59%, 92.22%, and 93.99% for respectively the bSSFP, T2-SPAIR, and LGE sequences of the MS-CMR. Moreover, the statistical significance analysis tests with p-value < 0.05 compared with transformer-based methods and some CNN-based approaches demonstrated that the CapNet, though having fewer training parameters, is statistically significant. The promising evaluation metrics show comparative results in both Dice and IoU indices compared to SOTA CNN-based and Transformer-based architectures.

Motion-Compensated MR CINE Reconstruction With Reconstruction-Driven Motion Estimation

In cardiac CINE, motion-compensated MR reconstruction (MCMR) is an effective approach to address highly undersampled acquisitions by incorporating motion information between frames. In this work, we propose a novel perspective for addressing the MCMR problem and a more integrated and efficient solution to the MCMR field. Contrary to state-of-the-art (SOTA) MCMR methods which break the original problem into two sub-optimization problems, i.e. motion estimation and reconstruction, we formulate this problem as a single entity with one single optimization. Our approach is unique in that the motion estimation is directly driven by the ultimate goal, reconstruction, but not by the canonical motion-warping loss (similarity measurement between motion-warped images and target images). We align the objectives of motion estimation and reconstruction, eliminating the drawbacks of artifacts-affected motion estimation and therefore error-propagated reconstruction. Further, we can deliver high-quality reconstruction and realistic motion without applying any regularization/smoothness loss terms, circumventing the non-trivial weighting factor tuning. We evaluate our method on two datasets: 1) an in-house acquired 2D CINE dataset for the retrospective study and 2) the public OCMR cardiac dataset for the prospective study. The conducted experiments indicate that the proposed MCMR framework can deliver artifact-free motion estimation and high-quality MR images even for imaging accelerations up to 20x, outperforming SOTA non-MCMR and MCMR methods in both qualitative and quantitative evaluation across all experiments. The code is available at https://github.com/JZPeterPan/MCMR-Recon-Driven-Motion.

The beating heart: artificial intelligence for cardiovascular application in the clinic

Artificial intelligence (AI) integration in cardiac magnetic resonance imaging presents new and exciting avenues for advancing patient care, automating post-processing tasks, and enhancing diagnostic precision and outcomes. The use of AI significantly streamlines the examination workflow through the reduction of acquisition and postprocessing durations, coupled with the automation of scan planning and acquisition parameters selection. This has led to a notable improvement in examination workflow efficiency, a reduction in operator variability, and an enhancement in overall image quality. Importantly, AI unlocks new possibilities to achieve spatial resolutions that were previously unattainable in patients. Furthermore, the potential for low-dose and contrast-agent-free imaging represents a stride toward safer and more patient-friendly diagnostic procedures. Beyond these benefits, AI facilitates precise risk stratification and prognosis evaluation by adeptly analysing extensive datasets. This comprehensive review article explores recent applications of AI in the realm of cardiac magnetic resonance imaging, offering insights into its transformative potential in the field.

Automated assessment of cardiac dynamics in aging and dilated cardiomyopathy Drosophila models using machine learning

The Drosophila model is pivotal in deciphering the pathophysiological underpinnings of various human ailments, notably aging and cardiovascular diseases. Cutting-edge imaging techniques and physiology yield vast high-resolution videos, demanding advanced analysis methods. Our platform leverages deep learning to segment optical microscopy images of Drosophila hearts, enabling the quantification of cardiac parameters in aging and dilated cardiomyopathy (DCM). Validation using experimental datasets confirms the efficacy of our aging model. We employ two innovative approaches deep-learning video classification and machine-learning based on cardiac parameters to predict fly aging, achieving accuracies of 83.3% (AUC 0.90) and 79.1%, (AUC 0.87) respectively. Moreover, we extend our deep-learning methodology to assess cardiac dysfunction associated with the knock-down of oxoglutarate dehydrogenase (OGDH), revealing its potential in studying DCM. This versatile approach promises accelerated cardiac assays for modeling various human diseases in Drosophila and holds promise for application in animal and human cardiac physiology under diverse conditions.

A Motion-Aware DNN Model with Edge Focus Loss and Quality Control for Short-Axis Left Ventricle Segmentation of Cine MR Sequences

Accurate segmentation of the left ventricle myocardium is the key step of automatic assessment of cardiac function. However, the current methods mainly focus on the end-diastolic and the end-systolic frames in cine MR sequences and lack the attention to myocardial motion in the cardiac cycle. Additionally, due to the lack of fine segmentation tools, the simplified approach, excluding papillary muscles and trabeculae from myocardium, is applied in clinical practice. To solve these problems, we propose a motion-aware DNN model with edge focus loss and quality control in this paper. Specifically, the bidirectional ConvLSTM layer and a new motion attention layer are proposed to encode motion-aware feature maps, and an edge focus loss function is proposed to train the model to generate the fine segmentation results. Additionally, a quality control method is proposed to filter out the abnormal segmentations before subsequent analyses. Compared with state-of-the-art segmentation models on the public dataset and the in-house dataset, the proposed method has obtained high segmentation accuracy. On the 17-segment model, the proposed method has obtained the highest Pearson correlation coefficient at 14 of 17 segments, and the mean PCC of 85%. The experimental results highlight the segmentation accuracy of the proposed method as well as its availability to substitute for the manually annotated boundaries for the automatic assessment of cardiac function.

A Dual Enrichment Synergistic Strategy to Handle Data Heterogeneity for Domain Incremental Cardiac Segmentation

Upon remarkable progress in cardiac image segmentation, contemporary studies dedicate to further upgrading model functionality toward perfection, through progressively exploring the sequentially delivered datasets over time by domain incremental learning. Existing works mainly concentrated on addressing the heterogeneous style variations, but overlooked the critical shape variations across domains hidden behind the sub-disease composition discrepancy. In case the updated model catastrophically forgets the sub-diseases that were learned in past domains but are no longer present in the subsequent domains, we proposed a dual enrichment synergistic strategy to incrementally broaden model competence for a growing number of sub-diseases. The data-enriched scheme aims to diversify the shape composition of current training data via displacement-aware shape encoding and decoding, to gradually build up the robustness against cross-domain shape variations. Meanwhile, the model-enriched scheme intends to strengthen model capabilities by progressively appending and consolidating the latest expertise into a dynamically-expanded multi-expert network, to gradually cultivate the generalization ability over style-variated domains. The above two schemes work in synergy to collaboratively upgrade model capabilities in two-pronged manners. We have extensively evaluated our network with the ACDC and M&Ms datasets in single-domain and compound-domain incremental learning settings. Our approach outperformed other competing methods and achieved comparable results to the upper bound.

Image2Flow: A proof-of-concept hybrid image and graph convolutional neural network for rapid patient-specific pulmonary artery segmentation and CFD flow field calculation from 3D cardiac MRI data

Computational fluid dynamics (CFD) can be used for non-invasive evaluation of hemodynamics. However, its routine use is limited by labor-intensive manual segmentation, CFD mesh creation, and time-consuming simulation. This study aims to train a deep learning model to both generate patient-specific volume-meshes of the pulmonary artery from 3D cardiac MRI data and directly estimate CFD flow fields. This proof-of-concept study used 135 3D cardiac MRIs from both a public and private dataset. The pulmonary arteries in the MRIs were manually segmented and converted into volume-meshes. CFD simulations were performed on ground truth meshes and interpolated onto point-point correspondent meshes to create the ground truth dataset. The dataset was split 110/10/15 for training, validation, and testing. Image2Flow, a hybrid image and graph convolutional neural network, was trained to transform a pulmonary artery template to patient-specific anatomy and CFD values, taking a specific inlet velocity as an additional input. Image2Flow was evaluated in terms of segmentation, and the accuracy of predicted CFD was assessed using node-wise comparisons. In addition, the ability of Image2Flow to respond to increasing inlet velocities was also evaluated. Image2Flow achieved excellent segmentation accuracy with a median Dice score of 0.91 (IQR: 0.86-0.92). The median node-wise normalized absolute error for pressure and velocity magnitude was 11.75% (IQR: 9.60-15.30%) and 9.90% (IQR: 8.47-11.90), respectively. Image2Flow also showed an expected response to increased inlet velocities with increasing pressure and velocity values. This proof-of-concept study has shown that it is possible to simultaneously perform patient-specific volume-mesh based segmentation and pressure and flow field estimation using Image2Flow. Image2Flow completes segmentation and CFD in ~330ms, which is ~5000 times faster than manual methods, making it more feasible in a clinical environment.

Immunotherapy efficacy prediction through a feature re-calibrated 2.5D neural network

BACKGROUND AND OBJECTIVE

Lung cancer continues to be a leading cause of cancer-related mortality worldwide, with immunotherapy emerging as a promising therapeutic strategy for advanced non-small cell lung cancer (NSCLC). Despite its potential, not all patients experience benefits from immunotherapy, and the current biomarkers used for treatment selection possess inherent limitations. As a result, the implementation of imaging-based biomarkers to predict the efficacy of lung cancer treatments offers a promising avenue for improving therapeutic outcomes.

METHODS

This study presents an automatic system for immunotherapy efficacy prediction on the subjects with lung cancer, facilitating significant clinical implications. Our model employs an advanced 2.5D neural network that incorporates 2D intra-slice feature extraction and 3D inter-slice feature aggregation. We further present a lesion-focused prior to guide the re-calibration for intra-slice features, and a attention-based re-calibration for the inter-slice features. Finally, we design an accumulated back-propagation strategy to optimize network parameters in a memory-efficient fashion.

RESULTS

We demonstrate that the proposed method achieves impressive performance on an in-house clinical dataset, surpassing existing state-of-the-art models. Furthermore, the proposed model exhibits increased efficiency in inference for each subject on average. To further validate the effectiveness of our model and its components, we conducted comprehensive and in-depth ablation experiments and discussions.

CONCLUSION

The proposed model showcases the potential to enhance physicians' diagnostic performance due to its impressive performance in predicting immunotherapy efficacy, thereby offering significant clinical application value. Moreover, we conduct adequate comparison experiments of the proposed methods and existing advanced models. These findings contribute to our understanding of the proposed model's effectiveness and serve as motivation for future work in immunotherapy efficacy prediction.

Exploring UMAP in hybrid models of entropy-based and representativeness sampling for active learning in biomedical segmentation

In this work, we study various hybrid models of entropy-based and representativeness sampling techniques in the context of active learning in medical segmentation, in particular examining the role of UMAP (Uniform Manifold Approximation and Projection) as a technique for capturing representativeness. Although UMAP has been shown viable as a general purpose dimension reduction method in diverse areas, its role in deep learning-based medical segmentation has yet been extensively explored. Using the cardiac and prostate datasets in the Medical Segmentation Decathlon for validation, we found that a novel hybrid combination of Entropy-UMAP sampling technique achieved a statistically significant Dice score advantage over the random baseline (3.2% for cardiac, 4.5% for prostate), and attained the highest Dice coefficient among the spectrum of 10 distinct active learning methodologies we examined. This provides preliminary evidence that there is an interesting synergy between entropy-based and UMAP methods when the former precedes the latter in a hybrid model of active learning.

Cardiac MRI segmentation using shifted-window multilayer perceptron mixer networks

Objectives. In this work, we proposed a deep-learning segmentation algorithm for cardiac magnetic resonance imaging to aid in contouring of the left ventricle, right ventricle, and Myocardium (Myo).Approach.We proposed a shifted window multilayer perceptron (Swin-MLP) mixer network which is built upon a 3D U-shaped symmetric encoder-decoder structure. We evaluated our proposed network using public data from 100 individuals. The network performance was quantitatively evaluated using 3D volume similarity between the ground truth contours and the predictions using Dice score coefficient, sensitivity, and precision as well as 2D surface similarity using Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMSD). We benchmarked the performance against two other current leading edge networks known as Dynamic UNet and Swin-UNetr on the same public dataset.Results.The proposed network achieved the following volume similarity metrics when averaged over three cardiac segments: Dice = 0.952 ± 0.017, precision = 0.948 ± 0.016, sensitivity = 0.956 ± 0.022. The average surface similarities were HD = 1.521 ± 0.121 mm, MSD = 0.266 ± 0.075 mm, and RMSD = 0.668 ± 0.288 mm. The network shows statistically significant improvement in comparison to the Dynamic UNet and Swin-UNetr algorithms for most volumetric and surface metrics withp-value less than 0.05. Overall, the proposed Swin-MLP mixer network demonstrates better or comparable performance than competing methods.Significance.The proposed Swin-MLP mixer network demonstrates more accurate segmentation performance compared to current leading edge methods. This robust method demonstrates the potential to streamline clinical workflows for multiple applications.

Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation

Precise delineation of multiple organs or abnormal regions in the human body from medical images plays an essential role in computer-aided diagnosis, surgical simulation, image-guided interventions, and especially in radiotherapy treatment planning. Thus, it is of great significance to explore automatic segmentation approaches, among which deep learning-based approaches have evolved rapidly and witnessed remarkable progress in multi-organ segmentation. However, obtaining an appropriately sized and fine-grained annotated dataset of multiple organs is extremely hard and expensive. Such scarce annotation limits the development of high-performance multi-organ segmentation models but promotes many annotation-efficient learning paradigms. Among these, studies on transfer learning leveraging external datasets, semi-supervised learning including unannotated datasets and partially-supervised learning integrating partially-labeled datasets have led the dominant way to break such dilemmas in multi-organ segmentation. We first review the fully supervised method, then present a comprehensive and systematic elaboration of the 3 abovementioned learning paradigms in the context of multi-organ segmentation from both technical and methodological perspectives, and finally summarize their challenges and future trends.

Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography

Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.

Masked autoencoders with generalizable self-distillation for skin lesion segmentation

In the field of skin lesion image segmentation, accurate identification and partitioning of diseased regions is of vital importance for in-depth analysis of skin cancer. Self-supervised learning, i.e., MAE, has emerged as a potent force in the medical imaging domain, which autonomously learns and extracts latent features from unlabeled data, thereby yielding pre-trained models that greatly assist downstream tasks. To encourage pre-trained models to more comprehensively learn the global structural and local detail information inherent in dermoscopy images, we introduce a Teacher-Student architecture, named TEDMAE, by incorporating a self-distillation mechanism, it learns holistic image feature information to improve the generalizable global knowledge learning of the student MAE model. To make the image features learned by the model suitable for unknown test images, two optimization strategies are, Exterior Conversion Augmentation (EC) utilizes random convolutional kernels and linear interpolation to effectively transform the input image into one with the same shape but altered intensities and textures, while Dynamic Feature Generation (DF) employs a nonlinear attention mechanism for feature merging, enhancing the expressive power of the features, are proposed to enhance the generalizability of global features learned by the teacher model, thereby improving the overall generalization capability of the pre-trained models. Experimental results from the three public skin disease datasets, ISIC2019, ISIC2017, and PH2 indicate that our proposed TEDMAE method outperforms several similar approaches. Specifically, TEDMAE demonstrated optimal segmentation and generalization performance on the ISIC2017 and PH2 datasets, with Dice scores reaching 82.1% and 91.2%, respectively. The best Jaccard values were 72.6% and 84.5%, while the optimal HD95% values were 13.0% and 8.9%, respectively.