Cardiac MRI segmentation using shifted-window multilayer perceptron mixer networks

Objectives. In this work, we proposed a deep-learning segmentation algorithm for cardiac magnetic resonance imaging to aid in contouring of the left ventricle, right ventricle, and Myocardium (Myo).Approach.We proposed a shifted window multilayer perceptron (Swin-MLP) mixer network which is built upon a 3D U-shaped symmetric encoder-decoder structure. We evaluated our proposed network using public data from 100 individuals. The network performance was quantitatively evaluated using 3D volume similarity between the ground truth contours and the predictions using Dice score coefficient, sensitivity, and precision as well as 2D surface similarity using Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMSD). We benchmarked the performance against two other current leading edge networks known as Dynamic UNet and Swin-UNetr on the same public dataset.Results.The proposed network achieved the following volume similarity metrics when averaged over three cardiac segments: Dice = 0.952 ± 0.017, precision = 0.948 ± 0.016, sensitivity = 0.956 ± 0.022. The average surface similarities were HD = 1.521 ± 0.121 mm, MSD = 0.266 ± 0.075 mm, and RMSD = 0.668 ± 0.288 mm. The network shows statistically significant improvement in comparison to the Dynamic UNet and Swin-UNetr algorithms for most volumetric and surface metrics withp-value less than 0.05. Overall, the proposed Swin-MLP mixer network demonstrates better or comparable performance than competing methods.Significance.The proposed Swin-MLP mixer network demonstrates more accurate segmentation performance compared to current leading edge methods. This robust method demonstrates the potential to streamline clinical workflows for multiple applications.

Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging

Background

Segmentation of cardiac structures is an important step in evaluation of the heart on imaging. There has been growing interest in how artificial intelligence (AI) methods-particularly deep learning (DL)-can be used to automate this process. Existing AI approaches to cardiac segmentation have mostly focused on cardiac MRI. This systematic review aimed to appraise the performance and quality of supervised DL tools for the segmentation of cardiac structures on CT.

Methods

Embase and Medline databases were searched to identify related studies from January 1, 2013 to December 4, 2023. Original research studies published in peer-reviewed journals after January 1, 2013 were eligible for inclusion if they presented supervised DL-based tools for the segmentation of cardiac structures and non-coronary great vessels on CT. The data extracted from eligible studies included information about cardiac structure(s) being segmented, study location, DL architectures and reported performance metrics such as the Dice similarity coefficient (DSC). The quality of the included studies was assessed using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM).

Results

18 studies published after 2020 were included. The DSC scores median achieved for the most commonly segmented structures were left atrium (0.88, IQR 0.83-0.91), left ventricle (0.91, IQR 0.89-0.94), left ventricle myocardium (0.83, IQR 0.82-0.92), right atrium (0.88, IQR 0.83-0.90), right ventricle (0.91, IQR 0.85-0.92), and pulmonary artery (0.92, IQR 0.87-0.93). Compliance of studies with CLAIM was variable. In particular, only 58% of studies showed compliance with dataset description criteria and most of the studies did not test or validate their models on external data (81%).

Conclusion

Supervised DL has been applied to the segmentation of various cardiac structures on CT. Most showed similar performance as measured by DSC values. Existing studies have been limited by the size and nature of the training datasets, inconsistent descriptions of ground truth annotations and lack of testing in external data or clinical settings.

Systematic Review Registration

[www.crd.york.ac.uk/prospero/], PROSPERO [CRD42023431113].

Deep Learning Paradigm and Its Bias for Coronary Artery Wall Segmentation in Intravascular Ultrasound Scans: A Closer Look

BACKGROUND AND MOTIVATION

Coronary artery disease (CAD) has the highest mortality rate; therefore, its diagnosis is vital. Intravascular ultrasound (IVUS) is a high-resolution imaging solution that can image coronary arteries, but the diagnosis software via wall segmentation and quantification has been evolving. In this study, a deep learning (DL) paradigm was explored along with its bias.

METHODS

Using a PRISMA model, 145 best UNet-based and non-UNet-based methods for wall segmentation were selected and analyzed for their characteristics and scientific and clinical validation. This study computed the coronary wall thickness by estimating the inner and outer borders of the coronary artery IVUS cross-sectional scans. Further, the review explored the bias in the DL system for the first time when it comes to wall segmentation in IVUS scans. Three bias methods, namely (i) ranking, (ii) radial, and (iii) regional area, were applied and compared using a Venn diagram. Finally, the study presented explainable AI (XAI) paradigms in the DL framework.

FINDINGS AND CONCLUSIONS

UNet provides a powerful paradigm for the segmentation of coronary walls in IVUS scans due to its ability to extract automated features at different scales in encoders, reconstruct the segmented image using decoders, and embed the variants in skip connections. Most of the research was hampered by a lack of motivation for XAI and pruned AI (PAI) models. None of the UNet models met the criteria for bias-free design. For clinical assessment and settings, it is necessary to move from a paper-to-practice approach.

Artificial intelligence in cardiac computed tomography

Artificial Intelligence (AI) is a broad discipline of computer science and engineering. Modern application of AI encompasses intelligent models and algorithms for automated data analysis and processing, data generation, and prediction with applications in visual perception, speech understanding, and language translation. AI in healthcare uses machine learning (ML) and other predictive analytical techniques to help sort through vast amounts of data and generate outputs that aid in diagnosis, clinical decision support, workflow automation, and prognostication. Coronary computed tomography angiography (CCTA) is an ideal union for these applications due to vast amounts of data generation and analysis during cardiac segmentation, coronary calcium scoring, plaque quantification, adipose tissue quantification, peri-operative planning, fractional flow reserve quantification, and cardiac event prediction. In the past 5 years, there has been an exponential increase in the number of studies exploring the use of AI for cardiac computed tomography (CT) image acquisition, de-noising, analysis, and prognosis. Beyond image processing, AI has also been applied to improve the imaging workflow in areas such as patient scheduling, urgent result notification, report generation, and report communication. In this review, we discuss algorithms applicable to AI and radiomic analysis; we then present a summary of current and emerging clinical applications of AI in cardiac CT. We conclude with AI's advantages and limitations in this new field.

An automatic segmentation method with self-attention mechanism on left ventricle in gated PET/CT myocardial perfusion imaging

OBJECTIVES

We aimed to propose an automatic segmentation method for left ventricular (LV) from 16 electrocardiogram (ECG) -gated ¹³N-NH₃ PET/CT myocardial perfusion imaging (MPI) to improve the performance of LV function assessment.

METHODS

Ninety-six cases with confirmed or suspected obstructive coronary artery disease (CAD) were enrolled in this research. The LV myocardial contours were delineated by physicians as ground truth. We developed an automatic segmentation method, which introduces the self-attention mechanism into 3D U-Net to capture global information of images so as to achieve fine segmentation of LV. Three cross-validation tests were performed on each gate (64 vs. 32 for training vs. validation). The effectiveness was validated by quantitative metrics (modified hausdorff distance, MHD; dice ratio, DR; 3D MHD) as well as cardiac functional parameters (end-systolic volume, ESV; end-diastolic volume, EDV; ejection fraction, EF). Furthermore, the feasibility of the proposed method was also evaluated by intra- and inter-observers with DR and 3D-MHD.

RESULTS

Compared with backbone network, the proposed approach improved the average DR from 0.905 ± 0.0193 to 0.9202 ± 0.0164, and decreased the average 3D MHD from 0.4611 ± 0.0349 to 0.4304 ± 0.0339. The average relative error of LV volume between proposed method and ground truth is 1.09±3.66%, and the correlation coefficient is 0.992 ± 0.007 (P < 0.001). The EDV, ESV, EF deduced from the proposed approach were highly correlated with ground truth (r ≥ 0.864, P < 0.001), and the correlation with commercial software is fair (r ≥ 0.871, P < 0.001). DR and 3D MHD of contours and myocardium from two observers are higher than 0.899 and less than 0.5194.

CONCLUSION

The proposed approach is highly feasible for automatic segmentation of the LV cavity and myocardium, with potential to benefit the precision of LV function assessment.

Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping Review

Machine-learning (ML) and deep-learning (DL) algorithms are part of a group of modeling algorithms that grasp the hidden patterns in data based on a training process, enabling them to extract complex information from the input data. In the past decade, these algorithms have been increasingly used for image processing, specifically in the medical domain. Cardiothoracic imaging is one of the early adopters of ML/DL research, and the COVID-19 pandemic resulted in more research focus on the feasibility and applications of ML/DL in cardiothoracic imaging. In this scoping review, we systematically searched available peer-reviewed medical literature on cardiothoracic imaging and quantitatively extracted key data elements in order to get a big picture of how ML/DL have been used in the rapidly evolving cardiothoracic imaging field. During this report, we provide insights on different applications of ML/DL and some nuances pertaining to this specific field of research. Finally, we provide general suggestions on how researchers can make their research more than just a proof-of-concept and move toward clinical adoption.

RPS-Net: An effective retinal image projection segmentation network for retinal vessels and foveal avascular zone based on OCTA data

BACKGROUND

Optical coherence tomography angiography (OCTA) is an advanced imaging technology that can present the three-dimensional (3D) structure of retinal vessels (RVs). Quantitative analysis of retinal vessel density and foveal avascular zone (FAZ) area is of great significance in clinical diagnosis and the automatic semantic segmentation at the pixel level helps quantitative analysis. The existing segmentation methods cannot effectively use the volume data and projection map data of the OCTA image at the same time and lack the trade-off between global perception and local details, which lead to problems such as discontinuity of segmentation results and deviation of morphological estimation.

PURPOSE

In order to better assist physicians in clinical diagnosis and treatment, the segmentation accuracy of RVs and FAZ needs to be further improved. In this work, we propose an effective retinal image projection segmentation network (RPS-Net) to achieve accurate RVs and FAZ segmentation. Experiments show that this network exhibits good performance and outperforms other existing methods.

METHODS

Our method considers three aspects. First, we use two parallel projection paths to learn global perceptual features and local supplementary details. Secondly, we use the dual-way projection learning module (DPLM) to reduce the depth of the 3D data and learn image spatial features. Finally, we merged the two-dimensional features learned from the volume data with the two-dimensional projection data, and used a U-shaped network to further learn and generate the final result.

RESULTS

We validated our model on the OCTA-500, which is a large multi-modal, multi-task retinal dataset. The experimental results showed that our method achieved state-of-the-art performance, the mean Dice coefficients for RVs are 89.89 ± 2.60 (%) and 91.40 ± 9.18 (%) on the two subsets, while the Dice coefficients for FAZ are 91.55 ± 2.05 (%) and 97.80 ± 2.75 (%), respectively.

CONCLUSIONS

Our method can make full use of the information of 3D data and 2D data to generate segmented images with higher continuity and accuracy. Code is available at https://github.com/hchuanZ/MFFN/tree/master. This article is protected by copyright. All rights reserved.

Attention-guided duplex adversarial U-net for pancreatic segmentation from computed tomography images

Purpose

Segmenting the organs from computed tomography (CT) images is crucial to early diagnosis and treatment. Pancreas segmentation is especially challenging because the pancreas has a small volume and a large variation in shape.

Methods

To mitigate this issue, an attention‐guided duplex adversarial U‐Net (ADAU‐Net) for pancreas segmentation is proposed in this work. First, two adversarial networks are integrated into the baseline U‐Net to ensure the obtained prediction maps resemble the ground truths. Then, attention blocks are applied to preserve much contextual information for segmentation. The implementation of the proposed ADAU‐Net consists of two steps: 1) backbone segmentor selection scheme is introduced to select an optimal backbone segmentor from three two‐dimensional segmentation model variants based on a conventional U‐Net and 2) attention blocks are integrated into the backbone segmentor at several locations to enhance the interdependency among pixels for a better segmentation performance, and the optimal structure is selected as a final version.

Results

The experimental results on the National Institutes of Health Pancreas‐CT dataset show that our proposed ADAU‐Net outperforms the baseline segmentation network by 6.39% in dice similarity coefficient and obtains a competitive performance compared with the‐state‐of‐art methods for pancreas segmentation.

Conclusion

The ADAU‐Net achieves satisfactory segmentation results on the public pancreas dataset, indicating that the proposed model can segment pancreas outlines from CT images accurately.

Correlation between quantification of myocardial area at risk and ischemic burden at cardiac computed tomography

Purpose

This study aims to investigate the correlation between myocardial area at risk at coronary computed tomography angiography (CCTA) and the ischemic burden derived from myocardial computed tomography perfusion (CTP) by using the 17-segment model.

Methods

Forty-two patients with chest pain complaints who underwent a combined CCTA and CTP protocol were identified. Patients with reversible ischemia at CTP and at least one stenosis of ≥ 50% at CCTA were selected. Myocardial area at risk was calculated using a Voronoi-based segmentation algorithm at CCTA and was defined as the sum of all territories related to a ≥ 50% stenosis as a percentage of the total left ventricular (LV) mass. The latter was calculated using LV contours which were automatically drawn using a machine learning algorithm. Subsequently, the ischemic burden was defined as the number of segments demonstrating relative hypoperfusion as a percentage of the total amount of segments (=17). Finally, correlations were tested between the myocardial area at risk and the ischemic burden using Pearson’s correlation coefficient.

Results

A total of 77 coronary lesions were assessed. Average myocardial area at risk and ischemic burden for all lesions was 59% and 23%, respectively. Correlations for ≥ 50% and ≥ 70% stenosis based myocardial area at risk compared to ischemic burden were moderate (r = 0.564; p < 0.01) and good (r = 0.708; p < 0.01), respectively.

Conclusion

The relation between myocardial area at risk as calculated by using a Voronoi-based algorithm at CCTA and ischemic burden as assessed by CTP is dependent on stenosis severity.

Artificial intelligence in cardiovascular CT: Current status and future implications

Artificial intelligence (AI) refers to the use of computational techniques to mimic human thought processes and learning capacity. The past decade has seen a rapid proliferation of AI developments for cardiovascular computed tomography (CT). These algorithms aim to increase efficiency, objectivity, and performance in clinical tasks such as image quality improvement, structure segmentation, quantitative measurements, and outcome prediction. By doing so, AI has the potential to streamline clinical workflow, increase interpretative speed and accuracy, and inform subsequent clinical pathways. This review covers state-of-the-art AI techniques in cardiovascular CT and the future role of AI as a clinical support tool.

Comparison of three U-Net family architectures for left ventricular myocardial wall automatic segmentation

Left ventricular (LV) segmentation is an important process which can provide quantitative clinical measurements such as volume, wall thickness and ejection fraction. The development of an automatic LV segmentation procedure is a challenging and complicated task mainly due to the variation of the heart shape from patient to patient, especially for those with pathological and physiological changes. In this study, we focus on the implementation, evaluation and comparison of three different Deep Learning architectures of the U-Net family: the custom 2-D U-Net, the ResU-Net++ and the DenseU-Net, in order to segment the LV myocardial wall. Our approach was applied to cardiac CT datasets specifically derived from patients with hypertrophic cardiomyopathy. The results of the models demonstrated high performance in the segmentation process with minor losses. The model revealed a dice score for U-Net, Res-U-net++ and Dense U-Net, 0.81, 0.82 and 0.84, respectively.

Automatic segmentation of organs-at-risks of nasopharynx cancer and lung cancer by cross-layer attention fusion network with TELD-Loss

PURPOSE

Radiotherapy is one of the main treatments of nasopharyngeal cancer (NPC) and lung cancer. Accurate segmentation of organs at risks (OARs) in CT images is a key step in radiotherapy planning for NPC and lung cancer. However, the segmentation of OARs is influenced by the highly imbalanced size of organs, which often results in very poor segmentation results for small and difficult-to-segment organs. In addition, the complex morphological changes and fuzzy boundaries of OARs also pose great challenges to the segmentation task. In this paper, we propose a cross-layer attention fusion network (CLAF-CNN) to solve the problem of accurately segmenting OARs.

METHODS

In CLAF-CNN, we integrate the spatial attention maps of the adjacent spatial attention modules to make the segmentation targets more accurately focused, so that the network can capture more target-related features. In this way, the spatial attention modules in the network can be learned and optimized together. In addition, we introduce a new Top-K exponential logarithmic Dice loss (TELD-Loss) to solve the imbalance problem in OAR segmentation. The TELD-Loss further introduces a Top-K optimization mechanism based on Dice loss and exponential logarithmic loss, which makes the network pay more attention to small organs and difficult-to-segment organs, so as to enhance the overall performance of the segmentation model.

RESULTS

We validated our framework on the OAR segmentation datasets of the head and neck and lung CT images in the StructSeg 2019 challenge. Experiments show that the CLAF-CNN outperforms the state-of-the-art attention-based segmentation methods in the OAR segmentation task with average Dice coefficient of 79.65% for head and neck OARs and 88.39% for lung OARs.

CONCLUSIONS

This work provides a new network named CLAF-CNN which contains cross-layer spatial attention map fusion architecture and TELD-Loss for OAR segmentation. Results demonstrated that the proposed method could obtain accurate segmentation results for OARs, which has a potential of improving the efficiency of radiotherapy planning for nasopharynx cancer and lung cancer.

A review of deep learning based methods for medical image multi-organ segmentation

Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.

An 8-layer residual U-Net with deep supervision for segmentation of the left ventricle in cardiac CT angiography

BACKGROUND AND OBJECTIVES

Accurate segmentation of left ventricle (LV) is a fundamental step in evaluation of cardiac function. Cardiac CT angiography (CCTA) has become an important clinical diagnostic method for cardio-vascular disease (CVD) due to its non-invasive, short exam time, and low cost. To obtain the segmentation of the LV in CCTA scans, we present a deep learning method based on an 8-layer residual U-Net with deep supervision.

METHODS

Based on the original 4-layer U-Net, our method deepened the network to eight layers, which increased the fitting capacity of the network, thus greatly improved its LV recognition capability. Residual blocks were incorporated to optimize the network from the increased depth. Auxiliary paths as deep supervision were introduced to supervise the intermediate information to improve the segmentation quality. In this study, we collected CCTA scans of 100 patients. Eighty patients with 1600 discrete slices were used to train the LV segmentation and the remaining 20 patients with 400 discrete slices were used for testing our method. An interactive graph cut algorithm was utilized reliably to annotate the LV reference standard that was further confirmed by cardiologists. Online data augmentation was performed in the training process to improve the generalization and robustness of our method.

RESULTS

Compared with the segmentation results from the original U-Net and FC-DenseNet56 with Dice similarity coefficient (DSC) of 0.878±0.230 and 0.897±0.189, respectively, our method demonstrated higher segmentation accuracy and robustness for varying LV shape, size, and contrast, achieving DSC of 0.927±0.139. Without online data augmentation, our method resulted in inferior performance with DSC of 0.911±0.170. In addition, compared with the provided results from other existing studies in the LV segmentation of cardiac CT images, our method achieved a competitive performance for the LV segmentation.

CONCLUSIONS

The proposed 8-layer residual U-Net with deep supervision accurately and efficiently segments the LV in CCTA scans. This method has potential advantages to be a reliable segmentation method and useful for the evaluation of cardiac function in the future study.

Thyroid gland delineation in noncontrast-enhanced CTs using deep convolutional neural networks

The purpose of this study is to develop a deep learning method for thyroid delineation with high accuracy, efficiency, and robustness in noncontrast-enhanced head and neck CTs. The cross-sectional analysis consisted of six tests, including randomized cross-validation and hold-out experiments, tests of prediction accuracy between cancer and benign and cross-gender analysis were performed to evaluate the proposed deep-learning-based performance method. CT images of 1977 patients with suspected thyroid carcinoma were retrospectively investigated. The automatically segmented thyroid gland volume was compared against physician-approved clinical contours using metrics, the Pearson correlation and Bland-Altman analysis. Quantitative metrics included: the Dice similarity coefficient (DSC), sensitivity, specificity, Jaccard index (JAC), Hausdorff distance (HD), mean surface distance (MSD), residual mean square distance (RMSD) and the center of mass distance (CMD). The robustness of the proposed method was further tested using the nonparametric Kruskal-Wallis test to assess the equality of distribution of DSC values. The proposed method's accuracy remained high through all the tests, with the median DSC, JAC, sensitivity and specificity higher than 0.913, 0.839, 0.856 and 0.979, respectively. The proposed method also resulted in median MSD, RMSD, HD and CMD, of less than 0.31 mm, 0.48 mm, 2.06 mm and 0.50 mm, respectively. The MSD and RMSD were 0.40 ± 0.29 mm and 0.70 ± 0.46 mm, respectively. Concurrent testing of the proposed method with 3D U-Net and V-Net showed that the proposed method had significantly improved performance. The proposed deep-learning method achieved accurate and robust performance through six cross-sectional analysis tests.

MAD-UNet: A deep U-shaped network combined with an attention mechanism for pancreas segmentation in CT images

PURPOSE

Pancreas segmentation is a difficult task because of the high intrapatient variability in the shape, size, and location of the organ, as well as the low contrast and small footprint of the CT scan. At present, the U-Net model is likely to lead to the problems of intraclass inconsistency and interclass indistinction in pancreas segmentation. To solve this problem, we improved the contextual and semantic feature information acquisition method of the biomedical image segmentation model (U-Net) based on a convolutional network and proposed an improved segmentation model called the multiscale attention dense residual U-shaped network (MAD-UNet).

METHODS

There are two aspects considered in this method. First, we adopted dense residual blocks and weighted binary cross-entropy to enhance the semantic features to learn the details of the pancreas. Using such an approach can reduce the effects of intraclass inconsistency. Second, we used an attention mechanism and multiscale convolution to enrich the contextual information and suppress learning in unrelated areas. We let the model be more sensitive to pancreatic marginal information and reduced the impact of interclass indistinction.

RESULTS

We evaluated our model using fourfold cross-validation on 82 abdominal enhanced three-dimensional (3D) CT scans from the National Institutes of Health (NIH-82) and 281 3D CT scans from the 2018 MICCAI segmentation decathlon challenge (MSD). The experimental results showed that our method achieved state-of-the-art performance on the two pancreatic datasets. The mean Dice coefficients were 86.10% ± 3.52% and 88.50% ± 3.70%.

CONCLUSIONS

Our model can effectively solve the problems of intraclass inconsistency and interclass indistinction in the segmentation of the pancreas, and it has value in clinical application. Code is available at https://github.com/Mrqins/pancreas-segmentation.