A deep-learning semantic segmentation approach to fully automated MRI-based left-ventricular deformation analysis in cardiotoxicity

The role of artificial intelligence in cardiovascular magnetic resonance imaging

Cardiovascular magnetic resonance (CMR) imaging is the gold standard test for myocardial tissue characterization and chamber volumetric and functional evaluation. However, manual CMR analysis can be time-consuming and is subject to intra- and inter-observer variability. Artificial intelligence (AI) is a field that permits automated task performance through the identification of high-level and complex data relationships. In this review, we review the rapidly growing role of AI in CMR, including image acquisition, sequence prescription, artifact detection, reconstruction, segmentation, and data reporting and analysis including quantification of volumes, function, myocardial infarction (MI) and scar detection, and prediction of outcomes. We conclude with a discussion of the emerging challenges to widespread adoption and solutions that will allow for successful, broader uptake of this powerful technology.

Machine Learning in Cardio-Oncology: New Insights from an Emerging Discipline

A growing body of evidence on a wide spectrum of adverse cardiac events following oncologic therapies has led to the emergence of cardio-oncology as an increasingly relevant interdisciplinary specialty. This also calls for better risk-stratification for patients undergoing cancer treatment. Machine learning (ML), a popular branch discipline of artificial intelligence that tackles complex big data problems by identifying interaction patterns among variables, has seen increasing usage in cardio-oncology studies for risk stratification. The objective of this comprehensive review is to outline the application of ML approaches in cardio-oncology, including deep learning, artificial neural networks, random forest and summarize the cardiotoxicity identified by ML. The current literature shows that ML has been applied for the prediction, diagnosis and treatment of cardiotoxicity in cancer patients. In addition, role of ML in gender and racial disparities for cardiac outcomes and potential future directions of cardio-oncology are discussed. It is essential to establish dedicated multidisciplinary teams in the hospital and educate medical professionals to become familiar and proficient in ML in the future.

Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.

Can global longitudinal strain (GLS) with magnetic resonance prognosticate early cancer therapy-related cardiac dysfunction (CTRCD) in breast cancer patients, a prospective study?

PURPOSE

To determine if Artificial Intelligence-based computation of global longitudinal strain (GLS) from left ventricular (LV) MRI is an early prognostic factor of cancer therapy-related cardiac dysfunction (CTRCD) in breast cancer patients. The main hypothesis based on the patients receiving antineoplastic chemotherapy treatment was CTRCD risk analysis with GLS that was independent of LV ejection fraction (LVEF).

METHODS

Displacement Encoding with Stimulated Echoes (DENSE) MRI was acquired on 32 breast cancer patients at baseline and 3- and 6-month follow-ups after chemotherapy. Two DeepLabV3+ Fully Convolutional Networks (FCNs) were deployed to automate image segmentation for LV chamber quantification and phase-unwrapping for 3D strains, computed with the Radial Point Interpolation Method. CTRCD risk (cardiotoxicity and adverse cardiac events) was analyzed with Cox Proportional Hazards (PH) models with clinical and contractile prognostic factors.

RESULTS

GLS worsened from baseline to the 3- and 6-month follow-ups (-19.1 ± 2.1%, -16.0 ± 3.1%, -16.1 ± 3.0%; P < 0.001). Univariable Cox regression showed the 3-month GLS significantly associated as an agonist (hazard ratio [HR]-per-SD: 2.1; 95% CI: 1.4-3.1; P < 0.001) and LVEF as a protector (HR-per-SD: 0.8; 95% CI: 0.7-0.9; P = 0.001) for CTRCD occurrence. Bivariable regression showed the 3-month GLS (HR-per-SD: 2.0; 95% CI: 1.2-3.4; P = 0.01) as a CTRCD prognostic factor independent of other covariates, including LVEF (HR-per-SD: 1.0; 95% CI: 0.9-1.2; P = 0.9).

CONCLUSIONS

The end-point analyses proved the hypothesis that GLS is an early, independent prognosticator of incident CTRCD risk. This novel GLS-guided approach to CTRCD risk analysis could improve antineoplastic treatment with further validation in a larger clinical trial.

Automated segmentation of long and short axis DENSE cardiovascular magnetic resonance for myocardial strain analysis using spatio-temporal convolutional neural networks

BACKGROUND

Cine Displacement Encoding with Stimulated Echoes (DENSE) facilitates the quantification of myocardial deformation, by encoding tissue displacements in the cardiovascular magnetic resonance (CMR) image phase, from which myocardial strain can be estimated with high accuracy and reproducibility. Current methods for analyzing DENSE images still heavily rely on user input, making this process time-consuming and subject to inter-observer variability. The present study sought to develop a spatio-temporal deep learning model for segmentation of the left-ventricular (LV) myocardium, as spatial networks often fail due to contrast-related properties of DENSE images.

METHODS

2D + time nnU-Net-based models have been trained to segment the LV myocardium from DENSE magnitude data in short- and long-axis images. A dataset of 360 short-axis and 124 long-axis slices was used to train the networks, from a combination of healthy subjects and patients with various conditions (hypertrophic and dilated cardiomyopathy, myocardial infarction, myocarditis). Segmentation performance was evaluated using ground-truth manual labels, and a strain analysis using conventional methods was performed to assess strain agreement with manual segmentation. Additional validation was performed using an externally acquired dataset to compare the inter- and intra-scanner reproducibility with respect to conventional methods.

RESULTS

Spatio-temporal models gave consistent segmentation performance throughout the cine sequence, while 2D architectures often failed to segment end-diastolic frames due to the limited blood-to-myocardium contrast. Our models achieved a DICE score of 0.83 ± 0.05 and a Hausdorff distance of 4.0 ± 1.1 mm for short-axis segmentation, and 0.82 ± 0.03 and 7.9 ± 3.9 mm respectively for long-axis segmentations. Strain measurements obtained from automatically estimated myocardial contours showed good to excellent agreement with manual pipelines, and remained within the limits of inter-user variability estimated in previous studies.

CONCLUSION

Spatio-temporal deep learning shows increased robustness for the segmentation of cine DENSE images. It provides excellent agreement with manual segmentation for strain extraction. Deep learning will facilitate the analysis of DENSE data, bringing it one step closer to clinical routine.

Mutually communicated model based on multi-parametric MRI for automated segmentation and classification of prostate cancer

BACKGROUND

Multiparametric magnetic resonance imaging (mp-MRI) is introduced and established as a noninvasive alternative for prostate cancer (PCa) detection and characterization.

PURPOSE

To develop and evaluate a mutually communicated deep learning segmentation and classification network (MC-DSCN) based on mp-MRI for prostate segmentation and PCa diagnosis.

METHODS

The proposed MC-DSCN can transfer mutual information between segmentation and classification components and facilitate each other in a bootstrapping way. For classification task, the MC-DSCN can transfer the masks produced by the coarse segmentation component to the classification component to exclude irrelevant regions and facilitate classification. For segmentation task, this model can transfer the high-quality localization information learned by the classification component to the fine segmentation component to mitigate the impact of inaccurate localization on segmentation results. Consecutive MRI exams of patients were retrospectively collected from two medical centers (referred to as center A and B). Two experienced radiologists segmented the prostate regions, and the ground truth of the classification refers to the prostate biopsy results. MC-DSCN was designed, trained, and validated using different combinations of distinct MRI sequences as input (e.g., T2-weighted and apparent diffusion coefficient) and the effect of different architectures on the network's performance was tested and discussed. Data from center A were used for training, validation, and internal testing, while another center's data were used for external testing. The statistical analysis is performed to evaluate the performance of the MC-DSCN. The DeLong test and paired t-test were used to assess the performance of classification and segmentation, respectively.

RESULTS

In total, 134 patients were included. The proposed MC-DSCN outperforms the networks that were designed solely for segmentation or classification. Regarding the segmentation task, the classification localization information helped to improve the IOU in center A: from 84.5% to 87.8% (p < 0.01) and in center B: from 83.8% to 87.1% (p < 0.01), while the area under curve (AUC) of PCa classification was improved in center A: from 0.946 to 0.991 (p < 0.02) and in center B: from 0.926 to 0.955 (p < 0.01) as a result of the additional information provided by the prostate segmentation.

CONCLUSION

The proposed architecture could effectively transfer mutual information between segmentation and classification components and facilitate each other in a bootstrapping way, thus outperforming the networks designed to perform only one task.

Automated extraction of pod phenotype data from micro-computed tomography

INTRODUCTION

Plant image datasets have the potential to greatly improve our understanding of the phenotypic response of plants to environmental and genetic factors. However, manual data extraction from such datasets are known to be time-consuming and resource intensive. Therefore, the development of efficient and reliable machine learning methods for extracting phenotype data from plant imagery is crucial.

METHODS

In this paper, a current gold standard computed vision method for detecting and segmenting objects in three-dimensional imagery (StartDist-3D) is applied to X-ray micro-computed tomography scans of oilseed rape (Brassica napus) mature pods.

RESULTS

With a relatively minimal training effort, this fine-tuned StarDist-3D model accurately detected (Validation F1-score = 96.3%,Testing F1-score = 99.3%) and predicted the shape (mean matched score = 90%) of seeds.

DISCUSSION

This method then allowed rapid extraction of data on the number, size, shape, seed spacing and seed location in specific valves that can be integrated into models of plant development or crop yield. Additionally, the fine-tuned StarDist-3D provides an efficient way to create a dataset of segmented images of individual seeds that could be used to further explore the factors affecting seed development, abortion and maturation synchrony within the pod. There is also potential for the fine-tuned Stardist-3D method to be applied to imagery of seeds from other plant species, as well as imagery of similarly shaped plant structures such as beans or wheat grains, provided the structures targeted for detection and segmentation can be described as star-convex polygons.

Improved U-Net based on cross-layer connection for pituitary adenoma MRI image segmentation

Pituitary adenoma is a common neuroendocrine neoplasm, and most of its MR images are characterized by blurred edges, high noise and similar to surrounding normal tissues. Therefore, it is extremely difficult to accurately locate and outline the lesion of pituitary adenoma. To sovle these limitations, we design a novel deep learning framework for pituitary adenoma MRI image segmentation. Under the framework of U-Net, a newly cross-layer connection is introduced to capture richer multi-scale features and contextual information. At the same time, full-scale skip structure can reasonably utilize the above information obtained by different layers. In addition, an improved inception-dense block is designed to replace the classical convolution layer, which can enlarge the effectiveness of the receiving field and increase the depth of our network. Finally, a novel loss function based on binary cross-entropy and Jaccard losses is utilized to eliminate the problem of small samples and unbalanced data. The sample data were collected from 30 patients in Quzhou People's Hospital, with a total of 500 lesion images. Experimental results show that although the amount of patient sample is small, the proposed method has better performance in pituitary adenoma image compared with existing algorithms, and its Dice, Intersection over Union (IoU), Matthews correlation coefficient (Mcc) and precision reach 88.87, 80.67, 88.91 and 97.63%, respectively.

Automated Detection of Corneal Ulcer Using Combination Image Processing and Deep Learning

A corneal ulcers are one of the most common eye diseases. They come from various infections, such as bacteria, viruses, or parasites. They may lead to ocular morbidity and visual disability. Therefore, early detection can reduce the probability of reaching the visually impaired. One of the most common techniques exploited for corneal ulcer screening is slit-lamp images. This paper proposes two highly accurate automated systems to localize the corneal ulcer region. The designed approaches are image processing techniques with Hough transform and deep learning approaches. The two methods are validated and tested on the publicly available SUSTech-SYSU database. The accuracy is evaluated and compared between both systems. Both systems achieve an accuracy of more than 90%. However, the deep learning approach is more accurate than the traditional image processing techniques. It reaches 98.9% accuracy and Dice similarity 99.3%. However, the first method does not require parameters to optimize an explicit training model. The two approaches can perform well in the medical field. Moreover, the first model has more leverage than the deep learning model because the last one needs a large training dataset to build reliable software in clinics. Both proposed methods help physicians in corneal ulcer level assessment and improve treatment efficiency.

Diagnosis of fetal total anomalous pulmonary venous connection based on the post-left atrium space ratio using artificial intelligence

OBJECTIVE

To explore whether the post-left atrium space (PLAS) ratio would be useful for prenatal diagnosis of total anomalous pulmonary venous connection (TAPVC) using echocardiography and artificial intelligence.

METHODS

We retrospectively included 642 frames of four-chamber view from 319 fetuses (32 with TAPVC and 287 without TAPVC) in end-systolic and end-diastolic periods with multiple apex directions. The average gestational age was 25.6±2.7 weeks. No other cardiac or extracardiac malformations were observed. The dataset was divided into a training set (n=540; 48 with TAPVC and 492 without TAPVC) and test set (n=102; 20 with TAPVC and 82 without TAPVC). The PLAS ratio was defined as the ratio of the epicardium-descending aortic distance to the center of the heart-descending aortic distance. Supervised learning was used in DeepLabv3+, FastFCN, PSPNet, and DenseASPP segmentation models. The area under the curve (AUC) was used on the test set.

RESULTS

Expert annotations showed that this ratio was not related to the period or apex direction. It was higher in the TAPVC group than in the control group detected by expert and the four models. The AUC of expert annotations, DeepLabv3+, FastFCN, PSPNet, and DenseASPP were 0.977, 0.941, 0.925, 0.856, and 0.887, respectively.

CONCLUSION

Segmentation models achieve good diagnostic accuracy for TAPVC based on the PLAS ratio. This article is protected by copyright. All rights reserved.

Deep learning-based automatic segmentation of images in cardiac radiography: A promising challenge

BACKGROUND

Due to the advancement of medical imaging and computer technology, machine intelligence to analyze clinical image data increases the probability of disease prevention and successful treatment. When diagnosing and detecting heart disease, medical imaging can provide high-resolution scans of every organ or tissue in the heart. The diagnostic results obtained by the imaging method are less susceptible to human interference. They can process numerous patient information, assist doctors in early detection of heart disease, intervene and treat patients, and improve the understanding of heart disease symptoms and clinical diagnosis of great significance. In a computer-aided diagnosis system, accurate segmentation of cardiac scan images is the basis and premise of subsequent thoracic function analysis and 3D image reconstruction.

EXISTING TECHNIQUES

This paper systematically reviews automatic methods and some difficulties for cardiac segmentation in radiographic images. Combined with recent advanced deep learning techniques, the feasibility of using deep learning network models for image segmentation is discussed, and the commonly used deep learning frameworks are compared.

DEVELOPED INSIGHTS

There are many standard methods for medical image segmentation, such as traditional methods based on regions and edges and methods based on deep learning. Because of characteristics of non-uniform grayscale, individual differences, artifacts and noise of medical images, the above image segmentation methods have certain limitations. It is tough to obtain the needed results sensitivity and accuracy when performing heart segmentation. The deep learning model proposed has achieved good results in image segmentation. Accurate segmentation improves the accuracy of disease diagnosis and reduces subsequent irrelevant computations.

SUMMARY

There are two requirements for accurate segmentation of radiological images. One is to use image segmentation to improve the development of computer-aided diagnosis. The other is to achieve complete segmentation of the heart. When there are lesions or deformities in the heart, there will be some abnormalities in the radiographic images, and the segmentation algorithm needs to segment the heart altogether. The quantity of processing inside a certain range will no longer be a restriction for real-time detection with the advancement of deep learning and the enhancement of hardware device performance.

Direct left-ventricular global longitudinal strain (GLS) computation with a fully convolutional network

This study's purpose was to develop a direct MRI-based, deep-learning semantic segmentation approach for computing global longitudinal strain (GLS), a known metric for detecting left-ventricular (LV) cardiotoxicity in breast cancer. Displacement Encoding with Stimulated Echoes cardiac image phases acquired from 30 breast cancer patients and 30 healthy females were unwrapped via a DeepLabV3 + fully convolutional network (FCN). Myocardial strains were directly computed from the unwrapped phases with the Radial Point Interpolation Method. FCN-unwrapped phases of a phantom's rotating gel were validated against quality-guided phase-unwrapping (QGPU) and robust transport of intensity equation (RTIE) phase-unwrapping. FCN performance on unwrapping human LV data was measured with F1 and Dice scores versus QGPU ground-truth. The reliability of FCN-based strains was assessed against RTIE-based strains with Cronbach's alpha (C-α) intraclass correlation coefficient. Mean squared error (MSE) of unwrapping the phantom experiment data at 0 dB signal-to-noise ratio were 1.6, 2.7 and 6.1 with FCN, QGPU and RTIE techniques. Human data classification accuracies were F1 = 0.95 (Dice = 0.96) with FCN and F1 = 0.94 (Dice = 0.95) with RTIE. GLS results from FCN and RTIE were -16 ± 3% vs. -16 ± 3% (C-α = 0.9) for patients and -20 ± 3% vs. -20 ± 3% (C-α = 0.9) for healthy subjects. The low MSE from the phantom validation demonstrates accuracy of phase-unwrapping with the FCN and comparable human subject results versus RTIE demonstrate GLS analysis accuracy. A deep-learning methodology for phase-unwrapping in medical images and GLS computation was developed and validated in a heterogeneous cohort.