Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture

Identification of Myocardial Scarring Using Contrast-Free Cardiac MRI in Patients With Autoimmune Rheumatic Diseases

BACKGROUND

Late gadolinium enhancement (LGE) cardiac MRI is the method of choice in revealing the presence of myocardial scarring, but its availability remains limited in clinical practice.

PURPOSE

To assess myocardial scarring in patients with autoimmune rheumatic diseases (ARDs) using contrast-free cardiac MRI with a radiomics model.

STUDY TYPE

Retrospective.

POPULATION

One hundred ninety-two patients (mean age, 41 years ± 15, 62 men) with or without ARDs, grouped into a training set of 153 patients and a testing set of 39 patients.

FIELD STRENGTH/SEQUENCE

3.0 T/ cine imaging with a balanced steady-state free precession sequence, T1 mapping with a modified Look-Locker inversion recovery sequence, and LGE imaging with a phase-sensitive inversion recovery gradient echo sequence.

ASSESSMENT

LGE assessment was the reference standard for identifying myocardial scarring. Based on motion features extracted from cine images and tissue characterization features extracted from native T1 maps, a fully automated radiomics model with T1, cine MRI, or combined inputs was developed.

STATISTICAL TESTS

Logistic regression model was used to detect myocardial scarring using contrast-free cardiac MRI parameters. Receiver operating characteristic curves were analyzed to assess the accuracy, sensitivity, and specificity in detecting myocardial scarring. Sensitivities of the models were further assessed in patients with various myocardial scarring proportions. Z-statistic and dice coefficient were assessed to compare the performance. P-values <0.05 were considered significant.

RESULTS

The multivariable regression model exhibited an accuracy of 85.3%, a sensitivity of 93.5%, and a specificity of 50.0%. The radiomics model with T1 and cine MRI input exhibited an accuracy of 75.7%, a sensitivity of 60.9%, and a specificity of 85.5%. Moreover, the radiomics model showed a sensitivity of 90.9% among patients with >25% myocardial scarring.

DATA CONCLUSIONS

The proposed radiomics model allowed for the identification of myocardial scarring similar to LGE, but on contrast-free cardiac MRI in patients with ARDs.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 1.

Deep learning approaches for the detection of scar presence from cine cardiac magnetic resonance adding derived parametric images

This work proposes a convolutional neural network (CNN) that utilizes different combinations of parametric images computed from cine cardiac magnetic resonance (CMR) images, to classify each slice for possible myocardial scar tissue presence. The CNN performance comparison in respect to expert interpretation of CMR with late gadolinium enhancement (LGE) images, used as ground truth (GT), was conducted on 206 patients (158 scar, 48 control) from Centro Cardiologico Monzino (Milan, Italy) at both slice- and patient-levels. Left ventricle dynamic features were extracted in non-enhanced cine images using parametric images based on both Fourier and monogenic signal analyses. The CNN, fed with cine images and Fourier-based parametric images, achieved an area under the ROC curve of 0.86 (accuracy 0.79, F1 0.81, sensitivity 0.9, specificity 0.65, and negative (NPV) and positive (PPV) predictive values 0.83 and 0.77, respectively), for individual slice classification. Remarkably, it exhibited 1.0 prediction accuracy (F1 0.98, sensitivity 1.0, specificity 0.9, NPV 1.0, and PPV 0.97) in patient classification as a control or pathologic. The proposed approach represents a first step towards scar detection in contrast-free CMR images. Patient-level results suggest its preliminary potential as a screening tool to guide decisions regarding LGE-CMR prescription, particularly in cases where indication is uncertain.

Prediction of Freezing of Gait in Parkinson's disease based on multi-channel time-series neural network

Freezing of Gait (FOG) is a noticeable symptom of Parkinson's disease, like being stuck in place and increasing the risk of falls. The wearable multi-channel sensor system is an efficient method to predict and monitor the FOG, thus warning the wearer to avoid falls and improving the quality of life. However, the existing approaches for the prediction of FOG mainly focus on a single sensor system and cannot handle the interference between multi-channel wearable sensors. Hence, we propose a novel multi-channel time-series neural network (MCT-Net) approach to merge multi-channel gait features into a comprehensive prediction framework, alerting patients to FOG symptoms in advance. Owing to the causal distributed convolution, MCT-Net is a real-time method available to give optimal prediction earlier and implemented in remote devices. Moreover, intra-channel and inter-channel transformers of MCT-Net extract and integrate different sensor position features into a unified deep learning model. Compared with four other state-of-the-art FOG prediction baselines, the proposed MCT-Net obtains 96.21% in accuracy and 80.46% in F1-score on average 2 s before FOG occurrence, demonstrating the superiority of MCT-Net.

The beating heart: artificial intelligence for cardiovascular application in the clinic

Artificial intelligence (AI) integration in cardiac magnetic resonance imaging presents new and exciting avenues for advancing patient care, automating post-processing tasks, and enhancing diagnostic precision and outcomes. The use of AI significantly streamlines the examination workflow through the reduction of acquisition and postprocessing durations, coupled with the automation of scan planning and acquisition parameters selection. This has led to a notable improvement in examination workflow efficiency, a reduction in operator variability, and an enhancement in overall image quality. Importantly, AI unlocks new possibilities to achieve spatial resolutions that were previously unattainable in patients. Furthermore, the potential for low-dose and contrast-agent-free imaging represents a stride toward safer and more patient-friendly diagnostic procedures. Beyond these benefits, AI facilitates precise risk stratification and prognosis evaluation by adeptly analysing extensive datasets. This comprehensive review article explores recent applications of AI in the realm of cardiac magnetic resonance imaging, offering insights into its transformative potential in the field.

Spatiotemporal knowledge teacher-student reinforcement learning to detect liver tumors without contrast agents

Detecting Liver tumors without contrast agents (CAs) has shown great potential to advance liver cancer screening. It enables the provision of a reliable liver tumor-detecting result from non-enhanced MR images comparable to the radiologists' results from CA-enhanced MR images, thus eliminating the high risk of CAs, preventing an experience gap between radiologists and simplifying clinical workflows. In this paper, we proposed a novel spatiotemporal knowledge teacher-student reinforcement learning (SKT-RL) as a safe, speedy, and inexpensive contrast-free technology for liver tumor detection. Our SKT-RL builds a teacher-student framework to realize the exploring of explicit liver tumor knowledge from a teacher network on clear contrast-enhanced images to guide a student network to detect tumors from non-enhanced images directly. Importantly, our STK-RL enables three novelties in aspects of construction, transferring, and optimization to tumor knowledge to improve the guide effect. (1) A new spatiotemporal ternary knowledge set enables the construction of accurate knowledge that allows understanding of DRL's behavior (what to do) and reason (why to do it) behind reliable detection within each state and between their related historical states. (2) A novel pixel momentum transferring strategy enables detailed and controlled knowledge transfer ability. It transfers knowledge at a pixel level to enlarge the explorable space of transferring and control how much knowledge is transferred to prevent over-rely of the student to the teacher. (3) A phase-trend reward function designs different evaluations according to different detection phases to optimal for each phase in high-precision but also allows reward trend to constraint the evaluation to improve stability. Comprehensive experiments on a generalized liver tumor dataset with 375 patients (including hemangiomas, hepatocellular carcinoma, and normal controls) show that our novel SKT-RL attains a new state-of-the-art performance (improved precision by at least 4% when comparing the six recent advanced methods) in the task of liver tumor detection without CAs. The results proved that our SKT-DRL has greatly promoted the development and deployment of contrast-free liver tumor technology.

Selecting the best optimizers for deep learning-based medical image segmentation

Purpose

The goal of this work is to explore the best optimizers for deep learning in the context of medical image segmentation and to provide guidance on how to design segmentation networks with effective optimization strategies.

Approach

Most successful deep learning networks are trained using two types of stochastic gradient descent (SGD) algorithms: adaptive learning and accelerated schemes. Adaptive learning helps with fast convergence by starting with a larger learning rate (LR) and gradually decreasing it. Momentum optimizers are particularly effective at quickly optimizing neural networks within the accelerated schemes category. By revealing the potential interplay between these two types of algorithms [LR and momentum optimizers or momentum rate (MR) in short], in this article, we explore the two variants of SGD algorithms in a single setting. We suggest using cyclic learning as the base optimizer and integrating optimal values of learning rate and momentum rate. The new optimization function proposed in this work is based on the Nesterov accelerated gradient optimizer, which is more efficient computationally and has better generalization capabilities compared to other adaptive optimizers.

Results

We investigated the relationship of LR and MR under an important problem of medical image segmentation of cardiac structures from MRI and CT scans. We conducted experiments using the cardiac imaging dataset from the ACDC challenge of MICCAI 2017, and four different architectures were shown to be successful for cardiac image segmentation problems. Our comprehensive evaluations demonstrated that the proposed optimizer achieved better results (over a 2% improvement in the dice metric) than other optimizers in the deep learning literature with similar or lower computational cost in both single and multi-object segmentation settings.

Conclusions

We hypothesized that the combination of accelerated and adaptive optimization methods can have a drastic effect in medical image segmentation performances. To this end, we proposed a new cyclic optimization method (Cyclic Learning/Momentum Rate) to address the efficiency and accuracy problems in deep learning-based medical image segmentation. The proposed strategy yielded better generalization in comparison to adaptive optimizers.

Siamese pyramidal deep learning network for strain estimation in 3D cardiac cine-MR

Strain represents the quantification of regional tissue deformation within a given area. Myocardial strain has demonstrated considerable utility as an indicator for the assessment of cardiac function. Notably, it exhibits greater sensitivity in detecting subtle myocardial abnormalities compared to conventional cardiac function indices, like left ventricle ejection fraction (LVEF). Nonetheless, the estimation of strain poses considerable challenges due to the necessity for precise tracking of myocardial motion throughout the complete cardiac cycle. This study introduces a novel deep learning-based pipeline, designed to automatically and accurately estimate myocardial strain from three-dimensional (3D) cine-MR images. Consequently, our investigation presents a comprehensive pipeline for the precise quantification of local and global myocardial strain. This pipeline incorporates a supervised Convolutional Neural Network (CNN) for accurate segmentation of the cardiac muscle and an unsupervised CNN for robust left ventricle motion tracking, enabling the estimation of strain in both artificial phantoms and real cine-MR images. Our investigation involved a comprehensive comparison of our findings with those obtained from two commonly utilized commercial software in this field. This analysis encompassed the examination of both intra- and inter-user variability. The proposed pipeline exhibited demonstrable reliability and reduced divergence levels when compared to alternative systems. Additionally, our approach is entirely independent of previous user data, effectively eliminating any potential user bias that could influence the strain analyses.

Artificial Intelligence Applications in Cardiovascular Magnetic Resonance Imaging: Are We on the Path to Avoiding the Administration of Contrast Media?

In recent years, cardiovascular imaging examinations have experienced exponential growth due to technological innovation, and this trend is consistent with the most recent chest pain guidelines. Contrast media have a crucial role in cardiovascular magnetic resonance (CMR) imaging, allowing for more precise characterization of different cardiovascular diseases. However, contrast media have contraindications and side effects that limit their clinical application in determinant patients. The application of artificial intelligence (AI)-based techniques to CMR imaging has led to the development of non-contrast models. These AI models utilize non-contrast imaging data, either independently or in combination with clinical and demographic data, as input to generate diagnostic or prognostic algorithms. In this review, we provide an overview of the main concepts pertaining to AI, review the existing literature on non-contrast AI models in CMR, and finally, discuss the strengths and limitations of these AI models and their possible future development.

Synthetic Inflammation Imaging with PatchGAN Deep Learning Networks

Background: Gadolinium (Gd)-enhanced Magnetic Resonance Imaging (MRI) is crucial in several applications, including oncology, cardiac imaging, and musculoskeletal inflammatory imaging. One use case is rheumatoid arthritis (RA), a widespread autoimmune condition for which Gd MRI is crucial in imaging synovial joint inflammation, but Gd administration has well-documented safety concerns. As such, algorithms that could synthetically generate post-contrast peripheral joint MR images from non-contrast MR sequences would have immense clinical utility. Moreover, while such algorithms have been investigated for other anatomies, they are largely unexplored for musculoskeletal applications such as RA, and efforts to understand trained models and improve trust in their predictions have been limited in medical imaging. Methods: A dataset of 27 RA patients was used to train algorithms that synthetically generated post-Gd IDEAL wrist coronal T₁-weighted scans from pre-contrast scans. UNets and PatchGANs were trained, leveraging an anomaly-weighted L₁ loss and global generative adversarial network (GAN) loss for the PatchGAN. Occlusion and uncertainty maps were also generated to understand model performance. Results: UNet synthetic post-contrast images exhibited stronger normalized root mean square error (nRMSE) than PatchGAN in full volumes and the wrist, but PatchGAN outperformed UNet in synovial joints (UNet nRMSEs: volume = 6.29 ± 0.88, wrist = 4.36 ± 0.60, synovial = 26.18 ± 7.45; PatchGAN nRMSEs: volume = 6.72 ± 0.81, wrist = 6.07 ± 1.22, synovial = 23.14 ± 7.37; n = 7). Occlusion maps showed that synovial joints made substantial contributions to PatchGAN and UNet predictions, while uncertainty maps showed that PatchGAN predictions were more confident within those joints. Conclusions: Both pipelines showed promising performance in synthesizing post-contrast images, but PatchGAN performance was stronger and more confident within synovial joints, where an algorithm like this would have maximal clinical utility. Image synthesis approaches are therefore promising for RA and synthetic inflammatory imaging.

MyoPS: A benchmark of myocardial pathology segmentation combining three-sequence cardiac magnetic resonance images

Assessment of myocardial viability is essential in diagnosis and treatment management of patients suffering from myocardial infarction, and classification of pathology on the myocardium is the key to this assessment. This work defines a new task of medical image analysis, i.e., to perform myocardial pathology segmentation (MyoPS) combining three-sequence cardiac magnetic resonance (CMR) images, which was first proposed in the MyoPS challenge, in conjunction with MICCAI 2020. Note that MyoPS refers to both myocardial pathology segmentation and the challenge in this paper. The challenge provided 45 paired and pre-aligned CMR images, allowing algorithms to combine the complementary information from the three CMR sequences for pathology segmentation. In this article, we provide details of the challenge, survey the works from fifteen participants and interpret their methods according to five aspects, i.e., preprocessing, data augmentation, learning strategy, model architecture and post-processing. In addition, we analyze the results with respect to different factors, in order to examine the key obstacles and explore the potential of solutions, as well as to provide a benchmark for future research. The average Dice scores of submitted algorithms were 0.614±0.231 and 0.644±0.153 for myocardial scars and edema, respectively. We conclude that while promising results have been reported, the research is still in the early stage, and more in-depth exploration is needed before a successful application to the clinics. MyoPS data and evaluation tool continue to be publicly available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/myops20/).

MyoPS-Net: Myocardial pathology segmentation with flexible combination of multi-sequence CMR images

Myocardial pathology segmentation (MyoPS) can be a prerequisite for the accurate diagnosis and treatment planning of myocardial infarction. However, achieving this segmentation is challenging, mainly due to the inadequate and indistinct information from an image. In this work, we develop an end-to-end deep neural network, referred to as MyoPS-Net, to flexibly combine five-sequence cardiac magnetic resonance (CMR) images for MyoPS. To extract precise and adequate information, we design an effective yet flexible architecture to extract and fuse cross-modal features. This architecture can tackle different numbers of CMR images and complex combinations of modalities, with output branches targeting specific pathologies. To impose anatomical knowledge on the segmentation results, we first propose a module to regularize myocardium consistency and localize the pathologies, and then introduce an inclusiveness loss to utilize relations between myocardial scars and edema. We evaluated the proposed MyoPS-Net on two datasets, i.e., a private one consisting of 50 paired multi-sequence CMR images and a public one from MICCAI2020 MyoPS Challenge. Experimental results showed that MyoPS-Net could achieve state-of-the-art performance in various scenarios. Note that in practical clinics, the subjects may not have full sequences, such as missing LGE CMR or mapping CMR scans. We therefore conducted extensive experiments to investigate the performance of the proposed method in dealing with such complex combinations of different CMR sequences. Results proved the superiority and generalizability of MyoPS-Net, and more importantly, indicated a practical clinical application. The code has been released via https://github.com/QJYBall/MyoPS-Net.

A New X-ray Medical-Image-Enhancement Method Based on Multiscale Shannon-Cosine Wavelet

Because of noise interference, improper exposure, and the over thickness of human tissues, the detailed information of DR (digital radiography) images can be masked, including unclear edges and reduced contrast. An image-enhancement algorithm based on wavelet multiscale decomposition is proposed to address the shortcomings of existing single-scale image-enhancement algorithms. The proposed algorithm is based on Shannon-Cosine wavelets by taking advantage of the interpolation, smoothness, tight support, and normalization properties. Next a multiscale interpolation wavelet operator is constructed to divide the image into several sub-images from high frequency to low frequency, and to perform different multi-scale wavelet transforms on the detailed image of each channel. So that the most subtle and diagnostically useful information in the image can be effectively enhanced. Moreover, the image will not be over-enhanced and combined with the high contrast sensitivity of the human eye's visual system in smooth regions, different attenuation coefficients are used for different regions to achieve the purpose of suppressing noise while enhancing details. The results obtained by some simulations show that this method can effectively eliminate the noise in the DR image, and the enhanced DR image detail information is clearer than before while having high effectiveness and robustness.

Using artificial intelligence in the development of diagnostic models of coronary artery disease with imaging markers: A scoping review

Background

Coronary artery disease (CAD) is a progressive disease of the blood vessels supplying the heart, which leads to coronary artery stenosis or obstruction and is life-threatening. Early diagnosis of CAD is essential for timely intervention. Imaging tests are widely used in diagnosing CAD, and artificial intelligence (AI) technology is used to shed light on the development of new imaging diagnostic markers.

Objective

We aim to investigate and summarize how AI algorithms are used in the development of diagnostic models of CAD with imaging markers.

Methods

This scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guideline. Eligible articles were searched in PubMed and Embase. Based on the predefined included criteria, articles on coronary heart disease were selected for this scoping review. Data extraction was independently conducted by two reviewers, and a narrative synthesis approach was used in the analysis.

Results

A total of 46 articles were included in the scoping review. The most common types of imaging methods complemented by AI included single-photon emission computed tomography (15/46, 32.6%) and coronary computed tomography angiography (15/46, 32.6%). Deep learning (DL) (41/46, 89.2%) algorithms were used more often than machine learning algorithms (5/46, 10.8%). The models yielded good model performance in terms of accuracy, sensitivity, specificity, and AUC. However, most of the primary studies used a relatively small sample (n < 500) in model development, and only few studies (4/46, 8.7%) carried out external validation of the AI model.

Conclusion

As non-invasive diagnostic methods, imaging markers integrated with AI have exhibited considerable potential in the diagnosis of CAD. External validation of model performance and evaluation of clinical use aid in the confirmation of the added value of markers in practice.

Systematic review registration

[https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022306638], identifier [CRD42022306638].

Classification of myocardial fibrosis in DE-MRI based on semi-supervised semantic segmentation and dual attention mechanism

OBJECTIVE

It is essential to utilize cardiac delayed-enhanced magnetic resonance imaging (DE-MRI) to diagnose cardiovascular disease. By segmenting myocardium DE-MRI images, it provides critical information for the evaluation and treatment of myocardial infarction. As a consequence, it is vital to investigate the segmentation and classification technique of myocardial DE-MRI.

METHODS

Firstly, an end-to-end minimally supervised and semi-supervised semantic DE-MRI myocardial fibrosis segmentation framework is proposed, which combines image classification and semantic segmentation branches based on the self-attention mechanism. Following that, a residual hole network fused with the dual attention mechanism was built, and a double attention metabolic pathway classification method for cardiac fibrosis in DE-MRI images was developed.

RESULTS

By adding pixel-level labels to an extra 40 training images, the segmentation model may enhance semantic segmentation performance by 2.6 percent (from 61.2 percent to 63.8 percent). When the number of pixel-level labels is increased to 80, semi-supervised feature extraction increases by 4.7 percent when compared to weakly guided semantic segmentation. Adding an attention mechanism to the critical network DRN (Deep Residual Network) can increase the classifier's performance by a small amount. Experiments revealed that the models worked effectively.

CONCLUSION

This paper investigates the segmentation and classification of cardiac fibrosis in DE-MRI data using a semi-supervised semantic segmentation and dual attention mechanism, dealing with the issue that existing segmentation algorithms have difficulty segmenting myocardial fibrosis tissue. In the future, we can consider optimizing the design of the attention module to reduce the module computation.

CMRSegTools: An open-source software enabling reproducible research in segmentation of acute myocardial infarct in CMR images

In the last decade, a large number of clinical trials have been deployed using Cardiac Magnetic Resonance (CMR) to evaluate cardioprotective strategies aiming at reducing the irreversible myocardial damage at the time of reperfusion. In these studies, segmentation and quantification of myocardial infarct lesion are often performed with a commercial software or an in-house closed-source code development thus creating a barrier for reproducible research. This paper introduces CMRSegTools: an open-source application software designed for the segmentation and quantification of myocardial infarct lesion enabling full access to state-of-the-art segmentation methods and parameters, easy integration of new algorithms and standardised results sharing. This post-processing tool has been implemented as a plug-in for the OsiriX/Horos DICOM viewer leveraging its database management functionalities and user interaction features to provide a bespoke tool for the analysis of cardiac MR images on large clinical cohorts. CMRSegTools includes, among others, user-assisted segmentation of the left-ventricle, semi- and automatic lesion segmentation methods, advanced statistical analysis and visualisation based on the American Heart Association 17-segment model. New segmentation methods can be integrated into the plug-in by developing components based on image processing and visualisation libraries such as ITK and VTK in C++ programming language. CMRSegTools allows the creation of training and testing data sets (labeled features such as lesion, microvascular obstruction and remote ROI) for supervised Machine Learning methods, and enables the comparative assessment of lesion segmentation methods via a single and integrated platform. The plug-in has been successfully used by several CMR imaging studies.

Brain Tumor Segmentation Based on Bendlet Transform and Improved Chan-Vese Model

Automated segmentation of brain tumors is a difficult procedure due to the variability and blurred boundary of the lesions. In this study, we propose an automated model based on Bendlet transform and improved Chan-Vese (CV) model for brain tumor segmentation. Since the Bendlet system is based on the principle of sparse approximation, Bendlet transform is applied to describe the images and map images to the feature space and, thereby, first obtain the feature set. This can help in effectively exploring the mapping relationship between brain lesions and normal tissues, and achieving multi-scale and multi-directional registration. Secondly, the SSIM region detection method is proposed to preliminarily locate the tumor region from three aspects of brightness, structure, and contrast. Finally, the CV model is solved by the Hermite-Shannon-Cosine wavelet homotopy method, and the boundary of the tumor region is more accurately delineated by the wavelet transform coefficient. We randomly selected some cross-sectional images to verify the effectiveness of the proposed algorithm and compared with CV, Ostu, K-FCM, and region growing segmentation methods. The experimental results showed that the proposed algorithm had higher segmentation accuracy and better stability.

Lightweight Compound Scaling Network for Nasopharyngeal Carcinoma Segmentation from MR Images

Nasopharyngeal carcinoma (NPC) is a category of tumours with a high incidence in head-and-neck. To treat nasopharyngeal cancer, doctors invariably need to perform focal segmentation. However, manual segmentation is time consuming and laborious for doctors and the existing automatic segmentation methods require large computing resources, which makes some small and medium-sized hospitals unaffordable. To enable small and medium-sized hospitals with limited computational resources to run the model smoothly and improve the accuracy of structure, we propose a new LW-UNet network. The network utilises lightweight modules to form the Compound Scaling Encoder and combines the benefits of UNet to make the model both lightweight and accurate. Our model achieves a high accuracy with a Dice coefficient value of 0.813 with 3.55 M parameters and 7.51 G of FLOPs within 0.1 s (testing time in GPU), which is the best result compared with four other state-of-the-art models.

Predicting post-contrast information from contrast agent free cardiac MRI using machine learning: Challenges and methods

Objectives

Currently, administering contrast agents is necessary for accurately visualizing and quantifying presence, location, and extent of myocardial infarction (MI) with cardiac magnetic resonance (CMR). In this study, our objective is to investigate and analyze pre- and post-contrast CMR images with the goal of predicting post-contrast information using pre-contrast information only. We propose methods and identify challenges.

Methods

The study population consists of 272 retrospectively selected CMR studies with diagnoses of MI (n = 108) and healthy controls (n = 164). We describe a pipeline for pre-processing this dataset for analysis. After data feature engineering, 722 cine short-axis (SAX) images and segmentation mask pairs were used for experimentation. This constitutes 506, 108, and 108 pairs for the training, validation, and testing sets, respectively. We use deep learning (DL) segmentation (UNet) and classification (ResNet50) models to discover the extent and location of the scar and classify between the ischemic cases and healthy cases (i.e., cases with no regional myocardial scar) from the pre-contrast cine SAX image frames, respectively. We then capture complex data patterns that represent subtle signal and functional changes in the cine SAX images due to MI using optical flow, rate of change of myocardial area, and radiomics data. We apply this dataset to explore two supervised learning methods, namely, the support vector machines (SVM) and the decision tree (DT) methods, to develop predictive models for classifying pre-contrast cine SAX images as being a case of MI or healthy.

Results

Overall, for the UNet segmentation model, the performance based on the mean Dice score for the test set (n = 108) is 0.75 (±0.20) for the endocardium, 0.51 (±0.21) for the epicardium and 0.20 (±0.17) for the scar. For the classification task, the accuracy, F1 and precision scores of 0.68, 0.69, and 0.64, respectively, were achieved with the SVM model, and of 0.62, 0.63, and 0.72, respectively, with the DT model.

Conclusion

We have presented some promising approaches involving DL, SVM, and DT methods in an attempt to accurately predict contrast information from non-contrast images. While our initial results are modest for this challenging task, this area of research still poses several open problems.

Automatic myocardium strain quantification in MR synthetic images with Deep Leaning

Accurate quantification of myocardium strain in magnetic resonance images is important to correctly diagnose and monitor cardiac diseases. Currently, available methods to estimate motion are based on tracking brightness pattern differences between images. In cine-MR images, the myocardium interior presents an inhered homogeneity, which reduces the accuracy in estimated motion, and consequently strain. Neural networks have recently been shown to be an important tool for a variety of applications, including motion estimation. In this work, we investigate the feasibility of quantifying myocardium strain in cardiac resonance synthetic images using motion generated by a compact and powerful network called Pyramid, Warping, and Cost Volume (PWC). Using the motion generated by the neural network, the radial myocardium strain obtained presents a mean average error of 12.30% +- 6.50%, and in the circumferential direction 1.20% +-0.61 %, better than the two classical methods evaluated. Clinical Relevance- This work demonstrates the feasibility of estimating myocardium strain using motion estimated by a convolutional neural network.

Cardiovascular Disease Diagnosis from DXA Scan and Retinal Images Using Deep Learning

Cardiovascular diseases (CVD) are the leading cause of death worldwide. People affected by CVDs may go undiagnosed until the occurrence of a serious heart failure event such as stroke, heart attack, and myocardial infraction. In Qatar, there is a lack of studies focusing on CVD diagnosis based on non-invasive methods such as retinal image or dual-energy X-ray absorptiometry (DXA). In this study, we aimed at diagnosing CVD using a novel approach integrating information from retinal images and DXA data. We considered an adult Qatari cohort of 500 participants from Qatar Biobank (QBB) with an equal number of participants from the CVD and the control groups. We designed a case-control study with a novel multi-modal (combining data from multiple modalities-DXA and retinal images)-to propose a deep learning (DL)-based technique to distinguish the CVD group from the control group. Uni-modal models based on retinal images and DXA data achieved 75.6% and 77.4% accuracy, respectively. The multi-modal model showed an improved accuracy of 78.3% in classifying CVD group and the control group. We used gradient class activation map (GradCAM) to highlight the areas of interest in the retinal images that influenced the decisions of the proposed DL model most. It was observed that the model focused mostly on the centre of the retinal images where signs of CVD such as hemorrhages were present. This indicates that our model can identify and make use of certain prognosis markers for hypertension and ischemic heart disease. From DXA data, we found higher values for bone mineral density, fat content, muscle mass and bone area across majority of the body parts in CVD group compared to the control group indicating better bone health in the Qatari CVD cohort. This seminal method based on DXA scans and retinal images demonstrate major potentials for the early detection of CVD in a fast and relatively non-invasive manner.

The Applications of Artificial Intelligence in Cardiovascular Magnetic Resonance-A Comprehensive Review

Cardiovascular disease remains an integral field on which new research in both the biomedical and technological fields is based, as it remains the leading cause of mortality and morbidity worldwide. However, despite the progress of cardiac imaging techniques, the heart remains a challenging organ to study. Artificial intelligence (AI) has emerged as one of the major innovations in the field of diagnostic imaging, with a dramatic impact on cardiovascular magnetic resonance imaging (CMR). AI will be increasingly present in the medical world, with strong potential for greater diagnostic efficiency and accuracy. Regarding the use of AI in image acquisition and reconstruction, the main role was to reduce the time of image acquisition and analysis, one of the biggest challenges concerning magnetic resonance; moreover, it has been seen to play a role in the automatic correction of artifacts. The use of these techniques in image segmentation has allowed automatic and accurate quantification of the volumes and masses of the left and right ventricles, with occasional need for manual correction. Furthermore, AI can be a useful tool to directly help the clinician in the diagnosis and derivation of prognostic information of cardiovascular diseases. This review addresses the applications and future prospects of AI in CMR imaging, from image acquisition and reconstruction to image segmentation, tissue characterization, diagnostic evaluation, and prognostication.

Two-stage segmentation network with feature aggregation and multi-level attention mechanism for multi-modality heart images

Accurate segmentation of cardiac substructures in multi-modality heart images is an important prerequisite for the diagnosis and treatment of cardiovascular diseases. However, the segmentation of cardiac images remains a challenging task due to (1) the interference of multiple targets, (2) the imbalance of sample size. Therefore, in this paper, we propose a novel two-stage segmentation network with feature aggregation and multi-level attention mechanism (TSFM-Net) to comprehensively solve these challenges. Firstly, in order to improve the effectiveness of multi-target features, we adopt the encoder-decoder structure as the backbone segmentation framework and design a feature aggregation module (FAM) to realize the multi-level feature representation (Stage1). Secondly, because the segmentation results obtained from Stage1 are limited to the decoding of single scale feature maps, we design a multi-level attention mechanism (MLAM) to assign more attention to the multiple targets, so as to get multi-level attention maps. We fuse these attention maps and concatenate the output of Stage1 to carry out the second segmentation to get the final segmentation result (Stage2). The proposed method has better segmentation performance and balance on 2017 MM-WHS multi-modality whole heart images than the state-of-the-art methods, which demonstrates the feasibility of TSFM-Net for accurate segmentation of heart images.

An Improved 3D Deep Learning-Based Segmentation of Left Ventricular Myocardial Diseases from Delayed-Enhancement MRI with Inclusion and Classification Prior Information U-Net (ICPIU-Net)

Accurate segmentation of the myocardial scar may supply relevant advancements in predicting and controlling deadly ventricular arrhythmias in subjects with cardiovascular disease. In this paper, we propose the architecture of inclusion and classification of prior information U-Net (ICPIU-Net) to efficiently segment the left ventricle (LV) myocardium, myocardial infarction (MI), and microvascular-obstructed (MVO) tissues from late gadolinium enhancement magnetic resonance (LGE-MR) images. Our approach was developed using two subnets cascaded to first segment the LV cavity and myocardium. Then, we used inclusion and classification constraint networks to improve the resulting segmentation of the diseased regions within the pre-segmented LV myocardium. This network incorporates the inclusion and classification information of the LGE-MRI to maintain topological constraints of pathological areas. In the testing stage, the outputs of each segmentation network obtained with specific estimated parameters from training were fused using the majority voting technique for the final label prediction of each voxel in the LGE-MR image. The proposed method was validated by comparing its results to manual drawings by experts from 50 LGE-MR images. Importantly, compared to various deep learning-based methods participating in the EMIDEC challenge, the results of our approach have a more significant agreement with manual contouring in segmenting myocardial diseases.

Robust optical flow algorithm for general single cell segmentation

Cell segmentation is crucial to the field of cell biology, as the accurate extraction of single-cell morphology, migration, and ultimately behavior from time-lapse live cell imagery are of paramount importance to elucidate and understand basic cellular processes. In an effort to increase available segmentation tools that can perform across research groups and platforms, we introduce a novel segmentation approach centered around optical flow and show that it achieves robust segmentation of single cells by validating it on multiple cell types, phenotypes, optical modalities, and in-vitro environments with or without labels. By leveraging cell movement in time-lapse imagery as a means to distinguish cells from their background and augmenting the output with machine vision operations, our algorithm reduces the number of adjustable parameters needed for manual optimization to two. We show that this approach offers the advantage of quicker processing times compared to contemporary machine learning based methods that require manual labeling for training, and in most cases achieves higher quality segmentation as well. This algorithm is packaged within MATLAB, offering an accessible means for general cell segmentation in a time-efficient manner.

Myocardial strain analysis of echocardiography based on deep learning

BACKGROUND

Strain analysis provides more thorough spatiotemporal signatures for myocardial contraction, which is helpful for early detection of cardiac insufficiency. The use of deep learning (DL) to automatically measure myocardial strain from echocardiogram videos has garnered recent attention. However, the development of key techniques including segmentation and motion estimation remains a challenge. In this work, we developed a novel DL-based framework for myocardial segmentation and motion estimation to generate strain measures from echocardiogram videos.

METHODS

Three-dimensional (3D) Convolutional Neural Network (CNN) was developed for myocardial segmentation and optical flow network for motion estimation. The segmentation network was used to define the region of interest (ROI), and the optical flow network was used to estimate the pixel motion in the ROI. We performed a model architecture search to identify the optimal base architecture for motion estimation. The final workflow design and associated hyperparameters are the result of a careful implementation. In addition, we compared the DL model with a traditional speck tracking algorithm on an independent, external clinical data. Each video was double-blind measured by an ultrasound expert and a DL expert using speck tracking echocardiography (STE) and DL method, respectively.

RESULTS

The DL method successfully performed automatic segmentation, motion estimation, and global longitudinal strain (GLS) measurements in all examinations. The 3D segmentation has better spatio-temporal smoothness, average dice correlation reaches 0.82, and the effect of target frame is better than that of previous 2D networks. The best motion estimation network achieved an average end-point error of 0.05 ± 0.03 mm per frame, better than previously reported state-of-the-art. The DL method showed no significant difference relative to the traditional method in GLS measurement, Spearman correlation of 0.90 (p < 0.001) and mean bias -1.2 ± 1.5%.

CONCLUSION

In conclusion, our method exhibits better segmentation and motion estimation performance and demonstrates the feasibility of DL method for automatic strain analysis. The DL approach helps reduce time consumption and human effort, which holds great promise for translational research and precision medicine efforts.

FA-GAN: Fused attentive generative adversarial networks for MRI image super-resolution

High-resolution magnetic resonance images can provide fine-grained anatomical information, but acquiring such data requires a long scanning time. In this paper, a framework called the Fused Attentive Generative Adversarial Networks(FA-GAN) is proposed to generate the super- resolution MR image from low-resolution magnetic resonance images, which can reduce the scanning time effectively but with high resolution MR images. In the framework of the FA-GAN, the local fusion feature block, consisting of different three-pass networks by using different convolution kernels, is proposed to extract image features at different scales. And the global feature fusion module, including the channel attention module, the self-attention module, and the fusion operation, is designed to enhance the important features of the MR image. Moreover, the spectral normalization process is introduced to make the discriminator network stable. 40 sets of 3D magnetic resonance images (each set of images contains 256 slices) are used to train the network, and 10 sets of images are used to test the proposed method. The experimental results show that the PSNR and SSIM values of the super-resolution magnetic resonance image generated by the proposed FA-GAN method are higher than the state-of-the-art reconstruction methods.

A rapid segmentation method of cell boundary for developing embryos using machine learning with a personal computer

Cell segmentation is crucial in the study of morphogenesis in developing embryos, but it had been limited in its accuracy until machine learning methods for image segmentation like U-Net. However, these methods take too much time. In this study, we provide a rapid method for cell segmentation using machine learning with a personal computer, termed Cell Segmentator using Machine Learning (CSML). CSML took four seconds per image with a personal computer for segmentation on average, much less than time to obtain an image. We observed that F-value of segmentation by CSML was around 0.97, showing better performance than state-of-the-art methods like RACE and watershed in assessing the segmentation of Xenopus ectodermal cells. CSML also showed slightly better performance and faster than other machine learning-based methods such as U-Net. CSML required only one whole embryo image for training a Fully Convolutional Network classifier and only two parameters. To validate its accuracy, we compared CSML to other methods in assessing several indicators of cell shape. We also examined the generality of this approach by measuring its performance of segmentation of independent images. Our data demonstrate the superiority of CSML, and we expect this application to improve efficiency in cell shape studies.

A weighted region-based level set method for image segmentation with intensity inhomogeneity

Image segmentation is a fundamental task in image processing and is still a challenging problem when processing images with high noise, low resolution and intensity inhomogeneity. In this paper, a weighted region-based level set method, which is based on the techniques of local statistical theory, level set theory and curve evolution, is proposed. Specifically, a new weighted pressure force function (WPF) is first presented to flexibly drive the closed contour to shrink or expand outside and inside of the object. Second, a faster and smoother regularization term is added to ensure the stability of the curve evolution and that there is no need for initialization in curve evolution. Third, the WPF is integrated into the region-based level set framework to accelerate the speed of the curve evolution and improve the accuracy of image segmentation. Experimental results on medical and natural images demonstrate that the proposed segmentation model is more efficient and robust to noise than other state-of-the-art models.

Use of Longitudinal Strain Bull’s-Eye Plot by Speckle Tracking Echocardiography for Evaluation of Homozygous Familial Hypercholesterolemia with Myocardial Ischemia

Background: In recent years, two-dimensional speckle tracking echocardiography (2D-STE) has been increasingly used to detect left ventricular myocardial ischemia with high accuracy for a reliable and comprehensive assessment of myocardial function by myocardial strain analysis. The present study aimed to assess whether the longitudinal strain (LS) bull’s eye plot could accurately detect left ventricular myocardial ischemia in homozygous familial hypercholesterolemia (HoFH) patients. Methods: A total of 28 HoHF patients, who underwent 2D-STE and myocardial perfusion imaging (MPI), were classified into two groups as diagnosed by MPI for myocardial ischemia. The myocardial ischemia score by MPI, LS bull’s-eye plot, and strain parameters were analyzed and compared. Results: Among the 28 HoFH patients, MPI detected 30.77% and 2D-STE showed 30.19% ischemic segments in 13 and 15 HoFH patients with myocardial ischemia, respectively. All segmental LSs in the left anterior descending artery (LAD) perfusion territory were significantly decreased in patients with myocardial ischemia. The diagnostic capability of 2D-STE for myocardial ischemia was 85.29%, 95.34%, and 93.91% for sensitivity, specificity and accuracy, respectively. Conclusion: Left ventricular LS bull’s-eye plot can assist rapid and accurate evaluation of myocardial ischemia in HoFH patients. The myocardial ischemia is mainly distributed in the LAD perfusion territory in these patients.

Automatic Segmentation of Color Tongue Image Using Deep Asymmetric Convolution Skip Net

Segmentation of the tongue body from color images is vital for tongue diagnoses in traditional Chinese medicine. In tongue images, the tongue body is easily confused with the skin and lips, and the shadow also causes incorrect segmentation. To address these issues, we proposed a novel ACS-Net for tongue image segmentation and implemented the end-to-end form. In our ACS-Net architecture, the following innovations proposed: (1) ordinary convolution was replaced with ACB Module, (2) decoder block restores the features extracted by the encoder block, (3) skip connections are implemented between and within blocks. We use our own datasets named S1 and S2 that collected from the partner hospital. The collection methods of these two datasets are different: S1 was collected by professionals while S2 was taken by nurses. The method achieved state-of-the-art results on both two datasets, we use two metrics to reflect the segmentation performance, which are accuracy (acc) and mean intersection over Union (mIoU), in which the acc reaches 0.984 on S1 and 0.981 on S2; the mIoU reaches 0.925 on S1 and 0.958 on S2.

Left ventricle motion estimation for cine MR images using sparse representation with shape constraint

PURPOSE

To propose a left ventricle (LV) motion estimation method based on sparse representation, in order to handle the spatial-varying intensity distortions caused by tissue deformation.

METHODS

For each myocardial landmark, an adaptive dictionary was generated by learning transformations from a training dataset. Then the landmark was tracked using sparse representation. Next, a point distribution model was applied to the overall tracking results. Finally, the dense displacement field of the LV myocardium was estimated based on the correspondence between each landmark. Using the dense displacement field estimated, the circumferential strain was calculated to assess the myocardial function. The performance of the proposed method was quantified by the average perpendicular distance (APD), the Dice metric, and the mean symmetric contour distance (SCD).

RESULTS

Comparing to the state-of-the-art techniques, the smallest value of APD and SCD, and the highest value of Dice can be obtained using the proposed method, for three public cardiac datasets. Moreover, the mean value of strain difference between the proposed method and the commercial software Medis Suite MR was -0.01, while the intraclass correlation coefficient between these two methods was 0.91.

CONCLUSIONS

The proposed method could estimate the dense displacement field of the LV accurately, which outperforms other state-of-the-art techniques. The circumferential strain derived from the proposed method was in excellent agreement with that derived from the Medis Suite MR software, while segmental strain abnormalities were detected for most of the subjects with heart diseases, which indicates the potential of the proposed method for clinical usage.

A LUNG IMAGE CLASSIFICATION METHOD: A CLASSIFIER CONSTRUCTED BY COMBINING IMPROVED VGG16 AND GRADIENT BOOSTING DECISION TREE

Early classification and diagnosis of lung diseases is essential to increase the best chance of patient recovery and survival. Using deep learning to make it possible, the key is how to improve the robustness of the deep learning model and the accuracy of lung image classification. In order to classify the five lung diseases, we used transfer learning to improve and fine-tune the fully connected layer of VGG16, and improve the cross entropy loss function, combined with the gradient boosting decision tree (GBDT), to establish a deep learning model called a classifier. The model was trained using the ChestX-ray14 dataset. On the test set, the classification accuracy of our model for the five lung diseases was 82.43%, 95.37%, 82.11%, 79.81%, 78.13%, which is better than the best published results. The F1 value is 0.456 (95% CI 0.415, 0.496). The robustness of the model exceeds CheXNet and average performance of doctors. This study clarified that the model has strong robustness and effectiveness in classifying five lung diseases.

AUTOMATED SCREENING OF DIABETIC RETINOPATHY WITH OPTIMIZED DEEP CONVOLUTIONAL NEURAL NETWORK: ENHANCED MOTH FLAME MODEL

Nowadays, analysis on retinal image exists as one of the challenging area for study. Numerous retinal diseases could be recognized by analyzing the variations taking place in retina. However, the main disadvantage among those studies is that, they do not have higher recognition accuracy. The proposed framework includes four phases namely, (i) Blood Vessel Segmentation (ii) Feature Extraction (iii) Optimal Feature Selection and (iv) Classification. Initially, the input fundus image is subjected to blood vessel segmentation from which two binary thresholded images (one from High Pass Filter (HPF) and other from top-hat reconstruction) are acquired. These two images are differentiated and the areas that are common to both are said to be the major vessels and the left over regions are fused to form vessel sub-image. These vessel sub-images are classified with Gaussian Mixture Model (GMM) classifier and the resultant is summed up with the major vessels to form the segmented blood vessels. The segmented images are subjected to feature extraction process, where the features like proposed Local Binary Pattern (LBP), Gray-Level Co-Occurrence Matrix (GLCM) and Gray Level Run Length Matrix (GLRM) are extracted. As the curse of dimensionality seems to be the greatest issue, it is important to select the appropriate features from the extracted one for classification. In this paper, a new improved optimization algorithm Moth Flame with New Distance Formulation (MF-NDF) is introduced for selecting the optimal features. Finally, the selected optimal features are subjected to Deep Convolutional Neural Network (DCNN) model for classification. Further, in order to make the precise diagnosis, the weights of DCNN are optimally tuned by the same optimization algorithm. The performance of the proposed algorithm will be compared against the conventional algorithms in terms of positive and negative measures.

Weakly-Supervised teacher-Student network for liver tumor segmentation from non-enhanced images

Accurate liver tumor segmentation without contrast agents (non-enhanced images) avoids the contrast-agent-associated time-consuming and high risk, which offers radiologists quick and safe assistance to diagnose and treat the liver tumor. However, without contrast agents enhancing, the tumor in liver images presents low contrast and even invisible to naked eyes. Thus the liver tumor segmentation from non-enhanced images is quite challenging. We propose a Weakly-Supervised Teacher-Student network (WSTS) to address the liver tumor segmentation in non-enhanced images by leveraging additional box-level-labeled data (labeled with a tumor bounding-box). WSTS deploys a weakly-supervised teacher-student framework (TCH-ST), namely, a Teacher Module learns to detect and segment the tumor in enhanced images during training, which facilitates a Student Module to detect and segment the tumor in non-enhanced images independently during testing. To detect the tumor accurately, the WSTS proposes a Dual-strategy DRL (DDRL), which develops two tumor detection strategies by creatively introducing a relative-entropy bias in the DRL. To accurately predict a tumor mask for the box-level-labeled enhanced image and thus improve tumor segmentation in non-enhanced images, the WSTS proposes an Uncertainty-Sifting Self-Ensembling (USSE). The USSE exploits the weakly-labeled data with self-ensembling and evaluates the prediction reliability with a newly-designed Multi-scale Uncertainty-estimation. WSTS is validated with a 2D MRI dataset, where the experiment achieves 83.11% of Dice and 85.12% of Recall in 50 patient testing data after training by 200 patient data (half amount data is box-level-labeled). Such a great result illustrates the competence of WSTS to segment the liver tumor from non-enhanced images. Thus, WSTS has excellent potential to assist radiologists by liver tumor segmentation without contrast-agents.

Analysis of the nonperfused volume ratio of adenomyosis from MRI images based on fewshot learning

The nonperfused volume (NPV) ratio is the key to the success of high intensity focused ultrasound (HIFU) ablation treatment of adenomyosis. However, there are no qualitative interpretation standards for predicting the NPV ratio of adenomyosis using magnetic resonance imaging (MRI) before HIFU ablation treatment, which leading to inter-reader variability. Convolutional neural networks (CNNs) have achieved state-of-the-art performance in the automatic disease diagnosis of MRI. Since the use of HIFU to treat adenomyosis is a novel treatment, there is not enough MRI data to support CNNs. We proposed a novel few-shot learning framework that extends CNNs to predict NPV ratio of HIFU ablation treatment for adenomyosis. We collected a dataset from 208 patients with adenomyosis who underwent MRI examination before and after HIFU treatment. Our proposed method was trained and evaluated by fourfold cross validation. This framework obtained sensitivity of 85.6%, 89.6% and 92.8% at 0.799, 0.980 and 1.180 FPs per patient. In the receiver operating characteristics analysis for NPV ratio of adenomyosis, our proposed method received the area under the curve of 0.8233, 0.8289, 0.8412, 0.8319, 0.7010, 0.7637, 0.8375, 0.8219, 0.8207, 0.9812 for the classifications of NPV ratio interval [0%-10%), [10%-20%), …, [90%-100%], respectively. The present study demonstrated that few-shot learning on NPV ratio prediction of HIFU ablation treatment for adenomyosis may contribute to the selection of eligible patients and the pre-judgment of clinical efficacy.

Review of Deep Learning Approaches for the Segmentation of Multiple Sclerosis Lesions on Brain MRI

In recent years, there have been multiple works of literature reviewing methods for automatically segmenting multiple sclerosis (MS) lesions. However, there is no literature systematically and individually review deep learning-based MS lesion segmentation methods. Although the previous review also included methods based on deep learning, there are some methods based on deep learning that they did not review. In addition, their review of deep learning methods did not go deep into the specific categories of Convolutional Neural Network (CNN). They only reviewed these methods in a generalized form, such as supervision strategy, input data handling strategy, etc. This paper presents a systematic review of the literature in automated multiple sclerosis lesion segmentation based on deep learning. Algorithms based on deep learning reviewed are classified into two categories through their CNN style, and their strengths and weaknesses will also be given through our investigation and analysis. We give a quantitative comparison of the methods reviewed through two metrics: Dice Similarity Coefficient (DSC) and Positive Predictive Value (PPV). Finally, the future direction of the application of deep learning in MS lesion segmentation will be discussed.

Deep learning analysis in coronary computed tomographic angiography imaging for the assessment of patients with coronary artery stenosis

BACKGROUND AND OBJECTIVE

Recently, deep convolutional neural network has significantly improved image classification and image segmentation. If coronary artery disease (CAD) can be diagnosed through machine learning and deep learning, it will significantly reduce the burdens of the doctors and accelerate the critical patient diagnoses. The purpose of the study is to assess the practicability of utilizing deep learning approaches to process coronary computed tomographic angiography (CCTA) imaging (termed CCTA-artificial intelligence, CCTA-AI) in coronary artery stenosis.

MATERIALS AND METHODS

A CCTA reconstruction pipeline was built by utilizing deep learning and transfer learning approaches to generate auto-reconstructed CCTA images based on a series of two-dimensional (2D) CT images. 150 patients who underwent successively CCTA and digital subtraction angiography (DSA) from June 2017 to December 2017 were retrospectively analyzed. The dataset was divided into two parts comprising training dataset and testing dataset. The training dataset included the CCTA images of 100 patients which are trained using convolutional neural networks (CNN) in order to further identify various plaque classifications and coronary stenosis. The other 50 CAD patients acted as testing dataset that is evaluated by comparing the auto-reconstructed CCTA images with traditional CCTA images on the condition that DSA images are regarded as the reference method. Receiver operating characteristic (ROC) analysis was used for statistical analysis to compare CCTA-AI with DSA and traditional CCTA in the aspect of detecting coronary stenosis and plaque features.

RESULTS

AI significantly reduces time for post-processing and diagnosis comparing to the traditional methods. In identifying various degrees of coronary stenosis, the diagnostic accuracy of CCTA-AI is better than traditional CCTA (AUC_AI = 0.870, AUC_CCTA = 0.781, P < 0.001). In identifying ≥ 50% stenotic vessels, the accuracy, sensitivity, specificity, positive predictive value and negative predictive value of CCTA-AI and traditional method are 86% and 83%, 88% and 59%, 85% and 94%, 73% and 84%, 94% and 83%, respectively. In the aspect of identifying plaque classification, accuracy of CCTA-AI is moderate compared to traditional CCTA (AUC = 0.750, P < 0.001).

CONCLUSION

The proposed CCTA-AI allows the generation of auto-reconstructed CCTA images from a series of 2D CT images. This approach is relatively accurate for detecting ≥50% stenosis and analyzing plaque features compared to traditional CCTA.

Sequential conditional reinforcement learning for simultaneous vertebral body detection and segmentation with modeling the spine anatomy

Accurate vertebral body (VB) detection and segmentation are critical for spine disease identification and diagnosis. Existing automatic VB detection and segmentation methods may cause false-positive results to the background tissue or inaccurate results to the desirable VB. Because they usually cannot take both the global spine pattern and the local VB appearance into consideration concurrently. In this paper, we propose a Sequential Conditional Reinforcement Learning network (SCRL) to tackle the simultaneous detection and segmentation of VBs from MR spine images. The SCRL, for the first time, applies deep reinforcement learning into VB detection and segmentation. It innovatively models the spatial correlation between VBs from top to bottom as sequential dynamic-interaction processes, thereby globally focusing detection and segmentation on each VB. Simultaneously, SCRL also perceives the local appearance feature of each desirable VB comprehensively, thereby achieving accurate detection and segmentation result. Particularly, SCRL seamlessly combines three parts: 1) Anatomy-Modeling Reinforcement Learning Network dynamically interacts with the image and focuses an attention-region on the VB; 2) Fully-Connected Residual Neural Network learns rich global context information of the VB including both the detailed low-level features and the abstracted high-level features to detect the accurate bounding-box of the VB based on the attention-region; 3) Y-shaped Network learns comprehensive detailed texture information of VB including multi-scale, coarse-to-fine features to segment the boundary of VB from the attention-region. On 240 subjects, SCRL achieves accurate detection and segmentation results, where on average the detection IoU is 92.3%, segmentation Dice is 92.6%, and classification mean accuracy is 96.4%. These excellent results demonstrate that SCRL can be an efficient aided-diagnostic tool to assist clinicians when diagnosing spinal diseases.

Machine Learning-Based Segmentation of Left Ventricular Myocardial Fibrosis from Magnetic Resonance Imaging

PURPOSE OF REVIEW

Myocardial fibrosis (MF) arises due to myocardial infarction and numerous cardiac diseases. MF may lead to several heart disorders, such as heart failure, arrhythmias, and ischemia. Cardiac magnetic resonance (CMR) imaging techniques, such as late gadolinium enhancement (LGE) CMR, enable non-invasive assessment of MF in the left ventricle (LV). Manual assessment of MF on CMR is a tedious and time-consuming task that is subject to high observer variability. Automated segmentation and quantification of MF is important for risk stratification and treatment planning in patients with heart disorders. This article aims to review the machine learning (ML)-based methodologies developed for MF quantification in the LV using CMR images.

RECENT FINDINGS

With the availability of relatively large labeled datasets supervised learning methods based on both conventional ML and state-of-the-art deep learning (DL) methods have been successfully applied for automated segmentation of MF. The incorporation of ML algorithms into imaging techniques such as 3D LGE CMR permits fast characterization of MF on CMR imaging and may enhance the diagnosis and prognosis of patients with heart disorders. Concurrently, the studies using cine CMR images have revealed that accurate segmentation of MF on non-contrast CMR imaging might be possible. The application of ML/DL tools in CMR image interpretation is likely to result in accurate and efficient quantification of MF.

Simultaneous left atrium anatomy and scar segmentations via deep learning in multiview information with attention

Three-dimensional late gadolinium enhanced (LGE) cardiac MR (CMR) of left atrial scar in patients with atrial fibrillation (AF) has recently emerged as a promising technique to stratify patients, to guide ablation therapy and to predict treatment success. This requires a segmentation of the high intensity scar tissue and also a segmentation of the left atrium (LA) anatomy, the latter usually being derived from a separate bright-blood acquisition. Performing both segmentations automatically from a single 3D LGE CMR acquisition would eliminate the need for an additional acquisition and avoid subsequent registration issues. In this paper, we propose a joint segmentation method based on multiview two-task (MVTT) recursive attention model working directly on 3D LGE CMR images to segment the LA (and proximal pulmonary veins) and to delineate the scar on the same dataset. Using our MVTT recursive attention model, both the LA anatomy and scar can be segmented accurately (mean Dice score of 93% for the LA anatomy and 87% for the scar segmentations) and efficiently ( ∼ 0.27 s to simultaneously segment the LA anatomy and scars directly from the 3D LGE CMR dataset with 60-68 2D slices). Compared to conventional unsupervised learning and other state-of-the-art deep learning based methods, the proposed MVTT model achieved excellent results, leading to an automatic generation of a patient-specific anatomical model combined with scar segmentation for patients in AF.

Automatic diagnosis of fungal keratitis using data augmentation and image fusion with deep convolutional neural network

BACKGROUND AND OBJECTIVES

Fungal keratitis is caused by inflammation of the cornea that results from infection by fungal organisms. The lack of an early effective diagnosis often results in serious complications even blindness. Confocal microscopy is one of the most effective methods in the diagnosis of fungal keratitis, but the diagnosis depends on the subjective judgment of medical experts.

METHODS

To address this problem, this paper proposes a novel convolutional neural network framework for the automatic diagnosis of fungal keratitis using data augmentation and image fusion. Firstly, a normal image is augmented by flipping to solve the problem of having a limited and imbalanced database. Secondly, a sub-area contrast stretching algorithm is proposed for image preprocessing to highlight the key structures in the images and to filter out irrelevant information. Thirdly, the histogram matching fusion algorithm is implemented, then the preprocessed image is fused with the original image to form a new algorithm framework and a new database. Finally, the traditional convolutional neural network is integrated into the novel algorithm framework to perform the experiments.

RESULTS

Experiments show that the accuracy of traditional AlexNet and VGGNet is 99.35% and 99.14%, that of AlexNet and VGGNet based on MF fusion is 99.80% and 99.83%, and that of AlexNet and VGGNet based on histogram matching fusion (HMF) is 99.95% and 99.89%. The experimental results show that the AlexNet framework using data augmentation and image fusion achieves a perfect trade-off between the diagnostic performance and the computational complexity, with a diagnostic accuracy of 99.95%.

CONCLUSIONS

These experimental results demonstrate the novel convolutional neural network framework perfectly balances the diagnostic performance and computational complexity, and can improve the effect and real-time performance in the diagnosis of fungal keratitis.

Deep Learning for Cardiac Image Segmentation: A Review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound and major anatomical structures of interest (ventricles, atria, and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

Multi-branch fusion network for Myocardial infarction screening from 12-lead ECG images

BACKGROUND AND OBJECTIVE

Myocardial infarction (MI) is a myocardial anoxic incapacitation caused by severe cardiovascular obstruction that can cause irreversible injury or even death. In medical field, the electrocardiogram (ECG) is a common and effective way to diagnose myocardial infarction, which often requires a wealth of medical knowledge. It is necessary to develop an approach that can detect the MI automatically.

METHODS

In this paper, we propose a multi-branch fusion framework for automatic MI screening from 12-lead ECG images, which consists of multi-branch network, feature fusion and classification network. First, we use text detection and position alignment to automatically separate twelve leads from ECG images. Then, those 12 leads are input into the multi-branch network constructed by a shallow neural network to get 12 feature maps. After concatenating those feature maps by depth fusion, classification is explored to judge the given ECG is MI or not.

RESULTS

Based on extensive experiments on an ECG image dataset, performances of different combinations of structures are analyzed. The proposed network is compared with other networks and also compared with physicians in the practical use. All the experiments verify that the proposed method is effective for MI screening based on ECG images, which achieves accuracy, sensitivity, specificity and F1-score of 94.73%, 96.41%, 95.94% and 93.79% respectively.

CONCLUSIONS

Rather than using the typical one-dimensional electrical ECG signal, this paper gives an effective model to screen MI by analyzing 12-lead ECG images. Extracting and analyzing these 12 leads from their corresponding ECG images is a good attempt in the application of MI screening.