Super-Resolution of Cardiac MR Cine Imaging using Conditional GANs and Unsupervised Transfer Learning

A novel GAN-based three-axis mutually supervised super-resolution reconstruction method for rectal cancer MR image

BACKGROUND AND OBJECTIVE

This study aims to enhance the resolution in the axial direction of rectal cancer magnetic resonance (MR) imaging scans to improve the accuracy of visual interpretation and quantitative analysis. MR imaging is a critical technique for the diagnosis and treatment planning of rectal cancer. However, obtaining high-resolution MR images is both time-consuming and costly. As a result, many hospitals store only a limited number of slices, often leading to low-resolution MR images, particularly in the axial plane. Given the importance of image resolution in accurate assessment, these low-resolution images frequently lack the necessary detail, posing substantial challenges for both human experts and computer-aided diagnostic systems. Image super-resolution (SR), a technique developed to enhance image resolution, was originally applied to natural images. Its success has since led to its application in various other tasks, especially in the reconstruction of low-resolution MR images. However, most existing SR methods fail to account for all anatomical planes during reconstruction, leading to unsatisfactory results when applied to rectal cancer MR images.

METHODS

In this paper, we propose a GAN-based three-axis mutually supervised super-resolution reconstruction method tailored for low-resolution rectal cancer MR images. Our approach involves performing one-dimensional (1D) intra-slice SR reconstruction along the axial direction for both the sagittal and coronal planes, coupled with inter-slice SR reconstruction based on slice synthesis in the axial direction. To further enhance the accuracy of super-resolution reconstruction, we introduce a consistency supervision mechanism across the reconstruction results of different axes, promoting mutual learning between each axis. A key innovation of our method is the introduction of Depth-GAN for synthesize intermediate slices in the axial plane, incorporating depth information and leveraging Generative Adversarial Networks (GANs) for this purpose. Additionally, we enhance the accuracy of intermediate slice synthesis by employing a combination of supervised and unsupervised interactive learning techniques throughout the process.

RESULTS

We conducted extensive ablation studies and comparative analyses with existing methods to validate the effectiveness of our approach. On the test set from Shanxi Cancer Hospital, our method achieved a Peak Signal-to-Noise Ratio (PSNR) of 34.62 and a Structural Similarity Index (SSIM) of 96.34 %. These promising results demonstrate the superiority of our method.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Complete spatiotemporal quantification of cardiac motion in mice through enhanced acquisition and super-resolution reconstruction

The quantification of cardiac motion using cardiac magnetic resonance imaging (CMR) has shown promise as an early-stage marker for cardiovascular diseases. Despite the growing popularity of CMR-based myocardial strain calculations, measures of complete spatiotemporal strains (i.e., three-dimensional strains over the cardiac cycle) remain elusive. Complete spatiotemporal strain calculations are primarily hampered by poor spatial resolution, with the rapid motion of the cardiac wall also challenging the reproducibility of such strains. We hypothesize that a super-resolution reconstruction (SRR) framework that leverages combined image acquisitions at multiple orientations will enhance the reproducibility of complete spatiotemporal strain estimation. Two sets of CMR acquisitions were obtained for five wild-type mice, combining short-axis scans with radial and orthogonal long-axis scans. Super-resolution reconstruction, integrated with tissue classification, was performed to generate full four-dimensional (4D) images. The resulting enhanced and full 4D images enabled complete quantification of the motion in terms of 4D myocardial strains. Additionally, the effects of SRR in improving accurate strain measurements were evaluated using an in-silico heart phantom. The SRR framework revealed near isotropic spatial resolution, high structural similarity, and minimal loss of contrast, which led to overall improvements in strain accuracy. In essence, a comprehensive methodology was generated to quantify complete and reproducible myocardial deformation, aiding in the much-needed standardization of complete spatiotemporal strain calculations.

Super-resolution of biomedical volumes with 2D supervision

Volumetric biomedical microscopy has the potential to increase the diagnostic information extracted from clinical tissue specimens and improve the diagnostic accuracy of both human pathologists and computational pathology models. Unfortunately, barriers to integrating 3-dimensional (3D) volumetric microscopy into clinical medicine include long imaging times, poor depth/z-axis resolution, and an insufficient amount of high-quality volumetric data. Leveraging the abundance of high-resolution 2D microscopy data, we introduce masked slice diffusion for super-resolution (MSDSR), which exploits the inherent equivalence in the data-generating distribution across all spatial dimensions of biological specimens. This intrinsic characteristic allows for super-resolution models trained on high-resolution images from one plane (e.g., XY) to effectively generalize to others (XZ, YZ), overcoming the traditional dependency on orientation. We focus on the application of MSDSR to stimulated Raman histology (SRH), an optical imaging modality for biological specimen analysis and intraoperative diagnosis, characterized by its rapid acquisition of high-resolution 2D images but slow and costly optical z-sectioning. To evaluate MSDSR's efficacy, we introduce a new performance metric, SliceFID, and demonstrate MSDSR's superior performance over baseline models through extensive evaluations. Our findings reveal that MSDSR not only significantly enhances the quality and resolution of 3D volumetric data, but also addresses major obstacles hindering the broader application of 3D volumetric microscopy in clinical diagnostics and biomedical research.

Super-resolution techniques for biomedical applications and challenges

Super-resolution (SR) techniques have revolutionized the field of biomedical applications by detailing the structures at resolutions beyond the limits of imaging or measuring tools. These techniques have been applied in various biomedical applications, including microscopy, magnetic resonance imaging (MRI), computed tomography (CT), X-ray, electroencephalogram (EEG), ultrasound, etc. SR methods are categorized into two main types: traditional non-learning-based methods and modern learning-based approaches. In both applications, SR methodologies have been effectively utilized on biomedical images, enhancing the visualization of complex biological structures. Additionally, these methods have been employed on biomedical data, leading to improvements in computational precision and efficiency for biomedical simulations. The use of SR techniques has resulted in more detailed and accurate analyses in diagnostics and research, essential for early disease detection and treatment planning. However, challenges such as computational demands, data interpretation complexities, and the lack of unified high-quality data persist. The article emphasizes these issues, underscoring the need for ongoing development in SR technologies to further improve biomedical research and patient care outcomes.

STADNet: Spatial-Temporal Attention-Guided Dual-Path Network for cardiac cine MRI super-resolution

Cardiac cine magnetic resonance imaging (MRI) is a commonly used clinical tool for evaluating cardiac function and morphology. However, its diagnostic accuracy may be compromised by the low spatial resolution. Current methods for cine MRI super-resolution reconstruction still have limitations. They typically rely on 3D convolutional neural networks or recurrent neural networks, which may not effectively capture long-range or non-local features due to their limited receptive fields. Optical flow estimators are also commonly used to align neighboring frames, which may cause information loss and inaccurate motion estimation. Additionally, pre-warping strategies may involve interpolation, leading to potential loss of texture details and complicated anatomical structures. To overcome these challenges, we propose a novel Spatial-Temporal Attention-Guided Dual-Path Network (STADNet) for cardiac cine MRI super-resolution. We utilize transformers to model long-range dependencies in cardiac cine MR images and design a cross-frame attention module in the location-aware spatial path, which enhances the spatial details of the current frame by using complementary information from neighboring frames. We also introduce a recurrent flow-enhanced attention module in the motion-aware temporal path that exploits the correlation between cine MRI frames and extracts the motion information of the heart. Experimental results demonstrate that STADNet outperforms SOTA approaches and has significant potential for clinical practice.

Novel Techniques in Imaging Congenital Heart Disease: JACC Scientific Statement

Recent years have witnessed exponential growth in cardiac imaging technologies, allowing better visualization of complex cardiac anatomy and improved assessment of physiology. These advances have become increasingly important as more complex surgical and catheter-based procedures are evolving to address the needs of a growing congenital heart disease population. This state-of-the-art review presents advances in echocardiography, cardiac magnetic resonance, cardiac computed tomography, invasive angiography, 3-dimensional modeling, and digital twin technology. The paper also highlights the integration of artificial intelligence with imaging technology. While some techniques are in their infancy and need further refinement, others have found their way into clinical workflow at well-resourced centers. Studies to evaluate the clinical value and cost-effectiveness of these techniques are needed. For techniques that enhance the value of care for congenital heart disease patients, resources will need to be allocated for education and training to promote widespread implementation.

Dual contrast attention-guided multi-frequency fusion for multi-contrast MRI super-resolution

Objective. Multi-contrast magnetic resonance (MR) imaging super-resolution (SR) reconstruction is an effective solution for acquiring high-resolution MR images. It utilizes anatomical information from auxiliary contrast images to improve the quality of the target contrast images. However, existing studies have simply explored the relationships between auxiliary contrast and target contrast images but did not fully consider different anatomical information contained in multi-contrast images, resulting in texture details and artifacts unrelated to the target contrast images.Approach. To address these issues, we propose a dual contrast attention-guided multi-frequency fusion (DCAMF) network to reconstruct SR MR images from low-resolution MR images, which adaptively captures relevant anatomical information and processes the texture details and low-frequency information from multi-contrast images in parallel. Specifically, after the feature extraction, a feature selection module based on a dual contrast attention mechanism is proposed to focus on the texture details of the auxiliary contrast images and the low-frequency features of the target contrast images. Then, based on the characteristics of the selected features, a high- and low-frequency fusion decoder is constructed to fuse these features. In addition, a texture-enhancing module is embedded in the high-frequency fusion decoder, to highlight and refine the texture details of the auxiliary contrast and target contrast images. Finally, the high- and low-frequency fusion process is constrained by integrating a deeply-supervised mechanism into the DCAMF network.Main results. The experimental results show that the DCAMF outperforms other state-of-the-art methods. The peak signal-to-noise ratio and structural similarity of DCAMF are 39.02 dB and 0.9771 on the IXI dataset and 37.59 dB and 0.9770 on the BraTS2018 dataset, respectively. The image recovery is further validated in segmentation tasks.Significance. Our proposed SR model can enhance the quality of MR images. The results of the SR study provide a reliable basis for clinical diagnosis and subsequent image-guided treatment.

Reconstruction and completion of high-resolution 3D cardiac shapes using anisotropic CMRI segmentations and continuous implicit neural representations

Since the onset of computer-aided diagnosis in medical imaging, voxel-based segmentation has emerged as the primary methodology for automatic analysis of left ventricle (LV) function and morphology in cardiac magnetic resonance images (CMRI). In standard clinical practice, simultaneous multi-slice 2D cine short-axis MR imaging is performed under multiple breath-holds resulting in highly anisotropic 3D images. Furthermore, sparse-view CMRI often lacks whole heart coverage caused by large slice thickness and often suffers from inter-slice misalignment induced by respiratory motion. Therefore, these volumes only provide limited information about the true 3D cardiac anatomy which may hamper highly accurate assessment of functional and anatomical abnormalities. To address this, we propose a method that learns a continuous implicit function representing 3D LV shapes by training an auto-decoder. For training, high-resolution segmentations from cardiac CT angiography are used. The ability of our approach to reconstruct and complete high-resolution shapes from manually or automatically obtained sparse-view cardiac shape information is evaluated by using paired high- and low-resolution CMRI LV segmentations. The results show that the reconstructed LV shapes have an unconstrained subvoxel resolution and appear smooth and plausible in through-plane direction. Furthermore, Bland-Altman analysis reveals that reconstructed high-resolution ventricle volumes are closer to the corresponding reference volumes than reference low-resolution volumes with bias of [limits of agreement] -3.51 [-18.87, 11.85] mL, and 12.96 [-10.01, 35.92] mL respectively. Finally, the results demonstrate that the proposed approach allows recovering missing shape information and can indirectly correct for limited motion-induced artifacts.

Generalizable Deep Learning Method for Suppressing Unseen and Multiple MRI Artifacts Using Meta-learning

Magnetic Resonance (MR) images suffer from various types of artifacts due to motion, spatial resolution, and under-sampling. Conventional deep learning methods deal with removing a specific type of artifact, leading to separately trained models for each artifact type that lack the shared knowledge generalizable across artifacts. Moreover, training a model for each type and amount of artifact is a tedious process that consumes more training time and storage of models. On the other hand, the shared knowledge learned by jointly training the model on multiple artifacts might be inadequate to generalize under deviations in the types and amounts of artifacts. Model-agnostic meta-learning (MAML), a nested bi-level optimization framework is a promising technique to learn common knowledge across artifacts in the outer level of optimization, and artifact-specific restoration in the inner level. We propose curriculum-MAML (CMAML), a learning process that integrates MAML with curriculum learning to impart the knowledge of variable artifact complexity to adaptively learn restoration of multiple artifacts during training. Comparative studies against Stochastic Gradient Descent and MAML, using two cardiac datasets reveal that CMAML exhibits (i) better generalization with improved PSNR for 83% of unseen types and amounts of artifacts and improved SSIM in all cases, and (ii) better artifact suppression in 4 out of 5 cases of composite artifacts (scans with multiple artifacts).Clinical relevance- Our results show that CMAML has the potential to minimize the number of artifact-specific models; which is essential to deploy deep learning models for clinical use. Furthermore, we have also taken another practical scenario of an image affected by multiple artifacts and show that our method performs better in 80% of cases.

Left ventricle segmentation combining deep learning and deformable models with anatomical constraints

Segmentation of the left ventricle is a key approach in Cardiac Magnetic Resonance Imaging for calculating biomarkers in diagnosis. Since there is substantial effort required from the expert, many automatic segmentation methods have been proposed, in which deep learning networks have obtained remarkable performance. However, one of the main limitations of these approaches is the production of segmentations that contain anatomical errors. To avoid this limitation, we propose a new fully-automatic left ventricle segmentation method combining deep learning and deformable models. We propose a new level set energy formulation that includes exam-specific information estimated from the deep learning segmentation and shape constraints. The method is part of a pipeline containing pre-processing steps and a failure correction post-processing step. Experiments were conducted with the Sunnybrook and ACDC public datasets, and a private dataset. Results suggest that the method is competitive, that it can produce anatomically consistent segmentations, has good generalization ability, and is often able to estimate biomarkers close to the expert.

Deep learning-based super-resolution gradient echo imaging of the pancreas: Improvement of image quality and reduction of acquisition time

PURPOSE

The purpose of this study was to evaluate the impact of a deep learning-based super-resolution technique on T1-weighted gradient-echo acquisitions (volumetric interpolated breath-hold examination; VIBE) on the assessment of pancreatic MRI at 1.5 T compared to standard VIBE imaging (VIBESTD).

MATERIALS AND METHODS

This retrospective single-center study was conducted between April 2021 and October 2021. Fifty patients with a total of 50 detectable pancreatic lesion entities were included in this study. There were 27 men and 23 women, with a mean age of 69 ± 13 (standard deviation [SD]) years (age range: 33-89 years). VIBESTD (precontrast, dynamic, postcontrast) was retrospectively processed with a deep learning-based super-resolution algorithm including a more aggressive partial Fourier setting leading to a simulated acquisition time reduction (VIBESR). Image analysis was performed by two radiologists regarding lesion detectability, noise levels, sharpness and contrast of pancreatic edges, as well as regarding diagnostic confidence using a 5-point Likert-scale with 5 being the best.

RESULTS

VIBESR was rated better than VIBESTD by both readers regarding lesion detectability (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5], for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5]) for reader 2; both P <0.001), noise levels (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5] for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5] for reader 2; both P <0.001), sharpness and contrast of pancreatic edges (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5] for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5] for reader 2; both P <0.001), as well as regarding diagnostic confidence (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5] for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5] for reader 2; both P <0.001). There were no significant differences between lesion sizes as measured by the two readers on VIBESR and VIBESTD images (P > 0.05). The mean acquisition time for VIBESTD (15 ± 1 [SD] s; range: 11-16 s) was longer than that for VIBESR (13 ± 1 [SD] s; range: 11-14 s) (P < 0.001).

CONCLUSION

Our results indicate that the newly developed deep learning-based super-resolution algorithm adapted to partial Fourier acquisitions has a positive influence not only on shortening the examination time but also on improvement of image quality in pancreatic MRI.

3D model-based super-resolution motion-corrected cardiac T1 mapping

Objective. To provide 3D high-resolution cardiac T1 maps using model-based super-resolution reconstruction (SRR).Approach. Due to signal-to-noise ratio limitations and the motion of the heart during imaging, often 2D T1 maps with only low through-plane resolution (i.e. slice thickness of 6-8 mm) can be obtained. Here, a model-based SRR approach is presented, which combines multiple stacks of 2D acquisitions with 6-8 mm slice thickness and generates 3D high-resolution T1 maps with a slice thickness of 1.5-2 mm. Every stack was acquired in a different breath hold (BH) and any misalignment between BH was corrected retrospectively. The novelty of the proposed approach is the BH correction and the application of model-based SRR on cardiac T1 Mapping. The proposed approach was evaluated in numerical simulations and phantom experiments and demonstrated in four healthy subjects.Main results. Alignment of BH states was essential for SRR even in healthy volunteers. In simulations, respiratory motion could be estimated with an RMS error of 0.18 ± 0.28 mm. SRR improved the visualization of small structures. High accuracy and precision (average standard deviation of 69.62 ms) of the T1 values was ensured by SRR while the detectability of small structures increased by 40%.Significance. The proposed SRR approach provided T1 maps with high in-plane and high through-plane resolution (1.3 × 1.3 × 1.5-2 mm³). The approach led to improvements in the visualization of small structures and precise T1 values.

Myocardial perfusion in excessively trabeculated hearts: Insights from imaging and histological studies

In gestation, the coronary circulation develops initially in the compact layer and it expands only in fetal development to the trabeculations. Conflicting data have been published as to whether the trabecular layer is hypoperfused relative to the compact wall after birth. If so, this could explain the poor pump function in patients with left ventricular excessive trabeculation, or so-called noncompaction. Here, we review direct and indirect assessments of myocardial perfusion in normal and excessively trabeculated hearts by in vivo imaging by magnetic resonance imaging (MRI), positron emission tomography (PET)/single photon emission computed tomography (SPECT), and echocardiography in addition to histology, injections of labelled microspheres in animals, and electrocardiography. In MRI, PET/SPECT, and echocardiography, flow of blood or myocardial uptake of blood-borne tracer molecules are measured. The imaged trabecular layer comprises trabeculations and blood-filled intertrabecular spaces whereas the compact layer comprises tissue only, and spatio-temporal resolution likely affects measurements of myocardial perfusion differently in the two layers. Overall, studies measuring myocardial uptake of tracers (PET/SPECT) suggest trabecular hypoperfusion. Studies measuring the quantity of blood (echocardiography and MRI) suggest trabecular hyperperfusion. These conflicting results are reconciled if the low uptake from intertrabecular spaces in PET/SPECT and the high signal from intertrabecular spaces in MRI and echocardiography are considered opposite biases. Histology on human hearts reveal a similar capillary density of trabecular and compact myocardium. Injections of labelled microspheres in animals reveal a similar perfusion of trabecular and compact myocardium. In conclusion, trabecular and compact muscle are likely equally perfused in normal hearts and most cases of excessive trabeculation.

[Artificial intelligence and radiomics : Value in cardiac MRI]

CLINICAL/METHODICAL ISSUE

Cardiac diseases are the leading cause of death. Many diseases can be specifically treated once a valid diagnosis is established. Cardiac magnetic resonance imaging (MRI) plays a central role in the workup of many cardiac pathologies. However, image acquisition as well as interpretation and related secondary image evaluation are time-consuming and complex.

STANDARD RADIOLOGICAL METHODS

Cardiac MRI is becoming increasingly established in international guidelines for the evaluation of cardiac function and differential diagnosis of a wide variety of cardiac diseases.

METHODOLOGICAL INNOVATIONS

Cardiac MRI has limited reproducibility due to the acquisition technique and interpretation of findings with complex secondary measurements. Artificial intelligence techniques and radiomics offer the potential to improve the acquisition, interpretation, and reproducibility of cardiac MRI.

PERFORMANCE

Research suggests that artificial intelligence and radiomic analysis can improve cardiac MRI in terms of image acquisition and also diagnostic and prognostic value. Furthermore, the implementation of artificial intelligence and radiomics may result in the identification of new biomarkers.

ACHIEVEMENTS AND PRACTICAL RECOMMENDATIONS

The implementation of artificial intelligence in cardiac MRI has great potential. However, the current level of evidence is still limited in some aspects; in particular there are too few prospective and large multicenter studies available. As a result, the algorithms developed are often not sufficiently validated scientifically and are not yet applied in clinical routine.

Comparison of DeepStrain and Feature Tracking for Cardiac MRI Strain Analysis

BACKGROUND

Myocardial feature tracking (FT) provides a comprehensive analysis of myocardial deformation from cine balanced steady-state free-precession images (bSSFP). However, FT remains time-consuming, precluding its clinical adoption.

PURPOSE

To compare left-ventricular global radial strain (GRS) and global circumferential strain (GCS) values measured using automated DeepStrain analysis of short-axis cine images to those calculated using manual commercially available FT analysis.

STUDY TYPE

Retrospective, single-center.

POPULATION

A total of 30 healthy subjects and 120 patients with cardiac disease for DeepStrain development. For evaluation, 47 healthy subjects (36 male, 53 ± 5 years) and 533 patients who had undergone a clinical cardiac MRI (373 male, 59 ± 14 years). FIELD STRENGTH/SEQUENCE: bSSFP sequence at 1.5 T (Phillips) and 3 T (Siemens).

ASSESSMENT

Automated DeepStrain measurements of GRS and GCS were compared to commercially available FT (Circle, cvi42) measures obtained by readers with 1 year and 3 years of experience. Comparisons were performed overall and stratified by scanner manufacturer.

STATISTICAL TESTS

Paired t-test, linear regression slope, Pearson correlation coefficient (r).

RESULTS

Overall, FT and DeepStrain measurements of GCS were not significantly different (P = 0.207), but measures of GRS were significantly different. Measurements of GRS from Philips (slope = 1.06 [1.03 1.08], r = 0.85) and Siemens (slope = 1.04 [0.99 1.09], r = 0.83) data showed a very strong correlation and agreement between techniques. Measurements of GCS from Philips (slope = 0.98 [0.98 1.01], r = 0.91) and Siemens (slope = 1.0 [0.96 1.03], r = 0.88) data similarly showed a very strong correlation. The average analysis time per subject was 4.1 ± 1.2 minutes for FT and 34.7 ± 3.3 seconds for DeepStrain, representing a 7-fold reduction in analysis time.

DATA CONCLUSION

This study demonstrated high correlation of myocardial GCS and GRS measurements between freely available fully automated DeepStrain and commercially available manual FT software, with substantial time-saving in the analysis.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 3.

Deep learning-based automatic segmentation of images in cardiac radiography: A promising challenge

BACKGROUND

Due to the advancement of medical imaging and computer technology, machine intelligence to analyze clinical image data increases the probability of disease prevention and successful treatment. When diagnosing and detecting heart disease, medical imaging can provide high-resolution scans of every organ or tissue in the heart. The diagnostic results obtained by the imaging method are less susceptible to human interference. They can process numerous patient information, assist doctors in early detection of heart disease, intervene and treat patients, and improve the understanding of heart disease symptoms and clinical diagnosis of great significance. In a computer-aided diagnosis system, accurate segmentation of cardiac scan images is the basis and premise of subsequent thoracic function analysis and 3D image reconstruction.

EXISTING TECHNIQUES

This paper systematically reviews automatic methods and some difficulties for cardiac segmentation in radiographic images. Combined with recent advanced deep learning techniques, the feasibility of using deep learning network models for image segmentation is discussed, and the commonly used deep learning frameworks are compared.

DEVELOPED INSIGHTS

There are many standard methods for medical image segmentation, such as traditional methods based on regions and edges and methods based on deep learning. Because of characteristics of non-uniform grayscale, individual differences, artifacts and noise of medical images, the above image segmentation methods have certain limitations. It is tough to obtain the needed results sensitivity and accuracy when performing heart segmentation. The deep learning model proposed has achieved good results in image segmentation. Accurate segmentation improves the accuracy of disease diagnosis and reduces subsequent irrelevant computations.

SUMMARY

There are two requirements for accurate segmentation of radiological images. One is to use image segmentation to improve the development of computer-aided diagnosis. The other is to achieve complete segmentation of the heart. When there are lesions or deformities in the heart, there will be some abnormalities in the radiographic images, and the segmentation algorithm needs to segment the heart altogether. The quantity of processing inside a certain range will no longer be a restriction for real-time detection with the advancement of deep learning and the enhancement of hardware device performance.

Generative Adversarial Networks in Cardiology

Generative Adversarial Networks (GANs) are state-of-the-art neural network models used to synthesize images and other data. GANs brought a considerable improvement to the quality of synthetic data, quickly becoming the standard for data generation tasks. In this work, we summarize the applications of GANs in the field of cardiology, including generation of realistic cardiac images, electrocardiography signals, and synthetic electronic health records. The utility of GAN-generated data is discussed with respect to research, clinical care, and academia. Moreover, we present illustrative examples of our GAN-generated cardiac magnetic resonance and echocardiography images, showing the evolution in image quality across six different models, which has become almost indistinguishable from real images. Finally, we discuss future applications, such as modality translation or patient trajectory modeling. Moreover, we discuss the pending challenges that GANs need to overcome, namely their training dynamics, the medical fidelity or the data regulations and ethics questions, to become integrated in cardiology workflows.

Higher spatial resolution improves the interpretation of the extent of ventricular trabeculation

The ventricular walls of the human heart comprise an outer compact layer and an inner trabecular layer. In the context of an increased pre-test probability, diagnosis left ventricular noncompaction cardiomyopathy is given when the left ventricle is excessively trabeculated in volume (trabecular vol >25% of total LV wall volume) or thickness (trabecular/compact (T/C) >2.3). Here, we investigated whether higher spatial resolution affects the detection of trabeculation and thus the assessment of normal and excessively trabeculated wall morphology. First, we screened left ventricles in 1112 post-natal autopsy hearts. We identified five excessively trabeculated hearts and this low prevalence of excessive trabeculation is in agreement with pathology reports but contrasts the prevalence of approximately 10% of the population found by in vivo non-invasive imaging. Using macroscopy, histology and low- and high-resolution MRI, the five excessively trabeculated hearts were compared with six normal hearts and seven abnormally trabeculated and excessive trabeculation-negative hearts. Some abnormally trabeculated hearts could be considered excessively trabeculated macroscopically because of a trabecular outflow or an excessive number of trabeculations, but they were excessive trabeculation-negative when assessed with MRI-based measurements (T/C <2.3 and vol <25%). The number of detected trabeculations and T/C ratio were positively correlated with higher spatial resolution. Using measurements on high resolution MRI and with histological validation, we could not replicate the correlation between trabeculations of the left and right ventricle that has been previously reported. In conclusion, higher spatial resolution may affect the sensitivity of diagnostic measurements and in addition could allow for novel measurements such as counting of trabeculations.

StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

Deep learning applications on computer vision involve the use of large-volume and representative data to obtain state-of-the-art results due to the massive number of parameters to optimise in deep models. However, data are limited with asymmetric distributions in industrial applications due to rare cases, legal restrictions, and high image-acquisition costs. Data augmentation based on deep learning generative adversarial networks, such as StyleGAN, has arisen as a way to create training data with symmetric distributions that may improve the generalisation capability of built models. StyleGAN generates highly realistic images in a variety of domains as a data augmentation strategy but requires a large amount of data to build image generators. Thus, transfer learning in conjunction with generative models are used to build models with small datasets. However, there are no reports on the impact of pre-trained generative models, using transfer learning. In this paper, we evaluate a StyleGAN generative model with transfer learning on different application domains—training with paintings, portraits, Pokémon, bedrooms, and cats—to generate target images with different levels of content variability: bean seeds (low variability), faces of subjects between 5 and 19 years old (medium variability), and charcoal (high variability). We used the first version of StyleGAN due to the large number of publicly available pre-trained models. The Fréchet Inception Distance was used for evaluating the quality of synthetic images. We found that StyleGAN with transfer learning produced good quality images, being an alternative for generating realistic synthetic images in the evaluated domains.