Motion Estimation by Deep Learning in 2D Echocardiography: Synthetic Dataset and Validation

A deep learning phase-based solution in 2D echocardiography motion estimation

In this paper, we propose a new deep learning method based on Quaternion Wavelet Transform (QWT) phases of 2D echocardiographic sequences to estimate the motion and strain of myocardium. The proposed method considers intensity and phases gained from QWT as the inputs of customized PWC-Net structure, a high-performance deep network in motion estimation. We have trained and tested our proposed method performance using two realistic simulated B-mode echocardiographic sequences. We have evaluated our proposed method in terms of both geometrical and clinical indices. Our method achieved an average endpoint error of 0.06 mm per frame and 0.59 mm between End Diastole and End Systole on a simulated dataset. Correlation analysis between ground truth and the computed strain shows a correlation coefficient of 0.89, much better than the most efficient methods in the state-of-the-art 2D echocardiography motion estimation. The results show the superiority of our proposed method in both geometrical and clinical indices.

Coordinate-Independent 3-D Ultrasound Principal Stretch and Direction Imaging

OBJECTIVE

In clinical ultrasound, current 2-D strain imaging faces challenges in quantifying three orthogonal normal strain components. This requires separate image acquisitions based on the pixel-dependent cardiac coordinate system, leading to additional computations and estimation discrepancies due to probe orientation. Most systems lack shear strain information, as displaying all components is challenging to interpret.

METHODS

This paper presents a 3-D high-spatial-resolution, coordinate-independent strain imaging approach based on principal stretch and axis estimation. All strain components are transformed into three principal stretches along three normal principal axes, enabling direct visualization of the primary deformation. We devised an efficient 3-D speckle tracking method with tilt filtering, incorporating randomized searching in a two-pass tracking framework and rotating the phase of the 3-D correlation function for robust filtering. The proposed speckle tracking approach significantly improves estimates of displacement gradients related to the axial displacement component. Non-axial displacement gradient estimates are enhanced using a correlation-weighted least-squares method constrained by tissue incompressibility.

RESULTS

Simulated and in vivo canine cardiac datasets were evaluated to estimate Lagrangian strains from end-diastole to end-systole. The proposed speckle tracking method improves displacement estimation by a factor of 4.3 to 10.5 over conventional 1-pass processing. Maximum principal axis/direction imaging enables better detection of local disease regions than conventional strain imaging.

CONCLUSION

Coordinate-independent tracking can identify myocardial abnormalities with high accuracy.

SIGNIFICANCE

This study offers enhanced accuracy and robustness in strain imaging compared to current methods.

Fast inter-frame motion correction in contrast-free ultrasound quantitative microvasculature imaging using deep learning

Contrast-free ultrasound quantitative microvasculature imaging shows promise in several applications, including the assessment of benign and malignant lesions. However, motion represents one of the major challenges in imaging tumor microvessels in organs that are prone to physiological motions. This study aims at addressing potential microvessel image degradation in in vivo human thyroid due to its proximity to carotid artery. The pulsation of the carotid artery induces inter-frame motion that significantly degrades microvasculature images, resulting in diagnostic errors. The main objective of this study is to reduce inter-frame motion artifacts in high-frame-rate ultrasound imaging to achieve a more accurate visualization of tumor microvessel features. We propose a low-complex deep learning network comprising depth-wise separable convolutional layers and hybrid adaptive and squeeze-and-excite attention mechanisms to correct inter-frame motion in high-frame-rate images. Rigorous validation using phantom and in-vivo data with simulated inter-frame motion indicates average improvements of 35% in Pearson correlation coefficients (PCCs) between motion corrected and reference data with respect to that of motion corrupted data. Further, reconstruction of microvasculature images using motion-corrected frames demonstrates PCC improvement from 31 to 35%. Another thorough validation using in-vivo thyroid data with physiological inter-frame motion demonstrates average improvement of 20% in PCC and 40% in mean inter-frame correlation. Finally, comparison with the conventional image registration method indicates the suitability of proposed network for real-time inter-frame motion correction with 5000 times reduction in motion corrected frame prediction latency.

Deep learning improves test-retest reproducibility of regional strain in echocardiography

Aims

The clinical utility of regional strain measurements in echocardiography is challenged by suboptimal reproducibility. In this study, we aimed to evaluate the test-retest reproducibility of regional longitudinal strain (RLS) per coronary artery perfusion territory (RLSTerritory) and basal-to-apical level of the left ventricle (RLSLevel), measured by a novel fully automated deep learning (DL) method based on point tracking.

Methods and results

We measured strain in a dual-centre test-retest data set that included 40 controls and 40 patients with suspected non-ST elevation acute coronary syndrome. Two consecutive echocardiograms per subject were recorded by different operators. The reproducibility of RLSTerritory and RLSLevel measured by the DL method and by three experienced observers using semi-automatic software (2D Strain, EchoPAC, GE HealthCare) was evaluated as minimal detectable change (MDC). The DL method had MDC for RLSTerritory and RLSLevel ranging from 3.6 to 4.3%, corresponding to a 33-35% improved reproducibility compared with the inter- and intraobserver scenarios (MDC 5.5-6.4% and 4.9-5.4%). Furthermore, the DL method had a lower variance of test-retest differences for both RLSTerritory and RLSLevel compared with inter- and intraobserver scenarios (all P < 0.001). Bland-Altman analyses demonstrated superior reproducibility by the DL method for the whole range of strain values compared with the best observer scenarios. The feasibility of the DL method was 93% and measurement time was only 1 s per echocardiogram.

Conclusion

The novel DL method provided fully automated measurements of RLS, with improved test-retest reproducibility compared with semi-automatic measurements by experienced observers. RLS measured by the DL method has the potential to advance patient care through a more detailed, more efficient, and less user-dependent clinical assessment of myocardial function.

Deep Learning-Derived Myocardial Strain

BACKGROUND

Echocardiographic strain measurements require extensive operator experience and have significant intervendor variability. Creating an automated, open-source, vendor-agnostic method to retrospectively measure global longitudinal strain (GLS) from standard echocardiography B-mode images would greatly improve post hoc research applications and may streamline patient analyses.

OBJECTIVES

This study was seeking to develop an automated deep learning strain (DLS) analysis pipeline and validate its performance across multiple applications and populations.

METHODS

Interobserver/-vendor variation of traditional GLS, and simulated effects of variation in contour on speckle-tracking measurements were assessed. The DLS pipeline was designed to take semantic segmentation results from EchoNet-Dynamic and derive longitudinal strain by calculating change in the length of the left ventricular endocardial contour. DLS was evaluated for agreement with GLS on a large external dataset and applied across a range of conditions that result in cardiac hypertrophy.

RESULTS

In patients scanned by 2 sonographers using 2 vendors, GLS had an intraclass correlation of 0.29 (95% CI: -0.01 to 0.53, P = 0.03) between vendor measurements and 0.63 (95% CI: 0.48-0.74, P < 0.001) between sonographers. With minor changes in initial input contour, step-wise pixel shifts resulted in a mean absolute error of 3.48% and proportional strain difference of 13.52% by a 6-pixel shift. In external validation, DLS maintained moderate agreement with 2-dimensional GLS (intraclass correlation coefficient [ICC]: 0.56, P = 0.002) with a bias of -3.31% (limits of agreement: -11.65% to 5.02%). The DLS method showed differences (P < 0.0001) between populations with cardiac hypertrophy and had moderate agreement in a patient population of advanced cardiac amyloidosis: ICC was 0.64 (95% CI: 0.53-0.72), P < 0.001, with a bias of 0.57%, limits of agreement of -4.87% to 6.01% vs 2-dimensional GLS.

CONCLUSIONS

The open-source DLS provides lower variation than human measurements and similar quantitative results. The method is rapid, consistent, vendor-agnostic, publicly released, and applicable across a wide range of imaging qualities.

Automatic Detection and Tracking of Anatomical Landmarks in Transesophageal Echocardiography for Quantification of Left Ventricular Function

OBJECTIVE

Evaluation of left ventricular (LV) function in critical care patients is useful for guidance of therapy and early detection of LV dysfunction, but the tools currently available are too time-consuming. To resolve this issue, we previously proposed a method for the continuous and automatic quantification of global LV function in critical care patients based on the detection and tracking of anatomical landmarks on transesophageal heart ultrasound. In the present study, our aim was to improve the performance of mitral annulus detection in transesophageal echocardiography (TEE).

METHODS

We investigated several state-of-the-art networks for both the detection and tracking of the mitral annulus in TEE. We integrated the networks into a pipeline for automatic assessment of LV function through estimation of the mitral annular plane systolic excursion (MAPSE), called autoMAPSE. TEE recordings from a total of 245 patients were collected from St. Olav's University Hospital and used to train and test the respective networks. We evaluated the agreement between autoMAPSE estimates and manual references annotated by expert echocardiographers in 30 Echolab patients and 50 critical care patients. Furthermore, we proposed a prototype of autoMAPSE for clinical integration and tested it in critical care patients in the intensive care unit.

RESULTS

Compared with manual references, we achieved a mean difference of 0.8 (95% limits of agreement: -2.9 to 4.7) mm in Echolab patients, with a feasibility of 85.7%. In critical care patients, we reached a mean difference of 0.6 (95% limits of agreement: -2.3 to 3.5) mm and a feasibility of 88.1%. The clinical prototype of autoMAPSE achieved real-time performance.

CONCLUSION

Automatic quantification of LV function had high feasibility in clinical settings. The agreement with manual references was comparable to inter-observer variability of clinical experts.

Ultrafast Cardiac Imaging Using Deep Learning for Speckle-Tracking Echocardiography

High-quality ultrafast ultrasound imaging is based on coherent compounding from multiple transmissions of plane waves (PW) or diverging waves (DW). However, compounding results in reduced frame rate, as well as destructive interferences from high-velocity tissue motion if motion compensation (MoCo) is not considered. While many studies have recently shown the interest of deep learning for the reconstruction of high-quality static images from PW or DW, its ability to achieve such performance while maintaining the capability of tracking cardiac motion has yet to be assessed. In this article, we addressed such issue by deploying a complex-weighted convolutional neural network (CNN) for image reconstruction and a state-of-the-art speckle-tracking method. The evaluation of this approach was first performed by designing an adapted simulation framework, which provides specific reference data, i.e., high-quality, motion artifact-free cardiac images. The obtained results showed that, while using only three DWs as input, the CNN-based approach yielded an image quality and a motion accuracy equivalent to those obtained by compounding 31 DWs free of motion artifacts. The performance was then further evaluated on nonsimulated, experimental in vitro data, using a spinning disk phantom. This experiment demonstrated that our approach yielded high-quality image reconstruction and motion estimation, under a large range of velocities and outperforms a state-of-the-art MoCo-based approach at high velocities. Our method was finally assessed on in vivo datasets and showed consistent improvement in image quality and motion estimation compared to standard compounding. This demonstrates the feasibility and effectiveness of deep learning reconstruction for ultrafast speckle-tracking echocardiography.

Lateral Strain Imaging Using Self-Supervised and Physically Inspired Constraints in Unsupervised Regularized Elastography

Convolutional Neural Networks (CNN) have shown promising results for displacement estimation in UltraSound Elastography (USE). Many modifications have been proposed to improve the displacement estimation of CNNs for USE in the axial direction. However, the lateral strain, which is essential in several downstream tasks such as the inverse problem of elasticity imaging, remains a challenge. The lateral strain estimation is complicated since the motion and the sampling frequency in this direction are substantially lower than the axial one, and a lack of carrier signal in this direction. In computer vision applications, the axial and the lateral motions are independent. In contrast, the tissue motion pattern in USE is governed by laws of physics which link the axial and lateral displacements. In this paper, inspired by Hooke's law, we, first propose Physically Inspired ConsTraint for Unsupervised Regularized Elastography (PICTURE), where we impose a constraint on the Effective Poisson's ratio (EPR) to improve the lateral strain estimation. In the next step, we propose self-supervised PICTURE (sPICTURE) to further enhance the strain image estimation. Extensive experiments on simulation, experimental phantom and in vivo data demonstrate that the proposed methods estimate accurate axial and lateral strain maps.

Fog-Assisted Deep-Learning-Empowered Intrusion Detection System for RPL-Based Resource-Constrained Smart Industries

The Internet of Things (IoT) is a prominent and advanced network communication technology that has familiarized the world with smart industries. The conveniently acquirable nature of IoT makes it susceptible to a diversified range of potential security threats. The literature has brought forth a plethora of solutions for ensuring secure communications in IoT-based smart industries. However, resource-constrained sectors still demand significant attention. We have proposed a fog-assisted deep learning (DL)-empowered intrusion detection system (IDS) for resource-constrained smart industries. The proposed Cuda-deep neural network gated recurrent unit (Cu-DNNGRU) framework was trained on the N-BaIoT dataset and was evaluated on judicious performance metrics, including accuracy, precision, recall, and F1-score. Additionally, the Cu-DNNGRU was empirically investigated alongside state-of-the-art classifiers, including Cu-LSTMDNN, Cu-BLSTM, and Cu-GRU. An extensive performance comparison was also undertaken among the proposed IDS and some outstanding solutions from the literature. The simulation results showed ample strength with respect to the validation of the proposed framework. The proposed Cu-DNNGRU achieved 99.39% accuracy, 99.09% precision, 98.89% recall, and an F1-score of 99.21%. In the performance comparison, the values were substantially higher than those of the benchmarked schemes, as well as competitive security solutions from the literature.

Artificial Intelligence-Enhanced Echocardiography for Systolic Function Assessment

The accurate assessment of left ventricular systolic function is crucial in the diagnosis and treatment of cardiovascular diseases. Left ventricular ejection fraction (LVEF) and global longitudinal strain (GLS) are the most critical indexes of cardiac systolic function. Echocardiography has become the mainstay of cardiac imaging for measuring LVEF and GLS because it is non-invasive, radiation-free, and allows for bedside operation and real-time processing. However, the human assessment of cardiac function depends on the sonographer’s experience, and despite their years of training, inter-observer variability exists. In addition, GLS requires post-processing, which is time consuming and shows variability across different devices. Researchers have turned to artificial intelligence (AI) to address these challenges. The powerful learning capabilities of AI enable feature extraction, which helps to achieve accurate identification of cardiac structures and reliable estimation of the ventricular volume and myocardial motion. Hence, the automatic output of systolic function indexes can be achieved based on echocardiographic images. This review attempts to thoroughly explain the latest progress of AI in assessing left ventricular systolic function and differential diagnosis of heart diseases by echocardiography and discusses the challenges and promises of this new field.

Artificial Intelligence-Enhanced Echocardiography for Systolic Function Assessment

The accurate assessment of left ventricular systolic function is crucial in the diagnosis and treatment of cardiovascular diseases. Left ventricular ejection fraction (LVEF) and global longitudinal strain (GLS) are the most critical indexes of cardiac systolic function. Echocardiography has become the mainstay of cardiac imaging for measuring LVEF and GLS because it is non-invasive, radiation-free, and allows for bedside operation and real-time processing. However, the human assessment of cardiac function depends on the sonographer's experience, and despite their years of training, inter-observer variability exists. In addition, GLS requires post-processing, which is time consuming and shows variability across different devices. Researchers have turned to artificial intelligence (AI) to address these challenges. The powerful learning capabilities of AI enable feature extraction, which helps to achieve accurate identification of cardiac structures and reliable estimation of the ventricular volume and myocardial motion. Hence, the automatic output of systolic function indexes can be achieved based on echocardiographic images. This review attempts to thoroughly explain the latest progress of AI in assessing left ventricular systolic function and differential diagnosis of heart diseases by echocardiography and discusses the challenges and promises of this new field.