Direct Segmentation-Based Full Quantification for Left Ventricle via Deep Multi-Task Regression Learning Network

A task-unified network with transformer and spatial-temporal convolution for left ventricular quantification

Quantification of the cardiac function is vital for diagnosing and curing the cardiovascular diseases. Left ventricular function measurement is the most commonly used measure to evaluate the function of cardiac in clinical practice, how to improve the accuracy of left ventricular quantitative assessment results has always been the subject of research by medical researchers. Although considerable efforts have been put forward to measure the left ventricle (LV) automatically using deep learning methods, the accurate quantification is yet a challenge work as a result of the changeable anatomy structure of heart in the systolic diastolic cycle. Besides, most methods used direct regression method which lacks of visual based analysis. In this work, a deep learning segmentation and regression task-unified network with transformer and spatial-temporal convolution is proposed to segment and quantify the LV simultaneously. The segmentation module leverages a U-Net like 3D Transformer model to predict the contour of three anatomy structures, while the regression module learns spatial-temporal representations from the original images and the reconstruct feature map from segmentation path to estimate the finally desired quantification metrics. Furthermore, we employ a joint task loss function to train the two module networks. Our framework is evaluated on the MICCAI 2017 Left Ventricle Full Quantification Challenge dataset. The results of experiments demonstrate the effectiveness of our framework, which achieves competitive cardiac quantification metric results and at the same time produces visualized segmentation results that are conducive to later analysis.

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

An Overview of Deep Learning Methods for Left Ventricle Segmentation

Cardiac health diseases are one of the key causes of death around the globe. The number of heart patients has considerably increased during the pandemic. Therefore, it is crucial to assess and analyze the medical and cardiac images. Deep learning architectures, specifically convolutional neural networks have profoundly become the primary choice for the assessment of cardiac medical images. The left ventricle is a vital part of the cardiovascular system where the boundary and size perform a significant role in the evaluation of cardiac function. Due to automatic segmentation and good promising results, the left ventricle segmentation using deep learning has attracted a lot of attention. This article presents a critical review of deep learning methods used for the left ventricle segmentation from frequently used imaging modalities including magnetic resonance images, ultrasound, and computer tomography. This study also demonstrates the details of the network architecture, software, and hardware used for training along with publicly available cardiac image datasets and self-prepared dataset details incorporated. The summary of the evaluation matrices with results used by different researchers is also presented in this study. Finally, all this information is summarized and comprehended in order to assist the readers to understand the motivation and methodology of various deep learning models, as well as exploring potential solutions to future challenges in LV segmentation.

A systematic review of multi-slice and multi-frame descriptors in cardiac MRI exams

Computer-Aided Diagnosis systems have been developed to help medical professional in their decision making routines towards a more accurate diagnosis. These systems process medical exams such as Magnetic Resonance (MRI) in order to quantify meaningful features. These can be used with similarity-measuring techniques in a Content-Based Image Retrieval context, or inputted into a machine learning classifier in order to support early disease detection. For cardiac MRIs, single slice descriptors have been proposed in the two-dimensional domain, shape descriptors have been proposed in the three-dimensional domain, and previous reviews have mapped these two descriptor categories. Nonetheless, no systematic review on these descriptors have looked at full cardiac MRI images sets. We have reviewed the literature by searching for descriptors that consider the whole slice set (multi-slice) or frames (multi-frame) in cardiac MRI exams. We discuss descriptors and techniques, the datasets that were used, and the different evaluation metrics. Finally, we highlight literature gaps and research opportunities.

TRSA-Net: Task Relation Spatial co-Attention for Joint Segmentation, Quantification and Uncertainty Estimation on Paired 2D Echocardiography

Clinical workflow of cardiac assessment on 2D echocardiography requires both accurate segmentation and quantification of the Left Ventricle (LV) from paired apical 4-chamber and 2-chamber. Moreover, uncertainty estimation is significant in clinically understanding the performance of a model. However, current research on 2D echocardiography ignores this vital task while joint segmentation with quantification, hence motivating the need for a unified optimization method. In this paper, we propose a multitask model with Task Relation Spatial co-Attention (referred as TRSA-Net) for joint segmentation, quantification, and uncertainty estimation on paired 2D echo. TRSA-Net achieves multitask joint learning by novelly exploring the spatial correlation between tasks. The task relation spatial co-attention learns the spatial mapping among task-specific features by non-local and co-excitation, which forcibly joints embedded spatial information in the segmentation and quantification. The Boundary-aware Structure Consistency (BSC) and Joint Indices Constraint (JIC) are integrated into the multitask learning optimization objective to guide the learning of segmentation and quantification paths. The BSC creatively promotes structural similarity of predictions, and JIC explores the internal relationship between three quantitative indices. We validate the efficacy of our TRSA-Net on the public CAMUS dataset. Extensive comparison and ablation experiments show that our approach can achieve competitive segmentation performance and highly accurate results on quantification.

Automatic Left Ventricle Segmentation from Short-Axis Cardiac MRI Images Based on Fully Convolutional Neural Network

Background: Left ventricle (LV) segmentation using a cardiac magnetic resonance imaging (MRI) dataset is critical for evaluating global and regional cardiac functions and diagnosing cardiovascular diseases. LV clinical metrics such as LV volume, LV mass and ejection fraction (EF) are frequently extracted based on the LV segmentation from short-axis MRI images. Manual segmentation to assess such functions is tedious and time-consuming for medical experts to diagnose cardiac pathologies. Therefore, a fully automated LV segmentation technique is required to assist medical experts in working more efficiently. Method: This paper proposes a fully convolutional network (FCN) architecture for automatic LV segmentation from short-axis MRI images. Several experiments were conducted in the training phase to compare the performance of the network and the U-Net model with various hyper-parameters, including optimization algorithms, epochs, learning rate, and mini-batch size. In addition, a class weighting method was introduced to avoid having a high imbalance of pixels in the classes of image’s labels since the number of background pixels was significantly higher than the number of LV and myocardium pixels. Furthermore, effective image conversion with pixel normalization was applied to obtain exact features representing target organs (LV and myocardium). The segmentation models were trained and tested on a public dataset, namely the evaluation of myocardial infarction from the delayed-enhancement cardiac MRI (EMIDEC) dataset. Results: The dice metric, Jaccard index, sensitivity, and specificity were used to evaluate the network’s performance, with values of 0.93, 0.87, 0.98, and 0.94, respectively. Based on the experimental results, the proposed network outperforms the standard U-Net model and is an advanced fully automated method in terms of segmentation performance. Conclusion: This proposed method is applicable in clinical practice for doctors to diagnose cardiac diseases from short-axis MRI images.

Transfer Learning of the ResNet-18 and DenseNet-121 Model Used to Diagnose Intracranial Hemorrhage in CT Scanning

OBJECTIVE

The aim of the study was to verify the ability of the deep learning model to identify five subtypes and normal images in non-contrast enhancement CT of intracranial hemorrhage.

METHODS

A total of 351 patients (39 patients in the normal group, 312 patients in the intracranial hemorrhage group) who underwent intracranial hemorrhage noncontrast enhanced CT were selected, obtaining 2768 images in total (514 images for the normal group, 398 images for the epidural hemorrhage group, 501 images for the subdural hemorrhage group, 497 images for the intraventricular hemorrhage group, 415 images for the cerebral parenchymal hemorrhage group, and 443 images for the subarachnoid hemorrhage group). Based on the diagnostic reports of two radiologists with more than 10 years of experience, the ResNet-18 and DenseNet-121 deep learning models were selected. Transfer learning was used. 80% of the data was used for training models, 10% was used for validating model performance against overfitting, and the last 10% was used for the final evaluation of the model. Assessment indicators included accuracy, sensitivity, specificity, and AUC values.

RESULTS

The overall accuracy of ResNet-18 and DenseNet-121 models was obtained as 89.64% and 82.5%, respectively. The sensitivity and specificity of identifying five subtypes and normal images were above 0.80. The sensitivity of the DenseNet-121 model to recognize intraventricular hemorrhage and cerebral parenchymal hemorrhage was lower than 0.80, 0.73, and 0.76, respectively. The AUC values of the two deep learning models were found to be above 0.9.

CONCLUSION

The deep learning model can accurately identify the five subtypes of intracranial hemorrhage and normal images, and it can be used as a new tool for clinical diagnosis in the future.

TSU-net: Two-stage multi-scale cascade and multi-field fusion U-net for right ventricular segmentation

Accurate segmentation of the right ventricle from cardiac magnetic resonance images (MRI) is a critical step in cardiac function analysis and disease diagnosis. It is still an open problem due to some difficulties, such as a large variety of object sizes and ill-defined borders. In this paper, we present a TSU-net network that grips deeper features and captures targets of different sizes with multi-scale cascade and multi-field fusion in the right ventricle. TSU-net mainly contains two major components: Dilated-Convolution Block (DB) and Multi-Layer-Pool Block (MB). DB extracts and aggregates multi-scale features for the right ventricle. MB mainly relies on multiple effective field-of-views to detect objects at different sizes and fill boundary features. Different from previous networks, we used DB and MB to replace the convolution layer in the encoding layer, thus, we can gather multi-scale information of right ventricle, detect different size targets and fill boundary information in each encoding layer. In addition, in the decoding layer, we used DB to replace the convolution layer, so that we can aggregate the multi-scale features of the right ventricle in each decoding layer. Furthermore, the two-stage U-net structure is used to further improve the utilization of DB and MB through a two-layer encoding/decoding layer. Our method is validated on the RVSC, a public right ventricular data set. The results demonstrated that TSU-net achieved an average Dice coefficient of 0.86 on endocardium and 0.90 on the epicardium, thereby outperforming other models. It effectively assists doctors to diagnose the disease and promotes the development of medical images. In addition, we also provide an intuitive explanation of our network, which fully explain MB and TSU-net's ability to detect targets of different sizes and fill in boundary features.

An FSCV Deep Neural Network: Development, Pruning, and Acceleration on an FPGA

Fast-scan cyclic voltammetry (FSCV) is an electrochemical technique for measuring rapid changes in the extracellular concentration of neurotransmitters within the brain. Due to its fast scan rate and large output-data size, the current analysis of the FSCV data is often conducted on a computer external to the FSCV device. Moreover, the analysis is semi-automated and requires a good understanding of the characteristics of the underlying chemistry to interpret, making it unsuitable for real-time implementation on low-resource FSCV devices. This paper presents a hardware-software co-design approach for the analysis of FSCV data. Firstly, a deep neural network (DNN) is developed to predict the concentration of a dopamine solution and identify the data recording electrode. Secondly, the DNN is pruned to decrease its computation complexity, and a custom overlay is developed to implement the pruned DNN on a low-resource FPGA-based platform. The pruned DNN attains a recognition accuracy of 97.2% with a compression ratio of 3.18. When the DNN overlay is implemented on a PYNQ-Z2 platform, it achieves the execution time of 13 ms and power consumption of 1.479 W on the entire PYNQ-Z2 board. This study demonstrates the possibility of operating the DNN for FSCV data analysis on portable FPGA-based platforms.

Dual attention enhancement feature fusion network for segmentation and quantitative analysis of paediatric echocardiography

Paediatric echocardiography is a standard method for screening congenital heart disease (CHD). The segmentation of paediatric echocardiography is essential for subsequent extraction of clinical parameters and interventional planning. However, it remains a challenging task due to (1) the considerable variation of key anatomic structures, (2) the poor lateral resolution affecting accurate boundary definition, (3) the existence of speckle noise and artefacts in echocardiographic images. In this paper, we propose a novel deep network to address these challenges comprehensively. We first present a dual-path feature extraction module (DP-FEM) to extract rich features via a channel attention mechanism. A high- and low-level feature fusion module (HL-FFM) is devised based on spatial attention, which selectively fuses rich semantic information from high-level features with spatial cues from low-level features. In addition, a hybrid loss is designed to deal with pixel-level misalignment and boundary ambiguities. Based on the segmentation results, we derive key clinical parameters for diagnosis and treatment planning. We extensively evaluate the proposed method on 4,485 two-dimensional (2D) paediatric echocardiograms from 127 echocardiographic videos. The proposed method consistently achieves better segmentation performance than other state-of-the-art methods, whichdemonstratesfeasibility for automatic segmentation and quantitative analysis of paediatric echocardiography. Our code is publicly available at https://github.com/end-of-the-century/Cardiac.

A novel solution of using deep learning for left ventricle detection: Enhanced feature extraction

BACKGROUND AND AIM

deep learning algorithms have not been successfully used for the left ventricle (LV) detection in echocardiographic images due to overfitting and vanishing gradient descent problem. This research aims to increase accuracy and improves the processing time of the left ventricle detection process by reducing the overfitting and vanishing gradient problem.

METHODOLOGY

the proposed system consists of an enhanced deep convolutional neural network with an extra convolutional layer, and dropout layer to solve the problem of overfitting and vanishing gradient. Data augmentation was used for increasing the accuracy of feature extraction for left ventricle detection.

RESULTS

four pathological groups of datasets were used for training and evaluation of the model: heart failure without infarction, heart failure with infarction, and hypertrophy, and healthy. The proposed model provided an accuracy of 94% in left ventricle detection for all the groups compared to the other current systems. The results showed that the processing time was reduced from 0.45 s to 0.34 s in an average.

CONCLUSION

the proposed system enhances accuracy and decreases processing time in the left ventricle detection. This paper solves the issues of overfitting of the data.

A 12-Lead ECG-Based System With Physiological Parameters and Machine Learning to Identify Right Ventricular Hypertrophy in Young Adults

OBJECTIVE

The presence of right ventricular hypertrophy (RVH) accounts for approximately 5-10% in young adults. The sensitivity estimated by commonly used 12-lead electrocardiographic (ECG) criteria for identifying the presence of RVH is under 20% in the general population. The aim of this study is to develop a 12-lead ECG system with the related information of age, body height and body weight via machine learning to increase the sensitivity and the precision for detecting RVH.

METHOD

In a sample of 1,701 males, aged 17-45 years, support vector machine is used for the training of 31 parameters including age, body height and body weight in addition to 28 ECG data such as axes, intervals and wave voltages as the inputs to link the output RVH. The RVH is defined on the echocardiographic finding for young males as right ventricular anterior wall thickness > 5.5 mm.

RESULTS

On the system goal for increasing sensitivity, the specificity is controlled around 70-75% and all data tested in the proposed method show competent sensitivity up to 70.3%. The values of area under curve of receiver operating characteristic curve and precision-recall curve using the proposed method are 0.780 and 0.285, respectively, which are better than 0.518 and 0.112 using the Sokolow-Lyon voltage criterion, respectively, for detecting unspecific RVH.

CONCLUSION

We present a method using simple physiological parameters with ECG data to effectively identify more than 70% of the RVH among young adults. Clinical Impact: This system provides a fast, precise and feasible diagnosis tool to screen RVH.

K-Net: Integrate Left Ventricle Segmentation and Direct Quantification of Paired Echo Sequence

The integration of segmentation and direct quantification on the left ventricle (LV) from the paired apical views(i.e., apical 4-chamber and 2-chamber together) echo sequence clinically achieves the comprehensive cardiac assessment: multiview segmentation for anatomical morphology, and multidimensional quantification for contractile function. Direct quantification of LV, i.e., to automatically quantify multiple LV indices directly from the image via task-aware feature representation and regression, avoids accumulative error from the inter-step target. This integration sequentially makes a stereoscopical reflection of cardiac activity jointly from the paired orthogonal cross views sequences, overcoming limited observation with a single plane. We propose a K-shaped Unified Network (K-Net), the first end-to-end framework to simultaneously segment LV from apical 4-chamber and 2-chamber views, and directly quantify LV from major- and minor-axis dimensions (1D), area (2D), and volume (3D), in sequence. It works via four components: 1) the K-Net architecture with the Attention Junction enables heterogeneous tasks learning of segmentation task of pixel-wise classification, and direct quantification task of image-wise regression, by interactively introducing the information from segmentation to jointly promote spatial attention map to guide quantification focusing on LV-related region, and transferring quantification feedback to make global constraint on segmentation; 2) the Bi-ResLSTMs distributed in K-Net layer-by-layer hierarchically extract spatial-temporal information in echo sequence, with bidirectional recurrent and short-cut connection to model spatial-temporal information among all frames; 3) the Information Valve tailing the Bi-ResLSTMs selectively exchanges information among multiple views, by stimulating complementary information and suppressing redundant information to make the efficient cross-flow for each view; 4) the Evolution Loss comprehensively guides sequential data learning, with static constraint for frame values, and dynamic constraint for inter-frame value changes. The experiments show that our K-Net gains high performance with a Dice coefficient up to 91.44% and a mean absolute error of the major-axis dimension down to 2.74mm, which reveal its clinical potential.

Machine Learning Based Suicide Ideation Prediction for Military Personnel

Military personnel have greater psychological stress and are at higher suicide attempt risk compared with the general population. High mental stress may cause suicide ideations which are crucially driving suicide attempts. However, traditional statistical methods could only find a moderate degree of correlation between psychological stress and suicide ideation in non-psychiatric individuals. This article utilizes machine learning techniques including logistic regression, decision tree, random forest, gradient boosting regression tree, support vector machine and multilayer perceptron to predict the presence of suicide ideation by six important psychological stress domains of the military males and females. The accuracies of all the six machine learning methods are over 98%. Among them, the multilayer perceptron and support vector machine provide the best predictions of suicide ideation approximately to 100%. As compared with the BSRS-5 score ≥7, a conventional criterion, for the presence of suicide ideation ≥1, the proposed algorithms can improve the performances of accuracy, sensitivity, specificity, precision, the AUC of ROC curve and the AUC of PR curve up to 5.7%, 35.9%, 4.6%, 65.2%, 4.3% and 53.2%, respectively; and for the presence of more severely intense suicide ideation ≥2, the improvements are 6.1%, 26.2%, 5.8%, 83.5%, 2.8% and 64.7%, respectively.

DeepCQ: Deep multi-task conditional quantification network for estimation of left ventricle parameters

BACKGROUND AND OBJECTIVE

Automatic cardiac left ventricle (LV) quantification plays an important role in assessing cardiac function. Although many advanced methods have been put forward to quantify related LV parameters, automatic cardiac LV quantification is still a challenge task due to the anatomy construction complexity of heart.

METHODS

In this work, we propose a novel deep multi-task conditional quantification learning model (DeepCQ) which contains Segmentation module, Quantification encoder, and Dynamic analysis module. Besides, we also use task uncertainty loss function to update the parameters of the network in training.

RESULTS

The proposed framework is validated on the dataset from Left Ventricle Full Quantification Challenge MICCAI 2018 (https://lvquan18.github.io/). The experimental results show that DeepCQ outperforms the other advanced methods.

CONCLUSIONS

It illustrates that our method has a great potential in comprehensive cardiac function assessment and could play an auxiliary role in clinicians' diagnosis.