Deep cross-modal feature learning applied to predict acutely decompensated heart failure using in-home collected electrocardiography and transthoracic bioimpedance

BACKGROUND

Deep learning has been successfully applied to ECG data to aid in the accurate and more rapid diagnosis of acutely decompensated heart failure (ADHF). Previous applications focused primarily on classifying known ECG patterns in well-controlled clinical settings. However, this approach does not fully capitalize on the potential of deep learning, which directly learns important features without relying on a priori knowledge. In addition, deep learning applications to ECG data obtained from wearable devices have not been well studied, especially in the field of ADHF prediction.

METHODS

We used ECG and transthoracic bioimpedance data from the SENTINEL-HF study, which enrolled patients (≥21 years) who were hospitalized with a primary diagnosis of heart failure or with ADHF symptoms. To build an ECG-based prediction model of ADHF, we developed a deep cross-modal feature learning pipeline, termed ECGX-Net, that utilizes raw ECG time series and transthoracic bioimpedance data from wearable devices. To extract rich features from ECG time series data, we first adopted a transfer learning approach in which ECG time series were transformed into 2D images, followed by feature extraction using ImageNet-pretrained DenseNet121/VGG19 models. After data filtering, we applied cross-modal feature learning in which a regressor was trained with ECG and transthoracic bioimpedance. Then, we concatenated the DenseNet121/VGG19 features with the regression features and used them to train a support vector machine (SVM) without bioimpedance information.

RESULTS

The high-precision classifier using ECGX-Net predicted ADHF with a precision of 94 %, a recall of 79 %, and an F1-score of 0.85. The high-recall classifier with only DenseNet121 had a precision of 80 %, a recall of 98 %, and an F1-score of 0.88. We found that ECGX-Net was effective for high-precision classification, while DenseNet121 was effective for high-recall classification.

CONCLUSION

We show the potential for predicting ADHF from single-channel ECG recordings obtained from outpatients, enabling timely warning signs of heart failure. Our cross-modal feature learning pipeline is expected to improve ECG-based heart failure prediction by handling the unique requirements of medical scenarios and resource limitations.

A review of pathobiological mechanisms and potential application of medicinal plants for vascular aging: focus on endothelial cell senescence

Endothelial cell (EC) senescence plays a pivotal role in aging and is essential for the pathomechanism of aging-related diseases. Drugs targeting cellular senescence, such as senolytic or senomorphic drugs, may prevent aging and age-related diseases, but these bullets remain undeveloped to target EC senescence. Some medicinal plants may have an anti-senescence property but remain undiscovered. Deep learning has become an emerging approach for drug discovery by simply analyzing cellular morphology-based deep learning. This precious tool would be useful for screening the herb candidate in senescent EC rejuvenescence. Of note, several medicinal plants that can be found in Indonesia such as Curcuma longa L., Piper retrofractum, Guazuma ulmifolia Lam, Centella asiatica (L.) Urb., and Garcinia mangostana L. might potentially possess an anti-senescence effect. This review highlighted the importance of targeting EC senescence, the use of deep learning for medicinal plant screening, and some potential anti-senescence plants originating from Indonesia.

Emerging machine learning approaches to phenotyping cellular motility and morphodynamics

Cells respond heterogeneously to molecular and environmental perturbations. Phenotypic heterogeneity, wherein multiple phenotypes coexist in the same conditions, presents challenges when interpreting the observed heterogeneity. Advances in live cell microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions. Phenotyping cellular dynamics, however, is a nontrivial task and requires machine learning (ML) approaches to discern phenotypic heterogeneity from live cell images. In recent years, ML has proven instrumental in biomedical research, allowing scientists to implement sophisticated computation in which computers learn and effectively perform specific analyses with minimal human instruction or intervention. In this review, we discuss how ML has been recently employed in the study of cell motility and morphodynamics to identify phenotypes from computer vision analysis. We focus on new approaches to extract and learn meaningful spatiotemporal features from complex live cell images for cellular and subcellular phenotyping.

Quantitative analysis of bony birth canal for periacetabular osteotomy patient by template fitting

Periacetabular osteotomy (PAO) is a joint preservation procedure for developmental dysplasia of the hip. Such a procedure requires osteotomy of the medial wall of the acetabulum, which may cause the narrow of the bony birth canal and increase the risk of complications during the childbirth process in the future. Using quantitative analysis of the bony birth canal to determine the risk of complications for the childbirth process remains a challenging task. The purpose of this paper is to explore a new 3D CT measurement method to quantify the narrowest parameters of the bony birth canal of the female patients with hip dysplasia before and after unilateral PAO surgery. By analyzing the impact of PAO surgery on the bony birth canal, the patient's risk of complications during the childbirth process in the future can be estimated, and it can be utilized for doctors to determine the impact of unilateral PAO for choosing appropriate delivery method. In this paper, a mean shape of the preoperative pelvises is obtained by using the statistical shape model algorithm, and the mean shape includes pelvic shape features of all the preoperative pelvises, and it can be utilized as the standard pelvic template. A bidirectional iterative algorithm is used to generate a standard bony birth canal path template. Then, the pelvic registration and principal plane deformation constraint are utilized to calculate the optimal bony birth canal path. The proposed method is verified in 31 cases of CT data with the approval of the institutional review board. The test data contain preoperative and postoperative CT images. Compared with the benchmark method, the measurement accuracy of the narrowest position and diameter of the bony birth canal is improved by 65% and 78%, respectively. In addition, the processing speed is increased by 32%. Experimental results demonstrate that the proposed method has high accuracy and validity for quantifying the bony birth canal. The proposed method can measure the anatomical parameters of the bony birth canal accurately. In addition, the doctor can make optimal planning for childbirth with the help of the quantitative analysis of the bony birth canal.

Dynamic spherical harmonics approach for shape classification of migrating cells

Cell migration involves dynamic changes in cell shape. Intricate patterns of cell shape can be analyzed and classified using advanced shape descriptors, including spherical harmonics (SPHARM). Though SPHARM have been used to analyze and classify migrating cells, such classification did not exploit SPHARM spectra in their dynamics. Here, we examine whether additional information from dynamic SPHARM improves classification of cell migration patterns. We combine the static and dynamic SPHARM approach with a support-vector-machine classifier and compare their classification accuracies. We demonstrate that the dynamic SPHARM analysis classifies cell migration patterns more accurately than the static one for both synthetic and experimental data. Furthermore, by comparing the computed accuracies with that of a naive classifier, we can identify the experimental conditions and model parameters that significantly affect cell shape. This capability should – in the future – help to pinpoint factors that play an essential role in cell migration.

Deep Learning-Based Single-Cell Optical Image Studies

Optical imaging technology that has the advantages of high sensitivity and cost-effectiveness greatly promotes the progress of nondestructive single-cell studies. Complex cellular image analysis tasks such as three-dimensional reconstruction call for machine-learning technology in cell optical image research. With the rapid developments of high-throughput imaging flow cytometry, big data cell optical images are always obtained that may require machine learning for data analysis. In recent years, deep learning has been prevalent in the field of machine learning for large-scale image processing and analysis, which brings a new dawn for single-cell optical image studies with an explosive growth of data availability. Popular deep learning techniques offer new ideas for multimodal and multitask single-cell optical image research. This article provides an overview of the basic knowledge of deep learning and its applications in single-cell optical image studies. We explore the feasibility of applying deep learning techniques to single-cell optical image analysis, where popular techniques such as transfer learning, multimodal learning, multitask learning, and end-to-end learning have been reviewed. Image preprocessing and deep learning model training methods are then summarized. Applications based on deep learning techniques in the field of single-cell optical image studies are reviewed, which include image segmentation, super-resolution image reconstruction, cell tracking, cell counting, cross-modal image reconstruction, and design and control of cell imaging systems. In addition, deep learning in popular single-cell optical imaging techniques such as label-free cell optical imaging, high-content screening, and high-throughput optical imaging cytometry are also mentioned. Finally, the perspectives of deep learning technology for single-cell optical image analysis are discussed. © 2020 International Society for Advancement of Cytometry.

A novel machine learning based approach for iPS progenitor cell identification

Identification of induced pluripotent stem (iPS) progenitor cells, the iPS forming cells in early stage of reprogramming, could provide valuable information for studying the origin and underlying mechanism of iPS cells. However, it is very difficult to identify experimentally since there are no biomarkers known for early progenitor cells, and only about 6 days after reprogramming initiation, iPS cells can be experimentally determined via fluorescent probes. What is more, the ratio of progenitor cells during early reprograming period is below 5%, which is too low to capture experimentally in the early stage. In this paper, we propose a novel computational approach for the identification of iPS progenitor cells based on machine learning and microscopic image analysis. Firstly, we record the reprogramming process using a live cell imaging system after 48 hours of infection with retroviruses expressing Oct4, Sox2 and Klf4, later iPS progenitor cells and normal murine embryonic fibroblasts (MEFs) within 3 to 5 days after infection are labeled by retrospectively tracing the time-lapse microscopic image. We then calculate 11 types of cell morphological and motion features such as area, speed, etc., and select best time windows for modeling and perform feature selection. Finally, a prediction model using XGBoost is built based on the selected six types of features and best time windows. Our model allows several missing values/frames in the sample datasets, thus it is applicable to a wide range of scenarios. Cross-validation, holdout validation and independent test experiments show that the minimum precision is above 52%, that is, the ratio of predicted progenitor cells within 3 to 5 days after viral infection is above 52%. The results also confirm that the morphology and motion pattern of iPS progenitor cells is different from that of normal MEFs, which helps with the machine learning methods for iPS progenitor cell identification.

Identification of induced pluripotent stem (iPS) progenitor cells could provide valuable information for studying the origin and underlying mechanism of iPS cells. However, it is very difficult to identify experimentally since there are no biomarkers known for early progenitor cells, and only after about 6 days of induction, iPS cells can be experimentally determined via fluorescent probes. What is more, the percentage of the progenitor cells during the early induction period is below 5%, too low to capture experimentally in early stage. In this work, we proposed an approach for the identification of iPS progenitor cells, the iPS forming cells, based on machine learning and microscopic image analysis. The aim is to help biologists to enrich iPS progenitor cells during the early stage of induction, which allows experimentalists to select iPS progenitor cells with much higher probability, and furthermore to study the biomarkers which trigger the reprogramming process.

Precise Quantitative Analysis of Cell Targeting by Particle-Based Agents Using Imaging Flow Cytometry and Convolutional Neural Network

Understanding the intricacies of particle-cell interactions is essential for many applications such as imaging, phototherapy, and drug/gene delivery, because it is the key to accurate control of the particle properties for the improvement of their therapeutic and diagnostic efficiency. Recently, high-throughput methods have emerged for the detailed investigation of these interactions. For example, imaging flow cytometry (IFC) collects up to 60,000 images of cells per second (in 12 optical channels) and provides information about morphology and organelle localization in combination with fluorescence and side scatter intensity data. However, analysis of IFC data is extremely difficult to perform using conventional methods that calculate integral parameters or use mask-based object recognition. Here, we show application of a convolutional neural network (CNN) for precise quantitative analysis of particle targeting of cells using IFC data. CNN provides high-throughput object detection with almost human precision but avoids the subjective choice of image processing parameters that often leads to incorrect data interpretation. The method allows accurate counting of cell-bound particles with reliable discrimination from the nonbound particles in the field of view. The proposed method expands capabilities of spot counting applications (such as organelle counting, quantification of cell-cell and cell-bacteria interactions) and is going to be useful not only for high-throughput analysis of IFC data but also for other imaging techniques. © 2019 International Society for Advancement of Cytometry.

Classification and Segmentation of Hyperspectral Data of Hepatocellular Carcinoma Samples Using 1-D Convolutional Neural Network

Pathological diagnosis plays an important role in the diagnosis and treatment of hepatocellular carcinoma (HCC). The traditional method of pathological diagnosis of most cancers requires freezing, slicing, hematoxylin and eosin staining, and manual analysis, limiting the speed of the diagnosis process. In this study, we designed a one-dimensional convolutional neural network to classify the hyperspectral data of HCC sample slices acquired by our hyperspectral imaging system. A weighted loss function was employed to promote the performance of the model. The proposed method allows us to accelerate the diagnosis process of identifying tumor tissues. Our deep learning model achieved good performance on our data set with sensitivity, specificity, and area under receiver operating characteristic curve of 0.871, 0.888, and 0.950, respectively. Meanwhile, our deep learning model outperformed the other machine learning methods assessed on our data set. The proposed method is a potential tool for the label-free and real-time pathologic diagnosis. © 2019 International Society for Advancement of Cytometry.

Analyzing temporal dynamics of cell deformation and intracellular movement with video feature aggregation

BACKGROUND

The research and analysis of cellular physiological properties has been an essential approach to studying some biological and biomedical problems. Temporal dynamics of cells therein are used as a quantifiable indicator of cellular response to extracellular cues and physiological stimuli.

METHODS

This work presents a novel image-based framework to profile and model the cell dynamics in live-cell videos. In the framework, the cell dynamics between frames are represented as frame-level features from cell deformation and intracellular movement. On the one hand, shape context is introduced to enhance the robustness of measuring the deformation of cellular contours. On the other hand, we employ Scale-Invariant Feature Transform (SIFT) flow to simultaneously construct the complementary movement field and appearance change field for the cytoplasmic streaming. Then, time series modeling is performed on these frame-level features. Specifically, temporal feature aggregation is applied to capture the video-wide temporal evolution of cell dynamics.

RESULTS

Our results demonstrate that the proposed cell dynamic features can effectively capture the cell dynamics in videos. They also prove that the Movement Field and Appearance Change Field Feature (MFAFF) can more precisely model the cytoplasmic streaming. Besides, temporal aggregation of cell dynamic features brings a substantial absolute increase of classification performance.

CONCLUSION

Experimental results demonstrate that the proposed framework outperforms competing mainstreaming approaches on the aforementioned datasets. Thus, our method has potential for cell dynamics analysis in videos.