Detection of COVID-19 in smartphone-based breathing recordings: A pre-screening deep learning tool

A Development of a Sound Recognition-Based Cardiopulmonary Resuscitation Training System

The objective of this study was to develop a sound recognition-based cardiopulmonary resuscitation (CPR) training system that is accessible, cost-effective, easy-to-maintain and provides accurate CPR feedback. Beep-CPR, a novel device with accordion squeakers that emit high-pitched sounds during compression, was developed. The sounds emitted by Beep-CPR were recorded using a smartphone, segmented into 2-second audio fragments, and then transformed into spectrograms. A total of 6,065 spectrograms were generated from approximately 40 minutes of audio data, which were then randomly split into training, validation, and test datasets. Each spectrogram was matched with the depth, rate, and release velocity of the compression measured at the same time interval by the ZOLL X Series monitor/defibrillator. Deep learning models utilizing spectrograms as input were trained using transfer learning based on EfficientNet to predict the depth (Depth model), rate (Rate model), and release velocity (Recoil model) of compressions. Results: The mean absolute error (MAE) for the Depth model was 0.30 cm (95% confidence interval [CI]: 0.27-0.33). The MAE of the Rate model was 3.6/min (95% CI: 3.2-3.9). For the Recoil model, the MAE was 2.3 cm/s (95% CI: 2.1-2.5). External validation of the models demonstrated acceptable performance across multiple conditions, including the utilization of a newly-manufactured device, a fatigued device, and evaluation in an environment with altered spatial dimensions. We have developed a novel sound recognition-based CPR training system, that accurately measures compression quality during training. Significance: Beep-CPR is a cost-effective and easy-to-maintain solution that can improve the efficacy of CPR training by facilitating decentralized at-home training with performance feedback.

TO-LAB model: Real time Touchless Lung Abnormality detection model using USRP based machine learning algorithm

BACKGROUND

Due to the increasing prevalence of respiratory diseases and the importance of early diagnosis. The need for non-invasive and touchless medical diagnostic solutions has become increasingly crucial in modern healthcare to detect lung abnormalities.

OBJECTIVE

Existing methods for lung abnormality detection often rely on invasive and time-consuming procedures limiting their effectiveness in real-time diagnosis. This work introduces a novel Touchless Lung Abnormality (TO-LAB) detection model utilizing universal software radio peripherals (USRP) and machine learning algorithms.

METHODS

The TO-LAB model integrates a blood pressure meter and an RGB-D depth-sensing camera to gather individual data without physical contact. Heart rate (HR) is analyzed through image conversion to IPPG signals, while blood pressure (BP) is obtained via analog conversion from the blood pressure meter. This touchless imaging setup facilitates the extraction of essential signal features crucial for respiratory pattern analysis. Advanced computer vision algorithms like Mel-frequency cepstral coefficients (MFCC) and Principal Component Analysis (PCA) process the acquired data to focus on breathing abnormalities. These features are then combined and inputted into a machine learning-based Multi-class SVM for breathing activity analysis. The Multi-class SVM categorizes breathing abnormalities as normal, shallow, or elevated based on the fused features. The efficiency of this TO-LAB model is evaluated with the simulated and real-time data.

RESULTS

According to the findings, the proposed TO-LAB model attains the maximum accuracy of 96.15% for real time data; however, the accuracy increases to 99.54% for simulated data for the efficient classification of breathing abnormalities.

CONCLUSION

From this analysis, our model attains better results in simulated data but it declines the accuracy while processing with real-time data. Moreover, this work has a significant medical impact since it presents a solution to the problem of gathering enough data during the epidemic to create a realistic model with a large dataset.

Smartphone-based device for point-of-care diagnostics of pulmonary inflammation using convolutional neural networks (CNNs)

In pulmonary inflammation diseases, like COVID-19, lung involvement and inflammation determine the treatment regime. Respiratory inflammation is typically arisen due to the cytokine storm and the leakage of the vessels for immune cells recruitment. Currently, such a situation is detected by the clinical judgment of a specialist or precisely by a chest CT scan. However, the lack of accessibility to the CT machines in many poor medical centers as well as its expensive service, demands more accessible methods for fast and cheap detection of lung inflammation. Here, we have introduced a novel method for tracing the inflammation and lung involvement in patients with pulmonary inflammation, such as COVID-19, by a simple electrolyte detection in their sputum samples. The presence of the electrolyte in the sputum sample results in the fern-like structures after air-drying. These fern patterns are different in the CT positive and negative cases that are detected by an AI application on a smartphone and using a low-cost and portable mini-microscope. Evaluating 160 patient-derived sputum sample images, this method demonstrated an interesting accuracy of 95%, as confirmed by CT-scan results. This finding suggests that the method has the potential to serve as a promising and reliable approach for recognizing lung inflammatory diseases, such as COVID-19.

Respiratory Diseases Diagnosis Using Audio Analysis and Artificial Intelligence: A Systematic Review

Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases' symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems.

Weighted aggregation through probability based ranking: An optimized federated learning architecture to classify respiratory diseases

Background and Objective Respiratory Diseases are one of the leading chronic illnesses in the world according to the reports by World Health Organization. Diagnosing these respiratory diseases is done through auscultation where a medical professional listens to sounds of air in the lungs for anomalies through a stethoscope. This method necessitates extensive experience and can also be misinterpreted by the medical professional. To address this issue, we introduce an AI-based solution that listens to the lung sounds and classifies the respiratory disease detected. Since the research work deals with medical data that is tightly under wraps due to privacy concerns in the medical field, we introduce a Deep learning solution to classify the diseases and a custom Federated learning (FL) approach to further improve the accuracy of the deep learning model and simultaneously maintain data privacy. Federated Learning architecture maintains data privacy and facilitates a distributed learning system for medical infrastructures. Methods The approach utilizes Generative Adversarial Networks (GAN) based Federated learning approach to ensure data privacy. Generative Adversarial Networks generate new data by synthesizing new lung sounds. This new synthesized data is then converted to spectrograms and trained on a neural network to classify four lung diseases, Heart Attack and Normal breathing patterns. Furthermore, to address performance loss during FL, we also propose a new "Weighted Aggregation through Probability-based Ranking (FedWAPR)" algorithm for optimizing the FL aggregation process. The FedWAPR aggregation takes inspiration from exponential distribution function and ranks better performing clients according to it. Results and Conclusion A test accuracy of about 92% was achieved by the trained model while classifying various respiratory diseases and heart failure. Additionally, we developed a novel FedWAPR approach that significantly outperformed the FedAVG approach for the FL aggregate function. A patient can be checked for respiratory diseases using this improved learning approach without the need for extensive sensitive data recording or for making sure the data sample obtained is secure. In a decentralized training runtime, the trained model successfully classifies various respiratory diseases and heart failure using lung sounds with a test accuracy on par with a centralized model.

Automatic Detection of Dyspnea in Real Human-Robot Interaction Scenarios

A respiratory distress estimation technique for telephony previously proposed by the authors is adapted and evaluated in real static and dynamic HRI scenarios. The system is evaluated with a telephone dataset re-recorded using the robotic platform designed and implemented for this study. In addition, the original telephone training data are modified using an environmental model that incorporates natural robot-generated and external noise sources and reverberant effects using room impulse responses (RIRs). The results indicate that the average accuracy and AUC are just 0.4% less than those obtained with matched training/testing conditions with simulated data. Quite surprisingly, there is not much difference in accuracy and AUC between static and dynamic HRI conditions. Moreover, the beamforming methods delay-and-sum and MVDR lead to average improvement in accuracy and AUC equal to 8% and 2%, respectively, when applied to training and testing data. Regarding the complementarity of time-dependent and time-independent features, the combination of both types of classifiers provides the best joint accuracy and AUC score.

Smartphone-based point-of-care testing of the SARS-CoV-2: A systematic review

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus's worldwide pandemic has highlighted the urgent need for reliable, quick, and affordable diagnostic tests for comprehending and controlling the epidemic by tracking the world population. Given how crucial it is to monitor and manage the pandemic, researchers have recently concentrated on creating quick detection techniques. Although PCR is still the preferred clinical diagnostic test, there is a pressing need for substitutes that are sufficiently rapid and cost-effective to provide a diagnosis at the time of use. The creation of a quick and simple POC equipment is necessary for home testing. Our review's goal is to provide an overview of the many methods utilized to identify SARS-CoV 2 in various samples utilizing portable devices, as well as any potential applications for smartphones in epidemiological research and detection. The point of care (POC) employs a range of microfluidic biosensors based on smartphones, including molecular sensors, immunological biosensors, hybrid biosensors, and imaging biosensors. For example, a number of tools have been created for the diagnosis of COVID-19, based on various theories. Integrated portable devices can be created using loop-mediated isothermal amplification, which combines isothermal amplification methods with colorimetric detection. Electrochemical approaches have been regarded as a potential substitute for optical sensing techniques that utilize fluorescence for detection and as being more beneficial to the Minimizing and simplicity of the tools used for detection, together with techniques that can amplify DNA or RNA under constant temperature conditions, without the need for repeated heating and cooling cycles. Many research have used smartphones for virus detection and data visualization, making these techniques more user-friendly and broadly distributed throughout nations. Overall, our research provides a review of different novel, non-invasive, affordable, and efficient methods for identifying COVID-19 contagious infected people and halting the disease's transmission.

Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection

This paper presents the Coswara dataset, a dataset containing diverse set of respiratory sounds and rich meta-data, recorded between April-2020 and February-2022 from 2635 individuals (1819 SARS-CoV-2 negative, 674 positive, and 142 recovered subjects). The respiratory sounds contained nine sound categories associated with variants of breathing, cough and speech. The rich metadata contained demographic information associated with age, gender and geographic location, as well as the health information relating to the symptoms, pre-existing respiratory ailments, comorbidity and SARS-CoV-2 test status. Our study is the first of its kind to manually annotate the audio quality of the entire dataset (amounting to 65 hours) through manual listening. The paper summarizes the data collection procedure, demographic, symptoms and audio data information. A COVID-19 classifier based on bi-directional long short-term (BLSTM) architecture, is trained and evaluated on the different population sub-groups contained in the dataset to understand the bias/fairness of the model. This enabled the analysis of the impact of gender, geographic location, date of recording, and language proficiency on the COVID-19 detection performance.

Diagnosis of COVID-19 from blood parameters using convolutional neural network

Asymptomatically presenting COVID-19 complicates the detection of infected individuals. Additionally, the virus changes too many genomic variants, which increases the virus's ability to spread. Because there isn't a specific treatment for COVID-19 in a short time, the essential goal is to reduce the virulence of the disease. Blood parameters, which contain essential clinical information about infectious diseases and are easy to access, have an important place in COVID-19 detection. The convolutional neural network (CNN) architecture, which is popular in image processing, produces highly successful results for COVID-19 detection models. When the literature is examined, it is seen that COVID-19 studies with CNN are generally done using lung images. In this study, one-dimensional (1D) blood parameters data were converted into two-dimensional (2D) image data after preprocessing, and COVID-19 detection was made with CNN. The t-distributed stochastic neighbor embedding method was applied to transfer the feature vectors to the 2D plane. All data were framed with convex hull and minimum bounding rectangle algorithms to obtain image data. The image data obtained by pixel mapping was presented to the developed 3-line CNN architecture. This study proposes an effective and successful model by providing a combination of low-cost and rapidly-accessible blood parameters and CNN architecture making image data processing highly successful for COVID-19 detection. Ultimately, COVID-19 detection was made with a success rate of 94.85%. This study has brought a new perspective to COVID-19 detection studies by obtaining 2D image data from 1D COVID-19 blood parameters and using CNN.

The voice of COVID-19: Breath and cough recording classification with temporal decision trees and random forests

Symbolic learning is the logic-based approach to machine learning, and its mission is to provide algorithms and methodologies to extract logical information from data and express it in an interpretable way. Interval temporal logic has been recently proposed as a suitable tool for symbolic learning, specifically via the design of an interval temporal logic decision tree extraction algorithm. In order to improve their performances, interval temporal decision trees can be embedded into interval temporal random forests, mimicking the corresponding schema at the propositional level. In this article we consider a dataset of cough and breath sample recordings of volunteer subjects, labeled with their COVID-19 status, originally collected by the University of Cambridge. By interpreting such recordings as multivariate time series, we study the problem of their automated classification using interval temporal decision trees and forests. While this problem has been approached with the same dataset as well as with other datasets, in all cases, non-symbolic learning methods (usually, deep learning-based) have been applied to solve it; in this article we apply a symbolic approach, and show that it does not only outperform the state-of-the-art obtained with the same dataset, but its results are also superior to those of most non-symbolic techniques applied on other datasets. As an added bonus, thanks to the symbolic nature of our approach, we are also able to extract explicit knowledge to help physicians characterize typical COVID-positive cough and breath.

Dyspnea Severity Assessment Based on Vocalization Behavior with Deep Learning on the Telephone

In this paper, a system to assess dyspnea with the mMRC scale, on the phone, via deep learning, is proposed. The method is based on modeling the spontaneous behavior of subjects while pronouncing controlled phonetization. These vocalizations were designed, or chosen, to deal with the stationary noise suppression of cellular handsets, to provoke different rates of exhaled air, and to stimulate different levels of fluency. Time-independent and time-dependent engineered features were proposed and selected, and a k-fold scheme with double validation was adopted to select the models with the greatest potential for generalization. Moreover, score fusion methods were also investigated to optimize the complementarity of the controlled phonetizations and features that were engineered and selected. The results reported here were obtained from 104 participants, where 34 corresponded to healthy individuals and 70 were patients with respiratory conditions. The subjects' vocalizations were recorded with a telephone call (i.e., with an IVR server). The system provided an accuracy of 59% (i.e., estimating the correct mMRC), a root mean square error equal to 0.98, false positive rate of 6%, false negative rate of 11%, and an area under the ROC curve equal to 0.97. Finally, a prototype was developed and implemented, with an ASR-based automatic segmentation scheme, to estimate dyspnea on line.

A Performance Study of CNN Architectures for the Autonomous Detection of COVID-19 Symptoms Using Cough and Breathing

Deep learning (DL) methods have the potential to be used for detecting COVID-19 symptoms. However, the rationale for which DL method to use and which symptoms to detect has not yet been explored. In this paper, we present the first performance study which compares various convolutional neural network (CNN) architectures for the autonomous preliminary COVID-19 detection of cough and/or breathing symptoms. We compare and analyze residual networks (ResNets), visual geometry Groups (VGGs), Alex neural networks (AlexNet), densely connected networks (DenseNet), squeeze neural networks (SqueezeNet), and COVID-19 identification ResNet (CIdeR) architectures to investigate their classification performance. We uniquely train and validate both unimodal and multimodal CNN architectures using the EPFL and Cambridge datasets. Performance comparison across all modes and datasets showed that the VGG19 and DenseNet-201 achieved the highest unimodal and multimodal classification performance. VGG19 and DensNet-201 had high F1 scores (0.94 and 0.92) for unimodal cough classification on the Cambridge dataset, compared to the next highest F1 score for ResNet (0.79), with comparable F1 scores to ResNet for the larger EPFL cough dataset. They also had consistently high accuracy, recall, and precision. For multimodal detection, VGG19 and DenseNet-201 had the highest F1 scores (0.91) compared to the other CNN structures (≤0.90), with VGG19 also having the highest accuracy and recall. Our investigation provides the foundation needed to select the appropriate deep CNN method to utilize for non-contact early COVID-19 detection.

Analyzing the effect of data preprocessing techniques using machine learning algorithms on the diagnosis of COVID-19

Real-time polymerase chain reaction (RT-PCR) known as the swab test is a diagnostic test that can diagnose COVID-19 disease through respiratory samples in the laboratory. Due to the rapid spread of the coronavirus around the world, the RT-PCR test has become insufficient to get fast results. For this reason, the need for diagnostic methods to fill this gap has arisen and machine learning studies have started in this area. On the other hand, studying medical data is a challenging area because the data it contains is inconsistent, incomplete, difficult to scale, and very large. Additionally, some poor clinical decisions, irrelevant parameters, and limited medical data adversely affect the accuracy of studies performed. Therefore, considering the availability of datasets containing COVID-19 blood parameters, which are less in number than other medical datasets today, it is aimed to improve these existing datasets. In this direction, to obtain more consistent results in COVID-19 machine learning studies, the effect of data preprocessing techniques on the classification of COVID-19 data was investigated in this study. In this study primarily, encoding categorical feature and feature scaling processes were applied to the dataset with 15 features that contain blood data of 279 patients, including gender and age information. Then, the missingness of the dataset was eliminated by using both K-nearest neighbor algorithm (KNN) and chain equations multiple value assignment (MICE) methods. Data balancing has been done with synthetic minority oversampling technique (SMOTE), which is a data balancing method. The effect of data preprocessing techniques on ensemble learning algorithms bagging, AdaBoost, random forest and on popular classifier algorithms KNN classifier, support vector machine, logistic regression, artificial neural network, and decision tree classifiers have been analyzed. The highest accuracies obtained with the bagging classifier were 83.42% and 83.74% with KNN and MICE imputations by applying SMOTE, respectively. On the other hand, the highest accuracy ratio reached with the same classifier without SMOTE was 83.91% for the KNN imputation. In conclusion, certain data preprocessing techniques are examined comparatively and the effect of these data preprocessing techniques on success is presented and the importance of the right combination of data preprocessing to achieve success has been demonstrated by experimental studies.

Cloud-based COVID-19 disease prediction system from X-Ray images using convolutional neural network on smartphone

COVID-19 has engulfed over 200 nations through human-to-human transmission, either directly or indirectly. Reverse Transcription-polymerase Chain Reaction (RT-PCR) has been endorsed as a standard COVID-19 diagnostic procedure but has caveats such as low sensitivity, the need for a skilled workforce, and is time-consuming. Coronaviruses show significant manifestation in Chest X-Ray (CX-Ray) images and, thus, can be a viable option for an alternate COVID-19 diagnostic strategy. An automatic COVID-19 detection system can be developed to detect the disease, thus reducing strain on the healthcare system. This paper discusses a real-time Convolutional Neural Network (CNN) based system for COVID-19 illness prediction from CX-Ray images on the cloud. The implemented CNN model displays exemplary results, with training accuracy being 99.94% and validation accuracy reaching 98.81%. The confusion matrix was utilized to assess the models' outcome and achieved 99% precision, 98% recall, 99% F1 score, 100% training area under the curve (AUC) and 98.3% validation AUC. The same CX-Ray dataset was also employed to predict the COVID-19 disease with deep Convolution Neural Networks (DCNN), such as ResNet50, VGG19, InceptonV3, and Xception. The prediction outcome demonstrated that the present CNN was more capable than the DCNN models. The efficient CNN model was deployed to the Platform as a Service (PaaS) cloud.

Rapid Convolutional Neural Networks for Gram-Stained Image Classification at Inference Time on Mobile Devices: Empirical Study from Transfer Learning to Optimization

Despite the emergence of mobile health and the success of deep learning (DL), deploying production-ready DL models to resource-limited devices remains challenging. Especially, during inference time, the speed of DL models becomes relevant. We aimed to accelerate inference time for Gram-stained analysis, which is a tedious and manual task involving microorganism detection on whole slide images. Three DL models were optimized in three steps: transfer learning, pruning and quantization and then evaluated on two Android smartphones. Most convolutional layers (≥80%) had to be retrained for adaptation to the Gram-stained classification task. The combination of pruning and quantization demonstrated its utility to reduce the model size and inference time without compromising model quality. Pruning mainly contributed to model size reduction by 15×, while quantization reduced inference time by 3× and decreased model size by 4×. The combination of two reduced the baseline model by an overall factor of 46×. Optimized models were smaller than 6 MB and were able to process one image in <0.6 s on a Galaxy S10. Our findings demonstrate that methods for model compression are highly relevant for the successful deployment of DL solutions to resource-limited devices.

The Use of Audio Signals for Detecting COVID-19: A Systematic Review

A systematic review on the topic of automatic detection of COVID-19 using audio signals was performed. A total of 48 papers were obtained after screening 659 records identified in the PubMed, IEEE Xplore, Embase, and Google Scholar databases. The reviewed studies employ a mixture of open-access and self-collected datasets. Because COVID-19 has only recently been investigated, there is a limited amount of available data. Most of the data are crowdsourced, which motivated a detailed study of the various pre-processing techniques used by the reviewed studies. Although 13 of the 48 identified papers show promising results, several have been performed with small-scale datasets (<200). Among those papers, convolutional neural networks and support vector machine algorithms were the best-performing methods. The analysis of the extracted features showed that Mel-frequency cepstral coefficients and zero-crossing rate continue to be the most popular choices. Less common alternatives, such as non-linear features, have also been proven to be effective. The reported values for sensitivity range from 65.0% to 99.8% and those for accuracy from 59.0% to 99.8%.

A comparison of the power of breathing sounds signals acquired with a smart stethoscope from a cohort of COVID-19 patients at peak disease, and pre-discharge from the hospital

Objectives

To characterize the frequencies of breathing sounds signals (BS) in COVID-19 patients at peak disease and pre-discharge from hospitalization using a Smart stethoscope.

Methods

Prospective cohort study conducted during the first COVID-19 wave (April-August 2020) in Israel. COVID-19 patients (n = 19) were validated by SARS-Cov-2 PCR test. The healthy control group was composed of 153 volunteers who stated that they were healthy. Power of BS was calculated in the frequency ranges of 0–20, 0–200, and 0–2000 Hz.

Results

The power calculated over frequency ranges 0–20, 20–200, and 200–2000 Hz contributed approximately 45%, 45%, and 10% to the total power calculated over the range 0–2000 Hz, respectively. Total power calculated from the right side of the back showed an increase of 45–80% during peak disease compared with the healthy controls (p < 0.05). The power calculated over the back, in the infrasound range, 0–20 Hz, and not in the 20–2000 Hz range, was greater for the healthy controls than for patients. Using all 3 ranges of frequencies for distinguishing peak disease from healthy controls resulted in sensitivity and specificity of 84% and 91%, respectively. Omitting the 0–20 Hz range resulted in sensitivity and specificity of 74% and 67%, respectively.

Discussion

The BS power acquired from COVID-19 patients at peak disease was significantly greater than that at pre-discharge from the hospital. The infrasound range had a significant contribution to the total power. Although the source of the infrasound is not presently clear, it may serve as an automated diagnostic tool when more clinical experience is gained with this method.

A novel deep learning model to detect COVID-19 based on wavelet features extracted from Mel-scale spectrogram of patients' cough and breathing sounds

The goal of this paper is to classify the various cough and breath sounds of COVID-19 artefacts in the signals from dynamic real-life environments. The main reason for choosing cough and breath sounds than other common symptoms to detect COVID-19 patients from the comfort of their homes, so that they do not overload the Medicare system and therefore do not unwittingly spread the disease by regularly monitoring themselves. The presented model includes two main phases. The first phase is the sound-to-image transformation, which is improved by the Mel-scale spectrogram approach. The second phase consists of extraction of features and classification using nine deep transfer models (ResNet18/34/50/100/101, GoogLeNet, SqueezeNet, MobileNetv2, and NasNetmobile). The dataset contains information data from almost 1600 people (1185 Male and 415 Female) from all over the world. Our classification model is the most accurate, its accuracy is 99.2% according to the SGDM optimizer. The accuracy is good enough that a large set of labelled cough and breath data may be used to check the possibility for generalization. The results demonstrate that ResNet18 is the best stable model for classifying cough and breath tones from a restricted dataset, with a sensitivity of 98.3% and a specificity of 97.8%. Finally, the presented model is shown to be more trustworthy and accurate than any other present model. Cough and breath study accuracy is promising enough to put extrapolation and generalization to the test.

Deep learning identifies cardiac coupling between mother and fetus during gestation

In the last two decades, stillbirth has caused around 2 million fetal deaths worldwide. Although current ultrasound tools are reliably used for the assessment of fetal growth during pregnancy, it still raises safety issues on the fetus, requires skilled providers, and has economic concerns in less developed countries. Here, we propose deep coherence, a novel artificial intelligence (AI) approach that relies on 1 min non-invasive electrocardiography (ECG) to explain the association between maternal and fetal heartbeats during pregnancy. We validated the performance of this approach using a trained deep learning tool on a total of 941 one minute maternal-fetal R-peaks segments collected from 172 pregnant women (20–40 weeks). The high accuracy achieved by the tool (90%) in identifying coupling scenarios demonstrated the potential of using AI as a monitoring tool for frequent evaluation of fetal development. The interpretability of deep learning was significant in explaining synchronization mechanisms between the maternal and fetal heartbeats. This study could potentially pave the way toward the integration of automated deep learning tools in clinical practice to provide timely and continuous fetal monitoring while reducing triage, side-effects, and costs associated with current clinical devices.

Development and Validation of a Multimodal-Based Prognosis and Intervention Prediction Model for COVID-19 Patients in a Multicenter Cohort

The ability to accurately predict the prognosis and intervention requirements for treating highly infectious diseases, such as COVID-19, can greatly support the effective management of patients, especially in resource-limited settings. The aim of the study is to develop and validate a multimodal artificial intelligence (AI) system using clinical findings, laboratory data and AI-interpreted features of chest X-rays (CXRs), and to predict the prognosis and the required interventions for patients diagnosed with COVID-19, using multi-center data. In total, 2282 real-time reverse transcriptase polymerase chain reaction-confirmed COVID-19 patients’ initial clinical findings, laboratory data and CXRs were retrospectively collected from 13 medical centers in South Korea, between January 2020 and June 2021. The prognostic outcomes collected included intensive care unit (ICU) admission and in-hospital mortality. Intervention outcomes included the use of oxygen (O2) supplementation, mechanical ventilation and extracorporeal membrane oxygenation (ECMO). A deep learning algorithm detecting 10 common CXR abnormalities (DLAD-10) was used to infer the initial CXR taken. A random forest model with a quantile classifier was used to predict the prognostic and intervention outcomes, using multimodal data. The area under the receiver operating curve (AUROC) values for the single-modal model, using clinical findings, laboratory data and the outputs from DLAD-10, were 0.742 (95% confidence interval [CI], 0.696−0.788), 0.794 (0.745−0.843) and 0.770 (0.724−0.815), respectively. The AUROC of the combined model, using clinical findings, laboratory data and DLAD-10 outputs, was significantly higher at 0.854 (0.820−0.889) than that of all other models (p < 0.001, using DeLong’s test). In the order of importance, age, dyspnea, consolidation and fever were significant clinical variables for prediction. The most predictive DLAD-10 output was consolidation. We have shown that a multimodal AI model can improve the performance of predicting both the prognosis and intervention in COVID-19 patients, and this could assist in effective treatment and subsequent resource management. Further, image feature extraction using an established AI engine with well-defined clinical outputs, and combining them with different modes of clinical data, could be a useful way of creating an understandable multimodal prediction model.