End-to-end sensor fusion and classification of atrial fibrillation using deep neural networks and smartphone mechanocardiography

OBJECTIVE

The purpose of this research is to develop a new deep learning framework for detecting atrial fibrillation (AFib), one of the most common heart arrhythmias, by analyzing the heart's mechanical functioning as reflected in seismocardiography (SCG) and gyrocardiography (GCG) signals. Jointly, SCG and GCG constitute the concept of mechanocardiography (MCG), a method used to measure precordial vibrations with the built-in inertial sensors of smartphones.

APPROACH

We present a modified deep residual neural network model for the classification of sinus rhythm (SR), AFib, and Noise categories from tri-axial SCG and GCG data derived from smartphones. In the model presented, pre-processing including automated early sensor fusion and spatial feature extraction are carried out using attention-based convolutional and residual blocks. Additionally, we use bidirectional long short-term memory layers on top of fully-connected layers to extract both spatial and spatiotemporal features of the multidimensional SCG and GCG signals. The dataset consisted of 728 short measurements recorded from 300 patients. Further, the measurements were divided into disjoint training, validation, and test sets, respectively, of 481 measurements, 140 measurements, and 107 measurements. Prior to ingestion by the model, measurements were split into 10-second segments with 75 percent overlap, pre-processed, and augmented.

MAIN RESULTS

On the unseen test set, the model delivered average micro- and macro-F1-score of 0.88 (0.87-0.89; 95% CI) and 0.83 (0.83-0.84; 95% CI) for the segment-wise classification as well as 0.95 (0.94-0.96; 95% CI) and 0.95 (0.94-0.96; 95% CI) for the measurement-wise classification, respectively.

SIGNIFICANCE

Our method not only can effectively fuse SCG and GCG signals but also can identify heart rhythms and abnormalities in the MCG signals with remarkable accuracy.

Effect of digital edge-enhancement on the visibility of normal parenchymal lung markings

Digital 'conventional-like' (C-L) and edge-enhanced (E-E) posteroanterior chest roentgenograms of 42 healthy individuals were ranked twice (interval of at least 5 days) in the order of increasing lung parenchymal markings (a total of four rankings). This was done by three radiologists, two residents, one medical student and one radiographer. There was a good general consistency of rankings for both the C-L and E-E images. The correlation coefficients were best (median 0.600) (P < 0.05) between the consequent rankings of C-L images compared to the subsequent rankings of E-E images (median 0.440) and to the various combinations between the rankings of both kinds of images. Subtle differences in normal lung parenchyma could, therefore, generally (five of the seven observers) best be demonstrated in C-L images, but two observers managed best with the E-E images.

Observed lung markings in normal chest roentgenograms

The digital chest posterior-anterior roentgenograms of 42 healthy individuals were ranked twice (interval of at least 5 days) in the order of increasing lung parenchymal markings. The evaluations were made by three radiologists, two residents, a medical student and a radiographer. All observers regardless of their radiological experience showed good intraobserver correlations between their two subsequent rankings (p < 0.05-0.001). The interobserver agreement on rankings was generally poor, even if the radiologists were considered. Radiologic training seemed to eliminate the influence of false leading factors (the object's respiration and body constitution). "The golden standard of evaluation" (= the added-up rankings by the radiologists) did not correlate either with the patient's ages (19-54 years) or smoking habits (0-45 pack-years, mean 4.2).