Towards the Development of a Mobile Phonopneumogram: Automatic Breath-Phase Classification Using Smartphones

Enabling Continuous Breathing-Phase Contextualization via Wearable-Based Impedance Pneumography and Lung Sounds: A Feasibility Study

Chronic respiratory diseases affect millions and are leading causes of death in the US and worldwide. Pulmonary auscultation provides clinicians with critical respiratory health information through the study of Lung Sounds (LS) and the context of the breathing-phase and chest location in which they are measured. Existing auscultation technologies, however, do not enable the simultaneous measurement of this context, thereby potentially limiting computerized LS analysis. In this work, LS and Impedance Pneumography (IP) measurements were obtained from 10 healthy volunteers while performing normal and forced-expiratory (FE) breathing maneuvers using our wearable IP and respiratory sounds (WIRS) system. Simultaneous auscultation was performed with the Eko CORE stethoscope (EKO). The breathing-phase context was extracted from the IP signals and used to compute phase-by-phase (Inspiratory (I), expiratory (E), and their ratio (I:E)) and breath-by-breath acoustic features. Their individual and added value was then elucidated through machine learning analysis. We found that the phase-contextualized features effectively captured the underlying acoustic differences between deep and FE breaths, yielding a maximum F1 Score of 84.1 ±11.4% with the phase-by-phase features as the strongest contributors to this performance. Further, the individual phase-contextualized models outperformed the traditional breath-by-breath models in all cases. The validity of the results was demonstrated for the LS obtained with WIRS, EKO, and their combination. These results suggest that incorporating breathing-phase context may enhance computerized LS analysis. Hence, multimodal sensing systems that enable this, such as WIRS, have the potential to advance LS clinical utility beyond traditional manual auscultation and improve patient care.

Breathing Chest Wall Kinematics Assessment through a Single Digital Camera: A Feasibility Study

The identification of respiratory patterns based on the movement of the chest wall can assist in monitoring an individual's health status, particularly those with neuromuscular disorders, such as hemiplegia and Duchenne muscular dystrophy. Thoraco-abdominal asynchrony (TAA) refers to the lack of coordination between the rib cage and abdominal movements, characterized by a time delay in their expansion. Motion capture systems, like optoelectronic plethysmography (OEP), are commonly employed to assess these asynchronous movements. However, alternative technologies able to capture chest wall movements without physical contact, such as RGB digital cameras and time-of-flight digital cameras, can also be utilized due to their accessibility, affordability, and non-invasive nature. This study explores the possibility of using a single RGB digital camera to record the kinematics of the thoracic and abdominal regions by placing four non-reflective markers on the torso. In order to choose the positions of these markers, we previously investigated the movements of 89 chest wall landmarks using OEP. Laboratory tests and volunteer experiments were conducted to assess the viability of the proposed system in capturing the kinematics of the chest wall and estimating various time-related respiratory parameters (i.e., fR, Ti, Te, and Ttot) as well as TAA indexes. The results demonstrate a high level of agreement between the detected chest wall kinematics and the reference data. Furthermore, the system shows promising potential in estimating time-related respiratory parameters and identifying phase shifts indicative of TAA, thus suggesting its feasibility in detecting abnormal chest wall movements without physical contact with a single RGB camera.

Non-invasive devices for respiratory sound monitoring

Respiratory diseases are leading causes of death and disability in the world. The recent COVID-19 pandemic is also affecting the respiratory system. Detecting and diagnosing respiratory diseases requires both medical professionals and the clinical environment. Most of the techniques used up to date were also invasive or expensive.

Some research groups are developing hardware devices and techniques to make possible a non-invasive or even remote respiratory sound acquisition. These sounds are then processed and analysed for clinical, scientific, or educational purposes.

We present the literature review of non-invasive sound acquisition devices and techniques.

The results are about a huge number of digital tools, like microphones, wearables, or Internet of Thing devices, that can be used in this scope.

Some interesting applications have been found. Some devices make easier the sound acquisition in a clinic environment, but others make possible daily monitoring outside that ambient. We aim to use some of these devices and include the non-invasive recorded respiratory sounds in a Digital Twin system for personalized health.

Efficiently Classifying Lung Sounds through Depthwise Separable CNN Models with Fused STFT and MFCC Features

Lung sounds remain vital in clinical diagnosis as they reveal associations with pulmonary pathologies. With COVID-19 spreading across the world, it has become more pressing for medical professionals to better leverage artificial intelligence for faster and more accurate lung auscultation. This research aims to propose a feature engineering process that extracts the dedicated features for the depthwise separable convolution neural network (DS-CNN) to classify lung sounds accurately and efficiently. We extracted a total of three features for the shrunk DS-CNN model: the short-time Fourier-transformed (STFT) feature, the Mel-frequency cepstrum coefficient (MFCC) feature, and the fused features of these two. We observed that while DS-CNN models trained on either the STFT or the MFCC feature achieved an accuracy of 82.27% and 73.02%, respectively, fusing both features led to a higher accuracy of 85.74%. In addition, our method achieved 16 times higher inference speed on an edge device and only 0.45% less accuracy than RespireNet. This finding indicates that the fusion of the STFT and MFCC features and DS-CNN would be a model design for lightweight edge devices to achieve accurate AI-aided detection of lung diseases.

A cost-effective smartphone-based device for ankle-brachial index (ABI) detection

BACKGROUND AND OBJECTIVE

Detectors of ankle-brachial index (ABI) are commonly used in cardiovascular patients who have high-risk levels of arteriosclerosis. Increased evidences suggest that patients with arteriosclerosis possess many risks of geriatric and chronic diseases. Meanwhile, new chronic treatments trend from the hospitals toward family and community health centers, but for arteriosclerosis cases have delivered benefits far below instrument costs. Compared to traditional devices based on cuff pressure, cuffless and non-invasive measures have wider application potential in home health care, especially in the case of physically-restricted or severely symptomatic patients.

METHODS

In this study, we developed a simple smartphone-based device for non-invasive ABI monitoring, which consists of four wireless cuffless limbs blood sensors. By identifying and tracking blood flow waveform, a multiparameter fusion (MPF) algorithm is used to estimate blood pressure and generate ABI value. An ARM-based chip STM32 has been adopted as the microcontroller. The ABI calculating program is embedded in C++ and executed by the processor. After generating data, ABI information can be delivered to the smartphone by using Bluetooth. Relying on mobile apps to visualize the data and display on the screen, doctors can monitor cardiovascular patients in real time and analyze the risk levels of arteriosclerosis online.

RESULTS

In this paper, the detection conducted by the classical Doppler equipment and prototype were recorded respectively. A statistical evaluation of the verification results obtained from 29 patients and 7 sub-health volunteers is given, which shows that our device can achieve 91.80% and 93.84% accuracy for patients and sub-health volunteers, respectively. In addition, the prototype can be performed stably for a continuous long time monitoring.

CONCLUSIONS

According to our studies, the accuracy of our device is sufficient for home medical and chronic disease monitoring within a certain time interval. The smartphone-based ABI device has several apparent advantages over traditional devices, such as portability, cost-effectiveness and energy-efficiency.

Recent development of respiratory rate measurement technologies

Respiratory rate (RR) is an important physiological parameter whose abnormality has been regarded as an important indicator of serious illness. In order to make RR monitoring simple to perform, reliable and accurate, many different methods have been proposed for such automatic monitoring. According to the theory of respiratory rate extraction, methods are categorized into three modalities: extracting RR from other physiological signals, RR measurement based on respiratory movements, and RR measurement based on airflow. The merits and limitations of each method are highlighted and discussed. In addition, current works are summarized to suggest key directions for the development of future RR monitoring methodologies.

Convolutional Neural Network for Breathing Phase Detection in Lung Sounds

We applied deep learning to create an algorithm for breathing phase detection in lung sound recordings, and we compared the breathing phases detected by the algorithm and manually annotated by two experienced lung sound researchers. Our algorithm uses a convolutional neural network with spectrograms as the features, removing the need to specify features explicitly. We trained and evaluated the algorithm using three subsets that are larger than previously seen in the literature. We evaluated the performance of the method using two methods. First, discrete count of agreed breathing phases (using 50% overlap between a pair of boxes), shows a mean agreement with lung sound experts of 97% for inspiration and 87% for expiration. Second, the fraction of time of agreement (in seconds) gives higher pseudo-kappa values for inspiration (0.73–0.88) than expiration (0.63–0.84), showing an average sensitivity of 97% and an average specificity of 84%. With both evaluation methods, the agreement between the annotators and the algorithm shows human level performance for the algorithm. The developed algorithm is valid for detecting breathing phases in lung sound recordings.

Detection of Respiratory Crackle Sounds via an Android Smartphone-based System

Pulmonary auscultation with traditional stethoscope, although useful, has limitations for detecting discontinuous adventitious respiratory sounds (crackles) that commonly occur in respiratory diseases. In this work, we present the development of a mobile health system for the automated detection of crackle sounds, comprised by an acoustical sensor, a smart phone device, and a mobile application (app) implemented in Android. The app allows the physician to record, store, reproduce, and analyze respiratory sounds directly on the smart phone. The algorithm for crackle detection was based on a time-varying autoregressive modeling. Performance of the automated detector was analyzed using synthetic fine and coarse crackle sounds randomly added to the basal respiratory sounds acquired from healthy subjects with different signal to noise ratios. Accuracy and sensitivity were found to range from 90.7% to 94.0% and from 91.2% to 94.2%, respectively. Application of the proposed mobile system to real acquired data from a patient with pulmonary fibrosis is also exemplified.

Combination of near-infrared and thermal imaging techniques for the remote and simultaneous measurements of breathing and heart rates under sleep situation

To achieve the simultaneous and unobtrusive breathing rate (BR) and heart rate (HR) measurements during nighttime, we leverage a far-infrared imager and an infrared camera equipped with IR-Cut lens and an infrared lighting array to develop a dual-camera imaging system. A custom-built cascade face classifier, containing the conventional Adaboost model and fully convolutional network trained by 32K images, was used to detect the face region in registered infrared images. The region of interest (ROI) inclusive of mouth and nose regions was afterwards confirmed by the discriminative regression and coordinate conversions of three selected landmarks. Subsequently, a tracking algorithm based on spatio-temporal context learning was applied for following the ROI in thermal video, and the raw signal was synchronously extracted. Finally, a custom-made time-domain signal analysis approach was developed for the determinations of BR and HR. A dual-mode sleep video database, including the videos obtained under environment where illumination intensity ranged from 0 to 3 Lux, was constructed to evaluate the effectiveness of the proposed system and algorithms. In linear regression analysis, the determination coefficient (R2) of 0.831 had been observed for the measured BR and reference BR, and this value was 0.933 for HR measurement. In addition, the Bland-Altman plots of BR and HR demonstrated that almost all the data points located within their own 95% limits of agreement. Consequently, the overall performance of the proposed technique is acceptable for BR and HR estimations during nighttime.

Synergetic use of thermal and visible imaging techniques for contactless and unobtrusive breathing measurement

We present a dual-mode imaging system operating on visible and long-wave infrared wavelengths for achieving the noncontact and nonobtrusive measurements of breathing rate and pattern, no matter whether the subjects use the nose and mouth simultaneously, alternately, or individually when they breathe. The improved classifiers in tandem with the biological characteristics outperformed the custom cascade classifiers using the Viola–Jones algorithm for the cross-spectrum detection of face and nose as well as mouth. In terms of breathing rate estimation, the results obtained by this system were verified to be consistent with those measured by reference method via the Bland–Altman plot with 95% limits of agreement from ? 2.998 to 2.391 and linear correlation analysis with a correlation coefficient of 0.971, indicating that this method was acceptable for the quantitative analysis of breathing. In addition, the breathing waveforms extracted by the dual-mode imaging system were basically the same as the corresponding standard breathing sequences. Since the validation experiments were conducted under challenging conditions, such as the significant positional and abrupt physiological variations, we stated that this dual-mode imaging system utilizing the respective advantages of RGB and thermal cameras was a promising breathing measurement tool for residential care and clinical applications.