Exploratory Study to Evaluate Respiratory Rate Using a Thermal Imaging Camera

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

As healthcare costs continue to rise, finding affordable and non-invasive ways to monitor vital signs is increasingly important. One of the key metrics for assessing overall health and identifying potential issues early on is respiratory rate (RR). Most of the existing methods require multiple steps that consist of image and signal processing. This might be difficult to deploy on edge devices that often do not have specialized digital signal processors (DSP). Therefore, the goal of this study is to develop a single neural network realizing the entire process of RR estimation in a single forward pass. The proposed solution builds on recent advances in video recognition, capturing both spatial and temporal information in a multi-path network. Both paths process the data at different sampling rates to capture rapid and slow changes that are associated with differences in the temperature of the nostril area during the breathing episodes. The preliminary results show that the introduced end-to-end solution achieves better performance compared to state-of-the-art methods, without requiring additional pre/post-processing steps and signal-processing techniques. In addition, the presented results demonstrate its robustness on low-resolution thermal video sequences that are often used at the embedded edge due to the size and power constraints of such systems. Taking that into account, the proposed approach has the potential for efficient and convenient respiratory rate estimation across various markets in solutions deployed locally, close to end users.

Machine learning empowered COVID-19 patient monitoring using non-contact sensing: An extensive review

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which caused the coronavirus disease 2019 (COVID-19) pandemic, has affected more than 400 million people worldwide. With the recent rise of new Delta and Omicron variants, the efficacy of the vaccines has become an important question. The goal of various studies has been to limit the spread of the virus by utilizing wireless sensing technologies to prevent human-to-human interactions, particularly for healthcare workers. In this paper, we discuss the current literature on invasive/contact and non-invasive/non-contact technologies (including Wi-Fi, radar, and software-defined radio) that have been effectively used to detect, diagnose, and monitor human activities and COVID-19 related symptoms, such as irregular respiration. In addition, we focused on cutting-edge machine learning algorithms (such as generative adversarial networks, random forest, multilayer perceptron, support vector machine, extremely randomized trees, and k-nearest neighbors) and their essential role in intelligent healthcare systems. Furthermore, this study highlights the limitations related to non-invasive techniques and prospective research directions.

•

This article describes cutting-edge technology (invasive/non-invasive) and its role in the recognition of COVID-19 symptoms.

•

This article summarizes state-of-art machine-learning algorithms and their roles in modern healthcare systems.

•

This article presents the challenges associated with wireless sensing techniques and potential future research directions.

A Neonatal Phantom for Vital Signs Simulation

Neonatal intensive care units provide vital medical support for premature infants. The key aspect in neonatal care is the continuous monitoring of vital signs measured using adhesive skin sensors. Since sensors can cause irritation of the skin and lead to infections, research focuses on contact-free, camera-based methods such as infrared thermography and photoplethysmography imaging. The development of image processing algorithms requires large datasets, but recording the necessary data from studies brings tremendous effort and costs. Therefore, realistic patient phantoms would be feasible to create a comprehensive dataset and validate image-based algorithms. This work describes the realization of a neonatal phantom which can simulate physiological vital parameters such as pulse rate and thermoregulation. It mimics the outer appearance of premature infants using a 3D printed base structure coated with several layers of modified, skin-colored silicone. A distribution of red and infrared LEDs in the scaffold enables the simulation of a PPG signal by mimicking pulsative light intensity changes on the skin. Additionally, the body temperature of the phantom is individually adjustable in several regions using heating elements. In the validation process for PPG simulation, the feasibility of setting different pulse frequencies and the variation of oxygen saturation levels was obtained. Furthermore, heating tests showed region-dependent temperature variations between 0.19 ^°C and 0.81 ^°C around the setpoint. In conclusion, the proposed neonatal phantom can be used to simulate a variety of vital parameters of preterm infants and, therefore, enables the implementation of image processing algorithms for the analysis of the medical state.

Evaluation of respiratory rate monitoring using a microwave Doppler sensor mounted on the ceiling of an intensive care unit: a prospective observational study

Continuous monitoring of the respiratory rate is crucial in an acute care setting. Contact respiratory monitoring modalities such as capnography and thoracic impedance pneumography are prone to artifacts, causing false alarms. Moreover, their cables can restrict patient behavior or interrupt patient care. A microwave Doppler sensor is a novel non-contact continuous respiratory rate monitor. We compared respiratory rate measurements performed with a microwave Doppler sensor mounted on the ceiling of an intensive care unit with those obtained by conventional methods in conscious and spontaneously breathing patients. Participants' respiratory rate was simultaneously measured by visual counting of chest wall movements for 60 s; a microwave Doppler sensor; capnography, using an oxygen mask; and thoracic impedance pneumography, using electrocardiogram electrodes. Bland-Altman analysis for repeated measures was performed to calculate bias and 95% limits of agreement between the respiratory rate measured by visual counting (reference) and that measured by each of the other methods. Among 52 participants, there were 336 (microwave Doppler sensor), 275 (capnography), and 336 (thoracic impedance pneumography) paired respiratory rate data points. Bias (95% limits of agreement) estimates were as follows: microwave Doppler sensor, 0.3 (- 6.1 to 6.8) breaths per minute (bpm); capnography, - 1.3 (- 8.6 to 6.0) bpm; and thoracic impedance pneumography, 0.1 (- 4.4 to 4.7) bpm. Compared to visual counting, the microwave Doppler sensor showed small bias; however, the limits of agreement were similar to those observed in other conventional methods. Our monitor and the conventional ones are not interchangeable with visual counting.Trial registration number: UMIN000032021, March/30/2018.

Advancements in Methods and Camera-Based Sensors for the Quantification of Respiration

Assessment of respiratory function allows early detection of potential disorders in the respiratory system and provides useful information for medical management. There is a wide range of applications for breathing assessment, from measurement systems in a clinical environment to applications involving athletes. Many studies on pulmonary function testing systems and breath monitoring have been conducted over the past few decades, and their results have the potential to broadly impact clinical practice. However, most of these works require physical contact with the patient to produce accurate and reliable measures of the respiratory function. There is still a significant shortcoming of non-contact measuring systems in their ability to fit into the clinical environment. The purpose of this paper is to provide a review of the current advances and systems in respiratory function assessment, particularly camera-based systems. A classification of the applicable research works is presented according to their techniques and recorded/quantified respiration parameters. In addition, the current solutions are discussed with regards to their direct applicability in different settings, such as clinical or home settings, highlighting their specific strengths and limitations in the different environments.

Contact-free sensor signals as a new digital biomarker for cardiovascular disease: chances and challenges

Multiple sensor systems are used to monitor physiological parameters, activities of daily living and behaviour. Digital biomarkers can be extracted and used as indicators for health and disease. Signal acquisition is either by object sensors, wearable sensors, or contact-free sensors including cameras, pressure sensors, non-contact capacitively coupled electrocardiogram (cECG), radar, and passive infrared motion sensors. This review summarizes contemporary knowledge of the use of contact-free sensors for patients with cardiovascular disease and healthy subjects following the PRISMA declaration. Chances and challenges are discussed. Thirty-six publications were rated to be of medium (31) or high (5) relevance. Results are best for monitoring of heart rate and heart rate variability using cardiac vibration, facial camera, or cECG; for respiration using cardiac vibration, cECG, or camera; and for sleep using ballistocardiography. Early results from radar sensors to monitor vital signs are promising. Contact-free sensors are little invasive, well accepted and suitable for long-term monitoring in particular in patient's homes. A major problem are motion artefacts. Results from long-term use in larger patient cohorts are still lacking, but the technology is about to emerge the market and we can expect to see more clinical results in the near future.

A Review of the State of the Art in Non-Contact Sensing for COVID-19

COVID-19, caused by SARS-CoV-2, has resulted in a global pandemic recently. With no approved vaccination or treatment, governments around the world have issued guidance to their citizens to remain at home in efforts to control the spread of the disease. The goal of controlling the spread of the virus is to prevent strain on hospitals. In this paper, we focus on how non-invasive methods are being used to detect COVID-19 and assist healthcare workers in caring for COVID-19 patients. Early detection of COVID-19 can allow for early isolation to prevent further spread. This study outlines the advantages and disadvantages and a breakdown of the methods applied in the current state-of-the-art approaches. In addition, the paper highlights some future research directions, which need to be explored further to produce innovative technologies to control this pandemic.

Non-contact thermography-based respiratory rate monitoring in a post-anesthetic care unit

In patients at high risk of respiratory complications, pulse oximetry may not adequately detect hypoventilation events. Previous studies have proposed using thermography, which relies on infrared imaging, to measure respiratory rate (RR). These systems lack support from real-world feasibility testing for widespread acceptance. This study enrolled 101 spontaneously ventilating patients in a post-anesthesia recovery unit. Patients were placed in a 45° reclined position while undergoing pulse oximetry and bioimpedance-based RR monitoring. A thermography camera was placed approximately 1 m from the patient and pointed at the patient’s face, recording continuously at 30 frames per second for 2 min. Simultaneously, RR was manually recorded. Offline imaging analysis identified the nares as a region of interest and then quantified nasal temperature changes frame by frame to estimate RR. The manually calculated RR was compared with both bioimpedance and thermographic estimates. The Pearson correlation coefficient between direct measurement and bioimpedance was 0.69 (R² = 0.48), and that between direct measurement and thermography was 0.95 (R² = 0.90). Limits of agreement analysis revealed a bias of 1.3 and limits of agreement of 10.8 (95% confidence interval 9.07 to 12.5) and − 8.13 (− 6.41 to − 9.84) between direct measurements and bioimpedance, and a bias of −0.139 and limits of agreement of 2.65 (2.14 to 3.15) and − 2.92 (− 2.41 to 3.42) between direct measurements and thermography. Thermography allowed tracking of the manually measured RR in the post-anesthesia recovery unit without requiring patient contact. Additional work is required for image acquisition automation and nostril identification.

Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review

Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.

A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum

OBJECTIVES

Camera-based vital sign estimation allows the contactless assessment of important physiological parameters. Seminal contributions were made in the 1930s, 1980s, and 2000s, and the speed of development seems ever increasing. In this suivey, we aim to overview the most recent works in this area, describe their common features as well as shortcomings, and highlight interesting "outliers".

METHODS

We performed a comprehensive literature research and quantitative analysis of papers published between 2016 and 2018. Quantitative information about the number of subjects, studies with healthy volunteers vs. pathological conditions, public datasets, laboratory vs. real-world works, types of camera, usage of machine learning, and spectral properties of data was extracted. Moreover, a qualitative analysis of illumination used and recent advantages in terms of algorithmic developments was also performed.

RESULTS

Since 2016, 116 papers were published on camera-based vital sign estimation and 59% of papers presented results on 20 or fewer subjects. While the average number of participants increased from 15.7 in 2016 to 22.9 in 2018, the vast majority of papers (n=100) were on healthy subjects. Four public datasets were used in 10 publications. We found 27 papers whose application scenario could be considered a real-world use case, such as monitoring during exercise or driving. These include 16 papers that dealt with non-healthy subjects. The majority of papers (n=61) presented results based on visual, red-green-blue (RGB) information, followed by RGB combined with other parts of the electromagnetic spectrum (n=18), and thermography only (n=12), while other works (n=25) used other mono- or polychromatic non-RGB data. Surprisingly, a minority of publications (n=39) made use of consumer-grade equipment. Lighting conditions were primarily uncontrolled or ambient. While some works focused on specialized aspects such as the removal of vital sign information from video streams to protect privacy or the influence of video compression, most algorithmic developments were related to three areas: region of interest selection, tracking, or extraction of a one-dimensional signal. Seven papers used deep learning techniques, 17 papers used other machine learning approaches, and 92 made no explicit use of machine learning.

CONCLUSION

Although some general trends and frequent shortcomings are obvious, the spectrum of publications related to camera-based vital sign estimation is broad. While many creative solutions and unique approaches exist, the lack of standardization hinders comparability of these techniques and of their performance. We believe that sharing algorithms and/ or datasets will alleviate this and would allow the application of newer techniques such as deep learning.