Monitoring of Cardiorespiratory Signals Using Thermal Imaging: A Pilot Study on Healthy Human Subjects

Remote photoplethysmography-based human vital sign prediction using cyclical algorithm

This article aims to predict vital signs like heart rate (HR), respiration rate, and arterial oxygen saturation using ambient light video, eliminating chronic distortions through improved frame quality with BER estimation. The study employs the cascade residual CNN-FPNR technique for preprocessing and SNR enhancement using energy variance maximization. The image cascade network (ICNet) facilitates segmentation, achieving strong segmentation in low-light ambient videos. Remote photoplethysmography (iPPG) enables noncontact vital sign monitoring, predicting HR and respiratory rate (RR). An innovative noninvasive temperature and cyclical algorithm, incorporating principal component analysis and fast Fourier transform, evaluate patient HR and RR. To address challenges related to involuntary movements, a dynamic time-warping-based optimization method is used for precise region selection. The study introduces an intensity variance-based threshold analysis for arterial oxygen saturation level determination. Ultimately, the support vector machine (SVM) classification technique evaluates the ground truth, showcasing the system's promising potential for remote and accurate vital sign assessment.

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

As healthcare costs continue to rise, finding affordable and non-invasive ways to monitor vital signs is increasingly important. One of the key metrics for assessing overall health and identifying potential issues early on is respiratory rate (RR). Most of the existing methods require multiple steps that consist of image and signal processing. This might be difficult to deploy on edge devices that often do not have specialized digital signal processors (DSP). Therefore, the goal of this study is to develop a single neural network realizing the entire process of RR estimation in a single forward pass. The proposed solution builds on recent advances in video recognition, capturing both spatial and temporal information in a multi-path network. Both paths process the data at different sampling rates to capture rapid and slow changes that are associated with differences in the temperature of the nostril area during the breathing episodes. The preliminary results show that the introduced end-to-end solution achieves better performance compared to state-of-the-art methods, without requiring additional pre/post-processing steps and signal-processing techniques. In addition, the presented results demonstrate its robustness on low-resolution thermal video sequences that are often used at the embedded edge due to the size and power constraints of such systems. Taking that into account, the proposed approach has the potential for efficient and convenient respiratory rate estimation across various markets in solutions deployed locally, close to end users.

Impact of Biological Sex on Radar-Measured Heart Sound Quality

Radar based contact-free technology has number of potential applications for monitoring the cardiopulmonary functions of patients. However, no study has evaluated the effect of gender on the quality of the recordings. This study makes an attempt to distinguish radar based recording of male and female subjects. The study analysed a publicly available dataset of radar-recorded heart sound signals from both male and female subjects. Here, we exploit the reference signal-to-noise ratio (RSNR) to quantify the signal's quality. The results indicate that there is a significant difference in the signal quality between males and females, with males having a higher RSNR value compared to females. This could be a limitation in the widespread use of the current radar based cardiopulmonary recording techniques and overcoming this should be considered for future research.Clinical relevance- This work has highlighted the gender based difference. By considering this, the radar based cardiopulmonary device has the potential for being used for patients requiring long-term monitoring.

Contactless Technologies, Sensors, and Systems for Cardiac and Respiratory Measurement during Sleep: A Systematic Review

Sleep is essential to physical and mental health. However, the traditional approach to sleep analysis-polysomnography (PSG)-is intrusive and expensive. Therefore, there is great interest in the development of non-contact, non-invasive, and non-intrusive sleep monitoring systems and technologies that can reliably and accurately measure cardiorespiratory parameters with minimal impact on the patient. This has led to the development of other relevant approaches, which are characterised, for example, by the fact that they allow greater freedom of movement and do not require direct contact with the body, i.e., they are non-contact. This systematic review discusses the relevant methods and technologies for non-contact monitoring of cardiorespiratory activity during sleep. Taking into account the current state of the art in non-intrusive technologies, we can identify the methods of non-intrusive monitoring of cardiac and respiratory activity, the technologies and types of sensors used, and the possible physiological parameters available for analysis. To do this, we conducted a literature review and summarised current research on the use of non-contact technologies for non-intrusive monitoring of cardiac and respiratory activity. The inclusion and exclusion criteria for the selection of publications were established prior to the start of the search. Publications were assessed using one main question and several specific questions. We obtained 3774 unique articles from four literature databases (Web of Science, IEEE Xplore, PubMed, and Scopus) and checked them for relevance, resulting in 54 articles that were analysed in a structured way using terminology. The result was 15 different types of sensors and devices (e.g., radar, temperature sensors, motion sensors, cameras) that can be installed in hospital wards and departments or in the environment. The ability to detect heart rate, respiratory rate, and sleep disorders such as apnoea was among the characteristics examined to investigate the overall effectiveness of the systems and technologies considered for cardiorespiratory monitoring. In addition, the advantages and disadvantages of the considered systems and technologies were identified by answering the identified research questions. The results obtained allow us to determine the current trends and the vector of development of medical technologies in sleep medicine for future researchers and research.

Respiratory Rate Extraction from Neonatal Near-Infrared Spectroscopy Signals

Background: Near-infrared spectroscopy (NIRS) relative concentration signals contain 'noise' from physiological processes such as respiration and heart rate. Simultaneous assessment of NIRS and respiratory rate (RR) using a single sensor would facilitate a perfectly time-synced assessment of (cerebral) physiology. Our aim was to extract respiratory rate from cerebral NIRS intensity signals in neonates admitted to a neonatal intensive care unit (NICU). Methods: A novel algorithm, NRR (NIRS RR), is developed for extracting RR from NIRS signals recorded from critically ill neonates. In total, 19 measurements were recorded from ten neonates admitted to the NICU with a gestational age and birth weight of 38 ± 5 weeks and 3092 ± 990 g, respectively. We synchronously recorded NIRS and reference RR signals sampled at 100 Hz and 0.5 Hz, respectively. The performance of the NRR algorithm is assessed in terms of the agreement and linear correlation between the reference and extracted RRs, and it is compared statistically with that of two existing methods. Results: The NRR algorithm showed a mean error of 1.1 breaths per minute (BPM), a root mean square error of 3.8 BPM, and Bland-Altman limits of agreement of 6.7 BPM averaged over all measurements. In addition, a linear correlation of 84.5% (p < 0.01) was achieved between the reference and extracted RRs. The statistical analyses confirmed the significant (p < 0.05) outperformance of the NRR algorithm with respect to the existing methods. Conclusions: We showed the possibility of extracting RR from neonatal NIRS in an intensive care environment, which showed high correspondence with the reference RR recorded. Adding the NRR algorithm to a NIRS system provides the opportunity to record synchronously different physiological sources of information about cerebral perfusion and respiration by a single monitoring system. This allows for a concurrent integrated analysis of the impact of breathing (including apnea) on cerebral hemodynamics.

Monitoring Pulse Rate in the Background Using Front Facing Cameras of Mobile Devices

We propose a novel framework to passively monitor pulse rate during the time spent by users on their personal mobile devices. Our framework is based on passively capturing the user's pulse signal using the front-facing camera. Signal capture is performed in the background, while the user is interacting with the device as he/she normally would, e.g., watch movies, read emails, text, and play games. The framework does not require subject participation with the monitoring procedure, thereby addressing the well-known problem of low adherence with such procedures. We investigate various techniques to suppress the impact of spontaneous user motion and fluctuations in ambient light conditions expected in non-participatory environments. Techniques include traditional signal processing, machine learning classifiers, and deep learning methods. Our performance evaluation is based on a clinical study encompassing 113 patients with a history of atrial fibrillation (Afib) who are passively monitored at home using a tablet for a period of two weeks. Our results show that the proposed framework accurately monitors pulse rate, thereby providing a gateway for long-term monitoring without relying on subject participation or the use of a dedicated wearable device.

Grimace scale, burrowing, and nest building for the assessment of post-surgical pain in mice and rats-A systematic review

Several studies suggested an informative value of behavioral and grimace scale parameters for the detection of pain. However, the robustness and reliability of the parameters as well as the current extent of implementation are still largely unknown. In this study, we aimed to systematically analyze the current evidence-base of grimace scale, burrowing, and nest building for the assessment of post-surgical pain in mice and rats. The following platforms were searched for relevant articles: PubMed, Embase via Ovid, and Web of Science. Only full peer-reviewed studies that describe the grimace scale, burrowing, and/or nest building as pain parameters in the post-surgical phase in mice and/or rats were included. Information about the study design, animal characteristics, intervention characteristics, and outcome measures was extracted from identified publications. In total, 74 papers were included in this review. The majority of studies have been conducted in young adult C57BL/6J mice and Sprague Dawley and Wistar rats. While there is an apparent lack of information about young animals, some studies that analyzed the grimace scale in aged rats were identified. The majority of studies focused on laparotomy-associated pain. Only limited information is available about other types of surgical interventions. While an impact of surgery and an influence of analgesia were rather consistently reported in studies focusing on grimace scales, the number of studies that assessed respective effects was rather low for nest building and burrowing. Moreover, controversial findings were evident for the impact of analgesics on post-surgical nest building activity. Regarding analgesia, a monotherapeutic approach was identified in the vast majority of studies with non-steroidal anti-inflammatory (NSAID) drugs and opioids being most commonly used. In conclusion, most evidence exists for grimace scales, which were more frequently used to assess post-surgical pain in rodents than the other behavioral parameters. However, our findings also point to relevant knowledge gaps concerning the post-surgical application in different strains, age levels, and following different surgical procedures. Future efforts are also necessary to directly compare the sensitivity and robustness of different readout parameters applied for the assessment of nest building and burrowing activities.

Goulart A, Desuó IC, Gomes G. Application of thermography to estimate respiratory rate in the emergency room: The journal Temperature toolbox

Among the vital signs collected during hospital triage, respiratory rate is an important parameter associated with physiological, pathophysiological, and emotional changes. In recent years, the importance of its verification in emergency centers due to the severe acute respiratory syndrome 2 (SARS2) pandemic has become very clear, although it is still one of the least evaluated and collected vital signs. In this context, infrared imaging has been shown to be a reliable estimator of respiratory rate, with the advantage of not requiring physical contact with patients. The objective of this study was to evaluate the potential of analyzing a sequence of thermal images as an estimator of respiratory rate in the clinical routine of an emergency room. We used an infrared thermal camera (T540, Flir Systems) to obtain the respiratory rate data of 136 patients, based on nostrils' temperature fluctuation, during the peak of the COVID-19 pandemic in Brazil and compared it with the chest incursion count method, commonly employed in the emergency screening procedures. We found a good agreement between both methods, with Bland-Altman limits of agreement ranging from -4 to 4 min-1, no proportional bias (R2 = 0.021, p = 0.095), and a strong correlation between them (r = 0.95, p < 0.001). Our results suggest that infrared thermography has potential to be a good estimator of respiratory rate in the routine of an emergency room.

Early Prediction of Hemodynamic Shock in Pediatric Intensive Care Units With Deep Learning on Thermal Videos

Shock is one of the major killers in intensive care units, and early interventions can potentially reverse it. In this study, we advance a noncontact thermal imaging modality for continuous monitoring of hemodynamic shock working on 1,03,936 frames from 406 videos recorded longitudinally upon 22 pediatric patients. Deep learning was used to preprocess and extract the Center-to-Peripheral Difference (CPD) in temperature values from the videos. This time-series data along with the heart rate was finally analyzed using Long-Short Term Memory models to predict the shock status up to the next 6 h. Our models achieved the best area under the receiver operating characteristic curve of 0.81 ± 0.06 and area under the precision-recall curve of 0.78 ± 0.05 at 5 h, providing sufficient time to stabilize the patient. Our approach, thus, provides a reliable shock prediction using an automated decision pipeline that can provide better care and save lives.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO₂). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO₂, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO₂ are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO₂ is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO₂ monitoring.

Noncontact Physiological Measurement Using a Camera: A Technical Review and Future Directions

Using a camera as an optical sensor to monitor physiological parameters has garnered considerable research interest in biomedical engineering in recent decades. Researchers have explored the use of a camera for monitoring a variety of physiological waveforms, together with the vital signs carried by these waveforms. Most of the obtained waveforms are related to the human respiratory and cardiovascular systems, and in addition of being indicative of overall health, they can also detect early signs of certain diseases. While using a camera for noncontact physiological signal monitoring offers the advantages of low cost and operational ease, it also has the disadvantages such as vulnerability to motion and lack of burden-free calibration solutions in some use cases. This study presents an overview of the existing camera-based methods that have been reported in recent years. It introduces the physiological principles behind these methods, signal acquisition approaches, various types of acquired signals, data processing algorithms, and application scenarios of these methods. It also discusses the technological gaps between the camera-based methods and traditional medical techniques, which are mostly contact-based. Furthermore, we present the manner in which noncontact physiological signal monitoring use has been extended, particularly over the recent years, to more day-to-day aspects of individuals' lives, so as to go beyond the more conventional use case scenarios. We also report on the development of novel approaches that facilitate easier measurement of less often monitored and recorded physiological signals. These have the potential of ushering a host of new medical and lifestyle applications. We hope this study can provide useful information to the researchers in the noncontact physiological signal measurement community.

Noncontact Sensing of Contagion

The World Health Organization (WHO) has declared COVID-19 a pandemic. We review and reduce the clinical literature on diagnosis of COVID-19 through symptoms that might be remotely detected as of early May 2020. Vital signs associated with respiratory distress and fever, coughing, and visible infections have been reported. Fever screening by temperature monitoring is currently popular. However, improved noncontact detection is sought. Vital signs including heart rate and respiratory rate are affected by the condition. Cough, fatigue, and visible infections are also reported as common symptoms. There are non-contact methods for measuring vital signs remotely that have been shown to have acceptable accuracy, reliability, and practicality in some settings. Each has its pros and cons and may perform well in some challenges but be inadequate in others. Our review shows that visible spectrum and thermal spectrum cameras offer the best options for truly noncontact sensing of those studied to date, thermal cameras due to their potential to measure all likely symptoms on a single camera, especially temperature, and video cameras due to their availability, cost, adaptability, and compatibility. Substantial supply chain disruptions during the pandemic and the widespread nature of the problem means that cost-effectiveness and availability are important considerations.

Non-contact thermography-based respiratory rate monitoring in a post-anesthetic care unit

In patients at high risk of respiratory complications, pulse oximetry may not adequately detect hypoventilation events. Previous studies have proposed using thermography, which relies on infrared imaging, to measure respiratory rate (RR). These systems lack support from real-world feasibility testing for widespread acceptance. This study enrolled 101 spontaneously ventilating patients in a post-anesthesia recovery unit. Patients were placed in a 45° reclined position while undergoing pulse oximetry and bioimpedance-based RR monitoring. A thermography camera was placed approximately 1 m from the patient and pointed at the patient’s face, recording continuously at 30 frames per second for 2 min. Simultaneously, RR was manually recorded. Offline imaging analysis identified the nares as a region of interest and then quantified nasal temperature changes frame by frame to estimate RR. The manually calculated RR was compared with both bioimpedance and thermographic estimates. The Pearson correlation coefficient between direct measurement and bioimpedance was 0.69 (R² = 0.48), and that between direct measurement and thermography was 0.95 (R² = 0.90). Limits of agreement analysis revealed a bias of 1.3 and limits of agreement of 10.8 (95% confidence interval 9.07 to 12.5) and − 8.13 (− 6.41 to − 9.84) between direct measurements and bioimpedance, and a bias of −0.139 and limits of agreement of 2.65 (2.14 to 3.15) and − 2.92 (− 2.41 to 3.42) between direct measurements and thermography. Thermography allowed tracking of the manually measured RR in the post-anesthesia recovery unit without requiring patient contact. Additional work is required for image acquisition automation and nostril identification.

Non-contact sensing of neonatal pulse rate using camera-based imaging: a clinical feasibility study

OBJECTIVE

Neonates and infants are patients who would benefit from less invasive vital sign sensing, especially from fewer cables and the avoidance of adhesive electrodes. Photoplethysmography imaging (PPGI) has been studied for medical applications in recent years: it is possible to assess various vital signs remotely, non-invasively, and without contact by using video cameras and light. However, studies on infants and especially on neonates in clinical settings are still rare. Hence, we conducted a single-center study to assess heart activity by estimating the pulse rate (PR) of 19 neonates.

APPROACH

Time series were generated from tracked regions of interest (ROIs) and PR was estimated via a joint time-frequency analysis using a short-time Fourier transform. Artifacts, for example, induced by movement, were detected and flagged by applying a signal quality index in the frequency domain.

MAIN RESULTS

The feasibility of PR estimation was demonstrated using visible light and near-infrared light at 850 nm and 940 nm, respectively: the estimated PR was as close as 3 heartbeats per minute in artifact-free time segments. Furthermore, an improvement could be shown when selecting the best performing ROI compared to the ROI containing the whole body. The main challenges are artifacts from motion, light sources, medical devices, and the detection and tracking of suitable regions for signal retrieval. Nonetheless, the PR extracted was found to be comparable to the contact-based photoplethysmography reference and is, therefore, a viable replacement if robust signal retrieval is ensured.

SIGNIFICANCE

Neonates are seldom measured by PPGI and studies reporting measurements on darker skin tones are rare. In this work, not only a single camera was used, but a synchronized camera setup using multiple wavelengths. Various ROIs were used for signal extraction to examine the capabilities of PPGI. In addition, qualitative observations regarding camera parameters and noise sources were reported and discussed.

Proton therapy delivery: what is needed in the next ten years?

Proton radiation therapy has been used clinically since 1952, and major advancements in the last 10 years have helped establish protons as a major clinical modality in the cancer-fighting arsenal. Technologies will always evolve, but enough major breakthroughs have been accomplished over the past 10 years to allow for a major revolution in proton therapy. This paper summarizes the major technology advancements with respect to beam delivery that are now ready for mass implementation in the proton therapy space and encourages vendors to bring these to market to benefit the cancer population worldwide. We state why these technologies are essential and ready for implementation, and we discuss how future systems should be designed to accommodate their required features.

Modelling and Validation of Computer Vision Techniques to Assess Heart Rate, Eye Temperature, Ear-Base Temperature and Respiration Rate in Cattle

Simple Summary

Animal monitoring normally requires procedures that are time- and labour-consuming. The implementation of novel non-invasive technologies could be a good approach to monitor animal health and welfare. This study aimed to evaluate the use of images and computer-based methods to track specific features of the face and to assess temperature; respiration rate and heart rate in cattle. The measurements were compared with measures obtained with conventional methods during the same time period. The data were collected from ten dairy cows that were recorded during six handling procedures across two consecutive days. The results from this study show over 92% of accuracy from the computer algorithm that was developed to track the areas selected on the videos collected. In addition, acceptable correlation was observed between the temperature calculated from thermal infrared images and temperature collected using intravaginal loggers. Moreover, there was acceptable correlation between the respiration rate calculated from infrared videos and from visual observation. Furthermore, a low to high relationship was found between the heart rate obtained from videos and from attached monitors. The study also showed that both the position of the cameras and the area analysed on the images are very important, as both had large impact on the accuracy of the methods. The positive outcomes and the limitations observed in this study suggest the need for further research

Abstract

Precision livestock farming has emerged with the aim of providing detailed information to detect and reduce problems related to animal management. This study aimed to develop and validate computer vision techniques to track required features of cattle face and to remotely assess eye temperature, ear-base temperature, respiration rate, and heart rate in cattle. Ten dairy cows were recorded during six handling procedures across two consecutive days using thermal infrared cameras and RGB (red, green, blue) video cameras. Simultaneously, core body temperature, respiration rate and heart rate were measured using more conventional ‘invasive’ methods to be compared with the data obtained with the proposed algorithms. The feature tracking algorithm, developed to improve image processing, showed an accuracy between 92% and 95% when tracking different areas of the face of cows. The results of this study also show correlation coefficients up to 0.99 between temperature measures obtained invasively and those obtained remotely, with the highest values achieved when the analysis was performed within individual cows. In the case of respiration rate, a positive correlation (r = 0.87) was found between visual observations and the analysis of non-radiometric infrared videos. Low to high correlation coefficients were found between the heart rates (0.09–0.99) obtained from attached monitors and from the proposed method. Furthermore, camera location and the area analysed appear to have a relevant impact on the performance of the proposed techniques. This study shows positive outcomes from the proposed computer vision techniques when measuring physiological parameters. Further research is needed to automate and improve these techniques to measure physiological changes in farm animals considering their individual characteristics.

Estimation of respiratory rate using infrared video in an inpatient population: an observational study

Respiratory rate (RR) is one of the most sensitive markers of a deteriorating patient. Despite this, there is significant inter-observer discrepancy when measured by clinical staff, and modalities used in clinical practice such as ECG bioimpedance are prone to error. This study utilized infrared thermography (IRT) to measure RR in a critically ill population in the Intensive Care Unit. This study was carried out in a Single Hospital Centre. Respiratory rate in 27 extubated ICU patients was counted by two observers and compared to ECG Bioimpedance and IRT-derived RR at distances of 0.4-0.6 m and > 1 m respectively. IRT-derived RR using two separate computer vision algorithms outperformed ECG derived RR at distances of 0.4-0.6 m. Using an Autocorrelation estimator, mean bias was - 0.667 breaths/min. Using a Fast Fourier Transform estimator, mean bias was - 1.000 breaths/min. At distances greater than 1 m no statistically significant signal could be obtained. Over all frequencies, there was a significant relationship between the RR estimated using IRT and via manual counting, with Pearson correlation coefficients between 0.796 and 0.943 (p < 0.001). Correlation between counting and ECG-derived RR demonstrated significance only at > 19 bpm (r = 0.562, p = 0.029). Overall agreement between IRT-derived RR at distances of 0.4-0.6 m and gold standard counting was satisfactory, and outperformed ECG derived bioimpedance. Contactless IRT derived RR may be feasible as a routine monitoring modality in wards and subacute inpatient settings.

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

We present a system that utilizes a range of image processing algorithms to allow fully automated thermal face analysis under both laboratory and real-world conditions. We implement methods for face detection, facial landmark detection, face frontalization and analysis, combining all of these into a fully automated workflow. The system is fully modular and allows implementing own additional algorithms for improved performance or specialized tasks. Our suggested pipeline contains a histogtam of oriented gradients support vector machine (HOG-SVM) based face detector and different landmark detecion methods implemented using feature-based active appearance models, deep alignment networks and a deep shape regression network. Face frontalization is achieved by utilizing piecewise affine transformations. For the final analysis, we present an emotion recognition system that utilizes HOG features and a random forest classifier and a respiratory rate analysis module that computes average temperatures from an automatically detected region of interest. Results show that our combined system achieves a performance which is comparable to current stand-alone state-of-the-art methods for thermal face and landmark datection and a classification accuracy of 65.75% for four basic emotions.

A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum

OBJECTIVES

Camera-based vital sign estimation allows the contactless assessment of important physiological parameters. Seminal contributions were made in the 1930s, 1980s, and 2000s, and the speed of development seems ever increasing. In this suivey, we aim to overview the most recent works in this area, describe their common features as well as shortcomings, and highlight interesting "outliers".

METHODS

We performed a comprehensive literature research and quantitative analysis of papers published between 2016 and 2018. Quantitative information about the number of subjects, studies with healthy volunteers vs. pathological conditions, public datasets, laboratory vs. real-world works, types of camera, usage of machine learning, and spectral properties of data was extracted. Moreover, a qualitative analysis of illumination used and recent advantages in terms of algorithmic developments was also performed.

RESULTS

Since 2016, 116 papers were published on camera-based vital sign estimation and 59% of papers presented results on 20 or fewer subjects. While the average number of participants increased from 15.7 in 2016 to 22.9 in 2018, the vast majority of papers (n=100) were on healthy subjects. Four public datasets were used in 10 publications. We found 27 papers whose application scenario could be considered a real-world use case, such as monitoring during exercise or driving. These include 16 papers that dealt with non-healthy subjects. The majority of papers (n=61) presented results based on visual, red-green-blue (RGB) information, followed by RGB combined with other parts of the electromagnetic spectrum (n=18), and thermography only (n=12), while other works (n=25) used other mono- or polychromatic non-RGB data. Surprisingly, a minority of publications (n=39) made use of consumer-grade equipment. Lighting conditions were primarily uncontrolled or ambient. While some works focused on specialized aspects such as the removal of vital sign information from video streams to protect privacy or the influence of video compression, most algorithmic developments were related to three areas: region of interest selection, tracking, or extraction of a one-dimensional signal. Seven papers used deep learning techniques, 17 papers used other machine learning approaches, and 92 made no explicit use of machine learning.

CONCLUSION

Although some general trends and frequent shortcomings are obvious, the spectrum of publications related to camera-based vital sign estimation is broad. While many creative solutions and unique approaches exist, the lack of standardization hinders comparability of these techniques and of their performance. We believe that sharing algorithms and/ or datasets will alleviate this and would allow the application of newer techniques such as deep learning.

Perspective review of optical imaging in welfare assessment in animal-based research

To refine animal research, vital signs, activity, stress, and pain must be monitored. In chronic studies, some measures can be assessed using telemetry sensors. Although this methodology provides high-precision data, an initial surgery for device implantation is necessary, potentially leading to stress, wound infections, and restriction of motion. Recently, camera systems have been adapted for animal research. We give an overview of parameters that can be assessed using imaging in the visible, near-infrared, and thermal spectrum of light. It focuses on heart activity, respiration, oxygen saturation, and motion, as well as on wound analysis. For each parameter, we offer recommendations on the minimum technical requirements of appropriate systems, regions of interest, and light conditions, among others. In general, these systems demonstrate great performance. For heart and respiratory rate, the error was <4  beats  /  min and 5 breaths/min. Furthermore, the systems are capable of tracking animals during different behavioral tasks. Finally, studies indicate that inhomogeneous temperature distribution around wounds might be an indicator of (pending) infections. In sum, camera-based techniques have several applications in animal research. As vital parameters are currently only assessed in sedated animals, the next step should be the integration of these modalities in home-cage monitoring.

A Clinically Evaluated Interferometric Continuous-Wave Radar System for the Contactless Measurement of Human Vital Parameters

Vital parameters are key indicators for the assessment of health. Conventional methods rely on direct contact with the patients’ skin and can hence cause discomfort and reduce autonomy. This article presents a bistatic 24 GHz radar system based on an interferometric six-port architecture and features a precision of 1 µm in distance measurements. Placed at a distance of 40 cm in front of the human chest, it detects vibrations containing respiratory movements, pulse waves and heart sounds. For the extraction of the respiration rate, time-domain approaches like autocorrelation, peaksearch and zero crossing rate are compared to the Fourier transform, while template matching and a hidden semi-Markov model are utilized for the detection of the heart rate from sphygmograms and heart sounds. A medical study with 30 healthy volunteers was conducted to collect 5.5 h of data, where impedance cardiogram and electrocardiogram were used as gold standard for synchronously recording respiration and heart rate, respectively. A low root mean square error for the breathing rate (0.828 BrPM) and a high overall F1 score for heartbeat detection (93.14%) could be achieved using the proposed radar system and signal processing.

Remote Welfare Monitoring of Rodents Using Thermal Imaging

Animal research has always played a crucial role in various medical and scientific breakthroughs. They offer, inter alia, insights into diseases mechanisms, genetic predisposition to a disease, and drug therapy. However, the use of animals for medical research is a cause of major controversies and debates in modern science. To warrant high bioethical standards, new directives have been being adopted to replace animal research whenever possible, to reduce the number of animals, and to refine the procedures to minimize stress and pain. Here, we present two new approaches, based on thermal imaging (a remote and passive technology), to assess respiratory rate (RR) as well as exploratory behavior and general activity in rodents. In animal research, these parameters are gold standards for welfare assessment. The approaches were validated in a study conducted with both rats and mice. To test the feasibility of our algorithm to estimate RR, thermal videos from anesthetized rodents were acquired. The capability of the second approach to monitor activity was tested with videos of Open Field tests. Regarding RR, a high agreement between thermal imaging and gold standard (electrocardiography-derived RR) was achieved. The mean relative error averaged 0.50 ± 0.15 breaths/min and 4.55 ± 2.94 breaths/min for rats and mice, respectively. The second approach was capable of monitoring and tracking the activity of the rodents very well. This paper demonstrates that thermal imaging is a promising and relevant alternative for monitoring of RR and activity in rodents, thus contributing to the remote assessment of animal welfare.