Remote heart rate monitoring - Assessment of the Facereader rPPg by Noldus

Emotion expressions and cognitive impairments in the elderly: review of the contactless detection approach

The aging population in Canada has been increasing continuously throughout the past decades. Amongst this demographic, around 11% suffer from some form of cognitive decline. While diagnosis through traditional means (i.e., Magnetic Resonance Imagings (MRIs), positron emission tomography (PET) scans, cognitive assessments, etc.) has been successful at detecting this decline, there remains unexplored measures of cognitive health that could reduce stress and cost for the elderly population, including approaches for early detection and preventive methods. Such efforts could additionally contribute to reducing the pressure and stress on the Canadian healthcare system, as well as improve the quality of life of the elderly population. Previous evidence has demonstrated emotional facial expressions being altered in individuals with various cognitive conditions such as dementias, mild cognitive impairment, and geriatric depression. This review highlights the commonalities among these cognitive health conditions, and research behind the contactless assessment methods to monitor the health and cognitive well-being of the elderly population through emotion expression. The contactless detection approach covered by this review includes automated facial expression analysis (AFEA), electroencephalogram (EEG) technologies and heart rate variability (HRV). In conclusion, a discussion of the potentials of the existing technologies and future direction of a novel assessment design through fusion of AFEA, EEG and HRV measures to increase detection of cognitive decline in a contactless and remote manner will be presented.

Non-Contact Vision-Based Techniques of Vital Sign Monitoring: Systematic Review

The development of non-contact techniques for monitoring human vital signs has significant potential to improve patient care in diverse settings. By facilitating easier and more convenient monitoring, these techniques can prevent serious health issues and improve patient outcomes, especially for those unable or unwilling to travel to traditional healthcare environments. This systematic review examines recent advancements in non-contact vital sign monitoring techniques, evaluating publicly available datasets and signal preprocessing methods. Additionally, we identified potential future research directions in this rapidly evolving field.

Video plethysmography for contactless blood pressure and heart rate measurement in perioperative care

The purpose of this study was to evaluate the feasibility and accuracy of remote Video Plethysmography (VPPG) for contactless measurements of blood pressure (BP) and heart rate (HR) in adult surgical patients in a hospital setting. An iPad Pro was used to record a 1.5-minute facial video of the participant's face and VPPG was used to extract vital signs measurements. A standard medical device (Welch Allyn) was used for comparison to measure BP and HR. Trial registration: NCT05165381. Two-hundred-sixteen participants consented and completed the contactless BP and HR monitoring (mean age 54.1 ± 16.8 years, 58% male). The consent rate was 75% and VPPG was 99% successful in capturing BP and HR. VPPG predicted SBP, DBP, and HR with a measurement bias ± SD, -8.18 ± 16.44 mmHg, - 6.65 ± 9.59 mmHg, 0.09 ± 6.47 beats/min respectively. Pearson's correlation for all measurements between VPPG and standard medical device was significant. Correlation for SBP was moderate (0.48), DBP was weak (0.29), and HR was strong (0.85). Most patients were satisfied with the non-contact technology with an average rating of 8.7/10 and would recommend it for clinical use. VPPG was highly accurate in measuring HR, and is currently not accurate in measuring BP in surgical patients. The VPPG BP algorithm showed limitations in capturing individual variations in blood pressure, highlighting the need for further improvements to render it clinically effective across all ranges. Contactless vital signs monitoring was well-received and earned a high satisfaction score.

Clinical applications of contactless photoplethysmography for monitoring in adults: A systematic review and meta-analysis

Contactless photoplethysmography (cPPG) is a method of physiological monitoring. It differs from conventional monitoring methods (e.g., saturation probe) by ensuring no contact with the subject by use of a camera. The majority of research on cPPG is conducted in a laboratory setting or in healthy populations. This review aims to evaluate the current literature on monitoring using cPPG in adults within a clinical setting. Adhering to the Preferred Items for Systematic Reviews and Meta-analysis (PRISMA, 2020) guidelines, OVID, Webofscience, Cochrane library, and clinicaltrials.org were systematically searched by two researchers. Research articles using cPPG for monitoring purposes in adults within a clinical setting were selected. Twelve studies with a total of 654 individuals were included. Heart rate (HR) was the most investigated vital sign (n = 8) followed by respiratory rate ((n = 2), Sp02 (n = 2), and HR variability (n = 2). Four studies were included in a meta-analysis of HR compared to ECG data which demonstrated a mean bias of -0.13 (95% CI, -1.22-0.96). This study demonstrates cPPG can be a useful tool in the remote monitoring of patients and has demonstrated accuracy for HR. However, further research is needed into the clinical applications of this method.

Monitoring Pulse Rate in the Background Using Front Facing Cameras of Mobile Devices

We propose a novel framework to passively monitor pulse rate during the time spent by users on their personal mobile devices. Our framework is based on passively capturing the user's pulse signal using the front-facing camera. Signal capture is performed in the background, while the user is interacting with the device as he/she normally would, e.g., watch movies, read emails, text, and play games. The framework does not require subject participation with the monitoring procedure, thereby addressing the well-known problem of low adherence with such procedures. We investigate various techniques to suppress the impact of spontaneous user motion and fluctuations in ambient light conditions expected in non-participatory environments. Techniques include traditional signal processing, machine learning classifiers, and deep learning methods. Our performance evaluation is based on a clinical study encompassing 113 patients with a history of atrial fibrillation (Afib) who are passively monitored at home using a tablet for a period of two weeks. Our results show that the proposed framework accurately monitors pulse rate, thereby providing a gateway for long-term monitoring without relying on subject participation or the use of a dedicated wearable device.

Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey

Over the last few years, a rich amount of research has been conducted on remote vital sign monitoring of the human body. Remote photoplethysmography (rPPG) is a camera-based, unobtrusive technology that allows continuous monitoring of changes in vital signs and thereby helps to diagnose and treat diseases earlier in an effective manner. Recent advances in computer vision and its extensive applications have led to rPPG being in high demand. This paper specifically presents a survey on different remote photoplethysmography methods and investigates all facets of heart rate analysis. We explore the investigation of the challenges of the video-based rPPG method and extend it to the recent advancements in the literature. We discuss the gap within the literature and suggestions for future directions.

Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving

In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

An Overview of the Sensors for Heart Rate Monitoring Used in Extramural Applications

This work presents an overview of the main strategies that have been proposed for non-invasive monitoring of heart rate (HR) in extramural and home settings. We discuss three categories of sensing according to what physiological effect is used to measure the pulsatile activity of the heart, and we focus on an illustrative sensing modality for each of them. Therefore, electrocardiography, photoplethysmography, and mechanocardiography are presented as illustrative modalities to sense electrical activity, mechanical activity, and the peripheral effect of heart activity. In this paper, we describe the physical principles underlying the three categories and the characteristics of the different types of sensors that belong to each class, and we touch upon the most used software strategies that are currently adopted to effectively and reliably extract HR. In addition, we investigate the strengths and weaknesses of each category linked to the different applications in order to provide the reader with guidelines for selecting the most suitable solution according to the requirements and constraints of the application.

Real-time realizable mobile imaging photoplethysmography

Photoplethysmography imaging (PPGI) sensors have attracted a significant amount of attention as they enable the remote monitoring of heart rates (HRs) and thus do not require any additional devices to be worn on fingers or wrists. In this study, we mounted PPGI sensors on a robot for active and autonomous HR (R-AAH) estimation. We proposed an algorithm that provides accurate HR estimation, which can be performed in real time using vision and robot manipulation algorithms. By simplifying the extraction of facial skin images using saturation (S) values in the HSV color space, and selecting pixels based on the most frequent S value within the face image, we achieved a reliable HR assessment. The results of the proposed algorithm using the R-AAH method were evaluated by rigorous comparison with the results of existing algorithms on the UBFC-RPPG dataset (n = 42). The proposed algorithm yielded an average absolute error (AAE) of 0.71 beats per minute (bpm). The developed algorithm is simple, with a processing time of less than 1 s (275 ms for an 8-s window). The algorithm was further validated on our own dataset (BAMI-RPPG dataset [n = 14]) with an AAE of 0.82 bpm.

A novel contact-free atrial fibrillation monitor: a pilot study

Aims

Atrial fibrillation (AF) is a major cause of morbidity and mortality. Current guidelines support performing electrocardiogram (ECG) screenings to spot AF in high-risk patients. The purpose of this study was to validate a new algorithm aimed to identify AF in patients measured with a recent FDA-cleared contact-free optical device.

Methods and results

Study participants were measured simultaneously using two devices: a contact-free optical system that measures chest motion vibrations (investigational device, 'Gili') and a standard reference bed-side ECG monitor (Mindray^®). Each reference ECG was evaluated by two board certified cardiologists that defined each trace as: regular rhythm, AF, other irregular rhythm or indecipherable/missing. A total of 3582, 30-s intervals, pertaining to 444 patients (41.9% with a history of AF) were made available for analysis. Distribution of patients with active AF, other irregular rhythm, and regular rhythm was 16.9%, 29.5%, and 53.6% respectively. Following application of cross-validated machine learning approach, the observed sensitivity and specificity were 0.92 [95% confidence interval (CI): 0.91-0.93] and 0.96 (95% CI: 0.95-0.96), respectively.

Conclusion

This study demonstrates for the first time the efficacy of a contact-free optical device for detecting AF.

Preclinical evaluation of noncontact vital signs monitoring using real-time IR-UWB radar and factors affecting its accuracy

Recently, noncontact vital sign monitors have attracted attention because of issues related to the transmission of contagious diseases. We developed a real-time vital sign monitor using impulse-radio ultrawideband (IR-UWB) radar with embedded processors and software; we then evaluated its accuracy in measuring heart rate (HR) and respiratory rate (RR) and investigated the factors affecting the accuracy of the radar-based measurements. In 50 patients visiting a cardiology clinic, HR and RR were measured using IR-UWB radar simultaneously with electrocardiography and capnometry. All patients underwent HR and RR measurements in 2 postures—supine and sitting—for 2 min each. There was a high agreement between the RR measured using radar and capnometry (concordance correlation coefficient [CCC] 0.925 [0.919–0.926]; upper and lower limits of agreement [LOA], − 2.21 and 3.90 breaths/min). The HR measured using radar was also in close agreement with the value measured using electrocardiography (CCC 0.749 [0.738–0.760]; upper and lower LOA, − 12.78 and 15.04 beats/min). Linear mixed effect models showed that the sitting position and an HR < 70 bpm were associated with an increase in the absolute biases of the HR, whereas the sitting position and an RR < 18 breaths/min were associated with an increase in the absolute biases of the RR. The IR-UWB radar sensor with embedded processors and software can measure the RR and HR in real time with high precision. The sitting position and a low RR or HR were associated with the accuracy of RR and HR measurement, respectively, using IR-UWB radar.

Effectiveness of consumer-grade contactless vital signs monitors: a systematic review and meta-analysis

The objective of this systematic review and meta-analysis was to analyze the effectiveness of contactless vital sign monitors that utilize a consumer-friendly camera versus medical grade instruments. A multiple database search was conducted from inception to September 2020. Inclusion criteria were as follows: studies that used a consumer-grade camera (smartphone/webcam) to examine contactless vital signs in adults; evaluated the non-contact device against a reference medical device; and used the participants’ face for measurement. Twenty-six studies were included in the review of which 16 were included in Pearson’s correlation and 14 studies were included in the Bland–Altman meta-analysis. Twenty-two studies measured heart rate (HR) (92%), three measured blood pressure (BP) (12%), and respiratory rate (RR) (12%). No study examined blood oxygen saturation (SpO₂). Most studies had a small sample size (≤ 30 participants) and were performed in a laboratory setting. Our meta-analysis found that consumer-grade contactless vital sign monitors were accurate in comparison to a medical device in measuring HR. Current contactless monitors have limitations such as motion, poor lighting, and lack of automatic face tracking. Currently available consumer-friendly contactless monitors measure HR accurately compared to standard medical devices. More studies are needed to assess the accuracy of contactless BP and RR monitors. Implementation of contactless vital sign monitors for clinical use will require validation in a larger population, in a clinical setting, and expanded to encompass other vital signs including BP, RR, and SpO₂.

Noncontact Monitoring of Heart Rate and Heart Rate Variability in Geriatric Patients Using Photoplethysmography Imaging

OBJECTIVE

Geriatric patients, especially those with dementia or in a delirious state, do not accept conventional contact-based monitoring. Therefore, we propose to measure heart rate (HR) and heart rate variability (HRV) of geriatric patients in a noncontact and unobtrusive way using photoplethysmography imaging (PPGI).

METHODS

PPGI video sequences were recorded from 10 geriatric patients and 10 healthy elderly people using a monochrome camera operating in the near-infrared spectrum and a colour camera operating in the visible spectrum. PPGI waveforms were extracted from both cameras using superpixel-based regions of interests (ROI). A classifier based on bagged trees was trained to automatically select artefact-free ROIs for HR estimation. HRV was calculated in the time-domain and frequency-domain.

RESULTS

an RMSE of 1.03 bpm and a correlation of 0.8 with the reference was achieved using the NIR camera for HR estimation. Using the RGB camera, RMSE and correlation improved to 0.48 bpm and 0.95, respectively. Correlation for HRV in the frequency-domain (LF/HF-ratio) was 0.50 using the NIR camera and 0.70 using the RGB camera.

CONCLUSION

We were able to demonstrate that PPGI is very suitable to measure HR and HRV in geriatric patients. We strongly believe that PPGI will become clinically relevant in monitoring of geriatric patients.

SIGNIFICANCE

we are the first group to measure both HR and HRV in awake geriatric patients using PPGI. Moreover, we systematically evaluate the effects of the spectrum (near-infrared vs. visible), ROI, and additional motion artefact reduction algorithms on the accuracy of estimated HR and HRV.

Contact-free sensor signals as a new digital biomarker for cardiovascular disease: chances and challenges

Multiple sensor systems are used to monitor physiological parameters, activities of daily living and behaviour. Digital biomarkers can be extracted and used as indicators for health and disease. Signal acquisition is either by object sensors, wearable sensors, or contact-free sensors including cameras, pressure sensors, non-contact capacitively coupled electrocardiogram (cECG), radar, and passive infrared motion sensors. This review summarizes contemporary knowledge of the use of contact-free sensors for patients with cardiovascular disease and healthy subjects following the PRISMA declaration. Chances and challenges are discussed. Thirty-six publications were rated to be of medium (31) or high (5) relevance. Results are best for monitoring of heart rate and heart rate variability using cardiac vibration, facial camera, or cECG; for respiration using cardiac vibration, cECG, or camera; and for sleep using ballistocardiography. Early results from radar sensors to monitor vital signs are promising. Contact-free sensors are little invasive, well accepted and suitable for long-term monitoring in particular in patient's homes. A major problem are motion artefacts. Results from long-term use in larger patient cohorts are still lacking, but the technology is about to emerge the market and we can expect to see more clinical results in the near future.

Blood pressure measurements with the OptiBP smartphone app validated against reference auscultatory measurements

Mobile health diagnostics have been shown to be effective and scalable for chronic disease detection and management. By maximizing the smartphones' optics and computational power, they could allow assessment of physiological information from the morphology of pulse waves and thus estimate cuffless blood pressure (BP). We trained the parameters of an existing pulse wave analysis algorithm (oBPM), previously validated in anaesthesia on pulse oximeter signals, by collecting optical signals from 51 patients fingertips via a smartphone while simultaneously acquiring BP measurements through an arterial catheter. We then compared smartphone-based measurements obtained on 50 participants in an ambulatory setting via the OptiBP app against simultaneously acquired auscultatory systolic blood pressure (SBP), diastolic blood pressure (DBP) and mean blood pressure (MBP) measurements. Patients were normotensive (70.0% for SBP versus 61.4% for DBP), hypertensive (17.1% vs. 13.6%) or hypotensive (12.9% vs. 25.0%). The difference in BP (mean ± standard deviation) between both methods were within the ISO 81,060-2:2018 standard for SBP (- 0.7 ± 7.7 mmHg), DBP (- 0.4 ± 4.5 mmHg) and MBP (- 0.6 ± 5.2 mmHg). These results demonstrate that BP can be measured with accuracy at the finger using the OptiBP smartphone app. This may become an important tool to detect hypertension in various settings, for example in low-income countries, where the availability of smartphones is high but access to health care is low.

Automated vs. manual pain coding and heart rate estimations based on videos of older adults with and without dementia

Introduction

Technological advances have allowed for the estimation of physiological indicators from video data. FaceReader™ is an automated facial analysis software that has been used widely in studies of facial expressions of emotion and was recently updated to allow for the estimation of heart rate (HR) using remote photoplethysmography (rPPG). We investigated FaceReader™-based heart rate and pain expression estimations in older adults in relation to manual coding by experts.

Methods

Using a video dataset of older adult patients with and without dementia, we assessed the relationship between FaceReader’s™ HR estimations against a well-established Video Magnification (VM) algorithm during baseline and pain conditions. Furthermore, we examined the correspondence between the Facial Action Coding System (FACS)-based pain scores obtained through FaceReader™ and manual coding.

Results

FaceReader’s™ HR estimations were correlated with VM algorithm in baseline and pain conditions. Non-verbal FaceReader™ pain scores and manual coding were also highly correlated despite discrepancies between the FaceReader™ and manual coding in the absolute value of scores based on pain-related facial action coding of the events preceding and following the pain response.

Conclusions

Compared to expert manual FACS coding and optimized VM algorithm, FaceReader™ showed good results in estimating HR values and non-verbal pain scores.