A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum

Video-based analysis of the blink reflex in Parkinson's disease patients

We developed a video-based tool to quantitatively assess the Glabellar Tap Reflex (GTR) in patients with idiopathic Parkinson's disease (iPD) as well as healthy age-matched participants. We also video-graphically assessed the effect of dopaminergic medication on the GTR in iPD patients, as well as the frequency and blinking duration of reflex and non-reflex blinks. The Glabellar Tap Reflex is a clinical sign seen in patients e.g. suffering from iPD. Reliable tools to quantify this sign are lacking.

METHODS

We recorded the GTR in 11 iPD patients and 12 healthy controls (HC) with a consumer-grade camera at a framerate of at least 180 images/s. In these videos, reflex and non-reflex blinks were analyzed for blink count and blinking duration in an automated fashion.

RESULTS

With our setup, the GTR can be extracted from high-framerate cameras using landmarks of the MediaPipe face algorithm. iPD patients did not habituate to the GTR; dopaminergic medication did not alter that response. iPD patients' non-reflex blinks were higher in frequency and higher in blinking duration (width at half prominence); dopaminergic medication decreased the median frequency (Before medication-HC: p < 0.001, After medication-HC: p = 0.0026) and decreased the median blinking duration (Before medication-HC: p = 0.8594, After medication-HC: p = 0.6943)-both in the direction of HC.

CONCLUSION

We developed a quantitative, video-based tool to assess the GTR and other blinking-specific parameters in HC and iPD patients. Further studies could compare the video data to electromyogram (EMG) data for accuracy and comparability, as well as evaluate the specificity of the GTR in patients with other neurodegenerative disorders, in whom the GTR can also be present.

SIGNIFICANCE

The video-based detection of the blinking parameters allows for unobtrusive measurement in patients, a safer and more comfortable option.

Methods and evaluation of physiological measurements with acoustic stimuli-a systematic review

Objective. The detection of psychological loads, such as stress reactions, is receiving greater attention and social interest, as stress can have long-term effects on health O'Connor, Thayer and Vedhara (2021Ann. Rev. Psychol.72, 663-688). Acoustic stimuli, especially noise, are investigated as triggering factors. The application of physiological measurements in the detection of psychological loads enables the recording of a further quantitative dimension that goes beyond purely perceptive questionnaires. Thus, unconscious reactions to acoustic stimuli can also be captured. The numerous physiological signals and possible experimental designs with acoustic stimuli may quickly lead to a challenging implementation of the study and an increased difficulty in reproduction or comparison between studies. An unsuitable experimental design or processing of the physiological data may result in conclusions about psychological loads that are not valid anymore.Approach. The systematic review according to the preferred reporting items for systematic reviews and meta-analysis standard presented here is therefore intended to provide guidance and a basis for further studies in this field. For this purpose, studies were identified in which the participants' short-term physiological responses to acoustic stimuli were investigated in the context of a listening test in a laboratory study.Main Results. A total of 37 studies met these criteria and data items were analysed in terms of the experimental design (studied psychological load, independent variables/acoustic stimuli, participants, playback, scenario/context, duration of test phases, questionnaires for perceptual comparison) and the physiological signals (measures, calculated features, systems, data processing methods, data analysis methods, results). The overviews show that stress is the most studied psychological load in response to acoustic stimuli. An ECG/PPG system and the measurement of skin conductance were most frequently used for the detection of psychological loads. A critical aspect is the numerous different methods of experimental design, which prevent comparability of the results. In the future, more standardized methods are needed to achieve more valid analyses of the effects of acoustic stimuli.

External Hardware and Sensors, for Improved MRI

Complex engineered systems are often equipped with suites of sensors and ancillary devices that monitor their performance and maintenance needs. MRI scanners are no different in this regard. Some of the ancillary devices available to support MRI equipment, the ones of particular interest here, have the distinction of actually participating in the image acquisition process itself. Most commonly, such devices are used to monitor physiological motion or variations in the scanner's imaging fields, allowing the imaging and/or reconstruction process to adapt as imaging conditions change. "Classic" examples include electrocardiography (ECG) leads and respiratory bellows to monitor cardiac and respiratory motion, which have been standard equipment in scan rooms since the early days of MRI. Since then, many additional sensors and devices have been proposed to support MRI acquisitions. The main physical properties that they measure may be primarily "mechanical" (eg acceleration, speed, and torque), "acoustic" (sound and ultrasound), "optical" (light and infrared), or "electromagnetic" in nature. A review of these ancillary devices, as currently available in clinical and research settings, is presented here. In our opinion, these devices are not in competition with each other: as long as they provide useful and unique information, do not interfere with each other and are not prohibitively cumbersome to use, they might find their proper place in future suites of sensors. In time, MRI acquisitions will likely include a plurality of complementary signals. A little like the microbiome that provides genetic diversity to organisms, these devices can provide signal diversity to MRI acquisitions and enrich measurements. Machine-learning (ML) algorithms are well suited at combining diverse input signals toward coherent outputs, and they could make use of all such information toward improved MRI capabilities. EVIDENCE LEVEL: 2 TECHNICAL EFFICACY: Stage 1.

Conditional Generative Adversarial Networks for Data Augmentation of a Neonatal Image Dataset

In today's neonatal intensive care units, monitoring vital signs such as heart rate and respiration is fundamental for neonatal care. However, the attached sensors and electrodes restrict movement and can cause medical-adhesive-related skin injuries due to the immature skin of preterm infants, which may lead to serious complications. Thus, unobtrusive camera-based monitoring techniques in combination with image processing algorithms based on deep learning have the potential to allow cable-free vital signs measurements. Since the accuracy of deep-learning-based methods depends on the amount of training data, proper validation of the algorithms is difficult due to the limited image data of neonates. In order to enlarge such datasets, this study investigates the application of a conditional generative adversarial network for data augmentation by using edge detection frames from neonates to create RGB images. Different edge detection algorithms were used to validate the input images' effect on the adversarial network's generator. The state-of-the-art network architecture Pix2PixHD was adapted, and several hyperparameters were optimized. The quality of the generated RGB images was evaluated using a Mechanical Turk-like multistage survey conducted by 30 volunteers and the FID score. In a fake-only stage, 23% of the images were categorized as real. A direct comparison of generated and real (manually augmented) images revealed that 28% of the fake data were evaluated as more realistic. An FID score of 103.82 was achieved. Therefore, the conducted study shows promising results for the training and application of conditional generative adversarial networks to augment highly limited neonatal image datasets.

Video-Based Hand Movement Analysis of Parkinson Patients before and after Medication Using High-Frame-Rate Videos and MediaPipe

Tremor is one of the common symptoms of Parkinson's disease (PD). Thanks to the recent evolution of digital technologies, monitoring of PD patients' hand movements employing contactless methods gained momentum. Objective: We aimed to quantitatively assess hand movements in patients suffering from PD using the artificial intelligence (AI)-based hand-tracking technologies of MediaPipe. Method: High-frame-rate videos and accelerometer data were recorded from 11 PD patients, two of whom showed classical Parkinsonian-type tremor. In the OFF-state and 30 Minutes after taking their standard oral medication (ON-state), video recordings were obtained. First, we investigated the frequency and amplitude relationship between the video and accelerometer data. Then, we focused on quantifying the effect of taking standard oral treatments. Results: The data extracted from the video correlated well with the accelerometer-based measurement system. Our video-based approach identified the tremor frequency with a small error rate (mean absolute error 0.229 (±0.174) Hz) and an amplitude with a high correlation. The frequency and amplitude of the hand movement before and after medication in PD patients undergoing medication differ. PD Patients experienced a decrease in the mean value for frequency from 2.012 (±1.385) Hz to 1.526 (±1.007) Hz and in the mean value for amplitude from 8.167 (±15.687) a.u. to 4.033 (±5.671) a.u. Conclusions: Our work achieved an automatic estimation of the movement frequency, including the tremor frequency with a low error rate, and to the best of our knowledge, this is the first paper that presents automated tremor analysis before/after medication in PD, in particular using high-frame-rate video data.

Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving

In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.

Camera fusion for real-time temperature monitoring of neonates using deep learning

Abstract

The continuous monitoring of vital signs is a crucial aspect of medical care in neonatal intensive care units. Since cable-based sensors pose a potential risk for the immature skin of preterm infants, unobtrusive monitoring techniques using camera systems are increasingly investigated. The combination of deep learning–based algorithms and camera modalities such as RGB and infrared thermography can improve the development of cable-free methods for the extraction of vital parameters. In this study, a real-time approach for local extraction of temperatures on the body surface of neonates using a multi-modal clinical dataset was implemented. Therefore, a trained deep learning–based keypoint detector was used for body landmark prediction in RGB. Image registration was conducted to transfer the RGB points to the corresponding thermographic recordings. These landmarks were used to extract the body surface temperature in various regions to determine the central-peripheral temperature difference. A validation of the keypoint detector showed a mean average precision of 0.82. The registration resulted in mean absolute errors of 16.4 px (8.2 mm) for x and 22.4 px (11.2 mm) for y. The evaluation of the temperature extraction revealed a mean absolute error of 0.55 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘C. A final performance of 31 fps was observed on the NVIDIA Jetson Xavier NX module, which proves real-time capability on an embedded GPU system. As a result, the approach can perform real-time temperature extraction on a low-cost GPU module.

Graphical abstract

Remote investigation and assessment of vital signs (RIA-VS)-proof of concept for contactless estimation of blood pressure, pulse, respiratory rate, and oxygen saturation in patients with suspicion of COVID-19

BACKGROUND

Vital signs are critical in assessing the severity and prognosis of infections, for example, COVID-19, influenza, sepsis, and pneumonia. This study aimed to evaluate a new method for rapid camera-based non-contact measurement of heart rate, blood oxygen saturation, respiratory rate, and blood pressure.

METHODS

Consecutive adult patients attending a hospital emergency department for suspected COVID-19 infection were invited to participate. Vital signs measured with a new camera-based method were compared to the corresponding standard reference methods. The camera device observed the patient's face for 30 s from ∼1 m.

RESULTS

Between 1 April and 1 October 2020, 214 subjects were included in the trial, 131 female (61%) and 83 male (39%). The mean age was 44 years (range 18-81 years). The new camera-based device's vital signs measurements were, on average, very close to the gold standard but the random variation was larger than the reference methods.

CONCLUSIONS

The principle of contactless measurement of blood pressure, pulse, respiratory rate, and oxygen saturation works, which is very promising. However, technical improvements to the equipment used in this study to reduce its random variability is required before clinical implementation. This will likely be a game changer once this is sorted out.

CLINICAL TRIAL REGISTRATION

Universal Trial Number (UTN) U1111-1251-4114 and the ClinicalTrials.gov Identifier NCT04383457.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO₂). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO₂, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO₂ are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO₂ is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO₂ monitoring.

Algorithmic insights of camera-based respiratory motion extraction

OBJECTIVE

Measuring the respiratory signal from a video based on body motion has been proposed and recently matured in products for contactless health monitoring. The core algorithm for this application is the measurement of tiny chest/abdominal motions induced by respiration (i.e. capturing sub-pixel displacement caused by subtle motion between subsequent video frames), and the fundamental challenge is motion sensitivity. Though prior art reported on the validation with real human subjects, there is no thorough or rigorous benchmark to quantify the sensitivities and boundary conditions of motion-based core respiratory algorithms.

APPROACH

A set-up was designed with a fully-controllable physical phantom to investigate the essence of core algorithms, together with a mathematical model incorporating two motion estimation strategies and three spatial representations, leading to six algorithmic combinations for respiratory signal extraction. Their promises and limitations are discussed and clarified through the phantom benchmark.

MAIN RESULTS

With the variation of phantom motion intensity between 0.5 mm and 8 mm, the recommended approach obtains an average precision, recall, coverage and MAE of 88.1%, 91.8%, 95.5% and 2.1bpm in the day-light condition, and 81.7%, 90.0%, 93.9% and 4.4 bpm in the night condition.

SIGNIFICANCE

The insights gained in this paper are intended to improve the understanding and applications of camera-based respiration measurement in health monitoring. The limitations of this study stem from the used physical phantom that does not consider human factors like body shape, sleeping posture, respiratory diseases, etc., and the investigated scenario is focused on sleep monitoring, not including scenarios with a sitting or standing patient like in clinical ward and triage.

A Setup for Camera-Based Detection of Simulated Pathological States Using a Neonatal Phantom

Premature infants are among the most vulnerable patients in a hospital. Due to numerous complications associated with immaturity, a continuous monitoring of vital signs with a high sensitivity and accuracy is required. Today, wired sensors are attached to the patient's skin. However, adhesive electrodes can be potentially harmful as they can damage the very thin immature skin. Although unobtrusive monitoring systems using cameras show the potential to replace cable-based techniques, advanced image processing algorithms are data-driven and, therefore, need much data to be trained. Due to the low availability of public neonatal image data, a patient phantom could help to implement algorithms for the robust extraction of vital signs from video recordings. In this work, a camera-based system is presented and validated using a neonatal phantom, which enabled a simulation of common neonatal pathologies such as hypo-/hyperthermia and brady-/tachycardia. The implemented algorithm was able to continuously measure and analyze the heart rate via photoplethysmography imaging with a mean absolute error of 0.91 bpm, as well as the distribution of a neonate's skin temperature with a mean absolute error of less than 0.55 °C. For accurate measurements, a temperature gain offset correction on the registered image from two infrared thermography cameras was performed. A deep learning-based keypoint detector was applied for temperature mapping and guidance for the feature extraction. The presented setup successfully detected several levels of hypo- and hyperthermia, an increased central-peripheral temperature difference, tachycardia and bradycardia.

Photoplethysmography for demarcation of cutaneous squamous cell carcinoma

A video processing algorithm designed to identify cancer suspicious skin areas is presented here. It is based on video recordings of squamous cell carcinoma in the skin. Squamous cell carcinoma is a common malignancy, normally treated by surgical removal. The surgeon should always balance sufficient tissue removal against unnecessary mutilation, and therefore methods for distinction of cancer boundaries are wanted. Squamous cell carcinoma has angiogenesis and increased blood supply. Remote photoplethysmography is an evolving technique for analysis of signal variations in video recordings in order to extract vital signs such as pulsation. We hypothesize that the remote photoplethysmography signal inside the area of a squamous cell carcinoma is significantly different from the surrounding healthy skin. Based on high speed video recordings of 13 patients with squamous cell carcinoma, we have examined temporal signal differences in cancer areas versus healthy skin areas. A significant difference in temporal signal changes between cancer areas and healthy areas was found. Our video processing algorithm showed promising results encouraging further investigation to clarify how detailed distinctions can be made.

A Neonatal Phantom for Vital Signs Simulation

Neonatal intensive care units provide vital medical support for premature infants. The key aspect in neonatal care is the continuous monitoring of vital signs measured using adhesive skin sensors. Since sensors can cause irritation of the skin and lead to infections, research focuses on contact-free, camera-based methods such as infrared thermography and photoplethysmography imaging. The development of image processing algorithms requires large datasets, but recording the necessary data from studies brings tremendous effort and costs. Therefore, realistic patient phantoms would be feasible to create a comprehensive dataset and validate image-based algorithms. This work describes the realization of a neonatal phantom which can simulate physiological vital parameters such as pulse rate and thermoregulation. It mimics the outer appearance of premature infants using a 3D printed base structure coated with several layers of modified, skin-colored silicone. A distribution of red and infrared LEDs in the scaffold enables the simulation of a PPG signal by mimicking pulsative light intensity changes on the skin. Additionally, the body temperature of the phantom is individually adjustable in several regions using heating elements. In the validation process for PPG simulation, the feasibility of setting different pulse frequencies and the variation of oxygen saturation levels was obtained. Furthermore, heating tests showed region-dependent temperature variations between 0.19 ^°C and 0.81 ^°C around the setpoint. In conclusion, the proposed neonatal phantom can be used to simulate a variety of vital parameters of preterm infants and, therefore, enables the implementation of image processing algorithms for the analysis of the medical state.

Effectiveness of consumer-grade contactless vital signs monitors: a systematic review and meta-analysis

The objective of this systematic review and meta-analysis was to analyze the effectiveness of contactless vital sign monitors that utilize a consumer-friendly camera versus medical grade instruments. A multiple database search was conducted from inception to September 2020. Inclusion criteria were as follows: studies that used a consumer-grade camera (smartphone/webcam) to examine contactless vital signs in adults; evaluated the non-contact device against a reference medical device; and used the participants’ face for measurement. Twenty-six studies were included in the review of which 16 were included in Pearson’s correlation and 14 studies were included in the Bland–Altman meta-analysis. Twenty-two studies measured heart rate (HR) (92%), three measured blood pressure (BP) (12%), and respiratory rate (RR) (12%). No study examined blood oxygen saturation (SpO₂). Most studies had a small sample size (≤ 30 participants) and were performed in a laboratory setting. Our meta-analysis found that consumer-grade contactless vital sign monitors were accurate in comparison to a medical device in measuring HR. Current contactless monitors have limitations such as motion, poor lighting, and lack of automatic face tracking. Currently available consumer-friendly contactless monitors measure HR accurately compared to standard medical devices. More studies are needed to assess the accuracy of contactless BP and RR monitors. Implementation of contactless vital sign monitors for clinical use will require validation in a larger population, in a clinical setting, and expanded to encompass other vital signs including BP, RR, and SpO₂.

Noncontact Monitoring of Heart Rate and Heart Rate Variability in Geriatric Patients Using Photoplethysmography Imaging

OBJECTIVE

Geriatric patients, especially those with dementia or in a delirious state, do not accept conventional contact-based monitoring. Therefore, we propose to measure heart rate (HR) and heart rate variability (HRV) of geriatric patients in a noncontact and unobtrusive way using photoplethysmography imaging (PPGI).

METHODS

PPGI video sequences were recorded from 10 geriatric patients and 10 healthy elderly people using a monochrome camera operating in the near-infrared spectrum and a colour camera operating in the visible spectrum. PPGI waveforms were extracted from both cameras using superpixel-based regions of interests (ROI). A classifier based on bagged trees was trained to automatically select artefact-free ROIs for HR estimation. HRV was calculated in the time-domain and frequency-domain.

RESULTS

an RMSE of 1.03 bpm and a correlation of 0.8 with the reference was achieved using the NIR camera for HR estimation. Using the RGB camera, RMSE and correlation improved to 0.48 bpm and 0.95, respectively. Correlation for HRV in the frequency-domain (LF/HF-ratio) was 0.50 using the NIR camera and 0.70 using the RGB camera.

CONCLUSION

We were able to demonstrate that PPGI is very suitable to measure HR and HRV in geriatric patients. We strongly believe that PPGI will become clinically relevant in monitoring of geriatric patients.

SIGNIFICANCE

we are the first group to measure both HR and HRV in awake geriatric patients using PPGI. Moreover, we systematically evaluate the effects of the spectrum (near-infrared vs. visible), ROI, and additional motion artefact reduction algorithms on the accuracy of estimated HR and HRV.

A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients

Infrared thermography for camera-based skin temperature measurement is increasingly used in medical practice, e.g., to detect fevers and infections, such as recently in the COVID-19 pandemic. This contactless method is a promising technology to continuously monitor the vital signs of patients in clinical environments. In this study, we investigated both skin temperature trend measurement and the extraction of respiration-related chest movements to determine the respiratory rate using low-cost hardware in combination with advanced algorithms. In addition, the frequency of medical examinations or visits to the patients was extracted. We implemented a deep learning-based algorithm for real-time vital sign extraction from thermography images. A clinical trial was conducted to record data from patients on an intensive care unit. The YOLOv4-Tiny object detector was applied to extract image regions containing vital signs (head and chest). The infrared frames were manually labeled for evaluation. Validation was performed on a hold-out test dataset of 6 patients and revealed good detector performance (0.75 intersection over union, 0.94 mean average precision). An optical flow algorithm was used to extract the respiratory rate from the chest region. The results show a mean absolute error of 2.69 bpm. We observed a computational performance of 47 fps on an NVIDIA Jetson Xavier NX module for YOLOv4-Tiny, which proves real-time capability on an embedded GPU system. In conclusion, the proposed method can perform real-time vital sign extraction on a low-cost system-on-module and may thus be a useful method for future contactless vital sign measurements.

Fast body part segmentation and tracking of neonatal video data using deep learning

Photoplethysmography imaging (PPGI) for non-contact monitoring of preterm infants in the neonatal intensive care unit (NICU) is a promising technology, as it could reduce medical adhesive-related skin injuries and associated complications. For practical implementations of PPGI, a region of interest has to be detected automatically in real time. As the neonates' body proportions differ significantly from adults, existing approaches may not be used in a straightforward way, and color-based skin detection requires RGB data, thus prohibiting the use of less-intrusive near-infrared (NIR) acquisition. In this paper, we present a deep learning-based method for segmentation of neonatal video data. We augmented an existing encoder-decoder semantic segmentation method with a modified version of the ResNet-50 encoder. This reduced the computational time by a factor of 7.5, so that 30 frames per second can be processed at 960 × 576 pixels. The method was developed and optimized on publicly available databases with segmentation data from adults. For evaluation, a comprehensive dataset consisting of RGB and NIR video recordings from 29 neonates with various skin tones recorded in two NICUs in Germany and India was used. From all recordings, 643 frames were manually segmented. After pre-training the model on the public adult data, parts of the neonatal data were used for additional learning and left-out neonates are used for cross-validated evaluation. On the RGB data, the head is segmented well (82% intersection over union, 88% accuracy), and performance is comparable with those achieved on large, public, non-neonatal datasets. On the other hand, performance on the NIR data was inferior. By employing data augmentation to generate additional virtual NIR data for training, results could be improved and the head could be segmented with 62% intersection over union and 65% accuracy. The method is in theory capable of performing segmentation in real time and thus it may provide a useful tool for future PPGI applications. Graphical Abstract This work presents the development of a customized, real-time capable Deep Learning architecture for segmenting of neonatal videos recorded in the intensive care unit. In addition to hand-annotated data, transfer learning is exploited to improve performance.

Contact-free sensor signals as a new digital biomarker for cardiovascular disease: chances and challenges

Multiple sensor systems are used to monitor physiological parameters, activities of daily living and behaviour. Digital biomarkers can be extracted and used as indicators for health and disease. Signal acquisition is either by object sensors, wearable sensors, or contact-free sensors including cameras, pressure sensors, non-contact capacitively coupled electrocardiogram (cECG), radar, and passive infrared motion sensors. This review summarizes contemporary knowledge of the use of contact-free sensors for patients with cardiovascular disease and healthy subjects following the PRISMA declaration. Chances and challenges are discussed. Thirty-six publications were rated to be of medium (31) or high (5) relevance. Results are best for monitoring of heart rate and heart rate variability using cardiac vibration, facial camera, or cECG; for respiration using cardiac vibration, cECG, or camera; and for sleep using ballistocardiography. Early results from radar sensors to monitor vital signs are promising. Contact-free sensors are little invasive, well accepted and suitable for long-term monitoring in particular in patient's homes. A major problem are motion artefacts. Results from long-term use in larger patient cohorts are still lacking, but the technology is about to emerge the market and we can expect to see more clinical results in the near future.

A Novel Non-contact Self-Injection-Locked Radar for Vital Sign Sensing and Body Movement Monitoring in COVID-19 Isolation Ward

Background

The outbreak of Coronavirus disease (COVID-19) pandemic has become the most serious global health issue. Isolation policy in hospitals is one of the most crucial protocols to prevent nosocomial infection of COVID-19. It is important to monitor and assess the physical conditions of the patients in isolation.

Methods

Our institution has installed the novel non-contact wireless sensor for vital sign sensing and body movement monitoring for patients in COVID-19 isolation ward.

Results

We have collected and compared data between the radar record with the nurse’s handover record of two patients, one recorded for 13 days and the other recorded for 5 days. The P value by Fisher’s exact test were 0.139 (temperature, P > 0.05) and 0.292 (heart beat rate, P > 0.05) respectively.

Conclusions

This is the first report about the application experience of this equipment. Therefore we attempted to share the experience and try to apply this equipment in COVID-19 patients in future to offer the more reliable and safe policy.

Electronic supplementary material

The online version of this article (10.1007/s10916-020-01637-z) contains supplementary material, which is available to authorized users.

Cyclostationary-Based Vital Signs Detection Using Microwave Radar at 2.5 GHz

Non-contact detection and estimation of vital signs such as respiratory and cardiac frequencies is a powerful tool for surveillance applications. In particular, the continuous wave bio-radar has been widely investigated to determine the physiological parameters in a non-contact manner. Since the RF-reflected signal from the human body is corrupted by noise and random body movements, traditional Fourier analysis fails to detect the heart and breathing frequencies. In this effort, cyclostationary analysis has been used to improve the radar performance for non-invasive measurement of respiratory rate and heart rate. However, the preliminary works focus only on one frequency and do not include the impact of attenuation and random movement of the body in the analysis. Hence in this paper, we evaluate the impact of distance and noise on the cyclic features of the reflected signal. Furthermore, we explore the assessment of second order cyclostationary signal processing performance by developing the cyclic mean, the conjugate cyclic autocorrelation and the cyclic cumulant. In addition, the analysis is carried out using a reduced number of samples to reduce the response time. Implementation of the cyclostationary technique using a bi-static radar configuration at 2.5 GHz is shown as an example to demonstrate the proposed approach.

Impact of makeup on remote-PPG monitoring

Camera-based remote photoplethysmography (remote-PPG) enables contactless measurement of blood volume pulse from the human skin. Skin visibility is essential to remote-PPG as the camera needs to capture the light reflected from the skin that penetrates deep into skin tissues and carries blood pulsation information. The use of facial makeup may jeopardize this measurement by reducing the amount of light penetrating into and reflecting from the skin. In this paper, we conduct an empirical study to thoroughly investigate the impact of makeup on remote-PPG monitoring, in both the visible (RGB) and invisible (Near Infrared, NIR) lighting conditions. The experiment shows that makeup has negative influence on remote-PPG, which reduces the relative PPG strength (AC/DC) at different wavelengths and changes the normalized PPG signature across multiple wavelengths. It makes (i) the pulse-rate extraction more difficult in both the RGB and NIR, although NIR is less affected than RGB, and (ii) the blood oxygen saturation extraction in NIR impossible. To the best of our knowledge, this is the first work that systematically investigate the impact of makeup on camera-based remote-PPG monitoring.

Modified RGB Cameras for Infrared Remote-PPG

Multi-wavelength cameras play an essential role in remote photoplethysmography (PPG). Whereas these are readily available for visible light, this is not the case for near infrared (NIR). We propose to modify existing RGB cameras to make them suited for NIR-PPG. In particular, we exploit the spectral leakage of the RGB channels in infrared in combination with a narrow dual-band optical filter. Such camera modification is simple, cost-effective, easy to implement, and it is shown to attain a pulse-rate extraction performance comparable to that of multiple narrow-band NIR cameras.