1
|
Henry B, Merz M, Hoang H, Abdulkarim G, Wosik J, Schoettker P. Cuffless Blood Pressure in clinical practice: challenges, opportunities and current limits. Blood Press 2024; 33:2304190. [PMID: 38245864 DOI: 10.1080/08037051.2024.2304190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 01/07/2024] [Indexed: 01/22/2024]
Abstract
Background: Cuffless blood pressure measurement technologies have attracted significant attention for their potential to transform cardiovascular monitoring.Methods: This updated narrative review thoroughly examines the challenges, opportunities, and limitations associated with the implementation of cuffless blood pressure monitoring systems.Results: Diverse technologies, including photoplethysmography, tonometry, and ECG analysis, enable cuffless blood pressure measurement and are integrated into devices like smartphones and smartwatches. Signal processing emerges as a critical aspect, dictating the accuracy and reliability of readings. Despite its potential, the integration of cuffless technologies into clinical practice faces obstacles, including the need to address concerns related to accuracy, calibration, and standardization across diverse devices and patient populations. The development of robust algorithms to mitigate artifacts and environmental disturbances is essential for extracting clear physiological signals. Based on extensive research, this review emphasizes the necessity for standardized protocols, validation studies, and regulatory frameworks to ensure the reliability and safety of cuffless blood pressure monitoring devices and their implementation in mainstream medical practice. Interdisciplinary collaborations between engineers, clinicians, and regulatory bodies are crucial to address technical, clinical, and regulatory complexities during implementation. In conclusion, while cuffless blood pressure monitoring holds immense potential to transform cardiovascular care. The resolution of existing challenges and the establishment of rigorous standards are imperative for its seamless incorporation into routine clinical practice.Conclusion: The emergence of these new technologies shifts the paradigm of cardiovascular health management, presenting a new possibility for non-invasive continuous and dynamic monitoring. The concept of cuffless blood pressure measurement is viable and more finely tuned devices are expected to enter the market, which could redefine our understanding of blood pressure and hypertension.
Collapse
Affiliation(s)
- Benoit Henry
- Service of Anesthesiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Maxime Merz
- Service of Anesthesiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Harry Hoang
- Service of Anesthesiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Ghaith Abdulkarim
- Neuro-Informatics Laboratory, Mayo Clinic College of Medicine and Science, Rochester, MN, USA
- Department of Neurological Surgery, Mayo Clinic, Rochester, MN, USA
| | - Jedrek Wosik
- Department of Medicine, Duke University School of Medicine, Durham, North Carolina
| | - Patrick Schoettker
- Service of Anesthesiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| |
Collapse
|
2
|
Di Lernia D, Finotti G, Tsakiris M, Riva G, Naber M. Remote photoplethysmography (rPPG) in the wild: Remote heart rate imaging via online webcams. Behav Res Methods 2024; 56:6904-6914. [PMID: 38632165 DOI: 10.3758/s13428-024-02398-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/12/2024] [Indexed: 04/19/2024]
Abstract
Remote photoplethysmography (rPPG) is a low-cost technique to measure physiological parameters such as heart rate by analyzing videos of a person. There has been growing attention to this technique due to the increased possibilities and demand for running psychological experiments on online platforms. Technological advancements in commercially available cameras and video processing algorithms have led to significant progress in this field. However, despite these advancements, past research indicates that suboptimal video recording conditions can severely compromise the accuracy of rPPG. In this study, we aimed to develop an open-source rPPG methodology and test its performance on videos collected via an online platform, without control of the hardware of the participants and the contextual variables, such as illumination, distance, and motion. Across two experiments, we compared the results of the rPPG extraction methodology to a validated dataset used for rPPG testing. Furthermore, we then collected 231 online video recordings and compared the results of the rPPG extraction to finger pulse oximeter data acquired with a validated mobile heart rate application. Results indicated that the rPPG algorithm was highly accurate, showing a significant degree of convergence with both datasets thus providing an improved tool for recording and analyzing heart rate in online experiments.
Collapse
Affiliation(s)
- Daniele Di Lernia
- Humane Technology Lab, Università Cattolica del Sacro Cuore, Largo Gemelli, 1, 20100, Milan, Italy.
- Applied Technology for Neuro-Psychology Lab, IRCCS Istituto Auxologico Italiano, Via Magnasco, 2, 20149, Milan, Italy.
- Department of Psychology, Università Cattolica del Sacro Cuore, Largo Gemelli, 1, 20100, Milan, Italy.
| | - Gianluca Finotti
- Lab of Action and Body, Department of Psychology, Royal Holloway, University of London, Egham Hill, Egham, TW20 0EX, UK
| | - Manos Tsakiris
- Lab of Action and Body, Department of Psychology, Royal Holloway, University of London, Egham Hill, Egham, TW20 0EX, UK
- Centre for the Politics of Feelings, School of Advanced Study, University of London, London, UK
| | - Giuseppe Riva
- Humane Technology Lab, Università Cattolica del Sacro Cuore, Largo Gemelli, 1, 20100, Milan, Italy
- Applied Technology for Neuro-Psychology Lab, IRCCS Istituto Auxologico Italiano, Via Magnasco, 2, 20149, Milan, Italy
| | - Marnix Naber
- Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584CS, Utrecht, The Netherlands
| |
Collapse
|
3
|
Nguyen N, Nguyen L, Li H, Bordallo López M, Álvarez Casado C. Evaluation of video-based rPPG in challenging environments: Artifact mitigation and network resilience. Comput Biol Med 2024; 179:108873. [PMID: 39053334 DOI: 10.1016/j.compbiomed.2024.108873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2024] [Revised: 07/05/2024] [Accepted: 07/08/2024] [Indexed: 07/27/2024]
Abstract
Video-based remote photoplethysmography (rPPG) has emerged as a promising technology for non-contact vital sign monitoring, especially under controlled conditions. However, the accurate measurement of vital signs in real-world scenarios faces several challenges, including artifacts induced by videocodecs, low-light noise, degradation, low dynamic range, occlusions, and hardware and network constraints. In this article, a systematic and comprehensive investigation of these issues is conducted, measuring their detrimental effects on the quality of rPPG measurements. Additionally, practical strategies are proposed for mitigating these challenges to improve the dependability and resilience of video-based rPPG systems. Methods for effective biosignal recovery in the presence of network limitations are detailed, along with denoising and inpainting techniques aimed at preserving video frame integrity. Compared to previous studies, this paper addresses a broader range of variables and demonstrates improved accuracy across various rPPG methods, emphasizing generalizability for practical applications in diverse scenarios with varying data quality. Extensive evaluations and direct comparisons demonstrate the effectiveness of these approaches in enhancing rPPG measurements under challenging environments, contributing to the development of more reliable and effective remote vital sign monitoring technologies.
Collapse
Affiliation(s)
- Nhi Nguyen
- Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Oulu, Finland.
| | - Le Nguyen
- Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Oulu, Finland.
| | - Honghan Li
- Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Oulu, Finland; Division of Bioengineering, Graduate School of Engineering Science, Osaka University, Osaka, Japan.
| | - Miguel Bordallo López
- Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Oulu, Finland; VTT Technical Research Center of Finland Ltd., Oulu, Finland.
| | | |
Collapse
|
4
|
Zhu S, Liu S, Jing X, Yang Y, She C. Innovative approaches in imaging photoplethysmography for remote blood oxygen monitoring. Sci Rep 2024; 14:19144. [PMID: 39160216 PMCID: PMC11333616 DOI: 10.1038/s41598-024-70192-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Accepted: 08/13/2024] [Indexed: 08/21/2024] Open
Abstract
Peripheral Capillary Oxygen Saturation (SpO2) has received increasing attention during the COVID-19 pandemic. Clinical investigations have demonstrated that individuals afflicted with COVID-19 exhibit notably reduced levels of SpO2 before the deterioration of their health status. To cost-effectively enable individuals to monitor their SpO2, this paper proposes a novel neural network model named "ITSCAN" based on Temporal Shift Module. Benefiting from the widespread use of smartphones, this model can assess an individual's SpO2 in real time, utilizing standard facial video footage, with a temporal granularity of seconds. The model is interweaved by two distinct branches: the motion branch, responsible for extracting spatiotemporal data features and the appearance branch, focusing on the correlation between feature channels and the location information of feature map using coordinate attention mechanisms. Accordingly, the SpO2 estimator generates the corresponding SpO2 value. This paper summarizes for the first time 5 loss functions commonly used in the SpO2 estimation model. Subsequently, a novel loss function has been contributed through the examination of various combinations and careful selection of hyperparameters. Comprehensive ablation experiments analyze the independent impact of each module on the overall model performance. Finally, the experimental results based on the public dataset (VIPL-HR) show that our model has obvious advantages in MAE (1.10%) and RMSE (1.19%) compared with related work, which implies more accuracy of the proposed method to contribute to public health.
Collapse
Affiliation(s)
- Shangwei Zhu
- School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| | - Shaohua Liu
- School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China.
| | - Xingjian Jing
- Department of Mechanical Engineering, Hong Kong City University, Hong Kong, 999077, China
| | - Yuchong Yang
- School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| | - Chundong She
- School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| |
Collapse
|
5
|
Cao M, Cheng X, Liu X, Jiang Y, Yu H, Shi J. ST-Phys: Unsupervised Spatio-Temporal Contrastive Remote Physiological Measurement. IEEE J Biomed Health Inform 2024; 28:4613-4624. [PMID: 38743531 DOI: 10.1109/jbhi.2024.3400869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
Remote photoplethysmography (rPPG) is a non-contact method that employs facial videos for measuring physiological parameters. Existing rPPG methods have achieved remarkable performance. However, the success mainly profits from supervised learning over massive labeled data. On the other hand, existing unsupervised rPPG methods fail to fully utilize spatio-temporal features and encounter challenges in low-light or noise environments. To address these problems, we propose an unsupervised contrast learning approach, ST-Phys. We incorporate a low-light enhancement module, a temporal dilated module, and a spatial enhanced module to better deal with long-term dependencies under the random low-light conditions. In addition, we design a circular margin loss, wherein rPPG signals originating from identical videos are attracted, while those from distinct videos are repelled. Our method is assessed on six openly accessible datasets, including RGB and NIR videos. Extensive experiments reveal the superior performance of our proposed ST-Phys over state-of-the-art unsupervised rPPG methods. Moreover, it offers advantages in parameter reduction and noise robustness.
Collapse
|
6
|
Khaleel Sallam Ma'aitah M, Helwan A. 3D DenseNet with temporal transition layer for heart rate estimation from real-life RGB videos. Technol Health Care 2024:THC241104. [PMID: 39058471 DOI: 10.3233/thc-241104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/28/2024]
Abstract
BACKGROUND Deep learning has demonstrated superior performance over traditional methods for the estimation of heart rates in controlled contexts. However, in less controlled scenarios this performance seems to vary based on the training dataset and the architecture of the deep learning models. OBJECTIVES In this paper, we develop a deep learning-based model leveraging the power of 3D convolutional neural networks (3DCNN) to extract temporal and spatial features that lead to an accurate heart rates estimation from RGB no pre-defined region of interest (ROI) videos. METHODS We propose a 3D DenseNet with a 3D temporal transition layer for the estimation of heart rates from a large-scale dataset of videos that appear more hospital-like and real-life than other existing facial video-based datasets. RESULTS Experimentally, our model was trained and tested on this less controlled dataset and showed heart rate estimation performance with root mean square error (RMSE) of 8.68 BPM and mean absolute error (MAE) of 3.34 BPM. CONCLUSION Moreover, we show that such a model can also achieve better results than the state-of-the-art models when tested on the VIPL-HR public dataset.
Collapse
|
7
|
Hsu JY, Jiang TY, Chao PCP. A Fast FPGA Hardware Accelerator for Remote Heart Rate Detection Based on RGB Vision. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2024; 18:592-607. [PMID: 38227402 DOI: 10.1109/tbcas.2024.3354505] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/17/2024]
Abstract
A fast hardware accelerator is created by this work via field programmable gate array (FPGA) to estimate heart rate (HR) through the video recorded by a RGB camera based on the technology of remote photoplethysmography (rPPG). The method of rPPG acquires physiological signals of a human body by analyzing the subtle color changes on the surface of the human skin. The hardware implementation of rPPG to estimate HR is proposed herein to aim for a much faster calculation speed than software for a number of applications, like heart failure pre-warning of an in-action athlete and drowsiness detection of a driver. In this accelerator, ICA (Independent Component Analysis) is used to recover the blood volume pulse from the raw signals of remote PPG, and then obtain the heart rate value. The architecture of the hardware circuit is described in Verilog HDL and verified by Quartus II, and also implemented in an Altera DE10-Standard FPGA board, which consists of image capture, heart rate algorithm and image display. A TRDB-D5M camera is utilized for image capture. Two experiments were conducted with image collecting duration of 16 seconds and 8 seconds respectively, and the commercial device Omron HEM-6111 was used as the golden value. The proposed system achieves an accuracy in (ME ± 1.96SD) of -0.76 ± 5.09 and -0.70 ± 8.71 bpm in the short periods of 16-second and 8-second versions, respectively, which outperforms all the reported prior works in combined computation time and accuracy.
Collapse
|
8
|
Xiang G, Yao S, Peng Y, Deng H, Wu X, Wang K, Li Y, Wu F. An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer. Phys Eng Sci Med 2024; 47:729-739. [PMID: 38504066 DOI: 10.1007/s13246-024-01401-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Accepted: 02/06/2024] [Indexed: 03/21/2024]
Abstract
Remote photoplethysmography (rPPG) technology is a non-contact physiological signal measurement method, characterized by non-invasiveness and ease of use. It has broad application potential in medical health, human factors engineering, and other fields. However, current rPPG technology is highly susceptible to variations in lighting conditions, head pose changes, and partial occlusions, posing significant challenges for its widespread application. In order to improve the accuracy of remote heart rate estimation and enhance model generalization, we propose PulseFormer, a dual-path network based on transformer. By integrating local and global information and utilizing fast and slow paths, PulseFormer effectively captures the temporal variations of key regions and spatial variations of the global area, facilitating the extraction of rPPG feature information while mitigating the impact of background noise variations. Heart rate estimation results on the popular rPPG dataset show that PulseFormer achieves state-of-the-art performance on public datasets. Additionally, we establish a dataset containing facial expressions and synchronized physiological signals in driving scenarios and test the pre-trained model from the public dataset on this collected dataset. The results indicate that PulseFormer exhibits strong generalization capabilities across different data distributions in cross-scenario settings. Therefore, this model is applicable for heart rate estimation of individuals in various scenarios.
Collapse
Affiliation(s)
- Guoliang Xiang
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| | - Song Yao
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| | - Yong Peng
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China.
| | - Hanwen Deng
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| | - Xianhui Wu
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| | - Kui Wang
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| | - Yingli Li
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| | - Fan Wu
- Key Laboratory of Traffic Safety on Track of Ministry of Education, School of Traffic & Transportation Engineering, Central South University, Changsha, 410075, China
| |
Collapse
|
9
|
Zhu F, Niu Q, Li X, Zhao Q, Su H, Shuai J. FM-FCN: A Neural Network with Filtering Modules for Accurate Vital Signs Extraction. RESEARCH (WASHINGTON, D.C.) 2024; 7:0361. [PMID: 38737196 PMCID: PMC11082448 DOI: 10.34133/research.0361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 04/01/2024] [Indexed: 05/14/2024]
Abstract
Neural networks excel at capturing local spatial patterns through convolutional modules, but they may struggle to identify and effectively utilize the morphological and amplitude periodic nature of physiological signals. In this work, we propose a novel network named filtering module fully convolutional network (FM-FCN), which fuses traditional filtering techniques with neural networks to amplify physiological signals and suppress noise. First, instead of using a fully connected layer, we use an FCN to preserve the time-dimensional correlation information of physiological signals, enabling multiple cycles of signals in the network and providing a basis for signal processing. Second, we introduce the FM as a network module that adapts to eliminate unwanted interference, leveraging the structure of the filter. This approach builds a bridge between deep learning and signal processing methodologies. Finally, we evaluate the performance of FM-FCN using remote photoplethysmography. Experimental results demonstrate that FM-FCN outperforms the second-ranked method in terms of both blood volume pulse (BVP) signal and heart rate (HR) accuracy. It substantially improves the quality of BVP waveform reconstruction, with a decrease of 20.23% in mean absolute error (MAE) and an increase of 79.95% in signal-to-noise ratio (SNR). Regarding HR estimation accuracy, FM-FCN achieves a decrease of 35.85% in MAE, 29.65% in error standard deviation, and 32.88% decrease in 95% limits of agreement width, meeting clinical standards for HR accuracy requirements. The results highlight its potential in improving the accuracy and reliability of vital sign measurement through high-quality BVP signal extraction. The codes and datasets are available online at https://github.com/zhaoqi106/FM-FCN.
Collapse
Affiliation(s)
- Fangfang Zhu
- Department of Physics, and Fujian Provincial Key Laboratory for Soft Functional Materials Research,
Xiamen University, Xiamen 361005, China
- National Institute for Data Science in Health and Medicine, and State Key Laboratory of Cellular Stress Biology, Innovation Center for Cell Signaling Network,
Xiamen University, Xiamen 361005, China
| | - Qichao Niu
- Vitalsilicon Technology Co. Ltd., Jiaxing, Zhejiang 314006, China
| | - Xiang Li
- Department of Physics, and Fujian Provincial Key Laboratory for Soft Functional Materials Research,
Xiamen University, Xiamen 361005, China
| | - Qi Zhao
- School of Computer Science and Software Engineering,
University of Science and Technology Liaoning, Anshan 114051, China
| | - Honghong Su
- Yangtze Delta Region Institute of Tsinghua University, Zhejiang, Jiaxing 314006, China
| | - Jianwei Shuai
- Wenzhou Institute,
University of Chinese Academy of Sciences, Wenzhou 325001, China
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Wenzhou 325001, China
| |
Collapse
|
10
|
Liu X, Yang X, Li X. HRUNet: Assessing Uncertainty in Heart Rates Measured From Facial Videos. IEEE J Biomed Health Inform 2024; 28:2955-2966. [PMID: 38345952 DOI: 10.1109/jbhi.2024.3363006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/07/2024]
Abstract
Video-based Photoplethysmography (VPPG) offers the capability to measure heart rate (HR) from facial videos. However, the reliability of the HR values extracted through this method remains uncertain, especially when videos are affected by various disturbances. Confronted by this challenge, we introduce an innovative framework for VPPG-based HR measurements, with a focus on capturing diverse sources of uncertainty in the predicted HR values. In this context, a neural network named HRUNet is structured for HR extraction from input facial videos. Departing from the conventional training approach of learning specific weight (and bias) values, we leverage the Bayesian posterior estimation to derive weight distributions within HRUNet. These distributions allow for sampling to encode uncertainty stemming from HRUNet's limited performance. On this basis, we redefine HRUNet's output as a distribution of potential HR values, as opposed to the traditional emphasis on the single most probable HR value. The underlying goal is to discover the uncertainty arising from inherent noise in the input video. HRUNet is evaluated across 1,098 videos from seven datasets, spanning three scenarios: undisturbed, motion-disturbed, and light-disturbed. The ensuing test outcomes demonstrate that uncertainty in the HR measurements increases significantly in the scenarios marked by disturbances, compared to that in the undisturbed scenario. Moreover, HRUNet outperforms state-of-the-art methods in HR accuracy when excluding HR values with 0.4 uncertainty. This underscores that uncertainty emerges as an informative indicator of potentially erroneous HR measurements. With enhanced reliability affirmed, the VPPG technique holds the promise for applications in safety-critical domains.
Collapse
|
11
|
Fan X, Liu F, Zhang J, Gao T, Fan Z, Huang Z, Xue W, Zhang J. Remote photoplethysmography based on reflected light angle estimation. Physiol Meas 2024; 45:035005. [PMID: 38430568 DOI: 10.1088/1361-6579/ad2f5d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 03/01/2024] [Indexed: 03/04/2024]
Abstract
Objective. In previous studies, the factors affecting the accuracy of imaging photoplethysmography (iPPG) heart rate (HR) measurement have been focused on the light intensity, facial reflection angle, and motion artifacts. However, the factor of specularly reflected light has not been studied in detail. We explored the effect of specularly reflected light on the accuracy of HR estimation and proposed an estimation method for the direction of specularly radiated light.Approach. To study the HR measurement accuracy influenced by specularly reflected light, we control the component of specularly reflected light by controlling its angle. A total of 100 videos from four different reflected light angles were collected, and 25 subjects participated in the dataset collection. We extracted angles and illuminations for 71 facial regions, fitting sample points through interpolation, and selecting the angle corresponding to the maximum weight in the fitted curve as the estimated reflected angle.Main results. The experimental results show that higher specularly reflected light compromises HR estimation accuracy under the same value of light intensity. Notably, at a 60° angle, the HR accuracy (ACC) increased by 0.7%, while the signal-to-noise ratio and Pearson correlation coefficient increased by 0.8 dB and 0.035, respectively, compared to 0°. The overall root mean squared error, standard deviation, and mean error of our proposed reflected light angle estimation method on the illumination multi-angle incidence (IMAI) dataset are 1.173°, 0.978°, and 0.773°. The average Pearson value is 0.8 in the PURE rotation dataset. In addition, the average ACC of HR measurements in the PURE dataset is improved by 1.73% in our method compared to the state-of-the-art traditional methods.Significance. Our method has great potential for clinical applications, especially in bright light environments such as during surgery, to improve accuracy and monitor blood volume changes in blood vessels.
Collapse
Affiliation(s)
- Xuanhe Fan
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Fangwu Liu
- Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai, People's Republic of China
| | - Jinjin Zhang
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Tong Gao
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Ziyang Fan
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Zhijie Huang
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Wei Xue
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - JingJing Zhang
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| |
Collapse
|
12
|
Trirongjitmoah S, Promking A, Kaewdang K, Phansiri N, Treeprapin K. Assessing heart rate and blood pressure estimation from image photoplethysmography using a digital blood pressure meter. Heliyon 2024; 10:e27113. [PMID: 38439889 PMCID: PMC10909774 DOI: 10.1016/j.heliyon.2024.e27113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 02/20/2024] [Accepted: 02/23/2024] [Indexed: 03/06/2024] Open
Abstract
This study presents a non-contact approach to measuring heart rate and blood pressure using an image photoplethysmography (iPPG) signal, and compares the results to those from an oscillometric blood pressure meter. Facial videos of 100 subjects were recorded via a webcam under ambient lighting conditions to extract iPPG signals. The results revealed a strong correlation between the heart rate derived from iPPG and that obtained from an oscillometric blood pressure meter. In addition, a continuous wavelet transform images with a 6-s duration were used as input for a custom convolutional neural network model, providing the most accurate blood pressure estimation. The proposed method received a grade A for diastolic and grade B for systolic blood pressure based on the British Hypertension Society's criteria. It also met the standards set by the Association for the Advancement of Medical Instrumentation. This non-contact framework shows promising potential for efficient screening purposes.
Collapse
Affiliation(s)
- Suchin Trirongjitmoah
- Department of Electrical and Electronics Engineering, Faculty of Engineering, Ubon Ratchathani University, 85 Sathonlamark, Warinchamrab, Ubon Ratchathani 34190, Thailand
| | - Arphorn Promking
- Department of Electrical and Electronics Engineering, Faculty of Engineering, Ubon Ratchathani University, 85 Sathonlamark, Warinchamrab, Ubon Ratchathani 34190, Thailand
| | - Khanittha Kaewdang
- Department of Electrical and Electronics Engineering, Faculty of Engineering, Ubon Ratchathani University, 85 Sathonlamark, Warinchamrab, Ubon Ratchathani 34190, Thailand
| | - Nisarut Phansiri
- Department of Electrical and Electronics Engineering, Faculty of Engineering, Ubon Ratchathani University, 85 Sathonlamark, Warinchamrab, Ubon Ratchathani 34190, Thailand
| | - Kriengsak Treeprapin
- Department of Mathematics, Statistics and Computers, Faculty of Science, Ubon Ratchathani University, 85 Sathonlamark, Warinchamrab, Ubon Ratchathani 34190, Thailand
| |
Collapse
|
13
|
Mather JD, Hayes LD, Mair JL, Sculthorpe NF. Validity of resting heart rate derived from contact-based smartphone photoplethysmography compared with electrocardiography: a scoping review and checklist for optimal acquisition and reporting. Front Digit Health 2024; 6:1326511. [PMID: 38486919 PMCID: PMC10937558 DOI: 10.3389/fdgth.2024.1326511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 02/08/2024] [Indexed: 03/17/2024] Open
Abstract
Background With the rise of smartphone ownership and increasing evidence to support the suitability of smartphone usage in healthcare, the light source and smartphone camera could be utilized to perform photoplethysmography (PPG) for the assessment of vital signs, such as heart rate (HR). However, until rigorous validity assessment has been conducted, PPG will have limited use in clinical settings. Objective We aimed to conduct a scoping review assessing the validity of resting heart rate (RHR) acquisition from PPG utilizing contact-based smartphone devices. Our four specific objectives of this scoping review were to (1) conduct a systematic search of the published literature concerning contact-based smartphone device-derived PPG, (2) map study characteristics and methodologies, (3) identify if methodological and technological advancements have been made, and (4) provide recommendations for the advancement of the investigative area. Methods ScienceDirect, PubMed and SPORTDiscus were searched for relevant studies between January 1st, 2007, and November 6th, 2022. Filters were applied to ensure only literature written in English were included. Reference lists of included studies were manually searched for additional eligible studies. Results In total 10 articles were included. Articles varied in terms of methodology including study characteristics, index measurement characteristics, criterion measurement characteristics, and experimental procedure. Additionally, there were variations in reporting details including primary outcome measure and measure of validity. However, all studies reached the same conclusion, with agreement ranging between good to very strong and correlations ranging from r = .98 to 1. Conclusions Smartphone applications measuring RHR derived from contact-based smartphone PPG appear to agree with gold standard electrocardiography (ECG) in healthy subjects. However, agreement was established under highly controlled conditions. Future research could investigate their validity and consider effective approaches that transfer these methods from laboratory conditions into the "real-world", in both healthy and clinical populations.
Collapse
Affiliation(s)
- James D. Mather
- Sport and Physical Activity Research Institute, School of Health and Life Sciences, University of the West of Scotland, Glasgow, United Kingdom
| | - Lawrence D. Hayes
- Sport and Physical Activity Research Institute, School of Health and Life Sciences, University of the West of Scotland, Glasgow, United Kingdom
| | - Jacqueline L. Mair
- Future Health Technologies, Singapore-ETH Centre, Campus for Research Excellence and Technological Enterprise (CREATE), Singapore, Singapore
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
| | - Nicholas F. Sculthorpe
- Sport and Physical Activity Research Institute, School of Health and Life Sciences, University of the West of Scotland, Glasgow, United Kingdom
| |
Collapse
|
14
|
Deng Y, Hu T, Chen J, Zeng J, Yang J, Ke Q, Miao L, Chen Y, Li R, Zhang R, Xu P. Non-invasive methods for heart rate measurement in fish based on photoplethysmography. J Exp Biol 2024; 227:jeb246464. [PMID: 38284767 DOI: 10.1242/jeb.246464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 01/22/2024] [Indexed: 01/30/2024]
Abstract
Heart rate is a crucial physiological indicator for fish, but current measurement methods are often invasive or require delicate manipulation. In this study, we introduced two non-invasive and easy-to-operate methods based on photoplethysmography, namely reflectance-type photoplethysmography (PPG) and remote photoplethysmography (rPPG), which we applied to the large yellow croaker (Larimichthys crocea). PPG showed perfect synchronization with electrocardiogram (ECG), with a Pearson's correlation coefficient of 0.99999. For rPPG, the results showed good agreement with ECG. Under active provision of green light, the Pearson's correlation coefficient was 0.966, surpassing the value of 0.947 under natural light. Additionally, the root mean square error was 0.810, which was lower than the value of 1.30 under natural light, indicating not only that the rPPG method had relatively high accuracy but also that green light may have the potential to further improve its accuracy.
Collapse
Affiliation(s)
- Yacheng Deng
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| | - Tianyu Hu
- School of Electronic Science and Engineering (National Model Microelectronics College), Xiamen University, Xiamen 361005, China
- Innovation Laboratory for Sciences and Technologies of Energy Materials of Fujian Province (IKKEM), Xiamen 361005, China
| | - Jia Chen
- School of Electronic Science and Engineering (National Model Microelectronics College), Xiamen University, Xiamen 361005, China
- Innovation Laboratory for Sciences and Technologies of Energy Materials of Fujian Province (IKKEM), Xiamen 361005, China
| | - Junjia Zeng
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| | - Jinqian Yang
- School of Electronic Science and Engineering (National Model Microelectronics College), Xiamen University, Xiamen 361005, China
- Innovation Laboratory for Sciences and Technologies of Energy Materials of Fujian Province (IKKEM), Xiamen 361005, China
| | - Qiaozhen Ke
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| | - Lingwei Miao
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| | - Yujia Chen
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| | - Rui Li
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| | - Rongxin Zhang
- School of Electronic Science and Engineering (National Model Microelectronics College), Xiamen University, Xiamen 361005, China
- Innovation Laboratory for Sciences and Technologies of Energy Materials of Fujian Province (IKKEM), Xiamen 361005, China
| | - Peng Xu
- State Key Laboratory of Mariculture Breeding, Xiamen University, Xiamen 361102, China
- Fujian Key Laboratory of Genetics and Breeding of Marine Organisms, College of Ocean and Earth Sciences, Xiamen University, Xiamen 361102, China
| |
Collapse
|
15
|
Helwan A, Azar D, Ma'aitah MKS. Conventional and deep learning methods in heart rate estimation from RGB face videos. Physiol Meas 2024; 45:02TR01. [PMID: 38081130 DOI: 10.1088/1361-6579/ad1458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 12/11/2023] [Indexed: 02/10/2024]
Abstract
Contactless vital signs monitoring is a fast-advancing scientific field that aims to employ monitoring methods that do not necessitate the use of leads or physical attachments to the patient in order to overcome the shortcomings and limits of traditional monitoring systems. Several traditional methods have been applied to extract the heart rate (HR) signal from the face. Moreover, machine learning has recently contributed majorly to the development of such a field in which deep networks and other deep learning methods are employed to extract the HR signal from RGB face videos. In this paper, we evaluate the state-of-the-art conventional and deep learning methods for HR estimates, focusing on the limits of deep learning methods and the availability of less-controlled face video datasets. We aim to present an extensive review that helps the various approaches of remote photoplethysmography extraction and HR estimation to be understood, in addition to their drawbacks and benefits.
Collapse
Affiliation(s)
| | | | - Mohamad Khaleel Sallam Ma'aitah
- Department of Robotics and Artificial Intelligence Engineering, Faculty of Engineering & Technology, Applied Science Private University, Amman, Jordan
| |
Collapse
|
16
|
Yang Z, Wang H, Liu B, Lu F. cbPPGGAN: A Generic Enhancement Framework for Unpaired Pulse Waveforms in Camera-Based Photoplethysmography. IEEE J Biomed Health Inform 2024; 28:598-608. [PMID: 37695961 DOI: 10.1109/jbhi.2023.3314282] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/13/2023]
Abstract
Camera-based photoplethysmography (cbP PG) is a non-contact technique that measures cardiac-related blood volume alterations in skin surface vessels through the analysis of facial videos. While traditional approaches can estimate heart rate (HR) under different illuminations, their accuracy can be affected by motion artifacts, leading to poor waveform fidelity and hindering further analysis of heart rate variability (HRV); deep learning-based approaches reconstruct high-quality pulse waveform, yet their performance significantly degrades under illumination variations. In this work, we aim to leverage the strength of these two methods and propose a framework that possesses favorable generalization capabilities while maintaining waveform fidelity. For this purpose, we propose the cbPPGGAN, an enhancement framework for cbPPG that enables the flexible incorporation of both unpaired and paired data sources in the training process. Based on the waveforms extracted by traditional approaches, the cbPPGGAN reconstructs high-quality waveforms that enable accurate HR estimation and HRV analysis. In addition, to address the lack of paired training data problems in real-world applications, we propose a cycle consistency loss that guarantees the time-frequency consistency before/after mapping. The method enhances the waveform quality of traditional POS approaches in different illumination tests (BH-rPPG) and cross-datasets (UBFC-rPPG) with mean absolute error (MAE) values of 1.34 bpm and 1.65 bpm, and average beat-to-beat (AVBB) values of 27.46 ms and 45.28 ms, respectively. Experimental results demonstrate that the cbPPGGAN enhances cbPPG signal quality and outperforms the state-of-the-art approaches in HR estimation and HRV analysis. The proposed framework opens a new pathway toward accurate HR estimation in an unconstrained environment.
Collapse
|
17
|
Othman W, Kashevnik A, Ali A, Shilov N, Ryumin D. Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild. SENSORS (BASEL, SWITZERLAND) 2024; 24:775. [PMID: 38339492 PMCID: PMC10857270 DOI: 10.3390/s24030775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Revised: 01/20/2024] [Accepted: 01/23/2024] [Indexed: 02/12/2024]
Abstract
Heart rate is an essential vital sign to evaluate human health. Remote heart monitoring using cheaply available devices has become a necessity in the twenty-first century to prevent any unfortunate situation caused by the hectic pace of life. In this paper, we propose a new method based on the transformer architecture with a multi-skip connection biLSTM decoder to estimate heart rate remotely from videos. Our method is based on the skin color variation caused by the change in blood volume in its surface. The presented heart rate estimation framework consists of three main steps: (1) the segmentation of the facial region of interest (ROI) based on the landmarks obtained by 3DDFA; (2) the extraction of the spatial and global features; and (3) the estimation of the heart rate value from the obtained features based on the proposed method. This paper investigates which feature extractor performs better by captioning the change in skin color related to the heart rate as well as the optimal number of frames needed to achieve better accuracy. Experiments were conducted using two publicly available datasets (LGI-PPGI and Vision for Vitals) and our own in-the-wild dataset (12 videos collected by four drivers). The experiments showed that our approach achieved better results than the previously published methods, making it the new state of the art on these datasets.
Collapse
Affiliation(s)
- Walaa Othman
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia; (W.O.); (N.S.); (D.R.)
| | - Alexey Kashevnik
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia; (W.O.); (N.S.); (D.R.)
| | - Ammar Ali
- Information Technology and Programming Faculty, ITMO University, 191002 St. Petersburg, Russia;
| | - Nikolay Shilov
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia; (W.O.); (N.S.); (D.R.)
| | - Dmitry Ryumin
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia; (W.O.); (N.S.); (D.R.)
| |
Collapse
|
18
|
Wang D, Chahl J. Simulating cardiac signals on 3D human models for photoplethysmography development. Front Robot AI 2024; 10:1266535. [PMID: 38269072 PMCID: PMC10806157 DOI: 10.3389/frobt.2023.1266535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 12/21/2023] [Indexed: 01/26/2024] Open
Abstract
Introduction: Image-based heart rate estimation technology offers a contactless approach to healthcare monitoring that could improve the lives of millions of people. In order to comprehensively test or optimize image-based heart rate extraction methods, the dataset should contain a large number of factors such as body motion, lighting conditions, and physiological states. However, collecting high-quality datasets with complete parameters is a huge challenge. Methods: In this paper, we introduce a bionic human model based on a three-dimensional (3D) representation of the human body. By integrating synthetic cardiac signal and body involuntary motion into the 3D model, five well-known traditional and four deep learning iPPG (imaging photoplethysmography) extraction methods are used to test the rendered videos. Results: To compare with different situations in the real world, four common scenarios (stillness, expression/talking, light source changes, and physical activity) are created on each 3D human. The 3D human can be built with any appearance and different skin tones. A high degree of agreement is achieved between the signals extracted from videos with the synthetic human and videos with a real human-the performance advantages and disadvantages of the selected iPPG methods are consistent for both real and 3D humans. Discussion: This technology has the capability to generate synthetic humans within various scenarios, utilizing precisely controlled parameters and disturbances. Furthermore, it holds considerable potential for testing and optimizing image-based vital signs methods in challenging situations where real people with reliable ground truth measurements are difficult to obtain, such as in drone rescue.
Collapse
Affiliation(s)
- Danyi Wang
- UniSA STEM, University of South Australia, Mawson Lakes, SA, Australia
| | - Javaan Chahl
- UniSA STEM, University of South Australia, Mawson Lakes, SA, Australia
- Platforms Division, Defence Science and Technology Group, Edinburgh, SA, Australia
| |
Collapse
|
19
|
Ontiveros RC, Elgendi M, Missale G, Menon C. Evaluating RGB channels in remote photoplethysmography: a comparative study with contact-based PPG. Front Physiol 2023; 14:1296277. [PMID: 38187134 PMCID: PMC10770840 DOI: 10.3389/fphys.2023.1296277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 11/28/2023] [Indexed: 01/09/2024] Open
Abstract
Remote photoplethysmography (rPPG) provides a non-contact method for measuring blood volume changes. In this study, we compared rPPG signals obtained from video cameras with traditional contact-based photoplethysmography (cPPG) to assess the effectiveness of different RGB channels in cardiac signal extraction. Our objective was to determine the most effective RGB channel for detecting blood volume changes and estimating heart rate. We employed dynamic time warping, Pearson's correlation coefficient, root-mean-square error, and Beats-per-minute Difference to evaluate the performance of each RGB channel relative to cPPG. The results revealed that the green channel was superior, outperforming the blue and red channels in detecting volumetric changes and accurately estimating heart rate across various activities. We also observed that the reliability of RGB signals varied based on recording conditions and subject activity. This finding underscores the importance of understanding the performance nuances of RGB inputs, crucial for constructing rPPG signals in algorithms. Our study is significant in advancing rPPG research, offering insights that could benefit clinical applications by improving non-contact methods for blood volume assessment.
Collapse
Affiliation(s)
- Rodrigo Castellano Ontiveros
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
- School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
| | - Giuseppe Missale
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
- Electronics and Telecommunications Department, Politecnico Di Torino, Torino, Italy
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
| |
Collapse
|
20
|
Lee S, Lee M, Sim JY. DSE-NN: Deeply Supervised Efficient Neural Network for Real-Time Remote Photoplethysmography. Bioengineering (Basel) 2023; 10:1428. [PMID: 38136019 PMCID: PMC10740871 DOI: 10.3390/bioengineering10121428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 12/11/2023] [Accepted: 12/12/2023] [Indexed: 12/24/2023] Open
Abstract
Non-contact remote photoplethysmography can be used in a variety of medical and healthcare fields by measuring vital signs continuously and unobtrusively. Recently, end-to-end deep learning methods have been proposed to replace the existing handcrafted features. However, since the existing deep learning methods are known as black box models, the problem of interpretability has been raised, and the same problem exists in the remote photoplethysmography (rPPG) network. In this study, we propose a method to visualize temporal and spectral representations for hidden layers, deeply supervise the spectral representation of intermediate layers through the depth of networks and optimize it for a lightweight model. The optimized network improves performance and enables fast training and inference times. The proposed spectral deep supervision helps to achieve not only high performance but also fast convergence speed through the regularization of the intermediate layers. The effect of the proposed methods was confirmed through a thorough ablation study on public datasets. As a result, similar or outperforming results were obtained in comparison to state-of-the-art models. In particular, our model achieved an RMSE of 1 bpm on the PURE dataset, demonstrating its high accuracy. Moreover, it excelled on the V4V dataset with an impressive RMSE of 6.65 bpm, outperforming other methods. We observe that our model began converging from the very first epoch, a significant improvement over other models in terms of learning efficiency. Our approach is expected to be generally applicable to models that learn spectral domain information as well as to the applications of regression that require the representations of periodicity.
Collapse
Affiliation(s)
| | | | - Joo Yong Sim
- Department of Mechanical Systems Engineering, Sookmyung Women’s University, Seoul 04310, Republic of Korea; (S.L.); (M.L.)
| |
Collapse
|
21
|
Huang B, Hu S, Liu Z, Lin CL, Su J, Zhao C, Wang L, Wang W. Challenges and prospects of visual contactless physiological monitoring in clinical study. NPJ Digit Med 2023; 6:231. [PMID: 38097771 PMCID: PMC10721846 DOI: 10.1038/s41746-023-00973-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 11/21/2023] [Indexed: 12/17/2023] Open
Abstract
The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.
Collapse
Affiliation(s)
- Bin Huang
- AI Research Center, Hangzhou Innovation Institute, Beihang University, 99 Juhang Rd., Binjiang Dist., Hangzhou, Zhejiang, China.
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China.
| | - Shen Hu
- Department of Obstetrics, The Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
- Department of Epidemiology, The Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Zimeng Liu
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
| | - Chun-Liang Lin
- College of Electrical Engineering and Computer Science, National Chung Hsing University, 145 Xingda Rd., South Dist., Taichung, Taiwan.
| | - Junfeng Su
- Department of General Intensive Care Unit, The Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
- Key Laboratory of Early Warning and Intervention of Multiple Organ Failure, China National Ministry of Education, Hangzhou, Zhejiang, China
| | - Changchen Zhao
- AI Research Center, Hangzhou Innovation Institute, Beihang University, 99 Juhang Rd., Binjiang Dist., Hangzhou, Zhejiang, China
| | - Li Wang
- Department of Rehabilitation Medicine, The First Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
| | - Wenjin Wang
- Department of Biomedical Engineering, Southern University of Science and Technology, 1088 Xueyuan Ave, Nanshan Dist., Shenzhen, Guangdong, China.
| |
Collapse
|
22
|
Arrow C, Ward M, Eshraghian J, Dwivedi G. Capturing the pulse: a state-of-the-art review on camera-based jugular vein assessment. BIOMEDICAL OPTICS EXPRESS 2023; 14:6470-6492. [PMID: 38420308 PMCID: PMC10898581 DOI: 10.1364/boe.507418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 11/02/2023] [Accepted: 11/05/2023] [Indexed: 03/02/2024]
Abstract
Heart failure is associated with a rehospitalisation rate of up to 50% within six months. Elevated central venous pressure may serve as an early warning sign. While invasive procedures are used to measure central venous pressure for guiding treatment in hospital, this becomes impractical upon discharge. A non-invasive estimation technique exists, where the clinician visually inspects the pulsation of the jugular veins in the neck, but it is less reliable due to human limitations. Video and signal processing technologies may offer a high-fidelity alternative. This state-of-the-art review analyses existing literature on camera-based methods for jugular vein assessment. We summarize key design considerations and suggest avenues for future research. Our review highlights the neck as a rich imaging target beyond the jugular veins, capturing comprehensive cardiac signals, and outlines factors affecting signal quality and measurement accuracy. Addressing an often quoted limitation in the field, we also propose minimum reporting standards for future studies.
Collapse
Affiliation(s)
- Coen Arrow
- School of Medicine, University of Western Australia, Perth, Australia
- Advanced Clinical and Translational Cardiovascular Imaging, Harry Perkins Institute of Medical Research, University of Western Australia, Perth, Australia
| | - Max Ward
- Department of Computer Science and Software Engineering, University of Western Australia, Perth, Australia
| | - Jason Eshraghian
- Department of Electrical and Computer Engineering, University of California (Santa Cruz), California, USA
| | - Girish Dwivedi
- School of Medicine, University of Western Australia, Perth, Australia
- Advanced Clinical and Translational Cardiovascular Imaging, Harry Perkins Institute of Medical Research, University of Western Australia, Perth, Australia
- Department of Cardiology, Fiona Stanley Hospital, Perth, Australia
| |
Collapse
|
23
|
Braun B, McDuff D, Baltrusaitis T, Holz C. Video-based sympathetic arousal assessment via peripheral blood flow estimation. BIOMEDICAL OPTICS EXPRESS 2023; 14:6607-6628. [PMID: 38420320 PMCID: PMC10898569 DOI: 10.1364/boe.507949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 10/27/2023] [Accepted: 10/27/2023] [Indexed: 03/02/2024]
Abstract
Electrodermal activity (EDA) is considered a standard marker of sympathetic activity. However, traditional EDA measurement requires electrodes in steady contact with the skin. Can sympathetic arousal be measured using only an optical sensor, such as an RGB camera? This paper presents a novel approach to infer sympathetic arousal by measuring the peripheral blood flow on the face or hand optically. We contribute a self-recorded dataset of 21 participants, comprising synchronized videos of participants' faces and palms and gold-standard EDA and photoplethysmography (PPG) signals. Our results show that we can measure peripheral sympathetic responses that closely correlate with the ground truth EDA. We obtain median correlations of 0.57 to 0.63 between our inferred signals and the ground truth EDA using only videos of the participants' palms or foreheads or PPG signals from the foreheads or fingers. We also show that sympathetic arousal is best inferred from the forehead, finger, or palm.
Collapse
Affiliation(s)
- Björn Braun
- Department of Computer Science, ETH Zürich, Switzerland
| | | | | | | |
Collapse
|
24
|
Charlton PH, Allen J, Bailón R, Baker S, Behar JA, Chen F, Clifford GD, Clifton DA, Davies HJ, Ding C, Ding X, Dunn J, Elgendi M, Ferdoushi M, Franklin D, Gil E, Hassan MF, Hernesniemi J, Hu X, Ji N, Khan Y, Kontaxis S, Korhonen I, Kyriacou PA, Laguna P, Lázaro J, Lee C, Levy J, Li Y, Liu C, Liu J, Lu L, Mandic DP, Marozas V, Mejía-Mejía E, Mukkamala R, Nitzan M, Pereira T, Poon CCY, Ramella-Roman JC, Saarinen H, Shandhi MMH, Shin H, Stansby G, Tamura T, Vehkaoja A, Wang WK, Zhang YT, Zhao N, Zheng D, Zhu T. The 2023 wearable photoplethysmography roadmap. Physiol Meas 2023; 44:111001. [PMID: 37494945 PMCID: PMC10686289 DOI: 10.1088/1361-6579/acead2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 04/04/2023] [Accepted: 07/26/2023] [Indexed: 07/28/2023]
Abstract
Photoplethysmography is a key sensing technology which is used in wearable devices such as smartwatches and fitness trackers. Currently, photoplethysmography sensors are used to monitor physiological parameters including heart rate and heart rhythm, and to track activities like sleep and exercise. Yet, wearable photoplethysmography has potential to provide much more information on health and wellbeing, which could inform clinical decision making. This Roadmap outlines directions for research and development to realise the full potential of wearable photoplethysmography. Experts discuss key topics within the areas of sensor design, signal processing, clinical applications, and research directions. Their perspectives provide valuable guidance to researchers developing wearable photoplethysmography technology.
Collapse
Affiliation(s)
- Peter H Charlton
- Department of Public Health and Primary Care, University of Cambridge, Cambridge, CB1 8RN, United Kingdom
- Research Centre for Biomedical Engineering, City, University of London, London, EC1V 0HB, United Kingdom
| | - John Allen
- Research Centre for Intelligent Healthcare, Coventry University, Coventry, CV1 5RW, United Kingdom
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, NE2 4HH, United Kingdom
| | - Raquel Bailón
- Biomedical Signal Interpretation and Computational Simulation (BSICoS) Group, Aragon Institute of Engineering Research (I3A), IIS Aragon, University of Zaragoza, E-50018 Zaragoza, Spain
- CIBER-BBN, Instituto de Salud Carlos III, C/Monforte de Lemos 3-5, E-28029 Madrid, Spain
| | - Stephanie Baker
- College of Science and Engineering, James Cook University, Cairns, 4878 Queensland, Australia
| | - Joachim A Behar
- Faculty of Biomedical Engineering, Technion Israel Institute of Technology, Haifa, 3200003, Israel
| | - Fei Chen
- Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, 518055 Guandong, People’s Republic of China
| | - Gari D Clifford
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, United States of America
- Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, United States of America
| | - David A Clifton
- Department of Engineering Science, University of Oxford, Oxford, OX3 7DQ, United Kingdom
| | - Harry J Davies
- Department of Electrical and Electronic Engineering, Imperial College London, London, SW7 2AZ, United Kingdom
| | - Cheng Ding
- Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, United States of America
- Department of Biomedical Engineering, Emory University, Atlanta, GA 30322, United States of America
| | - Xiaorong Ding
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 611731, People’s Republic of China
| | - Jessilyn Dunn
- Department of Biomedical Engineering, Duke University, Durham, NC 27708-0187, United States of America
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27708-0187, United States of America
- Duke Clinical Research Institute, Durham, NC 27705-3976, United States of America
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, 8008, Switzerland
| | - Munia Ferdoushi
- Department of Electrical and Computer Engineering, University of Southern California, 90089, Los Angeles, California, United States of America
- The Institute for Technology and Medical Systems (ITEMS), Keck School of Medicine, University of Southern California, Los Angeles, CA 90033, United States of America
| | - Daniel Franklin
- Institute of Biomedical Engineering, Translational Biology & Engineering Program, Ted Rogers Centre for Heart Research, University of Toronto, Toronto, M5G 1M1, Canada
| | - Eduardo Gil
- Biomedical Signal Interpretation and Computational Simulation (BSICoS) Group, Aragon Institute of Engineering Research (I3A), IIS Aragon, University of Zaragoza, E-50018 Zaragoza, Spain
- CIBER-BBN, Instituto de Salud Carlos III, C/Monforte de Lemos 3-5, E-28029 Madrid, Spain
| | - Md Farhad Hassan
- Department of Electrical and Computer Engineering, University of Southern California, 90089, Los Angeles, California, United States of America
- The Institute for Technology and Medical Systems (ITEMS), Keck School of Medicine, University of Southern California, Los Angeles, CA 90033, United States of America
| | - Jussi Hernesniemi
- Finnish Cardiovascular Research Center Tampere, Faculty of Medicine and Health Technology, Tampere University, Tampere, 33720, Finland
- Tampere Heart Hospital, Wellbeing Services County of Pirkanmaa, Tampere, 33520, Finland
| | - Xiao Hu
- Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, 30322, Georgia, United States of America
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, 30322, Georgia, United States of America
- Department of Computer Sciences, College of Arts and Sciences, Emory University, Atlanta, GA 30322, United States of America
| | - Nan Ji
- Hong Kong Center for Cerebrocardiovascular Health Engineering (COCHE), Hong Kong Science and Technology Park, Hong Kong, 999077, People’s Republic of China
| | - Yasser Khan
- Department of Electrical and Computer Engineering, University of Southern California, 90089, Los Angeles, California, United States of America
- The Institute for Technology and Medical Systems (ITEMS), Keck School of Medicine, University of Southern California, Los Angeles, CA 90033, United States of America
| | - Spyridon Kontaxis
- Biomedical Signal Interpretation and Computational Simulation (BSICoS) Group, Aragon Institute of Engineering Research (I3A), IIS Aragon, University of Zaragoza, E-50018 Zaragoza, Spain
- CIBER-BBN, Instituto de Salud Carlos III, C/Monforte de Lemos 3-5, E-28029 Madrid, Spain
| | - Ilkka Korhonen
- Finnish Cardiovascular Research Center Tampere, Faculty of Medicine and Health Technology, Tampere University, Tampere, 33720, Finland
| | - Panicos A Kyriacou
- Research Centre for Biomedical Engineering, City, University of London, London, EC1V 0HB, United Kingdom
| | - Pablo Laguna
- Biomedical Signal Interpretation and Computational Simulation (BSICoS) Group, Aragon Institute of Engineering Research (I3A), IIS Aragon, University of Zaragoza, E-50018 Zaragoza, Spain
- CIBER-BBN, Instituto de Salud Carlos III, C/Monforte de Lemos 3-5, E-28029 Madrid, Spain
| | - Jesús Lázaro
- Biomedical Signal Interpretation and Computational Simulation (BSICoS) Group, Aragon Institute of Engineering Research (I3A), IIS Aragon, University of Zaragoza, E-50018 Zaragoza, Spain
- CIBER-BBN, Instituto de Salud Carlos III, C/Monforte de Lemos 3-5, E-28029 Madrid, Spain
| | - Chungkeun Lee
- Digital Health Devices Division, Medical Device Evaluation Department, National Institute of Food and Drug Safety Evaluation, Ministry of Food and Drug Safety, Cheongju, 28159, Republic of Korea
| | - Jeremy Levy
- Faculty of Biomedical Engineering, Technion Israel Institute of Technology, Haifa, 3200003, Israel
- Faculty of Electrical and Computer Engineering, Technion Institute of Technology, Haifa, 3200003, Israel
| | - Yumin Li
- State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, People’s Republic of China
| | - Chengyu Liu
- State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, People’s Republic of China
| | - Jing Liu
- Analog Devices Inc, San Jose, CA 95124, United States of America
| | - Lei Lu
- Department of Engineering Science, University of Oxford, Oxford, OX3 7DQ, United Kingdom
| | - Danilo P Mandic
- Department of Electrical and Electronic Engineering, Imperial College London, London, SW7 2AZ, United Kingdom
| | - Vaidotas Marozas
- Department of Electronics Engineering, Kaunas University of Technology, 44249 Kaunas, Lithuania
- Biomedical Engineering Institute, Kaunas University of Technology, 44249 Kaunas, Lithuania
| | - Elisa Mejía-Mejía
- Research Centre for Biomedical Engineering, City, University of London, London, EC1V 0HB, United Kingdom
| | - Ramakrishna Mukkamala
- Department of Bioengineering and Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Meir Nitzan
- Department of Physics/Electro-Optic Engineering, Lev Academic Center, 91160 Jerusalem, Israel
| | - Tania Pereira
- INESC TEC—Institute for Systems and Computer Engineering, Technology and Science, Porto, 4200-465, Portugal
- Faculty of Engineering, University of Porto, Porto, 4200-465, Portugal
| | | | - Jessica C Ramella-Roman
- Department of Biomedical Engineering and Herbert Wertheim College of Medicine, Florida International University, Miami, FL 33174, United States of America
| | - Harri Saarinen
- Tampere Heart Hospital, Wellbeing Services County of Pirkanmaa, Tampere, 33520, Finland
| | - Md Mobashir Hasan Shandhi
- Department of Biomedical Engineering, Duke University, Durham, NC 27708-0187, United States of America
| | - Hangsik Shin
- Department of Digital Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, 05505, Republic of Korea
| | - Gerard Stansby
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, NE2 4HH, United Kingdom
- Northern Vascular Centre, Freeman Hospital, Newcastle upon Tyne, NE7 7DN, United Kingdom
| | - Toshiyo Tamura
- Future Robotics Organization, Waseda University, Tokyo, 1698050, Japan
| | - Antti Vehkaoja
- Finnish Cardiovascular Research Center Tampere, Faculty of Medicine and Health Technology, Tampere University, Tampere, 33720, Finland
- PulseOn Ltd, Espoo, 02150, Finland
| | - Will Ke Wang
- Department of Biomedical Engineering, Duke University, Durham, NC 27708-0187, United States of America
| | - Yuan-Ting Zhang
- Hong Kong Center for Cerebrocardiovascular Health Engineering (COCHE), Hong Kong Science and Technology Park, Hong Kong, 999077, People’s Republic of China
- Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, 999077, People’s Republic of China
| | - Ni Zhao
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong
| | - Dingchang Zheng
- Research Centre for Intelligent Healthcare, Coventry University, Coventry, CV1 5RW, United Kingdom
| | - Tingting Zhu
- Department of Engineering Science, University of Oxford, Oxford, OX3 7DQ, United Kingdom
| |
Collapse
|
25
|
Casado CA, Lopez MB. Face2PPG: An Unsupervised Pipeline for Blood Volume Pulse Extraction From Faces. IEEE J Biomed Health Inform 2023; 27:5530-5541. [PMID: 37610907 DOI: 10.1109/jbhi.2023.3307942] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/25/2023]
Abstract
Photoplethysmography (PPG) signals have become a key technology in many fields, such as medicine, well-being, or sports. Our work proposes a set of pipelines to extract remote PPG signals (rPPG) from the face robustly, reliably, and configurably. We identify and evaluate the possible choices in the critical steps of unsupervised rPPG methodologies. We assess a state-of-the-art processing pipeline in six different datasets, incorporating important corrections in the methodology that ensure reproducible and fair comparisons. In addition, we extend the pipeline by proposing three novel ideas; 1) a new method to stabilize the detected face based on a rigid mesh normalization; 2) a new method to dynamically select the different regions in the face that provide the best raw signals, and 3) a new RGB to rPPG transformation method, called Orthogonal Matrix Image Transformation (OMIT) based on QR decomposition, that increases robustness against compression artifacts. We show that all three changes introduce noticeable improvements in retrieving rPPG signals from faces, obtaining state-of-the-art results compared with unsupervised, non-learning-based methodologies and, in some databases, very close to supervised, learning-based methods. We perform a comparative study to quantify the contribution of each proposed idea. In addition, we depict a series of observations that could help in future implementations.
Collapse
|
26
|
Lin B, Tao J, Xu J, He L, Liu N, Zhang X. Estimation of vital signs from facial videos via video magnification and deep learning. iScience 2023; 26:107845. [PMID: 37790274 PMCID: PMC10542939 DOI: 10.1016/j.isci.2023.107845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 07/27/2023] [Accepted: 09/05/2023] [Indexed: 10/05/2023] Open
Abstract
The continuous monitoring of vital signs is one of the hottest topics in healthcare. Recent technological advances in sensors, signal processing, and image processing spawned the development of no-contact techniques such as remote photoplethysmography (rPPG). To solve the common problems of rPPG including weak extracted signals, body movements, and generalization with limited data resources, we proposed a dual-path estimation method based on video magnification and deep learning. First, image processes are applied to detect, track, and magnificate facial ROIs automatically. Then, the steady part of the wave of each processed ROI is used for the extraction of features including heart rate, PTT, and features of pulse wave waveform. The blood pressures are estimated from the features via a small CNN. Results comply with the current standard and promise potential clinical applications in the future.
Collapse
Affiliation(s)
- Bin Lin
- Key Laboratory of Opto-Electronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, Fujian 350117, China
| | - Jing Tao
- Key Laboratory of Opto-Electronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, Fujian 350117, China
| | - Jingjing Xu
- Key Laboratory of Opto-Electronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, Fujian 350117, China
| | - Liang He
- Key Laboratory of Opto-Electronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, Fujian 350117, China
| | - Nenrong Liu
- Fujian Provincial Key Laboratory of Quantum Manipulation and New Energy Materials, Fujian Provincial Collaborative Innovation Center for Advanced High-Field Superconducting Materials and Engineering, College of Physics and Energy, Fujian Normal University, Fuzhou, Fujian 350117, China
| | - Xianzeng Zhang
- Key Laboratory of Opto-Electronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, Fujian 350117, China
| |
Collapse
|
27
|
Zhang Q, Lin X, Zhang Y, Liu Q, Cai F. Non-contact high precision pulse-rate monitoring system for moving subjects in different motion states. Med Biol Eng Comput 2023; 61:2769-2783. [PMID: 37474842 DOI: 10.1007/s11517-023-02884-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 07/03/2023] [Indexed: 07/22/2023]
Abstract
Remote photoplethysmography (rPPG) enables contact-free monitoring of the pulse rate by using a color camera. The fundamental limitation is that motion artifacts and changes in ambient light conditions greatly affect the accuracy of pulse-rate monitoring. We propose use of a high-speed camera and a motion suppression algorithm with high computational efficiency. This system incorporates a number of major improvements including reproduction of pulse wave details, high-precision pulse-rate monitoring of moving subjects, and excellent scene scalability. A series of quantization methods were used to evaluate the effect of different frame rates and different algorithms in pulse-rate monitoring of moving subjects. The experimental results show that use of 180-fps video and a Plane-Orthogonal-to-Skin (POS) algorithm can produce high-precision pulse-rate monitoring results with mean absolute error can be less than 5 bpm and the relative accuracy reaching 94.5%. Thus, it has significant potential to improve personal health care and intelligent health monitoring.
Collapse
Affiliation(s)
- Qing Zhang
- School of Biomedical Engineering, Hainan University, Haikou, 570228, Hainan, China
| | - Xingsen Lin
- School of Biomedical Engineering, Hainan University, Haikou, 570228, Hainan, China
| | - Yuxin Zhang
- School of Biomedical Engineering, Hainan University, Haikou, 570228, Hainan, China
| | - Qian Liu
- School of Biomedical Engineering, Hainan University, Haikou, 570228, Hainan, China
| | - Fuhong Cai
- School of Biomedical Engineering, Hainan University, Haikou, 570228, Hainan, China.
| |
Collapse
|
28
|
Hino Y, Ashida K, Ogawa-Ochiai K, Tsumura N. Noise-Robust Pulse Wave Estimation from Near-Infrared Face Video Images Using the Wiener Estimation Method. J Imaging 2023; 9:202. [PMID: 37888309 PMCID: PMC10607892 DOI: 10.3390/jimaging9100202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 09/26/2023] [Accepted: 09/27/2023] [Indexed: 10/28/2023] Open
Abstract
In this paper, we propose a noise-robust pulse wave estimation method from near-infrared face video images. Pulse wave estimation in a near-infrared environment is expected to be applied to non-contact monitoring in dark areas. The conventional method cannot consider noise when performing estimation. As a result, the accuracy of pulse wave estimation in noisy environments is not very high. This may adversely affect the accuracy of heart rate data and other data obtained from pulse wave signals. Therefore, the objective of this study is to perform pulse wave estimation robust to noise. The Wiener estimation method, which is a simple linear computation that can consider noise, was used in this study. Experimental results showed that the combination of the proposed method and signal processing (detrending and bandpass filtering) increased the SNR (signal to noise ratio) by more than 2.5 dB compared to the conventional method and signal processing. The correlation coefficient between the pulse wave signal measured using a pulse wave meter and the estimated pulse wave signal was 0.30 larger on average for the proposed method. Furthermore, the AER (absolute error rate) between the heart rate measured with the pulse wave meter was 0.82% on average for the proposed method, which was lower than the value of the conventional method (12.53% on average). These results show that the proposed method is more robust to noise than the conventional method for pulse wave estimation.
Collapse
Affiliation(s)
- Yuta Hino
- Graduate School of Science and Engineering, Chiba University, Chiba 263-8522, Japan (N.T.)
| | - Koichi Ashida
- Graduate School of Science and Engineering, Chiba University, Chiba 263-8522, Japan (N.T.)
| | - Keiko Ogawa-Ochiai
- Kampo Clinical Center, Hiroshima University Hospital, Hiroshima 734-8511, Japan;
| | - Norimichi Tsumura
- Graduate School of Science and Engineering, Chiba University, Chiba 263-8522, Japan (N.T.)
- Kampo Clinical Center, Hiroshima University Hospital, Hiroshima 734-8511, Japan;
| |
Collapse
|
29
|
Sasaki S, Sugita N, Terai T, Yoshizawa M. Non-Contact Measurement of Blood Oxygen Saturation Using Facial Video Without Reference Values. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2023; 12:76-83. [PMID: 38088997 PMCID: PMC10712673 DOI: 10.1109/jtehm.2023.3318643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 09/05/2023] [Accepted: 09/20/2023] [Indexed: 12/18/2023]
Abstract
The continuous measurement of percutaneous oxygen saturation (SpO2) enables diseases that cause hypoxemia to be detected early and patients' conditions to be monitored. Currently, SpO2 is mainly measured using a pulse oximeter, which, owing to its simplicity, can be used in clinical settings and at home. However, the pulse oximeter requires a sensor to be in contact with the skin; therefore, prolonged use of the pulse oximeter for neonates or patients with sensitive skin may cause local inflammation or stress due to restricted movement. In addition, owing to COVID-19, there has been a growing demand for the contactless measurement of SpO2. Several studies on measuring SpO2 without contact used skin video images have been conducted. However, in these studies, the SpO2 values were estimated using a linear regression model or a look-up table that required reference values obtained using a contact-type pulse oximeter. In this study, we propose a new technique for the contactless measurement of SpO2 that does not require reference values. Specifically, we used certain approaches that reduced the influence of non-pulsating components and utilized different light wavelengths of video images that penetrated subcutaneously to different depths. We experimentally investigated the accuracy of SpO2 measurements using the proposed methods. The results indicate that the proposed methods were more accurate than the conventional method.
Collapse
Affiliation(s)
- Soma Sasaki
- Graduate School of EngineeringTohoku UniversitySendai9808579Japan
| | - Norihiro Sugita
- Graduate School of EngineeringTohoku UniversitySendai9808579Japan
- Cyberscience CenterTohoku UniversitySendai9808579Japan
| | - Takanori Terai
- Graduate School of EngineeringTohoku UniversitySendai9808579Japan
| | - Makoto Yoshizawa
- Center for Promotion of Innovation StrategyTohoku UniversitySendai9800845Japan
| |
Collapse
|
30
|
Mellouk W, Handouzi W. CNN-LSTM for automatic emotion recognition using contactless photoplythesmographic signals. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
|
31
|
Zheng K, Kong J, Tian L, Li B, Li H, Zhou J. Hand-over-face occlusion and distance adaptive heart rate detection based on imaging photoplethysmography and pixel distance in online learning. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104898] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
|
32
|
Li B, Zhang W, Fu H, Liu H, Xu F. Multi-Level Constrained Intra and Inter Subject Feature Representation for Facial Video Based BVP Signal Measurement. IEEE J Biomed Health Inform 2023; 27:3948-3957. [PMID: 37224379 DOI: 10.1109/jbhi.2023.3273557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Facial video-based blood volume pulse (BVP) signal measurement holds great potential for remote health monitoring, while existing methods have issues with convolutional kernel perceptual field constraints. This article proposes an end-to-end multi-level constrained spatiotemporal representation structure for facial video-based BVP signal measurement. First, an intra- and inter-subject feature representation is proposed to strengthen the BVP-related features generation at high, semantic, and shallow levels, respectively. Second, the global-local association is presented to enhance BVP signal period pattern learning, and the global temporal features are introduced into the local spatial convolution of each frame by adaptive kernel weights. Finally, the multi-dimensional fused features are mapped to one-dimensional BVP signals by the task-oriented signal estimator. The experimental results on the publicly available MMSE-HR dataset demonstrate that the proposed structure overperforms state-of-the-art methods (e.g., AutoHR) in BVP signal measurement, with a 20% and 40% reduction in mean absolute error and root mean squared error, respectively. The proposed structure would be a powerful tool for telemedical and non-contact heart health monitoring.
Collapse
|
33
|
Tang J, Chen K, Wang Y, Shi Y, Patel S, McDuff D, Liu X. MMPD: Multi-Domain Mobile Video Physiology Dataset. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-5. [PMID: 38083085 DOI: 10.1109/embc40787.2023.10340857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Remote photoplethysmography (rPPG) is an attractive method for noninvasive, convenient and concomitant measurement of physiological vital signals. Public benchmark datasets have served a valuable role in the development of this technology and improvements in accuracy over recent years. However, there remain gaps in the public datasets. First, despite the ubiquity of cameras on mobile devices, there are few datasets recorded specifically with mobile phone cameras. Second, most datasets are relatively small and therefore are limited in diversity, both in appearance (e.g., skin tone), behaviors (e.g., motion) and environment (e.g., lighting conditions). In an effort to help the field advance, we present the Multi-domain Mobile Video Physiology Dataset (MMPD), comprising 11 hours of recordings from mobile phones of 33 subjects. The dataset is designed to capture videos with greater representation across skin tone, body motion, and lighting conditions. MMPD is comprehensive with eight descriptive labels and can be used in conjunction with the rPPG-toolbox [1]. The reliability of the dataset is verified by mainstream unsupervised methods and neural methods. The GitHub repository of our dataset: https://github.com/THU-CS-PI/MMPD_rPPG_dataset.
Collapse
|
34
|
Tan C, Xiao C, Wang W. Camera-based Cardiovascular Screening based on Heart Rate and Its Variability In Pre- and Post-Exercise Conditions. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-5. [PMID: 38083672 DOI: 10.1109/embc40787.2023.10340871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Analysis of heart rate (HR) and heart rate variability (HRV) in pre- and post-exercise conditions can provide useful information about the health condition of cardiovascular system. Remote photoplethysmography (rPPG) that uses a contactless camera to measure vital signs from the human face might allow for ubiquitous applications in the fitness scenario. This paper benchmarks the accuracy of HR and HRV measured by ECG, PPG and rPPG, and investigates the difference of HR and HRV parameters between individuals with/without the exercise habit in pre- and post-exercise phases. We built a fitness benchmark consisting of 14 healthy subjects to perform running exercise on a treadmill, with video and reference data recorded simultaneously. The results show that rPPG has a similar performance as PPG in both the estimation of HR and HRV. The HRV parameters (Mean IBI, SDNN, LF, VLF, and SD2) of rPPG/PPG show good agreements with the ECG reference. Subjects with a regular exercise habit show lower resting HR and smaller values of HRV parameters in pre-exercise phase, fewer changes in HR and HRV at the same exercise intensity, and faster HR recovery after exercising, as compared with those without an exercise habit. The preliminary results suggest that rPPG is a promising surrogate of PPG for screening HR and HRV before and after exercise, to indicate the performance of cardiac training.
Collapse
|
35
|
Vatanparvar K, Li J, Gwak M, Zhu L, Kuang J, Gao A. Enhanced Contactless Heart Rate Monitoring Using Camera with Motion Artifact Removal During Physical Activities. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-5. [PMID: 38082654 DOI: 10.1109/embc40787.2023.10340279] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Contactless monitoring of heart rate (HR) can improve passive and continuous tracking of cardiovascular activities and overall people's health. Remote photoplethysmography (rPPG) using a camera eliminates the need for a wearable device. rPPG-based HR has shown promising results to be accurate and comparable to conventional methods such as contact PPG. Most experiments use stationary subjects while motion is known to affect the accuracy of remote PPG. In this paper, a novel methodology is introduced to enhance the accuracy and reliability of HR monitoring based on rPPG in the presence of physical activities like Yoga. This method quickly and accurately tracks HR and analyzes head motion to exclude unreliable data within short windows of rPPG signals. The method was tested with smartphone video data collected from 60 subjects when they are doing activities with varying levels of movement. Results show that our method without motion removal improves the accuracy of the HR readings by 0.7 bpm, reaching 3.57 bpm on average for a 30-sec-window. The accuracy is further improved by another 1.3 bpm after removing the motion artifacts, and reaches 2.29 bpm.Clinical relevance- The enhancement of HR readings from shorter rPPG signal with motion tolerance during physical activities can ultimately help with a more reliable HR tracking of people in uncontrolled settings like home which is a critical step towards remote health-care or wellness tracking.
Collapse
|
36
|
Fleischhauer V, Bruhn J, Rasche S, Zaunseder S. Photoplethysmography upon cold stress-impact of measurement site and acquisition mode. Front Physiol 2023; 14:1127624. [PMID: 37324389 PMCID: PMC10267461 DOI: 10.3389/fphys.2023.1127624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 05/15/2023] [Indexed: 06/17/2023] Open
Abstract
Photoplethysmography (PPG) allows various statements about the physiological state. It supports multiple recording setups, i.e., application to various body sites and different acquisition modes, rendering the technique a versatile tool for various situations. Owing to anatomical, physiological and metrological factors, PPG signals differ with the actual setup. Research on such differences can deepen the understanding of prevailing physiological mechanisms and path the way towards improved or novel methods for PPG analysis. The presented work systematically investigates the impact of the cold pressor test (CPT), i.e., a painful stimulus, on the morphology of PPG signals considering different recording setups. Our investigation compares contact PPG recorded at the finger, contact PPG recorded at the earlobe and imaging PPG (iPPG), i.e., non-contact PPG, recorded at the face. The study bases on own experimental data from 39 healthy volunteers. We derived for each recording setup four common morphological PPG features from three intervals around CPT. For the same intervals, we derived blood pressure and heart rate as reference. To assess differences between the intervals, we used repeated measures ANOVA together with paired t-tests for each feature and we calculated Hedges' g to quantify effect sizes. Our analyses show a distinct impact of CPT. As expected, blood pressure shows a highly significant and persistent increase. Independently of the recording setup, all PPG features show significant changes upon CPT as well. However, there are marked differences between recording setups. Effect sizes generally differ with the finger PPG showing the strongest response. Moreover, one feature (pulse width at half amplitude) shows an inverse behavior in finger PPG and head PPG (earlobe PPG and iPPG). In addition, iPPG features behave partially different from contact PPG features as they tend to return to baseline values while contact PPG features remain altered. Our findings underline the importance of recording setup and physiological as well as metrological differences that relate to the setups. The actual setup must be considered in order to properly interpret features and use PPG. The existence of differences between recording setups and a deepened knowledge on such differences might open up novel diagnostic methods in the future.
Collapse
Affiliation(s)
- Vincent Fleischhauer
- Laboratory for Advanced Measurements and Biomedical Data Analysis, Faculty of Information Technology, FH Dortmund, Dortmund, Germany
| | - Jan Bruhn
- Laboratory for Advanced Measurements and Biomedical Data Analysis, Faculty of Information Technology, FH Dortmund, Dortmund, Germany
| | - Stefan Rasche
- Faculty of Medicine Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Sebastian Zaunseder
- Laboratory for Advanced Measurements and Biomedical Data Analysis, Faculty of Information Technology, FH Dortmund, Dortmund, Germany
- Professorship for Diagnostic Sensing, Faculty of Applied Computer Science, University Augsburg, Augsburg, Germany
| |
Collapse
|
37
|
Boiko A, Martínez Madrid N, Seepold R. Contactless Technologies, Sensors, and Systems for Cardiac and Respiratory Measurement during Sleep: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23115038. [PMID: 37299762 DOI: 10.3390/s23115038] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 05/22/2023] [Accepted: 05/23/2023] [Indexed: 06/12/2023]
Abstract
Sleep is essential to physical and mental health. However, the traditional approach to sleep analysis-polysomnography (PSG)-is intrusive and expensive. Therefore, there is great interest in the development of non-contact, non-invasive, and non-intrusive sleep monitoring systems and technologies that can reliably and accurately measure cardiorespiratory parameters with minimal impact on the patient. This has led to the development of other relevant approaches, which are characterised, for example, by the fact that they allow greater freedom of movement and do not require direct contact with the body, i.e., they are non-contact. This systematic review discusses the relevant methods and technologies for non-contact monitoring of cardiorespiratory activity during sleep. Taking into account the current state of the art in non-intrusive technologies, we can identify the methods of non-intrusive monitoring of cardiac and respiratory activity, the technologies and types of sensors used, and the possible physiological parameters available for analysis. To do this, we conducted a literature review and summarised current research on the use of non-contact technologies for non-intrusive monitoring of cardiac and respiratory activity. The inclusion and exclusion criteria for the selection of publications were established prior to the start of the search. Publications were assessed using one main question and several specific questions. We obtained 3774 unique articles from four literature databases (Web of Science, IEEE Xplore, PubMed, and Scopus) and checked them for relevance, resulting in 54 articles that were analysed in a structured way using terminology. The result was 15 different types of sensors and devices (e.g., radar, temperature sensors, motion sensors, cameras) that can be installed in hospital wards and departments or in the environment. The ability to detect heart rate, respiratory rate, and sleep disorders such as apnoea was among the characteristics examined to investigate the overall effectiveness of the systems and technologies considered for cardiorespiratory monitoring. In addition, the advantages and disadvantages of the considered systems and technologies were identified by answering the identified research questions. The results obtained allow us to determine the current trends and the vector of development of medical technologies in sleep medicine for future researchers and research.
Collapse
Affiliation(s)
- Andrei Boiko
- Ubiquitous Computing Laboratory, Department of Computer Science, HTWG Konstanz-University of Applied Sciences, Alfred-Wachtel-Str. 8, 78462 Konstanz, Germany
| | - Natividad Martínez Madrid
- Internet of Things Laboratory, School of Informatics, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany
| | - Ralf Seepold
- Ubiquitous Computing Laboratory, Department of Computer Science, HTWG Konstanz-University of Applied Sciences, Alfred-Wachtel-Str. 8, 78462 Konstanz, Germany
| |
Collapse
|
38
|
Kolosov D, Kelefouras V, Kourtessis P, Mporas I. Contactless Camera-Based Heart Rate and Respiratory Rate Monitoring Using AI on Hardware. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23094550. [PMID: 37177754 PMCID: PMC10181491 DOI: 10.3390/s23094550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Revised: 05/01/2023] [Accepted: 05/05/2023] [Indexed: 05/15/2023]
Abstract
Detecting vital signs by using a contactless camera-based approach can provide several advantages over traditional clinical methods, such as lower financial costs, reduced visit times, increased comfort, and enhanced safety for healthcare professionals. Specifically, Eulerian Video Magnification (EVM) or Remote Photoplethysmography (rPPG) methods can be utilised to remotely estimate heart rate and respiratory rate biomarkers. In this paper two contactless camera-based health monitoring architectures are developed using EVM and rPPG, respectively; to this end, two different CNNs, (Mediapipe's BlazeFace and FaceMesh) are used to extract suitable regions of interest from incoming video frames. These two methods are implemented and deployed on four off-the-shelf edge devices as well as on a PC and evaluated in terms of latency (in each stage of the application's pipeline), throughput (FPS), power consumption (Watt), efficiency (throughput/Watt), and value (throughput/cost). This work provides important insights about the computational costs and bottlenecks of each method on each hardware platform, as well as which platform to use depending on the target metric. One of our insights shows that the Jetson Xavier NX platform is the best platform in terms of throughput and efficiency, while Raspberry Pi 4 8 GB is the best platform in terms of value.
Collapse
Affiliation(s)
- Dimitrios Kolosov
- School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK
| | - Vasilios Kelefouras
- School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth PL4 8AA, UK
| | - Pandelis Kourtessis
- School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK
| | - Iosif Mporas
- School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK
| |
Collapse
|
39
|
Savur C, Dautov R, Bukum K, Xia X, Couderc JP, Tsouri GR. Monitoring Pulse Rate in the Background Using Front Facing Cameras of Mobile Devices. IEEE J Biomed Health Inform 2023; 27:2208-2218. [PMID: 35939479 PMCID: PMC10244025 DOI: 10.1109/jbhi.2022.3197076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
We propose a novel framework to passively monitor pulse rate during the time spent by users on their personal mobile devices. Our framework is based on passively capturing the user's pulse signal using the front-facing camera. Signal capture is performed in the background, while the user is interacting with the device as he/she normally would, e.g., watch movies, read emails, text, and play games. The framework does not require subject participation with the monitoring procedure, thereby addressing the well-known problem of low adherence with such procedures. We investigate various techniques to suppress the impact of spontaneous user motion and fluctuations in ambient light conditions expected in non-participatory environments. Techniques include traditional signal processing, machine learning classifiers, and deep learning methods. Our performance evaluation is based on a clinical study encompassing 113 patients with a history of atrial fibrillation (Afib) who are passively monitored at home using a tablet for a period of two weeks. Our results show that the proposed framework accurately monitors pulse rate, thereby providing a gateway for long-term monitoring without relying on subject participation or the use of a dedicated wearable device.
Collapse
|
40
|
闫 昊, 李 霞, 朱 田, 陈 修, 龚 宁, 周 秦. [A non-contact continuous blood pressure measurement method based on video stream]. SHENG WU YI XUE GONG CHENG XUE ZA ZHI = JOURNAL OF BIOMEDICAL ENGINEERING = SHENGWU YIXUE GONGCHENGXUE ZAZHI 2023; 40:249-256. [PMID: 37139755 PMCID: PMC10162922 DOI: 10.7507/1001-5515.202211071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 02/28/2023] [Indexed: 05/05/2023]
Abstract
Hypertension is the primary disease that endangers human health. A convenient and accurate blood pressure measurement method can help to prevent the hypertension. This paper proposed a continuous blood pressure measurement method based on facial video signal. Firstly, color distortion filtering and independent component analysis were used to extract the video pulse wave of the region of interest in the facial video signal, and the multi-dimensional feature extraction of the pulse wave was preformed based on the time-frequency domain and physiological principles; Secondly, an integrated feature selection method was designed to extract the universal optimal feature subset; After that, we compared the single person blood pressure measurement models established by Elman neural network based on particle swarm optimization, support vector machine (SVM) and deep belief network; Finally, we used SVM algorithm to build a general blood pressure prediction model, which was compared and evaluated with the real blood pressure value. The experimental results showed that the blood pressure measurement results based on facial video were in good agreement with the standard blood pressure values. Comparing the estimated blood pressure from the video with standard blood pressure value, the mean absolute error (MAE) of systolic blood pressure was 4.9 mm Hg with a standard deviation (STD) of 5.9 mm Hg, and the MAE of diastolic blood pressure was 4.6 mm Hg with a STD of 5.0 mm Hg, which met the AAMI standards. The non-contact blood pressure measurement method based on video stream proposed in this paper can be used for blood pressure measurement.
Collapse
Affiliation(s)
- 昊 闫
- 西安交通大学 生命科学与技术学院(西安 710049)School of Life Science and Technology, Xi’an Jiaotong University, Xi’an 710049, P. R. China
| | - 霞 李
- 西安交通大学 生命科学与技术学院(西安 710049)School of Life Science and Technology, Xi’an Jiaotong University, Xi’an 710049, P. R. China
| | - 田杨 朱
- 西安交通大学 生命科学与技术学院(西安 710049)School of Life Science and Technology, Xi’an Jiaotong University, Xi’an 710049, P. R. China
| | - 修强 陈
- 西安交通大学 生命科学与技术学院(西安 710049)School of Life Science and Technology, Xi’an Jiaotong University, Xi’an 710049, P. R. China
| | - 宁 龚
- 西安交通大学 生命科学与技术学院(西安 710049)School of Life Science and Technology, Xi’an Jiaotong University, Xi’an 710049, P. R. China
| | - 秦武 周
- 西安交通大学 生命科学与技术学院(西安 710049)School of Life Science and Technology, Xi’an Jiaotong University, Xi’an 710049, P. R. China
| |
Collapse
|
41
|
Dominguez CT, Martinelli MB, Bachmann L, Cardoso GC. Arterial pulsation modulates the optical attenuation coefficient of skin. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:C87-C92. [PMID: 37132976 DOI: 10.1364/josaa.482939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Photoplethysmographic (PPG) signals arise from the modulation of light reflectivity on the skin due to changes of physiological origin. Imaging plethysmography (iPPG) is a video-based PPG method that can remotely monitor vital signs in a non-invasive manner. iPPG signals result from skin reflectivity modulation. The origin of such reflectivity modulation is still a subject of debate. Here, we have used optical coherence tomography (OCT) imaging to find whether iPPG signals may result from skin optical properties being directly or indirectly modulated by arterial transmural pressure propagation. The light intensity across the tissue was modeled through a simple exponential decay (Beer-Lambert law) to analyze in vivo the modulation of the optical attenuation coefficient of the skin by arterial pulsation. The OCT transversal images were acquired from a forearm of three subjects in a pilot study. The results show that the optical attenuation coefficient of skin changes at the same frequency as the arterial pulsation due to transmural pressure propagation (local ballistographic effect), but we cannot discard the contribution of global ballistographic effects.
Collapse
|
42
|
Wu J, Zhu Y, Jiang X, Liu Y, Lin J. Local attention and long-distance interaction of rPPG for deepfake detection. THE VISUAL COMPUTER 2023:1-12. [PMID: 37361461 PMCID: PMC10052279 DOI: 10.1007/s00371-023-02833-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 03/06/2023] [Indexed: 06/28/2023]
Abstract
With the development of generative models, abused Deepfakes have aroused public concerns. As a defense mechanism, face forgery detection methods have been intensively studied. Remote photoplethysmography (rPPG) technology extract heartbeat signal from recorded videos by examining the subtle changes in skin color caused by cardiac activity. Since the face forgery process inevitably disrupts the periodic changes in facial color, rPPG signal proves to be a powerful biological indicator for Deepfake detection. Motivated by the key observation that rPPG signals produce unique rhythmic patterns in terms of different manipulation methods, we regard Deepfake detection also as a source detection task. The Multi-scale Spatial-Temporal PPG map is adopted to further exploit heartbeat signal from multiple facial regions. Moreover, to capture both spatial and temporal inconsistencies, we propose a two-stage network consisting of a Mask-Guided Local Attention module (MLA) to capture unique local patterns of PPG maps, and a Temporal Transformer to interact features of adjacent PPG maps in long distance. Abundant experiments on FaceForensics + + and Celeb-DF datasets prove the superiority of our method over all other rPPG-based approaches. Visualization also demonstrates the effectiveness of the proposed method.
Collapse
Affiliation(s)
- Jiahui Wu
- School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237 China
| | - Yu Zhu
- School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237 China
- Shanghai Engineering Research Center of Internet of Things for Respiratory Medicine, Shanghai, 200032 China
| | - Xiaoben Jiang
- School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237 China
| | - Yatong Liu
- School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237 China
| | - Jiajun Lin
- School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237 China
| |
Collapse
|
43
|
Manni A, Caroppo A, Rescio G, Siciliano P, Leone A. Benchmarking of Contactless Heart Rate Measurement Systems in ARM-Based Embedded Platforms. SENSORS (BASEL, SWITZERLAND) 2023; 23:3507. [PMID: 37050566 PMCID: PMC10098566 DOI: 10.3390/s23073507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/23/2023] [Accepted: 03/26/2023] [Indexed: 06/19/2023]
Abstract
Heart rate monitoring is especially important for aging individuals because it is associated with longevity and cardiovascular risk. Typically, this vital parameter can be measured using wearable sensors, which are widely available commercially. However, wearable sensors have some disadvantages in terms of acceptability, especially when used by elderly people. Thus, contactless solutions have increasingly attracted the scientific community in recent years. Camera-based photoplethysmography (also known as remote photoplethysmography) is an emerging method of contactless heart rate monitoring that uses a camera and a processing unit on the hardware side, and appropriate image processing methodologies on the software side. This paper describes the design and implementation of a novel pipeline for heart rate estimation using a commercial and low-cost camera as the input device. The pipeline's performance was tested and compared on a desktop PC, a laptop, and three different ARM-based embedded platforms (Raspberry Pi 4, Odroid N2+, and Jetson Nano). The results showed that the designed and implemented pipeline achieved an average accuracy of about 96.7% for heart rate estimation, with very low variance (between 1.5% and 2.5%) across processing platforms, user distances from the camera, and frame resolutions. Furthermore, benchmark analysis showed that the Odroid N2+ platform was the most convenient in terms of CPU load, RAM usage, and average execution time of the algorithmic pipeline.
Collapse
|
44
|
For Heart Rate Assessments from Drone Footage in Disaster Scenarios. Bioengineering (Basel) 2023; 10:bioengineering10030336. [PMID: 36978727 PMCID: PMC10045207 DOI: 10.3390/bioengineering10030336] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 02/26/2023] [Accepted: 03/02/2023] [Indexed: 03/09/2023] Open
Abstract
The ability to use drones to obtain important vital signs could be very valuable for emergency personnel during mass-casualty incidents. The rapid and robust remote assessment of heart rates could serve as a life-saving decision aid for first-responders. With the flight sensor data of a specialized drone, a pipeline was developed to achieve a robust, non-contact assessment of heart rates through remote photoplethysmography (rPPG). This robust assessment was achieved through adaptive face-aware exposure and comprehensive de-noising of a large number of predicted noise sources. In addition, we performed a proof-of-concept study that involved 18 stationary subjects with clean skin and 36 recordings of their vital signs, using the developed pipeline in outdoor conditions. In this study, we could achieve a single-value heart-rate assessment with an overall root-mean-squared error of 14.3 beats-per-minute, demonstrating the basic feasibility of our approach. However, further research is needed to verify the applicability of our approach in actual disaster situations, where remote photoplethysmography readings could be impacted by other factors, such as blood, dirt, and body positioning.
Collapse
|
45
|
Ouzar Y, Djeldjli D, Bousefsaf F, Maaoui C. X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation. Comput Biol Med 2023; 154:106592. [PMID: 36709517 DOI: 10.1016/j.compbiomed.2023.106592] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 12/07/2022] [Accepted: 01/22/2023] [Indexed: 01/26/2023]
Abstract
Pulse rate (PR) is one of the most important markers for assessing a person's health. With the increasing demand for long-term health monitoring, much attention is being paid to contactless PR estimation using imaging photoplethysmography (iPPG). This non-invasive technique is based on the analysis of subtle changes in skin color. Despite efforts to improve iPPG, the existing algorithms are vulnerable to less-constrained scenarios (i.e., head movements, facial expressions, and environmental conditions). In this article, we propose a novel end-to-end spatio-temporal network, namely X-iPPGNet, for instantaneous PR estimation directly from facial video recordings. Unlike most existing systems, our model learns the iPPG concept from scratch without incorporating any prior knowledge or going through the extraction of blood volume pulse signals. Inspired by the Xception network architecture, color channel decoupling is used to learn additional photoplethysmographic information and to effectively reduce the computational cost and memory requirements. Moreover, X-iPPGNet predicts the pulse rate from a short time window (2 s), which has advantages with high and sharply fluctuating pulse rates. The experimental results revealed high performance under all conditions including head motions, facial expressions, and skin tone. Our approach significantly outperforms all current state-of-the-art methods on three benchmark datasets: MMSE-HR (MAE = 4.10 ; RMSE = 5.32 ; r = 0.85), UBFC-rPPG (MAE = 4.99 ; RMSE = 6.26 ; r = 0.67), MAHNOB-HCI (MAE = 3.17 ; RMSE = 3.93 ; r = 0.88).
Collapse
|
46
|
Cai Y, Li X, Li J. Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23052455. [PMID: 36904659 PMCID: PMC10007272 DOI: 10.3390/s23052455] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 02/18/2023] [Accepted: 02/21/2023] [Indexed: 06/12/2023]
Abstract
In recent years, the rapid development of sensors and information technology has made it possible for machines to recognize and analyze human emotions. Emotion recognition is an important research direction in various fields. Human emotions have many manifestations. Therefore, emotion recognition can be realized by analyzing facial expressions, speech, behavior, or physiological signals. These signals are collected by different sensors. Correct recognition of human emotions can promote the development of affective computing. Most existing emotion recognition surveys only focus on a single sensor. Therefore, it is more important to compare different sensors or unimodality and multimodality. In this survey, we collect and review more than 200 papers on emotion recognition by literature research methods. We categorize these papers according to different innovations. These articles mainly focus on the methods and datasets used for emotion recognition with different sensors. This survey also provides application examples and developments in emotion recognition. Furthermore, this survey compares the advantages and disadvantages of different sensors for emotion recognition. The proposed survey can help researchers gain a better understanding of existing emotion recognition systems, thus facilitating the selection of suitable sensors, algorithms, and datasets.
Collapse
|
47
|
Yu Z, Shen Y, Shi J, Zhao H, Cui Y, Zhang J, Torr P, Zhao G. PhysFormer++: Facial Video-Based Physiological Measurement with SlowFast Temporal Difference Transformer. Int J Comput Vis 2023. [DOI: 10.1007/s11263-023-01758-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/17/2023]
Abstract
AbstractRemote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote healthcare and affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, which neglect the long-range spatio-temporal perception and interaction for rPPG modeling. In this paper, we propose two end-to-end video transformer based architectures, namely PhysFormer and PhysFormer++, to adaptively aggregate both local and global spatio-temporal features for rPPG representation enhancement. As key modules in PhysFormer, the temporal difference transformers first enhance the quasi-periodic rPPG features with temporal difference guided global attention, and then refine the local spatio-temporal representation against interference. To better exploit the temporal contextual and periodic rPPG clues, we also extend the PhysFormer to the two-pathway SlowFast based PhysFormer++ with temporal difference periodic and cross-attention transformers. Furthermore, we propose the label distribution learning and a curriculum learning inspired dynamic constraint in frequency domain, which provide elaborate supervisions for PhysFormer and PhysFormer++ and alleviate overfitting. Comprehensive experiments are performed on four benchmark datasets to show our superior performance on both intra- and cross-dataset testings. Unlike most transformer networks needed pretraining from large-scale datasets, the proposed PhysFormer family can be easily trained from scratch on rPPG datasets, which makes it promising as a novel transformer baseline for the rPPG community.
Collapse
|
48
|
Haugg F, Elgendi M, Menon C. GRGB rPPG: An Efficient Low-Complexity Remote Photoplethysmography-Based Algorithm for Heart Rate Estimation. Bioengineering (Basel) 2023; 10:bioengineering10020243. [PMID: 36829737 PMCID: PMC9952130 DOI: 10.3390/bioengineering10020243] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 01/29/2023] [Accepted: 02/10/2023] [Indexed: 02/15/2023] Open
Abstract
Remote photoplethysmography (rPPG) is a promising contactless technology that uses videos of faces to extract health parameters, such as heart rate. Several methods for transforming red, green, and blue (RGB) video signals into rPPG signals have been introduced in the existing literature. The RGB signals represent variations in the reflected luminance from the skin surface of an individual over a given period of time. These methods attempt to find the best combination of color channels to reconstruct an rPPG signal. Usually, rPPG methods use a combination of prepossessed color channels to convert the three RGB signals to one rPPG signal that is most influenced by blood volume changes. This study examined simple yet effective methods to convert the RGB to rPPG, relying only on RGB signals without applying complex mathematical models or machine learning algorithms. A new method, GRGB rPPG, was proposed that outperformed most machine-learning-based rPPG methods and was robust to indoor lighting and participant motion. Moreover, the proposed method estimated the heart rate better than well-established rPPG methods. This paper also discusses the results and provides recommendations for further research.
Collapse
Affiliation(s)
- Fridolin Haugg
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Department of Mechanical Engineering, Karlsruher Institute for Technology, 76131 Karlsruhe, Germany
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Correspondence: (M.E.); (C.M.)
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Correspondence: (M.E.); (C.M.)
| |
Collapse
|
49
|
Hamoud B, Kashevnik A, Othman W, Shilov N. Neural Network Model Combination for Video-Based Blood Pressure Estimation: New Approach and Evaluation. SENSORS (BASEL, SWITZERLAND) 2023; 23:1753. [PMID: 36850349 PMCID: PMC9959092 DOI: 10.3390/s23041753] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/18/2022] [Revised: 01/23/2023] [Accepted: 02/02/2023] [Indexed: 06/18/2023]
Abstract
One of the most effective vital signs of health conditions is blood pressure. It has such an impact that changes your state from completely relaxed to extremely unpleasant, which makes the task of blood pressure monitoring a main procedure that almost everyone undergoes whenever there is something wrong or suspicious with his/her health condition. The most popular and accurate ways to measure blood pressure are cuff-based, inconvenient, and pricey, but on the bright side, many experimental studies prove that changes in the color intensities of the RGB channels represent variation in the blood that flows beneath the skin, which is strongly related to blood pressure; hence, we present a novel approach to blood pressure estimation based on the analysis of human face video using hybrid deep learning models. We deeply analyzed proposed approaches and methods to develop combinations of state-of-the-art models that were validated by their testing results on the Vision for Vitals (V4V) dataset compared to the performance of other available proposed models. Additionally, we came up with a new metric to evaluate the performance of our models using Pearson's correlation coefficient between the predicted blood pressure of the subjects and their respiratory rate at each minute, which is provided by our own dataset that includes 60 videos of operators working on personal computers for almost 20 min in each video. Our method provides a cuff-less, fast, and comfortable way to estimate blood pressure with no need for any equipment except the camera of your smartphone.
Collapse
Affiliation(s)
- Batol Hamoud
- Information Technology and Programming Faculty, ITMO University, St. Petersburg 197101, Russia
| | - Alexey Kashevnik
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg 199178, Russia
- Department of Applied Mathematics and Cybernetics, Institute of Mathematics and Information Technologies, Petrozavodsk State University (PetrSU), Petrozavodsk 185910, Russia
| | - Walaa Othman
- Information Technology and Programming Faculty, ITMO University, St. Petersburg 197101, Russia
| | - Nikolay Shilov
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg 199178, Russia
| |
Collapse
|
50
|
van Es VAA, Lopata RGP, Scilingo EP, Nardelli M. Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23031505. [PMID: 36772543 PMCID: PMC9919512 DOI: 10.3390/s23031505] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 01/16/2023] [Accepted: 01/20/2023] [Indexed: 05/27/2023]
Abstract
Despite the notable recent developments in the field of remote photoplethysmography (rPPG), extracting a reliable pulse rate variability (PRV) signal still remains a challenge. In this study, eight image-based photoplethysmography (iPPG) extraction methods (GRD, AGRD, PCA, ICA, LE, SPE, CHROM, and POS) were compared in terms of pulse rate (PR) and PRV features. The algorithms were made robust for motion and illumination artifacts by using ad hoc pre- and postprocessing steps. Then, they were systematically tested on the public dataset UBFC-RPPG, containing data from 42 subjects sitting in front of a webcam (30 fps) while playing a time-sensitive mathematical game. The performances of the algorithms were evaluated by statistically comparing iPPG-based and finger-PPG-based PR and PRV features in terms of Spearman's correlation coefficient, normalized root mean square error (NRMSE), and Bland-Altman analysis. The study revealed POS and CHROM techniques to be the most robust for PR estimation and the assessment of overall autonomic nervous system (ANS) dynamics by using PRV features in time and frequency domains. Furthermore, we demonstrated that a reliable characterization of the vagal tone is made possible by computing the Poincaré map of PRV series derived from the POS and CHROM methods. This study supports the use of iPPG systems as promising tools to obtain clinically useful and specific information about ANS dynamics.
Collapse
Affiliation(s)
- Valerie A. A. van Es
- Department of Biomedical Engineering, University of Technology, P.O. Box 513, 5600 Eindhoven, The Netherlands
| | - Richard G. P. Lopata
- Department of Biomedical Engineering, University of Technology, P.O. Box 513, 5600 Eindhoven, The Netherlands
| | - Enzo Pasquale Scilingo
- Bioengineering and Robotics Research Centre E. Piaggio, Dipartimento di Ingegneria dell’Informazione, University of Pisa, Largo Lucio Lazzarino 1, 56122 Pisa, Italy
| | - Mimma Nardelli
- Bioengineering and Robotics Research Centre E. Piaggio, Dipartimento di Ingegneria dell’Informazione, University of Pisa, Largo Lucio Lazzarino 1, 56122 Pisa, Italy
| |
Collapse
|