1
|
Baumann S, Stone R, Kim JYM. Introducing the Pi-CON Methodology to Overcome Usability Deficits during Remote Patient Monitoring. SENSORS (BASEL, SWITZERLAND) 2024; 24:2260. [PMID: 38610471 PMCID: PMC11014368 DOI: 10.3390/s24072260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Revised: 03/25/2024] [Accepted: 03/27/2024] [Indexed: 04/14/2024]
Abstract
The adoption of telehealth has soared, and with that the acceptance of Remote Patient Monitoring (RPM) and virtual care. A review of the literature illustrates, however, that poor device usability can impact the generated data when using Patient-Generated Health Data (PGHD) devices, such as wearables or home use medical devices, when used outside a health facility. The Pi-CON methodology is introduced to overcome these challenges and guide the definition of user-friendly and intuitive devices in the future. Pi-CON stands for passive, continuous, and non-contact, and describes the ability to acquire health data, such as vital signs, continuously and passively with limited user interaction and without attaching any sensors to the patient. The paper highlights the advantages of Pi-CON by leveraging various sensors and techniques, such as radar, remote photoplethysmography, and infrared. It illustrates potential concerns and discusses future applications Pi-CON could be used for, including gait and fall monitoring by installing an omnipresent sensor based on the Pi-CON methodology. This would allow automatic data collection once a person is recognized, and could be extended with an integrated gateway so multiple cameras could be installed to enable data feeds to a cloud-based interface, allowing clinicians and family members to monitor patient health status remotely at any time.
Collapse
Affiliation(s)
| | | | - Joseph Yun-Ming Kim
- Industrial and Manufacturing Systems Engineering, Iowa State University, 2529 Union Dr, Ames, IA 50011, USA; (S.B.); (R.S.)
| |
Collapse
|
2
|
Fan X, Liu F, Zhang J, Gao T, Fan Z, Huang Z, Xue W, Zhang J. Remote photoplethysmography based on reflected light angle estimation. Physiol Meas 2024; 45:035005. [PMID: 38430568 DOI: 10.1088/1361-6579/ad2f5d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 03/01/2024] [Indexed: 03/04/2024]
Abstract
Objective. In previous studies, the factors affecting the accuracy of imaging photoplethysmography (iPPG) heart rate (HR) measurement have been focused on the light intensity, facial reflection angle, and motion artifacts. However, the factor of specularly reflected light has not been studied in detail. We explored the effect of specularly reflected light on the accuracy of HR estimation and proposed an estimation method for the direction of specularly radiated light.Approach. To study the HR measurement accuracy influenced by specularly reflected light, we control the component of specularly reflected light by controlling its angle. A total of 100 videos from four different reflected light angles were collected, and 25 subjects participated in the dataset collection. We extracted angles and illuminations for 71 facial regions, fitting sample points through interpolation, and selecting the angle corresponding to the maximum weight in the fitted curve as the estimated reflected angle.Main results. The experimental results show that higher specularly reflected light compromises HR estimation accuracy under the same value of light intensity. Notably, at a 60° angle, the HR accuracy (ACC) increased by 0.7%, while the signal-to-noise ratio and Pearson correlation coefficient increased by 0.8 dB and 0.035, respectively, compared to 0°. The overall root mean squared error, standard deviation, and mean error of our proposed reflected light angle estimation method on the illumination multi-angle incidence (IMAI) dataset are 1.173°, 0.978°, and 0.773°. The average Pearson value is 0.8 in the PURE rotation dataset. In addition, the average ACC of HR measurements in the PURE dataset is improved by 1.73% in our method compared to the state-of-the-art traditional methods.Significance. Our method has great potential for clinical applications, especially in bright light environments such as during surgery, to improve accuracy and monitor blood volume changes in blood vessels.
Collapse
Affiliation(s)
- Xuanhe Fan
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Fangwu Liu
- Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai, People's Republic of China
| | - Jinjin Zhang
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Tong Gao
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Ziyang Fan
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Zhijie Huang
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - Wei Xue
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| | - JingJing Zhang
- China University of Geosciences, Wuhan, China, School of Automation, Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, People's Republic of China
| |
Collapse
|
3
|
Ontiveros RC, Elgendi M, Missale G, Menon C. Evaluating RGB channels in remote photoplethysmography: a comparative study with contact-based PPG. Front Physiol 2023; 14:1296277. [PMID: 38187134 PMCID: PMC10770840 DOI: 10.3389/fphys.2023.1296277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 11/28/2023] [Indexed: 01/09/2024] Open
Abstract
Remote photoplethysmography (rPPG) provides a non-contact method for measuring blood volume changes. In this study, we compared rPPG signals obtained from video cameras with traditional contact-based photoplethysmography (cPPG) to assess the effectiveness of different RGB channels in cardiac signal extraction. Our objective was to determine the most effective RGB channel for detecting blood volume changes and estimating heart rate. We employed dynamic time warping, Pearson's correlation coefficient, root-mean-square error, and Beats-per-minute Difference to evaluate the performance of each RGB channel relative to cPPG. The results revealed that the green channel was superior, outperforming the blue and red channels in detecting volumetric changes and accurately estimating heart rate across various activities. We also observed that the reliability of RGB signals varied based on recording conditions and subject activity. This finding underscores the importance of understanding the performance nuances of RGB inputs, crucial for constructing rPPG signals in algorithms. Our study is significant in advancing rPPG research, offering insights that could benefit clinical applications by improving non-contact methods for blood volume assessment.
Collapse
Affiliation(s)
- Rodrigo Castellano Ontiveros
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
- School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
| | - Giuseppe Missale
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
- Electronics and Telecommunications Department, Politecnico Di Torino, Torino, Italy
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, Zurich, Switzerland
| |
Collapse
|
4
|
Braun B, McDuff D, Baltrusaitis T, Holz C. Video-based sympathetic arousal assessment via peripheral blood flow estimation. BIOMEDICAL OPTICS EXPRESS 2023; 14:6607-6628. [PMID: 38420320 PMCID: PMC10898569 DOI: 10.1364/boe.507949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 10/27/2023] [Accepted: 10/27/2023] [Indexed: 03/02/2024]
Abstract
Electrodermal activity (EDA) is considered a standard marker of sympathetic activity. However, traditional EDA measurement requires electrodes in steady contact with the skin. Can sympathetic arousal be measured using only an optical sensor, such as an RGB camera? This paper presents a novel approach to infer sympathetic arousal by measuring the peripheral blood flow on the face or hand optically. We contribute a self-recorded dataset of 21 participants, comprising synchronized videos of participants' faces and palms and gold-standard EDA and photoplethysmography (PPG) signals. Our results show that we can measure peripheral sympathetic responses that closely correlate with the ground truth EDA. We obtain median correlations of 0.57 to 0.63 between our inferred signals and the ground truth EDA using only videos of the participants' palms or foreheads or PPG signals from the foreheads or fingers. We also show that sympathetic arousal is best inferred from the forehead, finger, or palm.
Collapse
Affiliation(s)
- Björn Braun
- Department of Computer Science, ETH Zürich, Switzerland
| | | | | | | |
Collapse
|
5
|
Lie WN, Le DQ, Lai CY, Fang YS. Heart Rate Estimation from Facial Image Sequences of a Dual-Modality RGB-NIR Camera. SENSORS (BASEL, SWITZERLAND) 2023; 23:6079. [PMID: 37447928 DOI: 10.3390/s23136079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 06/18/2023] [Accepted: 06/27/2023] [Indexed: 07/15/2023]
Abstract
This paper presents an RGB-NIR (Near Infrared) dual-modality technique to analyze the remote photoplethysmogram (rPPG) signal and hence estimate the heart rate (in beats per minute), from a facial image sequence. Our main innovative contribution is the introduction of several denoising techniques such as Modified Amplitude Selective Filtering (MASF), Wavelet Decomposition (WD), and Robust Principal Component Analysis (RPCA), which take advantage of RGB and NIR band characteristics to uncover the rPPG signals effectively through this Independent Component Analysis (ICA)-based algorithm. Two datasets, of which one is the public PURE dataset and the other is the CCUHR dataset built with a popular Intel RealSense D435 RGB-D camera, are adopted in our experiments. Facial video sequences in the two datasets are diverse in nature with normal brightness, under-illumination (i.e., dark), and facial motion. Experimental results show that the proposed method has reached competitive accuracies among the state-of-the-art methods even at a shorter video length. For example, our method achieves MAE = 4.45 bpm (beats per minute) and RMSE = 6.18 bpm for RGB-NIR videos of 10 and 20 s in the CCUHR dataset and MAE = 3.24 bpm and RMSE = 4.1 bpm for RGB videos of 60-s in the PURE dataset. Our system has the advantages of accessible and affordable hardware, simple and fast computations, and wide realistic applications.
Collapse
Affiliation(s)
- Wen-Nung Lie
- Department of Electrical Engineering, Center for Innovative Research on Aging Society (CIRAS), and Advanced Institute of Manufacturing with High-Tech Innovations (AIM-HI), National Chung Cheng University, Chia-Yi 621, Taiwan
| | - Dao-Quang Le
- Department of Electrical Engineering, Center for Innovative Research on Aging Society (CIRAS), and Advanced Institute of Manufacturing with High-Tech Innovations (AIM-HI), National Chung Cheng University, Chia-Yi 621, Taiwan
| | - Chun-Yu Lai
- Department of Electrical Engineering, Center for Innovative Research on Aging Society (CIRAS), and Advanced Institute of Manufacturing with High-Tech Innovations (AIM-HI), National Chung Cheng University, Chia-Yi 621, Taiwan
| | - Yu-Shin Fang
- Department of Electrical Engineering, Center for Innovative Research on Aging Society (CIRAS), and Advanced Institute of Manufacturing with High-Tech Innovations (AIM-HI), National Chung Cheng University, Chia-Yi 621, Taiwan
| |
Collapse
|
6
|
Fleischhauer V, Bruhn J, Rasche S, Zaunseder S. Photoplethysmography upon cold stress-impact of measurement site and acquisition mode. Front Physiol 2023; 14:1127624. [PMID: 37324389 PMCID: PMC10267461 DOI: 10.3389/fphys.2023.1127624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 05/15/2023] [Indexed: 06/17/2023] Open
Abstract
Photoplethysmography (PPG) allows various statements about the physiological state. It supports multiple recording setups, i.e., application to various body sites and different acquisition modes, rendering the technique a versatile tool for various situations. Owing to anatomical, physiological and metrological factors, PPG signals differ with the actual setup. Research on such differences can deepen the understanding of prevailing physiological mechanisms and path the way towards improved or novel methods for PPG analysis. The presented work systematically investigates the impact of the cold pressor test (CPT), i.e., a painful stimulus, on the morphology of PPG signals considering different recording setups. Our investigation compares contact PPG recorded at the finger, contact PPG recorded at the earlobe and imaging PPG (iPPG), i.e., non-contact PPG, recorded at the face. The study bases on own experimental data from 39 healthy volunteers. We derived for each recording setup four common morphological PPG features from three intervals around CPT. For the same intervals, we derived blood pressure and heart rate as reference. To assess differences between the intervals, we used repeated measures ANOVA together with paired t-tests for each feature and we calculated Hedges' g to quantify effect sizes. Our analyses show a distinct impact of CPT. As expected, blood pressure shows a highly significant and persistent increase. Independently of the recording setup, all PPG features show significant changes upon CPT as well. However, there are marked differences between recording setups. Effect sizes generally differ with the finger PPG showing the strongest response. Moreover, one feature (pulse width at half amplitude) shows an inverse behavior in finger PPG and head PPG (earlobe PPG and iPPG). In addition, iPPG features behave partially different from contact PPG features as they tend to return to baseline values while contact PPG features remain altered. Our findings underline the importance of recording setup and physiological as well as metrological differences that relate to the setups. The actual setup must be considered in order to properly interpret features and use PPG. The existence of differences between recording setups and a deepened knowledge on such differences might open up novel diagnostic methods in the future.
Collapse
Affiliation(s)
- Vincent Fleischhauer
- Laboratory for Advanced Measurements and Biomedical Data Analysis, Faculty of Information Technology, FH Dortmund, Dortmund, Germany
| | - Jan Bruhn
- Laboratory for Advanced Measurements and Biomedical Data Analysis, Faculty of Information Technology, FH Dortmund, Dortmund, Germany
| | - Stefan Rasche
- Faculty of Medicine Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Sebastian Zaunseder
- Laboratory for Advanced Measurements and Biomedical Data Analysis, Faculty of Information Technology, FH Dortmund, Dortmund, Germany
- Professorship for Diagnostic Sensing, Faculty of Applied Computer Science, University Augsburg, Augsburg, Germany
| |
Collapse
|
7
|
Manni A, Caroppo A, Rescio G, Siciliano P, Leone A. Benchmarking of Contactless Heart Rate Measurement Systems in ARM-Based Embedded Platforms. SENSORS (BASEL, SWITZERLAND) 2023; 23:3507. [PMID: 37050566 PMCID: PMC10098566 DOI: 10.3390/s23073507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/23/2023] [Accepted: 03/26/2023] [Indexed: 06/19/2023]
Abstract
Heart rate monitoring is especially important for aging individuals because it is associated with longevity and cardiovascular risk. Typically, this vital parameter can be measured using wearable sensors, which are widely available commercially. However, wearable sensors have some disadvantages in terms of acceptability, especially when used by elderly people. Thus, contactless solutions have increasingly attracted the scientific community in recent years. Camera-based photoplethysmography (also known as remote photoplethysmography) is an emerging method of contactless heart rate monitoring that uses a camera and a processing unit on the hardware side, and appropriate image processing methodologies on the software side. This paper describes the design and implementation of a novel pipeline for heart rate estimation using a commercial and low-cost camera as the input device. The pipeline's performance was tested and compared on a desktop PC, a laptop, and three different ARM-based embedded platforms (Raspberry Pi 4, Odroid N2+, and Jetson Nano). The results showed that the designed and implemented pipeline achieved an average accuracy of about 96.7% for heart rate estimation, with very low variance (between 1.5% and 2.5%) across processing platforms, user distances from the camera, and frame resolutions. Furthermore, benchmark analysis showed that the Odroid N2+ platform was the most convenient in terms of CPU load, RAM usage, and average execution time of the algorithmic pipeline.
Collapse
|
8
|
Haugg F, Elgendi M, Menon C. GRGB rPPG: An Efficient Low-Complexity Remote Photoplethysmography-Based Algorithm for Heart Rate Estimation. Bioengineering (Basel) 2023; 10:bioengineering10020243. [PMID: 36829737 PMCID: PMC9952130 DOI: 10.3390/bioengineering10020243] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 01/29/2023] [Accepted: 02/10/2023] [Indexed: 02/15/2023] Open
Abstract
Remote photoplethysmography (rPPG) is a promising contactless technology that uses videos of faces to extract health parameters, such as heart rate. Several methods for transforming red, green, and blue (RGB) video signals into rPPG signals have been introduced in the existing literature. The RGB signals represent variations in the reflected luminance from the skin surface of an individual over a given period of time. These methods attempt to find the best combination of color channels to reconstruct an rPPG signal. Usually, rPPG methods use a combination of prepossessed color channels to convert the three RGB signals to one rPPG signal that is most influenced by blood volume changes. This study examined simple yet effective methods to convert the RGB to rPPG, relying only on RGB signals without applying complex mathematical models or machine learning algorithms. A new method, GRGB rPPG, was proposed that outperformed most machine-learning-based rPPG methods and was robust to indoor lighting and participant motion. Moreover, the proposed method estimated the heart rate better than well-established rPPG methods. This paper also discusses the results and provides recommendations for further research.
Collapse
Affiliation(s)
- Fridolin Haugg
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Department of Mechanical Engineering, Karlsruher Institute for Technology, 76131 Karlsruhe, Germany
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Correspondence: (M.E.); (C.M.)
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Correspondence: (M.E.); (C.M.)
| |
Collapse
|
9
|
Kiddle A, Barham H, Wegerif S, Petronzio C. Dynamic region of interest selection in remote photoplethysmography: proof of principle (Preprint). JMIR Form Res 2022; 7:e44575. [PMID: 36995742 PMCID: PMC10131655 DOI: 10.2196/44575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 02/08/2023] [Accepted: 02/08/2023] [Indexed: 02/10/2023] Open
Abstract
BACKGROUND Remote photoplethysmography (rPPG) can record vital signs (VSs) by detecting subtle changes in the light reflected from the skin. Lifelight (Xim Ltd) is a novel software being developed as a medical device for the contactless measurement of VSs using rPPG via integral cameras on smart devices. Research to date has focused on extracting the pulsatile VS from the raw signal, which can be influenced by factors such as ambient light, skin thickness, facial movements, and skin tone. OBJECTIVE This preliminary proof-of-concept study outlines a dynamic approach to rPPG signal processing wherein green channel signals from the most relevant areas of the face (the midface, comprising the cheeks, nose, and top of the lip) are optimized for each subject using tiling and aggregation (T&A) algorithms. METHODS High-resolution 60-second videos were recorded during the VISION-MD study. The midface was divided into 62 tiles of 20×20 pixels, and the signals from multiple tiles were evaluated using bespoke algorithms through weighting according to signal-to-noise ratio in the frequency domain (SNR-F) score or segmentation. Midface signals before and after T&A were categorized by a trained observer blinded to the data processing as 0 (high quality, suitable for algorithm training), 1 (suitable for algorithm testing), or 2 (inadequate quality). On secondary analysis, observer categories were compared for signals predicted to improve categories following T&A based on the SNR-F score. Observer ratings and SNR-F scores were also compared before and after T&A for Fitzpatrick skin tones 5 and 6, wherein rPPG is hampered by light absorption by melanin. RESULTS The analysis used 4310 videos recorded from 1315 participants. Category 2 and 1 signals had lower mean SNR-F scores than category 0 signals. T&A improved the mean SNR-F score using all algorithms. Depending on the algorithm, 18% (763/4212) to 31% (1306/4212) of signals improved by at least one category, with up to 10% (438/4212) improving into category 0, and 67% (2834/4212) to 79% (3337/4212) remaining in the same category. Importantly, 9% (396/4212) to 21% (875/4212) improved from category 2 (not usable) into category 1. All algorithms showed improvements. No more than 3% (137/4212) of signals were assigned to a lower-quality category following T&A. On secondary analysis, 62% of signals (32/52) were recategorized, as predicted from the SNR-F score. T&A improved SNR-F scores in darker skin tones; 41% of signals (151/369) improved from category 2 to 1 and 12% (44/369) from category 1 to 0. CONCLUSIONS The T&A approach to dynamic region of interest selection improved signal quality, including in dark skin tones. The method was verified by comparison with a trained observer's rating. T&A could overcome factors that compromise whole-face rPPG. This method's performance in estimating VS is currently being assessed. TRIAL REGISTRATION ClinicalTrials.gov NCT04763746; https://clinicaltrials.gov/ct2/show/NCT04763746.
Collapse
|
10
|
Kim DY, Cho SY, Lee K, Sohn CB. A Study of Projection-Based Attentive Spatial-Temporal Map for Remote Photoplethysmography Measurement. Bioengineering (Basel) 2022; 9:638. [PMID: 36354549 PMCID: PMC9687348 DOI: 10.3390/bioengineering9110638] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 10/21/2022] [Accepted: 10/28/2022] [Indexed: 08/27/2023] Open
Abstract
The photoplethysmography (PPG) signal contains various information that is related to CVD (cardiovascular disease). The remote PPG (rPPG) is a method that can measure a PPG signal using a face image taken with a camera, without a PPG device. Deep learning-based rPPG methods can be classified into three main categories. First, there is a 3D CNN approach that uses a facial image video as input, which focuses on the spatio-temporal changes in the facial video. The second approach is a method that uses a spatio-temporal map (STMap), and the video image is pre-processed using the point where it is easier to analyze changes in blood flow in time order. The last approach uses a preprocessing model with a dichromatic reflection model. This study proposed the concept of an axis projection network (APNET) that complements the drawbacks, in which the 3D CNN method requires significant memory; the STMap method requires a preprocessing method; and the dyschromatic reflection model (DRM) method does not learn long-term temporal characteristics. We also showed that the proposed APNET effectively reduced the network memory size, and that the low-frequency signal was observed in the inferred PPG signal, suggesting that it can provide meaningful results to the study when developing the rPPG algorithm.
Collapse
Affiliation(s)
- Dae-Yeol Kim
- AI Research Team, Tvstorm, Seoul 13875, Korea
- Department of Electronics and Communications Engineering, Kwangwoon University, Seoul 01897, Korea
| | - Soo-Young Cho
- Department of Information Contents, Kwangwoon University, Seoul 01897, Korea
| | | | - Chae-Bong Sohn
- Department of Electronics and Communications Engineering, Kwangwoon University, Seoul 01897, Korea
| |
Collapse
|
11
|
Man PK, Cheung KL, Sangsiri N, Shek WJ, Wong KL, Chin JW, Chan TT, So RHY. Blood Pressure Measurement: From Cuff-Based to Contactless Monitoring. Healthcare (Basel) 2022; 10:healthcare10102113. [PMID: 36292560 PMCID: PMC9601911 DOI: 10.3390/healthcare10102113] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 09/26/2022] [Accepted: 10/02/2022] [Indexed: 11/04/2022] Open
Abstract
Blood pressure (BP) determines whether a person has hypertension and offers implications as to whether he or she could be affected by cardiovascular disease. Cuff-based sphygmomanometers have traditionally provided both accuracy and reliability, but they require bulky equipment and relevant skills to obtain precise measurements. BP measurement from photoplethysmography (PPG) signals has become a promising alternative for convenient and unobtrusive BP monitoring. Moreover, the recent developments in remote photoplethysmography (rPPG) algorithms have enabled new innovations for contactless BP measurement. This paper illustrates the evolution of BP measurement techniques from the biophysical theory, through the development of contact-based BP measurement from PPG signals, and to the modern innovations of contactless BP measurement from rPPG signals. We consolidate knowledge from a diverse background of academic research to highlight the importance of multi-feature analysis for improving measurement accuracy. We conclude with the ongoing challenges, opportunities, and possible future directions in this emerging field of research.
Collapse
Affiliation(s)
- Ping-Kwan Man
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Correspondence:
| | - Kit-Leong Cheung
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| | - Nawapon Sangsiri
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| | - Wilfred Jin Shek
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Biomedical Sciences, King’s College London, London WC2R 2LS, UK
| | - Kwan-Long Wong
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
| | - Jing-Wei Chin
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
| | - Tsz-Tai Chan
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
| | - Richard Hau-Yue So
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
| |
Collapse
|
12
|
Haugg F, Elgendi M, Menon C. Effectiveness of Remote PPG Construction Methods: A Preliminary Analysis. Bioengineering (Basel) 2022; 9:bioengineering9100485. [PMID: 36290452 PMCID: PMC9598377 DOI: 10.3390/bioengineering9100485] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Revised: 09/05/2022] [Accepted: 09/09/2022] [Indexed: 11/24/2022] Open
Abstract
The contactless recording of a photoplethysmography (PPG) signal with a Red-Green-Blue (RGB) camera is known as remote photoplethysmography (rPPG). Studies have reported on the positive impact of using this technique, particularly in heart rate estimation, which has led to increased research on this topic among scientists. Therefore, converting from RGB signals to constructing an rPPG signal is an important step. Eight rPPG methods (plant-orthogonal-to-skin (POS), local group invariance (LGI), the chrominance-based method (CHROM), orthogonal matrix image transformation (OMIT), GREEN, independent component analysis (ICA), principal component analysis (PCA), and blood volume pulse (PBV) methods) were assessed using dynamic time warping, power spectrum analysis, and Pearson’s correlation coefficient, with different activities (at rest, during exercising in the gym, during talking, and while head rotating) and four regions of interest (ROI): the forehead, the left cheek, the right cheek, and a combination of all three ROIs. The best performing rPPG methods in all categories were the POS, LGI, and OMI methods; each performed well in all activities. Recommendations for future work are provided.
Collapse
Affiliation(s)
- Fridolin Haugg
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Department of Mechanical Engineering, Karlsruher Institute for Technology, 76131 Karlsruhe, Germany
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Correspondence:
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
| |
Collapse
|
13
|
Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism. SENSORS 2022; 22:s22020688. [PMID: 35062649 PMCID: PMC8781886 DOI: 10.3390/s22020688] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Revised: 01/12/2022] [Accepted: 01/14/2022] [Indexed: 12/04/2022]
Abstract
Remote photoplethysmography (rPPG) is a video-based non-contact heart rate measurement technology. It is a fact that most existing rPPG methods fail to deal with the spatiotemporal features of the video, which is significant for the extraction of the rPPG signal. In this paper, we propose a 3D central difference convolutional network (CDCA-rPPGNet) to measure heart rate, with an attention mechanism to combine spatial and temporal features. First, we crop and stitch the region of interest together through facial landmarks. Next, the high-quality regions of interest are fed to CDCA-rPPGNet based on a central difference convolution, which can enhance the spatiotemporal representation and capture rich relevant time contexts by collecting time difference information. In addition, we integrate the attention module into the neural network, aiming to strengthen the ability of the neural network to extract video channels and spatial features, so as to obtain more accurate rPPG signals. In summary, the three main contributions of this paper are as follows: (1) the proposed network base on central difference convolution could better capture the subtle color changes to recover the rPPG signals; (2) the proposed ROI extraction method provides high-quality input to the network; (3) the attention module is used to strengthen the ability of the network to extract features. Extensive experiments are conducted on two public datasets—the PURE dataset and the UBFC-rPPG dataset. In terms of the experiment results, our proposed method achieves 0.46 MAE (bpm), 0.90 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the PURE dataset, and 0.60 MAE (bpm), 1.38 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the UBFC dataset, which proves the effectiveness of our proposed approach.
Collapse
|