1
|
Chen W, Yi Z, Lim LJR, Lim RQR, Zhang A, Qian Z, Huang J, He J, Liu B. Deep learning and remote photoplethysmography powered advancements in contactless physiological measurement. Front Bioeng Biotechnol 2024; 12:1420100. [PMID: 39104628 PMCID: PMC11298756 DOI: 10.3389/fbioe.2024.1420100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Accepted: 06/27/2024] [Indexed: 08/07/2024] Open
Abstract
In recent decades, there has been ongoing development in the application of computer vision (CV) in the medical field. As conventional contact-based physiological measurement techniques often restrict a patient's mobility in the clinical environment, the ability to achieve continuous, comfortable and convenient monitoring is thus a topic of interest to researchers. One type of CV application is remote imaging photoplethysmography (rPPG), which can predict vital signs using a video or image. While contactless physiological measurement techniques have an excellent application prospect, the lack of uniformity or standardization of contactless vital monitoring methods limits their application in remote healthcare/telehealth settings. Several methods have been developed to improve this limitation and solve the heterogeneity of video signals caused by movement, lighting, and equipment. The fundamental algorithms include traditional algorithms with optimization and developing deep learning (DL) algorithms. This article aims to provide an in-depth review of current Artificial Intelligence (AI) methods using CV and DL in contactless physiological measurement and a comprehensive summary of the latest development of contactless measurement techniques for skin perfusion, respiratory rate, blood oxygen saturation, heart rate, heart rate variability, and blood pressure.
Collapse
Affiliation(s)
- Wei Chen
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
| | - Zhe Yi
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
| | - Lincoln Jian Rong Lim
- Department of Medical Imaging, Western Health, Footscray Hospital, Footscray, VIC, Australia
- Department of Surgery, The University of Melbourne, Melbourne, VIC, Australia
| | - Rebecca Qian Ru Lim
- Department of Hand & Reconstructive Microsurgery, Singapore General Hospital, Singapore, Singapore
| | - Aijie Zhang
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
| | - Zhen Qian
- Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging, Beijing, China
| | - Jiaxing Huang
- Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jia He
- Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Bo Liu
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
- Beijing Research Institute of Traumatology and Orthopaedics, Beijing, China
| |
Collapse
|
2
|
Zheng K, Kong J, Tian L, Li B, Li H, Zhou J. Hand-over-face occlusion and distance adaptive heart rate detection based on imaging photoplethysmography and pixel distance in online learning. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104898] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
|
3
|
Zhao L, Zhang M, Ding H, Cui X. Fine-grained deepfake detection based on cross-modality attention. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08271-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
|
4
|
Javaid A, Zghyer F, Kim C, Spaulding EM, Isakadze N, Ding J, Kargillis D, Gao Y, Rahman F, Brown DE, Saria S, Martin SS, Kramer CM, Blumenthal RS, Marvel FA. Medicine 2032: The future of cardiovascular disease prevention with machine learning and digital health technology. Am J Prev Cardiol 2022; 12:100379. [PMID: 36090536 PMCID: PMC9460561 DOI: 10.1016/j.ajpc.2022.100379] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 08/21/2022] [Accepted: 08/28/2022] [Indexed: 11/30/2022] Open
Abstract
Machine learning (ML) refers to computational algorithms that iteratively improve their ability to recognize patterns in data. The digitization of our healthcare infrastructure is generating an abundance of data from electronic health records, imaging, wearables, and sensors that can be analyzed by ML algorithms to generate personalized risk assessments and promote guideline-directed medical management. ML's strength in generating insights from complex medical data to guide clinical decisions must be balanced with the potential to adversely affect patient privacy, safety, health equity, and clinical interpretability. This review provides a primer on key advances in ML for cardiovascular disease prevention and how they may impact clinical practice.
Collapse
Affiliation(s)
- Aamir Javaid
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Fawzi Zghyer
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Chang Kim
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Erin M. Spaulding
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Nino Isakadze
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Jie Ding
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Daniel Kargillis
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Yumin Gao
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Faisal Rahman
- Division of Cardiology, Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Donald E. Brown
- School of Data Science, University of Virginia, Charlottesville, VA, USA
| | - Suchi Saria
- Machine Learning and Healthcare Laboratory, Departments of Computer Science, Statistics, and Health Policy, Malone Center for Engineering in Healthcare, and Armstrong Institute for Patient Safety and Quality, Johns Hopkins University, Baltimore, MD, USA
| | - Seth S. Martin
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Christopher M. Kramer
- Cardiovascular Division, Department of Medicine, University of Virginia Health, Charlottesville, VA, USA
| | - Roger S. Blumenthal
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| | - Francoise A. Marvel
- Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease, Division of Cardiology, Department of Medicine, Johns Hopkins University School of Medicine, 600 N. Wolfe St, Carnegie 591, Baltimore, MD 21287, USA
| |
Collapse
|
5
|
Jaiswal KB, Meenpal T. rPPG-FuseNet: Non-contact heart rate estimation from facial video via RGB/MSR signal fusion. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.104002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
6
|
Shahzad HF, Rustam F, Flores ES, Luís Vidal Mazón J, de la Torre Diez I, Ashraf I. A Review of Image Processing Techniques for Deepfakes. SENSORS (BASEL, SWITZERLAND) 2022; 22:4556. [PMID: 35746333 PMCID: PMC9230855 DOI: 10.3390/s22124556] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 06/09/2022] [Accepted: 06/13/2022] [Indexed: 02/06/2023]
Abstract
Deep learning is used to address a wide range of challenging issues including large data analysis, image processing, object detection, and autonomous control. In the same way, deep learning techniques are also used to develop software and techniques that pose a danger to privacy, democracy, and national security. Fake content in the form of images and videos using digital manipulation with artificial intelligence (AI) approaches has become widespread during the past few years. Deepfakes, in the form of audio, images, and videos, have become a major concern during the past few years. Complemented by artificial intelligence, deepfakes swap the face of one person with the other and generate hyper-realistic videos. Accompanying the speed of social media, deepfakes can immediately reach millions of people and can be very dangerous to make fake news, hoaxes, and fraud. Besides the well-known movie stars, politicians have been victims of deepfakes in the past, especially US presidents Barak Obama and Donald Trump, however, the public at large can be the target of deepfakes. To overcome the challenge of deepfake identification and mitigate its impact, large efforts have been carried out to devise novel methods to detect face manipulation. This study also discusses how to counter the threats from deepfake technology and alleviate its impact. The outcomes recommend that despite a serious threat to society, business, and political institutions, they can be combated through appropriate policies, regulation, individual actions, training, and education. In addition, the evolution of technology is desired for deepfake identification, content authentication, and deepfake prevention. Different studies have performed deepfake detection using machine learning and deep learning techniques such as support vector machine, random forest, multilayer perceptron, k-nearest neighbors, convolutional neural networks with and without long short-term memory, and other similar models. This study aims to highlight the recent research in deepfake images and video detection, such as deepfake creation, various detection algorithms on self-made datasets, and existing benchmark datasets.
Collapse
Affiliation(s)
- Hina Fatima Shahzad
- Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan;
| | - Furqan Rustam
- Department of Software Engineering, University of Management and Technology, Lahore 544700, Pakistan;
| | - Emmanuel Soriano Flores
- Higher Polytechnic School, Universidad Europea del Atlántico (UNEATLANTICO), Isabel Torres 21, 39011 Santander, Spain; (E.S.F.); (J.L.V.M.)
- Department of Project Management, Universidad Internacional Iberoamericana (UNINI-MX), Campeche 24560, Mexico
| | - Juan Luís Vidal Mazón
- Higher Polytechnic School, Universidad Europea del Atlántico (UNEATLANTICO), Isabel Torres 21, 39011 Santander, Spain; (E.S.F.); (J.L.V.M.)
- Project Department, Universidade Internacional do Cuanza, Municipio do Kuito, Bairro Sede, EN250, Bié, Angola
| | - Isabel de la Torre Diez
- Department of Signal Theory and Communications and Telematic Engineering, University of Valladolid, Paseo de Belén 15, 47011 Valladolid, Spain
| | - Imran Ashraf
- Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 38541, Korea
| |
Collapse
|
7
|
Selvaraju V, Spicher N, Wang J, Ganapathy N, Warnecke JM, Leonhardt S, Swaminathan R, Deserno TM. Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:4097. [PMID: 35684717 PMCID: PMC9185528 DOI: 10.3390/s22114097] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 05/18/2022] [Accepted: 05/18/2022] [Indexed: 02/04/2023]
Abstract
In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.
Collapse
Affiliation(s)
- Vinothini Selvaraju
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
- Non-Invasive Imaging and Diagnostic Laboratory, Biomedical Engineering, Department of Applied Mechanics, Indian Institute of Technology Madras, Chennai 600036, India;
| | - Nicolai Spicher
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Ju Wang
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Nagarajan Ganapathy
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Joana M. Warnecke
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Steffen Leonhardt
- Chair for Medical Information Technology, Helmholtz-Institute for Biomedical Engineering, RWTH Aachen University, D-52074 Aachen, Germany;
| | - Ramakrishnan Swaminathan
- Non-Invasive Imaging and Diagnostic Laboratory, Biomedical Engineering, Department of Applied Mechanics, Indian Institute of Technology Madras, Chennai 600036, India;
| | - Thomas M. Deserno
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| |
Collapse
|
8
|
Zheng K, Shen J, Sun G, Li H, Li Y. Shielding facial physiological information in video. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2022; 19:5153-5168. [PMID: 35430858 DOI: 10.3934/mbe.2022241] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
With the recent development of non-contact physiological signal detection methods based on videos, it is possible to obtain the physiological parameters through the ordinary video only, such as heart rate and its variability of an individual. Therefore, personal physiological information may be leaked unknowingly with the spread of videos, which may cause privacy or security problems. In this paper a new method is proposed, which can shield physiological information in the video without reducing the video quality significantly. Firstly, the principle of the most widely used physiological signal detection algorithm: remote photoplethysmography (rPPG) was analyzed. Then the region of interest (ROI) of face contain physiological information with high signal to noise ratio was selected. Two physiological information forgery operation: single-channel periodic noise addition with blur filtering and brightness fine-tuning are conducted on the ROIs. Finally, the processed ROI images are merged into video frames to obtain the processed video. Experiments were performed on the VIPL-HR video dataset. The interference efficiencies of the proposed method on two mainly used rPPG methods: Independent Component Analysis (ICA) and Chrominance-based Method (CHROM) are 82.9 % and 84.6 % respectively, which demonstrated the effectiveness of the proposed method.
Collapse
Affiliation(s)
- Kun Zheng
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Junjie Shen
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Guangmin Sun
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Hui Li
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Yu Li
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| |
Collapse
|
9
|
Evaluation of biases in remote photoplethysmography methods. NPJ Digit Med 2021; 4:91. [PMID: 34083724 PMCID: PMC8175478 DOI: 10.1038/s41746-021-00462-z] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Accepted: 05/03/2021] [Indexed: 12/21/2022] Open
Abstract
This work investigates the estimation biases of remote photoplethysmography (rPPG) methods for pulse rate measurement across diverse demographics. Advances in photoplethysmography (PPG) and rPPG methods have enabled the development of contact and noncontact approaches for continuous monitoring and collection of patient health data. The contagious nature of viruses such as COVID-19 warrants noncontact methods for physiological signal estimation. However, these approaches are subject to estimation biases due to variations in environmental conditions and subject demographics. The performance of contact-based wearable sensors has been evaluated, using off-the-shelf devices across demographics. However, the measurement uncertainty of rPPG methods that estimate pulse rate has not been sufficiently tested across diverse demographic populations or environments. Quantifying the efficacy of rPPG methods in real-world conditions is critical in determining their potential viability as health monitoring solutions. Currently, publicly available face datasets accompanied by physiological measurements are typically captured in controlled laboratory settings, lacking diversity in subject skin tones, age, and cultural artifacts (e.g, bindi worn by Indian women). In this study, we collect pulse rate and facial video data from human subjects in India and Sierra Leone, in order to quantify the uncertainty in noncontact pulse rate estimation methods. The video data are used to estimate pulse rate using state-of-the-art rPPG camera-based methods, and compared against ground truth measurements captured using an FDA-approved contact-based pulse rate measurement device. Our study reveals that rPPG methods exhibit similar biases when compared with a contact-based device across demographic groups and environmental conditions. The mean difference between pulse rates measured by rPPG methods and the ground truth is found to be ~2% (1 beats per minute (b.p.m.)), signifying agreement of rPPG methods with the ground truth. We also find that rPPG methods show pulse rate variability of ~15% (11 b.p.m.), as compared to the ground truth. We investigate factors impacting rPPG methods and discuss solutions aimed at mitigating variance.
Collapse
|
10
|
Affiliation(s)
- Peipeng Yu
- Engineering Research Centre of Digital Forensics Ministry of Education School of Computer and Software Jiangsu Engineering Centre of Network Monitoring Jiangsu Collaborative Innovation Centre on Atmospheric Environment and Equipment Technology Nanjing University of Information Science & Technology Jiangsu Province Nanjing China
| | - Zhihua Xia
- College of Cyber Security, Jinan University Guangzhou China
- Engineering Research Center of Digital Forensics, Nanjing University of Information Science & Technology Nanjing China
| | - Jianwei Fei
- Engineering Research Centre of Digital Forensics Ministry of Education School of Computer and Software Jiangsu Engineering Centre of Network Monitoring Jiangsu Collaborative Innovation Centre on Atmospheric Environment and Equipment Technology Nanjing University of Information Science & Technology Jiangsu Province Nanjing China
| | - Yujiang Lu
- Engineering Research Centre of Digital Forensics Ministry of Education School of Computer and Software Jiangsu Engineering Centre of Network Monitoring Jiangsu Collaborative Innovation Centre on Atmospheric Environment and Equipment Technology Nanjing University of Information Science & Technology Jiangsu Province Nanjing China
| |
Collapse
|
11
|
Shoushan MM, Reyes BA, Rodriguez AM, Chong JW. Non-Contact HR Monitoring via Smartphone and Webcam During Different Respiratory Maneuvers and Body Movements. IEEE J Biomed Health Inform 2021; 25:602-612. [PMID: 32750916 DOI: 10.1109/jbhi.2020.2998399] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
As a reliable indicator for individual's healthiness conditions, heart rate (HR) has been widely considered and used. Imaging photoplethysmography (iPPG) is recently highlighted as a promising HR measurement method, due to its non-contact characteristics, by extracting the HR from facial video recordings. In this study, we propose a camera-based HR monitoring technique that estimates HR information from iPPG signals extracted from a video sequence. Videos were recorded using a smartphone or a laptop camera. We adopted the plane-orthogonal-to-skin (POS) method to compute iPPG. The proposed method is evaluated by applying it to extract HR of 9 subjects at rest and during two motion conditions (lateral and frontal) while they were performing several respiratory maneuvers-spontaneous, metronome, and forced. Automatic face detection algorithms were implemented in the proposed method. Our experimental results show that mean values of HR have 0.56% error and 99.4% accuracy when compared to HR calculated from the gold-standard electrocardiography (ECG) reference in diverse conditions of motions and respiratory maneuvers.
Collapse
|
12
|
Non-Contact Heart Rate Detection When Face Information Is Missing during Online Learning. SENSORS 2020; 20:s20247021. [PMID: 33302477 PMCID: PMC7763013 DOI: 10.3390/s20247021] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Revised: 12/04/2020] [Accepted: 12/05/2020] [Indexed: 11/17/2022]
Abstract
Research shows that physiological signals can provide objective data support for the analysis of human emotions. At present, non-contact heart rate data have been employed in the research of medicine, intelligent transportation, smart education, etc. However, it is hard to detect heart rate data using non-contact traditional methods during head rotation, especially when face information is missing in scenarios such as online teaching/learning. Traditional remote photoplethysmography (rPPG) methods require a static, full frontal face within a fixed distance for heart rate detection. These strict requirements make it impractical to measure heart rate data in real-world scenarios, as a lot of videos only partially record the subjects' face information, such as profile, too small distance, and wearing a mask. The current algorithm aims to solve the problem of head deflections between 30 degrees and 45 degrees by employing a symmetry substitution method, which can replace the undetected region of interest (ROI) with the detectable one. When face information is partially missing, our algorithm uses face-eye location to determine ROI. The results show that the method in this paper can solve certain practical problems related to heart rate detection, with a root mean square error (RMSE) under 7.64 bpm.
Collapse
|
13
|
Mamontov OV, Krasnikova TV, Volynsky MA, Anokhina NA, Shlyakhto EV, Kamshilin AA. Novel instrumental markers of proximal scleroderma provided by imaging photoplethysmography. Physiol Meas 2020; 41:044004. [DOI: 10.1088/1361-6579/ab807c] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
|
14
|
Li F, Zhao Y, Kong L, Dong L, Liu M, Hui M, Liu X. A camera-based ballistocardiogram heart rate measurement method. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2020; 91:054105. [PMID: 32486732 DOI: 10.1063/1.5128806] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Accepted: 04/09/2020] [Indexed: 06/11/2023]
Abstract
Recent studies have shown that head movements associated with cardiac activity contain a heart rate (HR) signal. In most previous studies, subjects were required to remain stationary in a specific environment during HR measurements, and measurement accuracy depended on the choice of target in the scene, i.e., the specified region of the face. In this paper, we proposed a robust HR measurement method based on ballistocardiogram (BCG) technology. This method requires only a camera and does not require that users establish a complex measurement environment. In addition, a bidirectional optical flow algorithm is designed to select and track valid feature points in the video captured by using the camera. Experiments with 11 subjects show that the HR values measured using the proposed method differ slightly from the reference values, and the average error is only 1.09%. Overall, this method can improve the accuracy of BCG without limitations related to skin tone, illumination, the state of the subject, or the test location.
Collapse
Affiliation(s)
- Fen Li
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Yuejin Zhao
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Lingqin Kong
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Liquan Dong
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Ming Liu
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Mei Hui
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Xiaohua Liu
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| |
Collapse
|
15
|
Soon S, Svavarsdottir H, Downey C, Jayne DG. Wearable devices for remote vital signs monitoring in the outpatient setting: an overview of the field. ACTA ACUST UNITED AC 2020. [DOI: 10.1136/bmjinnov-2019-000354] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
Early detection of physiological deterioration has been shown to improve patient outcomes. Due to recent improvements in technology, comprehensive outpatient vital signs monitoring is now possible. This is the first review to collate information on all wearable devices on the market for outpatient physiological monitoring.A scoping review was undertaken. The monitors reviewed were limited to those that can function in the outpatient setting with minimal restrictions on the patient’s normal lifestyle, while measuring any or all of the vital signs: heart rate, ECG, oxygen saturation, respiration rate, blood pressure and temperature.A total of 270 papers were included in the review. Thirty wearable monitors were examined: 6 patches, 3 clothing-based monitors, 4 chest straps, 2 upper arm bands and 15 wristbands. The monitoring of vital signs in the outpatient setting is a developing field with differing levels of evidence for each monitor. The most common clinical application was heart rate monitoring. Blood pressure and oxygen saturation measurements were the least common applications. There is a need for clinical validation studies in the outpatient setting to prove the potential of many of the monitors identified.Research in this area is in its infancy. Future research should look at aggregating the results of validity and reliability and patient outcome studies for each monitor and between different devices. This would provide a more holistic overview of the potential for the clinical use of each device.
Collapse
|
16
|
Qi L, Yu H, Xu L, Mpanda RS, Greenwald SE. Robust heart-rate estimation from facial videos using Project_ICA. Physiol Meas 2019; 40:085007. [PMID: 31479423 DOI: 10.1088/1361-6579/ab2c9f] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Remote photoplethysmography (rPPG) can achieve non-contact measurement of heart rate (HR) from a continuous video sequence by scanning the skin surface. However, practical applications are still limited by factors such as non-rigid facial motion and head movement. In this work, a detailed system framework for remotely estimating heart rate from facial videos under various movement conditions is described. APPROACH After the rPPG signal has been obtained from a defined region of the facial skin, a method, termed 'Project_ICA', based on a skin reflection model, is employed to extract the pulse signal from the original signal. MAIN RESULTS To evaluate the performance of the proposed algorithm, a dataset containing 112 videos including the challenges of various skin tones, body motion and HR recovery after exercise was created from 28 participants. SIGNIFICANCE The results show that Project_ICA, when evaluated by several criteria, provides a more accurate and robust estimate of HR than most existing methods, although problems remain in obtaining reliable measurements from dark-skinned subjects.
Collapse
Affiliation(s)
- Lin Qi
- Sino-Dutch Biomedical and Information Engineering School, Northeastern University, Shenyang, Liaoning, People's Republic of China
| | | | | | | | | |
Collapse
|