1
|
Quigley KS, Gianaros PJ, Norman GJ, Jennings JR, Berntson GG, de Geus EJC. Publication guidelines for human heart rate and heart rate variability studies in psychophysiology-Part 1: Physiological underpinnings and foundations of measurement. Psychophysiology 2024; 61:e14604. [PMID: 38873876 DOI: 10.1111/psyp.14604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 12/22/2023] [Accepted: 04/04/2024] [Indexed: 06/15/2024]
Abstract
This Committee Report provides methodological, interpretive, and reporting guidance for researchers who use measures of heart rate (HR) and heart rate variability (HRV) in psychophysiological research. We provide brief summaries of best practices in measuring HR and HRV via electrocardiographic and photoplethysmographic signals in laboratory, field (ambulatory), and brain-imaging contexts to address research questions incorporating measures of HR and HRV. The Report emphasizes evidence for the strengths and weaknesses of different recording and derivation methods for measures of HR and HRV. Along with this guidance, the Report reviews what is known about the origin of the heartbeat and its neural control, including factors that produce and influence HRV metrics. The Report concludes with checklists to guide authors in study design and analysis considerations, as well as guidance on the reporting of key methodological details and characteristics of the samples under study. It is expected that rigorous and transparent recording and reporting of HR and HRV measures will strengthen inferences across the many applications of these metrics in psychophysiology. The prior Committee Reports on HR and HRV are several decades old. Since their appearance, technologies for human cardiac and vascular monitoring in laboratory and daily life (i.e., ambulatory) contexts have greatly expanded. This Committee Report was prepared for the Society for Psychophysiological Research to provide updated methodological and interpretive guidance, as well as to summarize best practices for reporting HR and HRV studies in humans.
Collapse
Affiliation(s)
- Karen S Quigley
- Department of Psychology, Northeastern University, Boston, Massachusetts, USA
| | - Peter J Gianaros
- Department of Psychology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Greg J Norman
- Department of Psychology, The University of Chicago, Chicago, Illinois, USA
| | - J Richard Jennings
- Department of Psychiatry & Psychology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gary G Berntson
- Department of Psychology & Psychiatry, The Ohio State University, Columbus, Ohio, USA
| | - Eco J C de Geus
- Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
2
|
Khanam FTZ, Perera AG, Al-Naji A, Mcintyre TD, Chahl J. Integrating RGB-thermal image sensors for non-contact automatic respiration rate monitoring. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2024; 41:1140-1151. [PMID: 38856428 DOI: 10.1364/josaa.520757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Accepted: 04/23/2024] [Indexed: 06/11/2024]
Abstract
Respiration rate (RR) holds significance as a human health indicator. Presently, the conventional RR monitoring system requires direct physical contact, which may cause discomfort and pain. Therefore, this paper proposes a non-contact RR monitoring system integrating RGB and thermal imaging through RGB-thermal image alignment. The proposed method employs an advanced image processing algorithm for automatic region of interest (ROI) selection. The experimental results demonstrated a close correlation and a lower error rate between measured thermal, measured RGB, and reference data. In summary, the proposed non-contact system emerges as a promising alternative to conventional contact-based approaches without the associated discomfort and pain.
Collapse
|
3
|
Song K, Hirose K, Niitsu K, Sui T, Kojima H, Fujie T, Umezu S. A combination of logical judging circuit and water-resistant ultrathin film PEDOT: PSS electrode for noninvasive ECG measurement. DISCOVER NANO 2024; 19:45. [PMID: 38483679 PMCID: PMC10940549 DOI: 10.1186/s11671-024-03988-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 03/06/2024] [Indexed: 03/17/2024]
Abstract
Heart disease-related deaths have increased in recent decades, with most patients dying of sudden cardiac arrest. In such instances, the effect of regular electrocardiogram (ECG) measurements is minimal. Therefore, long-term ECG monitoring has become increasingly important. In this paper, we report a non-adhesive high accuracy ECG monitoring system that can be used in various scenarios without interfering with daily activities. The ECG ultra-thin film electrode is made by water-resistant material based on poly(3,4-ethylenedioxythiophene) poly(4-styrenesulfonate) (PEDOT: PSS) electrode doped with ethylene glycol (EG) and xylitol, to improve the noise signal caused by sweat. The optimal ratio of the three ingredients of PEDOT: PSS/xylitol/EG was determined experimentally to accommodate the ECG monitoring. By using the proposed selectively closed multi-channel single-lead logic circuit, the noise of ECG signal received from the proposed film electrode can be successfully reduced during broad-area electrode measurements, thus to improve ECG measurement accuracy.
Collapse
Affiliation(s)
- Kewei Song
- Department of Modern Mechanical Engineering, Graduate School of Creative Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan
| | - Kayo Hirose
- Anesthesiology and Pain Relief Center, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Kioto Niitsu
- Department of Integrative Bioscience and Biomedical Engineering, Graduate School of Advanced Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan
| | - Tsubasa Sui
- Department of Modern Mechanical Engineering, Graduate School of Creative Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan
| | - Hiroto Kojima
- Department of Integrative Bioscience and Biomedical Engineering, Graduate School of Advanced Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan
| | - Toshinori Fujie
- School of Life Science and Technology, Tokyo Institute of Technology, B-50, 4259 Nagatsuta-Cho, Midori-ku, Yokohama, 226-8501, Japan.
| | - Shinjiro Umezu
- Department of Modern Mechanical Engineering, Graduate School of Creative Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan.
- Department of Integrative Bioscience and Biomedical Engineering, Graduate School of Advanced Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan.
- Department of Modern Mechanical Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555, Japan.
| |
Collapse
|
4
|
Yuan Z, Lu S, He Y, Liu X, Fang J. Nmr-VSM: Non-Touch Motion-Robust Vital Sign Monitoring via UWB Radar Based on Deep Learning. MICROMACHINES 2023; 14:1479. [PMID: 37512790 PMCID: PMC10386750 DOI: 10.3390/mi14071479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 07/03/2023] [Accepted: 07/20/2023] [Indexed: 07/30/2023]
Abstract
In recent years, biometric radar has gained increasing attention in the field of non-touch vital sign monitoring due to its high accuracy and strong ability to detect fine-grained movements. However, most current research on biometric radar can only achieve heart rate or respiration rate monitoring in static environments, which have strict monitoring requirements and single monitoring parameters. Moreover, most studies have not applied the collected data despite their significant potential for applications. In this paper, we introduce a non-touch motion-robust vital sign monitoring system via ultra-wideband (UWB) radar based on deep learning. Nmr-VSM not only enables multi-dimensional vital sign monitoring under human motion environments but also implements cardiac anomaly detection. The design of Nmr-VSM includes three key components. Firstly, we design a UWB radar that can perform multi-dimensional vital sign monitoring, including heart rate, respiratory rate, distance, and motion status. Secondly, we collect real experimental data and analyze the impact of eight factors, such as motion status and distance, on heart rate monitoring. We then propose a deep neural network (DNN)-based heart rate data correction model that achieves high robustness in motion environments. Finally, we model the heart rate variability (HRV) of the human body and propose a convolutional neural network (CNN)-based anomaly detection model that achieves low-latency detection of heart diseases, such as ventricular tachycardia and ventricular fibrillation. Experimental results in a real environment demonstrate that Nmr-VSM can not only accurately monitor heart rate but also achieve anomaly detection with low latency.
Collapse
Affiliation(s)
- Zhonghang Yuan
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Shuaibing Lu
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Yi He
- School of Software Engineering, Beijing Jiaotong University, Beijing 100091, China
| | - Xuetao Liu
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Juan Fang
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| |
Collapse
|
5
|
Tan C, Xiao C, Wang W. Camera-based Cardiovascular Screening based on Heart Rate and Its Variability In Pre- and Post-Exercise Conditions. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-5. [PMID: 38083672 DOI: 10.1109/embc40787.2023.10340871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Analysis of heart rate (HR) and heart rate variability (HRV) in pre- and post-exercise conditions can provide useful information about the health condition of cardiovascular system. Remote photoplethysmography (rPPG) that uses a contactless camera to measure vital signs from the human face might allow for ubiquitous applications in the fitness scenario. This paper benchmarks the accuracy of HR and HRV measured by ECG, PPG and rPPG, and investigates the difference of HR and HRV parameters between individuals with/without the exercise habit in pre- and post-exercise phases. We built a fitness benchmark consisting of 14 healthy subjects to perform running exercise on a treadmill, with video and reference data recorded simultaneously. The results show that rPPG has a similar performance as PPG in both the estimation of HR and HRV. The HRV parameters (Mean IBI, SDNN, LF, VLF, and SD2) of rPPG/PPG show good agreements with the ECG reference. Subjects with a regular exercise habit show lower resting HR and smaller values of HRV parameters in pre-exercise phase, fewer changes in HR and HRV at the same exercise intensity, and faster HR recovery after exercising, as compared with those without an exercise habit. The preliminary results suggest that rPPG is a promising surrogate of PPG for screening HR and HRV before and after exercise, to indicate the performance of cardiac training.
Collapse
|
6
|
Li H, Li L, Zhang H, Wei J, Xu Z, Chen T. Cell Differentiation-Inspired, Salt-Induced Multifunctional Gels for an Intelligent Soft Robot with an Artificial Reflex Arc. ACS APPLIED MATERIALS & INTERFACES 2023; 15:5910-5920. [PMID: 36657404 DOI: 10.1021/acsami.2c20172] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
To make soft robotics intelligent, dazzling artificial skin and actuators have been created. However, compared to rigid commercial robots, the sophisticated demands of raw materials become a key challenge for autonomous soft actuators to realize manufacturing repeatability and reproducibility. Inspired by the stem cell, which has the potential to differentiate into multifunctional cells with the same original compositions, a potential multifunctional gel is presented. With well-designed polymer chains, the gel has a salt-induced regulating module, conductivity, and other bionic properties. Making use of this advantage, the gel acts as a double-duty electrode for both the actuator and sensor. An artificial reflex arc is therefore formed by their tight integration via an e-brain: a computing unit that specifically responds to organism intervention. This efficient strategy to obtain diverse components with minimal raw materials is promising for effortlessly fabricating fully soft robotics.
Collapse
Affiliation(s)
- Huijing Li
- Key Laboratory of Marine Materials and Related Technologies, Zhejiang Key Laboratory of Marine Materials and Protective Technologies, Ningbo Institute of Material Technology and Engineering, Chinese Academy of Sciences, Ningbo315201, China
- School of Chemical Science, University of Chinese Academy of Sciences, 19A Yuquan Road, Beijing100049, China
| | - Long Li
- Key Laboratory of Marine Materials and Related Technologies, Zhejiang Key Laboratory of Marine Materials and Protective Technologies, Ningbo Institute of Material Technology and Engineering, Chinese Academy of Sciences, Ningbo315201, China
| | - Hao Zhang
- School of Chemical Science, University of Chinese Academy of Sciences, 19A Yuquan Road, Beijing100049, China
| | - Junjie Wei
- Key Laboratory of Marine Materials and Related Technologies, Zhejiang Key Laboratory of Marine Materials and Protective Technologies, Ningbo Institute of Material Technology and Engineering, Chinese Academy of Sciences, Ningbo315201, China
- School of Chemical Science, University of Chinese Academy of Sciences, 19A Yuquan Road, Beijing100049, China
| | - Zhenyu Xu
- Key Laboratory of Marine Materials and Related Technologies, Zhejiang Key Laboratory of Marine Materials and Protective Technologies, Ningbo Institute of Material Technology and Engineering, Chinese Academy of Sciences, Ningbo315201, China
- School of Chemical Science, University of Chinese Academy of Sciences, 19A Yuquan Road, Beijing100049, China
| | - Tao Chen
- Key Laboratory of Marine Materials and Related Technologies, Zhejiang Key Laboratory of Marine Materials and Protective Technologies, Ningbo Institute of Material Technology and Engineering, Chinese Academy of Sciences, Ningbo315201, China
- School of Chemical Science, University of Chinese Academy of Sciences, 19A Yuquan Road, Beijing100049, China
| |
Collapse
|
7
|
di Biase L, Pecoraro PM, Pecoraro G, Caminiti ML, Di Lazzaro V. Markerless Radio Frequency Indoor Monitoring for Telemedicine: Gait Analysis, Indoor Positioning, Fall Detection, Tremor Analysis, Vital Signs and Sleep Monitoring. SENSORS (BASEL, SWITZERLAND) 2022; 22:8486. [PMID: 36366187 PMCID: PMC9656920 DOI: 10.3390/s22218486] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Revised: 10/20/2022] [Accepted: 11/01/2022] [Indexed: 06/16/2023]
Abstract
Quantitative indoor monitoring, in a low-invasive and accurate way, is still an unmet need in clinical practice. Indoor environments are more challenging than outdoor environments, and are where patients experience difficulty in performing activities of daily living (ADLs). In line with the recent trends of telemedicine, there is an ongoing positive impulse in moving medical assistance and management from hospitals to home settings. Different technologies have been proposed for indoor monitoring over the past decades, with different degrees of invasiveness, complexity, and capabilities in full-body monitoring. The major classes of devices proposed are inertial-based sensors (IMU), vision-based devices, and geomagnetic and radiofrequency (RF) based sensors. In recent years, among all available technologies, there has been an increasing interest in using RF-based technology because it can provide a more accurate and reliable method of tracking patients' movements compared to other methods, such as camera-based systems or wearable sensors. Indeed, RF technology compared to the other two techniques has higher compliance, low energy consumption, does not need to be worn, is less susceptible to noise, is not affected by lighting or other physical obstacles, has a high temporal resolution without a limited angle of view, and fewer privacy issues. The aim of the present narrative review was to describe the potential applications of RF-based indoor monitoring techniques and highlight their differences compared to other monitoring technologies.
Collapse
Affiliation(s)
- Lazzaro di Biase
- Research Unit of Neurology, Neurophysiology and Neurobiology, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Via Alvaro del Portillo 21, 00128 Rome, Italy
- Operative Research Unit of Neurology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128 Rome, Italy
- Brain Innovations Lab, Università Campus Bio-Medico di Roma, Via Álvaro del Portillo 21, 00128 Rome, Italy
| | - Pasquale Maria Pecoraro
- Research Unit of Neurology, Neurophysiology and Neurobiology, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Via Alvaro del Portillo 21, 00128 Rome, Italy
- Operative Research Unit of Neurology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128 Rome, Italy
| | - Giovanni Pecoraro
- Department of Electronics Engineering, University of Rome Tor Vergata, 00133 Rome, Italy
| | - Maria Letizia Caminiti
- Research Unit of Neurology, Neurophysiology and Neurobiology, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Via Alvaro del Portillo 21, 00128 Rome, Italy
- Operative Research Unit of Neurology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128 Rome, Italy
| | - Vincenzo Di Lazzaro
- Research Unit of Neurology, Neurophysiology and Neurobiology, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Via Alvaro del Portillo 21, 00128 Rome, Italy
- Operative Research Unit of Neurology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128 Rome, Italy
| |
Collapse
|
8
|
Sharma P, Zhang Z, Conroy TB, Hui X, Kan EC. Attention Detection by Heartbeat and Respiratory Features from Radio-Frequency Sensor. SENSORS (BASEL, SWITZERLAND) 2022; 22:8047. [PMID: 36298396 PMCID: PMC9610852 DOI: 10.3390/s22208047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 10/13/2022] [Accepted: 10/17/2022] [Indexed: 06/16/2023]
Abstract
This work presents a study on users' attention detection with reference to a relaxed inattentive state using an over-the-clothes radio-frequency (RF) sensor. This sensor couples strongly to the internal heart, lung, and diaphragm motion based on the RF near-field coherent sensing principle, without requiring a tension chest belt or skin-contact electrocardiogram. We use cardiac and respiratory features to distinguish attention-engaging vigilance tasks from a relaxed, inattentive baseline state. We demonstrate high-quality vitals from the RF sensor compared to the reference electrocardiogram and respiratory tension belts, as well as similar performance for attention detection, while improving user comfort. Furthermore, we observed a higher vigilance-attention detection accuracy using respiratory features rather than heartbeat features. A high influence of the user's baseline emotional and arousal levels on the learning model was noted; thus, individual models with personalized prediction were designed for the 20 participants, leading to an average accuracy of 83.2% over unseen test data with a high sensitivity and specificity of 85.0% and 79.8%, respectively.
Collapse
|
9
|
Matsumoto T, Okumura S, Hirata S. Non-contact respiratory measurement in a horse in standing position using millimeter-wave array radar. J Vet Med Sci 2022; 84:1340-1344. [PMID: 35944981 PMCID: PMC9586033 DOI: 10.1292/jvms.22-0238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
This study aimed to apply radar technology to a large quadruped animal. We first
developed a non-contact respiration measurement system using millimeter-wave array radar
for a horse in standing position. Specifically, we measured the respiration of a
stationary domestic horse in stables. Simultaneously, we measured the respiration rate
using infrared thermography and developed a method for analyzing the radar information
while verifying the rate of agreement. Our results suggested that the radar technology
detected breathing and accurately measured the respiration of a horse, despite variation
in the breathing frequency. To the best of our knowledge, this is the first study to apply
a non-contact respiration measurement system using millimeter-wave array radar has been
applied to large animals in an upright position, thereby demonstrating its potential
application in animal husbandry and welfare.
Collapse
Affiliation(s)
- Takuya Matsumoto
- Faculty of Science, Shinshu University.,Research Institute for Humanity and Nature
| | | | | |
Collapse
|
10
|
Gupta A, Ravelo-García AG, Dias FM. Availability and performance of face based non-contact methods for heart rate and oxygen saturation estimations: A systematic review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106771. [PMID: 35390724 DOI: 10.1016/j.cmpb.2022.106771] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 03/03/2022] [Accepted: 03/21/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND Consumer-level cameras have provided an advantage of designing cost-effective, non-contact physiological parameters estimation approaches which is not possible with gold standard estimation techniques. This encourages the development of non-contact estimation methods using camera technology. Therefore, this work aims to present a systematic review summarizing the currently existing face-based non-contact methods along with their performance. METHODS This review includes all heart rate (HR) and oxygen saturation (SpO2) studies published in journals and a few reputed conferences, which have compared the proposed estimation methods with one or more standard reference devices. The articles were collected from the following research databases: Institute of Electrical and Electronics Engineers (IEEE), PubMed, Web of Science (WoS), Science Direct, and Association of Computer Machinery (ACM) digital library. All database searches were completed on May 20, 2021. Each study was assessed using a finite set of identified factors for reporting bias. RESULTS Out of 332 identified studies, 32 studies were selected for the final review. Additionally, 18 studies were included by thoroughly checking these studies. 3 out of 50 (6%) studies were performed in clinical conditions, while the remaining studies were carried out on a healthy population. 42 out of 50 (84%) studies have estimated HR, while 5/50 (10%) studies have measured SpO2 only. The remaining three studies have estimated both parameters. The majority of the studies have used 1-3 min videos for estimation. Among the estimation methods, Deep Learning and Independent component analysis (ICA) were used by 11/42 (26.19%) and 9/42 (21.42%) studies, respectively. According to the Bland-Altman analysis, only 8/45 (17.77%) HR studies achieved the clinically accepted error limits whereas, for SpO2, 4/5 (80%) studies have matched the industry standards (±3%). DISCUSSION Deep Learning and ICA have been predominantly used for HR estimations. Among deep learning estimation methods, convolutional neural networks have been employed till date due to their good generalization ability. Most non-contact HR estimation methods need significant improvements to implement these methods in a clinical environment. Furthermore, these methods need to be tested on the subjects suffering from any related disease. SpO2 estimation studies are challenging and need to be tested by conducting hypoxemic events. The authors would encourage reporting the detailed information about the study population, the use of longer videos, and appropriate performance metrics and testing under abnormal HR and SpO2 ranges for future estimation studies.
Collapse
Affiliation(s)
- Ankit Gupta
- Interactive Technologies Institute/Larsys/Madeira Interactive Technologies Institute, Caminho da Penteada, Funchal, 9020-105, Portugal; Universidade da Madeira, Caminho da Penteada, Funchal, 9020-105, Portugal.
| | - Antonio G Ravelo-García
- Interactive Technologies Institute/Larsys/Madeira Interactive Technologies Institute, Caminho da Penteada, Funchal, 9020-105, Portugal; Universidad de Las Palmas de Gran Canaria, C. Juan de Quesada, 30, Las Palmas, 35001, Spain.
| | - Fernando Morgado Dias
- Interactive Technologies Institute/Larsys/Madeira Interactive Technologies Institute, Caminho da Penteada, Funchal, 9020-105, Portugal; Universidade da Madeira, Caminho da Penteada, Funchal, 9020-105, Portugal.
| |
Collapse
|
11
|
|
12
|
Contactless radar-based breathing monitoring of premature infants in the neonatal intensive care unit. Sci Rep 2022; 12:5150. [PMID: 35338172 PMCID: PMC8956695 DOI: 10.1038/s41598-022-08836-3] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 03/03/2022] [Indexed: 01/18/2023] Open
Abstract
Vital sign monitoring systems are essential in the care of hospitalized neonates. Due to the immaturity of their organs and immune system, premature infants require continuous monitoring of their vital parameters and sensors need to be directly attached to their fragile skin. Besides mobility restrictions and stress, these sensors often cause skin irritation and may lead to pressure necrosis. In this work, we show that a contactless radar-based approach is viable for breathing monitoring in the Neonatal intensive care unit (NICU). For the first time, different scenarios common to the NICU daily routine are investigated, and the challenges of monitoring in a real clinical setup are addressed through different contributions in the signal processing framework. Rather than just discarding measurements under strong interference, we present a novel random body movement mitigation technique based on the time-frequency decomposition of the recovered signal. In addition, we propose a simple and accurate frequency estimator which explores the harmonic structure of the breathing signal. As a result, the proposed radar-based solution is able to provide reliable breathing frequency estimation, which is close to the reference cabled device values most of the time. Our findings shed light on the strengths and limitations of this technology and lay the foundation for future studies toward a completely contactless solution for vital signs monitoring.
Collapse
|
13
|
Molinaro N, Schena E, Silvestri S, Bonotti F, Aguzzi D, Viola E, Buccolini F, Massaroni C. Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview. Front Physiol 2022; 13:801709. [PMID: 35250612 PMCID: PMC8895203 DOI: 10.3389/fphys.2022.801709] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 01/20/2022] [Indexed: 01/26/2023] Open
Abstract
The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.
Collapse
Affiliation(s)
- Nunzia Molinaro
- Unit of Measurements and Biomedical Instrumentation, Departmental Faculty of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Emiliano Schena
- Unit of Measurements and Biomedical Instrumentation, Departmental Faculty of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Sergio Silvestri
- Unit of Measurements and Biomedical Instrumentation, Departmental Faculty of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy
| | | | - Damiano Aguzzi
- BHOHB – Biometrical Holistic of Human Body S.r.l., Rome, Italy
| | - Erika Viola
- BHOHB – Biometrical Holistic of Human Body S.r.l., Rome, Italy
| | - Fabio Buccolini
- BHOHB – Biometrical Holistic of Human Body S.r.l., Rome, Italy
| | - Carlo Massaroni
- Unit of Measurements and Biomedical Instrumentation, Departmental Faculty of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy
| |
Collapse
|
14
|
Efficient Spatiotemporal Attention Network for Remote Heart Rate Variability Analysis. SENSORS 2022; 22:s22031010. [PMID: 35161756 PMCID: PMC8840211 DOI: 10.3390/s22031010] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 01/24/2022] [Accepted: 01/25/2022] [Indexed: 02/01/2023]
Abstract
Studies have shown that ordinary color cameras can detect the subtle color changes of the skin caused by the heartbeat cycle. Therefore, cameras can be used to remotely monitor the pulse in a non-contact manner. The technology for non-contact physiological measurement in this way is called remote photoplethysmography (rPPG). Heart rate variability (HRV) analysis, as a very important physiological feature, requires us to be able to accurately recover the peak time locations of the rPPG signal. This paper proposes an efficient spatiotemporal attention network (ESA-rPPGNet) to recover high-quality rPPG signal for heart rate variability analysis. First, 3D depth-wise separable convolution and a structure based on mobilenet v3 are used to greatly reduce the time complexity of the network. Next, a lightweight attention block called 3D shuffle attention (3D-SA), which integrates spatial attention and channel attention, is designed to enable the network to effectively capture inter-channel dependencies and pixel-level dependencies. Moreover, ConvGRU is introduced to further improve the network’s ability to learn long-term spatiotemporal feature information. Compared with existing methods, the experimental results show that the method proposed in this paper has better performance and robustness on the remote HRV analysis.
Collapse
|
15
|
Zhang C, Tian J, Li D, Hou X, Wang L. Comparative study on the effect of color spaces and color formats on heart rate measurement using the imaging photoplethysmography (IPPG) method. Technol Health Care 2022; 30:391-402. [PMID: 35124614 PMCID: PMC9028635 DOI: 10.3233/thc-thc228036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
Abstract
BACKGROUND The imaging photoplethysmography (IPPG) technology has been demonstrated to be an effective method for heart rate (HR) monitoring. However, some interference caused by the ambient illumination variation and facial motion severely influences the accuracy of the HR measurement. Some color spaces and color formats are assumed to reduce the interference, and enhance the accuracy of HR estimation. OBJECTIVE The aim is to identify the optimal color space and format for IPPG based HR measurement. METHODS Six color spaces and 3 color formats are compared in this study, based on an IPPG based HR measurement system. 424 pieces of videos captured by the system are used for the selection of the optimal color channel and color space; while 10 pieces of videos are for the identification of the optimal color format. RESULTS The results shows that the green channel of RGB space is the optimal color channel, and RGB is the optimal color space, in respect of the mean squared error of HR estimation. BayerBG 8bit is found to be the optimal color format for video recording, which can significantly reduce the HR estimation error. CONCLUSIONS BayerBG 8bit color format for video recording, and RGB color space for video analysis is suggested for the IPPG based HR measurement system. The suitable configuration of color space and format could enhance the accuracy of HR measurement.
Collapse
Affiliation(s)
- Chi Zhang
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Jing Tian
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Deyu Li
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Xiaoxu Hou
- National Institutes for Food and Drug Control, Beijing, China
| | - Li Wang
- Beijing Research Center of Urban System Engineering, Beijing, China
| |
Collapse
|
16
|
Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app112210913] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.
Collapse
|
17
|
Wang L, Yang Y, Su S. Nonparametric Modelling Based Model Predictive Control for Human Heart Rate Regulation during Treadmill Exercise. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:148-151. [PMID: 34891259 DOI: 10.1109/embc46164.2021.9629548] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper applies a kernel-based nonparametric modelling method to estimate the heart rate response during treadmill exercise and proposes a model predictive control (MPC) method to perform heart rate control for an automated treadmill system. This kernel-based method introduces a kernel regularisation term, which brings prior information to the model estimation phase. By adding this prior information, the experimental protocol can be significantly simplified and only a small amount of model training experiments are needed. The model parameters were experimentally estimated from 12 participants for the treadmill exercise with a short and practical exercise protocol. The modelling results show that the model identified using the proposed method can accurately describe the heart rate response to the treadmill exercise. Based on the identified model, an MPC controller is designed to track a predefined reference heart rate profile. An advantage is the speed and acceleration of the treadmill can be limited to within a safe range for vulnerable exercisers. The proposed controller was experimentally validated in a self-developed automated treadmill system. The tracking results indicate that the desired automatic treadmill system can regulate the participants' heart rate to follow the reference profile efficiently and safely.
Collapse
|
18
|
Nagumo K, Oiwa K, Nozawa A. Spatial normalization of facial thermal images using facial landmarks. ARTIFICIAL LIFE AND ROBOTICS 2021. [DOI: 10.1007/s10015-021-00703-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
AbstractHuman–computer interaction (HCI) is an interaction for mutual communication between humans and computers. HCI needs to recognize the human state quantitatively and in real-time. Although it is possible to quantitatively evaluate the human condition by measuring biological signals, the challenge is that it often requires physical constraints. There is an increasing interest in a non-contact method of estimating physiological and psychological states by measuring facial skin temperature using infrared thermography. However, due to individual differences in face shape, the accuracy of physiological and psychological state estimation using facial thermal images was sometimes low. To solve this problem, we hypothesized that spatial normalization of facial thermal image (SN-FTI) could reduce the effect of individual differences in facial shape. The objective of this study is to develop a method for SN-FTI and to evaluate the effect of SN-FTI on the estimation of physiological and psychological states. First, we attempted spatial normalization using facial features. The results suggested that SN-FTI would result in the same face shape among individuals. Since there are individual differences in facial skin temperature distribution, the inter-individual correlation coefficient is suggested to be lower than the intra-individual correlation coefficient. Next, we modeled the estimated drowsiness level using SN-FTIs and compared it with Normal. The results showed that SN-FTI slightly improved the discrimination rate of drowsiness level. SN-FTIs were suggested to reduce the effect of individual differences in facial structure on the estimation of physiological and psychological states.
Collapse
|
19
|
Khanam FTZ, Perera AG, Al-Naji A, Gibson K, Chahl J. Non-Contact Automatic Vital Signs Monitoring of Infants in a Neonatal Intensive Care Unit Based on Neural Networks. J Imaging 2021; 7:122. [PMID: 34460758 PMCID: PMC8404938 DOI: 10.3390/jimaging7080122] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Revised: 07/18/2021] [Accepted: 07/19/2021] [Indexed: 12/28/2022] Open
Abstract
Infants with fragile skin are patients who would benefit from non-contact vital sign monitoring due to the avoidance of potentially harmful adhesive electrodes and cables. Non-contact vital signs monitoring has been studied in clinical settings in recent decades. However, studies on infants in the Neonatal Intensive Care Unit (NICU) are still limited. Therefore, we conducted a single-center study to remotely monitor the heart rate (HR) and respiratory rate (RR) of seven infants in NICU using a digital camera. The region of interest (ROI) was automatically selected using a convolutional neural network and signal decomposition was used to minimize the noise artefacts. The experimental results have been validated with the reference data obtained from an ECG monitor. They showed a strong correlation using the Pearson correlation coefficients (PCC) of 0.9864 and 0.9453 for HR and RR, respectively, and a lower error rate with RMSE 2.23 beats/min and 2.69 breaths/min between measured data and reference data. A Bland-Altman analysis of the data also presented a close correlation between measured data and reference data for both HR and RR. Therefore, this technique may be applicable in clinical environments as an economical, non-contact, and easily deployable monitoring system, and it also represents a potential application in home health monitoring.
Collapse
Affiliation(s)
- Fatema-Tuz-Zohra Khanam
- UniSA STEM, Mawson Lakes Campus, University of South Australia, Mawson Lakes, SA 5095, Australia; (A.G.P.); (A.A.-N.); (J.C.)
| | - Asanka G. Perera
- UniSA STEM, Mawson Lakes Campus, University of South Australia, Mawson Lakes, SA 5095, Australia; (A.G.P.); (A.A.-N.); (J.C.)
| | - Ali Al-Naji
- UniSA STEM, Mawson Lakes Campus, University of South Australia, Mawson Lakes, SA 5095, Australia; (A.G.P.); (A.A.-N.); (J.C.)
- Electrical Engineering Technical College, Middle Technical University, Baghdad 10022, Iraq
| | - Kim Gibson
- Clinical and Health Sciences, City East Campus, University of South Australia, North Terrace, Adelaide, SA 5000, Australia;
| | - Javaan Chahl
- UniSA STEM, Mawson Lakes Campus, University of South Australia, Mawson Lakes, SA 5095, Australia; (A.G.P.); (A.A.-N.); (J.C.)
| |
Collapse
|
20
|
Ali M, Elsayed A, Mendez A, Savaria Y, Sawan M. Contact and Remote Breathing Rate Monitoring Techniques: A Review. IEEE SENSORS JOURNAL 2021; 21:14569-14586. [PMID: 35789086 PMCID: PMC8769001 DOI: 10.1109/jsen.2021.3072607] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 03/25/2021] [Accepted: 03/26/2021] [Indexed: 06/01/2023]
Abstract
Breathing rate monitoring is a must for hospitalized patients with the current coronavirus disease 2019 (COVID-19). We review in this paper recent implementations of breathing monitoring techniques, where both contact and remote approaches are presented. It is known that with non-contact monitoring, the patient is not tied to an instrument, which improves patients' comfort and enhances the accuracy of extracted breathing activity, since the distress generated by a contact device is avoided. Remote breathing monitoring allows screening people infected with COVID-19 by detecting abnormal respiratory patterns. However, non-contact methods show some disadvantages such as the higher set-up complexity compared to contact ones. On the other hand, many reported contact methods are mainly implemented using discrete components. While, numerous integrated solutions have been reported for non-contact techniques, such as continuous wave (CW) Doppler radar and ultrawideband (UWB) pulsed radar. These radar chips are discussed and their measured performances are summarized and compared.
Collapse
Affiliation(s)
- Mohamed Ali
- Department of Electrical EngineeringPolytechnique MontréalMontrealQCH3T IJ4Canada
- Department of MicroelectronicsElectronics Research InstituteCairo12622Egypt
| | - Ali Elsayed
- Nanotechnology and Nanoelectronics ProgramUniversity of Science and Technology, Zewail City of Science, Technology and InnovationGiza12578Egypt
| | - Arnaldo Mendez
- Department of Electrical EngineeringPolytechnique MontréalMontrealQCH3T IJ4Canada
| | - Yvon Savaria
- Department of Electrical EngineeringPolytechnique MontréalMontrealQCH3T IJ4Canada
| | - Mohamad Sawan
- Department of Electrical EngineeringPolytechnique MontréalMontrealQCH3T IJ4Canada
- School of EngineeringWestlake Institute for Advanced Study, Westlake UniversityHangzhou310024China
| |
Collapse
|
21
|
Ni A, Azarang A, Kehtarnavaz N. A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods. SENSORS 2021; 21:s21113719. [PMID: 34071736 PMCID: PMC8198867 DOI: 10.3390/s21113719] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 05/18/2021] [Accepted: 05/24/2021] [Indexed: 02/07/2023]
Abstract
The interest in contactless or remote heart rate measurement has been steadily growing in healthcare and sports applications. Contactless methods involve the utilization of a video camera and image processing algorithms. Recently, deep learning methods have been used to improve the performance of conventional contactless methods for heart rate measurement. After providing a review of the related literature, a comparison of the deep learning methods whose codes are publicly available is conducted in this paper. The public domain UBFC dataset is used to compare the performance of these deep learning methods for heart rate measurement. The results obtained show that the deep learning method PhysNet generates the best heart rate measurement outcome among these methods, with a mean absolute error value of 2.57 beats per minute and a mean square error value of 7.56 beats per minute.
Collapse
|
22
|
Classification of Prefrontal Cortex Activity Based on Functional Near-Infrared Spectroscopy Data upon Olfactory Stimulation. Brain Sci 2021; 11:brainsci11060701. [PMID: 34073372 PMCID: PMC8228245 DOI: 10.3390/brainsci11060701] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2021] [Revised: 04/30/2021] [Accepted: 05/19/2021] [Indexed: 11/17/2022] Open
Abstract
The sense of smell is one of the most important organs in humans, and olfactory imaging can detect signals in the anterior orbital frontal lobe. This study assessed olfactory stimuli using support vector machines (SVMs) with signals from functional near-infrared spectroscopy (fNIRS) data obtained from the prefrontal cortex. These data included odor stimuli and air state, which triggered the hemodynamic response function (HRF), determined from variations in oxyhemoglobin (oxyHb) and deoxyhemoglobin (deoxyHb) levels; photoplethysmography (PPG) of two wavelengths (raw optical red and near-infrared data); and the ratios of data from two optical datasets. We adopted three SVM kernel functions (i.e., linear, quadratic, and cubic) to analyze signals and compare their performance with the HRF and PPG signals. The results revealed that oxyHb yielded the most efficient single-signal data with a quadratic kernel function, and a combination of HRF and PPG signals yielded the most efficient multi-signal data with the cubic function. Our results revealed superior SVM analysis of HRFs for classifying odor and air status using fNIRS data during olfaction in humans. Furthermore, the olfactory stimulation can be accurately classified by using quadratic and cubic kernel functions in SVM, even for an individual participant data set.
Collapse
|
23
|
Purnomo AT, Lin DB, Adiprabowo T, Hendria WF. Non-Contact Monitoring and Classification of Breathing Pattern for the Supervision of People Infected by COVID-19. SENSORS 2021; 21:s21093172. [PMID: 34063576 PMCID: PMC8125653 DOI: 10.3390/s21093172] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 04/26/2021] [Accepted: 04/28/2021] [Indexed: 02/06/2023]
Abstract
During the pandemic of coronavirus disease-2019 (COVID-19), medical practitioners need non-contact devices to reduce the risk of spreading the virus. People with COVID-19 usually experience fever and have difficulty breathing. Unsupervised care to patients with respiratory problems will be the main reason for the rising death rate. Periodic linearly increasing frequency chirp, known as frequency-modulated continuous wave (FMCW), is one of the radar technologies with a low-power operation and high-resolution detection which can detect any tiny movement. In this study, we use FMCW to develop a non-contact medical device that monitors and classifies the breathing pattern in real time. Patients with a breathing disorder have an unusual breathing characteristic that cannot be represented using the breathing rate. Thus, we created an Xtreme Gradient Boosting (XGBoost) classification model and adopted Mel-frequency cepstral coefficient (MFCC) feature extraction to classify the breathing pattern behavior. XGBoost is an ensemble machine-learning technique with a fast execution time and good scalability for predictions. In this study, MFCC feature extraction assists machine learning in extracting the features of the breathing signal. Based on the results, the system obtained an acceptable accuracy. Thus, our proposed system could potentially be used to detect and monitor the presence of respiratory problems in patients with COVID-19, asthma, etc.
Collapse
Affiliation(s)
- Ariana Tulus Purnomo
- Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan;
- Correspondence: (A.T.P.); (D.-B.L.)
| | - Ding-Bing Lin
- Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan;
- Correspondence: (A.T.P.); (D.-B.L.)
| | - Tjahjo Adiprabowo
- Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan;
| | - Willy Fitra Hendria
- Department of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea;
| |
Collapse
|
24
|
Kaneko D, Stuldreher I, Reuten AJC, Toet A, van Erp JBF, Brouwer AM. Comparing Explicit and Implicit Measures for Assessing Cross-Cultural Food Experience. FRONTIERS IN NEUROERGONOMICS 2021; 2:646280. [PMID: 38235219 PMCID: PMC10790875 DOI: 10.3389/fnrgo.2021.646280] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/25/2020] [Accepted: 02/17/2021] [Indexed: 01/19/2024]
Abstract
The present study investigated the potential of implicit physiological measures to provide objective measures of affective food experience in contrast to explicit self-report ratings in a cross-cultural context. Dutch and Thai participants viewed 120 food images portraying universal food image categories (regular and molded food) and cultural food image categories (typically Dutch and Thai food). The universal food images were taken as ground truth high and low valence stimuli, where we assumed no genuine difference in affective experience between nationalities. In contrast, for the cultural food images, we did expect a genuine difference between nationalities. Participants were asked to rate valence, arousal and liking of each food image. In addition, heart rate (HR) and phasic electrodermal activity (EDA) responses to the images were recorded. Typically Asian and Western response biases were found for explicit ratings of regular and molded food with an extreme response style for Dutch, and a middle response style for Thai participants. However, such bias was not observed in HR. For cultural food image categories, HR showed the hypothesized interaction between participant nationality and food image category, reflecting the expected genuine difference between nationalities in affective food experience. Besides presenting participants with images, we also asked participants to taste typically Thai and Dutch drinks. Similar to images, a significant interaction between participant nationality and cultural food category was found for HR. An interaction was also found for sip size, while this was not seen in explicit measures. We attribute this to differences in the moment that these measures were taken. In this study, phasic EDA did not appear to be a sensitive measure of affective food experience, possibly since stimuli mostly differed in valence rather than arousal. To conclude, our study constitutes an example where cultural bias negatively affected the accuracy of self-reports, and only the implicit physiological measures followed the prior expectations of genuine food experience, indicating the potential of these measures to study cross-cultural food experience.
Collapse
Affiliation(s)
- Daisuke Kaneko
- Kikkoman Europe R&D Laboratory B.V., Wageningen, Netherlands
- Microbiology and Systems Biology, Netherlands Organisation for Applied Scientific Research TNO, Zeist, Netherlands
| | - Ivo Stuldreher
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research TNO, Soesterberg, Netherlands
- Human Media Interaction Lab, University of Twente, Enschede, Netherlands
| | - Anne J. C. Reuten
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research TNO, Soesterberg, Netherlands
| | - Alexander Toet
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research TNO, Soesterberg, Netherlands
| | - Jan B. F. van Erp
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research TNO, Soesterberg, Netherlands
- Human Media Interaction Lab, University of Twente, Enschede, Netherlands
| | - Anne-Marie Brouwer
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research TNO, Soesterberg, Netherlands
| |
Collapse
|
25
|
Khan MU, Aziz S, Akram T, Amjad F, Iqtidar K, Nam Y, Khan MA. Expert Hypertension Detection System Featuring Pulse Plethysmograph Signals and Hybrid Feature Selection and Reduction Scheme. SENSORS 2021; 21:s21010247. [PMID: 33401652 PMCID: PMC7794944 DOI: 10.3390/s21010247] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 12/22/2020] [Accepted: 12/24/2020] [Indexed: 12/27/2022]
Abstract
Hypertension is an antecedent to cardiac disorders. According to the World Health Organization (WHO), the number of people affected with hypertension will reach around 1.56 billion by 2025. Early detection of hypertension is imperative to prevent the complications caused by cardiac abnormalities. Hypertension usually possesses no apparent detectable symptoms; hence, the control rate is significantly low. Computer-aided diagnosis based on machine learning and signal analysis has recently been applied to identify biomarkers for the accurate prediction of hypertension. This research proposes a new expert hypertension detection system (EHDS) from pulse plethysmograph (PuPG) signals for the categorization of normal and hypertension. The PuPG signal data set, including rich information of cardiac activity, was acquired from healthy and hypertensive subjects. The raw PuPG signals were preprocessed through empirical mode decomposition (EMD) by decomposing a signal into its constituent components. A combination of multi-domain features was extracted from the preprocessed PuPG signal. The features exhibiting high discriminative characteristics were selected and reduced through a proposed hybrid feature selection and reduction (HFSR) scheme. Selected features were subjected to various classification methods in a comparative fashion in which the best performance of 99.4% accuracy, 99.6% sensitivity, and 99.2% specificity was achieved through weighted k-nearest neighbor (KNN-W). The performance of the proposed EHDS was thoroughly assessed by tenfold cross-validation. The proposed EHDS achieved better detection performance in comparison to other electrocardiogram (ECG) and photoplethysmograph (PPG)-based methods.
Collapse
Affiliation(s)
- Muhammad Umar Khan
- Department of Electronics Engineering, University of Engineering and Technology Taxila, Taxila 47050, Pakistan; (M.U.K.); (F.A.)
| | - Sumair Aziz
- Department of Electronics Engineering, University of Engineering and Technology Taxila, Taxila 47050, Pakistan; (M.U.K.); (F.A.)
- Correspondence: (S.A.); (Y.N.)
| | - Tallha Akram
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, Wah Cantonment, Islamabad 45550, Pakistan;
| | - Fatima Amjad
- Department of Electronics Engineering, University of Engineering and Technology Taxila, Taxila 47050, Pakistan; (M.U.K.); (F.A.)
| | - Khushbakht Iqtidar
- Department of Computer and Software Engineering, College of Electrical and Mechanical Engineering, National University of Sciences and Technology, Islamabad 44000, Pakistan;
| | - Yunyoung Nam
- Department of Computer Science and Engineering, Soonchunhyang University, Asan 31538, Korea
- Correspondence: (S.A.); (Y.N.)
| | | |
Collapse
|
26
|
Jin H, Yu J, Lin S, Gao S, Yang H, Haick H, Hua C, Deng S, Yang T, Liu Y, Shen W, Zhang X, Zhang X, Shan S, Ren T, Wang L, Cheung W, Kam W, Miao J, Chen D, Cui D. Nanosensor-Based Flexible Electronic Assisted with Light Fidelity Communicating Technology for Volatolomics-Based Telemedicine. ACS NANO 2020; 14:15517-15532. [PMID: 33141556 DOI: 10.1021/acsnano.0c06137] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Telemedicine provides an attractive vision for tele-monitoring human health conditions and, thus, offers the opportunity for timely preventing chronic disease. A key limitation of promoting telemedicine in clinic application is the lack of a noninvasive med-tech and effective monitoring platform, which should be wearable and capable of high-performance tele-monitoring of health risk. Here we proposed a volatolomics-based telemedicine for continuously and noninvasively assessing human health status through continuously tracking the variation of volatile markers derived from human breath or skin. Particularly, a nanosensor-based flexible electronic was specifically designed to serve as a powerful platform for implementing the proposed cost-effective healthcare. An all-flexible and highly packed makeup (all functional units were integrated in a 2*2*0.19 cm3 plate) enables an electronic, compact configuration and the capability of resisting negative impact derived from customers' daily movement. Notably, the nanosensor-based electronic demonstrates high specificity, quick response rate (t90% = 4.5 s), and desirable low detection limit (down to 0.117 ppm) in continuous tele-monitoring chronic-disease-related volatile marker (e.g., acetone). Assisted by the power saved light fidelity (Li-Fi) communicating technology, a clinic proof on the specifically designed electronic for noninvasively and uninterrupted assessing potential health risk (e.g., diabetics) is successfully implemented, with the accuracy of around 81%. A further increase in the accuracy of prewarning is predicted by excluding the impact of individual differences such as the gender, age, and smoking status of the customer. These promising pilot results indicate a bright future for the tailor-made nanosensing-device-supported volatolomics-based telemedicine in preventing chronic diseases and increasing patients' survival rate.
Collapse
Affiliation(s)
- Han Jin
- Institute of Micro-Nano Science and Technology, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
- National Engineering Research Center for Nanotechnology, Shanghai 200241, P. R. China
| | - Junkan Yu
- School of of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, P. R. China
| | - Shujing Lin
- Institute of Micro-Nano Science and Technology, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Shan Gao
- State Key Laboratory of Pathogen and Biosecurity, Institute of Microbiology and Epidemiology, Academy of Military Medical Sciences, Beijing 100071, P. R. China
| | - Hao Yang
- State Key Laboratory of Pathogen and Biosecurity, Institute of Microbiology and Epidemiology, Academy of Military Medical Sciences, Beijing 100071, P. R. China
| | - Hossam Haick
- The Department of Chemical Engineering and The Russell Berrie Nanotechnology Institute, Technion - Israel Institute of Technology, Haifa 3200003, Israel
| | - Changzhou Hua
- School of of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, P. R. China
| | - Shengwei Deng
- College of Chemical Engineering, Zhejiang University of Technology, Hangzhou 310014, P. R. China
| | - Tingqiang Yang
- State Key Laboratory of Silicate Materials for Architectures, School of Materials Science and Engineering, Wuhan University of Technology, Wuhan 430070, P. R. China
| | - Yueli Liu
- State Key Laboratory of Silicate Materials for Architectures, School of Materials Science and Engineering, Wuhan University of Technology, Wuhan 430070, P. R. China
| | - Wenfeng Shen
- Ningbo Materials Science and Technology Institute, Chinese Academy of Sciences, Ningbo 315201, P. R. China
| | - Xin Zhang
- School of of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, P. R. China
| | - Xiaowei Zhang
- School of of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, P. R. China
| | - Shan Shan
- Department of Respiration Medicine, Shanghai Jiao Tong University Affiliated Sixth People's Hospital, Shanghai 200233, P. R. China
| | - Tao Ren
- Department of Respiration Medicine, Shanghai Jiao Tong University Affiliated Sixth People's Hospital, Shanghai 200233, P. R. China
| | - Liwei Wang
- School of Marine Sciences, Guangxi University, Nanning 530004, P. R. China
| | | | | | - Jianmin Miao
- Institute of Micro-Nano Science and Technology, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Di Chen
- Institute of Micro-Nano Science and Technology, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Daxiang Cui
- Institute of Micro-Nano Science and Technology, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
- National Engineering Research Center for Nanotechnology, Shanghai 200241, P. R. China
| |
Collapse
|
27
|
Nazir S, Pateau V, Bert J, Clement JF, Fayad H, l'Her E, Visvikis D. Surface imaging for real-time patient respiratory function assessment in intensive care. Med Phys 2020; 48:142-155. [PMID: 33118190 DOI: 10.1002/mp.14557] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Revised: 10/08/2020] [Accepted: 10/20/2020] [Indexed: 11/11/2022] Open
Abstract
PURPOSE Monitoring of physiological parameters is a major concern in Intensive Care Units (ICU) given their role in the assessment of vital organ function. Within this context, one issue is the lack of efficient noncontact techniques for respiratory monitoring. In this paper, we present a novel noncontact solution for real-time respiratory monitoring and function assessment of ICU patients. METHODS The proposed system uses a Time-of-Flight depth sensor to analyze the patient's chest wall morphological changes in order to estimate multiple respiratory function parameters. The automatic detection of the patient's torso is also proposed using a deep neural network model trained on the COCO dataset. The evaluation of the proposed system was performed on a mannequin and on 16 mechanically ventilated patients (a total of 216 recordings) admitted in the ICU of the Brest University Hospital. RESULTS The estimation of respiratory parameters (respiratory rate and tidal volume) showed high correlation with the reference method (r = 0.99; P < 0.001 and r = 0.99; P < 0.001) in the mannequin recordings and (r = 0.95, P < 0.001 and r = 0.90, P < 0.001) for patients. CONCLUSION This study describes and evaluates a novel noncontact monitoring system suitable for continuous monitoring of key respiratory parameters for disease assessment of critically ill patients.
Collapse
Affiliation(s)
- Souha Nazir
- INSERM, UMR1101, LaTIM, University of Brest, Brest, 29200, France
| | | | - Julien Bert
- INSERM, UMR1101, LaTIM, University of Brest, Brest, 29200, France
| | | | - Hadi Fayad
- INSERM, UMR1101, LaTIM, University of Brest, Brest, 29200, France.,Hamad Medical Corporation OHS, PET/CT center Doha, Doha, Qatar
| | - Erwan l'Her
- INSERM, UMR1101, LaTIM, University of Brest, Brest, 29200, France.,CHRU, Brest, 29200, France
| | | |
Collapse
|
28
|
Villarroel M, Jorge J, Meredith D, Sutherland S, Pugh C, Tarassenko L. Non-contact vital-sign monitoring of patients undergoing haemodialysis treatment. Sci Rep 2020; 10:18529. [PMID: 33116150 PMCID: PMC7595175 DOI: 10.1038/s41598-020-75152-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Accepted: 10/12/2020] [Indexed: 12/12/2022] Open
Abstract
A clinical study was designed to record a wide range of physiological values from patients undergoing haemodialysis treatment in the Renal Unit of the Churchill Hospital in Oxford. Video was recorded for a total of 84 dialysis sessions from 40 patients during the course of 1 year, comprising an overall video recording time of approximately 304.1 h. Reference values were provided by two devices in regular clinical use. The mean absolute error between the heart rate estimates from the camera and the average from two reference pulse oximeters (positioned at the finger and earlobe) was 2.8 beats/min for over 65% of the time the patient was stable. The mean absolute error between the respiratory rate estimates from the camera and the reference values (computed from the Electrocardiogram and a thoracic expansion sensor-chest belt) was 2.1 breaths/min for over 69% of the time for which the reference signals were valid. To increase the robustness of the algorithms, novel methods were devised for cancelling out aliased frequency components caused by the artificial light sources in the hospital, using auto-regressive modelling and pole cancellation. Maps of the spatial distribution of heart rate and respiratory rate information were developed from the coefficients of the auto-regressive models. Most of the periods for which the camera could not produce a reliable heart rate estimate lasted under 3 min, thus opening the possibility to monitor heart rate continuously in a clinical environment.
Collapse
Affiliation(s)
- Mauricio Villarroel
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, UK.
| | - João Jorge
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| | - David Meredith
- Oxford Kidney Unit, Oxford University Hospitals National Health Service Trust, Oxford, UK
| | - Sheera Sutherland
- Oxford Kidney Unit, Oxford University Hospitals National Health Service Trust, Oxford, UK
| | - Chris Pugh
- Nuffield Department of Medicine, University of Oxford, Oxford, UK
| | - Lionel Tarassenko
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| |
Collapse
|
29
|
Singh A, Rehman SU, Yongchareon S, Chong PHJ. Modelling of Chest Wall Motion for Cardiorespiratory Activity for Radar-Based NCVS Systems. SENSORS (BASEL, SWITZERLAND) 2020; 20:E5094. [PMID: 32906804 PMCID: PMC7570465 DOI: 10.3390/s20185094] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/21/2020] [Revised: 09/02/2020] [Accepted: 09/03/2020] [Indexed: 11/16/2022]
Abstract
Chest wall motion can provide information on critical vital signs, including respiration and heartbeat. Mathematical modelling of chest wall motion can reduce an extensive requirement of human testing in the development of many biomedical applications. In this paper, we propose a mathematical model that simulates a chest wall motion due to cardiorespiratory activity. Chest wall motion due to respiration is simulated based on the optimal chemical-mechanical respiratory control-based mechanics. The theory of relaxation oscillation system is applied to model the motion due to cardiac activity. The proposed mathematical chest wall model can be utilized in designing and optimizing different design parameters for radar-based non-contact vital sign (NCVS) systems.
Collapse
Affiliation(s)
- Anuradha Singh
- Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand; (S.U.R.); (P.H.J.C.)
| | - Saeed Ur Rehman
- Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand; (S.U.R.); (P.H.J.C.)
- College of Science and Engineering, Flinders University, Adelaide 5042, Australia
| | - Sira Yongchareon
- Department of Information Technology and Software Engineering, Auckland University of Technology, Auckland 1010, New Zealand;
| | - Peter Han Joo Chong
- Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand; (S.U.R.); (P.H.J.C.)
| |
Collapse
|
30
|
Measuring Drivers’ Physiological Response to Different Vehicle Controllers in Highly Automated Driving (HAD): Opportunities for Establishing Real-Time Values of Driver Discomfort. INFORMATION 2020. [DOI: 10.3390/info11080390] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
This study investigated how driver discomfort was influenced by different types of automated vehicle (AV) controllers, compared to manual driving, and whether this response changed in different road environments, using heart-rate variability (HRV) and electrodermal activity (EDA). A total of 24 drivers were subjected to manual driving and four AV controllers: two modelled to depict “human-like” driving behaviour, one conventional lane-keeping assist controller, and a replay of their own manual drive. Each drive lasted for ~15 min and consisted of rural and urban environments, which differed in terms of average speed, road geometry and road-based furniture. Drivers showed higher skin conductance response (SCR) and lower HRV during manual driving, compared to the automated drives. There were no significant differences in discomfort between the AV controllers. SCRs and subjective discomfort ratings showed significantly higher discomfort in the faster rural environments, when compared to the urban environments. Our results suggest that SCR values are more sensitive than HRV-based measures to continuously evolving situations that induce discomfort. Further research may be warranted in investigating the value of this metric in assessing real-time driver discomfort levels, which may help improve acceptance of AV controllers.
Collapse
|
31
|
Stress levels estimation from facial video based on non-contact measurement of pulse wave. ARTIFICIAL LIFE AND ROBOTICS 2020. [DOI: 10.1007/s10015-020-00624-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
32
|
Zia J, Kimball J, Rolfes C, Hahn JO, Inan OT. Enabling the assessment of trauma-induced hemorrhage via smart wearable systems. SCIENCE ADVANCES 2020; 6:eabb1708. [PMID: 32766449 PMCID: PMC7375804 DOI: 10.1126/sciadv.abb1708] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/05/2020] [Accepted: 06/05/2020] [Indexed: 05/08/2023]
Abstract
As the leading cause of trauma-related mortality, blood loss due to hemorrhage is notoriously difficult to triage and manage. To enable timely and appropriate care for patients with trauma, this work elucidates the externally measurable physiological features of exsanguination, which were used to develop a globalized model for assessing blood volume status (BVS) or the relative severity of blood loss. These features were captured via both a multimodal wearable system and a catheter-based reference and used to accurately infer BVS in a porcine model of hemorrhage (n = 6). Ultimately, high-level features of cardiomechanical function were shown to strongly predict progression toward cardiovascular collapse and used to estimate BVS with a median error of 15.17 and 18.17% for the catheter-based and wearable systems, respectively. Exploring the nexus of biomedical theory and practice, these findings lay the groundwork for digital biomarkers of hemorrhage severity and warrant further study in human subjects.
Collapse
Affiliation(s)
- Jonathan Zia
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
- Emory University School of Medicine, Atlanta, GA 30322, USA
| | - Jacob Kimball
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - Christopher Rolfes
- Translational Training and Testing Laboratories Inc., Atlanta, GA 30313, USA
| | - Jin-Oh Hahn
- Department of Mechanical Engineering, University of Maryland, College Park, MD 20742, USA
| | - Omer T. Inan
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| |
Collapse
|
33
|
Frisch S, Werner P, Al-Hamadi A, Traue HC, Gruss S, Walter S. [From external assessment of pain to automated multimodal measurement of pain intensity : Narrative review of state of research and clinical perspectives]. Schmerz 2020; 34:376-387. [PMID: 32382799 DOI: 10.1007/s00482-020-00473-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
BACKGROUND In patients with limited communication skills, the use of conventional scales or external assessment is only possible to a limited extent or not at all. Multimodal pain recognition based on artificial intelligence (AI) algorithms could be a solution. OBJECTIVE Overview of the methods of automated multimodal pain measurement and their recognition rates that were calculated with AI algorithms. METHODS In April 2018, 101 studies on automated pain recognition were found in the Web of Science database to illustrate the current state of research. A selective literature review with special consideration of recognition rates of automated multimodal pain measurement yielded 14 studies, which are the focus of this review. RESULTS The variance in recognition rates was 52.9-55.0% (pain threshold) and 66.8-85.7%; in nine studies the recognition rate was ≥80% (pain tolerance), while one study reported recognition rates of 79.3% (pain threshold) and 90.9% (pain tolerance). CONCLUSION Pain is generally recorded multimodally, based on external observation scales. With regard to automated pain recognition and on the basis of the 14 selected studies, there is to date no conclusive evidence that multimodal automated pain recognition is superior to unimodal pain recognition. In the clinical context, multimodal pain recognition could be advantageous, because this approach is more flexible. In the case of one modality not being available, e.g., electrodermal activity in hand burns, the algorithm could use other modalities (video) and thus compensate for missing information.
Collapse
Affiliation(s)
- S Frisch
- Sektion Medizinische Psychologie, Klinik für Psychosomatische Medizin und Psychotherapie, Universitätsklinikum Ulm, Frauensteige 6, 89075, Ulm, Deutschland
- Praxis für Neurologie und Psychiatrie Leutkirch, Leutkirch, Deutschland
| | - P Werner
- Neuro-Informationstechnik, Institut für Informations- und Kommunikationstechnik, Otto-von-Guericke-Universität Magdeburg, Magdeburg, Deutschland
| | - A Al-Hamadi
- Neuro-Informationstechnik, Institut für Informations- und Kommunikationstechnik, Otto-von-Guericke-Universität Magdeburg, Magdeburg, Deutschland
| | - H C Traue
- Sektion Medizinische Psychologie, Klinik für Psychosomatische Medizin und Psychotherapie, Universitätsklinikum Ulm, Frauensteige 6, 89075, Ulm, Deutschland
| | - S Gruss
- Sektion Medizinische Psychologie, Klinik für Psychosomatische Medizin und Psychotherapie, Universitätsklinikum Ulm, Frauensteige 6, 89075, Ulm, Deutschland
| | - S Walter
- Sektion Medizinische Psychologie, Klinik für Psychosomatische Medizin und Psychotherapie, Universitätsklinikum Ulm, Frauensteige 6, 89075, Ulm, Deutschland.
| |
Collapse
|
34
|
Zheng Y, Yu Z, Mao G, Li Y, Pravarthana D, Asghar W, Liu Y, Qu S, Shang J, Li R. A Wearable Capacitive Sensor Based on Ring/Disk-Shaped Electrode and Porous Dielectric for Noncontact Healthcare Monitoring. GLOBAL CHALLENGES (HOBOKEN, NJ) 2020; 4:1900079. [PMID: 32328287 PMCID: PMC7175022 DOI: 10.1002/gch2.201900079] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 02/22/2020] [Accepted: 02/24/2020] [Indexed: 05/31/2023]
Abstract
Wearable sensors are gradually enabling decentralized healthcare systems. However, these sensors need to be closely attached to skin, which is unsuitable for long-term dynamic health monitoring of the patients, such as infants or persons with burn injuries. Here, a wearable capacitive sensor based on the capacitively coupled effect for healthcare monitoring in noncontact mode is reported. It consists of a ring-shaped top electrode, a disk-shaped bottom electrode, and a porous dielectric layer with low permittivity. This unique design enhanced the capacitively coupled effect of the sensor, which enables a high noncontact detectivity of capacitance change. When an object approaches the sensor, its capacitance change (ΔC/C i = -38.7%) is 3-5 times higher than that of previously reported sensors. Meanwhile, the sensor is insensitive to the stretching strain and pressure (ΔC/C i < 5%) due to the unique ring-shaped electrode and the incompressible closed cells of the porous dielectric material, respectively. Finally, various human physiological signals (pulse and respiratory) are recorded in noncontact mode, where a person wears loose and soft clothes implanted with the sensor. Thus, it is promising to build smart healthcare clothes based on it to develop wearable decentralized healthcare systems.
Collapse
Affiliation(s)
- Ya‐Nan Zheng
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- College of Materials Science and Opto‐Electronic TechnologyUniversity of Chinese Academy of SciencesBeijing100049P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| | - Zhe Yu
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- College of Materials Science and Opto‐Electronic TechnologyUniversity of Chinese Academy of SciencesBeijing100049P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| | - Guoyong Mao
- State Key Laboratory of Fluid Power and Mechatronic SystemKey Laboratory of Soft Machines and Smart Devices of Zhejiang ProvinceDepartment of Engineering MechanicsZhejiang UniversityHangzhou310027China
| | - Yunyao Li
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- College of Information EngineeringNanjing University of Finance and EconomicsNanjing210046China
| | - Dhanapal Pravarthana
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| | - Waqas Asghar
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| | - Yiwei Liu
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| | - Shaoxing Qu
- State Key Laboratory of Fluid Power and Mechatronic SystemKey Laboratory of Soft Machines and Smart Devices of Zhejiang ProvinceDepartment of Engineering MechanicsZhejiang UniversityHangzhou310027China
| | - Jie Shang
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| | - Run‐Wei Li
- CAS Key Laboratory of Magnetic Materials and DevicesNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
- Zhejiang Province Key Laboratory of Magnetic Materials and Application TechnologyNingbo Institute of Materials Technology and EngineeringChinese Academy of SciencesNingbo315201P. R. China
| |
Collapse
|
35
|
Nie L, Berckmans D, Wang C, Li B. Is Continuous Heart Rate Monitoring of Livestock a Dream or Is It Realistic? A Review. SENSORS 2020; 20:s20082291. [PMID: 32316511 PMCID: PMC7219037 DOI: 10.3390/s20082291] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 04/08/2020] [Accepted: 04/15/2020] [Indexed: 12/11/2022]
Abstract
For all homoeothermic living organisms, heart rate (HR) is a core variable to control the metabolic energy production in the body, which is crucial to realize essential bodily functions. Consequently, HR monitoring is becoming increasingly important in research of farm animals, not only for production efficiency, but also for animal welfare. Real-time HR monitoring for humans has become feasible though there are still shortcomings for continuously accurate measuring. This paper is an effort to estimate whether it is realistic to get a continuous HR sensor for livestock that can be used for long term monitoring. The review provides the reported techniques to monitor HR of living organisms by emphasizing their principles, advantages, and drawbacks. Various properties and capabilities of these techniques are compared to check the potential to transfer the mostly adequate sensor technology of humans to livestock in term of application. Based upon this review, we conclude that the photoplethysmographic (PPG) technique seems feasible for implementation in livestock. Therefore, we present the contributions to overcome challenges to evolve to better solutions. Our study indicates that it is realistic today to develop a PPG sensor able to be integrated into an ear tag for mid-sized and larger farm animals for continuously and accurately monitoring their HRs.
Collapse
Affiliation(s)
- Luwei Nie
- Department of Agricultural Structure and Bioenvironmental Engineering, College of Water Resources and Civil Engineering, China Agricultural University, Beijing 100083, China; (L.N.); (B.L.)
- Key Laboratory of Agricultural Engineering in Structure and Environment, Ministry of Agriculture and Rural Affairs, Beijing 100083, China
| | - Daniel Berckmans
- M3-BIORES KU Leuven, Department BioSystems, Kasteelpark Arenberg 30, 3001 Leuven, Belgium;
| | - Chaoyuan Wang
- Department of Agricultural Structure and Bioenvironmental Engineering, College of Water Resources and Civil Engineering, China Agricultural University, Beijing 100083, China; (L.N.); (B.L.)
- Key Laboratory of Agricultural Engineering in Structure and Environment, Ministry of Agriculture and Rural Affairs, Beijing 100083, China
- Correspondence: ; Tel.: +86-10-6273-8635
| | - Baoming Li
- Department of Agricultural Structure and Bioenvironmental Engineering, College of Water Resources and Civil Engineering, China Agricultural University, Beijing 100083, China; (L.N.); (B.L.)
- Key Laboratory of Agricultural Engineering in Structure and Environment, Ministry of Agriculture and Rural Affairs, Beijing 100083, China
| |
Collapse
|
36
|
Abstract
Recent developments in computer science and digital image processing have enabled the extraction of an individual’s heart pulsations from pixel changes in recorded video images of human skin surfaces. This method is termed remote photoplethysmography (rPPG) and can be achieved with consumer-level cameras (e.g., a webcam or mobile camera). The goal of the present publication is two-fold. First, we aim to organize future rPPG software developments in a tractable and nontechnical manner, such that the public gains access to a basic open-source rPPG code, comes to understand its utility, and can follow its most recent progressions. The second goal is to investigate rPPG’s accuracy in detecting heart rates from the skin surfaces of several body parts after physical exercise and under ambient lighting conditions with a consumer-level camera. We report that rPPG is highly accurate when the camera is aimed at facial skin tissue, but that the heart rate recordings from wrist regions are less reliable, and recordings from the calves are unreliable. Facial rPPG remained accurate despite the high heart rates after exercise. The proposed research procedures and the experimental findings provide guidelines for future studies on rPPG.
Collapse
|
37
|
Hiromatsu R, Yamamoto K, Toyoda K, Ohtsuki T. Novel CA-CFAR Approach for Improvement of Doppler Sensor-based Heart Rate Variability Estimation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:796-799. [PMID: 31946015 DOI: 10.1109/embc.2019.8857350] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Heart rate variability is one of major physiological parameters to reflect our stress, which has motivated researchers to investigate a Doppler sensor-based non-contact heartbeat interval estimation algorithm. As one of such methods, we have previously proposed a spectrogram-based method. In this method, the spectrum that might be due to heartbeats is integrated over a spectrogram, and then heartbeat interval is estimated by detecting peaks over the integrated spectrum. However, when a subject moves, undesired peaks with large amplitude appear, which causes the incorrect peak detection. As one of the technique to eliminate the undesired peaks with large amplitude, there is CA-CFAR (Cell Average-Constant False Alarm Rate). CA-CFAR is the technique to detect a signal, when the amplitude of a signal exceeds a threshold calculated with average amplitude of signals before and after the investigated one. However, depending on the duration of body movements, the influence of body movements might be included within the signals used for the threshold calculation, which might results in the detection failure of undesired peaks. This is because the length of GT (Guard Time) is fixed, where GT is the time to prevent the signal used for the threshold calculation from including the investigated signal components. To solve this problem, we propose a novel CA-CFAR in which the length of GT is set as the latest peak interval and only the signal before the investigated one is used so that the influence of body movements does not affect the threshold calculation. Through the experiments where a subject moves, i.e., typing, we confirmed that our spectrogram-based heart rate variability estimation method with the proposed CA-CFAR outperformed the one with CA-CFAR based on fixed GT by the RMSE (Root Mean Squared Error) between the estimated heartbeat interval and the ground truth value of the one.
Collapse
|
38
|
Maki Y, Monno Y, Yoshizaki K, Tanaka M, Okutomi M. Inter-Beat Interval Estimation from Facial Video Based on Reliability of BVP Signals. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:6525-6528. [PMID: 31947336 DOI: 10.1109/embc.2019.8857081] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Inter-beat interval (IBI) and heart rate variability (HRV) are important cardiac parameters that provide physiological and emotional states of a person. In this paper, we present a framework for accurate IBI and HRV estimation from a facial video based on the reliability of extracted blood volume pulse (BVP) signals. Our framework first extracts candidate BVP signals from randomly sampled multiple face patches. The BVP signals are then assessed based on a reliability metric to select the most reliable BVP signal, from which IBI and HRV are calculated. In experiments, we evaluate three reliability metrics and demonstrate that our framework can estimate IBI and HRV more accurately than a conventional single face region-based framework.
Collapse
|
39
|
Non-contact Health Monitoring with LDV. Bioanalysis 2020. [DOI: 10.1007/978-3-030-46691-6_1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022] Open
|
40
|
Rahman H, Ahmed MU, Barua S, Begum S. Non-contact-based driver’s cognitive load classification using physiological and vehicular parameters. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2019.101634] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
41
|
Villarroel M, Chaichulee S, Jorge J, Davis S, Green G, Arteta C, Zisserman A, McCormick K, Watkinson P, Tarassenko L. Non-contact physiological monitoring of preterm infants in the Neonatal Intensive Care Unit. NPJ Digit Med 2019; 2:128. [PMID: 31872068 PMCID: PMC6908711 DOI: 10.1038/s41746-019-0199-5] [Citation(s) in RCA: 39] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Accepted: 11/14/2019] [Indexed: 11/09/2022] Open
Abstract
The implementation of video-based non-contact technologies to monitor the vital signs of preterm infants in the hospital presents several challenges, such as the detection of the presence or the absence of a patient in the video frame, robustness to changes in lighting conditions, automated identification of suitable time periods and regions of interest from which vital signs can be estimated. We carried out a clinical study to evaluate the accuracy and the proportion of time that heart rate and respiratory rate can be estimated from preterm infants using only a video camera in a clinical environment, without interfering with regular patient care. A total of 426.6 h of video and reference vital signs were recorded for 90 sessions from 30 preterm infants in the Neonatal Intensive Care Unit (NICU) of the John Radcliffe Hospital in Oxford. Each preterm infant was recorded under regular ambient light during daytime for up to four consecutive days. We developed multi-task deep learning algorithms to automatically segment skin areas and to estimate vital signs only when the infant was present in the field of view of the video camera and no clinical interventions were undertaken. We propose signal quality assessment algorithms for both heart rate and respiratory rate to discriminate between clinically acceptable and noisy signals. The mean absolute error between the reference and camera-derived heart rates was 2.3 beats/min for over 76% of the time for which the reference and camera data were valid. The mean absolute error between the reference and camera-derived respiratory rate was 3.5 breaths/min for over 82% of the time. Accurate estimates of heart rate and respiratory rate could be derived for at least 90% of the time, if gaps of up to 30 seconds with no estimates were allowed.
Collapse
Affiliation(s)
- Mauricio Villarroel
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Sitthichok Chaichulee
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - João Jorge
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Sara Davis
- Neonatal Unit, John Radcliffe Hospital, Oxford University Hospitals Trust, Oxford, UK
| | - Gabrielle Green
- Neonatal Unit, John Radcliffe Hospital, Oxford University Hospitals Trust, Oxford, UK
| | - Carlos Arteta
- Visual Geometry Group, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Andrew Zisserman
- Visual Geometry Group, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Kenny McCormick
- Neonatal Unit, John Radcliffe Hospital, Oxford University Hospitals Trust, Oxford, UK
| | - Peter Watkinson
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Lionel Tarassenko
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| |
Collapse
|
42
|
A Pilot Study for Estimating the Cardiopulmonary Signals of Diverse Exotic Animals Using a Digital Camera. SENSORS 2019; 19:s19245445. [PMID: 31835550 PMCID: PMC6960731 DOI: 10.3390/s19245445] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/12/2019] [Revised: 12/01/2019] [Accepted: 12/09/2019] [Indexed: 11/17/2022]
Abstract
Monitoring the cardiopulmonary signal of animals is a challenge for veterinarians in conditions when contact with a conscious animal is inconvenient, difficult, damaging, distressing or dangerous to personnel or the animal subject. In this pilot study, we demonstrate a computer vision-based system and use examples of exotic, untamed species to demonstrate this means to extract the cardiopulmonary signal. Subject animals included the following species: Giant panda (Ailuropoda melanoleuca), African lions (Panthera leo), Sumatran tiger (Panthera tigris sumatrae), koala (Phascolarctos cinereus), red kangaroo (Macropus rufus), alpaca (Vicugna pacos), little blue penguin (Eudyptula minor), Sumatran orangutan (Pongo abelii) and Hamadryas baboon (Papio hamadryas). The study was done without need for restriction, fixation, contact or disruption of the daily routine of the subjects. The pilot system extracts the signal from the abdominal-thoracic region, where cardiopulmonary activity is most likely to be visible using image sequences captured by a digital camera. The results show motion on the body surface of the subjects that is characteristic of cardiopulmonary activity and is likely to be useful to estimate physiological parameters (pulse rate and breathing rate) of animals without any physical contact. The results of the study suggest that a fully controlled study against conventional physiological monitoring equipment is ethically warranted, which may lead to a novel approach to non-contact physiological monitoring and remotely sensed health assessment of animals. The method shows promise for applications in veterinary practice, conservation and game management, animal welfare and zoological and behavioral studies.
Collapse
|
43
|
Non-contact heart and respiratory rate monitoring of preterm infants based on a computer vision system: a method comparison study. Pediatr Res 2019; 86:738-741. [PMID: 31351437 DOI: 10.1038/s41390-019-0506-5] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/13/2019] [Revised: 07/04/2019] [Accepted: 07/11/2019] [Indexed: 11/08/2022]
Abstract
BACKGROUND Non-contact heart rate (HR) and respiratory rate (RR) monitoring is necessary for preterm infants due to the potential for the adhesive electrodes of conventional electrocardiogram (ECG) to cause damage to the epidermis. This study was performed to evaluate the agreement between HR and RR measurements of preterm infants using a non-contact computer vision system with comparison to measurements obtained by the ECG. METHODS A single-centre, cross-sectional observational study was conducted in a Neonatal Unit. Ten infants and their ECG monitors were videoed using two Nikon cameras for 10 min. HR and RR measurements obtained from the non-contact system were extracted using advanced signal processing techniques and later compared to the ECG readings using Bland-Altman analysis. RESULTS The non-contact system was able to detect an apnoea when the ECG determined movement as respirations. Although the mean bias between both methods was relatively low, the limits of agreement for HR were -8.3 to 17.4 beats per minute (b.p.m.) and for RR, -22 to 23.6 respirations per minute (r.p.m.). CONCLUSIONS This study provides necessary data for improving algorithms to address confounding variables common to the neonatal population. Further studies investigating the robustness of the proposed system for premature infants are therefore required.
Collapse
|
44
|
Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9204474] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.
Collapse
|
45
|
3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9204364] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
Remote pulse rate measurement from facial video has gained particular attention over the last few years. Research exhibits significant advancements and demonstrates that common video cameras correspond to reliable devices that can be employed to measure a large set of biomedical parameters without any contact with the subject. A new framework for measuring and mapping pulse rate from video is presented in this pilot study. The method, which relies on convolutional 3D networks, is fully automatic and does not require any special image preprocessing. In addition, the network ensures concurrent mapping by producing a prediction for each local group of pixels. A particular training procedure that employs only synthetic data is proposed. Preliminary results demonstrate that this convolutional 3D network can effectively extract pulse rate from video without the need for any processing of frames. The trained model was compared with other state-of-the-art methods on public data. Results exhibit significant agreement between estimated and ground-truth measurements: the root mean square error computed from pulse rate values assessed with the convolutional 3D network is equal to 8.64 bpm, which is superior to 10 bpm for the other state-of-the-art methods. The robustness of the method to natural motion and increases in performance correspond to the two main avenues that will be considered in future works.
Collapse
|
46
|
Przybyło J. Continuous Distant Measurement of the User's Heart Rate in Human-Computer Interaction Applications. SENSORS 2019; 19:s19194205. [PMID: 31569798 PMCID: PMC6806289 DOI: 10.3390/s19194205] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/09/2019] [Revised: 08/30/2019] [Accepted: 09/25/2019] [Indexed: 11/25/2022]
Abstract
In real world scenarios, the task of estimating heart rate (HR) using video plethysmography (VPG) methods is difficult because many factors could contaminate the pulse signal (i.e., a subjects’ movement, illumination changes). This article presents the evaluation of a VPG system designed for continuous monitoring of the user’s heart rate during typical human-computer interaction scenarios. The impact of human activities while working at the computer (i.e., reading and writing text, playing a game) on the accuracy of HR VPG measurements was examined. Three commonly used signal extraction methods were evaluated: green (G), green-red difference (GRD), blind source separation (ICA). A new method based on an excess green (ExG) image representation was proposed. Three algorithms for estimating pulse rate were used: power spectral density (PSD), autoregressive modeling (AR) and time domain analysis (TIME). In summary, depending on the scenario being studied, different combinations of signal extraction methods and the pulse estimation algorithm ensure optimal heart rate detection results. The best results were obtained for the ICA method: average RMSE = 6.1 bpm (beats per minute). The proposed ExG signal representation outperforms other methods except ICA (RMSE = 11.2 bpm compared to 14.4 bpm for G and 13.0 bmp for GRD). ExG also is the best method in terms of proposed success rate metric (sRate).
Collapse
Affiliation(s)
- Jaromir Przybyło
- AGH University of Science and Technology, 30 Mickiewicza Ave., 30-059 Krakow, Poland.
| |
Collapse
|
47
|
Zaunseder S, Trumpp A, Wedekind D, Malberg H. Cardiovascular assessment by imaging photoplethysmography - a review. ACTA ACUST UNITED AC 2019; 63:617-634. [PMID: 29897880 DOI: 10.1515/bmt-2017-0119] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2017] [Accepted: 05/04/2018] [Indexed: 12/12/2022]
Abstract
Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique's background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.
Collapse
Affiliation(s)
- Sebastian Zaunseder
- TU Dresden, Institute of Biomedical Engineering, Helmholtzstraße 18, Dresden, 01069 Saxony, Germany
| | - Alexander Trumpp
- TU Dresden, Institute of Biomedical Engineering, Helmholtzstraße 18, Dresden, 01069 Saxony, Germany
| | - Daniel Wedekind
- TU Dresden, Institute of Biomedical Engineering, Helmholtzstraße 18, Dresden, 01069 Saxony, Germany
| | - Hagen Malberg
- TU Dresden, Institute of Biomedical Engineering, Helmholtzstraße 18, Dresden, 01069 Saxony, Germany
| |
Collapse
|
48
|
A System for Monitoring Breathing Activity Using an Ultrasonic Radar Detection with Low Power Consumption. JOURNAL OF SENSOR AND ACTUATOR NETWORKS 2019. [DOI: 10.3390/jsan8020032] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Continuous monitoring of breathing activity plays a major role in detecting and classifying a breathing abnormality. This work aims to facilitate detection of abnormal breathing syndromes, including tachypnea, bradypnea, central apnea, and irregular breathing by tracking of thorax movement resulting from respiratory rhythms based on ultrasonic radar detection. This paper proposes a non-contact, non-invasive, low cost, low power consumption, portable, and precise system for simultaneous monitoring of normal and abnormal breathing activity in real-time using an ultrasonic PING sensor and microcontroller PIC18F452. Moreover, the obtained abnormal breathing syndrome is reported to the concerned physician’s mobile telephone through a global system for mobile communication (GSM) modem to handle the case depending on the patient’s emergency condition. In addition, the power consumption of the proposed monitoring system is reduced via a duty cycle using an energy-efficient sleep/wake scheme. Experiments were conducted on 12 participants without any physical contact at different distances of 0.5, 1, 2, and 3 m and the breathing rates measured with the proposed system were then compared with those measured by a piezo respiratory belt transducer. The experimental results illustrate the feasibility of the proposed system to extract breathing rate and detect the related abnormal breathing syndromes with a high degree of agreement, strong correlation coefficient, and low error ratio. The results also showed that the total current consumption of the proposed monitoring system based on the sleep/wake scheme was 6.936 mA compared to 321.75 mA when the traditional operation was used instead. Consequently, this led to a 97.8% of power savings and extended the battery life time from 8 h to approximately 370 h. The proposed monitoring system could be used in both clinical and home settings.
Collapse
|
49
|
Rahman H, Ahmed MU, Begum S. Non-Contact Physiological Parameters Extraction Using Facial Video Considering Illumination, Motion, Movement and Vibration. IEEE Trans Biomed Eng 2019; 67:88-98. [PMID: 31095471 DOI: 10.1109/tbme.2019.2908349] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
OBJECTIVE In this paper, four physiological parameters, i.e., heart rate (HR), inter-beat-interval (IBI), heart rate variability (HRV), and oxygen saturation (SpO2), are extracted from facial video recordings. METHODS Facial videos were recorded for 10 min each in 30 test subjects while driving a simulator. Four regions of interest (ROIs) are automatically selected in each facial image frame based on 66 facial landmarks. Red-green-blue color signals are extracted from the ROIs and four physiological parameters are extracted from the color signals. For the evaluation, physiological parameters are also recorded simultaneously using a traditional sensor "cStress," which is attached to hands and fingers of test subjects. RESULTS The Bland Altman plots show 95% agreement between the camera system and "cStress" with the highest correlation coefficient R = 0.96 for both HR and SpO2. The quality index is estimated for IBI considering 100 ms R-peak error; the accumulated percentage achieved is 97.5%. HRV features in both time and frequency domains are compared and the highest correlation coefficient achieved is 0.93. One-way analysis of variance test shows that there are no statistically significant differences between the measurements by camera and reference sensors. CONCLUSION These results present high degrees of accuracy of HR, IBI, HRV, and SpO2 extraction from facial image sequences. SIGNIFICANCE The proposed non-contact approach could broaden the dimensionality of physiological parameters extraction using cameras. This proposed method could be applied for driver monitoring application under realistic conditions, i.e., illumination, motion, movement, and vibration.
Collapse
|
50
|
Macwan R, Benezeth Y, Mansouri A. Heart rate estimation using remote photoplethysmography with multi-objective optimization. Biomed Signal Process Control 2019. [DOI: 10.1016/j.bspc.2018.10.012] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|