1
|
Jansen TS, Güney G, Ganse B, Monje MHG, Schulz JB, Dafotakis M, Hoog Antink C, Braczynski AK. Video-based analysis of the blink reflex in Parkinson's disease patients. Biomed Eng Online 2024; 23:43. [PMID: 38654246 PMCID: PMC11036732 DOI: 10.1186/s12938-024-01236-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 04/10/2024] [Indexed: 04/25/2024] Open
Abstract
We developed a video-based tool to quantitatively assess the Glabellar Tap Reflex (GTR) in patients with idiopathic Parkinson's disease (iPD) as well as healthy age-matched participants. We also video-graphically assessed the effect of dopaminergic medication on the GTR in iPD patients, as well as the frequency and blinking duration of reflex and non-reflex blinks. The Glabellar Tap Reflex is a clinical sign seen in patients e.g. suffering from iPD. Reliable tools to quantify this sign are lacking. METHODS We recorded the GTR in 11 iPD patients and 12 healthy controls (HC) with a consumer-grade camera at a framerate of at least 180 images/s. In these videos, reflex and non-reflex blinks were analyzed for blink count and blinking duration in an automated fashion. RESULTS With our setup, the GTR can be extracted from high-framerate cameras using landmarks of the MediaPipe face algorithm. iPD patients did not habituate to the GTR; dopaminergic medication did not alter that response. iPD patients' non-reflex blinks were higher in frequency and higher in blinking duration (width at half prominence); dopaminergic medication decreased the median frequency (Before medication-HC: p < 0.001, After medication-HC: p = 0.0026) and decreased the median blinking duration (Before medication-HC: p = 0.8594, After medication-HC: p = 0.6943)-both in the direction of HC. CONCLUSION We developed a quantitative, video-based tool to assess the GTR and other blinking-specific parameters in HC and iPD patients. Further studies could compare the video data to electromyogram (EMG) data for accuracy and comparability, as well as evaluate the specificity of the GTR in patients with other neurodegenerative disorders, in whom the GTR can also be present. SIGNIFICANCE The video-based detection of the blinking parameters allows for unobtrusive measurement in patients, a safer and more comfortable option.
Collapse
Affiliation(s)
- Talisa S Jansen
- Department of Neurology, RWTH University Hospital, Aachen, Germany
- Department of Child and Adolescent Psychiatry, Psychosomatics, and Psychotherapy, Medical Faculty, RWTH Aachen University, Aachen, Germany
| | - Gökhan Güney
- KIS*MED (AI Systems in Medicine Lab) Technische Universität Darmstadt, Darmstadt, Germany
| | - Bergita Ganse
- Innovative Implant Development, Saarland University, Homburg, Germany
| | - Mariana H G Monje
- Department of Neurology, Northwestern University Feinberg School of Medicine, Chicago, USA
| | - Jörg B Schulz
- Department of Neurology, RWTH University Hospital, Aachen, Germany
- Jülich Aachen Research Alliance (JARA), JARA-Institute Molecular Neuroscience and Neuroimaging, FZ Jülich and RWTH University, Jülich, Germany
| | - Manuel Dafotakis
- Department of Neurology, RWTH University Hospital, Aachen, Germany
| | - Christoph Hoog Antink
- KIS*MED (AI Systems in Medicine Lab) Technische Universität Darmstadt, Darmstadt, Germany.
| | - Anne K Braczynski
- Department of Neurology, RWTH University Hospital, Aachen, Germany
- Institut für Physikalische Biologie, Düsseldorf, Heinrich-Heine University, Düsseldorf, Germany
- Institute of Biological Information Processing (IBI-7: Structural Biochemistry), Forschungszentrum Jülich, Jülich, Germany
| |
Collapse
|
2
|
Huang B, Hu S, Liu Z, Lin CL, Su J, Zhao C, Wang L, Wang W. Challenges and prospects of visual contactless physiological monitoring in clinical study. NPJ Digit Med 2023; 6:231. [PMID: 38097771 PMCID: PMC10721846 DOI: 10.1038/s41746-023-00973-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 11/21/2023] [Indexed: 12/17/2023] Open
Abstract
The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.
Collapse
Affiliation(s)
- Bin Huang
- AI Research Center, Hangzhou Innovation Institute, Beihang University, 99 Juhang Rd., Binjiang Dist., Hangzhou, Zhejiang, China.
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China.
| | - Shen Hu
- Department of Obstetrics, The Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
- Department of Epidemiology, The Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Zimeng Liu
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
| | - Chun-Liang Lin
- College of Electrical Engineering and Computer Science, National Chung Hsing University, 145 Xingda Rd., South Dist., Taichung, Taiwan.
| | - Junfeng Su
- Department of General Intensive Care Unit, The Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
- Key Laboratory of Early Warning and Intervention of Multiple Organ Failure, China National Ministry of Education, Hangzhou, Zhejiang, China
| | - Changchen Zhao
- AI Research Center, Hangzhou Innovation Institute, Beihang University, 99 Juhang Rd., Binjiang Dist., Hangzhou, Zhejiang, China
| | - Li Wang
- Department of Rehabilitation Medicine, The First Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
| | - Wenjin Wang
- Department of Biomedical Engineering, Southern University of Science and Technology, 1088 Xueyuan Ave, Nanshan Dist., Shenzhen, Guangdong, China.
| |
Collapse
|
3
|
Zhang B, Dong L, Kong L, Liu M, Zhao Y, Hui M, Chu X. Prediction of Impulsive Aggression Based on Video Images. Bioengineering (Basel) 2023; 10:942. [PMID: 37627827 PMCID: PMC10451168 DOI: 10.3390/bioengineering10080942] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 07/27/2023] [Accepted: 08/01/2023] [Indexed: 08/27/2023] Open
Abstract
In response to the subjectivity, low accuracy, and high concealment of existing attack behavior prediction methods, a video-based impulsive aggression prediction method that integrates physiological parameters and facial expression information is proposed. This method uses imaging equipment to capture video and facial expression information containing the subject's face and uses imaging photoplethysmography (IPPG) technology to obtain the subject's heart rate variability parameters. Meanwhile, the ResNet-34 expression recognition model was constructed to obtain the subject's facial expression information. Based on the random forest classification model, the physiological parameters and facial expression information obtained are used to predict individual impulsive aggression. Finally, an impulsive aggression induction experiment was designed to verify the method. The experimental results show that the accuracy of this method for predicting the presence or absence of impulsive aggression was 89.39%. This method proves the feasibility of applying physiological parameters and facial expression information to predict impulsive aggression. This article has important theoretical and practical value for exploring new impulsive aggression prediction methods. It also has significance in safety monitoring in special and public places such as prisons and rehabilitation centers.
Collapse
Affiliation(s)
- Borui Zhang
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
| | - Liquan Dong
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
- Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
| | - Lingqin Kong
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
- Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
| | - Ming Liu
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
- Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
| | - Yuejin Zhao
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
- Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
| | - Mei Hui
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
| | - Xuhong Chu
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing 100081, China
- Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
| |
Collapse
|
4
|
Rahman MM, Cook J, Taebi A. Non-contact heart vibration measurement using computer vision-based seismocardiography. Sci Rep 2023; 13:11787. [PMID: 37479720 PMCID: PMC10362031 DOI: 10.1038/s41598-023-38607-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Accepted: 07/11/2023] [Indexed: 07/23/2023] Open
Abstract
Seismocardiography (SCG) is the noninvasive measurement of local vibrations of the chest wall produced by the mechanical activity of the heart and has shown promise in providing clinical information for certain cardiovascular diseases including heart failure and ischemia. Conventionally, SCG signals are recorded by placing an accelerometer on the chest. In this paper, we propose a novel contactless SCG measurement method to extract them from chest videos recorded by a smartphone. Our pipeline consists of computer vision methods including the Lucas-Kanade template tracking to track an artificial target attached to the chest, and then estimate the SCG signals from the tracked displacements. We evaluated our pipeline on 14 healthy subjects by comparing the vision-based SCG[Formula: see text] estimations with the gold-standard SCG[Formula: see text] measured simultaneously using accelerometers attached to the chest. The similarity between SCG[Formula: see text] and SCG[Formula: see text] was measured in the time and frequency domains using the Pearson correlation coefficient, a similarity index based on dynamic time warping (DTW), and wavelet coherence. The average DTW-based similarity index between the signals was 0.94 and 0.95 in the right-to-left and head-to-foot directions, respectively. Furthermore, SCG[Formula: see text] signals were utilized to estimate the heart rate, and these results were compared to the gold-standard heart rate obtained from ECG signals. The findings indicated a good agreement between the estimated heart rate values and the gold-standard measurements (bias = 0.649 beats/min). In conclusion, this work shows promise in developing a low-cost and widely available method for remote monitoring of cardiovascular activity using smartphone videos.
Collapse
Affiliation(s)
- Mohammad Muntasir Rahman
- Department of Agricultural and Biological Engineering, Mississippi State University, Mississippi, 39762, USA
| | - Jadyn Cook
- Department of Agricultural and Biological Engineering, Mississippi State University, Mississippi, 39762, USA
| | - Amirtahà Taebi
- Department of Agricultural and Biological Engineering, Mississippi State University, Mississippi, 39762, USA.
| |
Collapse
|
5
|
Boiko A, Martínez Madrid N, Seepold R. Contactless Technologies, Sensors, and Systems for Cardiac and Respiratory Measurement during Sleep: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23115038. [PMID: 37299762 DOI: 10.3390/s23115038] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 05/22/2023] [Accepted: 05/23/2023] [Indexed: 06/12/2023]
Abstract
Sleep is essential to physical and mental health. However, the traditional approach to sleep analysis-polysomnography (PSG)-is intrusive and expensive. Therefore, there is great interest in the development of non-contact, non-invasive, and non-intrusive sleep monitoring systems and technologies that can reliably and accurately measure cardiorespiratory parameters with minimal impact on the patient. This has led to the development of other relevant approaches, which are characterised, for example, by the fact that they allow greater freedom of movement and do not require direct contact with the body, i.e., they are non-contact. This systematic review discusses the relevant methods and technologies for non-contact monitoring of cardiorespiratory activity during sleep. Taking into account the current state of the art in non-intrusive technologies, we can identify the methods of non-intrusive monitoring of cardiac and respiratory activity, the technologies and types of sensors used, and the possible physiological parameters available for analysis. To do this, we conducted a literature review and summarised current research on the use of non-contact technologies for non-intrusive monitoring of cardiac and respiratory activity. The inclusion and exclusion criteria for the selection of publications were established prior to the start of the search. Publications were assessed using one main question and several specific questions. We obtained 3774 unique articles from four literature databases (Web of Science, IEEE Xplore, PubMed, and Scopus) and checked them for relevance, resulting in 54 articles that were analysed in a structured way using terminology. The result was 15 different types of sensors and devices (e.g., radar, temperature sensors, motion sensors, cameras) that can be installed in hospital wards and departments or in the environment. The ability to detect heart rate, respiratory rate, and sleep disorders such as apnoea was among the characteristics examined to investigate the overall effectiveness of the systems and technologies considered for cardiorespiratory monitoring. In addition, the advantages and disadvantages of the considered systems and technologies were identified by answering the identified research questions. The results obtained allow us to determine the current trends and the vector of development of medical technologies in sleep medicine for future researchers and research.
Collapse
Affiliation(s)
- Andrei Boiko
- Ubiquitous Computing Laboratory, Department of Computer Science, HTWG Konstanz-University of Applied Sciences, Alfred-Wachtel-Str. 8, 78462 Konstanz, Germany
| | - Natividad Martínez Madrid
- Internet of Things Laboratory, School of Informatics, Reutlingen University, Alteburgstr. 150, 72762 Reutlingen, Germany
| | - Ralf Seepold
- Ubiquitous Computing Laboratory, Department of Computer Science, HTWG Konstanz-University of Applied Sciences, Alfred-Wachtel-Str. 8, 78462 Konstanz, Germany
| |
Collapse
|
6
|
Bautista MJ, Kowal M, Cave DGW, Downey C, Jayne DG. Clinical applications of contactless photoplethysmography for monitoring in adults: A systematic review and meta-analysis. J Clin Transl Sci 2023; 7:e129. [PMID: 37313385 PMCID: PMC10260340 DOI: 10.1017/cts.2023.547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 04/24/2023] [Accepted: 05/05/2023] [Indexed: 06/15/2023] Open
Abstract
Contactless photoplethysmography (cPPG) is a method of physiological monitoring. It differs from conventional monitoring methods (e.g., saturation probe) by ensuring no contact with the subject by use of a camera. The majority of research on cPPG is conducted in a laboratory setting or in healthy populations. This review aims to evaluate the current literature on monitoring using cPPG in adults within a clinical setting. Adhering to the Preferred Items for Systematic Reviews and Meta-analysis (PRISMA, 2020) guidelines, OVID, Webofscience, Cochrane library, and clinicaltrials.org were systematically searched by two researchers. Research articles using cPPG for monitoring purposes in adults within a clinical setting were selected. Twelve studies with a total of 654 individuals were included. Heart rate (HR) was the most investigated vital sign (n = 8) followed by respiratory rate ((n = 2), Sp02 (n = 2), and HR variability (n = 2). Four studies were included in a meta-analysis of HR compared to ECG data which demonstrated a mean bias of -0.13 (95% CI, -1.22-0.96). This study demonstrates cPPG can be a useful tool in the remote monitoring of patients and has demonstrated accuracy for HR. However, further research is needed into the clinical applications of this method.
Collapse
|
7
|
Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey. INFORMATICS 2022. [DOI: 10.3390/informatics9030057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Over the last few years, a rich amount of research has been conducted on remote vital sign monitoring of the human body. Remote photoplethysmography (rPPG) is a camera-based, unobtrusive technology that allows continuous monitoring of changes in vital signs and thereby helps to diagnose and treat diseases earlier in an effective manner. Recent advances in computer vision and its extensive applications have led to rPPG being in high demand. This paper specifically presents a survey on different remote photoplethysmography methods and investigates all facets of heart rate analysis. We explore the investigation of the challenges of the video-based rPPG method and extend it to the recent advancements in the literature. We discuss the gap within the literature and suggestions for future directions.
Collapse
|
8
|
Selvaraju V, Spicher N, Swaminathan R, Deserno TM. Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:2967-2971. [PMID: 36085768 DOI: 10.1109/embc48229.2022.9871416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.
Collapse
|
9
|
Li CH, Ly FS, Woodhouse K, Chen J, Cheng Z, Santander T, Ashar N, Turki E, Yang HT, Miller M, Petzold L, Hansma PK. Dynamic Phase Extraction: Applications in Pulse Rate Variability. Appl Psychophysiol Biofeedback 2022; 47:213-222. [PMID: 35704121 DOI: 10.1007/s10484-022-09549-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/06/2022] [Indexed: 11/02/2022]
Abstract
Pulse rate variability is a physiological parameter that has been extensively studied and correlated with many physical ailments. However, the phase relationship between inter-beat interval, IBI, and breathing has very rarely been studied. Develop a technique by which the phase relationship between IBI and breathing can be accurately and efficiently extracted from photoplethysmography (PPG) data. A program based on Lock-in Amplifier technology was written in Python to implement a novel technique, Dynamic Phase Extraction. It was tested using a breath pacer and a PPG sensor on 6 subjects who followed a breath pacer at varied breathing rates. The data were then analyzed using both traditional methods and the novel technique (Dynamic Phase Extraction) utilizing a breath pacer. Pulse data was extracted using a PPG sensor. Dynamic Phase Extraction (DPE) gave the magnitudes of the variation in IBI associated with breathing [Formula: see text] measured with photoplethysmography during paced breathing (with premature ventricular contractions, abnormal arrhythmias, and other artifacts edited out). [Formula: see text] correlated well with two standard measures of pulse rate variability: the Standard Deviation of the inter-beat interval (SDNN) (ρ = 0.911) and with the integrated value of the Power Spectral Density between 0.04 and 0.15 Hz (Low Frequency Power or LF Power) (ρ = 0.885). These correlations were comparable to the correlation between the SDNN and the LF Power (ρ = 0.877). In addition to the magnitude [Formula: see text], Dynamic Phase Extraction also gave the phase between the breath pacer and the changes in the inter-beat interval (IBI) due to respiratory sinus arrythmia (RSA), and correlated well with the phase extracted using a Fourier transform (ρ = 0.857). Dynamic Phase Extraction can extract both the phase between the breath pacer and the changes in IBI due to the respiratory sinus arrhythmia component of pulse rate variability ([Formula: see text], but is limited by needing a breath pacer.
Collapse
Affiliation(s)
- Christopher H Li
- Department of Physics, University of California, Santa Barbara, Santa Barbara, USA.
| | - Franklin S Ly
- Department of Mechanical Engineering, University of California, Santa Barbara, Santa Barbara, USA
| | - Kegan Woodhouse
- Department of Mechanical Engineering, University of California, Santa Barbara, Santa Barbara, USA
| | - John Chen
- Department of Mechanical Engineering, University of California, Santa Barbara, Santa Barbara, USA
| | - Zhuowei Cheng
- Department of Computer Science, University of California, Santa Barbara, Santa Barbara, USA
| | - Tyler Santander
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, Santa Barbara, USA
| | - Nirmit Ashar
- Department of Computer Science, University of California, Santa Barbara, Santa Barbara, USA
| | - Elyes Turki
- Department of Physics, University of California, Santa Barbara, Santa Barbara, USA
| | - Henry T Yang
- Department of Mechanical Engineering, University of California, Santa Barbara, Santa Barbara, USA
| | - Michael Miller
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, Santa Barbara, USA
| | - Linda Petzold
- Department of Mechanical Engineering, University of California, Santa Barbara, Santa Barbara, USA.,Department of Computer Science, University of California, Santa Barbara, Santa Barbara, USA
| | - Paul K Hansma
- Department of Physics, University of California, Santa Barbara, Santa Barbara, USA.,Neuroscience Research Institute, University of California, Santa Barbara, Santa Barbara, USA
| |
Collapse
|
10
|
Gupta A, Ravelo-García AG, Dias FM. Availability and performance of face based non-contact methods for heart rate and oxygen saturation estimations: A systematic review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106771. [PMID: 35390724 DOI: 10.1016/j.cmpb.2022.106771] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 03/03/2022] [Accepted: 03/21/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND Consumer-level cameras have provided an advantage of designing cost-effective, non-contact physiological parameters estimation approaches which is not possible with gold standard estimation techniques. This encourages the development of non-contact estimation methods using camera technology. Therefore, this work aims to present a systematic review summarizing the currently existing face-based non-contact methods along with their performance. METHODS This review includes all heart rate (HR) and oxygen saturation (SpO2) studies published in journals and a few reputed conferences, which have compared the proposed estimation methods with one or more standard reference devices. The articles were collected from the following research databases: Institute of Electrical and Electronics Engineers (IEEE), PubMed, Web of Science (WoS), Science Direct, and Association of Computer Machinery (ACM) digital library. All database searches were completed on May 20, 2021. Each study was assessed using a finite set of identified factors for reporting bias. RESULTS Out of 332 identified studies, 32 studies were selected for the final review. Additionally, 18 studies were included by thoroughly checking these studies. 3 out of 50 (6%) studies were performed in clinical conditions, while the remaining studies were carried out on a healthy population. 42 out of 50 (84%) studies have estimated HR, while 5/50 (10%) studies have measured SpO2 only. The remaining three studies have estimated both parameters. The majority of the studies have used 1-3 min videos for estimation. Among the estimation methods, Deep Learning and Independent component analysis (ICA) were used by 11/42 (26.19%) and 9/42 (21.42%) studies, respectively. According to the Bland-Altman analysis, only 8/45 (17.77%) HR studies achieved the clinically accepted error limits whereas, for SpO2, 4/5 (80%) studies have matched the industry standards (±3%). DISCUSSION Deep Learning and ICA have been predominantly used for HR estimations. Among deep learning estimation methods, convolutional neural networks have been employed till date due to their good generalization ability. Most non-contact HR estimation methods need significant improvements to implement these methods in a clinical environment. Furthermore, these methods need to be tested on the subjects suffering from any related disease. SpO2 estimation studies are challenging and need to be tested by conducting hypoxemic events. The authors would encourage reporting the detailed information about the study population, the use of longer videos, and appropriate performance metrics and testing under abnormal HR and SpO2 ranges for future estimation studies.
Collapse
Affiliation(s)
- Ankit Gupta
- Interactive Technologies Institute/Larsys/Madeira Interactive Technologies Institute, Caminho da Penteada, Funchal, 9020-105, Portugal; Universidade da Madeira, Caminho da Penteada, Funchal, 9020-105, Portugal.
| | - Antonio G Ravelo-García
- Interactive Technologies Institute/Larsys/Madeira Interactive Technologies Institute, Caminho da Penteada, Funchal, 9020-105, Portugal; Universidad de Las Palmas de Gran Canaria, C. Juan de Quesada, 30, Las Palmas, 35001, Spain.
| | - Fernando Morgado Dias
- Interactive Technologies Institute/Larsys/Madeira Interactive Technologies Institute, Caminho da Penteada, Funchal, 9020-105, Portugal; Universidade da Madeira, Caminho da Penteada, Funchal, 9020-105, Portugal.
| |
Collapse
|
11
|
Faust O, Hong W, Loh HW, Xu S, Tan RS, Chakraborty S, Barua PD, Molinari F, Acharya UR. Heart rate variability for medical decision support systems: A review. Comput Biol Med 2022; 145:105407. [DOI: 10.1016/j.compbiomed.2022.105407] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Revised: 03/09/2022] [Accepted: 03/12/2022] [Indexed: 12/22/2022]
|
12
|
Selvaraju V, Spicher N, Wang J, Ganapathy N, Warnecke JM, Leonhardt S, Swaminathan R, Deserno TM. Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:4097. [PMID: 35684717 PMCID: PMC9185528 DOI: 10.3390/s22114097] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 05/18/2022] [Accepted: 05/18/2022] [Indexed: 02/04/2023]
Abstract
In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.
Collapse
Affiliation(s)
- Vinothini Selvaraju
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
- Non-Invasive Imaging and Diagnostic Laboratory, Biomedical Engineering, Department of Applied Mechanics, Indian Institute of Technology Madras, Chennai 600036, India;
| | - Nicolai Spicher
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Ju Wang
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Nagarajan Ganapathy
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Joana M. Warnecke
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| | - Steffen Leonhardt
- Chair for Medical Information Technology, Helmholtz-Institute for Biomedical Engineering, RWTH Aachen University, D-52074 Aachen, Germany;
| | - Ramakrishnan Swaminathan
- Non-Invasive Imaging and Diagnostic Laboratory, Biomedical Engineering, Department of Applied Mechanics, Indian Institute of Technology Madras, Chennai 600036, India;
| | - Thomas M. Deserno
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, D-38106 Braunschweig, Germany; (V.S.); (N.S.); (J.W.); (N.G.); (J.M.W.)
| |
Collapse
|
13
|
Chen Q, Wang Y, Liu X, Long X, Yin B, Chen C, Chen W. Camera-based heart rate estimation for hospitalized newborns in the presence of motion artifacts. Biomed Eng Online 2021; 20:122. [PMID: 34863194 PMCID: PMC8642856 DOI: 10.1186/s12938-021-00958-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Accepted: 11/15/2021] [Indexed: 02/07/2023] Open
Abstract
Background Heart rate (HR) is an important vital sign for evaluating the physiological condition of a newborn infant. Recently, for measuring HR, novel RGB camera-based non-contact techniques have demonstrated their specific superiority compared with other techniques, such as dopplers and thermal cameras. However, they still suffered poor robustness in infants’ HR measurements due to frequent body movement. Methods This paper introduces a framework to improve the robustness of infants’ HR measurements by solving motion artifact problems. Our solution is based on the following steps: morphology-based filtering, region-of-interest (ROI) dividing, Eulerian video magnification and majority voting. In particular, ROI dividing improves ROI information utilization. The majority voting scheme improves the statistical robustness by choosing the HR with the highest probability. Additionally, we determined the dividing parameter that leads to the most accurate HR measurements. In order to examine the performance of the proposed method, we collected 4 hours of videos and recorded the corresponding electrocardiogram (ECG) of 9 hospitalized neonates under two different conditions—rest still and visible movements. Results Experimental results indicate a promising performance: the mean absolute error during rest still and visible movements are 3.39 beats per minute (BPM) and 4.34 BPM, respectively, which improves at least 2.00 and 1.88 BPM compared with previous works. The Bland-Altman plots also show the remarkable consistency of our results and the HR derived from the ground-truth ECG. Conclusions To the best of our knowledge, this is the first study aimed at improving the robustness of neonatal HR measurement under motion artifacts using an RGB camera. The preliminary results have shown the promising prospects of the proposed method, which hopefully reduce neonatal mortality in hospitals.
Collapse
Affiliation(s)
- Qiong Chen
- Center for Intelligent Medical Electronics, School of Information Science and Technology, Fudan University, Shanghai, China
| | - Yalin Wang
- Center for Intelligent Medical Electronics, School of Information Science and Technology, Fudan University, Shanghai, China
| | - Xiangyu Liu
- School of Art Design and Media, East China University of Science and Technology, Shanghai, China
| | - Xi Long
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands.
| | - Bin Yin
- Connected Care and Personal Health Department, Philips Research, Shanghai, China
| | - Chen Chen
- Human Phenome Institute, Fudan University, Shanghai, China.
| | - Wei Chen
- Center for Intelligent Medical Electronics, School of Information Science and Technology, Fudan University, Shanghai, China.
| |
Collapse
|
14
|
Purnomo AT, Lin DB, Adiprabowo T, Hendria WF. Non-Contact Monitoring and Classification of Breathing Pattern for the Supervision of People Infected by COVID-19. SENSORS 2021; 21:s21093172. [PMID: 34063576 PMCID: PMC8125653 DOI: 10.3390/s21093172] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 04/26/2021] [Accepted: 04/28/2021] [Indexed: 02/06/2023]
Abstract
During the pandemic of coronavirus disease-2019 (COVID-19), medical practitioners need non-contact devices to reduce the risk of spreading the virus. People with COVID-19 usually experience fever and have difficulty breathing. Unsupervised care to patients with respiratory problems will be the main reason for the rising death rate. Periodic linearly increasing frequency chirp, known as frequency-modulated continuous wave (FMCW), is one of the radar technologies with a low-power operation and high-resolution detection which can detect any tiny movement. In this study, we use FMCW to develop a non-contact medical device that monitors and classifies the breathing pattern in real time. Patients with a breathing disorder have an unusual breathing characteristic that cannot be represented using the breathing rate. Thus, we created an Xtreme Gradient Boosting (XGBoost) classification model and adopted Mel-frequency cepstral coefficient (MFCC) feature extraction to classify the breathing pattern behavior. XGBoost is an ensemble machine-learning technique with a fast execution time and good scalability for predictions. In this study, MFCC feature extraction assists machine learning in extracting the features of the breathing signal. Based on the results, the system obtained an acceptable accuracy. Thus, our proposed system could potentially be used to detect and monitor the presence of respiratory problems in patients with COVID-19, asthma, etc.
Collapse
Affiliation(s)
- Ariana Tulus Purnomo
- Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan;
- Correspondence: (A.T.P.); (D.-B.L.)
| | - Ding-Bing Lin
- Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan;
- Correspondence: (A.T.P.); (D.-B.L.)
| | - Tjahjo Adiprabowo
- Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan;
| | - Willy Fitra Hendria
- Department of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea;
| |
Collapse
|
15
|
Saner H, Knobel SEJ, Schuetz N, Nef T. Contact-free sensor signals as a new digital biomarker for cardiovascular disease: chances and challenges. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2020; 1:30-39. [PMID: 36713967 PMCID: PMC9707864 DOI: 10.1093/ehjdh/ztaa006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/03/2020] [Revised: 09/26/2020] [Accepted: 11/18/2020] [Indexed: 02/01/2023]
Abstract
Multiple sensor systems are used to monitor physiological parameters, activities of daily living and behaviour. Digital biomarkers can be extracted and used as indicators for health and disease. Signal acquisition is either by object sensors, wearable sensors, or contact-free sensors including cameras, pressure sensors, non-contact capacitively coupled electrocardiogram (cECG), radar, and passive infrared motion sensors. This review summarizes contemporary knowledge of the use of contact-free sensors for patients with cardiovascular disease and healthy subjects following the PRISMA declaration. Chances and challenges are discussed. Thirty-six publications were rated to be of medium (31) or high (5) relevance. Results are best for monitoring of heart rate and heart rate variability using cardiac vibration, facial camera, or cECG; for respiration using cardiac vibration, cECG, or camera; and for sleep using ballistocardiography. Early results from radar sensors to monitor vital signs are promising. Contact-free sensors are little invasive, well accepted and suitable for long-term monitoring in particular in patient's homes. A major problem are motion artefacts. Results from long-term use in larger patient cohorts are still lacking, but the technology is about to emerge the market and we can expect to see more clinical results in the near future.
Collapse
Affiliation(s)
- Hugo Saner
- ARTORG Center for Biomedical Engineering Research, University of Bern, Murtenstrasse 50, CH 3008 Bern, Switzerland,Department of Preventive Cardiology, University Hospital Bern, Inselspital, Freiburgstrasse 18, CH 3010 Bern, Switzerland,Corresponding author. Tel: +41 79 209 11 82,
| | - Samuel Elia Johannes Knobel
- ARTORG Center for Biomedical Engineering Research, University of Bern, Murtenstrasse 50, CH 3008 Bern, Switzerland
| | - Narayan Schuetz
- ARTORG Center for Biomedical Engineering Research, University of Bern, Murtenstrasse 50, CH 3008 Bern, Switzerland
| | - Tobias Nef
- ARTORG Center for Biomedical Engineering Research, University of Bern, Murtenstrasse 50, CH 3008 Bern, Switzerland,Department of Neurology, University Hospital Bern, Inselspital, Freiburgstrasse 18, CH 3010 Bern, Switzerland
| |
Collapse
|