1
|
Cheng CH, Yuen Z, Chen S, Wong KL, Chin JW, Chan TT, So RHY. Contactless Blood Oxygen Saturation Estimation from Facial Videos Using Deep Learning. Bioengineering (Basel) 2024; 11:251. [PMID: 38534525 DOI: 10.3390/bioengineering11030251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 02/26/2024] [Accepted: 03/02/2024] [Indexed: 03/28/2024] Open
Abstract
Blood oxygen saturation (SpO2) is an essential physiological parameter for evaluating a person's health. While conventional SpO2 measurement devices like pulse oximeters require skin contact, advanced computer vision technology can enable remote SpO2 monitoring through a regular camera without skin contact. In this paper, we propose novel deep learning models to measure SpO2 remotely from facial videos and evaluate them using a public benchmark database, VIPL-HR. We utilize a spatial-temporal representation to encode SpO2 information recorded by conventional RGB cameras and directly pass it into selected convolutional neural networks to predict SpO2. The best deep learning model achieves 1.274% in mean absolute error and 1.71% in root mean squared error, which exceed the international standard of 4% for an approved pulse oximeter. Our results significantly outperform the conventional analytical Ratio of Ratios model for contactless SpO2 measurement. Results of sensitivity analyses of the influence of spatial-temporal representation color spaces, subject scenarios, acquisition devices, and SpO2 ranges on the model performance are reported with explainability analyses to provide more insights for this emerging research field.
Collapse
Affiliation(s)
- Chun-Hong Cheng
- Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK
| | - Zhikun Yuen
- Department of Computer Science, University of Ottawa, Ottawa, ON K1H 8M5, Canada
| | - Shutao Chen
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
| | - Kwan-Long Wong
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
| | - Jing-Wei Chin
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
| | - Tsz-Tai Chan
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
| | - Richard H Y So
- PanopticAI, Hong Kong Science and Technology Parks, New Territories, Hong Kong, China
- Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
| |
Collapse
|
2
|
Huang B, Hu S, Liu Z, Lin CL, Su J, Zhao C, Wang L, Wang W. Challenges and prospects of visual contactless physiological monitoring in clinical study. NPJ Digit Med 2023; 6:231. [PMID: 38097771 PMCID: PMC10721846 DOI: 10.1038/s41746-023-00973-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 11/21/2023] [Indexed: 12/17/2023] Open
Abstract
The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.
Collapse
Affiliation(s)
- Bin Huang
- AI Research Center, Hangzhou Innovation Institute, Beihang University, 99 Juhang Rd., Binjiang Dist., Hangzhou, Zhejiang, China.
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China.
| | - Shen Hu
- Department of Obstetrics, The Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
- Department of Epidemiology, The Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Zimeng Liu
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
| | - Chun-Liang Lin
- College of Electrical Engineering and Computer Science, National Chung Hsing University, 145 Xingda Rd., South Dist., Taichung, Taiwan.
| | - Junfeng Su
- Department of General Intensive Care Unit, The Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
- Key Laboratory of Early Warning and Intervention of Multiple Organ Failure, China National Ministry of Education, Hangzhou, Zhejiang, China
| | - Changchen Zhao
- AI Research Center, Hangzhou Innovation Institute, Beihang University, 99 Juhang Rd., Binjiang Dist., Hangzhou, Zhejiang, China
| | - Li Wang
- Department of Rehabilitation Medicine, The First Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
| | - Wenjin Wang
- Department of Biomedical Engineering, Southern University of Science and Technology, 1088 Xueyuan Ave, Nanshan Dist., Shenzhen, Guangdong, China.
| |
Collapse
|