A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients

Contactless face video based vital signs detection framework for continuous health monitoring using feature optimization and hybrid neural network

Continuous monitoring of vital signs such as respiration and heart rate is essential to detect and predict conditions that may affect the patient's well-being. To detect these vital signs most medical systems use contact sensors. They are not feasible for long term monitoring and are not repeatable. Vital signs using facial video-noncontact monitoring are becoming increasingly important. Researchers in the last few years although considerable progress has been made, challenging datasets absence timing of assessment process and the technology still has some limitations such as time consuming nature and lack of computer portability. To solve those problems, we propose a contactless video based vital signs detection framework for continuous health monitoring using feature optimization and hybrid neural network. In the proposed technique, modified war strategy optimization algorithm is proposed to segment the face portion from the input video frames. Then, we utilize the known data acquisition models to extract vital signs from the segmented face portions are heart rate, blood pressure, respiratory rate and oxygen saturation. An improved neural network structure (Lifting Net) is further used to achieve the adaptive extraction of deep hidden features for specific signs, for realizing the high precision of human health monitoring. The Hughes effect or dimensionality issue affects detection accuracy in sign classification when there are fewer training instances relative to the number of spectral features. The problem can be overcome through feature optimization here Northern goshawk optimization algorithm is used to select optimal best features which reduces the data dimensionality issue. Furthermore, hybrid deep ensemble reinforcement learning classifier is proposed for the human vital sign detection and classification which ensures the early detection of patient abnormality. Finally, we validate our framework using benchmark video datasets such as TokyoTechrPPG, PURE and COHFACE. To proves the effectiveness of proposed technique using simulation results and comparative analysis.

Non-invasive evaluation of vascular permeability in formalin-induced orofacial pain model using infrared thermography

Enhanced vascular permeability at the site of injury is a prominent feature in acute inflammatory pain models, commonly assessed through the Evans Blue test. However, this invasive test requires euthanasia, thereby precluding further investigations on the same animal. Due to these limitations, the integration of non-invasive tools such as IRT has been sought. Here, we aimed to evaluate the use of thermography in a common orofacial pain model that employs formalin as a chemical irritant to induce local orofacial inflammation. Male Hannover rats (290-300 g, N = 43) were used. In the first approach, radiometric images were taken before and after formalin administration, assessing temperature changes and extravasated Evans Blue. The second approach included capturing pre- and post-formalin test radiometric images, followed by cytokine measurements in excised vibrissae tissue. Rats were anesthetized for vibrissae tissue collection, allowing correlations between thermographic patterns, nocifensive behavior duration, and cytokine levels in this area. Our findings revealed a positive correlation between local temperature, measured via thermography, and vascular permeability in the contralateral (r2 = 0.3483) and ipsilateral (r2 = 0.4502) side, measured using spectrophotometry. The obtained data supports the notion that thermography-based temperature assessment can effectively evaluate vascular permeability in the orofacial region.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Exploring alternative approaches to precision medicine through genomics and artificial intelligence - a systematic review

The core idea behind precision medicine is to pinpoint the subpopulations that differ from one another in terms of disease risk, drug responsiveness, and treatment outcomes due to differences in biology and other traits. Biomarkers are found through genomic sequencing. Multi-dimensional clinical and biological data are created using these biomarkers. Better analytic methods are needed for these multidimensional data, which can be accomplished by using artificial intelligence (AI). An updated review of 80 latest original publications is presented on four main fronts-preventive medicine, medication development, treatment outcomes, and diagnostic medicine-All these studies effectively illustrated the significance of AI in precision medicine. Artificial intelligence (AI) has revolutionized precision medicine by swiftly analyzing vast amounts of data to provide tailored treatments and predictive diagnostics. Through machine learning algorithms and high-resolution imaging, AI assists in precise diagnoses and early disease detection. AI's ability to decode complex biological factors aids in identifying novel therapeutic targets, allowing personalized interventions and optimizing treatment outcomes. Furthermore, AI accelerates drug discovery by navigating chemical structures and predicting drug-target interactions, expediting the development of life-saving medications. With its unrivaled capacity to comprehend and interpret data, AI stands as an invaluable tool in the pursuit of enhanced patient care and improved health outcomes. It's evident that AI can open a new horizon for precision medicine by translating complex data into actionable information. To get better results in this regard and to fully exploit the great potential of AI, further research is required on this pressing subject.

AI-Guided Computing Insights into a Thermostat Monitoring Neonatal Intensive Care Unit (NICU)

In any healthcare setting, it is important to monitor and control airflow and ventilation with a thermostat. Computational fluid dynamics (CFD) simulations can be carried out to investigate the airflow and heat transfer taking place inside a neonatal intensive care unit (NICU). In this present study, the NICU is modeled based on the realistic dimensions of a single-patient room in compliance with the appropriate square footage allocated per incubator. The physics of flow in NICU is predicted based on the Navier-Stokes conservation equations for an incompressible flow, according to suitable thermophysical characteristics of the climate. The results show sensible flow structures and heat transfer as expected from any indoor climate with this configuration. Furthermore, machine learning (ML) in an artificial intelligence (AI) model has been adopted to take the important geometric parameter values as input from our CFD settings. The model provides accurate predictions of the thermal performance (i.e., temperature evaluation) associated with that design in real time. Besides the geometric parameters, there are three thermophysical variables of interest: the mass flow rate (i.e., inlet velocity), the heat flux of the radiator (i.e., heat source), and the temperature gradient caused by the convection. These thermophysical variables have significantly recovered the physics of convective flows and enhanced the heat transfer throughout the incubator. Importantly, the AI model is not only trained to improve the turbulence modeling but also to capture the large temperature gradient occurring between the infant and surrounding air. These physics-informed (Pi) computing insights make the AI model more general by reproducing the flow of fluid and heat transfer with high levels of numerical accuracy. It can be concluded that AI can aid in dealing with large datasets such as those produced in NICU, and in turn, ML can identify patterns in data and help with the sensor readings in health care.

Non-intrusive RF sensing for early diagnosis of spinal curvature syndrome disorders

The recent developments in communication and information ease people's lives to sit in one place and access any information from anywhere. However, the longevity of sitting and sitting in different postures raises the issues of spinal curvature. It necessitates a physical examination to identify the spinal illness in its early stages. This article aims to develop an intelligent monitoring framework for detecting and monitoring spinal curvature syndrome problems based on Software Defined Radio Frequency (SDRF) sensing and verify its feasibility for diagnosing actual patients. The proposed SDRF-based system identifies irregular spinal curvature syndrome and offers feedback signals when an incorrect posture is identified. We design the system using wireless university software-defined radio peripheral (USRP) kits to transmit and receive RF signals and record the wireless channel state information (WCSI) for kyphosis, Lordosis, and scoliosis spinal disorders. The statistical measures are extracted from the WCSI and apply machine learning algorithms to identify and classify the type of disorders. We record and test the system using 11 subjects with the spinal disorders kyphosis, Lordosis, and scoliosis. We acquire the WCSI, extract various statistical measures in terms of time and frequency domain features, and evaluate machine learning classifiers to identify and classify the spinal disorder. The performance comparison of the machine learning algorithms showed overall and each spinal curvature disorder recognition accuracy of more than 99%.

VitRad: A low-cost continuous wave Doppler radar system with 3D-printed horn antennas for human vital sign detection

In this study, a low-cost continuous wave (CW) radar system with 3D-printed high-gain horn antennas called VitRad is proposed for human vital sign detection. The CW radar consists of 3D-printed high-gain horn antennas, commercially available low-cost surface-mounting devices, and monolithic ICs. The CW radar system operates at a frequency band of 5.8 GHz, and the backscattered I/Q data are collected using a digital storage oscilloscope (DSO). The data is processed on MATLAB to determine vital sign information such as respiratory and heartbeat rates. It is demonstrated that the proposed CW radar system for vital-sign monitoring can effectively measure respiratory and heartbeat rates.

Using a graph-based image segmentation algorithm for remote vital sign estimation and monitoring

Reliable and contactless measurements of vital signs, such as respiration and heart rate, are still unmet needs in clinical and home settings. Mm-wave radar and video-based technologies are promising, but currently, the signal processing-based vital sign extraction methods are prone to body motion disruptions or illumination variations in the surrounding environment. Here we propose an image segmentation-based method to extract vital signs from the recorded video and mm-wave radar signals. The proposed method analyses time–frequency spectrograms obtained from Short-Time Fourier Transform rather than individual time-domain signals. This leads to much-improved robustness and accuracy of the heart rate and respiration rate extraction over existing methods. The experiments were conducted under pre- and post-exercise conditions and were repeated on multiple individuals. The results are evaluated by using four metrics against the gold standard contact-based measurements. Significant improvements were observed in terms of precision, accuracy, and stability. The performance was reflected by achieving an averaged Pearson correlation coefficient (PCC) of 93.8% on multiple subjects. We believe that the proposed estimation method will help address the needs for the increasingly popular remote cardiovascular sensing and diagnosing posed by Covid-19.

Non-Invasive Data Acquisition and IoT Solution for Human Vital Signs Monitoring: Applications, Limitations and Future Prospects

The rapid development of technology has brought about a revolution in healthcare stimulating a wide range of smart and autonomous applications in homes, clinics, surgeries and hospitals. Smart healthcare opens the opportunity for a qualitative advance in the relations between healthcare providers and end-users for the provision of healthcare such as enabling doctors to diagnose remotely while optimizing the accuracy of the diagnosis and maximizing the benefits of treatment by enabling close patient monitoring. This paper presents a comprehensive review of non-invasive vital data acquisition and the Internet of Things in healthcare informatics and thus reports the challenges in healthcare informatics and suggests future work that would lead to solutions to address the open challenges in IoT and non-invasive vital data acquisition. In particular, the conducted review has revealed that there has been a daunting challenge in the development of multi-frequency vital IoT systems, and addressing this issue will help enable the vital IoT node to be reachable by the broker in multiple area ranges. Furthermore, the utilization of multi-camera systems has proven its high potential to increase the accuracy of vital data acquisition, but the implementation of such systems has not been fully developed with unfilled gaps to be bridged. Moreover, the application of deep learning to the real-time analysis of vital data on the node/edge side will enable optimal, instant offline decision making. Finally, the synergistic integration of reliable power management and energy harvesting systems into non-invasive data acquisition has been omitted so far, and the successful implementation of such systems will lead to a smart, robust, sustainable and self-powered healthcare system.

Camera fusion for real-time temperature monitoring of neonates using deep learning

Abstract

The continuous monitoring of vital signs is a crucial aspect of medical care in neonatal intensive care units. Since cable-based sensors pose a potential risk for the immature skin of preterm infants, unobtrusive monitoring techniques using camera systems are increasingly investigated. The combination of deep learning–based algorithms and camera modalities such as RGB and infrared thermography can improve the development of cable-free methods for the extraction of vital parameters. In this study, a real-time approach for local extraction of temperatures on the body surface of neonates using a multi-modal clinical dataset was implemented. Therefore, a trained deep learning–based keypoint detector was used for body landmark prediction in RGB. Image registration was conducted to transfer the RGB points to the corresponding thermographic recordings. These landmarks were used to extract the body surface temperature in various regions to determine the central-peripheral temperature difference. A validation of the keypoint detector showed a mean average precision of 0.82. The registration resulted in mean absolute errors of 16.4 px (8.2 mm) for x and 22.4 px (11.2 mm) for y. The evaluation of the temperature extraction revealed a mean absolute error of 0.55 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘C. A final performance of 31 fps was observed on the NVIDIA Jetson Xavier NX module, which proves real-time capability on an embedded GPU system. As a result, the approach can perform real-time temperature extraction on a low-cost GPU module.

Graphical abstract

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

Potentials and Challenges of Pervasive Sensing in the Intensive Care Unit

Patients in critical care settings often require continuous and multifaceted monitoring. However, current clinical monitoring practices fail to capture important functional and behavioral indices such as mobility or agitation. Recent advances in non-invasive sensing technology, high throughput computing, and deep learning techniques are expected to transform the existing patient monitoring paradigm by enabling and streamlining granular and continuous monitoring of these crucial critical care measures. In this review, we highlight current approaches to pervasive sensing in critical care and identify limitations, future challenges, and opportunities in this emerging field.

Deep Learning-Based Pain Classifier Based on the Facial Expression in Critically Ill Patients

Objective

Pain assessment based on facial expressions is an essential issue in critically ill patients, but an automated assessment tool is still lacking. We conducted this prospective study to establish the deep learning-based pain classifier based on facial expressions.

Methods

We enrolled critically ill patients during 2020–2021 at a tertiary hospital in central Taiwan and recorded video clips with labeled pain scores based on facial expressions, such as relaxed (0), tense (1), and grimacing (2). We established both image- and video-based pain classifiers through using convolutional neural network (CNN) models, such as Resnet34, VGG16, and InceptionV1 and bidirectional long short-term memory networks (BiLSTM). The performance of classifiers in the test dataset was determined by accuracy, sensitivity, and F1-score.

Results

A total of 63 participants with 746 video clips were eligible for analysis. The accuracy of using Resnet34 in the polychromous image-based classifier for pain scores 0, 1, 2 was merely 0.5589, and the accuracy of dichotomous pain classifiers between 0 vs. 1/2 and 0 vs. 2 were 0.7668 and 0.8593, respectively. Similar accuracy of image-based pain classifier was found using VGG16 and InceptionV1. The accuracy of the video-based pain classifier to classify 0 vs. 1/2 and 0 vs. 2 was approximately 0.81 and 0.88, respectively. We further tested the performance of established classifiers without reference, mimicking clinical scenarios with a new patient, and found the performance remained high.

Conclusions

The present study demonstrates the practical application of deep learning-based automated pain assessment in critically ill patients, and more studies are warranted to validate our findings.

A Setup for Camera-Based Detection of Simulated Pathological States Using a Neonatal Phantom

Premature infants are among the most vulnerable patients in a hospital. Due to numerous complications associated with immaturity, a continuous monitoring of vital signs with a high sensitivity and accuracy is required. Today, wired sensors are attached to the patient's skin. However, adhesive electrodes can be potentially harmful as they can damage the very thin immature skin. Although unobtrusive monitoring systems using cameras show the potential to replace cable-based techniques, advanced image processing algorithms are data-driven and, therefore, need much data to be trained. Due to the low availability of public neonatal image data, a patient phantom could help to implement algorithms for the robust extraction of vital signs from video recordings. In this work, a camera-based system is presented and validated using a neonatal phantom, which enabled a simulation of common neonatal pathologies such as hypo-/hyperthermia and brady-/tachycardia. The implemented algorithm was able to continuously measure and analyze the heart rate via photoplethysmography imaging with a mean absolute error of 0.91 bpm, as well as the distribution of a neonate's skin temperature with a mean absolute error of less than 0.55 °C. For accurate measurements, a temperature gain offset correction on the registered image from two infrared thermography cameras was performed. A deep learning-based keypoint detector was applied for temperature mapping and guidance for the feature extraction. The presented setup successfully detected several levels of hypo- and hyperthermia, an increased central-peripheral temperature difference, tachycardia and bradycardia.

Estimation of Tidal Volume Using Load Cells on a Hospital Bed

Although respiratory failure is one of the primary causes of admission to intensive care, the importance placed on measurement of respiratory parameters is commonly overshadowed compared to cardiac parameters. With the increased demand for unobtrusive yet quantifi- able respiratory monitoring, many technologies have been proposed recently. However, there are challenges to be addressed for such technologies to enable widespread use. In this work, we explore the feasibility of using load cell sensors embedded on a hospital bed for monitoring respi- ratory rate (RR) and tidal volume (TV). We propose a globalized machine learning (ML)-based algorithm for estimating TV without the requirement of subject-specific calibration or training. In a study of 15 healthy subjects performing respiratory tasks in four different postures, the outputs from four load cell channels and the reference spirometer were recorded simultaneously. A signal processing pipeline was implemented to extract features that capture respira- tory movement and the respiratory effects on the cardiac (i.e., ballistocardiogram, BCG) signals. The proposed RR estimation algorithm achieved a root mean square error (RMSE) of 0.6 breaths per minute (brpm) against the ground truth RR from the spirometer. The TV estimation results demonstrated that combining all three axes of the low- frequency force signals and the BCG heartbeat features best quantifies the respiratory effects of TV. The model resulted in a correlation and RMSE between the estimated and true TV values of 0.85 and 0.23 L, respectively, in the posture independent model without electrocardiogram (ECG) signals. This study suggests that load cell sensors already existing in certain hospital beds can be used for convenient and continuous respiratory monitoring in general care settings.

A method for improving semantic segmentation using thermographic images in infants

Background

Regulation of temperature is clinically important in the care of neonates because it has a significant impact on prognosis. Although probes that make contact with the skin are widely used to monitor temperature and provide spot central and peripheral temperature information, they do not provide details of the temperature distribution around the body. Although it is possible to obtain detailed temperature distributions using multiple probes, this is not clinically practical. Thermographic techniques have been reported for measurement of temperature distribution in infants. However, as these methods require manual selection of the regions of interest (ROIs), they are not suitable for introduction into clinical settings in hospitals. Here, we describe a method for segmentation of thermal images that enables continuous quantitative contactless monitoring of the temperature distribution over the whole body of neonates.

Methods

The semantic segmentation method, U-Net, was applied to thermal images of infants. The optimal combination of Weight Normalization, Group Normalization, and Flexible Rectified Linear Unit (FReLU) was evaluated. U-Net Generative Adversarial Network (U-Net GAN) was applied to thermal images, and a Self-Attention (SA) module was finally applied to U-Net GAN (U-Net GAN + SA) to improve precision. The semantic segmentation performance of these methods was evaluated.

Results

The optimal semantic segmentation performance was obtained with application of FReLU and Group Normalization to U-Net, showing accuracy of 92.9% and Mean Intersection over Union (mIoU) of 64.5%. U-Net GAN improved the performance, yielding accuracy of 93.3% and mIoU of 66.9%, and U-Net GAN + SA showed further improvement with accuracy of 93.5% and mIoU of 70.4%.

Conclusions

FReLU and Group Normalization are appropriate semantic segmentation methods for application to neonatal thermal images. U-Net GAN and U-Net GAN + SA significantly improved the mIoU of segmentation.

Implementation of Thermal Camera for Non-Contact Physiological Measurement: A Systematic Review

Non-contact physiological measurements based on image sensors have developed rapidly in recent years. Among them, thermal cameras have the advantage of measuring temperature in the environment without light and have potential to develop physiological measurement applications. Various studies have used thermal camera to measure the physiological signals such as respiratory rate, heart rate, and body temperature. In this paper, we provided a general overview of the existing studies by examining the physiological signals of measurement, the used platforms, the thermal camera models and specifications, the use of camera fusion, the image and signal processing step (including the algorithms and tools used), and the performance evaluation. The advantages and challenges of thermal camera-based physiological measurement were also discussed. Several suggestions and prospects such as healthcare applications, machine learning, multi-parameter, and image fusion, have been proposed to improve the physiological measurement of thermal camera in the future.

FluNet: An AI-Enabled Influenza-Like Warning System

Influenza is an acute viral respiratory disease that is currently causing severe financial and resource strains worldwide. With the COVID-19 pandemic exceeding 153 million cases worldwide, there is a need for a low-cost and contactless surveillance system to detect symptomatic individuals. The objective of this study was to develop FluNet, a novel, proof-of-concept, low-cost and contactless device for the detection of high-risk individuals. The system conducts face detection in the LWIR with a precision rating of 0.98, a recall of 0.91, an F-score of 0.96, and a mean intersection over union of 0.74 while sequentially taking the temperature trend of faces with a thermal accuracy of ± 1 K. In parallel, determining if someone is coughing by using a custom lightweight deep convolutional neural network with a precision rating of 0.95, a recall of 0.92, an F-score of 0.94 and an AUC of 0.98. We concluded this study by testing the accuracy of the direction of arrival estimation for the cough detection revealing an error of ± 4.78°. If a subject is symptomatic, a photo is taken with a specified region of interest using a visible light camera. Two datasets have been constructed, one for face detection in the LWIR consisting of 250 images of 20 participants' faces at various rotations and coverings, including face masks. The other for the real-time detection of coughs comprised of 40,482 cough / not cough sounds. These findings could be helpful for future low-cost edge computing applications for influenza-like monitoring.

Automatic Separation of Respiratory Flow from Motion in Thermal Videos for Infant Apnea Detection

Both Respiratory Flow (RF) and Respiratory Motion (RM) are visible in thermal recordings of infants. Monitoring these two signals usually requires landmark detection for the selection of a region of interest. Other approaches combine respiratory signals coming from both RF and RM, obtaining a Mixed Respiratory (MR) signal. The detection and classification of apneas, particularly common in preterm infants with low birth weight, would benefit from monitoring both RF and RM, or MR, signals. Therefore, we propose in this work an automatic RF pixel detector not based on facial/body landmarks. The method is based on the property of RF pixels in thermal videos, which are in areas with a smooth circular gradient. We defined 5 features combined with the use of a bank of Gabor filters that together allow selection of the RF pixels. The algorithm was tested on thermal recordings of 9 infants amounting to a total of 132 min acquired in a neonatal ward. On average the percentage of correctly identified RF pixels was 84%. Obstructive Apneas (OAs) were simulated as a proof of concept to prove the advantage in monitoring the RF signal compared to the MR signal. The sensitivity in the simulated OA detection improved for the RF signal reaching 73% against the 23% of the MR signal. Overall, the method yielded promising results, although the positioning and number of cameras used could be further optimized for optimal RF visibility.

Estimation of Respiratory Rate from Thermography Using Respiratory Likelihood Index

Respiration is a key vital sign used to monitor human health status. Monitoring respiratory rate (RR) under non-contact is particularly important for providing appropriate pre-hospital care in emergencies. We propose an RR estimation system using thermal imaging cameras, which are increasingly being used in the medical field, such as recently during the COVID-19 pandemic. By measuring temperature changes during exhalation and inhalation, we aim to track the respiration of the subject in a supine or seated position in real-time without any physical contact. The proposed method automatically selects the respiration-related regions from the detected facial regions and estimates the respiration rate. Most existing methods rely on signals from nostrils and require close-up or high-resolution images, while our method only requires the facial region to be captured. Facial region is detected using YOLO v3, an object detection model based on deep learning. The detected facial region is divided into subregions. By calculating the respiratory likelihood of each segmented region using the newly proposed index, called the Respiratory Quality Index, the respiratory region is automatically selected and the RR is estimated. An evaluation of the proposed RR estimation method was conducted on seven subjects in their early twenties, with four 15 s measurements being taken. The results showed a mean absolute error of 0.66 bpm. The proposed method can be useful as an RR estimation method.

A Heterogeneous Hardware Accelerator for Image Classification in Embedded Systems

Convolutional neural networks (CNN) have been extensively employed for image classification due to their high accuracy. However, inference is a computationally-intensive process that often requires hardware acceleration to operate in real time. For mobile devices, the power consumption of graphics processors (GPUs) is frequently prohibitive, and field-programmable gate arrays (FPGA) become a solution to perform inference at high speed. Although previous works have implemented CNN inference on FPGAs, their high utilization of on-chip memory and arithmetic resources complicate their application on resource-constrained edge devices. In this paper, we present a scalable, low power, low resource-utilization accelerator architecture for inference on the MobileNet V2 CNN. The architecture uses a heterogeneous system with an embedded processor as the main controller, external memory to store network data, and dedicated hardware implemented on reconfigurable logic with a scalable number of processing elements (PE). Implemented on a XCZU7EV FPGA running at 200 MHz and using four PEs, the accelerator infers with 87% top-5 accuracy and processes an image of 224×224 pixels in 220 ms. It consumes 7.35 W of power and uses less than 30% of the logic and arithmetic resources used by other MobileNet FPGA accelerators.