Non-contact physiological monitoring of preterm infants in the Neonatal Intensive Care Unit

Deep learning to quantify care manipulation activities in neonatal intensive care units

Early-life exposure to stress results in significantly increased risk of neurodevelopmental impairments with potential long-term effects into childhood and even adulthood. As a crucial step towards monitoring neonatal stress in neonatal intensive care units (NICUs), our study aims to quantify the duration, frequency, and physiological responses of care manipulation activities, based on bedside videos and physiological signals. Leveraging 289 h of video recordings and physiological data within 330 sessions collected from 27 neonates in 2 NICUs, we develop and evaluate a deep learning method to detect manipulation activities from the video, to estimate their duration and frequency, and to further integrate physiological signals for assessing their responses. With a 13.8% relative error tolerance for activity duration and frequency, our results were statistically equivalent to human annotations. Further, our method proved effective for estimating short-term physiological responses, for detecting activities with marked physiological deviations, and for quantifying the neonatal infant stressor scale scores.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

Artificial Intelligence-Based Diagnostic Support System for Patent Ductus Arteriosus in Premature Infants

Background: Patent ductus arteriosus (PDA) is a prevalent congenital heart defect in premature infants, associated with significant morbidity and mortality. Accurate and timely diagnosis of PDA is crucial, given the vulnerability of this population. Methods: We introduce an artificial intelligence (AI)-based PDA diagnostic support system designed to assist medical professionals in diagnosing PDA in premature infants. This study utilized electronic health record (EHR) data from 409 premature infants spanning a decade at Severance Children's Hospital. Our system integrates a data viewer, data analyzer, and AI-based diagnosis supporter, facilitating comprehensive data presentation, analysis, and early symptom detection. Results: The system's performance was evaluated through diagnostic tests involving medical professionals. This early detection model achieved an accuracy rate of up to 84%, enabling detection up to 3.3 days in advance. In diagnostic tests, medical professionals using the system with the AI-based diagnosis supporter outperformed those using the system without the supporter. Conclusions: Our AI-based PDA diagnostic support system offers a comprehensive solution for medical professionals to accurately diagnose PDA in a timely manner in premature infants. The collaborative integration of medical expertise and technological innovation demonstrated in this study underscores the potential of AI-driven tools in advancing neonatal diagnosis and care.

Health Care Professionals' and Parents' Perspectives on the Use of AI for Pain Monitoring in the Neonatal Intensive Care Unit: Multisite Qualitative Study

BACKGROUND

The use of artificial intelligence (AI) for pain assessment has the potential to address historical challenges in infant pain assessment. There is a dearth of information on the perceived benefits and barriers to the implementation of AI for neonatal pain monitoring in the neonatal intensive care unit (NICU) from the perspective of health care professionals (HCPs) and parents. This qualitative analysis provides novel data obtained from 2 large tertiary care hospitals in Canada and the United Kingdom.

OBJECTIVE

The aim of the study is to explore the perspectives of HCPs and parents regarding the use of AI for pain assessment in the NICU.

METHODS

In total, 20 HCPs and 20 parents of preterm infants were recruited and consented to participate from February 2020 to October 2022 in interviews asking about AI use for pain assessment in the NICU, potential benefits of the technology, and potential barriers to use.

RESULTS

The 40 participants included 20 HCPs (17 women and 3 men) with an average of 19.4 (SD 10.69) years of experience in the NICU and 20 parents (mean age 34.4, SD 5.42 years) of preterm infants who were on average 43 (SD 30.34) days old. Six themes from the perspective of HCPs were identified: regular use of technology in the NICU, concerns with regard to AI integration, the potential to improve patient care, requirements for implementation, AI as a tool for pain assessment, and ethical considerations. Seven parent themes included the potential for improved care, increased parental distress, support for parents regarding AI, the impact on parent engagement, the importance of human care, requirements for integration, and the desire for choice in its use. A consistent theme was the importance of AI as a tool to inform clinical decision-making and not replace it.

CONCLUSIONS

HCPs and parents expressed generally positive sentiments about the potential use of AI for pain assessment in the NICU, with HCPs highlighting important ethical considerations. This study identifies critical methodological and ethical perspectives from key stakeholders that should be noted by any team considering the creation and implementation of AI for pain monitoring in the NICU.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Unobstructive Heartbeat Monitoring of Sleeping Infants and Young Children Using Sheet-Type PVDF Sensors

Techniques for noninvasively acquiring the vital information of infants and young children are considered very useful in the fields of healthcare and medical care. An unobstructive measurement method for sleeping infants and young children under the age of 6 years using a sheet-type vital sensor with a polyvinylidene fluoride (PVDF) pressure-sensitive layer is demonstrated. The signal filter conditions to obtain the ballistocardiogram (BCG) and phonocardiogram (PCG) are discussed from the waveform data of infants and young children. The difference in signal processing conditions was caused by the physique of the infants and young children. The peak-to-peak interval (PPI) extracted from the BCG or PCG during sleep showed an extremely high correlation with the R-to-R interval (RRI) extracted from the electrocardiogram (ECG). The vital changes until awakening in infants monitored using a sheet sensor were also investigated. In infants under one year of age that awakened spontaneously, the distinctive vital changes during awakening were observed. Understanding the changes in the heartbeat and respiration signs of infants and young children during sleep is essential for improving the accuracy of abnormality detection by unobstructive sensors.

Multimodality Video Acquisition System for the Assessment of Vital Distress in Children

In children, vital distress events, particularly respiratory, go unrecognized. To develop a standard model for automated assessment of vital distress in children, we aimed to construct a prospective high-quality video database for critically ill children in a pediatric intensive care unit (PICU) setting. The videos were acquired automatically through a secure web application with an application programming interface (API). The purpose of this article is to describe the data acquisition process from each PICU room to the research electronic database. Using an Azure Kinect DK and a Flir Lepton 3.5 LWIR attached to a Jetson Xavier NX board and the network architecture of our PICU, we have implemented an ongoing high-fidelity prospectively collected video database for research, monitoring, and diagnostic purposes. This infrastructure offers the opportunity to develop algorithms (including computational models) to quantify vital distress in order to evaluate vital distress events. More than 290 RGB, thermographic, and point cloud videos of each 30 s have been recorded in the database. Each recording is linked to the patient's numerical phenotype, i.e., the electronic medical health record and high-resolution medical database of our research center. The ultimate goal is to develop and validate algorithms to detect vital distress in real time, both for inpatient care and outpatient management.

Predicting the need for intubation within 3 h in the neonatal intensive care unit using a multimodal deep neural network

Respiratory distress is a common chief complaint in neonates admitted to the neonatal intensive care unit. Despite the increasing use of non-invasive ventilation in neonates with respiratory difficulty, some of them require advanced airway support. Delayed intubation is associated with increased morbidity, particularly in urgent unplanned cases. Early and accurate prediction of the need for intubation may provide more time for preparation and increase safety margins by avoiding the late intubation at high-risk infants. This study aimed to predict the need for intubation within 3 h in neonates initially managed with non-invasive ventilation for respiratory distress during the first 48 h of life using a multimodal deep neural network. We developed a multimodal deep neural network model to simultaneously analyze four time-series data collected at 1-h intervals and 19 variables including demographic, physiological and laboratory parameters. Evaluating the dataset of 128 neonates with respiratory distress who underwent non-invasive ventilation, our model achieved an area under the curve of 0.917, sensitivity of 85.2%, and specificity of 89.2%. These findings demonstrate promising results for the multimodal model in predicting neonatal intubation within 3 h.

Device for remote and realtime monitoring of neonatal vital signs in neonatal intensive care unit using internet of things: proof-of-concept study

BACKGROUND

Realtime and remote monitoring of neonatal vital signs is a crucial part of providing appropriate care in neonatal intensive care units (NICU) to reduce mortality and morbidity of newborns. In this study, a new approach, a device for remote and real-time monitoring of neonatal vital signs (DRRMNVS) in the neonatal intensive care unit using the internet of things (IoT), was proposed. The system integrates four vital signs: oxygen saturation, pulse rate, body temperature and respiration rate for continuous monitoring using the Blynk app and ThingSpeak IoT platforms.

METHODS

The Wemos D1 mini, a Wi-Fi microcontroller, was used to acquire the four biological biomarkers from sensors, process them and display the result on an OLED display for point of care monitoring and on the Blynk app and ThingSpeak for remote and continuous monitoring of vital signs. The Bland-Altman test was employed to test the agreement of DRRMNVS measurement with reference standards by taking measurements from ten healthy adults.

RESULTS

The prototype of the proposed device was successfully developed and tested. Bias [limits of agreement] were: Oxygen saturation (SpO2): -0.1 [- 1.546 to + 1.346] %; pulse rate: -0.3 [- 2.159 to + 1.559] bpm; respiratory rate: -0.7 [- 0.247 to + 1.647] breaths/min; temperature: 0.21 [+ 0.015˚C to + 0.405˚C] ˚C. The proof-of-concept prototype was developed for $33.19.

CONCLUSION

The developed DRRMNVS device was cheap and had acceptable measurement accuracy of vital signs in a controlled environment. The system has the potential to advance healthcare service delivery for neonates with further development from this proof-of-concept level.

Multi-modal body part segmentation of infants using deep learning

BACKGROUND

Monitoring the body temperature of premature infants is vital, as it allows optimal temperature control and may provide early warning signs for severe diseases such as sepsis. Thermography may be a non-contact and wireless alternative to state-of-the-art, cable-based methods. For monitoring use in clinical practice, automatic segmentation of the different body regions is necessary due to the movement of the infant.

METHODS

This work presents and evaluates algorithms for automatic segmentation of infant body parts using deep learning methods. Based on a U-Net architecture, three neural networks were developed and compared. While the first two only used one imaging modality (visible light or thermography), the third applied a feature fusion of both. For training and evaluation, a dataset containing 600 visible light and 600 thermography images from 20 recordings of infants was created and manually labeled. In addition, we used transfer learning on publicly available datasets of adults in combination with data augmentation to improve the segmentation results.

RESULTS

Individual optimization of the three deep learning models revealed that transfer learning and data augmentation improved segmentation regardless of the imaging modality. The fusion model achieved the best results during the final evaluation with a mean Intersection-over-Union (mIoU) of 0.85, closely followed by the RGB model. Only the thermography model achieved a lower accuracy (mIoU of 0.75). The results of the individual classes showed that all body parts were well-segmented, only the accuracy on the torso is inferior since the models struggle when only small areas of the skin are visible.

CONCLUSION

The presented multi-modal neural networks represent a new approach to the problem of infant body segmentation with limited available data. Robust results were obtained by applying feature fusion, cross-modality transfer learning and classical augmentation strategies.

For Heart Rate Assessments from Drone Footage in Disaster Scenarios

The ability to use drones to obtain important vital signs could be very valuable for emergency personnel during mass-casualty incidents. The rapid and robust remote assessment of heart rates could serve as a life-saving decision aid for first-responders. With the flight sensor data of a specialized drone, a pipeline was developed to achieve a robust, non-contact assessment of heart rates through remote photoplethysmography (rPPG). This robust assessment was achieved through adaptive face-aware exposure and comprehensive de-noising of a large number of predicted noise sources. In addition, we performed a proof-of-concept study that involved 18 stationary subjects with clean skin and 36 recordings of their vital signs, using the developed pipeline in outdoor conditions. In this study, we could achieve a single-value heart-rate assessment with an overall root-mean-squared error of 14.3 beats-per-minute, demonstrating the basic feasibility of our approach. However, further research is needed to verify the applicability of our approach in actual disaster situations, where remote photoplethysmography readings could be impacted by other factors, such as blood, dirt, and body positioning.

Novel approaches to capturing and using continuous cardiorespiratory physiological data in hospitalized children

Continuous cardiorespiratory physiological monitoring is a cornerstone of care in hospitalized children. The data generated by monitoring devices coupled with machine learning could transform the way we provide care. This scoping review summarizes existing evidence on novel approaches to continuous cardiorespiratory monitoring in hospitalized children. We aimed to identify opportunities for the development of monitoring technology and the use of machine learning to analyze continuous physiological data to improve the outcomes of hospitalized children. We included original research articles published on or after January 1, 2001, involving novel approaches to collect and use continuous cardiorespiratory physiological data in hospitalized children. OVID Medline, PubMed, and Embase databases were searched. We screened 2909 articles and performed full-text extraction of 105 articles. We identified 58 articles describing novel devices or approaches, which were generally small and single-center. In addition, we identified 47 articles that described the use of continuous physiological data in prediction models, but only 7 integrated multidimensional data (e.g., demographics, laboratory results). We identified three areas for development: (1) further validation of promising novel devices; (2) more studies of models integrating multidimensional data with continuous cardiorespiratory data; and (3) further dissemination, implementation, and validation of prediction models using continuous cardiorespiratory data. IMPACT: We performed a comprehensive scoping review of novel approaches to capture and use continuous cardiorespiratory physiological data for monitoring, diagnosis, providing care, and predicting events in hospitalized infants and children, from novel devices to machine learning-based prediction models. We identified three key areas for future development: (1) further validation of promising novel devices; (2) more studies of models integrating multidimensional data with continuous cardiorespiratory data; and (3) further dissemination, implementation, and validation of prediction models using cardiorespiratory data.

Artificial intelligence-driven wearable technologies for neonatal cardiorespiratory monitoring. Part 2: artificial intelligence

BACKGROUND

With the development of Artificial Intelligence (AI) techniques, smart health monitoring, particularly neonatal cardiorespiratory monitoring with wearable devices, is becoming more popular. To this end, it is crucial to investigate the trend of AI and wearable sensors being developed in this domain.

METHODS

We performed a review of papers published in IEEE Xplore, Scopus, and PubMed from the year 2000 onwards, to understand the use of AI for neonatal cardiorespiratory monitoring with wearable technologies. We reviewed the advances in AI development for this application and potential future directions. For this review, we assimilated machine learning (ML) algorithms developed for neonatal cardiorespiratory monitoring, designed a taxonomy, and categorised the methods based on their learning capabilities and performance.

RESULTS

For AI related to wearable technologies for neonatal cardio-respiratory monitoring, 63% of studies utilised traditional ML techniques and 35% utilised deep learning techniques, including 6% that applied transfer learning on pre-trained models.

CONCLUSIONS

A detailed review of AI methods for neonatal cardiorespiratory wearable sensors is presented along with their advantages and disadvantages. Hierarchical models and suggestions for future developments are highlighted to translate these AI technologies into patient benefit.

IMPACT

State-of-the-art review in artificial intelligence used for wearable neonatal cardiorespiratory monitoring. Taxonomy design for artificial intelligence methods. Comparative study of AI methods based on their advantages and disadvantages.

Non-contact respiratory rate monitoring using thermal and visible imaging: a pilot study on neonates

Respiratory rate (RR) monitoring is essential in neonatal intensive care units. Despite its importance, RR is still monitored intermittently by manual counting instead of continuous monitoring due to the risk of skin damage with prolonged use of contact electrodes in preterm neonates and false signals due to displacement of electrodes. Thermal imaging has recently gained significance as a non-contact method for RR detection because of its many advantages. However, due to the lack of information in thermal images, the selection and tracking of the region of interest (ROI) in thermal images for neonates are challenging. This paper presents the integration of visible (RGB) and thermal (T) image sequences for the selection and tracking of ROI for breathing rate extraction. The deep-learning based tracking-by-detection approach is employed to detect the ROI in the RGB images, and it is mapped to the thermal images using the RGB-T image registration. The mapped ROI in thermal spectrum sequences gives the respiratory rate. The study was conducted first on healthy adults in different modes, including steady, motion, talking, and variable respiratory order. Subsequently, the method is tested on neonates in a clinical settings. The findings have been validated with a contact-based reference method.The average absolute error between the proposed and belt-based contact method in healthy adults reached 0.1 bpm and for more challenging conditions was approximately 1.5 bpm and 1.8 bpm, respectively. In the case of neonates, the average error is 1.5 bpm, which are promising results. The Bland–Altman analysis showed a good agreement of estimated RR with the reference method RR and this pilot study provided the evidence of using the proposed approach as a contactless method for the respiratory rate detection of neonates in clinical settings.

Contactless skin perfusion monitoring with video cameras: tracking pharmacological vasoconstriction and vasodilation using photoplethysmographic changes

Objectives.Clinical assessment of skin perfusion informs prognosis in critically ill patients. Video camera monitoring could provide an objective, continuous method to monitor skin perfusion. In this prospective, interventional study of healthy volunteers, we tested whether video camera-derived photoplethysmography imaging and colour measurements could detect drug-induced skin perfusion changes.Approach.We monitored the lower limbs of 30 volunteers using video cameras while administering phenylephrine (a vasoconstrictor) and glyceryl trinitrate (a vasodilator). We report relative pixel intensity changes from baseline, as absolute values are sensitive to environmental factors. The primary outcome was the pre- to peak- infusion green channel amplitude change in the pulsatile PPGi waveform component. Secondary outcomes were pre-to-peak changes in the photoplethysmographic imaging waveform baseline, skin colour hue and skin colour saturation.Main results.The 30 participants had a median age of 29 years (IQR 25-34), sixteen (53%) were male. A 34.7% (p= 0.0001) mean decrease in the amplitude of the pulsatile photoplethysmographic imaging waveform occurred following phenylephrine infusion. A 30.7% (p= 0.000004) mean increase occurred following glyceryl trinitrate infusion. The photoplethysmographic imaging baseline decreased with phenylephrine by 2.1% (p= 0.000 02) and increased with glyceryl trinitrate by 0.5% (p= 0.026). Skin colour hue changed in opposite direction with phenylephrine (-0.0013,p= 0.0002) and glyceryl trinitrate (+0.0006,p= 0.019). Skin colour saturation decreased with phenylephrine by 0.0022 (p= 0.0002), with no significant change observed with glyceryl trinitrate (+0.0005,p= 0.21).Significance.Drug-induced vasoconstriction and vasodilation are associated with detectable changes in photoplethysmographic imaging waveform parameters and skin hue. Our findings suggest video cameras have great potential for continuous, contactless skin perfusion monitoring.

Deep learning based non-contact physiological monitoring in Neonatal Intensive Care Unit

Preterm babies in the Neonatal Intensive Care Unit (NICU) have to undergo continuous monitoring of their cardiac health. Conventional monitoring approaches are contact-based, making the neonates prone to various nosocomial infections. Video-based monitoring approaches have opened up potential avenues for contactless measurement. This work presents a pipeline for remote estimation of cardiopulmonary signals from videos in NICU setup. We have proposed an end-to-end deep learning (DL) model that integrates a non-learning-based approach to generate surrogate ground truth (SGT) labels for supervision, thus refraining from direct dependency on true ground truth labels. We have performed an extended qualitative and quantitative analysis to examine the efficacy of our proposed DL-based pipeline and achieved an overall average mean absolute error of 4.6 beats per minute (bpm) and root mean square error of 6.2 bpm in the estimated heart rate.

Camera fusion for real-time temperature monitoring of neonates using deep learning

Abstract

The continuous monitoring of vital signs is a crucial aspect of medical care in neonatal intensive care units. Since cable-based sensors pose a potential risk for the immature skin of preterm infants, unobtrusive monitoring techniques using camera systems are increasingly investigated. The combination of deep learning–based algorithms and camera modalities such as RGB and infrared thermography can improve the development of cable-free methods for the extraction of vital parameters. In this study, a real-time approach for local extraction of temperatures on the body surface of neonates using a multi-modal clinical dataset was implemented. Therefore, a trained deep learning–based keypoint detector was used for body landmark prediction in RGB. Image registration was conducted to transfer the RGB points to the corresponding thermographic recordings. These landmarks were used to extract the body surface temperature in various regions to determine the central-peripheral temperature difference. A validation of the keypoint detector showed a mean average precision of 0.82. The registration resulted in mean absolute errors of 16.4 px (8.2 mm) for x and 22.4 px (11.2 mm) for y. The evaluation of the temperature extraction revealed a mean absolute error of 0.55 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘C. A final performance of 31 fps was observed on the NVIDIA Jetson Xavier NX module, which proves real-time capability on an embedded GPU system. As a result, the approach can perform real-time temperature extraction on a low-cost GPU module.

Graphical abstract

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

Contactless radar-based breathing monitoring of premature infants in the neonatal intensive care unit

Vital sign monitoring systems are essential in the care of hospitalized neonates. Due to the immaturity of their organs and immune system, premature infants require continuous monitoring of their vital parameters and sensors need to be directly attached to their fragile skin. Besides mobility restrictions and stress, these sensors often cause skin irritation and may lead to pressure necrosis. In this work, we show that a contactless radar-based approach is viable for breathing monitoring in the Neonatal intensive care unit (NICU). For the first time, different scenarios common to the NICU daily routine are investigated, and the challenges of monitoring in a real clinical setup are addressed through different contributions in the signal processing framework. Rather than just discarding measurements under strong interference, we present a novel random body movement mitigation technique based on the time-frequency decomposition of the recovered signal. In addition, we propose a simple and accurate frequency estimator which explores the harmonic structure of the breathing signal. As a result, the proposed radar-based solution is able to provide reliable breathing frequency estimation, which is close to the reference cabled device values most of the time. Our findings shed light on the strengths and limitations of this technology and lay the foundation for future studies toward a completely contactless solution for vital signs monitoring.

Diagnostic Features and Potential Applications of PPG Signal in Healthcare: A Systematic Review

Recent research indicates that Photoplethysmography (PPG) signals carry more information than oxygen saturation level (SpO2) and can be utilized for affordable, fast, and noninvasive healthcare applications. All these encourage the researchers to estimate its feasibility as an alternative to many expansive, time-wasting, and invasive methods. This systematic review discusses the current literature on diagnostic features of PPG signal and their applications that might present a potential venue to be adapted into many health and fitness aspects of human life. The research methodology is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines 2020. To this aim, papers from 1981 to date are reviewed and categorized in terms of the healthcare application domain. Along with consolidated research areas, recent topics that are growing in popularity are also discovered. We also highlight the potential impact of using PPG signals on an individual's quality of life and public health. The state-of-the-art studies suggest that in the years to come PPG wearables will become pervasive in many fields of medical practices, and the main domains include cardiology, respiratory, neurology, and fitness. Main operation challenges, including performance and robustness obstacles, are identified.

The newborn delivery room of tomorrow: emerging and future technologies

Advances in neonatal care have resulted in improved outcomes for high-risk newborns with technologies playing a significant part although many were developed for the neonatal intensive care unit. The care provided in the delivery room (DR) during the first few minutes of life can impact short- and long-term neonatal outcomes. Increasingly, technologies have a critical role to play in the DR particularly with monitoring and information provision. However, the DR is a unique environment and has major challenges around the period of foetal to neonatal transition that need to be overcome when developing new technologies. This review focuses on current DR technologies as well as those just emerging and further over the horizon. We identify what key opinion leaders in DR care think of current technologies, what the important DR measures are to them, and which technologies might be useful in the future. We link these with key technologies including respiratory function monitors, electoral impedance tomography, videolaryngoscopy, augmented reality, video recording, eye tracking, artificial intelligence, and contactless monitoring. Encouraging funders and industry to address the unique technological challenges of newborn care in the DR will allow the continued improvement of outcomes of high-risk infants from the moment of birth. IMPACT: Technological advances for newborn delivery room care require consideration of the unique environment, the variable patient characteristics, and disease states, as well as human factor challenges. Neonatology as a speciality has embraced technology, allowing its rapid progression and improved outcomes for infants, although innovation in the delivery room often lags behind that in the intensive care unit. Investing in new and emerging technologies can support healthcare providers when optimising care and could improve training, safety, and neonatal outcomes.

Using High-Fidelity Avatars to Advance Camera-based Cardiac Pulse Measurement

Non-contact physiological measurement has the potential to provide low-cost, non-invasive health monitoring. However, machine vision approaches are often limited by the availability and diversity of annotated video datasets resulting in poor generalization to complex real-life conditions. To address these challenges, this work proposes the use of synthetic avatars that display facial blood flow changes and allow for systematic generation of samples under a wide variety of conditions. Our results show that training on both simulated and real video data can lead to performance gains under challenging conditions. We show strong performance on three large benchmark datasets and improved robustness to skin type and motion. These results highlight the promise of synthetic data for training camera-based pulse measurement; however, further research and validation is needed to establish whether synthetic data alone could be sufficient for training models.

A Setup for Camera-Based Detection of Simulated Pathological States Using a Neonatal Phantom

Premature infants are among the most vulnerable patients in a hospital. Due to numerous complications associated with immaturity, a continuous monitoring of vital signs with a high sensitivity and accuracy is required. Today, wired sensors are attached to the patient's skin. However, adhesive electrodes can be potentially harmful as they can damage the very thin immature skin. Although unobtrusive monitoring systems using cameras show the potential to replace cable-based techniques, advanced image processing algorithms are data-driven and, therefore, need much data to be trained. Due to the low availability of public neonatal image data, a patient phantom could help to implement algorithms for the robust extraction of vital signs from video recordings. In this work, a camera-based system is presented and validated using a neonatal phantom, which enabled a simulation of common neonatal pathologies such as hypo-/hyperthermia and brady-/tachycardia. The implemented algorithm was able to continuously measure and analyze the heart rate via photoplethysmography imaging with a mean absolute error of 0.91 bpm, as well as the distribution of a neonate's skin temperature with a mean absolute error of less than 0.55 °C. For accurate measurements, a temperature gain offset correction on the registered image from two infrared thermography cameras was performed. A deep learning-based keypoint detector was applied for temperature mapping and guidance for the feature extraction. The presented setup successfully detected several levels of hypo- and hyperthermia, an increased central-peripheral temperature difference, tachycardia and bradycardia.

Non-contact physiological monitoring of post-operative patients in the intensive care unit

Prolonged non-contact camera-based monitoring in critically ill patients presents unique challenges, but may facilitate safe recovery. A study was designed to evaluate the feasibility of introducing a non-contact video camera monitoring system into an acute clinical setting. We assessed the accuracy and robustness of the video camera-derived estimates of the vital signs against the electronically-recorded reference values in both day and night environments. We demonstrated non-contact monitoring of heart rate and respiratory rate for extended periods of time in 15 post-operative patients. Across day and night, heart rate was estimated for up to 53.2% (103.0 h) of the total valid camera data with a mean absolute error (MAE) of 2.5 beats/min in comparison to two reference sensors. We obtained respiratory rate estimates for 63.1% (119.8 h) of the total valid camera data with a MAE of 2.4 breaths/min against the reference value computed from the chest impedance pneumogram. Non-contact estimates detected relevant changes in the vital-sign values between routine clinical observations. Pivotal respiratory events in a post-operative patient could be identified from the analysis of video-derived respiratory information. Continuous vital-sign monitoring supported by non-contact video camera estimates could be used to track early signs of physiological deterioration during post-operative care.

Measuring respiratory and heart rate using a fiber optic interferometer: A pilot study in a neonate model

INTRODUCTION

The study aim was to test the safety and efficacy of a pad with optic fibers developed for monitoring newborn respiratory rate (RR) and heart rate (HR).

METHODS

Thirty New Zealand White rabbits were included, divided by weight into three groups. RR and HR were measured using two methods for each rabbit: ECG electrodes as the reference method and a newly developed pad with an experimental fiber optic system (EFOS) as the experimental method.

RESULTS

Analysis was performed on data for 29 rabbits (10 female, 34%; 19 male, 66%). EFOS performed better at measuring RR compared with HR. RR values did not differ significantly between the methods for the whole group (p = 0.151) or within each sex (female: p > 0.999; male: p = 0.075). Values for HR, however, did differ between methods for the whole group of animals (p < 0.001) and also within groups by sex (female: p < 0.001; male: p = 0.006).

CONCLUSION

The results of this preclinical study demonstrate the potential of this non-invasive method using a fiber optic pad to measure HR and RR.

Contactless Monitoring of Breathing Pattern and Thoracoabdominal Asynchronies in Preterm Infants Using Depth Cameras: A Feasibility Study

Objective: Monitoring infants’ breathing activity is crucial in research and clinical applications but remains a challenge. This study aims to develop a contactless method to monitor breathing patterns and thoracoabdominal asynchronies in infants inside the incubator, using depth cameras. Methods: We proposed an algorithm to extract the 3D displacements of the ribcage and abdomen from the analysis of depth images. We evaluated the accuracy of the system in-vitro vs. a reference motion capture analyzer. We also conducted a feasibility study on 12 patients receiving non-invasive respiratory support to estimate the mean and the variability of the chest wall displacements in preterm infants and evaluate the suitability of the proposed system in the clinical setting. Results: In-vitro, the mean (95% CI) error in the measurement of amplitude, frequency and phase shift between compartmental displacements was −0.14 (−0.57, 0.28) mm, 0.02 (−0.99, 1.03) bpm, and −0.40 (−1.76, 0.95)°, respectively. In-vivo, the mean (95% CI) amplitude of the ribcage and abdomen displacements were 0.99 (0.34, 2.67) mm and 1.20 (0.40, 2.15) mm, respectively. Conclusions: The developed system proved accurate in-vitro and was suitable for the clinical environment. Clinical Impact: The proposed method has value for evaluating infants’ breathing patterns in research applications and, after further development, may represent a simple monitoring tool for infants’ respiratory activity inside the incubator.

Bottle-Feeding Intervention Detection in the NICU

Video-based monitoring of patients in the neonatal intensive care unit (NICU) has great potential for improving patient care. Video-based detection of clinical events, such as bottle feeding, would represent a step towards semi-automated charting of clinical events. Recording such events contemporaneously would address the limitations associated with retrospective charting. Such a system could also support oral feeding assessment tools, as the patient's feeding skills and nutrition are pivotal in monitoring their growth. We therefore leverage transfer learning using a pretrained VGG-16 model to classify images obtained during an intervention, to determine if a bottle-feeding event is occurring. Additionally, we explore a data expansion technique by extracting similar-feature images from publicly available sources to supplement our dataset of real NICU patients to address data scarcity. This work also visualizes and quantifies the gap between the source domain (ImageNet data subset) and target domain (NICU dataset), thereby illustrating the impact of expanding our training set for knowledge transfer proficiency. Results show an increase of over 18% in sensitivity after data expansion. Analysis of network activation maps indicates that data expansion is able to reduce the distance between the source and target domains.

Asymmetric Three-dimensional Convolutions For Preterm Infants' Pose Estimation

Computer-assisted tools for preterm infants' movement monitoring in neonatal intensive care unit (NICU) could support clinicians in highlighting preterm-birth complications. With such a view, in this work we propose a deep-learning framework for preterm infants' pose estimation from depth videos acquired in the actual clinical practice. The pipeline consists of two consecutive convolutional neural networks (CNNs). The first CNN (inherited from our previous work) acts to roughly predict joints and joint-connections position, while the second CNN (Asy-regression CNN) refines such predictions to trace the limb pose. Asy-regression relies on asymmetric convolutions to temporally optimize both the training and predictions phase. Compared to its counterpart without asymmetric convolutions, Asy-regression experiences a reduction in training and prediction time of 66% , while keeping the root mean square error, computed against manual pose annotation, merely unchanged. Research mostly works to develop highly accurate models, few efforts have been invested to make the training and deployment of such models time-effective. With a view to make these monitoring technologies sustainable, here we focused on the second aspect and addressed the problem of designing a framework as trade-off between reliability and efficiency.

RGB-D scene analysis in the NICU

Continuity of care is achieved in the neonatal intensive care unit (NICU) through careful documentation of all events of clinical significance, including clinical interventions and routine care events (e.g., feeding, diaper change, weighing, etc.). As a step towards automating this documentation process, we propose a scene recognition algorithm that can automatically identify key features in a single image of the patient environment, paired with a rule-based sentence generator to caption the scene. Color and depth video were obtained from 29 newborn patients from the Children's Hospital of Eastern Ontario (CHEO) using an Intel RealSense SR300 RGB-D camera and manual bedside event annotation. Image processing techniques are implemented to classify two lighting conditions: brightness level and phototherapy. A deep neural network is developed for three image classification tasks: on-going intervention, bed occupancy, and patient coverage. Transfer learning is leveraged in the feature extraction layers, such that weights learned from a generic data-rich task are applied to the clinical domain where data collection is complex and costly. Different depth fusion techniques are implemented and compared among classification tasks, where the depth and color data are fused as an RGB-D image (image fusion) or separately at various layers in the network (network fusion). Promising results were obtained with >84% sensitivity and >73% F1 measure across all context variables despite the large class imbalance. RGBD-based models are shown to outperform RGB models on most tasks. In general, a 4-channel image fusion and network fusion at the 11th layer of the VGG-16 architecture were preferred. Ultimately, achieving complete scene understanding through multimodal computer vision could form the basis for a semi-automated charting system to assist clinical staff.

Automatic Separation of Respiratory Flow from Motion in Thermal Videos for Infant Apnea Detection

Both Respiratory Flow (RF) and Respiratory Motion (RM) are visible in thermal recordings of infants. Monitoring these two signals usually requires landmark detection for the selection of a region of interest. Other approaches combine respiratory signals coming from both RF and RM, obtaining a Mixed Respiratory (MR) signal. The detection and classification of apneas, particularly common in preterm infants with low birth weight, would benefit from monitoring both RF and RM, or MR, signals. Therefore, we propose in this work an automatic RF pixel detector not based on facial/body landmarks. The method is based on the property of RF pixels in thermal videos, which are in areas with a smooth circular gradient. We defined 5 features combined with the use of a bank of Gabor filters that together allow selection of the RF pixels. The algorithm was tested on thermal recordings of 9 infants amounting to a total of 132 min acquired in a neonatal ward. On average the percentage of correctly identified RF pixels was 84%. Obstructive Apneas (OAs) were simulated as a proof of concept to prove the advantage in monitoring the RF signal compared to the MR signal. The sensitivity in the simulated OA detection improved for the RF signal reaching 73% against the 23% of the MR signal. Overall, the method yielded promising results, although the positioning and number of cameras used could be further optimized for optimal RF visibility.

Non-Contact Automatic Vital Signs Monitoring of Infants in a Neonatal Intensive Care Unit Based on Neural Networks

Infants with fragile skin are patients who would benefit from non-contact vital sign monitoring due to the avoidance of potentially harmful adhesive electrodes and cables. Non-contact vital signs monitoring has been studied in clinical settings in recent decades. However, studies on infants in the Neonatal Intensive Care Unit (NICU) are still limited. Therefore, we conducted a single-center study to remotely monitor the heart rate (HR) and respiratory rate (RR) of seven infants in NICU using a digital camera. The region of interest (ROI) was automatically selected using a convolutional neural network and signal decomposition was used to minimize the noise artefacts. The experimental results have been validated with the reference data obtained from an ECG monitor. They showed a strong correlation using the Pearson correlation coefficients (PCC) of 0.9864 and 0.9453 for HR and RR, respectively, and a lower error rate with RMSE 2.23 beats/min and 2.69 breaths/min between measured data and reference data. A Bland-Altman analysis of the data also presented a close correlation between measured data and reference data for both HR and RR. Therefore, this technique may be applicable in clinical environments as an economical, non-contact, and easily deployable monitoring system, and it also represents a potential application in home health monitoring.

Physiological vital sign reference ranges for well late preterm newborns calculated during a typical two-hour newborn period between 2 hours and 7 days of life

Objectives To calculate 95% reference ranges for heart rate, respiratory rate, oxygen saturation, temperature and blood pressure for well late preterm newborns between 34+0/7 and 36+6/7 weeks of gestation during typical neonatal behaviour. Approach A single site, prospective cohort study in a major Australian quaternary hospital between February and September 2019. A total of 120 late preterm newborns had their heart rate, respiratory rate and oxygen saturation measurements recorded every two seconds for up to two hours with unconditional 95% reference ranges determined using a linear mixed model with random intercept for total standard deviation calculation including repeated measures. Temperature and blood pressure measurements were collected twice - at the start and conclusion of the data recording period, with weighted 2.5th and 97.5th percentiles calculated using the mean value. Main results A total of 364,577 heart rate, 365,208 respiratory rate, 360,494 peripheral oxygen saturation, and 240 temperature and blood pressure values were obtained. The 95% reference ranges were: heart rate 102 - 164 bpm; respiratory rate 15 - 67 rpm; oxygen saturation 94 - 100%; temperature 36.4 - 37.6°C; systolic blood pressure 51 - 86 mmHg; diastolic blood pressure 28 - 61 mmHg; mean arterial pressure 35 - 68 mmHg. Significance Seven vital sign references ranges were reported for the late preterm population during a typical newborn period (such as crying, sleeping, feeding, awake and alert, and during nappy hygiene cares); internal and external validation should be completed prior to clinical use. Cut off points for escalation of care have previously been generalised to all newborns irrespective of gestational age which may result in over-treatment or a delay in recognising subtle signs of deterioration.

Respiratory synchrony comparison between preterm and full-term neonates using inertial sensors

INTRODUCTION

Due to inefficient respiratory control, newborns become prone to asynchronous thoracoabdominal (TA) movements. The present study quantitatively estimated the synchrony of TA in preterm and full-term newborns through an inertial and magnetic measurement units (IMMUs) system.

METHODS

This cross-sectional study was conducted with 20 newborns divided into Preterm Group (PTG, n = 10) and Full-Term Group (FTG, n = 10). Each neonate had IMMUs placed on the sternum and near the umbilicus, thus the TA motion was estimated through the resultant inclination angles calculated using a sensor fusion filter. The respiratory incursions were also manually counted and video-recorded for two minutes, then used to validate a Matlab custom-written routine for their automatic identification. The respiratory cycles were used to calculate the phase change angle (φ) between the thoracic and abdominal compartments. Association between the manual and automatic methods were verified by Pearson's correlation and root mean squared errors (RMSE), and the comparison between the groups was performed through the Student's t test with α = .05.

RESULTS

The values of respiratory incursions measured by both methods showed a high association and low measurement error (r = .96, RMSE = 9.8, p < .001). The FTG presented a higher occurrence of TA synchrony (p = .049) while the PTG group presented a higher occurrence of TA asynchrony (p = .036). No difference was found between the groups regarding the paradoxical classification (p = .071).

CONCLUSION

The proposed method was valid to quantitatively assess the TA synchrony of hospitalized neonates. Preterm infants had a higher occurrence of the asynchronous respiratory pattern in comparison to full-term infants.

A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods

The interest in contactless or remote heart rate measurement has been steadily growing in healthcare and sports applications. Contactless methods involve the utilization of a video camera and image processing algorithms. Recently, deep learning methods have been used to improve the performance of conventional contactless methods for heart rate measurement. After providing a review of the related literature, a comparison of the deep learning methods whose codes are publicly available is conducted in this paper. The public domain UBFC dataset is used to compare the performance of these deep learning methods for heart rate measurement. The results obtained show that the deep learning method PhysNet generates the best heart rate measurement outcome among these methods, with a mean absolute error value of 2.57 beats per minute and a mean square error value of 7.56 beats per minute.

Preterm Newborn Presence Detection in Incubator and Open Bed Using Deep Transfer Learning

Video-based motion analysis recently appeared to be a promising approach in neonatal intensive care units for monitoring the state of preterm newborns since it is contact-less and noninvasive. However it is important to remove periods when the newborn is absent or an adult is present from the analysis. In this paper, we propose a method for automatic detection of preterm newborn presence in incubator and open bed. We learn a specific model for each bed type as the camera placement differs a lot and the encountered situations are different between both. We break the problem down into two binary classifications based on deep transfer learning that are fused afterwards: newborn presence detection on the one hand and adult presence detection on the other hand. Moreover, we adopt a strategy of decision intervals fusion in order to take advantage of temporal consistency. We test three deep neural network that were pre-trained on ImageNet: VGG16, MobileNetV2 and InceptionV3. Two classifiers are compared: support vector machine and a small neural network. Our experiments are conducted on a database of 120 newborns. The whole method is evaluated on a subset of 25 newborns including 66 days of video recordings. In incubator, we reach a balanced accuracy of 86%. In open bed, the performance is lower because of a much wider variety of situations whereas less data are available.

A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video

Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

Towards Continuous Camera-Based Respiration Monitoring in Infants

Aiming at continuous unobtrusive respiration monitoring, motion robustness is paramount. However, some types of motion can completely hide the respiration information and the detection of these events is required to avoid incorrect rate estimations. Therefore, this work proposes a motion detector optimized to specifically detect severe motion of infants combined with a respiration rate detection strategy based on automatic pixels selection, which proved to be robust to motion of the infants involving head and limbs. A dataset including both thermal and RGB (Red Green Blue) videos was used amounting to a total of 43 h acquired on 17 infants. The method was successfully applied to both RGB and thermal videos and compared to the chest impedance signal. The Mean Absolute Error (MAE) in segments where some motion is present was 1.16 and 1.97 breaths/min higher than the MAE in the ideal moments where the infants were still for testing and validation set, respectively. Overall, the average MAE on the testing and validation set are 3.31 breaths/min and 5.36 breaths/min, using 64.00% and 69.65% of the included video segments (segments containing events such as interventions were excluded based on a manual annotation), respectively. Moreover, we highlight challenges that need to be overcome for continuous camera-based respiration monitoring. The method can be applied to different camera modalities, does not require skin visibility, and is robust to some motion of the infants.

Non-contact breathing rate monitoring in newborns: A review

The neonatal period - the first 4 weeks of life - is the most critical time for a child's survival. Breathing rate is a vital indicator of the health condition and requires continuous monitoring in case of sickness or preterm birth. Breathing movements can be counted by contact and non-contact methods. In the case of newborn infants, the non-contact breathing rate monitoring need is high, as a contact-based approach may interfere while providing care and is subject to interference by non-breathing movements. This review article delivers a factual summary, and describes the methods and processing involved in non-contact based breathing rate monitoring. The article also provides the advantages, limitations, and clinical applications of these methods. Additionally, signal processing, feasibility, and future direction of different non-contact neonatal breathing rate monitoring are discussed.

Wireless skin sensors for physiological monitoring of infants in low-income and middle-income countries

Globally, neonatal mortality remains unacceptability high. Physiological monitoring is foundational to the care of these vulnerable patients to assess neonatal cardiopulmonary status, guide medical intervention, and determine readiness for safe discharge. However, most existing physiological monitoring systems require multiple electrodes and sensors, which are linked to wires tethered to wall-mounted display units, to adhere to the skin. For neonates, these systems can cause skin injury, prevent kangaroo mother care, and complicate basic clinical care. Novel, wireless, and biointegrated sensors provide opportunities to enhance monitoring capabilities, reduce iatrogenic injuries, and promote family-centric care. Early validation data have shown performance equivalent to (and sometimes exceeding) standard-of-care monitoring systems in premature neonates cared for in high-income countries. The reusable nature of these sensors and compatibility with low-cost mobile phones have the future potential to enable substantially lower monitoring costs compared with existing systems. Deployment at scale, in low-income countries, holds the promise of substantial improvements in neonatal outcomes.

Machine Learning-Based Automatic Classification of Video Recorded Neonatal Manipulations and Associated Physiological Parameters: A Feasibility Study

Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO₂)) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates <32 weeks’ gestation, diaper changes were associated with significant changes in HR and SpO₂, and, for neonates ≥32 weeks’ gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.

Non-contact vital-sign monitoring of patients undergoing haemodialysis treatment

A clinical study was designed to record a wide range of physiological values from patients undergoing haemodialysis treatment in the Renal Unit of the Churchill Hospital in Oxford. Video was recorded for a total of 84 dialysis sessions from 40 patients during the course of 1 year, comprising an overall video recording time of approximately 304.1 h. Reference values were provided by two devices in regular clinical use. The mean absolute error between the heart rate estimates from the camera and the average from two reference pulse oximeters (positioned at the finger and earlobe) was 2.8 beats/min for over 65% of the time the patient was stable. The mean absolute error between the respiratory rate estimates from the camera and the reference values (computed from the Electrocardiogram and a thoracic expansion sensor-chest belt) was 2.1 breaths/min for over 69% of the time for which the reference signals were valid. To increase the robustness of the algorithms, novel methods were devised for cancelling out aliased frequency components caused by the artificial light sources in the hospital, using auto-regressive modelling and pole cancellation. Maps of the spatial distribution of heart rate and respiratory rate information were developed from the coefficients of the auto-regressive models. Most of the periods for which the camera could not produce a reliable heart rate estimate lasted under 3 min, thus opening the possibility to monitor heart rate continuously in a clinical environment.

Multi-camera infrared thermography for infant respiration monitoring

Respiration is monitored in neonatal wards using chest impedance (CI), which is obtrusive and can cause skin damage to the infants. Therefore, unobtrusive solutions based on infrared thermography are being investigated. This work proposes an algorithm to merge multiple thermal camera views and automatically detect the pixels containing respiration motion or flow using three features. The method was tested on 152 minutes of recordings acquired on seven infants. We performed a comparison with the CI respiration rate yielding a mean absolute error equal to 2.07 breaths/min. Merging the three features resulted in reducing the dependency on the window size typical of spectrum-based features.