Intelligent neonatal monitoring based on a virtual thermal sensor

Flow-Field Inference for Turbulent Exhale Flow Measurement

BACKGROUND

Vision-based pulmonary diagnostics present a unique approach for tracking and measuring natural breathing behaviors through remote imaging. While many existing methods correlate chest and diaphragm movements to respiratory behavior, we look at how the direct visualization of thermal CO2 exhale flow patterns can be tracked to directly measure expiratory flow.

METHODS

In this work, we present a novel method for isolating and extracting turbulent exhale flow signals from thermal image sequences through flow-field prediction and optical flow measurement. The objective of this work is to introduce a respiratory diagnostic tool that can be used to capture and quantify natural breathing, to identify and measure respiratory metrics such as breathing rate, flow, and volume. One of the primary contributions of this work is a method for capturing and measuring natural exhale behaviors that describe individualized pulmonary traits. By monitoring subtle individualized respiratory traits, we can perform secondary analysis to identify unique personalized signatures and abnormalities to gain insight into pulmonary function. In our study, we perform data acquisition within a clinical setting to train an inference model (FieldNet) that predicts flow-fields to quantify observed exhale behaviors over time.

RESULTS

Expiratory flow measurements capturing individualized flow signatures from our initial cohort demonstrate how the proposed flow field model can be used to isolate and analyze turbulent exhale behaviors and measure anomalous behavior.

CONCLUSIONS

Our results illustrate that detailed spatial flow analysis can contribute to unique signatures for identifying patient specific natural breathing behaviors and abnormality detection. This provides the first-step towards a non-contact respiratory technology that directly captures effort-independent behaviors based on the direct measurement of imaged CO2 exhaled airflow patterns.

Continuous monitoring using thermography can capture the heat oscillations maintaining body temperature in neonates

The body temperature of infants at equilibrium with their surroundings is balanced between heat production from metabolism and the transfer of heat to the environment. Total heat production is related to body size, which is closely related to metabolic rate and oxygen consumption. Body temperature control is a crucial aspect of neonatal medicine but we have often struggled with temperature measures. Contactless infrared thermography (IRT) is useful for vulnerable neonates and may be able to assess their spontaneous thermal metabolism. The present study focused on heat oscillations and their cause. IRT was used to measure the skin temperature every 15 s of neonates in an incubator. We analyzed the thermal data of 27 neonates (32 measurements), calculated the average temperature within specified regions, and extracted two frequency components-Components A and B-using the Savitzky-Golay method. Furthermore, we derived an equation describing the cycle-named cycle T-for maintaining body temperature according to body weight. A positive correlation was observed between cycle T and Component B (median [IQR]: 368 [300-506] s). This study sheds light on the physiological thermoregulatory function of newborns and will lead to improved temperature management methods for newborns, particularly premature, low-birth-weight infants.

Camera fusion for real-time temperature monitoring of neonates using deep learning

Abstract

The continuous monitoring of vital signs is a crucial aspect of medical care in neonatal intensive care units. Since cable-based sensors pose a potential risk for the immature skin of preterm infants, unobtrusive monitoring techniques using camera systems are increasingly investigated. The combination of deep learning–based algorithms and camera modalities such as RGB and infrared thermography can improve the development of cable-free methods for the extraction of vital parameters. In this study, a real-time approach for local extraction of temperatures on the body surface of neonates using a multi-modal clinical dataset was implemented. Therefore, a trained deep learning–based keypoint detector was used for body landmark prediction in RGB. Image registration was conducted to transfer the RGB points to the corresponding thermographic recordings. These landmarks were used to extract the body surface temperature in various regions to determine the central-peripheral temperature difference. A validation of the keypoint detector showed a mean average precision of 0.82. The registration resulted in mean absolute errors of 16.4 px (8.2 mm) for x and 22.4 px (11.2 mm) for y. The evaluation of the temperature extraction revealed a mean absolute error of 0.55 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘C. A final performance of 31 fps was observed on the NVIDIA Jetson Xavier NX module, which proves real-time capability on an embedded GPU system. As a result, the approach can perform real-time temperature extraction on a low-cost GPU module.

Graphical abstract

Infrared Thermography with High Accuracy in a Neonatal Incubator

As the accuracy of body temperature measurement is especially critical in premature infants on admission to the neonatal intensive care unit (NICU), noninvasive measurement using infrared thermography (IRT) has not been widely adopted in the NICU due to a lack of evidence regarding its accuracy. We have established a new calibration method for IRT in an incubator, and evaluated its accuracy and reliability at different incubator settings using a variable-temperature blackbody furnace. This method improved the accuracy and reliability of IRT with an increase in percentage of data with mean absolute error (MAE) < 0.3 °C to 93.1% compared to 4.2% using the standard method. Two of three IRTs had MAE < 0.1 °C under all conditions examined. This method provided high accuracy not only for measurements at specific times but also for continuous monitoring. It will also contribute to avoiding the risk of neonates' skin trouble caused by attaching a thermistor. This study will facilitate the development of novel means of administering neonatal body temperature.

A method for improving semantic segmentation using thermographic images in infants

Background

Regulation of temperature is clinically important in the care of neonates because it has a significant impact on prognosis. Although probes that make contact with the skin are widely used to monitor temperature and provide spot central and peripheral temperature information, they do not provide details of the temperature distribution around the body. Although it is possible to obtain detailed temperature distributions using multiple probes, this is not clinically practical. Thermographic techniques have been reported for measurement of temperature distribution in infants. However, as these methods require manual selection of the regions of interest (ROIs), they are not suitable for introduction into clinical settings in hospitals. Here, we describe a method for segmentation of thermal images that enables continuous quantitative contactless monitoring of the temperature distribution over the whole body of neonates.

Methods

The semantic segmentation method, U-Net, was applied to thermal images of infants. The optimal combination of Weight Normalization, Group Normalization, and Flexible Rectified Linear Unit (FReLU) was evaluated. U-Net Generative Adversarial Network (U-Net GAN) was applied to thermal images, and a Self-Attention (SA) module was finally applied to U-Net GAN (U-Net GAN + SA) to improve precision. The semantic segmentation performance of these methods was evaluated.

Results

The optimal semantic segmentation performance was obtained with application of FReLU and Group Normalization to U-Net, showing accuracy of 92.9% and Mean Intersection over Union (mIoU) of 64.5%. U-Net GAN improved the performance, yielding accuracy of 93.3% and mIoU of 66.9%, and U-Net GAN + SA showed further improvement with accuracy of 93.5% and mIoU of 70.4%.

Conclusions

FReLU and Group Normalization are appropriate semantic segmentation methods for application to neonatal thermal images. U-Net GAN and U-Net GAN + SA significantly improved the mIoU of segmentation.

A Neonatal Phantom for Vital Signs Simulation

Neonatal intensive care units provide vital medical support for premature infants. The key aspect in neonatal care is the continuous monitoring of vital signs measured using adhesive skin sensors. Since sensors can cause irritation of the skin and lead to infections, research focuses on contact-free, camera-based methods such as infrared thermography and photoplethysmography imaging. The development of image processing algorithms requires large datasets, but recording the necessary data from studies brings tremendous effort and costs. Therefore, realistic patient phantoms would be feasible to create a comprehensive dataset and validate image-based algorithms. This work describes the realization of a neonatal phantom which can simulate physiological vital parameters such as pulse rate and thermoregulation. It mimics the outer appearance of premature infants using a 3D printed base structure coated with several layers of modified, skin-colored silicone. A distribution of red and infrared LEDs in the scaffold enables the simulation of a PPG signal by mimicking pulsative light intensity changes on the skin. Additionally, the body temperature of the phantom is individually adjustable in several regions using heating elements. In the validation process for PPG simulation, the feasibility of setting different pulse frequencies and the variation of oxygen saturation levels was obtained. Furthermore, heating tests showed region-dependent temperature variations between 0.19 ^°C and 0.81 ^°C around the setpoint. In conclusion, the proposed neonatal phantom can be used to simulate a variety of vital parameters of preterm infants and, therefore, enables the implementation of image processing algorithms for the analysis of the medical state.

BMP8 and activated brown adipose tissue in human newborns

The classical dogma states that brown adipose tissue (BAT) plays a major role in the regulation of temperature in neonates. However, although BAT has been studied in infants for more than a century, the knowledge about its physiological features at this stage of life is rather limited. This has been mainly due to the lack of appropriate investigation methods, ethically suitable for neonates. Here, we have applied non-invasive infrared thermography (IRT) to investigate neonatal BAT activity. Our data show that BAT temperature correlates with body temperature and that mild cold stimulus promotes BAT activation in newborns. Notably, a single short-term cold stimulus during the first day of life improves the body temperature adaption to a subsequent cold event. Finally, we identify that bone morphogenic protein 8B (BMP8B) is associated with the BAT thermogenic response in neonates. Overall, our data uncover key features of the setup of BAT thermogenesis in newborns.

Automatic non-contact monitoring of the respiratory rate of neonates using a structured light camera

This paper introduces an automatic non-contact monitoring method for measuring the respiratory rate of neonates using a structured light camera. The current monitoring bears several issues causing pressure marks, skin irritations and eczema. A structured light camera provides distance data. Our non-contact approach detects the thorax area automatically using a plane segmentation and calculates the respiratory rate from the movement of the thorax. Our method was tested and validated using the baby simulator SimBaby by Laerdal. We used different breathing rates corresponding to preterm neonates, mature neonates and babies aged up to nine months as well as two different breathing modes with differing breathing strokes. Furthermore, measurements were taken of two positions: the baby lying on its back and on its stomach.

Inexpensive Home Infrared Living/Environment Sensor with Regional Thermal Information for Infant Physical and Psychological Development

The use of home-based image sensors for biological and environmental monitoring provides novel insight into health and development but it is difficult to evaluate people during their normal activities in their home. Therefore, we developed a low-cost infrared (IR) technology-based motion, location, temperature and thermal environment detection system that can be used non-invasively for long-term studies in the home environment. We tested this technology along with the associated analysis algorithm to visualize the effects of parental care and thermal environment on developmental state change in a non-human primate model, the common marmoset (Callithrix jacchus). To validate this system, we first compared it to a manual analysis technique and we then assessed the development of circadian rhythms in common marmosets from postnatal day 15–45. The semi-automatically tracked biological indices of locomotion velocity (BV) and body surface temperature (BT) and the potential psychological index of place preference toward the door (BD), showed age-dependent shifts in circadian phase patterns. Although environmental variables appeared to affect circadian rhythm development, principal component analysis and signal superimposing imaging methods revealed a novel phasic pattern of BD-BT correlation day/night switching in animals older than postnatal day 38 (approximately equivalent to one year of age in humans). The origin of this switch was related to earlier development of body temperature (BT) rhythms and alteration of psychological behavior rhythms (BD) around earlier feeding times. We propose that this cost-effective, inclusive sensing and analytic technique has value for understanding developmental care conditions for which continual home non-invasive monitoring would be beneficial and further suggest the potential to adapt this technique for use in humans.

Business Process Management and Digital Innovations: A Systematic Literature Review

Emerging technologies have capabilities to reshape business process management (BPM) from its traditional version to a more explorative variant. However, to exploit the full benefits of new IT, it is essential to reveal BPM’s research potential and to detect recent trends in practice. Therefore, this work presents a systematic literature review (SLR) with 231 recent academic articles (from 2014 until May 2019) that integrate BPM with digital innovations (DI). We position those articles against seven future BPM-DI trends that were inductively derived from an expert panel. By complementing the expected trends in practice with a state-of-the-art literature review, we are able to derive covered and uncovered themes in order to help bridge a rigor-relevance gap. The major technological impacts within the BPM field seem to focus on value creation, customer engagement and managing human-centric and knowledge-intensive business processes. Finally, our findings are categorized into specific calls for research and for action to let scholars and organizations better prepare for future digital needs.

Thermal imaging applications in neonatal care: a scoping review

Background

In neonatal care, assessment of the temperature of the neonate is essential to confirm on-going health, and as an early signal of potential pathology. However, some methods of temperature assessment involve disturbing the baby, disrupting essential sleep patterns, and interrupting maternal/infant interaction. Thermal imaging is a completely non-invasive and non-contact method of assessing emitted temperature, but it is not a standard method for neonatal thermal monitoring. To examine the potential utility of using thermal imaging in neonatal care, we conducted a comprehensive systematic scoping review of thermal imaging applications in this context.

Methods

We searched EMBASE, MEDLINE and MIDIRS.

Results

From 442 hits, 21 met the inclusion criteria and were included in the review. A significant number (n = 9) were published in the last 8 years. All the studies were observational studies, with 20 out of 21 undertaken in North America or Europe. Most of them had small cohorts (range 4–29 participants). The findings were analysed narratively, to establish the issues identified in the included studies. Five broad themes emerged for future examination. These were: general thermal physiology; heat loss and respiratory monitoring; identification of internal pathologies, including necrotising enterocolitis; other uses of thermal imaging; and technical concerns. The findings suggest that thermal imaging is a reliable and non-invasive method for continuous monitoring of the emitted temperature of the neonates, with potential for contributing to the assurance of wellbeing, and to the diagnosis of pathologies, including internal abnormalities. However, the introduction of thermal imaging into everyday neonatology practice has several methodological challenges, including environmental parameters, especially when infants are placed in incubators or open radiant warmers.

Conclusion

In conclusion, although the first attempt at using thermal imaging in neonatal care started in the early-1970s, with promising results, and subsequent small cohort studies have recently reinforced this potential, there have not been any large prospective studies in this area that examine both the benefits and the barriers to its use in practice.

Video-Based Actigraphy for Monitoring Wake and Sleep in Healthy Infants: A Laboratory Study

Prolonged monitoring of infant sleep is paramount for parents and healthcare professionals for interpreting and evaluating infants’ sleep quality. Wake-sleep patterns are often studied to assess this. Video cameras have received a lot of attention in infant sleep monitoring because they are unobtrusive and easy to use at home. In this paper, we propose a method using motion data detected from infrared video frames (video-based actigraphy) to identify wake and sleep states. The motion, mostly caused by infant body movement, is known to be substantially associated with infant wake and sleep states. Two features were calculated from the video-based actigraphy, and a Bayesian-based linear discriminant classification model was employed to classify the two states. Leave-one-subject-out cross validation was performed to validate our proposed wake and sleep classification model. From a total of 11.6 h of infrared video recordings of 10 healthy term infants in a laboratory pilot study, we achieved a reliable classification performance with a Cohen’s kappa coefficient of 0.733 ± 0.204 (mean ± standard deviation) and an overall accuracy of 92.0% ± 4.6%.

Video and audio processing in paediatrics: a review

OBJECTIVE

Video and sound acquisition and processing technologies have seen great improvements in recent decades, with many applications in the biomedical area. The aim of this paper is to review the overall state of the art of advances within these topics in paediatrics and to evaluate their potential application for monitoring in the neonatal intensive care unit (NICU).

APPROACH

For this purpose, more than 150 papers dealing with video and audio processing were reviewed. For both topics, clinical applications are described according to the considered cohorts-full-term newborns, infants and toddlers or preterm newborns. Then, processing methods are presented, in terms of data acquisition, feature extraction and characterization.

MAIN RESULTS

The paper first focuses on the exploitation of video recordings; these began to be automatically processed in the 2000s and we show that they have mainly been used to characterize infant motion. Other applications, including respiration and heart rate estimation and facial analysis, are also presented. Audio processing is then reviewed, with a focus on the analysis of crying. The first studies in this field focused on induced-pain cries and the newest ones deal with spontaneous cries; the analyses are mainly based on frequency features. Then, some papers dealing with non-cry signals are also discussed.

SIGNIFICANCE

Finally, we show that even if recent improvements in digital video and signal processing allow for increased automation of processing, the context of the NICU makes a fully automated analysis of long recordings problematic. A few proposals for overcoming some of the limitations are given.

Review of Biomedical Applications of Contactless Imaging of Neonates Using Infrared Thermography and Beyond

The sick preterm infant monitoring is an intriguing job that medical staff in Neonatal Intensive Care Units (NICU) must deal with on a daily basis. As a standards monitoring procedure, preterm infants are monitored via sensors and electrodes that are firmly attached to their fragile and delicate skin and connected to processing monitors. However, an alternative exists in contactless imaging to record such physiological signals (we call it as Physio-Markers), detecting superficial changes and internal structures activities which can be used independently of, or aligned with, conventional monitors. Countless advantages can be gained from unobtrusive monitoring not limited to: (1) quick data generation; (2) decreasing physical and direct contact with skin, which reduces skin breakdown and minimizes risk of infection; and (3) reduction of electrodes and probes connected to clinical monitors and attached to the skin, which allows greater body surface-area for better care. This review is an attempt to build a solid ground for and to provide a clear perspective of the potential clinical applications of technologies inside NICUs that use contactless imaging modalities such as Visible Light Imaging (VLI), Near Infrared Spectroscopy (NIRS), and Infrared Thermography (IRT).

Effect of Timing of the First Bath on a Healthy Newborn's Temperature

OBJECTIVE

To determine if a healthy newborn's age in hours (3, 6, or 9 hours after birth) affects thermoregulatory status after the first bath as indicated by axillary and skin temperatures.

DESIGN

Quasi-experimental, mixed-model (between subjects and within subjects) design with hours of age as the nonrepeated variable and prebath and postbath temperatures as the repeated variables.

SETTING

Family-centered care unit at an urban hospital in the southwestern United States.

PARTICIPANTS

Healthy newborns (N = 75) 37 weeks or more completed gestation.

METHODS

Mothers chose time of first bath based on available time slots (n = 25 newborns in each age group). Research nurses sponge bathed the newborns in the mothers' rooms. Axillary temperature, an index of core temperature, was measured with a digital thermometer, and skin temperature, an index of body surface temperature, was measured with a thermography camera. Temperatures were taken before the bath; immediately after the bath; and 5, 30, 60, and 120 minutes after the bath. Immediately after the bath, newborns were placed in skin-to-skin care (SSC) for 60 or more minutes.

RESULTS

We found a difference (p = .0372) in axillary temperatures between the 3- and 9-hour age groups, although this difference was not clinically significant (0.18 °F [0.10 °C]). We found no statistically significant differences in skin temperatures among the three age groups. Regardless of age group, axillary and skin temperatures initially decreased and then recovered after the bath.

CONCLUSION

For up to 2 hours postbath, axillary and skin temperatures were not different between healthy newborns bathed at 3, 6, or 9 hours of age. Thermography holds promise for learning about thermoregulation, bathing, and SSC.

Body temperature in premature infants during the first week of life: Exploration using infrared thermal imaging

BACKGROUND

Hypothermia is a problem for very premature infants after birth and leads to increased morbidity and mortality. Previously we found very premature infants exhibit abnormal thermal patterns, keeping foot temperatures warmer than abdominal temperatures for their first 12h of life.

PURPOSE

We explored the utility of infrared thermography as a non-invasive method for measuring body temperature in premature infants in an attempt to regionally examine differential temperatures.

RESULTS

Our use of infrared imaging to measure abdominal and foot temperature for extremely premature infants in heated, humid incubators was successful and in close agreement using Bland and Altman technique with temperatures measured by skin thermistors.

CONCLUSIONS

Our study methods demonstrated that it was feasible to capture full body temperatures of extremely premature infants while they were resting in a heated, humid incubator using a Flir SC640 infrared camera. This technology offers researchers and clinicians a method to examine acute changes in perfusion differentials in premature infants which may lead to morbidity.

Infrared thermography in paediatrics: a narrative review of clinical use

BACKGROUND

Infrared thermography (IRT) has been used in adult medicine for decades, but recent improvements in quality of imaging and increasing computer processing power have allowed for a diversification of clinical applications. The specific usage of IRT in a paediatric population has not been widely explored, so this article aims to summarise the available literature in this area. IRT involves the non-contact, accurate measurement of skin surface temperature to identify temperature changes suggesting disease. IRT could well have unique applications in paediatric medicine.

METHODS

Electronic searches were performed independently by two authors, using the databases of MEDLINE (via Web of Science), the Cochrane Library, CINAHL (EBSCO) and Scopus, including articles published from 1990 to July 2016. The search strategy that was used aimed to include articles that covered the topics of IRT and children, including studies with participants 18 years old or younger. Articles were screened by title and abstract by two authors. Meta-analysis was not performed due to the marked heterogeneity in applications, study design and outcomes: this is a narrative summary of the available literature.

RESULTS

IRT has been shown to be an effective additional diagnostic tool in a number of different paediatric specialties, namely in fracture screening, burns assessment and neonatal monitoring. Small measurable skin temperature changes can effectively add to the clinical picture, while computer-tracking systems can be reliably used to focus investigations on particular areas of the body.

CONCLUSION

Throughout this review of the available literature, there has been a general consensus that this non-invasive, non-irradiating and relatively inexpensive technology may well have a place in the management of paediatric patients in the future.

Continuous non-contact vital sign monitoring in neonatal intensive care unit

Current technologies to allow continuous monitoring of vital signs in pre-term infants in the hospital require adhesive electrodes or sensors to be in direct contact with the patient. These can cause stress, pain, and also damage the fragile skin of the infants. It has been established previously that the colour and volume changes in superficial blood vessels during the cardiac cycle can be measured using a digital video camera and ambient light, making it possible to obtain estimates of heart rate or breathing rate. Most of the papers in the literature on non-contact vital sign monitoring report results on adult healthy human volunteers in controlled environments for short periods of time. The authors' current clinical study involves the continuous monitoring of pre-term infants, for at least four consecutive days each, in the high-dependency care area of the Neonatal Intensive Care Unit (NICU) at the John Radcliffe Hospital in Oxford. The authors have further developed their video-based, non-contact monitoring methods to obtain continuous estimates of heart rate, respiratory rate and oxygen saturation for infants nursed in incubators. In this Letter, it is shown that continuous estimates of these three parameters can be computed with an accuracy which is clinically useful. During stable sections with minimal infant motion, the mean absolute error between the camera-derived estimates of heart rate and the reference value derived from the ECG is similar to the mean absolute error between the ECG-derived value and the heart rate value from a pulse oximeter. Continuous non-contact vital sign monitoring in the NICU using ambient light is feasible, and the authors have shown that clinically important events such as a bradycardia accompanied by a major desaturation can be identified with their algorithms for processing the video signal.