Thermal Infrared Imaging-Based Computational Psychophysiology for Psychometrics

Facial thermal response to non-painful stressor in premature and term neonates

BACKGROUND

This study is a preliminary clinical investigation with the objective to evaluate the facial thermal response of premature and term neonates to a non-painful stressor (hunger) using infrared thermography (IRT). The development of objective and reliable parameters to monitor pain and stress is of relevance for optimal neonatal outcome and achieving a better management of patient comfort.

METHODS

We enrolled 12 neonates ranging from 27 to 39 weeks gestation (median: 34) and aged 3-79 days (median: 13). Recordings were performed before and after feeding, with and without hunger. Six regions of interest were chosen for evaluation (nose tip, periorbital and corrugator region, forehead, perioral and chin region).

RESULTS

There was an increase in the facial temperature in infants immediately prior to their next feed relative to infants who were not hungry, with the nasal tip being the facial evaluation site with the greatest temperature change.

CONCLUSIONS

The IRT appears to be a feasible and suitable method to detect changes in the neonatal patient. The thermal variations observed seem to reflect an arousal mediated by the parasympathetic nervous system, which has been described in existing infant stress research.

IMPACT

This is the first study to examine the use of infrared thermography (IRT) in monitoring the facial thermal response to a mild stressor (hunger) in premature and term neonates. Hunger as a mild, non-pain-associated stressor showed a significant effect on the facial temperature. The thermal signature of the regions of interest chosen showed hunger-related thermal variations. Results suggest the feasibility and suitability of IRT as an objective diagnostic tool to approach stress and changes in the condition of the neonatal patient.

Can Data-Driven Supervised Machine Learning Approaches Applied to Infrared Thermal Imaging Data Estimate Muscular Activity and Fatigue?

Surface electromyography (sEMG) is the acquisition, from the skin, of the electrical signal produced by muscle activation. Usually, sEMG is measured through electrodes with electrolytic gel, which often causes skin irritation. Capacitive contactless electrodes have been developed to overcome this limitation. However, contactless EMG devices are still sensitive to motion artifacts and often not comfortable for long monitoring. In this study, a non-invasive contactless method to estimate parameters indicative of muscular activity and fatigue, as they are assessed by EMG, through infrared thermal imaging (IRI) and cross-validated machine learning (ML) approaches is described. Particularly, 10 healthy participants underwent five series of bodyweight squats until exhaustion interspersed by 1 min of rest. During exercising, the vastus medialis activity and its temperature were measured through sEMG and IRI, respectively. The EMG average rectified value (ARV) and the median frequency of the power spectral density (MDF) of each series were estimated through several ML approaches applied to IRI features, obtaining good estimation performances (r = 0.886, p < 0.001 for ARV, and r = 0.661, p < 0.001 for MDF). Although EMG and IRI measure physiological processes of a different nature and are not interchangeable, these results suggest a potential link between skin temperature and muscle activity and fatigue, fostering the employment of contactless methods to deliver metrics of muscular activity in a non-invasive and comfortable manner in sports and clinical applications.

Accuracy and reliability of the optoelectronic plethysmography and the heart rate systems for measuring breathing rates compared with the spirometer

Measuring breathing rates without a mouthpiece is of interest in clinical settings. Electrocardiogram devices and, more recently, optoelectronic plethysmography (OEP) methods can estimate breathing rates with only a few electrodes or motion-capture markers placed on the patient. This study estimated and compared the accuracy and reliability of three non-invasive devices: an OEP system with 12 markers, an electrocardiogram device and the conventional spirometer. Using the three devices simultaneously, we recorded 72 six-minute epochs on supine subjects. Our results show that the OEP system has a very low limit of agreement and a bias lower than 0.4% compared with the spirometer, indicating that these devices can be used interchangeably. We observed comparable results for electrocardiogram devices. The OEP system facilitates breathing rate measurements and offers a more complete chest-lung volume analysis that can be easily associated with heart rate analysis without any synchronisation process, for useful features for clinical applications and intensive care.

Deep learning techniques for automated detection of autism spectrum disorder based on thermal imaging

Children with autism spectrum disorder have impairments in emotional processing which leads to the inability in recognizing facial expressions. Since emotion is a vital criterion for having fine socialisation, it is incredibly important for the autistic children to recognise emotions. In our study, we have chosen the facial skin temperature as a biomarker to measure emotions. To assess the facial skin temperature, the thermal imaging modality has been used in this study, since it has been recognised as a promising technique to evaluate emotional responses. The aim of this study was the following: (1) to compare the facial skin temperature of autistic and non-autistic children by using thermal imaging across various emotions; (2) to classify the thermal images obtained from the study using the customised convolutional neural network compared with the ResNet 50 network. Fifty autistic and fifty non-autistic participants were included for the study. Thermal imaging was used to obtain the temperature of specific facial regions such as the eyes, cheek, forehead and nose while we evoked emotions (Happiness, anger and sadness) in children using an audio-visual stimulus. Among the emotions considered, the emotion anger had the highest temperature difference between the autistic and non-autistic participants in the region's eyes (1.9%), cheek (2.38%) and nose (12.6%). The accuracy obtained by classifying the thermal images of the autistic and non-autistic children using Customised Neural Network and ResNet 50 Network was 96% and 90% respectively. This computer aided diagnostic tool can be a predictable and a steadfast method in the diagnosis of the autistic individuals.

Recent Advances in Atherosclerotic Disease Screening Using Pervasive Healthcare

Atherosclerosis screening helps the medical model transform from therapeutic medicine to preventive medicine by assessing degree of atherosclerosis prior to the occurrence of fatal vascular events. Pervasive screening emphasizes atherosclerotic monitoring with easy access, quick process, and advanced computing. In this work, we introduced five cutting-edge pervasive technologies including imaging photoplethysmography (iPPG), laser Doppler, radio frequency (RF), thermal imaging (TI), optical fiber sensing and piezoelectric sensor. IPPG measures physiological parameters by using video images that record the subtle skin color changes consistent with cardiac-synchronous blood volume changes in subcutaneous arteries and capillaries. Laser Doppler obtained the information on blood flow by analyzing the spectral components of backscattered light from the illuminated tissues surface. RF is based on Doppler shift caused by the periodic movement of the chest wall induced by respiration and heartbeat. TI measures vital signs by detecting electromagnetic radiation emitted by blood flow. The working principle of optical fiber sensor is to detect the change of light properties caused by the interaction between the measured physiological parameter and the entering light. Piezoelectric sensors are based on the piezoelectric effect of dielectrics. All these pervasive technologies are noninvasive, mobile, and can detect physiological parameters related to atherosclerosis screening.

Identifying the Optimal Features in Multimodal Deception Detection

Facial thermal imaging is a non-contact technology which can be useful for ubiquitous deceptive anxiety recognition. To date, studies investigating this technology have produced equivocal results in classification accuracy and finding the most correlated regions on the face. This study was conducted using our dataset with 41 subjects using two different protocols and three modalities (thermal, GSR and PPG). We selected and tracked five regions of interest (ROI) on each facial thermal imprint including periorbital, forehead, cheek, perinasal and chin that were mostly used in previous papers. By employing six statistical features, four feature reduction techniques and three classifiers, we attempted to identify the ROIs which are mostly associated with activation of the sympathetic nervous system to increase the final classification accuracy rate. The results of linear classification models show significant improvement of classification accuracy by using ROC feature selection method. We achieved 90.1% and 74.7% accuracy rate for thermal features in mock crime and best friend scenarios, respectively. Our experimental results show that perinasal and cheek areas have greater discriminatory power in comparison with other ROIs on the face.

Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review

Over recent years, robots are increasingly being employed in several aspects of modern society. Among others, social robots have the potential to benefit education, healthcare, and tourism. To achieve this purpose, robots should be able to engage humans, recognize users’ emotions, and to some extent properly react and "behave" in a natural interaction. Most robotics applications primarily use visual information for emotion recognition, which is often based on facial expressions. However, the display of emotional states through facial expression is inherently a voluntary controlled process that is typical of human–human interaction. In fact, humans have not yet learned to use this channel when communicating with a robotic technology. Hence, there is an urgent need to exploit emotion information channels not directly controlled by humans, such as those that can be ascribed to physiological modulations. Thermal infrared imaging-based affective computing has the potential to be the solution to such an issue. It is a validated technology that allows the non-obtrusive monitoring of physiological parameters and from which it might be possible to infer affective states. This review is aimed to outline the advantages and the current research challenges of thermal imaging-based affective computing for human–robot interaction.

Incremental Low Rank Noise Reduction for Robust Infrared Tracking of Body Temperature during Medical Imaging

Thermal imagery for monitoring of body temperature provides a powerful tool to decrease health risks (e.g., burning) for patients during medical imaging (e.g., magnetic resonance imaging). The presented approach discusses an experiment to simulate radiology conditions with infrared imaging along with an automatic thermal monitoring/tracking system. The thermal tracking system uses an incremental low-rank noise reduction applying incremental singular value decomposition (SVD) and applies color based clustering for initialization of the region of interest (ROI) boundary. Then a particle filter tracks the ROI(s) from the entire thermal stream (video sequence). The thermal database contains 15 subjects in two positions (i.e., sitting, and lying) in front of thermal camera. This dataset is created to verify the robustness of our method with respect to motion-artifacts and in presence of additive noise (2–20%—salt and pepper noise). The proposed approach was tested for the infrared images in the dataset and was able to successfully measure and track the ROI continuously (100% detecting and tracking the temperature of participants), and provided considerable robustness against noise (unchanged accuracy even in 20% additive noise), which shows promising performance.

Lateral Sinus Floor Elevation Performed with Trapezoidal and Modified Triangular Flap Designs: A Randomized Pilot Study of Post-Operative Pain Using Thermal Infrared Imaging

Purpose: Post-operative pain and swelling are frequently observed after sinus lift procedures. The aim of the present study was the clinical evaluation of swelling and pain of two different sinus flap lift techniques using a visual analogue scale (VAS), verbal rating scale (VRS), and infrared thermal imaging (i.e., thermography). Materials Methods: A randomized controlled trial was conducted with 15 patients (30 sinuses in total) randomly allocated into two groups. For the sinuses of Group I a trapezoidal flap was used, while for Group II a modified triangular flap without anterior release was utilized. Postoperative pain was scored by means of a 100-mm VAS ranging from 0 (no pain) to 100 (worst pain imaginable), and was recorded at 2, 4, 6 and 14 days after surgery. Swelling was recorded by a verbal rating scale (VRS) and was classified into four categories: a score of 1 referred the absence of swelling, patients with intra-oral swelling in the surgical zone scored 2, any extra-oral swelling in the surgical zone scored 3, and intense swelling exhibited by extra-oral swelling extending beyond the surgical zone scored 4. The facial temperature was recorded before and after sinus augmentation, and at 2, 4, 6, and 14 days post-surgery to check the course of healing. Results: In Group I pain intensity was recorded at 2 days after surgery with a mean score of 38.67 ± 6.4 mm. Swelling was greater at 2 and 4 days, and was absent at day 6. The facial temperature difference before and after the procedure was 4.737 &deg;C ± 0.37. In Group II the pain score were lower than in Group I (p < 0.05). The score for swelling was 2 on the first and second days, and was reduced on day 4. After the second day the difference in temperature was significantly reduced as compared to the day of surgery (0.77 &deg;C); at 2 and 4 days no difference was registered. Conclusions: The results of this clinical study show the significant effectiveness of the modified triangular flap in the sinus lift procedure for reducing pain and swelling.

Surgeons' and surgical trainees' acute stress in real operations or simulation: A systematic review

BACKGROUND AND PURPOSE

Acute stress in surgery is ubiquitous and has an immediate impact on surgical performance and patient safety. Surgeons react with several coping strategies; however, they recognise the necessity of formal stress management training. Thus, stress assessment is a direct need. Surgical simulation is a validated standardised training milieu designed to replicate real-life situations. It replicates stress, prevents biases, and provides objective metrics. The complexity of stress mechanisms makes stress measurement difficult to quantify and interpret. This systematic review aims to identify studies that have used acute stress estimation measurements in surgeons or surgical trainees during real operations or surgical simulation, and to collectively present the rationale of these tools, with special emphasis in salivary markers.

METHODS

A search strategy was implemented to retrieve relevant articles from MEDLINE and SCOPUS databases. The 738 articles retrieved were reviewed for further evaluation according to the predetermined inclusion/exclusion criteria.

RESULTS

Thirty-three studies were included in this systematic review. The methods for acute stress assessment varied greatly among studies with the non-invasive techniques being the most commonly used. Subjective and objective tests for surgeons' acute stress assessment are being presented.

CONCLUSION

There is a broad spectrum of acute mental stress assessment tools in the surgical field and simulation and salivary biomarkers have recently gained popularity. There is a need to maintain a consistent methodology in future research, towards a deeper understanding of acute stress in the surgical field.

Breathing Analysis Using Thermal and Depth Imaging Camera Video Records

The paper is devoted to the study of facial region temperature changes using a simple thermal imaging camera and to the comparison of their time evolution with the pectoral area motion recorded by the MS Kinect depth sensor. The goal of this research is to propose the use of video records as alternative diagnostics of breathing disorders allowing their analysis in the home environment as well. The methods proposed include (i) specific image processing algorithms for detecting facial parts with periodic temperature changes; (ii) computational intelligence tools for analysing the associated videosequences; and (iii) digital filters and spectral estimation tools for processing the depth matrices. Machine learning applied to thermal imaging camera calibration allowed the recognition of its digital information with an accuracy close to 100% for the classification of individual temperature values. The proposed detection of breathing features was used for monitoring of physical activities by the home exercise bike. The results include a decrease of breathing temperature and its frequency after a load, with mean values -0.16 °C/min and -0.72 bpm respectively, for the given set of experiments. The proposed methods verify that thermal and depth cameras can be used as additional tools for multimodal detection of breathing patterns.