Convolutional Neural Networks for Differential Diagnosis of Raynaud’s Phenomenon Based on Hands Thermal Patterns

Thermal imaging of local skin temperature as part of quality and safety assessment of injectable drugs

Injection of high-quality drugs can occasionally cause unexpected and unexplained local complications. As the current standard for drug quality control does not include an assessment of the local irritation effects of drugs, this effect may cause postinjection complications. Simultaneously, local irritation effects of the drugs can be assessed based on local tissue inflammation. The dynamics of local temperature can assess inflammation. Infrared monitoring of local skin temperature dynamics at subcutaneous, intramuscular, and intravenous injection sites of drugs under experimental and clinical conditions can improve their quality and safety. Therefore, there is a need to include dynamic thermography in the standard of biological evaluation of the quality and safety of drugs in the dosage form "solution for injections." This eliminates the local irritation and necrotizing activity of drugs and minimizes the development of local pain syndrome, aseptic inflammation, necrosis, and abscess.

Reference Standards for Digital Infrared Thermography Measuring Surface Temperature of the Upper Limbs

(1) Background: although digital infrared thermographic imaging (DITI) is used for diverse medical conditions of the upper limbs, no reference standards have been established. This study aims to establish reference standards by analyzing DITI results of the upper limbs. (2) Methods: we recruited 905 healthy Korean adults and conducted thermography on six regions (dorsal arm, ventral arm, lateral arm, medial arm, dorsal hand, and ventral hand region). We analyzed the data based on the proximity of regions of interest (ROIs), sex, and age. (3) Results: the average temperature (°C) and temperature discrepancy between the right and the left sides (ΔT) of each ROI varied significantly (p < 0.001), ranging from 28.45 ± 5.71 to 29.74 ± 5.14 and from 0.01 ± 0.49 to 0.15 ± 0.62, respectively. The temperature decreased towards the distal ROIs compared to proximal ROIs. The average temperatures of the same ROIs were significantly higher for men than women in all regions (p < 0.001). Across all regions, except the dorsal hand region, average temperatures tended to increase with age, particularly in individuals in their 30s and older (p < 0.001). (4) Conclusions: these data could be used as DITI reference standards to identify skin temperature abnormalities of the upper limbs. However, it is important to consider various confounding factors, and further research is required to validate the accuracy of our results under pathological conditions.

Intraoperative thermal infrared imaging in neurosurgery: machine learning approaches for advanced segmentation of tumors

Surgical resection is one of the most relevant practices in neurosurgery. Finding the correct surgical extent of the tumor is a key question and so far several techniques have been employed to assist the neurosurgeon in preserving the maximum amount of healthy tissue. Some of these methods are invasive for patients, not always allowing high precision in the detection of the tumor area. The aim of this study is to overcome these limitations, developing machine learning based models, relying on features obtained from a contactless and non-invasive technique, the thermal infrared (IR) imaging. The thermal IR videos of thirteen patients with heterogeneous tumors were recorded in the intraoperative context. Time (TD)- and frequency (FD)-domain features were extracted and fed different machine learning models. Models relying on FD features have proven to be the best solutions for the optimal detection of the tumor area (Average Accuracy = 90.45%; Average Sensitivity = 84.64%; Average Specificity = 93,74%). The obtained results highlight the possibility to accurately detect the tumor lesion boundary with a completely non-invasive, contactless, and portable technology, revealing thermal IR imaging as a very promising tool for the neurosurgeon.

Low-cost thermal imaging with machine learning for non-invasive diagnosis and therapeutic monitoring of pneumonia

Rapid screening and early treatment of lung infection are essential for effective control of many epidemics such as Coronavirus Disease 2019 (COVID-19). Recent studies have demonstrated the potential correlation between lung infection and the change of back skin temperature distribution. Based on these findings, we propose to use low-cost, portable and rapid thermal imaging in combination with image-processing algorithms and machine learning analysis for non-invasive and safe detection of pneumonia. The proposed method was tested in 69 subjects (30 normal adults, 11 cases of fever without pneumonia, 19 cases of general pneumonia and 9 cases of COVID-19) where both RGB and thermal images were acquired from the back of each subject. The acquired images were processed automatically in order to extract multiple location and shape features that distinguish normal subjects from pneumonia patients at a high accuracy of 93 % . Furthermore, daily assessment of two pneumonia patients by the proposed method accurately predicted the clinical outcomes, coincident with those of laboratory tests. Our pilot study demonstrated the technical feasibility of portable and intelligent thermal imaging for screening and therapeutic assessment of pneumonia. The method can be potentially implemented in under-resourced regions for more effective control of respiratory epidemics.

Depth-Wise Separable Convolution Attention Module for Garbage Image Classification

Currently, how to deal with the massive garbage produced by various human activities is a hot topic all around the world. In this paper, a preliminary and essential step is to classify the garbage into different categories. However, the mainstream waste classification mode relies heavily on manual work, which consumes a lot of labor and is very inefficient. With the rapid development of deep learning, convolutional neural networks (CNN) have been successfully applied to various application fields. Therefore, some researchers have directly adopted CNNs to classify garbage through their images. However, compared with other images, the garbage images have their own characteristics (such as inter-class similarity, intra-class variance and complex background). Thus, neglecting these characteristics would impair the classification accuracy of CNN. To overcome the limitations of existing garbage image classification methods, a Depth-wise Separable Convolution Attention Module (DSCAM) is proposed in this paper. In DSCAM, the inherent relationships of channels and spatial positions in garbage image features are captured by two attention modules with depth-wise separable convolutions, so that our method could only focus on important information and ignore the interference. Moreover, we also adopt a residual network as the backbone of DSCAM to enhance its discriminative ability. We conduct the experiments on five garbage datasets. The experimental results demonstrate that the proposed method could effectively classify the garbage images and that it outperforms some classical methods.

Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach

Extensive possibilities of applications have rendered emotion recognition ineluctable and challenging in the fields of computer science as well as in human-machine interaction and affective computing. Fields that, in turn, are increasingly requiring real-time applications or interactions in everyday life scenarios. However, while extremely desirable, an accurate and automated emotion classification approach remains a challenging issue. To this end, this study presents an automated emotion recognition model based on easily accessible physiological signals and deep learning (DL) approaches. As a DL algorithm, a Feedforward Neural Network was employed in this study. The network outcome was further compared with canonical machine learning algorithms such as random forest (RF). The developed DL model relied on the combined use of wearables and contactless technologies, such as thermal infrared imaging. Such a model is able to classify the emotional state into four classes, derived from the linear combination of valence and arousal (referring to the circumplex model of affect’s four-quadrant structure) with an overall accuracy of 70% outperforming the 66% accuracy reached by the RF model. Considering the ecological and agile nature of the technique used the proposed model could lead to innovative applications in the affective computing field.

Preparing infection detection technology for hospital at home after lower limb external fixation

Background

Patients with severe bone fractures and complex bone deformities are treated by orthopedic surgeons with external fixation for several months. During this long treatment period, there is a high risk of inflammation and infection at the superficial skin area (pin site). This can develop into a devastating, sometimes fatal, and always costly condition of deep bone infection.

Objective

For pin site infection surveillance, thermography technology could be the solution to build an objective and continuous home-based remote monitoring tool to avoid frequent nursing care and hospital visits. However, future studies of infection monitoring require a preliminary step to automate the process of locating and detecting the pin sites in thermal images reliably for temperature measurement, and this step is the aim of this study.

Methods

This study presents an automatic approach for identifying and annotating pin sites on visible images using bounding boxes and transferring them to the corresponding thermal images for temperature measurement. The pin site is detected by applying deep learning-based object detection architecture YOLOv5 with a novel loss evaluation and regression method, control distance intersection over union. Furthermore, we address detecting pin sites in a practical environment (home setting) accurately through transfer learning.

Results and conclusion

The proposed model offers the pin site detection in 1.8 ms with a high precision of 0.98 and enables temperature information extraction. Our work for automatic pin site annotation on thermography paves the way for future research on infection assessment on thermography.

The Prediction of Running Velocity during the 30-15 Intermittent Fitness Test Using Accelerometry-Derived Metrics and Physiological Parameters: A Machine Learning Approach

Measuring exercise variables is one of the most important points to consider to maximize physiological adaptations. High-intensity interval training (HIIT) is a useful method to improve both cardiovascular and neuromuscular performance. The 30–15_IFT is a field test reflecting the effort elicited by HIIT, and the final velocity reached in the test is used to set the intensity of HIIT during the training session. In order to have a valid measure of the velocity during training, devices such as GPS can be used. However, in several situations (e.g., indoor setting), such devices do not provide reliable measures. The aim of the study was to predict exact running velocity during the 30–15_IFT using accelerometry-derived metrics (i.e., Player Load and Average Net Force) and heart rate (HR) through a machine learning (ML) approach (i.e., Support Vector Machine) with a leave-one-subject-out cross-validation. The SVM approach showed the highest performance to predict running velocity (r = 0.91) when compared to univariate approaches using PL (r = 0.62), AvNetForce (r = 0.73) and HR only (r = 0.87). In conclusion, the presented multivariate ML approach is able to predict running velocity better than univariate ones, and the model is generalizable across subjects.

Improving Human-Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.

Regions of interest selection and thermal imaging data analysis in sports and exercise science: a narrative review

OBJECTIVE

Infrared thermography (IRT) is a non-invasive, contactless and low-cost technology that allows recording of the radiating energy that is released from a body, providing an estimate of its superficial temperature. Thanks to the improvement of infrared thermal detectors, this technique is widely used in the biomedical field to monitor the skin temperature for different purposes (e.g. assessing circulatory diseases, psychophysiological state, affective computing). Particularly, in sports and exercise science, thermography is extensively used to assess sports performance, to investigate superficial vascular changes induced by physical exercise, and to monitor injuries. However, the methods of analysis employed to treat IRT data are not standardized, and hence introduce variability in the results.

APPROACH

This review focuses on the methods of analysis currently used for thermal imaging in sports and exercise science.

MAIN RESULTS

Firstly, the procedures employed for the selection of regions of interest (ROIs) from anatomical body districts are reviewed, paying attention also to the potentialities of morphing algorithms to increase the reproducibility of thermal results. Secondly, the statistical approaches utilized to characterize the temperature frequency and spatial distributions within ROIs are investigated, showing their strengths and weaknesses. Moreover, the importance of employing tracking methods to analyze the temporal thermal oscillations within ROIs is discussed. Thirdly, the capability of employing procedures of investigation based on machine learning frameworks on thermal imaging in sports science is examined.

SIGNIFICANCE

Finally, some proposals to improve the standardization and the reproducibility of IRT data analysis are provided, in order to facilitate the development of a common database of thermal images and to improve the effectiveness of IRT in sports science.