The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset

Sensing behavior change in chronic pain: a scoping review of sensor technology for use in daily life

ABSTRACT

Technology offers possibilities for quantification of behaviors and physiological changes of relevance to chronic pain, using wearable sensors and devices suitable for data collection in daily life contexts. We conducted a scoping review of wearable and passive sensor technologies that sample data of psychological interest in chronic pain, including in social situations. Sixty articles met our criteria from the 2783 citations retrieved from searching. Three-quarters of recruited people were with chronic pain, mostly musculoskeletal, and the remainder with acute or episodic pain; those with chronic pain had a mean age of 43 (few studies sampled adolescents or children) and 60% were women. Thirty-seven studies were performed in laboratory or clinical settings and the remainder in daily life settings. Most used only 1 type of technology, with 76 sensor types overall. The commonest was accelerometry (mainly used in daily life contexts), followed by motion capture (mainly in laboratory settings), with a smaller number collecting autonomic activity, vocal signals, or brain activity. Subjective self-report provided "ground truth" for pain, mood, and other variables, but often at a different timescale from the automatically collected data, and many studies reported weak relationships between technological data and relevant psychological constructs, for instance, between fear of movement and muscle activity. There was relatively little discussion of practical issues: frequency of sampling, missing data for human or technological reasons, and the users' experience, particularly when users did not receive data in any form. We conclude the review with some suggestions for content and process of future studies in this field.

Guarding and flow in the movements of people with chronic pain: A qualitative study of physiotherapists' observations

BACKGROUND

Among the adaptations of movement consistently associated with disability in chronic pain, guarding is common. Based on previous work, we sought to understand better the constituents of guarding; we also used the concept of flow to explore the description of un/naturalness that emerged from physiotherapists' descriptions of movement in chronic pain. The aim was to inform the design of technical systems to support people with chronic pain in everyday activities.

METHODS

Sixteen physiotherapists, experts in chronic pain, were interviewed while repeatedly watching short video clips of people with chronic low back pain doing simple movements; physiotherapists described the movements, particularly in relation to guarding and flow. The transcribed interviews were analysed thematically to elaborate these constructs.

RESULTS

Moderate agreement emerged on the extent of guarding in the videos, with good agreement that guarding conveyed caution about movement, distinct from biomechanical variables of stiffness or slow speed. Physiotherapists' comments on flow showed slightly better agreement, and described the overall movement in terms of restriction (where there was no flow or only some flow), of tempo of the entire movement, and as naturalness (distinguished from normality of movement).

CONCLUSIONS

These qualities of movement may be useful in designing technical systems to support self-management of chronic pain.

SIGNIFICANCE

Drawing on the descriptions of movements of people with chronic low back pain provided by expert physiotherapists to standard stimuli, two key concepts were elaborated. Guarding was distinguished from stiffness (a physical limitation) or slowness as motivated by fear or worry about movement. Flow served to describe harmonious and continuous movement, even when adapted around restrictions of pain. Movement behaviours associated with pain are better understood in terms of their particular function than aggregated without reference to function.

A Comprehensive Study on Pain Assessment from Multimodal Sensor Data

Pain assessment is a critical aspect of healthcare, influencing timely interventions and patient well-being. Traditional pain evaluation methods often rely on subjective patient reports, leading to inaccuracies and disparities in treatment, especially for patients who present difficulties to communicate due to cognitive impairments. Our contributions are three-fold. Firstly, we analyze the correlations of the data extracted from biomedical sensors. Then, we use state-of-the-art computer vision techniques to analyze videos focusing on the facial expressions of the patients, both per-frame and using the temporal context. We compare them and provide a baseline for pain assessment methods using two popular benchmarks: UNBC-McMaster Shoulder Pain Expression Archive Database and BioVid Heat Pain Database. We achieved an accuracy of over 96% and over 94% for the F1 Score, recall and precision metrics in pain estimation using single frames with the UNBC-McMaster dataset, employing state-of-the-art computer vision techniques such as Transformer-based architectures for vision tasks. In addition, from the conclusions drawn from the study, future lines of work in this area are discussed.

Cooperative Learning for Personalized Context-Aware Pain Assessment From Wearable Data

Despite the promising performance of automated pain assessment methods, current methods suffer from performance generalization due to the lack of relatively large, diverse, and annotated pain datasets. Further, the majority of current methods do not allow responsible interaction between the model and user, and do not take different internal and external factors into consideration during the model's design and development. This article aims to provide an efficient cooperative learning framework for the lack of annotated data while facilitating responsible user communication and taking individual differences into consideration during the development of pain assessment models. Our results using body and muscle movement data, collected from wearable devices, demonstrate that the proposed framework is effective in leveraging both the human and the machine to efficiently learn and predict pain.

Children's Pain Identification Based on Skin Potential Signal

Pain management is a crucial concern in medicine, particularly in the case of children who may struggle to effectively communicate their pain. Despite the longstanding reliance on various assessment scales by medical professionals, these tools have shown limitations and subjectivity. In this paper, we present a pain assessment scheme based on skin potential signals, aiming to convert subjective pain into objective indicators for pain identification using machine learning methods. We have designed and implemented a portable non-invasive measurement device to measure skin potential signals and conducted experiments involving 623 subjects. From the experimental data, we selected 358 valid records, which were then divided into 218 silent samples and 262 pain samples. A total of 38 features were extracted from each sample, with seven features displaying superior performance in pain identification. Employing three classification algorithms, we found that the random forest algorithm achieved the highest accuracy, reaching 70.63%. While this identification rate shows promise for clinical applications, it is important to note that our results differ from state-of-the-art research, which achieved a recognition rate of 81.5%. This discrepancy arises from the fact that our pain stimuli were induced by clinical operations, making it challenging to precisely control the stimulus intensity when compared to electrical or thermal stimuli. Despite this limitation, our pain assessment scheme demonstrates significant potential in providing objective pain identification in clinical settings. Further research and refinement of the proposed approach may lead to even more accurate and reliable pain management techniques in the future.

Association Between Chronic Back Pain and Protective Behaviors is Subjective and Context Dependent

Chronic lower back (CLB) pain limits patients' day-to-day activities, increases their missed days of work, and causes emotional distress. Developing adequate and individual-tailored treatment for CLB patients requires a better understanding of pain and protective behaviors, and how these behaviors are modulated or altered by context and subjectivity. In this work, we conducted experiments to investigate 1) the relationship between pain and protective behaviors in patients with CLB pain, 2) whether individual differences and context are relevant factors in the relationship, and 3) the impact of this relationship and its factors on the performance of current automated models for pain and protective behavior perception. Our results show 1) significant association (p - value < 0.05) between pain and protective behaviors in patients with CLB pain and 2) subjectivity and context are influential factors in this association. Further, our results show that considering this association along with its factors significantly (p-value < 0.05) improves the performance of automated pain and protective behaviors perception. These findings highlight the role of this association on pain and protective behaviors perception and raise several questions about the robustness of existing automated models that do not take this association into account.

Artificial Intelligence for Automatic Pain Assessment: Research Methods and Perspectives

Although proper pain evaluation is mandatory for establishing the appropriate therapy, self-reported pain level assessment has several limitations. Data-driven artificial intelligence (AI) methods can be employed for research on automatic pain assessment (APA). The goal is the development of objective, standardized, and generalizable instruments useful for pain assessment in different clinical contexts. The purpose of this article is to discuss the state of the art of research and perspectives on APA applications in both research and clinical scenarios. Principles of AI functioning will be addressed. For narrative purposes, AI-based methods are grouped into behavioral-based approaches and neurophysiology-based pain detection methods. Since pain is generally accompanied by spontaneous facial behaviors, several approaches for APA are based on image classification and feature extraction. Language features through natural language strategies, body postures, and respiratory-derived elements are other investigated behavioral-based approaches. Neurophysiology-based pain detection is obtained through electroencephalography, electromyography, electrodermal activity, and other biosignals. Recent approaches involve multimode strategies by combining behaviors with neurophysiological findings. Concerning methods, early studies were conducted by machine learning algorithms such as support vector machine, decision tree, and random forest classifiers. More recently, artificial neural networks such as convolutional and recurrent neural network algorithms are implemented, even in combination. Collaboration programs involving clinicians and computer scientists must be aimed at structuring and processing robust datasets that can be used in various settings, from acute to different chronic pain conditions. Finally, it is crucial to apply the concepts of explainability and ethics when examining AI applications for pain research and management.

Automatic assessment of pain based on deep learning methods: A systematic review

BACKGROUND AND OBJECTIVE

The automatic assessment of pain is vital in designing optimal pain management interventions focused on reducing suffering and preventing the functional decline of patients. In recent years, there has been a surge in the adoption of deep learning algorithms by researchers attempting to encode the multidimensional nature of pain into meaningful features. This systematic review aims to discuss the models, the methods, and the types of data employed in establishing the foundation of a deep learning-based automatic pain assessment system.

METHODS

The systematic review was conducted by identifying original studies searching digital libraries, namely Scopus, IEEE Xplore, and ACM Digital Library. Inclusion and exclusion criteria were applied to retrieve and select those of interest, published until December 2021.

RESULTS

A total of one hundred and ten publications were identified and categorized by the number of information channels used (unimodal versus multimodal approaches) and whether the temporal dimension was also used.

CONCLUSIONS

This review demonstrates the importance of multimodal approaches for automatic pain estimation, especially in clinical settings, and also reveals that significant improvements are observed when the temporal exploitation of modalities is included. It provides suggestions regarding better-performing deep architectures and learning methods. Also, it provides suggestions for adopting robust evaluation protocols and interpretation methods to provide objective and comprehensible results. Furthermore, the review presents the limitations of the available pain databases for optimally supporting deep learning model development, validation, and application as decision-support tools in real-life scenarios.

Multisensory Perception and Learning: Linking Pedagogy, Psychophysics, and Human–Computer Interaction

In this review, we discuss how specific sensory channels can mediate the learning of properties of the environment. In recent years, schools have increasingly been using multisensory technology for teaching. However, it still needs to be sufficiently grounded in neuroscientific and pedagogical evidence. Researchers have recently renewed understanding around the role of communication between sensory modalities during development. In the current review, we outline four principles that will aid technological development based on theoretical models of multisensory development and embodiment to foster in-depth, perceptual, and conceptual learning of mathematics. We also discuss how a multidisciplinary approach offers a unique contribution to development of new practical solutions for learning in school. Scientists, engineers, and pedagogical experts offer their interdisciplinary points of view on this topic. At the end of the review, we present our results, showing that one can use multiple sensory inputs and sensorimotor associations in multisensory technology to improve the discrimination of angles, but also possibly for educational purposes. Finally, we present an application, the ‘RobotAngle’ developed for primary (i.e., elementary) school children, which uses sounds and body movements to learn about angles.

Enterprise Strategic Management From the Perspective of Business Ecosystem Construction Based on Multimodal Emotion Recognition

Emotion recognition (ER) is an important part of building an intelligent human-computer interaction system and plays an important role in human-computer interaction. Often, people express their feelings through a variety of symbols, such as words and facial expressions. A business ecosystem is an economic community based on interacting organizations and individuals. Over time, they develop their capabilities and roles together and tend to develop themselves in the direction of one or more central enterprises. This paper aims to study a multimodal ER method based on attention mechanism. It analyzes the current emotional state of consumers and the development direction of enterprises through multi-modal ER of human emotions and analysis of market trends, so as to provide the most appropriate response or plan. This paper firstly describes the related methods of multimodal ER and deep learning in detail, and briefly outlines the meaning of enterprise strategy in the business ecosystem. Then, two datasets, CMU-MOSI and CMU-MOSEI, are selected to design the scheme for multimodal ER based on self-attention mechanism. Through the comparative analysis of the accuracy of single-modal and multi-modal ER, the self-attention mechanism is applied in the experiment. The experimental results show that the average recognition accuracy of happy under multimodal ER reaches 91.5%.

Computer mediated automatic detection of pain-related behavior: prospect, progress, perils

Pain is often characterized as a fundamentally subjective phenomenon; however, all pain assessment reduces the experience to observables, with strengths and limitations. Most evidence about pain derives from observations of pain-related behavior. There has been considerable progress in articulating the properties of behavioral indices of pain; especially, but not exclusively those based on facial expression. An abundant literature shows that a limited subset of facial actions, with homologs in several non-human species, encode pain intensity across the lifespan. Unfortunately, acquiring such measures remains prohibitively impractical in many settings because it requires trained human observers and is laborious. The advent of the field of affective computing, which applies computer vision and machine learning (CVML) techniques to the recognition of behavior, raised the prospect that advanced technology might overcome some of the constraints limiting behavioral pain assessment in clinical and research settings. Studies have shown that it is indeed possible, through CVML, to develop systems that track facial expressions of pain. There has since been an explosion of research testing models for automated pain assessment. More recently, researchers have explored the feasibility of multimodal measurement of pain-related behaviors. Commercial products that purport to enable automatic, real-time measurement of pain expression have also appeared. Though progress has been made, this field remains in its infancy and there is risk of overpromising on what can be delivered. Insufficient adherence to conventional principles for developing valid measures and drawing appropriate generalizations to identifiable populations could lead to scientifically dubious and clinically risky claims. There is a particular need for the development of databases containing samples from various settings in which pain may or may not occur, meticulously annotated according to standards that would permit sharing, subject to international privacy standards. Researchers and users need to be sensitive to the limitations of the technology (for e.g., the potential reification of biases that are irrelevant to the assessment of pain) and its potentially problematic social implications.

Learning Pain from Action Unit Combinations: A Weakly Supervised Approach via Multiple Instance Learning

Patient pain can be detected highly reliably from facial expressions using a set of facial muscle-based action units (AUs) defined by the Facial Action Coding System (FACS). A key characteristic of facial expression of pain is the simultaneous occurrence of pain-related AU combinations, whose automated detection would be highly beneficial for efficient and practical pain monitoring. Existing general Automated Facial Expression Recognition (AFER) systems prove inadequate when applied specifically for detecting pain as they either focus on detecting individual pain-related AUs but not on combinations or they seek to bypass AU detection by training a binary pain classifier directly on pain intensity data but are limited by lack of enough labeled data for satisfactory training. In this paper, we propose a new approach that mimics the strategy of human coders of decoupling pain detection into two consecutive tasks: one performed at the individual video-frame level and the other at video-sequence level. Using state-of-the-art AFER tools to detect single AUs at the frame level, we propose two novel data structures to encode AU combinations from single AU scores. Two weakly supervised learning frameworks namely multiple instance learning (MIL) and multiple clustered instance learning (MCIL) are employed corresponding to each data structure to learn pain from video sequences. Experimental results show an 87% pain recognition accuracy with 0.94 AUC (Area Under Curve) on the UNBC-McMaster Shoulder Pain Expression dataset. Tests on long videos in a lung cancer patient video dataset demonstrates the potential value of the proposed system for pain monitoring in clinical settings.

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies.

Automatic Detection of Pain from Facial Expressions: A Survey

Pain sensation is essential for survival, since it draws attention to physical threat to the body. Pain assessment is usually done through self-reports. However, self-assessment of pain is not available in the case of noncommunicative patients, and therefore, observer reports should be relied upon. Observer reports of pain could be prone to errors due to subjective biases of observers. Moreover, continuous monitoring by humans is impractical. Therefore, automatic pain detection technology could be deployed to assist human caregivers and complement their service, thereby improving the quality of pain management, especially for noncommunicative patients. Facial expressions are a reliable indicator of pain, and are used in all observer-based pain assessment tools. Following the advancements in automatic facial expression analysis, computer vision researchers have tried to use this technology for developing approaches for automatically detecting pain from facial expressions. This paper surveys the literature published in this field over the past decade, categorizes it, and identifies future research directions. The survey covers the pain datasets used in the reviewed literature, the learning tasks targeted by the approaches, the features extracted from images and image sequences to represent pain-related information, and finally, the machine learning methods used.

Machine Learning in Chronic Pain Research: A Scoping Review

Given the high prevalence and associated cost of chronic pain, it has a significant impact on individuals and society. Improvements in the treatment and management of chronic pain may increase patients’ quality of life and reduce societal costs. In this paper, we evaluate state-of-the-art machine learning approaches in chronic pain research. A literature search was conducted using the PubMed, IEEE Xplore, and the Association of Computing Machinery (ACM) Digital Library databases. Relevant studies were identified by screening titles and abstracts for keywords related to chronic pain and machine learning, followed by analysing full texts. Two hundred and eighty-seven publications were identified in the literature search. In total, fifty-three papers on chronic pain research and machine learning were reviewed. The review showed that while many studies have emphasised machine learning-based classification for the diagnosis of chronic pain, far less attention has been paid to the treatment and management of chronic pain. More research is needed on machine learning approaches to the treatment, rehabilitation, and self-management of chronic pain. As with other chronic conditions, patient involvement and self-management are crucial. In order to achieve this, patients with chronic pain need digital tools that can help them make decisions about their own treatment and care.

Multimodal Signal Analysis for Pain Recognition in Physiotherapy Using Wavelet Scattering Transform

Fascial therapy is an effective, yet painful, procedure. Information about pain level is essential for the physiotherapist to adjust the therapy course and avoid potential tissue damage. We have developed a method for automatic pain-related reaction assessment in physiotherapy due to the subjectivity of a self-report. Based on a multimodal data set, we determine the feature vector, including wavelet scattering transforms coefficients. The AdaBoost classification model distinguishes three levels of reaction (no-pain, moderate pain, and severe pain). Because patients vary in pain reactions and pain resistance, our survey assumes a subject-dependent protocol. The results reflect an individual perception of pain in patients. They also show that multiclass evaluation outperforms the binary recognition.

PLAAN: Pain Level Assessment with Anomaly-detection based Network

Automatic chronic pain assessment and pain intensity estimation has been attracting growing attention due to its widespread applications. One of the prevalent issues in automatic pain analysis is inadequate balanced expert-labelled data for pain estimation. This work proposes an anomaly detection based network addressing one of the existing limitations of automatic pain assessment. The evaluation of the network is performed on pain intensity estimation and protective behaviour estimation tasks from body movements in the EmoPain Challenge dataset. The EmoPain dataset consists of body part based sensor data for both the tasks. The proposed network, PLAAN (Pain Level Assessment with Anomaly-detection based Network), is a lightweight LSTM-DNN network which considers features based on sensor data as the input and predicts intensity level of pain and presence or absence of protective behaviour in chronic low back pain patients. Joint training considering body movement patterns, such as exercise type, corresponding to pain exhibition as a label improves the performance of the network. However, contrary to perception, protective behaviour rather exists sporadically alongside pain in the EmoPain dataset. This induces yet another complication in accurate estimation of protective behaviour. This problem is resolved by incorporating anomaly detection in the network. A detailed comparison of different networks with varied features is outlined in the paper, presenting a significant improvement with the final proposed anomaly detection based network.

Analysis of the analgesic effects of tricyclic antidepressants on neuropathic pain, diabetic neuropathic pain, and fibromyalgia in rat models

OBJECTIVE

To investigate the analgesic effect of amitriptyline on neuropathic pain model rats, diabetic neuropathic pain model rats and fibromyalgia model rats.

METHODS

The healthy male Sprague wrote - Dawley (SD) rats were taken as the research object, and they were randomly divided into model group (group A), beside the sciatic nerve and injection of 5 mm amitriptyline group (group B), beside the sciatic nerve and injection of 10 mm amitriptyline group (group C), beside the sciatic nerve and injection of 15 mm amitriptyline group (group D), intraperitoneal injection of amitriptyline group (group E). Pain induced by selective injury of sciatic nerve branches in rats, pain induced by chronic compression of sciatic nerve, diabetic neuropathic pain and fibromyalgia were conducted to determine the pain threshold of mechanical stimulation in rats after drug administration.

RESULTS

The pain threshold of mechanical stimulation in the local amitriptyline group (group B, C, D) was significantly higher than that in the group A and group E at each time point after drug treatment, and the pain threshold of mechanical stimulation gradually increased with the increase of concentration. There was no statistically significant difference in mechanical stimulation pain threshold between group A and group E at each time point after drug treatment.

CONCLUSION

Para-sciatic injection of amitriptyline at different concentrations has analgesic effects on neuropathic pain, diabetic neuropathic pain and fibromyalgia in rat models, and amitriptyline directly ACTS on the local sciatic nerve.

Deep-Learning-Based Models for Pain Recognition: A Systematic Review

Traditional standards employed for pain assessment have many limitations. One such limitation is reliability linked to inter-observer variability. Therefore, there have been many approaches to automate the task of pain recognition. Recently, deep-learning methods have appeared to solve many challenges such as feature selection and cases with a small number of data sets. This study provides a systematic review of pain-recognition systems that are based on deep-learning models for the last two years. Furthermore, it presents the major deep-learning methods used in the review papers. Finally, it provides a discussion of the challenges and open issues.

Prospective Study Evaluating a Pain Assessment Tool in a Postoperative Environment: Protocol for Algorithm Testing and Enhancement

Background

Assessment of pain is critical to its optimal treatment. There is a high demand for accurate objective pain assessment for effectively optimizing pain management interventions. However, pain is a multivalent, dynamic, and ambiguous phenomenon that is difficult to quantify, particularly when the patient’s ability to communicate is limited. The criterion standard of pain intensity assessment is self-reporting. However, this unidimensional model is disparaged for its oversimplification and limited applicability in several vulnerable patient populations. Researchers have attempted to develop objective pain assessment tools through analysis of physiological pain indicators, such as electrocardiography, electromyography, photoplethysmography, and electrodermal activity. However, pain assessment by using only these signals can be unreliable, as various other factors alter these vital signs and the adaptation of vital signs to pain stimulation varies from person to person. Objective pain assessment using behavioral signs such as facial expressions has recently gained attention.

Objective

Our objective is to further the development and research of a pain assessment tool for use with patients who are likely experiencing mild to moderate pain. We will collect observational data through wearable technologies, measuring facial electromyography, electrocardiography, photoplethysmography, and electrodermal activity.

Methods

This protocol focuses on the second phase of a larger study of multimodal signal acquisition through facial muscle electrical activity, cardiac electrical activity, and electrodermal activity as indicators of pain and for building predictive models. We used state-of-the-art standard sensors to measure bioelectrical electromyographic signals and changes in heart rate, respiratory rate, and oxygen saturation. Based on the results, we further developed the pain assessment tool and reconstituted it with modern wearable sensors, devices, and algorithms. In this second phase, we will test the smart pain assessment tool in communicative patients after elective surgery in the recovery room.

Results

Our human research protections application for institutional review board review was approved for this part of the study. We expect to have the pain assessment tool developed and available for further research in early 2021. Preliminary results will be ready for publication during fall 2020.

Conclusions

This study will help to further the development of and research on an objective pain assessment tool for monitoring patients likely experiencing mild to moderate pain.

International Registered Report Identifier (IRRID)

DERR1-10.2196/17783

[From external assessment of pain to automated multimodal measurement of pain intensity : Narrative review of state of research and clinical perspectives]

BACKGROUND

In patients with limited communication skills, the use of conventional scales or external assessment is only possible to a limited extent or not at all. Multimodal pain recognition based on artificial intelligence (AI) algorithms could be a solution.

OBJECTIVE

Overview of the methods of automated multimodal pain measurement and their recognition rates that were calculated with AI algorithms.

METHODS

In April 2018, 101 studies on automated pain recognition were found in the Web of Science database to illustrate the current state of research. A selective literature review with special consideration of recognition rates of automated multimodal pain measurement yielded 14 studies, which are the focus of this review.

RESULTS

The variance in recognition rates was 52.9-55.0% (pain threshold) and 66.8-85.7%; in nine studies the recognition rate was ≥80% (pain tolerance), while one study reported recognition rates of 79.3% (pain threshold) and 90.9% (pain tolerance).

CONCLUSION

Pain is generally recorded multimodally, based on external observation scales. With regard to automated pain recognition and on the basis of the 14 selected studies, there is to date no conclusive evidence that multimodal automated pain recognition is superior to unimodal pain recognition. In the clinical context, multimodal pain recognition could be advantageous, because this approach is more flexible. In the case of one modality not being available, e.g., electrodermal activity in hand burns, the algorithm could use other modalities (video) and thus compensate for missing information.

A review of computational approaches for evaluation of rehabilitation exercises

Recent advances in data analytics and computer-aided diagnostics stimulate the vision of patient-centric precision healthcare, where treatment plans are customized based on the health records and needs of every patient. In physical rehabilitation, the progress in machine learning and the advent of affordable and reliable motion capture sensors have been conducive to the development of approaches for automated assessment of patient performance and progress toward functional recovery. The presented study reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems. Such approaches will play an important role in supplementing traditional rehabilitation assessment performed by trained clinicians, and in assisting patients participating in home-based rehabilitation. The reviewed computational methods for exercise evaluation are grouped into three main categories: discrete movement score, rule-based, and template-based approaches. The review places an emphasis on the application of machine learning methods for movement evaluation in rehabilitation. Related work in the literature on data representation, feature engineering, movement segmentation, and scoring functions is presented. The study also reviews existing sensors for capturing rehabilitation movements and provides an informative listing of pertinent benchmark datasets. The significance of this paper is in being the first to provide a comprehensive review of computational methods for evaluation of patient performance in rehabilitation programs.

Analysis of Facial Information for Healthcare Applications: A Survey on Computer Vision-Based Approaches

This paper gives an overview of the cutting-edge approaches that perform facial cue analysis in the healthcare area. The document is not limited to global face analysis but it also concentrates on methods related to local cues (e.g., the eyes). A research taxonomy is introduced by dividing the face in its main features: eyes, mouth, muscles, skin, and shape. For each facial feature, the computer vision-based tasks aiming at analyzing it and the related healthcare goals that could be pursued are detailed.

Two-Stream Attention Network for Pain Recognition from Video Sequences

Several approaches have been proposed for the analysis of pain-related facial expressions. These approaches range from common classification architectures based on a set of carefully designed handcrafted features, to deep neural networks characterised by an autonomous extraction of relevant facial descriptors and simultaneous optimisation of a classification architecture. In the current work, an end-to-end approach based on attention networks for the analysis and recognition of pain-related facial expressions is proposed. The method combines both spatial and temporal aspects of facial expressions through a weighted aggregation of attention-based neural networks' outputs, based on sequences of Motion History Images (MHIs) and Optical Flow Images (OFIs). Each input stream is fed into a specific attention network consisting of a Convolutional Neural Network (CNN) coupled to a Bidirectional Long Short-Term Memory (BiLSTM) Recurrent Neural Network (RNN). An attention mechanism generates a single weighted representation of each input stream (MHI sequence and OFI sequence), which is subsequently used to perform specific classification tasks. Simultaneously, a weighted aggregation of the classification scores specific to each input stream is performed to generate a final classification output. The assessment conducted on both the BioVid Heat Pain Database (Part A) and SenseEmotion Database points at the relevance of the proposed approach, as its classification performance is on par with state-of-the-art classification approaches proposed in the literature.

Sensor Technologies to Manage the Physiological Traits of Chronic Pain: A Review

Non-oncologic chronic pain is a common high-morbidity impairment worldwide and acknowledged as a condition with significant incidence on quality of life. Pain intensity is largely perceived as a subjective experience, what makes challenging its objective measurement. However, the physiological traces of pain make possible its correlation with vital signs, such as heart rate variability, skin conductance, electromyogram, etc., or health performance metrics derived from daily activity monitoring or facial expressions, which can be acquired with diverse sensor technologies and multisensory approaches. As the assessment and management of pain are essential issues for a wide range of clinical disorders and treatments, this paper reviews different sensor-based approaches applied to the objective evaluation of non-oncological chronic pain. The space of available technologies and resources aimed at pain assessment represent a diversified set of alternatives that can be exploited to address the multidimensional nature of pain.

The relationship between guarding, pain, and emotion

INTRODUCTION

Pain-related behavior in people with chronic pain is often overlooked in a focus on increasing the amount of activity, yet it may limit activity and maintain pain and disability. Targeting it in treatment requires better understanding of the role of beliefs, emotion, and pain in pain behavior.

OBJECTIVES

This study aimed to clarify the interrelationships between guarding, pain, anxiety, and confidence in movement in people with chronic pain in everyday movements.

METHODS

Physiotherapists rated extent of guarding on videos of people with chronic pain and healthy controls making specific movements. Bayesian modelling was used to determine how guarding was related to self-reported pain intensity, anxiety, and emotional distress, and observer-rated confidence in movement.

RESULTS

The absence of guarding was associated with low levels of pain, anxiety, distress, and higher movement self-efficacy, but guarding behavior occurred at high and low levels of each of those variables. Guarding was not directly dependent on pain but on anxiety; the relationship between pain and guarding was mediated by anxiety, with a high probability. Nor was guarding directly related to the broader distress score, but to self-efficacy for movement, again with a high probability.

CONCLUSION

Pain-related guarding is more likely to be effectively addressed by intervention to reduce anxiety rather than pain (such as analgesia); more attention to how people move with chronic pain, rather than only how much they move, is likely to help to extend activity.

The KIMORE Dataset: KInematic Assessment of MOvement and Clinical Scores for Remote Monitoring of Physical REhabilitation

This paper proposes a free dataset, available at the following link,¹named KIMORE, regarding different rehabilitation exercises collected by a RGB-D sensor. Three data inputs including RGB, depth videos, and skeleton joint positions were recorded during five physical exercises, specific for low back pain and accurately selected by physicians. For each exercise, the dataset also provides a set of features, specifically defined by the physicians, and relevant to describe its scope. These features, validated with respect to a stereophotogrammetric system, can be analyzed to compute a score for the subject's performance. The dataset also contains an evaluation of the same performance provided by the clinicians, through a clinical questionnaire. The impact of KIMORE has been analyzed by comparing the output obtained by an example of rule and template-based approaches and the clinical score. The dataset presented is intended to be used as a benchmark for human movement assessment in a rehabilitation scenario in order to test the effectiveness and the reliability of different computational approaches. Unlike other existing datasets, the KIMORE merges a large heterogeneous population of 78 subjects, divided into 2 groups with 44 healthy subjects and 34 with motor dysfunctions. It provides the most clinically-relevant features and the clinical score for each exercise.¹https://univpm-my.sharepoint.com/:f:/g/personal/p008099_staff_univpm_it/EiwbKIzk6N9NoJQx4J8aubIBx0o7tIa1XwclWp1NmRkA-w?e=F3jtBk.

A Data Set of Human Body Movements for Physical Rehabilitation Exercises

The article presents University of Idaho - Physical Rehabilitation Movement Data (UI-PRMD) - a publically available data set of movements related to common exercises performed by patients in physical rehabilitation programs. For the data collection, 10 healthy subjects performed 10 repetitions of different physical therapy movements, with a Vicon optical tracker and a Microsoft Kinect sensor used for the motion capturing. The data are in a format that includes positions and angles of full-body joints. The objective of the data set is to provide a basis for mathematical modeling of therapy movements, as well as for establishing performance metrics for evaluation of patient consistency in executing the prescribed rehabilitation exercises.

Objectively measuring pain using facial expression: is the technology finally ready?

Currently, clinicians observe pain-related behaviors and use patient self-report measures in order to determine pain severity. This paper reviews the evidence when facial expression is used as a measure of pain. We review the literature reporting the relevance of facial expression as a diagnostic measure, which facial movements are indicative of pain, and whether such movements can be reliably used to measure pain. We conclude that although the technology for objective pain measurement is not yet ready for use in clinical settings, the potential benefits to patients in improved pain management, combined with the advances being made in sensor technology and artificial intelligence, provide opportunities for research and innovation.

Head movements and postures as pain behavior

Pain assessment can benefit from observation of pain behaviors, such as guarding or facial expression, and observational pain scales are widely used in clinical practice with nonverbal patients. However, little is known about head movements and postures in the context of pain. In this regard, we analyze videos of three publically available datasets. The BioVid dataset was recorded with healthy participants subjected to painful heat stimuli. In the BP4D dataset, healthy participants performed a cold-pressor test and several other tasks (meant to elicit emotion). The UNBC dataset videos show shoulder pain patients during range-of-motion tests to their affected and unaffected limbs. In all videos, participants were sitting in an upright position. We studied head movements and postures that occurred during the painful and control trials by measuring head orientation from video over time, followed by analyzing posture and movement summary statistics and occurrence frequencies of typical postures and movements. We found significant differences between pain and control trials with analyses of variance and binomial tests. In BioVid and BP4D, pain was accompanied by head movements and postures that tend to be oriented downwards or towards the pain site. We also found differences in movement range and speed in all three datasets. The results suggest that head movements and postures should be considered for pain assessment and research. As additional pain indicators, they possibly might improve pain management whenever behavior is assessed, especially in nonverbal individuals such as infants or patients with dementia. However, in advance more research is needed to identify specific head movements and postures in pain patients.