Automatic assessment of pain based on deep learning methods: A systematic review

Advancing the integration of biosignal-based automated pain assessment methods into a comprehensive model for addressing cancer pain

BACKGROUND

Tailoring effective strategies for cancer pain management requires a careful analysis of multiple factors that influence pain phenomena and, ultimately, guide the therapy. While there is a wealth of research on automatic pain assessment (APA), its integration with clinical data remains inadequately explored. This study aimed to address the potential correlations between subjective and APA-derived objectives variables in a cohort of cancer patients.

METHODS

A multidimensional statistical approach was employed. Demographic, clinical, and pain-related variables were examined. Objective measures included electrodermal activity (EDA) and electrocardiogram (ECG) signals. Sensitivity analysis, multiple factorial analysis (MFA), hierarchical clustering on principal components (HCPC), and multivariable regression were used for data analysis.

RESULTS

The study analyzed data from 64 cancer patients. MFA revealed correlations between pain intensity, type, Eastern Cooperative Oncology Group Performance status (ECOG), opioids, and metastases. Clustering identified three distinct patient groups based on pain characteristics, treatments, and ECOG. Multivariable regression analysis showed associations between pain intensity, ECOG, type of breakthrough cancer pain, and opioid dosages. The analyses failed to find a correlation between subjective and objective pain variables.

CONCLUSIONS

The reported pain perception is unrelated to the objective variables of APA. An in-depth investigation of APA is required to understand the variables to be studied, the operational modalities, and above all, strategies for appropriate integration with data obtained from self-reporting.

TRIAL REGISTRATION

This study is registered with ClinicalTrials.gov, number (NCT04726228), registered 27 January 2021, https://classic.

CLINICALTRIALS

gov/ct2/show/NCT04726228?term=nct04726228&draw=2&rank=1.

From facial expressions to algorithms: a narrative review of animal pain recognition technologies

Facial expressions are essential for communication and emotional expression across species. Despite the improvements brought by tools like the Horse Grimace Scale (HGS) in pain recognition in horses, their reliance on human identification of characteristic traits presents drawbacks such as subjectivity, training requirements, costs, and potential bias. Despite these challenges, the development of facial expression pain scales for animals has been making strides. To address these limitations, Automated Pain Recognition (APR) powered by Artificial Intelligence (AI) offers a promising advancement. Notably, computer vision and machine learning have revolutionized our approach to identifying and addressing pain in non-verbal patients, including animals, with profound implications for both veterinary medicine and animal welfare. By leveraging the capabilities of AI algorithms, we can construct sophisticated models capable of analyzing diverse data inputs, encompassing not only facial expressions but also body language, vocalizations, and physiological signals, to provide precise and objective evaluations of an animal's pain levels. While the advancement of APR holds great promise for improving animal welfare by enabling better pain management, it also brings forth the need to overcome data limitations, ensure ethical practices, and develop robust ground truth measures. This narrative review aimed to provide a comprehensive overview, tracing the journey from the initial application of facial expression recognition for the development of pain scales in animals to the recent application, evolution, and limitations of APR, thereby contributing to understanding this rapidly evolving field.

Incorporation of "Artificial Intelligence" for Objective Pain Assessment: A Comprehensive Review

Pain is a significant health issue, and pain assessment is essential for proper diagnosis, follow-up, and effective management of pain. The conventional methods of pain assessment often suffer from subjectivity and variability. The main issue is to understand better how people experience pain. In recent years, artificial intelligence (AI) has been playing a growing role in improving clinical diagnosis and decision-making. The application of AI offers promising opportunities to improve the accuracy and efficiency of pain assessment. This review article provides an overview of the current state of AI in pain assessment and explores its potential for improving accuracy, efficiency, and personalized care. By examining the existing literature, research gaps, and future directions, this article aims to guide further advancements in the field of pain management. An online database search was conducted via multiple websites to identify the relevant articles. The inclusion criteria were English articles published between January 2014 and January 2024). Articles that were available as full text clinical trials, observational studies, review articles, systemic reviews, and meta-analyses were included in this review. The exclusion criteria were articles that were not in the English language, not available as free full text, those involving pediatric patients, case reports, and editorials. A total of (47) articles were included in this review. In conclusion, the application of AI in pain management could present promising solutions for pain assessment. AI can potentially increase the accuracy, precision, and efficiency of objective pain assessment.

Application of AI in Multilevel Pain Assessment Using Facial Images: Systematic Review and Meta-Analysis

BACKGROUND

The continuous monitoring and recording of patients' pain status is a major problem in current research on postoperative pain management. In the large number of original or review articles focusing on different approaches for pain assessment, many researchers have investigated how computer vision (CV) can help by capturing facial expressions. However, there is a lack of proper comparison of results between studies to identify current research gaps.

OBJECTIVE

The purpose of this systematic review and meta-analysis was to investigate the diagnostic performance of artificial intelligence models for multilevel pain assessment from facial images.

METHODS

The PubMed, Embase, IEEE, Web of Science, and Cochrane Library databases were searched for related publications before September 30, 2023. Studies that used facial images alone to estimate multiple pain values were included in the systematic review. A study quality assessment was conducted using the Quality Assessment of Diagnostic Accuracy Studies, 2nd edition tool. The performance of these studies was assessed by metrics including sensitivity, specificity, log diagnostic odds ratio (LDOR), and area under the curve (AUC). The intermodal variability was assessed and presented by forest plots.

RESULTS

A total of 45 reports were included in the systematic review. The reported test accuracies ranged from 0.27-0.99, and the other metrics, including the mean standard error (MSE), mean absolute error (MAE), intraclass correlation coefficient (ICC), and Pearson correlation coefficient (PCC), ranged from 0.31-4.61, 0.24-2.8, 0.19-0.83, and 0.48-0.92, respectively. In total, 6 studies were included in the meta-analysis. Their combined sensitivity was 98% (95% CI 96%-99%), specificity was 98% (95% CI 97%-99%), LDOR was 7.99 (95% CI 6.73-9.31), and AUC was 0.99 (95% CI 0.99-1). The subgroup analysis showed that the diagnostic performance was acceptable, although imbalanced data were still emphasized as a major problem. All studies had at least one domain with a high risk of bias, and for 20% (9/45) of studies, there were no applicability concerns.

CONCLUSIONS

This review summarizes recent evidence in automatic multilevel pain estimation from facial expressions and compared the test accuracy of results in a meta-analysis. Promising performance for pain estimation from facial images was established by current CV algorithms. Weaknesses in current studies were also identified, suggesting that larger databases and metrics evaluating multiclass classification performance could improve future studies.

TRIAL REGISTRATION

PROSPERO CRD42023418181; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=418181.

Multimodal automatic assessment of acute pain through facial videos and heart rate signals utilizing transformer-based architectures

Accurate and objective pain evaluation is crucial in developing effective pain management protocols, aiming to alleviate distress and prevent patients from experiencing decreased functionality. A multimodal automatic assessment framework for acute pain utilizing video and heart rate signals is introduced in this study. The proposed framework comprises four pivotal modules: the Spatial Module, responsible for extracting embeddings from videos; the Heart Rate Encoder, tasked with mapping heart rate signals into a higher dimensional space; the AugmNet, designed to create learning-based augmentations in the latent space; and the Temporal Module, which utilizes the extracted video and heart rate embeddings for the final assessment. The Spatial-Module undergoes pre-training on a two-stage strategy: first, with a face recognition objective learning universal facial features, and second, with an emotion recognition objective in a multitask learning approach, enabling the extraction of high-quality embeddings for the automatic pain assessment. Experiments with the facial videos and heart rate extracted from electrocardiograms of the BioVid database, along with a direct comparison to 29 studies, demonstrate state-of-the-art performances in unimodal and multimodal settings, maintaining high efficiency. Within the multimodal context, 82.74% and 39.77% accuracy were achieved for the binary and multi-level pain classification task, respectively, utilizing 9.62 million parameters for the entire framework.

Automated facial recognition system using deep learning for pain assessment in adults with cerebral palsy

Objective

Assessing pain in individuals with neurological conditions like cerebral palsy is challenging due to limited self-reporting and expression abilities. Current methods lack sensitivity and specificity, underlining the need for a reliable evaluation protocol. An automated facial recognition system could revolutionize pain assessment for such patients.The research focuses on two primary goals: developing a dataset of facial pain expressions for individuals with cerebral palsy and creating a deep learning-based automated system for pain assessment tailored to this group.

Methods

The study trained ten neural networks using three pain image databases and a newly curated CP-PAIN Dataset of 109 images from cerebral palsy patients, classified by experts using the Facial Action Coding System.

Results

The InceptionV3 model demonstrated promising results, achieving 62.67% accuracy and a 61.12% F1 score on the CP-PAIN dataset. Explainable AI techniques confirmed the consistency of crucial features for pain identification across models.

Conclusion

The study underscores the potential of deep learning in developing reliable pain detection systems using facial recognition for individuals with communication impairments due to neurological conditions. A more extensive and diverse dataset could further enhance the models' sensitivity to subtle pain expressions in cerebral palsy patients and possibly extend to other complex neurological disorders. This research marks a significant step toward more empathetic and accurate pain management for vulnerable populations.

Human vs machine towards neonatal pain assessment: A comprehensive analysis of the facial features extracted by health professionals, parents, and convolutional neural networks

Neonates are not able to verbally communicate pain, hindering the correct identification of this phenomenon. Several clinical scales have been proposed to assess pain, mainly using the facial features of the neonate, but a better comprehension of these features is yet required, since several related works have shown the subjectivity of these scales. Meanwhile, computational methods have been implemented to automate neonatal pain assessment and, although performing accurately, these methods still lack the interpretability of the corresponding decision-making processes. To address this issue, we propose in this work a facial feature extraction framework to gather information and investigate the human and machine neonatal pain assessments, comparing the visual attention of the facial features perceived by health-professionals and parents of neonates with the most relevant ones extracted by eXplainable Artificial Intelligence (XAI) methods, considering the VGG-Face and N-CNN deep learning architectures. Our experimental results show that the information extracted by the computational methods are clinically relevant to neonatal pain assessment, but yet do not agree with the facial visual attention of health-professionals and parents, suggesting that humans and machines can learn from each other to improve their decision-making processes. We believe that these findings might advance our understanding of how humans and machines code and decode neonatal facial responses to pain, enabling further improvements in clinical scales widely used in practical situations and in face-based automatic pain assessment tools as well.

Face-based automatic pain assessment: challenges and perspectives in neonatal intensive care units

OBJECTIVE

To describe the challenges and perspectives of the automation of pain assessment in the Neonatal Intensive Care Unit.

DATA SOURCES

A search for scientific articles published in the last 10 years on automated neonatal pain assessment was conducted in the main Databases of the Health Area and Engineering Journal Portals, using the descriptors: Pain Measurement, Newborn, Artificial Intelligence, Computer Systems, Software, Automated Facial Recognition.

SUMMARY OF FINDINGS

Fifteen articles were selected and allowed a broad reflection on first, the literature search did not return the various automatic methods that exist to date, and those that exist are not effective enough to replace the human eye; second, computational methods are not yet able to automatically detect pain on partially covered faces and need to be tested during the natural movement of the neonate and with different light intensities; third, for research to advance in this area, databases are needed with more neonatal facial images available for the study of computational methods.

CONCLUSION

There is still a gap between computational methods developed for automated neonatal pain assessment and a practical application that can be used at the bedside in real-time, that is sensitive, specific, and with good accuracy. The studies reviewed described limitations that could be minimized with the development of a tool that identifies pain by analyzing only free facial regions, and the creation and feasibility of a synthetic database of neonatal facial images that is freely available to researchers.

Development of a binary classifier model from extended facial codes toward video-based pain recognition in cancer patients

OBJECTIVES

The Automatic Pain Assessment (APA) relies on the exploitation of objective methods to evaluate the severity of pain and other pain-related characteristics. Facial expressions are the most investigated pain behavior features for APA. We constructed a binary classifier model for discriminating between the absence and presence of pain through video analysis.

METHODS

A brief interview lasting approximately two-minute was conducted with cancer patients, and video recordings were taken during the session. The Delaware Pain Database and UNBC-McMaster Shoulder Pain dataset were used for training. A set of 17 Action Units (AUs) was adopted. For each image, the OpenFace toolkit was used to extract the considered AUs. The collected data were grouped and split into train and test sets: 80 % of the data was used as a training set and the remaining 20 % as the validation set. For continuous estimation, the entire patient video with frame prediction values of 0 (no pain) or 1 (pain), was imported into an annotator (ELAN 6.4). The developed Neural Network classifier consists of two dense layers. The first layer contains 17 nodes associated with the facial AUs extracted by OpenFace for each image. The output layer is a classification label of "pain" (1) or "no pain" (0).

RESULTS

The classifier obtained an accuracy of ∼94 % after about 400 training epochs. The Area Under the ROC curve (AUROC) value was approximately 0.98.

CONCLUSIONS

This study demonstrated that the use of a binary classifier model developed from selected AUs can be an effective tool for evaluating cancer pain. The implementation of an APA classifier can be useful for detecting potential pain fluctuations. In the context of APA research, further investigations are necessary to refine the process and particularly to combine this data with multi-parameter analyses such as speech analysis, text analysis, and data obtained from physiological parameters.

Pose-invariant and occlusion-robust neonatal facial pain assessment

Neonatal Facial Pain Assessment (NFPA) is essential to improve neonatal pain management. Pose variation and occlusion, which can significantly alter the facial appearance, are two major and still unstudied barriers to NFPA. We bridge this gap in terms of method and dataset. Techniques to tackle both challenges in other tasks either expect pose/occlusion-invariant deep learning methods or first generate a normal version of the input image before feature extraction, combining these we argue that it is more effective to jointly perform adversarial learning and end-to-end classification for their mutual benefit. To this end, we propose a Pose-invariant Occlusion-robust Pain Assessment (POPA) framework, with two novelties. We incorporate adversarial learning-based disturbance mitigation for end-to-end pain-level classification and propose a novel composite loss function for facial representation learning; compared to the vanilla discriminator that implicitly determines occlusion and pose conditions, we propose a multi-scale discriminator that determines explicitly, while incorporating local discriminators to enhance the discrimination of key regions. For a comprehensive evaluation, we built the first neonatal pain dataset with disturbance annotation involving 1091 neonates and also applied the proposed POPA to the facial expression recognition task. Extensive qualitative and quantitative experiments prove the superiority of the POPA.

The experience of pain and its ontological modelling from a philosophical point of view: Phenomenological description and ontological revision of the McGill Pain Questionnaire

The aim of the article is to identify, on the basis of the phenomenological and ontological analysis of the experience of pain and the ways in which this experience is expressed in natural language, an ontological modelling of the language of pain and, at the same time, a revision of the traditional version of the McGill questionnaire. The purpose is to provide a different characterisation and an adequate evaluation of the phenomenon of pain, and, consequently, an effective measure of the actual experience of the suffering subject.

A Quantitative Measure of Pain with Current Perception Threshold, Pain Equivalent Current, and Quantified Pain Degree: A Retrospective Study

Background: As a subjective sensation, pain is difficult to evaluate objectively. The assessment of pain degree is largely dependent on subjective methods such as the numeric rating scale (NRS). The PainVisionTM system has recently been introduced as an objective pain degree measurement tool. The purpose of this study was to analyze correlations between the NRS and the current perception threshold (CPT), pain equivalent current (PEC), and quantified pain degree (QPD). Methods: Medical records of 398 subjects who visited the pain clinic in a university hospital from March 2017 to February 2019 were retrospectively reviewed. To evaluate the pain degree, NRS, CPT, PEC, and QPD were measured. Subjects were categorized into two groups: the Pain group (n = 355) and the No-pain group (n = 43). Results: The NRS showed a negative correlation with CPT (R = -0.10, p = 0.054) and a positive correlation with QPD (R = 0.13, p = 0.008). Among various diseases, only spinal disease patients showed a negative correlation between CPT and NRS (R = -0.22, p = 0.003). Additionally, there were significant differences in CPT and QPD between the Pain and No-pain groups (p = 0.005 and p = 0.002, respectively). Conclusions: CPT and QPD measured using the PainVisionTM system could be used to estimate pain intensity and the presence of pain. These parameters would be considered useful for predicting pain itself and its intensity.

A Full Transformer-based Framework for Automatic Pain Estimation using Videos

The automatic estimation of pain is essential in designing an optimal pain management system offering reliable assessment and reducing the suffering of patients. In this study, we present a novel full transformer-based framework consisting of a Transformer in Transformer (TNT) model and a Transformer leveraging cross-attention and self-attention blocks. Elaborating on videos from the BioVid database, we demonstrate state-of-the-art performances, showing the efficacy, efficiency, and generalization capability across all the primary pain estimation tasks.

A Review of Voice-Based Pain Detection in Adults Using Artificial Intelligence

Pain is a complex and subjective experience, and traditional methods of pain assessment can be limited by factors such as self-report bias and observer variability. Voice is frequently used to evaluate pain, occasionally in conjunction with other behaviors such as facial gestures. Compared to facial emotions, there is less available evidence linking pain with voice. This literature review synthesizes the current state of research on the use of voice recognition and voice analysis for pain detection in adults, with a specific focus on the role of artificial intelligence (AI) and machine learning (ML) techniques. We describe the previous works on pain recognition using voice and highlight the different approaches to voice as a tool for pain detection, such as a human effect or biosignal. Overall, studies have shown that AI-based voice analysis can be an effective tool for pain detection in adult patients with various types of pain, including chronic and acute pain. We highlight the high accuracy of the ML-based approaches used in studies and their limitations in terms of generalizability due to factors such as the nature of the pain and patient population characteristics. However, there are still potential challenges, such as the need for large datasets and the risk of bias in training models, which warrant further research.