1
|
Narimisaei J, Naeim M, Imannezhad S, Samian P, Sobhani M. Exploring emotional intelligence in artificial intelligence systems: a comprehensive analysis of emotion recognition and response mechanisms. Ann Med Surg (Lond) 2024; 86:4657-4663. [PMID: 39118764 PMCID: PMC11305735 DOI: 10.1097/ms9.0000000000002315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2024] [Accepted: 06/17/2024] [Indexed: 08/10/2024] Open
Abstract
This study aims to dissect the current state of emotion recognition and response mechanisms in artificial intelligence (AI) systems, exploring the progress made, challenges faced, and implicit operations of integrating emotional intelligence into AI. This study utilized a comprehensive review approach to investigate the integration of emotional intelligence (EI) into artificial intelligence (AI) systems, concentrating on emotion recognition and response mechanisms. The review process entailed formulating research questions, systematically searching academic databases such as PubMed, Scopus, and Web of Science, critically evaluating relevant literature, synthesizing the data, and presenting the findings in a comprehensive format. The study highlights the advancements in emotion recognition models, including the use of deep literacy ways and multimodal data emulsion. It discusses the challenges in emotion recognition, similar to variability in mortal expressions and the need for real-time processing. The integration of contextual information and individual traits is emphasized as enhancing the understanding of mortal feelings. The study also addresses ethical enterprises, similar as sequestration and impulses in training data. The integration of emotional intelligence into AI systems presents openings to revise mortal-computer relations. Emotion recognition and response mechanisms have made significant progress, but challenges remain. Unborn exploration directions include enhancing the robustness and interpretability of emotion recognition models, exploring cross-cultural and environment-apprehensive emotion understanding, and addressing long-term emotion shadowing and adaption. By further exploring emotional intelligence in AI systems, further compassionate and responsive machines can be developed, enabling deeper emotional connections with humans.
Collapse
Affiliation(s)
- Jale Narimisaei
- Department of Computer, Energy and Data Science Faculty, Behbahan Khatam Alanbia University of Technology, Behbahan
| | - Mahdi Naeim
- Department of Research, Psychology and Counseling Organization, Tehran
| | - Shima Imannezhad
- Department of Pediatrics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad
| | - Pooya Samian
- Department of Educational Sciences, Faculty of Educational Sciences and Psychology, Shahid Madani University of Azerbaijan, Tabriz, Iran
| | | |
Collapse
|
2
|
Moscato S, Orlandi S, Di Gregorio F, Lullini G, Pozzi S, Sabattini L, Chiari L, La Porta F. Feasibility interventional study investigating PAIN in neurorehabilitation through wearabLE SensorS (PAINLESS): a study protocol. BMJ Open 2023; 13:e073534. [PMID: 37993169 PMCID: PMC10668325 DOI: 10.1136/bmjopen-2023-073534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 07/28/2023] [Indexed: 11/24/2023] Open
Abstract
INTRODUCTION Millions of people survive injuries to the central or peripheral nervous system for which neurorehabilitation is required. In addition to the physical and cognitive impairments, many neurorehabilitation patients experience pain, often not widely recognised and inadequately treated. This is particularly true for multiple sclerosis (MS) patients, for whom pain is one of the most common symptoms. In clinical practice, pain assessment is usually conducted based on a subjective estimate. This approach can lead to inaccurate evaluations due to the influence of numerous factors, including emotional or cognitive aspects. To date, no objective and simple to use clinical methods allow objective quantification of pain and the diagnostic differentiation between the two main types of pain (nociceptive vs neuropathic). Wearable technologies and artificial intelligence (AI) have the potential to bridge this gap by continuously monitoring patients' health parameters and extracting meaningful information from them. Therefore, we propose to develop a new automatic AI-powered tool to assess pain and its characteristics during neurorehabilitation treatments using physiological signals collected by wearable sensors. METHODS AND ANALYSIS We aim to recruit 15 participants suffering from MS undergoing physiotherapy treatment. During the study, participants will wear a wristband for three consecutive days and be monitored before and after their physiotherapy sessions. Measurement of traditionally used pain assessment questionnaires and scales (ie, painDETECT, Doleur Neuropathique 4 Questions, EuroQoL-5-dimension-3-level) and physiological signals (photoplethysmography, electrodermal activity, skin temperature, accelerometer data) will be collected. Relevant parameters from physiological signals will be identified, and AI algorithms will be used to develop automatic classification methods. ETHICS AND DISSEMINATION The study has been approved by the local Ethical Committee (285-2022-SPER-AUSLBO). Participants are required to provide written informed consent. The results will be disseminated through contributions to international conferences and scientific journals, and they will also be included in a doctoral dissertation. TRIAL REGISTRATION NUMBER NCT05747040.
Collapse
Affiliation(s)
- Serena Moscato
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, Alma Mater Studiorum University of Bologna, Bologna, Italy
| | - Silvia Orlandi
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, Alma Mater Studiorum University of Bologna, Bologna, Italy
- Health Science and Technologies - Interdepartmental Center for Industrial Research (CIRI-SDV), Alma Mater Studiorum University of Bologna, Bologna, Italy
| | - Francesco Di Gregorio
- UOC Medicina Riabilitativa e Neuroriabilitazione, Azienda Unità Sanitaria Locale di Bologna, Bologna, Italy
- Centro studi e ricerche in Neuroscienze Cognitive, Dipartimento di Psicologia, Alma Mater Studiorum - Università di Bologna, Cesena, Italy
| | - Giada Lullini
- IRCCS Istituto delle Scienze Neurologuche di Bologna, Bologna, Italy
| | - Stefania Pozzi
- DATER Riabilitazione Ospedaliera, UA Riabilitazione, Azienda Unità Sanitaria Locale di Bologna, Bologna, Italy
| | | | - Lorenzo Chiari
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, Alma Mater Studiorum University of Bologna, Bologna, Italy
- Health Science and Technologies - Interdepartmental Center for Industrial Research (CIRI-SDV), Alma Mater Studiorum University of Bologna, Bologna, Italy
| | - Fabio La Porta
- IRCCS Istituto delle Scienze Neurologuche di Bologna, Bologna, Italy
| |
Collapse
|
3
|
Alhejaili R, Alomainy A. The Use of Wearable Technology in Providing Assistive Solutions for Mental Well-Being. SENSORS (BASEL, SWITZERLAND) 2023; 23:7378. [PMID: 37687834 PMCID: PMC10490605 DOI: 10.3390/s23177378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Revised: 08/08/2023] [Accepted: 08/11/2023] [Indexed: 09/10/2023]
Abstract
The main goal of this manuscript is to provide an extensive literature review and analysis of certain biomarkers, which are frequently used to identify stress, anxiety, and other emotions, leading to potential solutions for the monitoring of mental wellness using wearable technologies. It is possible to see the impacts of several biomarkers in detecting stress levels and their effectiveness with an investigation into the literature on this subject. Biofeedback training has demonstrated some psychological effects, such as a reduction in anxiety and self-control enhancement. This survey demonstrates backed up by evidence that wearable devices are assistive in providing health and mental wellness solutions. Because physical activity tracing would reduce the stress stressors, which affect the subject's body, therefore, it would also affect the mental activity and would lead to a reduction in cognitive mental load.
Collapse
Affiliation(s)
- Reham Alhejaili
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
- Department of Computer Science and Artificial Intelligence, College of Computer Science and Engineering, University of Jeddah, Jeddah 23218, Saudi Arabia
| | - Akram Alomainy
- Antennas and Electromagnetics Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK;
| |
Collapse
|
4
|
Aminosharieh Najafi T, Affanni A, Rinaldo R, Zontone P. Drivers' Mental Engagement Analysis Using Multi-Sensor Fusion Approaches Based on Deep Convolutional Neural Networks. SENSORS (BASEL, SWITZERLAND) 2023; 23:7346. [PMID: 37687801 PMCID: PMC10490517 DOI: 10.3390/s23177346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 08/15/2023] [Accepted: 08/16/2023] [Indexed: 09/10/2023]
Abstract
In this paper, we present a comprehensive assessment of individuals' mental engagement states during manual and autonomous driving scenarios using a driving simulator. Our study employed two sensor fusion approaches, combining the data and features of multimodal signals. Participants in our experiment were equipped with Electroencephalogram (EEG), Skin Potential Response (SPR), and Electrocardiogram (ECG) sensors, allowing us to collect their corresponding physiological signals. To facilitate the real-time recording and synchronization of these signals, we developed a custom-designed Graphical User Interface (GUI). The recorded signals were pre-processed to eliminate noise and artifacts. Subsequently, the cleaned data were segmented into 3 s windows and labeled according to the drivers' high or low mental engagement states during manual and autonomous driving. To implement sensor fusion approaches, we utilized two different architectures based on deep Convolutional Neural Networks (ConvNets), specifically utilizing the Braindecode Deep4 ConvNet model. The first architecture consisted of four convolutional layers followed by a dense layer. This model processed the synchronized experimental data as a 2D array input. We also proposed a novel second architecture comprising three branches of the same ConvNet model, each with four convolutional layers, followed by a concatenation layer for integrating the ConvNet branches, and finally, two dense layers. This model received the experimental data from each sensor as a separate 2D array input for each ConvNet branch. Both architectures were evaluated using a Leave-One-Subject-Out (LOSO) cross-validation approach. For both cases, we compared the results obtained when using only EEG signals with the results obtained by adding SPR and ECG signals. In particular, the second fusion approach, using all sensor signals, achieved the highest accuracy score, reaching 82.0%. This outcome demonstrates that our proposed architecture, particularly when integrating EEG, SPR, and ECG signals at the feature level, can effectively discern the mental engagement of drivers.
Collapse
Affiliation(s)
- Taraneh Aminosharieh Najafi
- Polytechnic Department of Engineering and Architecture, University of Udine, Via Delle Scienze 206, 33100 Udine, Italy; (A.A.); (R.R.); (P.Z.)
| | | | | | | |
Collapse
|
5
|
Baldassarri S, García de Quirós J, Beltrán JR, Álvarez P. Wearables and Machine Learning for Improving Runners' Motivation from an Affective Perspective. SENSORS (BASEL, SWITZERLAND) 2023; 23:1608. [PMID: 36772647 PMCID: PMC9920630 DOI: 10.3390/s23031608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Revised: 01/17/2023] [Accepted: 01/30/2023] [Indexed: 06/18/2023]
Abstract
Wearable technology is playing an increasing role in the development of user-centric applications. In the field of sports, this technology is being used to implement solutions that improve athletes' performance, reduce the risk of injury, or control fatigue, for example. Emotions are involved in most of these solutions, but unfortunately, they are not monitored in real-time or used as a decision element that helps to increase the quality of training sessions, nor are they used to guarantee the health of athletes. In this paper, we present a wearable and a set of machine learning models that are able to deduce runners' emotions during their training. The solution is based on the analysis of runners' electrodermal activity, a physiological parameter widely used in the field of emotion recognition. As part of the DJ-Running project, we have used these emotions to increase runners' motivation through music. It has required integrating the wearable and the models into the DJ-Running mobile application, which interacts with the technological infrastructure of the project to select and play the most suitable songs at each instant of the training.
Collapse
Affiliation(s)
- Sandra Baldassarri
- Computer Science and Systems Engineering Department, Engineering Research Institute of Aragon (I3A), University of Zaragoza, 50018 Zaragoza, Spain
| | - Jorge García de Quirós
- Computer Science and Systems Engineering Department, Engineering Research Institute of Aragon (I3A), University of Zaragoza, 50018 Zaragoza, Spain
| | - José Ramón Beltrán
- Electronic Engineering and Communications Department, Engineering Research Institute of Aragon (I3A), University of Zaragoza, 50009 Zaragoza, Spain
| | - Pedro Álvarez
- Computer Science and Systems Engineering Department, Engineering Research Institute of Aragon (I3A), University of Zaragoza, 50018 Zaragoza, Spain
| |
Collapse
|
6
|
Machine Learning Algorithms for Detection and Classifications of Emotions in Contact Center Applications. SENSORS 2022; 22:s22145311. [PMID: 35890994 PMCID: PMC9321989 DOI: 10.3390/s22145311] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 06/27/2022] [Accepted: 07/13/2022] [Indexed: 12/04/2022]
Abstract
Over the past few years, virtual assistant solutions used in Contact Center systems are gaining popularity. One of the main tasks of the virtual assistant is to recognize the intentions of the customer. It is important to note that quite often the actual intention expressed in a conversation is also directly influenced by the emotions that accompany that conversation. Unfortunately, scientific literature has not identified what specific types of emotions in Contact Center applications are relevant to the activities they perform. Therefore, the main objective of this work was to develop an Emotion Classification for Machine Detection of Affect-Tinged Conversational Contents dedicated directly to the Contact Center industry. In the conducted study, Contact Center voice and text channels were considered, taking into account the following families of emotions: anger, fear, happiness, sadness vs. affective neutrality of the statements. The obtained results confirmed the usefulness of the proposed classification—for the voice channel, the highest efficiency was obtained using the Convolutional Neural Network (accuracy, 67.5%; precision, 80.3; F1-Score, 74.5%), while for the text channel, the Support Vector Machine algorithm proved to be the most efficient (accuracy, 65.9%; precision, 58.5; F1-Score, 61.7%).
Collapse
|
7
|
Bustos-López M, Cruz-Ramírez N, Guerra-Hernández A, Sánchez-Morales LN, Cruz-Ramos NA, Alor-Hernández G. Wearables for Engagement Detection in Learning Environments: A Review. BIOSENSORS 2022; 12:509. [PMID: 35884312 PMCID: PMC9312492 DOI: 10.3390/bios12070509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 07/03/2022] [Accepted: 07/05/2022] [Indexed: 11/16/2022]
Abstract
Appropriate teaching-learning strategies lead to student engagement during learning activities. Scientific progress and modern technology have made it possible to measure engagement in educational settings by reading and analyzing student physiological signals through sensors attached to wearables. This work is a review of current student engagement detection initiatives in the educational domain. The review highlights existing commercial and non-commercial wearables for student engagement monitoring and identifies key physiological signals involved in engagement detection. Our findings reveal that common physiological signals used to measure student engagement include heart rate, skin temperature, respiratory rate, oxygen saturation, blood pressure, and electrocardiogram (ECG) data. Similarly, stress and surprise are key features of student engagement.
Collapse
Affiliation(s)
- Maritza Bustos-López
- Instituto de Investigaciones en Inteligencia Artificial, Universidad Veracruzana, Xalapa, Veracruz 91097, Mexico; (M.B.-L.); (N.C.-R.); (A.G.-H.)
| | - Nicandro Cruz-Ramírez
- Instituto de Investigaciones en Inteligencia Artificial, Universidad Veracruzana, Xalapa, Veracruz 91097, Mexico; (M.B.-L.); (N.C.-R.); (A.G.-H.)
| | - Alejandro Guerra-Hernández
- Instituto de Investigaciones en Inteligencia Artificial, Universidad Veracruzana, Xalapa, Veracruz 91097, Mexico; (M.B.-L.); (N.C.-R.); (A.G.-H.)
| | - Laura Nely Sánchez-Morales
- Division of Research and Postgraduate Studies, CONACYT-Tecnológico Nacional de México/I. T. Orizaba, Av. Oriente 9 852 Col. Emiliano Zapata, Orizaba, Veracruz 94320, Mexico;
| | - Nancy Aracely Cruz-Ramos
- Division of Research and Postgraduate Studies, Tecnológico Nacional de México/I. T. Orizaba, Av. Oriente 9 852 Col. Emiliano Zapata, Orizaba, Veracruz 94320, Mexico;
| | - Giner Alor-Hernández
- Division of Research and Postgraduate Studies, Tecnológico Nacional de México/I. T. Orizaba, Av. Oriente 9 852 Col. Emiliano Zapata, Orizaba, Veracruz 94320, Mexico;
| |
Collapse
|
8
|
Li R, Yuizono T, Li X. Affective computing of multi-type urban public spaces to analyze emotional quality using ensemble learning-based classification of multi-sensor data. PLoS One 2022; 17:e0269176. [PMID: 35657805 PMCID: PMC9165821 DOI: 10.1371/journal.pone.0269176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Accepted: 05/15/2022] [Indexed: 11/18/2022] Open
Abstract
The quality of urban public spaces affects the emotional response of users; therefore, the emotional data of users can be used as indices to evaluate the quality of a space. Emotional response can be evaluated to effectively measure public space quality through affective computing and obtain evidence-based support for urban space renewal. We proposed a feasible evaluation method for multi-type urban public spaces based on multiple physiological signals and ensemble learning. We built binary, ternary, and quinary classification models based on participants’ physiological signals and self-reported emotional responses through experiments in eight public spaces of five types. Furthermore, we verified the effectiveness of the model by inputting data collected from two other public spaces. Three observations were made based on the results. First, the highest accuracies of the binary and ternary classification models were 92.59% and 91.07%, respectively. After external validation, the highest accuracies were 80.90% and 65.30%, respectively, which satisfied the preliminary requirements for evaluating the quality of actual urban spaces. However, the quinary classification model could not satisfy the preliminary requirements. Second, the average accuracy of ensemble learning was 7.59% higher than that of single classifiers. Third, reducing the number of physiological signal features and applying the synthetic minority oversampling technique to solve unbalanced data improved the evaluation ability.
Collapse
Affiliation(s)
- Ruixuan Li
- School of Art and Design, Dalian Polytechnic University, Dalian City, Liaoning Province, China
- Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, Japan
- * E-mail:
| | - Takaya Yuizono
- Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, Japan
| | - Xianghui Li
- School of Art and Design, Dalian Polytechnic University, Dalian City, Liaoning Province, China
- Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, Japan
| |
Collapse
|
9
|
Moscato S, Cortelli P, Chiari L. Physiological responses to pain in cancer patients: A systematic review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 217:106682. [PMID: 35172252 DOI: 10.1016/j.cmpb.2022.106682] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 01/23/2022] [Accepted: 02/04/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Pain is one of the most debilitating symptoms in persons with cancer. Still, its assessment is often neglected both by patients and healthcare professionals. There is increasing interest in conducting pain assessment and monitoring via physiological signals that promise to overcome the limitations of state-of-the-art pain assessment tools. This systematic review aims to evaluate existing experimental studies to identify the most promising methods and results for objectively quantifying cancer patients' pain experience. METHODS Four electronic databases (Pubmed, Compendex, Scopus, Web of Science) were systematically searched for articles published up to October 2020. RESULTS Fourteen studies (528 participants) were included in the review. The selected studies analyzed seven physiological signals. Blood pressure and ECG were the most used signals. Sixteen physiological parameters showed significant changes in association with pain. The studies were fairly consistent in stating that heart rate, the low-frequency to high-frequency component ratio (LF/HF), and systolic blood pressure positively correlate with the pain. CONCLUSIONS Current evidence supports the hypothesis that physiological signals can help objectively quantify, at least in part, cancer patients' pain experience. While there is much more to be done to obtain a reliable pain assessment method, this review takes an essential first step by highlighting issues that should be taken into account in future research: use of a wearable device for pervasive recording in a real-world context, implementation of a big-data approach possibly supported by AI, including multiple stratification factors (e.g., cancer site and stage, source of pain, demographic and psychosocial data), and better-defined recording procedures. Improved methods and algorithms could then become valuable add-ons in taking charge of cancer patients.
Collapse
Affiliation(s)
- Serena Moscato
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, University of Bologna, Bologna, Italy.
| | - Pietro Cortelli
- IRCCS Istituto Delle Scienze Neurologiche Di Bologna, UOC Clinica Neurologica NeuroMet, Ospedale Bellaria, Bologna, Italy; Department of Biomedical and Neuromotor Sciences (DIBINEM), University of Bologna, Bologna, Italy
| | - Lorenzo Chiari
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, University of Bologna, Bologna, Italy; Health Sciences and Technologies, Interdepartmental Center for Industrial Research (CIRI-SDV), University of Bologna, Bologna, Italy
| |
Collapse
|
10
|
Wan C, Chen D, Huang Z, Luo X. A Wearable Head Mounted Display Bio-Signals Pad System for Emotion Recognition. SENSORS (BASEL, SWITZERLAND) 2021; 22:142. [PMID: 35009684 PMCID: PMC8749721 DOI: 10.3390/s22010142] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/22/2021] [Revised: 12/14/2021] [Accepted: 12/22/2021] [Indexed: 06/14/2023]
Abstract
Multimodal bio-signals acquisition based on wearable devices and using virtual reality (VR) as stimulus source are promising techniques in emotion recognition research field. Numerous studies have shown that emotional states can be better evoked through Immersive Virtual Environments (IVE). The main goal of this paper is to provide researchers with a system for emotion recognition in VR environments. In this paper, we present a wearable forehead bio-signals acquisition pad which is attached to Head-Mounted Displays (HMD), termed HMD Bio Pad. This system can simultaneously record emotion-related two-channel electroencephalography (EEG), one-channel electrodermal activity (EDA), photoplethysmograph (PPG) and skin temperature (SKT) signals. In addition, we develop a human-computer interaction (HCI) interface which researchers can carry out emotion recognition research using VR HMD as stimulus presentation device. To evaluate the performance of the proposed system, we conducted different experiments to validate the multimodal bio-signals quality, respectively. To validate EEG signal, we have assessed the performance in terms of EEG eyes-blink task and eyes-open and eyes-closed task. The EEG eyes-blink task indicates that the proposed system can achieve comparable EEG signal quality in comparison to the dedicated bio-signals measuring device. The eyes-open and eyes-closed task proves that the proposed system can efficiently record alpha rhythm. Then we used signal-to-noise ratio (SNR) and Skin Conductance Reaction (SCR) signal to validate the performance for EDA acquisition system. A filtered EDA signal, with a high mean SNR of 28.52 dB, is plotted on HCI interface. Moreover, the SCR signal related to stimulus response can be correctly extracted from EDA signal. The SKT acquisition system has been validated effectively by the temperature change experiment when subjects are in unpleasant emotion. The pulse rate (PR) estimated from PPG signal achieved the low mean average absolute error (AAE), which is 1.12 beats per minute (BPM) over 8 recordings. In summary, the proposed HMD Bio Pad offers a portable, comfortable and easy-to-wear device for recording bio-signals. The proposed system could contribute to emotion recognition research in VR environments.
Collapse
Affiliation(s)
- Chunting Wan
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; (C.W.); (Z.H.); (X.L.)
- School of Electronic Engineering and Automation, Guilin University of Electronic Science and Technology, Guilin 541004, China
| | - Dongyi Chen
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; (C.W.); (Z.H.); (X.L.)
| | - Zhiqi Huang
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; (C.W.); (Z.H.); (X.L.)
| | - Xi Luo
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; (C.W.); (Z.H.); (X.L.)
| |
Collapse
|
11
|
Sheikh M, Qassem M, Kyriacou PA. Wearable, Environmental, and Smartphone-Based Passive Sensing for Mental Health Monitoring. Front Digit Health 2021; 3:662811. [PMID: 34713137 PMCID: PMC8521964 DOI: 10.3389/fdgth.2021.662811] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Accepted: 03/02/2021] [Indexed: 12/21/2022] Open
Abstract
Collecting and analyzing data from sensors embedded in the context of daily life has been widely employed for the monitoring of mental health. Variations in parameters such as movement, sleep duration, heart rate, electrocardiogram, skin temperature, etc., are often associated with psychiatric disorders. Namely, accelerometer data, microphone, and call logs can be utilized to identify voice features and social activities indicative of depressive symptoms, and physiological factors such as heart rate and skin conductance can be used to detect stress and anxiety disorders. Therefore, a wide range of devices comprising a variety of sensors have been developed to capture these physiological and behavioral data and translate them into phenotypes and states related to mental health. Such systems aim to identify behaviors that are the consequence of an underlying physiological alteration, and hence, the raw sensor data are captured and converted into features that are used to define behavioral markers, often through machine learning. However, due to the complexity of passive data, these relationships are not simple and need to be well-established. Furthermore, parameters such as intrapersonal and interpersonal differences need to be considered when interpreting the data. Altogether, combining practical mobile and wearable systems with the right data analysis algorithms can provide a useful tool for the monitoring and management of mental disorders. The current review aims to comprehensively present and critically discuss all available smartphone-based, wearable, and environmental sensors for detecting such parameters in relation to the treatment and/or management of the most common mental health conditions.
Collapse
Affiliation(s)
- Mahsa Sheikh
- Research Centre for Biomedical Engineering, School of Mathematics, Computer Science & Engineering, City, University of London, London, United Kingdom
| | - M Qassem
- Research Centre for Biomedical Engineering, School of Mathematics, Computer Science & Engineering, City, University of London, London, United Kingdom
| | - Panicos A Kyriacou
- Research Centre for Biomedical Engineering, School of Mathematics, Computer Science & Engineering, City, University of London, London, United Kingdom
| |
Collapse
|
12
|
Moscato S, Sichi V, Giannelli A, Palumbo P, Ostan R, Varani S, Pannuti R, Chiari L. Virtual Reality in Home Palliative Care: Brief Report on the Effect on Cancer-Related Symptomatology. Front Psychol 2021; 12:709154. [PMID: 34630217 PMCID: PMC8497744 DOI: 10.3389/fpsyg.2021.709154] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2021] [Accepted: 08/26/2021] [Indexed: 12/12/2022] Open
Abstract
Virtual reality (VR) has been used as a complementary therapy for managing psychological and physical symptoms in cancer patients. In palliative care, the evidence about the use of VR is still inadequate. This study aims to assess the effect of an immersive VR-based intervention conducted at home on anxiety, depression, and pain over 4days and to evaluate the short-term effect of VR sessions on cancer-related symptomatology. Participants were advanced cancer patients assisted at home who were provided with a VR headset for 4days. On days one and four, anxiety and depression were measured by the Hospital Anxiety and Depression Scale (HADS) and pain by the Brief Pain Inventory (BPI). Before and after each VR session, symptoms were collected by the Edmonton Symptom Assessment Scale (ESAS). Participants wore a smart wristband measuring physiological signals associated with pain, anxiety, and depression. Fourteen patients (mean age 47.2±14.2years) were recruited. Anxiety, depression (HADS), and pain (BPI) did not change significantly between days one and four. However, the ESAS items related to pain, depression, anxiety, well-being, and shortness of breath collected immediately after the VR sessions showed a significant improvement (p<0.01). A progressive reduction in electrodermal activity has been observed comparing the recordings before, during, and after the VR sessions, although these changes were not statistically significant. This brief research report supports the idea that VR could represent a suitable complementary tool for psychological treatment in advanced cancer patients assisted at home.
Collapse
Affiliation(s)
- Serena Moscato
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, University of Bologna, Bologna, Italy
| | - Vittoria Sichi
- National Tumor Assistance (ANT) Foundation, Bologna, Italy
| | | | - Pierpaolo Palumbo
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, University of Bologna, Bologna, Italy
| | - Rita Ostan
- National Tumor Assistance (ANT) Foundation, Bologna, Italy
| | - Silvia Varani
- National Tumor Assistance (ANT) Foundation, Bologna, Italy
| | | | - Lorenzo Chiari
- Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi" - DEI, University of Bologna, Bologna, Italy
- Health Sciences and Technologies - Interdepartmental Center for Industrial Research (CIRI-SDV), University of Bologna, Bologna, Italy
| |
Collapse
|
13
|
Pal S, Mukhopadhyay S, Suryadevara N. Development and Progress in Sensors and Technologies for Human Emotion Recognition. SENSORS 2021; 21:s21165554. [PMID: 34451002 PMCID: PMC8402266 DOI: 10.3390/s21165554] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/19/2021] [Revised: 08/08/2021] [Accepted: 08/13/2021] [Indexed: 11/25/2022]
Abstract
With the advancement of human-computer interaction, robotics, and especially humanoid robots, there is an increasing trend for human-to-human communications over online platforms (e.g., zoom). This has become more significant in recent years due to the Covid-19 pandemic situation. The increased use of online platforms for communication signifies the need to build efficient and more interactive human emotion recognition systems. In a human emotion recognition system, the physiological signals of human beings are collected, analyzed, and processed with the help of dedicated learning techniques and algorithms. With the proliferation of emerging technologies, e.g., the Internet of Things (IoT), future Internet, and artificial intelligence, there is a high demand for building scalable, robust, efficient, and trustworthy human recognition systems. In this paper, we present the development and progress in sensors and technologies to detect human emotions. We review the state-of-the-art sensors used for human emotion recognition and different types of activity monitoring. We present the design challenges and provide practical references of such human emotion recognition systems in the real world. Finally, we discuss the current trends in applications and explore the future research directions to address issues, e.g., scalability, security, trust, privacy, transparency, and decentralization.
Collapse
Affiliation(s)
- Shantanu Pal
- School of Computer Science, Faculty of Science, Queensland University of Technology, Brisbane, QLD 4000, Australia;
| | - Subhas Mukhopadhyay
- School of Engineering, Faculty of Science and Engineering, Macquarie University, Sydney, NSW 2109, Australia
- Correspondence: ; Tel.: +61-2-9850-6510
| | - Nagender Suryadevara
- School of Computer and Information Sciences, University of Hyderabad, Hyderabad, Telangana 500046, India;
| |
Collapse
|
14
|
Vavrinsky E, Stopjakova V, Kopani M, Kosnacova H. The Concept of Advanced Multi-Sensor Monitoring of Human Stress. SENSORS (BASEL, SWITZERLAND) 2021; 21:3499. [PMID: 34067895 PMCID: PMC8157129 DOI: 10.3390/s21103499] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Revised: 05/10/2021] [Accepted: 05/12/2021] [Indexed: 12/23/2022]
Abstract
Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.
Collapse
Affiliation(s)
- Erik Vavrinsky
- Institute of Electronics and Photonics, Faculty of Electrical Engineering and Information Technology, Slovak University of Technology, Ilkovicova 3, 81219 Bratislava, Slovakia;
- Institute of Medical Physics, Biophysics, Informatics and Telemedicine, Faculty of Medicine, Comenius University, Sasinkova 2, 81272 Bratislava, Slovakia;
| | - Viera Stopjakova
- Institute of Electronics and Photonics, Faculty of Electrical Engineering and Information Technology, Slovak University of Technology, Ilkovicova 3, 81219 Bratislava, Slovakia;
| | - Martin Kopani
- Institute of Medical Physics, Biophysics, Informatics and Telemedicine, Faculty of Medicine, Comenius University, Sasinkova 2, 81272 Bratislava, Slovakia;
| | - Helena Kosnacova
- Department of Simulation and Virtual Medical Education, Faculty of Medicine, Comenius University, Sasinkova 4, 81272 Bratislava, Slovakia
- Department of Molecular Oncology, Cancer Research Institute, Biomedical Research Center of the Slovak Academy of Sciences, Dúbravská Cesta 9, 84505 Bratislava, Slovakia
| |
Collapse
|
15
|
Costadopoulos N, Islam MZ, Tien D. A knowledge discovery and visualisation method for unearthing emotional states from physiological data. INT J MACH LEARN CYB 2020. [DOI: 10.1007/s13042-020-01205-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
16
|
Yang N, Dey N, Sherratt RS, Shi F. Recognize basic emotional statesin speech by machine learning techniques using mel-frequency cepstral coefficient features. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2020. [DOI: 10.3233/jifs-179963] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Speech Emotion Recognition (SER) has been widely used in many fields, such as smart home assistants commonly found in the market. Smart home assistants that could detect the user’s emotion would improve the communication between a user and the assistant enabling the assistant to offer more productive feedback. Thus, the aim of this work is to analyze emotional states in speech and propose a suitable algorithm considering performance verses complexity for deployment in smart home devices. The four emotional speech sets were selected from the Berlin Emotional Database (EMO-DB) as experimental data, 26 MFCC features were extracted from each type of emotional speech to identify the emotions of happiness, anger, sadness and neutrality. Then, speaker-independent experiments for our Speech emotion Recognition (SER) were conducted by using the Back Propagation Neural Network (BPNN), Extreme Learning Machine (ELM), Probabilistic Neural Network (PNN) and Support Vector Machine (SVM). Synthesizing the recognition accuracy and processing time, this work shows that the performance of SVM was the best among the four methods as a good candidate to be deployed for SER in smart home devices. SVM achieved an overall accuracy of 92.4% while offering low computational requirements when training and testing. We conclude that the MFCC features and the SVM classification models used in speaker-independent experiments are highly effective in the automatic prediction of emotion.
Collapse
Affiliation(s)
- Ningning Yang
- First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Nilanjan Dey
- Department of Information Technology, Techno India College of Technology, West Bengal, India
| | | | - Fuqian Shi
- First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
17
|
Zamkah A, Hui T, Andrews S, Dey N, Shi F, Sherratt RS. Identification of Suitable Biomarkers for Stress and Emotion Detection for Future Personal Affective Wearable Sensors. BIOSENSORS-BASEL 2020; 10:bios10040040. [PMID: 32316280 PMCID: PMC7235866 DOI: 10.3390/bios10040040] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Revised: 04/10/2020] [Accepted: 04/13/2020] [Indexed: 02/07/2023]
Abstract
Skin conductivity (i.e., sweat) forms the basis of many physiology-based emotion and stress detection systems. However, such systems typically do not detect the biomarkers present in sweat, and thus do not take advantage of the biological information in the sweat. Likewise, such systems do not detect the volatile organic components (VOC’s) created under stressful conditions. This work presents a review into the current status of human emotional stress biomarkers and proposes the major potential biomarkers for future wearable sensors in affective systems. Emotional stress has been classified as a major contributor in several social problems, related to crime, health, the economy, and indeed quality of life. While blood cortisol tests, electroencephalography and physiological parameter methods are the gold standards for measuring stress; however, they are typically invasive or inconvenient and not suitable for wearable real-time stress monitoring. Alternatively, cortisol in biofluids and VOCs emitted from the skin appear to be practical and useful markers for sensors to detect emotional stress events. This work has identified antistress hormones and cortisol metabolites as the primary stress biomarkers that can be used in future sensors for wearable affective systems.
Collapse
Affiliation(s)
- Abdulaziz Zamkah
- Biomedical Sciences and Biomedical Engineering, The University of Reading, Reading RG6 6AY, UK; (A.Z.); (T.H.); (S.A.)
| | - Terence Hui
- Biomedical Sciences and Biomedical Engineering, The University of Reading, Reading RG6 6AY, UK; (A.Z.); (T.H.); (S.A.)
| | - Simon Andrews
- Biomedical Sciences and Biomedical Engineering, The University of Reading, Reading RG6 6AY, UK; (A.Z.); (T.H.); (S.A.)
| | - Nilanjan Dey
- Department of Information Technology, Techno India College of Technology, West Bengal 700156, India;
| | - Fuqian Shi
- Rutgers Cancer Institute of New Jersey, Rutgers University, New Brunswick, NJ 08903, USA;
| | - R. Simon Sherratt
- Biomedical Sciences and Biomedical Engineering, The University of Reading, Reading RG6 6AY, UK; (A.Z.); (T.H.); (S.A.)
- Correspondence: ; Tel.: +44-118-378-8588
| |
Collapse
|
18
|
Baldassarre A, Mucci N, Lecca LI, Tomasini E, Parcias-do-Rosario MJ, Pereira CT, Arcangeli G, Oliveira PAB. Biosensors in Occupational Safety and Health Management: A Narrative Review. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:E2461. [PMID: 32260295 PMCID: PMC7177223 DOI: 10.3390/ijerph17072461] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Revised: 03/31/2020] [Accepted: 04/02/2020] [Indexed: 12/20/2022]
Abstract
A sensor is a device used to gather information registered by some biological, physical or chemical change, and then convert the information into a measurable signal. The first biosensor prototype was conceived more than a century ago, in 1906, but a properly defined biosensor was only developed later in 1956. Some of them have reached the commercial stage and are routinely used in environmental and agricultural applications, and especially, in clinical laboratory and industrial analysis, mostly because it is an economical, simple and efficient instrument for the in situ detection of the bioavailability of a broad range of environmental pollutants. We propose a narrative review, that found 32 papers and aims to discuss the possible uses of biosensors, focusing on their use in the area of occupational safety and health (OSH).
Collapse
Affiliation(s)
- Antonio Baldassarre
- Department of Experimental and Clinical Medicine, University of Florence, 50134 Florence, Italy
| | - Nicola Mucci
- Department of Experimental and Clinical Medicine, University of Florence, 50134 Florence, Italy
| | - Luigi Isaia Lecca
- Department of Experimental and Clinical Medicine, University of Florence, 50134 Florence, Italy
| | - Emanuela Tomasini
- Department of Experimental and Clinical Medicine, University of Florence, 50134 Florence, Italy
| | | | - Carolina Tauil Pereira
- Hospital De Clinicas, Serviço de Medicina Ocupacional, Porto Alegre 90035-007, Rio Grande do Sul, Brazil
| | - Giulio Arcangeli
- Department of Experimental and Clinical Medicine, University of Florence, 50134 Florence, Italy
| | | |
Collapse
|
19
|
Dzedzickis A, Kaklauskas A, Bucinskas V. Human Emotion Recognition: Review of Sensors and Methods. SENSORS (BASEL, SWITZERLAND) 2020; 20:E592. [PMID: 31973140 PMCID: PMC7037130 DOI: 10.3390/s20030592] [Citation(s) in RCA: 99] [Impact Index Per Article: 24.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/11/2019] [Revised: 01/10/2020] [Accepted: 01/12/2020] [Indexed: 11/16/2022]
Abstract
Automated emotion recognition (AEE) is an important issue in various fields of activities which use human emotional reactions as a signal for marketing, technical equipment, or human-robot interaction. This paper analyzes scientific research and technical papers for sensor use analysis, among various methods implemented or researched. This paper covers a few classes of sensors, using contactless methods as well as contact and skin-penetrating electrodes for human emotion detection and the measurement of their intensity. The results of the analysis performed in this paper present applicable methods for each type of emotion and their intensity and propose their classification. The classification of emotion sensors is presented to reveal area of application and expected outcomes from each method, as well as their limitations. This paper should be relevant for researchers using human emotion evaluation and analysis, when there is a need to choose a proper method for their purposes or to find alternative decisions. Based on the analyzed human emotion recognition sensors and methods, we developed some practical applications for humanizing the Internet of Things (IoT) and affective computing systems.
Collapse
Affiliation(s)
- Andrius Dzedzickis
- Faculty of Mechanics, Vilnius Gediminas Technical University, J. Basanaviciaus g. 28, LT-03224 Vilnius, Lithuania;
| | - Artūras Kaklauskas
- Faculty of Civil engineering, Vilnius Gediminas Technical University, Sauletekio ave. 11, LT-10223 Vilnius, Lithuania;
| | - Vytautas Bucinskas
- Faculty of Mechanics, Vilnius Gediminas Technical University, J. Basanaviciaus g. 28, LT-03224 Vilnius, Lithuania;
| |
Collapse
|
20
|
Kaczor EE, Carreiro S, Stapp J, Chapman B, Indic P. Objective Measurement of Physician Stress in the Emergency Department Using a Wearable Sensor. PROCEEDINGS OF THE ... ANNUAL HAWAII INTERNATIONAL CONFERENCE ON SYSTEM SCIENCES. ANNUAL HAWAII INTERNATIONAL CONFERENCE ON SYSTEM SCIENCES 2020; 2020:3729-3738. [PMID: 32015695 PMCID: PMC6996921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Physician stress, and resultant consequences such as burnout, have become increasingly recognized pervasive problems, particularly within the specialty of Emergency Medicine. Stress is difficult to measure objectively, and research predominantly relies on self-reported measures. The present study aims to characterize digital biomarkers of stress as detected by a wearable sensor among Emergency Medicine physicians. Physiologic data were continuously collected using a wearable sensor during clinical work in the emergency department, and participants were asked to self-identify episodes of stress. Machine learning algorithms were used to classify self-reported episodes of stress. Comparing baseline sensor data to data in the 20-minute period preceding self-reported stress episodes demonstrated the highest prediction accuracy for stress. With further study, detection of stress via wearable sensors could be used to facilitate evidence-based stress research and just-in-time interventions for emergency physicians and other high-stress professionals.
Collapse
Affiliation(s)
- Eric E Kaczor
- Department of Emergency Medicine, Division of Medical Toxicology, University of Massachusetts, Medical School, Worcester, MA
| | - Stephanie Carreiro
- Department of Emergency Medicine, Division of Medical Toxicology, University of Massachusetts Medical, School, Worcester, MA
| | - Joshua Stapp
- Department of Electrical Engineering, University of Texas at Tyler, Tyler, TX
| | - Brittany Chapman
- Department of Emergency Medicine, Division of Medical Toxicology, University of Massachusetts Medical School, Worcester, MA
| | - Premananda Indic
- Department of Electrical Engineering, University of Texas at Tyler, Tyler, TX
| |
Collapse
|
21
|
|
22
|
Kyriakou K, Resch B, Sagl G, Petutschnig A, Werner C, Niederseer D, Liedlgruber M, Wilhelm F, Osborne T, Pykett J. Detecting Moments of Stress from Measurements of Wearable Physiological Sensors. SENSORS (BASEL, SWITZERLAND) 2019; 19:E3805. [PMID: 31484366 PMCID: PMC6749249 DOI: 10.3390/s19173805] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/29/2019] [Revised: 08/28/2019] [Accepted: 08/31/2019] [Indexed: 12/28/2022]
Abstract
There is a rich repertoire of methods for stress detection using various physiological signals and algorithms. However, there is still a gap in research efforts moving from laboratory studies to real-world settings. A small number of research has verified when a physiological response is a reaction to an extrinsic stimulus of the participant's environment in real-world settings. Typically, physiological signals are correlated with the spatial characteristics of the physical environment, supported by video records or interviews. The present research aims to bridge the gap between laboratory settings and real-world field studies by introducing a new algorithm that leverages the capabilities of wearable physiological sensors to detect moments of stress (MOS). We propose a rule-based algorithm based on galvanic skin response and skin temperature, combing empirical findings with expert knowledge to ensure transferability between laboratory settings and real-world field studies. To verify our algorithm, we carried out a laboratory experiment to create a "gold standard" of physiological responses to stressors. We validated the algorithm in real-world field studies using a mixed-method approach by spatially correlating the participant's perceived stress, geo-located questionnaires, and the corresponding real-world situation from the video. Results show that the algorithm detects MOS with 84% accuracy, showing high correlations between measured (by wearable sensors), reported (by questionnaires and eDiary entries), and recorded (by video) stress events. The urban stressors that were identified in the real-world studies originate from traffic congestion, dangerous driving situations, and crowded areas such as tourist attractions. The presented research can enhance stress detection in real life and may thus foster a better understanding of circumstances that bring about physiological stress in humans.
Collapse
Affiliation(s)
- Kalliopi Kyriakou
- Department of Geoinformatics, University of Salzburg, 5020 Salzburg, Austria.
| | - Bernd Resch
- Center for Geographic Analysis, Harvard University, Cambridge, MA 02138, USA
| | - Günther Sagl
- Department of Geoinformatics, University of Salzburg, 5020 Salzburg, Austria
| | - Andreas Petutschnig
- Department of Geoinformatics, University of Salzburg, 5020 Salzburg, Austria
| | - Christian Werner
- Department of Geoinformatics, University of Salzburg, 5020 Salzburg, Austria
| | - David Niederseer
- Department of Cardiology, University Hospital Zurich, 8091 Zurich, Switzerland
| | | | - Frank Wilhelm
- Department of Psychology, University of Salzburg, 5020 Salzburg, Austria
| | - Tess Osborne
- Department of Demography, Faculty of Spatial Sciences, University of Groningen, PO Box 800, 9700 AV Groningen, The Netherlands
| | - Jessica Pykett
- School of Geography, Earth and Environmental Sciences, University of Birmingham, Birmingham B15 2TT, UK
| |
Collapse
|
23
|
Shi F, Dey N, Ashour AS, Sifaki-Pistolla D, Sherratt RS. Meta-KANSEI Modeling with Valence-Arousal fMRI Dataset of Brain. Cognit Comput 2018. [DOI: 10.1007/s12559-018-9614-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
24
|
Taj-Eldin M, Ryan C, O'Flynn B, Galvin P. A Review of Wearable Solutions for Physiological and Emotional Monitoring for Use by People with Autism Spectrum Disorder and Their Caregivers. SENSORS (BASEL, SWITZERLAND) 2018; 18:E4271. [PMID: 30518133 PMCID: PMC6308558 DOI: 10.3390/s18124271] [Citation(s) in RCA: 53] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Revised: 11/30/2018] [Accepted: 12/01/2018] [Indexed: 12/28/2022]
Abstract
The goal of real-time feedback on physiological changes, stress monitoring and even emotion detection is becoming a technological reality. People in their daily life experience varying emotional states, some of which are negative and which can lead to decreased attention, decreased productivity and ultimately, reduced quality of life. Therefore, having a solution that continuously monitors the physiological signals of the person and assesses his or her emotional well-being could be a very valuable tool. This paper aims to review existing physiological and motional monitoring devices, highlight their features and compare their sensing capabilities. Such technology would be particularly useful for certain populations who experience rapidly changing emotional states such as people with autism spectrum disorder and people with intellectual disabilities. Wearable sensing devices present a potential solution that can support and complement existing behavioral interventions. This paper presents a review of existing and emerging products in the market. It reviews the literature on state-of-the-art prototypes and analyzes their usefulness, clinical validity, and discusses clinical perspectives. A small number of products offer reliable physiological internal state monitoring and may be suitable for people with Autism Spectrum Disorder (ASD). It is likely that more promising solutions will be available in the near future. Therefore, caregivers should be careful in their selection of devices that meet the care-receiver's personal needs and have strong research support for reliability and validity.
Collapse
Affiliation(s)
- Mohammed Taj-Eldin
- Tyndall National Institute, University College Cork, T12 R5CP Cork, Ireland.
| | - Christian Ryan
- School of Applied Psychology, University College Cork, T12 R5CP Cork, Ireland.
| | - Brendan O'Flynn
- Tyndall National Institute, University College Cork, T12 R5CP Cork, Ireland.
| | - Paul Galvin
- Tyndall National Institute, University College Cork, T12 R5CP Cork, Ireland.
| |
Collapse
|