1
|
Systematic Review of Affective Computing Techniques for Infant Robot Interaction. Int J Soc Robot 2023. [DOI: 10.1007/s12369-023-00985-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/09/2023]
Abstract
AbstractResearch studies on social robotics and human-robot interaction have gained insights into factors that influence people’s perceptions and behaviors towards robots. However, adults’ perceptions of robots may differ significantly from those of infants. Consequently, extending this knowledge also to infants’ attitudes toward robots is a growing field of research. Indeed, infant-robot interaction (IRI) is emerging as a critical and necessary area of research as robots are increasingly used in social environments, such as caring for infants with all types of disabilities, companionship, and education. Although studies have been conducted on the ability of robots to positively engage infants, little is known about the infants’ affective state when interacting with a robot. In this systematic review, technologies for infant affective state recognition relevant to IRI applications are presented and surveyed. Indeed, adapting techniques currently employed for infant’s emotion recognition to the field of IRI results to be a complex task, since it requires timely response while not interfering with the infant’s behavior. Those aspects have a crucial impact on the selection of the emotion recognition techniques and the related metrics to be used for this purpose. Therefore, this review is intended to shed light on the advantages and the current research challenges of the infants’ affective state recognition approaches in the IRI field, elucidates a roadmap for their use in forthcoming studies as well as potentially provide support to future developments of emotion-aware robots.
Collapse
|
2
|
Human-Computer Interaction for Recognizing Speech Emotions Using Multilayer Perceptron Classifier. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:6005446. [PMID: 35388315 PMCID: PMC8979705 DOI: 10.1155/2022/6005446] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Revised: 02/28/2022] [Accepted: 03/04/2022] [Indexed: 12/02/2022]
Abstract
Human-computer interaction (HCI) has seen a paradigm shift from textual or display-based control toward more intuitive control modalities such as voice, gesture, and mimicry. Particularly, speech has a great deal of information, conveying information about the speaker's inner condition and his/her aim and desire. While word analysis enables the speaker's request to be understood, other speech features disclose the speaker's mood, purpose, and motive. As a result, emotion recognition from speech has become critical in current human-computer interaction systems. Moreover, the findings of the several professions involved in emotion recognition are difficult to combine. Many sound analysis methods have been developed in the past. However, it was not possible to provide an emotional analysis of people in a live speech. Today, the development of artificial intelligence and the high performance of deep learning methods bring studies on live data to the fore. This study aims to detect emotions in the human voice using artificial intelligence methods. One of the most important requirements of artificial intelligence works is data. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) open-source dataset was used in the study. The RAVDESS dataset contains more than 2000 data recorded as speeches and songs by 24 actors. Data were collected for eight different moods from the actors. It was aimed at detecting eight different emotion classes, including neutral, calm, happy, sad, angry, fearful, disgusted, and surprised moods. The multilayer perceptron (MLP) classifier, a widely used supervised learning algorithm, was preferred for classification. The proposed model's performance was compared with that of similar studies, and the results were evaluated. An overall accuracy of 81% was obtained for classifying eight different emotions by using the proposed model on the RAVDESS dataset.
Collapse
|
3
|
Patel P,
R
R, Annavarapu RN. EEG-based human emotion recognition using entropy as a feature extraction measure. Brain Inform 2021; 8:20. [PMID: 34609639 PMCID: PMC8492873 DOI: 10.1186/s40708-021-00141-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Accepted: 09/04/2021] [Indexed: 11/10/2022] Open
Abstract
Many studies on brain-computer interface (BCI) have sought to understand the emotional state of the user to provide a reliable link between humans and machines. Advanced neuroimaging methods like electroencephalography (EEG) have enabled us to replicate and understand a wide range of human emotions more precisely. This physiological signal, i.e., EEG-based method is in stark comparison to traditional non-physiological signal-based methods and has been shown to perform better. EEG closely measures the electrical activities of the brain (a nonlinear system) and hence entropy proves to be an efficient feature in extracting meaningful information from raw brain waves. This review aims to give a brief summary of various entropy-based methods used for emotion classification hence providing insights into EEG-based emotion recognition. This study also reviews the current and future trends and discusses how emotion identification using entropy as a measure to extract features, can accomplish enhanced identification when using EEG signal.
Collapse
Affiliation(s)
- Pragati Patel
- Department of Physics, School of Physical, Chemical, and Applied Sciences, Pondicherry University, Puducherry, 605014 India
| | -
Raghunandan
R
- Department of Physics, School of Physical, Chemical, and Applied Sciences, Pondicherry University, Puducherry, 605014 India
| | - Ramesh Naidu Annavarapu
- Department of Physics, School of Physical, Chemical, and Applied Sciences, Pondicherry University, Puducherry, 605014 India
| |
Collapse
|
4
|
Mahzoon H, Okazaki M, Yoshikawa Y, Ishiguro H. Effect of the projection of robot’s talk information on the perception of communicating human. Adv Robot 2021. [DOI: 10.1080/01691864.2021.1964597] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Hamed Mahzoon
- Institute for Open and Transdisciplinary Research Initiatives (OTRI), Osaka University, Osaka, Japan
| | - Momoko Okazaki
- Graduate School of Engineering Science, Osaka University, Osaka, Japan
| | | | - Hiroshi Ishiguro
- Graduate School of Engineering Science, Osaka University, Osaka, Japan
| |
Collapse
|
5
|
KIM DAECHANG, LEE SEUNGBONG, KIM SUNGMIN, JEONG JAEHOON. CONFIRMATION OF THE CLINICAL VALUE AND THE EFFECT OF SOUND BIOFEEDBACK ON THE AUTONOMIC NERVOUS SYSTEM THROUGH HEART RATE VARIABILITY ANALYSIS. J MECH MED BIOL 2021. [DOI: 10.1142/s0219519421500366] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The purpose of this study is to suggest sound biofeedback, which is a new technique of early stress relief effect by observing change in the heart rate variability (HRV). The sound biofeedback imitating heart rate of the comfortable and stress state is termed parasympathetic stimulation sound (PSS) and sympathetic stimulation sound (SSS), respectively. Twelve subjects were selected without previous history of cardiovascular diseases and mental illness, such as arrhythmia, myocardial infarction, depression and panic disorder. To confirm the changes in the low-frequency (LF), high-frequency (HF) and LF/HF values of HRV as stress evaluation indicators, the HRV of subjects was measured by photoplethysmogram. Signals were processed using the peak detect algorithm, and fast Fourier transform. Results were obtained using power specific densities. During the PSS stimulation, the LF/HF tended to decrease generally. On the other hand, during the SSS stimulation, LF/HF tended to increase. The LF/HF Mean change value ([Formula: see text]) using the PSS stimulation is similar to the effect of Transcutaneous Vagal Nerve Stimulation (tVNS). In addition, the quantitative effect of sound biofeedback was confirmed by judging changes in the parasympathetic and sympathetic nerves in the autonomic nervous system (ANS) through [Formula: see text]-score normalized data. These experimental results suggest that sound biofeedback has the same stimulation location and clinical applicability as tVNS. As a result, sound biofeedback may be used as a new method for stress reduction.
Collapse
Affiliation(s)
- DAECHANG KIM
- Department of Medical Biotechnology, Dongguk University-Bio Medi Campus, Gyeonggi-do 10326, Korea
| | - SEUNGBONG LEE
- Department of Medical Biotechnology, Dongguk University-Bio Medi Campus, Gyeonggi-do 10326, Korea
| | - SUNGMIN KIM
- Department of Medical Devices Industry, Dongguk University-Seoul, Seoul 04620, Korea
| | - JAEHOON JEONG
- Department of Medical Devices Industry, Dongguk University-Seoul, Seoul 04620, Korea
| |
Collapse
|
6
|
NeuroSense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2020.12.098] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
7
|
M. M, A. M. Facial geometric feature extraction based emotional expression classification using machine learning algorithms. PLoS One 2021; 16:e0247131. [PMID: 33600467 PMCID: PMC7891769 DOI: 10.1371/journal.pone.0247131] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2020] [Accepted: 02/01/2021] [Indexed: 11/18/2022] Open
Abstract
Emotion plays a significant role in interpersonal communication and also improving social life. In recent years, facial emotion recognition is highly adopted in developing human-computer interfaces (HCI) and humanoid robots. In this work, a triangulation method for extracting a novel set of geometric features is proposed to classify six emotional expressions (sadness, anger, fear, surprise, disgust, and happiness) using computer-generated markers. The subject's face is recognized by using Haar-like features. A mathematical model has been applied to positions of eight virtual markers in a defined location on the subject's face in an automated way. Five triangles are formed by manipulating eight markers' positions as an edge of each triangle. Later, these eight markers are uninterruptedly tracked by Lucas- Kanade optical flow algorithm while subjects' articulating facial expressions. The movement of the markers during facial expression directly changes the property of each triangle. The area of the triangle (AoT), Inscribed circle circumference (ICC), and the Inscribed circle area of a triangle (ICAT) are extracted as features to classify the facial emotions. These features are used to distinguish six different facial emotions using various types of machine learning algorithms. The inscribed circle area of the triangle (ICAT) feature gives a maximum mean classification rate of 98.17% using a Random Forest (RF) classifier compared to other features and classifiers in distinguishing emotional expressions.
Collapse
Affiliation(s)
- Murugappan M.
- Department of Electronics and Communication Engineering, Intelligent Signal Processing (ISP) Research Lab, Kuwait College of Science and Technology (A Private University), Al-Jahra, Kuwait
| | - Mutawa A.
- Computer Engineering Department, College of Engineering and Petroleum, Kuwait University, Kumait, Kuwait
| |
Collapse
|
8
|
Vukelić M, Lingelbach K, Pollmann K, Peissner M. Oscillatory EEG Signatures of Affective Processes during Interaction with Adaptive Computer Systems. Brain Sci 2020; 11:35. [PMID: 33396330 PMCID: PMC7824422 DOI: 10.3390/brainsci11010035] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Revised: 12/16/2020] [Accepted: 12/24/2020] [Indexed: 11/23/2022] Open
Abstract
Affect monitoring is being discussed as a novel strategy to make adaptive systems more user-oriented. Basic knowledge about oscillatory processes and functional connectivity underlying affect during naturalistic human-computer interactions (HCI) is, however, scarce. This study assessed local oscillatory power entrainment and distributed functional connectivity in a close-to-naturalistic HCI-paradigm. Sixteen participants interacted with a simulated assistance system which deliberately evoked positive (supporting goal-achievement) and negative (impeding goal-achievement) affective reactions. Electroencephalography (EEG) was used to examine the reactivity of the cortical system during the interaction by studying both event-related (de-)synchronization (ERD/ERS) and event-related functional coupling of cortical networks towards system-initiated assistance. Significantly higher α-band and β-band ERD in centro-parietal and parieto-occipital regions and β-band ERD in bi-lateral fronto-central regions were observed during impeding system behavior. Supportive system behavior activated significantly higher γ-band ERS in bi-hemispheric parietal-occipital regions. This was accompanied by functional coupling of remote β-band and γ-band activity in the medial frontal, left fronto-central and parietal regions, respectively. Our findings identify oscillatory signatures of positive and negative affective processes as reactions to system-initiated assistance. The findings contribute to the development of EEG-based neuroadaptive assistance loops by suggesting a non-obtrusive method for monitoring affect in HCI.
Collapse
Affiliation(s)
- Mathias Vukelić
- Fraunhofer Institute for Industrial Engineering IAO, 70569 Stuttgart, Germany; (K.P.); (M.P.)
| | - Katharina Lingelbach
- Institute of Human Factors and Technology Management IAT, University of Stuttgart, 70569 Stuttgart, Germany;
- Department of Psychology, University of Oldenburg, 26129 Oldenburg, Germany
| | - Kathrin Pollmann
- Fraunhofer Institute for Industrial Engineering IAO, 70569 Stuttgart, Germany; (K.P.); (M.P.)
| | - Matthias Peissner
- Fraunhofer Institute for Industrial Engineering IAO, 70569 Stuttgart, Germany; (K.P.); (M.P.)
| |
Collapse
|
9
|
Khowaja SA, Prabono AG, Setiawan F, Yahya BN, Lee SL. Toward soft real-time stress detection using wrist-worn devices for human workspaces. Soft comput 2020. [DOI: 10.1007/s00500-020-05338-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
10
|
GIS-Based Emotional Computing: A Review of Quantitative Approaches to Measure the Emotion Layer of Human–Environment Relationships. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION 2020. [DOI: 10.3390/ijgi9090551] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
In recent years, with the growing accessibility of abundant contextual emotion information, which is benefited by the numerous georeferenced user-generated content and the maturity of artificial intelligence (AI)-based emotional computing technics, the emotion layer of human–environment relationship is proposed for enriching traditional methods of various related disciplines such as urban planning. This paper proposes the geographic information system (GIS)-based emotional computing concept, which is a novel framework for applying GIS methods to collective human emotion. The methodology presented in this paper consists of three key steps: (1) collecting georeferenced data containing emotion and environment information such as social media and official sites, (2) detecting emotions using AI-based emotional computing technics such as natural language processing (NLP) and computer vision (CV), and (3) visualizing and analyzing the spatiotemporal patterns with GIS tools. This methodology is a great synergy of multidisciplinary cutting-edge techniques, such as GIScience, sociology, and computer science. Moreover, it can effectively and deeply explore the connection between people and their surroundings with the help of GIS methods. Generally, the framework provides a standard workflow to calculate and analyze the new information layer for researchers, in which a measured human-centric perspective onto the environment is possible.
Collapse
|
11
|
Lim JZ, Mountstephens J, Teo J. Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges. SENSORS (BASEL, SWITZERLAND) 2020; 20:E2384. [PMID: 32331327 PMCID: PMC7219342 DOI: 10.3390/s20082384] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/10/2020] [Revised: 03/31/2020] [Accepted: 03/31/2020] [Indexed: 12/12/2022]
Abstract
The ability to detect users' emotions for the purpose of emotion engineering is currently one of the main endeavors of machine learning in affective computing. Among the more common approaches to emotion detection are methods that rely on electroencephalography (EEG), facial image processing and speech inflections. Although eye-tracking is fast in becoming one of the most commonly used sensor modalities in affective computing, it is still a relatively new approach for emotion detection, especially when it is used exclusively. In this survey paper, we present a review on emotion recognition using eye-tracking technology, including a brief introductory background on emotion modeling, eye-tracking devices and approaches, emotion stimulation methods, the emotional-relevant features extractable from eye-tracking data, and most importantly, a categorical summary and taxonomy of the current literature which relates to emotion recognition using eye-tracking. This review concludes with a discussion on the current open research problems and prospective future research directions that will be beneficial for expanding the body of knowledge in emotion detection using eye-tracking as the primary sensor modality.
Collapse
Affiliation(s)
- Jia Zheng Lim
- Evolutionary Computing Laboratory, Faculty of Computing and Informatics, Universiti Malaysia Sabah, Jalan UMS, Kota Kinabalu 88400, Sabah, Malaysia;
| | - James Mountstephens
- Faculty of Computing and Informatics, Universiti Malaysia Sabah, Jalan UMS, Kota Kinabalu 88400, Sabah, Malaysia;
| | - Jason Teo
- Faculty of Computing and Informatics, Universiti Malaysia Sabah, Jalan UMS, Kota Kinabalu 88400, Sabah, Malaysia;
| |
Collapse
|
12
|
Lyaqini S, Quafafou M, Nachaoui M, Chakib A. Supervised learning as an inverse problem based on non-smooth loss function. Knowl Inf Syst 2020. [DOI: 10.1007/s10115-020-01439-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|
13
|
Towards Mixed-Initiative Human-Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction. SENSORS 2020; 20:s20010296. [PMID: 31948046 PMCID: PMC6982852 DOI: 10.3390/s20010296] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/02/2019] [Revised: 12/31/2019] [Accepted: 01/02/2020] [Indexed: 11/17/2022]
Abstract
The design of human–robot interactions is a key challenge to optimize operational performance. A promising approach is to consider mixed-initiative interactions in which the tasks and authority of each human and artificial agents are dynamically defined according to their current abilities. An important issue for the implementation of mixed-initiative systems is to monitor human performance to dynamically drive task allocation between human and artificial agents (i.e., robots). We, therefore, designed an experimental scenario involving missions whereby participants had to cooperate with a robot to fight fires while facing hazards. Two levels of robot automation (manual vs. autonomous) were randomly manipulated to assess their impact on the participants’ performance across missions. Cardiac activity, eye-tracking, and participants’ actions on the user interface were collected. The participants performed differently to an extent that we could identify high and low score mission groups that also exhibited different behavioral, cardiac and ocular patterns. More specifically, our findings indicated that the higher level of automation could be beneficial to low-scoring participants but detrimental to high-scoring ones, and vice versa. In addition, inter-subject single-trial classification results showed that the studied behavioral and physiological features were relevant to predict mission performance. The highest average balanced accuracy (74%) was reached using the features extracted from all input devices. These results suggest that an adaptive HRI driving system, that would aim at maximizing performance, would be capable of analyzing such physiological and behavior markers online to further change the level of automation when it is relevant for the mission purpose.
Collapse
|
14
|
Bevilacqua F, Engström H, Backlund P. Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games. SENSORS 2019; 19:s19132877. [PMID: 31261716 PMCID: PMC6650833 DOI: 10.3390/s19132877] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/28/2019] [Revised: 06/21/2019] [Accepted: 06/25/2019] [Indexed: 12/24/2022]
Abstract
Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments ( n = 20 and n = 62 ) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games.
Collapse
Affiliation(s)
- Fernando Bevilacqua
- Computer Science, Federal University of Fronteira Sul, Chapecó 89802 112, Brazil
| | - Henrik Engström
- School of Informatics, University of Skövde, 541 28 Skövde, Sweden.
| | - Per Backlund
- School of Informatics, University of Skövde, 541 28 Skövde, Sweden
| |
Collapse
|
15
|
Egger M, Ley M, Hanke S. Emotion Recognition from Physiological Signal Analysis: A Review. ACTA ACUST UNITED AC 2019. [DOI: 10.1016/j.entcs.2019.04.009] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
16
|
Machine learning approaches to understand the influence of urban environments on human’s physiological response. Inf Sci (N Y) 2019. [DOI: 10.1016/j.ins.2018.09.061] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
17
|
Song Z, Li W, Bao Y, Han F, Gao L, Xu J, Ma Y, Han D, Niu L. Breathable and Skin-Mountable Strain Sensor with Tunable Stretchability, Sensitivity, and Linearity via Surface Strain Delocalization for Versatile Skin Activities' Recognition. ACS APPLIED MATERIALS & INTERFACES 2018; 10:42826-42836. [PMID: 30499657 DOI: 10.1021/acsami.8b14365] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Metal film/elastomer-based strain sensors usually exhibit small rupture strain (<5%) because of the strain localization and necking effect of the metal film under tension. To achieve both high stretchability and wide linear region is still challenging for metal film-based strain sensors. Here, we propose a low-cost yet effective strategy for fabricating ultrathin, breathable, and skin-mountable strain sensors with high sensitivity (gauge factor from 7.2 to 474.8), high stretchability (up to 140%), and good linearity by regulating the surface strain delocalization in the metal film on elastomer substrate. On the basis of this phenomenon of strain delocalization, the sensitivity and linearity are further enhanced based on a novel diffraction-induced Au film with gradient thickness. Meanwhile, by means of the strain redistribution and Poisson effect, a novel biaxial strain sensor is designed for recognition of complex human motion. On the basis of the enhanced stretchability, linearity, skin-mountable, and breathable properties, the low-cost metal film-based strain sensors can be broadened as disposable wearables for human motion detection, emotional expression recognition, human interaction, and virtual reality.
Collapse
Affiliation(s)
- Zhongqian Song
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
- University of Chinese Academy of Sciences , Beijing 100039 , China
| | - Weiyan Li
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
| | - Yu Bao
- Center for Advanced Analytical Science, c/o School of Chemistry and Chemical Engineering , Guangzhou University , Guangzhou 510006 , P.R. China
| | - Fangjie Han
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
| | - Lifang Gao
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
- University of Chinese Academy of Sciences , Beijing 100039 , China
| | - Jianan Xu
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
- University of Chinese Academy of Sciences , Beijing 100039 , China
| | - Yingming Ma
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
| | - Dongxue Han
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
- Center for Advanced Analytical Science, c/o School of Chemistry and Chemical Engineering , Guangzhou University , Guangzhou 510006 , P.R. China
| | - Li Niu
- State Key Laboratory of Electroanalytical Chemistry, c/o Engineering Laboratory for Modern Analytical Techniques, CAS Center for Excellence in Nanoscience, Changchun Institute of Applied Chemistry , Chinese Academy of Sciences , Changchun 130022 Jilin , China
- Center for Advanced Analytical Science, c/o School of Chemistry and Chemical Engineering , Guangzhou University , Guangzhou 510006 , P.R. China
| |
Collapse
|
18
|
Hassanien AE, Kilany M, Houssein EH, AlQaheri H. Intelligent human emotion recognition based on elephant herding optimization tuned support vector regression. Biomed Signal Process Control 2018. [DOI: 10.1016/j.bspc.2018.05.039] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
19
|
Abstract
Supervised machine-learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world?
Collapse
|
20
|
Wang Z, Majewicz Fey A. Human-centric predictive model of task difficulty for human-in-the-loop control tasks. PLoS One 2018; 13:e0195053. [PMID: 29621301 PMCID: PMC5886487 DOI: 10.1371/journal.pone.0195053] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2017] [Accepted: 03/07/2018] [Indexed: 11/18/2022] Open
Abstract
Quantitatively measuring the difficulty of a manipulation task in human-in-the-loop control systems is ill-defined. Currently, systems are typically evaluated through task-specific performance measures and post-experiment user surveys; however, these methods do not capture the real-time experience of human users. In this study, we propose to analyze and predict the difficulty of a bivariate pointing task, with a haptic device interface, using human-centric measurement data in terms of cognition, physical effort, and motion kinematics. Noninvasive sensors were used to record the multimodal response of human user for 14 subjects performing the task. A data-driven approach for predicting task difficulty was implemented based on several task-independent metrics. We compare four possible models for predicting task difficulty to evaluated the roles of the various types of metrics, including: (I) a movement time model, (II) a fusion model using both physiological and kinematic metrics, (III) a model only with kinematic metrics, and (IV) a model only with physiological metrics. The results show significant correlation between task difficulty and the user sensorimotor response. The fusion model, integrating user physiology and motion kinematics, provided the best estimate of task difficulty (R2 = 0.927), followed by a model using only kinematic metrics (R2 = 0.921). Both models were better predictors of task difficulty than the movement time model (R2 = 0.847), derived from Fitt's law, a well studied difficulty model for human psychomotor control.
Collapse
Affiliation(s)
- Ziheng Wang
- Department of Mechanical Engineering, The University of Texas at Dallas, Richardson, TX 75080, United States of America
| | - Ann Majewicz Fey
- Department of Mechanical Engineering, The University of Texas at Dallas, Richardson, TX 75080, United States of America
- Department of Surgery, UT Southwestern Medical Center, Dallas, TX 75390, United States of America
| |
Collapse
|
21
|
Kara S, Bilgiç V, Akar SA. Respiratory Variability during Different Auditory Stimulation Periods in Schizophrenia Patients. Methods Inf Med 2018; 51:29-38. [DOI: 10.3414/me10-01-0087] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2010] [Accepted: 05/12/2011] [Indexed: 11/09/2022]
Abstract
SummaryBackground: Schizophrenic patients are known to have difficulty processing emotions and to exhibit impairment in stimuli discrimination. However, there is limited knowledge regarding their physiological responsivity to auditory stimuli.Objectives: The purpose of this study was to compare the respiratory effects of two types of auditory stimuli with emotional content, classical Turkish music (CTM) and white noise (WN), on schizophrenia patients and healthy control subjects.Methods: Forty-six individuals participated in the experiment, and respiratory signals derived from a strain-gauge were recorded. Two important respiratory patterns, respiration rate and depth, were analyzed.Results: The results indicated that the patients presented a significantly higher respiration rate than control subjects during the initial baseline and WN exposure periods. Although CTM evoked an increase in respiration rates and a decrease in respiration depths in the control group, no significant differences were found during the stimulation periods in the patient group. The respiration rate was lower in the post-stimulation period than during the initial baseline period, and no respiration depth differences were found for the WN, music or post-stimulation periods in the schizophrenia group. Patients exhibited a greater respiration depth than the control subjects over all periods; however, a significant difference between the patient and control groups was obtained in the second resting condition and CTM exposure period. Furthermore, to analyze the effect of symptom severity on respiratory patterns, patients were divided into two classes according to their Positive and Negative Syndrome Scale score.Conclusions: Further studies are needed to correlate respiratory differences with emotionally evocative stimuli and to refine our understanding of the dynamics of these types of stimuli in relation to clinical state and medication effects.
Collapse
|
22
|
Beyond Human-in-the-Loop: Empowering End-Users with Transparent Machine Learning. HUMAN AND MACHINE LEARNING 2018. [DOI: 10.1007/978-3-319-90403-0_3] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
|
23
|
Ghaderyan P, Abbasi A. A novel cepstral-based technique for automatic cognitive load estimation. Biomed Signal Process Control 2018. [DOI: 10.1016/j.bspc.2017.07.020] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
24
|
Ajoudani A, Zanchettin AM, Ivaldi S, Albu-Schäffer A, Kosuge K, Khatib O. Progress and prospects of the human–robot collaboration. Auton Robots 2017. [DOI: 10.1007/s10514-017-9677-2] [Citation(s) in RCA: 235] [Impact Index Per Article: 33.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
25
|
Paiva JS, Cardoso J, Pereira T. Supervised learning methods for pathological arterial pulse wave differentiation: A SVM and neural networks approach. Int J Med Inform 2017; 109:30-38. [PMID: 29195703 DOI: 10.1016/j.ijmedinf.2017.10.011] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2017] [Revised: 09/29/2017] [Accepted: 10/16/2017] [Indexed: 11/17/2022]
Abstract
OBJECTIVE The main goal of this study was to develop an automatic method based on supervised learning methods, able to distinguish healthy from pathologic arterial pulse wave (APW), and those two from noisy waveforms (non-relevant segments of the signal), from the data acquired during a clinical examination with a novel optical system. MATERIALS AND METHODS The APW dataset analysed was composed by signals acquired in a clinical environment from a total of 213 subjects, including healthy volunteers and non-healthy patients. The signals were parameterised by means of 39pulse features: morphologic, time domain statistics, cross-correlation features, wavelet features. Multiclass Support Vector Machine Recursive Feature Elimination (SVM RFE) method was used to select the most relevant features. A comparative study was performed in order to evaluate the performance of the two classifiers: Support Vector Machine (SVM) and Artificial Neural Network (ANN). RESULTS AND DISCUSSION SVM achieved a statistically significant better performance for this problem with an average accuracy of 0.9917±0.0024 and a F-Measure of 0.9925±0.0019, in comparison with ANN, which reached the values of 0.9847±0.0032 and 0.9852±0.0031 for Accuracy and F-Measure, respectively. A significant difference was observed between the performances obtained with SVM classifier using a different number of features from the original set available. CONCLUSION The comparison between SVM and NN allowed reassert the higher performance of SVM. The results obtained in this study showed the potential of the proposed method to differentiate those three important signal outcomes (healthy, pathologic and noise) and to reduce bias associated with clinical diagnosis of cardiovascular disease using APW.
Collapse
Affiliation(s)
- Joana S Paiva
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Rua Dr. Roberto Frias, 4200, Porto, Portugal; Physics and Astronomy Department, Sciences Faculty, University of Porto, Rua do Campo Alegre, 4169-007 Porto, Portugal
| | - João Cardoso
- LIBPhys-UC, Physics Department, University of Coimbra, Rua Larga, 3004-516 Coimbra, Portugal
| | - Tânia Pereira
- LIBPhys-UC, Physics Department, University of Coimbra, Rua Larga, 3004-516 Coimbra, Portugal.
| |
Collapse
|
26
|
Paiva JS, Dias D, Cunha JPS. Beat-ID: Towards a computationally low-cost single heartbeat biometric identity check system based on electrocardiogram wave morphology. PLoS One 2017; 12:e0180942. [PMID: 28719614 PMCID: PMC5515426 DOI: 10.1371/journal.pone.0180942] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2017] [Accepted: 06/08/2017] [Indexed: 11/25/2022] Open
Abstract
In recent years, safer and more reliable biometric methods have been developed. Apart from the need for enhanced security, the media and entertainment sectors have also been applying biometrics in the emerging market of user-adaptable objects/systems to make these systems more user-friendly. However, the complexity of some state-of-the-art biometric systems (e.g., iris recognition) or their high false rejection rate (e.g., fingerprint recognition) is neither compatible with the simple hardware architecture required by reduced-size devices nor the new trend of implementing smart objects within the dynamic market of the Internet of Things (IoT). It was recently shown that an individual can be recognized by extracting features from their electrocardiogram (ECG). However, most current ECG-based biometric algorithms are computationally demanding and/or rely on relatively large (several seconds) ECG samples, which are incompatible with the aforementioned application fields. Here, we present a computationally low-cost method (patent pending), including simple mathematical operations, for identifying a person using only three ECG morphology-based characteristics from a single heartbeat. The algorithm was trained/tested using ECG signals of different duration from the Physionet database on more than 60 different training/test datasets. The proposed method achieved maximal averaged accuracy of 97.450% in distinguishing each subject from a ten-subject set and false acceptance and rejection rates (FAR and FRR) of 5.710±1.900% and 3.440±1.980%, respectively, placing Beat-ID in a very competitive position in terms of the FRR/FAR among state-of-the-art methods. Furthermore, the proposed method can identify a person using an average of 1.020 heartbeats. It therefore has FRR/FAR behavior similar to obtaining a fingerprint, yet it is simpler and requires less expensive hardware. This method targets low-computational/energy-cost scenarios, such as tiny wearable devices (e.g., a smart object that automatically adapts its configuration to the user). A hardware proof-of-concept implementation is presented as an annex to this paper.
Collapse
Affiliation(s)
- Joana S. Paiva
- Biomedical Research And INnovation (BRAIN), Centre for Biomedical Engineering Research (C-BER), INESC Technology and Science, Porto, Portugal
- Physics and Astronomy Department, Faculty of Sciences, University of Porto, Porto, Portugal
| | - Duarte Dias
- Biomedical Research And INnovation (BRAIN), Centre for Biomedical Engineering Research (C-BER), INESC Technology and Science, Porto, Portugal
| | - João P. S. Cunha
- Biomedical Research And INnovation (BRAIN), Centre for Biomedical Engineering Research (C-BER), INESC Technology and Science, Porto, Portugal
- Faculty of Engineering, University of Porto, Porto, Portugal
| |
Collapse
|
27
|
Ghaderyan P, Abbasi A. An efficient automatic workload estimation method based on electrodermal activity using pattern classifier combinations. Int J Psychophysiol 2016; 110:91-101. [PMID: 27780715 DOI: 10.1016/j.ijpsycho.2016.10.013] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2016] [Revised: 10/16/2016] [Accepted: 10/17/2016] [Indexed: 12/17/2022]
Abstract
Automatic workload estimation has received much attention because of its application in error prevention, diagnosis, and treatment of neural system impairment. The development of a simple but reliable method using minimum number of psychophysiological signals is a challenge in automatic workload estimation. To address this challenge, this paper presented three different decomposition techniques (Fourier, cepstrum, and wavelet transforms) to analyze electrodermal activity (EDA). The efficiency of various statistical and entropic features was investigated and compared. To recognize different levels of an arithmetic task, the features were processed by principal component analysis and machine-learning techniques. These methods have been incorporated into a workload estimation system based on two types: feature-level and decision-level combinations. The results indicated the reliability of the method for automatic and real-time inference of psychological states. This method provided a quantitative estimation of the workload levels and a bias-free evaluation approach. The high-average accuracy of 90% and cost effective requirement were the two important attributes of the proposed workload estimation system. New entropic features were proved to be more sensitive measures for quantifying time and frequency changes in EDA. The effectiveness of these measures was also compared with conventional tonic EDA measures, demonstrating the superiority of the proposed method in achieving accurate estimation of workload levels.
Collapse
Affiliation(s)
- Peyvand Ghaderyan
- Computational Neuroscience Laboratory, Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran.
| | - Ataollah Abbasi
- Computational Neuroscience Laboratory, Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran.
| |
Collapse
|
28
|
Motion codeword generation using selective subsequence clustering for human action recognition. INTEL SERV ROBOT 2016. [DOI: 10.1007/s11370-016-0208-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
29
|
Cheung S, Han E, Kushki A, Anagnostou E, Biddiss E. Biomusic: An Auditory Interface for Detecting Physiological Indicators of Anxiety in Children. Front Neurosci 2016; 10:401. [PMID: 27625593 PMCID: PMC5003931 DOI: 10.3389/fnins.2016.00401] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2016] [Accepted: 08/15/2016] [Indexed: 11/13/2022] Open
Abstract
For children with profound disabilities affecting communication, it can be extremely challenging to identify salient emotions such as anxiety. If left unmanaged, anxiety can lead to hypertension, cardiovascular disease, and other psychological diagnoses. Physiological signals of the autonomic nervous system are indicative of anxiety, but can be difficult to interpret for non-specialist caregivers. This paper evaluates an auditory interface for intuitive detection of anxiety from physiological signals. The interface, called "Biomusic," maps physiological signals to music (i.e., electrodermal activity to melody; skin temperature to musical key; heart rate to drum beat; respiration to a "whooshing" embellishment resembling the sound of an exhalation). The Biomusic interface was tested in two experiments. Biomusic samples were generated from physiological recordings of typically developing children (n = 10) and children with autism spectrum disorders (n = 5) during relaxing and anxiety-provoking conditions. Adult participants (n = 16) were then asked to identify "anxious" or "relaxed" states by listening to the samples. In a classification task with 30 Biomusic samples (1 relaxed state, 1 anxious state per child), classification accuracy, sensitivity, and specificity were 80.8% [standard error (SE) = 2.3], 84.9% (SE = 3.0), and 76.8% (SE = 3.9), respectively. Participants were able to form an early and accurate impression of the anxiety state within 12.1 (SE = 0.7) seconds of hearing the Biomusic with very little training (i.e., < 10 min) and no contextual information. Biomusic holds promise for monitoring, communication, and biofeedback systems for anxiety management.
Collapse
Affiliation(s)
- Stephanie Cheung
- Institute of Biomaterials and Biomedical Engineering, University of TorontoToronto, ON, Canada; Bloorview Research Institute, Holland Bloorview Kids Rehabilitation HospitalToronto, ON, Canada
| | - Elizabeth Han
- Institute of Biomaterials and Biomedical Engineering, University of TorontoToronto, ON, Canada; Bloorview Research Institute, Holland Bloorview Kids Rehabilitation HospitalToronto, ON, Canada
| | - Azadeh Kushki
- Institute of Biomaterials and Biomedical Engineering, University of TorontoToronto, ON, Canada; Bloorview Research Institute, Holland Bloorview Kids Rehabilitation HospitalToronto, ON, Canada
| | - Evdokia Anagnostou
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation HospitalToronto, ON, Canada; Department of Paediatrics, University of TorontoToronto, ON, Canada
| | - Elaine Biddiss
- Institute of Biomaterials and Biomedical Engineering, University of TorontoToronto, ON, Canada; Bloorview Research Institute, Holland Bloorview Kids Rehabilitation HospitalToronto, ON, Canada
| |
Collapse
|
30
|
Cummings ML, Gao F, Thornburg KM. Boredom in the Workplace: A New Look at an Old Problem. HUMAN FACTORS 2016; 58:279-300. [PMID: 26490443 DOI: 10.1177/0018720815609503] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/28/2014] [Accepted: 09/04/2015] [Indexed: 06/05/2023]
Abstract
OBJECTIVE We review historical and more recent efforts in boredom research and related fields. A framework is presented that organizes the various facets of boredom, particularly in supervisory control settings, and research gaps and future potential areas for study are highlighted. BACKGROUND Given the ubiquity of boredom across a wide spectrum of work environments--exacerbated by increasingly automated systems that remove humans from direct, physical system interaction and possibly increasing tedium in the workplace--there is a need not only to better understand the multiple facets of boredom in work environments but to develop targeted mitigation strategies. METHOD To better understand the relationships between the various influences and outcomes of boredom, a systems-based framework, called the Boredom Influence Diagram, is proposed that describes various elements of boredom and their interrelationships. RESULTS Boredom is closely related to vigilance, attention management, and task performance. This review highlights the need to develop more naturalistic experiments that reflect the characteristics of a boring work environment. CONCLUSION With the increase in automation, boredom in the workplace will likely become a more prevalent issue for motivation and retention. In addition, developing continuous measures of boredom based on physiological signals is critical. APPLICATION Personnel selection and improvements in system and task design can potentially mitigate boredom. However, more work is needed to develop and evaluate other potential interventions.
Collapse
Affiliation(s)
| | - Fei Gao
- Massachusetts Institute of Technology, Cambridge
| | | |
Collapse
|
31
|
Maruthapillai V, Murugappan M. Optimal Geometrical Set for Automated Marker Placement to Virtualized Real-Time Facial Emotions. PLoS One 2016; 11:e0149003. [PMID: 26859884 PMCID: PMC4747560 DOI: 10.1371/journal.pone.0149003] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2016] [Accepted: 01/26/2016] [Indexed: 11/18/2022] Open
Abstract
In recent years, real-time face recognition has been a major topic of interest in developing intelligent human-machine interaction systems. Over the past several decades, researchers have proposed different algorithms for facial expression recognition, but there has been little focus on detection in real-time scenarios. The present work proposes a new algorithmic method of automated marker placement used to classify six facial expressions: happiness, sadness, anger, fear, disgust, and surprise. Emotional facial expressions were captured using a webcam, while the proposed algorithm placed a set of eight virtual markers on each subject's face. Facial feature extraction methods, including marker distance (distance between each marker to the center of the face) and change in marker distance (change in distance between the original and new marker positions), were used to extract three statistical features (mean, variance, and root mean square) from the real-time video sequence. The initial position of each marker was subjected to the optical flow algorithm for marker tracking with each emotional facial expression. Finally, the extracted statistical features were mapped into corresponding emotional facial expressions using two simple non-linear classifiers, K-nearest neighbor and probabilistic neural network. The results indicate that the proposed automated marker placement algorithm effectively placed eight virtual markers on each subject's face and gave a maximum mean emotion classification rate of 96.94% using the probabilistic neural network.
Collapse
Affiliation(s)
- Vasanthan Maruthapillai
- School of Mechatronic Engineering, Universiti Malaysia Perlis, 02600, Ulu Pauh, Arau, Perlis, West Malaysia
| | - Murugappan Murugappan
- School of Mechatronic Engineering, Universiti Malaysia Perlis, 02600, Ulu Pauh, Arau, Perlis, West Malaysia
| |
Collapse
|
32
|
A Survey of Autonomous Human Affect Detection Methods for Social Robots Engaged in Natural HRI. J INTELL ROBOT SYST 2015. [DOI: 10.1007/s10846-015-0259-2] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
33
|
Khair NM, Hariharan M, Yaacob S, Basah SN. Locality sensitivity discriminant analysis-based feature ranking of human emotion actions recognition. J Phys Ther Sci 2015; 27:2649-53. [PMID: 26357453 PMCID: PMC4563335 DOI: 10.1589/jpts.27.2649] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2015] [Accepted: 05/18/2015] [Indexed: 11/24/2022] Open
Abstract
[Purpose] Computational intelligence similar to pattern recognition is frequently confronted with high-dimensional data. Therefore, the reduction of the dimensionality is critical to make the manifold features amenable. Procedures that are analytically or computationally manageable in smaller amounts of data and low-dimensional space can become important to produce a better classification performance. [Methods] Thus, we proposed two stage reduction techniques. Feature selection-based ranking using information gain (IG) and Chi-square (Chisq) are used to identify the best ranking of the features selected for emotion classification in different actions including knocking, throwing, and lifting. Then, feature reduction-based locality sensitivity discriminant analysis (LSDA) and principal component analysis (PCA) are used to transform the selected feature to low-dimensional space. Two-stage feature selection-reduction methods such as IG-PCA, IG-LSDA, Chisq-PCA, and Chisq-LSDA are proposed. [Results] The result confirms that applying feature ranking combined with a dimensional-reduction method increases the performance of the classifiers. [Conclusion] The dimension reduction was performed using LSDA by denoting the features of the highest importance determined using IG and Chisq to not only improve the effectiveness but also reduce the computational time.
Collapse
Affiliation(s)
- Nurnadia M Khair
- School of Mechatronic Engineering, Universiti Malaysia Perlis (UniMAP), Malaysia
| | - M Hariharan
- School of Mechatronic Engineering, Universiti Malaysia Perlis (UniMAP), Malaysia
| | - S Yaacob
- Universiti Kuala Lumpur Malaysian Spanish Institute, Kulim Hi-Tech Park, Malaysia
| | - Shafriza Nisha Basah
- School of Mechatronic Engineering, Universiti Malaysia Perlis (UniMAP), Malaysia
| |
Collapse
|
34
|
de Vries J(GJ, Pauws SC, Biehl M. Insightful stress detection from physiology modalities using Learning Vector Quantization. Neurocomputing 2015. [DOI: 10.1016/j.neucom.2014.10.008] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
35
|
Bajaj V, Pachori RB. Detection of Human Emotions Using Features Based on the Multiwavelet Transform of EEG Signals. BRAIN-COMPUTER INTERFACES 2015. [DOI: 10.1007/978-3-319-10978-7_8] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|
36
|
Shirzad N, Van der Loos HFM. Adaptation of task difficulty in rehabilitation exercises based on the user's motor performance and physiological responses. IEEE Int Conf Rehabil Robot 2014; 2013:6650429. [PMID: 24187247 DOI: 10.1109/icorr.2013.6650429] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Although robot-assisted rehabilitation regimens are as effective, functionally, as conventional therapies, they still lack features to increase patients' engagement in the regimen. Providing rehabilitation tasks at a "desirable difficulty" is one of the ways to address this issue and increase the motivation of a patient to continue with the therapy program. Then the problem is to design a system that is capable of estimating the user's desirable difficulty, and ultimately, modifying the task based on this prediction. In this paper we compared the performance of three machine learning algorithms in predicting a user's desirable difficulty during a typical reaching motion rehabilitation task. Different levels of error amplification were used as different levels of task difficulty. We explored the usefulness of using participants' motor performance and physiological signals during the reaching task in prediction of their desirable difficulties. Results showed that a Neural Network approach gives higher prediction accuracy in comparison with models based on k-Nearest Neighbor and Discriminant Analysis methods.
Collapse
|
37
|
Feature selection for speech emotion recognition in Spanish and Basque: on the use of machine learning to improve human-computer interaction. PLoS One 2014; 9:e108975. [PMID: 25279686 PMCID: PMC4184843 DOI: 10.1371/journal.pone.0108975] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2014] [Accepted: 09/05/2014] [Indexed: 11/19/2022] Open
Abstract
Study of emotions in human–computer interaction is a growing research area. This paper shows an attempt to select the most significant features for emotion recognition in spoken Basque and Spanish Languages using different methods for feature selection. RekEmozio database was used as the experimental data set. Several Machine Learning paradigms were used for the emotion classification task. Experiments were executed in three phases, using different sets of features as classification variables in each phase. Moreover, feature subset selection was applied at each phase in order to seek for the most relevant feature subset. The three phases approach was selected to check the validity of the proposed approach. Achieved results show that an instance-based learning algorithm using feature subset selection techniques based on evolutionary algorithms is the best Machine Learning paradigm in automatic emotion recognition, with all different feature sets, obtaining a mean of 80,05% emotion recognition rate in Basque and a 74,82% in Spanish. In order to check the goodness of the proposed process, a greedy searching approach (FSS-Forward) has been applied and a comparison between them is provided. Based on achieved results, a set of most relevant non-speaker dependent features is proposed for both languages and new perspectives are suggested.
Collapse
|
38
|
Analysis of heart rate variability during auditory stimulation periods in patients with schizophrenia. J Clin Monit Comput 2014; 29:153-62. [DOI: 10.1007/s10877-014-9580-8] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2013] [Accepted: 05/06/2014] [Indexed: 02/06/2023]
|
39
|
Bekele E, Sarkar N. Psychophysiological Feedback for Adaptive Human–Robot Interaction (HRI). HUMAN–COMPUTER INTERACTION SERIES 2014. [DOI: 10.1007/978-1-4471-6392-3_7] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
40
|
Walter S, Wendt C, Böhnke J, Crawcour S, Tan JW, Chan A, Limbrecht K, Gruss S, Traue HC. Similarities and differences of emotions in human-machine and human-human interactions: what kind of emotions are relevant for future companion systems? ERGONOMICS 2013; 57:374-386. [PMID: 23924061 DOI: 10.1080/00140139.2013.822566] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
Cognitive-technical intelligence is envisioned to be constantly available and capable of adapting to the user's emotions. However, the question is: what specific emotions should be reliably recognised by intelligent systems? Hence, in this study, we have attempted to identify similarities and differences of emotions between human-human (HHI) and human-machine interactions (HMI). We focused on what emotions in the experienced scenarios of HMI are retroactively reflected as compared with HHI. The sample consisted of N = 145 participants, who were divided into two groups. Positive and negative scenario descriptions of HMI and HHI were given by the first and second groups, respectively. Subsequently, the participants evaluated their respective scenarios with the help of 94 adjectives relating to emotions. The correlations between the occurrences of emotions in the HMI versus HHI were very high. The results do not support the statement that only a few emotions in HMI are relevant.
Collapse
|
41
|
|
42
|
Bekele ET, Lahiri U, Swanson AR, Crittendon JA, Warren ZE, Sarkar N. A step towards developing adaptive robot-mediated intervention architecture (ARIA) for children with autism. IEEE Trans Neural Syst Rehabil Eng 2013; 21:289-99. [PMID: 23221831 PMCID: PMC3860752 DOI: 10.1109/tnsre.2012.2230188] [Citation(s) in RCA: 93] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Emerging technology, especially robotic technology, has been shown to be appealing to children with autism spectrum disorders (ASD). Such interest may be leveraged to provide repeatable, accurate and individualized intervention services to young children with ASD based on quantitative metrics. However, existing robot-mediated systems tend to have limited adaptive capability that may impact individualization. Our current work seeks to bridge this gap by developing an adaptive and individualized robot-mediated technology for children with ASD. The system is composed of a humanoid robot with its vision augmented by a network of cameras for real-time head tracking using a distributed architecture. Based on the cues from the child's head movement, the robot intelligently adapts itself in an individualized manner to generate prompts and reinforcements with potential to promote skills in the ASD core deficit area of early social orienting. The system was validated for feasibility, accuracy, and performance. Results from a pilot usability study involving six children with ASD and a control group of six typically developing (TD) children are presented.
Collapse
Affiliation(s)
- Esubalew T Bekele
- Electrical Engineering and Computer Science Department, Vanderbilt University, Nashville, TN 37212, USA.
| | | | | | | | | | | |
Collapse
|
43
|
Evaluating Classifiers for Emotion Recognition Using EEG. FOUNDATIONS OF AUGMENTED COGNITION 2013. [DOI: 10.1007/978-3-642-39454-6_53] [Citation(s) in RCA: 47] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
|
44
|
CHUEH TUNGHUNG, CHEN TAIBEEN, LU HENRYHORNGSHING, JU SHANSHAN, TAO TEHHO, SHAW JIUNNHAUR. STATISTICAL PREDICTION OF EMOTIONAL STATES BY PHYSIOLOGICAL SIGNALS WITH MANOVA AND MACHINE LEARNING. INT J PATTERN RECOGN 2012. [DOI: 10.1142/s0218001412500085] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
For the importance of communication between human and machine interface, it would be valuable to develop an implement which has the ability to recognize emotional states. In this paper, we proposed an approach which can deal with the daily dependence and personal dependence in the data of multiple subjects and samples. 30 features were extracted from the physiological signals of subject for three states of emotion. The physiological signals measured were: electrocardiogram (ECG), skin temperature (SKT) and galvanic skin response (GSR). After removing the daily dependence and personal dependence by the statistical technique of MANOVA, six machine learning methods including Bayesian network learning, naive Bayesian classification, SVM, decision tree of C4.5, Logistic model and K-nearest-neighbor (KNN) were implemented to differentiate the emotional states. The results showed that Logistic model gives the best classification accuracy and the statistical technique of MANOVA can significantly improve the performance of all six machine learning methods in emotion recognition system.
Collapse
Affiliation(s)
- TUNG-HUNG CHUEH
- Green Energy and Environment Research Laboratories, Industrial Technology Research Institute, Chutung, Hsinchu 310, Taiwan, R. O. C
| | - TAI-BEEN CHEN
- Department of Medical Imaging and Radiological Sciences, I-Shou University Kaohsiung 824, Taiwan, R. O. C
| | - HENRY HORNG-SHING LU
- Institute of Statistics, National Chiao Tung University, Hsinchu 300, Taiwan, R. O. C
| | - SHAN-SHAN JU
- Center for Measurement Standards, Industrial Technology Research Institute, Hsinchu 300, Taiwan, R. O. C
| | - TEH-HO TAO
- Center for Measurement Standards, Industrial Technology Research Institute, Hsinchu 300, Taiwan, R. O. C
| | - JIUNN-HAUR SHAW
- Center for Measurement Standards, Industrial Technology Research Institute, Hsinchu 300, Taiwan, R. O. C
| |
Collapse
|
45
|
Yongwook Chae, Jaeseung Jeong, Sungho Jo. Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI. IEEE T ROBOT 2012. [DOI: 10.1109/tro.2012.2201310] [Citation(s) in RCA: 137] [Impact Index Per Article: 11.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
46
|
Hyun KH, Kim EH, Kwak YK. Emotional Feature Extraction Method Based on the Concentration of Phoneme Influence for Human–Robot Interaction. Adv Robot 2012. [DOI: 10.1163/016918609x12585530487822] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Kyung Hak Hyun
- a Department of Mechanical Engineering, KAIST, 335 Gwahangro, Yuseong-Gu, Daejeon 305-701, South Korea;,
| | - Eun Ho Kim
- b Department of Mechanical Engineering, KAIST, 335 Gwahangro, Yuseong-Gu, Daejeon 305-701, South Korea
| | - Yoon Keun Kwak
- c Department of Mechanical Engineering, KAIST, 335 Gwahangro, Yuseong-Gu, Daejeon 305-701, South Korea
| |
Collapse
|
47
|
Chanel G, Rebetez C, Bétrancourt M, Pun T. Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty. ACTA ACUST UNITED AC 2011. [DOI: 10.1109/tsmca.2011.2116000] [Citation(s) in RCA: 250] [Impact Index Per Article: 19.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
48
|
Mower E, Mataric MJ, Narayanan S. A Framework for Automatic Human Emotion Classification Using Emotion Profiles. ACTA ACUST UNITED AC 2011. [DOI: 10.1109/tasl.2010.2076804] [Citation(s) in RCA: 132] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
49
|
Welch KC, Lahiri U, Warren Z, Sarkar N. An Approach to the Design of Socially Acceptable Robots for Children with Autism Spectrum Disorders. Int J Soc Robot 2010. [DOI: 10.1007/s12369-010-0063-x] [Citation(s) in RCA: 57] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
50
|
Affective Man-Machine Interface: Unveiling Human Emotions through Biosignals. BIOMEDICAL ENGINEERING SYSTEMS AND TECHNOLOGIES 2010. [DOI: 10.1007/978-3-642-11721-3_2] [Citation(s) in RCA: 37] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|