An empirical study of machine learning techniques for affect recognition in human–robot interaction

Systematic Review of Affective Computing Techniques for Infant Robot Interaction

Research studies on social robotics and human-robot interaction have gained insights into factors that influence people’s perceptions and behaviors towards robots. However, adults’ perceptions of robots may differ significantly from those of infants. Consequently, extending this knowledge also to infants’ attitudes toward robots is a growing field of research. Indeed, infant-robot interaction (IRI) is emerging as a critical and necessary area of research as robots are increasingly used in social environments, such as caring for infants with all types of disabilities, companionship, and education. Although studies have been conducted on the ability of robots to positively engage infants, little is known about the infants’ affective state when interacting with a robot. In this systematic review, technologies for infant affective state recognition relevant to IRI applications are presented and surveyed. Indeed, adapting techniques currently employed for infant’s emotion recognition to the field of IRI results to be a complex task, since it requires timely response while not interfering with the infant’s behavior. Those aspects have a crucial impact on the selection of the emotion recognition techniques and the related metrics to be used for this purpose. Therefore, this review is intended to shed light on the advantages and the current research challenges of the infants’ affective state recognition approaches in the IRI field, elucidates a roadmap for their use in forthcoming studies as well as potentially provide support to future developments of emotion-aware robots.

Human-Computer Interaction for Recognizing Speech Emotions Using Multilayer Perceptron Classifier

Human-computer interaction (HCI) has seen a paradigm shift from textual or display-based control toward more intuitive control modalities such as voice, gesture, and mimicry. Particularly, speech has a great deal of information, conveying information about the speaker's inner condition and his/her aim and desire. While word analysis enables the speaker's request to be understood, other speech features disclose the speaker's mood, purpose, and motive. As a result, emotion recognition from speech has become critical in current human-computer interaction systems. Moreover, the findings of the several professions involved in emotion recognition are difficult to combine. Many sound analysis methods have been developed in the past. However, it was not possible to provide an emotional analysis of people in a live speech. Today, the development of artificial intelligence and the high performance of deep learning methods bring studies on live data to the fore. This study aims to detect emotions in the human voice using artificial intelligence methods. One of the most important requirements of artificial intelligence works is data. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) open-source dataset was used in the study. The RAVDESS dataset contains more than 2000 data recorded as speeches and songs by 24 actors. Data were collected for eight different moods from the actors. It was aimed at detecting eight different emotion classes, including neutral, calm, happy, sad, angry, fearful, disgusted, and surprised moods. The multilayer perceptron (MLP) classifier, a widely used supervised learning algorithm, was preferred for classification. The proposed model's performance was compared with that of similar studies, and the results were evaluated. An overall accuracy of 81% was obtained for classifying eight different emotions by using the proposed model on the RAVDESS dataset.

EEG-based human emotion recognition using entropy as a feature extraction measure

Many studies on brain-computer interface (BCI) have sought to understand the emotional state of the user to provide a reliable link between humans and machines. Advanced neuroimaging methods like electroencephalography (EEG) have enabled us to replicate and understand a wide range of human emotions more precisely. This physiological signal, i.e., EEG-based method is in stark comparison to traditional non-physiological signal-based methods and has been shown to perform better. EEG closely measures the electrical activities of the brain (a nonlinear system) and hence entropy proves to be an efficient feature in extracting meaningful information from raw brain waves. This review aims to give a brief summary of various entropy-based methods used for emotion classification hence providing insights into EEG-based emotion recognition. This study also reviews the current and future trends and discusses how emotion identification using entropy as a measure to extract features, can accomplish enhanced identification when using EEG signal.

CONFIRMATION OF THE CLINICAL VALUE AND THE EFFECT OF SOUND BIOFEEDBACK ON THE AUTONOMIC NERVOUS SYSTEM THROUGH HEART RATE VARIABILITY ANALYSIS

The purpose of this study is to suggest sound biofeedback, which is a new technique of early stress relief effect by observing change in the heart rate variability (HRV). The sound biofeedback imitating heart rate of the comfortable and stress state is termed parasympathetic stimulation sound (PSS) and sympathetic stimulation sound (SSS), respectively. Twelve subjects were selected without previous history of cardiovascular diseases and mental illness, such as arrhythmia, myocardial infarction, depression and panic disorder. To confirm the changes in the low-frequency (LF), high-frequency (HF) and LF/HF values of HRV as stress evaluation indicators, the HRV of subjects was measured by photoplethysmogram. Signals were processed using the peak detect algorithm, and fast Fourier transform. Results were obtained using power specific densities. During the PSS stimulation, the LF/HF tended to decrease generally. On the other hand, during the SSS stimulation, LF/HF tended to increase. The LF/HF Mean change value ([Formula: see text]) using the PSS stimulation is similar to the effect of Transcutaneous Vagal Nerve Stimulation (tVNS). In addition, the quantitative effect of sound biofeedback was confirmed by judging changes in the parasympathetic and sympathetic nerves in the autonomic nervous system (ANS) through [Formula: see text]-score normalized data. These experimental results suggest that sound biofeedback has the same stimulation location and clinical applicability as tVNS. As a result, sound biofeedback may be used as a new method for stress reduction.

M. Facial geometric feature extraction based emotional expression classification using machine learning algorithms

Emotion plays a significant role in interpersonal communication and also improving social life. In recent years, facial emotion recognition is highly adopted in developing human-computer interfaces (HCI) and humanoid robots. In this work, a triangulation method for extracting a novel set of geometric features is proposed to classify six emotional expressions (sadness, anger, fear, surprise, disgust, and happiness) using computer-generated markers. The subject's face is recognized by using Haar-like features. A mathematical model has been applied to positions of eight virtual markers in a defined location on the subject's face in an automated way. Five triangles are formed by manipulating eight markers' positions as an edge of each triangle. Later, these eight markers are uninterruptedly tracked by Lucas- Kanade optical flow algorithm while subjects' articulating facial expressions. The movement of the markers during facial expression directly changes the property of each triangle. The area of the triangle (AoT), Inscribed circle circumference (ICC), and the Inscribed circle area of a triangle (ICAT) are extracted as features to classify the facial emotions. These features are used to distinguish six different facial emotions using various types of machine learning algorithms. The inscribed circle area of the triangle (ICAT) feature gives a maximum mean classification rate of 98.17% using a Random Forest (RF) classifier compared to other features and classifiers in distinguishing emotional expressions.

Oscillatory EEG Signatures of Affective Processes during Interaction with Adaptive Computer Systems

Affect monitoring is being discussed as a novel strategy to make adaptive systems more user-oriented. Basic knowledge about oscillatory processes and functional connectivity underlying affect during naturalistic human-computer interactions (HCI) is, however, scarce. This study assessed local oscillatory power entrainment and distributed functional connectivity in a close-to-naturalistic HCI-paradigm. Sixteen participants interacted with a simulated assistance system which deliberately evoked positive (supporting goal-achievement) and negative (impeding goal-achievement) affective reactions. Electroencephalography (EEG) was used to examine the reactivity of the cortical system during the interaction by studying both event-related (de-)synchronization (ERD/ERS) and event-related functional coupling of cortical networks towards system-initiated assistance. Significantly higher α-band and β-band ERD in centro-parietal and parieto-occipital regions and β-band ERD in bi-lateral fronto-central regions were observed during impeding system behavior. Supportive system behavior activated significantly higher γ-band ERS in bi-hemispheric parietal-occipital regions. This was accompanied by functional coupling of remote β-band and γ-band activity in the medial frontal, left fronto-central and parietal regions, respectively. Our findings identify oscillatory signatures of positive and negative affective processes as reactions to system-initiated assistance. The findings contribute to the development of EEG-based neuroadaptive assistance loops by suggesting a non-obtrusive method for monitoring affect in HCI.

GIS-Based Emotional Computing: A Review of Quantitative Approaches to Measure the Emotion Layer of Human–Environment Relationships

In recent years, with the growing accessibility of abundant contextual emotion information, which is benefited by the numerous georeferenced user-generated content and the maturity of artificial intelligence (AI)-based emotional computing technics, the emotion layer of human–environment relationship is proposed for enriching traditional methods of various related disciplines such as urban planning. This paper proposes the geographic information system (GIS)-based emotional computing concept, which is a novel framework for applying GIS methods to collective human emotion. The methodology presented in this paper consists of three key steps: (1) collecting georeferenced data containing emotion and environment information such as social media and official sites, (2) detecting emotions using AI-based emotional computing technics such as natural language processing (NLP) and computer vision (CV), and (3) visualizing and analyzing the spatiotemporal patterns with GIS tools. This methodology is a great synergy of multidisciplinary cutting-edge techniques, such as GIScience, sociology, and computer science. Moreover, it can effectively and deeply explore the connection between people and their surroundings with the help of GIS methods. Generally, the framework provides a standard workflow to calculate and analyze the new information layer for researchers, in which a measured human-centric perspective onto the environment is possible.

Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges

The ability to detect users' emotions for the purpose of emotion engineering is currently one of the main endeavors of machine learning in affective computing. Among the more common approaches to emotion detection are methods that rely on electroencephalography (EEG), facial image processing and speech inflections. Although eye-tracking is fast in becoming one of the most commonly used sensor modalities in affective computing, it is still a relatively new approach for emotion detection, especially when it is used exclusively. In this survey paper, we present a review on emotion recognition using eye-tracking technology, including a brief introductory background on emotion modeling, eye-tracking devices and approaches, emotion stimulation methods, the emotional-relevant features extractable from eye-tracking data, and most importantly, a categorical summary and taxonomy of the current literature which relates to emotion recognition using eye-tracking. This review concludes with a discussion on the current open research problems and prospective future research directions that will be beneficial for expanding the body of knowledge in emotion detection using eye-tracking as the primary sensor modality.

Towards Mixed-Initiative Human-Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction

The design of human–robot interactions is a key challenge to optimize operational performance. A promising approach is to consider mixed-initiative interactions in which the tasks and authority of each human and artificial agents are dynamically defined according to their current abilities. An important issue for the implementation of mixed-initiative systems is to monitor human performance to dynamically drive task allocation between human and artificial agents (i.e., robots). We, therefore, designed an experimental scenario involving missions whereby participants had to cooperate with a robot to fight fires while facing hazards. Two levels of robot automation (manual vs. autonomous) were randomly manipulated to assess their impact on the participants’ performance across missions. Cardiac activity, eye-tracking, and participants’ actions on the user interface were collected. The participants performed differently to an extent that we could identify high and low score mission groups that also exhibited different behavioral, cardiac and ocular patterns. More specifically, our findings indicated that the higher level of automation could be beneficial to low-scoring participants but detrimental to high-scoring ones, and vice versa. In addition, inter-subject single-trial classification results showed that the studied behavioral and physiological features were relevant to predict mission performance. The highest average balanced accuracy (74%) was reached using the features extracted from all input devices. These results suggest that an adaptive HRI driving system, that would aim at maximizing performance, would be capable of analyzing such physiological and behavior markers online to further change the level of automation when it is relevant for the mission purpose.

Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games

Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments ( n = 20 and n = 62 ) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games.

Breathable and Skin-Mountable Strain Sensor with Tunable Stretchability, Sensitivity, and Linearity via Surface Strain Delocalization for Versatile Skin Activities' Recognition

Metal film/elastomer-based strain sensors usually exhibit small rupture strain (<5%) because of the strain localization and necking effect of the metal film under tension. To achieve both high stretchability and wide linear region is still challenging for metal film-based strain sensors. Here, we propose a low-cost yet effective strategy for fabricating ultrathin, breathable, and skin-mountable strain sensors with high sensitivity (gauge factor from 7.2 to 474.8), high stretchability (up to 140%), and good linearity by regulating the surface strain delocalization in the metal film on elastomer substrate. On the basis of this phenomenon of strain delocalization, the sensitivity and linearity are further enhanced based on a novel diffraction-induced Au film with gradient thickness. Meanwhile, by means of the strain redistribution and Poisson effect, a novel biaxial strain sensor is designed for recognition of complex human motion. On the basis of the enhanced stretchability, linearity, skin-mountable, and breathable properties, the low-cost metal film-based strain sensors can be broadened as disposable wearables for human motion detection, emotional expression recognition, human interaction, and virtual reality.

The Mythos of Model Interpretability

Supervised machine-learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world?

Human-centric predictive model of task difficulty for human-in-the-loop control tasks

Quantitatively measuring the difficulty of a manipulation task in human-in-the-loop control systems is ill-defined. Currently, systems are typically evaluated through task-specific performance measures and post-experiment user surveys; however, these methods do not capture the real-time experience of human users. In this study, we propose to analyze and predict the difficulty of a bivariate pointing task, with a haptic device interface, using human-centric measurement data in terms of cognition, physical effort, and motion kinematics. Noninvasive sensors were used to record the multimodal response of human user for 14 subjects performing the task. A data-driven approach for predicting task difficulty was implemented based on several task-independent metrics. We compare four possible models for predicting task difficulty to evaluated the roles of the various types of metrics, including: (I) a movement time model, (II) a fusion model using both physiological and kinematic metrics, (III) a model only with kinematic metrics, and (IV) a model only with physiological metrics. The results show significant correlation between task difficulty and the user sensorimotor response. The fusion model, integrating user physiology and motion kinematics, provided the best estimate of task difficulty (R2 = 0.927), followed by a model using only kinematic metrics (R2 = 0.921). Both models were better predictors of task difficulty than the movement time model (R2 = 0.847), derived from Fitt's law, a well studied difficulty model for human psychomotor control.

Respiratory Variability during Different Auditory Stimulation Periods in Schizophrenia Patients

Background: Schizophrenic patients are known to have difficulty processing emotions and to exhibit impairment in stimuli discrimination. However, there is limited knowledge regarding their physiological responsivity to auditory stimuli.

Objectives: The purpose of this study was to compare the respiratory effects of two types of auditory stimuli with emotional content, classical Turkish music (CTM) and white noise (WN), on schizophrenia patients and healthy control subjects.

Methods: Forty-six individuals participated in the experiment, and respiratory signals derived from a strain-gauge were recorded. Two important respiratory patterns, respiration rate and depth, were analyzed.

Results: The results indicated that the patients presented a significantly higher respiration rate than control subjects during the initial baseline and WN exposure periods. Although CTM evoked an increase in respiration rates and a decrease in respiration depths in the control group, no significant differences were found during the stimulation periods in the patient group. The respiration rate was lower in the post-stimulation period than during the initial baseline period, and no respiration depth differences were found for the WN, music or post-stimulation periods in the schizophrenia group. Patients exhibited a greater respiration depth than the control subjects over all periods; however, a significant difference between the patient and control groups was obtained in the second resting condition and CTM exposure period. Furthermore, to analyze the effect of symptom severity on respiratory patterns, patients were divided into two classes according to their Positive and Negative Syndrome Scale score.

Conclusions: Further studies are needed to correlate respiratory differences with emotionally evocative stimuli and to refine our understanding of the dynamics of these types of stimuli in relation to clinical state and medication effects.

Supervised learning methods for pathological arterial pulse wave differentiation: A SVM and neural networks approach

OBJECTIVE

The main goal of this study was to develop an automatic method based on supervised learning methods, able to distinguish healthy from pathologic arterial pulse wave (APW), and those two from noisy waveforms (non-relevant segments of the signal), from the data acquired during a clinical examination with a novel optical system.

MATERIALS AND METHODS

The APW dataset analysed was composed by signals acquired in a clinical environment from a total of 213 subjects, including healthy volunteers and non-healthy patients. The signals were parameterised by means of 39pulse features: morphologic, time domain statistics, cross-correlation features, wavelet features. Multiclass Support Vector Machine Recursive Feature Elimination (SVM RFE) method was used to select the most relevant features. A comparative study was performed in order to evaluate the performance of the two classifiers: Support Vector Machine (SVM) and Artificial Neural Network (ANN).

RESULTS AND DISCUSSION

SVM achieved a statistically significant better performance for this problem with an average accuracy of 0.9917±0.0024 and a F-Measure of 0.9925±0.0019, in comparison with ANN, which reached the values of 0.9847±0.0032 and 0.9852±0.0031 for Accuracy and F-Measure, respectively. A significant difference was observed between the performances obtained with SVM classifier using a different number of features from the original set available.

CONCLUSION

The comparison between SVM and NN allowed reassert the higher performance of SVM. The results obtained in this study showed the potential of the proposed method to differentiate those three important signal outcomes (healthy, pathologic and noise) and to reduce bias associated with clinical diagnosis of cardiovascular disease using APW.

Beat-ID: Towards a computationally low-cost single heartbeat biometric identity check system based on electrocardiogram wave morphology

In recent years, safer and more reliable biometric methods have been developed. Apart from the need for enhanced security, the media and entertainment sectors have also been applying biometrics in the emerging market of user-adaptable objects/systems to make these systems more user-friendly. However, the complexity of some state-of-the-art biometric systems (e.g., iris recognition) or their high false rejection rate (e.g., fingerprint recognition) is neither compatible with the simple hardware architecture required by reduced-size devices nor the new trend of implementing smart objects within the dynamic market of the Internet of Things (IoT). It was recently shown that an individual can be recognized by extracting features from their electrocardiogram (ECG). However, most current ECG-based biometric algorithms are computationally demanding and/or rely on relatively large (several seconds) ECG samples, which are incompatible with the aforementioned application fields. Here, we present a computationally low-cost method (patent pending), including simple mathematical operations, for identifying a person using only three ECG morphology-based characteristics from a single heartbeat. The algorithm was trained/tested using ECG signals of different duration from the Physionet database on more than 60 different training/test datasets. The proposed method achieved maximal averaged accuracy of 97.450% in distinguishing each subject from a ten-subject set and false acceptance and rejection rates (FAR and FRR) of 5.710±1.900% and 3.440±1.980%, respectively, placing Beat-ID in a very competitive position in terms of the FRR/FAR among state-of-the-art methods. Furthermore, the proposed method can identify a person using an average of 1.020 heartbeats. It therefore has FRR/FAR behavior similar to obtaining a fingerprint, yet it is simpler and requires less expensive hardware. This method targets low-computational/energy-cost scenarios, such as tiny wearable devices (e.g., a smart object that automatically adapts its configuration to the user). A hardware proof-of-concept implementation is presented as an annex to this paper.

An efficient automatic workload estimation method based on electrodermal activity using pattern classifier combinations

Automatic workload estimation has received much attention because of its application in error prevention, diagnosis, and treatment of neural system impairment. The development of a simple but reliable method using minimum number of psychophysiological signals is a challenge in automatic workload estimation. To address this challenge, this paper presented three different decomposition techniques (Fourier, cepstrum, and wavelet transforms) to analyze electrodermal activity (EDA). The efficiency of various statistical and entropic features was investigated and compared. To recognize different levels of an arithmetic task, the features were processed by principal component analysis and machine-learning techniques. These methods have been incorporated into a workload estimation system based on two types: feature-level and decision-level combinations. The results indicated the reliability of the method for automatic and real-time inference of psychological states. This method provided a quantitative estimation of the workload levels and a bias-free evaluation approach. The high-average accuracy of 90% and cost effective requirement were the two important attributes of the proposed workload estimation system. New entropic features were proved to be more sensitive measures for quantifying time and frequency changes in EDA. The effectiveness of these measures was also compared with conventional tonic EDA measures, demonstrating the superiority of the proposed method in achieving accurate estimation of workload levels.

Biomusic: An Auditory Interface for Detecting Physiological Indicators of Anxiety in Children

For children with profound disabilities affecting communication, it can be extremely challenging to identify salient emotions such as anxiety. If left unmanaged, anxiety can lead to hypertension, cardiovascular disease, and other psychological diagnoses. Physiological signals of the autonomic nervous system are indicative of anxiety, but can be difficult to interpret for non-specialist caregivers. This paper evaluates an auditory interface for intuitive detection of anxiety from physiological signals. The interface, called "Biomusic," maps physiological signals to music (i.e., electrodermal activity to melody; skin temperature to musical key; heart rate to drum beat; respiration to a "whooshing" embellishment resembling the sound of an exhalation). The Biomusic interface was tested in two experiments. Biomusic samples were generated from physiological recordings of typically developing children (n = 10) and children with autism spectrum disorders (n = 5) during relaxing and anxiety-provoking conditions. Adult participants (n = 16) were then asked to identify "anxious" or "relaxed" states by listening to the samples. In a classification task with 30 Biomusic samples (1 relaxed state, 1 anxious state per child), classification accuracy, sensitivity, and specificity were 80.8% [standard error (SE) = 2.3], 84.9% (SE = 3.0), and 76.8% (SE = 3.9), respectively. Participants were able to form an early and accurate impression of the anxiety state within 12.1 (SE = 0.7) seconds of hearing the Biomusic with very little training (i.e., < 10 min) and no contextual information. Biomusic holds promise for monitoring, communication, and biofeedback systems for anxiety management.

Boredom in the Workplace: A New Look at an Old Problem

OBJECTIVE

We review historical and more recent efforts in boredom research and related fields. A framework is presented that organizes the various facets of boredom, particularly in supervisory control settings, and research gaps and future potential areas for study are highlighted.

BACKGROUND

Given the ubiquity of boredom across a wide spectrum of work environments--exacerbated by increasingly automated systems that remove humans from direct, physical system interaction and possibly increasing tedium in the workplace--there is a need not only to better understand the multiple facets of boredom in work environments but to develop targeted mitigation strategies.

METHOD

To better understand the relationships between the various influences and outcomes of boredom, a systems-based framework, called the Boredom Influence Diagram, is proposed that describes various elements of boredom and their interrelationships.

RESULTS

Boredom is closely related to vigilance, attention management, and task performance. This review highlights the need to develop more naturalistic experiments that reflect the characteristics of a boring work environment.

CONCLUSION

With the increase in automation, boredom in the workplace will likely become a more prevalent issue for motivation and retention. In addition, developing continuous measures of boredom based on physiological signals is critical.

APPLICATION

Personnel selection and improvements in system and task design can potentially mitigate boredom. However, more work is needed to develop and evaluate other potential interventions.

Optimal Geometrical Set for Automated Marker Placement to Virtualized Real-Time Facial Emotions

In recent years, real-time face recognition has been a major topic of interest in developing intelligent human-machine interaction systems. Over the past several decades, researchers have proposed different algorithms for facial expression recognition, but there has been little focus on detection in real-time scenarios. The present work proposes a new algorithmic method of automated marker placement used to classify six facial expressions: happiness, sadness, anger, fear, disgust, and surprise. Emotional facial expressions were captured using a webcam, while the proposed algorithm placed a set of eight virtual markers on each subject's face. Facial feature extraction methods, including marker distance (distance between each marker to the center of the face) and change in marker distance (change in distance between the original and new marker positions), were used to extract three statistical features (mean, variance, and root mean square) from the real-time video sequence. The initial position of each marker was subjected to the optical flow algorithm for marker tracking with each emotional facial expression. Finally, the extracted statistical features were mapped into corresponding emotional facial expressions using two simple non-linear classifiers, K-nearest neighbor and probabilistic neural network. The results indicate that the proposed automated marker placement algorithm effectively placed eight virtual markers on each subject's face and gave a maximum mean emotion classification rate of 96.94% using the probabilistic neural network.

Locality sensitivity discriminant analysis-based feature ranking of human emotion actions recognition

[Purpose] Computational intelligence similar to pattern recognition is frequently confronted with high-dimensional data. Therefore, the reduction of the dimensionality is critical to make the manifold features amenable. Procedures that are analytically or computationally manageable in smaller amounts of data and low-dimensional space can become important to produce a better classification performance. [Methods] Thus, we proposed two stage reduction techniques. Feature selection-based ranking using information gain (IG) and Chi-square (Chisq) are used to identify the best ranking of the features selected for emotion classification in different actions including knocking, throwing, and lifting. Then, feature reduction-based locality sensitivity discriminant analysis (LSDA) and principal component analysis (PCA) are used to transform the selected feature to low-dimensional space. Two-stage feature selection-reduction methods such as IG-PCA, IG-LSDA, Chisq-PCA, and Chisq-LSDA are proposed. [Results] The result confirms that applying feature ranking combined with a dimensional-reduction method increases the performance of the classifiers. [Conclusion] The dimension reduction was performed using LSDA by denoting the features of the highest importance determined using IG and Chisq to not only improve the effectiveness but also reduce the computational time.

Adaptation of task difficulty in rehabilitation exercises based on the user's motor performance and physiological responses

Although robot-assisted rehabilitation regimens are as effective, functionally, as conventional therapies, they still lack features to increase patients' engagement in the regimen. Providing rehabilitation tasks at a "desirable difficulty" is one of the ways to address this issue and increase the motivation of a patient to continue with the therapy program. Then the problem is to design a system that is capable of estimating the user's desirable difficulty, and ultimately, modifying the task based on this prediction. In this paper we compared the performance of three machine learning algorithms in predicting a user's desirable difficulty during a typical reaching motion rehabilitation task. Different levels of error amplification were used as different levels of task difficulty. We explored the usefulness of using participants' motor performance and physiological signals during the reaching task in prediction of their desirable difficulties. Results showed that a Neural Network approach gives higher prediction accuracy in comparison with models based on k-Nearest Neighbor and Discriminant Analysis methods.

Feature selection for speech emotion recognition in Spanish and Basque: on the use of machine learning to improve human-computer interaction

Study of emotions in human–computer interaction is a growing research area. This paper shows an attempt to select the most significant features for emotion recognition in spoken Basque and Spanish Languages using different methods for feature selection. RekEmozio database was used as the experimental data set. Several Machine Learning paradigms were used for the emotion classification task. Experiments were executed in three phases, using different sets of features as classification variables in each phase. Moreover, feature subset selection was applied at each phase in order to seek for the most relevant feature subset. The three phases approach was selected to check the validity of the proposed approach. Achieved results show that an instance-based learning algorithm using feature subset selection techniques based on evolutionary algorithms is the best Machine Learning paradigm in automatic emotion recognition, with all different feature sets, obtaining a mean of 80,05% emotion recognition rate in Basque and a 74,82% in Spanish. In order to check the goodness of the proposed process, a greedy searching approach (FSS-Forward) has been applied and a comparison between them is provided. Based on achieved results, a set of most relevant non-speaker dependent features is proposed for both languages and new perspectives are suggested.

Similarities and differences of emotions in human-machine and human-human interactions: what kind of emotions are relevant for future companion systems?

Cognitive-technical intelligence is envisioned to be constantly available and capable of adapting to the user's emotions. However, the question is: what specific emotions should be reliably recognised by intelligent systems? Hence, in this study, we have attempted to identify similarities and differences of emotions between human-human (HHI) and human-machine interactions (HMI). We focused on what emotions in the experienced scenarios of HMI are retroactively reflected as compared with HHI. The sample consisted of N = 145 participants, who were divided into two groups. Positive and negative scenario descriptions of HMI and HHI were given by the first and second groups, respectively. Subsequently, the participants evaluated their respective scenarios with the help of 94 adjectives relating to emotions. The correlations between the occurrences of emotions in the HMI versus HHI were very high. The results do not support the statement that only a few emotions in HMI are relevant.

A step towards developing adaptive robot-mediated intervention architecture (ARIA) for children with autism

Emerging technology, especially robotic technology, has been shown to be appealing to children with autism spectrum disorders (ASD). Such interest may be leveraged to provide repeatable, accurate and individualized intervention services to young children with ASD based on quantitative metrics. However, existing robot-mediated systems tend to have limited adaptive capability that may impact individualization. Our current work seeks to bridge this gap by developing an adaptive and individualized robot-mediated technology for children with ASD. The system is composed of a humanoid robot with its vision augmented by a network of cameras for real-time head tracking using a distributed architecture. Based on the cues from the child's head movement, the robot intelligently adapts itself in an individualized manner to generate prompts and reinforcements with potential to promote skills in the ASD core deficit area of early social orienting. The system was validated for feasibility, accuracy, and performance. Results from a pilot usability study involving six children with ASD and a control group of six typically developing (TD) children are presented.

STATISTICAL PREDICTION OF EMOTIONAL STATES BY PHYSIOLOGICAL SIGNALS WITH MANOVA AND MACHINE LEARNING

For the importance of communication between human and machine interface, it would be valuable to develop an implement which has the ability to recognize emotional states. In this paper, we proposed an approach which can deal with the daily dependence and personal dependence in the data of multiple subjects and samples. 30 features were extracted from the physiological signals of subject for three states of emotion. The physiological signals measured were: electrocardiogram (ECG), skin temperature (SKT) and galvanic skin response (GSR). After removing the daily dependence and personal dependence by the statistical technique of MANOVA, six machine learning methods including Bayesian network learning, naive Bayesian classification, SVM, decision tree of C4.5, Logistic model and K-nearest-neighbor (KNN) were implemented to differentiate the emotional states. The results showed that Logistic model gives the best classification accuracy and the statistical technique of MANOVA can significantly improve the performance of all six machine learning methods in emotion recognition system.