Summarization of videos by analyzing affective state of the user through crowdsource

Predicting Moral Elevation Conveyed in Danmaku Comments Using EEGs

Moral elevation, the emotion that arises when individuals observe others' moral behaviors, plays an important role in determining moral behaviors in real life. While recent research has demonstrated the potential to decode basic emotions with brain signals, there has been limited exploration of affective computing for moral elevation, an emotion related to social cognition. To address this gap, we recorded electroencephalography (EEG) signals from 23 participants while they viewed videos that were expected to elicit moral elevation. More than 30,000 danmaku comments were extracted as a crowdsourcing tagging method to label moral elevation continuously at a 1-s temporal resolution. Then, by employing power spectra features and the least absolute shrinkage and selection operator regularized regression analyses, we achieved a promising prediction performance for moral elevation (prediction r = 0.44 ± 0.11). Our findings indicate that it is possible to decode moral elevation using EEG signals. Moreover, the small-sample neural data can predict the continuous moral elevation experience conveyed in danmaku comments from a large population.

Musical Emotions Recognition Using Entropy Features and Channel Optimization Based on EEG

The dynamic of music is an important factor to arouse emotional experience, but current research mainly uses short-term artificial stimulus materials, which cannot effectively awaken complex emotions and reflect their dynamic brain response. In this paper, we used three long-term stimulus materials with many dynamic emotions inside: the "Waltz No. 2" containing pleasure and excitement, the "No. 14 Couplets" containing excitement, briskness, and nervousness, and the first movement of "Symphony No. 5 in C minor" containing passion, relaxation, cheerfulness, and nervousness. Approximate entropy (ApEn) and sample entropy (SampEn) were applied to extract the non-linear features of electroencephalogram (EEG) signals under long-term dynamic stimulation, and the K-Nearest Neighbor (KNN) method was used to recognize emotions. Further, a supervised feature vector dimensionality reduction method was proposed. Firstly, the optimal channel set for each subject was obtained by using a particle swarm optimization (PSO) algorithm, and then the number of times to select each channel in the optimal channel set of all subjects was counted. If the number was greater than or equal to the threshold, it was a common channel suitable for all subjects. The recognition results based on the optimal channel set demonstrated that each accuracy of two categories of emotions based on "Waltz No. 2" and three categories of emotions based on "No. 14 Couplets" was generally above 80%, respectively, and the recognition accuracy of four categories based on the first movement of "Symphony No. 5 in C minor" was about 70%. The recognition accuracy based on the common channel set was about 10% lower than that based on the optimal channel set, but not much different from that based on the whole channel set. This result suggested that the common channel could basically reflect the universal features of the whole subjects while realizing feature dimension reduction. The common channels were mainly distributed in the frontal lobe, central region, parietal lobe, occipital lobe, and temporal lobe. The channel number distributed in the frontal lobe was greater than the ones in other regions, indicating that the frontal lobe was the main emotional response region. Brain region topographic map based on the common channel set showed that there were differences in entropy intensity between different brain regions of the same emotion and the same brain region of different emotions. The number of times to select each channel in the optimal channel set of all 30 subjects showed that the principal component channels representing five brain regions were Fp1/F3 in the frontal lobe, CP5 in the central region, Pz in the parietal lobe, O2 in the occipital lobe, and T8 in the temporal lobe, respectively.

Affective Video Tagging Framework using Human Attention Modelling through EEG Signals

The explosion of multimedia content over the past years is not surprising, thus their efficient management and analysis methods are always in demand. The affectiveness of any multimedia content deals with analyzing human perception and cognition while watching it. Human attention is also one of the important parameters, as it describes the engagement and interestingness of the user while watching that content. Considering this aspect, a video tagging framework is proposed in which the EEG signals of participants are used to analyze human perception while watching videos. A rigorous analysis has been performed on different scalp locations and frequency rhythms of brain signals to formulate significant features corresponding to affective and interesting video content. The analysis presented in this paper shows that the extracted human attention-based features are generating promising results with the accuracy of 93.2% using SVM-based classification model which supports the applicability of the model for various BCI-based applications for automatic classification of multimedia content.

A Light-Weight Practical Framework for Feces Detection and Trait Recognition

Fecal trait examinations are critical in the clinical diagnosis of digestive diseases, and they can effectively reveal various aspects regarding the health of the digestive system. An automatic feces detection and trait recognition system based on a visual sensor could greatly alleviate the burden on medical inspectors and overcome many sanitation problems, such as infections. Unfortunately, the lack of digital medical images acquired with camera sensors due to patient privacy has obstructed the development of fecal examinations. In general, the computing power of an automatic fecal diagnosis machine or a mobile computer-aided diagnosis device is not always enough to run a deep network. Thus, a light-weight practical framework is proposed, which consists of three stages: illumination normalization, feces detection, and trait recognition. Illumination normalization effectively suppresses the illumination variances that degrade the recognition accuracy. Neither the shape nor the location is fixed, so shape-based and location-based object detection methods do not work well in this task. Meanwhile, this leads to a difficulty in labeling the images for training convolutional neural networks (CNN) in detection. Our segmentation scheme is free from training and labeling. The feces object is accurately detected with a well-designed threshold-based segmentation scheme on the selected color component to reduce the background disturbance. Finally, the preprocessed images are categorized into five classes with a light-weight shallow CNN, which is suitable for feces trait examinations in real hospital environments. The experiment results from our collected dataset demonstrate that our framework yields a satisfactory accuracy of 98.4%, while requiring low computational complexity and storage.

Ten challenges for EEG-based affective computing

The emerging field of affective computing focuses on enhancing computers’ ability to understand and appropriately respond to people’s affective states in human-computer interactions, and has revealed significant potential for a wide spectrum of applications. Recently, the electroencephalography (EEG) based affective computing has gained increasing interest for its good balance between mechanistic exploration and real-world practical application. The present work reviewed ten theoretical and operational challenges for the existing affective computing researches from an interdisciplinary perspective of information technology, psychology, and neuroscience. On the theoretical side, we suggest that researchers should be well aware of the limitations of the commonly used emotion models, and be cautious about the widely accepted assumptions on EEG-emotion relationships as well as the transferability of findings based on different research paradigms. On the practical side, we propose several operational recommendations for the challenges about data collection, feature extraction, model implementation, online system design, as well as the potential ethical issues. The present review is expected to contribute to an improved understanding of EEG-based affective computing and promote further applications.