Spontaneous blink rate as an index of attention and emotion during film clips viewing

Rapid Voluntary Blinking as a Clinical Marker of Parkinson's Disease

Reduced spontaneous blinking is a recognized Parkinson's disease (PD) feature. In contrast, voluntary blinking has been less studied and might serve as a measurable marker of facial bradykinesia. We tested 31 PD patients and 31 controls. Participants were filmed during conversation and a rapid blinking task. Both tasks were videorecorded to count the number of blinks per second. PD patients had lower blink rates. Rapid blinking accurately discriminated between groups with 77% sensitivity and 71% specificity. To conclude, rapid blinking may be a simple and quantifiable task of facial bradykinesia.

Ingestible pills reveal gastric correlates of emotions

Although it is generally held that gastrointestinal (GI) signals are related to emotions, direct evidence for such a link is currently lacking. One of the reasons why the internal milieu of the GI system is poorly investigated is because visceral organs are difficult to access and monitor. To directly measure the influence of endoluminal markers of GI activity on the emotional experience, we asked a group of healthy male participants to ingest a pill that measured pH, pressure, and temperature of their GI tract while they watched video clips that consistently induced disgust, fear, happiness, sadness, or a control neutral state. In addition to the objective physiological markers of GI activity, subjective ratings of perceived emotions and visceral (i.e. gastric, respiratory and cardiac) sensations were recorded, as well as changes in heart rate (HR), heart rate variability (HRV) and spontaneous eyes blinks as non-gastric behavioral and autonomic markers of the emotional experience. We found that when participants observed fearful and disgusting video clips, they reported to perceive not only cardiac and respiratory sensations but also gastric sensations, such as nausea. Moreover, we found that there was a clear relation between the physiology of the stomach and the perceived emotions. Specifically, when disgusting video clips were displayed, the more acidic the pH, the more participants reported feelings of disgust and fear; the less acidic the pH, the more they reported happiness. Complementing the results found in the deep gastric realm, we found that disgusting stimuli induced a significant increase in HRV compared to the neutral scenarios, and together with fearful video clips a decrease in HR. Our findings suggest that gastric signals contribute to unique emotional states and that ingestible pills may open new avenues for exploring the deep-body physiology of emotions.

Lemniscate of Bernoulli's map quantifiers: innovative measures for EEG emotion recognition

Thanks to the advent of affective computing, designing an automatic human emotion recognition system for clinical and non-clinical applications has attracted the attention of many researchers. Currently, multi-channel electroencephalogram (EEG)-based emotion recognition is a fundamental but challenging issue. This experiment envisioned developing a new scheme for automated EEG affect recognition. An innovative nonlinear feature engineering approach was presented based on Lemniscate of Bernoulli's Map (LBM), which belongs to the family of chaotic maps, in line with the EEG's nonlinear nature. As far as the authors know, LBM has not been utilized for biological signal analysis. Next, the map was characterized using several graphical indices. The feature vector was imposed on the feature selection algorithm while evaluating the role of the feature vector dimension on emotion recognition rates. Finally, the efficiency of the features on emotion recognition was appraised using two conventional classifiers and validated using the Database for Emotion Analysis using Physiological signals (DEAP) and SJTU Emotion EEG Dataset-IV (SEED-IV) benchmark databases. The experimental results showed a maximum accuracy of 92.16% for DEAP and 90.7% for SEED-IV. Achieving higher recognition rates compared to the state-of-art EEG emotion recognition systems suggest the proposed method based on LBM could have potential both in characterizing bio-signal dynamics and detecting affect-deficit disorders.

Non-invasive detection of mental fatigue in construction equipment operators through geometric measurements of facial features

INTRODUCTION

Prolonged operation of construction equipment could lead to mental fatigue, which can increase the chances of human error-related accidents as well as operators' ill-health. The objective detection of operators' mental fatigue is crucial for reducing accident risk and ensuring operator health. Electroencephalography, photoplethysmography, electrodermal activity, and eye-tracking technology have been used to mitigate this issue. These technologies are invasive and wearable sensors that can cause irritation and discomfort. Geometric measurements of facial features can serve as a noninvasive alternative approach. Its application in detecting mental fatigue of construction equipment operators has not been reported in the literature. Although the application of facial features has been widespread in other domains, such as drivers and other occupation scenarios, their ecological validity for construction excavator operators remains a knowledge gap.

METHOD

This study proposed employing geometric measurements of facial features to detect mental fatigue in construction equipment operators' facial features. In this study, seventeen operators performed excavation operations. Mental fatigue was labeled subjectively and objectively using NASA-TLX scores and EDA values. Based on geometric measurements, facial features (eyebrow, mouth outer, mouth corners, head motion, eye area, and face area) were extracted.

RESULTS

The results showed that there was significant difference in the measured metrics for high fatigue compared to low fatigue. Specifically, the most noteworthy variation was for the eye and face area metrics, with mean differences of 45.88% and 26.9%, respectively.

CONCLUSIONS

The findings showed that geometrical measurements of facial features are a useful, noninvasive approach for detecting the mental fatigue of construction equipment operators.

Arousal, interindividual differences and temporal binding a psychophysiological study

The sense of agency varies as a function of arousal in negative emotional contexts. As yet, it is unknown whether the same is true for positive affect, and how inter-individual characteristics might predict these effects. Temporal binding, an implicit measure of the sense of agency, was measured in 59 participants before and after watching either an emotionally neutral film clip or a positive film clip with high or low arousal. Analyses included participants' individual differences in subjective affective ratings, physiological arousal (pupillometry, skin conductance, heart rate), striatal dopamine levels via eye blink rates, and psychopathy. Linear mixed models showed that sexual arousal decreased temporal binding whereas calm pleasure had no facilitation effect on binding. Striatal dopamine levels were positively linked whereas subjective and physiological arousal may be negatively associated with binding towards actions. Psychopathic traits reduced the effect of high arousal on binding towards actions. These results provide evidence that individual differences influence the extent to which the temporal binding is affected by high arousing states with positive valence.

Blink-related arousal network surges are shaped by cortical vigilance states

The vigilance state and the excitability of cortical networks impose wide-range effects on brain dynamics that arousal surges could promptly modify. We previously reported an association between spontaneous eye-blinks and BOLD activation in the brain arousal ascending network (AAN) and in thalamic nuclei based on 3T MR resting state brain images. Here we aimed to replicate our analyses using 7T MR images in a larger cohort of participants collected from the Human Connectome Project (HCP), which also contained simultaneous eye-tracking recordings, and to assess the interaction between the blink-associated arousal surges and the vigilance states. For this purpose, we compared blink associated BOLD activity under a vigilant versus a drowsy state, a classification made based on the pupillary data obtained during the fMRI scans. We conducted two main analyses: i) Cross-correlation analysis between the BOLD signal and blink events (eye blink time-series were convolved with the canonical and also with the temporal derivative of the Hemodynamic Response Function, HRF) within preselected regions of interests (ROIs) (i.e., brainstem AAN, thalamic and cerebellar nuclei) together with an exploratory voxel-wise analyses to assess the whole-brain, and ii) blink-event analysis of the BOLD signals to reveal the signal changes onset to the blinks in the preselected ROIs. Consistent with our prior findings on 3T MRI, we showed significant positive cross correlations between BOLD peaks in brainstem and thalamic nuclei that preceded or were overlapping with blink moments and that sharply decreased post-blink. Whole brain analysis revealed blink-related activation that was strongest in cerebellum, insula, lateral geniculate nucleus (LGN) and visual cortex. Drowsiness impacted HRF BOLD (enhancing it), time-to-peak (delaying it) and post-blink BOLD activity (accentuating decreases). Responses in the drowsy state could be related to the differences in the excitability of cortical, subcortical and cerebellar tissue, such that cerebellar and thalamic regions involved in visual attention processing were more responsive for the vigilant state, but AAN ROIs, as well as cerebellar and thalamic ROIs connected to pre-motor, frontal, temporal and DMN regions were less responsive. Such qualitative and quantitative differences in the blink related BOLD signal changes could reflect delayed cortical processing and the ineffectiveness of arousal surges during states of drowsiness. Future studies that manipulate arousal are needed to corroborate a mechanistic interaction of arousal surges with vigilance states and cortical excitability.

The psychophysiology of Mastermind: Characterizing response times and blinking in a high-stakes television game show

Television game shows have proven to be a valuable resource for studying human behavior under conditions of high stress and high stakes. However, previous work has focused mostly on choices-ignoring much of the rich visual information that is available on screen. Here, we take a first step to extracting more of this information by investigating the response times and blinking of contestants in the BBC show Mastermind. In Mastermind, contestants answer rapid-fire quiz questions while a camera slowly zooms in on their faces. By labeling contestants' behavior and blinks from 25 episodes, we asked how accuracy, response times, and blinking varied over the course of the game. For accuracy and response times, we tested whether contestants responded more accurately and more slowly after an error-exhibiting the "post-error increase in accuracy" and "post-error slowing" which has been repeatedly observed in the lab. For blinking, we tested whether blink rates varied according to the cognitive demands of the game-decreasing during periods of cognitive load, such as when pondering a response, and increasing at event boundaries in the task, such as the start of a question. In contrast to the lab, evidence for post-error changes in accuracy and response time was weak, with only marginal effects observed. In line with the lab, blinking varied over the course of the game much as we predicted. Overall, our findings demonstrate the potential of extracting dynamic signals from game shows to study the psychophysiology of behavior in the real world.

Attentional modulation of eye blinking is altered by sex, age, and task structure

Spontaneous eye blinking is gaining popularity as a proxy for higher cognitive functions, as it is readily modulated by both environmental demands and internal processes. Prior studies were impoverished in sample size, sex representation and age distribution, making it difficult to establish a complete picture of the behavior. Here we present eye-tracking data from a large cohort of normative participants (n=604, 393 F, aged 5-93 years) performing two tasks: one with structured, discrete trials (interleaved pro/anti-saccade task; IPAST) and one with a less structured, continuous organization in which participants watch movies (free-viewing; FV). Sex- and age-based analyses revealed that females had higher blink rates between the ages of 22 and 58 years in the IPAST, and 22 and 34 years in FV. We derived a continuous measure of blink probability to reveal behavioral changes driven by stimulus appearance in both paradigms. In the IPAST, blinks were suppressed near stimulus appearance, particularly on correct anti-saccade trials, which we attribute to the stronger inhibitory control required for anti-saccades compared to pro-saccades. In FV, blink suppression occurred immediately after scene changes, and the effect was sustained on scenes where gaze clustered among participants (indicating engagement of attention). Females were more likely than males to blink during appearance of novel stimuli in both tasks, but only within the age bin of 18-44 years. The consistency of blink patterns in each paradigm endorses blinking as a sensitive index for changes in visual processing and attention, while sex and age differences drive interindividual variability.Significance Statement Eye-tracking is becoming useful as a non-invasive tool for detecting preclinical markers of neurological and psychiatric disease. Blinks are understudied despite being an important supplement to saccade and pupil eye-tracking metrics. The present study is a crucial step in developing a healthy baseline for blink behavior to compare to clinical groups. While many prior blink studies suffered from small sample sizes with relatively low age- and sex-diversity (review by Jongkees & Colzato, 2016), our large cohort of healthy participants has permitted a more detailed analysis of sex and age effects in blink behavior. Furthermore, our analysis techniques are robust to temporal changes in blink probability, greatly clarifying the relationship between blinking, visual processing, and inhibitory control mechanisms on visual tasks.

Investigation of different ML approaches in classification of emotions induced by acute stress

Background

Machine learning is becoming a common tool in monitoring emotion. However, methodological studies of the processing pipeline are scarce, especially ones using subjective appraisals as ground truth.

New method

A novel protocol was used to induce cognitive load and physical discomfort, and emotional dimensions (arousal, valence, and dominance) were reported after each task. The performance of five common ML models with a versatile set of features (physiological features, task performance data, and personality trait) was compared in binary classification of subjectively assessed emotions.

Results

The psychophysiological responses proved the protocol was successful in changing the mental state from baseline, also the cognitive and physical tasks were different. The optimization and performance of ML models used for emotion detection were evaluated. Additionally, methods to account for imbalanced classes were applied and shown to improve the classification performance.

Comparison with existing methods

Classification of human emotional states often assumes the states are determined by the stimuli. However, individual appraisals vary. None of the past studies have classified subjective emotional dimensions with a set of features including biosignals, personality and behavior.

Conclusion

Our data represent a typical setup in affective computing utilizing psychophysiological monitoring: N is low compared to number of features, inter-individual variability is high, and class imbalance cannot be avoided. Our observations are a) if possible, include features representing physiology, behavior and personality, b) use simple models and limited number of features to improve interpretability, c) address the possible imbalance, d) if the data size allows, use nested cross-validation.

Explainable stress type classification captures physiologically relevant responses in the Maastricht Acute Stress Test

Introduction

Current stress detection methods concentrate on identification of stress and non-stress states despite the existence of various stress types. The present study performs a more specific, explainable stress classification, which could provide valuable information on the physiological stress reactions.

Methods

Physiological responses were measured in the Maastricht Acute Stress Test (MAST), comprising alternating trials of cold pressor (inducing physiological stress and pain) and mental arithmetics (eliciting cognitive and social-evaluative stress). The responses in these subtasks were compared to each other and to the baseline through mixed model analysis. Subsequently, stress type detection was conducted with a comprehensive analysis of several machine learning components affecting classification. Finally, explainable artificial intelligence (XAI) methods were applied to analyze the influence of physiological features on model behavior.

Results

Most of the investigated physiological reactions were specific to the stressors, and the subtasks could be distinguished from baseline with up to 86.5% balanced accuracy. The choice of the physiological signals to measure (up to 25%-point difference in balanced accuracy) and the selection of features (up to 7%-point difference) were the two key components in classification. Reflection of the XAI analysis to mixed model results and human physiology revealed that the stress detection model concentrated on physiological features relevant for the two stressors.

Discussion

The findings confirm that multimodal machine learning classification can detect different types of stress reactions from baseline while focusing on physiologically sensible changes. Since the measured signals and feature selection affected classification performance the most, data analytic choices left limited input information uncompensated.

Emotion Recognition Using a Novel Granger Causality Quantifier and Combined Electrodes of EEG

Electroencephalogram (EEG) connectivity patterns can reflect neural correlates of emotion. However, the necessity of evaluating bulky data for multi-channel measurements increases the computational cost of the EEG network. To date, several approaches have been presented to pick the optimal cerebral channels, mainly depending on available data. Consequently, the risk of low data stability and reliability has increased by reducing the number of channels. Alternatively, this study suggests an electrode combination approach in which the brain is divided into six areas. After extracting EEG frequency bands, an innovative Granger causality-based measure was introduced to quantify brain connectivity patterns. The feature was subsequently subjected to a classification module to recognize valence-arousal dimensional emotions. A Database for Emotion Analysis Using Physiological Signals (DEAP) was used as a benchmark database to evaluate the scheme. The experimental results revealed a maximum accuracy of 89.55%. Additionally, EEG-based connectivity in the beta-frequency band was able to effectively classify dimensional emotions. In sum, combined EEG electrodes can efficiently replicate 32-channel EEG information.

Psycho-Cognitive Profile and NGF and BDNF Levels in Tears and Serum: A Pilot Study in Patients with Graves' Disease

Nerve Growth Factor (NGF) and Brain derived Neurotrophic Factor (BDNF) mature/precursor imbalance in tears and serum is suggested as a risk factor and symptomatology aggravation in ophthalmology and neuropsychiatric disturbances. Cognitive and mood alterations are reported by patients with Graves' Orbitopathy (GO), indicating neurotrophin alterations might be involved. To address this question, the expression levels of NGF and BDNF and their precursors in serum and tears of GO patients were analyzed and correlated with the ophthalmological and psycho-cognitive symptoms. Hamilton Rating Scale for Anxiety (HAM-A) and Depression (HAM-D), Temperament and Character Inventory (TCI), and Cambridge Neuropsychological Test Automated Battery (CANTAB) test were used as a score. NGF and BDNF levels were measured using ELISA and Western Blot and statistically analyzed for psychiatric/ocular variable trend association. GO patients show memorization time and level of distraction increase, together with high irritability and impulsiveness. HAM-A and CANTAB variables association, and some TCI dimensions are also found. NGF and BDNF expression correlates with ophthalmological symptoms only in tears, while mature/precursor NGF and BDNF correlate with the specific psycho-cognitive variables both in tears and serum. Our study is the first to show that changes in NGF and BDNF processing in tears and serum might profile ocular and cognitive alterations in patients.

Red deer Cervus elaphus blink more in larger groups

Most animals need to spend time being vigilant for predators, at the expense of other activities such as foraging. Group-living animals can benefit from the shared vigilance effort of other group members, with individuals reducing personal vigilance effort as group size increases. Behaviors like active scanning or head lifting are usually used to quantify vigilance but may not be accurate measures of this. We suggest that measuring an animal's blinking rate gives a meaningful measure of vigilance: increased blinking implies reduced vigilance, as the animal cannot detect predators when its eyes are closed. We describe an observational study of a captive population of red deer, where we measured the blinking rates of individual deer from groups of differing sizes (where mean group size ranged between 1 and 42.7 individuals). We demonstrate that as group size increases in red deer, individuals increase their blink rate, confirming the prediction that vigilance should decrease. Blinking is a simple non-invasive measure and offers a useful metric for assessing the welfare of animals experiencing an increase in perceived predation risk or other stressors.

The distribution of blinkrate among Malawian young adults: a cross-sectional study

Blink rate is a critical sign for numerous systemic and ocular conditions in medicine, however, the literature reports varying values for the parameter. Hence, the aim of this study was to establish the cut-off blink rate value among Malawian young adults including the effects of sex and age on the parameter. This was a quantitative descriptive cross-sectional study conducted among students at Mzuzu University in Malawi. The study recruited 98 participants, 50% male and 50% female. The age ranged from 17-45 years. The blink rate was measured manually by observing the number of blinks per minute. The average blink rate was 16.04 (SD = 6.417) blinks per minute. The Blink rate was not significantly correlated with age (P = 0.066) and sex (P = 0.8143). Our study confirms that blink rate varies according to geographical location as a factor of different weather conditions. Moreover, we found no age and sex-related differences in blink rate.

Time-synchronic comments on video streaming website reveal core structures of audience engagement in movie viewing

To what extent movie viewers are swept into a fictional world has long been pondered by psychologists and filmmakers. With the development of time-synchronic comments on online viewing platforms, we can now analyze viewers' immediate responses toward movies. In this study, we collected over 3 million Chinese time-synchronic comments from a video streaming website. We first assessed emotion and cognition-related word rates in these comments with the Simplified Chinese version of the Linguistic Inquiry and Word Count (SCLIWC) and applied time-series clustering to the word rates. Then Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) was conducted on the text to investigate the prevalent topics among the comments. We found different commenting behaviors in front of various movies and prototypical diachronic trajectories of the psychological engagement of the audience. We further identified how topics are discussed through time, and tried to account for viewer's engagement, considering successively movie genres, topics and movie content. Among other points, we finally discussed the challenge in explaining the trajectories of engagement and the disconnection with narrative content. Overall, our study provides a new perspective on using social media data to answer questions from psychology and film studies. It underscores the potential of time-synchronic comments as a resource for detecting real-time human responses to specific events.

Activation of brain arousal networks coincident with eye blinks during resting state

Eye-blinking has been implicated in arousal and attention. Here we test the hypothesis that blinking-moments represent arousal surges associated with activation of the ascending arousal network (AAN) and its thalamic projections. For this purpose, we explored the temporal relationship between eye-blinks and fMRI BOLD activity in AAN and thalamic nuclei, as well as whole brain cluster corrected activations during eyes-open, resting-state fMRI scanning. We show that BOLD activations in the AAN nuclei peaked prior to the eye blinks and in thalamic nuclei peaked prior to and during the blink, consistent with the role of eye blinking in arousal surges. Additionally, we showed visual cortex peak activation prior to the eye blinks, providing further evidence of the visual cortex's role in arousal, and document cerebellar peak activation post eye blinks, which might reflect downstream engagement from arousal surges.

Seeing the Forrest through the trees: Oculomotor metrics are linked to heart rate

Fluctuations in a person’s arousal accompany mental states such as drowsiness, mental effort, or motivation, and have a profound effect on task performance. Here, we investigated the link between two central instances affected by arousal levels, heart rate and eye movements. In contrast to heart rate, eye movements can be inferred remotely and unobtrusively, and there is evidence that oculomotor metrics (i.e., fixations and saccades) are indicators for aspects of arousal going hand in hand with changes in mental effort, motivation, or task type. Gaze data and heart rate of 14 participants during film viewing were used in Random Forest models, the results of which show that blink rate and duration, and the movement aspect of oculomotor metrics (i.e., velocities and amplitudes) link to heart rate–more so than the amount or duration of fixations and saccades. We discuss that eye movements are not only linked to heart rate, but they may both be similarly influenced by the common underlying arousal system. These findings provide new pathways for the remote measurement of arousal, and its link to psychophysiological features.

Task difficulty and physiological measures of mental workload in air traffic control: a scoping review

This study provides a systematic synthesis of empirical research on mental workload (MWL) in air traffic control (ATC). MWL is a key concept in research on innovative technologies, because the assessment of MWL is crucial to the evaluation of such technologies. Our specific focus was on physiological measures of MWL. The used search strategy identified 39 peer-reviewed publications that analysed ATC tasks, examined different levels of difficulty of the ATC task, and considered at least one physiological measure of MWL. Positive relations between measures of MWL and task difficulty were observed most frequently, indicating that the measures indeed allowed the assessment of MWL. The most commonly used physiological measures were brain measures (EEG and fNIR) and heart rate measures. The review revealed a need for more precise descriptions of crucial experimental parameters in order to permit a transition of the field towards more interactive and dynamic types of analysis. Practitioner summary: Research on innovative technology in air traffic control (ATC) depends on assessments of mental workload (MWL). We reviewed empirical research on MWL in ATC. Brain and heart measures often allow assessments of MWL. Better descriptions of experiments are needed to allow comparisons among studies and more dynamic and interactive analyses.

Innovative Poincare's plot asymmetry descriptors for EEG emotion recognition

Given the importance of emotion recognition in both medical and non-medical applications, designing an automatic system has captured the attention of several scholars. Currently, EEG-based emotion recognition has a special position, which has not fulfilled the desired accuracy rates yet. This experiment intended to provide novel EEG asymmetry measures to improve emotion recognition rates. Four emotional states have been classified using the k-nearest neighbor (kNN), support vector machine, and Naïve Bayes. Feature selection has been performed, and the role of employing a different number of top-ranked features on emotion recognition rates has been assessed. To validate the efficiency of the proposed scheme, two public databases, including the SJTU Emotion EEG Dataset-IV (SEED-IV) and a Database for Emotion Analysis using Physiological signals (DEAP) were evaluated. The experimental results indicated that kNN outperformed the other classifiers with a maximum accuracy of 95.49 and 98.63% using SEED-IV and DEAP datasets, respectively. In conclusion, the results of the proposed novel EEG-asymmetry measures make the framework a superior one compared to the state-of-art EEG emotion recognition approaches.

An Effective Gaze-Based Authentication Method with the Spatiotemporal Feature of Eye Movement

Eye movement has become a new behavioral feature for biometric authentication. In the eye movement-based authentication methods that use temporal features and artificial design features, the required duration of eye movement recordings are too long to be applied. Therefore, this study aims at using eye movement recordings with shorter duration to realize authentication. And we give out a reasonable eye movement recording duration that should be less than 12 s, referring to the changing pattern of the deviation degree between the gaze point and the stimulus point on the screen. In this study, the temporal motion features of the gaze points and the spatial distribution features of the saccade are using to represent the personal identity. Two datasets are constructed for the experiments, including 5 s and 12 s of eye movement recordings. On the datasets constructed in this paper, the open-set authentication results show that the Equal Error Rate of our proposed methods can reach 10.62% when recording duration is 12 s and 12.48% when recording duration is 5 s. The closed-set authentication results show that the Equal Error Rate of our proposed methods can reach 5.25% when recording duration is 12 s and 7.82% when recording duration is 5 s. It demonstrates that the proposed method provides a reference for the eye movements data-based identity authentication.

Temporal dynamics of affect in the brain: Evidence from human imaging and animal models

Emotions are time-varying internal states that promote survival in the face of dynamic environments and shifting homeostatic needs. Research in non-human organisms has recently afforded specific insights into the neural mechanisms that support the emergence, persistence, and decay of affective states. Concurrently, a separate affective neuroscience literature has begun to dissect the neural bases of affective dynamics in humans. However, the circuit-level mechanisms identified in animals lack a clear mapping to the human neuroscience literature. As a result, critical questions pertaining to the neural bases of affective dynamics in humans remain unanswered. To address these shortcomings, the present review integrates findings from humans and non-human organisms to highlight the neural mechanisms that govern the temporal features of emotional states. Using the theory of affective chronometry as an organizing framework, we describe the specific neural mechanisms and modulatory factors that arbitrate the rise-time, intensity, and duration of emotional states.

Eye Movement and Pupil Measures: A Review

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first describe the main oculomotor events studied in the literature, and their characteristics exploited by different measures. Next, we review various eye movement and pupil measures from prior literature. Finally, we discuss our observations based on applications of these measures, the benefits and practical challenges involving these measures, and our recommendations on future eye-tracking research directions.

Neural signatures of attentional engagement during narratives and its consequences for event memory

As we comprehend narratives, our attentional engagement fluctuates over time. Despite theoretical conceptions of narrative engagement as emotion-laden attention, little empirical work has characterized the cognitive and neural processes that comprise subjective engagement in naturalistic contexts or its consequences for memory. Here, we relate fluctuations in narrative engagement to patterns of brain coactivation and test whether neural signatures of engagement predict subsequent memory. In behavioral studies, participants continuously rated how engaged they were as they watched a television episode or listened to a story. Self-reported engagement was synchronized across individuals and driven by the emotional content of the narratives. In functional MRI datasets collected as different individuals watched the same show or listened to the same story, engagement drove neural synchrony, such that default mode network activity was more synchronized across individuals during more engaging moments of the narratives. Furthermore, models based on time-varying functional brain connectivity predicted evolving states of engagement across participants and independent datasets. The functional connections that predicted engagement overlapped with a validated neuromarker of sustained attention and predicted recall of narrative events. Together, our findings characterize the neural signatures of attentional engagement in naturalistic contexts and elucidate relationships among narrative engagement, sustained attention, and event memory.

Eye-tracking research on sensory and consumer science: A review, pitfalls and future directions

Visual processing is a core cognitive element of sensory and consumer science. Consumers visually attend to food types, packaging, label design, advertisements, supermarket shelves, food menus, and other visible information. During the past decade, sensory and consumer science have used eye tracking to elucidate visual processing by consumers. This review paper summarizes earlier findings in terms of bottom-up (i.e., stimulus-driven) processing such as visual salience, size, and top-down (i.e., goal-driven) processing such as goals, task instructions, task complexity, and emotions. Downstream effects of gaze on choice are also reviewed. Pitfalls and future directions of eye-tracking research on sensory and consumer science are also discussed.

The effect of anime character's facial expressions and eye blinking on donation behavior

Studies of the use of artificial agents and robots to solicit donations from people have suggested that the design of the agents must consider facial expressions. However, there has not been sufficient evidence to generalize the finding that the emotions conveyed by agents' facial expressions can induce donations. In the present study, we conducted an experiment with an animated character that has intermediate realism and a different appearance from those in previous studies to replicate the finding that facial expressions represented by changes in the shapes of the eyes and mouth cause people to become more prosocial and to test whether we can extend this finding to the emotional expressions presented by changes in the dynamic properties of eyes. In the experiment, participants ([Formula: see text]) played a hypothetical dictator game with an avatar that expressed its emotions by changing the shapes of its eyes, eyebrows and mouth and by changing the frequency of eye blinking. The results showed that the emotions expressed by changes in the shape of the facial parts contributed to eliciting a higher donation amount, consistent with previous studies. However, we could not find an additive effect of the emotional expression shown by eye blinking. The results suggest that, regardless of appearance, emotional expression is useful in the design of a virtual agent's face, but it might not be necessary to consider the dynamic properties of the eyes.

Viewers Change Eye-Blink Rate by Predicting Narrative Content

Eye blinks provoke a loss of visual information. However, we are not constantly making conscious decisions about the appropriate moment to blink. The presence or absence of eye blinks also denotes levels of attention. We presented three movies with the exact same narrative but different styles of editing and recorded participants' eye blinks. We found that moments of increased or decreased eye blinks by viewers coincided with the same content in the different movie styles. The moments of increased eye blinks corresponded to those when the actor leaves the scene and when the movie repeats the same action for a while. The moments of decreased eye blinks corresponded to actions where visual information was crucial to proper understanding of the scene presented. According to these results, viewers' attention is more related to narrative content than to the style of editing when watching movies.

'Defrosting' music chills with naltrexone: The role of endogenous opioids for the intensity of musical pleasure

The endogenous opioid system has been implicated during experiences of pleasure (i.e., from food or sex). Music can elicit intense emotional and bodily sensations of pleasure, called 'Chills'. We investigated the effects of an opioid antagonist (50 mg naltrexone) or placebo (40 μg d3-vitamin) while listening to self-selected music or other 'control' music selected by another participant. We used a novel technique of continuous measurement of pleasantness with an eye tracker system, where participants shifted their eyes along a visual analogue scale, in the semblance of a thermometer so that, as the music unfolded, gaze positions indicated the self-reported hedonic experience. Simultaneously, we obtained pupil diameters. Self-reported pleasure remained unchanged by naltrexone, which - however - selectively decreased pupillary diameters during 'Chills'. Hence, the endogenous μ-opioid signaling is not necessary for subjective enjoyment of music but an opioid blockade dampens pupil responses to peak pleasure, consistent with decreased arousal to the music.

Phylogenetically-controlled correlates of primate blinking behaviour

Eye blinking is an essential maintenance behaviour for many terrestrial animals, but is also a risky behaviour as the animal is unable to scan the environment and detect hazards while its eyes are temporarily closed. It is therefore likely that the length of time that the eyes are closed and the length of the gap between blinks for a species may reflect aspects of the ecology of that species, such as its social or physical environment. An earlier published study conducted a comparative study linking blinking behaviour and ecology, and detailed a dataset describing the blinking behaviour of a large number of primate species that was collected from captive animals, but the analysis presented did not control for the nonindependence of the data due to common evolutionary history. In the present study, the dataset is reanalysed using phylogenetic comparative methods, after reconsideration of the parameters describing the physical and social environments of the species. I find that blink rate is best described by the locomotion mode of a species, where species moving through arboreal environments blink least, ground-living species blink most, and species that use both environments show intermediate rates. The duration of a blink was also related to locomotion mode, and positively correlated with both mean species group size and mean species body mass, although the increase in relation to group size is small. How a species moves through the environment therefore appears to be important for determining blinking behaviour, and suggests that complex arboreal environments may require less interruption to visual attention. Given that the data were collected with captive individuals, caution is recommended for interpreting the correlations found.

Recognition of Blinks Activity Patterns during Stress Conditions Using CNN and Markovian Analysis

This paper investigates eye behaviour through blinks activity during stress conditions. Although eye blinking is a semi-voluntary action, it is considered to be affected by one’s emotional states such as arousal or stress. The blinking rate provides information towards this direction, however, the analysis on the entire eye aperture timeseries and the corresponding blinking patterns provide enhanced information on eye behaviour during stress conditions. Thus, two experimental protocols were established to induce affective states (neutral, relaxed and stress) systematically through a variety of external and internal stressors. The study populations included 24 and 58 participants respectively performing 12 experimental affective trials. After the preprocessing phase, the eye aperture timeseries and the corresponding features were extracted. The behaviour of inter-blink intervals (IBI) was investigated using the Markovian Analysis to quantify incidence dynamics in sequences of blinks. Moreover, Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) network models were employed to discriminate stressed versus neutral tasks per cognitive process using the sequence of IBI. The classification accuracy reached a percentage of 81.3% which is very promising considering the unimodal analysis and the noninvasiveness modality used.

EEG gamma activity to emotional movies in individuals with high traits of primary "successful" psychopathy

This study aimed to investigate emotional alterations in a community sample with primary psychopathic traits. Sixty males selected from a larger sample and divided in two groups, High (HP) and Low (LP) primary Psychopathy, watched 15 validated emotional movies divided in five categories (Erotic, Neutral, Fear, and the new Scenery and Compassion). Subjective responses and cortical activity in the EEG Gamma band (30-49 Hz) were recorded. Concerning self-reports, HP participants felt less anxious and anguished in response to Fear clips and less sad, touched and anguished by Compassion excerpts. Negative clips induced larger EEG Gamma activity in both groups compared to neutral and erotic movies, but Gamma activity to Fear movies was reduced in HP. This group showed also a small cortical response to positive Scenery movies, at the level of the Neutral ones. Source analysis showed in HP participants a reduced cortical activation to Fear in a large brain network, comprising the right prefrontal and temporal cortices and bilateral inferior parietal cortex. Results showed that primary psychopathy, in addition to the impaired response to Fear, was associated with a reduced response also to other specific categories of emotional stimuli, suggesting an altered affect on a broader scale.

Fluctuation in Pupil Size and Spontaneous Blinks Reflect Story Transportation

Thirty-nine participants listened to 28 neutral and horror excerpts of Stephen King short stories while constantly tracking their emotional arousal. Pupil size was measured with an Eyelink 1000+, and participants rated valence and transportation after each story. In addition to computing mean pupil size across 1-sec intervals, we extracted blink count and used detrended fluctuation analysis (DFA) to obtain the scaling exponents of long-range temporal correlations (LRTCs) in pupil size time-series. Pupil size was expected to be sensitive also to emotional arousal, whereas blink count and LRTC's were expected to reflect cognitive engagement. The results showed that self-reported arousal increased, pupil size was overall greater, and the decreasing slope of pupil size was flatter for horror than for neutral stories. Horror stories induced higher transportation than neutral stories. High transportation was associated with a steeper increase in self-reported arousal across time, stronger LRTCs in pupil size fluctuations, and lower blink count. These results indicate that pupil size reflects emotional arousal induced by the text content, while LRTCs and blink count are sensitive to cognitive engagement associated with transportation, irrespective of the text type. The study demonstrates the utility of pupillometric measures and blink count to study literature reception.

E-MOVIE - Experimental MOVies for Induction of Emotions in neuroscience: An innovative film database with normative data and sex differences

The need for a validated set of emotional clips to elicit emotions in more ecological experiments is increasing. Here we present the validation of a new database of emotional films, named E-MOVIE, which includes, in this first validation phase, 39 excerpts arranged in six categories, three negative (Fear, Sadness and Compassion), two positive (Erotic and Scenery) and a Neutral category. Notably, Compassion and Scenery are new in the field as they were not included in other databases. The clips in E-MOVIE are characterized by homogenous durations of approximately two minutes, which make them suitable for psychophysiological research. In order to study the affective profile prompted by each category 174 participants (112 women) rated the movies on multiple dimensions, namely valence and arousal, intensity and discreteness of the induction of one of the six basic emotions and, finally, intensity of the experience of the emotional states defined by a series of emotional adjectives. Erotic clips were effective in the elicitation of a positive emotional state, characterized by high levels of arousal and excitement. On the other hand, Fear clips (selected without blood to avoid disgust reaction) prompted an affect characterized by high arousal, low valence and high levels of reported fear and anxiety. Women reported greater unpleasantness, distress, anxiety and jittery than men to the three negative categories. Compassion clips, characterized by the depiction of crying characters, were able to induce an affective state dominated by sadness and feeling touched, consistent with an empathic reaction to emotional sufferance. Sadness clips, instead, elicited an affective state characterized by sadness together with distress and angst. We also demonstrated that clips depicting natural environments (i.e. Scenery) prompted in the viewer a surprised, inspired affective state, characterized by high valence and arousal (especially in males), a result which suggests that their past categorization as neutral stimuli was inaccurate and problematic.

Affective and cortical EEG gamma responses to emotional movies in women with high vs low traits of empathy

The present study sought to investigate how differences in trait empathy can influence emotional reactivity to a specific set of affective categories. Forty-one female students, divided in High (HE, n = 20) and Low (LE, n = 21) trait empathy, watched eight validated movie clips divided in four emotional categories (Erotic, Fear, Compassion and Neutral) while subjective evaluation of emotion and EEG gamma activity were recorded. Analysis of self-reports revealed that HE compared to LE exhibited an increased arousal level to all emotional clips. Concerning EEG data, the HE group showed a greater cortical gamma to all the emotional categories compared to the Neutral, while the LE group had greater response only to the negative clips. Participants in the HE group also showed a strong positive correlation between subjective arousal and cortical activity in response to Fear and Compassion clips. The greatest correlation was found to Compassion clips and was located in the right inferior parietal lobe (r₍₁₈₎ = 0.63), an important hub for both sensory-emotion integration and empathic sharing of others' emotions. Results suggest that high empathy was associated with enhanced gamma activity and greater self-reported arousal to all emotional stimuli. Furthermore, in this group, scenes with crying characters prompted a distinctive and localized cortical processing.