Mutual information between heart rate variability and respiration for emotion characterization

Gradients of O-information highlight synergy and redundancy in physiological applications

The study of high order dependencies in complex systems has recently led to the introduction of statistical synergy, a novel quantity corresponding to a form of emergence in which patterns at large scales are not traceable from lower scales. As a consequence, several works in the last years dealt with the synergy and its counterpart, the redundancy. In particular, the O-information is a signed metric that measures the balance between redundant and synergistic statistical dependencies. In spite of its growing use, this metric does not provide insight about the role played by low-order scales in the formation of high order effects. To fill this gap, the framework for the computation of the O-information has been recently expanded introducing the so-called gradients of this metric, which measure the irreducible contribution of a variable (or a group of variables) to the high order informational circuits of a system. Here, we review the theory behind the O-information and its gradients and present the potential of these concepts in the field of network physiology, showing two new applications relevant to brain functional connectivity probed via functional resonance imaging and physiological interactions among the variability of heart rate, arterial pressure, respiration and cerebral blood flow.

Changes in the autonomic cardiorespiratory activity in parturient women with severe and moderate features of preeclampsia

Background

Cardiorespiratory coupling (CRC) is a physiological phenomenon that reflects the mutual interaction between the cardiac and respiratory control systems. It is mainly associated with efferent vagal activity from the central autonomic network. Few studies have explored the autonomic changes of CRC in preeclampsia, a critical obstetric complication related to possible autonomic dysfunctions and inflammatory disturbances. This study examined the autonomic mechanisms of CRC in women with severe and moderate preeclampsia and healthy controls by applying nonlinear methods based on information theory, such as mutual information (MI) and Renyi's mutual information (RMI) and the linear and nonlinear analysis of the Pulse-Respiration Quotient (PRQ).

Methods

We studied three groups of parturient women in the third trimester of pregnancy with a clinical diagnosis of preeclampsia without severe symptoms (P, 38.5 ± 1.4 weeks of pregnancy, n=19), preeclampsia with severe symptoms (SP, 37.5 ± 0.9 weeks of pregnancy, n=22), and normotensive control women (C, 39.1 ± 1.3 weeks of pregnancy, n=20). 10-minutes of abdominal electrocardiograms (ECG) and respiratory signals (RESP) were recorded in all the participants. Subsequently, we obtained the maternal beat-to-beat (RR) and breath-to-breath (BB) time series from ECG and RESP, respectively. The CRC between RR and BB was quantified by nonlinear methods based on information theory, such as MI and RMI, along with the analysis of the novel index of PRQ. Subsequently, we computed the mean PRQ (mPRQ) and the normalized permutation entropy (nPermEn_PRQ) from the PRQ time series generated from BB and RR. In addition, we examined the vagal activity in the three groups by the logarithm of the median of the distribution of the absolute values of successive RR differences (logRSA).

Results

The MI and RMI values were significantly lower (p<0.05) in the preeclamptic groups compared to the control group. However, no significant differences were found between the preeclamptic groups. The logRSA and nPermEn_PRQ indices were significantly lower (p<0.05) in SP compared to C and P.

Conclusion

Our data suggest that parturient women with severe and mild preeclampsia may manifest an altered cardiorespiratory coupling compared with normotensive control women. Disrupted CRC in severe preeclampsia could be associated with vagal withdrawal and less complex cardiorespiratory dynamics. The difference in vagal activity between the preeclamptic groups may suggest a further reduction in vagal activity associated with the severity of the disease.

Using Wearable Passive Sensing to Predict Binge Eating in Response to Negative Affect Among Individuals With Transdiagnostic Binge Eating: Protocol for an Observational Study

BACKGROUND

Binge eating (BE), characterized by eating a large amount of food accompanied by a sense of loss of control over eating, is a public health crisis. Negative affect is a well-established antecedent for BE. The affect regulation model of BE posits that elevated negative affect increases momentary risk for BE, as engaging in BE alleviates negative affect and reinforces the behavior. The eating disorder field's capacity to identify moments of elevated negative affect, and thus BE risk, has exclusively relied on ecological momentary assessment (EMA). EMA involves the completion of surveys in real time on one's smartphone to report behavioral, cognitive, and emotional symptoms throughout the day. Although EMA provides ecologically valid information, EMA surveys are often delivered only 5-6 times per day, involve self-report of affect intensity only, and are unable to assess affect-related physiological arousal. Wearable, psychophysiological sensors that measure markers of affect arousal including heart rate, heart rate variability, and electrodermal activity may augment EMA surveys to improve accurate real-time prediction of BE. These sensors can objectively and continuously measure biomarkers of nervous system arousal that coincide with affect, thus allowing them to measure affective trajectories on a continuous timescale, detect changes in negative affect before the individual is consciously aware of them, and reduce user burden to improve data completeness. However, it is unknown whether sensor features can distinguish between positive and negative affect states, given that physiological arousal may occur during both negative and positive affect states.

OBJECTIVE

The aims of this study are (1) to test the hypothesis that sensor features will distinguish positive and negative affect states in individuals with BE with >60% accuracy and (2) test the hypothesis that a machine learning algorithm using sensor data and EMA-reported negative affect to predict the occurrence of BE will predict BE with greater accuracy than an algorithm using EMA-reported negative affect alone.

METHODS

This study will recruit 30 individuals with BE who will wear Fitbit Sense 2 wristbands to passively measure heart rate and electrodermal activity and report affect and BE on EMA surveys for 4 weeks. Machine learning algorithms will be developed using sensor data to distinguish instances of high positive and high negative affect (aim 1) and to predict engagement in BE (aim 2).

RESULTS

This project will be funded from November 2022 to October 2024. Recruitment efforts will be conducted from January 2023 through March 2024. Data collection is anticipated to be completed in May 2024.

CONCLUSIONS

This study is anticipated to provide new insight into the relationship between negative affect and BE by integrating wearable sensor data to measure affective arousal. The findings from this study may set the stage for future development of more effective digital ecological momentary interventions for BE.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/47098.

Deep learning-based EEG emotion recognition: Current trends and future perspectives

Automatic electroencephalogram (EEG) emotion recognition is a challenging component of human-computer interaction (HCI). Inspired by the powerful feature learning ability of recently-emerged deep learning techniques, various advanced deep learning models have been employed increasingly to learn high-level feature representations for EEG emotion recognition. This paper aims to provide an up-to-date and comprehensive survey of EEG emotion recognition, especially for various deep learning techniques in this area. We provide the preliminaries and basic knowledge in the literature. We review EEG emotion recognition benchmark data sets briefly. We review deep learning techniques in details, including deep belief networks, convolutional neural networks, and recurrent neural networks. We describe the state-of-the-art applications of deep learning techniques for EEG emotion recognition in detail. We analyze the challenges and opportunities in this field and point out its future directions.

Confounding effects of heart rate, breathing rate, and frontal fNIRS on interoception

Recent studies have established that cardiac and respiratory phases can modulate perception and related neural dynamics. While heart rate and respiratory sinus arrhythmia possibly affect interoception biomarkers, such as heartbeat-evoked potentials, the relative changes in heart rate and cardiorespiratory dynamics in interoceptive processes have not yet been investigated. In this study, we investigated the variation in heart and breathing rates, as well as higher functional dynamics including cardiorespiratory correlation and frontal hemodynamics measured with fNIRS, during a heartbeat counting task. To further investigate the functional physiology linked to changes in vagal activity caused by specific breathing rates, we performed the heartbeat counting task together with a controlled breathing rate task. The results demonstrate that focusing on heartbeats decreases breathing and heart rates in comparison, which may be part of the physiological mechanisms related to "listening" to the heart, the focus of attention, and self-awareness. Focusing on heartbeats was also observed to increase frontal connectivity, supporting the role of frontal structures in the neural monitoring of visceral inputs. However, cardiorespiratory correlation is affected by both heartbeats counting and controlled breathing tasks. Based on these results, we concluded that variations in heart and breathing rates are confounding factors in the assessment of interoceptive abilities and relative fluctuations in breathing and heart rates should be considered to be a mode of covariate measurement of interoceptive processes.

Affective State Recognition in Livestock—Artificial Intelligence Approaches

Simple Summary

Emotions or affective states recognition in farm animals is an underexplored research domain. Despite significant advances in animal welfare research, animal affective state computing through the development and application of devices and platforms that can not only recognize but interpret and process the emotions, are in a nascent stage. The analysis and measurement of unique behavioural, physical, and biological characteristics offered by biometric sensor technologies and the affiliated complex and large data sets, opens the pathway for novel and realistic identification of individual animals amongst a herd or a flock. By capitalizing on the immense potential of biometric sensors, artificial intelligence enabled big data methods offer substantial advancement of animal welfare standards and meet the urgent needs of caretakers to respond effectively to maintain the wellbeing of their animals.

Abstract

Farm animals, numbering over 70 billion worldwide, are increasingly managed in large-scale, intensive farms. With both public awareness and scientific evidence growing that farm animals experience suffering, as well as affective states such as fear, frustration and distress, there is an urgent need to develop efficient and accurate methods for monitoring their welfare. At present, there are not scientifically validated ‘benchmarks’ for quantifying transient emotional (affective) states in farm animals, and no established measures of good welfare, only indicators of poor welfare, such as injury, pain and fear. Conventional approaches to monitoring livestock welfare are time-consuming, interrupt farming processes and involve subjective judgments. Biometric sensor data enabled by artificial intelligence is an emerging smart solution to unobtrusively monitoring livestock, but its potential for quantifying affective states and ground-breaking solutions in their application are yet to be realized. This review provides innovative methods for collecting big data on farm animal emotions, which can be used to train artificial intelligence models to classify, quantify and predict affective states in individual pigs and cows. Extending this to the group level, social network analysis can be applied to model emotional dynamics and contagion among animals. Finally, ‘digital twins’ of animals capable of simulating and predicting their affective states and behaviour in real time are a near-term possibility.

Linking Multi-Layer Dynamical GCN With Style-Based Recalibration CNN for EEG-Based Emotion Recognition

Electroencephalography (EEG)-based emotion computing has become one of the research hotspots of human-computer interaction (HCI). However, it is difficult to effectively learn the interactions between brain regions in emotional states by using traditional convolutional neural networks because there is information transmission between neurons, which constitutes the brain network structure. In this paper, we proposed a novel model combining graph convolutional network and convolutional neural network, namely MDGCN-SRCNN, aiming to fully extract features of channel connectivity in different receptive fields and deep layer abstract features to distinguish different emotions. Particularly, we add style-based recalibration module to CNN to extract deep layer features, which can better select features that are highly related to emotion. We conducted two individual experiments on SEED data set and SEED-IV data set, respectively, and the experiments proved the effectiveness of MDGCN-SRCNN model. The recognition accuracy on SEED and SEED-IV is 95.08 and 85.52%, respectively. Our model has better performance than other state-of-art methods. In addition, by visualizing the distribution of different layers features, we prove that the combination of shallow layer and deep layer features can effectively improve the recognition performance. Finally, we verified the important brain regions and the connection relationships between channels for emotion generation by analyzing the connection weights between channels after model learning.

Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach

Extensive possibilities of applications have rendered emotion recognition ineluctable and challenging in the fields of computer science as well as in human-machine interaction and affective computing. Fields that, in turn, are increasingly requiring real-time applications or interactions in everyday life scenarios. However, while extremely desirable, an accurate and automated emotion classification approach remains a challenging issue. To this end, this study presents an automated emotion recognition model based on easily accessible physiological signals and deep learning (DL) approaches. As a DL algorithm, a Feedforward Neural Network was employed in this study. The network outcome was further compared with canonical machine learning algorithms such as random forest (RF). The developed DL model relied on the combined use of wearables and contactless technologies, such as thermal infrared imaging. Such a model is able to classify the emotional state into four classes, derived from the linear combination of valence and arousal (referring to the circumplex model of affect’s four-quadrant structure) with an overall accuracy of 70% outperforming the 66% accuracy reached by the RF model. Considering the ecological and agile nature of the technique used the proposed model could lead to innovative applications in the affective computing field.

Respiration Based Non-Invasive Approach for Emotion Recognition Using Impulse Radio Ultra Wide Band Radar and Machine Learning

Emotion recognition gained increasingly prominent attraction from a multitude of fields recently due to their wide use in human-computer interaction interface, therapy, and advanced robotics, etc. Human speech, gestures, facial expressions, and physiological signals can be used to recognize different emotions. Despite the discriminating properties to recognize emotions, the first three methods have been regarded as ineffective as the probability of human's voluntary and involuntary concealing the real emotions can not be ignored. Physiological signals, on the other hand, are capable of providing more objective, and reliable emotion recognition. Based on physiological signals, several methods have been introduced for emotion recognition, yet, predominantly such approaches are invasive involving the placement of on-body sensors. The efficacy and accuracy of these approaches are hindered by the sensor malfunctioning and erroneous data due to human limbs movement. This study presents a non-invasive approach where machine learning complements the impulse radio ultra-wideband (IR-UWB) signals for emotion recognition. First, the feasibility of using IR-UWB for emotion recognition is analyzed followed by determining the state of emotions into happiness, disgust, and fear. These emotions are triggered using carefully selected video clips to human subjects involving both males and females. The convincing evidence that different breathing patterns are linked with different emotions has been leveraged to discriminate between different emotions. Chest movement of thirty-five subjects is obtained using IR-UWB radar while watching the video clips in solitude. Extensive signal processing is applied to the obtained chest movement signals to estimate respiration rate per minute (RPM). The RPM estimated by the algorithm is validated by repeated measurements by a commercially available Pulse Oximeter. A dataset is maintained comprising gender, RPM, age, and associated emotions which are further used with several machine learning algorithms for automatic recognition of human emotions. Experiments reveal that IR-UWB possesses the potential to differentiate between different human emotions with a decent accuracy of 76% without placing any on-body sensors. Separate analysis for male and female participants reveals that males experience high arousal for happiness while females experience intense fear emotions. For disgust emotion, no large difference is found for male and female participants. To the best of the authors' knowledge, this study presents the first non-invasive approach using the IR-UWB radar for emotion recognition.

Assessment of Cognitive Student Engagement Using Heart Rate Data in Distance Learning during COVID-19

Student engagement allows educational institutions to make better decisions regarding teaching methodologies, methods for evaluating the quality of education, and ways to provide timely feedback. Due to the COVID-19 pandemic, identifying cognitive student engagement in distance learning has been a challenge in higher education institutions. In this study, we implemented a non-self-report method assessing students’ heart rate data to identify the cognitive engagement during active learning activities. Additionally, as a supplementary tool, we applied a previously validated self-report method. This study was performed in distance learning lessons on a group of university students in Bogota, Colombia. After data analysis, we validated five hypotheses and compared the results from both methods. The results confirmed that the heart rate assessment had a statistically significant difference with respect to the baseline during active learning activities, and this variance could be positive or negative. In addition, the results show that if students are previously advised that they will have to develop an a new task after a passive learning activity (such as a video projection), their heart rate will tend to increase and consequently, their cognitive engagement will also increase. We expect this study to provide input for future research assessing student cognitive engagement using physiological parameters as a tool.

Postural effects of interoceptive imagery as a function of hypnotizability

The aim of the study was to investigate the effects of pleasant and unpleasant interoceptive imagery on postural control in participants with different hypnotizability, interoceptive sensibility and ability of imagery. Forty-one healthy individuals classified as high (highs), medium (mediums) and low hypnotizables (lows) according to the Stanford Hypnotic Susceptibility Scale, form A (SHSS, A) were characterized for Interoceptive Sensitivity (IS) through the Multisensory Assessment of Interoceptive Awareness (MAIA) and for organic mental imagery (ORG) through Betts' questionnaire. The experimental session included baseline closed eyes conditions preceding tasks of pleasant (P) and unpleasant imagery (U) and a neutral cognitive task (NT) while standing on a stabilometric platform. Subjective reports of vividness and pleasantness/unpleasantness of mental images were collected. Postural variables, ECG and pneumogram were acquired. Highs exhibited greater vividness of imagery than mediums/lows and larger Area of the Centre of Pressure (CoP), while mediums and lows decreased it during all tasks with respect to baseline conditions. Significant differences moderated by IS and ORG were found between highs and lows in the CoP Area. In all groups significant task related differences in the CoP Area were moderated only by IS. Cardiovascular variables were similar in the three groups, but differed among tasks and were influenced by IS and ORG. Our findings extend earlier observations on the role of hypnotisability in the postural correlates of sensorimotor imagery to interoceptive imagery, support the hypothesis that interoceptive sensibility moderates postural control and cardiorespiratory variables during interoceptive imagery, and confirm earlier reports of the absence of hypnotisability-related modulation of cardiorespiratory variables during emotional tasks.

Measuring the Rate of Information Exchange in Point-Process Data With Application to Cardiovascular Variability

The amount of information exchanged per unit of time between two dynamic processes is an important concept for the analysis of complex systems. Theoretical formulations and data-efficient estimators have been recently introduced for this quantity, known as the mutual information rate (MIR), allowing its continuous-time computation for event-based data sets measured as realizations of coupled point processes. This work presents the implementation of MIR for point process applications in Network Physiology and cardiovascular variability, which typically feature short and noisy experimental time series. We assess the bias of MIR estimated for uncoupled point processes in the frame of surrogate data, and we compensate it by introducing a corrected MIR (cMIR) measure designed to return zero values when the two processes do not exchange information. The method is first tested extensively in synthetic point processes including a physiologically-based model of the heartbeat dynamics and the blood pressure propagation times, where we show the ability of cMIR to compensate the negative bias of MIR and return statistically significant values even for weakly coupled processes. The method is then assessed in real point-process data measured from healthy subjects during different physiological conditions, showing that cMIR between heartbeat and pressure propagation times increases significantly during postural stress, though not during mental stress. These results document that cMIR reflects physiological mechanisms of cardiovascular variability related to the joint neural autonomic modulation of heart rate and arterial compliance.

A New Methodology Based on EMD and Nonlinear Measurements for Sudden Cardiac Death Detection

Heart diseases are among the most common death causes in the population. Particularly, sudden cardiac death (SCD) is the cause of 10% of the deaths around the world. For this reason, it is necessary to develop new methodologies that can predict this event in the earliest possible stage. This work presents a novel methodology to predict when a person can develop an SCD episode before it occurs. It is based on the adroit combination of the empirical mode decomposition, nonlinear measurements, such as the Higuchi fractal and permutation entropy, and a neural network. The obtained results show that the proposed methodology is capable of detecting an SCD episode 25 min before it appears with a 94% accuracy. The main benefits of the proposal are: (1) an improved detection time of 25% compared with previously published works, (2) moderate computational complexity since only two features are used, and (3) it uses the raw ECG without any preprocessing stage, unlike recent previous works.