1
|
Scagliarini T, Sparacino L, Faes L, Marinazzo D, Stramaglia S. Gradients of O-information highlight synergy and redundancy in physiological applications. FRONTIERS IN NETWORK PHYSIOLOGY 2024; 3:1335808. [PMID: 38264338 PMCID: PMC10803408 DOI: 10.3389/fnetp.2023.1335808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 12/21/2023] [Indexed: 01/25/2024]
Abstract
The study of high order dependencies in complex systems has recently led to the introduction of statistical synergy, a novel quantity corresponding to a form of emergence in which patterns at large scales are not traceable from lower scales. As a consequence, several works in the last years dealt with the synergy and its counterpart, the redundancy. In particular, the O-information is a signed metric that measures the balance between redundant and synergistic statistical dependencies. In spite of its growing use, this metric does not provide insight about the role played by low-order scales in the formation of high order effects. To fill this gap, the framework for the computation of the O-information has been recently expanded introducing the so-called gradients of this metric, which measure the irreducible contribution of a variable (or a group of variables) to the high order informational circuits of a system. Here, we review the theory behind the O-information and its gradients and present the potential of these concepts in the field of network physiology, showing two new applications relevant to brain functional connectivity probed via functional resonance imaging and physiological interactions among the variability of heart rate, arterial pressure, respiration and cerebral blood flow.
Collapse
Affiliation(s)
- Tomas Scagliarini
- Dipartimento di Fisica e Astronomia G. Galilei, Università degli Studi di Padova, Padova, Italy
| | - Laura Sparacino
- Dipartimento di Ingegneria, Università di Palermo, Palermo, Italy
| | - Luca Faes
- Dipartimento di Ingegneria, Università di Palermo, Palermo, Italy
| | | | - Sebastiano Stramaglia
- Dipartimento Interateneo di Fisica, Università degli Studi di Bari Aldo Moro, Bari, Italy
- Center of Innovative Technologies for Signal Detection and Processing (TIRES), Università degli Studi di Bari Aldo Moro, Bari, Italy
| |
Collapse
|
2
|
Pichardo-Carmona EY, Reyes-Lagos JJ, Ceballos-Juárez RG, Ledesma-Ramírez CI, Mendieta-Zerón H, Peña-Castillo MÁ, Nsugbe E, Porta-García MÁ, Mina-Paz Y. Changes in the autonomic cardiorespiratory activity in parturient women with severe and moderate features of preeclampsia. Front Immunol 2023; 14:1190699. [PMID: 37724103 PMCID: PMC10505439 DOI: 10.3389/fimmu.2023.1190699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 08/03/2023] [Indexed: 09/20/2023] Open
Abstract
Background Cardiorespiratory coupling (CRC) is a physiological phenomenon that reflects the mutual interaction between the cardiac and respiratory control systems. It is mainly associated with efferent vagal activity from the central autonomic network. Few studies have explored the autonomic changes of CRC in preeclampsia, a critical obstetric complication related to possible autonomic dysfunctions and inflammatory disturbances. This study examined the autonomic mechanisms of CRC in women with severe and moderate preeclampsia and healthy controls by applying nonlinear methods based on information theory, such as mutual information (MI) and Renyi's mutual information (RMI) and the linear and nonlinear analysis of the Pulse-Respiration Quotient (PRQ). Methods We studied three groups of parturient women in the third trimester of pregnancy with a clinical diagnosis of preeclampsia without severe symptoms (P, 38.5 ± 1.4 weeks of pregnancy, n=19), preeclampsia with severe symptoms (SP, 37.5 ± 0.9 weeks of pregnancy, n=22), and normotensive control women (C, 39.1 ± 1.3 weeks of pregnancy, n=20). 10-minutes of abdominal electrocardiograms (ECG) and respiratory signals (RESP) were recorded in all the participants. Subsequently, we obtained the maternal beat-to-beat (RR) and breath-to-breath (BB) time series from ECG and RESP, respectively. The CRC between RR and BB was quantified by nonlinear methods based on information theory, such as MI and RMI, along with the analysis of the novel index of PRQ. Subsequently, we computed the mean PRQ (mPRQ) and the normalized permutation entropy (nPermEn_PRQ) from the PRQ time series generated from BB and RR. In addition, we examined the vagal activity in the three groups by the logarithm of the median of the distribution of the absolute values of successive RR differences (logRSA). Results The MI and RMI values were significantly lower (p<0.05) in the preeclamptic groups compared to the control group. However, no significant differences were found between the preeclamptic groups. The logRSA and nPermEn_PRQ indices were significantly lower (p<0.05) in SP compared to C and P. Conclusion Our data suggest that parturient women with severe and mild preeclampsia may manifest an altered cardiorespiratory coupling compared with normotensive control women. Disrupted CRC in severe preeclampsia could be associated with vagal withdrawal and less complex cardiorespiratory dynamics. The difference in vagal activity between the preeclamptic groups may suggest a further reduction in vagal activity associated with the severity of the disease.
Collapse
Affiliation(s)
| | | | | | | | - Hugo Mendieta-Zerón
- School of Medicine, Autonomous University of the State of Mexico (UAEMéx), Toluca, Mexico
- Mónica Pretelini Sáenz Maternal-Perinatal Hospital, Health Institute of the State of Mexico (ISEM), Toluca, Mexico
| | | | - Ejay Nsugbe
- Nsugbe Research Labs, Swindon, United Kingdom
| | | | - Yecid Mina-Paz
- Faculty of Health Sciences, Universidad Libre Seccional Cali, Cali, Colombia
| |
Collapse
|
3
|
Presseller EK, Lampe EW, Zhang F, Gable PA, Guetterman TC, Forman EM, Juarascio AS. Using Wearable Passive Sensing to Predict Binge Eating in Response to Negative Affect Among Individuals With Transdiagnostic Binge Eating: Protocol for an Observational Study. JMIR Res Protoc 2023; 12:e47098. [PMID: 37410522 PMCID: PMC10360009 DOI: 10.2196/47098] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 05/02/2023] [Accepted: 05/05/2023] [Indexed: 07/07/2023] Open
Abstract
BACKGROUND Binge eating (BE), characterized by eating a large amount of food accompanied by a sense of loss of control over eating, is a public health crisis. Negative affect is a well-established antecedent for BE. The affect regulation model of BE posits that elevated negative affect increases momentary risk for BE, as engaging in BE alleviates negative affect and reinforces the behavior. The eating disorder field's capacity to identify moments of elevated negative affect, and thus BE risk, has exclusively relied on ecological momentary assessment (EMA). EMA involves the completion of surveys in real time on one's smartphone to report behavioral, cognitive, and emotional symptoms throughout the day. Although EMA provides ecologically valid information, EMA surveys are often delivered only 5-6 times per day, involve self-report of affect intensity only, and are unable to assess affect-related physiological arousal. Wearable, psychophysiological sensors that measure markers of affect arousal including heart rate, heart rate variability, and electrodermal activity may augment EMA surveys to improve accurate real-time prediction of BE. These sensors can objectively and continuously measure biomarkers of nervous system arousal that coincide with affect, thus allowing them to measure affective trajectories on a continuous timescale, detect changes in negative affect before the individual is consciously aware of them, and reduce user burden to improve data completeness. However, it is unknown whether sensor features can distinguish between positive and negative affect states, given that physiological arousal may occur during both negative and positive affect states. OBJECTIVE The aims of this study are (1) to test the hypothesis that sensor features will distinguish positive and negative affect states in individuals with BE with >60% accuracy and (2) test the hypothesis that a machine learning algorithm using sensor data and EMA-reported negative affect to predict the occurrence of BE will predict BE with greater accuracy than an algorithm using EMA-reported negative affect alone. METHODS This study will recruit 30 individuals with BE who will wear Fitbit Sense 2 wristbands to passively measure heart rate and electrodermal activity and report affect and BE on EMA surveys for 4 weeks. Machine learning algorithms will be developed using sensor data to distinguish instances of high positive and high negative affect (aim 1) and to predict engagement in BE (aim 2). RESULTS This project will be funded from November 2022 to October 2024. Recruitment efforts will be conducted from January 2023 through March 2024. Data collection is anticipated to be completed in May 2024. CONCLUSIONS This study is anticipated to provide new insight into the relationship between negative affect and BE by integrating wearable sensor data to measure affective arousal. The findings from this study may set the stage for future development of more effective digital ecological momentary interventions for BE. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/47098.
Collapse
Affiliation(s)
- Emily K Presseller
- Department of Psychological and Brain Sciences, Drexel University, Philadelphia, PA, United States
- Center for Weight, Eating, and Lifestyle Science, Drexel University, Philadelphia, PA, United States
| | - Elizabeth W Lampe
- Department of Psychological and Brain Sciences, Drexel University, Philadelphia, PA, United States
- Center for Weight, Eating, and Lifestyle Science, Drexel University, Philadelphia, PA, United States
| | - Fengqing Zhang
- Department of Psychological and Brain Sciences, Drexel University, Philadelphia, PA, United States
| | - Philip A Gable
- Department of Psychological and Brain Sciences, University of Delaware, Newark, DE, United States
| | - Timothy C Guetterman
- Department of Family Medicine, University of Michigan Medical School, Ann Arbor, MI, United States
| | - Evan M Forman
- Department of Psychological and Brain Sciences, Drexel University, Philadelphia, PA, United States
- Center for Weight, Eating, and Lifestyle Science, Drexel University, Philadelphia, PA, United States
| | - Adrienne S Juarascio
- Department of Psychological and Brain Sciences, Drexel University, Philadelphia, PA, United States
- Center for Weight, Eating, and Lifestyle Science, Drexel University, Philadelphia, PA, United States
| |
Collapse
|
4
|
Wang X, Ren Y, Luo Z, He W, Hong J, Huang Y. Deep learning-based EEG emotion recognition: Current trends and future perspectives. Front Psychol 2023; 14:1126994. [PMID: 36923142 PMCID: PMC10009917 DOI: 10.3389/fpsyg.2023.1126994] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 01/11/2023] [Indexed: 03/03/2023] Open
Abstract
Automatic electroencephalogram (EEG) emotion recognition is a challenging component of human-computer interaction (HCI). Inspired by the powerful feature learning ability of recently-emerged deep learning techniques, various advanced deep learning models have been employed increasingly to learn high-level feature representations for EEG emotion recognition. This paper aims to provide an up-to-date and comprehensive survey of EEG emotion recognition, especially for various deep learning techniques in this area. We provide the preliminaries and basic knowledge in the literature. We review EEG emotion recognition benchmark data sets briefly. We review deep learning techniques in details, including deep belief networks, convolutional neural networks, and recurrent neural networks. We describe the state-of-the-art applications of deep learning techniques for EEG emotion recognition in detail. We analyze the challenges and opportunities in this field and point out its future directions.
Collapse
Affiliation(s)
- Xiaohu Wang
- School of Intelligent Manufacturing and Mechanical Engineering, Hunan Institute of Technology, Hengyang, China
| | - Yongmei Ren
- School of Electrical and Information Engineering, Hunan Institute of Technology, Hengyang, China
| | - Ze Luo
- School of Intelligent Manufacturing and Mechanical Engineering, Hunan Institute of Technology, Hengyang, China
| | - Wei He
- School of Electrical and Information Engineering, Hunan Institute of Technology, Hengyang, China
| | - Jun Hong
- School of Intelligent Manufacturing and Mechanical Engineering, Hunan Institute of Technology, Hengyang, China
| | - Yinzhen Huang
- School of Computer and Information Engineering, Hunan Institute of Technology, Hengyang, China
| |
Collapse
|
5
|
Candia-Rivera D, Sappia MS, Horschig JM, Colier WNJM, Valenza G. Confounding effects of heart rate, breathing rate, and frontal fNIRS on interoception. Sci Rep 2022; 12:20701. [PMID: 36450811 PMCID: PMC9712694 DOI: 10.1038/s41598-022-25119-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Accepted: 11/24/2022] [Indexed: 12/02/2022] Open
Abstract
Recent studies have established that cardiac and respiratory phases can modulate perception and related neural dynamics. While heart rate and respiratory sinus arrhythmia possibly affect interoception biomarkers, such as heartbeat-evoked potentials, the relative changes in heart rate and cardiorespiratory dynamics in interoceptive processes have not yet been investigated. In this study, we investigated the variation in heart and breathing rates, as well as higher functional dynamics including cardiorespiratory correlation and frontal hemodynamics measured with fNIRS, during a heartbeat counting task. To further investigate the functional physiology linked to changes in vagal activity caused by specific breathing rates, we performed the heartbeat counting task together with a controlled breathing rate task. The results demonstrate that focusing on heartbeats decreases breathing and heart rates in comparison, which may be part of the physiological mechanisms related to "listening" to the heart, the focus of attention, and self-awareness. Focusing on heartbeats was also observed to increase frontal connectivity, supporting the role of frontal structures in the neural monitoring of visceral inputs. However, cardiorespiratory correlation is affected by both heartbeats counting and controlled breathing tasks. Based on these results, we concluded that variations in heart and breathing rates are confounding factors in the assessment of interoceptive abilities and relative fluctuations in breathing and heart rates should be considered to be a mode of covariate measurement of interoceptive processes.
Collapse
Affiliation(s)
- Diego Candia-Rivera
- Bioengineering and Robotics Research Center E. Piaggio & Department of Information Engineering, School of Engineering, University of Pisa, 56122, Pisa, Italy.
| | - M Sofía Sappia
- Artinis Medical Systems, B.V., Einsteinweg 17, 6662 PW, Elst, The Netherlands
- Donders Institute for Brain, Behaviour and Cognition, Radboud University Nijmegen, 6525 EN, Nijmegen, The Netherlands
| | - Jörn M Horschig
- Artinis Medical Systems, B.V., Einsteinweg 17, 6662 PW, Elst, The Netherlands
| | - Willy N J M Colier
- Artinis Medical Systems, B.V., Einsteinweg 17, 6662 PW, Elst, The Netherlands
| | - Gaetano Valenza
- Bioengineering and Robotics Research Center E. Piaggio & Department of Information Engineering, School of Engineering, University of Pisa, 56122, Pisa, Italy
| |
Collapse
|
6
|
Affective State Recognition in Livestock—Artificial Intelligence Approaches. Animals (Basel) 2022; 12:ani12060759. [PMID: 35327156 PMCID: PMC8944789 DOI: 10.3390/ani12060759] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 03/15/2022] [Accepted: 03/16/2022] [Indexed: 12/21/2022] Open
Abstract
Simple Summary Emotions or affective states recognition in farm animals is an underexplored research domain. Despite significant advances in animal welfare research, animal affective state computing through the development and application of devices and platforms that can not only recognize but interpret and process the emotions, are in a nascent stage. The analysis and measurement of unique behavioural, physical, and biological characteristics offered by biometric sensor technologies and the affiliated complex and large data sets, opens the pathway for novel and realistic identification of individual animals amongst a herd or a flock. By capitalizing on the immense potential of biometric sensors, artificial intelligence enabled big data methods offer substantial advancement of animal welfare standards and meet the urgent needs of caretakers to respond effectively to maintain the wellbeing of their animals. Abstract Farm animals, numbering over 70 billion worldwide, are increasingly managed in large-scale, intensive farms. With both public awareness and scientific evidence growing that farm animals experience suffering, as well as affective states such as fear, frustration and distress, there is an urgent need to develop efficient and accurate methods for monitoring their welfare. At present, there are not scientifically validated ‘benchmarks’ for quantifying transient emotional (affective) states in farm animals, and no established measures of good welfare, only indicators of poor welfare, such as injury, pain and fear. Conventional approaches to monitoring livestock welfare are time-consuming, interrupt farming processes and involve subjective judgments. Biometric sensor data enabled by artificial intelligence is an emerging smart solution to unobtrusively monitoring livestock, but its potential for quantifying affective states and ground-breaking solutions in their application are yet to be realized. This review provides innovative methods for collecting big data on farm animal emotions, which can be used to train artificial intelligence models to classify, quantify and predict affective states in individual pigs and cows. Extending this to the group level, social network analysis can be applied to model emotional dynamics and contagion among animals. Finally, ‘digital twins’ of animals capable of simulating and predicting their affective states and behaviour in real time are a near-term possibility.
Collapse
|
7
|
Bao G, Yang K, Tong L, Shu J, Zhang R, Wang L, Yan B, Zeng Y. Linking Multi-Layer Dynamical GCN With Style-Based Recalibration CNN for EEG-Based Emotion Recognition. Front Neurorobot 2022; 16:834952. [PMID: 35280845 PMCID: PMC8907537 DOI: 10.3389/fnbot.2022.834952] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 01/24/2022] [Indexed: 11/25/2022] Open
Abstract
Electroencephalography (EEG)-based emotion computing has become one of the research hotspots of human-computer interaction (HCI). However, it is difficult to effectively learn the interactions between brain regions in emotional states by using traditional convolutional neural networks because there is information transmission between neurons, which constitutes the brain network structure. In this paper, we proposed a novel model combining graph convolutional network and convolutional neural network, namely MDGCN-SRCNN, aiming to fully extract features of channel connectivity in different receptive fields and deep layer abstract features to distinguish different emotions. Particularly, we add style-based recalibration module to CNN to extract deep layer features, which can better select features that are highly related to emotion. We conducted two individual experiments on SEED data set and SEED-IV data set, respectively, and the experiments proved the effectiveness of MDGCN-SRCNN model. The recognition accuracy on SEED and SEED-IV is 95.08 and 85.52%, respectively. Our model has better performance than other state-of-art methods. In addition, by visualizing the distribution of different layers features, we prove that the combination of shallow layer and deep layer features can effectively improve the recognition performance. Finally, we verified the important brain regions and the connection relationships between channels for emotion generation by analyzing the connection weights between channels after model learning.
Collapse
Affiliation(s)
- Guangcheng Bao
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Kai Yang
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Li Tong
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Jun Shu
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Rongkai Zhang
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Linyuan Wang
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Bin Yan
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Ying Zeng
- Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, China
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- *Correspondence: Ying Zeng
| |
Collapse
|
8
|
Filippini C, Di Crosta A, Palumbo R, Perpetuini D, Cardone D, Ceccato I, Di Domenico A, Merla A. Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach. SENSORS 2022; 22:s22051789. [PMID: 35270936 PMCID: PMC8914721 DOI: 10.3390/s22051789] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 02/21/2022] [Accepted: 02/22/2022] [Indexed: 12/18/2022]
Abstract
Extensive possibilities of applications have rendered emotion recognition ineluctable and challenging in the fields of computer science as well as in human-machine interaction and affective computing. Fields that, in turn, are increasingly requiring real-time applications or interactions in everyday life scenarios. However, while extremely desirable, an accurate and automated emotion classification approach remains a challenging issue. To this end, this study presents an automated emotion recognition model based on easily accessible physiological signals and deep learning (DL) approaches. As a DL algorithm, a Feedforward Neural Network was employed in this study. The network outcome was further compared with canonical machine learning algorithms such as random forest (RF). The developed DL model relied on the combined use of wearables and contactless technologies, such as thermal infrared imaging. Such a model is able to classify the emotional state into four classes, derived from the linear combination of valence and arousal (referring to the circumplex model of affect’s four-quadrant structure) with an overall accuracy of 70% outperforming the 66% accuracy reached by the RF model. Considering the ecological and agile nature of the technique used the proposed model could lead to innovative applications in the affective computing field.
Collapse
Affiliation(s)
- Chiara Filippini
- Department of Neurosciences, Imaging and Clinical Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (C.F.); (D.P.); (D.C.); (I.C.)
| | - Adolfo Di Crosta
- Department of Psychological, Health and Territorial Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (A.D.C.); (R.P.); (A.D.D.)
| | - Rocco Palumbo
- Department of Psychological, Health and Territorial Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (A.D.C.); (R.P.); (A.D.D.)
| | - David Perpetuini
- Department of Neurosciences, Imaging and Clinical Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (C.F.); (D.P.); (D.C.); (I.C.)
| | - Daniela Cardone
- Department of Neurosciences, Imaging and Clinical Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (C.F.); (D.P.); (D.C.); (I.C.)
| | - Irene Ceccato
- Department of Neurosciences, Imaging and Clinical Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (C.F.); (D.P.); (D.C.); (I.C.)
| | - Alberto Di Domenico
- Department of Psychological, Health and Territorial Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (A.D.C.); (R.P.); (A.D.D.)
| | - Arcangelo Merla
- Department of Neurosciences, Imaging and Clinical Sciences, University G. D’Annunzio of Chieti-Pescara, 9, 66100 Chieti, Italy; (C.F.); (D.P.); (D.C.); (I.C.)
- Correspondence: ; Tel.: +39-0871-3556-954
| |
Collapse
|
9
|
Siddiqui HUR, Shahzad HF, Saleem AA, Khan Khakwani AB, Rustam F, Lee E, Ashraf I, Dudley S. Respiration Based Non-Invasive Approach for Emotion Recognition Using Impulse Radio Ultra Wide Band Radar and Machine Learning. SENSORS 2021; 21:s21248336. [PMID: 34960430 PMCID: PMC8707312 DOI: 10.3390/s21248336] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 12/09/2021] [Accepted: 12/09/2021] [Indexed: 12/22/2022]
Abstract
Emotion recognition gained increasingly prominent attraction from a multitude of fields recently due to their wide use in human-computer interaction interface, therapy, and advanced robotics, etc. Human speech, gestures, facial expressions, and physiological signals can be used to recognize different emotions. Despite the discriminating properties to recognize emotions, the first three methods have been regarded as ineffective as the probability of human's voluntary and involuntary concealing the real emotions can not be ignored. Physiological signals, on the other hand, are capable of providing more objective, and reliable emotion recognition. Based on physiological signals, several methods have been introduced for emotion recognition, yet, predominantly such approaches are invasive involving the placement of on-body sensors. The efficacy and accuracy of these approaches are hindered by the sensor malfunctioning and erroneous data due to human limbs movement. This study presents a non-invasive approach where machine learning complements the impulse radio ultra-wideband (IR-UWB) signals for emotion recognition. First, the feasibility of using IR-UWB for emotion recognition is analyzed followed by determining the state of emotions into happiness, disgust, and fear. These emotions are triggered using carefully selected video clips to human subjects involving both males and females. The convincing evidence that different breathing patterns are linked with different emotions has been leveraged to discriminate between different emotions. Chest movement of thirty-five subjects is obtained using IR-UWB radar while watching the video clips in solitude. Extensive signal processing is applied to the obtained chest movement signals to estimate respiration rate per minute (RPM). The RPM estimated by the algorithm is validated by repeated measurements by a commercially available Pulse Oximeter. A dataset is maintained comprising gender, RPM, age, and associated emotions which are further used with several machine learning algorithms for automatic recognition of human emotions. Experiments reveal that IR-UWB possesses the potential to differentiate between different human emotions with a decent accuracy of 76% without placing any on-body sensors. Separate analysis for male and female participants reveals that males experience high arousal for happiness while females experience intense fear emotions. For disgust emotion, no large difference is found for male and female participants. To the best of the authors' knowledge, this study presents the first non-invasive approach using the IR-UWB radar for emotion recognition.
Collapse
Affiliation(s)
- Hafeez Ur Rehman Siddiqui
- Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan; (H.U.R.S.); (H.F.S.); (A.A.S.); (F.R.)
| | - Hina Fatima Shahzad
- Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan; (H.U.R.S.); (H.F.S.); (A.A.S.); (F.R.)
| | - Adil Ali Saleem
- Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan; (H.U.R.S.); (H.F.S.); (A.A.S.); (F.R.)
| | - Abdul Baqi Khan Khakwani
- Management and Information Technology, Jubail Industrial College, Al Jubail 35718, Saudi Arabia;
| | - Furqan Rustam
- Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan; (H.U.R.S.); (H.F.S.); (A.A.S.); (F.R.)
| | - Ernesto Lee
- Department of Computer Science, Broward College, Broward County, FL 33301, USA
- Correspondence: (E.L.); (I.A.)
| | - Imran Ashraf
- Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 38541, Korea
- Correspondence: (E.L.); (I.A.)
| | - Sandra Dudley
- School of Engineering, London South Bank University, London SE1 0AA, UK;
| |
Collapse
|
10
|
Assessment of Cognitive Student Engagement Using Heart Rate Data in Distance Learning during COVID-19. EDUCATION SCIENCES 2021. [DOI: 10.3390/educsci11090540] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Student engagement allows educational institutions to make better decisions regarding teaching methodologies, methods for evaluating the quality of education, and ways to provide timely feedback. Due to the COVID-19 pandemic, identifying cognitive student engagement in distance learning has been a challenge in higher education institutions. In this study, we implemented a non-self-report method assessing students’ heart rate data to identify the cognitive engagement during active learning activities. Additionally, as a supplementary tool, we applied a previously validated self-report method. This study was performed in distance learning lessons on a group of university students in Bogota, Colombia. After data analysis, we validated five hypotheses and compared the results from both methods. The results confirmed that the heart rate assessment had a statistically significant difference with respect to the baseline during active learning activities, and this variance could be positive or negative. In addition, the results show that if students are previously advised that they will have to develop an a new task after a passive learning activity (such as a video projection), their heart rate will tend to increase and consequently, their cognitive engagement will also increase. We expect this study to provide input for future research assessing student cognitive engagement using physiological parameters as a tool.
Collapse
|
11
|
Manetti R, Manzoni D, Orsini P, Sebastiani L, Santarcangelo EL. Postural effects of interoceptive imagery as a function of hypnotizability. Physiol Behav 2021; 229:113222. [PMID: 33127462 DOI: 10.1016/j.physbeh.2020.113222] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2020] [Revised: 10/20/2020] [Accepted: 10/24/2020] [Indexed: 11/30/2022]
Abstract
The aim of the study was to investigate the effects of pleasant and unpleasant interoceptive imagery on postural control in participants with different hypnotizability, interoceptive sensibility and ability of imagery. Forty-one healthy individuals classified as high (highs), medium (mediums) and low hypnotizables (lows) according to the Stanford Hypnotic Susceptibility Scale, form A (SHSS, A) were characterized for Interoceptive Sensitivity (IS) through the Multisensory Assessment of Interoceptive Awareness (MAIA) and for organic mental imagery (ORG) through Betts' questionnaire. The experimental session included baseline closed eyes conditions preceding tasks of pleasant (P) and unpleasant imagery (U) and a neutral cognitive task (NT) while standing on a stabilometric platform. Subjective reports of vividness and pleasantness/unpleasantness of mental images were collected. Postural variables, ECG and pneumogram were acquired. Highs exhibited greater vividness of imagery than mediums/lows and larger Area of the Centre of Pressure (CoP), while mediums and lows decreased it during all tasks with respect to baseline conditions. Significant differences moderated by IS and ORG were found between highs and lows in the CoP Area. In all groups significant task related differences in the CoP Area were moderated only by IS. Cardiovascular variables were similar in the three groups, but differed among tasks and were influenced by IS and ORG. Our findings extend earlier observations on the role of hypnotisability in the postural correlates of sensorimotor imagery to interoceptive imagery, support the hypothesis that interoceptive sensibility moderates postural control and cardiorespiratory variables during interoceptive imagery, and confirm earlier reports of the absence of hypnotisability-related modulation of cardiorespiratory variables during emotional tasks.
Collapse
Affiliation(s)
- Rachele Manetti
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
| | - Diego Manzoni
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
| | - Paolo Orsini
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
| | - Laura Sebastiani
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
| | - Enrica L Santarcangelo
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy.
| |
Collapse
|
12
|
Behar JA, Liu C, Zigel Y, Laguna P, Clifford GD. Editorial on Remote Health Monitoring: from chronic diseases to pandemics. Physiol Meas 2021; 41:100401. [PMID: 33393486 DOI: 10.1088/1361-6579/abbb6d] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
13
|
Mijatovic G, Pernice R, Perinelli A, Antonacci Y, Busacca A, Javorka M, Ricci L, Faes L. Measuring the Rate of Information Exchange in Point-Process Data With Application to Cardiovascular Variability. FRONTIERS IN NETWORK PHYSIOLOGY 2021; 1:765332. [PMID: 36925567 PMCID: PMC10013020 DOI: 10.3389/fnetp.2021.765332] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Accepted: 11/26/2021] [Indexed: 02/01/2023]
Abstract
The amount of information exchanged per unit of time between two dynamic processes is an important concept for the analysis of complex systems. Theoretical formulations and data-efficient estimators have been recently introduced for this quantity, known as the mutual information rate (MIR), allowing its continuous-time computation for event-based data sets measured as realizations of coupled point processes. This work presents the implementation of MIR for point process applications in Network Physiology and cardiovascular variability, which typically feature short and noisy experimental time series. We assess the bias of MIR estimated for uncoupled point processes in the frame of surrogate data, and we compensate it by introducing a corrected MIR (cMIR) measure designed to return zero values when the two processes do not exchange information. The method is first tested extensively in synthetic point processes including a physiologically-based model of the heartbeat dynamics and the blood pressure propagation times, where we show the ability of cMIR to compensate the negative bias of MIR and return statistically significant values even for weakly coupled processes. The method is then assessed in real point-process data measured from healthy subjects during different physiological conditions, showing that cMIR between heartbeat and pressure propagation times increases significantly during postural stress, though not during mental stress. These results document that cMIR reflects physiological mechanisms of cardiovascular variability related to the joint neural autonomic modulation of heart rate and arterial compliance.
Collapse
Affiliation(s)
- Gorana Mijatovic
- Faculty of Technical Science, University of Novi Sad, Novi Sad, Serbia
| | - Riccardo Pernice
- Department of Engineering, University of Palermo, Palermo, Italy
| | - Alessio Perinelli
- CIMeC, Center for Mind/Brain Sciences, University of Trento, Rovereto, Italy
| | - Yuri Antonacci
- Department of Physics and Chemistry "Emilio Segrè," University of Palermo, Palermo, Italy
| | | | - Michal Javorka
- Department of Physiology and Biomedical Center Martin, Jessenius Faculty of Medicine, Comenius University, Martin, Slovakia
| | - Leonardo Ricci
- Department of Physics, University of Trento, Trento, Italy
| | - Luca Faes
- Department of Engineering, University of Palermo, Palermo, Italy
| |
Collapse
|
14
|
Vargas-Lopez O, Amezquita-Sanchez JP, De-Santiago-Perez JJ, Rivera-Guillen JR, Valtierra-Rodriguez M, Toledano-Ayala M, Perez-Ramirez CA. A New Methodology Based on EMD and Nonlinear Measurements for Sudden Cardiac Death Detection. SENSORS (BASEL, SWITZERLAND) 2019; 20:E9. [PMID: 31861320 PMCID: PMC6983035 DOI: 10.3390/s20010009] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2019] [Revised: 12/06/2019] [Accepted: 12/11/2019] [Indexed: 02/01/2023]
Abstract
Heart diseases are among the most common death causes in the population. Particularly, sudden cardiac death (SCD) is the cause of 10% of the deaths around the world. For this reason, it is necessary to develop new methodologies that can predict this event in the earliest possible stage. This work presents a novel methodology to predict when a person can develop an SCD episode before it occurs. It is based on the adroit combination of the empirical mode decomposition, nonlinear measurements, such as the Higuchi fractal and permutation entropy, and a neural network. The obtained results show that the proposed methodology is capable of detecting an SCD episode 25 min before it appears with a 94% accuracy. The main benefits of the proposal are: (1) an improved detection time of 25% compared with previously published works, (2) moderate computational complexity since only two features are used, and (3) it uses the raw ECG without any preprocessing stage, unlike recent previous works.
Collapse
Affiliation(s)
- Olivia Vargas-Lopez
- ENAP RG, Department of Biomedical Engineering, Faculty of Engineering, Autonomous University of Queretaro, Queretaro 76144, Mexico; (O.V.-L.); (J.P.A.-S.)
| | - Juan P. Amezquita-Sanchez
- ENAP RG, Department of Biomedical Engineering, Faculty of Engineering, Autonomous University of Queretaro, Queretaro 76144, Mexico; (O.V.-L.); (J.P.A.-S.)
- ENAP RG, Department of Electromechanical Engineering, Faculty of Engineering, Autonomous University of Queretaro, San Juan del Rio, Queretaro 76807, Mexico; (J.J.D.-S.-P.); (J.R.R.-G.); (M.V.-R.)
| | - J. Jesus De-Santiago-Perez
- ENAP RG, Department of Electromechanical Engineering, Faculty of Engineering, Autonomous University of Queretaro, San Juan del Rio, Queretaro 76807, Mexico; (J.J.D.-S.-P.); (J.R.R.-G.); (M.V.-R.)
| | - Jesus R. Rivera-Guillen
- ENAP RG, Department of Electromechanical Engineering, Faculty of Engineering, Autonomous University of Queretaro, San Juan del Rio, Queretaro 76807, Mexico; (J.J.D.-S.-P.); (J.R.R.-G.); (M.V.-R.)
| | - Martin Valtierra-Rodriguez
- ENAP RG, Department of Electromechanical Engineering, Faculty of Engineering, Autonomous University of Queretaro, San Juan del Rio, Queretaro 76807, Mexico; (J.J.D.-S.-P.); (J.R.R.-G.); (M.V.-R.)
| | | | - Carlos A. Perez-Ramirez
- ENAP RG, Department of Biomedical Engineering, Faculty of Engineering, Autonomous University of Queretaro, Queretaro 76144, Mexico; (O.V.-L.); (J.P.A.-S.)
| |
Collapse
|