1
|
Cui X, Wang J, Xue S, Qin Z, Peng CK. Quantifying the accuracy of inter-beat intervals acquired from consumer-grade photoplethysmography wristbands using an electrocardiogram-aided information-based similarity approach. Physiol Meas 2024; 45:035002. [PMID: 38387061 DOI: 10.1088/1361-6579/ad2c14] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 02/22/2024] [Indexed: 02/24/2024]
Abstract
Objective. Although inter-beat intervals (IBI) and the derived heart rate variability (HRV) can be acquired through consumer-grade photoplethysmography (PPG) wristbands and have been applied in a variety of physiological and psychophysiological conditions, their accuracy is still unsatisfactory.Approach.In this study, 30 healthy participants concurrently wore two wristbands (E4 and Honor 5) and a gold-standard electrocardiogram (ECG) device under four conditions: resting, deep breathing with a frequency of 0.17 Hz and 0.1 Hz, and mental stress tasks. To quantitatively validate the accuracy of IBI acquired from PPG wristbands, this study proposed to apply an information-based similarity (IBS) approach to quantify the pattern similarity of the underlying dynamical temporal structures embedded in IBI time series simultaneously recorded using PPG wristbands and the ECG system. The occurrence frequency of basic patterns and their rankings were analyzed to calculate the IBS distance from gold-standard IBI, and to further calculate the signal-to-noise ratio (SNR) of the wristband IBI time series.Main results.The accuracies of both HRV and mental state classification were not satisfactory due to the low SNR in the wristband IBI. However, by rejecting data segments of SNR < 25, the Pearson correlation coefficients between the wristbands' HRV and the gold-standard HRV were increased from 0.542 ± 0.235 to 0.922 ± 0.120 for E4 and from 0.596 ± 0.227 to 0.859 ± 0.145 for Honor 5. The average accuracy of four-class mental state classification increased from 77.3% to 81.9% for E4 and from 79.3% to 83.3% for Honor 5.Significance.Consumer-grade PPG wristbands are acceptable for HR and HRV monitoring when removing low SNR segments. The proposed method can be applied for quantifying the accuracies of IBI and HRV indices acquired via any non-ECG system.
Collapse
Affiliation(s)
- Xingran Cui
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, People's Republic of China
- Center for Nonlinear Dynamics in Medicine, Southeast University, Nanjing, People's Republic of China
| | - Jing Wang
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, People's Republic of China
| | - Shan Xue
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, People's Republic of China
| | - Zeguang Qin
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, People's Republic of China
| | - Chung-Kang Peng
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, People's Republic of China
- Center for Nonlinear Dynamics in Medicine, Southeast University, Nanjing, People's Republic of China
- Center for Dynamical Biomarkers, Beth Israel Deaconess Medical Center/Harvard Medical School, Boston, United States of America
| |
Collapse
|
2
|
Gomez-Zaragoza L, Marin-Morales J, Vargas EP, Giglioli IAC, Raya MA. An Online Attachment Style Recognition System Based on Voice and Machine Learning. IEEE J Biomed Health Inform 2023; 27:5576-5587. [PMID: 37566508 DOI: 10.1109/jbhi.2023.3304369] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/13/2023]
Abstract
Attachment styles are known to have significant associations with mental and physical health. Specifically, insecure attachment leads individuals to higher risk of suffering from mental disorders and chronic diseases. The aim of this study is to develop an attachment recognition model that can distinguish between secure and insecure attachment styles from voice recordings, exploring the importance of acoustic features while also evaluating gender differences. A total of 199 participants recorded their responses to four open questions intended to trigger their attachment system using a web-based interrogation system. The recordings were processed to obtain the standard acoustic feature set eGeMAPS, and recursive feature elimination was applied to select the relevant features. Different supervised machine learning models were trained to recognize attachment styles using both gender-dependent and gender-independent approaches. The gender-independent model achieved a test accuracy of 58.88%, whereas the gender-dependent models obtained 63.88% and 83.63% test accuracy for women and men respectively, indicating a strong influence of gender on attachment style recognition and the need to consider them separately in further studies. These results also demonstrate the potential of acoustic properties for remote assessment of attachment style, enabling fast and objective identification of this health risk factor, and thus supporting the implementation of large-scale mobile screening systems.
Collapse
|
3
|
Maghfira TN, Krisnadhi AA, Basaruddin T, Pudjiati SRR. The Indonesian Young-Adult Attachment (IYAA): An audio-video dataset for behavioral young-adult attachment assessment. Data Brief 2023; 50:109599. [PMID: 37780464 PMCID: PMC10539883 DOI: 10.1016/j.dib.2023.109599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 09/14/2023] [Accepted: 09/15/2023] [Indexed: 10/03/2023] Open
Abstract
The attachment system is an innate human instinct to gain a sense of security as a form of self-defense from threats. Adults with secure attachment can maintain the balance of their relationships with themselves and significant others such as parents, romantic partners, and close friends. Generally, the adult attachment assessment data are collected primarily from subjective responses through questionnaires or interviews, which are closed to the research community. Attachment assessment from behavioral traits has also not been studied in depth because attachment-related behavioral data are still not openly available for research. This limits the scope of attachment assessment to new alternative innovations, such as the application of machine learning and deep learning-based approaches. This paper presents the Indonesian Young Adult Attachment (IYAA) dataset, a facial expression and speech audio dataset of Indonesian young adults in attachment projective-based assessment. The assessment contains two stages: exposure and response of 14 attachment-based stimuli. IYAA consists of audio-video data from age groups between 18-29 years old, with 20 male and 67 female subjects. It contains 1216 exposure videos, 1217 response videos, and 1217 speech response audios. Each data has a varying duration; the duration for exposure video ranges from 25 seconds to 1 minute 39 seconds, while for response video and speech response audio ranges from 40 seconds to 8 minutes and 25 seconds. The IYAA dataset is annotated into two kinds of labels: emotion and attachment. First, emotion labeling is annotated on each stimulus for all subject data (exposure videos, response videos, speech response audios). Each data is annotated into one or more labels among eight basic emotion categories (neutral, happy, sad, contempt, anger, disgust, surprised, fear) since each attachment-related event involves unconscious mental processes characterized by emotional changes. Second, each subject is annotated into one among three attachment style labels: secure, insecure-anxious, and insecure-avoidance. Given these two kinds of labeling, the IYAA dataset supports several research purposes, either using one kind of label separately or using them together for attachment classification research. It also supports innovative approaches to build automatic attachment classification through collaboration between the study of Behavioral, Developmental, and Social Psychology with Social Signal Processing.
Collapse
Affiliation(s)
| | | | - T. Basaruddin
- Computer Science Department, Universitas Indonesia, Depok 16424 Indonesia
| | | |
Collapse
|
4
|
Parra F, Benezeth Y, Yang F. Automatic Assessment of Emotion Dysregulation in American, French, and Tunisian Adults and New Developments in Deep Multimodal Fusion: Cross-sectional Study. JMIR Ment Health 2022; 9:e34333. [PMID: 35072643 PMCID: PMC8822434 DOI: 10.2196/34333] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 11/10/2021] [Accepted: 11/23/2021] [Indexed: 12/28/2022] Open
Abstract
BACKGROUND Emotion dysregulation is a key dimension of adult psychological functioning. There is an interest in developing a computer-based, multimodal, and automatic measure. OBJECTIVE We wanted to train a deep multimodal fusion model to estimate emotion dysregulation in adults based on their responses to the Multimodal Developmental Profile, a computer-based psychometric test, using only a small training sample and without transfer learning. METHODS Two hundred and forty-eight participants from 3 different countries took the Multimodal Developmental Profile test, which exposed them to 14 picture and music stimuli and asked them to express their feelings about them, while the software extracted the following features from the video and audio signals: facial expressions, linguistic and paralinguistic characteristics of speech, head movements, gaze direction, and heart rate variability derivatives. Participants also responded to the brief version of the Difficulties in Emotional Regulation Scale. We separated and averaged the feature signals that corresponded to the responses to each stimulus, building a structured data set. We transformed each person's per-stimulus structured data into a multimodal codex, a grayscale image created by projecting each feature's normalized intensity value onto a cartesian space, deriving each pixel's position by applying the Uniform Manifold Approximation and Projection method. The codex sequence was then fed to 2 network types. First, 13 convolutional neural networks dealt with the spatial aspect of the problem, estimating emotion dysregulation by analyzing each of the codified responses. These convolutional estimations were then fed to a transformer network that decoded the temporal aspect of the problem, estimating emotional dysregulation based on the succession of responses. We introduce a Feature Map Average Pooling layer, which computes the mean of the convolved feature maps produced by our convolution layers, dramatically reducing the number of learnable weights and increasing regularization through an ensembling effect. We implemented 8-fold cross-validation to provide a good enough estimation of the generalization ability to unseen samples. Most of the experiments mentioned in this paper are easily replicable using the associated Google Colab system. RESULTS We found an average Pearson correlation (r) of 0.55 (with an average P value of <.001) between ground truth emotion dysregulation and our system's estimation of emotion dysregulation. An average mean absolute error of 0.16 and a mean concordance correlation coefficient of 0.54 were also found. CONCLUSIONS In psychometry, our results represent excellent evidence of convergence validity, suggesting that the Multimodal Developmental Profile could be used in conjunction with this methodology to provide a valid measure of emotion dysregulation in adults. Future studies should replicate our findings using a hold-out test sample. Our methodology could be implemented more generally to train deep neural networks where only small training samples are available.
Collapse
Affiliation(s)
- Federico Parra
- LE2I EA 7508, Université Bourgogne Franche-Comté, Dijon, France
| | | | - Fan Yang
- LE2I EA 7508, Université Bourgogne Franche-Comté, Dijon, France
| |
Collapse
|
5
|
Ostovar S, Griffiths MD, Raeisi T, Hashim IHM. Path Analysis of the Relationship Between Optimism, Humor, Affectivity, and Marital Satisfaction Among Infertile Couples. Int J Ment Health Addict 2020. [DOI: 10.1007/s11469-020-00341-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/28/2022] Open
|
6
|
Schröder M, Lüdtke J, Fux E, Izat Y, Bolten M, Gloger-Tippelt G, Suess GJ, Schmid M. Attachment disorder and attachment theory - Two sides of one medal or two different coins? Compr Psychiatry 2019; 95:152139. [PMID: 31706154 DOI: 10.1016/j.comppsych.2019.152139] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/27/2019] [Revised: 10/10/2019] [Accepted: 10/14/2019] [Indexed: 01/04/2023] Open
Abstract
INTRODUCTION Currently, attachment quality and attachment disorder exist in parallel, but the mutual association is still insufficiently clarified. For policy makers and clinical experts, it can be difficult to differentiate between these constructs, but the distinction is crucial to develop mental-health services and effective treatment concepts. We aimed to investigate the association between attachment representations (AR) and attachment disorders (AD), including Reactive Attachment Disorder (RAD) and Disinhibited Social Engagement Disorder (DSED) in children aged between 5 and 9. METHODS A total of 135 children aged between 5 and 9 years (M=7.17 years, SD=1.40, 63% male) and their primary caregivers participated in the study. Children were interviewed with the story stem method to assess AR, and the primary caregiver completed diagnostic interviews and questionnaires on mental disorders, AD, emotional and behavioral problems, and intelligence and development. RESULTS The prevalence of AR in children with AD was 28.6% for the 'secure' form of AR, 17.1% for the 'insecure-avoidant' form, 25.7% for the 'insecure-ambivalent' form, and 28.6% for the 'disorganized' form. Prevalences of the various AR forms did not differ statistically significantly, indicating that AR is conceptionally distinct from AD. Children with disorganized attachment scored significantly lower on language and intelligence skills than children with secure attachment. AD was significantly associated with a higher number of comorbidities, emotional and behavioral problems, and lower language skills. CONCLUSIONS Longitudinal studies using standardized assessment instruments are needed to systematically provide comparable and reliable empirical findings to improve current understanding of AR and AD as well as their etiological models.
Collapse
Affiliation(s)
- Martin Schröder
- Psychiatric University Clinics (UPK), Department of Child and Adolescent Psychiatry (UPKKJ), University of Basel, Research Department, Schanzenstrasse 13, 4056, Basel, Switzerland; University of Lüneburg, Faculty of Education, Institute of Social work and Social Education, Universitätsallee 1, 21339, Lüneburg, Germany.
| | - Janine Lüdtke
- Psychiatric University Clinics (UPK), Department of Child and Adolescent Psychiatry (UPKKJ), University of Basel, Research Department, Schanzenstrasse 13, 4056, Basel, Switzerland
| | - Elodie Fux
- Psychiatric University Clinics (UPK), Department of Child and Adolescent Psychiatry (UPKKJ), University of Basel, Research Department, Schanzenstrasse 13, 4056, Basel, Switzerland
| | - Yonca Izat
- Vivantes Clinic Friedrichshain, Child and Adolescent Psychiatry Berlin, Child and Adolescent Psychiatry, Psychotherapy, Psychosomatic, Zadekstrasse 53, 12351, Berlin, Germany
| | - Margarete Bolten
- Psychiatric University Clinics (UPK), Department of Child and Adolescent Psychiatry (UPKKJ), University of Basel, Research Department, Schanzenstrasse 13, 4056, Basel, Switzerland
| | | | - Gerhard J Suess
- Hamburg University of Applied Sciences, Faculty Business & Social Sciences, Department Social Work, Alexanderstraße 1, 20099, Hamburg, Germany
| | - Marc Schmid
- Psychiatric University Clinics (UPK), Department of Child and Adolescent Psychiatry (UPKKJ), University of Basel, Research Department, Schanzenstrasse 13, 4056, Basel, Switzerland
| |
Collapse
|
7
|
Menghini L, Gianfranchi E, Cellini N, Patron E, Tagliabue M, Sarlo M. Stressing the accuracy: Wrist-worn wearable sensor validation over different conditions. Psychophysiology 2019; 56:e13441. [PMID: 31332802 DOI: 10.1111/psyp.13441] [Citation(s) in RCA: 61] [Impact Index Per Article: 12.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2018] [Revised: 04/29/2019] [Accepted: 06/24/2019] [Indexed: 01/26/2023]
Abstract
Wearable sensors are promising instruments for conducting both laboratory and ambulatory research in psychophysiology. However, scholars should be aware of their measurement error and the conditions in which accuracy is achieved. This study aimed to assess the accuracy of a wearable sensor designed for research purposes, the E4 wristband (Empatica, Milan, Italy), in measuring heart rate (HR), heart rate variability (HRV), and skin conductance (SC) over five laboratory conditions widely used in stress reactivity research (seated rest, paced breathing, orthostatic, Stroop, speech task) and two ecological conditions (slow walking, keyboard typing). Forty healthy participants concurrently wore the wristband and two gold standard measurement systems (i.e., electrocardiography and finger SC sensor). The wristband accuracy was determined by evaluating the signal quality and the correlations with and the Bland-Altman plots against gold standard-derived measurements. Moreover, exploratory analyses were performed to assess predictors of measurement error. Mean HR measures showed the best accuracy over all conditions. HRV measures showed satisfactory accuracy in seated rest, paced breathing, and recovery conditions but not in dynamic conditions, including speaking. Accuracy was diminished by wrist movements, cognitive and emotional stress, nonstationarity, and larger wrist circumferences. Wrist SC measures showed neither correlation nor visual resemblance with finger SC signal, suggesting that the two sites may reflect different phenomena. Future studies are needed to assess the responsivity of wrist SC to emotional and cognitive stress. Limitations and implications for laboratory and ambulatory research are discussed.
Collapse
Affiliation(s)
- Luca Menghini
- Department of General Psychology, University of Padova, Padova, Italy
| | | | - Nicola Cellini
- Department of General Psychology, University of Padova, Padova, Italy
| | - Elisabetta Patron
- Department of General Psychology, University of Padova, Padova, Italy
| | | | - Michela Sarlo
- Department of General Psychology, University of Padova, Padova, Italy
| |
Collapse
|
8
|
Biosignals reflect pair-dynamics in collaborative work: EDA and ECG study of pair-programming in a classroom environment. Sci Rep 2018; 8:3138. [PMID: 29453408 PMCID: PMC5816605 DOI: 10.1038/s41598-018-21518-3] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2017] [Accepted: 02/06/2018] [Indexed: 02/04/2023] Open
Abstract
Collaboration is a complex phenomenon, where intersubjective dynamics can greatly affect the productive outcome. Evaluation of collaboration is thus of great interest, and can potentially help achieve better outcomes and performance. However, quantitative measurement of collaboration is difficult, because much of the interaction occurs in the intersubjective space between collaborators. Manual observation and/or self-reports are subjective, laborious, and have a poor temporal resolution. The problem is compounded in natural settings where task-activity and response-compliance cannot be controlled. Physiological signals provide an objective mean to quantify intersubjective rapport (as synchrony), but require novel methods to support broad deployment outside the lab. We studied 28 student dyads during a self-directed classroom pair-programming exercise. Sympathetic and parasympathetic nervous system activation was measured during task performance using electrodermal activity and electrocardiography. Results suggest that (a) we can isolate cognitive processes (mental workload) from confounding environmental effects, and (b) electrodermal signals show role-specific but correlated affective response profiles. We demonstrate the potential for social physiological compliance to quantify pair-work in natural settings, with no experimental manipulation of participants required. Our objective approach has a high temporal resolution, is scalable, non-intrusive, and robust.
Collapse
|
9
|
Calvo RA, Dinakar K, Picard R, Christensen H, Torous J. Toward Impactful Collaborations on Computing and Mental Health. J Med Internet Res 2018; 20:e49. [PMID: 29426812 PMCID: PMC5889813 DOI: 10.2196/jmir.9021] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2017] [Revised: 12/14/2017] [Accepted: 12/16/2017] [Indexed: 12/26/2022] Open
Abstract
We describe an initiative to bring mental health researchers, computer scientists, human-computer interaction researchers, and other communities together to address the challenges of the global mental ill health epidemic. Two face-to-face events and one special issue of the Journal of Medical Internet Research were organized. The works presented in these events and publication reflect key state-of-the-art research in this interdisciplinary collaboration. We summarize the special issue articles and contextualize them to present a picture of the most recent research. In addition, we describe a series of collaborative activities held during the second symposium and where the community identified 5 challenges and their possible solutions.
Collapse
Affiliation(s)
- Rafael Alejandro Calvo
- Wellbeing Supportive Technology Lab, School of Electrical and Information Engineering, University of Sydney, Sydney, Australia
| | - Karthik Dinakar
- Massachusetts Institute of Technology Media Lab, Cambridge, MA, United States
| | - Rosalind Picard
- Massachusetts Institute of Technology Media Lab, Cambridge, MA, United States
| | | | - John Torous
- Division of Clinical Informatics, Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States.,Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| |
Collapse
|