1
|
Bian Y, Küster D, Liu H, Krumhuber EG. Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models. SENSORS (BASEL, SWITZERLAND) 2023; 24:126. [PMID: 38202988 PMCID: PMC10781259 DOI: 10.3390/s24010126] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 11/30/2023] [Accepted: 12/21/2023] [Indexed: 01/12/2024]
Abstract
This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.
Collapse
Affiliation(s)
- Yifan Bian
- Department of Experimental Psychology, University College London, London WC1H 0AP, UK;
| | - Dennis Küster
- Department of Mathematics and Computer Science, University of Bremen, 28359 Bremen, Germany; (D.K.); (H.L.)
| | - Hui Liu
- Department of Mathematics and Computer Science, University of Bremen, 28359 Bremen, Germany; (D.K.); (H.L.)
| | - Eva G. Krumhuber
- Department of Experimental Psychology, University College London, London WC1H 0AP, UK;
| |
Collapse
|
2
|
McDonald DQ, DeJardin E, Sariyanidi E, Herrington JD, Tunç B, Zampella CJ, Schultz RT. Predicting Autism from Head Movement Patterns during Naturalistic Social Interactions. PROCEEDINGS OF THE 2023 7TH INTERNATIONAL CONFERENCE ON MEDICAL AND HEALTH INFORMATICS (ICMHI 2023) : MAY 12-14, 2023, KYOTO, JAPAN. INTERNATIONAL CONFERENCE ON MEDICAL AND HEALTH INFORMATICS (7TH : 2023 : KYOTO, JAPAN) 2023; 2023:55-60. [PMID: 38699395 PMCID: PMC11064057 DOI: 10.1145/3608298.3608309] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2024]
Abstract
Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized in part by difficulties in verbal and nonverbal social communication. Evidence indicates that autistic people, compared to neurotypical peers, exhibit differences in head movements, a key form of nonverbal communication. Despite the crucial role of head movements in social communication, research on this nonverbal cue is relatively scarce compared to other forms of nonverbal communication, such as facial expressions and gestures. There is a need for scalable, reliable, and accurate instruments for measuring head movements directly within the context of social interactions. In this study, we used computer vision and machine learning to examine the head movement patterns of neurotypical and autistic individuals during naturalistic, face-to-face conversations, at both the individual (monadic) and interpersonal (dyadic) levels. Our model predicts diagnostic status using dyadic head movement data with an accuracy of 80%, highlighting the value of head movement as a marker of social communication. The monadic data pipeline had lower accuracy (69.2%) compared to the dyadic approach, emphasizing the importance of studying back-and-forth social communication cues within a true social context. The proposed classifier is not intended for diagnostic purposes, and future research should replicate our findings in larger, more representative samples.
Collapse
Affiliation(s)
| | - Ellis DeJardin
- Children's Hospital of Philadelphia Philadelphia, PA, USA
| | | | - John D Herrington
- Children's Hospital of Philadelphia Philadelphia, PA, USA
- University of Pennsylvania Philadelphia, PA, USA
| | - Birkan Tunç
- Children's Hospital of Philadelphia Philadelphia, PA, USA
- University of Pennsylvania Philadelphia, PA, USA
| | | | - Robert T Schultz
- Children's Hospital of Philadelphia Philadelphia, PA, USA
- University of Pennsylvania Philadelphia, PA, USA
| |
Collapse
|
3
|
Onal Ertugrul I, Ahn YA, Bilalpur M, Messinger DS, Speltz ML, Cohn JF. Infant AFAR: Automated facial action recognition in infants. Behav Res Methods 2023; 55:1024-1035. [PMID: 35538295 PMCID: PMC9646921 DOI: 10.3758/s13428-022-01863-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/13/2022] [Indexed: 11/08/2022]
Abstract
Automated detection of facial action units in infants is challenging. Infant faces have different proportions, less texture, fewer wrinkles and furrows, and unique facial actions relative to adults. For these and related reasons, action unit (AU) detectors that are trained on adult faces may generalize poorly to infant faces. To train and test AU detectors for infant faces, we trained convolutional neural networks (CNN) in adult video databases and fine-tuned these networks in two large, manually annotated, infant video databases that differ in context, head pose, illumination, video resolution, and infant age. AUs were those central to expression of positive and negative emotion. AU detectors trained in infants greatly outperformed ones trained previously in adults. Training AU detectors across infant databases afforded greater robustness to between-database differences than did training database specific AU detectors and outperformed previous state-of-the-art in infant AU detection. The resulting AU detection system, which we refer to as Infant AFAR (Automated Facial Action Recognition), is available to the research community for further testing and applications in infant emotion, social interaction, and related topics.
Collapse
|
4
|
McDonald DQ, Zampella CJ, Sariyanidi E, Manakiwala A, DeJardin E, Herrington JD, Schultz RT, Tunç B. Head Movement Patterns during Face-to-Face Conversations Vary with Age. ICMI'22 COMPANION : COMPANION PUBLICATION OF THE 2022 INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION : NOVEMBER 7-11, 2022, BANGALORE, INDIA. ICMI (CONFERENCE) (2022 : BANGALORE, INDIA) 2022; 2022:185-195. [PMID: 37975062 PMCID: PMC10652276 DOI: 10.1145/3536220.3563366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/19/2023]
Abstract
Advances in computational behavior analysis have the potential to increase our understanding of behavioral patterns and developmental trajectories in neurotypical individuals, as well as in individuals with mental health conditions marked by motor, social, and emotional difficulties. This study focuses on investigating how head movement patterns during face-to-face conversations vary with age from childhood through adulthood. We rely on computer vision techniques due to their suitability for analysis of social behaviors in naturalistic settings, since video data capture can be unobtrusively embedded within conversations between two social partners. The methods in this work include unsupervised learning for movement pattern clustering, and supervised classification and regression as a function of age. The results demonstrate that 3-minute video recordings of head movements during conversations show patterns that distinguish between participants that are younger vs. older than 12 years with 78% accuracy. Additionally, we extract relevant patterns of head movement upon which the age distinction was determined by our models.
Collapse
Affiliation(s)
| | | | | | - Aashvi Manakiwala
- University of Pennsylvania, Philadelphia, PA, USA
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
| | - Ellis DeJardin
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
| | - John D Herrington
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
- University of Pennsylvania, Philadelphia, PA, USA
| | - Robert T Schultz
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
- University of Pennsylvania, Philadelphia, PA, USA
| | - Birkan Tunç
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
- University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
5
|
Narrative as co-regulation: A review of embodied narrative in infant development. Infant Behav Dev 2022; 68:101747. [PMID: 35839557 DOI: 10.1016/j.infbeh.2022.101747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Revised: 07/01/2022] [Accepted: 07/03/2022] [Indexed: 11/21/2022]
Abstract
We review evidence of non-verbal, embodied narratives in human infancy to better understand their form and function as generators of common experience, regulation, and learning. We examine their development prior to the onset of language, with a view to improve understanding of narrative as regular motifs or schemas of early experience in both solitary and social engagement. Embodied narratives are composed of regular patterns of interest, arousal, affect, and intention that yield a characteristic four-part structure of (i) introduction, (ii) development, (iii) climax, and (iv) resolution. Made with others these form co-created shared acts of meaning, and are parsed in time with discreet beginnings and endings that allow a regular pattern to frame and give predictive understanding for prospective regulation (especially important within social contexts) that safely returns to baseline again. This characteristic pattern, co-created between infant and adult from the beginning of life, allows the infant to contribute to, and learn, the patterns of its culture. We conclude with a view on commonalities and differences of co-created narrative in non-human primates, and discuss implications of disruption to narrative co-creation for developmental psychopathology.
Collapse
|
6
|
Provenza NR, Sheth SA, Dastin-van Rijn EM, Mathura RK, Ding Y, Vogt GS, Avendano-Ortega M, Ramakrishnan N, Peled N, Gelin LFF, Xing D, Jeni LA, Ertugrul IO, Barrios-Anderson A, Matteson E, Wiese AD, Xu J, Viswanathan A, Harrison MT, Bijanki KR, Storch EA, Cohn JF, Goodman WK, Borton DA. Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in obsessive-compulsive disorder. Nat Med 2021; 27:2154-2164. [PMID: 34887577 PMCID: PMC8800455 DOI: 10.1038/s41591-021-01550-z] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Accepted: 09/22/2021] [Indexed: 01/02/2023]
Abstract
Detection of neural signatures related to pathological behavioral states could enable adaptive deep brain stimulation (DBS), a potential strategy for improving efficacy of DBS for neurological and psychiatric disorders. This approach requires identifying neural biomarkers of relevant behavioral states, a task best performed in ecologically valid environments. Here, in human participants with obsessive-compulsive disorder (OCD) implanted with recording-capable DBS devices, we synchronized chronic ventral striatum local field potentials with relevant, disease-specific behaviors. We captured over 1,000 h of local field potentials in the clinic and at home during unstructured activity, as well as during DBS and exposure therapy. The wide range of symptom severity over which the data were captured allowed us to identify candidate neural biomarkers of OCD symptom intensity. This work demonstrates the feasibility and utility of capturing chronic intracranial electrophysiology during daily symptom fluctuations to enable neural biomarker identification, a prerequisite for future development of adaptive DBS for OCD and other psychiatric disorders.
Collapse
Affiliation(s)
- Nicole R Provenza
- Brown University School of Engineering, Providence, RI, USA
- Charles Stark Draper Laboratory, Cambridge, MA, USA
| | - Sameer A Sheth
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA
| | | | - Raissa K Mathura
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA
| | - Yaohan Ding
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Gregory S Vogt
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Michelle Avendano-Ortega
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Nithya Ramakrishnan
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Noam Peled
- MGH/HST Martinos Center for Biomedical Imaging, Charlestown, MA, USA
- Harvard Medical School, Cambridge, MA, USA
| | | | - David Xing
- Brown University School of Engineering, Providence, RI, USA
| | - Laszlo A Jeni
- Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Itir Onal Ertugrul
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, the Netherlands
| | | | - Evan Matteson
- Brown University School of Engineering, Providence, RI, USA
| | - Andrew D Wiese
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
- Department of Psychology, University of Missouri-Kansas City, Kansas City, MO, USA
| | - Junqian Xu
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
- Department of Radiology, Baylor College of Medicine, Houston, TX, USA
| | - Ashwin Viswanathan
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA
| | | | - Kelly R Bijanki
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Eric A Storch
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Jeffrey F Cohn
- Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA
| | - Wayne K Goodman
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - David A Borton
- Brown University School of Engineering, Providence, RI, USA.
- Carney Institute for Brain Science, Brown University, Providence, RI, USA.
- Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs, Providence, RI, USA.
| |
Collapse
|
7
|
Chen M, Chow SM, Hammal Z, Messinger DS, Cohn JF. A Person- and Time-Varying Vector Autoregressive Model to Capture Interactive Infant-Mother Head Movement Dynamics. MULTIVARIATE BEHAVIORAL RESEARCH 2021; 56:739-767. [PMID: 32530313 PMCID: PMC8763288 DOI: 10.1080/00273171.2020.1762065] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Head movement is an important but often overlooked component of emotion and social interaction. Examination of regularity and differences in head movements of infant-mother dyads over time and across dyads can shed light on whether and how mothers and infants alter their dynamics over the course of an interaction to adapt to each others. One way to study these emergent differences in dynamics is to allow parameters that govern the patterns of interactions to change over time, and according to person- and dyad-specific characteristics. Using two estimation approaches to implement variations of a vector-autoregressive model with time-varying coefficients, we investigated the dynamics of automatically-tracked head movements in mothers and infants during the Face-Face/Still-Face Procedure (SFP) with 24 infant-mother dyads. The first approach requires specification of a confirmatory model for the time-varying parameters as part of a state-space model, whereas the second approach handles the time-varying parameters in a semi-parametric ("mostly" model-free) fashion within a generalized additive modeling framework. Results suggested that infant-mother head movement dynamics varied in time both within and across episodes of the SFP, and varied based on infants' subsequently-assessed attachment security. Code for implementing the time-varying vector-autoregressive model using two R packages, dynr and mgcv, is provided.
Collapse
Affiliation(s)
| | | | - Zakia Hammal
- The Robotics Institute, Carnegie Mellon University
| | | | - Jeffrey F Cohn
- The Robotics Institute, Carnegie Mellon University
- University of Pittsburgh
| |
Collapse
|
8
|
Guida E, Scano A, Storm F, Biffi E, Reni G, Montirosso R. Mother-Infant Interaction Kinect Analysis (MIIKA): An automatic kinematic-based methodology for the investigation of interpersonal distance during early exchanges. Infant Behav Dev 2021; 63:101567. [PMID: 33894631 DOI: 10.1016/j.infbeh.2021.101567] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Revised: 01/29/2021] [Accepted: 04/13/2021] [Indexed: 11/29/2022]
Abstract
Interpersonal distance is a core aspect of mother-child interaction. While conventional measures based on human coders do not fully capture the dynamics of this feature, computational methods provide automatic measures which can detect even small changes and more accurate estimates both spatially and temporally. Using RGB-D sensors (Microsoft Kinect V2), the present study describes a setup to automatically examine interpersonal distance during mother-child interactions, termed Mother-Infant Interaction Kinect Analysis (MIIKA). First, the laboratory setting and the data extraction method are described. By using an ad-hoc algorithm for kinematic data extraction, MIIKA returns three metrics: barycenter position (distance and velocity of approach and separation), movements (number of small, medium and large approaches and separations) and contributions (proportional contributions of mother and child to approaches and separations). Secondly, preliminary MIIKA metrics are described for a non-clinical mother-child dyad as an exemplification of the protocol. As interpersonal distance can be affected by contingent situations, we detected mother-infant full skeleton during three interactional contexts characterized by different kinds of dyadic exchanges: a free play session, a task-oriented activity and an emotionally arousing condition. Results highlighted similarities and differences between the three interactional contexts. MIIKA appears to be a promising setup to automatically examine interpersonal distance in early mother-child interactions.
Collapse
Affiliation(s)
- Elena Guida
- 0-3 Center for the at-Risk Infant, Scientific Institute IRCCS "Eugenio Medea", Bosisio Parini, Lecco, Italy
| | - Alessandro Scano
- Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing (STIIMA), National Research Council of Italy (CNR), Milan, Italy
| | - Fabio Storm
- Bioengineering Laboratory, Scientific Institute, IRCCS "Eugenio Medea", Bosisio Parini, Lecco, Italy
| | - Emilia Biffi
- Bioengineering Laboratory, Scientific Institute, IRCCS "Eugenio Medea", Bosisio Parini, Lecco, Italy
| | - Gianluigi Reni
- Bioengineering Laboratory, Scientific Institute, IRCCS "Eugenio Medea", Bosisio Parini, Lecco, Italy
| | - Rosario Montirosso
- 0-3 Center for the at-Risk Infant, Scientific Institute IRCCS "Eugenio Medea", Bosisio Parini, Lecco, Italy.
| |
Collapse
|
9
|
Chong E, Clark-Whitney E, Southerland A, Stubbs E, Miller C, Ajodan EL, Silverman MR, Lord C, Rozga A, Jones RM, Rehg JM. Detection of eye contact with deep neural networks is as accurate as human experts. Nat Commun 2020; 11:6386. [PMID: 33318484 PMCID: PMC7736573 DOI: 10.1038/s41467-020-19712-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2020] [Accepted: 10/14/2020] [Indexed: 01/10/2023] Open
Abstract
Eye contact is among the most primary means of social communication used by humans. Quantification of eye contact is valuable as a part of the analysis of social roles and communication skills, and for clinical screening. Estimating a subject's looking direction is a challenging task, but eye contact can be effectively captured by a wearable point-of-view camera which provides a unique viewpoint. While moments of eye contact from this viewpoint can be hand-coded, such a process tends to be laborious and subjective. In this work, we develop a deep neural network model to automatically detect eye contact in egocentric video. It is the first to achieve accuracy equivalent to that of human experts. We train a deep convolutional network using a dataset of 4,339,879 annotated images, consisting of 103 subjects with diverse demographic backgrounds. 57 subjects have a diagnosis of Autism Spectrum Disorder. The network achieves overall precision of 0.936 and recall of 0.943 on 18 validation subjects, and its performance is on par with 10 trained human coders with a mean precision 0.918 and recall 0.946. Our method will be instrumental in gaze behavior analysis by serving as a scalable, objective, and accessible tool for clinicians and researchers.
Collapse
Affiliation(s)
- Eunji Chong
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA.
| | - Elysha Clark-Whitney
- Center for Autism and the Developing Brain, Weill Cornell Medicine, New York, USA
| | - Audrey Southerland
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA
| | - Elizabeth Stubbs
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA
| | - Chanel Miller
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA
| | - Eliana L Ajodan
- Center for Autism and the Developing Brain, Weill Cornell Medicine, New York, USA
| | - Melanie R Silverman
- Center for Autism and the Developing Brain, Weill Cornell Medicine, New York, USA
| | - Catherine Lord
- School of Medicine, University of California, Los Angeles, USA
| | - Agata Rozga
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA
| | - Rebecca M Jones
- Center for Autism and the Developing Brain, Weill Cornell Medicine, New York, USA
| | - James M Rehg
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA
| |
Collapse
|
10
|
Behavior and interaction imaging at 9 months of age predict autism/intellectual disability in high-risk infants with West syndrome. Transl Psychiatry 2020; 10:54. [PMID: 32066713 PMCID: PMC7026100 DOI: 10.1038/s41398-020-0743-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/07/2019] [Revised: 01/13/2020] [Accepted: 01/16/2020] [Indexed: 02/07/2023] Open
Abstract
Automated behavior analysis are promising tools to overcome current assessment limitations in psychiatry. At 9 months of age, we recorded 32 infants with West syndrome (WS) and 19 typically developing (TD) controls during a standardized mother-infant interaction. We computed infant hand movements (HM), speech turn taking of both partners (vocalization, pause, silences, overlap) and motherese. Then, we assessed whether multimodal social signals and interactional synchrony at 9 months could predict outcomes (autism spectrum disorder (ASD) and intellectual disability (ID)) of infants with WS at 4 years. At follow-up, 10 infants developed ASD/ID (WS+). The best machine learning reached 76.47% accuracy classifying WS vs. TD and 81.25% accuracy classifying WS+ vs. WS-. The 10 best features to distinguish WS+ and WS- included a combination of infant vocalizations and HM features combined with synchrony vocalization features. These data indicate that behavioral and interaction imaging was able to predict ASD/ID in high-risk children with WS.
Collapse
|
11
|
Infants’ Motor Activity During a Mother–Infant Interaction Alternating Silent and Singing Phases. JOURNAL OF MOTOR LEARNING AND DEVELOPMENT 2019. [DOI: 10.1123/jmld.2019-0001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Infants interact with their caregivers, using visual contact, smiling, vocalizations, and motor activity. Most of the studies on infant motor activity during dyadic interaction provided qualitative microanalyses. Few documented changes in infants’ general motor activity when facing social stimuli, but the movement analyses were carried out over large time windows. Following on from a previous study, we sought to explore rapid variations in motor activity during an interaction between mothers and their 6-month-old infants. The interaction featured short alternating silent and singing phases. Results showed that infants’ head, hand, and foot motor activity was related to their mothers’ behavior. Head movements, in particular, decreased during the songs and increased between them. The interindividual variability was strong at the hand and foot level and a k-mean cluster analysis showed three different group tendencies. Head stillness certainly expressed the infants’ attentional engagement in the interaction. Hand and foot movements appeared to be more variable and, at the foot level, presumably depended on the mother–infant communicative routines. Infants’ general motor activity could be an indicator of the infant’s engagement and reflect the dyad’s communicative routines during interactions.
Collapse
|
12
|
Egmose I, Væver MS, Smith-Nielsen J, Varni G, Køppe S. Motor activity and spatial proximity: Relationships to infant emotions and maternal postpartum depression. Infant Behav Dev 2019; 57:101335. [DOI: 10.1016/j.infbeh.2019.101335] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2018] [Revised: 06/11/2019] [Accepted: 06/16/2019] [Indexed: 12/27/2022]
|
13
|
Dynamics of Face and Head Movement in Infants with and without Craniofacial Microsomia: An Automatic Approach. PLASTIC AND RECONSTRUCTIVE SURGERY-GLOBAL OPEN 2019; 7:e2081. [PMID: 30859039 PMCID: PMC6382247 DOI: 10.1097/gox.0000000000002081] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2018] [Accepted: 10/26/2018] [Indexed: 11/26/2022]
Abstract
Supplemental Digital Content is available in the text. Background: Craniofacial microsomia (CFM) is a congenital condition associated with malformations of the bone and soft tissue of the face and the facial nerves, all of which have the potential to impair facial expressiveness. We investigated whether CFM-related variation in expressiveness is evident as early as infancy. Methods: Participants were 113 ethnically diverse 13-month-old infants (n = 63 cases with CFM and n = 50 unaffected matched controls). They were observed in 2 emotion induction tasks designed to elicit positive and negative effects. Facial and head movement was automatically measured using a computer vision–based approach. Expressiveness was quantified as the displacement, velocity, and acceleration of 49 facial landmarks (eg, lip corners) and head pitch and yaw. Results: For both cases and controls, all measures of expressiveness strongly differed between tasks. Case–control differences were limited to infants with microtia plus mandibular hypoplasia and other associated CFM features, which were the most common phenotypes and were characterized by decreased expressiveness relative to control infants. Conclusions: Infants with microtia plus mandibular hypoplasia and those with other associated CFM phenotypes were less facially expressive than same-aged peers. Both phenotypes were associated with more severe involvement than microtia alone, suggesting that infants with more severe CFM begin to diverge in expressiveness from controls by age 13 months. Further research is needed to both replicate the current findings and elucidate their developmental implications.
Collapse
|
14
|
Provenzi L, Scotto di Minico G, Giusti L, Guida E, Müller M. Disentangling the Dyadic Dance: Theoretical, Methodological and Outcomes Systematic Review of Mother-Infant Dyadic Processes. Front Psychol 2018; 9:348. [PMID: 29615947 PMCID: PMC5868133 DOI: 10.3389/fpsyg.2018.00348] [Citation(s) in RCA: 54] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2017] [Accepted: 03/01/2018] [Indexed: 02/01/2023] Open
Abstract
Background: During the last decades, the research on mother-infant dyad has produced a great amount of data, methods and theories, which largely contributed to set a revolution in the way we look at developmental changes during infancy and childhood. Very different constructs depict the different aspects of the "dyadic dance" occurring between a mother and her infant; nonetheless, a comprehensive and consistent systematization of these concepts in a coherent theoretical landscape is still lacking. Aim: In the present work, we aim at disentangling the different theoretical and methodological definitions of 9 dyadic constructs and we highlight their effects on infants' and children developmental outcomes. Methods: A literature search has been conducted on three databases-PubMed, Scopus, Web of Science. Three different reviews are reported here: (1) a review on the theoretical definitions of dyadic constructs; (2) a review of operational definitions, settings and methods of dyadic processes; (3) a systematic review of dyadic processes' outcomes for infants' and children developmental trajectories. Results: Two constructs emerged as wide meta-theoretical concepts (reciprocity and mutuality) and seven described specific processes (attunement, contingency, coordination, matching, mirroring, reparation, synchrony). A global model resuming the relationships among different processes is reported, which highlights the emergence of two specific cycles of dyadic functioning (i.e., matching-mismatching-reparation-synchrony; contingency, coordination, attunement, mirroring). A comprehensive review of the adopted measures is also provided. Finally, all the processes provided significant contributions to infants' behavioral, cognitive, and socio-emotional development during the first 3 years of age, but limited research has been conducted on specific processes (e.g. reparation and mirroring). Conclusion: The present study provides an original research-grounded framework to consider the different nature of mother-infant dyadic processes within a unified dyadic eco-system. Different levels of evidence emerged for the role of diverse mother-infant dyadic processes on infants' and children development. Open questions and future research directions are highlighted.
Collapse
Affiliation(s)
- Livio Provenzi
- Center for the at-Risk Infant, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Giunia Scotto di Minico
- Center for the at-Risk Infant, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Lorenzo Giusti
- Center for the at-Risk Infant, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Elena Guida
- Center for the at-Risk Infant, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Mitho Müller
- Department of Psychology, Ludwig-Maximilians-University Munich, Munich, Germany
| |
Collapse
|
15
|
Spodenkiewicz M, Aigrain J, Bourvis N, Dubuisson S, Chetouani M, Cohen D. Distinguish self- and hetero-perceived stress through behavioral imaging and physiological features. Prog Neuropsychopharmacol Biol Psychiatry 2018; 82:107-114. [PMID: 29191571 DOI: 10.1016/j.pnpbp.2017.11.023] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/05/2017] [Revised: 11/20/2017] [Accepted: 11/27/2017] [Indexed: 11/25/2022]
Abstract
Stress reactivity is a complex phenomenon associated to multiple and multimodal expressions. Response to stressors has an obvious survival function and may be seen as an internal regulation to adapt to threat or danger. The intensity of this internal response can be assessed as the self-perception of the stress response. In species with social organization, this response also serves a communicative function, so-called hetero-perception. Our study presents multimodal stress detection assessment - a new methodology combining behavioral imaging and physiological monitoring for analyzing stress from these two perspectives. The system is based on automatic extraction of 39 behavioral (2D+3D video recording) and 62 physiological (Nexus-10 recording) features during a socially evaluated mental arithmetic test. The analysis with machine learning techniques for automatic classification using Support Vector Machine (SVM) show that self-perception and hetero-perception of social stress are both close but different phenomena: self-perception was significantly correlated with hetero-perception but significantly differed from it. Also, assessing stress with SVM through multimodality gave excellent classification results (F1 score values: 0.9±0.012 for hetero-perception and 0.87±0.021 for self-perception). In the best selected feature subsets, we found some common behavioral and physiological features that allow classification of both self- and hetero-perceived stress. However, we also found the contributing features for automatic classifications had opposite distributions: self-perception classification was mainly based on physiological features and hetero-perception was mainly based on behavioral features.
Collapse
Affiliation(s)
- Michel Spodenkiewicz
- Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique, Paris, France; CEPOI EA 7388, Unité de Pédopsychiatrie de Liaison, Pôle de Santé Mentale, CHU Sud Réunion, Université de la Réunion, Saint-Pierre, France; CESP Inserm U1178, Groupe IPSOM, Paris, France.
| | - Jonathan Aigrain
- Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique, Paris, France
| | - Nadège Bourvis
- Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique, Paris, France; Département de Psychiatrie de l'Enfant et de l'Adolescent, AP-HP, GHU Pitié-Salpêtrière, Paris, France
| | - Séverine Dubuisson
- Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique, Paris, France
| | - Mohamed Chetouani
- Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique, Paris, France
| | - David Cohen
- Sorbonne Universités, UPMC Univ Paris 06, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique, Paris, France; Département de Psychiatrie de l'Enfant et de l'Adolescent, AP-HP, GHU Pitié-Salpêtrière, Paris, France.
| |
Collapse
|
16
|
Werner P, Al-Hamadi A, Limbrecht-Ecklundt K, Walter S, Traue HC. Head movements and postures as pain behavior. PLoS One 2018; 13:e0192767. [PMID: 29444153 PMCID: PMC5812618 DOI: 10.1371/journal.pone.0192767] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2017] [Accepted: 01/30/2018] [Indexed: 11/19/2022] Open
Abstract
Pain assessment can benefit from observation of pain behaviors, such as guarding or facial expression, and observational pain scales are widely used in clinical practice with nonverbal patients. However, little is known about head movements and postures in the context of pain. In this regard, we analyze videos of three publically available datasets. The BioVid dataset was recorded with healthy participants subjected to painful heat stimuli. In the BP4D dataset, healthy participants performed a cold-pressor test and several other tasks (meant to elicit emotion). The UNBC dataset videos show shoulder pain patients during range-of-motion tests to their affected and unaffected limbs. In all videos, participants were sitting in an upright position. We studied head movements and postures that occurred during the painful and control trials by measuring head orientation from video over time, followed by analyzing posture and movement summary statistics and occurrence frequencies of typical postures and movements. We found significant differences between pain and control trials with analyses of variance and binomial tests. In BioVid and BP4D, pain was accompanied by head movements and postures that tend to be oriented downwards or towards the pain site. We also found differences in movement range and speed in all three datasets. The results suggest that head movements and postures should be considered for pain assessment and research. As additional pain indicators, they possibly might improve pain management whenever behavior is assessed, especially in nonverbal individuals such as infants or patients with dementia. However, in advance more research is needed to identify specific head movements and postures in pain patients.
Collapse
Affiliation(s)
- Philipp Werner
- Neuro-Information Technology group, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
| | - Ayoub Al-Hamadi
- Neuro-Information Technology group, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
| | | | - Steffen Walter
- Medical Psychology, University Clinic for Psychosomatic Medicine and Psychotherapy, Ulm, Germany
| | - Harald C. Traue
- Medical Psychology, University Clinic for Psychosomatic Medicine and Psychotherapy, Ulm, Germany
| |
Collapse
|
17
|
Bourvis N, Singer M, Saint Georges C, Bodeau N, Chetouani M, Cohen D, Feldman R. Pre-linguistic infants employ complex communicative loops to engage mothers in social exchanges and repair interaction ruptures. ROYAL SOCIETY OPEN SCIENCE 2018; 5:170274. [PMID: 29410790 PMCID: PMC5792867 DOI: 10.1098/rsos.170274] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/13/2017] [Accepted: 12/14/2017] [Indexed: 06/07/2023]
Abstract
Language has long been identified as a powerful communicative tool among humans. Yet, pre-linguistic communication, which is common in many species, is also used by human infants prior to the acquisition of language. The potential communicational value of pre-linguistic vocal interactions between human infants and mothers has been studied in the past decades. With 120 dyads (mothers and three- or six-month-old infants), we used the classical Still Face Paradigm (SFP) in which mothers interact freely with their infants, then refrain from communication (Still Face, SF), and finally resume play. We employed innovative automated techniques to measure infant and maternal vocalization and pause, and dyadic parameters (infant response to mother, joint silence and overlap) and the emotional component of Infant Directed Speech (e-IDS) throughout the interaction. We showed that: (i) during the initial free play mothers use longer vocalizations and more e-IDS when they interact with older infants and (ii) infant boys exhibit longer vocalizations and shorter pauses than girls. (iii) During the SF and reunion phases, infants show marked and sustained changes in vocalizations but their mothers do not and (iv) mother-infant dyadic parameters increase in the reunion phase. Our quantitative results show that infants, from the age of three months, actively participate to restore the interactive loop after communicative ruptures long before vocalizations show clear linguistic meaning. Thus, auditory signals provide from early in life a channel by which infants co-create interactions, enhancing the mother-infant bond.
Collapse
Affiliation(s)
- Nadège Bourvis
- Service de Psychiatrie de l'Enfant et de l'Adolescent, Groupe Hospitalier Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Paris, France
- Equipe IMI2S, Institut des Systèmes Intelligents et de Robotique, UMR 7222, Université Paris-Sorbonne, Paris, France
- Pôle de Psychiatrie Infanto-Juvénile, Centre Hospitalier Intercommunal Toulon-La Seyne, Toulon, France
| | - Magi Singer
- Baruch Ivcher School of Psychology, Interdisciplinary Center, Herzlia, Israel
| | - Catherine Saint Georges
- Service de Psychiatrie de l'Enfant et de l'Adolescent, Groupe Hospitalier Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Paris, France
- Equipe IMI2S, Institut des Systèmes Intelligents et de Robotique, UMR 7222, Université Paris-Sorbonne, Paris, France
| | - Nicolas Bodeau
- Equipe IMI2S, Institut des Systèmes Intelligents et de Robotique, UMR 7222, Université Paris-Sorbonne, Paris, France
| | - Mohamed Chetouani
- Equipe IMI2S, Institut des Systèmes Intelligents et de Robotique, UMR 7222, Université Paris-Sorbonne, Paris, France
| | - David Cohen
- Service de Psychiatrie de l'Enfant et de l'Adolescent, Groupe Hospitalier Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Paris, France
- Equipe IMI2S, Institut des Systèmes Intelligents et de Robotique, UMR 7222, Université Paris-Sorbonne, Paris, France
| | - Ruth Feldman
- Baruch Ivcher School of Psychology, Interdisciplinary Center, Herzlia, Israel
| |
Collapse
|
18
|
Egmose I, Varni G, Cordes K, Smith-Nielsen J, Væver MS, Køppe S, Cohen D, Chetouani M. Relations between Automatically Extracted Motion Features and the Quality of Mother-Infant Interactions at 4 and 13 Months. Front Psychol 2017; 8:2178. [PMID: 29326626 PMCID: PMC5733485 DOI: 10.3389/fpsyg.2017.02178] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2017] [Accepted: 11/29/2017] [Indexed: 11/13/2022] Open
Abstract
Bodily movements are an essential component of social interactions. However, the role of movement in early mother-infant interaction has received little attention in the research literature. The aim of the present study was to investigate the relationship between automatically extracted motion features and interaction quality in mother-infant interactions at 4 and 13 months. The sample consisted of 19 mother-infant dyads at 4 months and 33 mother-infant dyads at 13 months. The coding system Coding Interactive Behavior (CIB) was used for rating the quality of the interactions. Kinetic energy of upper-body, arms and head motion was calculated and used as segmentation in order to extract coarse- and fine-grained motion features. Spearman correlations were conducted between the composites derived from the CIB and the coarse- and fine-grained motion features. At both 4 and 13 months, longer durations of maternal arm motion and infant upper-body motion were associated with more aversive interactions, i.e., more parent-led interactions and more infant negativity. Further, at 4 months, the amount of motion silence was related to more adaptive interactions, i.e., more sensitive and child-led interactions. Analyses of the fine-grained motion features showed that if the mother coordinates her head movements with her infant's head movements, the interaction is rated as more adaptive in terms of less infant negativity and less dyadic negative states. We found more and stronger correlations between the motion features and the interaction qualities at 4 compared to 13 months. These results highlight that motion features are related to the quality of mother-infant interactions. Factors such as infant age and interaction set-up are likely to modify the meaning and importance of different motion features.
Collapse
Affiliation(s)
- Ida Egmose
- Early Child Developmental Unit, Babylab, Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - Giovanna Varni
- LTCI, Télécom ParisTech, Université Paris-Saclay, Paris, France
| | - Katharina Cordes
- Early Child Developmental Unit, Babylab, Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - Johanne Smith-Nielsen
- Early Child Developmental Unit, Babylab, Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - Mette S Væver
- Early Child Developmental Unit, Babylab, Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - Simo Køppe
- Early Child Developmental Unit, Babylab, Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - David Cohen
- LTCI, Télécom ParisTech, Université Paris-Saclay, Paris, France
- Department of Child and Adolescent Psychiatry, Pitié-Salpêtrière Hospital, Sorbonne Université Pierre et Marie Curie, Paris, France
| | | |
Collapse
|
19
|
Viaux-Savelon S, Rosenblum O, Guedeney A, Diene G, Çabal-Berthoumieu S, Fichaux-Bourin P, Molinas C, Faye S, Valette M, Bascoul C, Cohen D, Tauber M. Dyssynchrony and perinatal psychopathology impact of child disease on parents-child interactions, the paradigm of Prader Willi syndrom. ACTA ACUST UNITED AC 2017; 110:427-433. [PMID: 28823614 DOI: 10.1016/j.jphysparis.2017.08.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2016] [Revised: 08/16/2017] [Accepted: 08/16/2017] [Indexed: 10/18/2022]
Abstract
BACKGROUND Infant-mother interaction is a set of bidirectional processes, where the baby is not only affected by the influences of his caregiver, but is also at the origin of considerable modifications. The recent discovery of biological correlates of synchrony during interaction validated its crucial value during child development. Here, we focus on the paradigmatic case of Prader-Willi Syndrome (PWS) where early endocrinal dysfunction is associated with severe hypotonia and early feeding disorder. As a consequence, parent-infant interaction is impaired. In a recent study (Tauber et al., 2017), OXT intranasal infusion was able to partially reverse the feeding phenotype, infant's behavior and brain connectivity. This article details the interaction profile found during feeding in these dyads and their improvement after OXT treatment. METHODS Eighteen infants (≤6months) with PWS were recruited and hospitalized 9days in a French reference center for PWS where they were treated with a short course of intranasal OXT. Social withdrawal behavior and mother-infant interaction were assessed on videos of feeding before and after treatment using the Alarm Distress Baby (ADBB) Scale and the Coding Interactive Behavior (CIB) Scale. Raters were blind to treatment status. RESULTS At baseline, infants with PWS showed hypotonia, low expressiveness of affects, fatigability and poor involvement in the relationship with severe withdrawal. Parents tended to adapt to their child difficulties, but the interaction was perturbed, tense, restricted and frequently intrusive with a forcing component during the feeding situation. After OXT treatment, infants were more alert, less fatigable, more expressive, and had less social withdrawal. They initiated mutual activities and were more engaged in relationships through gaze, behavior, and vocalizations. They had a better global tonicity with better handling. These modifications helped the parents to be more sensitive and the synchrony of the dyad was in a positive transactional spiral. CONCLUSION Dys-synchrony can be induced by children's pathology as well as parental pathology with emotional and developmental impact in the both cases. The PWS paradigm shows us the necessity to sustain early parents-child relationship to avoid establishment of a negative transactional pattern of interaction that can impact child's development.
Collapse
Affiliation(s)
- Sylvie Viaux-Savelon
- Departement of Child and Adolescent Psychiatry, AP-HP, Universtity Hospital Pitie Salpetriere, Paris, France; Institut des Systemes Intelligents et de Robotiques, CNRS, UMR 7222, University Paris 6, Pierre et Marie Curie, Paris, France.
| | - Ouriel Rosenblum
- Departement of Child and Adolescent Psychiatry, AP-HP, Universtity Hospital Pitie Salpetriere, Paris, France
| | - Antoine Guedeney
- Department of Child and Adolescent Psychiatry, University Hospital Bichat Claude Bernard, APHP, University Paris 7, Paris, France; Research Unit, INSERM U669 PSYGIAM, Paris, France
| | - Gwenaelle Diene
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France; INSERM UMR 1027-Université Toulouse III Hôpital Paule de Viguier, Toulouse, France
| | - Sophie Çabal-Berthoumieu
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France; Service de Psychiatrie de l'Enfant et de l'Adolescent, Hôpital des Enfants, Toulouse, France
| | - Pascale Fichaux-Bourin
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France
| | - Catherine Molinas
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France; Axe Pédiatrique du CIC 9302/INSERM, Hôpital des Enfants, Toulouse, France; INSERM U1043, Centre de Physiopathologie de Toulouse Purpan, UPS, France
| | - Sandy Faye
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France; Axe Pédiatrique du CIC 9302/INSERM, Hôpital des Enfants, Toulouse, France
| | - Marion Valette
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France; Axe Pédiatrique du CIC 9302/INSERM, Hôpital des Enfants, Toulouse, France
| | - Céline Bascoul
- Service de Psychiatrie de l'Enfant et de l'Adolescent, Hôpital des Enfants, Toulouse, France
| | - David Cohen
- Departement of Child and Adolescent Psychiatry, AP-HP, Universtity Hospital Pitie Salpetriere, Paris, France; Institut des Systemes Intelligents et de Robotiques, CNRS, UMR 7222, University Paris 6, Pierre et Marie Curie, Paris, France
| | - Maïthé Tauber
- Unité d'Endocrinologie, Obésité, Maladies Osseuses, Génétique et Gynécologie Médicale, Centre de Référence du Syndrome de Prader-Willi, Hôpital des Enfants, Toulouse, France; Axe Pédiatrique du CIC 9302/INSERM, Hôpital des Enfants, Toulouse, France; INSERM U1043, Centre de Physiopathologie de Toulouse Purpan, UPS, France
| |
Collapse
|
20
|
Interaction and behaviour imaging: a novel method to measure mother-infant interaction using video 3D reconstruction. Transl Psychiatry 2016; 6:e816. [PMID: 27219342 PMCID: PMC5070050 DOI: 10.1038/tp.2016.82] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2015] [Revised: 11/10/2015] [Accepted: 11/13/2015] [Indexed: 11/09/2022] Open
Abstract
Studying early interaction is essential for understanding development and psychopathology. Automatic computational methods offer the possibility to analyse social signals and behaviours of several partners simultaneously and dynamically. Here, 20 dyads of mothers and their 13-36-month-old infants were videotaped during mother-infant interaction including 10 extremely high-risk and 10 low-risk dyads using two-dimensional (2D) and three-dimensional (3D) sensors. From 2D+3D data and 3D space reconstruction, we extracted individual parameters (quantity of movement and motion activity ratio for each partner) and dyadic parameters related to the dynamics of partners heads distance (contribution to heads distance), to the focus of mutual engagement (percentage of time spent face to face or oriented to the task) and to the dynamics of motion activity (synchrony ratio, overlap ratio, pause ratio). Features are compared with blind global rating of the interaction using the coding interactive behavior (CIB). We found that individual and dyadic parameters of 2D+3D motion features perfectly correlates with rated CIB maternal and dyadic composite scores. Support Vector Machine classification using all 2D-3D motion features classified 100% of the dyads in their group meaning that motion behaviours are sufficient to distinguish high-risk from low-risk dyads. The proposed method may present a promising, low-cost methodology that can uniquely use artificial technology to detect meaningful features of human interactions and may have several implications for studying dyadic behaviours in psychiatry. Combining both global rating scales and computerized methods may enable a continuum of time scale from a summary of entire interactions to second-by-second dynamics.
Collapse
|