1
|
Martin EA, Lian W, Oltmanns JR, Jonas KG, Samaras D, Hallquist MN, Ruggero CJ, Clouston SAP, Kotov R. Behavioral meaures of psychotic disorders: Using automatic facial coding to detect nonverbal expressions in video. J Psychiatr Res 2024; 176:9-17. [PMID: 38830297 DOI: 10.1016/j.jpsychires.2024.05.056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 04/11/2024] [Accepted: 05/29/2024] [Indexed: 06/05/2024]
Abstract
Emotional deficits in psychosis are prevalent and difficult to treat. In particular, much remains unknown about facial expression abnormalities, and a key reason is that expressions are very labor-intensive to code. Automatic facial coding (AFC) can remove this barrier. The current study sought to both provide evidence for the utility of AFC in psychosis for research purposes and to provide evidence that AFC are valid measures of clinical constructs. Changes of facial expressions and head position of participants-39 with schizophrenia/schizoaffective disorder (SZ), 46 with other psychotic disorders (OP), and 108 never psychotic individuals (NP)-were assessed via FaceReader, a commercially available automated facial expression analysis software, using video recorded during a clinical interview. We first examined the behavioral measures of the psychotic disorder groups and tested if they can discriminate between the groups. Next, we evaluated links of behavioral measures with clinical symptoms, controlling for group membership. We found the SZ group was characterized by significantly less variation in neutral expressions, happy expressions, arousal, and head movements compared to NP. These measures discriminated SZ from NP well (AUC = 0.79, sensitivity = 0.79, specificity = 0.67) but discriminated SZ from OP less well (AUC = 0.66, sensitivity = 0.77, specificity = 0.46). We also found significant correlations between clinician-rated symptoms and most behavioral measures (particularly happy expressions, arousal, and head movements). Taken together, these results suggest that AFC can provide useful behavioral measures of psychosis, which could improve research on non-verbal expressions in psychosis and, ultimately, enhance treatment.
Collapse
Affiliation(s)
- Elizabeth A Martin
- Department of Psychological Science, University of California, Irvine, CA, USA.
| | - Wenxuan Lian
- Department of Materials Science and Engineering and Department of Applied Math and Statistics, Stony Brook University, Stony Brook, NY, USA
| | - Joshua R Oltmanns
- Department of Psychiatry, Stony Brook University, Stony Brook, NY, USA
| | - Katherine G Jonas
- Department of Psychiatry, Stony Brook University, Stony Brook, NY, USA
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY, USA
| | - Michael N Hallquist
- Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Camilo J Ruggero
- Department of Psychology, University of Texas at Dallas, Richardson, TX, USA
| | - Sean A P Clouston
- Program in Public Health and Department of Family, Population, and Preventive Medicine, Renaissance School of Medicine, Stony Brook University, Stony Brook, NY, USA
| | - Roman Kotov
- Department of Psychiatry, Stony Brook University, Stony Brook, NY, USA.
| |
Collapse
|
2
|
Olson GM, Damme KSF, Cowan HR, Alliende LM, Mittal VA. Emotional tone in clinical high risk for psychosis: novel insights from a natural language analysis approach. Front Psychiatry 2024; 15:1389597. [PMID: 38803678 PMCID: PMC11128650 DOI: 10.3389/fpsyt.2024.1389597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Accepted: 04/29/2024] [Indexed: 05/29/2024] Open
Abstract
Background Individuals at clinical high risk (CHR) for psychosis experience subtle emotional disturbances that are traditionally difficult to assess, but natural language processing (NLP) methods may provide novel insight into these symptoms. We predicted that CHR individuals would express more negative emotionality and less emotional language when compared to controls. We also examined associations with symptomatology. Methods Participants included 49 CHR individuals and 42 healthy controls who completed a semi-structured narrative interview. Interview transcripts were analyzed using Linguistic Inquiry and Word Count (LIWC) to assess the emotional tone of the language (tone -the ratio of negative to positive language) and count positive/negative words used. Participants also completed clinical symptom assessments to determine CHR status and characterize symptoms (i.e., positive and negative symptom domains). Results The CHR group had more negative emotional tone compared to healthy controls (t=2.676, p=.009), which related to more severe positive symptoms (r2=.323, p=.013). The percentages of positive and negative words did not differ between groups (p's>.05). Conclusions Language analyses provided accessible, ecologically valid insight into affective dysfunction and psychosis risk symptoms. Natural language processing analyses unmasked differences in language for CHR that captured language tendencies that were more nuanced than the words that are chosen.
Collapse
Affiliation(s)
- Gabrielle M. Olson
- Department of Psychology, Northwestern University, Evanston, IL, United States
| | - Katherine S. F. Damme
- Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, United States
- Department of Psychiatry, Northwestern University, Chicago, IL, United States
| | - Henry R. Cowan
- Department of Psychiatry and Behavioral Health, The Ohio State University, Columbus, OH, United States
- Department of Psychology, Michigan State University, East Lansing, MI, United States
| | - Luz Maria Alliende
- Department of Psychology, Northwestern University, Evanston, IL, United States
- Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, United States
| | - Vijay A. Mittal
- Department of Psychology, Northwestern University, Evanston, IL, United States
- Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, United States
- Department of Psychiatry, Northwestern University, Chicago, IL, United States
- Medical Social Sciences, Northwestern University, Chicago, IL, United States
- Institute for Policy Research (IPR), Northwestern University, Chicago, IL, United States
| |
Collapse
|
3
|
Rodosky SE, Stephens JE, Hittner EF, Rompilla DB, Mittal VA, Haase CM. Facial expressions in adolescent-parent interactions and mental health: A proof-of-concept study. Emotion 2023; 23:2110-2115. [PMID: 36729505 PMCID: PMC10394109 DOI: 10.1037/emo0001216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
Parent-child relationships are hotbeds of emotion and play a key role in mental health. The present proof-of-concept study examined facial expressions of emotion during adolescent-parent interactions and links with internalizing mental health symptoms. Neutral, negative, and positive facial expressions were objectively measured in 28 parent-adolescent dyads during three 10-min dyadic interactions. Internalizing mental health symptoms were measured using anxiety and depressive symptom questionnaires. Data were analyzed using actor-partner interdependence modeling. Results revealed that higher levels of (a) adolescents' neutral facial expressions as well as (b) parents' negative facial expressions were associated with higher levels of adolescents' mental health symptoms. Findings did not support a robust link between (c) positive expressions and mental health symptoms. Together, these results demonstrate the utility of facial expressions of emotion during parent-child interactions as behavioral correlates of adolescents' internalizing mental health symptoms, highlight the need for replication with larger samples, and suggest directions for future research. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Collapse
Affiliation(s)
| | - Jacquelyn E Stephens
- Department of Medical Social Sciences, Osher Center for Integrative Medicine, Northwestern University Feinberg School of Medicine
| | - Emily F Hittner
- School of Education and Social Policy, Northwestern University
| | | | | | - Claudia M Haase
- School of Education and Social Policy, Northwestern University
| |
Collapse
|
4
|
Gupta T, Osborne KJ, Nadig A, Haase CM, Mittal VA. Alterations in facial expressions in individuals at risk for psychosis: a facial electromyography approach using emotionally evocative film clips. Psychol Med 2023; 53:5829-5838. [PMID: 36285533 PMCID: PMC10130238 DOI: 10.1017/s0033291722003087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
BACKGROUND Negative symptoms such as blunted facial expressivity are characteristic of schizophrenia. However, it is not well-understood if and what abnormalities are present in individuals at clinical high-risk (CHR) for psychosis. METHODS This experimental study employed facial electromyography (left zygomaticus major and left corrugator supercilia) in a sample of CHR individuals (N = 34) and healthy controls (N = 32) to detect alterations in facial expressions in response to emotionally evocative film clips and to determine links with symptoms. RESULTS Findings revealed that the CHR group showed facial blunting manifested in reduced zygomatic activity in response to an excitement (but not amusement, fear, or sadness) film clip compared to controls. Reductions in zygomatic activity in the CHR group emerged in response to the emotionally evocative peak period of the excitement film clip. Lower zygomaticus activity during the excitement clip was related to anxiety while lower rates of change in zygomatic activity during the excitement video clip were related to higher psychosis risk conversion scores. CONCLUSIONS Together, these findings inform vulnerability/disease driving mechanisms and biomarker and treatment development.
Collapse
Affiliation(s)
- Tina Gupta
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA
| | - K. Juston Osborne
- Department of Psychology, Northwestern University, Evanston, IL, USA
| | - Ajay Nadig
- Harvard/MIT MD-PhD Program, Harvard Medical School, Boston, MA, 02115
| | - Claudia M. Haase
- Department of Psychology, Northwestern University, Evanston, IL, USA
- School of Education and Social Policy, Northwestern University, Evanston, IL, USA
| | - Vijay A. Mittal
- Department of Psychology, Northwestern University, Evanston, IL, USA
| |
Collapse
|
5
|
Loch AA, Gondim JM, Argolo FC, Lopes-Rocha AC, Andrade JC, van de Bilt MT, de Jesus LP, Haddad NM, Cecchi GA, Mota NB, Gattaz WF, Corcoran CM, Ara A. Detecting at-risk mental states for psychosis (ARMS) using machine learning ensembles and facial features. Schizophr Res 2023; 258:45-52. [PMID: 37473667 PMCID: PMC10448183 DOI: 10.1016/j.schres.2023.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 04/26/2023] [Accepted: 07/10/2023] [Indexed: 07/22/2023]
Abstract
AIMS Our study aimed to develop a machine learning ensemble to distinguish "at-risk mental states for psychosis" (ARMS) subjects from control individuals from the general population based on facial data extracted from video-recordings. METHODS 58 non-help-seeking medication-naïve ARMS and 70 healthy subjects were screened from a general population sample. At-risk status was assessed with the Structured Interview for Prodromal Syndromes (SIPS), and "Subject's Overview" section was filmed (5-10 min). Several features were extracted, e.g., eye and mouth aspect ratio, Euler angles, coordinates from 51 facial landmarks. This elicited 649 facial features, which were further selected using Gradient Boosting Machines (AdaBoost combined with Random Forests). Data was split in 70/30 for training, and Monte Carlo cross validation was used. RESULTS Final model reached 83 % of mean F1-score, and balanced accuracy of 85 %. Mean area under the curve for the receiver operator curve classifier was 93 %. Convergent validity testing showed that two features included in the model were significantly correlated with Avolition (SIPS N2 item) and expression of emotion (SIPS N3 item). CONCLUSION Our model capitalized on short video-recordings from individuals recruited from the general population, effectively distinguishing between ARMS and controls. Results are encouraging for large-screening purposes in low-resource settings.
Collapse
Affiliation(s)
- Alexandre Andrade Loch
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil; Instituto Nacional de Biomarcadores em Neuropsiquiatria (INBION), Conselho Nacional de Desenvolvimento Científico e Tecnológico, Brazil.
| | - João Medrado Gondim
- Instituto de Computação, Universidade Federal da Bahia, Salvador, BA, Brazil
| | - Felipe Coelho Argolo
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil
| | - Ana Caroline Lopes-Rocha
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil
| | - Julio Cesar Andrade
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil
| | - Martinus Theodorus van de Bilt
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil; Instituto Nacional de Biomarcadores em Neuropsiquiatria (INBION), Conselho Nacional de Desenvolvimento Científico e Tecnológico, Brazil
| | - Leonardo Peroni de Jesus
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil
| | - Natalia Mansur Haddad
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil
| | | | - Natalia Bezerra Mota
- Instituto de Psiquiatria (IPUB), Departamento de Psiquiatria e Medicina Legal, Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil; Research Department at Motrix Lab - Motrix, Rio de Janeiro, Brazil
| | - Wagner Farid Gattaz
- Laboratório de Neurociencias (LIM 27), Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Brazil; Instituto Nacional de Biomarcadores em Neuropsiquiatria (INBION), Conselho Nacional de Desenvolvimento Científico e Tecnológico, Brazil
| | - Cheryl Mary Corcoran
- Icahn School of Medicine at Mount Sinai, New York, NY, USA; James J. Peters VA Medical Center Bronx, NY, USA
| | - Anderson Ara
- Statistics Department, Federal University of Paraná, Curitiba, PR, Brazil
| |
Collapse
|
6
|
Martin JC, Clark SR, Schubert KO. Towards a Neurophenomenological Understanding of Self-Disorder in Schizophrenia Spectrum Disorders: A Systematic Review and Synthesis of Anatomical, Physiological, and Neurocognitive Findings. Brain Sci 2023; 13:845. [PMID: 37371325 DOI: 10.3390/brainsci13060845] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 05/18/2023] [Accepted: 05/18/2023] [Indexed: 06/29/2023] Open
Abstract
The concept of anomalous self-experience, also termed Self-Disorder, has attracted both clinical and research interest, as empirical studies suggest such experiences specifically aggregate in and are a core feature of schizophrenia spectrum disorders. A comprehensive neurophenomenological understanding of Self-Disorder may improve diagnostic and therapeutic practice. This systematic review aims to evaluate anatomical, physiological, and neurocognitive correlates of Self-Disorder (SD), considered a core feature of Schizophrenia Spectrum Disorders (SSDs), towards developing a neurophenomenological understanding. A search of the PubMed database retrieved 285 articles, which were evaluated for inclusion using PRISMA guidelines. Non-experimental studies, studies with no validated measure of Self-Disorder, or those with no physiological variable were excluded. In total, 21 articles were included in the review. Findings may be interpreted in the context of triple-network theory and support a core dysfunction of signal integration within two anatomical components of the Salience Network (SN), the anterior insula and dorsal anterior cingulate cortex, which may mediate connectivity across both the Default Mode Network (DMN) and Fronto-Parietal Network (FPN). We propose a theoretical Triple-Network Model of Self-Disorder characterized by increased connectivity between the Salience Network (SN) and the DMN, increased connectivity between the SN and FPN, decreased connectivity between the DMN and FPN, and increased connectivity within both the DMN and FPN. We go on to describe translational opportunities for clinical practice and provide suggestions for future research.
Collapse
Affiliation(s)
- James C Martin
- Discipline of Psychiatry, Adelaide Medical School, The University of Adelaide, Adelaide, SA 5000, Australia
| | - Scott R Clark
- Discipline of Psychiatry, Adelaide Medical School, The University of Adelaide, Adelaide, SA 5000, Australia
- Basil Hetzel Institute, Woodville, SA 5011, Australia
| | - K Oliver Schubert
- Discipline of Psychiatry, Adelaide Medical School, The University of Adelaide, Adelaide, SA 5000, Australia
- Division of Mental Health, Northern Adelaide Local Health Network, SA Health, Adelaide, SA 5000, Australia
- Headspace Early Psychosis, Sonder, Adelaide, SA 5000, Australia
| |
Collapse
|
7
|
Ricard JR, Gupta T, Vargas T, Haase CM, Mittal VA. Genuine and non-genuine smiles in individuals meeting criteria for a clinical high-risk syndrome. Early Interv Psychiatry 2022; 16:875-882. [PMID: 34725928 PMCID: PMC9056581 DOI: 10.1111/eip.13233] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 08/21/2021] [Accepted: 10/19/2021] [Indexed: 01/17/2023]
Abstract
AIM Psychosis is characterized by both alterations in emotional functioning and environmental stressors including bullying victimization. Recent evidence suggests that some alterations in emotional functioning (e.g., blunted positive facial expressions) are already present in the psychosis risk period. Yet, some clinically relevant facial emotions have not been investigated such as genuine smiles (thought to reflect genuine positive emotions) and non-genuine smiles (thought to fake positive or mask negative emotions) in individuals meeting criteria for a clinical high-risk (CHR) syndrome. Further, despite a compelling conceptual basis to suggest a link between affective expression and exposure to environmental stress, to date, no investigations have sought to examine this association. Here, we aim to assess differences between a sample of CHR (N = 65) and control (N = 67) individuals in genuine and non-genuine smiles and associations with bullying victimization. METHODS Smiles (i.e., genuine; non-genuine) were objectively coded on a second-by-second basis using the Facial Action Coding System during a digitally recorded clinical interview segment. Bullying victimization was measured via parent report. RESULTS Findings revealed that the CHR group (1) showed blunted genuine (but not non-genuine) smiles compared to controls. Moreover, (2) bullying victimization was related to blunted genuine smiles, but not non-genuine smiles. CONCLUSION These findings expand our understanding of emotional alterations in this group with implications for diagnosis (highlighting blunted genuine smiles as a specific marker) and etiology (underscoring the role of bullying victimization in the etiology of emotional dysfunction).
Collapse
Affiliation(s)
- Jordyn R Ricard
- School of Education and Social Policy, Northwestern University, Evanston, Illinois, USA
| | - Tina Gupta
- Department of Psychology, Northwestern University, Evanston, Illinois, USA
| | - Teresa Vargas
- Department of Psychology, Northwestern University, Evanston, Illinois, USA
| | - Claudia M Haase
- School of Education and Social Policy and (by courtesy) Department of Psychology, Northwestern University, Evanston, Illinois, USA
| | - Vijay A Mittal
- Department of Psychology, Department of Psychiatry, Institute for Policy Research (IPR), Department of Medical Social Sciences, and Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston, Illinois, USA
| |
Collapse
|
8
|
Cowan T, Strauss GP, Raugh IM, Le TP, Cohen AS. How do social factors relate to blunted facial affect in schizophrenia? A digital phenotyping study using ambulatory video recordings. J Psychiatr Res 2022; 150:96-104. [PMID: 35366600 PMCID: PMC10036138 DOI: 10.1016/j.jpsychires.2022.03.024] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/17/2021] [Revised: 03/09/2022] [Accepted: 03/21/2022] [Indexed: 01/05/2023]
Abstract
Clinical interviews and laboratory-based emotional induction paradigms provide consistent evidence that facial affect is blunted in many individuals with schizophrenia. Although it is clear that blunted facial affect is not a by-product of diminished emotional experience in schizophrenia, factors contributing to blunted affect remain unclear. The current study used a combination of ambulatory video recordings that were evaluated via computerized facial affect analysis and concurrently completed ecological momentary assessment surveys to assess whether blunted affect reflects insufficient reactivity to affective or contextual factors. Specifically, whether individuals with schizophrenia require more intense affective experiences to produce expression, or whether they are less reactive to social factors (i.e. being in the presence of others, social motivation). Participants included outpatients with schizophrenia (n = 33) and healthy controls (n = 31) who completed six days of study procedures. Multilevel linear models were evaluated using both Null-Hypothesis Statistical Testing and Bayesian analyses. Individuals with schizophrenia displayed comparable expression of positive and negative emotion to controls during daily life, and no evidence was found for a different intensity of experience required for expression in either group. However, social factors differentially influenced facial expression in schizophrenia compared to controls, such that individuals with schizophrenia did not modulate their expressions based on social motivation to the same extent as controls. These findings suggest that social motivation may play an important role in determining when blunting occurs.
Collapse
Affiliation(s)
- Tovah Cowan
- Department of Psychology, Louisiana State University, United States; Center for Computation and Technology, Louisiana State University, United States
| | | | - Ian M Raugh
- Department of Psychology, University of Georgia, United States
| | - Thanh P Le
- Department of Psychology, Louisiana State University, United States
| | - Alex S Cohen
- Department of Psychology, Louisiana State University, United States; Center for Computation and Technology, Louisiana State University, United States.
| |
Collapse
|
9
|
Computerized analysis of facial expressions in serious mental illness. Schizophr Res 2022; 241:44-51. [PMID: 35074531 PMCID: PMC8978090 DOI: 10.1016/j.schres.2021.12.026] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 11/19/2021] [Accepted: 12/18/2021] [Indexed: 12/30/2022]
Abstract
Blunted facial affect is a transdiagnostic component of Serious Mental Illness (SMI) and is associated with a host of negative outcomes. However, blunted facial affect is a poorly understood phenomenon, with no known cures or treatments. A critical step in better understanding its phenotypic expression involves clarifying which facial expressions are altered in specific ways and under what contexts. The current literature suggests that individuals with SMI show decreased positive facial expressions, but typical, or even increased negative facial expressions during laboratory tasks. While this literature has coalesced around general trends, significantly more nuance is available regarding what components facial expressions are atypical and how those components are associated with increased severity of clinical ratings. The present project leveraged computerized facial analysis to test whether clinician-rated blunted affect is driven by decreases in duration, intensity, or frequency of positive versus other facial expressions during a structured clinical interview. Stable outpatients meeting criteria for SMI (N = 59) were examined. Facial expression did not generally vary as a function of clinical diagnosis. Overall, clinically-rated blunted affect was not associated with positive expressions, but was associated with decreased surprise and increased anger, sadness, and fear expressions. Blunted affect is not a monolithic lack of expressivity, and increased precision in operationally defining it is critical for uncovering its causes and maintaining factors. Our discussion focuses on this effort, and on advancing digital phenotyping of blunted facial affect more generally.
Collapse
|
10
|
Birnbaum ML, Abrami A, Heisig S, Ali A, Arenare E, Agurto C, Lu N, Kane JM, Cecchi G. Acoustic and Facial Features From Clinical Interviews for Machine Learning-Based Psychiatric Diagnosis: Algorithm Development. JMIR Ment Health 2022; 9:e24699. [PMID: 35072648 PMCID: PMC8822433 DOI: 10.2196/24699] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Revised: 04/29/2021] [Accepted: 12/01/2021] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND In contrast to all other areas of medicine, psychiatry is still nearly entirely reliant on subjective assessments such as patient self-report and clinical observation. The lack of objective information on which to base clinical decisions can contribute to reduced quality of care. Behavioral health clinicians need objective and reliable patient data to support effective targeted interventions. OBJECTIVE We aimed to investigate whether reliable inferences-psychiatric signs, symptoms, and diagnoses-can be extracted from audiovisual patterns in recorded evaluation interviews of participants with schizophrenia spectrum disorders and bipolar disorder. METHODS We obtained audiovisual data from 89 participants (mean age 25.3 years; male: 48/89, 53.9%; female: 41/89, 46.1%): individuals with schizophrenia spectrum disorders (n=41), individuals with bipolar disorder (n=21), and healthy volunteers (n=27). We developed machine learning models based on acoustic and facial movement features extracted from participant interviews to predict diagnoses and detect clinician-coded neuropsychiatric symptoms, and we assessed model performance using area under the receiver operating characteristic curve (AUROC) in 5-fold cross-validation. RESULTS The model successfully differentiated between schizophrenia spectrum disorders and bipolar disorder (AUROC 0.73) when aggregating face and voice features. Facial action units including cheek-raising muscle (AUROC 0.64) and chin-raising muscle (AUROC 0.74) provided the strongest signal for men. Vocal features, such as energy in the frequency band 1 to 4 kHz (AUROC 0.80) and spectral harmonicity (AUROC 0.78), provided the strongest signal for women. Lip corner-pulling muscle signal discriminated between diagnoses for both men (AUROC 0.61) and women (AUROC 0.62). Several psychiatric signs and symptoms were successfully inferred: blunted affect (AUROC 0.81), avolition (AUROC 0.72), lack of vocal inflection (AUROC 0.71), asociality (AUROC 0.63), and worthlessness (AUROC 0.61). CONCLUSIONS This study represents advancement in efforts to capitalize on digital data to improve diagnostic assessment and supports the development of a new generation of innovative clinical tools by employing acoustic and facial data analysis.
Collapse
Affiliation(s)
- Michael L Birnbaum
- Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States.,The Feinstein Institute for Medical Research, Northwell Health, Manhasset, NY, United States.,The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, United States
| | - Avner Abrami
- Computational Biology Center, IBM Research, Yorktown Heights, NY, United States
| | - Stephen Heisig
- Icahn School of Medicine at Mount Sinai, New York City, NY, United States
| | - Asra Ali
- Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States.,The Feinstein Institute for Medical Research, Northwell Health, Manhasset, NY, United States
| | - Elizabeth Arenare
- Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States.,The Feinstein Institute for Medical Research, Northwell Health, Manhasset, NY, United States
| | - Carla Agurto
- Computational Biology Center, IBM Research, Yorktown Heights, NY, United States
| | - Nathaniel Lu
- Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States.,The Feinstein Institute for Medical Research, Northwell Health, Manhasset, NY, United States
| | - John M Kane
- Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States.,The Feinstein Institute for Medical Research, Northwell Health, Manhasset, NY, United States.,The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, United States
| | - Guillermo Cecchi
- Computational Biology Center, IBM Research, Yorktown Heights, NY, United States
| |
Collapse
|
11
|
Damme KSF, Gupta T, Haase CM, Mittal VA. Responses to positive affect and unique resting-state connectivity in individuals at clinical high-risk for psychosis. Neuroimage Clin 2022; 33:102946. [PMID: 35091254 PMCID: PMC8800133 DOI: 10.1016/j.nicl.2022.102946] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2021] [Revised: 12/10/2021] [Accepted: 01/19/2022] [Indexed: 11/25/2022]
Abstract
Individuals at clinical high-risk for psychosis (CHR) report dampened positive affect, while this deficit appears to be an important clinical marker, our current understanding of underlying causes is limited. Dysfunctional regulatory strategies (i.e., abnormal use of dampening, self-focused, or emotion-focused strategies) may account for dampening affect but has not yet been examined. Participants (57 CHR and 56 healthy controls) completed the Response to Positive Affect Scale, clinical interviews, and resting-state scan examining nucleus accumbens (NAcc) connectivity. Individuals at CHR for psychosis showed greater dampening (but no differences in self/emotion-focus) in self-reported response to positive affect compared to healthy controls. In individuals at CHR, higher levels of dampening and lower levels of self-focus were associated with higher positive and lower negative symptoms. Dampening responses were related to decreased dorsal and rostral anterior cingulate cortex-NAcc resting-state connectivity in the CHR group but increased dorsal and rostral anterior cingulate cortex-NAcc resting-state connectivity in the healthy control group. Self-focused responses were related to increased dorsolateral prefrontal cortex-NAcc resting-state connectivity in the CHR group but decreased resting-state connectivity in the healthy control group. Self-reported dampening of positive affect was elevated in individuals at CHR for psychosis. Dampening and self-focused responses were associated with distinct resting-state connectivity compared to peers, suggesting unique mechanisms underlying these emotion regulation strategies. Responses to positive affect may be a useful target for cognitive treatment, but individuals at CHR show distinct neurocorrelates and may require a tailored approach.
Collapse
Affiliation(s)
- Katherine S F Damme
- Department of Psychology, Northwestern University, Evanston, IL, USA; Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, USA.
| | - Tina Gupta
- Department of Psychology, Northwestern University, Evanston, IL, USA; Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, USA
| | - Claudia M Haase
- Department of Psychology, Northwestern University, Evanston, IL, USA; Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, USA; Human Development and Social Policy, Northwestern University, Evanston, IL, USA; Institute for Policy Research (IPR), Northwestern University, Chicago, IL, USA
| | - Vijay A Mittal
- Department of Psychology, Northwestern University, Evanston, IL, USA; Institute for Innovations in Developmental Sciences (DevSci), Northwestern University, Evanston and Chicago, IL, USA; Human Development and Social Policy, Northwestern University, Evanston, IL, USA; Institute for Policy Research (IPR), Northwestern University, Chicago, IL, USA; Department of Psychiatry, Northwestern University, Chicago, IL, USA; Medical Social Sciences, Northwestern University, Chicago, IL, USA
| |
Collapse
|
12
|
Hitczenko K, Cowan HR, Goldrick M, Mittal VA. Racial and Ethnic Biases in Computational Approaches to Psychopathology. Schizophr Bull 2021; 48:285-288. [PMID: 34729605 PMCID: PMC8886581 DOI: 10.1093/schbul/sbab131] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- Kasia Hitczenko
- Department of Linguistics, Northwestern University, Evanston, IL, USA,To whom correspondence should be addressed; Department of Linguistics, Northwestern University, 2016 Sheridan Road, Evanston, IL 60208, USA; tel: 847-491-5831, fax: 847-491-3770, e-mail:
| | - Henry R Cowan
- Department of Psychology, Northwestern University, Evanston, IL, USA
| | - Matthew Goldrick
- Department of Linguistics, Northwestern University, Evanston, IL, USA,Department of Psychology, Northwestern University, Evanston, IL, USA,Institute for Innovations in Developmental Sciences, Northwestern University, Evanston/Chicago, IL, USA
| | - Vijay A Mittal
- Department of Psychology, Northwestern University, Evanston, IL, USA,Institute for Innovations in Developmental Sciences, Northwestern University, Evanston/Chicago, IL, USA,Department of Psychiatry, Northwestern University, Chicago, IL, USA,Institute for Policy Research, Northwestern University, Evanston, IL, USA,Medical Social Sciences, Northwestern University, Chicago, IL, USA
| |
Collapse
|
13
|
Reciprocal Social Behavior and Related Social Outcomes in Individuals at Clinical High Risk for Psychosis. Psychiatry Res 2021; 306:114224. [PMID: 34610542 PMCID: PMC8643304 DOI: 10.1016/j.psychres.2021.114224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2021] [Revised: 09/19/2021] [Accepted: 09/21/2021] [Indexed: 11/20/2022]
Abstract
Reciprocal social behavior (RSB) deficits have been noted in formal psychotic disorders and may play a role in the clinical high-risk for psychosis (CHR) syndrome. The present study examined RSB deficits and clinical and social functioning correlates in 45 individuals meeting criteria for a CHR syndrome and 47 healthy comparisons (HC). Further, this study examined associations with number of friends, problematic social Internet use, and perceived social support. Compared to the HC group, the CHR group exhibited greater deficits in total RSB and in all RSB subdomains. Total RSB deficits were associated with greater negative but not positive symptom severity in the CHR group, and greater social functional impairment. RSB deficits also may have related to fewer friendships, more problematic social Internet use, and less perceived belonging and tangible social support, although relationships with Internet use and perceived social support did not survive FDR-correction. These findings provide further evidence that RSB is impaired in the CHR syndrome and suggest specific social outcomes that may be affected. Further investigations with larger, diverse samples and repeated measures can confirm these findings.
Collapse
|
14
|
Gupta T, Strauss GP, Cowan HR, Pelletier-Baldelli A, Ellman LM, Schiffman J, Mittal VA. Secondary Sources of Negative Symptoms in Those Meeting Criteria for a Clinical High-Risk Syndrome. BIOLOGICAL PSYCHIATRY GLOBAL OPEN SCIENCE 2021; 1:210-218. [PMID: 35415704 PMCID: PMC8996819 DOI: 10.1016/j.bpsgos.2021.05.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 04/30/2021] [Accepted: 05/18/2021] [Indexed: 11/20/2022] Open
Abstract
BACKGROUND Negative symptoms are diagnostic characteristics of schizophrenia. They can result from primary (i.e., idiopathic) or secondary (i.e., due to other factors such as depression, anxiety, psychosis, disorganization, medication effects) features of the illness. Although secondary sources of negative symptoms are prevalent among individuals meeting criteria for clinical high-risk syndromes that are due to high rates of comorbidity, the extent to which secondary sources account for variance in negative symptom domains is unknown. Addressing this gap is an important step in informing vulnerability models and treatments for negative symptoms. This study aimed to investigate secondary sources of negative symptoms in those meeting criteria for a clinical high-risk syndrome (N = 192). METHODS Simultaneous regression and hierarchical partitioning methods were used to determine the proportion of variance explained by selective serotonin reuptake inhibitor use, anxiety, depression, unusual thought content, and disorganized communication in predicting severity of five negative symptom domains (avolition, anhedonia, asociality, blunted affect, and alogia). RESULTS Findings revealed that depression explained the largest proportion of variance in avolition, asociality, and anhedonia. Anxiety was the most predictive of blunted affect, and selective serotonin reuptake inhibitor use explained the most variance in alogia. Analyses within male and female samples revealed that in males, depression explained a large proportion of variance in several negative symptom domains, while in females, selective serotonin reuptake inhibitor use explained variance in alogia. CONCLUSIONS Results highlight heterogeneity in variance explained by secondary sources of negative symptoms. These findings guide treatment development for secondary sources of negative symptoms. Furthermore, results inform etiologic models of psychosis and negative symptom conceptualizations.
Collapse
Affiliation(s)
- Tina Gupta
- Department of Psychology, Northwestern University, Evanston, Illinois
| | | | - Henry R. Cowan
- Department of Psychology, Northwestern University, Evanston, Illinois
| | | | - Lauren M. Ellman
- Department of Psychology, Temple University, Philadelphia, Pennsylvania
| | - Jason Schiffman
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland
- Department of Psychological Science, University of California, Irvine, Irvine, California
| | - Vijay A. Mittal
- Department of Psychology, Northwestern University, Evanston, Illinois
- Department of Psychiatry, Northwestern University, Evanston, Illinois
- Department of Medical Social Science, Northwestern University, Evanston, Illinois
- Institute for Policy Research, Northwestern University, Evanston, Illinois
| |
Collapse
|
15
|
Abstract
Individuals diagnosed with psychotic disorders exhibit abnormalities in the perception of expressive behaviors, which are linked to symptoms and visual information processing domains. Specifically, literature suggests these groups have difficulties perceiving gestures that accompany speech. While our understanding of gesture perception in psychotic disorders is growing, gesture perception abnormalities and clues about potential causes and consequences among individuals meeting criteria for a clinical high-risk (CHR) syndrome is limited. Presently, 29 individuals with a CHR syndrome and 32 healthy controls completed an eye-tracking gesture perception paradigm. In this task, participants viewed an actor using abstract and literal gestures while presenting a story and eye gaze data (eg, fixation counts and total fixation time) was collected. Furthermore, relationships between fixation variables and both symptoms (positive, negative, anxiety, and depression) and measures of visual information processing (working memory and attention) were examined. Findings revealed that the CHR group gazed at abstract gestures fewer times than the control group. When individuals in the CHR group did gaze at abstract gestures, on average, they spent significantly less time fixating compared to controls. Furthermore, reduced fixation (ie, count and time) was related to depression and slower response time on an attentional task. While a similar pattern of group differences in the same direction appeared for literal gestures, the effect was not significant. These data highlight the importance of integrating gesture perception abnormalities into vulnerability models of psychosis and inform the development of targeted treatments for social communicative deficits.
Collapse
Affiliation(s)
- Tina Gupta
- Department of Psychology, Northwestern University, Evanston, IL,To whom correspondence should be addressed; Department of Psychology, Northwestern University, 2029 Sheridan Road, Evanston, IL 60208, US; tel: 847-467-5907, fax: 847-467-5707, e-mail:
| | | | - Vijay A Mittal
- Department of Psychology, Northwestern University, Evanston, IL
| |
Collapse
|
16
|
Corcoran CM, Mittal VA, Bearden CE, E Gur R, Hitczenko K, Bilgrami Z, Savic A, Cecchi GA, Wolff P. Language as a biomarker for psychosis: A natural language processing approach. Schizophr Res 2020; 226:158-166. [PMID: 32499162 PMCID: PMC7704556 DOI: 10.1016/j.schres.2020.04.032] [Citation(s) in RCA: 70] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2020] [Revised: 04/22/2020] [Accepted: 04/24/2020] [Indexed: 12/21/2022]
Abstract
Human ratings of conceptual disorganization, poverty of content, referential cohesion and illogical thinking have been shown to predict psychosis onset in prospective clinical high risk (CHR) cohort studies. The potential value of linguistic biomarkers has been significantly magnified, however, by recent advances in natural language processing (NLP) and machine learning (ML). Such methodologies allow for the rapid and objective measurement of language features, many of which are not easily recognized by human raters. Here we review the key findings on language production disturbance in psychosis. We also describe recent advances in the computational methods used to analyze language data, including methods for the automatic measurement of discourse coherence, syntactic complexity, poverty of content, referential coherence, and metaphorical language. Linguistic biomarkers of psychosis risk are now undergoing cross-validation, with attention to harmonization of methods. Future directions in extended CHR networks include studies of sources of variance, and combination with other promising biomarkers of psychosis risk, such as cognitive and sensory processing impairments likely to be related to language. Implications for the broader study of social communication, including reciprocal prosody, face expression and gesture, are discussed.
Collapse
Affiliation(s)
- Cheryl M Corcoran
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Vijay A Mittal
- Department of Psychology, Northwestern University, Evanston, IL, USA
| | - Carrie E Bearden
- Department of Psychiatry and Biobehavioral Sciences, University of California Los Angeles, CA, USA; Department of Psychology, Semel Institute for Neuroscience and Human Behavior, Brain Research Institute, University of California Los Angeles, CA, USA; Department of Psychology, University of California Los Angeles, CA USA
| | - Raquel E Gur
- Brain Behavior Laboratory, Neuropsychiatry Division, Department of Psychiatry, Philadelphia, PA 19104, USA
| | - Kasia Hitczenko
- Department of Linguistics, Northwestern University, Evanston, IL, USA
| | - Zarina Bilgrami
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Aleksandar Savic
- Department of Diagnostics and Intensive Care, University Psychiatric Hospital Vrapce, Zagreb, Croatia
| | - Guillermo A Cecchi
- Computational Biology Center-Neuroscience, IBM T.J. Watson Research Center, Yorktown Heights, NY, USA
| | - Phillip Wolff
- Department of Psychology, Emory University, Atlanta, GA, USA.
| |
Collapse
|
17
|
Hitczenko K, Mittal VA, Goldrick M. Understanding Language Abnormalities and Associated Clinical Markers in Psychosis: The Promise of Computational Methods. Schizophr Bull 2020; 47:344-362. [PMID: 33205155 PMCID: PMC8480175 DOI: 10.1093/schbul/sbaa141] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
The language and speech of individuals with psychosis reflect their impairments in cognition and motor processes. These language disturbances can be used to identify individuals with and at high risk for psychosis, as well as help track and predict symptom progression, allowing for early intervention and improved outcomes. However, current methods of language assessment-manual annotations and/or clinical rating scales-are time intensive, expensive, subject to bias, and difficult to administer on a wide scale, limiting this area from reaching its full potential. Computational methods that can automatically perform linguistic analysis have started to be applied to this problem and could drastically improve our ability to use linguistic information clinically. In this article, we first review how these automated, computational methods work and how they have been applied to the field of psychosis. We show that across domains, these methods have captured differences between individuals with psychosis and healthy controls and can classify individuals with high accuracies, demonstrating the promise of these methods. We then consider the obstacles that need to be overcome before these methods can play a significant role in the clinical process and provide suggestions for how the field should address them. In particular, while much of the work thus far has focused on demonstrating the successes of these methods, we argue that a better understanding of when and why these models fail will be crucial toward ensuring these methods reach their potential in the field of psychosis.
Collapse
Affiliation(s)
- Kasia Hitczenko
- Department of Linguistics, Northwestern University, Evanston,
IL,To whom correspondence should be addressed; Northwestern University, 2016
Sheridan Road, Evanston, IL 60208; tel: 847-491-5831, fax: 847-491-3770, e-mail:
| | - Vijay A Mittal
- Department of Psychology, Northwestern University, Evanston, IL,Department of Psychiatry, Northwestern University, Chicago, IL,Institute for Policy Research, Northwestern University, Evanston,
IL,Medical Social Sciences, Northwestern University, Chicago, IL,Institute for Innovations in Developmental Sciences, Northwestern
University, Evanston and Chicago, IL
| | - Matthew Goldrick
- Department of Linguistics, Northwestern University, Evanston,
IL,Institute for Innovations in Developmental Sciences, Northwestern
University, Evanston and Chicago, IL
| |
Collapse
|
18
|
Cohen AS, Cox CR, Le TP, Cowan T, Masucci MD, Strauss GP, Kirkpatrick B. Using machine learning of computerized vocal expression to measure blunted vocal affect and alogia. NPJ SCHIZOPHRENIA 2020; 6:26. [PMID: 32978400 PMCID: PMC7519104 DOI: 10.1038/s41537-020-00115-2] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/06/2020] [Accepted: 08/06/2020] [Indexed: 11/16/2022]
Abstract
Negative symptoms are a transdiagnostic feature of serious mental illness (SMI) that can be potentially “digitally phenotyped” using objective vocal analysis. In prior studies, vocal measures show low convergence with clinical ratings, potentially because analysis has used small, constrained acoustic feature sets. We sought to evaluate (1) whether clinically rated blunted vocal affect (BvA)/alogia could be accurately modelled using machine learning (ML) with a large feature set from two separate tasks (i.e., a 20-s “picture” and a 60-s “free-recall” task), (2) whether “Predicted” BvA/alogia (computed from the ML model) are associated with demographics, diagnosis, psychiatric symptoms, and cognitive/social functioning, and (3) which key vocal features are central to BvA/Alogia ratings. Accuracy was high (>90%) and was improved when computed separately by speaking task. ML scores were associated with poor cognitive performance and social functioning and were higher in patients with schizophrenia versus depression or mania diagnoses. However, the features identified as most predictive of BvA/Alogia were generally not considered critical to their operational definitions. Implications for validating and implementing digital phenotyping to reduce SMI burden are discussed.
Collapse
Affiliation(s)
- Alex S Cohen
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA. .,Center for Computation and Technology, Louisiana State University, Baton Rouge, LA, 70803, USA.
| | - Christopher R Cox
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | - Thanh P Le
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA.,Center for Computation and Technology, Louisiana State University, Baton Rouge, LA, 70803, USA
| | - Tovah Cowan
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA.,Center for Computation and Technology, Louisiana State University, Baton Rouge, LA, 70803, USA
| | - Michael D Masucci
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA.,Center for Computation and Technology, Louisiana State University, Baton Rouge, LA, 70803, USA
| | | | - Brian Kirkpatrick
- Department of Psychiatry and Behavioral Sciences, University of Nevada, Reno, USA
| |
Collapse
|
19
|
Gupta T, Haase CM, Strauss GP, Cohen AS, Ricard JR, Mittal VA. Alterations in facial expressions of emotion: Determining the promise of ultrathin slicing approaches and comparing human and automated coding methods in psychosis risk. Emotion 2020; 22:714-724. [PMID: 32584067 DOI: 10.1037/emo0000819] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Alterations in facial expressions of emotion are a hallmark of psychopathology and may be present before the onset of mental illness. Technological advances have spurred interest in examining alterations based on "thin slices" of behavior using automated approaches. However, questions remain. First, can alterations be detected in ultrathin slices of behavior? Second, how do automated approaches converge with human coding techniques? The present study examined ultrathin (i.e., 1-min) slices of video-recorded clinical interviews of 42 individuals at clinical high risk (CHR) for psychosis and 42 matched controls. Facial expressions of emotion (e.g., joy, anger) were examined using two automated facial analysis programs and coded by trained human raters (using the Expressive Emotional Behavior Coding System). Results showed that ultrathin (i.e., 1-min) slices of behavior were sufficient to reveal alterations in facial expressions of emotion, specifically blunted joy expressions in individuals at CHR (with supplementary analyses probing links with attenuated positive symptoms and functioning). Furthermore, both automated analysis programs converged in the ability to detect blunted joy expressions and were consistent with human coding at the level of both second-by-second and aggregate data. Finally, there were areas of divergence across approaches for other emotional expressions beyond joy. These data suggest that ultrathin slices of behavior can yield clues about emotional dysfunction. Further, automated approaches (which do not require lengthy training and coder time but do lend well to mobile assessment and computational modeling) show promise, but careful evaluation of convergence with human coding is needed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
|
20
|
Cohen AS, Schwartz E, Le TP, Cowan T, Kirkpatrick B, Raugh IM, Strauss GP. Digital phenotyping of negative symptoms: the relationship to clinician ratings. Schizophr Bull 2020; 47:44-53. [PMID: 32467967 PMCID: PMC7825094 DOI: 10.1093/schbul/sbaa065] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Negative symptoms are a critical, but poorly understood, aspect of schizophrenia. Measurement of negative symptoms primarily relies on clinician ratings, an endeavor with established reliability and validity. There have been increasing attempts to digitally phenotype negative symptoms using objective biobehavioral technologies, eg, using computerized analysis of vocal, speech, facial, hand and other behaviors. Surprisingly, biobehavioral technologies and clinician ratings are only modestly inter-related, and findings from individual studies often do not replicate or are counterintuitive. In this article, we document and evaluate this lack of convergence in 4 case studies, in an archival dataset of 877 audio/video samples, and in the extant literature. We then explain this divergence in terms of "resolution"-a critical psychometric property in biomedical, engineering, and computational sciences defined as precision in distinguishing various aspects of a signal. We demonstrate how convergence between clinical ratings and biobehavioral data can be achieved by scaling data across various resolutions. Clinical ratings reflect an indispensable tool that integrates considerable information into actionable, yet "low resolution" ordinal ratings. This allows viewing of the "forest" of negative symptoms. Unfortunately, their resolution cannot be scaled or decomposed with sufficient precision to isolate the time, setting, and nature of negative symptoms for many purposes (ie, to see the "trees"). Biobehavioral measures afford precision for understanding when, where, and why negative symptoms emerge, though much work is needed to validate them. Digital phenotyping of negative symptoms can provide unprecedented opportunities for tracking, understanding, and treating them, but requires consideration of resolution.
Collapse
Affiliation(s)
- Alex S Cohen
- Department of Psychology, Louisiana State University, Baton Rouge, LA,Louisiana State University, Center for Computation and Technology, Baton Rouge, LA,To whom correspondence should be addressed; Department of Psychology, Louisiana State University, 236 Audubon Hall, Baton Rouge, LA 70803; tel: +1-225-578-7017, fax: +1-225-578-4125, e-mail:
| | - Elana Schwartz
- Department of Psychology, Louisiana State University, Baton Rouge, LA,Louisiana State University, Center for Computation and Technology, Baton Rouge, LA
| | - Thanh P Le
- Department of Psychology, Louisiana State University, Baton Rouge, LA,Louisiana State University, Center for Computation and Technology, Baton Rouge, LA
| | - Tovah Cowan
- Department of Psychology, Louisiana State University, Baton Rouge, LA,Louisiana State University, Center for Computation and Technology, Baton Rouge, LA
| | - Brian Kirkpatrick
- Department of Psychiatry and Behavioral Sciences, University of Nevada, Reno School of Medicine, Reno, NV
| | - Ian M Raugh
- Department of Psychology, University of Georgia, Athens, GA
| | | |
Collapse
|
21
|
DiLalla LF, John SG. A genetically informed examination of the relations between inaccurate emotion expression and recognition and experiencing peer victimization. SOCIAL DEVELOPMENT 2020. [DOI: 10.1111/sode.12410] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Lisabeth Fisher DiLalla
- Family and Community Medicine Southern Illinois University School of Medicine Carbondale Illinois
| | - Sufna Gheyara John
- Department of Psychiatry University of Arkansas for Medical Sciences Little Rock Arkansas
| |
Collapse
|