Multimodal Assessment of Schizophrenia and Depression Utilizing Video, Acoustic, Locomotor, Electroencephalographic, and Heart Rate Technology: Protocol for an Observational Study

Assessment of symptom severity in psychotic disorder patients based on heart rate variability and accelerometer mobility data

BACKGROUND

Advancement in mental health care requires easily accessible, efficient diagnostic and treatment assessment tools. Viable biomarkers could enable objectification and automation of the diagnostic and treatment process, currently dependent on a psychiatric interview. Available wearable technology and computational methods make it possible to incorporate heart rate variability (HRV), an indicator of autonomic nervous system (ANS) activity, into potential diagnostic and treatment assessment frameworks as a biomarker of disease severity in mental disorders, including schizophrenia and bipolar disorder (BD).

METHOD

We used a commercially available electrocardiography (ECG) chest strap with a built-in accelerometer, i.e. Polar H10, to record R-R intervals and physical activity of 30 hospitalized schizophrenia or BD patients and 30 control participants through ca. 1.5-2 h time periods. We validated a novel approach to data acquisition based on a flexible, patient-friendly and cost-effective setting. We analyzed the relationship between HRV and the Positive and Negative Syndrome Scale (PANSS) test scores, as well as the HRV and mobility coefficient. We also proposed a method of rest period selection based on R-R intervals and mobility data. The source code for reproducing all experiments is available on GitHub, while the dataset is published on Zenodo.

RESULTS

Mean HRV values were lower in the patient compared to the control group and negatively correlated with the results of the PANSS general subcategory. For the control group, we also discovered the inversely proportional dependency between the mobility coefficient, based on accelerometer data, and HRV. This relationship was less pronounced for the treatment group.

CONCLUSIONS

HRV value itself, as well as the relationship between HRV and mobility, may be promising biomarkers in disease diagnostics. These findings can be used to develop a flexible monitoring system for symptom severity assessment.

Mental Disorders Among Mothers in Contact with the Criminal Justice System: A Scoping Review and Meta-analysis

This scoping review critically assessed evidence regarding mental disorders among mothers involved with the criminal justice system (CJS) and provided pooled prevalence rates of mental disorders. In total, 27 studies were included in the review, with 23 studies from the United States of America and 26 focused on incarcerated mothers. The findings supported the evidence on substantial burden of mental disorders, among CJS-involved mothers. Several factors contributing to mental disorders were identified, including history of abuse/incarceration/mental illness, a greater number of pregnancies, child-rearing responsibilities, less contact with children, and poor social support, which were organised using a socioecological model. However, significant gaps in the current evidence base were apparent, including inconsistencies in methodologies and outcomes assessed and a lack of large, longitudinal studies. The study highlights the importance of high-quality longitudinal research to extend knowledge around causal pathways between different risk or protective factors and mental disorders among CJS-involved mothers.

Multimodal Mental Health Digital Biomarker Analysis From Remote Interviews Using Facial, Vocal, Linguistic, and Cardiovascular Patterns

OBJECTIVE

Psychiatric evaluation suffers from subjectivity and bias, and is hard to scale due to intensive professional training requirements. In this work, we investigated whether behavioral and physiological signals, extracted from tele-video interviews, differ in individuals with psychiatric disorders.

METHODS

Temporal variations in facial expression, vocal expression, linguistic expression, and cardiovascular modulation were extracted from simultaneously recorded audio and video of remote interviews. Averages, standard deviations, and Markovian process-derived statistics of these features were computed from 73 subjects. Four binary classification tasks were defined: detecting 1) any clinically-diagnosed psychiatric disorder, 2) major depressive disorder, 3) self-rated depression, and 4) self-rated anxiety. Each modality was evaluated individually and in combination.

RESULTS

Statistically significant feature differences were found between psychiatric and control subjects. Correlations were found between features and self-rated depression and anxiety scores. Heart rate dynamics provided the best unimodal performance with areas under the receiver-operator curve (AUROCs) of 0.68-0.75 (depending on the classification task). Combining multiple modalities provided AUROCs of 0.72-0.82.

CONCLUSION

Multimodal features extracted from remote interviews revealed informative characteristics of clinically diagnosed and self-rated mental health status.

SIGNIFICANCE

The proposed multimodal approach has the potential to facilitate scalable, remote, and low-cost assessment for low-burden automated mental health services.

Using HIPAA (Health Insurance Portability and Accountability Act)-Compliant Transcription Services for Virtual Psychiatric Interviews: Pilot Comparison Study

BACKGROUND

Automatic speech recognition (ASR) technology is increasingly being used for transcription in clinical contexts. Although there are numerous transcription services using ASR, few studies have compared the word error rate (WER) between different transcription services among different diagnostic groups in a mental health setting. There has also been little research into the types of words ASR transcriptions mistakenly generate or omit.

OBJECTIVE

This study compared the WER of 3 ASR transcription services (Amazon Transcribe [Amazon.com, Inc], Zoom-Otter AI [Zoom Video Communications, Inc], and Whisper [OpenAI Inc]) in interviews across 2 different clinical categories (controls and participants experiencing a variety of mental health conditions). These ASR transcription services were also compared with a commercial human transcription service, Rev (Rev.Com, Inc). Words that were either included or excluded by the error in the transcripts were systematically analyzed by their Linguistic Inquiry and Word Count categories.

METHODS

Participants completed a 1-time research psychiatric interview, which was recorded on a secure server. Transcriptions created by the research team were used as the gold standard from which WER was calculated. The interviewees were categorized into either the control group (n=18) or the mental health condition group (n=47) using the Mini-International Neuropsychiatric Interview. The total sample included 65 participants. Brunner-Munzel tests were used for comparing independent sets, such as the diagnostic groupings, and Wilcoxon signed rank tests were used for correlated samples when comparing the total sample between different transcription services.

RESULTS

There were significant differences between each ASR transcription service's WER (P<.001). Amazon Transcribe's output exhibited significantly lower WERs compared with the Zoom-Otter AI's and Whisper's ASR. ASR performances did not significantly differ across the 2 different clinical categories within each service (P>.05). A comparison between the human transcription service output from Rev and the best-performing ASR (Amazon Transcribe) demonstrated a significant difference (P<.001), with Rev having a slightly lower median WER (7.6%, IQR 5.4%-11.35 vs 8.9%, IQR 6.9%-11.6%). Heat maps and spider plots were used to visualize the most common errors in Linguistic Inquiry and Word Count categories, which were found to be within 3 overarching categories: Conversation, Cognition, and Function.

CONCLUSIONS

Overall, consistent with previous literature, our results suggest that the WER between manual and automated transcription services may be narrowing as ASR services advance. These advances, coupled with decreased cost and time in receiving transcriptions, may make ASR transcriptions a more viable option within health care settings. However, more research is required to determine if errors in specific types of words impact the analysis and usability of these transcriptions, particularly for specific applications and in a variety of populations in terms of clinical diagnosis, literacy level, accent, and cultural origin.

Multimodal mental health assessment with remote interviews using facial, vocal, linguistic, and cardiovascular patterns

Objective

The current clinical practice of psychiatric evaluation suffers from subjectivity and bias, and requires highly skilled professionals that are often unavailable or unaffordable. Objective digital biomarkers have shown the potential to address these issues. In this work, we investigated whether behavioral and physiological signals, extracted from remote interviews, provided complimentary information for assessing psychiatric disorders.

Methods

Time series of multimodal features were derived from four conceptual modes: facial expression, vocal expression, linguistic expression, and cardiovascular modulation. The features were extracted from simultaneously recorded audio and video of remote interviews using task-specific and foundation models. Averages, standard deviations, and hidden Markov model-derived statistics of these features were computed from 73 subjects. Four binary classification tasks were defined: detecting 1) any clinically-diagnosed psychiatric disorder, 2) major depressive disorder, 3) self-rated depression, and 4) self-rated anxiety. Each modality was evaluated individually and in combination.

Results

Statistically significant feature differences were found between controls and subjects with mental health conditions. Correlations were found between features and self-rated depression and anxiety scores. Visual heart rate dynamics achieved the best unimodal performance with areas under the receiver-operator curve (AUROCs) of 0.68-0.75 (depending on the classification task). Combining multiple modalities achieved AUROCs of 0.72-0.82. Features from task-specific models outperformed features from foundation models.

Conclusion

Multimodal features extracted from remote interviews revealed informative characteristics of clinically diagnosed and self-rated mental health status.

Significance

The proposed multimodal approach has the potential to facilitate objective, remote, and low-cost assessment for low-burden automated mental health services.