Digital Phenotyping Using Multimodal Data

Exercise behaviours and motivation after a first psychotic episode: A digital intervention

AIM

Research has demonstrated that participation in aerobic exercise can have significant beneficial effects across both physical and mental health domains for individuals who are in the early phase of schizophrenia. Despite these notable benefits of exercise, deficits in motivation and a lack of methods to increase engagement are significant barriers for exercise participation, limiting these potentially positive effects. Fortunately, digital health tools have the potential to improve adherence to an exercise program. The present study examined the role of motivation for exercise and the effects of an automated digital text messaging program on participation in an aerobic exercise program.

METHODS

A total of 46 first-episode psychosis participants from an ongoing 12-month randomized clinical trial (Enhancing Cognitive Training through Exercise Following a First Schizophrenia Episode (CT&E-RCT)) were included in an analysis to examine the efficacy of motivational text messaging. Personalized motivational text message reminders were sent to participants with the aim of increasing engagement in the exercise program.

RESULTS

We found that participants with higher levels of intrinsic motivation to participate in a text messaging program and in an exercise intervention completed a higher proportion of individual, at-home exercise sessions. In a between groups analysis, participants who received motivational text messages, compared to those who did not, completed a higher proportion of at-home exercise sessions.

CONCLUSION

These results indicate the importance of considering a person's level of motivation for exercise and the potential utility of using individualized and interactive mobile text messaging reminders to increase engagement in aerobic exercise in the early phase of psychosis. We emphasize the need for understanding how individualized patient preferences and needs interplay between intrinsic motivation and digital health interventions for young adults.

Exploring a multimodal approach for utilizing digital biomarkers for childhood mental health screening

Background

Depression and anxiety are prevalent mental health concerns among children and adolescents. The application of conventional assessment methods, such as survey questionnaires to children, may lead to self-reporting issues. Digital biomarkers provide extensive data, reducing bias in mental health self-reporting, and significantly influence patient screening. Our primary objectives were to accurately assess children's mental health and to investigate the feasibility of using various digital biomarkers.

Methods

This study included a total of 54 boys and girls aged between 7 to 11 years. Each participant's mental state was assessed using the Depression, Anxiety, and Stress Scale. Subsequently, the subjects participated in digital biomarker collection tasks. Heart rate variability (HRV) data were collected using a camera sensor. Eye-tracking data were collected through tasks displaying emotion-face stimuli. Voice data were obtained by recording the participants' voices while they engaged in free speech and description tasks.

Results

Depressive symptoms were positively correlated with low frequency (LF, 0.04-0.15 Hz of HRV) in HRV and negatively associated with eye-tracking variables. Anxiety symptoms had a negative correlation with high frequency (HF, 0.15-0.40 Hz of HRV) in HRV and a positive association with LF/HF. Regarding stress, eye-tracking variables indicated a positive correlation, while pNN50, which represents the proportion of NN50 (the number of pairs of successive R-R intervals differing by more than 50 milliseconds) divided by the total number of NN (R-R) intervals, exhibited a negative association. Variables identified for childhood depression included LF and the total time spent looking at a sad face. Those variables recognized for anxiety were LF/HF, heart rate (HR), and pNN50. For childhood stress, HF, LF, and Jitter showed different correlation patterns between the two grade groups.

Discussion

We examined the potential of multimodal biomarkers in children, identifying features linked to childhood depression, particularly LF and the Sad.TF:time. Anxiety was most effectively explained by HRV features. To explore reasons for non-replication of previous studies, we categorized participants by elementary school grades into lower grades (1st, 2nd, 3rd) and upper grades (4th, 5th, 6th).

Conclusion

This study confirmed the potential use of multimodal digital biomarkers for children's mental health screening, serving as foundational research.

Benchmarking Mental Health Status Using Passive Sensor Data: Protocol for a Prospective Observational Study

BACKGROUND

Computational psychiatry has the potential to advance the diagnosis, mechanistic understanding, and treatment of mental health conditions. Promising results from clinical samples have led to calls to extend these methods to mental health risk assessment in the general public; however, data typically used with clinical samples are neither available nor scalable for research in the general population. Digital phenotyping addresses this by capitalizing on the multimodal and widely available data created by sensors embedded in personal digital devices (eg, smartphones) and is a promising approach to extending computational psychiatry methods to improve mental health risk assessment in the general population.

OBJECTIVE

Building on recommendations from existing computational psychiatry and digital phenotyping work, we aim to create the first computational psychiatry data set that is tailored to studying mental health risk in the general population; includes multimodal, sensor-based behavioral features; and is designed to be widely shared across academia, industry, and government using gold standard methods for privacy, confidentiality, and data integrity.

METHODS

We are using a stratified, random sampling design with 2 crossed factors (difficulties with emotion regulation and perceived life stress) to recruit a sample of 400 community-dwelling adults balanced across high- and low-risk for episodic mental health conditions. Participants first complete self-report questionnaires assessing current and lifetime psychiatric and medical diagnoses and treatment, and current psychosocial functioning. Participants then complete a 7-day in situ data collection phase that includes providing daily audio recordings, passive sensor data collected from smartphones, self-reports of daily mood and significant events, and a verbal description of the significant daily events during a nightly phone call. Participants complete the same baseline questionnaires 6 and 12 months after this phase. Self-report questionnaires will be scored using standard methods. Raw audio and passive sensor data will be processed to create a suite of daily summary features (eg, time spent at home).

RESULTS

Data collection began in June 2022 and is expected to conclude by July 2024. To date, 310 participants have consented to the study; 149 have completed the baseline questionnaire and 7-day intensive data collection phase; and 61 and 31 have completed the 6- and 12-month follow-up questionnaires, respectively. Once completed, the proposed data set will be made available to academic researchers, industry, and the government using a stepped approach to maximize data privacy.

CONCLUSIONS

This data set is designed as a complementary approach to current computational psychiatry and digital phenotyping research, with the goal of advancing mental health risk assessment within the general population. This data set aims to support the field's move away from siloed research laboratories collecting proprietary data and toward interdisciplinary collaborations that incorporate clinical, technical, and quantitative expertise at all stages of the research process.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/53857.

Rationale and Challenges for a New Instrument for Remote Measurement of Negative Symptoms

There is a broad consensus that the commonly used clinician-administered rating scales for assessment of negative symptoms share significant limitations, including (1) reliance upon accurate self-report and recall from the patient and caregiver; (2) potential for sampling bias and thus being unrepresentative of daily-life experiences; (3) subjectivity of the symptom scoring process and limited sensitivity to change. These limitations led a work group from the International Society of CNS Clinical Trials and Methodology (ISCTM) to initiate the development of a multimodal negative symptom instrument. Experts from academia and industry reviewed the current methods of assessing the domains of negative symptoms including diminished (1) affect; (2) sociality; (3) verbal communication; (4) goal-directed behavior; and (5) Hedonic drives. For each domain, they documented the limitations of the current methods and recommended new approaches that could potentially be included in a multimodal instrument. The recommended methods for assessing negative symptoms included ecological momentary assessment (EMA), in which the patient self-reports their condition upon receipt of periodic prompts from a smartphone or other device during their daily routine; and direct inference of negative symptoms through detection and analysis of the patient's voice, appearance or activity from audio/visual or sensor-based (eg, global positioning systems, actigraphy) recordings captured by the patient's smartphone or other device. The process for developing an instrument could resemble the NIMH MATRICS process that was used to develop a battery for measuring cognition in schizophrenia. Although the EMA and other digital measures for negative symptoms are at relatively early stages of development/maturity and development of such an instrument faces substantial challenges, none of them are insurmountable.

The Use of Mobile Assessments for Monitoring Mental Health in Youth: Umbrella Review

BACKGROUND

Improving mental health in youth is a major concern. Future approaches to monitor and intervene in youth mental health problems should rely on mobile tools that allow for the daily monitoring of mental health both actively (eg, using ecological momentary assessments [EMAs]) and passively (eg, digital phenotyping) by capturing individuals' data.

OBJECTIVE

This umbrella review aims to (1) report the main characteristics of existing reviews on mental health and young people, including mobile approaches to mental health; (2) describe EMAs and trace data and the mental health conditions investigated; (3) report the main results; and (4) outline promises, limitations, and directions for future research.

METHODS

A systematic literature search was carried out in 9 scientific databases (Communication & Mass Media Complete, Psychology and Behavioral Sciences Collection, PsycINFO, CINAHL, ERIC, MEDLINE, the ProQuest Sociology Database, Web of Science, and PubMed) on January 30, 2022, coupled with a hand search and updated in July 2022. We included (systematic) reviews of EMAs and trace data in the context of mental health, with a specific focus on young populations, including children, adolescents, and young adults. The quality of the included reviews was evaluated using the AMSTAR (Assessment of Multiple Systematic Reviews) checklist.

RESULTS

After the screening process, 30 reviews (published between 2016 and 2022) were included in this umbrella review, of which 21 (70%) were systematic reviews and 9 (30%) were narrative reviews. The included systematic reviews focused on symptoms of depression (5/21, 24%); bipolar disorders, schizophrenia, or psychosis (6/21, 29%); general ill-being (5/21, 24%); cognitive abilities (2/21, 9.5%); well-being (1/21, 5%); personality (1/21, 5%); and suicidal thoughts (1/21, 5%). Of the 21 systematic reviews, 15 (71%) summarized studies that used mobile apps for tracing, 2 (10%) summarized studies that used them for intervention, and 4 (19%) summarized studies that used them for both intervention and tracing. Mobile tools used in the systematic reviews were smartphones only (8/21, 38%), smartphones and wearable devices (6/21, 29%), and smartphones with other tools (7/21, 33%). In total, 29% (6/21) of the systematic reviews focused on EMAs, including ecological momentary interventions; 33% (7/21) focused on trace data; and 38% (8/21) focused on both. Narrative reviews mainly focused on the discussion of issues related to digital phenotyping, existing theoretical frameworks used, new opportunities, and practical examples.

CONCLUSIONS

EMAs and trace data in the context of mental health assessments and interventions are promising tools. Opportunities (eg, using mobile approaches in low- and middle-income countries, integration of multimodal data, and improving self-efficacy and self-awareness on mental health) and limitations (eg, absence of theoretical frameworks, difficulty in assessing the reliability and effectiveness of such approaches, and need to appropriately assess the quality of the studies) were further discussed.

TRIAL REGISTRATION

PROSPERO CRD42022347717; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=347717.

Identifying Medications Underlying Communication Atypicalities in Psychotic and Affective Disorders: A Pharmacovigilance Study Within the FDA Adverse Event Reporting System

PURPOSE

Communication atypicalities are considered promising markers of a broad range of clinical conditions. However, little is known about the mechanisms and confounders underlying them. Medications might have a crucial, relatively unknown role both as potential confounders and offering an insight on the mechanisms at work. The integration of regulatory documents with disproportionality analyses provides a more comprehensive picture to account for in future investigations of communication-related markers. The aim of this study was to identify a list of drugs potentially associated with communicative atypicalities within psychotic and affective disorders.

METHOD

We developed a query using the Medical Dictionary for Regulatory Activities to search for communicative atypicalities within the FDA Adverse Event Reporting System (updated June 2021). A Bonferroni-corrected disproportionality analysis (reporting odds ratio) was separately performed on spontaneous reports involving psychotic, affective, and non-neuropsychiatric disorders, to account for the confounding role of different underlying conditions. Drug-adverse event associations not already reported in the Side Effect Resource database of labeled adverse drug reactions (unexpected) were subjected to further robustness analyses to account for expected biases.

RESULTS

A list of 291 expected and 91 unexpected potential confounding medications was identified, including drugs that may irritate (inhalants) or desiccate (anticholinergics) the larynx, impair speech motor control (antipsychotics), or induce nodules (acitretin) or necrosis (vascular endothelial growth factor receptor inhibitors) on vocal cords; sedatives and stimulants; neurotoxic agents (anti-infectives); and agents acting on neurotransmitter pathways (dopamine agonists).

CONCLUSIONS

We provide a list of medications to account for in future studies of communication-related markers in affective and psychotic disorders. The current test case illustrates rigorous procedures for digital phenotyping, and the methodological tools implemented for large-scale disproportionality analyses can be considered a road map for investigations of communication-related markers in other clinical populations.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23721345.

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders.

Using Automated Speech Processing for Repeated Measurements in a Clinical Setting of the Behavioral Variability in the Stroop Task

The Stroop interference task is indispensable to current neuropsychological practice. Despite this, it is limited in its potential for repeated administration, its sensitivity and its demands on professionals and their clients. We evaluated a digital Stroop deployed using a smart device. Spoken responses were timed using automated speech recognition. Participants included adult nonpatients (N = 113; k = 5 sessions over 5 days) and patients with psychiatric diagnoses (N = 85; k = 3-4 sessions per week over 4 weeks). Traditional interference (difference in response time between color incongruent words vs. color neutral words; M = 0.121 s) and facilitation (neutral vs. color congruent words; M = 0.085 s) effects were robust and temporally stable over testing sessions (ICCs 0.50-0.86). The performance showed little relation to clinical symptoms for a two-week window for either nonpatients or patients but was related to self-reported concentration at the time of testing for both groups. Performance was also related to treatment outcomes in patients. The duration of response word utterances was longer in patients than in nonpatients. Measures of intra-individual variability showed promise for understanding clinical state and treatment outcome but were less temporally stable than measures based solely on average response time latency. This framework of remote assessment using speech processing technology enables the fine-grained longitudinal charting of cognition and verbal behavior. However, at present, there is a problematic lower limit to the absolute size of the effects that can be examined when using voice in such a brief 'out-of-the-laboratory condition' given the temporal resolution of the speech-to-text detection system (in this case, 10 ms). This resolution will limit the parsing of meaningful effect sizes.

Treatment engagement in first-episode schizophrenia: Associations between intrinsic motivation and attendance during cognitive training and an aerobic exercise program

Systematic cognitive training and aerobic exercise programs have emerged as promising interventions to improve cognitive deficits in first-episode schizophrenia, with successful outcomes closely linked with greater treatment engagement (e.g., higher attendance and homework completion rates). Unfortunately, treatment disengagement from these services remains a persistent issue. Intrinsic motivation, or the willingness to exert effort because a task is inherently interesting or meaningful, has emerged as a promising malleable personal factor to enhance treatment engagement. This study investigated whether early task-specific intrinsic motivation and its domains (e.g., interest, perceived competence, and value) predicted treatment engagement within the context of intensive cognitive training and aerobic exercise interventions over a 6-month period. Thirty-nine participants with first-episode schizophrenia were administered baseline measures of task-specific intrinsic motivation inventories, one for cognitive training and one for exercise, and completed a 6-month randomized clinical trial comparing a neuroplasticity-based cognitive training plus aerobic exercise program against the same cognitive training alone. Results indicated that higher baseline scores of intrinsic motivation for cognitive training, specifically early perceptions of task interest and value, were predictive of greater cognitive training and exercise group attendance. Scores for exercise-specific intrinsic motivation were generally unrelated to indices of exercise participation, with the exception that the gain over time in perceived choice for exercise was linked with greater exercise homework completion and a similar directional tendency for greater in-clinic exercise attendance. This study provides support for monitoring and enhancing motivation early during service delivery to maximize engagement and the likelihood of successful treatment outcomes.

Digital phenotyping in psychiatry: A scoping review

BACKGROUND

Digital phenotyping has been defined as the moment-by-moment assessment of an illness state through digital means, promising objective, quantifiable data on psychiatric patients' conditions, and could potentially improve diagnosis and management of mental illness. As it is a rapidly growing field, it is to be expected that new literature is being published frequently.

OBJECTIVE

We conducted this scoping review to assess the current state of literature on digital phenotyping and offer some discussion on the current trends and future direction of this area of research.

METHODS

We searched four databases, PubMed, Ovid MEDLINE, PsycINFO and Web of Science, from inception to August 25th, 2021. We included studies written in English that 1) investigated or applied their findings to diagnose psychiatric disorders and 2) utilized passive sensing for management or diagnosis. Protocols were excluded. A narrative synthesis approach was used, due to the heterogeneity and variability in outcomes and outcome types reported.

RESULTS

Of 10506 unique records identified, we included a total of 107 articles. The number of published studies has increased over tenfold from 2 in 2014 to 28 in 2020, illustrating the field's rapid growth. However, a significant proportion of these (49% of all studies and 87% of primary studies) were proof of concept, pilot or correlational studies examining digital phenotyping's potential. Most (62%) of the primary studies published evaluated individuals with depression (21%), BD (18%) and SZ (23%) (Appendix 1).

CONCLUSION

There is promise shown in certain domains of data and their clinical relevance, which have yet to be fully elucidated. A consensus has yet to be reached on the best methods of data collection and processing, and more multidisciplinary collaboration between physicians and other fields is needed to unlock the full potential of digital phenotyping and allow for statistically powerful clinical trials to prove clinical utility.

Ambulatory audio and video recording for digital phenotyping in schizophrenia: Adherence & data usability

Ambulatory audio and video recording provides a wealth of information which can be used for a broad range of applications, including digital phenotyping, telepsychiatry, and telepsychology. However, these technologies are in their infancy, and guidelines for their use and analysis have yet to be established. The current project used ambulatory assessment data from individuals with schizophrenia (N = 52) and controls (N = 55) over a week to assess factors influencing sufficiency and useability of video and audio data. Logistic multilevel models examined the effect of relevant variables on video provision and video quality. There was no difference by group in video provision or quality. Videos were less likely to be provided later in the study and later in the day. Video quality was lower later in the day, particularly for controls. Participants were more likely to provide videos if alone or at home than in other settings. Black participants were less likely to have analyzable video frames than White participants. These results suggest potential racial disparities in camera technologies and/or facial analysis algorithms. Implications of these findings and recommendations for future study development, such as instructions to provide to participants to optimize video quality, are discussed.

The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality

As the COVID-19 pandemic has largely increased the utilization of telehealth, mobile mental health technologies - such as smartphone apps, vir-tual reality, chatbots, and social media - have also gained attention. These digital health technologies offer the potential of accessible and scalable interventions that can augment traditional care. In this paper, we provide a comprehensive update on the overall field of digital psychiatry, covering three areas. First, we outline the relevance of recent technological advances to mental health research and care, by detailing how smartphones, social media, artificial intelligence and virtual reality present new opportunities for "digital phenotyping" and remote intervention. Second, we review the current evidence for the use of these new technological approaches across different mental health contexts, covering their emerging efficacy in self-management of psychological well-being and early intervention, along with more nascent research supporting their use in clinical management of long-term psychiatric conditions - including major depression; anxiety, bipolar and psychotic disorders; and eating and substance use disorders - as well as in child and adolescent mental health care. Third, we discuss the most pressing challenges and opportunities towards real-world implementation, using the Integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework to explain how the innovations themselves, the recipients of these innovations, and the context surrounding innovations all must be considered to facilitate their adoption and use in mental health care systems. We conclude that the new technological capabilities of smartphones, artificial intelligence, social media and virtual reality are already changing mental health care in unforeseen and exciting ways, each accompanied by an early but promising evidence base. We point out that further efforts towards strengthening implementation are needed, and detail the key issues at the patient, provider and policy levels which must now be addressed for digital health technologies to truly improve mental health research and treatment in the future.

Digital phenotyping adherence, feasibility, and tolerability in outpatients with schizophrenia

Digital phenotyping has potential for use as an objective and ecologically valid form of symptom assessment in clinical trials for schizophrenia. However, there are critical methodological factors that must be addressed before digital phenotyping can be used for this purpose. The current study evaluated levels of adherence, feasibility, and tolerability for active (i.e., signal and event contingent ecological momentary assessment surveys) and passive (i.e., geolocation, accelerometry, and ambulatory psychophysiology) digital phenotyping methods recorded from smartphone and smartband devices. Participants included outpatients diagnosed with schizophrenia (SZ: n = 54) and demographically matched healthy controls (CN: n = 55), who completed 6 days of digital phenotyping. Adherence was significantly lower in SZ than CN for active recordings, but not markedly different for passive recordings. Some forms of passive recordings had lower adherence (ambulatory psychophysiology) than others (accelerometry and geolocation). Active digital phenotyping adherence was predicted by higher psychosocial functioning, whereas passive digital phenotyping adherence was predicted by education, positive symptoms, negative symptoms, and psychosocial functioning in people with SZ. Both groups found digital phenotyping methods tolerable and feasibility was supported by low frequency of invalid responding, brief survey completion times, and similar impediments to study completion. Digital phenotyping methods can be completed by individuals with SZ with good adherence, feasibility, and tolerability. Recommendations are provided for using digital phenotyping methods in clinical trials for SZ.