Naturalistic smartphone keyboard typing reflects processing speed and executive function

OBJECTIVE

The increase in smartphone usage has enabled the possibility of more accessible ways to conduct neuropsychological evaluations. The objective of this study was to determine the feasibility of using smartphone typing dynamics with mood scores to supplement cognitive assessment through trail making tests.

METHODS

Using a custom-built keyboard, naturalistic keypress dynamics were unobtrusively recorded in individuals with bipolar disorder (n = 11) and nonbipolar controls (n = 8) on an Android smartphone. Keypresses were matched to digital trail making tests part B (dTMT-B) administered daily in two periods and weekly mood assessments. Following comparison of dTMT-Bs to the pencil-and-paper equivalent, longitudinal mixed-effects models were used to analyze daily dTMT-B performance as a function of typing and mood.

RESULTS

Comparison of the first dTMT-B to paper TMT-B showed adequate reliability (intraclass correlations = 0.74). In our model, we observed that participants who typed slower took longer to complete dTMT-B (b = 0.189, p < .001). This trend was also seen in individual fluctuations in typing speed and dTMT-B performance (b = 0.032, p = .004). Moreover, participants who were more depressed completed the dTMT-B slower than less depressed participants (b = 0.189, p < .001). A practice effect was observed for the dTMT-Bs.

CONCLUSION

Typing speed in combination with depression scores has the potential to infer aspects of cognition (visual attention, processing speed, and task switching) in people's natural environment to complement formal in-person neuropsychological assessments that commonly include the trail making test.

Effects of mood and aging on keystroke dynamics metadata and their diurnal patterns in a large open-science sample: A BiAffect iOS study

OBJECTIVE

Ubiquitous technologies can be leveraged to construct ecologically relevant metrics that complement traditional psychological assessments. This study aims to determine the feasibility of smartphone-derived real-world keyboard metadata to serve as digital biomarkers of mood.

MATERIALS AND METHODS

BiAffect, a real-world observation study based on a freely available iPhone app, allowed the unobtrusive collection of typing metadata through a custom virtual keyboard that replaces the default keyboard. User demographics and self-reports for depression severity (Patient Health Questionnaire-8) were also collected. Using >14 million keypresses from 250 users who reported demographic information and a subset of 147 users who additionally completed at least 1 Patient Health Questionnaire, we employed hierarchical growth curve mixed-effects models to capture the effects of mood, demographics, and time of day on keyboard metadata.

RESULTS

We analyzed 86 541 typing sessions associated with a total of 543 Patient Health Questionnaires. Results showed that more severe depression relates to more variable typing speed (P < .001), shorter session duration (P < .001), and lower accuracy (P < .05). Additionally, typing speed and variability exhibit a diurnal pattern, being fastest and least variable at midday. Older users exhibit slower and more variable typing, as well as more pronounced slowing in the evening. The effects of aging and time of day did not impact the relationship of mood to typing variables and were recapitulated in the 250-user group.

CONCLUSIONS

Keystroke dynamics, unobtrusively collected in the real world, are significantly associated with mood despite diurnal patterns and effects of age, and thus could serve as a foundation for constructing digital biomarkers.

The Effects of Bipolar Disorder Risk on a Mobile Phone Keystroke Dynamics Based Biomarker of Brain Age

Background: Research by our group and others have demonstrated the feasibility of using mobile phone derived metadata to model mood and cognition. Given the effects of age and mood on cognitive performance, it was hypothesized that using such data a model could be built to predict chronological age and that differences between predicted age and actual age could be a marker of pathology. Methods: These data were collected via the ongoing BiAffect study. Participants complete the Mood Disorders Questionnaire (MDQ), a screening questionnaire for bipolar disorder, and self-reported their birth year. Data were split into training and validation sets. Features derived from the smartphone kinematics were used to train random forest regression models to predict age. Prediction errors were compared between participants screening positive and negative on the MDQ. Results: Three hundred forty-four participants had analyzable data of which 227 had positive screens for bipolar disorder and 117 had negative screens. The absolute prediction error tended to be lower for participants with positive screens (median 4.50 years) than those with negative screens (median 7.92 years) (W = 508, p = 0.0049). The raw prediction error tended to be lower for participants with negative screens (median = -5.95 years) than those with positive screens (median = 0.55 years) (W = 1,037, p= 0.037). Conclusions: The tendency to underestimate the chronological age of participants screening negative for bipolar disorder compared to those screening positive is consistent with the finding that bipolar disorder may be associated with brain changes that could reflect pathological aging. This interesting result could also reflect that those who screen negative for bipolar disorder and who engaged in the study were more likely to have higher premorbid functioning. This work demonstrates that age-related changes may be detected via a passive smartphone kinematics based digital biomarker.

Predicting Mood Disturbance Severity with Mobile Phone Keystroke Metadata: A BiAffect Digital Phenotyping Study

Background

Mood disorders are common and associated with significant morbidity and mortality. Better tools are needed for their diagnosis and treatment. Deeper phenotypic understanding of these disorders is integral to the development of such tools. This study is the first effort to use passively collected mobile phone keyboard activity to build deep digital phenotypes of depression and mania.

Objective

The objective of our study was to investigate the relationship between mobile phone keyboard activity and mood disturbance in subjects with bipolar disorders and to demonstrate the feasibility of using passively collected mobile phone keyboard metadata features to predict manic and depressive signs and symptoms as measured via clinician-administered rating scales.

Methods

Using a within-subject design of 8 weeks, subjects were provided a mobile phone loaded with a customized keyboard that passively collected keystroke metadata. Subjects were administered the Hamilton Depression Rating Scale (HDRS) and Young Mania Rating Scale (YMRS) weekly. Linear mixed-effects models were created to predict HDRS and YMRS scores. The total number of keystrokes was 626,641, with a weekly average of 9791 (7861), and that of accelerometer readings was 6,660,890, with a weekly average 104,076 (68,912).

Results

A statistically significant mixed-effects regression model for the prediction of HDRS-17 item scores was created: conditional R²=.63, P=.01. A mixed-effects regression model for YMRS scores showed the variance accounted for by random effect was zero, and so an ordinary least squares linear regression model was created: R²=.34, P=.001. Multiple significant variables were demonstrated for each measure.

Conclusions

Mood states in bipolar disorder appear to correlate with specific changes in mobile phone usage. The creation of these models provides evidence for the feasibility of using passively collected keyboard metadata to detect and monitor mood disturbances.