Smartphone-derived Virtual Keyboard Dynamics Coupled with Accelerometer Data as a Window into Understanding Brain Health: Smartphone Keyboard and Accelerometer as Window into Brain Health

Smartphone keyboard dynamics predict affect in suicidal ideation

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = -0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

Smartphone keyboard dynamics predict affect in suicidal ideation

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = -0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

A Review on Smartphone Keystroke Dynamics as a Digital Biomarker for Understanding Neurocognitive Functioning

Can digital technologies provide a passive unobtrusive means to observe and study cognition outside of the laboratory? Previously, cognitive assessments and monitoring were conducted in a laboratory or clinical setting, allowing for a cross-sectional glimpse of cognitive states. In the last decade, researchers have been utilizing technological advances and devices to explore ways of assessing cognition in the real world. We propose that the virtual keyboard of smartphones, an increasingly ubiquitous digital device, can provide the ideal conduit for passive data collection to study cognition. Passive data collection occurs without the active engagement of a participant and allows for near-continuous, objective data collection. Most importantly, this data collection can occur in the real world, capturing authentic datapoints. This method of data collection and its analyses provide a more comprehensive and potentially more suitable insight into cognitive states, as intra-individual cognitive fluctuations over time have shown to be an early manifestation of cognitive decline. We review different ways passive data, centered around keystroke dynamics, collected from smartphones, have been used to assess and evaluate cognition. We also discuss gaps in the literature where future directions of utilizing passive data can continue to provide inferences into cognition and elaborate on the importance of digital data privacy and consent.