Digital assessment of working memory and processing speed in everyday life: Feasibility, validation, and lessons-learned

Measuring Situational Cognitive Performance in the Wild: A Psychometric Evaluation of Three Brief Smartphone-Based Test Procedures

Mobile devices provide new opportunities to draw conclusions about cognitive performance in everyday situations. To gain insights into cognitive performance patterns in healthy adult populations, we adapted three established cognitive tests for smartphone use: the Digit Symbol Substitution Task (DSST), Sustained Attention to Response Task (SART), and Psychomotor Vigilance Task (PVT). To increase their feasibility for ambulatory assessment, we identified the minimum measurement durations that provide reliable and valid state measures of cognitive performance. Over 2 weeks, 46 participants performed each test once per day at random times, along with self-reports (e.g., on concentration, mood, and mental demands). The validity and reliability of change are promising for the 30-second PVT and 90-second DSST and SART. The DSST and SART provide fruitful outcomes for ambulatory field studies linked to mood, stress, and mental demands. We provide digital versions of the adapted DSST and SART online for free.

Neuromuscular electrical stimulation to combat cognitive aging in people with spinal cord injury: protocol for a single case experimental design study

INTRODUCTION

Individuals with spinal cord injury (SCI) can experience accelerated cognitive aging. Myokines (factors released from muscle cells during contractions), such as brain-derived neurotrophic factor (BDNF), are thought to have beneficial effects on cognition. Neuromuscular electrical stimulation (NMES) was shown to elicit a large release of myokines. However, the effects of NMES on cognitive function have not been studied.

OBJECTIVE

To present the study protocol for a clinical trial evaluating the effects of NMES aimed at improving cognition and BDNF.

METHODS

A replicated randomized three-phases single-case experimental design (SCED) with sequential multiple baseline time series and a single-armed prospective trial will be conducted with 15 adults with chronic SCI (> 12 months after injury) above L1 neurological level undergoing 30-min quadriceps NMES, 3 days per week for 12 weeks.

MAIN STUDY ENDPOINTS

Primary endpoint is cognitive performance (assessed by a smartphone test) conducted three times per week during the baseline phase with random duration of 3 to 8 weeks, the intervention phase of 12 weeks, and the follow-up phase of 3 weeks after a no measurement rest period of 12 weeks. Secondary endpoints are changes in BDNF levels and cognitive performance measured before the baseline period, before and after intervention and after a 12 weeks follow-up.

CONCLUSION

This will be the first study investigating the effects of 12 weeks NMES on both cognition and BDNF levels in individuals with SCI. The SCED results provide information on individual treatment effect courses which may direct future research.

TRIAL REGISTRATION

ClinicalTrials.gov (NCT05822297, 12/01/2023).

Purpose in Life and Cognitive Function: Evidence for Momentary Associations in Daily Life

Background and Objectives

Purpose in life is associated with healthier cognitive outcomes in older adulthood. This research examines within-person dynamics between momentary purpose and cognitive function to provide proof of concept that increases in purpose are associated with better cognitive performance.

Research Design and Methods

Participants (N = 303; 54% female; Mage = 51.71, SD = 7.32) completed smartphone-based momentary assessments of purpose and short cognitive tasks 3 times a day for 8 days.

Results

In moments when participants felt more purpose driven than their average, they had faster processing speed (b = -1.240, SE = 0.194; p < .001), independent of person, temporal, and contextual factors and practice effects. Momentary purpose was unrelated to visual working memory performance (b = -0.001, SE = 0.001; p = .475). In contrast to purpose, momentary hedonic affect (e.g., happiness) was unrelated to momentary cognition.

Discussion and Implications

Feeling more momentary purpose may support faster processing speed in daily life. Such evidence provides stage 0 support for a purpose-based intervention for healthier cognition, which may be particularly useful in middle adulthood and the transition to older adulthood before the onset of cognitive impairment.

Feedback based on experience sampling data: Examples of current approaches and considerations for future research

Methodologies such as the Experience Sampling Method (ESM) or Ecological Momentary Assessment allow the gathering of fine-graded, dynamic, personal data within a patient's daily life. Currently, it is studied whether feedback based on experience sampling data (ESM-based feedback) can be used as a clinical tool to inform shared decision-making in clinical practice. Although the potential of feedback is recognized, little is known on how to generate, use, and implement it. This article (i) presents n = 15 ongoing ESM projects within the Belgian-Dutch network for ESM research wherein ESM-based feedback is provided to various patient populations, and (ii) summarizes qualitative data on experiences with ESM-based feedback of researchers (n = 8) with extensive expertise with ESM (average of 10 years) involved in these ongoing studies. The following aspects appear to be of relevance when providing ESM-based feedback: training for healthcare professionals and researchers, the use of online interfaces and graphical visualizations to present data, and interacting with patients in a face-to-face setting when discussing the contextual relevance and potential implications. Prospectively, research may build on these aspects and create coherent consensus-based guidelines for the use of ESM-based feedback.

The role of sense of presence in expressing cognitive abilities in a virtual reality task: an initial validation study

There is a raised interest in literature to use Virtual Reality (VR) technology as an assessment tool for cognitive domains. One of the essential advantages of transforming tests in an immersive virtual environment is the possibility of automatically calculating the test's score, a time-consuming process under natural conditions. Although the characteristics of VR can deliver different degrees of immersion in a virtual environment, the sense of presence could jeopardize the evolution of these practices. The sense of presence results from a complex interaction between human, contextual factors, and the VR environment. The present study has two aims: firstly, it contributes to the validation of a virtual version of the naturalistic action test (i.e., virtual reality action test); second, it aims to evaluate the role of sense of presence as a critical booster of the expression of cognitive abilities during virtual reality tasks. The study relies on healthy adults tested in virtual and real conditions in a cross-over research design. The study's results support the validity of the virtual reality action test. Furthermore, two structural equation models are tested to comprehend the role of sense of presence as a moderator in the relationship between cognitive abilities and virtual task performance.

Reliability and Validity of Noncognitive Ecological Momentary Assessment Survey Response Times as an Indicator of Cognitive Processing Speed in People's Natural Environment: Intensive Longitudinal Study

BACKGROUND

Various populations with chronic conditions are at risk for decreased cognitive performance, making assessment of their cognition important. Formal mobile cognitive assessments measure cognitive performance with greater ecological validity than traditional laboratory-based testing but add to participant task demands. Given that responding to a survey is considered a cognitively demanding task itself, information that is passively collected as a by-product of ecological momentary assessment (EMA) may be a means through which people's cognitive performance in their natural environment can be estimated when formal ambulatory cognitive assessment is not feasible. We specifically examined whether the item response times (RTs) to EMA questions (eg, mood) can serve as approximations of cognitive processing speed.

OBJECTIVE

This study aims to investigate whether the RTs from noncognitive EMA surveys can serve as approximate indicators of between-person (BP) differences and momentary within-person (WP) variability in cognitive processing speed.

METHODS

Data from a 2-week EMA study investigating the relationships among glucose, emotion, and functioning in adults with type 1 diabetes were analyzed. Validated mobile cognitive tests assessing processing speed (Symbol Search task) and sustained attention (Go-No Go task) were administered together with noncognitive EMA surveys 5 to 6 times per day via smartphones. Multilevel modeling was used to examine the reliability of EMA RTs, their convergent validity with the Symbol Search task, and their divergent validity with the Go-No Go task. Other tests of the validity of EMA RTs included the examination of their associations with age, depression, fatigue, and the time of day.

RESULTS

Overall, in BP analyses, evidence was found supporting the reliability and convergent validity of EMA question RTs from even a single repeatedly administered EMA item as a measure of average processing speed. BP correlations between the Symbol Search task and EMA RTs ranged from 0.43 to 0.58 (P<.001). EMA RTs had significant BP associations with age (P<.001), as expected, but not with depression (P=.20) or average fatigue (P=.18). In WP analyses, the RTs to 16 slider items and all 22 EMA items (including the 16 slider items) had acceptable (>0.70) WP reliability. After correcting for unreliability in multilevel models, EMA RTs from most combinations of items showed moderate WP correlations with the Symbol Search task (ranged from 0.29 to 0.58; P<.001) and demonstrated theoretically expected relationships with momentary fatigue and the time of day. The associations between EMA RTs and the Symbol Search task were greater than those between EMA RTs and the Go-No Go task at both the BP and WP levels, providing evidence of divergent validity.

CONCLUSIONS

Assessing the RTs to EMA items (eg, mood) may be a method of approximating people's average levels of and momentary fluctuations in processing speed without adding tasks beyond the survey questions.

Remote Digital Technologies for the Early Detection and Monitoring of Cognitive Decline in Patients With Type 2 Diabetes: Insights From Studies of Neurodegenerative Diseases

Type 2 diabetes (T2D) is a risk factor for cognitive decline. In neurodegenerative disease research, remote digital cognitive assessments and unobtrusive sensors are gaining traction for their potential to improve early detection and monitoring of cognitive impairment. Given the high prevalence of cognitive impairments in T2D, these digital tools are highly relevant. Further research incorporating remote digital biomarkers of cognition, behavior, and motor functioning may enable comprehensive characterizations of patients with T2D and may ultimately improve clinical care and equitable access to research participation. The aim of this commentary article is to review the feasibility, validity, and limitations of using remote digital cognitive tests and unobtrusive detection methods to identify and monitor cognitive decline in neurodegenerative conditions and apply these insights to patients with T2D.

Novel digital methods for gathering intensive time series data in mental health research: scoping review of a rapidly evolving field

Recent technological advances enable the collection of intensive longitudinal data. This scoping review aimed to provide an overview of methods for collecting intensive time series data in mental health research as well as basic principles, current applications, target constructs, and statistical methods for this type of data.In January 2021, the database MEDLINE was searched. Original articles were identified that (1) used active or passive data collection methods to gather intensive longitudinal data in daily life, (2) had a minimum sample size of N ⩾ 100 participants, and (3) included individuals with subclinical or clinical mental health problems.In total, 3799 original articles were identified, of which 174 met inclusion criteria. The most widely used methods were diary techniques (e.g. Experience Sampling Methodology), various types of sensors (e.g. accelerometer), and app usage data. Target constructs included affect, various symptom domains, cognitive processes, sleep, dysfunctional behaviour, physical activity, and social media use. There was strong evidence on feasibility of, and high compliance with, active and passive data collection methods in diverse clinical settings and groups. Study designs, sampling schedules, and measures varied considerably across studies, limiting the generalisability of findings.Gathering intensive longitudinal data has significant potential to advance mental health research. However, more methodological research is required to establish and meet critical quality standards in this rapidly evolving field. Advanced approaches such as digital phenotyping, ecological momentary interventions, and machine-learning methods will be required to efficiently use intensive longitudinal data and deliver personalised digital interventions and services for improving public mental health.

A Novel Experience Sampling Method Tool Integrating Momentary Assessments of Cognitive Biases: Two Compliance, Usability, and Measurement Reactivity Studies

BACKGROUND

Experience sampling methods (ESMs) are increasingly being used to study ecological emotion dynamics in daily functioning through repeated assessments taken over several days. However, most of these ESM approaches are only based on self-report assessments, and therefore, studies on the ecological trajectories of their underlying mechanisms are scarce (ie, cognitive biases) and require evaluation through experimental tasks. We developed a novel ESM tool that integrates self-report measures of emotion and emotion regulation with a previously validated app-based cognitive task that allows for the assessment of underlying mechanisms during daily functioning.

OBJECTIVE

The objective of the study is to test this new tool and study its usability and the possible factors related to compliance with it in terms of latency and missing responses. Among the compliance predictors, we considered psychological and time-related variables, as well as usability, measurement reactivity, and participants' satisfaction with the tool.

METHODS

We conducted 2 extensive ESM studies-study 1 (N=84; a total of 3 assessments per day for 5 days) and study 2 (N=135; a total of 3 assessments per day for 10 days).

RESULTS

In both studies, participants found the tool highly usable (average usability score >81). By using mixed regression models, we found both common and specific results for the compliance predictors. In both study 1 and study 2, latency was significantly predicted by the day (P<.001 and P=.003, respectively). Participants showed slower responses to the notification as the days of the study progressed. In study 2 but not in study 1, latency was further predicted by individual differences in overload with the use of the app, and missing responses were accounted for by individual differences in stress reactivity to notifications (P=.04). Thus, by using a more extensive design, participants who experienced higher overload during the study were characterized by slower responses to notifications (P=.01), whereas those who experienced higher stress reactivity to the notification system were characterized by higher missing responses.

CONCLUSIONS

The new tool had high levels of usability. Furthermore, the study of compliance is of enormous importance when implementing novel ESM methods, including app-based cognitive tasks. The main predictors of latency and missing responses found across studies, specifically when using extensive ESM protocols (study 2), are methodology-related variables. Future research that integrates cognitive tasks in ESM designs should take these results into consideration by performing accurate estimations of participants' response rates to facilitate the optimal quality of novel eHealth approaches, as in this study.

Incorporating ecological momentary assessment into multimethod investigations of cognitive aging: Promise and practical considerations

Ecological momentary assessment (EMA) represents a promising approach to study cognitive aging. In contrast to laboratory-based studies, EMA involves the repeated sampling of experiences in daily life contexts, enabling investigators to gain access to dynamic processes (e.g., situational contexts, intraindividual variability) that are likely to strongly contribute to aging and age-related change across the adult life-span. As such, EMA approaches complement the prevailing research methods in the field of cognitive aging (e.g., laboratory-based paradigms, neuroimaging), while also providing the opportunity to replicate and extend findings from the laboratory in more naturalistic contexts. Following an overview of the methodological and conceptual strengths of EMA approaches in cognitive aging research, we discuss best practices for researchers interested in implementing EMA studies. A key goal is to highlight the tremendous potential for combining EMA methods with other laboratory-based approaches, in order to increase the robustness, replicability, and real-world implications of research findings in the field of cognitive aging. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Between-person and within-person associations of sleep and working-memory in the everyday lives of old and very old adults: initial level, learning, and variability

STUDY OBJECTIVES

Sleep duration affects various aspects of cognitive performance, such as working-memory and learning, among children and adults. However, it remains open, whether similar or even stronger associations exist in old and very old age when changes in sleep and cognitive decrements are common.

METHODS

Using repeated daily-life assessments from a sample of 121 young-old (66-69 years old) and 39 old-old adults (84-90 years old), we assessed links between sleep duration and different aspects of working-memory (initial level, practice-related learning, and residualized variability) between and within persons. Participants reported their sleep durations every morning and performed a numerical working-memory updating task six times a day for seven consecutive days.

RESULTS

Both people who slept longer and those who slept shorter than the sample average showed lower initial performance levels, but a stronger increase of WM over time (i.e. larger learning effects), relative to people with average sleep. Sleep duration did not predict performance variability. Within-person associations were found for people sleeping relatively little on average: For them, working-memory performance was lower on days with shorter than average sleep, yet higher on days with longer than average sleep. Except for lower initial levels of working-memory in old-old adults, no differences between young-old and old-old adults were observed.

CONCLUSION

We conclude that sufficient sleep remains important for working-memory performance in older adults and that it is relevant to include different aspects of working-memory performance, because effects differed for initial performance and learning.

Feasibility and Psychometric Integrity of Mobile Phone-Based Intensive Measurement of Cognition in Older Adults

Background:There is a pressing need for assessment approaches that can be deployed remotely to measure cognitive outcomes in clinical trials and longitudinal aging cohorts. We evaluated the utility of a mobile phone-based intensive measurement study for this purpose. Method:A small cohort of healthy older adults (N = 17, mean age = 73) completed five assessment "bursts" over 12 months, with each measurement burst involving two assessments daily for five consecutive days. Each assessment included brief tests of visual short-term memory and information processing speed, as well as surveys measuring state factors that can affect cognition. Results:At study endpoint we had 94% retention, 97% compliance, and high participant satisfaction. Mobile cognitive test scores demonstrated good reliability, moderate correlations with in-person baseline neuropsychological testing, and significant associations with participant age and education level. Conclusions: Mobile phone-based intensive measurement designs represent a promising assessment approach for measuring cognition longitudinally in older adults.

Wearable Computing Technology for Assessment of Cognitive Functioning of Bipolar Patients and Healthy Controls

Mobile cognitive tests have been emerged to first, bring the assessments outside the clinics and second, frequently measure individuals' cognitive performance in their free-living environment. Patients with Bipolar Disorder (BD) suffer from cognitive impairments and poor sleep quality negatively affects their cognitive performance. Wearables are capable of unobtrusively collecting multivariate data including activity and sleep features. In this study, we analyzed daily attention, working memory, and executive functions of patients with BD and healthy controls by using a smartwatch-based tool called UbiCAT to 1) investigate its concurrent validity and feasibility, 2) identify digital phenotypes of mental health using cognitive and mobile sensor data, and 3) classify patients and healthy controls on the basis of their daily cognitive and mobile data. Our findings demonstrated that UbiCAT is feasible with valid measures for in-the-wild cognitive assessments. The analysis showed that the patients responded more slowly during the attention task than the healthy controls, which could indicate a lower alertness of this group. Furthermore, sleep duration correlated positively with participants' working memory performance the next day. Statistical analysis showed that features including cognitive measures of attention and executive functions, sleep duration, time in bed, awakening frequency and duration, and step counts are the digital phenotypes of mental health diagnosis. Supervised learning models was used to classify individuals' mental health diagnosis using their daily observations. Overall, we achieved accuracy of approximately 74% using K-Nearest Neighbour (KNN) method.

Smartphone-Based Experience Sampling in People With Mild Cognitive Impairment: Feasibility and Usability Study

BACKGROUND

Daily functioning of people with cognitive disorders such as mild cognitive impairment (MCI) is usually depicted by retrospective questionnaires, which can be memory-biased and neglect fluctuations over time or contexts.

OBJECTIVE

This study examines the feasibility and usability of applying the experience sampling method (ESM) in people with MCI to provide a detailed and dynamic picture of behavioral, emotional, and cognitive patterns in everyday life.

METHODS

For 6 consecutive days, 21 people with MCI used an ESM app on their smartphones. At 8 semi-random timepoints per day, participants filled in momentary questionnaires on mood, activities, social context, and subjective cognitive complaints. Feasibility was determined through self-reports and observable human-technology interactions. Usability was demonstrated on an individual and group level.

RESULTS

Of the 21 participants, 3 dropped out due to forgetting to carry their smartphones or forgetting the study instructions. In the remaining 18 individuals, the compliance rate was high, at 78.7%. Participants reported that momentary questions reflected their daily experiences well. Of the 18 participants, 13 (72%) experienced the increase in awareness of their own memory functions as pleasant or neutral.

CONCLUSIONS

Support was found for the general feasibility of smartphone-based experience sampling in people with MCI. However, many older adults with MCI are currently not in possession of smartphones, and study adherence seems challenging for a minority of individuals. Momentary data can increase the insights into daily patterns and may guide the person-tailored development of self-management strategies in clinical settings.

Feasibility and initial validation of 'HD-Mobile', a smartphone application for remote self-administration of performance-based cognitive measures in Huntington's disease

OBJECTIVE

Smartphone-based cognitive assessment measures allow efficient, rapid, and convenient collection of cognitive datasets. Establishment of feasibility and validity is essential for the widespread use of this approach. We describe a novel smartphone application (HD-Mobile) that includes three performance-based cognitive tasks with four key outcome measures, for use with Huntington's disease (HD) samples. We describe known groups and concurrent validity, test-retest reliability, sensitivity, and feasibility properties of the tasks.

METHODS

Forty-two HD CAG-expanded participants (20 manifest, 22 premanifest) and 28 healthy controls completed HD-Mobile cognitive tasks three times across an 8-day period, on days 1, 4, and 8. A subsample of participants had pen-and-paper cognitive task data available from their most recent assessment from their participation in a separate observational longitudinal study, Enroll-HD.

RESULTS

Manifest-HD participants performed worse than healthy controls for three of four HD-Mobile cognitive measures, and worse than premanifest-HD participants for two of four measures. We found robust test-retest reliability for manifest-HD participants (ICC = 0.71-0.96) and with some exceptions, in premanifest-HD (ICC = 0.52-0.96) and healthy controls (0.54-0.96). Correlations between HD-Mobile and selected Enroll-HD cognitive tasks were mostly medium to strong (r = 0.36-0.68) as were correlations between HD-Mobile cognitive tasks and measures of expected disease progression and motor symptoms for the HD CAG-expanded participants (r = - 0.34 to - 0.54).

CONCLUSIONS

Results indicated robust known-groups, test-retest, concurrent validity, and sensitivity of HD-Mobile cognitive tasks. The study demonstrates the feasibility and utility of HD-Mobile for conducting convenient, frequent, and potentially ongoing assessment of HD samples without the need for in-person assessment.