Feasibility and validity of mobile cognitive testing in the investigation of age-related cognitive decline

Characterizing Longitudinal Patterns in Cognition, Mood, And Activity in Depression With 6-Week High-Frequency Wearable Assessment: Observational Study

BACKGROUND

Cognitive symptoms are an underrecognized aspect of depression that are often untreated. High-frequency cognitive assessment holds promise for improving disease and treatment monitoring. Although we have previously found it feasible to remotely assess cognition and mood in this capacity, further work is needed to ascertain the optimal methodology to implement and synthesize these techniques.

OBJECTIVE

The objective of this study was to examine (1) longitudinal changes in mood, cognition, activity levels, and heart rate over 6 weeks; (2) diurnal and weekday-related changes; and (3) co-occurrence of fluctuations between mood, cognitive function, and activity.

METHODS

A total of 30 adults with current mild-moderate depression stabilized on antidepressant monotherapy responded to testing delivered through an Apple Watch (Apple Inc) for 6 weeks. Outcome measures included cognitive function, assessed with 3 brief n-back tasks daily; self-reported depressed mood, assessed once daily; daily total step count; and average heart rate. Change over a 6-week duration, diurnal and day-of-week variations, and covariation between outcome measures were examined using nonlinear and multilevel models.

RESULTS

Participants showed initial improvement in the Cognition Kit N-Back performance, followed by a learning plateau. Performance reached 90% of individual learning levels on average 10 days after study onset. N-back performance was typically better earlier and later in the day, and step counts were lower at the beginning and end of each week. Higher step counts overall were associated with faster n-back learning, and an increased daily step count was associated with better mood on the same (P<.001) and following day (P=.02). Daily n-back performance covaried with self-reported mood after participants reached their learning plateau (P=.01).

CONCLUSIONS

The current results support the feasibility and sensitivity of high-frequency cognitive assessments for disease and treatment monitoring in patients with depression. Methods to model the individual plateau in task learning can be used as a sensitive approach to better characterize changes in behavior and improve the clinical relevance of cognitive data. Wearable technology allows assessment of activity levels, which may influence both cognition and mood.

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

Remote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App

BACKGROUND

Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians' ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics' time, encourage broader screening practices, and increase early detection of cognitive decline.

OBJECTIVE

The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile?

METHODS

A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items.

RESULTS

All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants' rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants' abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items.

CONCLUSIONS

Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.

Adherence type impacts completion rates of frequent mobile cognitive assessments among older adults with and without cognitive impairment

Background

Prior to a diagnosis of Alzheimer's disease, many individuals experience cognitive and behavioral fluctuations that are not detected during a single session of traditional neuropsychological assessment. Mobile applications now enable high-frequency cognitive data to be collected remotely, introducing new opportunities and challenges. Emerging evidence suggests cognitively impaired older adults are capable of completing mobile assessments frequently, but no study has observed whether completion rates vary by assessment frequency or adherence type.

Methods

Thirty-three older adults were recruited from the Boston University Alzheimer's Disease Research Center (mean age = 73.5 years; 27.3% cognitively impaired; 57.6% female; 81.8% White, 18.2% Black). Participants remotely downloaded and completed the DANA Brain Vital application on their own mobile devices throughout the study. The study schedule included seventeen assessments to be completed over the course of a year. Specific periods during which assessments were expected to be completed were defined as subsegments, while segments consisted of multiple subsegments. The first segment included three subsegments to be completed within one week, the second segment included weekly subsegments and spanned three weeks, and the third and fourth segments included monthly subsegments spanning five and six months, respectively. Three distinct adherence types - subsegment adherence, segment adherence, and cumulative adherence - were examined to determine how completion rates varied depending on assessment frequency and adherence type.

Results

Adherence type significantly impacted whether the completion rates declined. When utilizing subsegment adherence, the completion rate significantly declined (p = 0.05) during the fourth segment. However, when considering completion rates from the perspective of segment adherence, a decline in completion rate was not observed. Overall adherence rates increased as adherence parameters were broadened from subsegment adherence (60.6%) to segment adherence (78.8%), to cumulative adherence (90.9%).

Conclusions

Older adults, including those with cognitive impairment, are able to complete remote cognitive assessments at a high-frequency, but may not necessarily adhere to prescribed schedules.

Daily Assessment of Executive Functioning and Expressive Suppression Predict Daily Functioning among Community-Dwelling Older Adults

OBJECTIVE

Executive functioning (EF) is known to be associated with performance of instrumental activities of daily living (IADLs). However, prior research has found that the degree to which EF fluctuates was more predictive of self-reported cognitive and IADL lapses than was average EF performance. One source of such EF fluctuations is engagement in an emotion regulation strategy known as expressive suppression (ES). Importantly, ES has also been shown to relate to IADL performance, presumably due to its impact on EF. However, past research is limited due to assessing IADLs only in the laboratory or via self-report. The present study examined (a) the association of daily EF and ES fluctuations with performance of actual IADL tasks in participants' homes, and (b) whether any significant association between ES fluctuations and daily IADLs would be mediated by daily EF variability.

METHOD

Participants were 52 older adults aged 60 to 95. Over the course of 18 days while at home, participants completed daily IADL tasks as well as daily measures of EF and ES via ecological momentary assessment.

RESULTS

Contrary to our hypothesis, average EF across days predicted at-home IADLs above and beyond daily EF variability, which itself was also predictive. ES variability also predicted daily IADLs, and this association was fully mediated by average daily EF.

CONCLUSIONS

Daily fluctuations in ES appear to have a deleterious impact on performance of IADLs at home, likely due to the impact of such fluctuations on EF, although the average level of EF capacity is also important.

Ecological validity reconsidered: the Night Out Task versus the D-KEFS

INTRODUCTION

Although executive functioning (EF) correlates with execution of instrumental activities of daily living (IADLs), tests of EF have been criticized for having poor ecological validity. Attempts have been made to develop new tests that approximate naturalistic daily tasks. However, the incremental utility of such tests has not been convincingly demonstrated. The Night Out Task (NOT) is a novel measure designed to increase ecological validity. This study examined whether the NOT correlates with traditional lab- and home-based measures of EF and IADLs, and whether it outperforms traditional measures of EF in predicting IADLs.

METHOD

Participants (50 adults aged 60 to 95) completed (1) the Delis Kaplan Executive Function System (D-KEFS) and IADLs in the laboratory, and (2) ecological momentary assessment of EF and daily IADL tasks at home across three weeks (using the Daily Assessment of Independent Living and Executive Skills protocol; DAILIES).

RESULTS

The NOT correlated with a lab-based measure of EF beyond covariates, and lab-based IADLs beyond covariates and beyond the D-KEFS. However, it was unrelated to at-home variables beyond covariates. In contrast, the D-KEFS was a significant predictor of at-home IADLs, and this association was mediated by at-home EF performance.

CONCLUSION

This study provides a preliminary validation of the NOT as a correlate of office-based performances in a primarily college educated white sample. Despite its high face validity, the NOT does not appear to sufficiently tap EF processes needed for home-based IADLs as measured by the DAILIES, although small sample size limits the interpretability of this negative finding.

Ecological momentary facial emotion recognition in psychotic disorders

BACKGROUND

Cognitive tasks delivered during ecological momentary assessment (EMA) may elucidate the short-term dynamics and contextual influences on cognition and judgements of performance. This paper provides initial validation of a smartphone task of facial emotion recognition in serious mental illness.

METHODS

A total of 86 participants with psychotic disorders (non-affective and affective psychosis), aged 19-65, were administered in-lab 'gold standard' affect recognition, neurocognition, and symptom assessments. They subsequently completed 10 days of the mobile facial emotion recognition task, assessing both accuracy and self-assessed performance, along with concurrent EMA of psychotic symptoms and mood. Validation focused on task adherence and predictors of adherence, gold standard convergent validity, and symptom and diagnostic group variation.

RESULTS

The mean rate of adherence to the task was 79%; no demographic or clinical variables predicted adherence. Convergent validity was observed with in-lab measures of facial emotion recognition, and no practice effects were observed on the mobile facial emotion recognition task. EMA reports of more severe voices, sadness, and paranoia were associated with worse performance, whereas mood more strongly associated with self-assessed performance.

CONCLUSION

The mobile facial emotion recognition task was tolerated and demonstrated convergent validity with in-lab measures of the same construct. Social cognitive performance, and biased judgements previously shown to predict function, can be evaluated in real-time in naturalistic environments.

Validation of a Remote and Fully Automated Story Recall Task to Assess for Early Cognitive Impairment in Older Adults: Longitudinal Case-Control Observational Study

Background

Story recall is a simple and sensitive cognitive test that is commonly used to measure changes in episodic memory function in early Alzheimer disease (AD). Recent advances in digital technology and natural language processing methods make this test a candidate for automated administration and scoring. Multiple parallel test stimuli are required for higher-frequency disease monitoring.

Objective

This study aims to develop and validate a remote and fully automated story recall task, suitable for longitudinal assessment, in a population of older adults with and without mild cognitive impairment (MCI) or mild AD.

Methods

The “Amyloid Prediction in Early Stage Alzheimer’s disease” (AMYPRED) studies recruited participants in the United Kingdom (AMYPRED-UK: NCT04828122) and the United States (AMYPRED-US: NCT04928976). Participants were asked to complete optional daily self-administered assessments remotely on their smart devices over 7 to 8 days. Assessments included immediate and delayed recall of 3 stories from the Automatic Story Recall Task (ASRT), a test with multiple parallel stimuli (18 short stories and 18 long stories) balanced for key linguistic and discourse metrics. Verbal responses were recorded and securely transferred from participants’ personal devices and automatically transcribed and scored using text similarity metrics between the source text and retelling to derive a generalized match score. Group differences in adherence and task performance were examined using logistic and linear mixed models, respectively. Correlational analysis examined parallel-forms reliability of ASRTs and convergent validity with cognitive tests (Logical Memory Test and Preclinical Alzheimer’s Cognitive Composite with semantic processing). Acceptability and usability data were obtained using a remotely administered questionnaire.

Results

Of the 200 participants recruited in the AMYPRED studies, 151 (75.5%)—78 cognitively unimpaired (CU) and 73 MCI or mild AD—engaged in optional remote assessments. Adherence to daily assessment was moderate and did not decline over time but was higher in CU participants (ASRTs were completed each day by 73/106, 68.9% participants with MCI or mild AD and 78/94, 83% CU participants). Participants reported favorable task usability: infrequent technical problems, easy use of the app, and a broad interest in the tasks. Task performance improved modestly across the week and was better for immediate recall. The generalized match scores were lower in participants with MCI or mild AD (Cohen d=1.54). Parallel-forms reliability of ASRT stories was moderate to strong for immediate recall (mean rho 0.73, range 0.56-0.88) and delayed recall (mean rho=0.73, range=0.54-0.86). The ASRTs showed moderate convergent validity with established cognitive tests.

Conclusions

The unsupervised, self-administered ASRT task is sensitive to cognitive impairments in MCI and mild AD. The task showed good usability, high parallel-forms reliability, and high convergent validity with established cognitive tests. Remote, low-cost, low-burden, and automatically scored speech assessments could support diagnostic screening, health care, and treatment monitoring.

Feasibility and validity of ecological momentary cognitive testing among older adults with mild cognitive impairment

It is critical to intervene early in the mild cognitive impairment (MCI) stage of the Alzheimer's disease trajectory, but traditional cognitive testing methods are costly, burdensome, and difficult to access. We examined adherence and validity data to a 30-day self-administered ecological momentary cognitive testing protocol among a sample of older adults with MCI and cognitively normal controls to evaluate feasibility, tolerability, and initial validity in comparison to standard neuropsychological tests. Participants included 48 participants with MCI (Mean age = 72 years, SD = 7 years) and 46 demographically-matched cognitively normal (NC) control participants (Mean age = 70 years, SD = 7 years). Participants completed traditional neuropsychological testing to determine MCI status, followed by 30 days of remote ecological momentary cognitive testing. Ecological momentary assessment (EMA) surveys were administered 3 times per day for 30 days (possible total = 90), and mobile cognitive tests were administered every other day (for a total of 15 administrations). Mobile cognitive tests included the Variable Difficulty List Memory Test (VLMT; measure of learning and memory), Memory Matrix (measure of visual working memory), and the Color Trick Test (measure of executive function). EMA and mobile cognitive test adherence, fatigue effects, mobile cognitive test performance and group differences, and psychometrics (reliability, convergent validity, ceiling effects, and practice effects) were examined. Overall mean-level adherence to the mobile cognitive tests was 85% and did not differ by MCI status. The reliability of stable between-person individual differences for the VLMT and Memory Matrix were very high. Moreover, although the reliability of within-person change for Memory Matrix was adequate, the corresponding reliability for VLMT was somewhat low. Averaged performance on the mobile cognitive tests was correlated with lab-based tests measuring the same construct. Participants with MCI performed worse than NCs on the VLMT and Color Trick Test, and there was no evidence of fatigue effects for these two tests. These findings support the feasibility and potential for ecological momentary cognitive testing to support clinical trials and for measuring cognitive changes over time in persons with increased risk for Alzheimer's disease such as those with MCI.

Initial Psychometric Properties of 7 NeuroUX Remote Ecological Momentary Cognitive Tests Among People With Bipolar Disorder: Validation Study

BACKGROUND

As smartphone technology has become nearly ubiquitous, there is a growing body of literature suggesting that ecological momentary cognitive testing (EMCT) offers advantages over traditional pen-and-paper psychological assessment. We introduce a newly developed platform for the self-administration of cognitive tests in ecologically valid ways.

OBJECTIVE

The aim of this study is to develop a Health Insurance Portability and Accountability Act-compliant EMCT smartphone-based platform for the frequent and repeated testing of cognitive abilities in everyday life. This study examines the psychometric properties of 7 mobile cognitive tests covering domains of processing speed, visual working memory, recognition memory, and response inhibition within our platform among persons with and without bipolar disorder (BD). Ultimately, if shown to have adequate psychometric properties, EMCTs may be useful in research on BD and other neurological and psychiatric illnesses.

METHODS

A total of 45 persons with BD and 21 demographically comparable healthy volunteer participants (aged 18-65 years) completed smartphone-based EMCTs 3 times daily for 14 days. Each EMCT session lasted approximately 1.5 minutes. Only 2 to 3 tests were administered in any given session, no test was administered more than once per day, and alternate test versions were administered in each session.

RESULTS

The mean adherence to the EMCT protocol was 69.7% (SD 20.5%), resulting in 3965 valid and complete tests across the full sample. Participants were significantly more likely to miss tests on later versus earlier study days. Adherence did not differ by diagnostic status, suggesting that BD does not interfere with EMCT participation. In most tests, age and education were related to EMCT performance in expected directions. The average performances on most EMCTs were moderately to strongly correlated with the National Institutes of Health Toolbox Cognition Battery. Practice effects were observed in 5 tests, with significant differences in practice effects by BD status in 3 tests.

CONCLUSIONS

Although additional reliability and validity data are needed, this study provides initial psychometric support for EMCTs in the assessment of cognitive performance in real-world contexts in BD.

Ecological Momentary Assessment of Creative Ideation

Abstract. Creative ideas in daily life show substantial variation in quality. Yet, most studies investigate the creative ideation process in highly controlled laboratory contexts, which challenges the ecological validity of creativity research findings. In this article, we advocate the use of ambulatory assessments of creative ideation to gain deeper insight into the variability of ideation processes (between- and within-subjects) in everyday life. We demonstrate this approach by the example of the ambulatory battery of creativity (ABC), which constitutes a reliable and valid approach to assess divergent thinking ability in the verbal and figural domain in everyday life context. Furthermore, it differentiates between-person and within-person variation of creative ideation performance. The first part of this paper will shortly describe the general approach using ABC as an example. In the second part, we use the 7 C’s heuristic to explore applications and implications of this novel method for creativity research. We focus on four C’s with special relevance for ambulatory assessment: Creator, Creating, Context, and Curricula. To this end, we review the findings of strongly controlled laboratory studies and discuss and illustrate applications of the ambulatory assessment. We conclude that the assessment of creative ideation performance in the field might help move the spotlight of creative ideation research from the laboratory to more naturalistic settings. This would increase the ecological validity of creative ideation research and facilitate fresh or unprecedented perspectives on past and future questions on a person’s creative potential and its moment-to-moment fluctuation.

A Neuropsychological Perspective on Defining Cognitive Impairment in the Clinical Study of Alzheimer’s Disease: Towards a More Continuous Approach

The global fight against Alzheimer’s disease (AD) poses unique challenges for the field of neuropsychology. Along with the increased focus on early detection of AD pathophysiology, characterizing the earliest clinical stage of the disease has become a priority. We believe this is an important time for neuropsychology to consider how our approach to the characterization of cognitive impairment can be improved to detect subtle cognitive changes during early-stage AD. The present article aims to provide a critical examination of how we define and measure cognitive status in the context of aging and AD. First, we discuss pitfalls of current methods for defining cognitive impairment within the context of research shifting to earlier (pre)symptomatic disease stages. Next, we introduce a shift towards a more continuous approach for identifying early markers of cognitive decline and characterizing progression and discuss how this may be facilitated by novel assessment approaches. Finally, we summarize potential implications and challenges of characterizing cognitive status using a continuous approach.

Validity of Digital Assessments in Screening for HIV-Related Cognitive Impairment: a Review

PURPOSE OF REVIEW

While traditional neuropsychological tests are the gold standard in screening for HIV-related cognitive impairment, computerized neuropsychological assessment devices (CNADs) offer an alternative to these time- and resource-intensive batteries and may prove to be particularly useful for remote assessments or longitudinal monitoring. This review seeks to describe the benefits, limitations, and validity of CNADs in the evaluation of HIV-associated neurocognitive disorder (HAND).

RECENT FINDINGS

We identified eight CNADs that have undergone validity testing for cognitive impairment in the setting of HIV. Included among these are batteries that have been modeled after the traditional neuropsychological exam, as well as others that implement new technologies, such as simulated reality and daily ecological assessments in their testing. Currently, these digital batteries do not yet have the ability to supplant gold standard neuropsychological tests in screening for HAND. However, many have the potential to become effective clinical screening tools.

Remote Digital Psychiatry for Mobile Mental Health Assessment and Therapy: MindLogger Platform Development Study

Background

Universal access to assessment and treatment of mental health and learning disorders remains a significant and unmet need. There are many people without access to care because of economic, geographic, and cultural barriers, as well as the limited availability of clinical experts who could help advance our understanding and treatment of mental health.

Objective

This study aims to create an open, configurable software platform to build clinical measures, mobile assessments, tasks, and interventions without programming expertise. Specifically, our primary requirements include an administrator interface for creating and scheduling recurring and customized questionnaires where end users receive and respond to scheduled notifications via an iOS or Android app on a mobile device. Such a platform would help relieve overwhelmed health systems and empower remote and disadvantaged subgroups in need of accurate and effective information, assessment, and care. This platform has the potential to advance scientific research by supporting the collection of data with instruments tailored to specific scientific questions from large, distributed, and diverse populations.

Methods

We searched for products that satisfy these requirements. We designed and developed a new software platform called MindLogger, which exceeds the requirements. To demonstrate the platform’s configurability, we built multiple applets (collections of activities) within the MindLogger mobile app and deployed several of them, including a comprehensive set of assessments underway in a large-scale, longitudinal mental health study.

Results

Of the hundreds of products we researched, we found 10 that met our primary requirements with 4 that support end-to-end encryption, 2 that enable restricted access to individual users’ data, 1 that provides open-source software, and none that satisfy all three. We compared features related to information presentation and data capture capabilities; privacy and security; and access to the product, code, and data. We successfully built MindLogger mobile and web applications, as well as web browser–based tools for building and editing new applets and for administering them to end users. MindLogger has end-to-end encryption, enables restricted access, is open source, and supports a variety of data collection features. One applet is currently collecting data from children and adolescents in our mental health study, and other applets are in different stages of testing and deployment for use in clinical and research settings.

Conclusions

We demonstrated the flexibility and applicability of the MindLogger platform through its deployment in a large-scale, longitudinal, mobile mental health study and by building a variety of other mental health–related applets. With this release, we encourage a broad range of users to apply the MindLogger platform to create and test applets to advance health care and scientific research. We hope that increasing the availability of applets designed to assess and administer interventions will facilitate access to health care in the general population.

Relationships between daily mood states and real-time cognitive performance in individuals with bipolar disorder and healthy comparators: A remote ambulatory assessment study

OBJECTIVE

Neuropsychological impairments are observed in individuals with Bipolar Disorder (BD), yet knowledge of how cognitive deficits unfold in real-time remains limited. Given intraindividual variability in mood observed in people with BD, and the potential for mood and cognition to be mutually influential, we employed ambulatory assessment technologies to examine potential contemporaneous (same survey) and lagged (next survey) relationships of congition and mood.

METHODS

Outpatients with BD (n = 46) or no psychiatric disorders (heathy volunteers [HV]; n = 20) completed in-laboratory neurobehavioral assessments and 14 days of smartphone-administered mobile cognitive tests and ratings of affective variables. Linear mixed-effects models were used to analyze real-time relationships between mobile cognitive test performance and mood.

RESULTS

On in-laboratory tests, participants with BD showed worse cognitive performance than HVs as well as mild depression severity; mood and cognitive performance were unrelated. On mobile cognitive tests and surveys, participants with BD showed somewhat worse cognitive performance and ratings of lower energy and greater sadness relative to HV participants. Among those with BD, mania and sadness earlier in the day related to worse processing speed and better working memory performance, respectively, on the next survey. In contrast, same survey ratings of greater stress related to better working memory, and greater happiness related to better processing speed.

CONCLUSIONS

Real-time assessments of mood and cognition provide incremental information beyond what can be gleaned from laboratory assessments. Understanding how these affect-related changes in processing speed emerge and play out in daily life may provide clinically useful information for treatment planning.

The Cognitive Online Self-Test Amsterdam (COST-A): Establishing norm scores in a community-dwelling population

BACKGROUND

Heightened public awareness about Alzheimer's disease and dementia increases the need for at-home cognitive self-testing. We offered Cognitive Online Self-Test Amsterdam (COST-A) to independent groups of cognitively normal adults and investigated the robustness of a norm-score formula and cutoff.

METHODS

Three thousand eighty-eight participants (mean age ± standard deviation = 61 ± 12 years, 70% female) completed COST-A and evaluated it. Demographically adjusted norm scores were the difference between expected COST-A scores, based on age, gender, and education, and actual scores. We applied the resulting norm-score formula to two independent cohorts.

RESULTS

Participants evaluated COST-A to be of adequate difficulty and duration. Our norm-score formula was shown to be robust: ≈8% of participants in two cognitively normal cohorts had abnormal scores. A cutoff of -1.5 standard deviations proved optimal for distinguishing normal from impaired cognition.

CONCLUSION

With robust norm scores, COST-A is a promising new tool for research and clinical practice, providing low cost and minimally invasive remote assessment of cognitive functioning.

Current advances in digital cognitive assessment for preclinical Alzheimer's disease

There is a pressing need to capture and track subtle cognitive change at the preclinical stage of Alzheimer's disease (AD) rapidly, cost-effectively, and with high sensitivity. Concurrently, the landscape of digital cognitive assessment is rapidly evolving as technology advances, older adult tech-adoption increases, and external events (i.e., COVID-19) necessitate remote digital assessment. Here, we provide a snapshot review of the current state of digital cognitive assessment for preclinical AD including different device platforms/assessment approaches, levels of validation, and implementation challenges. We focus on articles, grants, and recent conference proceedings specifically querying the relationship between digital cognitive assessments and established biomarkers for preclinical AD (e.g., amyloid beta and tau) in clinically normal (CN) individuals. Several digital assessments were identified across platforms (e.g., digital pens, smartphones). Digital assessments varied by intended setting (e.g., remote vs. in-clinic), level of supervision (e.g., self vs. supervised), and device origin (personal vs. study-provided). At least 11 publications characterize digital cognitive assessment against AD biomarkers among CN. First available data demonstrate promising validity of this approach against both conventional assessment methods (moderate to large effect sizes) and relevant biomarkers (predominantly weak to moderate effect sizes). We discuss levels of validation and issues relating to usability, data quality, data protection, and attrition. While still in its infancy, digital cognitive assessment, especially when administered remotely, will undoubtedly play a major future role in screening for and tracking preclinical AD.

Remote Ecological Momentary Testing of Learning and Memory in Adults With Serious Mental Illness

Smartphone-based ecological mobile cognitive tests (EMCTs) can measure cognitive abilities in the real world, complementing traditional neuropsychological assessments. We evaluated the validity of an EMCT of recognition memory designed for use with people with serious mental illness, as well as relevant contextual influences on performance. Participants with schizophrenia (SZ), schizoaffective disorder, and bipolar disorder (BD) completed in-lab assessments of memory (Hopkins Verbal Learning Test, HVLT), other cognitive abilities, functional capacity, and symptoms, followed by 30 days of EMCTs during which they completed our Mobile Variable Difficulty List Memory Test (VLMT) once every other day (3 trials per session). List length on the VLMT altered between 6, 12, and 18 items. On average, participants completed 75.3% of EMCTs. Overall performance on VLMT 12 and 18 items was positively correlated with HVLT (ρ = 0.52, P < .001). People with BD performed better on the VLMT than people with SZ. Intraindividual variability on the VLMT was more specifically associated with HVLT than nonmemory tests and not associated with symptoms. Performance during experienced distraction, low effort, and out of the home location was reduced yet still correlated with the in-lab HVLT. The VLMT converged with in-lab memory assessment, demonstrating variability within person and by different contexts. Ambulatory cognitive testing on participants' personal mobile devices offers more a cost-effective and "ecologically valid" measurement of real-world cognitive performance.

Smartphone-Based Measurement of Executive Function in Older Adults with and without HIV

OBJECTIVE

To examine feasibility, convergent validity and biases associated with a mobile color-word interference test (mCWIT) among older persons living with HIV (PLHIV).

METHOD

Over a 14-day period, 58 PLHIV and 32 HIV-uninfected individuals (aged 50-74) completed the mCWIT on smartphones once per day in real-world settings. Participants also completed a comprehensive laboratory-based neuropsychological evaluation.

RESULTS

A high rate of compliance was observed (86%) in the repeated administration of the mCWIT. A practice effect was observed in the overall sample concerning mCWIT subscores, and these learning effects were greater for PLHIV. Stabilization of performance was observed after 6 (HIV+) and 7 days (HIV-) for completion time and after 2 (HIV-) and 3 days (HIV+) for mCWIT errors. A minor fatigue effect was observed in the overall sample which was unassociated with group status. Moderate to strong correlations were found between mCWIT completion time and mCWIT errors with global neurocognition and with all of the individual neurocognitive domains. The strongest associations were with mCWIT completion time and laboratory-based global neurocognition, executive function, and working memory scores.

CONCLUSIONS

Cognitive testing administered within the context of a person's daily life provides qualitatively different data than neuropsychological testing completed in clinical settings, and it may constitute a more ecologically valid indicator of cognitive performance than traditional methods. Mobile cognitive testing has potential to help characterize real-time cognitive performance and serve as a complementary assessment tool to traditional methods.

Daily Activities Related to Mobile Cognitive Performance in Middle-Aged and Older Adults: An Ecological Momentary Cognitive Assessment Study

Background

Daily activities have been associated with neurocognitive performance. However, much of this research has used in-person neuropsychological testing that requires participants to travel to a laboratory or clinic, which may not always be feasible and does not allow for the examination of real-time relationships between cognition and behavior. Thus, there is a need to understand the real-time relationship between activities in the real world and neurocognitive functioning to improve tracking of symptoms or disease states and aid in the early identification of neurocognitive deficits among at-risk individuals.

Objective

We used a smartphone-based ecological momentary cognitive assessment (EMCA) platform to examine real-time relationships between daily activities and neurocognitive performance (executive functioning and verbal learning) in the everyday environment of middle-aged and older adults with and without HIV.

Methods

A total of 103 adults aged 50-74 years (67 persons with HIV; mean age 59 years, SD 6.4) were recruited from the University of California, San Diego HIV Neurobehavioral Research Program and the San Diego community. Participants completed our EMCA protocol for 14 days. Participants reported their current daily activities 4 times per day; following 2 of the 4 daily ecological momentary assessment (EMA) surveys, participants were administered the mobile Color-Word Interference Test (mCWIT) and mobile Verbal Learning Test (mVLT), each once per day. Activities were categorized into cognitively stimulating activities, passive leisure activities, and instrumental activities of daily living (IADLs). We used multilevel modeling to examine the same-survey and lagged within-person and between-person effects of each activity type on mobile cognitive performance.

Results

On average, participants completed 91% of the EMA surveys, 85% of the mCWIT trials, and 80% of the mVLT trials, and they reported engaging in cognitively stimulating activities on 17% of surveys, passive leisure activities on 33% of surveys, and IADLs on 20% of surveys. Adherence and activity percentages did not differ by HIV status. Within-persons, engagement in cognitively stimulating activities was associated with better mCWIT performance (β=−1.12; P=.007), whereas engagement in passive leisure activities was associated with worse mCWIT performance (β=.94; P=.005). There were no lagged associations. At the aggregate between-person level, a greater percentage of time spent in cognitively stimulating activities was associated with better mean mVLT performance (β=.07; P=.02), whereas a greater percentage of time spent in passive leisure activities was associated with worse mean mVLT performance (β=−.07; P=.01). IADLs were not associated with mCWIT or mVLT performance.

Conclusions

Smartphones present unique opportunities for assessing neurocognitive performance and behavior in middle-aged and older adults’ own environment. Measurement of cognition and daily functioning outside of clinical settings may generate novel insights on the dynamic association of daily behaviors and neurocognitive performance and may add new dimensions to understanding the complexity of human behavior.

Cognition in Context: Understanding the Everyday Predictors of Cognitive Performance in a New Era of Measurement

Background

Research suggests that variability in attention and working memory scores, as seen across time points, may be a sensitive indicator of impairment compared with a singular score at one point in time. Given that fluctuation in cognitive performance is a meaningful metric of real-world function and trajectory, it is valuable to understand the internal state-based and environmental factors that could be driving these fluctuations in performance.

Objective

In this viewpoint, we argue for the use of repeated mobile assessment as a way to better understand how context shapes moment-to-moment cognitive performance. To elucidate potential factors that give rise to intraindividual variability, we highlight existing literature that has linked both internal and external modifying variables to a number of cognitive domains. We identify ways in which these variables could be measured using mobile assessment to capture them in ecologically meaningful settings (ie, in daily life). Finally, we describe a number of studies that have already begun to use mobile assessment to measure changes in real time cognitive performance in people’s daily environments and the ways in which this burgeoning methodology may continue to advance the field.

Methods

This paper describes selected literature on contextual factors that examined how experimentally induced or self-reported contextual variables (ie, affect, motivation, time of day, environmental noise, physical activity, and social activity) related to tests of cognitive performance. We also selected papers that used mobile assessment of cognition; these papers were chosen for their use of high-frequency time-series measurement of cognition using a mobile device.

Results

Upon review of the relevant literature, it is evident that contextual factors have the potential to meaningfully impact cognitive performance when measured in laboratory and daily life environments. Although this research has shed light on the question of what gives rise to real-life variability in cognitive function (eg, affect and activity), many of the studies were limited by traditional methods of data collection (eg, involving retrospective recall). Furthermore, cognition has often been measured in one domain or in one age group, which does not allow us to extrapolate results to other cognitive domains and across the life span. On the basis of the literature reviewed, mobile assessment of cognition shows high levels of feasibility and validity and could be a useful method for capturing individual cognitive variability in real-world contexts via passive and active measures.

Conclusions

We propose that, through the use of mobile assessment, there is an opportunity to combine multiple sources of contextual and cognitive data. These data have the potential to provide individualized digital signatures that could improve diagnostic precision and lead to meaningful clinical outcomes in a wide range of psychiatric and neurological disorders.

Age-Associated Capacity to Progress When Playing Cognitive Mobile Games: Ecological Retrospective Observational Study

Background

The decline of cognitive function is an important issue related to aging. Over the last few years, numerous mobile apps have been developed to challenge the brain with cognitive exercises; however, little is currently known about how age influences capacity for performance improvement when playing cognitive mobile games.

Objective

The objective of this study was to analyze the score data of cognitive mobile games over a period of 100 gaming sessions to determine age-related learning ability for new cognitive tasks by measuring the level of score improvement achieved by participants of different ages.

Methods

Scores from 9000 individuals of different ages for 7 cognitive mobile games over 100 gaming sessions were analyzed. Scores from the first session were compared between age groups using one-way analysis of variance. Mixed models were subsequently used to investigate the progression of scores over 100 sessions.

Results

Statistically significant differences were found between age groups for the initial scores of 6 of the 7 games (linear trend, P<.001). Cognitive mobile game scores increased for all participants (P<.001) suggesting that all participants were able to improve their performance. The rate of improvement was, however, strongly influenced by the age of the participant with slower progression for older participants (P<.001).

Conclusions

This study provides evidence to support two interesting insights—cognitive mobile game scores appear to be sensitive to the changes in cognitive ability that occur with advancing age; therefore, these games could be a convenient way to monitor cognitive function over long-term follow-up, and users who train with the cognitive mobile games improve regardless of age.

Bridging the gap between performance-based assessment and self-reported everyday functioning: An ecological momentary assessment approach

Objective: In the real-world environment, multiple and interacting state-dependent factors (e.g., fatigue, distractions) can cause cognitive failures and negatively impact everyday activities. This study used ecological momentary assessment (EMA) and a n-back task to examine the relationship between fluctuating levels of cognition measured in the real-world environment and self-report and performance-based measures of functional status.Method: Thirty-five community-dwelling older adults (M age = 71.80) completed a brief battery of objective and self-report measures of cognitive and functional status. After completing 100, 45-second trials to reach stable performance on a n-back task, EMA data collection began. Four times daily for one week, participants received prompts on a tablet to complete a n-back task and a brief survey. From the EMA n-back trials, measures of EMA average performance and intra-individual variability (IIV) across performances were created.Results: For the EMA n-back, the correlation between IIV and EMA average was weak and non-significant. IIV associated with self-report measures, and EMA average with the objective, performance-based functional status composite. Hierarchical regressions further revealed that IIV was a significant predictor of self-reported functional status and cognitive failures over and above EMA average performance and global cognitive status. In contrast, for the objective, functional status composite, IIV did not explain additional variance.Conclusions: The findings suggest that IIV and self-report measures of functional status and cognitive failures may capture a real-world cognitive capacity that fluctuates over time and with context; one that may not easily be captured by objective, performance-based measures designed to assess optimal function.

Evaluating the Feasibility of Frequent Cognitive Assessment Using the Mezurio Smartphone App: Observational and Interview Study in Adults With Elevated Dementia Risk

Background

By enabling frequent, sensitive, and economic remote assessment, smartphones will facilitate the detection of early cognitive decline at scale. Previous studies have sustained participant engagement with remote cognitive assessment over a week; extending this to a period of 1 month clearly provides a greater opportunity for measurement. However, as study durations are increased, the need to understand how participant burden and scientific value might be optimally balanced also increases.

Objective

This study explored the little but often approach to assessment employed by the Mezurio app when prompting participants to interact every day for over a month. Specifically, this study aimed to understand whether this extended duration of remote study is feasible, and which factors promote sustained participant engagement over such periods.

Methods

A total of 35 adults (aged 40-59 years) with no diagnosis of cognitive impairment were prompted to interact with the Mezurio smartphone app platform for up to 36 days, completing short, daily episodic memory tasks in addition to optional executive function and language tests. A subset (n=20) of participants completed semistructured interviews focused on their experience of using the app.

Results

Participants complied with 80% of the daily learning tasks scheduled for subsequent tests of episodic memory, with 88% of participants still actively engaged by the final task. A thematic analysis of the participants’ experiences highlighted schedule flexibility, a clear user interface, and performance feedback as important considerations for engagement with remote digital assessment.

Conclusions

Despite the extended study duration, participants demonstrated high compliance with the schedule of daily learning tasks and were extremely positive about their experiences. Long durations of remote digital interaction are therefore definitely feasible but only when careful attention is paid to the design of the users’ experience.

Digital assessment of working memory and processing speed in everyday life: Feasibility, validation, and lessons-learned

OBJECTIVES

Cognitive functioning is often impaired in mental and neurological conditions and might fluctuate throughout the day. An existing experience-sampling tool was upgraded to assess individual's cognition in everyday life. The objectives were to test the feasibility and validity of two momentary cognition tasks.

METHODS

The momentary Visuospatial Working Memory Task (mVSWMT) and momentary Digit Symbol Substitution Task (mDSST) were add-ons to an experience sampling method (ESM) smartphone app. Healthy adults (n = 49) between 19 and 73 years of age performed the tasks within an ESM questionnaire 8 times a day, over 6 consecutive days. Feasibility was determined through completion rate and participant experience. Validity was assessed through contextualization of cognitive performance within intrapersonal and situational factors in everyday life.

FINDINGS

Participants experienced the tasks as pleasant, felt motivated, and the completion rate was high (71%). Social context, age, and distraction influenced cognitive performance in everyday life. The mVSWMT was too difficult as only 37% of recalls were correct and thus requires adjustments (i.e. fixed time between encoding and recall; more trials per moment). The mDSST speed outcome seems the most sensitive outcome measure to capture between- and within-person variance.

CONCLUSIONS

Short momentary cognition tasks for repeated assessment are feasible and hold promise, but more research is needed to improve validity and applicability in different samples. Recommendations for teams engaging in the field include matching task design with traditional neuropsychological tests and involving a multidisciplinary team as well as users. Special attention for individual needs can improve motivation and prevent frustration. Finally, tests should be attractive and competitive to stimulate engagement, but still reflect actual cognitive functioning.

Gallery Game: Smartphone-based assessment of long-term memory in adults at risk of Alzheimer's disease

Introduction: Gallery Game, deployed within the Mezurio smartphone app, targets the processes of episodic memory hypothesized to be first vulnerable to neurofibrillary tau-related degeneration in Alzheimer's Disease, prioritizing both perirhinal and entorhinal cortex/hippocampal demands.Methods: Thirty-five healthy adults (aged 40-59 years), biased toward those at elevated familial risk of dementia, completed daily Gallery Game tasks for a month. Assessments consisted of cross-modal paired-associate learning, with subsequent tests of recognition and free recall following delays ranging from one to 13 days.Results: Retention intervals of at least three days were needed to evidence significant forgetting at both recognition and paired-associate recall test. The association between Gallery Game outcomes and established in-clinic memory assessments were small but numerically in the anticipated direction. In addition, there was preliminary support for utilizing the perirhinal-dependent pattern of semantic false alarms during object recognition as a marker of early impairment.Conclusions: These results support the need for tests of longer-term memory to sensitively record behavioral differences in adults with no diagnosis of cognitive impairment. Aggregate behavioral outcomes promote Gallery Game's utility as a digital assessment of episodic memory, aligning with established theoretical models of object memory and showing small yet uniform associations with existing in-clinic tests. Initial support for the discriminatory value of perirhinal-targeted outcomes justifies ongoing large-sample validation against traditional biomarkers of Alzheimer's disease.

Measuring within-day cognitive performance using the experience sampling method: A pilot study in a healthy population

Introduction

People with depression, anxiety, or psychosis often complain of confusion, problems concentrating or difficulties cognitively appraising contextual cues. The same applies to people with neurodegenerative diseases or brain damage such as dementia or stroke. Assessments of those cognitive difficulties often occurs in cross-sectional and controlled clinical settings. Information on daily moment-to-moment cognitive fluctuations and its relation to affect and context is lacking. The development and evaluation of a digital cognition task is presented. It enables the fine-grained mapping of cognition and its relation to mood, intrapersonal factors and context.

Methods

The momentary Digit Symbol Substitution Task is a modified digital version of the original paper-and-pencil task, with a duration of 30 seconds and implemented in an experience sampling protocol (8 semi-random assessments a day on 6 consecutive days). It was tested in the healthy population (N = 40). Descriptive statistics and multilevel regression analyses were used to determine initial feasibility and assess cognitive patterns in everyday life. Cognition outcome measures were the number of trials within the 30-second sessions and the percentage of correct trials.

Results

Subjects reported the task to be easy, pleasant and do-able. On average, participants completed 11 trials with 97% accuracy per 30-second session. Cognitive variation was related to mood, with an interaction between positive and negative affect for accuracy (% correct) (p = .001) and an association between positive affect and speed (number of trials) (p = .01). Specifically, cheerful, irritated and anxious seem to covary with cognition. Distraction and location are relevant contextual factors. The number of trials showed a learning effect (p < .001) and was sensitive to age (p < .001).

Conclusion

Implementing a digital cognition task within an experience-sampling paradigm shows promise. Fine-tuning in further research and in clinical samples is needed. Gaining insight into cognitive functioning could help patients navigate and adjust the demands of daily life.

Remote Assessment of Cognitive Impairment Level Based on Serious Mobile Game Performance: An Initial Proof of Concept

Individuals with cognitive impairments are evaluated using clinically validated cognitive assessment tools, which need to be administered by trained therapists. This serves as a major barrier for frequent and longitudinal monitoring of patients' cognitive impairment level. We introduce Neuro-World, a set of six mobile games designed to challenge visuospatial short-term memory and selective attention, which allows one to self-administer the assessment of his/her cognitive impairment level. Game performance is analyzed to estimate a widely accepted clinical measure, the mini mental state examination (MMSE), which highlights the translational impact of the system in real-world settings. We collected game-specific performance data from 12 post-stroke patients at baseline and a three-month follow-up, which were used to train supervised machine learning models to estimate the corresponding MMSE scores. The results presented herein show that the proposed approach can estimate the MMSE scores with a normalized root mean square error of 5.75%. We also validate the system's responsiveness to longitudinal changes in cognitive impairment level and demonstrate the system's positive usability in cognitively impaired individuals and their willingness to adhere to the longitudinal use. This study demonstrates that Neuro-world has great potential to be used to evaluate the cognitive impairment level and monitor its long-term change. This study enables new clinical and research opportunities for accurate, longitudinal assessment of cognitive function via mobile games.

Promising technological innovations in cognitive training to treat eating-related behavior

One potential reason for the suboptimal outcomes of treatments targeting appetitive behavior, such as eating and alcohol consumption, is that they do not target the implicit cognitive processes that may be driving these behaviors. Two groups of related neurocognitive processes that are robustly associated with dysregulated eating and drinking are attention bias (AB; selective attention to specific stimuli) and executive function (EF; a set of cognitive control processes such as inhibitory control, working memory, set shifting, that govern goal-directed behaviors). An increasing body of work suggests that EF and AB training programs improve regulation of appetitive behaviors, especially if trainings are frequent and sustained. However, several key challenges, such as adherence to the trainings in the long term, and overall potency of the training, remain. The current manuscript describes five technological innovations that have the potential to address difficulties related to the effectiveness and feasibility of EF and AB trainings: (1) deployment of training in the home, (2) training via smartphone, (3) gamification, (4) virtual reality, and (5) personalization. The drawbacks of these innovations, as well as areas for future research, are also discussed. The above-mentioned innovations are likely to be instrumental in the future empirical work to develop and evaluate effective EF and AB trainings for appetitive behaviors.

Applications for self-administered mobile cognitive assessments in clinical research: A systematic review

Frequent, brief and repeated self-administered mobile assessments of cognitive function, conducted in everyday life settings, are a promising complementary tool to traditional assessment approaches. Mobile cognitive assessments promote patient-centered care and might enhance capacity to inform individual-level outcomes over time (i.e. detecting subtle declines in cognitive function), as well as in assessing cognition and its correlates in the naturalistic environment. The goal of this systematic review was to assess the feasibility and psychometric properties of mobile cognitive assessments. Through a comprehensive search, we identified 12 articles using self-administered, mobile phone-based cognitive assessments. Studies sampled participants between 1 and 6 times per day for 1-14 days. Samples ranged in age from 14 to 83 years old and were generally healthy populations without cognitive impairment. Working memory was the most frequently-assessed cognitive domain (n = 7), followed by attention/reaction time (n = 4). Seven studies reported adherence, with mean adherence rates of 79.2%. In addition to positive evidence of feasibility, there was general support for high levels of between- and within-person reliability and construct validity. While research has only begun to explore the utility of mobile cognitive assessments, studies to-date indicate they may be a promising complementary tool to traditional assessment methods with potential to improve clinical care and research.

Feasibility and validity of mobile cognitive testing in the investigation of age-related cognitive decline

Mobile cognitive testing may be used to help characterize subtle deficits at the earliest stages of cognitive decline. Despite growing interest in this approach, comprehensive information concerning its feasibility and validity has been lacking in elderly samples. Over a one-week period, this study applied mobile cognitive tests of semantic memory, episodic memory and executive functioning in a cohort of 114 elderly non-demented community residents. While the study acceptance rate was moderate (66%), the majority of recruited individuals met minimal compliance thresholds and responded to an average of 82% of the repeated daily assessments. Missing data did not increase over the course of the study, but practice effects were observed for several test scores. However, even when controlling for practice effects, traditional neuropsychological tests were significantly associated with mobile cognitive test scores. In particular, the Isaacs Set Test was associated with mobile assessments of semantic memory (γ = 0.084, t = 5.598, p < 0.001), the Grober and Buschke with mobile assessments of episodic memory (γ = 0.069, t = 3.156, p < 0.01, and the Weschler symbol coding with mobile assessments of executive functioning (γ = 0.168, t = 4.562, p < 0.001). Mobile cognitive testing in the elderly may provide complementary and potentially more sensitive data relative to traditional neuropsychological assessment. Copyright © 2016 John Wiley & Sons, Ltd.