Paradigm Shift Toward Digital Neuropsychology and High-Dimensional Neuropsychological Assessments: Review

Using machine learning to detect noncredible cognitive test performance

Objective: Advanced algorithmic methods may improve the assessment of performance validity during neuropsychological testing. This study investigated whether unsupervised machine learning (ML) could serve as one such method. Method: Participants were 359 adult outpatients who underwent a neuropsychological evaluation for various referral reasons. Data relating to participants' performance validity test scores, medical and psychiatric history, referral reason, litigation status, and disability status were examined in an unsupervised ML model. The model was programmed to synthesize the data into an unspecified number of clusters, which were then compared to predetermined ratings of whether patients had valid or invalid test performance. Ratings were established according to multiple empirical performance validity test scores. To further understand the model, we examined which data were most helpful in its clustering decision-making process. Results: Similar to the clinical determination of patients' performance on neuropsychological testing, the model identified a two-cluster profile consisting of valid and invalid data. The model demonstrated excellent predictive accuracy (area under the curve of .92 [95% CI .88, .97]) when referenced against participants' predetermined validity status. Performance validity test scores were the most influential in the differentiation of clusters, but medical history, referral reason, and disability status were also contributory. Conclusions: These findings serve as a proof of concept that unsupervised ML can accurately assess performance validity using various data obtained during a neuropsychological evaluation. The manner in which unsupervised ML evaluates such data may circumvent some of the limitations with traditional validity assessment approaches. Importantly, unsupervised ML is adaptable to emerging digital technologies within neuropsychology that can be used to further improve the assessment of performance validity.

Remote Digital Technologies for the Early Detection and Monitoring of Cognitive Decline in Patients With Type 2 Diabetes: Insights From Studies of Neurodegenerative Diseases

Type 2 diabetes (T2D) is a risk factor for cognitive decline. In neurodegenerative disease research, remote digital cognitive assessments and unobtrusive sensors are gaining traction for their potential to improve early detection and monitoring of cognitive impairment. Given the high prevalence of cognitive impairments in T2D, these digital tools are highly relevant. Further research incorporating remote digital biomarkers of cognition, behavior, and motor functioning may enable comprehensive characterizations of patients with T2D and may ultimately improve clinical care and equitable access to research participation. The aim of this commentary article is to review the feasibility, validity, and limitations of using remote digital cognitive tests and unobtrusive detection methods to identify and monitor cognitive decline in neurodegenerative conditions and apply these insights to patients with T2D.

The future of neuropsychology is digital, theory-driven, and Bayesian: a paradigmatic study of cognitive flexibility

Introduction

This study explores the transformative potential of digital, theory-driven, and Bayesian paradigms in neuropsychology by combining digital technologies, a commitment to evaluating theoretical frameworks, and Bayesian statistics. The study also examines theories of executive function and cognitive flexibility in a large sample of neurotypical individuals (N = 489).

Methods

We developed an internet-based Wisconsin Card-Sorting Task (iWCST) optimized for online assessment of perseveration errors (PE). Predictions of the percentage of PE, PE (%), in non-repetitive versus repetitive situations were derived from the established supervisory attention system (SAS) theory, non-repetitive PE (%) < repetitive PE (%), and the novel goal-directed instrumental control (GIC) theory, non-repetitive PE (%) > repetitive PE (%).

Results

Bayesian t-tests revealed the presence of a robust error suppression effect (ESE) indicating that PE are less likely in repetitive situations than in non-repetitive situations, contradicting SAS theory with posterior model probability p < 0.001 and confirming GIC theory with posterior model probability p > 0.999. We conclude that repetitive situations support cognitive set switching in the iWCST by facilitating the retrieval of goal-directed, instrumental memory that associates stimulus features, actions, and outcomes, thereby generating the ESE in neurotypical individuals. We also report exploratory data analyses, including a Bayesian network analysis of relationships between iWCST measures.

Discussion

Overall, this study serves as a paradigmatic model for combining digital technologies, theory-driven research, and Bayesian statistics in neuropsychology. It also provides insight into how this integrative, innovative approach can advance the understanding of executive function and cognitive flexibility and inform future research and clinical applications.

From Practice to Public Health: Broadening Neuropsychology's Reach & Value-An Introduction to the National Academy of Neuropsychology's 2022 Annual Conference Special Issue

This special issue is centered around presentations from the National Academy of Neuropsychology 2022 Annual Conference. The theme of the conference, "From Practice to Public Health: Broadening Neuropsychology's Reach & Value" is pivotal for the field's future. With an ever-shifting technological landscape and recent changes in clinical practice post-COVID, we are left wondering how neuropsychology will develop. How will we use biomedical and technological advances, such as blood-based Alzheimer's disease biomarkers or passive digital recordings, to improve clinical care and further expand our understanding of disease mechanisms? As neuropsychologists, how can we use our expertise to empirically inform public health policy? The diagnosis and treatment of post-acute sequelae of COVID-19, the identification and characterization of post-pandemic educational setbacks, and the adaptation of new technological and diagnostic advances into clinical practice workflows represent a vital set of new challenges and opportunities poised to disrupt traditional modes of practice. The articles in this special issue convey the role of neuropsychology in addressing these emerging issues and illustrate how and why neuropsychology is well positioned to be at the forefront of clinical practice and scientific advancements.

Overlapping and distinct phenotypic profiles in Alzheimer's disease and late onset epilepsy: a biologically-based approach

Due to shared hippocampal dysfunction, patients with Alzheimer's dementia and late-onset epilepsy (LOE) report memory decline. Multiple studies have described the epidemiological, pathological, neurophysiological, and behavioral overlap between Alzheimer's Disease and LOE, implying a bi-directional relationship. We describe the neurobiological decline occurring at different spatial in AD and LOE patients, which may explain why their phenotypes overlap and differ. We provide suggestions for clinical recognition of dual presentation and novel approaches for behavioral testing that reflect an "inside-out," or biologically-based approach to testing memory. New memory and language assessments could detect-and treat-memory impairment in AD and LOE at an earlier, actionable stage.

The Reliability and Clinical Validation of Automatically-Derived Verbal Memory Features of the Verbal Learning Test in Early Diagnostics of Cognitive Impairment

BACKGROUND

Previous research has shown that verbal memory accurately measures cognitive decline in the early phases of neurocognitive impairment. Automatic speech recognition from the verbal learning task (VLT) can potentially be used to differentiate between people with and without cognitive impairment.

OBJECTIVE

Investigate whether automatic speech recognition (ASR) of the VLT is reliable and able to differentiate between subjective cognitive decline (SCD) and mild cognitive impairment (MCI).

METHODS

The VLT was recorded and processed via a mobile application. Following, verbal memory features were automatically extracted. The diagnostic performance of the automatically derived features was investigated by training machine learning classifiers to distinguish between participants with SCD versus MCI/dementia.

RESULTS

The ICC for inter-rater reliability between the clinical and automatically derived features was 0.87 for the total immediate recall and 0.94 for the delayed recall. The full model including the total immediate recall, delayed recall, recognition count, and the novel verbal memory features had an AUC of 0.79 for distinguishing between participants with SCD versus MCI/dementia. The ten best differentiating VLT features correlated low to moderate with other cognitive tests such as logical memory tasks, semantic verbal fluency, and executive functioning.

CONCLUSIONS

The VLT with automatically derived verbal memory features showed in general high agreement with the clinical scoring and distinguished well between SCD and MCI/dementia participants. This might be of added value in screening for cognitive impairment.

Digital Health Technologies for Alzheimer's Disease and Related Dementias: Initial Results from a Landscape Analysis and Community Collaborative Effort

Digital health technologies offer valuable advantages to dementia researchers and clinicians as screening tools, diagnostic aids, and monitoring instruments. To support the use and advancement of these resources, a comprehensive overview of the current technological landscape is essential. A multi-stakeholder working group, convened by the Digital Medicine Society (DiMe), conducted a landscape review to identify digital health technologies for Alzheimer's disease and related dementia populations. We searched studies indexed in PubMed, Embase, and APA PsycInfo to identify manuscripts published between May 2003 to May 2023 reporting analytical validation, clinical validation, or usability/feasibility results for relevant digital health technologies. Additional technologies were identified through community outreach. We collated peer-reviewed manuscripts, poster presentations, or regulatory documents for 106 different technologies for Alzheimer's disease and related dementia assessment covering diverse populations such as Lewy Body, vascular dementias, frontotemporal dementias, and all severities of Alzheimer's disease. Wearable sensors represent 32% of included technologies, non-wearables 61%, and technologies with components of both account for the remaining 7%. Neurocognition is the most prevalent concept of interest, followed by physical activity and sleep. Clinical validation is reported in 69% of evidence, analytical validation in 34%, and usability/feasibility in 20% (not mutually exclusive). These findings provide clinicians and researchers a landscape overview describing the range of technologies for assessing Alzheimer's disease and related dementias. A living library of technologies is presented for the clinical and research communities which will keep findings up-to-date as the field develops.

A review of the reliability of remote neuropsychological assessment

The provision of clinical neuropsychological services has predominately been undertaken by way of standardized administration in a face-to-face setting. Interpretation of psychometric findings in this context is dependent on the use of normative comparison. When the standardization in which such psychometric measures are employed deviates from how they were employed in the context of the development of its associated norms, one is left to question the reliability and hence, validity of any such findings and in turn, diagnostic decision making. In light of the current COVID-19 pandemic and resultant social distancing direction, face-to-face neuropsychological assessment has been challenging to undertake. As such, remote (i.e., virtual) neuropsychological assessment has become an obvious solution. Here, and before the results from remote neuropsychological assessment can be said to stand on firm scientific grounds, it is paramount to ensure that results garnered remotely are reliable and valid. To this end, we undertook a review of the literature and present an overview of the landscape. To date, the literature shows evidence for the reliability of remote administration and the clinical implications are paramount. When and where needed, neuropsychologists, psychometric technicians and examinees may no longer need to be in the same physical space to undergo an assessment. These findings are most relevant given the physical distancing practices because of COVID-19. And whilst remote assessment should never supplant face-to-face neuropsychological assessments, it does serve as a valid alternative when necessary.

Neural network process simulations support a distributed memory system and aid design of a novel computer adaptive digital memory test for preclinical and prodromal Alzheimer's disease

OBJECTIVE

Growing evidence supports the importance of learning as a central deficit in preclinical/prodromal Alzheimer's disease. The aims of this study were to conduct a series of neural network simulations to develop a functional understanding of a distributed, nonmodular memory system that can learn efficiently without interference. This understanding is applied to the development of a novel digital memory test.

METHOD

Simulations using traditional feed forward neural network architectures to learn simple logic problems are presented. The simulations demonstrate three limitations: (a) inefficiency, (b) an inability to learn problems consistently, and (c) catastrophic interference when given multiple problems. A new mirrored cascaded architecture is introduced to address these limitations, with support provided by a series of simulations.

RESULTS

The mirrored cascaded architecture demonstrates efficient and consistent learning relative to feed forward networks but also suffers from catastrophic interference. Addition of context values to add the capability of distinguishing features as part of learning eliminates the problem of interference in the mirrored cascaded, but not the feed forward, architectures.

CONCLUSIONS

A mirrored cascaded architecture addresses the limitations of traditional feed forward neural networks, provides support for a distributed memory system, and emphasizes the importance of context to avoid interference. These process models contributed to the design of a digital computer-adaptive word list learning test that places maximum stress on the capability to distinguish specific episodes of learning. Process simulations provide a useful method of testing models of brain function and contribute to new approaches to neuropsychological assessment. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Ecological Momentary Assessment of Cognition in Clinical and Community Samples: Reliability and Validity Study

BACKGROUND

The current methods of evaluating cognitive functioning typically rely on a single time point to assess and characterize an individual's performance. However, cognitive functioning fluctuates within individuals over time in relation to environmental, psychological, and physiological contexts. This limits the generalizability and diagnostic utility of single time point assessments, particularly among individuals who may exhibit large variations in cognition depending on physiological or psychological context (eg, those with type 1 diabetes [T1D], who may have fluctuating glucose concentrations throughout the day).

OBJECTIVE

We aimed to report the reliability and validity of cognitive ecological momentary assessment (EMA) as a method for understanding between-person differences and capturing within-person variation in cognition over time in a community sample and sample of adults with T1D.

METHODS

Cognitive performance was measured 3 times a day for 15 days in the sample of adults with T1D (n=198, recruited through endocrinology clinics) and for 10 days in the community sample (n=128, recruited from TestMyBrain, a web-based citizen science platform) using ultrabrief cognitive tests developed for cognitive EMA. Our cognitive EMA platform allowed for remote, automated assessment in participants' natural environments, enabling the measurement of within-person cognitive variation without the burden of repeated laboratory or clinic visits. This allowed us to evaluate reliability and validity in samples that differed in their expected degree of cognitive variability as well as the method of recruitment.

RESULTS

The results demonstrate excellent between-person reliability (ranging from 0.95 to 0.99) and construct validity of cognitive EMA in both the sample of adults with T1D and community sample. Within-person reliability in both samples (ranging from 0.20 to 0.80) was comparable with that observed in previous studies in healthy older adults. As expected, the full-length baseline and EMA versions of TestMyBrain tests correlated highly with one another and loaded together on the expected cognitive domains when using exploratory factor analysis. Interruptions had higher negative impacts on accuracy-based outcomes (β=-.34 to -.26; all P values <.001) than on reaction time-based outcomes (β=-.07 to -.02; P<.001 to P=.40).

CONCLUSIONS

We demonstrated that ultrabrief mobile assessments are both reliable and valid across 2 very different clinic versus community samples, despite the conditions in which cognitive EMAs are administered, which are often associated with more noise and variability. The psychometric characteristics described here should be leveraged appropriately depending on the goals of the cognitive assessment (eg, diagnostic vs everyday functioning) and the population being studied.

Innovations in Neuropsychology: Future Applications in Neurosurgical Patient Care

Over the last century, collaboration between clinical neuropsychologists and neurosurgeons has advanced the state of the science in both disciplines. These advances have provided the field of neuropsychology with many opportunities for innovation in the care of patients prior to, during, and following neurosurgical intervention. Beyond giving a general overview of how present-day advances in technology are being applied in the practice of neuropsychology within a neurological surgery department, this article outlines new developments that are currently unfolding. Improvements in remote platform, computer interface, "real-time" analytics, mobile devices, and immersive virtual reality have the capacity to increase the customization, precision, and accessibility of neuropsychological services. In doing so, such innovations have the potential to improve outcomes and ameliorate health care disparities.

Associations between smartphone keystroke dynamics and cognition in MS

Objective

Examine the associations between smartphone keystroke dynamics and cognitive functioning among persons with multiple sclerosis (MS).

Methods

Sixteen persons with MS with no self-reported upper extremity or typing difficulties and 10 healthy controls (HCs) completed six weeks of remote monitoring of their keystroke dynamics (i.e., how they typed on their smartphone keyboards). They also completed a comprehensive neuropsychological assessment and symptom ratings about fatigue, depression, and anxiety at baseline.

Results

A total of 1,335,787 keystrokes were collected, which were part of 30,968 typing sessions. The MS group typed slower (P < .001) and more variably (P = .032) than the HC group. Faster typing speed was associated with better performance on measures of processing speed (P = .016), attention (P = .022), and executive functioning (cognitive flexibility: P = .029; behavioral inhibition: P = .002; verbal fluency: P = .039), as well as less severe impact from fatigue (P < .001) and less severe anxiety symptoms (P = .007). Those with better cognitive functioning and less severe symptoms showed a stronger correlation between the use of backspace and autocorrection events (P < .001).

Conclusion

Typing speed may be sensitive to cognitive functions subserved by the frontal-subcortical brain circuits. Individuals with better cognitive functioning and less severe symptoms may be better at monitoring their typing errors. Keystroke dynamics have the potential to be used as an unobtrusive remote monitoring method for real-life cognitive functioning among persons with MS, which may improve the detection of relapses, evaluate treatment efficacy, and track disability progression.

What makes for a competitive fellowship candidate? A survey of clinical neuropsychology postdoctoral training directors

To obtain objective data about the factors that clinical neuropsychology postdoctoral training directors (TDs) look for and prioritize in their review and selection of fellowship candidates.

We identified 167 TDs who were overseeing postdoctoral training programs that provided training consistent with the Houston Conference Guidelines. We invited all TDs to complete an anonymous online survey that assessed their expectations as they relate to the selection of fellowship candidates. Eighty-eight TDs completed the survey in full. We used descriptive statistics to analyze the data and investigate potential between-group differences in TDs' responses across patient populations, training settings, and APPCN member program status.

TDs ranked the intensity of candidates' neuropsychology education and training experiences, their fellowship interviews, and letters of recommendation as most important. Increasing the representation of under-represented minorities and other factors were ranked lower. Minimum benchmarks related to candidates' scholarly productivity, dissertation progress, and the time they spent engaged in clinical neuropsychology activities during internship were revealed. There were relatively few differences in TDs' responses when compared across patient populations, training settings, or APPCN member program status.

Students may increase their competitiveness for clinical neuropsychology fellowships by obtaining intensive education and training experiences in the specialty, which includes clinical training and coursework, and by producing scholarly work. Students may also benefit from improving their interviewing skills, completing an internship with at least 40% of time spent in neuropsychological activities, and at minimum by having their dissertation data collected before their fellowship interviews.

An Immersive and Interactive Platform for Cognitive Assessment and Rehabilitation (bWell): Design and Iterative Development Process

BACKGROUND

Immersive technologies like virtual reality can enable clinical care that meaningfully aligns with real-world deficits in cognitive functioning. However, options in immersive 3D environments are limited, partly because of the unique challenges presented by the development of a clinical care platform. These challenges include selecting clinically relevant features, enabling tasks that capture the full breadth of deficits, ensuring longevity in a rapidly changing technology landscape, and performing the extensive technical and clinical validation required for digital interventions. Complicating development, is the need to integrate recommendations from domain experts at all stages.

OBJECTIVE

The Cognitive Health Technologies team at the National Research Council Canada aims to overcome these challenges with an iterative process for the development of bWell, a cognitive care platform providing multisensory cognitive tasks for adoption by treatment providers.

METHODS

The team harnessed the affordances of immersive technologies while taking an interdisciplinary research and developmental approach, obtaining active input from domain experts with iterative deliveries of the platform. The process made use of technology readiness levels, agile software development, and human-centered design to advance four main activities: identification of basic requirements and key differentiators, prototype design and foundational research to implement components, testing and validation in lab settings, and recruitment of external clinical partners.

RESULTS

bWell was implemented according to the findings from the design process. The main features of bWell include multimodal (fully, semi, or nonimmersive) and multiplatform (extended reality, mobile, and PC) implementation, configurable exercises that pair standardized assessment with adaptive and gamified variants for therapy, a therapist-facing user interface for task administration and dosing, and automated activity data logging. bWell has been designed to serve as a broadly applicable toolkit, targeting general aspects of cognition that are commonly impacted across many disorders, rather than focusing on 1 disorder or a specific cognitive domain. It comprises 8 exercises targeting different domains: states of attention (Egg), visual working memory (Theater), relaxation (Tent), inhibition and cognitive control (Mole), multitasking (Lab), self-regulation (Butterfly), sustained attention (Stroll), and visual search (Cloud). The prototype was tested and validated with healthy adults in a laboratory environment. In addition, a cognitive care network (5 sites across Canada and 1 in Japan) was established, enabling access to domain expertise and providing iterative input throughout the development process.

CONCLUSIONS

Implementing an interdisciplinary and iterative approach considering technology maturity brought important considerations for the development of bWell. Altogether, this harnesses the affordances of immersive technology and design for a broad range of applications, and for use in both cognitive assessment and rehabilitation. The technology has attained a maturity level of prototype implementation with preliminary validation carried out in laboratory settings, with next steps to perform the validation required for its eventual adoption as a clinical tool.