Digitally translated Self-Administered Gerocognitive Examination (eSAGE): relationship with its validated paper version, neuropsychological evaluations, and clinical assessments

BACKGROUND

The original paper Self-Administered Gerocognitive Examination (SAGE) is a valid and reliable cognitive assessment tool used to identify individuals with mild cognitive impairment (MCI) or early dementia. We evaluated identical test questions in a digital format (eSAGE) made for tablet use with the goals of calibrating it against SAGE and establishing its association with other neuropsychological tests and clinical assessments of cognitive impairment.

METHODS

Subjects aged 50 and over who had taken SAGE were recruited from community and clinic settings. Subjects were randomly selected to participate in a clinical evaluation including neuropsychological evaluations. SAGE and eSAGE were administered using a crossover design. Subjects were identified as dementia, MCI, or normal based on standard clinical criteria. Associations were investigated using Spearman correlations, linear regression, and sensitivity and specificity measures.

RESULTS

Of the 426 subjects screened, 66 completed the evaluation. eSAGE score correlation to a battery of neuropsychological tests was 0.73 (p < 0.0001) with no significant difference between the paper and digital format. Spearman correlation of SAGE versus eSAGE was 0.88 (p < 0.0001), and they are related by the formula: eSAGE score = -1.05 + 0.99 × SAGE score. Since the slope is very close to 1 (p = 0.86) there is strong evidence that the scaling is identical between eSAGE and SAGE, with no scale bias. Overall, eSAGE scores are lower by an average of 1.21 and the decrease is statistically significant (p < 0.0001). For those subjects familiar with smartphones or tablets (one measure of digital proficiency), eSAGE scores are lower by an average of 0.83 points (p = 0.029). With a score 16 and higher being classified as normal, eSAGE had 90% specificity and 71% sensitivity in detecting those with cognitive impairment from normal subjects.

CONCLUSIONS

Tablet-based eSAGE shows a strong association with the validated paper SAGE and a neuropsychological battery. It shows no scale bias compared to SAGE. Both have the advantage of self-administration, brevity, four interchangeable forms, and high sensitivity and specificity in detecting cognitive impairment from normal subjects. Their potential widespread availability will be a major factor in overcoming the many obstacles in identifying early cognitive changes.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02544074 . Registered on 18 March 2015.

Remote Ecological Momentary Testing of Learning and Memory in Adults With Serious Mental Illness

Smartphone-based ecological mobile cognitive tests (EMCTs) can measure cognitive abilities in the real world, complementing traditional neuropsychological assessments. We evaluated the validity of an EMCT of recognition memory designed for use with people with serious mental illness, as well as relevant contextual influences on performance. Participants with schizophrenia (SZ), schizoaffective disorder, and bipolar disorder (BD) completed in-lab assessments of memory (Hopkins Verbal Learning Test, HVLT), other cognitive abilities, functional capacity, and symptoms, followed by 30 days of EMCTs during which they completed our Mobile Variable Difficulty List Memory Test (VLMT) once every other day (3 trials per session). List length on the VLMT altered between 6, 12, and 18 items. On average, participants completed 75.3% of EMCTs. Overall performance on VLMT 12 and 18 items was positively correlated with HVLT (ρ = 0.52, P < .001). People with BD performed better on the VLMT than people with SZ. Intraindividual variability on the VLMT was more specifically associated with HVLT than nonmemory tests and not associated with symptoms. Performance during experienced distraction, low effort, and out of the home location was reduced yet still correlated with the in-lab HVLT. The VLMT converged with in-lab memory assessment, demonstrating variability within person and by different contexts. Ambulatory cognitive testing on participants' personal mobile devices offers more a cost-effective and "ecologically valid" measurement of real-world cognitive performance.

Age effects in autobiographical memory depend on the measure

Studies examining age effects in autobiographical memory have produced inconsistent results. This study examined whether a set of typical autobiographical memory measures produced equivalent results in a single participant sample. Five memory tests (everyday memory, autobiographical memory from the past year, autobiographical memory from age 11-17, word-cued autobiographical memory, and word-list recall) were administered in a single sample of young and older adults. There was significant variance in the tests' sensitivity to age: word-cued autobiographical memory produced the largest deficit in older adults, similar in magnitude to word-list recall. In contrast, older adults performed comparatively well on the other measures. The pattern of findings was broadly consistent with the results of previous investigations, suggesting that (1) the results of the different AM tasks are reliable, and (2) variable age effects in the autobiographical memory literature are at least partly due to the use of different tasks, which cannot be considered interchangeable measures of autobiographical memory ability. The results are also consistent with recent work dissociating measures of specificity and detail in autobiographical memory, and suggest that specificity is particularly sensitive to ageing. In contrast, detail is less sensitive to ageing, but is influenced by retention interval and event type. The extent to which retention interval and event type interact with age remains unclear; further research using specially designed autobiographical memory tasks could resolve this issue.

Validation of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) in a Colombian Population

BACKGROUND

Cognitive impairment has been recognized as an important factor in multiple sclerosis (MS) in the past few years. One brief, reliable and valid tool to assess cognition in MS is the BICAMS, which encompasses the Symbol Digit Modalities Test (SDMT), the California Verbal Learning Test (CVLT II), and the Brief Visuospatial Memory Test - Revised (BVMT-R). Continuing with the international initiative to validate the BICAMS in different countries, here we present the results obtained from the efforts in validating such test in the Colombian population.

METHOD

100 healthy controls and 50 MS patients participated in the study, group matched for age, education and gender. Subjects completed all three tests of the BICAMS. Instead of the CVLT-II, the Colombian validated form PAMCL (Prueba de Aprendizaje y Memoria con Codificación Libre) was used. Test-retest measures were obtained for 16 patients in order to test for reliability.

RESULTS

Evidence of criterion validity was obtained, MS group performing significantly worse than HC group in all three tests (SDMT: p= .001, d= 0.59; PAMCL: p= .03, d= 0.38; BVMT-R: p<.001, d= 0.58). Test-retest was also obtained, finding significant correlations for all three tests (SDMT: r=0.932, p<.00; BVMT-R: r=0.863, p<.001; PAMCL: r=0.889, p<.001). Standardization of raw scores to uncontrolled scaled scores was done and these scores were then adjusted for age and years of schooling using a multiple linear regression.

CONCLUSIONS

The BICAMS proved to be a valid and sensitive tool to screen for cognitive impairment in MS patients.

Utility of abbreviated versions of the test of memory malingering in children with traumatic brain injury

We evaluated the classification accuracy of two abbreviated versions of the Test of Memory Malingering (TOMM) in prediction of results on the full-length instrument in a sample of 126 children who were evaluated within 1-12 months after traumatic brain injury. Both a version based on administration of Trial 1 and a version based on administration of only the first 10 items of Trial 1 had acceptable specificity (i.e., > .90) and sensitivity (i.e., > .60) with regard to prediction of pass/fail results on the complete TOMM. Failure on the TOMM suppressed performance on measures of processing speed that were otherwise sensitive to severity of traumatic brain injury. We conclude that these abbreviated versions of the TOMM can be used in clinical practice with children with traumatic brain injury, at the discretion of the neuropsychologist.

Detecting memory performance validity with DETECTS: A computerized performance validity test

Evaluating performance validity is essential in neuropsychological and forensic assessments. Nonetheless, most psychological assessment tests are unable to detect performance validity and other methods must be used for this purpose. A new Performance Validity Test (DETECTS - Memory Performance Validity Test) was developed with several characteristics that enhance test utility. Moreover, precise response time measurement was added to DETECTS. Two groups of participants (normative and simulator group) completed DETECTS and three memory tests from the Wechsler Memory Scale III. Simulators achieved considerably lower scores (hits) and higher response times in DETECTS compared with the normative group. All participants in the normative group were classified correctly and no simulator was classified as having legitimate memory deficits. Thus, DETECTS seems to be a valuable computerized Performance Validity Test with reduced application time and effective cut-off scores as well as high sensitivity, specificity, and positive and negative predictive power values. Lastly, response time may be a very useful measure for detecting memory malingering.

Utility of learning ratio scores from the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) Word List Memory Test in distinguishing patterns of cognitive decline in veterans referred for neuropsychological evaluation

Background: The Learning Ratio (LR) is a novel learning score that has shown improved utility over other learning metrics in detecting Alzheimer's disease (AD) across multiple memory tasks. However, its utility on the Consortium to Establish a Registry for Alzheimer's Disease Word List Memory Test (CERAD WLMT), a widely used list learning measure sensitive to decline in neurodegenerative disease, is unknown. The goal of the current study was to determine the utility of LR on the CERAD WLMT in differentiating between diagnostic (MiNCD vs MaNCD) and etiologic groups (VaD vs AD) in a veteran sample. Methods: Raw learning slope (RLS) and LR scores were examined in 168 veterans diagnosed with major neurocognitive disorder (MaNCD), mild neurocognitive disorder (MiNCD), or normal aging following neuropsychological evaluation. Patients with MaNCD were further classified by suspected etiology (i.e. microvascular disease vs AD). Results: Whereas RLS scores were not significantly different between MiNCD and MaNCD, LR scores were significantly different between all diagnostic groups (p's < .05). Those with AD had lower LR scores and RLS scores compared to those with VaD (p's < .05). LR classification accuracy was acceptable for MiNCD (AUC = .76), excellent for MaNCD (AUC = .86) and VaD (AUC = .81), and outstanding for AD (AUC = .91). Optimal cutoff scores for WLMT LR were derived from Youden's index. Conclusion: Results support the use of LR scores over RLS when interpreting the CERAD WLMT and highlight the clinical utility of LR in differentiating between diagnostic groups and identifying suspected etiology.

The Grenada Learning and Memory Scale: Psychometric features and normative data in Caribbean preschool children

OBJECTIVE

Neuropsychological assessment of preschool children is essential for early detection of delays and referral for intervention prior to school entry. This is especially pertinent in low- and middle-income countries (LMICs), which are disproportionately impacted by micronutrient deficiencies and teratogenic exposures. The Grenada Learning and Memory Scale (GLAMS) was created for use in limited resource settings and includes a shopping list and face-name association test. Here, we present psychometric and normative data for the GLAMS in a Grenadian preschool sample.

METHODS

Typically developing children between 36 and 72 months of age, primarily English speaking, were recruited from public preschools in Grenada. Trained Early Childhood Assessors administered the GLAMS and NEPSY-II in schools, homes, and clinics. GLAMS score distributions, reliability, and convergent/divergent validity against NEPSY-II were evaluated.

RESULTS

The sample consisted of 400 children (190 males, 210 females). GLAMS internal consistency, inter-rater agreement, and test-retest reliability were acceptable. Principal components analysis revealed two latent factors, aligned with expected verbal/visual memory constructs. A female advantage was observed in verbal memory. Moderate age effects were observed on list learning/recall and small age effects on face-name learning/recall. All GLAMS subtests were correlated with NEPSY-II Sentence Repetition, supporting convergent validity with a measure of verbal working memory.

CONCLUSIONS

The GLAMS is a psychometrically sound measure of learning and memory in Grenadian preschool children. Further adaptation and scale-up to global LMICs are recommended.

CVLT-II short form forced choice recognition in a clinical dementia sample: Cautions for performance validity assessment

Performance validity tests are susceptible to false positives from genuine cognitive impairment (e.g., dementia); this has not been explored with the short form of the California Verbal Learning Test II (CVLT-II-SF). In a memory clinic sample, we examined whether CVLT-II-SF Forced Choice Recognition (FCR) scores differed across diagnostic groups, and how the severity of impairment [Clinical Dementia Rating Sum of Boxes (CDR-SOB) or Mini-Mental State Examination (MMSE)] modulated test performance. Three diagnostic groups were identified: subjective cognitive impairment (SCI; n = 85), amnestic mild cognitive impairment (a-MCI; n = 17), and dementia due to Alzheimer's Disease (AD; n = 50). Significant group differences in FCR were observed using one-way ANOVA; post-hoc analysis indicated the AD group performed significantly worse than the other groups. Using multiple regression, FCR performance was modeled as a function of the diagnostic group, severity (MMSE or CDR-SOB), and their interaction. Results yielded significant main effects for MMSE and diagnostic group, with a significant interaction. CDR-SOB analyses were non-significant. Increases in impairment disproportionately impacted FCR performance for persons with AD, adding caution to research-based cutoffs for performance validity in dementia. Caution is warranted when assessing performance validity in dementia populations. Future research should examine whether CVLT-II-SF-FCR is appropriately specific for best-practice testing batteries for dementia.

The modified Ruche visuospatial learning test (RUCHE-M) for the assessment of visuospatial episodic memory in patients with temporal lobe epilepsy: Preliminary evidence for the investigation of memory binding

INTRODUCTION

The Ruche test is a visuospatial form of the Rey auditory verbal learning test (RAVLT), with initial evidence of utility in the diagnosis of temporal lobe epilepsy (TLE)-related memory disorders.

AIMS

To present the translation to Brazilian Portuguese and modification of the Ruche test (RUCHE-M) and compare the RUCHE-M and RAVLT performance between patients with right and left TLE.

METHODS

Twenty-five neuropsychologists participated in instrument adaptation. Thirty-seven patients with right (n = 19) and left (n = 18) TLE participated. Data were compared with the Mann-Whitney U test.

RESULTS

All specialists considered the final RUCHE-M to be adequate. The RUCHE-M forgetting speed index (FSI) score and several RAVLT scores differed significantly between patients with right and left TLE.

CONCLUSION

The RUCHE-M showed limited utility for the assessment of visuospatial episodic memory in patients with TLE. The manipulation of memory binding as demonstrated by FSI score seems to be a promising paradigm for the assessment of right hippocampal function.

Response time measures on the Word Memory Test do not add incremental validity to accuracy scores in predicting noncredible neurocognitive dysfunction in mild traumatic brain injury litigants

The objective of the current study was to investigate whether response time measures on the Word Memory Test (WMT) increase predictive validity on determining noncredible neurocognitive dysfunction in a large sample of mild traumatic brain injury (MTBI) litigants. Participants included 203 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT). Participants failing PVTs exhibited significantly slower response times and lower accuracy on the WMT compared to participants who passed PVTs. Response time measures did not add significant incremental validity beyond that afforded by WMT accuracy measures alone. The best predictor of PVT status was the WMT Consistency Score (CNS) which was associated with an extremely large effect size (d = 16.44), followed by Immediate Recognition (IR: d = 10.68) and Delayed Recognition (DR: d = 10.10).

Lowered cutoffs to reduce false positives on the Word Memory Test

OBJECTIVE

To adjust the decision criterion for the Word Memory Test (WMT, Green, 2003) to minimize the frequency of false positives.

METHOD

Archival data were combined into a database (n = 3,210) to examine the best cut score for the WMT. We compared results based on the original scoring rules and those based on adjusted scoring rules using a criterion based on 16 performance validity tests (PVTs) exclusive of the WMT. Cutoffs based on peer-reviewed publications and test manuals were used. The resulting PVT composite was considered the best estimate of validity status. We focused on a specificity of .90 with a false-positive rate of less than .10 across multiple samples.

RESULTS

Each examinee was administered the WMT, as well as on average 5.5 (SD = 2.5) other PVTs. Based on the original scoring rules of the WMT, 31.8% of examinees failed. Using a single failure on the criterion PVT (C-PVT), the base rate of failure was 45.9%. When requiring two or more failures on the C-PVT, the failure rate dropped to 22.8%. Applying a contingency analysis (i.e., X2) to the two failures model on the C-PVT measure and using the original rules for the WMT resulted in only 65.3% agreement. However, using our adjusted rules for the WMT, which consisted of relying on only the IR and DR WMT subtest scores with a cutoff of 77.5%, agreement between the adjusted and the C-PVT criterion equaled 80.8%, for an improvement of 12.1% identified. The adjustmeny resulted in a 49.2% reduction in false positives while preserving a sensitivity of 53.6%. The specificity for the new rules was 88.8%, for a false positive rate of 11.2%.

CONCLUSIONS

Results supported lowering of the cut score for correct responding from 82.5% to 77.5% correct. We also recommend discontinuing the use of the Consistency subtest score in the determination of WMT failure.

Detecting feigned cognitive impairment using pupillometry on the Warrington Recognition Memory Test for Words

OBJECTIVE

Pupillometry provides information about physiological and psychological processes related to cognitive load, familiarity, and deception, and it is outside of conscious control. This study examined pupillary dilation patterns during a performance validity test (PVT) among adults with true and feigned impairment of traumatic brain injury (TBI).

PARTICIPANTS AND METHODS

Participants were 214 adults in three groups: adults with bona fide moderate to severe TBI (TBI; n = 51), healthy comparisons instructed to perform their best (HC; n = 72), and healthy adults instructed and incentivized to simulate cognitive impairment due to TBI (SIM; n = 91). The Recognition Memory Test (RMT) was administered in the context of a comprehensive neuropsychological battery. Three pupillary indices were evaluated. Two pure pupil dilation (PD) indices assessed a simple measure of baseline arousal (PD-Baseline) and a nuanced measure of dynamic engagement (PD-Range). A pupillary-behavioral index was also evaluated. Dilation-response inconsistency (DRI) captured the frequency with which examinees displayed a pupillary familiarity response to the correct answer but selected the unfamiliar stimulus (incorrect answer).

RESULTS

All three indices differed significantly among the groups, with medium-to-large effect sizes. PD-Baseline appeared sensitive to oculomotor dysfunction due to TBI; adults with TBI displayed significantly lower chronic arousal as compared to the two groups of healthy adults (SIM, HC). Dynamic engagement (PD-Range) yielded a hierarchical structure such that SIM were more dynamically engaged than TBI followed by HC. As predicted, simulators engaged in DRI significantly more frequently than other groups. Moreover, subgroup analyses indicated that DRI differed significantly for simulators who scored in the invalid range on the RMT (n = 45) versus adults with genuine TBI who scored invalidly (n = 15).

CONCLUSIONS

The findings support continued research on the application of pupillometry to performance validity assessment: Overall, the findings highlight the promise of biometric indices in multimethod assessments of performance validity.