The effect of literacy and culture on cognitive effort test performance: An examination of the Test of Memory Malingering in Colombia

Validity assessment in Eastern Europe: cross-validation of the Dot Counting Test and MODEMM against the TOMM-1 and Rey-15 in a Romanian mixed clinical sample

OBJECTIVE

This study investigated performance validity in the understudied Romanian clinical population by exploring classification accuracies of the Dot Counting Test (DCT) and the first Romanian performance validity test (PVT) (Memory of Objects and Digits and Evaluation of Memory Malingering/MODEMM) in a heterogeneous clinical sample.

METHODS

We evaluated 54 outpatients (26 females; MAge = 62.02; SDAge = 12.3; MEducation = 2.41, SDEducation = 2.82) with the Test of Memory Malingering 1 (TOMM-1), Rey Fifteen Items Test (Rey-15) (free recall and recognition trials), DCT, MODEMM, and MMSE/MoCA as part of their neuropsychological assessment. Accuracy parameters and base failure rates were computed for the DCT and MODEMM indicators against the TOMM-1 and Rey-15. Two patient groups were constructed according to psychometrically defined credible/noncredible performance (i.e., pass/fail both TOMM-1 and Rey-15).

RESULTS

Similar to other cultures, a cutoff of ≥18 on the DCT E score produced the best combination between sensitivity (0.50-0.57) and specificity (≥0.90). MODEMM indicators based on recognition accuracy, inconsistencies, and inclusion false positives generated 0.75-0.86 sensitivities at ≥0.90 specificities. Multivariable models of MODEMM indicators reached perfect sensitivities at ≥0.90 specificities against two PVTs. Patients who failed the TOMM-1 and Rey-15 were significantly more likely to fail the DCT and MODEMM than patients who passed both PVTs.

CONCLUSIONS

Our results offer proof of concept for the DCT's cross-cultural validity and the applicability of the MODEMM on Romanian clinical examinees, further recommending the use of heterogeneous validity indicators in clinical assessments.

Best Practices and Methodological Strategies for Addressing Generalizability in Neuropsychological Assessment

Generalizability considerations are widely discussed and a core foundation for understanding when and why treatment effects will replicate across sample demographics. However, guidelines on assessing and reporting generalizability-related factors differ across fields and are inconsistently applied. This paper synthesizes obstacles and best practices to apply recent work on measurement and sample diversity. We present a brief history of how knowledge in psychology has been constructed, with implications for who has been historically prioritized in research. We then review how generalizability remains a contemporary threat to neuropsychological assessment and outline best practices for researchers and clinical neuropsychologists. In doing so, we provide concrete tools to evaluate whether a given assessment is generalizable across populations and assist researchers in effectively testing and reporting treatment differences across sample demographics.

Illiteracy, Neuropsychological Assessment, and Cognitive Rehabilitation: A Narrative Review

OBJECT

Νeuropsychological assessment is particularly important for the accurate discrimination of cognitive abilities and weaknesses of patients in order to determine the appropriate therapeutic intervention. However, the reliability and validity of neuropsychological assessment appears to be influenced by a wide range of factors, including literacy and educational level.

AIM

This systematic review evaluates neuropsychological tests appropriate for the valid assessment of illiterate individuals and the effectiveness of cognitive rehabilitation programs for illiterate and/or low-educated individuals according to the results of English language studies that have been published in the PubMed/Medline electronic database until August 2022 (no initiation date).

RESULTS

49 studies were included for neuropsychological assessment and 4 studies for cognitive rehabilitation. In terms of investigating the validity and reliability of neuropsychological tests for the assessment of healthy illiterate individuals, most studies concluded that for the majority of neuropsychological tests there is a significant difference in performance between healthy illiterate and literate individuals. However, there was consensus among studies that the performance of illiterate subjects was equivalent to the performance of literate subjects on tasks depicting colored and real objects. Regarding cognitive rehabilitation programs, all four studies concluded that they are effective in improving the cognitive functions of illiterate and/or low-literate patients with mild cognitive impairment and/or mild dementia.

CONCLUSIONS

For the assessment of illiterate individuals, it is imperative that neuropsychological tests with high ecological validity (i.e., tests related to activities of daily living) be administered so as not to underestimate their cognitive functioning. At the same time, cognitive enhancement/stimulation programs seem to be effective in this population group; however, this area needs further investigation.

What are the predictors of TOMM failure in clinical TBI populations? A retrospective analysis

OBJECTIVES

To determine base rates of invalid performance on the Test of Memory Malingering (TOMM) in patients with traumatic brain injury (TBI) undertaking rehabilitation who were referred for clinical assessment, and the factors contributing to TOMM failure.

METHODS

Retrospective file review of consecutive TBI referrals for neuropsychological assessment over seven years. TOMM failure was conventionally defined as performance <45/50 on Trial 2 or Retention Trial. Demographic, injury, financial compensation, occupational, and medical variables were collected.

RESULTS

Four hundred and ninety one TBI cases (Median age = 40 years [IQR = 26-52], 79% male, 82% severe TBI) were identified. Overall, 48 cases (9.78%) failed the TOMM. Logistic regression analyses revealed that use of an interpreter during the assessment (adjusted odds ratio [aOR] = 8.25, 95%CI = 3.96-17.18), outpatient setting (aOR = 4.80, 95%CI = 1.87-12.31) and post-injury psychological distress (aOR = 2.77, 95%CI = 1.35-5.70) were significant multivariate predictors of TOMM failure. The TOMM failure rate for interpreter cases was 49% (21/43) in the outpatient setting vs. 7% (2/30) in the inpatient setting. By comparison, 9% (21/230) of non-interpreter outpatient cases failed the TOMM vs. 2% (4/188) of inpatient cases.

CONCLUSIONS

TOMM failure very rarely occurs in clinical assessment of TBI patients in the inpatient rehabilitation setting. It is more common in the outpatient setting, particularly in non-English-speaking people requiring an interpreter. The findings reinforce the importance of routinely administering stand-alone performance validity tests in assessments of clinical TBI populations, particularly in outpatient settings, to ensure that neuropsychological test results can be interpreted with a high degree of confidence.

Neurocognitive Profiles Associated With Limited English Proficiency in Cognitively Intact Adults

OBJECTIVE

The objective of the present study was to examine the neurocognitive profiles associated with limited English proficiency (LEP).

METHOD

A brief neuropsychological battery including measures with high (HVM) and low verbal mediation (LVM) was administered to 80 university students: 40 native speakers of English (NSEs) and 40 with LEP.

RESULTS

Consistent with previous research, individuals with LEP performed more poorly on HVM measures and equivalent to NSEs on LVM measures-with some notable exceptions.

CONCLUSIONS

Low scores on HVM tests should not be interpreted as evidence of acquired cognitive impairment in individuals with LEP, because these measures may systematically underestimate cognitive ability in this population. These findings have important clinical and educational implications.

Examining the cross-cultural validity of the test of memory malingering and the Rey 15-item test

OBJECTIVE

This study was designed to investigate the cross-cultural validity of two freestanding performance validity tests (PVTs), the Test of Memory Malingering - Trial 1 (TOMM-1) and the Rey Fifteen Item Test (Rey-15) in Romanian-speaking patients.

METHODS

The TOMM-1 and Rey-15 free recall (FR) and the combination score incorporating the recognition trial (COMB) were administered to a mixed clinical sample of 61 adults referred for cognitive evaluation, 24 of whom had external incentives to appear impaired. Average scores on PVTs were compared between the two groups. Classification accuracies were computed using one PVT against another.

RESULTS

Patients with identifiable external incentives to appear impaired produced significantly lower scores and more errors on validity indicators. The largest effect sizes emerged on TOMM-1 (Cohen's d = 1.00-1.19). TOMM-1 was a significant predictor of the Rey-15 COMB ≤20 (AUC = .80; .38 sensitivity; .89 specificity at a cutoff of ≤39). Similarly, both Rey-15 indicators were significant predictors of TOMM-1 at ≤39 as the criterion (AUCs = .73-.76; .33 sensitivity; .89-.90 specificity).

CONCLUSION

Results offer a proof of concept for the cross-cultural validity of the TOMM-1 and Rey-15 in a Romanian clinical sample.

Demographic differences in performance validity test failure

OBJECTIVE

The present study investigated demographic differences in performance validity test (PVT) failure in a Veteran sample.

METHOD

Data were extracted from clinical neuropsychological evaluations. Only veterans who identified as men, as either European American/White (EA) or African American/Black (AA) were included (n = 1261). We investigated whether performance on two frequently used PVTs, the Test of Memory Malingering (TOMM), and the Medical Symptom Validity Test (MSVT), differed by age, education, and race using separate logistic regressions.

RESULTS

Veterans with younger age, less education, and Veterans Affairs (VA) service-connected disability were significantly more likely to fail both PVTs. Race was not a significant predictor of MSVT failure, but AA patients were significantly more likely than EA patients to fail the TOMM. For all significant demographic predictors in the models, effects were small. In a subsample of patients who were given both PVTs (n = 461), the effects of race on performance remained.

CONCLUSIONS

Performance on the TOMM and MSVT differed by age and level of education. Performance on the TOMM differed between EA and AA patients, whereas performance on the MSVT did not. These results suggest that demographic factors may play a small but measurable role in performance on specific PVTs.

Cross-cultural neuropsychological assessment in the European Union: a Delphi expert study

OBJECTIVE

The increasing ethnic diversity in the European Union (EU) calls for adaptations to neuropsychological assessment practices. The aims of this study were to examine the current state of cross-cultural neuropsychological assessment in EU-15 countries and to provide recommendations for researchers and policy makers.

METHOD

Twelve experts from nine EU-15 countries participated in a Delphi consensus study involving two sequential rounds of web-based questionnaires and an in-person consensus meeting. The experts individually rated Delphi topics on the basis of importance (scale 1-10). The degree of consensus was determined by assessing first and third quartiles (Q1 and Q3) and medians.

RESULTS

Consensus outcomes showed the following priorities: (a) the development of tests (median importance rating 10, Q1-Q3: 9-10), (b) the collection of normative data (median importance rating 9, Q1-Q3: 8-10), and (c) more training, awareness, and knowledge regarding cross-cultural assessment among neuropsychologists in the EU (median importance rating 9, Q1-Q3: 8-10). Whereas memory tests were often available, tests measuring social cognition (median 9, Q1-Q3: 8-10) and language (median 9, Q1-Q3: 7-10) are particularly lacking. Recommendations were made regarding essential skills and knowledge necessary for cross-cultural neuropsychological assessment.

CONCLUSIONS

This study in a small group of experts suggests that the development and availability of cross-cultural tests and normative data should be prioritized, as well as the development and implementation of training initiatives. Furthermore, EU guidelines could be established for working with interpreters during neuropsychological assessment. Before implementing these recommendations, follow-up studies are recommended that include more minority neuropsychologists and community stakeholders.