BNT-15: Revised Performance Validity Cutoffs and Proposed Clinical Classification Ranges

The brief executive language screen: sensitivity and specificity in acute to early sub-acute stroke

INTRODUCTION

Propositional language and underlying executive functions can be impaired post-stroke and affect communication and quality of life. Current stroke screening tools are largely tailored to patients with aphasia, being either non-verbal or focussed on core language skills such as naming and repetition. The Brief Executive Language Screening Test (BELS) is a newly developed cognitive screening tool that assesses memory, oral apraxia, core language, as well as propositional language and associated executive functions that can be impacted and overlooked in stroke patients without aphasia. This study examines BELS sensitivity and specificity, and performance in acute to early sub-acute stroke relative to controls.

METHOD

Cross-sectional BELS data from 88 acute left and right hemisphere stroke patients (within 7 weeks of stroke) and 116 age-matched healthy controls were compared using independent samples t-tests. ROC Curve Analysis was performed to determine a cutoff score for the BELS.

RESULTS

Left and right stroke patients were reduced on all propositional language subtests, and executive function subtests of inhibition, strategy, and selection. Differences were also observed for Oral Apraxia, Naming, and Memory. By contrast, Word Comprehension and Repetition, and Sentence Completion Initiation (after corrections applied) did not differ between groups. A total BELS score of 79.25/100 was highly sensitive (.89) and specific (.89) when classifying stroke patients and healthy controls.

CONCLUSION

The BELS is brief, sensitive, suitable for bedside administration, and can aid in detection and rehabilitation of subtle executive language impairments. This in turn will help improve relationships and quality of life post-stroke.

Influence of Infarct Morphology and Patterns on Cognitive Outcomes After Endovascular Thrombectomy

BACKGROUND

To assess the association of qualitative and quantitative infarct characteristics and 3 cognitive outcome tests, namely the Montreal Cognitive Assessment (MOCA) for mild cognitive impairment, the Boston Naming Test for visual confrontation naming, and the Sunnybrook Neglect Assessment Procedure for neglect, in large vessel occlusion stroke.

METHODS

Secondary observational cohort study using data from the randomized-controlled ESCAPE-NA1 trial (Safety and Efficacy of Nerinetide in Subjects Undergoing Endovascular Thrombectomy for Stroke), in which patients with large vessel occlusion undergoing endovascular treatment were randomized to receive either intravenous Nerinetide or placebo. MOCA, Sunnybrook Neglect Assessment Procedure, and 15-item Boston Naming Test were obtained at 90 days. Total infarct volume, gray matter, and white matter infarct volumes were manually measured on 24-hour follow-up imaging. Infarcts were also visually classified as either involving the gray matter only or both the gray and white matter and scattered versus territorial. Associations of infarct variables and cognitive outcomes were analyzed using multivariable ordinal or binary logistic regression models.

RESULTS

Of 1105 patients enrolled in ESCAPE-NA1, 1026 patients with visible infarcts on 24-hour follow-up imaging were included. MOCA and Sunnybrook Neglect Assessment Procedure were available for 706 (68.8%) patients and the 15-item Boston Naming Test was available for 682 (66.5%) patients. Total infarct volume was associated with worse MOCA scores (adjusted common odds ratio per 10 mL increase, 1.05 [95% CI, 1.04-1.06]). After adjusting for baseline variables and total infarct volume, mixed gray and white matter involvement (versus gray matter-only adjusted common odds ratio, 1.92 [95% CI, 1.37-2.69]), white matter infarct volume (adjusted common odds ratio per 10 mL increase 1.36 [95% CI, 1.18-1.58]) and territorial (versus scattered) infarct pattern (adjusted common odds ratio, 1.65 [95% CI, 1.15-2.38]) were associated with worse MOCA scores. Results for Sunnybrook Neglect Assessment Procedure and 15-item Boston Naming Test were similar, except for the territorial infarct pattern, which did not reach statistical significance in multivariable analysis.

CONCLUSIONS

Besides total infarct volume, infarcts that involve the white matter and that show a territorial distribution were associated with worse cognitive outcomes, even after adjusting for total infarct volume.

Geographic variability in limited English proficiency: A cross-cultural study of cognitive profiles

OBJECTIVE

This study was designed to evaluate the effect of limited English proficiency (LEP) on neurocognitive profiles.

METHOD

Romanian (LEP-RO; n = 59) and Arabic (LEP-AR; n = 30) native speakers were compared to Canadian native speakers of English (NSE; n = 24) on a strategically selected battery of neuropsychological tests.

RESULTS

As predicted, participants with LEP demonstrated significantly lower performance on tests with high verbal mediation relative to US norms and the NSE sample (large effects). In contrast, several tests with low verbal mediation were robust to LEP. However, clinically relevant deviations from this general pattern were observed. The level of English proficiency varied significantly within the LEP-RO and was associated with a predictable performance pattern on tests with high verbal mediation.

CONCLUSIONS

The heterogeneity in cognitive profiles among individuals with LEP challenges the notion that LEP status is a unitary construct. The level of verbal mediation is an imperfect predictor of the performance of LEP examinees during neuropsychological testing. Several commonly used measures were identified that are robust to the deleterious effects of LEP. Administering tests in the examinee's native language may not be the optimal solution to contain the confounding effect of LEP in cognitive evaluations.

Replicating a Meta-Analysis: The Search for the Optimal Word Choice Test Cutoff Continues

This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.

From "below chance" to "a single error is one too many": Evaluating various thresholds for invalid performance on two forced choice recognition tests

This study was designed to empirically evaluate the classification accuracy of various definitions of invalid performance in two forced-choice recognition performance validity tests (PVTs; FCRCVLT-II and Test of Memory Malingering [TOMM-2]). The proportion of at and below chance level responding defined by the binomial theory and making any errors was computed across two mixed clinical samples from the United States and Canada (N = 470) and two sets of criterion PVTs. There was virtually no overlap between the binomial and empirical distributions. Over 95% of patients who passed all PVTs obtained a perfect score. At chance level responding was limited to patients who failed ≥2 PVTs (91% of them failed 3 PVTs). No one scored below chance level on FCRCVLT-II or TOMM-2. All 40 patients with dementia scored above chance. Although at or below chance level performance provides very strong evidence of non-credible responding, scores above chance level have no negative predictive value. Even at chance level scores on PVTs provide compelling evidence for non-credible presentation. A single error on the FCRCVLT-II or TOMM-2 is highly specific (0.95) to psychometrically defined invalid performance. Defining non-credible responding as below chance level scores is an unnecessarily restrictive threshold that gives most examinees with invalid profiles a Pass.

Cross-Cultural Performance Validity Testing: Managing False Positives in Examinees with Limited English Proficiency

Base rates of failure (BR_Fail) on performance validity tests (PVTs) were examined in university students with limited English proficiency (LEP). BR_Fail was calculated for several free-standing and embedded PVTs. All free-standing PVTs and certain embedded indicators were robust to LEP. However, LEP was associated with unacceptably high BR_Fail (20-50%) on several embedded PVTs with high levels of verbal mediation (even multivariate models of PVT could not contain BR_Fail). In conclusion, failing free-standing/dedicated PVTs cannot be attributed to LEP. However, the elevated BR_Fail on several embedded PVTs in university students suggest an unacceptably high overall risk of false positives associated with LEP.

Take Their Word for It: The Inventory of Problems Provides Valuable Information on Both Symptom and Performance Validity

This study was designed to compare the validity of the Inventory of Problems (IOP-29) and its newly developed memory module (IOP-M) in 150 patients clinically referred for neuropsychological assessment. Criterion groups were psychometrically derived based on established performance and symptom validity tests (PVTs and SVTs). The criterion-related validity of the IOP-29 was compared to that of the Negative Impression Management scale of the Personality Assessment Inventory (NIM_PAI) and the criterion-related validity of the IOP-M was compared to that of Trial-1 on the Test of Memory Malingering (TOMM-1). The IOP-29 correlated significantly more strongly (z = 2.50, p = .01) with criterion PVTs than the NIM_PAI (r_IOP-29 = .34; r_NIM-PAI = .06), generating similar overall correct classification values (OCC_IOP-29: 79-81%; OCC_NIM-PAI: 71-79%). Similarly, the IOP-M correlated significantly more strongly (z = 2.26, p = .02) with criterion PVTs than the TOMM-1 (r_IOP-M = .79; r_TOMM-1 = .59), generating similar overall correct classification values (OCC_IOP-M: 89-91%; OCC_TOMM-1: 84-86%). Findings converge with the cumulative evidence that the IOP-29 and IOP-M are valuable additions to comprehensive neuropsychological batteries. Results also confirm that symptom and performance validity are distinct clinical constructs, and domain specificity should be considered while calibrating instruments.

Neurocognitive Profiles Associated With Limited English Proficiency in Cognitively Intact Adults

OBJECTIVE

The objective of the present study was to examine the neurocognitive profiles associated with limited English proficiency (LEP).

METHOD

A brief neuropsychological battery including measures with high (HVM) and low verbal mediation (LVM) was administered to 80 university students: 40 native speakers of English (NSEs) and 40 with LEP.

RESULTS

Consistent with previous research, individuals with LEP performed more poorly on HVM measures and equivalent to NSEs on LVM measures-with some notable exceptions.

CONCLUSIONS

Low scores on HVM tests should not be interpreted as evidence of acquired cognitive impairment in individuals with LEP, because these measures may systematically underestimate cognitive ability in this population. These findings have important clinical and educational implications.