Low scores on BDAE Complex Ideational Material are associated with invalid performance in adults without aphasia

BNT-15: Revised Performance Validity Cutoffs and Proposed Clinical Classification Ranges

BACKGROUND

Abbreviated neurocognitive tests offer a practical alternative to full-length versions but often lack clear interpretive guidelines, thereby limiting their clinical utility.

OBJECTIVE

To replicate validity cutoffs for the Boston Naming Test-Short Form (BNT-15) and to introduce a clinical classification system for the BNT-15 as a measure of object-naming skills.

METHOD

We collected data from 43 university students and 46 clinical patients. Classification accuracy was computed against psychometrically defined criterion groups. Clinical classification ranges were developed using a z -score transformation.

RESULTS

Previously suggested validity cutoffs (≤11 and ≤12) produced comparable classification accuracy among the university students. However, a more conservative cutoff (≤10) was needed with the clinical patients to contain the false-positive rate (0.20-0.38 sensitivity at 0.92-0.96 specificity). As a measure of cognitive ability, a perfect BNT-15 score suggests above average performance; ≤11 suggests clinically significant deficits. Demographically adjusted prorated BNT-15 T-scores correlated strongly (0.86) with the newly developed z -scores.

CONCLUSION

Given its brevity (<5 minutes), ease of administration and scoring, the BNT-15 can function as a useful and cost-effective screening measure for both object-naming/English proficiency and performance validity. The proposed clinical classification ranges provide useful guidelines for practitioners.

The BNT-15 provides an accurate measure of English proficiency in cognitively intact bilinguals - a study in cross-cultural assessment

This study was designed to replicate earlier reports of the utility of the Boston Naming Test - Short Form (BNT-15) as an index of limited English proficiency (LEP). Twenty-eight English-Arabic bilingual student volunteers were administered the BNT-15 as part of a brief battery of cognitive tests. The majority (23) were women, and half had LEP. Mean age was 21.1 years. The BNT-15 was an excellent psychometric marker of LEP status (area under the curve: .990-.995). Participants with LEP underperformed on several cognitive measures (verbal comprehension, visuomotor processing speed, single word reading, and performance validity tests). Although no participant with LEP failed the accuracy cutoff on the Word Choice Test, 35.7% of them failed the time cutoff. Overall, LEP was associated with an increased risk of failing performance validity tests. Previously published BNT-15 validity cutoffs had unacceptably low specificity (.33-.52) among participants with LEP. The BNT-15 has the potential to serve as a quick and effective objective measure of LEP. Students with LEP may need academic accommodations to compensate for slower test completion time. Likewise, LEP status should be considered for exemption from failing performance validity tests to protect against false positive errors.

Outcomes in post-acute sequelae of COVID-19 (PASC) at 6 months post-infection Part 1: Cognitive functioning

OBJECTIVE

Long-term cognitive sequelae of COVID-19 have not been extensively studied. This study provides initial results on cognitive outcomes in Post-Acute Sequelae of COVID-19 (PASC).Participants and Methods: This study examined 53 consecutive outpatients diagnosed with COVID-19. Four participants were excluded due to performance validity test failure. All participants had positive COVID-19 tests, reported cognitive concerns, and completed neuropsychological tests to assess performance validity, attention/working memory, processing speed, memory, language, visual-spatial, executive functioning, motor, and emotional functioning. The sample was mostly white (89.8%), female (83.7%), and never hospitalized (69.4%) for COVID-19.

RESULTS

Analyses indicated no mean scores in the Impaired range (>2 standard deviations [SD] below normative mean) on objective cognitive testing and a low base rate of Impaired test scores. Higher (>20%) base rates of Borderline performance (1-2 SDs below normative mean) were found on some measures. There was also evidence for frequently elevated mean scores on mood measures which correlated with some cognitive measures and the number of Borderline scores per participants.

CONCLUSIONS

The results were noteworthy for infrequent Impaired scores, and significant correlations between cognition and mood/anxiety measures, but not between cognitive performance and premorbid vascular risk factors, psychiatric diagnoses, or COVID-19 disease severity. Results suggest that psychological distress was prominent in PASC and related to objective cognitive performance, but objective cognitive performance was unrelated to cognitive complaints. Other contributing factors may include fatigue/sleep issues. Neurologically based cognitive deficits were not suggested by the results.During the COVID-19 pandemic, researchers and clinicians have gained considerable knowledge of the SARS-CoV-2 virus that causes the COVID-19 illness. Research has implicated COVID-19 in a variety of neurological and psychiatric issues (Taquet et al., 2021) including stroke, depression/anxiety, and more rarely intracranial hemorrhage and psychotic disorders. Further, various vascular risk factors such as hypertension, diabetes, hyperlipidemia, and obstructive sleep apnea (OSA) have been associated with more severe COVID-19 symptoms (Bauer et al., 2021; Luk et al., 2021; Mishra et al., 2020; Saxena et al., 2021). Additionally, while many patients recover from COVID-19 in a few weeks, a substantial number continue to experience physical (e.g., fatigue, shortness of breath, cough, palpitations) and cognitive (e.g., "brain fog", concentration and memory complaints) symptoms for a considerable period (Raveendran et al., 2021). It has also been suggested that patients themselves, via social media, contributed to reifying the phenomenon of "Long COVID-19" or post-acute sequelae of SARS-CoV-2 infection (PASC; Callard & Perego, 2021). However, very limited research exists that directly investigates the cognitive sequelae of COVID-19 infection, particularly in the long term. The current study aims to address this limitation in the literature.

Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator

OBJECTIVE

: Replicate previous research on Logical Memory Recognition (LM_Recog) and perform a critical item analysis.

METHOD

: Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LM_Recog and nine critical items (CR-9) was computed.

RESULTS

: LM_Recog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LM_Recog ≤20; CR-9 ≥6 increased specificity by 6-8% over LM_Recog ≤20; CR-9 ≥7 increased specificity by 8-15%.

CONCLUSIONS

: Critical item analysis enhances the classification accuracy of the optimal LM_Recog cutoff (≤20).

The Impact of Primary Progressive Aphasia on Picture Naming and General Language Ability

BACKGROUND

Primary progressive aphasia (PPA) is a clinical syndrome that is characterized by progressive deterioration of language while other cognitive domains remain relatively intact. The extent to which print exposure and cortical volume atrophy jointly influence picture naming and general language ability in individuals with PPA remains underexplored.

OBJECTIVE

To investigate the language performance of individuals with the nonfluent variant of primary progressive aphasia (nfvPPA) and to explore the impact of print exposure and cortical volume atrophy on their language ability.

METHOD

We compared 14 Greek individuals with nfvPPA and similar age, education, disease duration, and cognitive ability with age-, gender- and education-matched Greek controls on picture naming and on language tasks of the Boston Diagnostic Aphasia Examination-Short Form, including oral word reading, word and sentence repetition, complex ideational material, and reading comprehension. The effects of print exposure and left-hemisphere cortical volume on the individuals' language performance were estimated through stepwise regression models.

RESULTS

The language performance of the individuals with nfvPPA was affected by print exposure and cortical volume atrophy. Picture naming and word reading were affected by print exposure. The highest contributions of cortical volume atrophy were found for the repetition, complex ideational material, and reading comprehension tasks.

CONCLUSION

Print exposure and cortical volume atrophy may help explain variability in the language performance of nfvPPA individuals with similar age, education, disease duration, and cognitive ability.

The emotion word fluency test as an embedded performance validity indicator - Alone and in a multivariate validity composite

OBJECTIVE

This project was designed to cross-validate existing performance validity cutoffs embedded within measures of verbal fluency (FAS and animals) and develop new ones for the Emotion Word Fluency Test (EWFT), a novel measure of category fluency.

METHOD

The classification accuracy of the verbal fluency tests was examined in two samples (70 cognitively healthy university students and 52 clinical patients) against psychometrically defined criterion measures.

RESULTS

A demographically adjusted T-score of ≤31 on the FAS was specific (.88-.97) to noncredible responding in both samples. Animals T ≤ 29 achieved high specificity (.90-.93) among students at .27-.38 sensitivity. A more conservative cutoff (T ≤ 27) was needed in the patient sample for a similar combination of sensitivity (.24-.45) and specificity (.87-.93). An EWFT raw score ≤5 was highly specific (.94-.97) but insensitive (.10-.18) to invalid performance. Failing multiple cutoffs improved specificity (.90-1.00) at variable sensitivity (.19-.45).

CONCLUSIONS

Results help resolve the inconsistency in previous reports, and confirm the overall utility of existing verbal fluency tests as embedded validity indicators. Multivariate models of performance validity assessment are superior to single indicators. The clinical utility and limitations of the EWFT as a novel measure are discussed.

Flipping the Script: Measuring Both Performance Validity and Cognitive Ability with the Forced Choice Recognition Trial of the RCFT

In this study we attempted to replicate the classification accuracy of the newly introduced Forced Choice Recognition trial (FCR) of the Rey Complex Figure Test (RCFT) in a clinical sample. We administered the RCFT_FCR and the earlier Yes/No Recognition trial from the RCFT to 52 clinically referred patients as part of a comprehensive neuropsychological test battery and incentivized a separate control group of 83 university students to perform well on these measures. We then computed the classification accuracies of both measures against criterion performance validity tests (PVTs) and compared results between the two samples. At previously published validity cutoffs (≤16 & ≤17), the RCFT_FCR remained specific (.84-1.00) to psychometrically defined non-credible responding. Simultaneously, the RCFT_FCR was more sensitive to examinees' natural variability in visual-perceptual and verbal memory skills than the Yes/No Recognition trial. Even after being reduced to a seven-point scale (18-24) by the validity cutoffs, both RCFT recognition scores continued to provide clinically useful information on visual memory. This is the first study to validate the RCFT_FCR as a PVT in a clinical sample. Our data also support its use for measuring cognitive ability. Replication studies with more diverse samples and different criterion measures are still needed before large-scale clinical application of this scale.

This will only take a minute: Time cutoffs are superior to accuracy cutoffs on the forced choice recognition trial of the Hopkins Verbal Learning Test - Revised

OBJECTIVE

This study was designed to evaluate the classification accuracy of the recently introduced forced-choice recognition trial to the Hopkins Verbal Learning Test - Revised (FCR_HVLT-R) as a performance validity test (PVT) in a clinical sample. Time-to-completion (T2C) for FCR_HVLT-R was also examined.

METHOD

Forty-three students were assigned to either the control or the experimental malingering (expMAL) condition. Archival data were collected from 52 adults clinically referred for neuropsychological assessment. Invalid performance was defined using expMAL status, two free-standing PVTs and two validity composites.

RESULTS

Among students, FCR_HVLT-R ≤11 or T2C ≥45 seconds was specific (0.86-0.93) to invalid performance. Among patients, an FCR_HVLT-R ≤11 was specific (0.94-1.00), but relatively insensitive (0.38-0.60) to non-credible responding0. T2C ≥35 s produced notably higher sensitivity (0.71-0.89), but variable specificity (0.83-0.96). The T2C achieved superior overall correct classification (81-86%) compared to the accuracy score (68-77%). The FCR_HVLT-R provided incremental utility in performance validity assessment compared to previously introduced validity cutoffs on Recognition Discrimination.

CONCLUSIONS

Combined with T2C, the FCR_HVLT-R has the potential to function as a quick, inexpensive and effective embedded PVT. The time-cutoff effectively attenuated the low ceiling of the accuracy scores, increasing sensitivity by 19%. Replication in larger and more geographically and demographically diverse samples is needed before the FCR_HVLT-R can be endorsed for routine clinical application.

Neurocognitive deficits in severe COVID-19 infection: Case series and proposed model

Objective: To date, very few studies investigating neurocognitive deficits in COVID-19 have been published. This case series addresses cognition in post-COVID-19 patient by describing three patients in acute rehabilitation to inform a model of cognitive sequelae of COVID-19. Methods: Three English-speaking inpatients with severe symptoms and long-term intensive care unit (ICU) treatment are described. All patients had a premorbid history of hypertension and hyperlipidemia and experienced delirium and hypoxemia when hospitalized. Patient 1 is a 62-year-old male with 15 years of education with additional history of obstructive sleep apnea and type 2 diabetes. Patient 2 is a 73-year-old female with 12 years of education with a premorbid medical history of alcohol use disorder and Guillain-Barre syndrome. Patient 3 is a 75-year-old male with 14 years of education. No patients had premorbid psychiatric histories. Results: The three patients demonstrated deficits on formal neuropsychological testing, particularly with encoding and verbal fluency. Memory measures improved with a more structured story memory task compared to a less-structured verbal list-learning task, suggesting executive dysfunction impacted learning. None of the patients demonstrated rapid forgetting of information. Two patients endorsed new depressive and/or anxiety symptoms. Conclusions: The results suggest evidence for neurocognitive deficits after severe COVID-19 infection, particularly in encoding and verbal fluency. These results were interpreted with caution given the limited number of patients and the telephone-based battery. The specific mechanism that caused these cognitive deficits in these individuals remains unclear. A proposed three-stage model of cognitive dysfunction is described to help guide future research.

Stronger Together: The Wechsler Adult Intelligence Scale-Fourth Edition as a Multivariate Performance Validity Test in Patients with Traumatic Brain Injury

OBJECTIVE

This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV).

METHOD

Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance.

RESULTS

Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91-.95) over univariate cutoffs (.78-.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43-.67) compared to univariate cutoffs (.11-.63) while maintaining consistently high specificity (.93-.95).

CONCLUSIONS

In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment.

BRIEF SUMMARY

Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.

Performance validity test failure in clinical populations-a systematic review

Performance validity tests (PVTs) are widely used in attempts to quantify effort and/or detect negative response bias during neuropsychological testing. However, it can be challenging to interpret the meaning of poor PVT performance in a clinical context. Compensation-seeking populations predominate in the PVT literature. We aimed to establish base rates of PVT failure in clinical populations without known external motivation to underperform. We searched MEDLINE, EMBASE and PsycINFO for studies reporting PVT failure rates in adults with defined clinical diagnoses, excluding studies of active or veteran military personnel, forensic populations or studies of participants known to be litigating or seeking disability benefits. Results were summarised by diagnostic group and implications discussed. Our review identified 69 studies, and 45 different PVTs or indices, in clinical populations with intellectual disability, degenerative brain disease, brain injury, psychiatric disorders, functional disorders and epilepsy. Various pass/fail cut-off scores were described. PVT failure was common in all clinical groups described, with failure rates for some groups and tests exceeding 25%. PVT failure is common across a range of clinical conditions, even in the absence of obvious incentive to underperform. Failure rates are no higher in functional disorders than in other clinical conditions. As PVT failure indicates invalidity of other attempted neuropsychological tests, the finding of frequent and unexpected failure in a range of clinical conditions raises important questions about the degree of objectivity afforded to neuropsychological tests in clinical practice and research.

Short-term effects of cannabis consumption on cognitive performance in medical cannabis patients

This observational study examined the acute cognitive effects of cannabis. We hypothesized that cognitive performance would be negatively affected by acute cannabis intoxication. Twenty-two medical cannabis patients from Southwestern Ontario completed the study. The majority (n = 13) were male. Mean age was 36.0 years, and mean level of education was 13.7 years. Participants were administered the same brief neurocognitive battery three times during a six-hour period: at baseline ("Baseline"), once after they consumed a 20% THC cannabis product ("THC"), and once again several hours later ("Recovery"). The average self-reported level of cannabis intoxication prior to the second assessment (i.e., during THC) was 5.1 out of 10. Contrary to expectations, performance on neuropsychological tests remained stable or even improved during the acute intoxication stage (THC; d: .49-.65, medium effect), and continued to increase during Recovery (d: .45-.77, medium-large effect). Interestingly, the failure rate on performance validity indicators increased during THC. Contrary to our hypothesis, there was no psychometric evidence for a decline in cognitive ability following THC intoxication. There are several possible explanations for this finding but, in the absence of a control group, no definitive conclusion can be reached at this time.

Brain networks' functional connectivity separates aphasic deficits in stroke

OBJECTIVE

To investigate whether different language deficits are distinguished by the relative strengths of their association with the functional connectivity (FC) at rest of the language network (LN) and cingulo-opercular network (CON) after aphasic stroke.

METHODS

In a group of patients with acute stroke and left-hemisphere damage, we identified 3 distinct, yet correlated, clusters of deficits including comprehension/lexical semantic, grapheme-phoneme knowledge, and verbal executive functions. We computed partial correlations in which the contributions of a behavioral cluster and network FC of no interest were statistically regressed out.

RESULTS

We observed a double dissociation such that impairment of grapheme-phoneme knowledge was more associated with lower FC of the LN within the left hemisphere than lower FC of the CON, whereas verbal executive deficits were more related to lower FC of the CON than the LN in the left hemisphere. Furthermore, the specific association between language deficits and FC was independent of the amount of structural damage to the LN and CON.

CONCLUSION

These findings indicate that after a left-hemisphere lesion, the type of language impairment is related to the abnormal pattern of correlated activity in different networks. Accordingly, they extend the concept of a neuropsychological double dissociation from structural damage to functional network abnormalities. Finally, current results strongly argue in favor of the behavioral specificity of intrinsic brain activity after focal structural damage.

Differentiating epilepsy from psychogenic nonepileptic seizures using neuropsychological test data

OBJECTIVE

Differentiating epileptic seizures (ES) from psychogenic nonepileptic seizures (PNES) represents a challenging differential diagnosis with important treatment implications. This study was designed to explore the utility of neuropsychological test scores in differentiating ES from PNES.

METHOD

Psychometric data from 72 patients with ES and 33 patients with PNES were compared on various tests of cognitive ability and performance validity. Individual measures that best discriminated the diagnoses were then entered as predictors in a logistic regression equation with group membership (ES vs. PNES) as the criterion.

RESULTS

On most tests of cognitive ability, the PNES sample outperformed the ES sample (medium-large effect) and was less likely to fail the Reliable Digit Span. However, patients with PNES failed two embedded validity indicators at significantly higher rates (risk ratios (RR): 2.45-4.16). There were no group differences on the Test of Memory Malingering (TOMM). A logistic regression equation based on seven neuropsychological tests correctly classified 85.1% of patients. The cutoff with perfect specificity was associated with 0.47 sensitivity.

CONCLUSIONS

Consistent with previous research, the utility of psychometric methods of differential diagnosis is limited by the complex neurocognitive profiles associated with ES and PNES. Although individual measures might help differentiate ES from PNES, multivariate assessment models have superior discriminant power. The strongest psychometric evidence for PNES appears to be a consistent lack of impairment on tests sensitive to diffuse neurocognitive deficits such as processing speed, working memory, and verbal fluency. While video-electroencephalogram (EEG) monitoring is the gold standard of differential diagnosis, psychometric testing has the potential to enhance clinical decision-making, particularly in complex or unclear cases such as patients with nondiagnostic video-EEGs. Adopting a standardized, fixed neuropsychological battery at epilepsy centers would advance research on the differential diagnostic power of psychometric testing.

Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment

This study was designed to examine the "domain specificity" hypothesis in performance validity tests (PVTs) and the epistemological status of an "indeterminate range" when evaluating the credibility of a neuropsychological profile using a multivariate model of performance validity assessment. While previous research suggests that aggregating PVTs produces superior classification accuracy compared to individual instruments, the effect of the congruence between the criterion and predictor variable on signal detection and the issue of classifying borderline cases remain understudied. Data from a mixed clinical sample of 234 adults referred for cognitive evaluation (M_Age = 46.6; M_Education = 13.5) were collected. Two validity composites were created: one based on five verbal PVTs (EI-5_VER) and one based on five nonverbal PVTs (EI-5_NV) and compared against several other PVTs. Overall, language-based tests of cognitive ability were more sensitive to elevations on the EI-5_VER compared to visual-perceptual tests; whereas, the opposite was observed with the EI-5_NV. However, the match between predictor and criterion variable had a more complex relationship with classification accuracy, suggesting the confluence of multiple factors (sensory modality, cognitive domain, testing paradigm). An "indeterminate range" of performance validity emerged that was distinctly different from both the Pass and the Fail group. Trichotomized criterion PVTs (Pass-Borderline-Fail) had a negative linear relationship with performance on tests of cognitive ability, providing further support for an "in-between" category separating the unequivocal Pass and unequivocal Fail classification range. The choice of criterion variable can influence classification accuracy in PVT research. Establishing a Borderline range between Pass and Fail more accurately reflected the distribution of scores on multiple PVTs. The traditional binary classification system imposes an artificial dichotomy on PVTs that was not fully supported by the data. Accepting "indeterminate" as a legitimate third outcome of performance validity assessment has the potential to improve the clinical utility of PVTs and defuse debates regarding "near-Passes" and "soft Fails."