Prevalence of Low Scores in Children and Adolescents on the Test of Verbal Conceptualization and Fluency

Multivariate Base Rates of Low Neuropsychological Test Scores in Cognitively Intact Older Adults with Subjective Cognitive Decline from a Specialist Memory Clinic

OBJECTIVE

To avoid misdiagnosing mild cognitive impairment (MCI), knowledge of the multivariate base rates (MVBRs) of low scores on neuropsychological tests is crucial. Base rates have typically been determined from normative population samples, which may differ from clinically referred samples. The current study addresses this limitation by calculating the MVBR of low or high cognitive scores in older adults who presented to a memory clinic experiencing subjective cognitive decline but were not diagnosed with MCI.

METHOD

We determined the MVBRs on the Kaplan-Baycrest Neurocognitive Assessment for 107 cognitively healthy older adults (M age = 75.81), by calculating the frequency of patients producing n scores below or above different cut-off values (i.e., 1, 1.5, 2.0, 2.5 SD from the mean), stratifying by education and gender.

RESULTS

Performing below or above cut-off was common, with more stringent cut-offs leading to lower base rates (≥1 low scores occurred in 84.1% of older adults at -1 SD, 55.1% at -1.5 SD, and 39.3% at -2 SD below the mean; ≥1 high scores occurred in 80.4% of older adults at +1 SD, 35.5% at +1.5 SD, and 16.8% at +2 SD above the mean). Higher education was associated with varying base rates. Overall, the MVBR of obtaining a low cognitive test score was higher in this clinic sample, compared with prior studies of normative samples.

CONCLUSIONS

MVBRs for clinically referred older adults experiencing memory complaints provide a diagnostic benefit, helping to prevent attributing normal variability to cognitive impairment and limiting false positive diagnoses.

Enhanced interpretation of the BRIEF2: multivariate base rates of elevated scores in the standardization samples

The Behavior Rating Inventory of Executive Function, Second Edition (BRIEF2) is a standardized rating (self, parent, and teacher) scale of executive functioning in children and adolescents. Here, we provide multivariate base rate (MBR) information (for the Self, Parent, and Teacher forms), which is not included in the BRIEF2 Professional Manual. Participants were children and adolescents for the BRIEF2 Self-Report (ages = 11-18; N = 803), Parent-Report (ages = 5-18; N = 1,400), and Teacher-Report (ages = 5-18; N = 1,400) standardization samples. We focused on cumulative (e.g., % of sample with oneor more elevated scores) MBRs across scales, which were examined at three elevation levels on each form: T≥ 60, ≥65, and ≥70. Across forms, MBRs predictably decreased with increasing number of elevated scores and at higher cutoffs. The cumulative MBR of having at least one score at T≥ 60 was common (37.5-42.2%), but less frequent at T≥ 70 (15.4-17.4%). The probability of having elevated scores on all scales was very low, irrespective of form, age, or elevation threshold (T≥ 60 = 2.4-4.4%; T≥ 65 = 1.0-1.4%; T≥ 70 = 0.0-0.7%). There was no clinically meaningful relation between demographic factors (age, gender, race, and parental education) and MBRs. These data provide clinicians and researchers with an enhanced way of concurrently interpreting multiple BRIEF2 scales.

Cognitive Impairment in Late-Life Depression: A Comparative Study of Healthy Older People, Late-Life Depression, and Mild Alzheimer's Disease Using Multivariate Base Rates of Low Scores

Late-Life Depression (LLD) is often associated with cognitive impairment. However, distinction between cognitive impairment due to LLD and those due to normal aging or mild Alzheimer's Disease (AD) remain difficult. The aim of this study was to present and compare the multivariate base rates of low scores in LLD, mild AD, and healthy control groups on a battery of neuropsychological tests. Participants (ages 60-89) were 352 older healthy adults, 390 patients with LLD, and 234 patients with mild AD (i.e., MMSE ≥ 20). Multivariate base rates of low scores (i.e., ≤ 5th percentile) were calculated for each participant group within different cognitive domains (verbal episodic memory, executive skills, mental processing speed, constructional praxis, and language/semantic memory). Obtaining at least one low score was relatively common in healthy older people controls (from 9.4 to 17.6%), and may thus result in a large number of false positives. By contrast, having at least two low scores was unusual (from 0.3 to 4.6%) and seems to be a more reliable criterion for identifying cognitive impairment in LLD. Having at least three low memory scores was poorly associated with LLD (5.9%) compared to mild AD (76.1%) and may provide a useful way to differentiate between these two conditions [χ ( 1 ) 2 = 329.8, p < 0.001; Odds Ratio = 50.7, 95% CI = 38.2-77.5]. The multivariate base rate information about low scores in healthy older people and mild AD may help clinicians to identify cognitive impairments in LLD patients, improve the clinical decision-making, and target those who require regular cognitive and clinical follow-up.

Multivariate Base Rates and Concussion Detection: A Comparative Study

Recent research has yielded multivariate base rates (MBRs) of low scores in healthy populations using a widely adopted concussion screening measure, Immediate Postconcussion and Cognitive Testing (ImPACT). However, the extent to which individuals with concussion obtain reliable changes at divergent frequencies relative to healthy individuals is largely unknown. The present study examined whether MBRs of reliable change accurately discriminated between those with and without concussion. This archival review consisted of 129 healthy individuals and 81 individuals with concussion. MBRs of reliable change scores were examined at varying cutoffs and frequencies between those with and without concussion. Composites showed small to medium effect sizes in differentiating between those with and without concussion. MBRs of reliable change scores on ImPACT provided limited discriminative utility in isolation. Computations of posttest probabilities using Bayes' Theorem yielded evidence for incremental gains when utilizing MBRs of reliable change under certain constraints.

Multivariate Base Rates of Low Scores on Tests of Executive Functions in a Multi-Country Latin American Sample

The objective of the study was to determine the prevalence of low scores in a diverse Latin American population for two neuropsychological commonly used tests to evaluate executive functions and to compare the number of low scores obtained using normative data from a Spanish-speaking population from Latin America versus an English-speaking population from U.S.A. Healthy adults (N = 5402) were administered the Modified Wisconsin Card Sorting Test and Stroop Color-Word. Low scores on measures of executive functioning are common. Clinicians working with Spanish-speaking adults should take into account the higher probability of low scores on these measures to reduce false-positive diagnoses of cognitive deficits in an individual.

Multivariate base rates for the assessment of executive functioning among children and adolescents

This study provides observed base rates of low executive functioning test scores among healthy children and adolescents, stratifies those base rates by narrow intellectual functioning and age groupings, and provides normative classification ranges to aid the interpretation of performances on the Delis-Kaplan Executive Function System (D-KEFS) in clinical practice and research. Participants included 875 children and adolescents between 8 and 19 years old from the D-KEFS normative sample (48% male; 52% female). Among these participants, 838 had complete data and were included in the current study. The racial/ethnic composition of the sample was: White (73.7%), African American (12.4%), Hispanic (11.1%), and other racial/ethnic backgrounds (2.7%). The Overall Test Battery Mean (OTBM) and the prevalence of low scores at various clinical cut-offs were calculated for the 13 primary scores from the D-KEFS Trail Making Test, Verbal Fluency Test, and Color-Word Interference Test. The OTBM and base rates were also calculated separately for those scores reflecting executive functioning (n = 7) and processing speed (n = 6). Healthy children and adolescents commonly obtained low scores on the D-KEFS tests considered here. Younger age, lower estimated full-scale intelligence quotient, and more test scores interpreted were associated with a greater frequency of low scores. Clinicians and researchers are encouraged to consider these multivariate base rates when assessing and attempting to identify executive functioning impairment among children and adolescents with the D-KEFS.

Brief iPad-Based Assessment of Cognitive Functioning with ImPACT® Pediatric

ImPACT® Pediatric is an examiner-administered iOS-based battery of neuropsychological tests designed to measure neurocognitive functioning in children ages 5-11. This study documented Multivariate Base Rates (prevalence of low scores when multiple test scores are considered simultaneously) in the ImPACT® Pediatric standardization sample (N = 892). In the total sample, it was common for children to obtain at least one low factor score using the 25th percentile [T 43; base rate (BR) = 54.2%], 16th percentile (T40, BR = 38.1%), and the 10th percentile (T37, BR = 31.1%). However, it was uncommon for children to obtain two (or more) low factor scores using any of the above-listed cutoff score.

Reliable Change on Memory Tests is Common in Healthy Children and Adolescents

Objective

Neuropsychologists interpret a large number of scores in their assessments, including numerous retest scores to determine change over time. The rate at which healthy children and adolescents obtain reliably improved or declined memory scores when retested has yet to be explored. The purpose of this study was to illustrate the prevalence of reliable change scores on memory test batteries in healthy children and adolescents.

Methods

Participants were children and adolescents from test-retest samples from two published memory test batteries (ChAMP and CMS). Reliable change scores (RCI with 90% confidence interval and practice effects) were calculated for the indexes and subtests of each battery. Multivariate base rates involved considering all change scores simultaneously within each battery and calculating the frequencies of healthy children obtaining one or more reliably declined or one or more reliably improved scores.

Results

Across both memory batteries, one or more reliably changed index or subtest score was common; however, reliable change on three or more scores was uncommon (i.e., found in <5% of the samples). Base rates of change scores did not differ by parent education.

Conclusions

Having a single reliably changed score on retest is common when interpreting these memory batteries. Multivariate interpretation is necessary when determining cognitive decline and cognitive recovery. Further research is warranted with other measures, other samples, and different retest intervals.

Prevalence of low test scores in a pediatric psychiatric inpatient population: Applying multivariate base rate analyses

The understanding of neuropsychological functioning in pediatric psychiatric inpatient populations is growing, but limited, resulting in interpretive challenges. This study examined the application of multivariate base rate (MVBR) analysis in a clinical sample to appraise its utility in characterizing the frequency of low scores, as well as predictors of low scores, when using a flexible test battery. Participants included 99 children from a psychiatric inpatient unit referred for neuropsychological testing. Children hospitalized with psychiatric disorders exhibited high rates of low scores at varying criteria across the battery of tests. Hierarchical multiple regression analyses revealed that after accounting for demographic and psychiatric factors, intellectual functioning accounted for approximately 26% of the variance in observed low scores. The results suggest that a substantial percentage of this population produces low scores on neuropsychological testing and, consistent with prior research, intellectual functioning is strongly associated with low score frequency. To our knowledge, this is the first study to examine the clinical application of MVBR analysis in a pediatric psychiatric inpatient population using a flexible test battery. Taken together, this investigation highlights the potential clinical utility of MVBR analysis when interpreting neuropsychological performance in clinical pediatric populations.

Empirical Derivation and Validation of a Clinical Case Definition for Neuropsychological Impairment in Children and Adolescents

Neuropsychological assessment aims to identify individual performance profiles in multiple domains of cognitive functioning; however, substantial variation exists in how deficits are defined and what cutoffs are used, and there is no universally accepted definition of neuropsychological impairment. The aim of this study was to derive and validate a clinical case definition rule to identify neuropsychological impairment in children and adolescents. An existing normative pediatric sample was used to calculate base rates of abnormal functioning on eight measures covering six domains of neuropsychological functioning. The dataset was analyzed by varying the range of cutoff levels [1, 1.5, and 2 standard deviations (SDs) below the mean] and number of indicators of impairment. The derived rule was evaluated by bootstrap, internal and external clinical validation (orthopedic and traumatic brain injury). Our neuropsychological impairment (NPI) rule was defined as "two or more test scores that fall 1.5 SDs below the mean." The rule identifies 5.1% of the total sample as impaired in the assessment battery and consistently targets between 3 and 7% of the population as impaired even when age, domains, and number of tests are varied. The NPI rate increases in groups known to exhibit cognitive deficits. The NPI rule provides a psychometrically derived method for interpreting performance across multiple tests and may be used in children 6-18 years. The rule may be useful to clinicians and scientists who wish to establish whether specific individuals or clinical populations present within expected norms versus impaired function across a battery of neuropsychological tests.

The value of Bayes' theorem for interpreting abnormal test scores in cognitively healthy and clinical samples

The base rates of abnormal test scores in cognitively normal samples have been a focus of recent research. The goal of the current study is to illustrate how Bayes' theorem uses these base rates--along with the same base rates in cognitively impaired samples and prevalence rates of cognitive impairment--to yield probability values that are more useful for making judgments about the absence or presence of cognitive impairment. Correlation matrices, means, and standard deviations were obtained from the Wechsler Memory Scale--4th Edition (WMS-IV) Technical and Interpretive Manual and used in Monte Carlo simulations to estimate the base rates of abnormal test scores in the standardization and special groups (mixed clinical) samples. Bayes' theorem was applied to these estimates to identify probabilities of normal cognition based on the number of abnormal test scores observed. Abnormal scores were common in the standardization sample (65.4% scoring below a scaled score of 7 on at least one subtest) and more common in the mixed clinical sample (85.6% scoring below a scaled score of 7 on at least one subtest). Probabilities varied according to the number of abnormal test scores, base rates of normal cognition, and cutoff scores. The results suggest that interpretation of base rates obtained from cognitively healthy samples must also account for data from cognitively impaired samples. Bayes' theorem can help neuropsychologists answer questions about the probability that an individual examinee is cognitively healthy based on the number of abnormal test scores observed.

Considering the base rates of low performance in cognitively healthy older adults improves the accuracy to identify neurocognitive impairment with the Consortium to Establish a Registry for Alzheimer's Disease-Neuropsychological Assessment Battery (CERAD-NAB)

It is common for some healthy older adults to obtain low test scores when a battery of neuropsychological tests is administered, which increases the risk of the clinician misdiagnosing cognitive impairment. Thus, base rates of healthy individuals' low scores are required to more accurately interpret neuropsychological results. At present, this information is not available for the German version of the Consortium to Establish a Registry for Alzheimer's Disease-Neuropsychological Assessment Battery (CERAD-NAB), a frequently used battery in the USA and in German-speaking Europe. This study aimed to determine the base rates of low scores for the CERAD-NAB and to tabulate a summary figure of cut-off scores and numbers of low scores to aid in clinical decision making. The base rates of low scores on the ten German CERAD-NAB subscores were calculated from the German CERAD-NAB normative sample (N = 1,081) using six different cut-off scores (i.e., 1st, 2.5th, 7th, 10th, 16th, and 25th percentile). Results indicate that high percentages of one or more "abnormal" scores were obtained, irrespective of the cut-off criterion. For example, 60.6% of the normative sample obtained one or more scores at or below the 10th percentile. These findings illustrate the importance of considering the prevalence of low scores in healthy individuals. The summary figure of CERAD-NAB base rates is an important supplement for test interpretation and can be used to improve the diagnostic accuracy of neurocognitive disorders.