NIH toolbox cognition tests following traumatic brain injury: Frequency of low scores

Development and Validation of a Vocabulary Measure in the Mobile Toolbox

OBJECTIVE

We describe the development of a new computer adaptive vocabulary test, Mobile Toolbox (MTB) Word Meaning, and validity evidence from 3 studies.

METHOD

Word Meaning was designed to be a multiple-choice synonym test optimized for self-administration on a personal smartphone. The items were first calibrated online in a sample of 7,525 participants to create the computer-adaptive test algorithm for the Word Meaning measure within the MTB app. In Study 1, 92 participants self-administered Word Meaning on study-provided smartphones in the lab and were administered external measures by trained examiners. In Study 2, 1,021 participants completed the external measures in the lab and Word Meaning was self-administered remotely on their personal smartphones. In Study 3, 141 participants self-administered Word Meaning remotely twice with a 2-week delay on personal iPhones.

RESULTS

The final bank included 1363 items. Internal consistency was adequate to good across samples (ρxx = 0.78 to 0.81, p < .001). Test-retest reliability was good (ICC = 0.65, p < .001), and the mean theta score was not significantly different upon the second administration. Correlations were moderate to large with measures of similar constructs (ρ = 0.67-0.75, p < .001) and non-significant with measures of dissimilar constructs. Scores demonstrated small to moderate correlations with age (ρ = 0.35 to 0.45, p < .001) and education (ρ = 0.26, p < .001).

CONCLUSION

The MTB Word Meaning measure demonstrated evidence of reliability and validity in three samples. Further validation studies in clinical samples are necessary.

Multivariate base rates of potentially problematic scores on the NIH Toolbox Emotion Battery

OBJECTIVE

To examine the normal frequency of obtaining one or more scores considered potentially problematic based on normative comparisons when completing the NIH Toolbox Emotion Battery (NIHTB-EB).

METHOD

Participants (N = 753; ages 18-85, 62.4% women, 66.4% non-Hispanic White) from the NIHTB norming study completed 17 scales of emotional functioning fitting into three subdomains (i.e., Negative Affect, Psychological Well-being, Social Satisfaction). Scores were considered potentially problematic if they were 1 SD above/below the mean, depending on the orientation of the scale, and cutoffs for 1.5 and 2 SD were also included for reference. Multivariate base rates quantified the rate at which participants obtained one or more potentially problematic scale or subdomain scores.

RESULTS

The portion of participants obtaining one or more potentially problematic scores on the NIHTB-EB scales and subdomains was 61.2 and 23.2%, respectively. Participants who were younger (i.e., 18-49) or had less education had higher rates of potentially problematic scores within specific subdomains. There were no significant differences by sex or race/ethnicity.

CONCLUSIONS

Elevated scores on the NIHTB-EB were common in the normative sample and related to education/age. The multivariate base rates provided indicate obtaining one or more potentially problematic scores on the NIHTB-EB is broadly normal among adults, which may guard against overinterpreting a single score as clinically significant. These base rates should be considered in the context of other assessment findings, such as interviews, medical history or informant reports, to ensure that true emotional problems are not dismissed, and normal variation in emotional functioning is not pathologized.

Degenerative cervical myelopathy: establishing severity thresholds for neuromotor dysfunction in the aging spine using the NIH Toolbox Assessment Scale

Degenerative cervical myelopathy (DCM) is a leading cause of age-related non-traumatic spinal cord disorders resulting from chronic degeneration of the cervical spine. While traditional clinical assessments rely on patient-reported measures, this study used the NIH Toolbox Motor Battery (NIHTBm) as an objective, quantitative measure to determine DCM severity. The objective is to define NIHTBm cutoff values that can accurately classify the severity of DCM neuromotor dysfunction. A case-controlled pilot study of patients with DCM and age-matched controls. The focus was an in-depth quantitative motor assessment using the NIHTBm to understand the severity of neuromotor deficits due to degenerative spine disease. Motor assessments, dexterity, grip strength, balance, and gait speed were measured in 45 DCM patients and 37 age-matched healthy subjects (HC). Receiver operating curve (ROC) analysis determined cutoff values for mild and moderate-to-severe myelopathy which were validated by comparing motor assessment scores with disability scores. The ROC curves identified thresholds for mild dexterity impairment (T-score range 38.4 - 33.5, AUC 0.77), moderate-to-severe dexterity impairment (< 33.5, AUC 0.70), mild grip strength impairment (47.4 - 32.0, AUC 0.80), moderate-to-severe grip strength impairment (< 32.0, AUC 0.75), mild balance impairment (36.4 - 33.0, AUC 0.61), and moderate-to-severe balance impairment (< 33.0, AUC 0.78). Mild gait speed impairment was defined as 0.78-0.6 m/sec (AUC 0.65), while moderate-to-severe gait speed impairment was < 0.6 m/sec (AUC 0.65). The NIHTB motor score cutoff points correlated negatively with the DCM neck disability index (NDI) and showed balance and dexterity measures as independent indicators of DCM dysfunction. The use of NIHTB allows for precise delineation of DCM severity by establishing cutoff values corresponding to mild and moderate-to-severe myelopathy. The use of NIHTB in DCM allows enhanced clinical precision, enabling clinicians to better pinpoint specific motor deficits in DCM and other neurological disorders with motor deficits, including stroke and traumatic brain injury (TBI). Furthermore, the utility of objective assessment, NIHTB, allows us to gain a better understanding of the heterogeneity of DCM, which will enhance treatment strategies. This study serves as a foundation for future research to facilitate the discovery of innovative treatment strategies for DCM and other neurological conditions.

Cognitive impairment indicator for the neuropsychological test batteries in the Canadian Longitudinal Study on Aging: definition and evidence for validity

BACKGROUND

Prevalence of overall cognitive impairment based on each participant's performance across a neuropsychological battery is challenging; consequently, we define and validate a dichotomous cognitive impairment/no cognitive indicator (CII) using a neuropsychological battery administered in a population-based study. This CII approximates the clinical practice of interpretation across a neuropsychological battery and can be applied to any neuropsychological dataset.

METHODS

Using data from participants aged 45-85 in the Canadian Longitudinal Study on Aging receiving a telephone-administered neuropsychological battery (Tracking, N = 21,241) or a longer in-person battery (Comprehensive, N = 30,097), impairment was determined for each neuropsychological test based on comparison with normative data. We adjusted for the joint probability of abnormally low scores on multiple neuropsychological tests using baserates of low scores demonstrated in the normative samples and created a dichotomous CII (i.e., cognitive impairment vs no cognitive impairment). Convergent and discriminant validity of the CII were assessed with logistic regression analyses.

RESULTS

Using the CII, the prevalence of cognitive impairment was 4.3% in the Tracking and 5.0% in the Comprehensive cohorts. The CII demonstrated strong convergent and discriminant validity.

CONCLUSIONS

The approach for the CII is a feasible method to identify participants who demonstrate cognitive impairment on a battery of tests. These methods can be applied in other epidemiological studies that use neuropsychological batteries.

Assessing Clinically Significant Cognitive Impairment Using the NIH Toolbox in Individuals with Co-occurring Serious Mental Illness and Alcohol Use Disorder

OBJECTIVE

Serious mental illnesses (SMI) and alcohol use disorder (AUD) co-occurrence (SMI-AUD) is common, yet little is known about the prevalence and risk factors of cognitive impairment for this population. We used the National Institutes of Health (NIH) Toolbox to identify clinically significant cognitive impairment (CSCI), describe the cognitive profile, and investigate whether psychiatric and AUD severity measures are associated with CSCI in individuals with SMI-AUD.

METHODS

CSCI was defined as 2 or more fully corrected fluid subtest T scores below a set threshold based on an individual's crystalized composite score. Psychiatric severity measures included the Structured Clinical Interview for DSM-V (SCID-5) for SMI diagnosis and the Positive and Negative Syndrome Scale. AUD severity measures included the SCID-5 for AUD symptom severity score, years of alcohol use, and urine ethyl glucuronide levels. A multivariable logistic regression was used to investigate the adjusted effects of each variable on the probability of CSCI.

RESULTS

Forty-one percent (N = 55/135) of our sample had CSCI compared with the base rate of 15% from the NIH Toolbox normative sample. Subtests measuring executive function most frequently contributed to meeting criteria for CSCI (Flanker and Dimensional Change Card Sort). A history of head injury ( P = 0.033), increased AUD symptom severity score ( P = 0.007) and increased negative symptom severity score ( P = 0.027) were associated with CSCI.

CONCLUSIONS

Cognition should be considered in the treatment of people with SMI-AUD, particularly in those with history of brain injury, higher AUD symptom severity, and/or negative symptom severity.

Isolating Neurologic Deficits in Cervical Spondylotic Myelopathy: A Case-Controlled Study, Using the NIH Toolbox Motor Battery

Background and Objectives

Patients with cervical spondylotic myelopathy (CSM) have motor impairments, including weakness, imbalance, and loss of dexterity. The reliable assessment of these symptoms is critical for treatment decisions. This study aimed to determine, for the first time, the use of the NIH Toolbox motor battery (NIHTBm) in the objective assessment of motor deficits in patients with CSM.

Methods

Patients with symptoms and MRI evidence of CSM and age-matched healthy controls (HC), with no evidence of spinal disorder or surgery were included in this case-control study based on our inclusion and exclusion criteria. We performed motor tests, dexterity, gait speed, grip strength, and balance tests, using the NIHTBm in patients with CSM and HCs. Motor impairment rates were determined in patients with CSM based on the NIHTBm scores. We determined the association between NIHTBm scores and patient-reported outcome scores; patient-reported outcome measures (the modified Japanese Orthopedic Association [mJOA] and Nurick grade) to determine the association. One-way analysis of variance was used to analyze group differences and the Spearman rank correlation to determine the relationship between assessment scores.

Results

We enrolled 24 patients with CSM with a mean age (SD) of 57.96 (10.61) years and 24 age-matched HCs with a mean age (SD) of 53.17 (6.04) years in this study. Overall, we observed a significant decrease in the motor function T-scores mean (SD): dexterity 31.54 (14.82) vs 51.54 (9.72), grip strength 32.00 (17.47) vs 56.79 (8.46), balance 27.58 (16.65) vs 40.21 (6.35), and gait speed 0.64 (0.18) vs 0.99 (0.17) m/s, in patients with CSM compared with that in HCs. The lower extremity dysfunction scores on the NIHTBm, balance (ρ = -0.67) and gait speed (ρ = -0.62), were associated with higher Nurick grades. We observed a similar but weaker association with the Nurick grades and NIHTBm tests: dexterity (ρ = -0.49) and grip strength (ρ = -0.31) scores. The total motor mJOA showed a positive but weak association with NIHTBm scores, gait speed (ρ = 0.38), balance (ρ = 0.49), grip strength (ρ = 0.41), and dexterity (ρ = 0.45).

Discussion

Patients with CSM had significantly lower NIHTBm scores compared with HCs. The results from the NIHTBm are consistent with the clinical presentation of CSM showing patients have motor impairments in both upper and lower extremities. As a neurologic-specific scale, NIHTBm should be used in the evaluation and clinical management of patients with CSM.

Neurocognitive monitoring in congenital central hypoventilation syndrome with the NIH Toolbox®

Congenital central hypoventilation syndrome (CCHS) is a rare neurocristopathy, caused by mutations in the paired-like homeobox gene PHOX2B, which alters control of breathing and autonomic nervous system regulation, necessitating artificial ventilation as life-support. A broad range of neurocognitive performance has been reported in CCHS, including an array of cognitive deficits. We administered the NIH Toolbox® Cognition Battery (NTCB), a novel technology comprised of seven tasks presented via an interactive computer tablet application, to a CCHS cohort and studied its convergent and divergent validity relative to traditional clinical neurocognitive measures. The NTCB was administered to 51 CCHS participants, including a subcohort of 24 who also received traditional clinical neurocognitive testing (Wechsler Intelligence Scales). Age-corrected NTCB scores from the overall sample and subcohort were compared to population norms. Associations between NTCB indices and Wechsler Intelligence scores were studied to determine the convergent and divergent validity of the NTCB. NTCB test results indicated reduced Fluid Cognition, which measures new learning and speeded information processing (p < 0.001), but intact Crystallized Cognition, which measures past learning, in CCHS relative to population norms. Moderate to strong associations (r > 0.60) were found between age-corrected NTCB Fluid and Crystallized indices and comparable Wechsler indices, supporting the convergent and discriminant validity of the NTCB. Results reveal deficits of Fluid Cognition in individuals with CCHS and indicate that the NTCB is a valid and sensitive measure of cognitive outcomes in this population. Our findings suggest that the NTCB may play a useful role in tracking neurocognition in CCHS.

Assessing Cognitive Decline in High-Functioning Spanish-Speaking Patients: High Score Base Rates on the Spanish-Language NIH Toolbox Cognition Battery

OBJECTIVE

Prior research has focused on the normal frequency of obtaining low scores on a neuropsychological test battery, but few studies have examined the normal frequency of obtaining high scores. This study involved the preparation of high score multivariate base rates for the Spanish-language National Institutes of Health (NIH) Toolbox Cognition Battery (NIHTB-CB).

METHOD

Participants included 250 healthy Latinx adults (age range: 19-80) from the Spanish-language NIHTB-CB normative sample who completed the full battery (two crystallized and four fluid cognition tests). Multivariate base rates, stratified by education, crystallized ability, and sociocultural characteristics, quantified the frequency at which participants obtained one or more "high" fluid test scores (i.e., ≥50th, ≥63rd, ≥75th, ≥84th, ≥91st, ≥95th, and ≥ 98th percentile).

RESULTS

It was common for participants to obtain one or more high scores (i.e., 50.8% obtained one or more scores ≥84th percentile) and uncommon for participants to obtain "no" high scores, especially when using non-conventional thresholds for defining a high score (i.e., 10.8% obtained no scores ≥50th percentile). High scores were more commonly obtained by participants with greater education and higher crystallized ability, who were bilingual, born and educated within the United States, and from higher income households.

CONCLUSIONS

This study demonstrated that participants administered multiple neuropsychological tests commonly obtain high scores, and that the frequency of high scores varies by education, crystallized ability, and sociocultural characteristics. The absence of high scores may be indicative of cognitive impairment among examinees who are higher functioning, have greater education, and have specific sociocultural characteristics (e.g., bilingualism, higher socioeconomic status).

Algorithms for Operationalizing Mild Cognitive Impairment Using the Spanish-Language NIH Toolbox Cognition Battery

OBJECTIVE

Neuropsychologists would benefit from flexible methods for operationalizing psychometric cognitive impairment in Spanish-speaking examinees that vary in association with intelligence, education, and sociocultural characteristics. The current study combined low and high score base rates (BRs) for the Spanish-language NIH Toolbox Cognition Battery (NIHTB-CB) to identify score combinations that are uncommon within different stratifications of the normative sample and may indicate cognitive impairment.

METHOD

The Spanish-language NIHTB-CB normative sample included 250 healthy Latinx adults with complete data on two crystallized and five fluid cognitive tests (M = 38.8 ± 13.7 years old, 72.0% women). Test performances were converted into age-adjusted and demographically adjusted normed scores, adjusting for age, gender, and education. The frequencies at which participants obtained one or more low scores or few to no high scores on fluid cognitive tests were combined into algorithms that occurred at BRs approximately 1 SD (~16%) or 1.5 SDs (~7%) below the mean.

RESULTS

Algorithms are provided for age-adjusted and demographically adjusted scores, with BRs stratified by crystallized ability, education, and sociocultural characteristics. Using demographically adjusted norms, the BR of obtaining any one of the following, 5 scores <50th, 4+ scores ≤25th, 3+ scores ≤16th, or 2+ scores ≤9th percentile, approximates 1 SD below the mean in participants born (BR = 16.2%) or educated abroad (BR = 18.6%), who are monolingual Spanish speakers (BR = 16.4%) or who reside in low-income households (BR = 13.6%).

CONCLUSIONS

These algorithms offer a flexible approach to operationalizing psychometric cognitive impairment, through which different definitions can be applied to different examinees based on varying crystallized ability, education, and sociocultural characteristics.

A Multivariate Interpretation of the Spanish-Language NIH Toolbox Cognition Battery: The Normal Frequency of Low Scores

OBJECTIVE

The current study involved the preparation of multivariate base rates for the Spanish-language NIH Toolbox Cognition Battery (NIHTB-CB) based on the U.S. normative sample, quantifying the normal frequency of low scores among healthy adults.

METHOD

Participants included 250 healthy Latinx adults (M = 38.8 ± 13.7 years old, range: 19-80; 72.0% women; education: M = 11.5 ± 3.9 years) who completed the full Spanish-language NIHTB-CB, including two tests of crystallized cognition and five tests of fluid cognition. Multivariate base rates quantified the frequency at which participants obtained 1 or more fluid scores ≤25th, ≤16th, ≤9th, ≤5th, and ≤2nd percentile, per age-adjusted or demographically adjusted (age, gender, education) normed scores.

RESULTS

A substantial minority of participants had 1 or more low scores (e.g., 40.4% had 1 or more age-adjusted score ≤16th percentile). The frequencies of low scores increased with fewer years of education and lower crystallized cognitive ability. Higher frequencies of low scores were observed among participants who were born and educated abroad, versus within the USA; monolingual Spanish speakers, versus bilingual Spanish/English speakers; and from households below the national median income, versus households above the national median.

CONCLUSION

Low scores were common and related to crystallized ability, education, and sociocultural variables. Although using demographically adjusted scores reduced group differences related to sociocultural variables, group differences were not eliminated, indicating that age, gender, and education score adjustments do not fully explain the associations between sociocultural variables and test performances. These stratified base rates may be useful when working with Spanish-speaking patients with diverse sociocultural characteristics.

Brief iPad-Based Assessment of Cognitive Functioning with ImPACT® Quick Test: Prevalence of Low Scores Using Multivariate Base Rates

OBJECTIVE

The ImPACT® Quick Test is a brief iPad-based battery of neurocognitive tests that has been standardized on a sample of children, adolescents, and adults (ages 12-70). This study provides information on the prevalence of ImPACT® Quick Test scores falling below specific percentiles in the normative sample to aid in clinical interpretation and reduce the risk of over-interpreting, or misinterpreting, a single low score.

METHOD

Participants were 683 individuals ranging in age from 12 to 70, who were assessed individually. The ImPACT® Quick Test includes five subtests, contributing to three factor scores: motor speed, memory, and attention tracker. The prevalence of low factor scores, stratified by age and sex, were calculated using multivariate base rates.

RESULTS

In the total sample, obtaining 1 or more scores below the 25th percentile was common (base rate, BR = 47.2%), but obtaining 2 or more scores in this range was uncommon (BR = 15.3%). Similarly, obtaining 1 or more scores below the 16th percentile was common (BR = 31.4%), but obtaining two or more scores in this range was uncommon (BR = 6.9%). There were small differences in BRs between sexes and the number of low scores was fairly similar across the age groups.

CONCLUSION

Results from this study parallel previous work illustrating that a substantial percentage of healthy individuals will obtain one or more low test scores when administered a brief battery of cognitive tests. Given that some healthy individuals will obtain a single score below expected cut-offs, clinicians should caution against overinterpreting a single low test score.

Patient and clinician experiences of a computerised cognitive battery for use after concussion: a preliminary qualitative study

Objective:

The Cognition Battery of the National Institute of Health (NIH) Toolbox for Assessment of Neurological and Behavioural Function is a computerised neuropsychological battery recommended for clinical practice, neurological research and clinical trials. We investigated the utility of the NIH Toolbox Cognition Battery (NIHTB-CB) for people with concussion.

Methods:

In this small qualitative study, semi-structured interviews were conducted with five adults with concussion who were participating in a larger study using the NIHTB-CB. Three clinician participants and two cultural advisors familiar with the tool were also interviewed. Interview transcripts were analysed using a general thematic approach and qualitative description.

Results:

Participants described both positive and negative experiences with the NIHTB-CB and using qualitative description, their experiences were organised into three broad themes: (1) using technology for cognitive testing made sense, (2) there were some cultural relevance questions and (3) cognitive testing after concussion could have challenges. They were positive about the computerised format and range of domains assessed for the concussion context but identified the contextual relevance of some content as having potential to impact on performances.

Conclusion:

This was a small study examining the experiences of a select group of participants, but nevertheless does suggest a need for future research validating the NIHTB-CB for use in different cultural and clinical contexts.

The clinical utility and construct validity of the NIH Toolbox Cognition Battery (NIHTB-CB) in individuals with disabilities

A State-of-the-Science conference on measurement with disability populations recommended "...the development of cognitive and psychosocial outcome measures, using computer-adaptive testing...that are low in respondent burden and valid across patient populations," (Clohan et al., 2007, p. 1537). Following this recommendation, the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) prioritized the development of measures of cognitive functioning for individuals with disabilities, noting that measures of cognitive functioning "have not been developed for systemic application in the field of medical rehabilitation. Cognition is both a rehabilitation outcome and a factor related to broader functional and community outcomes for individuals with a wide variety of disabling conditions" (Office of Special Education & Rehabilitation Services, 2009, p. 37193). From this came the NIH Toolbox for the Assessment of Neurological and Behavioral Function project (NIH Toolbox) which provides a comprehensive set of cognitive, motor, sensory, and emotional health and function measures for use in clinical, longitudinal, and epidemiological research. The nine papers comprising this special section of Rehabilitation Psychology reflect the sustained collaborative efforts of more than two dozen investigators working at six sites over the past 8 years. They are an initial attempt to validate the NIHTB-CB in disability samples, and they provide initial evidence that the NIHTB-CB can be used with individuals who have TBI, SCI, or stroke. The articles published here reflect the fulfillment of recommendations made during a state-of-the-science conference in 2007. (PsycINFO Database Record

Interpreting Patterns of Low Scores on the NIH Toolbox Cognition Battery

INTRODUCTION

The National Institutes of Health Toolbox for Assessment of Neurological and Behavioral Function Cognition Battery is comprised of seven cognitive tests, including two tests measuring crystallized cognitive ability (i.e., vocabulary and reading) and five tests measuring fluid cognitive functioning (i.e., working memory, memory, speed of processing, and executive functioning). This study presents comprehensive base rate tables for the frequency of low scores in adults and older adults from the normative sample.

METHODS

Participants were 843 adults, ages 20-85, from the NIH Toolbox standardization sample who completed all seven cognition tests. Rates of low scores were derived for standard age-adjusted and fully-demographically-adjusted scores at multiple cut-scores. Base rates were stratified by education, crystallized intellectual ability, and cognitive domain.

RESULTS

Using the five demographically-adjusted fluid cognitive test scores, 45.9% of adults obtained one or more scores at or below the 16th percentile, and 16.8% obtained one or more score at or below the 5th percentile, which is consistent with findings from other neurocognitive test batteries.

DISCUSSION

Based on the study findings, nearly 50% of adults in the general population would meet psychometric criteria for a diagnosis of the Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5) Mild Neurocognitive Disorder (MND). We developed new psychometric criteria for identifying MND using the NIH Toolbox Cognition Battery that reduce the false positive rate. Knowing these multivariate normative base rates will help researchers and clinicians interpret NIH Toolbox scores in people with neurodevelopmental, psychiatric, medical, neurological, and neurodegenerative disorders that affect cognitive functioning.