The clinical utility and construct validity of the NIH Toolbox Cognition Battery (NIHTB-CB) in individuals with disabilities

Design and Harmonization Approach for the Multi-Institutional Neurocognitive Discovery Study (MINDS) of Adult Congenital Heart Disease (ACHD) Neuroimaging Ancillary Study: A Technical Note

Dramatic advances in the management of congenital heart disease (CHD) have improved survival to adulthood from less than 10% in the 1960s to over 90% in the current era, such that adult CHD (ACHD) patients now outnumber their pediatric counterparts. ACHD patients demonstrate domain-specific neurocognitive deficits associated with reduced quality of life that include deficits in educational attainment and social interaction. Our hypothesis is that ACHD patients exhibit vascular brain injury and structural/physiological brain alterations that are predictive of specific neurocognitive deficits modified by behavioral and environmental enrichment proxies of cognitive reserve (e.g., level of education and lifestyle/social habits). This technical note describes an ancillary study to the National Heart, Lung, and Blood Institute (NHLBI)-funded Pediatric Heart Network (PHN) "Multi-Institutional Neurocognitive Discovery Study (MINDS) in Adult Congenital Heart Disease (ACHD)". Leveraging clinical, neuropsychological, and biospecimen data from the parent study, our study will provide structural-physiological correlates of neurocognitive outcomes, representing the first multi-center neuroimaging initiative to be performed in ACHD patients. Limitations of the study include recruitment challenges inherent to an ancillary study, implantable cardiac devices, and harmonization of neuroimaging biomarkers. Results from this research will help shape the care of ACHD patients and further our understanding of the interplay between brain injury and cognitive reserve.

Consensus-Based Evaluation of Outcome Measures in Pediatric Stroke Care: A Toolkit

Following a pediatric stroke, outcome measures selected for monitoring functional recovery and development vary widely. We sought to develop a toolkit of outcome measures that are currently available to clinicians, possess strong psychometric properties, and are feasible for use within clinical settings. A multidisciplinary group of clinicians and scientists from the International Pediatric Stroke Organization comprehensively reviewed the quality of measures in multiple domains described in pediatric stroke populations including global performance, motor and cognitive function, language, quality of life, and behavior and adaptive functioning. The quality of each measure was evaluated using guidelines focused on responsiveness and sensitivity, reliability, validity, feasibility, and predictive utility. A total of 48 outcome measures were included and were rated by experts based on the available evidence within the literature supporting the strengths of their psychometric properties and practical use. Only three measures were found to be validated for use in pediatric stroke: the Pediatric Stroke Outcome Measure, the Pediatric Stroke Recurrence and Recovery Questionnaire, and the Pediatric Stroke Quality of Life Measure. However, multiple additional measures were deemed to have good psychometric properties and acceptable utility for assessing pediatric stroke outcomes. Strengths and weaknesses of commonly used measures including feasibility are highlighted to guide evidence-based and practicable outcome measure selection. Improving the coherence of outcome assessment will facilitate comparison of studies and enhance research and clinical care in children with stroke. Further work is urgently needed to close the gap and validate measures across all clinically significant domains in the pediatric stroke population.

Reliability of the NIH toolbox cognitive battery in children and adolescents: a 3-year longitudinal examination

BACKGROUND

The Cognitive Battery of the National Institutes of Health Toolbox (NIH-TB) is a collection of assessments that have been adapted and normed for administration across the lifespan and is increasingly used in large-scale population-level research. However, despite increasing adoption in longitudinal investigations of neurocognitive development, and growing recommendations that the Toolbox be used in clinical applications, little is known about the long-term temporal stability of the NIH-TB, particularly in youth.

METHODS

The present study examined the long-term temporal reliability of the NIH-TB in a large cohort of youth (9-15 years-old) recruited across two data collection sites. Participants were invited to complete testing annually for 3 years.

RESULTS

Reliability was generally low-to-moderate, with intraclass correlation coefficients ranging between 0.31 and 0.76 for the full sample. There were multiple significant differences between sites, with one site generally exhibiting stronger temporal stability than the other.

CONCLUSIONS

Reliability of the NIH-TB Cognitive Battery was lower than expected given early work examining shorter test-retest intervals. Moreover, there were very few instances of tests meeting stability requirements for use in research; none of the tests exhibited adequate reliability for use in clinical applications. Reliability is paramount to establishing the validity of the tool, thus the constructs assessed by the NIH-TB may vary over time in youth. We recommend further refinement of the NIH-TB Cognitive Battery and its norming procedures for children before further adoption as a neuropsychological assessment. We also urge researchers who have already employed the NIH-TB in their studies to interpret their results with caution.

A novel measure of matching categories for early development: Item creation and pilot feasibility study

BACKGROUND

Many cognitive tests assess a limited developmental span, making longitudinal measurement for trials aimed at improving cognition challenging. Tests targeting transitional skills, which integrate foundational abilities into complex schemas, may be amenable to assessment across a wide developmental span. Furthermore, tablet-based tests permit computer adaptive testing (CAT), which is psychometrically more efficient and could increase testing motivation, especially for children with developmental delays. Such measures may be useful for research and clinical practice.

AIMS

Outline the creation of a novel, tablet-based concept formation test, and evaluate its feasibility in individuals with mental ages less than 24-months.

METHODS AND PROCEDURES

Item generation, user interface construction, and pre-piloting were conducted in consultation with subject matter experts. Item content and interface parameters underwent iterative revisions, resulting in the pilot test.

OUTCOMES AND RESULTS

We created and piloted a tablet-based test of concept formation suitable for CAT-based administration with items of increasing difficulty based on target salience. We show feasibility in individuals with mental ages less than 24-months-old.

CONCLUSIONS AND IMPLICATIONS

Tablet-based assessment of concept formation may be a useful outcome measure of an aspect of cognitive ability in young children. Future work will address optimizing the user interface and developing CAT administration.

Prerequisite Skills in Cognitive Testing: Innovations in theory and recommendations for practice

Testing cognitive skill development is important for diagnostic, prognostic, and monitoring purposes, especially for young children and individuals with neurodevelopmental disorders. Developmental tests have been created for infants and toddlers, while traditional IQ tests are often employed beginning in the later preschool period. However, IQ tests rely on developmental skills that are rapidly changing during early childhood. Here, we introduce the idea of prerequisite skills in developmental domains, which are discrete skills required for, but not explicitly tested by, traditional IQ tests. Focusing on general cognition, particularly among children with a chronological or mental age under 4 years, may fail to capture important nuances in skill development. New skill-based assessments are needed in general, and in particular for categorization, which is foundational to higher-order cognitive skills. Novel measures quantifying categorization skills would provide a more sensitive measure of development for young children and older individuals with low developmental levels.

Reliability and Validity of the Spanish-Language Version of the NIH Toolbox

The psychometric properties of the English-language NIH Toolbox for Assessment of Neurological and Behavioral Function (NIH Toolbox) have been examined in numerous populations. This study evaluated the reliability and validity of the Spanish-language NIH Toolbox. Participants were children aged 3 to 7 years and adults aged 18 to 85 years who took part in the NIH Toolbox norming study in Spanish. Results supported the internal consistency reliability of included measures. Test-retest reliability was strong for most tests, though it was weaker for the test of olfaction among children and the test of locomotion among adults. Spearman's correlations and general linear models showed Spanish tests were often associated with age, sex, and education. Convergent validity for the two language measures that underwent more intensive development, evaluated via Spearman's correlations with legacy measures, was strong. Results support using the Spanish-language NIH Toolbox to measure neurological and behavioral functioning among Spanish-speaking individuals in the United States.

Cognitive, Emotional, and Physical Functioning as Predictors of Paid Employment in People With Stroke, Traumatic Brain Injury, and Spinal Cord Injury

OBJECTIVE

Our objective was to examine demographic, cognitive, emotional, and physical factors that predict return to paid employment for people after neurological injury.

METHOD

Four hundred eighty adults with stroke (n = 149), traumatic brain injury (n = 155), and spinal cord injury (n = 176) completed an occupational outcome questionnaire and physical, emotional, and cognitive assessments at three rehabilitation facilities.

RESULTS

Odds of employment were predicted by being married or partnered, having more education, requiring fewer prompts for task sequencing, and having higher inhibitory control (but were not predicted by specific type of injury). Participants who returned to work within 3 mo were more likely to work with the same employer and to take a full-time position than those who returned later.

CONCLUSION

Executive functioning, in particular sequencing and inhibitory control, strongly predicts employment and highlights the importance of cognitive strategy training during occupational therapy with people who have sustained neurological injuries.

Utility of the NIH Toolbox for assessment of prodromal Alzheimer's disease and dementia

Introduction

The NIH Toolbox Cognition Battery (NIHTB-CB) is a computer-based protocol not yet validated for clinical assessment.

Methods

We administered the NIHTB-CB and traditional neuropsychological tests to 247 Memory Disorders and Alzheimer's Prevention Clinic patients with subjective cognitive decline, mild cognitive impairment, mild dementia due to Alzheimer's disease, and normal cognition. Principal component analysis, partial correlations, and univariate general linear model tests were performed to assess construct validity. Discriminant function analyses compared classification accuracy.

Results

Principal component analysis identified three conceptually coherent factors: memory (MEM_NIH), executive function (EF_NIH), and crystallized intelligence (CI_NIH). These factors were strongly associated with corresponding traditional tests and differed across diagnostic groups as expected. Both NIHTB and traditional batteries yielded strong overall discriminative ability (>80%).

Discussion

The NIHTB-CB is a valid method to assess neurocognitive domains pertinent to aging and dementia and has utility for applications in a memory clinic setting.

Test accommodations for individuals with neurological conditions completing the NIH Toolbox-Cognition Battery: An evaluation of frequency and appropriateness

OBJECTIVES

First, to evaluate the frequency with which individuals with neurological conditions require test administration accommodations for the NIH Toolbox-Cognition Battery (NIHTB-CB). Second, to evaluate the appropriateness of accommodations provided by administrators, including adherence to NIHTB-CB Reasonable Accommodations Guidelines.

METHOD

Adults with traumatic brain injury, spinal cord injury, or stroke (n = 604) completed the NIHTB-CB and other assessments as part of a multisite study. We provide a descriptive, secondary analysis of test administrator notes to determine use and appropriateness of accommodations.

RESULTS

Of the 604 participants, 450 (75%) completed the NIHTB-CB using standard administration procedures, but 137 (22.6%) encountered accessibility challenges that required accommodations. Participants with motor function impairments were most likely to receive at least 1 of 3 kinds of accommodations: (a) use of nonstandard methods of entering responses using standard input devices, (b) use of alternate input devices, or (c) help from the test administrator to enter a response. Fatigue and/or impulsivity led to nonstandard administration by 48 (7.9%) individuals. Post hoc audit of test administrator notes revealed that despite careful instructions and supervision, 49 (56.3%) of the accommodated administrations breached standardization and scores could not be interpreted using test norms.

CONCLUSION

Although the NIHTB-CB was developed for individuals without neurological impairment, most individuals with neurological conditions completed the standardized administration without accommodations. When accommodations were needed, administrators did not adhere to the official Reasonable Accommodations Guidelines in more than half of the cases. (PsycINFO Database Record

Construct validity of the NIH Toolbox Cognition Battery in individuals with stroke

OBJECTIVE

The National Institutes of Health (NIH) Toolbox (NIHTB) for the Assessment of Behavior and Neurological Function Cognition Battery (NIHTB-CB) provides a brief assessment (approximately 30 min) of key components of cognition. This article examines construct validity to support the clinical utility of the NIHTB-CB in individuals with stroke.

RESEARCH METHOD

A total of 131 individuals with stroke (n = 71 mild stroke; n = 60 moderate/severe stroke) completed the NIHTB-CB. Univariate analyses were conducted to examine the cognitive profiles of the two different stroke groups (mild vs. moderate/severe stroke) on NIHTB-CB measures and composite scores. Pearson correlations were conducted between NIHTB-CB and established measures to examine convergent and discriminant validity. Effect sizes and clinical impairment rates for the different NIHTB-CB measures and composite scores were also examined.

RESULTS

Participants experiencing moderate to severe stroke had poorer performance than did individuals with mild stroke on several of the NIHTB cognition measures. Evidence of convergent validity was provided by moderate to strong correlations between the NIHTB measures and the corresponding standard neuropsychological test (Pearson rs ranged from 0.31 to 0.88; median = .60). Evidence of discriminant validity was provided by smaller correlations between different cognitive domains than correlations of measures within the same domain. Effect sizes for composite and subtest scores regarding stroke severity were generally moderate-to-large. In addition, 42% of the sample were exhibiting mild cognitive impairment (i.e., ≥2 low scores on fluid tests).

CONCLUSIONS

Findings provide support for the construct validity of the NIHTB-CB in individuals with stroke. (PsycINFO Database Record

Factor structure of the NIH Toolbox Cognition Battery in individuals with acquired brain injury

OBJECTIVE

The National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) measures reading, vocabulary, episodic memory, working memory, executive functioning, and processing speed. While previous research has validated the factor structure in healthy adults, the factor structure has not been examined in adults with neurological impairments. Thus, this study evaluated the NIHTB-CB factor structure in individuals with acquired brain injury.

METHOD

A sample of 392 individuals (ages 18-84) with acquired brain injury (n = 182 TBI, n = 210 stroke) completed the NIHTB-CB along with neuropsychological tests as part of a larger, multisite research project.

RESULTS

Confirmatory factor analyses supported a 5-factor solution that included reading, vocabulary, episodic memory, working memory, and processing speed/executive functioning. This structure generally held in TBI and stroke subsamples as well as in subsamples of those with severe TBI and stroke injuries.

CONCLUSIONS

The factor structure of the NIHTB-CB is similar in adults with acquired brain injury to adults from the general population. We discuss the implications of these findings for clinical practice and clinical research. (PsycINFO Database Record