Test accommodations for individuals with neurological conditions completing the NIH Toolbox-Cognition Battery: An evaluation of frequency and appropriateness

Normative Reference Values for Crystallized-Fluid Discrepancy Scores for the NIH Toolbox Cognition Battery

INTRODUCTION

The purpose of this study was to translate NIH Toolbox Cognition Battery (NIHTB-CB) Crystallized-Fluid discrepancy scores into research and clinical practice with adults by providing normative data for discrepancy scores for both age-adjusted standard scores (SSs) and demographically adjusted T-scores.

METHOD

We included adult participants from the NIHTB-CB standardization sample who denied having neurodevelopmental, medical, psychiatric, or neurological conditions (n = 730; M = 47.4 years old, SD = 17.6, range: 18-85; 64.4% women; 63.1% White). Descriptive statistics were calculated for the Fluid and Crystallized composite scores and Crystallized-Fluid discrepancy score, along with correlations between the composite scores and reliability estimates of the discrepancy score. Percentiles were calculated for the discrepancy score, with stratifications by the gender, education, and Crystallized composite for the age-adjusted SSs and demographically adjusted T-scores (T).

RESULTS

Crystallized-Fluid discrepancy scores ranged from -40 to 44 (M = -0.63, SD = 14.89, Mdn = -1, interquartile range [IQR]: -11 to 10) for age-adjusted SSs and from -29 to 27 (M = -0.39, SD = 10.49, Mdn = -1, IQR = -8 to 7) for demographically adjusted T-scores. Crystallized-Fluid discrepancy scores of SS = 15 and T = 11 were at the 16th percentile (1 SD below the mean) and discrepancy scores of SS = 21 and T = 15 were at the 7th percentile (1.5 SD below the mean).

CONCLUSIONS

Crystallized-Fluid discrepancy scores may be, with future research, a useful within-person interpretive approach for detecting a decline from pre-injury or pre-disease levels of cognitive functioning. These normative reference values assist clinicians and researchers in determining the frequency at which given Crystallized-Fluid discrepancy scores occurred among healthy adults in the normative sample.

Uses of the NIH Toolbox® in Clinical Samples: A Scoping Review

Background and Objectives

The NIH Toolbox® for the Assessment of Neurologic and Behavioral Function is a compilation of computerized measures designed to assess sensory, motor, emotional, and cognitive functioning of individuals across the life span. The NIH Toolbox was initially developed for use with the general population and was not originally validated in clinical populations. The objective of this scoping review was to assess the extent to which the NIH Toolbox has been used with clinical populations.

Methods

Guided by the Joanna Briggs Methods Manual for Scoping Reviews, records were identified through searches of PubMed MEDLINE, PsycINFO, ClinicalTrials.gov, EMBASE, and ProQuest Dissertations and Theses Global (2008-2020). Database searches yielded 5,693 unique titles of original research that used at least one NIH Toolbox assessment in a sample characterized by any clinical diagnosis. Two reviewers screened titles, abstracts, and full texts for inclusion in duplicate. Conflicts at each stage of the review process were resolved by a group discussion.

Results

Ultimately, 281 publication records were included in this scoping review (nJournal Articles = 104, nConference Abstracts = 84, nClinical Trial Registrations = 86, and nTheses/Dissertations = 7). The NIH Toolbox Cognition Battery was by far the most used of the 4 batteries in the measurement system (nCognition = 225, nEmotion = 49, nMotor = 29, and nSensation = 16). The most represented clinical category was neurologic disorders (n = 111), followed by psychological disorders (n = 39) and cancer (n = 31). Most (96.8%) of the journal articles and conference abstracts reporting the use of NIH Toolbox measures with clinical samples were published in 2015 or later. As of May 2021, these records had been cited a total of nearly 1,000 times.

Discussion

The NIH Toolbox measures have been widely used among individuals with various clinical conditions across the life span. Our results lay the groundwork to support the feasibility and utility of administering the NIH Toolbox measures in research conducted with clinical populations and further suggest that these measures may be of value for implementation in fast-paced clinical settings as part of routine practice.

Assessment of language impairment and function

This chapter is written for the qualified neurologist or related professional working with persons who have had a stroke or other sudden brain injury. It is critical that the presence of aphasia is detected, no matter how mild the presentation, and to support that assertion, this chapter highlights the plight of persons with latent aphasia. At the individual level, the impact of aphasia is devastating, with overwhelming evidence that aphasia negatively impacts psychosocial outcomes. At the global level, sensitive detection and accurate diagnosis of aphasia are critical for accurate characterization and quantification of the global burden of aphasia. The word "LANGUAGE" is leveraged as an acronym to create a useful and memorable checklist to guide navigation of aphasia screening and assessment: it begins with the definition of language (L), followed by the definition and diagnostic criteria for aphasia (A). Then language abilities and characteristics to be considered in assessment are presented: naming (N); grammar and syntax (G); unintelligible words, jargon, and paraphasias (U); auditory comprehension and repetition (A); graphemic abilities-reading and writing (G); and everyday communication and discourse (E). Recommendations for improving procedural adherence are provided, and a list of potential brief assessment measures are introduced.

A novel measure of matching categories for early development: Item creation and pilot feasibility study

BACKGROUND

Many cognitive tests assess a limited developmental span, making longitudinal measurement for trials aimed at improving cognition challenging. Tests targeting transitional skills, which integrate foundational abilities into complex schemas, may be amenable to assessment across a wide developmental span. Furthermore, tablet-based tests permit computer adaptive testing (CAT), which is psychometrically more efficient and could increase testing motivation, especially for children with developmental delays. Such measures may be useful for research and clinical practice.

AIMS

Outline the creation of a novel, tablet-based concept formation test, and evaluate its feasibility in individuals with mental ages less than 24-months.

METHODS AND PROCEDURES

Item generation, user interface construction, and pre-piloting were conducted in consultation with subject matter experts. Item content and interface parameters underwent iterative revisions, resulting in the pilot test.

OUTCOMES AND RESULTS

We created and piloted a tablet-based test of concept formation suitable for CAT-based administration with items of increasing difficulty based on target salience. We show feasibility in individuals with mental ages less than 24-months-old.

CONCLUSIONS AND IMPLICATIONS

Tablet-based assessment of concept formation may be a useful outcome measure of an aspect of cognitive ability in young children. Future work will address optimizing the user interface and developing CAT administration.

Prerequisite Skills in Cognitive Testing: Innovations in theory and recommendations for practice

Testing cognitive skill development is important for diagnostic, prognostic, and monitoring purposes, especially for young children and individuals with neurodevelopmental disorders. Developmental tests have been created for infants and toddlers, while traditional IQ tests are often employed beginning in the later preschool period. However, IQ tests rely on developmental skills that are rapidly changing during early childhood. Here, we introduce the idea of prerequisite skills in developmental domains, which are discrete skills required for, but not explicitly tested by, traditional IQ tests. Focusing on general cognition, particularly among children with a chronological or mental age under 4 years, may fail to capture important nuances in skill development. New skill-based assessments are needed in general, and in particular for categorization, which is foundational to higher-order cognitive skills. Novel measures quantifying categorization skills would provide a more sensitive measure of development for young children and older individuals with low developmental levels.

Motor demands of cognitive testing may artificially reduce executive function scores in individuals with spinal cord injury

Objective: To determine whether the motor demands of cognitive tests contribute to differences in cognitive function scores in participants with and without spinal cord injury (SCI).Design: Cohort study.Setting: Rehabilitation research laboratory.Participants: 68 individuals without SCI ("NON") and 22 individuals with motor complete SCI ("SCI").Interventions: None.Outcome Measures: NIH Toolbox cognitive assessments, including two with motor demands and reaction-time based scoring (Dimensional Change Card Sort (DCCS), Flanker Inhibitory Control and Attention (Flanker) and two without timed scoring (List Sorting Working Memory (List Sorting), Picture Sequence Memory Test (Picture Sequence). Tests were administered with and without the assistance of a proctor on two randomly-determined days (>24 hr interval). For DCCS and Flanker, the motor-task score offset was estimated as the difference between the proctored and non-proctored scores.Results: For demographically-corrected data, proctoring reduced DCCS and Flanker scores (P < 0.001) but mitigated apparent differences between SCI and NON (all P > 0.403). SCI and NON did not differ for List Sorting (P > 0.072) but did differ significantly for Picture Sequence (P < 0.001). Significant practice effects existed for memory-based tests (List Sorting and Picture Sequence); all P < 0.015, effect size>0.645.Conclusions: DCCS and Flanker scores for individuals with SCI may be artificially reduced consequent to secondary motor demands of the tests. Proctoring and computation of a motor-response score offset enables comparisons to be made between individuals with SCI and a Non-SCI control cohort; however, further work is needed to determine whether offset-adjusted scores can be compared to standardized normative values.

Accessibility Considerations in the National Children's Study

In the National Children's Study (NCS), assessments were proposed and developed that used a wide range of modes of administration (e.g., direct in-person interviews, telephone interviews, computer assisted interviews, self-administered questionnaires, real time and recall observations, and physical examinations). These modes of administration may pose accessibility challenges for some people with disabilities. Accessibility of measurement is important to consider because systematic exclusion of people with disabilities from research can lead to measurement bias and systematic error in derived scores. We describe our approach to analyzing the accessibility of measures in the NCS and describe the work of the Accessibility Domain Team. Finally, we describe a decision process for creating and using accessible health research measures.

Using the NIH Toolbox Cognition Battery (NIHTB-CB) in individuals with traumatic brain injury

PURPOSE/OBJECTIVE

The NIH Toolbox for the Assessment of Neurological Behavior and Function Cognition Battery (NIHTB-CB) is a common data element for use in individuals with traumatic brain injury (TBI). This study evaluates its sensitivity and specificity in distinguishing individuals with complicated mild, moderate, or severe TBI, and provides support for the construct validity of the NIHTB-CB in individuals with TBI.

RESEARCH METHOD

One hundred eighty-two individuals with TBI (n = 83 complicated mild/moderate; n = 99 severe) completed the NIHTB-CB and neuropsychological criterion measures. Complete data were obtained on 158 participants. A control sample of 158 individuals without known neurological impairment was extracted from the NIHTB-CB normative sample. Multivariate analyses of variance determined the sensitivity of the NIHTB-CB measures to TBI and injury severity (complicated mild/moderate TBI, severe TBI, and controls) on the demographically corrected NIHTB-CB composite scores and seven subtests. A descriptive analysis of the sensitivity of each subtest was conducted. Finally, correlations between NIHTB-CB measures and criterion tests assessed convergent and discriminant validity.

RESULTS

Multivariate analyses indicated that there was a main effect for group (complicated mild/moderate vs. severe vs. controls) for fluid scores in the NIHTB-CB as opposed to only marginally significant results for the verbal scores. Moderate to strong relationships were found between the NIHTB-CB measures and their corresponding neuropsychological measures (convergent validity), whereas much smaller correlations were found between measures of different cognitive domains (discriminant validity).

CONCLUSIONS

Findings provide evidence of construct validity and the clinical utility of the NIHTB-CB in individuals with TBI. (PsycINFO Database Record

Cognition among community-dwelling individuals with spinal cord injury

PURPOSE/OBJECTIVE

To compare the cognitive profiles of a well-characterized sample of adults with and without spinal cord injury (SCI) using the NIH Toolbox Cognition Battery NIHTB-CB. Research Method/Design: Participants were 156 community-dwelling individuals with SCI recruited from 3 academic medical centers, and 156 individuals without SCI selected from the NIHTB-CB normative database (N = 312). The main outcome measures were the demographically adjusted NIHTB-CB subtest and composite scores.

RESULTS

Individuals with and without SCI performed equivalently on the NIHTB-CB crystallized composite score, suggesting comparable premorbid functioning. Individuals with SCI produced lower scores on the NIHTB-CB fluid composite score by an average of 4.5 T-score points (Cohen's d = 0.50; a medium effect size). As a group, individuals with SCI had the most difficulty on tests of processing speed and executive functions, and some difficulty on a test of episodic memory, although effect sizes were small. These differences remained even after accounting for fine motor speed and dexterity. Individuals with tetraplegia produced lower scores than individuals with paraplegia on tests of processing speed and executive functioning.

CONCLUSION/IMPLICATIONS

Community-dwelling individuals with SCI are at elevated risk of mild cognitive difficulties, particularly on tasks that rely on processing speed and executive functions. The NIHTB-CB is relatively brief, samples important cognitive domains, has good normative data, and is appropriate for some individuals with SCI (those who have functional use of one hand). The battery has standardized accommodations for individuals with minor motor limitations, but timed tests are inaccessible for individuals who are unable to perform rapid button presses. (PsycINFO Database Record

Motor-free composites from the National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) for people with disabilities

PURPOSE/OBJECTIVE

The National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) includes a group of brief measures (i.e., 30 min) designed to assess language, processing speed, working memory, episodic memory, and executive functioning. These subtests can be combined to create composite scores that reflect fluid and crystallized cognition, as well as overall cognition. The battery is of limited utility with individuals who have impaired upper extremity motor functioning. This manuscript examines the accuracy of the Oral Symbol Digit Modalities Test as a substitute for the Pattern Comparison Processing Speed Test for computing motor-free composite scores. Research Method/Design: Individuals with spinal cord injury (SCI; n = 188), traumatic brain injury (TBI; n = 159), or stroke (n = 180) completed the NIHTB-CB. We used the Oral Symbol Digit Modalities Test to create a Motor-Free Pattern Comparison score; this was used to create revised, Motor-Free Composite scores for Fluid Cognition and Overall Cognition.

RESULTS

Although there were statistically significant overall differences between the two Fluid and Overall Cognition composite scores for some of the clinical groups (scores based on the motor-free approach were significantly higher than the original score), these differences were small and partly because of overclassification of impaired processing speed in participants with motor impairment. There was good to substantial agreement with regard to "impairment" classification between the two sets of Original and Motor-Free composite scores.

CONCLUSIONS/IMPLICATIONS

Although the Motor-Free scores are not a perfect match for the Original Composite scores, they provide a reliable and valid way to examine overall and fluid cognition in individuals with upper extremity motor impairments. (PsycINFO Database Record

The clinical utility and construct validity of the NIH Toolbox Cognition Battery (NIHTB-CB) in individuals with disabilities

A State-of-the-Science conference on measurement with disability populations recommended "...the development of cognitive and psychosocial outcome measures, using computer-adaptive testing...that are low in respondent burden and valid across patient populations," (Clohan et al., 2007, p. 1537). Following this recommendation, the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) prioritized the development of measures of cognitive functioning for individuals with disabilities, noting that measures of cognitive functioning "have not been developed for systemic application in the field of medical rehabilitation. Cognition is both a rehabilitation outcome and a factor related to broader functional and community outcomes for individuals with a wide variety of disabling conditions" (Office of Special Education & Rehabilitation Services, 2009, p. 37193). From this came the NIH Toolbox for the Assessment of Neurological and Behavioral Function project (NIH Toolbox) which provides a comprehensive set of cognitive, motor, sensory, and emotional health and function measures for use in clinical, longitudinal, and epidemiological research. The nine papers comprising this special section of Rehabilitation Psychology reflect the sustained collaborative efforts of more than two dozen investigators working at six sites over the past 8 years. They are an initial attempt to validate the NIHTB-CB in disability samples, and they provide initial evidence that the NIHTB-CB can be used with individuals who have TBI, SCI, or stroke. The articles published here reflect the fulfillment of recommendations made during a state-of-the-science conference in 2007. (PsycINFO Database Record

Interdisciplinary Approach to the Development of Accessible Computer-Administered Measurement Instruments

Principles of fairness in testing require that all test takers, including people with disabilities, have an equal opportunity to demonstrate their capacity on the construct being measured. Measurement design features and assessment protocols can pose barriers for people with disabilities. Fairness in testing is a fundamental validity issue at all phases in the design, administration, and interpretation of measurement instruments in clinical practice and research. There is limited guidance for instrument developers on how to develop and evaluate the accessibility and usability of measurement instruments. This article describes a 6-stage iterative process for developing accessible computer-administered measurement instruments grounded in the procedures implemented across several major measurement initiatives. A key component of this process is interdisciplinary teams of accessibility experts, content and measurement experts, information technology experts, and people with disabilities working together to ensure that measurement instruments are accessible and usable by a wide range of users. The development of accessible measurement instruments is not only an ethical requirement, it also ensures better science by minimizing measurement bias, missing data, and attrition due to mismatches between the target population and test administration platform and protocols.