The NIH Toolbox: Overview of Development for Use with Hispanic Populations

Interpreting reliable change on the Spanish-language NIH toolbox cognition battery

This study applied a reliable change methodology to the test-retest data from the Spanish-language NIH Toolbox Cognition Battery (NIHTB-CB) normative sample. Participants included Spanish-speaking adults (n = 48; 54.2% women, 100% Latinx) evaluated twice within one to two weeks on the Spanish-language NIHTB-CB, consisting of two crystallized and five fluid cognitive tests. Test-retest means, standard deviations, and intraclass correlations were used to calculate upper and lower bounds of 70, 80, and 90% confidence intervals (CIs) around change scores, with these bounds used as cutoffs for inferring reliable change. Cutoffs were calculated for raw scores, age-adjusted standard scores (SS; M = 100, SD = 15), and demographic-adjusted T-scores (T; M = 50, SD = 10), adjusting for age, gender, and education. Test-retest change scores on the Spanish-language NIHTB-CB exceeding the following cutoffs indicate reliable change based on an 80% CI (i.e., values exceeding these cutoffs indicate greater decline or greater improvement than 90% of the sample): Dimensional Change Card Sort (SS ≥ 15/T ≥ 11), Flanker (SS ≥ 13/T ≥ 10), List Sorting (SS ≥ 13/T ≥ 9), Picture Sequence Memory (SS ≥ 14/T ≥ 9), Pattern Comparison (SS ≥ 14/T ≥ 10), Picture Vocabulary (SS ≥ 8/T ≥ 6), Oral Reading (SS ≥ 7/T ≥ 5), Fluid Cognition Composite (SS ≥ 12/T ≥ 9), Crystallized Cognition Composite (SS ≥ 6/T ≥ 5), and Total Cognition Composite (SS ≥ 8/T ≥ 7). These cutoffs are one of few resources to interpret cognitive change at retest among Spanish-speaking patients and participants.

Performance Validity Testing on the NIH Toolbox Cognition Battery: Base Rates of Failed Embedded Validity Indicators in the Adult Normative Sample

OBJECTIVE

The goal of this study was to determine the base rates of failing proposed embedded validity indicators (EVIs) for the National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) in the normative sample.

METHOD

Participants included adults in the NIHTB-CB normative sample with data to calculate age-adjusted standard scores (n = 855; ages: M(SD) = 46.9(17.3), range: 18-85; 65.0% women; education: M(SD) = 14.1(2.5) years) or demographically adjusted T-scores (n = 803; ages: M(SD) = 47.3(17.3), range: 18-85; 65.3% women; education: M(SD) = 14.2(2.5) years) for all tests. The NIHTB-CB includes two tests of crystallized cognition and five tests of fluid cognition. Individual norm-referenced test performances were categorized as falling above or below liberal and conservative cutoffs based on proposed univariate EVIs. The number of univariate EVI failures was summed to compute multivariable EVIs. EVI failure rates above 10% were considered high false-positive rates, indicating specificity < .90. Using chi-square analyses, the frequencies of EVI failures were compared based on gender, race/ethnicity, education, and crystallized composite.

RESULTS

The multivariable EVIs had predominantly low false-positive rates in the normative sample. EVI failure rates were most common among participants with low crystallized composites. Using age-adjusted standard scores, EVI failure rates varied by education, race/ethnicity, and estimated premorbid intelligence. These differences were mostly eliminated when using demographically adjusted T-scores.

CONCLUSIONS

Multivariable EVIs requiring ≥ 4 failures using liberal cutoffs or ≥ 3 failures using conservative cutoffs had acceptable false-positive rates (i.e., < 10%) using both age-adjusted standard scores and demographically adjusted T-scores. These multivariable EVIs could be applied to large data sets with NIHTB-CB data to screen for potentially invalid test performances.

A psychometric evaluation of the NIH Toolbox fluid cognition tests adapted for Swahili and Dholuo languages in Kenyan children and adolescents

OBJECTIVE

Our objective was to evaluate the psychometric properties of the culturally adapted NIH Toolbox African Languages® when used in Swahili and Dholuo-speaking children in western Kenya.

METHOD

Swahili-speaking participants were recruited from Eldoret and Dholuo-speaking participants from Ajigo; all were <14 years of age and enrolled in primary school. Participants completed a demographics questionnaire and five fluid cognition tests of the NIH Toolbox® African Languages program, including Flanker, Dimensional Change Card Sort (DCCS), Picture Sequence Memory, Pattern Comparison, and List Sorting tests. Statistical analyses examined aspects of reliability, including internal consistency (in both languages) and test-retest reliability (in Dholuo only).

RESULTS

Participants included 479 children (n = 239, Swahili-speaking; n = 240, Dholuo-speaking). Generally, the tests had acceptable psychometric properties for research use within Swahili- and Dholuo-speaking populations (mean age = 10.5; SD = 2.3). Issues related to shape identification and accuracy over speed limited the utility of DCCS for many participants, with approximately 25% of children unable to match based on shape. These cultural differences affected outcomes of reliability testing among the Dholuo-speaking cohort, where accuracy improved across all five tests, including speed.

CONCLUSIONS

There is preliminary evidence that the NIH Toolbox ® African Languages potentially offers a valid assessment of development and performance using tests of fluid cognition in Swahili and Dholuo among research settings. With piloting underway across other diverse settings, future research should gather additional evidence on the clinical utility and acceptability of these tests, specifically through the establishment of norming data among Kenyan regions and evaluating these psychometric properties.

Neurocognition as a biomarker in the rare autonomic disorders of CCHS and ROHHAD

PURPOSE

Congenital central hypoventilation syndrome (CCHS) and rapid-onset obesity with hypothalamic dysfunction, hypoventilation, and autonomic dysregulation (ROHHAD) are rare disorders of autonomic regulation with risk for disrupted neurocognitive development. Our aim is to summarize research on neurocognitive outcomes in these conditions, advance understanding of how to best support these individuals throughout development, and facilitate future research.

METHODS

We conducted a narrative review of literature on neurocognitive outcomes in CCHS and ROHHAD, supplemented with previously unpublished data from patients with CCHS and ROHHAD at our Center for Autonomic Medicine in Pediatrics (CAMP).

RESULTS

Individuals with CCHS and ROHHAD experience a wide range of neurocognitive functioning ranging from above average to below average, but are at particular risk for difficulties with working memory, processing speed, perceptual reasoning, and visuographic skills. An assessment framework emphasizing fluid cognition seems especially appropriate for these conditions. Owing to small cohorts and varied methods of data collection, it has been difficult to identify associations between disease factors (including CCHS PHOX2B genotypes) and cognitive outcomes. However, results suggest that early childhood is a period of particular vulnerability, perhaps due to the disruptive impact of recurrent intermittent hypoxic episodes on brain and cognitive development.

CONCLUSION

Neurocognitive monitoring is recommended as a component of routine clinical care in CCHS and ROHHAD as a marker of disease status and to ensure that educational support and disability accommodations are provided as early as possible. Collaborative efforts will be essential to obtain samples needed to enhance our understanding of neurocognitive outcomes in CCHS and ROHHAD.

Algorithms for Operationalizing Mild Cognitive Impairment Using the Spanish-Language NIH Toolbox Cognition Battery

OBJECTIVE

Neuropsychologists would benefit from flexible methods for operationalizing psychometric cognitive impairment in Spanish-speaking examinees that vary in association with intelligence, education, and sociocultural characteristics. The current study combined low and high score base rates (BRs) for the Spanish-language NIH Toolbox Cognition Battery (NIHTB-CB) to identify score combinations that are uncommon within different stratifications of the normative sample and may indicate cognitive impairment.

METHOD

The Spanish-language NIHTB-CB normative sample included 250 healthy Latinx adults with complete data on two crystallized and five fluid cognitive tests (M = 38.8 ± 13.7 years old, 72.0% women). Test performances were converted into age-adjusted and demographically adjusted normed scores, adjusting for age, gender, and education. The frequencies at which participants obtained one or more low scores or few to no high scores on fluid cognitive tests were combined into algorithms that occurred at BRs approximately 1 SD (~16%) or 1.5 SDs (~7%) below the mean.

RESULTS

Algorithms are provided for age-adjusted and demographically adjusted scores, with BRs stratified by crystallized ability, education, and sociocultural characteristics. Using demographically adjusted norms, the BR of obtaining any one of the following, 5 scores <50th, 4+ scores ≤25th, 3+ scores ≤16th, or 2+ scores ≤9th percentile, approximates 1 SD below the mean in participants born (BR = 16.2%) or educated abroad (BR = 18.6%), who are monolingual Spanish speakers (BR = 16.4%) or who reside in low-income households (BR = 13.6%).

CONCLUSIONS

These algorithms offer a flexible approach to operationalizing psychometric cognitive impairment, through which different definitions can be applied to different examinees based on varying crystallized ability, education, and sociocultural characteristics.

A methodology to engineering continuous monitoring of intrinsic capacity for elderly people

Continuous monitoring of the well-being state of elderly people is about to become an urgent need in the early future due to population aging. Aiming a unified notion of well-being, we find the Intrinsic Capacity concept in accordance with the SMART BEAR project goals. In this study, we mainly focus on the enabling infrastructure, mapping our models to interoperable repositories and to streaming/computing components that can foster monitoring. Our method is also innovative for explicitly combining personalized and risk levels in generating the Intrinsic Capacity score. Leveraging on synthetic data, we represent the outcome trajectories of some sample patients for 1-year continuous monitoring and discuss approaches to characterize them based on the exhibited tendency and evaluate the results from the predictability point of view providing by the entropy of time series concept. At the end, we discuss the possible data quality issues in health care studies using synthetic data.

Translation and Cultural Adaptation of NIH Toolbox Cognitive Tests into Swahili and Dholuo Languages for Use in Children in Western Kenya

OBJECTIVES

Performing high-quality and reliable cognitive testing requires significant resources and training. As a result, large-scale studies involving cognitive testing are difficult to perform in low- and middle-income settings, limiting access to critical knowledge to improve academic achievement and economic production in these populations. The NIH Toolbox® is a collection of cognitive, motor, sensory, and emotional tests that can be administered and scored using an iPad® tablet, reducing the need for training and quality monitoring; and thus, it is a potential solution to this problem.

METHODS

We describe our process for translation and cultural adaptation of the existing NIH Toolbox tests of fluid cognition into the Swahili and Dholuo languages for use in children aged 3-14 years in western Kenya. Through serial forward and back translations, cognitive interviews, group consensus, outside feedback, and support from the NIH Toolbox team, we produced translated tests that have both face validity and linguistic validation.

RESULTS

During our cognitive interviews, we found that the five chosen tests (one each of attention, cognitive flexibility, working memory, episodic memory, and processing speed) were generally well understood by children aged 7-14 years in our chosen populations. The cognitive interviews informed alterations in translation as well as slight changes in some images to culturally adapt the tests.

CONCLUSIONS

This study describes the process by which we translated five fluid cognition tests from the NIH Toolbox into the Swahili and Dholuo languages. The finished testing application will be available for future studies, including a pilot study for assessment of psychometric properties.

Assessing Cognitive Decline in High-Functioning Spanish-Speaking Patients: High Score Base Rates on the Spanish-Language NIH Toolbox Cognition Battery

OBJECTIVE

Prior research has focused on the normal frequency of obtaining low scores on a neuropsychological test battery, but few studies have examined the normal frequency of obtaining high scores. This study involved the preparation of high score multivariate base rates for the Spanish-language National Institutes of Health (NIH) Toolbox Cognition Battery (NIHTB-CB).

METHOD

Participants included 250 healthy Latinx adults (age range: 19-80) from the Spanish-language NIHTB-CB normative sample who completed the full battery (two crystallized and four fluid cognition tests). Multivariate base rates, stratified by education, crystallized ability, and sociocultural characteristics, quantified the frequency at which participants obtained one or more "high" fluid test scores (i.e., ≥50th, ≥63rd, ≥75th, ≥84th, ≥91st, ≥95th, and ≥ 98th percentile).

RESULTS

It was common for participants to obtain one or more high scores (i.e., 50.8% obtained one or more scores ≥84th percentile) and uncommon for participants to obtain "no" high scores, especially when using non-conventional thresholds for defining a high score (i.e., 10.8% obtained no scores ≥50th percentile). High scores were more commonly obtained by participants with greater education and higher crystallized ability, who were bilingual, born and educated within the United States, and from higher income households.

CONCLUSIONS

This study demonstrated that participants administered multiple neuropsychological tests commonly obtain high scores, and that the frequency of high scores varies by education, crystallized ability, and sociocultural characteristics. The absence of high scores may be indicative of cognitive impairment among examinees who are higher functioning, have greater education, and have specific sociocultural characteristics (e.g., bilingualism, higher socioeconomic status).

Cognitive and socio-emotional correlates of psychological well-being and mental health in Guatemalan adults

BACKGROUND

Little is known about associations of psychological and mental well-being with cognitive and socioemotional factors in low and middle-income countries, particularly among vulnerable populations born in adverse environments that may restrict developmental potential. This study aimed to examine the cognitive and socioemotional correlates of psychological well-being and mental health in a cohort of Guatemalan adults born in contexts of poverty and malnutrition.

METHODS

From Dec 2017 to Apr 2019, data were collected from 704 women and 564 men ages 40-57 years living in four rural villages in eastern Guatemala and Guatemala City. We measured latent domains of psychological well-being, spirituality and religion, emotional support, and executive function using Confirmatory Factor Analysis (CFA). Under a Structural Equation Modeling framework, we examined intercorrelations among latent domains and observed measures of intelligence and mental health.

RESULTS

CFA supported the construct validity of factor structures in this population. Correlations of psychological well-being with spirituality and religion were moderate in women (r = 0.68, p < 0.001) and men (r = 0.70, p < 0.001). Executive function was weakly correlated with psychological well-being in men (r = 0.23, p < 0.001) and showed no association in women. Correlations of psychological well-being with emotional support and IQ were weak in women (r = 0.34, and r = 0.15, respectively; p < 0.001 for both) and men (r = 0.35, and r = 0.25, respectively; p < 0.001 for both). Mental health and IQ were weakly correlated in men (r = 0.09, p < 0.05) and showed no association in women. Mental health showed weak correlations with emotional support (r = 0.18, p < 0.001 in women; r = 0.09, p < 0.05 in men), psychological well-being (r = 0.32 and r = 0.35, in women and men respectively; p < 0.001 for both) and showed no association with executive function in both sexes.

CONCLUSIONS

Of all examined factors, spirituality and religion made the greatest contribution to psychological well-being. These findings support the notion that in populations experiencing difficult circumstances, religion can perhaps make a greater contribution to well-being and aid coping. More research is needed to examine mediators of this association.

A Multivariate Interpretation of the Spanish-Language NIH Toolbox Cognition Battery: The Normal Frequency of Low Scores

OBJECTIVE

The current study involved the preparation of multivariate base rates for the Spanish-language NIH Toolbox Cognition Battery (NIHTB-CB) based on the U.S. normative sample, quantifying the normal frequency of low scores among healthy adults.

METHOD

Participants included 250 healthy Latinx adults (M = 38.8 ± 13.7 years old, range: 19-80; 72.0% women; education: M = 11.5 ± 3.9 years) who completed the full Spanish-language NIHTB-CB, including two tests of crystallized cognition and five tests of fluid cognition. Multivariate base rates quantified the frequency at which participants obtained 1 or more fluid scores ≤25th, ≤16th, ≤9th, ≤5th, and ≤2nd percentile, per age-adjusted or demographically adjusted (age, gender, education) normed scores.

RESULTS

A substantial minority of participants had 1 or more low scores (e.g., 40.4% had 1 or more age-adjusted score ≤16th percentile). The frequencies of low scores increased with fewer years of education and lower crystallized cognitive ability. Higher frequencies of low scores were observed among participants who were born and educated abroad, versus within the USA; monolingual Spanish speakers, versus bilingual Spanish/English speakers; and from households below the national median income, versus households above the national median.

CONCLUSION

Low scores were common and related to crystallized ability, education, and sociocultural variables. Although using demographically adjusted scores reduced group differences related to sociocultural variables, group differences were not eliminated, indicating that age, gender, and education score adjustments do not fully explain the associations between sociocultural variables and test performances. These stratified base rates may be useful when working with Spanish-speaking patients with diverse sociocultural characteristics.

Reliability and Validity of the Spanish-Language Version of the NIH Toolbox

The psychometric properties of the English-language NIH Toolbox for Assessment of Neurological and Behavioral Function (NIH Toolbox) have been examined in numerous populations. This study evaluated the reliability and validity of the Spanish-language NIH Toolbox. Participants were children aged 3 to 7 years and adults aged 18 to 85 years who took part in the NIH Toolbox norming study in Spanish. Results supported the internal consistency reliability of included measures. Test-retest reliability was strong for most tests, though it was weaker for the test of olfaction among children and the test of locomotion among adults. Spearman's correlations and general linear models showed Spanish tests were often associated with age, sex, and education. Convergent validity for the two language measures that underwent more intensive development, evaluated via Spearman's correlations with legacy measures, was strong. Results support using the Spanish-language NIH Toolbox to measure neurological and behavioral functioning among Spanish-speaking individuals in the United States.