Construct validity of the NIH toolbox cognitive domains: A comparison with conventional neuropsychological assessments

Neurocognition and its predictors in a linguistically and culturally diverse cohort of people with HIV

Objective: HIV/AIDS disproportionately affects Black and Latino people in the United States, yet there is a lack of research on predictors of neurocognitive outcomes in these groups. We examined neurocognitive performance and its key predictors across White, Black, and Latino people with HIV (PWH). Method: Participants included 586 PWH of White, Black, and Latino (English- and Spanish-speaking) background. Neurocognition was assessed via demographically-adjusted Fluid Cognition Composite T-scores from the NIH-Toolbox cognition battery, and individual tests comprising this composite. Predictors examined included sociodemographic and HIV disease characteristics, and medical, psychiatric and substance comorbidities. Results: Compared to White PWH, English-speaking Latino PWH had lower T-scores on the Fluid Cognition Composite, as well as Flanker Inhibition and Picture Sequence Memory tests. While there were no other significant group differences on Fluid Cognition, both Latino PWH language groups performed worse than Black PWH on Flanker Inhibition, and Black PWH performed worse than White PWH on List Sorting. Separate multivariable linear regression models by ethnic/racial/language group showed that significant correlates of worse Fluid Cognition included depressive symptoms among White PWH; hepatitis C co-infection among Black PWH; hypertension among English-speaking Latino PWH; and higher estimated duration of HIV disease and depressive symptoms in Spanish-speaking Latino PWH. Conclusions: Findings suggest worse neurocognition among English-speaking Latino PWH compared to Whites. Predictors of neurocognitive function among PWH differ across ethnic/racial and language groups. Consideration of these HIV disease characteristics and comorbidities may be valuable in developing targeted culturally-relevant interventions aimed at ameliorating neurocognitive dysfunction among diverse PWH.

Psychometrics of drift-diffusion model parameters derived from the Eriksen flanker task: Reliability and validity in two independent samples

The flanker task is a widely used measure of cognitive control abilities. Drift-diffusion modeling of flanker task behavior can yield separable parameters of cognitive control-related subprocesses, but the parameters' psychometrics are not well-established. We examined the reliability and validity of four behavioral measures: (1) raw accuracy, (2) reaction time (RT) interference, (3) NIH Toolbox flanker score, and (4) two drift-diffusion model (DDM) parameters-drift rate and boundary separation-capturing evidence accumulation efficiency and speed-accuracy trade-off, respectively. Participants from two independent studies - one cross-sectional (N = 381) and one with three timepoints (N = 83) - completed the flanker task while electroencephalography data were collected. Across both studies, drift rate and boundary separation demonstrated comparable split-half and test-retest reliability to accuracy, RT interference, and NIH Toolbox flanker score, but better incremental convergent validity with psychophysiological measures (i.e., the error-related negativity; ERN) and neuropsychological measures of cognitive control than the other behavioral indices. Greater drift rate (i.e., faster and more accurate responses) to congruent and incongruent stimuli, and smaller boundary separation to incongruent stimuli were related to 1) larger ERN amplitudes (in both studies) and 2) faster and more accurate inhibition and set-shifting over and above raw accuracy, reaction time, and NIH Toolbox flanker scores (in Study 1). Computational models, such as DDM, can parse behavioral performance into subprocesses that exhibit comparable reliability to other scoring approaches, but more meaningful relationships with other measures of cognitive control. The application of these computational models may be applied to existing data and enhance the identification of cognitive control deficits in psychiatric disorders.

The validity of studying healthy aging with cognitive tests measuring different constructs

A clinically useful characterization of the cognitive aging process requires the development of valid and robust behavioral tests, with an emphasis on explaining and understanding typical inter-individual variability in cognition. Here, using a dataset that includes behavioral scores collected with the National Institute of Health Toolbox Cognition Battery (NIHTB-CB) and other auxiliary tests, we examined (1) the differences between young and old adults across different cognitive domains, (2) the strength of across-subject correlations in behavioral test scores, (3) the consistency of low-dimensional behavioral representations across age using factor analysis, and (4) the accuracy of behavioral scores in predicting participants' age. Our results revealed that (1) elderly females had better verbal episodic memory scores than elderly males, (2) across-subject correlations between behavioral tests varied with age group, (3) although a three-factor model explained the behavioral data in both age groups, some tasks loaded to different factors between the two groups, and (4) age-performance relationship (i.e. a regression model linking age to cognitive scores) in one group cannot be extrapolated to predict age in the other group, indicating an inconsistency in age-performance relationships across groups. These findings suggest that executive function tests might tap into different cognitive processes in different age groups, which might ultimately suggest that a statistically significant between-group difference in test performance might not always reflect differences in the same underlying cognitive processes. Overall, this study calls for more caution when interpreting age-related differences and similarities between age groups with different cognitive abilities even when the same tests are used.

The Cognitive Profile of Older Adults With Treatment-Resistant Depression: An Analysis of the OPTIMUM Randomized Controlled Trial

OBJECTIVE

Major depressive disorder in older adults (late-life depression; LLD) is frequently associated with cognitive impairment, and some deficits (e.g., executive function) have been associated with a higher level of treatment resistance. However, the cognitive profile of treatment-resistant LLD (TR-LLD) has not been characterized. We hypothesized that patients with TR-LLD would show deficits in cognitive function, especially executive function, and that executive function deficits would predict poorer response to pharmacotherapy.

DESIGN

Secondary analysis of baseline cognitive data from OPTIMUM, a multicenter RCT evaluating pharmacotherapy strategies for TR-LLD.

SETTING

Five outpatient academic medical centers (4 US, 1 Canada).

PARTICIPANTS

About 369 participants aged 60 and older from the OPTIMUM study.

MEASUREMENTS

Baseline scores on individual tasks and composite scores from the NIH Toolbox-Cognition Battery were transformed into demographically-adjusted T-scores and compared to published norms. Impairments in the set shifting and inhibitory control tasks were investigated as predictors of depressive symptom change following treatment using ANCOVA models.

RESULTS

Participants had low performance on tasks evaluating inhibitory control, processing speed, verbal/nonverbal memory, and the fluid composite, but normative performance on working memory and set shifting. Participants had high estimated premorbid IQ (superior Performance on oral reading recognition). Age and physical comorbidity negatively associated with processing speed. Impairments in set shifting predicted less improvement in depressive symptoms; impairments in inhibitory control did not.

CONCLUSIONS

Participants with TR-LLD presented with broad cognitive deficits relative to healthy norms. Given poorer outcomes following standard pharmacotherapy associated with impaired set shifting, future research needs to identify alternative treatment strategies.

Unsupervised machine learning for identifying attention-deficit/hyperactivity disorder subtypes based on cognitive function and their implications for brain structure

BACKGROUND

Structural anomalies in the frontal lobe and basal ganglia have been reported in patients with attention-deficit/hyperactivity disorder (ADHD). However, these findings have been not always consistent because of ADHD diversity. This study aimed to identify ADHD subtypes based on cognitive function and find their distinct brain structural characteristics.

METHODS

Using the data of 656 children with ADHD from the Adolescent Brain Cognitive Development (ABCD) Study, we applied unsupervised machine learning to identify ADHD subtypes using the National Institutes of Health Toolbox Tasks. Moreover, we compared the regional brain volumes between each ADHD subtype and 6601 children without ADHD (non-ADHD).

RESULTS

Hierarchical cluster analysis automatically classified ADHD into three distinct subtypes: ADHD-A (n = 212, characterized by high-order cognitive ability), ADHD-B (n = 190, characterized by low cognitive control, processing speed, and episodic memory), and ADHD-C (n = 254, characterized by strikingly low cognitive control, working memory, episodic memory, and language ability). Structural analyses revealed that the ADHD-C type had significantly smaller volumes of the left inferior temporal gyrus and right lateral orbitofrontal cortex than the non-ADHD group, and the right lateral orbitofrontal cortex volume was positively correlated with language performance in the ADHD-C type. However, the volumes of the ADHD-A and ADHD-B types were not significantly different from those of the non-ADHD group.

CONCLUSIONS

These results indicate the presence of anomalies in the lateral orbitofrontal cortex associated with language deficits in the ADHD-C type. Subtype specificity may explain previous inconsistencies in brain structural anomalies reported in ADHD.

Accuracy of Reaction Time Measurement on Automated Neuropsychological Assessment Metric UltraMobile

OBJECTIVE

This observational study examined the accuracy of simple reaction time (RT) measurements on various touchscreen tablet devices using the Automated Neuropsychological Assessment Metric (ANAM) UltraMobile test battery. The study investigated the implications of interpreting ANAM UltraMobile with laptop-based normative data by analyzing the magnitude and variability of RT accuracy across devices.

METHOD

RT accuracy on 10 different tablets was assessed using a photodetector and robotic arm to respond to stimuli at predetermined response times. The recorded RT was compared with the true RT obtained from the robotic arm to calculate the RT error.

RESULTS

ANAM UltraMobile recorded slower RTs than the laptop version. Additionally, RT error varied considerably among the 10 tablet models, suggesting psychometrically significant implications that could lead to interpretive errors when using laptop-based normative data.

CONCLUSIONS

Relative to the RT error from the laptop-based version of ANAM, tablet data from ANAM UltraMobile are significantly slower and exhibit large variability between devices. These differences may have clinically significant implications for the comparability of the two versions. The findings suggest that further research with human participants is needed to assess the equivalence of ANAM UltraMobile with its predecessor.

Association of chronotype with language and episodic memory processing in children: implications for brain structure

Introduction

Chronotype refers to individual preference in circadian cycles and is associated with psychiatric problems. It is mainly classified into early (those who prefer to be active in the morning and sleep and wake up early) and late (those who prefer to be active in the evening and sleep and wake up late) chronotypes. Although previous research has demonstrated associations between chronotype and cognitive function and brain structure in adults, little is known regarding these associations in children. Here, we aimed to investigate the relationship between chronotype and cognitive function in children. Moreover, based on the significant association between chronotype and specific cognitive functions, we extracted regions-of-interest (ROI) and examined the association between chronotype and ROI volumes.

Methods

Data from 4,493 children (mean age of 143.06 months) from the Adolescent Brain Cognitive Development Study were obtained, wherein chronotype (mid-sleep time on free days corrected for sleep debt on school days) was assessed by the Munich Chronotype Questionnaire. Subsequently, the associations between chronotype, cognitive function, and ROI volumes were evaluated using linear mixed-effects models.

Results

Behaviorally, chronotype was negatively associated with vocabulary knowledge, reading skills, and episodic memory performance. Based on these associations, the ROI analysis focused on language-related and episodic memory-related areas revealed a negative association between chronotype and left precentral gyrus and right posterior cingulate cortex volumes. Furthermore, the precentral gyrus volume was positively associated with vocabulary knowledge and reading skills, while the posterior cingulate cortex volume was positively associated with episodic memory performance.

Discussion

These results suggest that children with late chronotype have lower language comprehension and episodic memory and smaller brain volumes in the left precentral gyrus and right posterior cingulate cortex associated with these cognitive functions.

Utility of the NIH Toolbox Cognition Battery in middle to older aged adults with longstanding type 1 diabetes: The DCCT/EDIC study

Objective: Adults with type 1 diabetes (T1D) face an increased risk for cognitive decline and dementia. Diabetes-related and vascular risk factors have been linked to cognitive decline using detailed neuropsychological testing; however, it is unclear if cognitive screening batteries can detect cognitive changes associated with aging in T1D. Method: 1,049 participants with T1D (median age 59 years; range 43-74) from the Diabetes Control and Complications Trial (DCCT), and the follow-up Epidemiology of Diabetes Interventions and Complications (EDIC) study, completed the NIH Toolbox Cognition Battery (NIHTB-C) and Montreal Cognitive Assessment (MoCA). Neuropsychological assessments, depression, glycated hemoglobin levels (HbA1c), severe hypoglycemia, T1D complications, and vascular risk factors were assessed repeatedly over 32 years to determine associations with current NIHTB-C performance. Available cognitive data was clinically adjudicated to determine cognitive impairment status. Results: NIHTB-C scores had moderate associations (r = 0.36-0.53) with concurrently administered neuropsychological tests. In multivariate models, prior severe hypoglycemic episodes, depression symptoms, nephropathy, lower BMI, and higher HbA1c and LDL cholesterol were associated with poorer NIHTB-C Fluid Cognition Composite scores. The NIHTB-C adequately detected adjudicated cognitive impairment (Area Under the Curve = 0.86; optimal cut score ≤90). The MoCA performed similarly (Area Under the Curve = 0.83; optimal cut score ≤25). Conclusions: The NIHTB-C is sensitive to the cognitive effects of diabetes-related and vascular risk factors, correlated with neuropsychological testing, and accurately detects adjudicated cognitive impairment. These data support its use as a screening test in middle to older aged adults with T1D to determine if referral for detailed neuropsychological assessment is needed.

Construct validation of NIH Toolbox Cognition Battery premorbid cognitive functioning scores in Black and White older Americans with and without mild cognitive impairment

OBJECTIVE

Valid estimates of premorbid cognitive functioning (PMIQ) are crucial for the assessment of older adults at risk for Alzheimer's disease. We investigated the relationship between the NIH Toolbox-Cognition Battery's (NIHTB-CB) Oral Reading Recognition (ORR) subtest and Wechsler Test of Adult Reading scores (WTAR, convergent validity). We also compared ORR to NIHTB-CB Flanker scores, where null relationships were expected (discriminant validity).

METHODS

The WTAR and NIHTB-CB were administered to 130 cognitively normal (CN) and 113 participants with mild cognitive impairment (MCI). Participants were community-dwelling, older Black and White adults, ages 55-88 years. Data analysis used uncorrected standard scores and Bayesian bivariate correlations. Supplemental materials include intraclass correlations.

RESULTS

ORR and WTAR scores were strongly positively associated, while ORR and Flanker scores were unrelated. This pattern held when restricting analyses to the two cognitive status groups, the two racial groups, and the four race-by-diagnosis subgroups.

CONCLUSION

The findings demonstrate convergent and discriminant validity and support NIHTB-CB ORR scores as valid estimates of scores on a PMIQ measure in older Black and White adults with and without MCI.

Evidence for a Compensatory Relationship between Left- and Right-Lateralized Brain Networks

The two hemispheres of the human brain are functionally asymmetric. At the network level, the language network exhibits left-hemisphere lateralization. While this asymmetry is widely replicated, the extent to which other functional networks demonstrate lateralization remains a subject of Investigation. Additionally, it is unknown how the lateralization of one functional network may affect the lateralization of other networks within individuals. We quantified lateralization for each of 17 networks by computing the relative surface area on the left and right cerebral hemispheres. After examining the ecological, convergent, and external validity and test-retest reliability of this surface area-based measure of lateralization, we addressed two hypotheses across multiple datasets (Human Connectome Project = 553, Human Connectome Project-Development = 343, Natural Scenes Dataset = 8). First, we hypothesized that networks associated with language, visuospatial attention, and executive control would show the greatest lateralization. Second, we hypothesized that relationships between lateralized networks would follow a dependent relationship such that greater left-lateralization of a network would be associated with greater right-lateralization of a different network within individuals, and that this pattern would be systematic across individuals. A language network was among the three networks identified as being significantly left-lateralized, and attention and executive control networks were among the five networks identified as being significantly right-lateralized. Next, correlation matrices, an exploratory factor analysis, and confirmatory factor analyses were used to test the second hypothesis and examine the organization of lateralized networks. We found general support for a dependent relationship between highly left- and right-lateralized networks, meaning that across subjects, greater left lateralization of a given network (such as a language network) was linked to greater right lateralization of another network (such as a ventral attention/salience network) and vice versa. These results further our understanding of brain organization at the macro-scale network level in individuals, carrying specific relevance for neurodevelopmental conditions characterized by disruptions in lateralization such as autism and schizophrenia.

Effects of age on the relationship between sleep quality and cognitive performance: Findings from the Human Connectome Project-Aging cohort

BACKGROUND

The association between sleep quality and cognition is widely established, but the role of aging in this relationship is largely unknown.

OBJECTIVE

To examine how age impacts the sleep-cognition relationship and determine whether there are sensitive ranges when the relationship between sleep and cognition is modified. This investigation could help identify individuals at risk for sleep-related cognitive impairment.

SUBJECTS

Sample included 711 individuals (ages 36.00-89.83, 59.66 ± 14.91, 55.7 % female) from the Human Connectome Project-Aging (HCP-A).

METHODS

The association between sleep quality (Pittsburgh Sleep Quality Index, PSQI) and cognition (Crystallized Cognition Composite and Fluid Cognition Composite from the NIH Toolbox, the Trail Making Test, TMT, and the Rey Auditory Verbal Learning Test, RAVLT) was measured using linear regression models, with sex, race, use of sleep medication, hypertension, and years of education as covariates. The interaction between sleep and age on cognition was tested using the moderation analysis, with age as both continuous linear and nonlinear (quadratic) terms.

RESULTS

There was a significant interaction term between the PSQI and nonlinear age term (age2) on TMT-B (p = 0.02) and NIH Toolbox crystallized cognition (p = 0.02), indicating that poor sleep quality was associated with worse performance on these measures (sensitive age ranges 50-75 years for TMT-B and 66-70 years for crystallized cognition).

CONCLUSIONS

The sleep-cognition relationship may be modified by age. Individuals in the middle age to early older adulthood age band may be most vulnerable to sleep-related cognitive impairment.

A psychometric evaluation of the NIH Toolbox fluid cognition tests adapted for Swahili and Dholuo languages in Kenyan children and adolescents

OBJECTIVE

Our objective was to evaluate the psychometric properties of the culturally adapted NIH Toolbox African Languages® when used in Swahili and Dholuo-speaking children in western Kenya.

METHOD

Swahili-speaking participants were recruited from Eldoret and Dholuo-speaking participants from Ajigo; all were <14 years of age and enrolled in primary school. Participants completed a demographics questionnaire and five fluid cognition tests of the NIH Toolbox® African Languages program, including Flanker, Dimensional Change Card Sort (DCCS), Picture Sequence Memory, Pattern Comparison, and List Sorting tests. Statistical analyses examined aspects of reliability, including internal consistency (in both languages) and test-retest reliability (in Dholuo only).

RESULTS

Participants included 479 children (n = 239, Swahili-speaking; n = 240, Dholuo-speaking). Generally, the tests had acceptable psychometric properties for research use within Swahili- and Dholuo-speaking populations (mean age = 10.5; SD = 2.3). Issues related to shape identification and accuracy over speed limited the utility of DCCS for many participants, with approximately 25% of children unable to match based on shape. These cultural differences affected outcomes of reliability testing among the Dholuo-speaking cohort, where accuracy improved across all five tests, including speed.

CONCLUSIONS

There is preliminary evidence that the NIH Toolbox ® African Languages potentially offers a valid assessment of development and performance using tests of fluid cognition in Swahili and Dholuo among research settings. With piloting underway across other diverse settings, future research should gather additional evidence on the clinical utility and acceptability of these tests, specifically through the establishment of norming data among Kenyan regions and evaluating these psychometric properties.

Oscillatory dynamics serving visual selective attention during a Simon task

Selective attention is an important component of cognitive control and is essential for day-to-day functioning. The Simon task is a common test of visual selective attention that has been widely used to probe response selection, inhibition and cognitive control. However, to date, there is a dearth of literature that has focused on the oscillatory dynamics serving task performance in the selective attention component of this task. In this study, 32 healthy adults (mean age: 33.09 years, SD: 7.27 years) successfully completed a modified version of the Simon task during magnetoencephalography. All magnetoencephalographic data were pre-processed and transformed into the time-frequency domain. Significant oscillatory brain responses were imaged using a beamforming approach, and peak task-related neural activity was extracted to examine the temporal dynamics. Across both congruent and Simon conditions, our results indicated robust decreases in alpha (8-12 Hz) activity in the bilateral occipital regions and cuneus during task performance, while increases in theta (3-6 Hz) oscillatory activity were detected in regions of the dorsal frontoparietal attention network, including the dorsolateral prefrontal cortex, frontal eye fields and insula. Lastly, whole-brain condition-wise analyses showed Simon interference effects in the theta range in the superior parietal region and the alpha range in the posterior cingulate and inferior frontal cortices. These findings provide network-specific insights into the oscillatory dynamics serving visual selective attention.

The effects of an exercise intervention on executive function among overweight adults with obstructive sleep apnea

PURPOSE

Obstructive sleep apnea (OSA) is associated with poorer executive function. This study examined the effects of a comprehensive exercise intervention on executive function in overweight adults with mild and moderate-to-severe OSA.

METHODS

Participants aged between 30 and 65 years, with a body mass index (BMI) ranging from 27 to 42 kg/m^2, participated in a 6-week exercise program. Standardized polysomnographic recording methods provided total Apnea-Hypopnea Index (AHI) and level of hypoxemia. Executive function was assessed using the NIH Toolbox Flanker Inhibitory Control Test. A submaximal treadmill exercise test evaluated cardiorespiratory fitness. Participants with baseline total AHI between 5 and 14.9 events/h were classified as mild OSA and participants with baseline total AHI 15 ≥ events/h were classified as moderate-to-severe OSA.

RESULTS

Fifteen participants completed 18 exercise sessions. Significant differences between OSA categories at baseline were observed for sleep characteristics, but not for fitness or executive function. Wilcoxon Signed Rank Tests showed significant increases in median values for the Flanker Test in the moderate-to-severe category only, z = 2.429, p < .015, η² = .737.

CONCLUSION

Six weeks of exercise improved executive function in overweight individuals with moderate-to-severe OSA, but not in those with mild OSA.

Link between fluid/crystallized intelligence and global/local visual abilities across adulthood

Human visual processing involves the extraction of both global and local information from a visual stimulus. Such processing may be related to cognitive abilities, which is likely going to change over time as we age. We aimed to investigate the impact of healthy aging on the association between visual global vs local processing and intelligence. In this context, we collected behavioral data during a visual search task in 103 adults (50 younger/53 older). We extracted three metrics reflecting global advantage (faster global than local processing), and visual interference in detecting either local or global features (based on interfering visual distractors). We found that older, but not younger, adults with higher levels of fluid and crystallized intelligence showed stronger signs of global advantage and interference effects during local processing, respectively. The present findings also provide promising clues regarding how participants consider and process their visual world in healthy aging.