Developing an Executive Functioning Composite Score for Research and Clinical Trials

Recent Advances in Neuropsychological Test Interpretation for Clinical Practice

Much attention in the field of clinical neuropsychology has focused on adapting to the modern healthcare environment by advancing telehealth and promoting technological innovation in assessment. Perhaps as important (but less discussed) are advances in the development and interpretation of normative neuropsychological test data. These techniques can yield improvement in diagnostic decision-making and treatment planning with little additional cost. Brooks and colleagues (Can Psychol 50: 196-209, 2009) eloquently summarized best practices in normative data creation and interpretation, providing a practical overview of norm development, measurement error, the base rates of low scores, and methods for assessing change. Since the publication of this seminal work, there have been several important advances in research on development and interpretation of normative neuropsychological test data, which may be less familiar to the practicing clinician. Specifically, we provide a review of the literature on regression-based normed scores, item response theory, multivariate base rates, summary/factor scores, cognitive intraindividual variability, and measuring change over time. For each topic, we include (1) an overview of the method, (2) a rapid review of the recent literature, (3) a relevant case example, and (4) a discussion of limitations and controversies. Our goal was to provide a primer for use of normative neuropsychological test data in neuropsychological practice.

Age-related hearing loss is not linked to cerebrospinal fluid levels of β-amyloid or p-tau181

INTRODUCTION

As Hearing loss and dementia affect people with the same profile, several epidemiological studies have evaluated their relationship. However, the link between age-related hearing loss and Alzheimer's disease is still unclear.

METHODS

We selected subjects with no history of exposure to loud noises, blasts, head trauma with hearing loss, or sudden sensorineural hearing loss from a cohort intended to study preclinical phases of Alzheimer's disease. Participants are volunteers over 55 years without cognitive impairment. We correlated the results of an objective auditory evaluation with brain amyloid and p-tau181 levels and with the outcomes of a comprehensive neuropsychological assessment.

RESULTS

Fifty-five subjects at different stages of the Alzheimer's disease continuum were evaluated. There were no statistically significant correlations between amyloid-β and p-tau levels and any of the objective auditory measures. A weak but significant correlation was found between amyloid-β values and the Hearing Handicap Inventory for the Elderly. The neuropsychological domains more correlated to hearing loss were executive function and processing speed.

DISCUSSION

Age-related hearing loss is not linked to any pathological markers of Alzheimer's disease nor to neuropsychological domains typically affected in this disease. The Hearing Handicap Inventory for the Elderly has an important component of subjectivity and further studies are needed to explore its relationship with amyloid-β levels.

A Clinic-Based Measure of Everyday Planning Ability: The Overnight Trip Task

OBJECTIVE

Traditional measures of cognitive abilities most used by neuropsychologists are generally low in ecological validity and only capture a small proportion of the variance of a person's true functional capacity. This study evaluates the association between clinic-based performance and parallel real-world completion of an everyday planning test, the Overnight Trip Task (OTT).

METHOD

A cross-sectional cohort of 65 community-dwelling older adults completed a battery of cognitive assessments and two formats of the OTT: the paper-and-pencil clinic OTT, which was completed remotely through video conferencing (Zoom), and the parallel real-world OTT (RW-OTT), which was completed at home between sessions. Both formats required participants to plan for what they would pack and prepare for a hypothetical overnight trip based on a provided story that included rules and embedded contingencies.

RESULTS

The clinic OTT demonstrated a small-to-moderate relationship with the RW-OTT (r = 0.35) and no relationship with measures of learning, long delay recall, an executive function composite, and a measure of everyday planning. Hierarchical regressions indicated that the clinic OTT demonstrated incremental validity above an executive function composite measure and global cognition when predicting self-reported everyday functioning and RW-OTT performance.

CONCLUSIONS

The clinic OTT showed only modest association with the RW-OTT and discriminate but not convergent validity was demonstrated. The clinic OTT showed incremental validity when predicting self-reported everyday functioning and RW-OTT performance above more traditional measures. Before the clinic OTT could be considered a reliable and valid clinic-based measure for predicting real-world behavior, additional research would be needed.

An Executive Functioning Composite Does Not Moderate the Relationship Between Combat Exposure and Posttraumatic Stress Disorder Symptom Clusters

OBJECTIVE

Posttraumatic stress disorder (PTSD) is prevalent among U.S. combat Veterans, and associated with poor health and wellbeing. As combat experiences are likely to significantly modify self-, other-, and society-oriented cognitions and heighten risk for PTSD, examination of related cognitive processes may yield new treatment strategies. The cognitive model of PTSD suggests that persistent threat perceptions contribute to symptom worsening. Thus, cognitive processes of shifting perspectives or generating novel interpretations may be particularly relevant to lessen PTSD symptoms. This cross-sectional study examined executive functioning as a moderator to the relationship between combat exposure and PTSD symptom clusters among post-9/11 Veterans.

METHOD

Data from 168 Veterans were drawn from a larger study examining post-deployment mental health and cognitive function. An executive functioning composite derived from Wisconsin Card Sorting Test Perseveration Errors, WAIS-III Similarities, Trail Making Test B, and Stroop Color-Word Inhibition scores was computed. Path analysis was used to test the moderation model.

RESULTS

After accounting for age, sex, and estimated premorbid functioning, results indicated that combat exposure was associated with all symptom clusters on the PTSD Checklist-Military. Executive functioning was not significantly associated with the PTSD symptom clusters and did not moderate the relationship between combat exposure and any of the PTSD symptom clusters.

CONCLUSIONS

Combat exposure is an important dimension of risk related to PTSD in Veterans that warrants regular screening. Moderation by executive functioning was not observed despite theoretical support. Future work could test methodological and sampling reasons for this finding to determine if theoretical adjustment is necessary.

Basal forebrain functional connectivity as a mediator of associations between cardiorespiratory fitness and cognition in healthy older women

Age-related cholinergic dysfunction within the basal forebrain (BF) is one of the key hallmarks for age-related cognitive decline. Given that higher cardiorespiratory fitness (CRF) induces neuroprotective effects that may differ by sex, we investigated the moderating effects of sex on the associations between CRF, BF cholinergic function, and cognitive function in older adults. 176 older adults (68.5 years) were included from the Nathan Kline Institute Rockland Sample. Functional connectivity (rsFC) of the BF subregions including the medial septal nucleus/diagonal band of Broca (MS/DB) and nucleus basalis of Meynert (NBM) were computed from resting-sate functional MRI. Modified Astrand-Ryhming submaximal cycle ergometer protocol was used to estimate CRF. Trail making task and inhibition performance during the color word interference test from the Delis-Kaplan Executive Function System and Rey Auditory Verbal Learning Test were used to examine cognitive function. Linear regression models were used to assess the associations between CRF, BF rsFC, and cognitive performance after controlling for age, sex, and years of education. Subsequently, we measured the associations between the variables in men and women separately to investigate the sex differences. There was an association between higher CRF and greater rsFC between the NBM and right middle frontal gyrus in older men and women. There were significant associations between CRF, NBM rsFC, and trail making task number-letter switching performance only in women. In women, greater NBM rsFC mediated the association between higher CRF and better trail making task number-letter switching performance. These findings provide evidence that greater NBM rsFC, particularly in older women, may be an underlying neural mechanism for the relationship between higher CRF and better executive function.

Initial Psychometric Properties of 7 NeuroUX Remote Ecological Momentary Cognitive Tests Among People With Bipolar Disorder: Validation Study

BACKGROUND

As smartphone technology has become nearly ubiquitous, there is a growing body of literature suggesting that ecological momentary cognitive testing (EMCT) offers advantages over traditional pen-and-paper psychological assessment. We introduce a newly developed platform for the self-administration of cognitive tests in ecologically valid ways.

OBJECTIVE

The aim of this study is to develop a Health Insurance Portability and Accountability Act-compliant EMCT smartphone-based platform for the frequent and repeated testing of cognitive abilities in everyday life. This study examines the psychometric properties of 7 mobile cognitive tests covering domains of processing speed, visual working memory, recognition memory, and response inhibition within our platform among persons with and without bipolar disorder (BD). Ultimately, if shown to have adequate psychometric properties, EMCTs may be useful in research on BD and other neurological and psychiatric illnesses.

METHODS

A total of 45 persons with BD and 21 demographically comparable healthy volunteer participants (aged 18-65 years) completed smartphone-based EMCTs 3 times daily for 14 days. Each EMCT session lasted approximately 1.5 minutes. Only 2 to 3 tests were administered in any given session, no test was administered more than once per day, and alternate test versions were administered in each session.

RESULTS

The mean adherence to the EMCT protocol was 69.7% (SD 20.5%), resulting in 3965 valid and complete tests across the full sample. Participants were significantly more likely to miss tests on later versus earlier study days. Adherence did not differ by diagnostic status, suggesting that BD does not interfere with EMCT participation. In most tests, age and education were related to EMCT performance in expected directions. The average performances on most EMCTs were moderately to strongly correlated with the National Institutes of Health Toolbox Cognition Battery. Practice effects were observed in 5 tests, with significant differences in practice effects by BD status in 3 tests.

CONCLUSIONS

Although additional reliability and validity data are needed, this study provides initial psychometric support for EMCTs in the assessment of cognitive performance in real-world contexts in BD.

Considerations for Selecting Cognitive Endpoints and Psychological Patient-Reported Outcomes for Clinical Trials in Pediatric Patients With Sickle Cell Disease

Pediatric patients with sickle cell disease (SCD) experience a range of medical complications that result in significant morbidity and mortality. Recent advances in prophylactic and curative treatment approaches have highlighted the need for sensitive and clinically-meaningful trial endpoints. The detrimental effects of cognitive and psychological difficulties on social and economic mobility are well described. Although numerous reviews have assessed cognitive outcomes in other rare genetic disorders, SCD has not received the same focus. This review describes the cognitive (i.e., executive function and processing speed) and psychological domains (i.e., depression and anxiety) that are consistently associated with SCD pathology and, therefore, may be of particular interest as clinical trial endpoints. We then discuss corresponding well-validated and reliable cognitive tests and patient-reported outcomes (PROs) that may be appropriate for clinical trials given their robust psychometric properties, ease of administration, and previous use in the SCD population. Further, we provide a discussion of potential pitfalls and considerations to guide endpoint selection. In line with the move toward patient-centered medicine, we identify specific tests (e.g., NIH Toolbox Cognition Module, Wechsler Cancellation Test) and psychological PROs (e.g., PROMIS depression and anxiety scales) that are sensitive to SCD morbidity and have the potential to capture changes that are clinically meaningful in the context of patients' day to day lives. In particularly vulnerable cognitive domains, such as executive function, we highlight the advantages of composite over single-test scores within the context of trials. We also identify general (i.e., practice effects, disease heterogeneity) and SCD-specific considerations (i.e., genotype, treatment course, and disease course, including degree of neurologic, pain, and sleep morbidity) for trial measures. Executive function composites hold particular promise as trial endpoints that are clinically meaningful, amenable to change, relatively easy to collect, and can be incorporated into the routine care of patients with SCD in various settings and countries.

Examining Test-Retest Reliability and Reliable Change for Cognition Endpoints for the CENTER-TBI Neuropsychological Test Battery

Objective: Seven candidate cognition composite scores have been developed and evaluated as part of a research program designed to validate a cognition endpoint for traumatic brain injury (TBI) research and clinical trials, but these composites have yet to be examined longitudinally. This study examined test-retest reliability and methods for determining reliable change for these seven candidate composite scores, using the neuropsychological test battery from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI).

Methods: Participants (18–59 years-old) with mild TBI (n = 124), orthopedic trauma without head injury (n = 67), and healthy community controls (n = 63) from the Trondheim MTBI follow-up study completed the CENTER-TBI neuropsychological test battery at 2 weeks and 3 months after injury. The battery included both traditional paper-and-pencil tests and computerized tests from the Cambridge Neuropsychological Test Automated Battery (CANTAB). Seven composite scores were calculated for the paper-and-pencil tests, the CANTAB tests, and all tests combined (i.e., 21 composites in total on each assessment): the overall test battery mean (OTBM); global deficit score (GDS); neuropsychological deficit score-weighted (NDS-W); low score composite (LSC); and the number of scores ≤5th percentile, ≤16th percentile, or <50th percentile. The OTBM was calculated by averaging T scores for all tests. The other composite scores were deficit-based scores, assigning different weights to low scores.

Results: All composites revealed better cognitive performance at the 3-month assessment compared to the 2-week assessment and the magnitude of improvement was similar across groups. Differences, in terms of effect sizes, were largest on the OTBMs. In the combined composites, the test-retest correlation was highest for the OTBM (Spearman's rho = 0.87, in the community control group) and lowest for the number of scores ≤5th percentile (rho = 0.41).

Conclusion: The high test-retest reliability of the OTBM appears to favor its use in TBI research; however, future studies are needed to examine these candidate composite scores in participants with more severe TBIs and cognitive deficits and the association of the composites with functional outcomes.

Developing Cognition Endpoints for the CENTER-TBI Neuropsychological Test Battery

Background: Measuring cognitive functioning is common in traumatic brain injury (TBI) research, but no universally accepted method for combining several neuropsychological test scores into composite, or summary, scores exists. This study examined several possible composite scores for the test battery used in the large-scale study Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI).

Methods: Participants with mild traumatic brain injury (MTBI; n = 140), orthopedic trauma (n = 72), and healthy community controls (n = 70) from the Trondheim MTBI follow-up study completed the CENTER-TBI test battery at 2 weeks after injury, which includes both traditional paper-and-pencil tests and tests from the Cambridge Neuropsychological Test Automated Battery (CANTAB). Seven composite scores were calculated for the paper and pencil tests, the CANTAB tests, and all tests combined (i.e., 21 composites): the overall test battery mean (OTBM); global deficit score (GDS); neuropsychological deficit score-weighted (NDS-W); low score composite (LSC); and the number of scores ≤5th percentile, ≤16th percentile, or <50th percentile.

Results: The OTBM and the number of scores <50th percentile composites had distributional characteristics approaching a normal distribution. The other composites were in general highly skewed and zero-inflated. When the MTBI group, the trauma control group, and the community control group were compared, effect sizes were negligible to small for all composites. Subgroups with vs. without loss of consciousness at the time of injury did not differ on the composite scores and neither did subgroups with complicated vs. uncomplicated MTBIs. Intercorrelations were high within the paper-and-pencil composites, the CANTAB composites, and the combined composites and lower between the paper-and-pencil composites and the CANTAB composites.

Conclusion: None of the composites revealed significant differences between participants with MTBI and the two control groups. Some of the composite scores were highly correlated and may be redundant. Additional research on patients with moderate to severe TBIs is needed to determine which scores are most appropriate for TBI clinical trials.