An evidence-based approach to the creation of normative data: base rates of impaired scores within a brief neuropsychological battery argue for age corrections, but against corrections for medical conditions

Recent Advances in Neuropsychological Test Interpretation for Clinical Practice

Much attention in the field of clinical neuropsychology has focused on adapting to the modern healthcare environment by advancing telehealth and promoting technological innovation in assessment. Perhaps as important (but less discussed) are advances in the development and interpretation of normative neuropsychological test data. These techniques can yield improvement in diagnostic decision-making and treatment planning with little additional cost. Brooks and colleagues (Can Psychol 50: 196-209, 2009) eloquently summarized best practices in normative data creation and interpretation, providing a practical overview of norm development, measurement error, the base rates of low scores, and methods for assessing change. Since the publication of this seminal work, there have been several important advances in research on development and interpretation of normative neuropsychological test data, which may be less familiar to the practicing clinician. Specifically, we provide a review of the literature on regression-based normed scores, item response theory, multivariate base rates, summary/factor scores, cognitive intraindividual variability, and measuring change over time. For each topic, we include (1) an overview of the method, (2) a rapid review of the recent literature, (3) a relevant case example, and (4) a discussion of limitations and controversies. Our goal was to provide a primer for use of normative neuropsychological test data in neuropsychological practice.

The association between shift work exposure and cognitive impairment among middle-aged and older adults: Results from the Canadian Longitudinal Study on Aging (CLSA)

BACKGROUND

Shift work, especially rotating and night shift work, has been linked to a wide range of detrimental health outcomes. Occupational factors like shift work and their potential impact on cognitive functions have received little attention, and the evidence is inconclusive. The objective of our study is to explore associations between shift work exposure and cognitive impairment indicators based on comparisons with the normative standards from the Canadian population.

METHODS

Cross-sectional analyses were performed using baseline Canadian Longitudinal Study on Aging database, including 47,811 middle-aged and older adults (45-85 years). Three derived shift work variables were utilized: ever exposed to shift work, shift work exposure in longest job, and shift work exposure in current job. Four cognitive function tests were utilized, Rey Auditory Verbal Learning Tests (immediate and delayed) representing memory domain, and Animal Fluency, and Mental Alteration, representing the executive function domain. All cognitive test scores included in study were normalized and adjusted for the participant's age, sex, education and language of test administration (English and French), which were then compared to normative data to create "cognitive impairment' variables. Unadjusted and adjusted multivariable logistic regression models were used to determine associations between shift work variables and cognitive impairment individually (memory and executive function domains), and also for overall cognitive impairment.

RESULT

Overall, one in every five individuals (21%) reported having been exposed to some kind of shift work during their jobs. Exposure to night shift work (both current and longest job) was associated with overall cognitive impairment. In terms of domain-based measures, night shift work (longest job) was associated with memory function impairment, and those exposed to rotating shift work (both current and longest job) showed impairment on executive function measures, when compared to daytime workers.

CONCLUSION

This study suggests disruption to the circadian rhythm, due to shift work has negative impact on cognitive function in middle-aged and older adults and this warrants further investigation.

Base rates of low neuropsychological test scores in older adults with subjective cognitive impairment: Findings from a tertiary memory clinic

Base rates of low scores are typically determined from normative samples, which differ from clinical samples. We examined base rates of spuriously low scores for 93 older adults with subjective cognitive impairment presenting to a memory clinic. Crawford's Monte Carlo simulation algorithm was used to estimate multivariate base rates by calculating the percentage of cognitively intact memory clinic patients who produced normed scores at or below the 5th percentile. Neuropsychological tests included: Weschler Adult Intelligence Scale block design, digit span backwards, coding, Weschler Memory Scale logical memory immediate/delayed; California Verbal Learning Test immediate/delayed memory; Brief Visuospatial Memory Test immediate/delayed; and Delis-Kaplan Executive Functioning category switching, letter number sequencing, and inhibition/switching. An estimated 33.58% of the cognitively intact memory clinic population would have one or more low scores, 14.7% two or more, 6.55% three or more, 2.94% four or more, and 1.31% 5 or more due to chance. Base rates were then applied to a subset of clinical data: all with dementia and most with MCI had low scores that exceeded the base rates. Determining base rates of spuriously low scores on a neuropsychological battery in clinical samples could reduce false positives by using empirical adjustments for expected low scores.

Methodological considerations when establishing reliable and valid normative data: Canadian Longitudinal Study on Aging (CLSA) neuropsychological battery

Objective: Creation of normative data with regression corrections for demographic covariates reduces risk of small cell sizes compared with traditional normative approaches. We explored whether methods of correcting for demographic covariates (e.g., full regression models versus hybrid models with stratification and regression) and choice of covariates (i.e., correcting for age with or without sex and/or education correction) impacted reliability and validity of normative data. Method: Measurement invariance for sex and education was explored in a brief telephone-administered cognitive battery from the Canadian Longitudinal Study on Aging (CLSA; after excluding persons with neurological conditions N = 12,350 responded in English and N = 1,760 in French). Results: Measurement invariance was supported in hybrid normative models where different age-based regression models were created for groups based on sex and education level. Measurement invariance was not supported in full regression models where age, sex, and education were simultaneous predictors. Evidence for reliability was demonstrated by precision defined as the 95% inter-percentile range of the 5^th percentile. Precision was higher for full regression models than for hybrid models but with negligible differences in precision for the larger English sample. Conclusions: We present normative data for a remotely administered brief neuropsychological battery that best mitigates measurement bias and are precise. In the smaller French speaking sample, only one model reduced measurement bias, but its estimates were less precise, underscoring the need for large sample sizes when creating normative data. The resulting normative data are appended in a syntax file.

Frailty among middle-aged and older Canadians: population norms for the frailty index using the Canadian Longitudinal Study on Aging

BACKGROUND

frailty is a public health priority now that the global population is ageing at a rapid rate. A scientifically sound tool to measure frailty and generate population-based reference values is a starting point.

OBJECTIVE

in this report, our objectives were to operationalize frailty as deficit accumulation using a standard frailty index (FI), describe levels of frailty in Canadians ≥45 years old and provide national normative data.

DESIGN

this is a secondary analysis of the Canadian Longitudinal Study on Aging (CLSA) baseline data.

SETTING/PARTICIPANTS

about 51,338 individuals (weighted to represent 13,232,651 Canadians), aged 45-85 years, from the tracking and comprehensive cohorts of CLSA.

METHODS

after screening all available variables in the pooled dataset, 52 items were selected to construct an FI. Descriptive statistics for the FI and normative data derived from quantile regressions were developed.

RESULTS

the average age of the participants was 60.3 years (95% confidence interval [CI]: 60.2-60.5), and 51.5% were female (95% CI: 50.8-52.2). The mean FI score was 0.07 (95% CI: 0.07-0.08) with a standard deviation of 0.06. Frailty was higher among females and with increasing age, and scores >0.2 were present in 4.2% of the sample. National normative data were identified for each year of age for males and females.

CONCLUSIONS

the standardized frailty tool and the population-based normative frailty values can help inform discussions about frailty, setting a new bar in the field. Such information can be used by clinicians, researchers, stakeholders and the general public to understand frailty, especially its relationship with age and sex.

Five-year change scores in old age for six neuropsychological tests and normative data for the Useful Field of View (UFOV) test: The influence of physical health

INTRODUCTION

Neuropsychological assessment of cognitive change over time is often conducted in clinical settings, but whether neuropsychological change scores are influenced by physical health has, as far as we know, not been examined previously.

METHOD

In a sample of 153 older Swedish adults (age range, 72-86 years), we evaluated the influence of common age-related diseases, terminal decline pathology, age, education, and gender, to provide (a) preliminary test-specific regression weights and 90% confidence intervals to assess significant change in performance after five years on tests of visual scanning, mental shifting, visual spatial ability, memory, reaction time, and selective attention, and (b) normative data for the Useful Field of View test (UFOV) from a single testing occasion.

RESULTS

Multiple regression analyses showed that test-retest changes were affected by physical health for mental shifting, visual spatial ability, memory, and reaction time, by age for mental shifting and visual reaction time, by education for visual spatial ability, and by Age × Education for auditory reaction time. Gender did not affect any of the change scores. The overall average of variance explained was 2.5%: up to 8.1% for physical health, 4.4% for age, and 3.6% for education. The UFOV scores were mostly influenced by age, but also by physical health and education.

CONCLUSIONS

The findings indicate that considering the influence of health on normative change scores in old age in addition to demographic factors leads to more accurate predictions of whether true change has occurred.