Differential item functioning in the cognitive screener used in the Longitudinal Aging Study in India

Education, urbanicity of residence, and cardiometabolic biomarkers among middle-aged and older populations in the US, Mexico, China, and India

Background

The relationship between education and cardiometabolic biomarkers is contextually dependent on both inter-country and intra-country factors. This study aimed to examine educational differences in cardiometabolic biomarkers among middle-aged and older adults in the US, Mexico, China, and India, and whether this relationship is modified by urbanicity of residence.

Methods

Data were from contemporary cross-sectional waves of the US Health and Retirement Study (HRS; 2016/17, n = 19,608), the Mexican Health and Aging Study (MHAS; 2015, n = 12,356), the China Health and Retirement Longitudinal Study (CHARLS; 2015/16, n = 13,268), and the Longitudinal Aging Study in India (LASI; 2017/19, n = 47,838). To account for substantial variations in educational distribution across the four countries, we measured education attainment in two ways: by categorizing education levels into binary classifications ('lower education: lower secondary education or below' vs. 'higher education: upper secondary education or above') to assess absolute education attainment, and by using within-country percentile ranks to capture relative education attainment. We assessed educational differences in four cardiometabolic biomarkers: body mass index (BMI), systolic blood pressure (SBP), glycated haemoglobin (HbA1c), and total cholesterol. We tested whether urbanicity of residence modified the relationship between education and these cardiometabolic biomarkers.

Results

The proportion of individuals with higher education was 82.6% in the US, 15.6% in Mexico, 10.6% in China, and 16.8% in India. In the US, higher education was associated with lower SBP (-2.74 mmHg, 95% CI: -3.62, -1.86) and HbA1c (-0.14%, 95% CI: -0.20, -0.08), but higher total cholesterol (3.33 mg/dL, 95% CI: 1.41, 5.25). In Mexico, higher education was associated with lower BMI only (-0.51 kg/m2, 95% CI: -0.76, -0.26). In China, higher education was not associated with any biomarker. In India, higher education was associated with higher BMI (1.61 kg/m2, 95% CI: 1.49, 1.73), SBP (1.67 mmHg, 95% CI: 1.16, 2.18), and HbA1c (0.35%, 95% CI: 0.19, 0.51). The association between education and cardiometabolic biomarkers was modified by urbanicity in China and India but not in the US or Mexico. In both China and India, relationships between education and cardiometabolic biomarkers were stronger among rural residents compared to those among urban residents. Results based on relative education attainment showed similar patterns in terms of the direction of the effect estimates, despite some discrepancies in statistical significance.

Interpretation

There is a complex relationship between education and cardiometabolic biomarkers across countries and by urbanicity of residence. This complexity underscores the importance of accounting for contextual factors when devising strategies to enhance cardiometabolic health in various settings.

Memory and language cognitive data harmonization across the United States and Mexico

INTRODUCTION

We used cultural neuropsychology-informed procedures to derive and validate harmonized scores representing memory and language across population-based studies in the United States and Mexico.

METHODS

Data were from the Health and Retirement Study Harmonized Cognitive Assessment Protocol (HRS-HCAP) and the Mexican Health and Aging Study (MHAS) Ancillary Study on Cognitive Aging (Mex-Cog). We statistically co-calibrated memory and language domains and performed differential item functioning (DIF) analysis using a cultural neuropsychological approach. We examined relationships among harmonized scores, age, and education.

RESULTS

We included 3170 participants from the HRS-HCAP (Mage = 76.6 [standard deviation (SD): 7.5], 60% female) and 2042 participants from the Mex-Cog (Mage = 68.1 [SD: 9.0], 59% female). Five of seven memory items and one of twelve language items demonstrated DIF by study. Harmonized memory and language scores showed expected associations with age and education.

DISCUSSION

A cultural neuropsychological approach to harmonization facilitates the generation of harmonized measures of memory and language function in cross-national studies.

HIGHLIGHTS

We harmonized memory and language scores across studies in the United States and Mexico.A cultural neuropsychological approach to data harmonization was used.Harmonized scores showed minimal measurement differences between cohorts.Future work can use these harmonized scores for cross-national studies of Alzheimer's disease and related dementias.

Differences in the measurement of cognition for the assessment of dementia across geographic contexts: Recommendations for cross-national research

INTRODUCTION

Most cognitive assessments have been developed in high-income countries but are used in diverse contexts. Differences in culture and context may affect the performance of cognitive items.

METHODS

We used the Harmonized Cognitive Assessment Protocol (HCAP) surveys in the United States, Mexico, India, England, and South Africa (combined N = 11,364) to quantify associations across countries between cognitive items and cognitive impairment status using age- and sex-adjusted logistic regression.

RESULTS

Associations were stronger in the United States (median odds ratio [OR] across items = 0.17) and England (median OR = 0.19), compared to South Africa (median OR = 0.23), India (median OR = 0.29), and Mexico (median OR = 0.28). Items assessing memory (e.g., delayed recall tasks) had the most consistent associations of the largest magnitudes across contexts.

DISCUSSION

Transporting cognitive items among countries and cultures warrants caution. Our results can guide the design of future instruments by identifying items that performed well either in individual contexts or across the range of contexts considered.

HIGHLIGHTS

Little quantitative evidence exists to guide the design of cognitive assessments in cross-national studies. The performance of cognitive items for the measurement of dementia varied across countries. Items with lower variation across countries (e.g., delayed word recall) should be used in future cross-national assessments. Across countries, there was variability in the performance of language assessments, with the exception of the animal naming task. Results can be used to design future cross-national or location-specific cognitive assessments.

The application of cross-sectionally derived dementia algorithms to longitudinal data in risk factor analyses

PURPOSE

Dementia algorithms are often developed in cross-sectional samples but implemented in longitudinal studies to ascertain incident dementia. However, algorithm performance may be higher in cross-sectional settings, and this may impact estimates of risk factor associations.

METHODS

We used data from the Religious Orders Study and the Memory and Aging Project (N = 3460) to assess the performance of example algorithms in classifying prevalent dementia in cross-sectional samples versus incident dementia in longitudinal samples. We used an applied example and simulation study to characterize the impact of varying sensitivity, specificity, and unequal sensitivity or specificity between exposure groups (differential performance) on estimated hazard ratios from Cox models.

RESULTS

Using all items, algorithm sensitivity was higher for prevalent (0.796) versus incident dementia (0.719); hazard ratios had slight bias. Sensitivity differences were larger using a subset of items (0.732 vs. 0.600) and hazard ratios were 13%-19% higher across adjustment sets compared to estimates using gold-standard dementia status. Simulations indicated specificity and differential algorithmic performance between exposure groups may have large effects on hazard ratios.

CONCLUSIONS

Algorithms developed using cross-sectional data may be adequate for longitudinal settings when performance is high and non-differential. Poor specificity or differential performance between exposure groups may lead to biases.

The effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods

Background

Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.

Methods

We used data from Visit 6 (2016–2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.

Results

When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.

Conclusions

Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01572-2.

Differential item functioning to validate setting of delivery compatibility in PROMIS-global health

PURPOSE

Patient-reported outcomes measures (PROMs) such as PROMIS are increasingly utilized in healthcare to assess patient perception and functional status, but the effect of delivery setting remains to be fully investigated. To our knowledge, no current study establishes the absence of differential item functioning (DIF) across delivery setting for these PROMIS- Global Health (PROMIS-GH) measures among orthopedic patients. We sought to investigate the correlation of PROMIS-GH scores across in-clinic versus remote delivery by evaluating DIF within the Global Physical Health (GPH) and Global Mental Health (GMH) items. We hypothesize that the setting of delivery of the GPH and GMH domains of PROMIS-GH will not impact the results of the measure, allowing direct comparison between the two delivery settings.

METHODS

Five thousand and seven hundred and eighty-five complete PROMIS-Global Health measures were analyzed retrospectively using the 'Lordif' package on the R platform. DIF was measured for GPH and GMH domains across setting of response (in-clinic vs remote) during the pre-operative period, immediate post-operative period, and 1-year post-operative period using Monte Carlo estimation. McFadden pseudo-R2 thresholds (> 0.02) were used to assess the magnitude of DIF for individual PROMIS items.

RESULTS

No GPH or GMH items contained in the PROMIS-GH instrument yielded DIF across in-clinic vs remote delivery setting during the pre-operative, immediate post-operative, or 1-year post-operative window.

CONCLUSION

The GPH and GMH domains within the PROMIS-GH instrument may be delivered in the clinic or remotely with comparable accuracy. This cross-delivery setting validation analysis may aid to improve the quality of patient care by allowing mixed platform PROMIS-GH data tailored to individual patient circumstance.

You Say Tomato, I Say Radish: Can Brief Cognitive Assessments in the U.S. Health Retirement Study Be Harmonized With Its International Partner Studies?

OBJECTIVES

To characterize the extent to which brief cognitive assessments administered in the population-representative U.S. Health and Retirement Study (HRS) and its International Partner Studies can be considered to be measuring a single, unidimensional latent cognitive function construct.

METHODS

Cognitive function assessments were administered in face-to-face interviews in 12 studies in 26 countries (N = 155,690), including the U.S. HRS and selected International Partner Studies. We used the time point of the first cognitive assessment for each study to minimize differential practice effects across studies and documented cognitive test item coverage across studies. Using confirmatory factor analysis models, we estimated single-factor general cognitive function models and bifactor models representing memory-specific and nonmemory-specific cognitive domains for each study. We evaluated model fits and factor loadings across studies.

RESULTS

Despite relatively sparse and inconsistent cognitive item coverage across studies, all studies had some cognitive test items in common with other studies. In all studies, the bifactor models with a memory-specific domain fit better than single-factor general cognitive function models. The data fit the models at reasonable thresholds for single-factor models in 6 of the 12 studies and for the bifactor models in all 12 of the 12 studies.

DISCUSSION

The cognitive assessments in the U.S. HRS and its International Partner Studies reflect comparable underlying cognitive constructs. We discuss the assumptions underlying our methods, present alternatives, and future directions for cross-national harmonization of cognitive aging data.

Use of multidimensional item response theory methods for dementia prevalence prediction: an example using the Health and Retirement Survey and the Aging, Demographics, and Memory Study

BACKGROUND

Data sparsity is a major limitation to estimating national and global dementia burden. Surveys with full diagnostic evaluations of dementia prevalence are prohibitively resource-intensive in many settings. However, validation samples from nationally representative surveys allow for the development of algorithms for the prediction of dementia prevalence nationally.

METHODS

Using cognitive testing data and data on functional limitations from Wave A (2001-2003) of the ADAMS study (n = 744) and the 2000 wave of the HRS study (n = 6358) we estimated a two-dimensional item response theory model to calculate cognition and function scores for all individuals over 70. Based on diagnostic information from the formal clinical adjudication in ADAMS, we fit a logistic regression model for the classification of dementia status using cognition and function scores and applied this algorithm to the full HRS sample to calculate dementia prevalence by age and sex.

RESULTS

Our algorithm had a cross-validated predictive accuracy of 88% (86-90), and an area under the curve of 0.97 (0.97-0.98) in ADAMS. Prevalence was higher in females than males and increased over age, with a prevalence of 4% (3-4) in individuals 70-79, 11% (9-12) in individuals 80-89 years old, and 28% (22-35) in those 90 and older.

CONCLUSIONS

Our model had similar or better accuracy as compared to previously reviewed algorithms for the prediction of dementia prevalence in HRS, while utilizing more flexible methods. These methods could be more easily generalized and utilized to estimate dementia prevalence in other national surveys.

Sex Differences in Cognitive Health Among Older Adults in India

OBJECTIVES

To document sex differences in late-life cognitive function and identify their early-life determinants among older Indian adults.

DESIGN

Harmonized Diagnostic Assessment of Dementia for Longitudinal Aging Study in India (LASI-DAD).

SETTING

Individual cognitive testing in hospital or household setting across 14 states of India.

PARTICIPANTS

Individuals aged 60 years and older from LASI-DAD (2017-2019) (N = 2,704; 53.5% female).

MEASUREMENTS

Given the low levels of literacy and numeracy among older Indian adults, we consider two composite cognitive scores as outcome variables. Score I is based on tests that do not require literacy or numeracy, whereas score II is based on tests that require such skills. Ordinary least squares is used to estimate models featuring a progressively increasing number of covariates. We add to the baseline specification, including a sex dummy, age, and state indicators, measures of early-life socioeconomic status (SES), early-life nutrition, as proxied by knee height, and education.

RESULTS

Across most cognitive domains, women perform significantly worse than for men: -0.4 standard deviations (SD) for score I and -0.8 SD for score II. Early-life SES, health, and education explain 90% of the gap for score I and 55% for score II. Results are similar across hospital-based and home testing.

CONCLUSION

In India, lower levels of early-life human capital investments in nutrition and education among women compared with men are associated with a female disadvantage in late-life cognitive health. This has important implications for public health policy, aiming at reducing the risk of cognitive decline and dementia-a nascent concern in India. J Am Geriatr Soc 68:S20-S28, 2020.

Lp Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups

The comparison of group means in latent variable models plays a vital role in empirical research in the social sciences. The present article discusses an extension of invariance alignment and Haberman linking by choosing the robust power loss function ρ(x)=|x|p(p>0). This power loss function with power values p smaller than one is particularly suited for item responses that are generated under partial invariance. For a general class of linking functions, asymptotic normality of estimates is shown. Moreover, the theory of M-estimation is applied for obtaining linking errors (i.e., inference with respect to a population of items) for this class of linking functions. In a simulation study, it is shown that invariance alignment and Haberman linking have comparable performance, and in some conditions, the newly proposed robust Haberman linking outperforms invariance alignment. In three examples, the influence of the choice of a particular linking function on the estimation of group means is demonstrated. It is concluded that the choice of the loss function in linking is related to structural assumptions about the pattern of noninvariance in item parameters.