Modifying measures based on differential item functioning (DIF) impact analyses

Measurement Invariance in Intellectual and Developmental Disability Research

Measurement invariance (MI) is a psychometric property of an instrument indicating the degree to which scores from an instrument are comparable across groups. In recent years, there has been a marked uptick in publications using MI in intellectual and developmental disability (IDD) samples. Our goal here is to provide an overview of why MI is important to IDD researchers and to describe some challenges to evaluating it, with an eye towards nudging our subfield into a more thoughtful and measured interpretation of studies using MI.

Establishing language and ethnic equivalence for health-related quality of life item banks and testing their efficiency via computerised adaptive testing simulations

PURPOSE

We aimed to (1) establish linguistic and ethnic equivalence (i.e. lack of bias) for the items in the English and Chinese versions of the Singapore Health and Well Being (SHAWS) Physical Functioning (PF), Positive Mindset (PM) and Social Relationship (SR) item banks (IBs); and (2) evaluate the preliminary efficiency of these IBs using Computer Adaptive Testing (CAT) simulations.

METHODS

In this cross-sectional study, 671, 670, and 672 subjects answered 55, 48 and 30 items of the PF, PM, and SR IBs, respectively. Rasch analysis was conducted to assess each IB's psychometric properties, particularly the presence of differential item functioning (DIF) for language and ethnicity. A set of performance criteria related to removing items that displayed notable DIF were employed. CAT simulations determined the mean number of items for high, moderate, and moderate-low measurement precisions (stopping rule: SEM 0.300, 0.387. 0.521, respectively).

RESULTS

Half of subjects were >50 years old (40.9% PF, 42.1% PM, 41.4% SR), Chinese (50.7% PF, 51.0% PM, 50.6% SR) and female (50.0% PF. 49.4% PM, 52.8% SR) respectively. Rasch analysis revealed 4 items with DIF for the PF IB, 9 items with DIF for the PM IB and 2 items with DIF for the SR IB. In CAT simulations, the mean number of items administered was 8.5, 21.6 and 14.5 for the PF, PM and SR IBs, respectively (SEM 0.300), 5.1, 13.0, 8.0 for PF, PM and SR IBs, respectively (SEM 0.387) and 3.1, 5.3 and 4.1 for PF, PM and SR IBs, respectively (SEM 0.521).

CONCLUSION

The PF, PM and SR IBs to measure health-related quality of life revealed minimal DIF for language and ethnicity after remedial efforts. CAT simulations demonstrated that these IBs were efficient, especially when the stopping rule was set at moderate precision, and support the implementation of the SHAWS IBs into routine clinical care.

Gender-Based Differential Item Function for the Positive and Negative Semantic Dimensions of the Relationship Satisfaction Scale with Item Response Theory

Relationship satisfaction is at the core of a robust social life and is essential to mental health. The positive and negative semantic dimensions of the relationship satisfaction (PN-SMD) scale is considered in the field of relationship studies to be a reliable tool for assessing the quality of a person's interpersonal relationships. This study evaluated the psychometric properties of the PN-SMD scale by conducting multidimensional item response theory (MIRT) and differential item functioning (DIF) analyses, both of which are emerging assessment methods that focus on individual items. We recruited 511 Chinese undergraduate students for this study. Construct validity, internal consistency, and concurrent validity were assessed, and MIRT and DIF analyses were conducted. Five of the 14 items were found to have gender-based DIF traits, affecting the scale's construct validity. A revised nine-item scale (DIF items excluded) had a significantly better model fit and demonstrated comparable concurrent validity to the original scale. The implications of our results and future research directions are discussed.

Different Responses to Social Capital Among Black People and White People: What Racial Differential Item Functioning Reveals for Racial Health Equity

Social capital has been conceptualized as features of social organization, such as networks, and norms that facilitate coordination and cooperation for mutual benefit. Because of long-standing anti-Black structural oppression in the United States, social capital may be associated with health differently for Black people than for other racial/ethnic groups. Our aim was to examine the psychometric properties of social capital indicators, comparing responses from Black and White people to identify whether there is differential item functioning (DIF) in social capital according to race. DIF examines how items are related to a latent construct and whether this relationship differs across groups such as different racial groups. We used data from respondents to the Southeastern Pennsylvania Household Health Survey in 2004, who lived in Philadelphia (n = 2,048), a city with a large Black population. We used item response theory analysis to test for racial DIF. We found DIF across the items, indicating measurement error, which could be related to the way these items were developed (i.e., based on cultural assumptions tested in mainstream White America). Hence, our findings underscore the need to interrogate the assumptions that underly existing social capital items through an equity-based lens, and to take corrective action when developing new items to ensure that they are racially and culturally congruent.

Animal welfare knowledge, attitudes, and practices among livestock holders in Ethiopia

Improving animal welfare is a human responsibility and influenced by a person's values and experiences. Thus, it is critical to have an in-depth understanding of the knowledge, attitudes, and practices (KAP) of animal welfare among animal owners. For livestock in Ethiopia, the greatest proportion of livestock are reared by pastoral and mixed crop-livestock communities. A cross-sectional survey covering a range of species and animal welfare aspects was carried out on a total of 197 household (117 pastoral and 80 crop-livestock owners) and recorded information on 34 animal welfare KAP items. Item response theory models (IRT) were fitted to the data from KAP items to estimate the probability of correctly answering an item. This was used as a function of the respondents' KAP level. Overall, the highest percentage of desirable scores was recorded for the knowledge scale (35.7%) and the lowest was for the practice scale (24.6%). A significant correlation (P < 0.01) was found between knowledge of the farmers and their attitude toward animal welfare and self-reported practices. Generally, households practicing mixed crop-livestock farming system had better animal welfare knowledge, attitude, and practice than pastoralist. Mixed crop-livestock farmers had better knowledge on items related to observing the nutrition condition of the animal, animal-human relationship, the importance of water, and health inspection compared to pastoralists. In contrast, pastoralists had better knowledge of items related to natural behavior expression, animal care, and animal suffering than mixed crop-livestock farmers. Pastoralists had 3.3-times higher odds than mixed crop-livestock farmers to have a positive attitude to train their animals without beating. KAP scores demonstrate the need for targeted training to improve animal well-being (i.e., housing, management, nutrition, disease prevention and treatment, responsible care, humane handling) across livestock holding communities in Ethiopia.

The effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods

Background

Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.

Methods

We used data from Visit 6 (2016–2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.

Results

When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.

Conclusions

Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01572-2.

Reliability and Validity of the Cornell Scale for Depression in Dementia and Invariance Between Black Versus White Residents in Nursing Homes

OBJECTIVES

The purpose of this study was to expand on prior work testing invariance on several depression measures in community-based older adults and explore the psychometric properties and evidence of invariance between racial groups based on the Cornell Scale for Depression in Dementia.

DESIGN

This was a descriptive measurement study.

SETTING AND PARTICIPANTS

This was a secondary data analysis using baseline data from 2 studies: Testing the Implementation of the Evidence Integration Triangle for Behavioral and Psychological Symptoms Associated with Dementia and the study Testing the Impact of Function and Behavior Focused Care for Nursing Home Residents with Dementia. Combined, 67 nursing homes participated from 2 states and 889 residents were recruited. The mean age of the participants was 86.58 (SD 10.31) and most were women (72%) and White (70%).

METHODS

This was a descriptive study, and a Rasch analysis was done to establish reliability based on internal consistency and evidence of differential item functioning (DIF) across races. Validity was based on item fit and model testing with structural equation modeling to compare models between White and Black participants.

RESULTS

There was evidence of internal consistency (alpha coefficient of 0.98) and no significant evidence of DIF. The item related to suicide had a high logit and did not significantly load onto the measurement model for Black individuals. There was not a good spread of the items across the concept of depression. The model had a better fit with the items when used with White versus Black participants.

CONCLUSIONS AND IMPLICATIONS

The findings indicate that it would be helpful to add some additional items that reflect depressive symptoms among this population. Further, the findings serve as a reminder that this measure may be biased toward identification of symptoms of depression among White versus Black residents.

Differential item functioning of the Beck Anxiety Inventory in a rural, multi-ethnic cohort

BACKGROUND

Evaluating measurement bias is vital to ensure equivalent assessment across diverse groups. One approach for evaluating test bias, differential item functioning (DIF), assesses item-level bias across specified groups by comparing item-level responses between groups that have the same overall score. Previous DIF studies of the Beck Anxiety Inventory (BAI) have only assessed bias across age, sex, and disease duration in monolingual samples. We expand this literature through DIF analysis of the BAI across age, sex, education, ethnicity, cognitive status, and test language.

METHODS

BAI data from a sample (n = 527, mean age=61.4 ± 12.7, mean education=10.9 ± 4.3, 69.3% female, 41.9% Hispanic/Latin American) from rural communities in West Texas, USA were analyzed. Item response theory (IRT) / logistic ordinal regression DIF was conducted across dichotomized demographic grouping factors. The Mann-Whitney U test and Hedge's g standardized mean differences were calculated before and after adjusting for the impact of DIF.

RESULTS

Significant DIF was demonstrated in 10/21 items. An adverse impact of DIF was not identified when demographics were assessed individually. Adverse DIF was identified for only one participant (1/527, 0.2%) when all demographics were aggregated.

LIMITATIONS

These results might not be generalizable to a sample with broader racial representation, more severe cognitive impairment, and higher levels of anxiety.

CONCLUSIONS

Minimal item-level bias was identified across demographic factors considered. These results support prior evidence that the BAI is valid for assessing anxiety across age and sex while contributing new evidence of its clinical relevance across education, ethnicity, cognitive status, and English/Spanish test language.

Measuring Problematic Facebook Use among Adolescents and Young Adults with the Bergen Facebook Addiction Scale: A Psychometric Analysis by Applying Item Response Theory

The Bergen Facebook Addiction Scale (BFAS) is widely used, but psychometric evidence by applying Item Response Theory (IRT) is lacking. Considering the advantages of this psychometric approach, the aim of study was to investigate the psychometric properties of the Italian version of the BFAS among adolescents and young adults. Participants were 1134 (50% males, Mean age = 20.7, SD = 3.5, range = 14-33 years) Italian high school students and undergraduates. The unidimensionality of the scale was confirmed (χ2/df = 2.8, CFI = 0.99, TLI = 0.98, and RMSEA = 0.04 [C.I. = 0.02-0.06]) and IRT analysis showed that the scale assesses medium and high levels of the trait, and that it is useful in order to discriminate different levels of Problematic Facebook use (PFU) within this range of trait, in which the scale is sufficiently informative. The relationships of BFAS θ scores with theoretically related constructs provided support to the validity of the scale. In accordance with previous studies, BFAS scores were positively correlated with Problematic Internet use and problematic Social Network use, negatively correlated with self-esteem, and positively related to loneliness. The Differential Item Functioning (DIF) analysis showed that BFAS is invariant across gender, and only one item had uniform and small-in-size DIF. Additionally, we tested age invariance. Since only 17% of the BFAS items were non-invariant, we determined that the BFAS exhibited minor non-invariance as a whole. An analysis of the adequacy of the polythetic and monothetic criteria to define the range of the trait indicative of problematic use was also conducted. Overall, this study offers evidence that BFAS is a valuable and useful scale for measuring high levels of PFU among Italian adolescents and young adults.

Differential Item Functioning Based on Autism Features, IQ, and Age on the Screen for Child Anxiety Related Disorders (SCARED) Among Youth on the Autism Spectrum

Anxiety commonly occurs among youth on the autism spectrum, yet measurement of anxiety in this population is complicated by a number of factors, including potentially overlapping symptomatology, the child's intellectual functioning, and changes in anxiety across development. Moreover, few studies have examined the psychometric properties of anxiety measures in this population, and no study to date has tested whether there are systematic differences in the measurement of anxiety, or differential item functioning (DIF), across the high degree of heterogeneity and the developmental course of autism. To test this possibility, data were combined across multiple studies using the National Database for Autism Research, an NIH-funded data repository. Parent-report on the Screen for Child Anxiety Related Disorders (SCARED) and Social Communication Questionnaire (SCQ) were used as measures of anxiety and autism features, respectively. A confirmatory factor analysis indicated good fit of the literature standard five-factor structure. Moderated nonlinear factor analysis (MNLFA) revealed multiple items with intercept and loading DIF based on level of autism features, IQ, and age, especially for items related to social behavior. Therefore, although the measure's factor structure is consistent with that found in the general population, the SCARED may not capture differences in anxiety equivalently for all children on the spectrum and across their development. Clinicians and researchers need to be especially vigilant in measuring anxiety symptoms in children with autism by removing items flagged for DIF from the SCARED and/or by using multiple measures and informants. LAY ABSTRACT: Autistic youth often experience clinical levels of anxiety. Many tools used to measure anxiety were developed for the general population, but not for use with autistic youth. This study found that the Screen for Child Anxiety Related Disorders (SCARED) measures the same five dimensions of anxiety as in the general population. Parents, however, may respond differently to questions on the SCARED based on their child's autism features, intellectual functioning, and age, which impacts our ability to accurately measure anxiety among autistic youth.

Universal Design for Measurement: Centering the Experiences of Individuals With Disabilities Within Health Measurement Research

People with disabilities comprise roughly 25% of the U.S. adult population yet remain underrepresented in mainstream public health and evaluation research. The lack of measures of common constructs that are validated in but not specific to this population may impede their inclusion. This article describes the use of Universal Design for Measurement (UDM), a novel method for developing self-report measurement instruments validated among broad populations to minimize the need for scale adaptation. We applied UDM to the development and content validation of a new body image scale. We assessed content validity by surveying subject matter experts (SMEs) and conducted a Delphi panel study to assess consensus about scale items among community women with (n = 18) and without (n = 15) disabilities. Most scale items were found acceptable by SMEs and community women. The Delphi panel study was useful toward evaluating consensus about scale items among women with and without disabilities. Findings support the use of UDM in developing inclusive and psychometrically sound measurement scales to ultimately facilitate the full inclusion of people with disabilities within health research.

State of the psychometric methods: patient-reported outcome measure development and refinement using item response theory

BACKGROUND

This paper is part of a series comparing different psychometric approaches to evaluate patient-reported outcome (PRO) measures using the same items and dataset. We provide an overview and example application to demonstrate 1) using item response theory (IRT) to identify poor and well performing items; 2) testing if items perform differently based on demographic characteristics (differential item functioning, DIF); and 3) balancing IRT and content validity considerations to select items for short forms.

METHODS

Model fit, local dependence, and DIF were examined for 51 items initially considered for the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Depression item bank. Samejima's graded response model was used to examine how well each item measured severity levels of depression and how well it distinguished between individuals with high and low levels of depression. Two short forms were constructed based on psychometric properties and consensus discussions with instrument developers, including psychometricians and content experts. Calibrations presented here are for didactic purposes and are not intended to replace official PROMIS parameters or to be used for research.

RESULTS

Of the 51 depression items, 14 exhibited local dependence, 3 exhibited DIF for gender, and 9 exhibited misfit, and these items were removed from consideration for short forms. Short form 1 prioritized content, and thus items were chosen to meet DSM-V criteria rather than being discarded for lower discrimination parameters. Short form 2 prioritized well performing items, and thus fewer DSM-V criteria were satisfied. Short forms 1-2 performed similarly for model fit statistics, but short form 2 provided greater item precision.

CONCLUSIONS

IRT is a family of flexible models providing item- and scale-level information, making it a powerful tool for scale construction and refinement. Strengths of IRT models include placing respondents and items on the same metric, testing DIF across demographic or clinical subgroups, and facilitating creation of targeted short forms. Limitations include large sample sizes to obtain stable item parameters, and necessary familiarity with measurement methods to interpret results. Combining psychometric data with stakeholder input (including people with lived experiences of the health condition and clinicians) is highly recommended for scale development and evaluation.

Using the Short Form of the MSBS to Assess State Boredom Among Adolescents: Psychometric Evidence by Applying Item Response Theory

The Multidimensional State Boredom Scale (MSBS) is widely used, but evidence regarding its psychometric properties among adolescents is lacking. In particular, the functioning of the scale across genders is unknown. As a result, we used item response theory (IRT) to investigate gender invariance of the Short Form of the MSBS (MSBS-SF) among adolescents. Four hundred and sixty-six Italian high school students (51% male; M = 16.7, SD = 1.44) were recruited. A confirmatory factor analysis demonstrated the unidimensionality of the scale, and IRT analyses indicated that the scale was sufficiently informative. Differential item functioning (DIF) across genders showed that only one item had DIF that was both nonuniform and small in size. Additionally, relationships with negative/positive urgency and present/future-oriented time perspectives were found. Overall, this study offers evidence that the MSBS-SF is a valuable and useful scale for measuring state boredom among male and female adolescents.

Study protocol for the validation of a new patient-reported outcome measure (PROM) of listening effort in cochlear implantation: the Listening Effort Questionnaire-Cochlear Implant (LEQ-CI)

INTRODUCTION

Listening effort may be defined as the cognitive resources needed to understand an auditory message. A sustained requirement for listening effort is known to have a negative impact on individuals' sense of social connectedness, well-being and quality of life. A number of hearing-specific patient-reported outcome measures (PROMs) exist currently; however, none adequately assess listening effort as it is experienced in the listening situations of everyday life. The Listening Effort Questionnaire-Cochlear Implant (LEQ-CI) is a new, hearing-specific PROM designed to assess perceived listening effort as experienced by adult CI patients. It is the aim of this study to conduct the first psychometric evaluation of the LEQ-CI's measurement properties.

METHODS AND ANALYSIS

This study is a phased, prospective, multi-site validation study in a UK population of adults with severe-profound sensorineural hearing loss who meet local candidacy criteria for CI. In phase 1, 250 CI patients from four National Health Service CI centres will self-complete a paper version of the LEQ-CI. Factor analysis will establish unidimensionality and Rasch analysis will evaluate item fit, differential item functioning, response scale ordering, targeting of persons and items, and reliability. Classical test theory methods will assess acceptability/data completeness, scaling assumptions, targeting and internal consistency reliability. Phase 1 results will inform refinements to the LEQ-CI. In phase 2, a new sample of adult CI patients (n=100) will self-complete the refined LEQ-CI, the Speech, Spatial and Qualities of Hearing Scale, the Nijmegen Cochlear Implant Questionnaire and the Fatigue Assessment Scale to assess construct validity.

ETHICS AND DISSEMINATION

This study was approved by the Abertawe Bro Morgannwg University Health Board/Swansea University Joint Study Review Committee and the Newcastle and North Tyneside 2 Research Ethics Committee, Ref: 18/NE/0320. Dissemination will be in high-quality journals, conference presentations and SEH's doctoral dissertation.

Item response theory analyses of the Delis-Kaplan Executive Function System card sorting subtest

In the current study, we examined the dimensionality of the 16-item Card Sorting subtest of the Delis-Kaplan Executive Functioning System assessment in a sample of 264 native English-speaking children between the ages of 9 and 15 years. We also tested for measurement invariance for these items across age and gender groups using item response theory (IRT). Results of the exploratory factor analysis indicated that a two-factor model that distinguished between verbal and perceptual items provided the best fit to the data. Although the items demonstrated measurement invariance across age groups, measurement invariance was violated for gender groups, with two items demonstrating differential item functioning for males and females. Multigroup analysis using all 16 items indicated that the items were more effective for individuals whose IRT scale scores were relatively high. A single-group explanatory IRT model using 14 non-differential item functioning items showed that for perceptual ability, females scored higher than males and that scores increased with age for both males and females; for verbal ability, the observed increase in scores across age differed for males and females. The implications of these findings are discussed.

The use of latent variable mixture models to identify invariant items in test construction

Purpose

Patient-reported outcome measures (PROMs) are frequently used in heterogeneous patient populations. PROM scores may lead to biased inferences when sources of heterogeneity (e.g., gender, ethnicity, and social factors) are ignored. Latent variable mixture models (LVMMs) can be used to examine measurement invariance (MI) when sources of heterogeneity in the population are not known a priori. The goal of this article is to discuss the use of LVMMs to identify invariant items within the context of test construction.

Methods

The Draper-Lindely-de Finetti (DLD) framework for the measurement of latent variables provides a theoretical context for the use of LVMMs to identify the most invariant items in test construction. In an expository analysis using 39 items measuring daily activities, LVMMs were conducted to compare 1- and 2-class item response theory models (IRT). If the 2-class model had better fit, item-level logistic regression differential item functioning (DIF) analyses were conducted to identify items that were not invariant. These items were removed and LVMMs and DIF testing repeated until all remaining items showed MI.

Results

The 39 items had an essentially unidimensional measurement structure. However, a 1-class IRT model resulted in many statistically significant bivariate residuals, indicating suboptimal fit due to remaining local dependence. A 2-class LVMM had better fit. Through subsequent rounds of LVMMs and DIF testing, nine items were identified as being most invariant.

Conclusions

The DLD framework and the use of LVMMs have significant potential for advancing theoretical developments and research on item selection and the development of PROMs for heterogeneous populations.

Measurement invariance across Genders on the Childhood Illness Attitude Scales (CIAS)

OBJECTIVE

The Childhood Illness Attitude Scales (CIAS) were created as a developmentally appropriate measure for symptoms of health anxiety (HA) in school-aged children. Despite overall sound psychometric properties reported in previous studies, more comprehensive examination of the latent structure and potential response bias in the CIAS is needed. The purpose of the present study was to cross-validate the latent structure of the CIAS across genders and to examine gender-specific variations in CIAS scores.

METHODS

The sample comprised data from 602 Canadian and Danish school-aged children (M_age=10.54, SD=0.99; 52.5% girls). Confirmatory factor analyses were conducted to test 3-, modified 3-, and 4-factor models in both samples. Multigroup confirmatory factor analysis was performed to test factor structure invariance across boys and girls in a combined sample. Differential Item Functioning (DIF) was assessed using test characteristic curves.

RESULTS

A modified 3-factor solution (i.e., fears=11 items, help-seeking=6 items, and symptom effects=4 items) provided the best fit to the data (χ² (364, N=602)=681.7, p<0.001; χ²/df=1.803; RMSEA=0.037; CFI=0.926). The factor structure was stable, well-fitting, and indicated measurement invariance across groups. DIF analyses revealed no gender-based response bias at the scale level.

CONCLUSION

Results support a revised 3-factor version of the CIAS that can be used with confidence to assess symptoms of HA in school-aged boys and girls.

Do people with and without medical conditions respond similarly to the short health anxiety inventory? An assessment of differential item functioning using item response theory

OBJECTIVE

Individuals with medical conditions are likely to have elevated health anxiety; however, research has not demonstrated how medical status impacts response patterns on health anxiety measures. Measurement bias can undermine the validity of a questionnaire by overestimating or underestimating scores in groups of individuals. We investigated whether the Short Health Anxiety Inventory (SHAI), a widely-used measure of health anxiety, exhibits medical condition-based bias on item and subscale levels, and whether the SHAI subscales adequately assess the health anxiety continuum.

METHODS

Data were from 963 individuals with diabetes, breast cancer, or multiple sclerosis, and 372 healthy individuals. Mantel-Haenszel tests and item characteristic curves were used to classify the severity of item-level differential item functioning in all three medical groups compared to the healthy group. Test characteristic curves were used to assess scale-level differential item functioning and whether the SHAI subscales adequately assess the health anxiety continuum.

RESULTS

Nine out of 14 items exhibited differential item functioning. Two items exhibited differential item functioning in all medical groups compared to the healthy group. In both Thought Intrusion and Fear of Illness subscales, differential item functioning was associated with mildly deflated scores in medical groups with very high levels of the latent traits. Fear of Illness items poorly discriminated between individuals with low and very low levels of the latent trait.

CONCLUSIONS

While individuals with medical conditions may respond differentially to some items, clinicians and researchers can confidently use the SHAI with a variety of medical populations without concern of significant bias.

Differential reporting of depressive symptoms across distinct clinical subpopulations: what DIFference does it make?

OBJECTIVE

To investigate the impact of differences in depressive symptom reporting across clinical groups (healthcare setting, chronic illness, depression diagnosis and anxiety diagnosis) on clinical interpretability and comparability of depression scores.

METHODS

Participants from the Netherlands Study of Depression and Anxiety (n=2981) completed the self-report Inventory of Depressive Symptomatology (IDS-SR). Differences in depressive symptom reporting between distinct clinical subpopulations were assessed using a Differential Item Functioning (DIF) analysis. The effects of DIF on symptom level were evaluated by examining whether DIF-adjustment had clinically relevant effects.

RESULTS

Significant DIF was detected across all tested clinical subpopulation groupings. Clinically relevant DIF was found on the symptom level for 13 IDS-SR items. However, impact of DIF on the aggregate level ranged from small to negligible: adjustment for DIF only led to salient changes in aggregate scores for 0.2-12.7% of individuals across tested sources of DIF.

CONCLUSION

Differences in endorsement patterns of depressive symptoms were observed across clinical populations, challenging the assumptions regarding the measurement properties of self-reported depression. However, effects of DIF on the aggregate level of IDS-SR total scores were found to be minimal and not clinically important. The IDS-SR thus seems robust against DIF across clinical populations.

Demographic characteristics do not decrease the utility of depressive symptoms assessments: examining the practical impact of item bias in four heterogeneous samples of older adults

OBJECTIVE

Previous studies have identified differential item function (DIF) in depressive symptoms measures, but the impact of DIF has been rarely reported. Given the critical importance of depressive symptoms assessment among older adults, we examined whether DIF due to demographic characteristics resulted in salient score changes in commonly used measures.

METHODS

Four longitudinal studies of cognitive aging provided a sample size of 3754 older adults and included individuals both with and without a clinical diagnosis of major depression. Each study administered at least one of the following measures: the Center for Epidemiologic Studies Depression scale (20-item ordinal response or 10-item dichotomous response versions), the Geriatric Depression Scale, and the Montgomery-Åsberg Depression Rating Scale. Hybrid logistic regression-item response theory methods were used to examine the presence and impact of DIF due to age, sex, race/ethnicity, and years of education on the depressive symptoms items.

RESULTS

Although statistically significant DIF due to demographic factors was present on several items, its cumulative impact on depressive symptoms scores was practically negligible.

CONCLUSIONS

The findings support substantive meaningfulness of previously reported demographic differences in depressive symptoms among older adults, showing that these individual differences were unlikely to have resulted from item bias attributable to demographic characteristics we examined.

Few items in the thyroid-related quality of life instrument ThyPRO exhibited differential item functioning

OBJECTIVE

To evaluate the extent of differential item functioning (DIF) within the thyroid-specific quality of life patient-reported outcome measure, ThyPRO, according to sex, age, education and thyroid diagnosis.

STUDY DESIGN AND SETTING

A total of 838 patients with benign thyroid diseases completed the ThyPRO questionnaire (84 five-point items, 13 scales). Uniform and nonuniform DIF were investigated using ordinal logistic regression, testing for both statistical significance and magnitude (∆R(2) > 0.02). Scale level was estimated by the sum score, after purification.

RESULTS

Twenty instances of DIF in 17 of the 84 items were found. Eight according to diagnosis, where the goiter scale was the one most affected, possibly due to differing perceptions in patients with auto-immune thyroid diseases compared to patients with simple goiter. Eight DIFs according to age were found, of which 5 were in positively worded items, which younger patients were more likely to endorse; one according to gender: women were more likely to report crying, and three according to educational level. The vast majority of DIF had only minor influence on the scale scores (0.1-2.3 points on the 0-100 scales), but two DIF corresponded to a difference of 4.6 and 9.8, respectively.

CONCLUSION

Ordinal logistic regression identified DIF in 17 of 84 items. The potential impact of this on the present scales was low, but items displaying DIF could be avoided when developing abbreviated scales, where the potential impact of DIF (due to fewer items) will be larger.