Depressive Response Sets due to gender and culture-based Differential Item Functioning

Differential Item Functioning of the Mini-BESTest Balance Measure: A Rasch Analysis Study

The Mini-Balance Evaluation Systems Test (Mini-BESTest), a 14-item scale, has high content validity for balance assessment. This study further examines the construct validity of the Mini-BESTest with an emphasis on its measurement invariance. The Mini-BESTest was administered to 292 neurological patients in two sessions (before and after rehabilitation) and evaluated with the Rasch analysis (Many-Facet Rating Scale Model: persons, items, sessions). Categories' order and fit to the model were assessed. Next, maps, dimensionality, and differential item functioning (DIF) were examined for construct validity evaluation. DIF was inspected for several clinically important variables, including session, diagnosis, and assistive devices. Mini-BESTest items had ordered categories and fitted the Rasch model. The item map did not flag severe construct underrepresentation. The dimensionality analysis showed that another variable extraneous to balance affected the score of a few items. However, this multidimensionality had only a modest impact on measures. Session did not cause DIF. DIF for assistive devices affected six items and caused a severe measurement artefact. The measurement artefact caused by DIF for diagnosis was negligible. The Mini-BESTest returns interval measures with robust construct validity and measurement invariance. However, caution should be used when comparing Mini-BESTest measures obtained with and without assistive devices.

Measuring stress in Australia: validation of the perceived stress scale (PSS-14) in a national sample

Background

In Australia, the stress levels have increased over the years, impacting on the physical and mental health of the general population. The aim of the present study was to evaluate the validity and reliability of the PSS-14 in an Australian population.

Methods

The PSS-14 was applied to a large national sample comprising 3857 Australians in the population-based cross-sectional study Australia’s National Survey of Adult Oral Health 2004–2006. The psychometric properties analyzed with the Rasch model and Graphical Log-linear Rasch models were: model fit, item fit, local dependence, differential item functioning, unidimensionality, reliability, targeting and criterion validity.

Results

The PSS-14 did not fit the pure RM (χ2 (55) = 3828.3, p = < 0.001) and the unidimensionality of the whole scale was rejected (p = < 0.001). The Perceived Stress (χ2 (27) = 1409.7, p = < 0.001) and Perceived Control (χ2 (27) = 713.4, p = < 0.001) subscales did not fit the pure RM. After the deletion of two items, the Perceived Stress subscale (χ2 (96) = 94.4, p = 0.440) fitted a GLLRM, while the Perceived Control scale (χ2 (55) = 62.50, p = 0.224) fitted a GLLRM after the exclusion of four misfitting items.

Conclusions

The Perceived Stress subscale displayed adequate psychometric properties after the deletion of two items; however, the majority of problems centered around the Perceived Control subscale. The presence of differential item functioning among four items indicates that adjustment of total scores is required to avoid measurement bias. Recommendations for future applications in Australia are provided.

Measuring postnatal demoralisation: adaptation of the Demoralisation Scale-II (DS-II) for postnatal use

OBJECTIVE

To examine the psychometric properties of the Demoralisation Scale II (DS-II) and adapt it for use with women in the postnatal period.

BACKGROUND

Demoralisation is a psychological state characterised by a sense of incompetence and feelings of helplessness and hopelessness in response to a stressful situation. The postnatal period is a life stage of many disruptions. Women may lose their confidence and become demoralised if feeling unprepared for the tasks of motherhood. The DS-II is a 16-item scale developed among cancer patients, but with content that is also relevant postnatally, including items on sense of failure, helplessness, hopelessness, isolation, entrapment and loss of purpose.

METHODS

Rasch analysis was used to investigate the psychometric properties of the DS-II and refine the scale for postnatal use.

RESULTS

Participants were 209 women admitted with their babies to a residential early parenting programme. A 14-item revised scale was derived, the Postnatal DS-II, showing good psychometric properties, discriminant validity and sensitivity to change, and being well targeted to the sample.

CONCLUSION

The Postnatal DS-II could have utility as an assessment tool, helping clinicians to understand better women's postnatal experiences, assess the effectiveness of interventions and communicate with women in a meaningful and non-stigmatising way.

Contrasting depression among African Americans and major depressive disorder in the DSM-V

Purpose

The purpose of this paper is to identify symptoms that constitute a shared cultural model of depression among African Americans and to compare these accounts with criteria for major depressive disorder (MDD) in the 5th edition of the Diagnostic Statistical Manual of mental disorders (DSM-V).

Design/methodology/approach

Data were collected in a disproportionately Black urban neighborhood in the USA and analyzed using cultural consensus analysis (CCA). In total, 34 African Americans participated in a free-listing exercise to elicit common indicators of depression in the same community. Another 40 key informants completed a survey to rate how common each indicator was in the same community. Factor analysis was performed, factor loadings were used to weight the responses of each informant in the survey and then aggregated to determine the most significant indicators or components of the shared model depression.

Findings

Indicators of depression included classic symptoms in the DSM-V such as sadness and lack of motivation. However, other indicators that are inconsistent with symptoms of MDD in the DSM-V such as paranoia and rage were common and constituted a shared model of depression in the sample.

Research limitations/implications

Some symptoms common among African Americans that are not in the DSM-V or on research instruments developed based on the DSM could be overlooked in epidemiological surveys and in clinical assessments of depression.

Practical implications

The provision of mental health care might benefit from a better understanding of how contextual factors shape expressions of distress among African Americans.

Originality/value

This study identify culturally salient symptoms of depression among African Americans independent of clinically defined criteria.

Common measure of quality of life for people with systemic sclerosis across seven European countries: a cross-sectional study

Objectives

The aim of this study was to adapt the Systemic Sclerosis Quality of Life Questionnaire (SScQoL) into six European cultures and validate it as a common measure of quality of life in systemic sclerosis (SSc).

Methods

This was a seven-country (Germany, France, Italy, Poland, Spain, Sweden and UK) cross-sectional study. A forward–backward translation process was used to adapt the English SScQoL into target languages. SScQoL was completed by patients with SSc, then data were validated against the Rasch model. To correct local response dependency, items were grouped into the following subscales: function, emotion, sleep, social and pain and reanalysed for fit to the model, unidimensionality and cross-cultural equivalence.

Results

The adaptation of the SScQoL was seamless in all countries except Germany. Cross-cultural validation included 1080 patients with a mean age 58.0 years (SD 13.9) and 87% were women. Local dependency was evident in individual country data. Grouping items into testlets corrected the local dependency in most country specific data. Fit to the model, reliability and unidimensionality was achieved in six-country data after cross-cultural adjustment for Italy in the social subscale. The SScQoL was then calibrated into an interval level scale.

Conclusion

The individual SScQoL items have translated well into five languages and overall, the scale maintained its construct validity, working well as a five-subscale questionnaire. Measures of quality of life in SSc can be directly compared across five countries (France, Poland Spain, Sweden and UK). Data from Italy are also comparable with the other five countries although require an adjustment.

Symptom endorsement in men versus women with a diagnosis of depression: A differential item functioning approach

BACKGROUND

There is some evidence that, in contrast to depressed women, depressed men tend to report alternative symptoms that are not listed as standard diagnostic criteria. This may possibly lead to an under- or misdiagnosis of depression in men.

AIMS

This study aims to clarify whether depressed men and women report different symptoms.

METHODS

This study used data from the 2007 Australian National Survey of Mental Health and Wellbeing that was collected using the World Health Organization's Composite International Diagnostic Interview. Participants with a diagnosis of a depressive disorder with 12-month symptoms (n = 663) were identified and included in this study. Differential item functioning (DIF) was used to test whether depressed men and women endorse different features associated with their condition.

RESULTS

Gender-related DIF was present for three symptoms associated with depression. Depressed women were more likely to report 'appetite/weight disturbance', whereas depressed men were more likely to report 'alcohol misuse' and 'substance misuse'.

CONCLUSION

While the results may reflect a greater risk of co-occurring alcohol and substance misuse in men, inclusion of these features in assessments may improve the detection of depression in men, especially if standard depressive symptoms are under-reported.

Factorial Structure of the Manic-Depressiveness Scale in American College Students

In a sample of 503 American college students, the correlational matrix (18 × 18) of the Thalbourne, et al. Manic-Depressiveness Scale (1994) was subjected to exploratory factor analysis with a varimax rotation, which showed 13 items had the correct assignment to one of the scales (six for depression and seven for mania).

Drive to Marry and Social Prescription in Chinese Online Daters

Individuals’ eagerness or desire to get married was investigated in a sample of online daters in China (n = 3,389) using a 6-item version of the Drive to Marry (DTM) Scale, which was modified with new questions about normative pressures to marry given the cultural emphasis on social prescription in Eastern versus Western culture. The questionnaire items conformed to a unidimensional Rasch scale with interval-level measurement, although two themes seemed inherent to DTM – positive feelings of excitement or anticipation and negative feelings of urgency or desperation. Consistent with previous theory and research, women exhibited stronger DTM than men and normative pressures resulted in greater perceived DTM. Finally, significant response biases by sex were found, indicating that men and women differ in their qualitative experience of DTM. The results offer a preliminary cross-cultural validation and perspective on DTM and expand the present conceptualization and measurement of the construct to guide future research and theory-building.

Evaluating the Mathematics Interest Inventory Using Item Response Theory

As the United States has begun to lag behind other developed countries in performance on mathematics and science, researchers have sought to explain this with theories of teaching, knowledge, and motivation. We expand this examination by further analyzing a measure of interest that has been linked to student performance in mathematics and intention to enroll in advanced mathematics classes. Using Item Response Theory (IRT), the purpose of this study was to evaluate the psychometric properties of the Mathematics Interest Inventory (MII). Six hundred sixty-six students (47.6% Hispanic, 48.2% Male) formed the sample. Results revealed minimal differential item functioning (DIF) between genders. Substantial DIF emerged between Hispanic and White students. Implications of the differential functioning and the measurement of interest in mathematics are discussed.

Performance of the 6-item Kessler scale for measuring serious mental illness in Hong Kong

BACKGROUND

The 6-item Kessler scale (K6) promises to be a valuable epidemiological tool for assessing serious mental illness (SMI) in communities with limited resources for psychiatric research and treatment. Its performance in Chinese community has not been studied with reference to clinically assessed SMI.

METHOD

From a representative telephone-based population survey (n = 3014) that administered the K6, 153 participants were readministered the K6 and, on the same day, interviewed face-to-face by clinicians using the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorder, Fourth Edition, Axis I Disorder. Predictive indicators such as McNemar χ(2), area under receiver operating characteristic curve and stratum-specific likelihood ratios were used to investigate the concordance between the K6 and clinical status of SMI, individual-level predicted probabilities of having SMI, and the weighted prevalence of SMI in the community.

RESULT

The K6 exhibited high internal consistency and test-retest reliability. Factor analysis revealed 2 correlating components composed of depression and anxiety. Matching of K6 caseness and SMI status showed that at the cutoff of 12/13, the area under receiver operating characteristic curve was moderate (0.69). The K6 had high specificity and was a stronger screen-out than screen-in tool for SMI. The weighted prevalence estimate of SMI in Hong Kong was 6.5%. A person scoring 13 or above on the K6 has a probability of at least 22.2% of having SMI.

CONCLUSION

The Chinese K6 is reliable and generates the likelihood of SMI with substantial concordance with face-to-face clinical interviews in Hong Kong. It is a valuable tool for screening SMI, behavioral risk factor surveillance, and community epidemiological surveys.

Evaluation of the "Consultation and Relational Empathy" (CARE) measure by means of Rasch-analysis at the example of cancer patients

OBJECTIVE

The aims of this study are: (1) analysis of unidimensionality of the German version of the "Consultation and Relational Empathy" (CARE) measure and (2) identification of moderating variables affecting the scale structure.

METHODS

The CARE-measure was evaluated by means of Rasch-analysis in a sample of N=326 cancer patients. Association of diagnosis and treatment as well as patients' characteristics was analyzed by person-fit measures and Differential Item Functioning.

RESULTS

Nine of the original 10 CARE items fit to the Rasch-model. For breast and prostate cancer, as well as for patients taking complementary and alternative medicine treatment, item biases affect the scale structure. Furthermore, older patients and patients with higher social support exhibit substantial deviations from model predictions.

CONCLUSIONS

Only the nine-item version of the CARE-measure allows for the unidimensional assessment of physician empathy. Especially for specific diagnosis and treatment groups, the CARE-items indicate variations in the underlying latent construct of physician empathy.

PRACTICE IMPLICATIONS

The CARER-measure provides a theory-based and psychometrically sound basis for the assessment of PE. It can be used to enhance the fairness of the assessment and to further identify valuable information about the influence of patient characteristics on the structure of the construct PE.

Screening for depression: Rasch analysis of the dimensional structure of the PHQ-9 and the HADS-D

BACKGROUND

Both the depression modules of the Hospital Anxiety and Depression Scale (HADS-D) and the Patient Health Questionnaire (PHQ-9) are widely used for the screening of depression. We analyzed the dimensionality and the item fit of both scales individually and across the scales. Moreover, we sought to identify items which evidenced item response bias associated with age and gender.

METHODS

The depression subscales HADS-D and the PHQ-9 were administered to 1271 patients (mean age 67.2; 22.5% women) undergoing coronary artery bypass graft surgery (CABG). Rasch analyses were performed to assess the overall fit of the model, individual item fit and differential item functioning (DIF).

RESULTS

Rasch analysis revealed that the HADS-D and the PHQ-9 feature a common core construct containing six items of the HADS-D and three items of the PHQ-9. Two of these items are identical with the 2-item short form of the PHQ-9. In addition, fatigability was the only somatic item that fitted the model. No substantial DIF was observed.

LIMITATIONS

The generalizability of these results might be restricted to patients awaiting CABG.

CONCLUSIONS

The short form of the PHQ-9 seems to be an economic and valid instrument for the screening of depression, which indicates the same latent construct that is captured by six items of the HADS-D. Further studies are needed to evaluate whether the addition of fatigability might enhance the validity of the PHQ-2 in this patient population.

Using the cumulative common log-odds ratio to identify differential item functioning of rating scale items in the exercise and sport sciences

One aspect of construct validity is the extent to which the measurement properties of a rating scale are invariant across the groups being compared. An increasingly used method for assessing between-group differences in the measurement properties of items of a scale is the framework of differential item functioning (DIF). In this paper we introduce the concept of DIF as a validation tool for scales used in physical education and exercise science, and illustrate the use of the cumulative common log-odds ratio for assessing the presence of DIF with data from the Exercise Imagery Inventory (EII; Giacobbi, Hausenblas, & Penfield, 2005).

An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS)

OBJECTIVES

To demonstrate the use of Rasch analysis by assessing the appropriateness of utilizing the Hospital Anxiety and Depression Scale (HADS) total score (HADS-14) as a measure of psychological distress.

DESIGN

Cross-sectional, using Rasch analysis.

METHODS

The HADS was administered to 296 patients attending an out-patient musculoskeletal rehabilitation program. Rasch analysis was conducted using RUMM2020 software to assess the overall fit of the model, the response scale used, individual item fit, differential item functioning (DIF) and person separation.

RESULTS

Rasch analysis supported the viability of the HADS-14 as a measure of psychological distress. It showed good person separation, little disordering of the thresholds and no evidence of DIE One anxiety item (item 11) showed some misfit to the model. The residuals patterned into the two subscales (anxiety and depression), but the person estimate derived from these two subscales was not statistically different to that derived from all items taken together, supporting the assumption of unidimensionality. A cut-point of 12 on the HADS-14 identified all cases that were classified as both anxious and depressed on the original individual HADS subscales.

CONCLUSIONS

The results of Rasch analysis support the use of the HADS-14 as a global measure of psychological distress. The study demonstrates the usefulness of Rasch analysis in assessing the psychometric properties of a scale and suggests that further use of this technique to assess the HADS-14 in other clinical groups is warranted.

Evaluation of the Edinburgh Post Natal Depression Scale using Rasch analysis

BACKGROUND

The Edinburgh Postnatal Depression Scale (EPDS) is a 10 item self-rating post-natal depression scale which has seen widespread use in epidemiological and clinical studies. Concern has been raised over the validity of the EPDS as a single summed scale, with suggestions that it measures two separate aspects, one of depressive feelings, the other of anxiety.

METHODS

As part of a larger cross-sectional study conducted in Melbourne, Australia, a community sample (324 women, ranging in age from 18 to 44 years: mean = 32 yrs, SD = 4.6), was obtained by inviting primiparous women to participate voluntarily in this study. Data from the EPDS were fitted to the Rasch measurement model and tested for appropriate category ordering, for item bias through Differential Item Functioning (DIF) analysis, and for unidimensionality through tests of the assumption of local independence.

RESULTS

Rasch analysis of the data from the ten item scale initially demonstrated a lack of fit to the model with a significant Item-Trait Interaction total chi-square (chi Square = 82.8, df = 40; p < .001). Removal of two items (items 7 and 8) resulted in a non-significant Item-Trait Interaction total chi-square with a residual mean value for items of -0.467 with a standard deviation of 0.850, showing fit to the model. No DIF existed in the final 8-item scale (EPDS-8) and all items showed fit to model expectations. Principal Components Analysis of the residuals supported the local independence assumption, and unidimensionality of the revised EPDS-8 scale. Revised cut points were identified for EPDS-8 to maintain the case identification of the original scale.

CONCLUSION

The results of this study suggest that EPDS, in its original 10 item form, is not a viable scale for the unidimensional measurement of depression. Rasch analysis suggests that a revised eight item version (EPDS-8) would provide a more psychometrically robust scale. The revised cut points of 7/8 and 9/10 for the EPDS-8 show high levels of agreement with the original case identification for the EPDS-10.

Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the framework of the Rasch model: the PRO-ESOR project

INTRODUCTION

In Europe it is common for outcome measures to be translated for use in other languages. This adaptation may be complicated by culturally specific approaches to certain tasks; for example, bathing. In this context the issue of cross-cultural validity becomes paramount.

OBJECTIVE

To facilitate the pooling of data in international studies, a project set out to evaluate the cross-cultural validity of impairment and activity limitation measures used in rehabilitation from the perspective of the Rasch measurement model.

METHODS

Cross-cultural validity is assessed through an analysis of Differential Item Functioning (DIF) within the context of additive conjoint measurement expressed through the Rasch model. Data from patients undergoing rehabilitation for stroke was provided from 62 centers across Europe. Two commonly used outcome measures, the Mini-Mental State Examination (MMSE) and the Functional Independence Measure (FIM) motor scale are used to illustrate the approach.

RESULTS

Pooled data from 3 countries for the MMSE were shown to fit the Rasch model with only 1 item displaying DIF by country. In contrast, many items from the FIM expressed DIF and misfit to the model. Consequently they were allowed to be unique across countries, so resolving the lack of fit to the model.

CONCLUSIONS

Where data are to be pooled for international studies, analysis of DIF by culture is essential. Where DIF is observed, adjustments can be made to allow for cultural differences in outcome measurement.

Rasch scaling paranormal belief and experience: structure and semantics of Thalbourne's Australian Sheep-Goat Scale

Research on the relation between demographic variables and paranormal belief remains controversial given the possible semantic distortions introduced by item and test level biases. We illustrate how Rasch scaling can be used to detect such biases and to quantify their effects, using the Australian Sheep-Goal Scale as a substantive example. Based on data from 1.822 respondents, this test was Rasch scalable, reliable, and unbiased at the test level. Consistent with other research in which unbiased measures of paranormal belief were used, extremely weak age and sex effects were found (partial eta2 = .005 and .012, respectively).

Age and sex differences in paranormal beliefs: a response to Vitulli, Tipton, and Rowe (1999)

Vitulli, Tipton, and Rowe (1999) report evidence of age and sex differences in the strength of paranormal beliefs. An alternative interpretation of their data is offered in terms of differential item functioning. It is suggested that respondents' interpretation of paranormal belief test items may vary with age and sex, and that such differences in the strength with which such beliefs are endorsed has not been conclusively established by Vitulli, et al.