An Analysis of (Dis)Ordered Categories, Thresholds, and Crossings in Difference and Divide-by-Total IRT Models for Ordered Responses

Are the Steps on Likert Scales Equidistant? Responses on Visual Analog Scales Allow Estimating Their Distances

A recurring question regarding Likert items is whether the discrete steps that this response format allows represent constant increments along the underlying continuum. This question appears unsolvable because Likert responses carry no direct information to this effect. Yet, any item administered in Likert format can identically be administered with a continuous response format such as a visual analog scale (VAS) in which respondents mark a position along a continuous line. Then, the operating characteristics of the item would manifest under both VAS and Likert formats, although perhaps differently as captured by the continuous response model (CRM) and the graded response model (GRM) in item response theory. This article shows that CRM and GRM item parameters hold a formal relation that is mediated by the form in which the continuous dimension is partitioned into intervals to render the discrete Likert responses. Then, CRM and GRM characterizations of the items in a test administered with VAS and Likert formats allow estimating the boundaries of the partition that renders Likert responses for each item and, thus, the distance between consecutive steps. The validity of this approach is first documented via simulation studies. Subsequently, the same approach is used on public data from three personality scales with 12, eight, and six items, respectively. The results indicate the expected correspondence between VAS and Likert responses and reveal unequal distances between successive pairs of Likert steps that also vary greatly across items. Implications for the scoring of Likert items are discussed.

Item response theory assumptions were adequately met by the Oxford hip and knee scores

OBJECTIVES

To develop item response theory (IRT) models for the Oxford hip and knee scores which convert patient responses into continuous scores with quantifiable precision and provide these as web applications for efficient score conversion.

STUDY DESIGN AND SETTING

Data from the National Health Service patient-reported outcome measures program were used to test the assumptions of IRT (unidimensionality, monotonicity, local independence, and measurement invariance) before fitting models to preoperative response patterns obtained from patients undergoing primary elective hip or knee arthroplasty. The hip and knee datasets contained 321,147 and 355,249 patients, respectively.

RESULTS

Scree plots, Kaiser criterion analyses, and confirmatory factor analyses confirmed unidimensionality and Mokken analysis confirmed monotonicity of both scales. In each scale, all item pairs shared a residual correlation of ≤ 0.20. At the test level, both scales showed measurement invariance by age and gender. Both scales provide precise measurement in preoperative settings but demonstrate poorer precision and ceiling effects in postoperative settings.

CONCLUSION

We provide IRT parameters and web applications that can convert Oxford Hip Score or Oxford Knee Score response sets into continuous measurements and quantify individual measurement error. These can be used in sensitivity analyses or to administer truncated and individualized computerized adaptive tests.

Psychometric Properties of and Measurement Invariance in the Questionnaire of Stereotypes Toward Older Adulthood in Health Care College Students and Health Professionals of Colombia: Psychometric Study

BACKGROUND

In health professionals, negative stereotypes toward older adulthood have been associated with the difficulty in recognizing pathological processes and the refusal to care for older patients because of assuming that communication with them will be uncomfortable and frustrating. For these reasons, research on stereotypes in these groups has acquired growing importance. The usual strategy to identify and evaluate agist stereotypes is to use scales and questionnaires. Although multiple scales are currently used, in Latin America, the Questionnaire for the Evaluation of Negative Stereotypes Toward Older Adulthood (Cuestionario de Estereotipos Negativos sobre la Vejez [CENVE]), developed in Spain, is widely used but without evidence of construct validity in our context. In addition, although in the original version, a factorial structure of 3 factors was found, in later studies, a unifactorial structure was obtained.

OBJECTIVE

The objective is to study the construct validity of the CENVE in a sample of Colombian health personnel to clarify its factorial structure and concurrent validity. Likewise, the measurement invariance according to gender and age was studied.

METHODS

A nonprobabilistic sample of 877 Colombian health professionals and intern health students was obtained. The data were collected online using the LimeSurvey tool. To study the factor structure of the CENVE, 2 confirmatory factor analysis (CFA) models were carried out, one to test a single factor and the other to test the 3-related-factor structure. The factor measurement reliability was evaluated with the composite reliability index (CRI) and the average variance extracted (AVE). The measurement invariance was studied according to gender (men and women) and age (emerging adults, 18-29 years old, and adults, 30 years old or older). Using a structural equation model, the relationship between age and the latent CENVE total score was studied to obtain evidence of concurrent validity, since studies indicate that the younger the age, the greater the number of stereotypes.

RESULTS

The 1-factor structure was confirmed. The reliability results indicated that both indices show adequate values. Likewise, the existence of a strong invariance in measurement by gender and age group was verified. After contrasting the means of the groups, the results showed that men show more negative stereotypes toward old age than women. Likewise, emerging adults also showed more stereotypes than adults. We also verified that age is inversely related to the latent score of the questionnaire, such that the younger the age, the greater the stereotype. These results are in agreement with those obtained by other authors.

CONCLUSIONS

The CENVE shows good construct and concurrent validity, as well as good reliability, and it can be used to assess stereotypes toward older adulthood in Colombian health professionals and health sciences college students. This will allow us to better understand the effect of stereotypes on agism.

Assessing the quality of life of people with chronic wounds by using the cross-culturally valid and revised Wound-QoL questionnaire

The Wound-QoL is an often used reliable and valid measure, originally developed in Germany. It has been sequentially translated and validated for other languages/countries, for the measurement of health-related quality of life (HRQoL) in patients with chronic wounds. However, a study from the United States postulated its benefits from further adaptations. Furthermore, some patients struggled to provide an answer for some of the items. We aimed to test the cross-cultural structure and psychometric performance of the questionnaire to suggest necessary revisions. This cross-sectional analysis of existing data sets included 1185 patients from Germany, the US, the Netherlands, Spain, Sweden, and Israel. Patients in the U.S. Wound Registry completed the Wound-QoL during routine care. Different studies comprised the data collection in the other countries. Almost half of the patients were women (48.4%). Furthermore, 42.6% were diagnosed with leg ulcers. Their average age was 66 years. We used a confirmatory factor analysis and an unconstrained graded response model. We revised and shortened the Wound-QoL from 17 to 14 items. In addition, we supported the cross-cultural metric invariance of the revised Wound-QoL questionnaire. The new version with 14 items and three dimensions revealed good psychometric properties with Cronbach's alpha (α) of 0.913 for the total score, and 0.709-0.907 for different dimensions. Furthermore, we provided strict invariance for different clinical variables. In conclusion, the revised Wound-QoL is a reliable and cross-cultural instrument to measure the HRQoL on patients with chronic wounds. Future studies should analyse the revised Wound-QoL for convergent validity with generic HRQoL questionnaires as well as for determining its sensitivity to clinical change.

Not Very Powerful

Abstract. Several guidelines on how to construct questionnaire items exist, even though the literature lacks empirical evidence for their effectiveness. To investigate whether the addition of negations and vague quantifiers worsens the psychometric properties of an established questionnaire, 872 participants completed one version of the Positive and Negative Affect Schedule (PANAS) – the German original, a negated version, a version with vague quantifiers or a version with both negations and vague quantifiers. Reliability estimates, item-total correlations, Confirmatory Factor Analysis (CFA) model fit, and fit to the Partial Credit Model (PCM) were compared among the four conditions. No PANAS version was clearly superior as no systematic pattern in the psychometric properties was found. Our findings question the general applicability of the guidelines of item construction as well as the effectiveness of widely used statistical analyses assessing the quality of scales. The results should encourage researchers to put a stronger focus on careful item construction as relying on psychometric properties might not be sufficient to develop valid questionnaires.

Validation and calibration of the patient-reported outcomes measurement information system: Pediatric PROMIS® Emotional Distress domain item banks, Portuguese version (Brazil/Portugal)

OBJECTIVE

This paper goal was to validate the Portuguese version (Brazil/Portugal) of the Anger, Anxiety, and Depressive Symptoms item banks of the Pediatric PROMIS^® Emotional Distress domain (version 1.0) for the Brazilian and Portuguese pediatric population.

METHOD

The total of 1216 participants answered a self-applied version of the Portuguese Anger, Anxiety, and Depressive Symptoms item banks. Reliability was assessed through internal consistency, test-retest reliability, and total information curve (TIC). Confirmatory Factor Analysis (CFA) with a bifactor model was used to confirm construct validity and IRT assumptions. Item calibration was performed according to Graded Response Model (GRM). Differential Item Functioning (DIF) was analyzed according the participants' age, gender, health condition (healthy versus chronic disease), and language.

RESULTS

Internal consistency reliability (Cronbach's alpha coefficient = 0.84) and test-retest reliability (intraclass correlation = 0.93) were accurate. Unidimensionality, Local Independency, and construct validity were verified by CFA (CFI = 0.93; TLI = 0.93; RMSEA = 0.05; χ² = 3052.4 with DIF = 557 and P value = 0.595). GRM was adjusted, and Emotional Distress had a satisfactory coverage. DIF was not significant.

CONCLUSION

The results obtained indicate the adequacy of the psychometric properties of the Portuguese version (Brazil/Portugal) of the Anger, Anxiety, and Depressive Symptoms item banks of the Pediatric PROMIS^® Emotional Distress domain.

State of the psychometric methods: comments on the ISOQOL SIG psychometric papers

BACKGROUND

Psychometric analyses of patient reported outcomes typically use either classical test theory (CTT), item response theory (IRT), or Rasch measurement theory (RTM). The three papers from the ISOQOL Psychometrics SIG examined the same data set using the tree different approaches. By comparing the results from these papers, the current paper aims to examine the extent to which conclusions about the validity and reliability of a PRO tool depends on the selected psychometric approach.

MAIN TEXT

Regarding the basic statistical model, IRT and RTM are relatively similar but differ notably from CTT. However, modern applications of CTT diminish these differences. In analyses of item discrimination, CTT and IRT gave very similar results, while RTM requires equal discrimination and therefore suggested exclusion of items deviating too much from this requirement. Thus, fewer items fitted the Rasch model. In analyses of item thresholds (difficulty), IRT and RMT provided fairly similar results. Item thresholds are typically not evaluated in CTT. Analyses of local dependence showed only moderate agreement between methods, partly due to different thresholds for important local dependence. Analyses of differential item function (DIF) showed good agreement between IRT and RMT. Agreement might be further improved by adjusting the thresholds for important DIF. Analyses of measurement precision across the score range showed high agreement between IRT and RMT methods. CTT assumes constant measurement precision throughout the score range and thus gave different results. Category orderings were examined in RMT analyses by checking for reversed thresholds. However, this approach is controversial within the RMT society. The same issue can be examined by the nominal categories IRT model.

CONCLUSIONS

While there are well-known differences between CTT, IRT and RMT, the comparison between three actual analyses revealed a great deal of agreement between the results from the methods. If the undogmatic attitude of the three current papers is maintained, the field will be well served.

Order-Constrained Estimation of Nominal Response Model Parameters to Assess the Empirical Order of Categories

Bock's nominal response model (NRM) is sometimes used to identify the empirical order of response categories in polytomous items but its application tags many items as having disordered categories. Disorderly estimated categories may not reflect a true characteristic of the items but, rather, a numerically best-fitting solution possibly equivalent to other solutions with orderly estimated categories. To investigate this possibility, an order-constrained variant of the NRM was developed that enforces the preassumed order of categories on parameter estimates, for a comparison of its outcomes with those of the original unconstrained NRM. For items with ordered categories, order-constrained and unconstrained solutions should account for the data equally well even if the latter solution estimated disordered categories for some items; for items with truly disordered categories, the unconstrained solution should outperform the order-constrained solution. Criteria for this comparative analysis are defined and their utility is tested in several simulation studies with items of diverse characteristics, including ordered and disordered categories. The results demonstrate that a comparison of order-constrained and unconstrained calibrations on such criteria provides the evidence needed to determine whether category disorder estimated on some items by the original unconstrained form of the NRM is authentic or spurious. Applications of this method to assess category order in existing data sets are presented and practical implications are discussed.

Using item response theory to address vulnerabilities in FFQ

The limitations for self-reporting of dietary patterns are widely recognised as a major vulnerability of FFQ and the dietary screeners/scales derived from FFQ. Such instruments can yield inconsistent results to produce questionable interpretations. The present article discusses the value of psychometric approaches and standards in addressing these drawbacks for instruments used to estimate dietary habits and nutrient intake. We argue that a FFQ or screener that treats diet as a 'latent construct' can be optimised for both internal consistency and the value of the research results. Latent constructs, a foundation for item response theory (IRT)-based scales (e.g. Patient Reported Outcomes Measurement Information System) are typically introduced in the design stage of an instrument to elicit critical factors that cannot be observed or measured directly. We propose an iterative approach that uses such modelling to refine FFQ and similar instruments. To that end, we illustrate the benefits of psychometric modelling by using items and data from a sample of 12 370 Soldiers who completed the 2012 US Army Global Assessment Tool (GAT). We used factor analysis to build the scale incorporating five out of eleven survey items. An IRT-driven assessment of response category properties indicates likely problems in the ordering or wording of several response categories. Group comparisons, examined with differential item functioning (DIF), provided evidence of scale validity across each Army sub-population (sex, service component and officer status). Such an approach holds promise for future FFQ.

[Experience with the management of subtalar dislocation]

In 10 and 3 years resp., material of two surgical departments - with great material - 10 cases of luxation - without fracture - occurred in the subtaler joint have been treated. All 10 patients suffered covered, subtaler dislocations. After conservative treatment all 10 patients recovered and are back at work.