Novel indices reveal that pollinator exposure to pesticides varies across biological compartments and crop surroundings

Declines in insect pollinators have been linked to a range of causative factors such as disease, loss of habitats, the quality and availability of food, and exposure to pesticides. Here, we analysed an extensive dataset generated from pesticide screening of foraging insects, pollen-nectar stores/beebread, pollen and ingested nectar across three species of bees collected at 128 European sites set in two types of crop. In this paper, we aimed to (i) derive a new index to summarise key aspects of complex pesticide exposure data and (ii) understand the links between pesticide exposures depicted by the different matrices, bee species and apple orchards versus oilseed rape crops. We found that summary indices were highly correlated with the number of pesticides detected in the related matrix but not with which pesticides were present. Matrices collected from apple orchards generally contained a higher number of pesticides (7.6 pesticides per site) than matrices from sites collected from oilseed rape crops (3.5 pesticides), with fungicides being highly represented in apple crops. A greater number of pesticides were found in pollen-nectar stores/beebread and pollen matrices compared with nectar and bee body matrices. Our results show that for a complete assessment of pollinator pesticide exposure, it is necessary to consider several different exposure routes and multiple species of bees across different agricultural systems.

Measuring patient activation: the utility of the Patient Activation Measure administered in an interview setting

BACKGROUND

Patient activation is an emerging field in healthcare research concerning knowledge, skills, and confidence of patients in managing their health. This is particularly important for patients with chronic diseases, who often require more complex care management and self-care skills. However, due to temporary or longer-lasting visual impairments, certain patient groups cannot answer a questionnaire independently. The main objective is to investigate the psychometric properties of the German Patient Activation Measure® (PAM) survey in an everyday clinical setting where it has to be read aloud.

METHODS

Outpatients with macular edema participated in this questionnaire-based cross-sectional study. The study assessed patient activation by the PAM® survey, self-rated health, self-efficacy, quality of life, and general mood. Interviewers read questionnaires aloud to patients. Psychometric properties of the PAM® survey were investigated by item response theory (IRT), Cronbach's α and trait-trait correlations.

RESULTS

The analysis included N = 554 patients. Median age was 69 (IQR 62.0-76.0) years and mean overall activation score 74.1 (SD 13.7). All items showed ceiling effects. Empirical reliability from the IRT model and Cronbach's α were 0.75. The PAM® survey showed a Spearman correlation of 0.54 with self-efficacy, 0.51 with quality of life and 0.34 with general mood.

CONCLUSION

The read-aloud PAM® survey has been shown to provide to adequate measurement precision and convergent validity to be used as a screening tool in an everyday clinical setting. Objective assessment in an interview setting with the PAM® survey is possible. PAM® items are good in distinguishing lower to middle activated patients, but not patients with high activation. Further, issues with structural validity need more investigation.

Healthy aging trajectories and their predictors among Chinese older adults: Evidence from a 7-year nationwide prospective cohort study

OBJECTIVES

This study aimed to identify healthy aging trajectories of Chinese older adults, and explore the factors contributing to these trajectories.

METHODS

We used data from four waves (2011-2018) of China Health and Retirement Longitudinal Study. We developed a healthy aging metric based on the healthy aging framework of World Health Organization (WHO) and Bayesian multilevel item response theory (IRT) method. The healthy aging trajectories were identified using the latent class growth analysis. The predictors of trajectories were explored using multinomial logistic regression analysis. Additionally, we developed two alternative metrics for healthy aging based on the Chinese Healthy Ageing Index (CHAI) and Rowe and Kahn's model of successful aging, respectively. We compared these metrics to the one developed based on the WHO's healthy aging framework.

RESULTS

We identified three distinct healthy aging trajectories with varying scores and decline rates. Individuals who were female, had lower educational levels, resided in rural areas, experienced depression, had more chronic diseases, participated in fewer social activities, had fewer childhood friends, experienced more adverse childhood events, and had worse family financial status in childhood were more likely to experience a worse healthy aging trajectory compared to their counterparts. Supplementary analysis showed that healthy aging metric based on WHO definition and IRT method had the strongest association with health outcomes compared to the metrics based on CHAI, as well as Rowe and Kahn model.

CONCLUSIONS

Our findings provide a foundation for the development of tailored interventions to enhance and sustain healthy aging among Chinese older adults.

Unveiling disparities: examining differential item functioning's impact on racial health equity among white and black populations

OBJECTIVES

This paper aims to examine the psychometric properties of social capital indicators, comparing Black and White respondents to identify the extent of measurement invariance in social capital by race.

STUDY DESIGN

We used data from the longitudinal study Midlife in the United States (MIDUS), waves 1 through 3 (1995-2016).

METHODS

Data were from 6513 respondents (5604 White and 909 Black respondents). Social capital indicators were social cohesion, contributions to community, and community involvement. We used Structural Equation Modeling and Item Response Theory methods to test for measurement invariance in social capital by race.

RESULTS

We observed violations of longitudinal and multi-group measurement invariance (MI) at configural and metric levels on two scales. Factor structures and indicator loadings were inconsistent over time. In IRT analysis, 'Many people come for advice' exhibited Differential Item Functioning (DIF), indicating a consistent advantage for White respondents on the contributions to community scale. Despite similar social capital levels (P(χ2,2) = 0.00), DIF was found in all contributions to community items and some community involvement items when examining race and education interaction.

CONCLUSIONS

Invariance issues in social capital items suggest potential biases in comparing Black and White respondents. Recognizing these biases is essential. Future social capital research should assess existing data assumptions and involve stakeholders from diverse communities in creating new items.

Measuring the variability of personality traits with interval responses: Psychometric properties of the dual-range slider response format : Measuring variability with interval responses

Measuring the variability in persons' behaviors and experiences using ecological momentary assessment is time-consuming and costly. We investigate whether interval responses provided through a dual-range slider (DRS) response format can be used as a simple and efficient alternative: Respondents indicate variability in their behavior in a retrospective rating by choosing a lower and an upper bound on a continuous, bounded scale. We investigate the psychometric properties of this response format as a prerequisite for further validation. First, we assess the test-retest reliability of factor-score estimates for the width of DRS intervals. Second, we test whether factor-score estimates of the visual analog scale (VAS) and the location of DRS intervals show convergent validity. Third, we investigate whether factor-score estimates for the DRS are uncorrelated between different personality scales. We present a longitudinal multitrait-multimethod study using two personality scales (Extraversion, Conscientiousness) and two response formats (VAS, DRS) at two measurement occasions (6-8 weeks apart) for which we estimate factor-score correlations in a joint item response theory model. The test-retest reliability of the width of DRS intervals was high (ρ ^ ≥ . 73 ). Also, convergent validity between location scores of VAS and DRS was high (ρ ^ ≥ . 88 ). Conversely, discriminant validity of the width of DRS intervals between Extraversion and Conscientiousness was poor (ρ ^ ≥ . 94 ). In conclusion, the DRS seems to be a reliable response format that could be used to measure the central tendency of a trait equivalently to the VAS. However, it might not be well suited for measuring intra-individual variability in personality traits.

A neural network paradigm for modeling psychometric data and estimating IRT model parameters: Cross estimation network

This paper presents a novel approach known as the cross estimation network (CEN) for fitting the datasets obtained from psychological or educational tests and estimating the parameters of item response theory (IRT) models. The CEN is comprised of two subnetworks: the person network (PN) and the item network (IN). The PN processes the response pattern of individual respondent and generates an estimate of the underlying ability, while the IN takes in the response pattern of individual item and outputs the estimates of the item parameters. Four simulation studies and an empirical study were comprehensively and rigorously conducted to investigate the performance of CEN on parameter estimation of the two-parameter logistic model under various testing scenarios. Results showed that CEN effectively fit the training data and produced accurate estimates of both person and item parameters. The trained PN and IN adhered to AI principles and acted as intelligent agents, delivering commendable evaluations for even unseen patterns of new respondents and items.

Development and validation of the patient-reported outcome for older people living with HIV/AIDS in China (PROHIV-OLD)

BACKGROUND

The involvement of quality of life as the UNAIDS fourth 90 target to monitor the global HIV response highlighted the development of patient-reported outcome (PRO) measures to help address the holistic needs of people living with HIV/AIDS (PLWHA) beyond viral suppression. This study developed and tested preliminary measurement properties of a new patient-reported outcome (PROHIV-OLD) measure designed specifically to capture influences of HIV on patients aged 50 and older in China.

METHODS

Ninety-three older people living with HIV/AIDS (PLWHA) were interviewed to solicit items and two rounds of patient cognitive interviews were conducted to modify the content and wording of the initial items. A validation study was then conducted to refine the initial instrument and evaluate measurement properties. Patients were recruited between February 2021 and November 2021, and followed six months later after the first investigation. Classical test theory (CTT) and item response theory (IRT) were used to select items using the baseline data. The follow-up data were used to evaluate the measurement properties of the final instrument.

RESULTS

A total of 600 patients were recruited at the baseline. Of the 485 patients who completed the follow-up investigation, 483 were included in the validation sample. The final scale of PROHIV-OLD contained 25 items describing five dimensions (physical symptoms, mental status, illness perception, family relationship, and treatment). All the PROHIV-OLD dimensions had satisfactory reliability with Cronbach's alpha coefficient, McDonald's ω, and composite reliability of each dimension being all higher than 0.85. Most dimensions met the test-retest reliability standard except for the physical symptoms dimension (ICC = 0.64). Confirmatory factor analysis supported the structural validity of the final scale, and the model fit index satisfied the criterion. The correlations between dimensions of PROHIV-OLD and MOS-HIV met hypotheses in general. Significant differences on scores of the PROHIV-OLD were found between demographic and clinical subgroups, supporting known-groups validity.

CONCLUSIONS

The PROHIV-OLD was found to have good feasibility, reliability and validity for evaluating health outcome of Chinese older PLWHA. Other measurement properties such as responsiveness and interpretability will be further examined.

Genetic associations with dementia-related proteinopathy: Application of item response theory

INTRODUCTION

Although dementia-related proteinopathy has a strong negative impact on public health, and is highly heritable, understanding of the related genetic architecture is incomplete.

METHODS

We applied multidimensional generalized partial credit modeling (GPCM) to test genetic associations with dementia-related proteinopathies. Data were analyzed to identify candidate single nucleotide variants for the following proteinopathies: Aβ, tau, α-synuclein, and TDP-43.

RESULTS

Final included data comprised 966 participants with neuropathologic and WGS data. Three continuous latent outcomes were constructed, corresponding to TDP-43-, Aβ/Tau-, and α-synuclein-related neuropathology endophenotype scores. This approach helped validate known genotype/phenotype associations: for example, TMEM106B and GRN were risk alleles for TDP-43 pathology; and GBA for α-synuclein/Lewy bodies. Novel suggestive proteinopathy-linked alleles were also discovered, including several (SDHAF1, TMEM68, and ARHGEF28) with colocalization analyses and/or high degrees of biologic credibility.

DISCUSSION

A novel methodology using GPCM enabled insights into gene candidates for driving misfolded proteinopathies.

HIGHLIGHTS

Latent factor scores for proteinopathies were estimated using a generalized partial credit model. The three latent continuous scores corresponded well with proteinopathy severity. Novel genes associated with proteinopathies were identified. Several genes had high degrees of biologic credibility for dementia risk factors.

Development of a rapid tool for screening financial toxicity based on the comprehensive score for financial toxicity

OBJECTIVE

This study aimed to use item response theory (IRT) to develop and validate a short version of the Chinese COmprehensive Score for financial Toxicity (COST-S-C) in patients with cancer.

METHODS

A total of 500 patients with cancer participated in the survey. Confirmatory factor analysis (CFA) was used to assess the factor structure of the COST-S-C. A two-parameter IRT model was used to examine the item fit of the COST-S-C. Item discrimination, item characteristic curves (ICCs), item information curves (IICs), and item fit were estimated. Differential item functioning (DIF) was assessed for participants' gender and age. Reliability, convergent and discriminant validity was examined as well as estimating the optimal cut-off points of the COST-S-C.

RESULTS

The CFA results supported the bi-factor structure of the COST-S-C. The IRT analysis findings revealed that several items showed problems with the ICCs, providing little information in terms of IICs as well as poor discrimination. Item 5 showed a negligible DIF problem with age. A valid 3-item COST-S-C scale and its optimal cut-off point was developed.

CONCLUSIONS

The COST-S-C is a valid and quick screening tool used to distinguish patients with cancer who report significant financial toxicity in various medical settings.

Can you tell people's cognitive ability level from their response patterns in questionnaires?

Questionnaires are ever present in survey research. In this study, we examined whether an indirect indicator of general cognitive ability could be developed based on response patterns in questionnaires. We drew on two established phenomena characterizing connections between cognitive ability and people's performance on basic cognitive tasks, and examined whether they apply to questionnaires responses. (1) The worst performance rule (WPR) states that people's worst performance on multiple sequential tasks is more indicative of their cognitive ability than their average or best performance. (2) The task complexity hypothesis (TCH) suggests that relationships between cognitive ability and performance increase with task complexity. We conceptualized items of a questionnaire as a series of cognitively demanding tasks. A graded response model was used to estimate respondents' performance for each item based on the difference between the observed and model-predicted response ("response error" scores). Analyzing data from 102 items (21 questionnaires) collected from a large-scale nationally representative sample of people aged 50+ years, we found robust associations of cognitive ability with a person's largest but not with their smallest response error scores (supporting the WPR), and stronger associations of cognitive ability with response errors for more complex than for less complex questions (supporting the TCH). Results replicated across two independent samples and six assessment waves. A latent variable of response errors estimated for the most complex items correlated .50 with a latent cognitive ability factor, suggesting that response patterns can be utilized to extract a rough indicator of general cognitive ability in survey research.

When better is the enemy of good: two cautionary tales of conceptual validity versus parsimony in clinical psychometric research

This paper presents an empirical challenge to the assumption that an item-response theory analysis always yields a better measure of a clinical construct. We summarize results from two measurement development studies that showed that such an analysis lost important content reflecting the conceptual model ("conceptual validity"). The cost of parsimony may thus be too high. Conceptual models that form the foundation of QOL measurement reflect the patient's experience. This experience may include concepts and items that are psychometrically "redundant" but capture distinct features of the concept. Good measurement is likely a balance between relying on IRT's quantitative metrics and recognizing the importance of conceptual validity and clinical utility.

Development and validation of the missed intensive nursing care scale

BACKGROUND

Missed nursing care is a pervasive issue in hospitals, nursing homes, and communities, posing a significant threat to patient safety and the quality of nursing care. It has adverse effects on patient satisfaction and the motivation of nursing staff. Understanding the causes and nature of these care omissions in clinical settings is essential for implementing effective interventions. This study aims to develop and validate a tool for assessing missed nursing care in adult intensive care units.

METHODS

Semi-structured interviews, expert consultations conducted via the Delphi method and item analysis were used to develop the initial scale. Our analysis involved data collected from 400 nurses and employed correlation coefficient analysis, critical ratio assessment, Cronbach's α coefficient evaluation, discrete trend analysis, and factor analysis, which were grounded in both classical test theory and item response theory, allowing us to scrutinize and refine the items in the scale. To validate the scale, we conveniently sampled 550 nurses and assessed structural validity, internal reliability, split-half reliability, and test-retest reliability to ensure the scale's robustness and accuracy.

RESULTS

The Missed Intensive Nursing Care Scale (MINCS) comprises three distinct components. Part A serves to collect general information about the participants. In Part B, the missed care elements are categorized into five domains, following the framework of Maslow's hierarchy of needs theory: physiology, safety, belongingness, esteem, and cognition. Part C is dedicated to detailing the reasons behind missed care, which encompass labor resources, material resources, communication factors, and managerial factors. Remarkably, the Cronbach's α coefficient for the MINCS stands at an impressive 0.951, with S-CVI values of 0.988 and 0.977 in Part B and C, respectively, underscoring the scale's exceptional reliability and validity. This demonstrates the scale's effectiveness in measuring missed nursing care while upholding rigorous standards of quality.

CONCLUSIONS

The MINCS emerges as a robust and dependable instrument for quantifying instances of missed care within the ICU. Its efficacy makes it a valuable resource for informing the development of strategies aimed at averting and mitigating the adverse effects associated with missed nursing care.

Benefits of the Curious Behavior of Bayesian Hierarchical Item Response Theory Models-An in-Depth Investigation and Bias Correction

When using Bayesian hierarchical modeling, a popular approach for Item Response Theory (IRT) models, researchers typically face a tradeoff between the precision and accuracy of the item parameter estimates. Given the pooling principle and variance-dependent shrinkage, the expected behavior of Bayesian hierarchical IRT models is to deliver more precise but biased item parameter estimates, compared to those obtained in nonhierarchical models. Previous research, however, points out the possibility that, in the context of the two-parameter logistic IRT model, the aforementioned tradeoff has not to be made. With a comprehensive simulation study, we provide an in-depth investigation into this possibility. The results show a superior performance, in terms of bias, RMSE and precision, of the hierarchical specifications compared to the nonhierarchical counterpart. Under certain conditions, the bias in the item parameter estimates is independent of the bias in the variance components. Moreover, we provide a bias correction procedure for item discrimination parameter estimates. In sum, we show that IRT models create a unique situation where the Bayesian hierarchical approach indeed yields parameter estimates that are not only more precise, but also more accurate, compared to nonhierarchical approaches. We discuss this beneficial behavior from both theoretical and applied point of views.

Unsupervised item response theory models for assessing sample heterogeneity in patient-reported outcomes measures

PURPOSE

Unsupervised item-response theory (IRT) models such as polytomous IRT based on recursive partitioning (IRTrees) and mixture IRT (MixIRT) models can be used to assess differential item functioning (DIF) in patient-reported outcome measures (PROMs) when the covariates associated with DIF are unknown a priori. This study examines the consistency of results for IRTrees and MixIRT models.

METHODS

Data were from 4478 individuals in the Alberta Provincial Project on Outcome Assessment in Coronary Heart Disease registry who received cardiac angiography in Alberta, Canada, and completed the Hospital Anxiety and Depression Scale (HADS) depression subscale items. The partial credit model (PCM) based on recursive partitioning (PCTree) and mixture PCM (MixPCM) were used to identify covariates associated with differential response patterns to HADS depression subscale items. Model covariates included demographic and clinical characteristics.

RESULTS

The median (interquartile range) age was 64.5(15.7) years, and 3522(78.5%) patients were male. The PCTree identified 4 terminal nodes (subgroups) defined by smoking status, age, and body mass index. A 3-class PCM fits the data well. The MixPCM latent classes were defined by age, disease indication, smoking status, comorbid diabetes, congestive heart failure, and chronic obstructive pulmonary disease.

CONCLUSION

PCTree and MixPCM were not consistent in detecting covariates associated with differential interpretations of PROM items. Future research will use computer simulations to assess these models' Type I error and statistical power for identifying covariates associated with DIF.

Psychometric evidence of the Acceptance and Action Questionnaire-II (AAQ-II): an item response theory analysis in university students from Chile

BACKGROUND

Experiential avoidance (EA) is a psychological mechanism associated with several mental health disorders and is regarded as a relevant target by third-generation cognitive behavioral therapies. It has been mainly assessed through self-report questionnaires, and the AAQ-II is the most used tool. Its psychometric evidence has been mostly tested through the classical test theory (CTT) and very scarcely assessed through Item Response Theory (IRT).

METHODS

We used the Graded Response Model to examine its psychometric properties in Spanish-speaking university students (n = 1503; women = 995 (66.2%), mean age = 19.29, SD = 2.45). We tested whether the empirical data fit the model's predictions and estimated the dispersion of persons and items along the experiential avoidance continuum. Moreover, we examined category probability curves to identify the response probability of each answer. Likewise, an item-person map was made where the measurement of persons and items, both on the same scale and along the experiential avoidance continuum, could be observed jointly. Finally, we tested the gender invariance of the scale.

RESULTS

We found that the values of the individuals and the items were in the established range to be considered an adequate measure of EA. Additionally, we observed high discrimination indices for all items. The current version with seven answer options could not be optimal and should be tested in future studies. Finally, we found evidence of differential functioning by gender in one of the seven items of the instrument.

CONCLUSIONS

Our results indicate that the AAQ-II is a suitable tool for measuring EA and accurately classifying and differentiating EA levels in university students.

Longitudinal joint modeling for assessing parallel interactive development of latent ability and processing speed using responses and response times

To measure the parallel interactive development of latent ability and processing speed using longitudinal item response accuracy (RA) and longitudinal response time (RT) data, we proposed three longitudinal joint modeling approaches from the structural equation modeling perspective, namely unstructured-covariance-matrix-based longitudinal joint modeling, latent growth curve-based longitudinal joint modeling, and autoregressive cross-lagged longitudinal joint modeling. The proposed modeling approaches can not only provide the developmental trajectories of latent ability and processing speed individually, but also exploit the relationship between the change in latent ability and processing speed through the across-time relationships of these two constructs. The results of two empirical studies indicate that (1) all three models are practically applicable and have highly consistent conclusions in terms of the changes in ability and speed in the analysis of the same data set, and (2) additional analysis of the RT data and acquisition of individual processing speed measurements can reveal the parallel interactive development phenomena that are difficult to detect using RA data alone. Furthermore, the results of our simulation study demonstrate that the proposed Bayesian Markov chain Monte Carlo estimation algorithm can ensure accurate model parameter recovery for all three proposed longitudinal joint models. Finally, the implications of our findings are discussed from the research and practice perspectives.

Modeling insufficient effort responses in mixed-worded scales

Surveys often add reverse-coded questions to monitor respondents with insufficient effort responses (IERs) but often wrongly assume that all respondents consistently answer all questions with full effort. By contrast, this study expanded the mixture model for IERs and ran a simulation via LatentGOLD to show the harmful consequences of ignoring IERs to positively and negatively worded questions: less test reliability, bias and less accuracy in slope and intercept parameters. We showed its practical application to two public data sets: Machiavellianism (five-point scale) and self-reported depression (four-point scale).

The U.S. PFAS exposure burden calculator for 2017-2018: Application to the HOME Study, with comparison of epidemiological findings from NHANES

BACKGROUND

The 2017-2018 U.S. PFAS exposure burden calculator was designed to provide a summary exposure score for per- and polyfluoroalkyl substances (PFAS) mixtures using targeted PFAS analyte data. Its aim was to place PFAS burden score estimates onto a common scale based on nationally representative U.S. reference ranges from 2017 to 2018, enabling comparisons of overall PFAS burden scores across studies even if they did not measure the same set of PFAS analytes.

OBJECTIVE

To use the U.S. PFAS exposure burden calculator for comparing the same mixture of PFAS compounds in similarly aged adolescents and their associations with cardiometabolic outcomes in the HOME Study and NHANES between 2015 and 2018.

METHODS

We applied the PFAS burden calculator to 8 PFAS analytes measured in the serum of adolescents from the HOME Study (Cincinnati, Ohio; age range 11-14 years; years: 2016-2019; n = 207) and NHANES (US; age range 12-14 years; years 2015-2018; n = 245). We used the non-parametric Mann-Whitney U test and chi-squared test to compare the two study samples. In both studies, we examined associations of PFAS burden scores with the same cardiometabolic outcomes, adjusted for the same core set of covariates using regression analyses. We conducted sensitivity analyses to verify robustness of exposure-outcome associations, by accounting for measurement error of PFAS burden scores.

RESULTS

PFAS burden scores were significantly different (p = 0.004) between the HOME Study (median: 0.00, interquartile range - 0.37, 0.34) and the NHANES samples (median: 0.04, IQR -0.11, 0.54), while no significant difference was found for PFAS summed concentrations (p = 0.661). In the HOME Study, an interquartile (IQR) increase in PFAS burden score was associated with higher total cholesterol [7.0 mg/dL, 95% CI: 0.6, 13.4]; HDL [2.8 mg/dL, 95% CI: 0.4, 5.2]; LDL [5.9 mg/dL, 95% CI: 0.5, 11.3], insulin [0.1 log(mIU/L), 95% CI: 0.01, 0.2], and HOMA-IR [0.1, 95% CI: 0.01, 0.2]. In NHANES, an IQR increase in PFAS burden score was associated with higher diastolic blood pressure [2.4 mmHg, 95% CI: 0.4, 4.4] but not with other outcomes. Sensitivity analyses in the HOME Study and NHANES were consistent with the main findings.

CONCLUSIONS

Performance of the U.S. PFAS exposure burden calculator was similar in a local versus national sample of adolescents, and may be a useful tool for the assessment of PFAS mixtures across studies.

A random item effects generalized partial credit model with a multiple imputation-based scoring procedure

PURPOSE

Random item effects item response theory (IRT) models have received much attention for more than a decade. However, more research is needed on random item effects IRT models for polytomous data. Additionally, to improve the utility of this new class of IRT models, the scoring issue must be addressed.

METHODS

We proposed a new random item effects generalized partial credit model (GPCM), which considers both random person and random item and category-specific effects. In addition, we introduced a multiple imputation (MI)-based scoring procedure that applies to various random item effects IRT models. To evaluate the proposed model and scoring procedure, we analyzed data from a Quality of Life (QoL) scale for the Chronically Mentally III and conducted a preliminary simulation study.

RESULTS

In the empirical data analysis, we found that patient scores generated based on the proposed model and scoring procedure were almost identical to those obtained through the conventional GPCM and scoring method. However, the standard errors (SEs) associated with the scores were slightly larger when the proposed approach was utilized. In the simulation study, we observed adequate recovery of the model parameters and patient scores.

CONCLUSION

The proposed model and MI-based scoring procedure contribute to the literature. The proposed model substantially reduces the number of free parameters in comparison to a conventional GPCM, which can be desired when sample sizes are small, e.g., special populations. In addition, the MI-based scoring procedure addresses the scoring issue and can be easily extended for scoring with other random item effects IRT models.

An item response theory analysis of the matrix reasoning item bank (MaRs-IB)

Matrix reasoning tasks are among the most widely used measures of cognitive ability in the behavioral sciences, but the lack of matrix reasoning tests in the public domain complicates their use. Here, we present an extensive investigation and psychometric validation of the matrix reasoning item bank (MaRs-IB), an open-access set of matrix reasoning items. In a first study, we calibrate the psychometric functioning of the items in the MaRs-IB in a large sample of adult participants (N = 1501). Using additive multilevel item structure models, we establish that the MaRs-IB has many desirable psychometric properties: its items span a wide range of difficulty, possess medium-to-large levels of discrimination, and exhibit robust associations between item complexity and difficulty. However, we also find that item clones are not always psychometrically equivalent and cannot be assumed to be exchangeable. In a second study, we demonstrate how experimenters can use the estimated item parameters to design new matrix reasoning tests using optimal item assembly. Specifically, we design and validate two new sets of test forms in an independent sample of adults (N = 600). We find these new tests possess good reliability and convergent validity with an established measure of matrix reasoning. We hope that the materials and results made available here will encourage experimenters to use the MaRs-IB in their research.

Face identification proficiency test designed using item response theory

Measures of face-identification proficiency are essential to ensure accurate and consistent performance by professional forensic face examiners and others who perform face-identification tasks in applied scenarios. Current proficiency tests rely on static sets of stimulus items and so cannot be administered validly to the same individual multiple times. To create a proficiency test, a large number of items of "known" difficulty must be assembled. Multiple tests of equal difficulty can be constructed then using subsets of items. We introduce the Triad Identity Matching (TIM) test and evaluate it using item response theory (IRT). Participants view face-image "triads" (N = 225) (two images of one identity, one image of a different identity) and select the different identity. In Experiment 3, university students (N = 197) showed wide-ranging accuracy on the TIM test, and IRT modeling demonstrated that the TIM items span various difficulty levels. In Experiment 3, we used IRT-based item metrics to partition the test into subsets of specific difficulties. Simulations showed that subsets of the TIM items yielded reliable estimates of subject ability. In Experiments 3a and b, we found that the student-derived IRT model reliably evaluated the ability of non-student participants and that ability generalized across different test sessions. In Experiment 3c, we show that TIM test performance correlates with other common face-recognition tests. In summary, the TIM test provides a starting point for developing a framework that is flexible and calibrated to measure proficiency across various ability levels (e.g., professionals or populations with face-processing deficits).

Validation of the satisfaction with life scale for Korean older adults using item response theory

As Korea is rapidly becoming an aging society, people are increasingly interested in the life satisfaction of the older adults. Therefore, the need for a tool that can precisely measure the life satisfaction of the older adults in Korea is also increasing. In this study, SWLS, which has been widely used to measure life satisfaction, was applied to Korean older adults. This study analyzed the life satisfaction of the older adults using item response theory that can supplement the limitations of existing research methods. Data from 3543 older adults who responded to the 2021 Korean Happiness Survey. As a result of the analysis, it was found that the goodness of fit of item 5 exceeded the standard value, and item 4 functioned differentially between the young-old and old-old. Based on the results of this study, it presented an empirical basis for the application of SWLS to the older adults.

Enhancing Patient-Reported Outcome Measures for Drug Hypersensitivity: A Validated 6-Item Quality-of-Life Questionnaire for Patients With Drug Hypersensitivity

BACKGROUND

Drug hypersensitivity reactions (DHRs) can significantly impair patients' health-related quality of life (HRQoL). However, tools for HRQoL assessment for patients with DHR are time-consuming and remain underutilized.

OBJECTIVE

To develop and validate an optimized version of the Drug Hypersensitivity Quality-of-Life Questionnaire (DrHy-Q) designed for everyday clinical use.

METHODS

Item response theory (IRT), a statistical framework for psychometric measurement, was used to evaluate the 15 questions from the original DrHy-Q for their respective item difficulty, discrimination, and information using prospective data from 243 patients with histories of suspected/confirmed DHR before allergy workup. Accordingly, the best-performing items were identified to develop a 6-item optimized version (DrHy-Q6), which was subsequently validated with another prospective cohort of 156 patients.

RESULTS

All 15 items of the original DrHy-Q demonstrated satisfactory parameters in IRT analysis, including very high discrimination (>1.7), appropriate difficulty (in between -1.5 and 1.5), and good information (a high and broad peak in the information curve). Six items with top-ranked IRT parameters were identified to construct an optimized version, which we named the DrHy-Q6. The DrHy-Q6 demonstrated a 1-factor structure with an improved fit compared with the original DrHy-Q (comparative fit index = 0.985, Tucker-Lewis index = 0.974), excellent convergent validity (unadjusted Pearson correlation with the full version = 0.955; adjusted = 0.894, P < .001), reliability (Cronbach's α and McDonald's ω = 0.93), divergent validity (Pearson correlation with all Short Form 12-item Health Survey Version 2 subscales <0.60, P < .001), and discriminant validity (significantly higher scores with multiple DHR labels [42.45 ± 27.26 vs 32.93 ± 26.66], P = .013).

CONCLUSIONS

From an IRT perspective, the DrHy-Q and all its constituent items are psychometrically valid for HRQoL assessment. We propose an optimized 6-item version (DrHy-Q6) as an abbreviated alternative for assessing HRQoL in patients with DHR, especially for routine use in clinical practice. Patients and physicians may benefit from its streamlined length and simpler scoring algorithm.

Development and validation of the patient-reported outcome scale for chronic kidney disease

PURPOSE

The patient-reported outcomes (PROs) measuring patient's experience and perception of disease are important components of approach to care. However, no tools are available to assess the PROs of chronic kidney disease (CKD). This study aims to develop and verify a PROs scale to evaluate clinical outcomes in CKD patients.

METHODS

The theoretical structure model and original item pool were formed through a literature review, patient interviews and references to relevant scales. The Delphi method, classical test theory methods and item response theory method were used to select items and adjust dimensions to form the final scale. Altogether 360 CKD patients were recruited through convenience sampling. CKD-PROs could be evaluated from four aspects, namely reliability, content validity, construct validity, responsibility, and feasibility.

RESULTS

The CKD-PROs scale covers 4 domains, including the physiological, psychological, social, and therapeutic domain, and 12 dimensions, 54 items. The Cronbach's α is 0.939, the split reliability coefficient is 0.945, and the correlation of the scores each item and domain's coefficients range from 0.413 to 0.669. The results of structure validity, content validity and reactivity showed that the multidimensional measurement of the scale met professional expectations. The recovery rate and effective rate of the scale were over 99%.

CONCLUSION

The CKD-PROs scale has great reliability, validity, reactivity, acceptability and is capable of being used as one of the evaluation tools for the clinical outcomes of CKD patients.

Developing the ACEIG-scale: An adverse childhood experience scale for Inuit youth in Greenland

BACKGROUND

Adverse Childhood Experiences (ACEs) have been identified as a major public health challenge in Greenland. No previous studies have created a multi-item ACE- scale among an Arctic Indigenous population.

OBJECTIVE

To develop a multi-item ACE-scale among Inuit youth in Greenland (the ACEIG scale).

METHODS

The ACEIG scale was created with data from the 'Wellbeing among Inuit youth in Greenland'-survey. Scale items were based on a recognised ACE-scale and further adapted to the context of Inuit youth in Greenland by adding items relevant for the population. The scale was validated through item response theory (IRT) and reliability was assessed by Cronbach's alpha.

RESULTS

Four items relevant for Inuit youth in Greenland were added to the recognised ACE scale (bullying, death of parent, gambling problems in close family, and suicide in close relations). The scale was reduced by IRT, as three items (bullying, divorce of parents and parents passing away) exceeded the difficulty index threshold. The ACEIG scale therefore consists of 10 items: alcohol problems in close family, marijuana use in close family, domestic violence, being victim of physical violence, being victim of psychological violence, any type of sexual abuse, sexual abuse (intercourse), sexual abuse (more than once), suicide in close relations, and gambling problems in close family. Cronbach's alpha was 0.7.

CONCLUSION

The ACEIG scale includes 10 items with acceptable reliability. The scale can inform future screening tools to identify vulnerable youth and target interventions. Future studies should investigate the association between the ACEIG scale and health outcomes.

Accounting for careless and insufficient effort responding in large-scale survey data-development, evaluation, and application of a screen-time-based weighting procedure

Careless and insufficient effort responding (C/IER) poses a major threat to the quality of large-scale survey data. Traditional indicator-based procedures for its detection are limited in that they are only sensitive to specific types of C/IER behavior, such as straight lining or rapid responding, rely on arbitrary threshold settings, and do not allow taking the uncertainty of C/IER classification into account. Overcoming these limitations, we develop a two-step screen-time-based weighting procedure for computer-administered surveys. The procedure allows considering the uncertainty in C/IER identification, is agnostic towards the specific types of C/IE response patterns, and can feasibly be integrated with common analysis workflows for large-scale survey data. In Step 1, we draw on mixture modeling to identify subcomponents of log screen time distributions presumably stemming from C/IER. In Step 2, the analysis model of choice is applied to item response data, with respondents' posterior class probabilities being employed to downweigh response patterns according to their probability of stemming from C/IER. We illustrate the approach on a sample of more than 400,000 respondents being administered 48 scales of the PISA 2018 background questionnaire. We gather supporting validity evidence by investigating relationships between C/IER proportions and screen characteristics that entail higher cognitive burden, such as screen position and text length, relating identified C/IER proportions to other indicators of C/IER as well as by investigating rank-order consistency in C/IER behavior across screens. Finally, in a re-analysis of the PISA 2018 background questionnaire data, we investigate the impact of the C/IER adjustments on country-level comparisons.

Using item response theory to estimate interpretation threshold values for the Frailty Index in community dwelling older adults

BACKGROUND

Although the frailty index (FI) is designed as a continuous measure of frailty, thresholds are often needed to guide its interpretation. This study aimed to introduce and demonstrate the utility of an item response theory (IRT) method in estimating FI interpretation thresholds in community-dwelling adults and to compare them with cutoffs estimated using the receiver operating characteristics (ROC) method.

METHODS

A sample of 1,149 community-dwelling adults (mean[SD], 68[7] years) participated in this cross-sectional study. Participants completed a multi-domain geriatric screen from which the 40-item FI and 3 clinical anchors were computed - namely, (i)self-reported mobility limitations (SRML), (ii)"fair" or "poor" self-rated health (SRH), and (iii) restricted life-space mobility (RLSM). Participants were classified as having SRML-1 if they responded "Yes" to either of the 2 questions regarding walking and stair climbing difficulty and SRML-2 if they reported having walking and stair climbing difficulty. Participants with a Life Space Assessment score <60 points were classified as having RLSM. Threshold values for all anchor questions were estimated using the IRT method and ROC analysis with Youden criterion.

RESULTS

The proportions of participants with SRML-1, SRML-2, Fair/Poor SRH, and RLSM were 21 %, 8 %, 22 %, and 9 %, respectively. The IRT-based thresholds for SRML-2 (0.26), fair/poor SRH (0.29), and RLSM (0.32) were significantly higher than those for SRML-1 (0.18). ROC-based FI cutoffs were significantly lower than IRT-based values for SRML-2, SRH, and RLSM (0.12 to 0.17), and they varied minimally and non-systematically across the anchors.

CONCLUSIONS

The IRT method identifies biologically plausible FI thresholds that could meaningfully complement and contextualize existing thresholds for defining frailty.

The effects of careless responding on the fit of confirmatory factor analysis and item response theory models

It is common to model responses to surveys within latent variable frameworks (e.g., item response theory [IRT], confirmatory factor analysis [CFA]) and use model fit indices to evaluate model-data congruence. Unfortunately, research shows that people occasionally engage in careless responding (CR) when completing online surveys. While CR has the potential to negatively impact model fit, this issue has not been systematically explored. To better understand the CR-fit linkage, two studies were conducted. In study 1, participants' response behaviors were experimentally shaped and used to embed aspects of a comprehensive simulation (study 2) with empirically informed data. For this simulation, 144 unique conditions (which varied the sample size, number of items, CR prevalence, CR severity, and CR type), two latent variable models (IRT, CFA), and six model fit indices (χ2, RMSEA, SRMSR [CFA] and M2, RMSEA, SRMSR [IRT]), were examined. The results indicated that CR deteriorates model fit under most circumstances, though these effects are nuanced, variable, and contingent on many factors. These findings can be leveraged by researchers and practitioners to improve survey methods, obtain more accurate survey results, develop more precise theories, and enable more justifiable data-driven decisions.

Psychometric evaluation of the WHOQOL-BREF and its shorter versions for general Thai population: confirmatory factor analysis and Rasch analysis

PURPOSE

Rasch analysis was employed to validate the World Health Organization Quality of Life-BREF (WHOQOL-BREF) and its existing shorter versions in the general Thai population.

METHODS

1200 respondents were randomly selected to complete the questionnaire. Confirmatory factor analysis (CFA) was employed to test the structure of the WHOQOL-BREF and its shorter versions with the random sub-sample of 900 respondents, while Rasch analysis was performed with a random sub-sample of 300 respondents.

RESULTS

The CFA confirmed the factor structure of WHOQOL-BREF and its shorter versions. The Rasch analysis revealed that the WHOQOL-BREF, when a four-domain structure was tested using a subtest approach, achieved acceptable model fit to the Rasch model and met the expectations of unidimensionality with high reliability (PSI = 0.87). Individual domain models were also unidimensional, but reliability of the 3-item social domain was inadequate. While the 8-item EUROHIS-QOL-8 and 5-item WHOQOL-5 achieved an overall acceptable fit and met the expectations of unidimensionality, the reliability of the WHOQOL-5 was below the acceptable threshold (PSI = 0.66). Reliability of the EUROHIS-QOL-8 was satisfactory (PSI = 0.79).

CONCLUSIONS

The WHOQOL-BREF is a valid instrument for use in the Thai general population, both as a total score as well as individual subscales. Rasch analysis also supports the use of EUROHIS-QOL-8, but the WHOQOL-5 lacks good reliability. While the reliability of the EUROHIS-QOL-8 is sufficiently high for between-group analysis, the Thai WHOQOL-BREF total score can also be used for within-participant analyses. Rasch investigation with a more varied health conditions of general Thai samples or patient groups is encouraged for future studies.

Comparison of Two Methods for Determining Item Characteristic Functions and Latent Variable Time-Course for Pharmacometric Item Response Models

There are examples in the literature demonstrating different approaches to defining the item characteristic functions (ICF) and characterizing the latent variable time-course within a pharmacometrics item response theory (IRT) framework. One such method estimates both the ICF and latent variable time-course simultaneously, and another method establishes the ICF first then models the latent variable directly. To date, a direct comparison of the "simultaneous" and "sequential" methodologies described in this work has not yet been systematically investigated. Item parameters from a graded response IRT model developed from Parkinson's Progression Marker Initiative (PPMI) study data were used as simulation parameters. Each method was evaluated under the following conditions: (i) with and without drug effect and (ii) slow progression rate with smaller sample size and rapid progression rate with larger sample size. Overall, the methods performed similarly, with low bias and good precision for key parameters and hypothesis testing for drug effect. The ICF parameters were well determined when the model was correctly specified, with an increase in precision in the scenario with rapid progression. In terms of drug effect, both methods had large estimation bias for the slow progression rate; however, this bias can be considered small relative to overall progression rate. Both methods demonstrated type 1 error control and similar discrimination between model with and without drug effect. The simultaneous method was slightly more precise than the sequential method while the sequential method was more robust towards longitudinal model misspecification and offers practical advantages in model building.

Analyzing the psychometric properties of the PHQ-9 using item response theory in a Chinese adolescent population

BACKGROUND

People are more likely to fall victim to depression during adolescence since it is a period of rapid biopsychosocial transformation. Despite this, most depression research has concentrated on clinical issues, and evaluating depressive symptoms in teenagers is not as widespread. This study used item response theory (IRT) to examine the psychometric properties of the Patient Health Report scale (PHQ-9) in Chinese adolescents. Meanwhile, item function difference tests were used to check whether there were differences in depression symptoms in this group based on education and gender.

METHODS

In this research, the PHQ-9 was employed as a measurement tool, and 5958 valid data points were obtained from 12 secondary schools in China (Mage = 13.484; SDage = 1.627; range 11-19 years; 52.17% boys).

RESULTS

IRT shows that all items of the PHQ-9 satisfy monotonicity, unidimensionality and local independence and that they have good psychometric properties. Furthermore, DIF analysis revealed gender and educational disparities in adolescent depressive symptoms.

CONCLUSION

The study indicates that the PHQ-9 possesses favourable psychometric properties for use in Chinese adolescents. As a result, it serves as a valuable tool for effectively screening depressive symptoms in adolescents. It provides a foundation for prioritizing the development of secondary school students' physical and mental health.

Evaluation of the psychometric properties of the family adaptability and cohesion scale (FACES III) through item response theory models in students from Chile and Colombia

BACKGROUND

A psychometric study of the Family Adaptability and Cohesion Scale (FACES III) has been conducted in Spanish-speaking countries from the perspective of the classical test theory. However, this approach has limitations that affect the psychometric understanding of this scale.

OBJECTIVE

Accordingly, this study used the item response theory to investigate the psychometric performance of the items. Furthermore, it evaluated the differential performance of the items for Colombia and Chile.

METHOD

For this purpose, 518 health science students from both countries participated. Confirmatory Factor Analysis was used.

RESULTS

The study results revealed that the cohesion and adaptability items presented adequate discrimination and difficulty indices. In addition, items 5, 8, 13, 17, and 19 of cohesion indicated differential functioning between students from both countries, with Chilean students exhibiting a greater discriminatory power. Further, the Colombian group exhibited a greater discriminatory power for item 18 of adaptability.

CONCLUSIONS

The study concluded that the items of FACES III indicated adequate psychometric performance in terms of their discriminative capacity and difficulty in Chile and Colombia.

Reliability, validity, and simplification of the Chinese version of the Global Pain Scale in patients with rheumatoid arthritis

BACKGROUND

Persistent pain is the most reported symptom in patients with rheumatoid arthritis (RA); however, effective and brief assessment tools are lacking. We validated the Chinese version of the Global Pain Scale (C-GPS) in Chinese patients with RA and proposed a short version of the C-GPS (s-C-GPS).

METHOD

The study was conducted using a face-to-face questionnaire survey with a multicenter cross-sectional design from March to December 2019. Patients aged > 18 years who met the RA diagnostic criteria were included. Based on the classical test theory (CTT) and the item response theory (IRT), we assessed the validity and reliability of the C-GPS and the adaptability of each item. An s-C-GPS was developed using IRT-based computerized adaptive testing (CAT) analytics.

RESULTS

In total, 580 patients with RA (mean age, 51.04 ± 24.65 years; mean BMI, 22.36 ± 4.07 kg/m2), including 513 (88.4%) women, were included. Most participants lived in a suburb (49.3%), were employed (72.2%) and married (91.2%), reported 9-12 years of education (66.9%), and had partial medical insurance (57.8%). Approximately 88.1% smoked and 84.5% drank alcohol. Analysis of the CTT demonstrated that all items in the C-GPS were positively correlated with the total scale score, and the factor loadings of all these items were > 0.870. A significant positive relationship was found between the Visual Analog Scale (VAS) and the C-GPS. IRT analysis showed that discrimination of the C-GPS was between 2.271 and 3.312, and items 6, 8, 13, 14, and 16 provided a large amount of information. Based on the CAT and clinical practice, six items covering four dimensions were included to form the s-C-GPS, all of which had very high discrimination. The s-C-GPS positively correlated with the VAS.

CONCLUSION

The C-GPS has good reliability and validity and can be used to evaluate pain in RA patients from a Chinese cultural background. The s-C-GPS, which contains six items, has good criterion validity and may be suitable for pain assessment in busy clinical practice.

TRIAL REGISTRATION

This cross-sectional study was registered in the Chinese Clinical Trial Registry (ChiCTR1800020343), granted on December 25, 2018.

A Time-Varying Dynamic Partial Credit Model to Analyze Polytomous and Multivariate Time Series Data

The accessibility to electronic devices and the novel statistical methodologies available have allowed researchers to comprehend psychological processes at the individual level. However, there are still great challenges to overcome as, in many cases, collected data are more complex than the available models are able to handle. For example, most methods assume that the variables in the time series are measured on an interval scale, which is not the case when Likert-scale items were used. Ignoring the scale of the variables can be problematic and bias the results. Additionally, most methods also assume that the time series are stationary, which is rarely the case. To tackle these disadvantages, we propose a model that combines the partial credit model (PCM) of the item response theory framework and the time-varying autoregressive model (TV-AR), which is a popular model used to study psychological dynamics. The proposed model is referred to as the time-varying dynamic partial credit model (TV-DPCM), which allows to appropriately analyze multivariate polytomous data and nonstationary time series. We test the performance and accuracy of the TV-DPCM in a simulation study. Lastly, by means of an example, we show how to fit the model to empirical data and interpret the results.

Screening for cannabis use disorder among young adults: Sensitivity, specificity, and item-level performance of the Cannabis Use Disorders Identification Test - Revised

The Cannabis Use Disorders Identification Test Revised (CUDIT-R) is a widely used screening tool. However, its utility in screening for cannabis use disorder (CUD) among young adults requires further investigation. The current study evaluated the accuracy of the CUDIT-R in distinguishing between young adults with and without CUD. We also examined the individual item properties of the CUDIT-R in this sample. A community sample of young adults (N = 153, ages 19-26 years) who reported at least one lifetime period of regular cannabis use (i.e., twice or more per month for at least six months) completed the CUDIT-R and the Structured Clinical Interview for DSM-5 (SCID-5) Substance Use Module. The CUDIT-R showed good accuracy in distinguishing between young adults with and without CUD as determined by the SCID-5 (area under the receiver operating characteristic curve = 0.84). A cut-point of nine yielded 87.80 % sensitivity and 70.42 % specificity, with positive and negative predictive values of 77.42 % and 83.33 %, respectively. Item response theory analyses revealed that CUDIT-R items assessing impaired control, failure to meet expectations, time devoted to use, and memory and concentration problems displayed high discrimination and were relatively difficult, whereas items assessing frequency of use and attempts to reduce use showed moderate discrimination and relatively low difficulty. Items assessing hours high and use with potential for physical hazard displayed poor discrimination. The CUDIT-R appears to validly distinguish between young adults with and without CUD and may be useful in screening for CUD in this high-risk population. Further research considering individual item properties is needed to refine the CUDIT-R for use among young adults.

Validation of the Chinese version of academic goals orientation questionnaire in nursing student: a study based on SEM and IRT multidimensional models

OBJECTIVE

To translate the Academic Goals Orientation Questionnaire (AGOQ) into Chinese and to determine the validity and reliability of the (AGOQ) in Chinese nursing students based on SEM and IRT multidimensional models.

METHODS

The participants were 654 nursing students with an age range of 17-26 years (mean age 21.61 ± 1.73 years). The psychometric properties of AGOQ were investigated based on a dual analytical perspective of structural equation modeling (SEM) and item response theory (IRT).

RESULTS

The Cronbach's α value of the questionnaire is 0.895. A four-factor model was obtained by exploratory factor analysis, which explained the variance of 71.892%. With confirmatory factor analysis, a new four-factors model was built and showed an acceptable goodness-of-fit, chi-square/degree of freedom (CMIN/DF) = 4.008, goodness of fit index (GFI) = 0.932, adjusted goodness of fit index (AGFI) = 0.905, comparative fit index (CFI) = 0.952, incremental fit index (IFI) = 0.952, Tucker Lewis index (TLI) = 0.941. In the analysis part of IRT, according to the comparison between Akek's information criterion (AIC) and Bayesian information criterion (BIC), we choose the Graded Response Model (GRM) for analysis. The results show that the difficulty value is monotonically increasing, and the discrimination of all items is greater than 0.19, which shows that 16 items can be retained.

CONCLUSIONS

This study tested the psychometric characteristics of AGOQ of nursing students in China. The results confirmed that the Chinese version of AGOQ has good psychometric characteristics and can be used to measure the academic goal orientation of nursing students in China.

Development and Psychometric Evaluation of the Open-Source Challenging Behavior Scale (OS-CBS)

At present, there are no brief, freely-available, informant-report measures that evaluate key challenging behaviors relevant to youth with autism spectrum disorder (ASD) or other developmental disabilities (DD). This paper describes the development, refinement, and initial psychometric evaluation of a new 18-item measure, the Open-Source Challenging Behavior Scale (OS-CBS). In a large sample (n = 2004, 169 with ASD, ages 2-17), results of psychometric analyses indicated a clear factor structure (property destruction, aggression, elopement, conduct problems, and self-injury and a general factor with high loadings from all items) based on exploratory structural equation modeling, good scale reliability (α = .66-.83 for subscales, α = .91 total scale), measurement invariance across demographics, and good construct validity. The OS-CBS is a psychometrically-sound instrument for screening and monitoring intervention progress.

Robustness of the performance of the optimized hierarchical two-parameter logistic IRT model for small-sample item calibration

Hierarchical Bayesian modeling is beneficial when complex models with many parameters of the same type, such as item response theory (IRT) models, are to be estimated with sparse data. Recently, Koenig et al. (Applied Psychological Measurement, 44, 311-326, 2020) illustrated in an optimized hierarchical Bayesian two-parameter logistic model (OH2PL) how to avoid bias due to unintended shrinkage or degeneracies of the posterior, and how to benefit from this approach in small samples. The generalizability of their findings, however, is limited because they investigated only a single specification of the hyperprior structure. Consequently, in a comprehensive simulation study, we investigated the robustness of the performance of the novel OH2PL in several specifications of their hyperpriors under a broad range of data conditions. We show that the novel OH2PL in the half-Cauchy or Exponential configuration yields unbiased (in terms of bias) model parameter estimates in small samples of N = 50. Moreover, it outperforms (especially in terms of the RMSE of the item discrimination parameters) marginal maximum likelihood (MML) estimation and its nonhierarchical counterpart. This further corroborates the possibility that hierarchical Bayesian IRT models behave differently than general hierarchical Bayesian models. We discuss these results regarding the applicability of complex IRT models in small-scale situations typical in psychological research, and illustrate the extended applicability of the 2PL IRT model with an empirical example.

Accelerating item factor analysis on GPU with Python package xifa

Item parameter estimation is a crucial step when conducting item factor analysis (IFA). From the view of frequentist estimation, marginal maximum likelihood (MML) seems to be the gold standard. However, fitting a high-dimensional IFA model by MML is still a challenging task. The current study demonstrates that with the help of a GPU (graphics processing unit) and carefully designed vectorization, the computational time of MML could be largely reduced for large-scale IFA applications. In particular, a Python package called xifa (accelerated item factor analysis) is developed, which implements a vectorized Metropolis-Hastings Robbins-Monro (VMHRM) algorithm. Our numerical experiments show that the VMHRM on a GPU may run 33 times faster than its CPU version. When the number of factors is at least five, VMHRM (on GPU) is much faster than the Bock-Aitkin expectation maximization, MHRM implemented by mirt (on CPU), and the importance-weighted autoencoder (on GPU). The GPU-implemented VMHRM is most appropriate for high-dimensional IFA with large data sets. We believe that GPU computing will play a central role in large-scale psychometric modeling in the near future.

The validity and IRT psychometric analysis of Chinese version of Difficult Doctor-Patient Relationship Questionnaire (DDPRQ-10)

OBJECTIVE

The doctor-patient relationship (DPR) plays a crucial role in the Chinese healthcare system, functioning to improve medical quality and reduce medical costs. This study examined the psychometric properties of the Chinese version of the Difficult Doctor-Patient Relationship Questionnaire (DDPRQ-10) among general hospital inpatients in China.

METHODS

The research recruited 38 resident doctors responsible for 120 participants, and factor analyses were used to assess the construct validity of the scale. Convergent validity was evaluated by examining the correlation between DDPRQ-10 and depressive symptoms, burnout, and self-efficacy, using the Patient Health Questionnaire Depression Scale-9 item (PHQ-9), and the Maslach Burnout Inventory (MBI). Both multidimensional item response theory (MIRT) and unidimensional item response theory (IRT) frameworks were used to estimate the parameters of each item.

RESULTS

The Chinese version of DDPRQ-10 showed satisfactory internal consistency (Cronbach's alpha = 0.931), and fitted in a modified two-factor model of positive feelings and negative feelings (χ2/df = 1.494, GFI = 0.925, RMSEA = 0.071, SRMR = 0.008, CFI = 0.985, NFI = 0.958, NNFI = 0.980, TLI = 0.980, IFI = 0.986). Significant correlations with PHQ-9 with DDPRQ-10 and both subscales were revealed (r = 0.293 ~ 0.333, p < .001), while DDPRQ-10 score also significantly correlated with doctors' MBI score (r = -0.467, p < .001). The MIRT model of full scale and IRT models of both subscales showed high discrimination of all items (a = 2.30 ~ 10.18), and the test information within the range of low-quality relationship was relatively high.

CONCLUSION

The Chinese version of DDPRQ-10 displayed satisfactory reliability and validity and thus was appropriate for measuring the DPR in Chinese medical settings.

Establishing the measurement precision of the patient health questionnaire in an adolescent sample

BACKGROUND

Technically sound measures are necessary for accurately identifying youth at risk for depression, but many studies rely on classical test theory metrics or adult samples to evaluate measures. This study examined the use of the PHQ-8, a common and freely available pediatric depression screener, in an adolescent sample using item response theory (IRT).

METHODS

Secondary analyses were conducted on data from a study conducted in Midwestern middle schools in which 1224 youth completed the PHQ-8 as part of a battery of surveys. Polytomous IRT analyses (a Graded Response Model) were used to evaluate the PHQ-8. Items were examined for their ability to distinguish between respondents of different latent depression severity and for differential item functioning (DIF) across demographic categories.

RESULTS

All PHQ-8 items had adequate discriminative abilities. Items measuring anhedonia and psychomotor disturbances performed relatively poorly, and items measuring somatic symptoms (appetite and sleep) were most informative when respondents endorsed extreme response options ("not at all" or "nearly every day"). No DIF was found across grade level or race, but several items were flagged for DIF by gender and student income level.

LIMITATIONS

These results might not be generalizable to a broader youth population due to administration setting and the unique demographic characteristics of this sample (76.0 % African American).

CONCLUSIONS

Tools such as the PHQ-8 are appropriate to quickly screen for depression in adolescents, but further scrutiny of adolescent response patterns is warranted. Future research should examine items measuring anhedonia and psychomotor and somatic disturbances in adolescents.

A Bayesian Approach Towards Missing Covariate Data in Multilevel Latent Regression Models

The measurement of latent traits and investigation of relations between these and a potentially large set of explaining variables is typical in psychology, economics, and the social sciences. Corresponding analysis often relies on surveyed data from large-scale studies involving hierarchical structures and missing values in the set of considered covariates. This paper proposes a Bayesian estimation approach based on the device of data augmentation that addresses the handling of missing values in multilevel latent regression models. Population heterogeneity is modeled via multiple groups enriched with random intercepts. Bayesian estimation is implemented in terms of a Markov chain Monte Carlo sampling approach. To handle missing values, the sampling scheme is augmented to incorporate sampling from the full conditional distributions of missing values. We suggest to model the full conditional distributions of missing values in terms of non-parametric classification and regression trees. This offers the possibility to consider information from latent quantities functioning as sufficient statistics. A simulation study reveals that this Bayesian approach provides valid inference and outperforms complete cases analysis and multiple imputation in terms of statistical efficiency and computation time involved. An empirical illustration using data on mathematical competencies demonstrates the usefulness of the suggested approach.

Psychometric properties of the Chinese version of the PROMIS-Cancer-Anxiety item bank assessed using a graded response model

Objective

This study aimed to examine the psychometric properties of the Chinese version of the Patient-Reported Outcome Measurement Information System (PROMIS)-Cancer-Anxiety item bank using a graded response model in a sample of patients with cancer.

Methods

A cross-sectional study was conducted and the Chinese version of the PROMIS-Cancer-Anxiety item bank was used to measure anxiety in patients with cancer. The unidimensional structure of the item bank was evaluated using principal component analysis. Residual correlations and the graphs of item mean scores conditional on the rest scores were examined to evaluate the local independence and monotonicity of the items, respectively. Item characteristics were described using item parameter estimates and item information. Operating characteristic curves (OCCs) and test information curve (TIC) were also plotted. Measurement invariance across age, gender, and education level was assessed to identify possible differential item functioning (DIF).

Results

A total of 1075 patients with cancer were enrolled. Under the assumptions of unidimensionality, local independence, and monotonicity, the discrimination parameters a ranged from 2.30 to 5.47, and the threshold parameters b ranged from b1 = -2.87 to b4 = 3.21 with proper intervals. Completely overlapped category curves were not observed among the OCCs of any items. Item information and TIC showed that the item bank had a wide measurement range. The DIFs for age, gender, and education level for all items were not remarkable.

Conclusions

The results supported using the Chinese version of the PROMIS-Cancer-Anxiety item bank to measure anxiety and develop a computerized adaptive testing (CAT) system for anxiety in patients with cancer.

Multidimensional item response theory models for testlet-based doubly bounded data

A testlet-based visual analogue scale (VAS) is a doubly bounded scaling approach (e.g., from 0% to 100% or from 0 to 1) composed of multiple adjectives, nouns, or sentences (statements/items) within testlets for measuring individuals' attitudes, opinions, or career interests. While testlet-based VASs have many advantages over Likert scales, such as reducing response style effects, the development of proper statistical models for analyzing testlet-based VAS data lags behind. This paper proposes a novel beta copula model and a competing logit-normal model based on the item response theory framework, assessed by Bayesian parameter estimation, model comparison, and goodness-of-fit statistics. An empirical career interest dataset based on a testlet-based VAS design was analyzed using the proposed models. Simulation studies were conducted to assess the two models' parameter recovery. The results show that the beta copula model had superior fit in the empirical data analysis, and also exhibited good parameter recovery in the simulation studies, suggesting that it is a promising statistical approach to testlet-based doubly bounded responses.

The development and evaluation of the worker-occupation fit inventory

BACKGROUND

Person-environment fit (PEF) theory, one of the foundational theories of occupational stress, has primarily found applications in organizational behavior and human resource management. Given the alignment between the definition of occupational stress and the essence of PEF, we introduced the concept of worker-occupation fit (WOF). To validate our theoretical model, the development of an instrument to measure WOF becomes imperative.

METHODS

The Worker-Occupation Fit Inventory (WOFI) comprises three dimensions: personal trait fit (PTF), need-supply fit (NSF) and demand-ability fit (DAF). Job-related mental disorders (JRMDs) were assessed using the DASS-21. During the pre-investigation, items of the WOFI underwent screening through classic test theory (CTT) analysis. In the formal investigation, item response theory (IRT) analysis was employed to evaluate the selected items. The relationship between WOF and JRMD was verified by Pearson's correlation analysis and multiple logistic regression analysis.

RESULTS

The initial version consisted of 26 items. Three common factors were extracted by exploratory factor analysis (EFA): 6 items were included in the PTF, 6 items were included in the NSF, 4 items were included in the DAF, and 10 items were deleted because of unacceptable factor loadings. The confirmatory factor analysis (CFA) verified the structure of the WOFI with χ2/df = 1.822, CFI = 0.947, and SRMSR = 0.056. The Cronbach's alpha coefficients of the PTF, NSF, and DAF were 0.91, 0.92, and 0.80, respectively. In IRT analysis, the discrimination values of all items ranged from 1.25 to 2.53, and the difficulty values of all items ranged from -6.28 to 1.30 (with no difficulty of reversal). The WOF was negatively related to job-related stress (r = -0.34, p<0.001), anxiety (r = -0.37, p<0.001), and depression (r = -0.41, p<0.001). The multiple logistic regression analysis suggested that a high level of WOF was a protective factor against job-related mental disorders, with ORs all less than 1 (p<0.001), and a low level of WOF was a risk factor for job-related mental disorders, with ORs all more than 1.0 (p<0.001).

CONCLUSIONS

The results of CTT and IRT analysis indicated that the WOFI exhibits reliability and validation. The WOF effectively predicted job-related mental disorders. Subsequent studies will delve into the influence of WOFI on diverse professions and various health outcomes.

Human ratings take time: A hierarchical facets model for the joint analysis of ratings and rating times

Performance assessments increasingly utilize onscreen or internet-based technology to collect human ratings. One of the benefits of onscreen ratings is the automatic recording of rating times along with the ratings. Considering rating times as an additional data source can provide a more detailed picture of the rating process and improve the psychometric quality of the assessment outcomes. However, currently available models for analyzing performance assessments do not incorporate rating times. The present research aims to fill this gap and advance a joint modeling approach, the "hierarchical facets model for ratings and rating times" (HFM-RT). The model includes two examinee parameters (ability and time intensity) and three rater parameters (severity, centrality, and speed). The HFM-RT successfully recovered examinee and rater parameters in a simulation study and yielded superior reliability indices. A real-data analysis of English essay ratings collected in a high-stakes assessment context revealed that raters exhibited considerably different speed measures, spent more time on high-quality than low-quality essays, and tended to rate essays faster with increasing severity. However, due to the significant heterogeneity of examinees' writing proficiency, the improvement in the assessment's reliability using the HFM-RT was not salient in the real-data example. This discussion focuses on the advantages of accounting for rating times as a source of information in rating quality studies and highlights perspectives from the HFM-RT for future research on rater cognition.

Item Banks for Measuring the Effect of Blood Pressure Dysregulation on Health-Related Quality of Life in Persons With Spinal Cord Injury

OBJECTIVE

To report on the development and calibration of the new Blood Pressure Dysregulation Measurement System (BPD-MS) item banks that assess the effect of BPD on health-related quality of life (HRQOL) and the daily activities of Veterans and non-Veterans with spinal cord injury (SCI).

DESIGN

Cross-sectional survey study.

SETTING

Two Veteran Affairs medical centers and a SCI model system site.

PARTICIPANTS

454 respondents with SCI (n=262 American Veterans and n=192 non-Veterans; N=454).

INTERVENTIONS

Not applicable MAIN OUTCOME MEASURES: The BPD-MS item banks.

RESULTS

BPD item pools were developed and refined using literature reviews, qualitative data from focus groups, and cognitive debriefing of persons with SCI and professional caregivers. The item banks then underwent expert review, reading level assessment, and translatability review prior to field testing. The items pools consisted of 180 unique questions (items). Exploratory and confirmatory factor analyses, item response theory modeling, and differential item function investigations resulted in item banks that included a total of 150 items: 75 describing the effect of autonomic dysreflexia on HRQOL, 55 describing the effect of low blood pressure (LBP) on HRQOL, and 20 describing the effect of LBP on daily activities. In addition, 10-item short forms were constructed based on item response theory-derived item information values and the clinical relevance of item content.

CONCLUSIONS

The new BPD-MS item banks and corresponding 10-item short forms were developed using established rigorous measurement development standards, which represents the first BPD-specific patient-reported outcomes measurement system unique for use in the SCI population.

Psychometric sensitivity analyses can identify bias related to measurement properties in trials that use patient-reported outcome measures: a secondary analysis of a clinical trial using the disabilities of the arm, shoulder, and hand questionnaire

OBJECTIVES

Demonstrate psychometric sensitivity analyses for testing the stability of study findings to assumptions made about patient-reported outcome measures.

STUDY DESIGN AND SETTING

We performed secondary analyses of Disability of Arm, Shoulder, and Hand (DASH) data collected within the Prevention of Shoulder Problems clinical trial, which compared upper limb function scores in women who had undergone breast cancer surgery, randomized to either an exercise program or usual care. We repeated the principal trial analyses after grouping DASH items into subscales suggested by factorial analyses in this dataset and applied item response theory to account for unequal item weighting. We checked for measurement invariance by participant age and response shift bias using established techniques.

RESULTS

Our analyses suggested that the DASH measured two constructs: motor function and sensory symptoms. The majority of the six-month difference in DASH score was driven by motor function. With item response theory scoring, we found differences in both constructs at 12 months (P = 0.019 and P = 0.007), but in neither construct at 6 months, contrary to the original trial results. We found no differential item function by age or between baseline and 12-month measurements.

CONCLUSIONS

Psychometric sensitivity analyses aid in the interpretation of the Prevention of Shoulder Problems trial's results.

A 13-item Health of the Nation Outcome Scale (HoNOS-13): validation by item response theory (IRT) in patients with substance use disorder

BACKGROUND

The Health of the Nation Outcome Scale (HoNOS) is a widely used 12-item tool to assess mental health and social functioning. The French version has an added 13th item measuring adherence to psychotropic medication. The aim of the current study is to uncover the unknown pattern of the new item 13 and to compare the unidimensional and multidimensional fit of the new HoNOS-13 using Item Response Theory (IRT). This research question was studied among inpatients with substance use disorder (SUD).

METHODS

Six hundred and nine valid questionnaires of HoNOS-13 were analyzed using unidimensional (one-factor) and multidimensional (two-factor) IRT modeling.

RESULTS

The multidimensional model suggesting a first factor capturing psychiatric/impairment-related issues and a second factor reflecting social-related issues yielded better goodness-of-fit values compared to the unidimensional solution. This resulted in an improvement of all slope parameters which in turn translates to better discriminative power. Significant improvement in item location parameters were observed as well. The new item 13 had a good discriminative power (1.17) and covered a wide range of the latent trait (- 0.14 to 2.64).

CONCLUSIONS

We were able to validate the 13-item questionnaire including medication compliance and suggest that the HoNOS-13 can be recommended as a clinical evaluation tool to assess the problems and treatment needs for inpatients with SUD. Interestingly, the majority of item response categories are endorsed by respondents who are below and above the average levels of HoNOS. This indicates that the scale is able to discriminate between participants both at the low and at the high ends of the latent trait continuum. More importantly, the new item 13 has a good discriminative power and covers a broad range of the latent trait below and above the mean. It therefore has the desired profile of a good item and is a useful measure for the assessment of mental health and social functioning. Trial registration ClinicalTrials.gov, Identifier: NCT03551301. Registered: 11.06.2018. Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03551301 .

Comparison between pystan and numpyro in Bayesian item response theory: evaluation of agreement of estimated latent parameters and sampling performance

Purpose

The purpose of this study is to compare two libraries dedicated to the Markov chain Monte Carlo method: pystan and numpyro. In the comparison, we mainly focused on the agreement of estimated latent parameters and the performance of sampling using the Markov chain Monte Carlo method in Bayesian item response theory (IRT).

Materials and methods

Bayesian 1PL-IRT and 2PL-IRT were implemented with pystan and numpyro. Then, the Bayesian 1PL-IRT and 2PL-IRT were applied to two types of medical data obtained from a published article. The same prior distributions of latent parameters were used in both pystan and numpyro. Estimation results of latent parameters of 1PL-IRT and 2PL-IRT were compared between pystan and numpyro. Additionally, the computational cost of the Markov chain Monte Carlo method was compared between the two libraries. To evaluate the computational cost of IRT models, simulation data were generated from the medical data and numpyro.

Results

For all the combinations of IRT types (1PL-IRT or 2PL-IRT) and medical data types, the mean and standard deviation of the estimated latent parameters were in good agreement between pystan and numpyro. In most cases, the sampling time using the Markov chain Monte Carlo method was shorter in numpyro than that in pystan. When the large-sized simulation data were used, numpyro with a graphics processing unit was useful for reducing the sampling time.

Conclusion

Numpyro and pystan were useful for applying the Bayesian 1PL-IRT and 2PL-IRT. Our results show that the two libraries yielded similar estimation result and that regarding to sampling time, the fastest libraries differed based on the dataset size.