Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests

Items with the presence of differential item functioning (DIF) will compromise the validity and fairness of a test. Studies have investigated the DIF effect in the context of cognitive diagnostic assessment (CDA), and some DIF detection methods have been proposed. Most of these methods are mainly designed to perform the presence of DIF between two groups; however, empirical situations may contain more than two groups. To date, only a handful of studies have detected the DIF effect with multiple groups in the CDA context. This study uses the generalized logistic regression (GLR) method to detect DIF items by using the estimated attribute profile as matching criteria. A simulation study is conducted to examine the performance of the two GLR methods, GLR-based Wald test (GLR-Wald) and GLR-based likelihood ratio test (GLR-LRT), in detecting the DIF items, the results based on the ordinary Wald test are also reported. Results show that (1) both GLR-Wald and GLR-LRT have more reasonable performance in controlling Type I error rates than the ordinary Wald test in most conditions; (2) the GLR method also produces higher empirical rejection rates than the ordinary Wald test in most conditions; and (3) using the estimated attribute profile as the matching criteria can produce similar Type I error rates and empirical rejection rates for GLR-Wald and GLR-LRT. A real data example is also analyzed to illustrate the application of these DIF detection methods in multiple groups.

A Review of the use of Propensity Score Methods with Multiple Treatment Groups in the General Internal Medicine Literature

BACKGROUND

Propensity score (PS) methods with two treatment groups (e.g., treated vs. control) is a well-established technique for reducing the effects of confounding in nonrandomized studies. However, researchers are often interested in comparing multiple interventions. PS methods have been modified to incorporate multiple exposures. We described available techniques for PS methods in multicategory exposures (≥ 3 groups) and examined their use in the medical literature.

METHODS

A comprehensive search was conducted for studies published in PubMed, Embase, Google Scholar and Web of Science until February 27, 2023. We included studies using PS methods for multiple groups in general internal medicine research.

RESULTS

The literature search yielded 4088 studies (2616 from PubMed, 86 from Embase, 85 from Google Scholar, 1671 from Web of Science, 5 from other sources). In total, 264 studies using PS method for multiple groups were identified; 61 studies were on general internal medicine topics and included. The most commonly used method was that of McCaffrey et al., which was used in 26 studies (43%), where the Toolkit for Weighting and Analysis of Nonequivalent Groups (TWANG) method and corresponding inverse probabilities of treatment weights were estimated via generalized boosted models. The next most commonly used method was pairwise propensity-matched comparisons, which was used in 20 studies (33%). The method by Imbens et al. using a generalized propensity score was implemented in 6 studies (10%). Four studies (7%) used a conditional probability of being in a particular group given a set of observed baseline covariates where a multiple propensity score was estimated using a non-parsimonious multinomial logistic regression model. Four studies (7%) used a technique that estimates generalized propensity scores and then creates 1:1:1 matched sets, and one study (2%) used the matching weight method.

CONCLUSIONS

Many propensity score methods for multiple groups have been adopted in the literature. The TWANG method is the most commonly used method in the general medical literature. This article is protected by copyright. All rights reserved.

Multiple teams, multiple projects, multiple groups at the intersection of (multiple) research fields: A bibliometric study

Multi-teaming is a concept studied across a variety of disciplines. While using a bibliometric approach on 255 research papers extracted from Web of Science, we aimed to depict the architecture of the multi-teaming concept across academic disciplines and time. Results of citation, co-citation and bibliographic coupling analyses identified four major fields looking at the concept of multi-teaming. The fields emerged over time from fragmentation to integration and acknowledging similarities. We identify gaps and propose (multi)-disciplinary research ideas that can benefit the field of multi-teaming.

Reliability coefficients for multiple group item response theory models

Reliability of scores from psychological or educational assessments provides important information regarding the precision of measurement. The reliability of scores is however population dependent and may vary across groups. In item response theory, this population dependence can be attributed to differential item functioning or to differences in the latent distributions between groups and needs to be accounted for when estimating the reliability of scores for different groups. Here, we introduce group-specific and overall reliability coefficients for sum scores and maximum likelihood ability estimates defined by a multiple group item response theory model. We derive confidence intervals using asymptotic theory and evaluate the empirical properties of estimators and the confidence intervals in a simulation study. The results show that the estimators are largely unbiased and that the confidence intervals are accurate with moderately large sample sizes. We exemplify the approach with the Montreal Cognitive Assessment (MoCA) in two groups defined by education level and give recommendations for applied work.

Multiple Group Decision Making for Selecting Emergency Alternatives: A Novel Method Based on the LDWPA Operator and LD-MABAC

When an emergency event occurs, it is critical to respond in the shortest possible time. Therefore, the rationality and effectiveness of emergency decisions are the key links in emergency management. In this paper, with aims to investigate the problem of emergency alternatives selection, in which a large number of experts from multiple groups consider the linguistic evaluations of emergency alternatives and prior orders of criteria, a novel emergency decision method is proposed. First, many experts from multiple subgroups are employed to evaluate the emergency alternatives associated with multiple criteria in the format of linguistic terms. Then, linguistic distribution evaluations for the emergency alternatives of the criteria associated with each subgroup are constructed. With respect to the linguistic distribution evaluations associated with the different subgroups, the linguistic distribution power average (LDPA) and linguistic distribution weighted power average (LDWPA) operators are developed so as to aggregate the subgroups' evaluations. Next, based on the linguistic distribution multi-attributive border approximation area comparison (LD-MABAC) method, the distance matrix of the emergency alternatives is calculated. Furthermore, the prior weights of the criteria are determined based on the distance values. Finally, the ranking result of the emergency alternatives is derived. A practical example of emergency alternatives selection is adopted to illustrate the availability and practicability of the proposed method.

A Hierarchical Multi-Unidimensional IRT Approach for Analyzing Sparse, Multi-Group Data for Integrative Data Analysis

The present paper proposes a hierarchical, multi-unidimensional two-parameter logistic item response theory (2PL-MUIRT) model extended for a large number of groups. The proposed model was motivated by a large-scale integrative data analysis (IDA) study which combined data (N = 24,336) from 24 independent alcohol intervention studies. IDA projects face unique challenges that are different from those encountered in individual studies, such as the need to establish a common scoring metric across studies and to handle missingness in the pooled data. To address these challenges, we developed a Markov chain Monte Carlo (MCMC) algorithm for a hierarchical 2PL-MUIRT model for multiple groups in which not only were the item parameters and latent traits estimated, but the means and covariance structures for multiple dimensions were also estimated across different groups. Compared to a few existing MCMC algorithms for multidimensional IRT models that constrain the item parameters to facilitate estimation of the covariance matrix, we adapted an MCMC algorithm so that we could directly estimate the correlation matrix for the anchor group without any constraints on the item parameters. The feasibility of the MCMC algorithm and the validity of the basic calibration procedure were examined using a simulation study. Results showed that model parameters could be adequately recovered, and estimated latent trait scores closely approximated true latent trait scores. The algorithm was then applied to analyze real data (69 items across 20 studies for 22,608 participants). The posterior predictive model check showed that the model fit all items well, and the correlations between the MCMC scores and original scores were overall quite high. An additional simulation study demonstrated robustness of the MCMC procedures in the context of the high proportion of missingness in data. The Bayesian hierarchical IRT model using the MCMC algorithms developed in the current study has the potential to be widely implemented for IDA studies or multi-site studies, and can be further refined to meet more complicated needs in applied research.

Sample size and power for a logrank test and Cox proportional hazards model with multiple groups and strata, or a quantitative covariate with multiple strata

I describe general expressions for the evaluation of sample size and power for the K group Mantel-logrank test or the Cox proportional hazards (PH) model score test. Under an exponential model, the method of Lachin and Foulkes for the 2 group case is extended to the K ⩾2 group case using the non-centrality parameter of the K - 1 df chi-square test. I also show similar results to apply to the K group score test in a Cox PH model. Lachin and Foulkes employed a truncated exponential distribution to provide for a non-linear rate of enrollment. I present expressions for the mean time of enrollment and the expected follow-up time in the presence of exponential losses to follow-up. When used with the expression for the noncentrality parameter for the test, equations are derived for the evaluation of sample size and power under specific designs with r years of recruitment and T years total duration. I also describe sample size and power for a stratified-adjusted K group test and for the assessment of a group by stratum interaction. Similarly, I describe computations for a stratified-adjusted analysis of a quantitative covariate and a test of a stratum by covariate interaction in the Cox PH model.