A Short Note on Estimating the Testlet Model With Different Estimators in Mplus

Construct validity of questionnaires for the original and revised reinforcement sensitivity theory

This study highlights psychometric properties and evidence of construct validity on parcel-level for questionnaires on the original and revised reinforcement sensitivity theory. Our data (N = 1,076) suggest good to very good psychometric properties and moderate to excellent internal consistencies. Confirmatory factor analysis (CFA) models suggest a very good model fit for the first-order, four factor models of the Carver-White BIS/BAS scales, Reinforcement Sensitivity Theory - Personality Questionnaire (RST-PQ), the two-factor model of revised Reinforcement Sensitivity Theory-Questionnaire (rRST-Q) and for the bifactor model of the Conflict Monitoring Questionnaire (CMQ-44). The CMQ-44 extends the psychometric measurement of previous trait-(r)BIS and trait-BAS scales. Factor scores of CMQ-44 cognitive demand correlate positively with factor scores of Carver-White BIS and all Carver-White BAS subfactors except RST-PQ-Impulsivity suggesting that CMQ-44 cognitive demand addresses Carver-White trait-BIS specifically and more generally the trait-BAS core. CMQ-44 anticipation of negative consequences and response adaptation correlate negatively with trait-BAS, whereas the second-order factor performance monitoring extends the rRST trait-space and correlates positively with trait-BAS.

A Motivational-Developmental Free Response Assessment Through a Bifactor Lens

The Motivational-Developmental Assessment (MDA) measures a university student’s motivational and developmental attributes by utilizing overlapping constructs measured across four writing prompts. The MDA’s format may lead to the violation of the local item independence (LII) assumption for unidimensional item response theory (IRT) scoring models, or the uncorrelated errors assumption for scoring models in classical test theory (CTT) due to the measurement of overlapping constructs within a prompt. This assumption violation is known as a testlet effect, which can be viewed as a method effect. The application of a unidimensional IRT or CTT model to score the MDA can result in imprecise parameter estimates when this effect is ignored. To control for this effect in the MDA responses, we first examined the presence of local dependence via a restricted bifactor model and Yen’s Q3 statistic. Second, we applied bifactor models to account for the testlet effect in the responses, as this effect is modeled as an additional latent variable in a factor model. Results support the presence of local dependence in two of the four MDA prompts, and the use of the restricted bifactor model to account for the testlet effect in the responses. Modeling the testlet effect through the restricted bifactor model supports a scoring inference in a validation argument framework. Implications are discussed.

A Short Note on Obtaining Point Estimates of the IRT Ability Parameter With MCMC Estimation in Mplus: How Many Plausible Values Are Needed?

Plausible values can be used to either estimate population-level statistics or compute point estimates of latent variables. While it is well known that five plausible values are usually sufficient for accurate estimation of population-level statistics in large-scale surveys, the minimum number of plausible values needed to obtain accurate latent variable point estimates is unclear. This is especially relevant when an item response theory (IRT) model is estimated with MCMC (Markov chain Monte Carlo) methods in Mplus and point estimates of the IRT ability parameter are of interest, as Mplus only estimates the posterior distribution of each ability parameter. In order to obtain point estimates of the ability parameter, a number of plausible values can be drawn from the posterior distribution of each individual ability parameter and their mean (the posterior mean ability estimate) can be used as an individual ability point estimate. In this note, we conducted a simulation study to investigate how many plausible values were needed to obtain accurate posterior mean ability estimates. The results indicate that 20 is the minimum number of plausible values required to obtain point estimates of the IRT ability parameter that are comparable to marginal maximum likelihood estimation(MMLE)/expected a posteriori (EAP) estimates. A real dataset was used to demonstrate the comparison between MMLE/EAP point estimates and posterior mean ability estimates based on different number of plausible values.