THE PENALTY FOR NOT GUESSING1

Scoring Multiple True/False Tests

A chievement tests in the health professions that utilize the multiple true-false (MTF) format apply variousformula scoring techniques to minimize the probability of inflated scores that result from guessing and the accompans'ing side effects of reduced score variability and attenuated discrimination between examinees. Despite the 50%i probability of a correctly guessed response, this article suggests that an MTF examination should provide sufficient score variabilit i'for adequate discrimination without formula scoring. Furthermore, since formula scoring may introduce factors that lower test validity, the scoring system advocated here is to direct examinees to respond to each item, with their scores based simply on the number of correct responses.

Fitting logistic IRT models: small wonder

State-of-the-art item response theory (IRT) models use logistic functions exclusively as their item response functions (IRFs). Logistic functions meet the requirements that their range is the unit interval and that they are monotonically increasing, but they impose a parameter space whose dimensions can only be assigned a metaphorical interpretation in the context of testing. Applications of IRT models require obtaining the set of values for logistic function parameters that best fit an empirical data set. However, success in obtaining such set of values does not guarantee that the constructs they represent actually exist, for the adequacy of a model is not sustained by the possibility of estimating parameters. This article illustrates how mechanical adoption of off-the-shelf logistic functions as IRFs for IRT models can result in off-the-shelf parameter estimates and fits to data. The results of a simulation study are presented, which show that logistic IRT models can fit a set of data generated by IRFs other than logistic functions just as well as they fit logistic data, even though the response processes and parameter spaces involved in each case are substantially different. An explanation of why logistic functions work as they do is offered, the theoretical and practical consequences of their behavior are discussed, and a testable alternative to logistic IRFs is commented upon.