Testing the validity of a self-report scale, author recognition test, and book counting as measures of lifetime exposure to print fiction

Some students are more equal: Performance in Author Recognition Test and Title Recognition Test modulated by print exposure and academic background

Reading is a key skill for university students. The Author Recognition Test (ART) and the Title Recognition Test (TRT) have both been used for decades to measure print exposure which correlates with reading and other linguistic skills. Given the available evidence for interindividual differences in reading skills, this study addresses three open issues. First, to what extent do ART and TRT scores correlate with individual differences regarding students' study programs? Second, how do these results correlate with the self-reported time spent reading different types of text genres (e.g., fiction, nonfiction) per week? And third, this study compares ART and TRT to one another. We tested students from six study programs in the humanities and (medical) sciences which vary in the amount and kind of reading material required for study success. We found that students perform significantly differently in the ART and the TRT depending on their field of study. Students in a study program focusing on fiction and literature perform best overall. We also replicated the well-known effect of age on ART and TRT scores: older students have better scores. We did not find reliable effects of reading time on test performance, whereas individual creative writing habits did positively predict ART/TRT test results. These results raise a number of important questions regarding the ART/TRT in general and regarding interindividual differences in personal reading and writing habits and the change in reading habits in times of media convergence.

Unipolar IRT and the Author Recognition Test (ART)

Item response theory (IRT) analyses are often used to evaluate measurement error in educational and psychological test instruments. In such contexts, the latent traits/proficiencies are typically assumed normally distributed and a cumulative normal/logistic measurement link function is applied. Such choices are consistent with constructs that are viewed as bipolar in nature and play a critical role in defining the latent proficiency metric against which the measurement error in the test is evaluated. Recently, alternative models that portray the construct as unipolar have been highlighted as being more appropriate for certain psychopathology and personality constructs. In this paper we extend consideration of unipolar IRT models for a recognition task measure, using several example datasets from various versions of the Author Recognition Test (ART), a measure of print exposure. We show how the decision between unipolar versus bipolar IRT modeling has substantial implications for the quantification and interpretation of measurement error in the ART. In sharp contrast to prior bipolar IRT analyses of the ART, under unipolar IRT measurement error in the ART is minimized at low levels of latent print exposure, and increases as latent print exposure increases. Implications for consideration of unipolar IRT with other constructs and measures (e.g., vocabulary, specialized forms of knowledge) that reflect a similar type of response process are considered in the discussion.

Using a willingness to wait design to assess how readers value text

What affects moment-to-moment motivation to read? Existing reading motivation questionnaires are trait-based and not well suited to capturing the dynamic, situational influences of text or social context. Drawing on the decision science literature, we have created a paradigm to measure situational enjoyment during reading. Using this paradigm, we find reading enjoyment is associated with further decision-making about the text and with reading comprehension.

Measuring reading behavior in large-scale assessments and surveys

Large amounts of studies have shown that reading behavior is an important aspect for the development of students' reading skills. The construct reading behavior as examined in large-scale assessments and surveys within the field of empirical educational research is operationalized through a wide range of reading and reading-related aspects (e.g., reading time, reading frequency, print exposure, reading engagement, book genre preferences, knowledge of authors or book titles) and a broad array of measurement methods are used. The approaches to measure the same variable - namely reading behavior - differ fundamentally from each other, while at the same time, a clear concept that would help to classify the used measurement instruments and to interpret them in relation to the superordinate construct of reading behavior is missing. Therefore, the present article aims to give an overview of methods to measure reading behavior within the context of large-scale assessments and surveys, and to discuss how they were implemented. Finally, we make some suggestions on how it might be possible to relate the applied measurement approaches to each other, especially in terms of their content and theoretical relationship to the overarching construct of reading behavior.

Validated tests for language research with university students whose native language is English: Tests of vocabulary, general knowledge, author recognition, and reading comprehension

We present five studies aimed at developing an L1 vocabulary test for English-speaking university students. Such a test is useful as an indicator of crystallized intelligence and because vocabulary size correlates well with reading comprehension. In the first study, we tested 100 written words with four answer alternatives, based on Nation's Vocabulary Size Test. Analysis suggested two factors, which we interpreted as the possible existence of two types of difficult words: unknown words for general knowledge and unknown words for specialized knowledge. In Study 2, we attempted to develop a vocabulary test for each type of word, and these tests were then validated in Study 3. Since the test for general words proved too easy for the target population, we improved it in a fourth study by creating and testing more difficult items. Finally, a fifth study was conducted to validate the new test. Unexpectedly, Study 5 found a high correlation (r = .82) between the general knowledge vocabulary test and the specialized knowledge vocabulary test, suggesting that they measure the same latent factor, contrary to our initial assumption. Both tests have high reliability (r > .85) and correlate well (r > .4) with general knowledge, author recognition, and reading comprehension. In addition, a collection of other language tests was used and improved to verify the validity of the vocabulary tests. An exploratory factor analysis of all tests identified three factors (text comprehension, crystallized intelligence, and reading speed), with the vocabulary tests loading on the factor crystallized intelligence, which in turn correlates with reading comprehension. Structural equation modeling confirmed the interpretation.