Scale Type Revisited: Some Misconceptions, Misinterpretations, and Recommendations

Stevens’s classification of scales into nominal, ordinal, interval, and ratio types is among the most controversial yet resilient ideas in psychological and educational measurement. In this essay, I challenge the notion that scale type is essential for the development of measures in these fields. I highlight how the concept of scale type, and of interval-level measurement in particular, is variously interpreted by many researchers. These (often unstated) differences in perspectives lead to confusion about what evidence is appropriate to demonstrate interval-level measurement, as well as the implications of scale type for research in practice. I then borrow from contemporary ideas in the philosophy of measurement to demonstrate that scale type can only be established in the context of well-developed theory and through experimentation. I conclude that current notions of scale type are of limited use, and that scale type ought to occupy a lesser role in psychometric discourse and pedagogy.

Rating scales institutionalise a network of logical errors and conceptual problems in research practices: A rigorous analysis showing ways to tackle psychology's crises

This article explores in-depth the metatheoretical and methodological foundations on which rating scales-by their very conception, design and application-are built and traces their historical origins. It brings together independent lines of critique from different scholars and disciplines to map out the problem landscape, which centres on the failed distinction between psychology's study phenomena (e.g., experiences, everyday constructs) and the means of their exploration (e.g., terms, data, scientific constructs)-psychologists' cardinal error. Rigorous analyses reveal a dense network of 12 complexes of problematic concepts, misconceived assumptions and fallacies that support each other, making it difficult to be identified and recognised by those (unwittingly) relying on them (e.g., various forms of reductionism, logical errors of operationalism, constructification, naïve use of language, quantificationism, statisticism, result-based data generation, misconceived nomotheticism). Through the popularity of rating scales for efficient quantitative data generation, uncritically interpreted as psychological measurement, these problems have become institutionalised in a wide range of research practices and perpetuate psychology's crises (e.g., replication, confidence, validation, generalizability). The article provides an in-depth understanding that is needed to get to the root of these problems, which preclude not just measurement but also the scientific exploration of psychology's study phenomena and thus its development as a science. From each of the 12 problem complexes; specific theoretical concepts, methodologies and methods are derived as well as key directions of development. The analyses-based on three central axioms for transdisciplinary research on individuals, (1) complexity, (2) complementarity and (3) anthropogenicity-highlight that psychologists must (further) develop an explicit metatheory and unambiguous terminology as well as concepts and theories that conceive individuals as living beings, open self-organising systems with complementary phenomena and dynamic interrelations across their multi-layered systemic contexts-thus, theories not simply of elemental properties and structures but of processes, relations, dynamicity, subjectivity, emergence, catalysis and transformation. Philosophical and theoretical foundations of approaches suited for exploring these phenomena must be developed together with methods of data generation and methods of data analysis that are appropriately adapted to the peculiarities of psychologists' study phenomena (e.g., intra-individual variation, momentariness, contextuality). Psychology can profit greatly from its unique position at the intersection of many other disciplines and can learn from their advancements to develop research practices that are suited to tackle its crises holistically.

Is measurement in psychology an empirical or a conceptual issue? A comment on David Franz

David Franz’s (2022) claim that it is not possible to give an empirically meaningful answer to the question “Are psychological attributes quantitative?” is rejected as incorrect. First, the ambiguous terminology identified by Franz in the measurement debate is not inherent to the debate itself, but reflects a biased understanding of the empirical meaning of axioms of measurement. Second, the problem of “hidden” or “unobservable” phenomena described by Franz as the root of conceptual confusion is not unique to psychology, but can also be found in physics. Therefore, it cannot be the cause for the failure to establish measurement in psychology. This comment concludes with the statement that, though both conceptual and empirical analyses are useful, Franz’s general claim that the former supersedes the latter must be rejected as inaccurate.

“Are psychological attributes quantitative?” is not an empirical question: Conceptual confusions in the measurement debate

Critics of psychological measurement have accused quantitative psychologists of ignoring the empirical hypothesis that psychological phenomena are quantitative (Michell), or have claimed that it is impossible in principle to find out whether psychological phenomena are actually quantitative (Trendler). By drawing on Bennett and Hacker (2003), I argue that both criticisms do not go far enough because they sidestep the fundamental conceptual problem of the measurement debate: It is impossible to give concrete formulations of the question “Are psychological attributes quantitative?” without transgressing the boundaries of meaningful language. Conceptual confusions and questionable philosophical assumptions have contributed to the misguided idea that the quantity of psychological phenomena must or can be demonstrated empirically. First, the measurement debate is characterized by misleading examples and ambiguous terminology. Second, the idea of psychological measurement is inherently Cartesian. In summary, psychological measurement is even more problematic than Michell and Trendler have argued.

Should We Say Goodbye to Latent Constructs to Overcome Replication Crisis or Should We Take Into Account Epistemological Considerations?

This paper discusses theoretical and epistemological problems concerning validity of psychological science in the context of latent constructs. I consider the use of latent constructs as one reason for the replicability crisis. At the moment, there exist different constructs describing the same psychological phenomena side by side, and different psychological phenomena that are reflected by the same latent construct. Hagger called them déjà-variables, which lead to a decreasing validity of measurements and inhibit a deeper understanding of psychological phenomena. To overcome this problem, I suggest a shift of theoretical and epistemological perspective on latent constructs. One main point is the explicit consideration of latent constructs as mental representations, which change objects and are changed by objects via assimilative and accommodative processes. The explicit orientation toward assimilation and accommodation allows the control of normally automatized processes that influence our understanding of psychological phenomena and their corresponding latent constructs. I argue that assimilation and accommodation are part of our research practice anyway and cause the mentioned problems. For example, taking a measurement is an assimilative process, and thus a high measurement error should lead to an increase of accommodative processes. Taking into account these considerations, I suggest consequences for research practices, for individual researchers and for the philosophy of science.

Measurability, systematic error, and the replication crisis: A reply to Michell (2019) and Krantz and Wallsten (2019)

Although Krantz and Wallsten (2019) claim that interval and ratio scales abound in psychology, they miss the opportunity to deliver specific evidence for their existence. Michell (2019), on the other hand, misconstrues my objection against the practical usefulness of conjoint measurement (Trendler, 2019). Furthermore, he underestimates the critical role humans play as measurement instruments—that is, as detectors of magnitudes of psychological attributes as derived quantities—and he also misunderstands the meaning of the Millean Quantity Objection. Finally, in answer to Krantz and Wallsten, I specify my position with regard to the connection between scientific stagnation, measurability, and reproducibility.

Comment on Trendler’s (2019) “Conjoint measurement undone”

Most of Trendler’s (2019) article, “Conjoint measurement undone,” seems wrong to us. We explain why we disagree completely with two of his assertions: (a) that cardinal measurement scales are absent in psychology and (b) that psychology has stagnated. We share three of his other concerns, but not his perspectives on them or the supposed links among them. These three points are: (a) fewer applications of additive conjoint measurement than initially expected, (b) flaws in the practice of statistics, and (c) need to improve the culture of replication in psychology. We provide our views on these points and also note two distinct strands in the foundational analysis of measurement—one derived from geometry, the other from probability. Trendler completely overlooked the latter.

Conjoint measurement underdone: Comment on Günter Trendler (2019)

Trendler’s (2019) critique of conjoint measurement fails because he neglects to distinguish standard sequences (human constructions) from series of equal magnitudes (features of quantitative structures). The latter, not the former, is presumed in conjoint measurement. Furthermore, in so far as some mental tests use humans as measuring instruments, the only questionable assumption involved is that the relevant psychological attributes are quantitative, and that assumption is potentially testable using conjoint measurement. Finally, contrary to Trendler, psychological phenomena can be captured and the structure of psychological attributes investigated using conjoint measurement.

Quantitative Data From Rating Scales: An Epistemological and Methodological Enquiry

Rating scales are popular methods for generating quantitative data directly by persons rather than automated technologies. But scholars increasingly challenge their foundations. This article contributes epistemological and methodological analyses of the processes involved in person-generated quantification. They are crucial for measurement because data analyses can reveal information about study phenomena only if relevant properties were encoded systematically in the data. The Transdisciplinary Philosophy-of-Science Paradigm for Research on Individuals (TPS-Paradigm) is applied to explore psychological and social-science concepts of measurement and quantification, including representational measurement theory, psychometric theories and their precursors in psychophysics. These are compared to theories from metrology specifying object-dependence of measurement processes and subject-independence of outcomes as key criteria, which allow tracing data to the instances measured and the ways they were quantified. Separate histories notwithstanding, the article's basic premise is that general principles of scientific measurement and quantification should apply to all sciences. It elaborates principles by which these metrological criteria can be implemented also in psychology and social sciences, while considering their research objects' peculiarities. Application of these principles is illustrated by quantifications of individual-specific behaviors ('personality'). The demands rating methods impose on data-generating persons are deconstructed and compared with the demands involved in other quantitative methods (e.g., ethological observations). These analyses highlight problematic requirements for raters. Rating methods sufficiently specify neither the empirical study phenomena nor the symbolic systems used as data nor rules of assignment between them. Instead, pronounced individual differences in raters' interpretation and use of items and scales indicate considerable subjectivity in data generation. Together with recoding scale categories into numbers, this introduces a twofold break in the traceability of rating data, compromising interpretability of findings. These insights question common reliability and validity concepts for ratings and provide novel explanations for replicability problems. Specifically, rating methods standardize only data formats but not the actual data generation. Measurement requires data generation processes to be adapted to the study phenomena's properties and the measurement-executing persons' abilities and interpretations, rather than to numerical outcome formats facilitating statistical analyses. Researchers must finally investigate how people actually generate ratings to specify the representational systems underlying rating data.