On Conducting Construct Validity Meta-Analyses for the Rorschach: A Reply to Tibon Czopp and Zeligman (2016)

Why We Need a Formal Systematic Approach to Validating Psychological Tests: The Case of the Rorschach Comprehensive System

This article documents and discusses the importance of using a formal systematic approach to validating psychological tests. To illustrate, results are presented from a systematic review of the validity findings cited in the Rorschach Comprehensive System (CS; Exner, 2003) test manual, originally conducted during the manuscript review process for Mihura, Meyer, Dumitrascu, and Bombel's (2013) CS meta-analyses. Our review documents (a) the degree to which the CS test manual reports validity findings for each test variable, (b) whether these findings are publicly accessible or unpublished studies coordinated by the test developer, and (c) the presence and nature of data discrepancies between the CS test manual and the cited source. Implications are discussed for the CS in particular, the Rorschach more generally, and psychological tests more broadly. Notably, a history of intensive scrutiny of the Rorschach has resulted in more stringent standards applied to it, even though its scales have more published and supportive construct validity meta-analyses than any other psychological test. Calls are made for (a) a mechanism to correct data errors in the scientific literature, (b) guidelines for test developers' key unpublished studies, and (c) systematic reviews and meta-analyses to become standard practice for all psychological tests.

A Survey of Challenges Experienced by New Learners Coding the Rorschach

Learning to code the imagery, communication, and behavior associated with Rorschach responding is challenging. Although there is some survey research on graduate students' impressions of their Rorschach training, research has not identified which coding decisions students find to be the most problematic and time-consuming. We surveyed students to identify what they struggled with most when learning coding and to quantify how difficult it is to learn how to code. Participants (n = 191) from the United States, Brazil, Denmark, Israel, and Italy rated 57 aspects of coding using a 4-point scale that encompassed both the time required to code and the subjective difficulty of doing so. Mean ratings for coding in general indicated that students considered the overall task challenging. Ratings also revealed that students struggled most with Cognitive Special Scores, Determinants, and extrapolating from the tables to code Form Quality for objects that were not specifically listed. The findings offer suggestions about how to improve the guidelines for some of the more difficult variables and where it is most necessary to focus teaching time. Taking these steps might help new students in learning the Rorschach.