A 25-year replication of Katz et al.’s (1988) metaphor norms

The RK processor: A program for analysing metaphor and word feature-listing data

Feature-listing tasks are an invaluable resource for exploring how words, categories, and metaphors are represented. However, manually coding the generated features is time-consuming and expensive, and involves subjective judgments from the experimenter. The purpose of this paper is to introduce the "RK processor", a program that was developed in our lab to analyse metaphor feature data but which can also be applied to other feature-listing data. After detailing the steps of processing, we demonstrate that the processed feature data align with previous findings in which metaphor features were processed manually and that the processed features predict dimensions of metaphor judgments pertaining to comprehensibility and metaphor goodness. Lastly, we present several other applications for research on word similarity, compound words, categories and concepts, semantic ambiguity, incongruity resolution and computational modelling. The RK processor offers researchers a valuable tool to save time and resources and to maintain consistency in processing.

Evaluating the predication model of metaphor comprehension: Using word2vec to model best/worst quality judgments of 622 novel metaphors

In this paper our goal is to undertake a systematic assessment of the first, most widely known, and simplest computational model of metaphor comprehension, the predication model developed by Kintsch (Cognitive Science, 25(2), 173-202, 2000). 622 metaphors of the form "x is a y" were selected from a much larger set generated randomly. The metaphors were judged for quality using best/worst judgments, which asks judges to pick the best and worst metaphor from among four presented metaphors. The metaphors and their judgments have been publicly released. We modeled the judgments by extending Kintsch's predication model (2000) by systematically walking through the parameter space of that model. Our model successfully differentiated metaphors rated as good (> 1.5z) from metaphors rated as bad (< -1.5z; Cohen's d = 0.72) and was able to successfully classify good metaphors with an accuracy of 82.9%. However, it achieved a true negative rate below chance at 36.3% and had a resultantly low kappa of 0.037. The model could not distinguish unselected random metaphors from those selected by humans as having metaphorical potential. In a follow-up study we showed that the model's quality estimates reliably predict metaphor decision times, with better metaphors being judged more quickly than worse metaphors.

Time to Throw in the Towel? No Evidence for Automatic Conceptual Metaphor Access in Idiom Processing

The goal of the current research was to determine if conceptual metaphors are activated when people read idioms within a text. Participants read passages that included idioms that were consistent (blow your top) or inconsistent (bite his head off) with an underlying conceptual metaphor (ANGER IS HEATED FLUID IN A CONTAINER) followed by target words that were related (heat) or unrelated (lead) to the conceptual metaphor. Reading time (Experiment 1) or lexical decision time (Experiment 2) for the target words were measured. We found no evidence supporting conceptual metaphor activation. Target word reading times were unaffected by whether they were related or unrelated to underlying conceptual metaphors. Lexical decision times were facilitated for related target words in both the consistent and inconsistent idiom conditions. We suggest that the conceptual (target) domain, not a specific underlying conceptual metaphor, facilitates processing of related target words.

Normative Data for Novel Nominal Metaphors, Novel Similes, Literal, and Anomalous Utterances in Polish and English

The two studies reported in the article provide normative measures for 120 novel nominal metaphors, 120 novel similes, 120 literal sentences, and 120 anomalous utterances in Polish (Study 1) and in English (Study 2). The presented set is ideally suited to addressing methodological requirements in research on metaphor processing. The critical (sentence-final) words of each utterance were controlled for in terms of their frequency per million, number of letters and syllables. For each condition in each language, the following variables are reported: cloze probability, meaningfulness, metaphoricity, and familiarity, whose results confirm that the sentences are well-matched. Consequently, the present paper provides materials that can be employed in order to test the new as well as existing theories of metaphor comprehension. The results obtained from the series of normative tests showed the same pattern in both studies, where the comparison structure present in similes (i.e., A is like B) facilitated novel metaphor comprehension, as compared to categorical statements (i.e., A is B). It therefore indicates that comparison mechanisms might be engaged in novel meaning construction irrespectively of language-specific syntactic rules.

Examining the Utility of the New Raney Vocabulary Measure Alongside the WAIS-III

Psychometric tests related to vocabulary assessments are, for the most part, restricted in their use by trained professionals and/or are costly. These restrictions limit their use, especially for research purposes. To circumvent these limitations, the Raney Vocabulary Measure was created for assessing vocabulary proficiency, specifically for research purposes. The measure consists of 30 questions where participants were instructed to choose the best definition of each word. The purpose of the study was to examine the utility of the new measure using the highly standardized but protected Wechsler Adult Intelligence Scale. Results from the linear combination of the subscales revealed the significant prediction of the Raney Vocabulary Measure, with the Vocabulary subtest contributing most to the unique variance. These results support that the test examines vocabulary ability. The current results are promising as the test would allow for greater accessibility for researchers who do not have access to restricted psychometric tests.

"Going to town": Large-scale norming and statistical analysis of 870 American English idioms

An idiom is classically defined as a formulaic sequence whose meaning is comprised of more than the sum of its parts. For this reason, idioms pose a unique problem for models of sentence processing, as researchers must take into account how idioms vary and along what dimensions, as these factors can modulate the ease with which an idiomatic interpretation can be activated. In order to help ensure external validity and comparability across studies, idiom research benefits from the availability of publicly available resources reporting ratings from a large number of native speakers. Resources such as the one outlined in the current paper facilitate opportunities for consensus across studies on idiom processing and help to further our goals as a research community. To this end, descriptive norms were obtained for 870 American English idioms from 2,100 participants along five dimensions: familiarity, meaningfulness, literal plausibility, global decomposability, and predictability. Idiom familiarity and meaningfulness strongly correlated with one another, whereas familiarity and meaningfulness were positively correlated with both global decomposability and predictability. Correlations with previous norming studies are also discussed.