1
|
Breithaupt F, Otenen E, Wright DR, Kruschke JK, Li Y, Tan Y. Humans create more novelty than ChatGPT when asked to retell a story. Sci Rep 2024; 14:875. [PMID: 38195660 PMCID: PMC10776760 DOI: 10.1038/s41598-023-50229-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 12/17/2023] [Indexed: 01/11/2024] Open
Abstract
We compare how humans retell stories to how ChatGPT retells stories in chains of three retellings by different people or different accounts on ChatGPT. ChatGPT provides competent summaries of the original narrative texts in one step of retelling. In subsequent retellings few additional changes occur. Human retellers, by contrast, reduce the original text incrementally and by creating 55-60% of novel words and concepts (synsets) at each iteration. The retellings by both ChatGPT and humans show very stable emotion ratings, which is a puzzle for human retellers given the high degree of novel inventions across retellings. ChatGPT maintains more nouns, adjectives, and prepositions and also uses language later acquired in life, while humans use more verbs, adverbs, and negations and use language acquired at a younger age. The results reveal that spontaneous retelling by humans involves ongoing creativity, anchored by emotions, beyond the default probabilistic wording of large language models such as ChatGPT.
Collapse
Affiliation(s)
- Fritz Breithaupt
- Cognitive Science Program, Indiana University Bloomington, 1001 E 10th St, Bloomington, IN, 47405, USA.
- Germanic Studies, Indiana University Bloomington, 355 N. Eagleson, Bloomington, IN, 47405, USA.
| | - Ege Otenen
- Cognitive Science Program, Indiana University Bloomington, 1001 E 10th St, Bloomington, IN, 47405, USA
- Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, 901 E 10th St, Bloomington, IN, 47405, USA
| | - Devin R Wright
- Cognitive Science Program, Indiana University Bloomington, 1001 E 10th St, Bloomington, IN, 47405, USA
- Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, 901 E 10th St, Bloomington, IN, 47405, USA
| | - John K Kruschke
- Cognitive Science Program, Indiana University Bloomington, 1001 E 10th St, Bloomington, IN, 47405, USA
- Department of Psychological and Brain Sciences, Indiana University Bloomington, 1101 E 10th St, Bloomington, IN, 47405, USA
| | - Ying Li
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, 16 Lincui Road, 100101, Beijing, China
| | - Yiyan Tan
- Cognitive Science Program, Indiana University Bloomington, 1001 E 10th St, Bloomington, IN, 47405, USA
- Department of Psychological and Brain Sciences, Indiana University Bloomington, 1101 E 10th St, Bloomington, IN, 47405, USA
| |
Collapse
|
2
|
Li Y, Breithaupt F, Hills T, Lin Z, Chen Y, Siew CSW, Hertwig R. How cognitive selection affects language change. Proc Natl Acad Sci U S A 2024; 121:e2220898120. [PMID: 38150495 PMCID: PMC10769849 DOI: 10.1073/pnas.2220898120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Accepted: 10/12/2023] [Indexed: 12/29/2023] Open
Abstract
Like biological species, words in language must compete to survive. Previously, it has been shown that language changes in response to cognitive constraints and over time becomes more learnable. Here, we use two complementary research paradigms to demonstrate how the survival of existing word forms can be predicted by psycholinguistic properties that impact language production. In the first study, we analyzed the survival of words in the context of interpersonal communication. We analyzed data from a large-scale serial-reproduction experiment in which stories were passed down along a transmission chain over multiple participants. The results show that words that are acquired earlier in life, more concrete, more arousing, and more emotional are more likely to survive retellings. We reason that the same trend might scale up to language evolution over multiple generations of natural language users. If that is the case, the same set of psycholinguistic properties should also account for the change of word frequency in natural language corpora over historical time. That is what we found in two large historical-language corpora (Study 2): Early acquisition, concreteness, and high arousal all predict increasing word frequency over the past 200 y. However, the two studies diverge with respect to the impact of word valence and word length, which we take up in the discussion. By bridging micro-level behavioral preferences and macro-level language patterns, our investigation sheds light on the cognitive mechanisms underlying word competition.
Collapse
Affiliation(s)
- Ying Li
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing100101, China
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin14195, Germany
| | - Fritz Breithaupt
- Department of Germanic Studies, Indiana University, Bloomington, IN001809
- Program of Cognitive Science, Indiana University, Bloomington, IN001809
| | - Thomas Hills
- Department of Psychology, University of Warwick, CoventryCV4 7AL, United Kingdom
| | - Ziyong Lin
- Center for Life Span Psychology, Max Planck Institute for Human Development, Berlin14195, Germany
| | - Yanyan Chen
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing100049, China
| | - Cynthia S. W. Siew
- Department of Psychology, National University of Singapore, Singapore119077, Singapore
| | - Ralph Hertwig
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin14195, Germany
| |
Collapse
|
3
|
He T, Breithaupt F, Kübler S, Hills TT. Quantifying the retention of emotions across story retellings. Sci Rep 2023; 13:2448. [PMID: 36774370 PMCID: PMC9922315 DOI: 10.1038/s41598-023-29178-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Accepted: 01/31/2023] [Indexed: 02/13/2023] Open
Abstract
Story retelling is a fundamental medium for the transmission of information between individuals and among social groups. Besides conveying factual information, stories also contain affective information. Though natural language processing techniques have advanced considerably in recent years, the extent to which machines can be trained to identify and track emotions across retellings is unknown. This study leverages the powerful RoBERTa model, based on a transformer architecture, to derive emotion-rich story embeddings from a unique dataset of 25,728 story retellings. The initial stories were centered around five emotional events (joy, sadness, embarrassment, risk, and disgust-though the stories did not contain these emotion words) and three intensities (high, medium, and low). Our results indicate (1) that RoBERTa can identify emotions in stories it was not trained on, (2) that the five emotions and their intensities are preserved when they are transmitted in the form of retellings, (3) that the emotions in stories are increasingly well-preserved as they experience additional retellings, and (4) that among the five emotions, risk and disgust are least well-preserved, compared with joy, sadness, and embarrassment. This work is a first step toward quantifying situation-driven emotions with machines.
Collapse
Affiliation(s)
- Tianyou He
- Department of Psychology, University of Warwick, Gibbet Hill Road, Coventry, CV47AL, UK
| | - Fritz Breithaupt
- Department of Germanic Studies, Indiana University, Bloomington, USA
- Cognitive Science Program, Indiana University, Bloomington, USA
| | - Sandra Kübler
- Cognitive Science Program, Indiana University, Bloomington, USA
- Department of Psychology, University of Warwick, Gibbet Hill Road, Coventry, UK
| | - Thomas T Hills
- Department of Psychology, University of Warwick, Gibbet Hill Road, Coventry, CV47AL, UK.
| |
Collapse
|