1
|
Kawakita G, Zeleznikow-Johnston A, Tsuchiya N, Oizumi M. Gromov-Wasserstein unsupervised alignment reveals structural correspondences between the color similarity structures of humans and large language models. Sci Rep 2024; 14:15917. [PMID: 38987348 PMCID: PMC11237038 DOI: 10.1038/s41598-024-65604-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 06/21/2024] [Indexed: 07/12/2024] Open
Abstract
Large Language Models (LLMs), such as the General Pre-trained Transformer (GPT), have shown remarkable performance in various cognitive tasks. However, it remains unclear whether these models have the ability to accurately infer human perceptual representations. Previous research has addressed this question by quantifying correlations between similarity response patterns of humans and LLMs. Correlation provides a measure of similarity, but it relies pre-defined item labels and does not distinguish category- and item- level similarity, falling short of characterizing detailed structural correspondence between humans and LLMs. To assess their structural equivalence in more detail, we propose the use of an unsupervised alignment method based on Gromov-Wasserstein optimal transport (GWOT). GWOT allows for the comparison of similarity structures without relying on pre-defined label correspondences and can reveal fine-grained structural similarities and differences that may not be detected by simple correlation analysis. Using a large dataset of similarity judgments of 93 colors, we compared the color similarity structures of humans (color-neurotypical and color-atypical participants) and two GPT models (GPT-3.5 and GPT-4). Our results show that the similarity structure of color-neurotypical participants can be remarkably well aligned with that of GPT-4 and, to a lesser extent, to that of GPT-3.5. These results contribute to the methodological advancements of comparing LLMs with human perception, and highlight the potential of unsupervised alignment methods to reveal detailed structural correspondences.
Collapse
Affiliation(s)
- Genji Kawakita
- Department of Bioengineering, Imperial College London, London, UK.
| | - Ariel Zeleznikow-Johnston
- School of Psychological Sciences, Monash University, Melbourne, Australia
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia
| | - Naotsugu Tsuchiya
- School of Psychological Sciences, Monash University, Melbourne, Australia
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka, Japan
- Department of Qualia Structure, ATR Computational Neuroscience Laboratories, Kyoto, Japan
| | - Masafumi Oizumi
- Graduate School of Arts and Science, The University of Tokyo, Tokyo, Japan.
| |
Collapse
|
2
|
Zettersten M, Bredemann C, Kaul M, Ellis K, Vlach HA, Kirkorian H, Lupyan G. Nameability supports rule-based category learning in children and adults. Child Dev 2024; 95:497-514. [PMID: 37728552 PMCID: PMC10922161 DOI: 10.1111/cdev.14008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Revised: 05/07/2023] [Accepted: 07/25/2023] [Indexed: 09/21/2023]
Abstract
The present study tested the hypothesis that verbal labels support category induction by providing compact hypotheses. Ninety-seven 4- to 6-year-old children (M = 63.2 months; 46 female, 51 male; 77% White, 8% more than one race, 4% Asian, and 3% Black; tested 2018) and 90 adults (M = 20.1 years; 70 female, 20 male) in the Midwestern United States learned novel categories with features that were easy (e.g., "red") or difficult (e.g., "mauve") to name. Adults (d = 1.06) and-to a lesser extent-children (d = 0.57; final training block) learned categories composed of more nameable features better. Children's knowledge of difficult-to-name color words predicted their learning for categories with difficult-to-name features. Rule-based category learning may be supported by the emerging ability to form verbal hypotheses.
Collapse
Affiliation(s)
- Martin Zettersten
- University of Wisconsin-Madison, Department of Psychology, 1202 W Johnson St, Madison, WI 53706, USA
- Princeton University, Department of Psychology, South Dr, Princeton, NJ 08540, USA
| | - Catherine Bredemann
- University of Wisconsin-Madison, Department of Educational Psychology, 1025 W Johnson St, Madison, WI 53706, USA
| | - Megan Kaul
- University of Wisconsin-Madison, Department of Educational Psychology, 1025 W Johnson St, Madison, WI 53706, USA
| | - Kaitlynn Ellis
- University of Wisconsin-Madison, Department of Educational Psychology, 1025 W Johnson St, Madison, WI 53706, USA
| | - Haley A. Vlach
- University of Wisconsin-Madison, Department of Educational Psychology, 1025 W Johnson St, Madison, WI 53706, USA
| | - Heather Kirkorian
- University of Wisconsin-Madison, Human Development and Family Studies Department, 1300 Linden Dr, Madison, WI 53706, USA
| | - Gary Lupyan
- University of Wisconsin-Madison, Department of Psychology, 1202 W Johnson St, Madison, WI 53706, USA
| |
Collapse
|
3
|
Saji N, Hong C, Wang C. Learning semantic categories of L2 verbs: The case of cutting and breaking verbs. PLoS One 2024; 19:e0296628. [PMID: 38241255 PMCID: PMC10798537 DOI: 10.1371/journal.pone.0296628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Accepted: 12/15/2023] [Indexed: 01/21/2024] Open
Abstract
To attain native-like proficiency in second-language word usage, learners have to discover intricate semantic categories in the target language. We investigated the factors influencing the development of two aspects of second-language learners' semantic categories: the category center and category boundary of word meanings. In the experiment, second-language learners of Japanese, whose first language is Mandarin, were asked to produce the best verb for 28 videos depicting various cutting and breaking events. Descriptive analyses were conducted to compare the verb patterns used by second-language learners with those of native speakers. The second-language learners' verb use pattern suggested their struggle in delineating the semantic ranges of breaking verbs in a native-like manner. Model analyses further revealed that different factors contribute to learning two different aspects of word meanings. The learning category center of word meaning depended on the similarity between the lexical domains in the first and second languages. On the contrary, the success of learning the semantic boundaries of verbs required a large input frequency and smaller semantic coverage, and smaller category ambiguity. The results suggest that constructing a semantic domain in the second language should be evaluated from at least two different aspects of semantic representation.
Collapse
Affiliation(s)
| | - Chunzi Hong
- Dalian University of Technology, Dalian, China
| | - Chong Wang
- Dalian University of Technology, Dalian, China
| |
Collapse
|
4
|
Forbes SH, Plunkett K. Colour perception changes with basic colour word comprehension. Dev Sci 2023; 26:e13406. [PMID: 37127947 DOI: 10.1111/desc.13406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 04/11/2023] [Accepted: 04/19/2023] [Indexed: 05/03/2023]
Abstract
Recent work has investigated the origin of infant colour categories, showing pre-linguistic infants categorise colour even in the absence of colour words. These infant categories are similar but not identical to adult categories, giving rise to an important question about how infant colour perception changes with the learning of colour words. Here we present two novel paradigms in which 12- and 19-month-old participants learning English as their first language were assessed on their perception of colour, while data on their colour word comprehension were also collected. Results indicate that participants' perception of colours close to the colour category boundaries dramatically change after colour word learning. The results highlight the shift made from infant colour categories to adult-like linguistically mediated colour categories that accompanies colour word learning. RESEARCH HIGHLIGHTS: We aimed to test whether colour perception is linguistically mediated in infants. We used novel eye-tracking and pupillometry paradigms to test infant colour perception either side of learning colour words. Infants' discrimination of colour changes after learning colour words, suggesting a shift due to colour word learning. A shift from pre-linguistic colour representation to linguistically mediated colour representation is discussed.
Collapse
|
5
|
Abstract
Color is a pervasive feature of our psychological experience, having a role in many aspects of human mind and behavior such as basic vision, scene perception, object recognition, aesthetics, and communication. Understanding how humans encode, perceive, talk about, and use color has been a major interdisciplinary effort. Here, we present the current state of knowledge on how color perception and cognition develop. We cover the development of various aspects of the psychological experience of color, ranging from low-level color vision to perceptual mechanisms such as color constancy to phenomena such as color naming and color preference. We also identify neurodiversity in the development of color perception and cognition and implications for clinical and educational contexts. We discuss the theoretical implications of the research for understanding mature color perception and cognition, for identifying the principles of perceptual and cognitive development, and for fostering a broader debate in the psychological sciences.
Collapse
Affiliation(s)
- John Maule
- The Sussex Colour Group & Baby Lab, School of Psychology, University of Sussex, Falmer, United Kingdom;
| | - Alice E Skelton
- The Sussex Colour Group & Baby Lab, School of Psychology, University of Sussex, Falmer, United Kingdom;
| | - Anna Franklin
- The Sussex Colour Group & Baby Lab, School of Psychology, University of Sussex, Falmer, United Kingdom;
| |
Collapse
|
6
|
Root N, Asano M, Melero H, Kim CY, Sidoroff-Dorso AV, Vatakis A, Yokosawa K, Ramachandran V, Rouw R. Do the colors of your letters depend on your language? Language-dependent and universal influences on grapheme-color synesthesia in seven languages. Conscious Cogn 2021; 95:103192. [PMID: 34500326 DOI: 10.1016/j.concog.2021.103192] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2020] [Revised: 08/05/2021] [Accepted: 08/06/2021] [Indexed: 12/24/2022]
Abstract
Grapheme-color synesthetes experience graphemes as having a consistent color (e.g., "N is turquoise"). Synesthetes' specific associations (which letter is which color) are often influenced by linguistic properties such as phonetic similarity, color terms ("Y is yellow"), and semantic associations ("D is for dog and dogs are brown"). However, most studies of synesthesia use only English-speaking synesthetes. Here, we measure the effect of color terms, semantic associations, and non-linguistic shape-color associations on synesthetic associations in Dutch, English, Greek, Japanese, Korean, Russian, and Spanish. The effect size of linguistic influences (color terms, semantic associations) differed significantly between languages. In contrast, the effect size of non-linguistic influences (shape-color associations), which we predicted to be universal, indeed did not differ between languages. We conclude that language matters (outcomes are influenced by the synesthete's language) and that synesthesia offers an exceptional opportunity to study influences on letter representations in different languages.
Collapse
Affiliation(s)
- Nicholas Root
- Brain and Cognition, Psychology Department, University of Amsterdam, Nieuwe Achtergracht 129-B, 1018 WT Amsterdam, Netherlands.
| | - Michiko Asano
- Department of Psychology, College of Contemporary Psychology, Rikkyo University, 1-2-26 Kitano, Niiza, Saitama 352-8558, Japan
| | - Helena Melero
- Department of Psychobiology and Methodology in Behavioral Sciences, Faculty of Psychology, Complutense University of Madrid, Av. Séneca, 2, 28040 Madrid, Spain
| | - Chai-Youn Kim
- Department of Psychology, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea
| | - Anton V Sidoroff-Dorso
- Department of Psychology, Moscow Pedagogical State University, Moscow 119435, Russia; Centre of Interdisciplinary Research of Contemporary Childhood, Moscow State University of Psychology and Education, Moscow 123290, Russia
| | - Argiro Vatakis
- Department of Psychology, Panteion University of Social and Political Sciences, 136 Syngrou Ave., 17671 Athens, Greece
| | - Kazuhiko Yokosawa
- Department of Psychology, Graduate School of Humanities and Sociology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Vilayanur Ramachandran
- Department of Psychology, University of California, San Diego, 9500 Gilman Drive, San Diego, CA 92093, United States
| | - Romke Rouw
- Brain and Cognition, Psychology Department, University of Amsterdam, Nieuwe Achtergracht 129-B, 1018 WT Amsterdam, Netherlands
| |
Collapse
|
7
|
Witzel C, Flack Z, Sanchez-Walker E, Franklin A. Colour category constancy and the development of colour naming. Vision Res 2021; 187:41-54. [PMID: 34167049 DOI: 10.1016/j.visres.2021.05.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Revised: 05/19/2021] [Accepted: 05/19/2021] [Indexed: 11/24/2022]
Abstract
In this study, we investigated the processes of coordination, adaptation, and calibration during the development of colour naming and colour constancy, and we tested whether colour term knowledge is related to colour constancy. We measured category membership and prototypes with 163 Munsell chips in preschool children (3- to 4-year-old) under neutral, green, and red illuminations, and compared their results to those of adults. We introduced an index of colour term maturity based on the similarity of children's colour term use to adults, and a colour category constancy index that quantifies the variation in colour categorisation that is specific to illumination changes. Results showed that illumination changes affected children's consistency of colour categorisation, but only to a small extent. However, colour term maturity and illumination-specific effects on consistency strongly varied in this age range. Correlations between colour term maturity and illumination-specific consistency indicated that colour constancy increases with colour term acquisition; but those results depended on the type of illumination changes (between neutral, green, and red). Together, our findings suggest that children progressively fine-tune and recalibrate the meaning of colour terms through processes of coordination and adaptation that are also involved in the calibration of colour constancy.
Collapse
Affiliation(s)
- Christoph Witzel
- School of Psychology, University of Southampton, United Kingdom.
| | - Zoe Flack
- School of Humanities and Social Science, University of Brighton, United Kingdom
| | - Emma Sanchez-Walker
- The Sussex Colour Group, School of Psychology, University of Sussex, Falmer, Brighton BN1 9RH, United Kingdom
| | - Anna Franklin
- The Sussex Colour Group, School of Psychology, University of Sussex, Falmer, Brighton BN1 9RH, United Kingdom
| |
Collapse
|
8
|
Grosse G, Streubel B, Gunzenhauser C, Saalbach H. Let's Talk About Emotions: the Development of Children's Emotion Vocabulary from 4 to 11 Years of Age. AFFECTIVE SCIENCE 2021; 2:150-162. [PMID: 36043167 PMCID: PMC9382957 DOI: 10.1007/s42761-021-00040-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 03/01/2021] [Indexed: 12/21/2022]
Abstract
Learning to use language in an adult-like way is a long-lasting process. This may particularly apply to complex conceptual domains such as emotions. The present study examined children's and adults' patterns of emotion word usage regarding their convergence and underlying semantic dimensions, and the factors influencing the ease of emotion word learning. We assessed the production of emotion words by 4- to 11-year-old children (N = 123) and 27 adults (M = 37 years) using a vignette test. We found that the older the children, the more emotion words they produced. Moreover, with increasing age, children's pattern of emotion word usage converged with adult usage. The analysis for semantic dimensions revealed one clear criterion-the differentiation of positive versus negative emotions-for all children and adults. We further found that broad covering emotion words are produced earlier and in a more adult-like way.
Collapse
Affiliation(s)
- Gerlind Grosse
- Department of Social and Education Sciences¸ Early Childhood Education Studies, Potsdam University of Applied Sciences, Kiepenheuerallee 5, 14469 Potsdam, Germany
- Leipzig Research Centre for Early Childhood Development, Leipzig University, Jahnallee 59, Leipzig, 04109 Germany
| | - Berit Streubel
- Leipzig Research Centre for Early Childhood Development, Leipzig University, Jahnallee 59, Leipzig, 04109 Germany
- Department of Educational Sciences, Leipzig University, Marschnerstraße 31, 04229 Leipzig, Germany
| | - Catherine Gunzenhauser
- Leipzig Research Centre for Early Childhood Development, Leipzig University, Jahnallee 59, Leipzig, 04109 Germany
- Department of Educational Sciences, University of Freiburg, Rempartstraße 11, Freiburg, 79098 Germany
| | - Henrik Saalbach
- Leipzig Research Centre for Early Childhood Development, Leipzig University, Jahnallee 59, Leipzig, 04109 Germany
- Department of Educational Sciences, Leipzig University, Marschnerstraße 31, 04229 Leipzig, Germany
| |
Collapse
|