1
|
Portelance E, Frank MC, Jurafsky D. Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model. Cogn Sci 2024; 48:e13448. [PMID: 38742768 DOI: 10.1111/cogs.13448] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Revised: 04/05/2024] [Accepted: 04/12/2024] [Indexed: 05/16/2024]
Abstract
Interpreting a seemingly simple function word like "or," "behind," or "more" can require logical, numerical, and relational reasoning. How are such words learned by children? Prior acquisition theories have often relied on positing a foundation of innate knowledge. Yet recent neural-network-based visual question answering models apparently can learn to use function words as part of answering questions about complex visual scenes. In this paper, we study what these models learn about function words, in the hope of better understanding how the meanings of these words can be learned by both models and children. We show that recurrent models trained on visually grounded language learn gradient semantics for function words requiring spatial and numerical reasoning. Furthermore, we find that these models can learn the meanings of logical connectives and and or without any prior knowledge of logical reasoning as well as early evidence that they are sensitive to alternative expressions when interpreting language. Finally, we show that word learning difficulty is dependent on the frequency of models' input. Our findings offer proof-of-concept evidence that it is possible to learn the nuanced interpretations of function words in a visually grounded context by using non-symbolic general statistical learning algorithms, without any prior knowledge of linguistic meaning.
Collapse
Affiliation(s)
- Eva Portelance
- Department of Linguistics, McGill University
- Mila - Quebec Artificial Intelligence Institute
| | | | | |
Collapse
|
2
|
Lavechin M, de Seyssel M, Métais M, Metze F, Mohamed A, Bredin H, Dupoux E, Cristia A. Modeling early phonetic acquisition from child-centered audio data. Cognition 2024; 245:105734. [PMID: 38335906 DOI: 10.1016/j.cognition.2024.105734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 12/29/2023] [Accepted: 01/22/2024] [Indexed: 02/12/2024]
Abstract
Infants learn their native language(s) at an amazing speed. Before they even talk, their perception adapts to the language(s) they hear. However, the mechanisms responsible for this perceptual attunement and the circumstances in which it takes place remain unclear. This paper presents the first attempt to study perceptual attunement using ecological child-centered audio data. We show that a simple prediction algorithm exhibits perceptual attunement when applied on unrealistic clean audio-book data, but fails to do so when applied on ecologically-valid child-centered data. In the latter scenario, perceptual attunement only emerges when the prediction mechanism is supplemented with inductive biases that force the algorithm to focus exclusively on speech segments while learning speaker-, pitch-, and room-invariant representations. We argue these biases are plausible given previous research on infants and non-human animals. More generally, we show that what our model learns and how it develops through exposure to speech depends exquisitely on the details of the input signal. By doing so, we illustrate the importance of considering ecologically valid input data when modeling language acquisition.
Collapse
Affiliation(s)
- Marvin Lavechin
- Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France; Cognitive Machine Learning Team, INRIA, Paris, France; Meta AI Research, Paris, France.
| | - Maureen de Seyssel
- Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France; Cognitive Machine Learning Team, INRIA, Paris, France; Laboratoire de linguistique formelle, Université de Paris, CNRS, Paris, France
| | - Marianne Métais
- Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France; Cognitive Machine Learning Team, INRIA, Paris, France
| | | | | | - Hervé Bredin
- Institut de Recherche en Informatique de Toulouse, Université de Toulouse, CNRS, Toulouse, France
| | - Emmanuel Dupoux
- Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France; Cognitive Machine Learning Team, INRIA, Paris, France; Meta AI Research, Paris, France
| | - Alejandrina Cristia
- Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France; Cognitive Machine Learning Team, INRIA, Paris, France
| |
Collapse
|
3
|
de Seyssel M, Lavechin M, Dupoux E. Realistic and broad-scope learning simulations: first results and challenges. JOURNAL OF CHILD LANGUAGE 2023; 50:1294-1317. [PMID: 37246513 DOI: 10.1017/s0305000923000272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
There is a current 'theory crisis' in language acquisition research, resulting from fragmentation both at the level of the approaches and the linguistic level studied. We identify a need for integrative approaches that go beyond these limitations, and propose to analyse the strengths and weaknesses of current theoretical approaches of language acquisition. In particular, we advocate that language learning simulations, if they integrate realistic input and multiple levels of language, have the potential to contribute significantly to our understanding of language acquisition. We then review recent results obtained through such language learning simulations. Finally, we propose some guidelines for the community to build better simulations.
Collapse
Affiliation(s)
- Maureen de Seyssel
- Laboratoire de Sciences Cognitives et de Psycholinguistique, Département d'Études Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France
- Laboratoire de Linguistique Formelle, Université Paris Cité, CNRS, Paris, France
| | - Marvin Lavechin
- Laboratoire de Sciences Cognitives et de Psycholinguistique, Département d'Études Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France
- Meta AI Research, Paris, France
| | - Emmanuel Dupoux
- Laboratoire de Sciences Cognitives et de Psycholinguistique, Département d'Études Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France
- Meta AI Research, Paris, France
| |
Collapse
|
4
|
Nikolaus M, Fourtassi A. Communicative Feedback in language acquisition. NEW IDEAS IN PSYCHOLOGY 2023. [DOI: 10.1016/j.newideapsych.2022.100985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
5
|
Cristia A. A systematic review suggests marked differences in the prevalence of infant-directed vocalization across groups of populations. Dev Sci 2023; 26:e13265. [PMID: 35429106 DOI: 10.1111/desc.13265] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Revised: 03/25/2022] [Accepted: 03/28/2022] [Indexed: 12/15/2022]
Abstract
Anthropological reports have long suggested that speaking to young children is very infrequent in certain populations (notably farming ones), which is in line with scattered quantitative studies. A systematic review was undertaken to use available literature in order to estimate the extent of population variation. Database searches, expert lists, and citation searches led to the discovery of 29 reports on the frequency of vocalizations directed to infants aged 24 months or younger, based on systematic observations of spontaneous activity in the infant's natural environment lasting at least 30 min in length. Together, these studies provide evidence on 1314 infants growing up in a range of communities (urban, foraging, farming). For populations located outside of North America, the frequency with which vocalization was directed to urban infants was much higher than that for rural infants (including both foraging and farming, medians = 12.6 vs. 3.6% of observations contained infant-directed vocalization behaviors). We benchmarked this effect against socio-economic status (SES) variation in the United States, which was much smaller. Infants in high SES American homes were spoken to only slightly more frequently than those in low SES homes (medians = 16.4 vs. 15.1% of observations contained infant-directed vocalization behaviors). Although published research represents a biased sample of the world's populations, these results invite further cross-population research to understand the causes and effects of such considerable population group differences.
Collapse
Affiliation(s)
- Alejandrina Cristia
- Laboratoire de Sciences Cognitives et de Psycholinguistique, Département d'Etudes cognitives, ENS, EHESS, CNRS, PSL University, Paris, France
| |
Collapse
|
6
|
DIANA, a Process-Oriented Model of Human Auditory Word Recognition. Brain Sci 2022; 12:brainsci12050681. [PMID: 35625067 PMCID: PMC9140177 DOI: 10.3390/brainsci12050681] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 05/05/2022] [Accepted: 05/10/2022] [Indexed: 02/04/2023] Open
Abstract
This article presents DIANA, a new, process-oriented model of human auditory word recognition, which takes as its input the acoustic signal and can produce as its output word identifications and lexicality decisions, as well as reaction times. This makes it possible to compare its output with human listeners’ behavior in psycholinguistic experiments. DIANA differs from existing models in that it takes more available neuro-physiological evidence on speech processing into account. For instance, DIANA accounts for the effect of ambiguity in the acoustic signal on reaction times following the Hick–Hyman law and it interprets the acoustic signal in the form of spectro-temporal receptive fields, which are attested in the human superior temporal gyrus, instead of in the form of abstract phonological units. The model consists of three components: activation, decision and execution. The activation and decision components are described in detail, both at the conceptual level (in the running text) and at the computational level (in the Appendices). While the activation component is independent of the listener’s task, the functioning of the decision component depends on this task. The article also describes how DIANA could be improved in the future in order to even better resemble the behavior of human listeners.
Collapse
|