Changes in visual object recognition precede the shape bias in early noun learning

Using deep neural networks to disentangle visual and semantic information in human perception and memory

Mental representations of familiar categories are composed of visual and semantic information. Disentangling the contributions of visual and semantic information in humans is challenging because they are intermixed in mental representations. Deep neural networks that are trained either on images or on text or by pairing images and text enable us now to disentangle human mental representations into their visual, visual-semantic and semantic components. Here we used these deep neural networks to uncover the content of human mental representations of familiar faces and objects when they are viewed or recalled from memory. The results show a larger visual than semantic contribution when images are viewed and a reversed pattern when they are recalled. We further reveal a previously unknown unique contribution of an integrated visual-semantic representation in both perception and memory. We propose a new framework in which visual and semantic information contribute independently and interactively to mental representations in perception and memory.

Word learning in ASD: the sensorimotor, the perceptual and the symbolic

Word learning requires successful pairing of form and meaning. A common hypothesis about the process of word learning is that initially, infants work on identifying the phonological segments corresponding to words (speech analysis), and subsequently map those segments onto meaning. A range of theories have been proposed to account for the underlying mechanisms and factors in this remarkable achievement. While some are mainly concerned with the sensorimotor affordances and perceptual properties of referents out in the world, other theories emphasize the importance of language as a system, and the relations among language units (other words or syntax). Recent approaches inspired by neuro-science suggest that the storage and processing of word meanings is supported by neural systems subserving both the representation of conceptual knowledge and its access and use (Lambon Ralph et al., Nature Reviews Neuroscience 18:42–55, 2017). Developmental disorders have been attested to impact on different aspects of word learning. While impaired word knowledge is not a hallmark of Autism Spectrum Disorder (ASD), and remains largely understudied in this population, there is evidence that there are, sometimes subtle, problems in that domain, reflected in both how such knowledge is acquired and how words are used (Vulchanova et al., Word knowledge and word usage: A cross-disciplinary guide to the mental lexicon, Mouton De Gruyter, 2020). In addition, experimental evidence suggests that children with autism present with specific problems in categorizing the referents of linguistic labels leading to subsequent problems with using those labels (Hartley and Allen, Autism 19:570–579, 2015). Furthermore, deficits have been reported in some of the underlying mechanisms, biases and use of cues in word learning, such as e.g., object shape (Field et al., Journal of Autism and Developmental Disorders 46:1210–1219, 2016; Tek et al., Autism Research 1:208–222, 2008). Finally, it is likely that symbol use might be impaired in ASD, however, the direction of the causal relationship between social and communication impairment in autism and symbolic skills is still an open question (Allen and Lewis, Journal of Autism and Developmental Disorders 45:1–3, 2015; Allen and Butler, British Journal of Developmental Psychology 38:345–362, 2020; Wainwright et al., Journal of Autism and Developmental Disorders 50:2941–2956, 2020). Further support for impaired symbol formation in autism comes from the well-attested problems with figurative, non-literal language use (e.g., metaphors, idioms, hyperbole, irony) (Vulchanova et al., Frontiers in Human Neuroscience 9:24, 2015). Here we propose that embodied theories of cognition which link perceptual experience with conceptual knowledge (see Eigsti, Frontiers in Psychology 4:224, 2013; Klin et al., Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 358:345–360, 2003) might be useful in explaining the difficulty in symbolic understanding that individuals with autism face during the word learning process.

Vocabulary Development and the Shape Bias in Children With Hearing Loss

Purpose Although children with hearing loss (HL) can benefit from cochlear implants (CIs) and hearing aids (HAs), they often show language delays. Moreover, little is known about the mechanisms by which children with HL learn words. One mechanism by which typically hearing (TH) children learn words is by acquiring word learning biases such as the "shape bias," that is, generalizing the names of novel solid objects by similarity in shape. In TH children, the shape bias emerges out of regularities in the early vocabulary and, once acquired, has consequences for subsequent vocabulary development. Method Here, we ask whether children with HL exhibit similar word learning biases as TH children. In the current study, nineteen 2- to 3.5-year-old children with HL generalized the names of novel objects by similarity in shape or material. We compared their performance to that of 20 TH children matched on age and 20 TH children matched on vocabulary size. Results Children with HL were significantly less likely than age-matched TH children and vocabulary-matched TH children to generalize novel names to objects of the same shape. However, there was also an interaction such that vocabulary has a stronger effect on novel noun generalization for those with HL than for those who are TH. Exploratory analyses of children with HL reveal similar novel noun generalization and vocabulary sizes in children who use CIs and those who use HAs, regardless of hearing age or degree of HL. Conclusion Together, the results suggest that, although vocabulary knowledge drives development of the shape bias in general for all children, it may be especially important for children with HL, who are at risk for language delays.

Adjective Learning by Spanish-Speakers Children and Adults. The Impact of Morphology and Semantic Information

The purpose of this research was to explore the kind of information Spanish-speaking 3-year-old children and adults use when learning adjectives in a joint picturebook reading situation. The impact of two linguistic clues was studied; a morphological clue (adjective suffix) and a semantic clue (descriptive information concerning the property). Results show that for children the description was decisive to map the new adjective with the property; for adults, instead, the presence of the suffix was crucial. These results illustrate a developmental shift in the sort of clues that shapes adjective learning.

Self-generated variability in object images predicts vocabulary growth

Object names are a major component of early vocabularies and learning object names depends on being able to visually recognize objects in the world. However, the fundamental visual challenge of the moment-to-moment variations in object appearances that learners must resolve has received little attention in word learning research. Here we provide the first evidence that image-level object variability matters and may be the link that connects infant object manipulation to vocabulary development. Using head-mounted eye tracking, the present study objectively measured individual differences in the moment-to-moment variability of visual instances of the same object, from infants' first-person views. Infants who generated more variable visual object images through manual object manipulation at 15 months of age experienced greater vocabulary growth over the next six months. Elucidating infants' everyday visual experiences with objects may constitute a crucial missing link in our understanding of the developmental trajectory of object name learning.

Brain Measures of Toddlers' Shape Recognition Predict Language and Cognitive Skills at 6-7 Years

While a number of studies have found that an improvement in object shape recognition is associated with language growth in infants and toddlers, no published studies have investigated the longitudinal relation between early shape recognition, and language abilities in later childhood. An electrophysiological measure of semantic processing (the N400) was used to assess shape recognition and general object recognition in a naming context in 20-month-olds. The measures of shape recognition strongly predicted language and cognitive abilities at 6–7 years even after controlling for toddler vocabulary size. The electrophysiological measures of general object recognition were not related to future language or cognitive abilities. These results suggest that early shape recognition abilities may play a role in language acquisition and influence even long-term language outcomes.

The Heterogeneity of Word Learning Biases in Late-Talking Children

Purpose The particular statistical approach researchers choose is intimately connected to the way they conceptualize their questions, which, in turn, can influence the conclusions they draw. One particularly salient area in which statistics influence our conclusions is in the context of atypical development. Traditional statistical approaches such as t tests or analysis of variance lend themselves to a focus on group differences, downplaying the heterogeneity that exists within so many atypically developing populations. Understanding such variability is important-classification of what a disorder is, an individual's diagnosis, and whether or not a child receives intervention all directly relate to an accurate classification of the disorder and individual's abilities compared to their typically developing peers. Method Here, we use word learning biases (i.e., shape and material biases) in late-talking children as a sample case and employ a variety of statistical approaches to compare the conclusions those approaches might warrant. Results We argue that advanced statistical approaches, such as mixed-effects regression, can help us make sense of heterogeneity and are more consistent with a modern dimensional view of language disorders. Conclusions Accurate characterization of late-talking children (and others at risk for delays) and their prognoses is necessary for accurate diagnosis and implementation of appropriate target interventions. It therefore requires rigorous statistical analyses that can capture and allow for interpretation of the heterogeneity inherent in populations with language delays and disorders.

The role of colour labels in mediating toddler visual attention

Toddlers, children and adults will spontaneously attend to a semantically- or perceptually-related object when a named target is absent from the visual scene: Upon hearing "strawberry", they will orient to a red plate rather than a yellow one. We examine the role that knowledge of feature labels plays in mediating visual attention to unnamed features. For example, does knowing the word "red", facilitate attending to red objects, though the label is not uttered? We show that toddlers systematically fixate a colour-related object, if and only if they know the name of the colour associated with the named object and the perceptually-related object. These findings suggest that knowledge of perceptual feature labels can play a central role in highlighting salient similarities between objects, both present and absent in the toddler's visual field. We discuss the implications and limitations of these findings beyond the realm of recognition of colour similarities between objects.

Reproducibility and a unifying explanation: Lessons from the shape bias

The goal of science is to advance our understanding of particular phenomena. However, in the field of development, the phenomena of interest are complex, multifaceted, and change over time. Here, we use three decades of research on the shape bias to argue that while replication is clearly an important part of the scientific process, integration across the findings of many studies that include variations in procedure is also critical to create a coherent understanding of the thoughts and behaviors of young children. The "shape bias," or the tendency to generalize a novel label to novel objects of the same shape, is a reliable and robust behavioral finding and has been shown to predict future vocabulary growth and possible language disorders. Despite the robustness of the phenomenon, the way in which the shape bias is defined and tested has varied across studies and laboratories. The current review argues that differences in performance that come from even seemingly minor changes to the participants or task can offer critical insight to underlying mechanisms, and that working to incorporate data from multiple labs is an important way to reveal how task variation and a child's individual pathway creates behavior-a key issue for understanding developmental phenomena.

Words affect visual perception by activating object shape representations

Linguistic labels are known to facilitate object recognition, yet the mechanism of this facilitation is not well understood. Previous psychophysical studies have suggested that words guide visual perception by activating information about visual object shape. Here we aimed to test this hypothesis at the neural level, and to tease apart the visual and semantic contribution of words to visual object recognition. We created a set of object pictures from two semantic categories with varying shapes, and obtained subjective ratings of their shape and category similarity. We then conducted a word-picture matching experiment, while recording participants’ EEG, and tested if the shape or the category similarity between the word’s referent and target picture explained the spatiotemporal pattern of the picture-evoked responses. The results show that hearing a word activates representations of its referent’s shape, which interacts with the visual processing of a subsequent picture within 100 ms from its onset. Furthermore, non-visual categorical information, carried by the word, affects the visual processing at later stages. These findings advance our understanding of the interaction between language and visual perception and provide insights into how the meanings of words are represented in the brain.

How self-generated labelling shapes transfer of learning during early childhood: The role of individual differences

Multiple factors influence imitation during toddlerhood, including task complexity, social contingency, and individual differences. We conducted a secondary data analysis of individual differences in self-generated labelling using data collected from a complex puzzle imitation task with 355 2- to 3-year-olds. This analysis indicated that toddlers' ability to label the completed puzzle (fish or boat) was associated with better imitation performance. Labelling occurs during social interactions; therefore, our second analysis tested how labelling differed as a function of the level of social scaffolding in each condition. This analysis revealed that self-generated labelling was lower when the social demonstrator was removed and the task was presented on a touchscreen. This study is one of the first to examine self-generated labelling during a complex imitation task in toddlers and increases our understanding of the complexity of memory processing needed for imitation learning. Statement of contribution What is already known on this subject? Toddlers exhibit a transfer of learning deficit from 2D media, including books, TV, and tablets. Self-generated labelling enhances children's learning, through attentional and cognitive mechanisms. Children are sensitive to reduced social cues in screen media contributing to the transfer deficit. What does this study add? Self-generated labelling is associated with better goal imitation performance. Self-generated labelling occurs more frequently under social conditions.

The Developing Infant Creates a Curriculum for Statistical Learning

New efforts are using head cameras and eye-trackers worn by infants to capture everyday visual environments from the point of view of the infant learner. From this vantage point, the training sets for statistical learning develop as the sensorimotor abilities of the infant develop, yielding a series of ordered datasets for visual learning that differ in content and structure between timepoints but are highly selective at each timepoint. These changing environments may constitute a developmentally ordered curriculum that optimizes learning across many domains. Future advances in computational models will be necessary to connect the developmentally changing content and statistics of infant experience to the internal machinery that does the learning.

A Developmental Approach to Machine Learning?

Visual learning depends on both the algorithms and the training material. This essay considers the natural statistics of infant- and toddler-egocentric vision. These natural training sets for human visual object recognition are very different from the training data fed into machine vision systems. Rather than equal experiences with all kinds of things, toddlers experience extremely skewed distributions with many repeated occurrences of a very few things. And though highly variable when considered as a whole, individual views of things are experienced in a specific order - with slow, smooth visual changes moment-to-moment, and developmentally ordered transitions in scene content. We propose that the skewed, ordered, biased visual experiences of infants and toddlers are the training data that allow human learners to develop a way to recognize everything, both the pervasively present entities and the rarely encountered ones. The joint consideration of real-world statistics for learning by researchers of human and machine learning seems likely to bring advances in both disciplines.

Is a Pink Cow Still a Cow? Individual Differences in Toddlers' Vocabulary Knowledge and Lexical Representations

When a toddler knows a word, what does she actually know? Many categories have multiple relevant properties; for example, shape and color are relevant to membership in the category banana. How do toddlers prioritize these properties when recognizing familiar words, and are there systematic differences among children? In this study, toddlers viewed pairs of objects associated with prototypical colors. On some trials, objects were typically colored (e.g., Holstein cow and pink pig); on other trials, colors were switched (e.g., pink cow and Holstein-patterned pig). On each trial, toddlers were directed to find a target object. Overall, recognition was disrupted when colors were switched, as measured by eye movements. Moreover, individual differences in vocabularies predicted recognition differences: Toddlers who say fewer shape-based words were more disrupted by color switches. "Knowing" a word may not mean the same thing for all toddlers; different toddlers prioritize different facets of familiar objects in their lexical representations.

How we categorize objects is related to how we remember them: The shape bias as a memory bias

The "shape bias" describes the phenomenon that, after a certain point in development, children and adults generalize object categories based on shape to a greater degree than other perceptual features. The focus of research on the shape bias has been to examine the types of information that learners attend to in one moment in time. The current work takes a different approach by examining whether learners' categorical biases are related to their retention of information across time. In three experiments, children's (N=72) and adults' (N=240) memory performance for features of objects was examined in relation to their categorical biases. The results of these experiments demonstrated that the number of shape matches chosen during the shape bias task significantly predicted shape memory. Moreover, children and adults with a shape bias were more likely to remember the shape of objects than the color and size of objects. Taken together, this work suggests that the development of a shape bias may engender better memory for shape information.

Individual and developmental differences in preschoolers' categorization biases and vocabulary across tasks

This study bridges prior research on young children's use of taxonomic versus thematic relations to categorize objects with prior research on their use of shared shape versus shared function to categorize artifacts. Specifically, this research examined associations in children's categorization tendencies across these two dichotomies, including assessments of individual differences, developmental trends, and vocabulary level. Preschoolers (3- to 5-year-olds) completed a receptive vocabulary assessment and two match-to-sample tasks: one pitting (superordinate) taxonomic and thematic relations against each other and one pitting shape and function similarity against each other. The results revealed individual and developmental variation in children's cross-task categorization biases, with a predominant tendency to focus on both thematic and function relations that became increasingly stronger with age. In 3- and 5-year-olds, function-based categorization was also positively associated with verb vocabulary. These findings demonstrate an emerging tendency to focus on relational information during the preschool years that, among other learning effects, may benefit verb acquisition. The results are discussed in terms of the real-time processing and developmental factors that might contribute to the development of strategies for learning about objects and categories during early childhood.

Impact of feature saliency on visual category learning

People have to sort numerous objects into a large number of meaningful categories while operating in varying contexts. This requires identifying the visual features that best predict the 'essence' of objects (e.g., edibility), rather than categorizing objects based on the most salient features in a given context. To gain this capacity, visual category learning (VCL) relies on multiple cognitive processes. These may include unsupervised statistical learning, that requires observing multiple objects for learning the statistics of their features. Other learning processes enable incorporating different sources of supervisory information, alongside the visual features of the categorized objects, from which the categorical relations between few objects can be deduced. These deductions enable inferring that objects from the same category may differ from one another in some high-saliency feature dimensions, whereas lower-saliency feature dimensions can best differentiate objects from distinct categories. Here I illustrate how feature saliency affects VCL, by also discussing kinds of supervisory information enabling reflective categorization. Arguably, principles debated here are often being ignored in categorization studies.

Relations among early object recognition skills: Objects and letters

Human visual object recognition is multifaceted, with several domains of expertise. Developmental relations between young children's letter recognition and their 3-dimensional object recognition abilities are implicated on several grounds but have received little research attention. Here, we ask how preschoolers' success in recognizing letters relates to their ability to recognize 3-dimensional objects from sparse shape information alone. A relation is predicted because perception of the spatial relations is critical in both domains. Seventy-three 2 ½- to 4-year-old children completed a Letter Recognition task, measuring the ability to identify a named letter among 3 letters with similar shapes, and a "Shape Caricature Recognition" task, measuring recognition of familiar objects from sparse, abstract information about their part shapes and the spatial relations among those parts. Children also completed a control "Shape Bias" task, in which success depends on recognition of overall object shape but not of relational structure. Children's success in letter recognition was positively related to their shape caricature recognition scores, but not to their shape bias scores. The results suggest that letter recognition builds upon developing skills in attending to and representing the relational structure of object shape, and that these skills are common to both 2-dimensional and 3-dimensional object perception.

Event-related potentials during word mapping to object shape predict toddlers' vocabulary size

What role does attention to different object properties play in early vocabulary development? This longitudinal study using event-related potentials in combination with behavioral measures investigated 20- and 24-month-olds' (n = 38; n = 34; overlapping n = 24) ability to use object shape and object part information in word-object mapping. The N400 component was used to measure semantic priming by images containing shape or detail information. At 20 months, the N400 to words primed by object shape varied in topography and amplitude depending on vocabulary size, and these differences predicted productive vocabulary size at 24 months. At 24 months, when most of the children had vocabularies of several hundred words, the relation between vocabulary size and the N400 effect in a shape context was weaker. Detached object parts did not function as word primes regardless of age or vocabulary size, although the part-objects were identified behaviorally. The behavioral measure, however, also showed relatively poor recognition of the part-objects compared to the shape-objects. These three findings provide new support for the link between shape recognition and early vocabulary development.

Using the axis of elongation to align shapes: developmental changes between 18 and 24 months of age

An object's axis of elongation serves as an important frame of reference for forming three-dimensional representations of object shape. By several recent accounts, the formation of these representations is also related to experiences of acting on objects. Four experiments examined 18- to 24-month-olds' (N=103) sensitivity to the elongated axis in action tasks that required extracting, comparing, and physically rotating an object so that its major axis was aligned with that of a visual standard. In Experiments 1 and 2, the older toddlers precisely rotated both simple and complexly shaped three-dimensional objects in insertion tasks where the visual standard was the rectangular contour defining the opening in a box. The younger toddlers performed poorly. Experiments 3 and 4 provide evidence on emerging abilities in extracting and using the most extended axis as a frame of reference for shape comparison. Experiment 3 showed that 18-month-olds could rotate an object to align its major axis with the direction of their own hand motion, and Experiment 4 showed that they could align the major axis of one object with that of another object of the exact same three-dimensional shape. The results are discussed in terms of theories of the development of three-dimensional shape representations, visual object recognition, and the role of action in these developments.

It's all connected: Pathways in visual object recognition and early noun learning

A developmental pathway may be defined as the route, or chain of events, through which a new structure or function forms. For many human behaviors, including object name learning and visual object recognition, these pathways are often complex and multicausal and include unexpected dependencies. This article presents three principles of development that suggest the value of a developmental psychology that explicitly seeks to trace these pathways and uses empirical evidence on developmental dependencies among motor development, action on objects, visual object recognition, and object name learning in 12- to 24-month-old infants to make the case. The article concludes with a consideration of the theoretical implications of this approach. (PsycINFO Database Record (c) 2013 APA, all rights reserved).