Spatial relation categorization in infants and deep neural networks

Spatial relations, such as above, below, between, and containment, are important mediators in children's understanding of the world (Piaget, 1954). The development of these relational categories in infancy has been extensively studied (Quinn, 2003) yet little is known about their computational underpinnings. Using developmental tests, we examine the extent to which deep neural networks, pretrained on a standard vision benchmark or egocentric video captured from one baby's perspective, form categorical representations for visual stimuli depicting relations. Notably, the networks did not receive any explicit training on relations. We then analyze whether these networks recover similar patterns to ones identified in development, such as reproducing the relative difficulty of categorizing different spatial relations and different stimulus abstractions. We find that the networks we evaluate tend to recover many of the patterns observed with the simpler relations of "above versus below" or "between versus outside", but struggle to match developmental findings related to "containment". We identify factors in the choice of model architecture, pretraining data, and experimental design that contribute to the extent the networks match developmental patterns, and highlight experimental predictions made by our modeling results. Our results open the door to modeling infants' earliest categorization abilities with modern machine learning tools and demonstrate the utility and productivity of this approach.

Perceptual reorganization from prior knowledge emerges late in childhood

Human vision relies heavily on prior knowledge. Here, we show for the first time that prior-knowledge-induced reshaping of visual inputs emerges gradually in late childhood. To isolate the effects of prior knowledge on perception, we presented 4- to 12-year-olds and adults with two-tone images - hard-to-recognize degraded photos. In adults, seeing the original photo triggers perceptual reorganization, causing mandatory recognition of the two-tone version. This involves top-down signaling from higher-order brain areas to early visual cortex. We show that children younger than 7-9 years do not experience this knowledge-guided shift, despite viewing the original photo immediately before each two-tone. To assess computations underlying this development, we compared human performance to three neural networks with varying architectures. The best-performing model behaved much like 4- to 5-year-olds, displaying feature-based rather than holistic processing strategies. The reconciliation of prior knowledge with sensory input undergoes a striking age-related shift, which may underpin the development of many perceptual abilities.

Overcoming bias in the comparison of human language and animal communication

Human language is a powerful communicative and cognitive tool. Scholars have long sought to characterize its uniqueness, but each time a property is proposed to set human language apart (e.g., reference, syntax), some (attenuated) version of that property is found in animals. Recently, the uniqueness argument has shifted from linguistic rules to cognitive capacities underlying them. Scholars argue that human language is unique because it relies on ostension and inference, while animal communication depends on simple associations and largely hardwired signals. Such characterizations are often borne out in published data, but these empirical findings are driven by radical differences in the ways animal and human communication are studied. The field of animal communication has been dramatically shaped by the "code model," which imagines communication as involving information packets that are encoded, transmitted, decoded, and interpreted. This framework standardized methods for studying meaning in animal signals, but it does not allow for the nuance, ambiguity, or contextual variation seen in humans. The code model is insidious. It is rarely referenced directly, but it significantly shapes how we study animals. To compare animal communication and human language, we must acknowledge biases resulting from the different theoretical models used. By incorporating new approaches that break away from searching for codes, we may find that animal communication and human language are characterized by differences of degree rather than kind.

The sound of silence: Reconsidering infants' object categorization in silence, with labels, and with nonlinguistic sounds

A large body of research based on a specific stimulus set (dinosaur/fish) has argued that auditory labels and novel communicative signals (such as beeps used in a communicative context) facilitate category formation in infants, that such effects can be attributed to the auditory signals' communicative nature, and that other auditory stimuli have no effect on categorization. A contrasting view, the auditory overshadowing hypothesis, maintains that auditory signals disrupt processing of visual information and, therefore, interfere with categorization, with more unfamiliar sounds having a more disruptive effect than familiar ones. Here, we used the dinosaur/fish stimulus set to test these contrasting theories in two experiments. In Experiment 1 (N = 17), we found that 6-month-old infants were able to form categories of these stimuli in silence, weakening the claim that labels facilitated their categorization in infants. These results imply that prior findings of no categorization of these stimuli in the presence of nonlinguistic sounds must be due to disruptive effects of such sounds. In Experiment 2 (N = 17), we showed that familiarity modulated the disruptive effect of nonlinguistic sounds on infants' categorization of these stimuli. Together, these results support the auditory overshadowing hypothesis and provide new insights into the interaction between visual and auditory information in infants' category formation.

Preverbal infants tune manual choices on subliminal affective information

Human behaviour is often shaped by unconscious emotional cues. From early on, infants are able to process emotional signals even if presented subliminally; however, whether subliminal emotional expressions are capable to affect infants' behaviour remains unknown. The current study aimed to fill this gap, recording 8-10-month-old infants' looking time and manual choice toward two objects previously associated to subliminal emotional faces. Results demonstrated that infants' manual choice, but not looking time, was guided by the previously presented subliminal emotional signal, as infants preferred to choose the object associated to the happy face. Overall, our findings show that preverbal infants tune their behaviour based on affective information, which drives them towards or away from previous encounters, even outside conscious awareness.

Real-world statistics at two timescales and a mechanism for infant learning of object names

Infants learn mappings between heard names and seen things before their first birthday and before they produce spoken language. Two challenges to explaining this early learning are the immaturity of infant memory systems and the infrequency of any individual object name in the heard language input. We quantified the frequency of visual referents, heard names, and the cooccurrences of referents and names in infant everyday experiences. We discovered statistical patterns at two timescales that align with a cortical mechanism of associative memory formation that supports the rapid formation of durable associative memories from very few experienced cooccurrences.

Infants begin learning the visual referents of nouns before their first birthday. Despite considerable empirical and theoretical effort, little is known about the statistics of the experiences that enable infants to break into object–name learning. We used wearable sensors to collect infant experiences of visual objects and their heard names for 40 early-learned categories. The analyzed data were from one context that occurs multiple times a day and includes objects with early-learned names: mealtime. The statistics reveal two distinct timescales of experience. At the timescale of many mealtime episodes (n = 87), the visual categories were pervasively present, but naming of the objects in each of those categories was very rare. At the timescale of single mealtime episodes, names and referents did cooccur, but each name–referent pair appeared in very few of the mealtime episodes. The statistics are consistent with incremental learning of visual categories across many episodes and the rapid learning of name–object mappings within individual episodes. The two timescales are also consistent with a known cortical learning mechanism for one-episode learning of associations: new information, the heard name, is incorporated into well-established memories, the seen object category, when the new information cooccurs with the reactivation of that slowly established memory.

Labels and object categorization in six- and nine-month-olds: tracking labels across varying carrier phrases

Language shapes object categorization in infants. This starts as a general enhanced attentional effect of language, which narrows to a specific link between labels and categories by twelve months. The current experiments examined this narrowing effect by investigating when infants track a consistent label across varied input. Six-month-old infants (N = 48) were familiarized to category exemplars, each presented with the exact same labeling phrase or the same label in different phrases. Evidence of object categorization at test was only found with the same phrase, suggesting that infants were not tracking the label's consistency, but rather that of the entire input. Nine-month-olds (N = 24) did show evidence of categorization across the varied phrases, suggesting that they were tracking the consistent label across the varied input.

Intuitive sociology

How do humans intuitively understand the structure of their society? How should psychologists study people's commonsense understanding of societal structure? The present chapter seeks to address both of these questions by describing the domain of "intuitive sociology." Drawing primarily from empirical research focused on how young children represent and reason about social groups, we propose that intuitive sociology consists of three core phenomena: social types (the identification of relevant groups and their attributes); social value (the worth of different groups); and social norms (shared expectations for how groups ought to be). After articulating each component of intuitive sociology, we end the chapter by considering both the emergence of intuitive sociology in infancy as well as transitions from intuitive to reflective representations of sociology later in life.

Do infants discriminate non-linguistic vocal expressions of positive emotions?

Adults are highly proficient in understanding emotional signals from both facial and vocal cues, including when communicating across cultural boundaries. However, the developmental origin of this ability is poorly understood, and in particular, little is known about the ontogeny of differentiation of signals with the same valence. The studies reported here employed a habituation paradigm to test whether preverbal infants discriminate between non-linguistic vocal expressions of relief and triumph. Infants as young as 6 months who had habituated to relief or triumph showed significant discrimination of relief and triumph tokens at test (i.e. greater recovery to the unhabituated stimulus type), when exposed to tokens from a single individual (Study 1). Infants habituated to expressions from multiple individuals showed less consistent discrimination in that consistent discrimination was only found when infants were habituated to relief tokens (Study 2). Further, infants tested with tokens from individuals from different cultures showed dishabituation only when habituated to relief tokens and only at 10-12 months (Study 3). These findings suggest that discrimination between positive emotional expressions develops early and is modulated by learning. Further, infants' categorical representations of emotional expressions, like those of speech sounds, are influenced by speaker-specific information.

Basic Emotions, Natural Kinds, Emotion Schemas, and a New Paradigm

Research on emotion flourishes in many disciplines and specialties, yet experts cannot agree on its definition. Theorists and researchers use the term emotion in ways that imply different processes and meanings. Debate continues about the nature of emotions, their functions, their relations to broad affective dimensions, the processes that activate them, and their role in our daily activities and pursuits. I will address these issues here, specifically in terms of basic emotions as natural kinds, the nature of emotion schemas, the development of emotion—cognition relations that lead to emotion schemas, and discrete emotions in relation to affective dimensions. Finally, I propose a new paradigm that assumes continual emotion as a factor in organizing consciousness and as an influence on mind and behavior. The evidence reviewed suggests that a theory that builds on concepts of both basic emotions and emotion schemas provides a viable research tool and is compatible with more holistic or dimensional approaches.

What goes where? Eye tracking reveals spatial relational memory during infancy

Episodic memory involves binding components of an event (who, what, when, and where) into a relational representation. The ability to encode information about the relative locations of objects (i.e., spatial relational memory) is a key component of episodic memory. Here we used eye tracking to test whether infants and toddlers learn about the spatial relations among objects. In Experiment 1, 9-, 18-, and 27-month olds were familiarized with an array of three objects. Following familiarization, they saw test arrays in which two of the objects had been replaced with novel ones (object switch condition) and arrays in which two of the objects had switched positions (location switch condition). Both 18- and 27-month olds looked significantly longer than would be predicted by chance at the objects that had switched spatial locations; however, 9-month olds did not. In Experiment 2, we showed that, given sufficient familiarization time, 9-month olds were also capable of detecting disruptions to the spatial relations among an array of objects. These results have important implications for our understanding of spatial relational memory development.

From perceptual to language-mediated categorization

From at least two months onwards, infants can form perceptual categories. During the first year of life, object knowledge develops from the ability to represent individual object features to representing correlations between attributes and to integrate information from different sources. At the end of the first year, these representations are shaped by labels, opening the way to conceptual knowledge. Here, we review the development of object knowledge and object categorization over the first year of life. We then present an artificial neural network model that models the transition from early perceptual categorization to categories mediated by labels. The model informs a current debate on the role of labels in object categorization by suggesting that although labels do not act as object features they nevertheless affect perceived similarity of perceptually distinct objects sharing the same label. The model presents the first step of an integrated account from early perceptual categorization to language-based concept learning.

Development of differential sensitivity for shape changes resulting from linear and nonlinear planar transformations

A shape bias for extending names to objects that look visually similar has been commonly accepted but it is hard to define which kind of shape dissimilarities are diagnostic for the identity of an object. Here, we present a transformational approach to describe shape differences that can incorporate many significant shape features. We introduce two kinds of transformations: one kind concerns linear transformations of the image plane (affine transformations), generally limiting shape variations within the borders of basic-level categories; the other kind concerns nonlinear continuous transformations of the image plane (topological transformations), allowing all kinds of shape variation crossing and not crossing the borders of basic-level categories. We administered stimulus pairs differing in these shape transformations to children of 3 years to 7 years old in a delayed match-to-sample task. With increasing age, especially between 5 years and 6 years, children became more sensitive to the topological deformations that are relevant for between-category distinctions, indicating that acquired categorical knowledge in early years induces perceptual learning of the relevant generic shape differences between categories.

Categorization in 3- and 4-month-old infants: an advantage of words over tones

Neonates prefer human speech to other nonlinguistic auditory stimuli. However, it remains an open question whether there are any conceptual consequences of words on object categorization in infants younger than 6 months. The current study examined the influence of words and tones on object categorization in forty-six 3- to 4-month-old infants. Infants were familiarized to different exemplars of a category accompanied by either a labeling phrase or a tone sequence. In test, infants viewed novel category and new within-category exemplars. Infants who heard labeling phrases provided evidence of categorization at test while infants who heard tone sequences did not, suggesting that infants as young as 3 months of age treat words and tones differently vis-à-vis object categorization.

Development of the other-race effect during infancy: evidence toward universality?

The other-race effect in face processing develops within the first year of life in Caucasian infants. It is currently unknown whether the developmental trajectory observed in Caucasian infants can be extended to other cultures. This is an important issue to investigate because recent findings from cross-cultural psychology have suggested that individuals from Eastern and Western backgrounds tend to perceive the world in fundamentally different ways. To this end, the current study investigated 3-, 6-, and 9-month-old Chinese infants' ability to discriminate faces within their own racial group and within two other racial groups (African and Caucasian). The 3-month-olds demonstrated recognition in all conditions, whereas the 6-month-olds recognized Chinese faces and displayed marginal recognition for Caucasian faces but did not recognize African faces. The 9-month-olds' recognition was limited to Chinese faces. This pattern of development is consistent with the perceptual narrowing hypothesis that our perceptual systems are shaped by experience to be optimally sensitive to stimuli most commonly encountered in one's unique cultural environment.

Young infants' reasoning about physical events involving inert and self-propelled objects

The present research examined whether 5- to 6.5-month-old infants would hold different expectations about various physical events involving a box after receiving evidence that it was either inert or self-propelled. Infants were surprised if the inert but not the self-propelled box: reversed direction spontaneously (Experiment 1); remained stationary when hit or pulled (Experiments 3 and 3A); remained stable when released in midair or with inadequate support from a platform (Experiment 4); or disappeared when briefly hidden by one of two adjacent screens (the second screen provided the self-propelled box with an alternative hiding place; Experiment 5). On the other hand, infants were surprised if the inert or the self-propelled box appeared to pass through an obstacle (Experiment 2) or disappeared when briefly hidden by a single screen (Experiment 5). The present results indicate that infants as young as 5 months of age distinguish between inert and self-propelled objects and hold different expectations for physical events involving these objects, even when incidental differences between the objects are controlled. These findings are consistent with the proposal by Gelman, R. (1990). First principles organize attention to and learning about relevant data: Number and the animate-inanimate distinction as examples. Cognitive Science, 14, 79-106, Leslie, A. M. (1994). ToMM, ToBY, and Agency: Core architecture and domain specificity. In L. A. Hirschfeld & S. A. Gelman (Eds.), Mapping the mind: Domain specificity in cognition and culture (pp. 119-148). New York: Cambridge University Press, and others that infants endow self-propelled objects with an internal source of energy. Possible links between infants' concepts of self-propelled object, agent, and animal are also discussed.

Categorical representation of facial expressions in the infant brain

Categorical perception, demonstrated as reduced discrimination of within-category relative to between-category differences in stimuli, has been found in a variety of perceptual domains in adults. To examine the development of categorical perception in the domain of facial expression processing, we used behavioral and event-related potential (ERP) methods to assess discrimination of within-category (happy-happy) and between-category (happy-sad) differences in facial expressions in 7-month-old infants. Data from a visual paired-comparison test and recordings of attention-sensitive ERPs showed no discrimination of facial expressions in the within-category condition whereas reliable discrimination was observed in the between-category condition. The results also showed that face-sensitive ERPs over occipital-temporal scalp (P400) were attenuated in the within-category condition relative to the between-category condition, suggesting a potential neural basis for the reduced within-category sensitivity. Together, these results suggest that the neural systems underlying categorical representation of facial expressions emerge during the early stages of postnatal development, before acquisition of language.

Infants' categorization of novel objects with more or less obvious features

In five experiments, 14- to 15-month-old infants' categorization of objects on the basis of more or less obvious features was investigated. Using an object examining paradigm, a total of 200 infants were familiarized with novel objects that shared either more obvious features (i.e., easily visible) or less obvious features (i.e., accessible by lifting a flap), followed by an in-category object and an out-of-category object. When only perceptual information was available, infants formed a category on the basis of the more obvious features but not on the basis of the less obvious features (Experiments 1 and 3). When infants were provided with animacy cues and/or object names, they formed categories on the basis of either more or less obvious features (Experiments 2, 4, and 5). The results of these studies delineate the role of animacy cues and object names in establishing categories on the basis of less obvious features.

Can chimpanzee infants (Pan troglodytes) form categorical representations in the same manner as human infants (Homo sapiens)?

We directly compared chimpanzee infants and human infants for categorical representations of three global-like categories (mammals, furniture and vehicles), using the familiarization-novelty preference technique. Neither species received any training during the experiments. We used the time that participants spent looking at the stimulus object while touching it as a measure. During the familiarization phase, participants were presented with four familiarization objects from one of three categories (e.g. mammals). Then, they were tested with a pair of novel objects, one was a familiar-category object and another was a novel-category object (e.g. vehicle) in the test phase. The chimpanzee infants did not show significant habituation, whereas human infants did. However, most important, both species showed significant novelty-preference in the test phase. This indicates that not only human infants, but also chimpanzee infants formed categorical representations of a global-like level. Implications for the shared origins and species-specificity of categorization abilities, and the cognitive operations underlying categorization, are discussed.

Is the asymmetry in young infants' categorization of humans versus nonhuman animals based on head, body, or global gestalt information?

Quinn and Eimas (1998) reported an asymmetry in the exclusivity of the category representations that young infants form for humans and nonhuman animals: category representations for nonhuman animal species were found to exclude humans, whereas a category representation for humans was found to include nonhuman animal species (i.e., cats, horses). The present experiment utilized the familiarization/novelty-preference procedure with 3- and 4-month-olds to determine the perceptual cues (i.e., whole stimulus, head alone, body alone) that provided the basis for this asymmetry. The data revealed the asymmetry to be observable only with the whole animal stimuli and not when infants were provided with information from just the head or the body of the exemplars. The results indicate that the incorporation of nonhuman animal species into a category representation for humans is based on holistic information.

Spatial representation by young infants: Categorization of spatial relations or sensitivity to a crossing primitive?

The spatial representation abilities of 3- to 4-month-old infants were examined in four experiments. Experiments 1 and 2 showed that infants familiarized with a diamond appearing in distinct locations to the left or right of a vertical bar subsequently preferred a stimulus depicting the diamond on the opposite side of the bar over a stimulus depicting the diamond in a novel location on the same side of the bar. Experiment 3 was a replication of Experiment 1, except that the bar was oriented at 45 degrees. In this instance, infants divided their attention between the stimulus depicting the diamond on the opposite side of the bar and the stimulus depicting the diamond in a novel location on the same side of the bar. Experiment 4 demonstrated that the results of Experiment 3 were not a consequence of a failure to process the diagonal bar. When considered with previous reports that infants can represent the categories of above and below (Quinn, 1994), the present results suggest that (1) infants can also represent the categories of left and right, and (2) performance cannot be interpreted as a response to an arbitrary crossing of one object relative to another. Although recent discussions of the relation between language and cognition have pointed to the ways in which spatial language influences spatial cognition (Bowerman & Levinson, 2001), the present findings are consistent with an influence in the opposite direction: Spatial cognition may in some instances shape spatial language.

Development of Subordinate-Level Categorization in 3- to 7-Month-Old Infants

Visual preference procedures were used to investigate development of perceptually based subordinate-level categorization in 3- to 7-month-old infants. Experiments 1 and 2 demonstrated that 3- to 4-month-olds did not form category representations for photographic exemplars of subordinate-level classes of cats and dogs (i.e., Siamese vs. Tabby, Beagle vs. Saint Bernard). Experiments 3 though 5 showed that 6- and 7-month-olds formed a category representation for Tabby that excluded Siamese and a category representation for Saint Bernard that excluded Beagle, but they did not form a category representation for Siamese that excluded Tabby or a category representation for Beagle that excluded Saint Bernard. The findings are consistent with a differentiation-driven view of early perceptual category development from global to basic to subordinate levels.

A test of the generality of perceptually based categories found in infants: attentional differences toward natural kinds by New World monkeys

A preference to novelty paradigm used to study human infants (Quinn, 2002) examined attention to novel animal pictures at subordinate, basic and superordinate levels in tamarins. First, pairs of pictures were presented in phases, starting with a monkey species (subordinate level) and ending with mammal and dinosaur sets (superordinate levels). After each phase, tests paired novel pictures from the familiarized set with a novel broader category. Look rates toward each picture were coded. Tamarins looked significantly longer at a novel species after being familiarized with a monkey species, a species-specific effect. Subjects attended equivalently to novel primate species after habituation to four monkey species, but looked significantly longer at pictures of mammals, marking a more global-level inclusion and exclusion. Superordinate testing revealed that more novel and diverse sets were differentiated attentionally. The evidence implies that natural categorical representation occurs at an attentional level in primates in ways similar to human infants, and is affected by recent exposure and category variability.

Representation of the gender of human faces by infants: a preference for female

Six experiments based on visual preference procedures were conducted to examine gender categorization of female versus male faces by infants aged 3 to 4 months. In experiment 1, infants familiarized with male faces preferred a female face over a novel male face, but infants familiarized with female faces divided their attention between a male face and a novel female face. Experiment 2 demonstrated that these asymmetrical categorization results were likely due to a spontaneous preference for females. Experiments 3 and 4 showed that the preference for females was based on processing of the internal facial features in their upright orientation, and not the result of external hair cues or higher-contrast internal facial features. While experiments 1 through 4 were conducted with infants reared with female primary caregivers, experiment 5 provided evidence that infants reared with male primary caregivers tend to show a spontaneous preference for males. Experiment 6 showed that infants reared with female primary caregivers displayed recognition memory for individual females, but not males. These results suggest that representation of information about human faces by young infants may be influenced by the gender of the primary caregiver.