Multisensory perception of looming and receding objects in human newborns

Perception of visual and audiovisual trajectories toward and away from the body in the first postnatal year

Perceiving motion in depth is important in everyday life, especially motion in relation to the body. Visual and auditory cues inform us about motion in space when presented in isolation from each other, but the most comprehensive information is obtained through the combination of both of these cues. We traced the development of infants' ability to discriminate between visual motion trajectories across peripersonal space and to match these with auditory cues specifying the same peripersonal motion. We measured 5-month-old (n = 20) and 9-month-old (n = 20) infants' visual preferences for visual motion toward or away from their body (presented simultaneously and side by side) across three conditions: (a) visual displays presented alone, (b) paired with a sound increasing in intensity, and (c) paired with a sound decreasing in intensity. Both groups preferred approaching motion in the visual-only condition. When the visual displays were paired with a sound increasing in intensity, neither group showed a visual preference. When a sound decreasing in intensity was played instead, the 5-month-olds preferred the receding (spatiotemporally congruent) visual stimulus, whereas the 9-month-olds preferred the approaching (spatiotemporally incongruent) visual stimulus. We speculate that in the approaching sound condition, the behavioral salience of the sound could have led infants to focus on the auditory information alone, in order to prepare a motor response, and to neglect the visual stimuli. In the receding sound condition, instead, the difference in response patterns in the two groups may have been driven by infants' emerging motor abilities and their developing predictive processing mechanisms supporting and influencing each other.

Trajectories of brain and behaviour development in the womb, at birth and through infancy

Birth is often seen as the starting point for studying effects of the environment on human development, with much research focused on the capacities of young infants. However, recent imaging advances have revealed that the complex behaviours of the fetus and the uterine environment exert influence. Birth is now viewed as a punctuate event along a developmental pathway of increasing autonomy of the child from their mother. Here we highlight (1) increasing physiological autonomy and perceptual sensitivity in the fetus, (2) physiological and neurochemical processes associated with birth that influence future behaviour, (3) the recalibration of motor and sensory systems in the newborn to adapt to the world outside the womb and (4) the effect of the prenatal environment on later infant behaviours and brain function. Taken together, these lines of evidence move us beyond nature-nurture issues to a developmental human lifespan view beginning within the womb.

Cortical signatures of auditory looming bias show cue-specific adaptation between newborns and young adults

Adaptive biases in favor of approaching, or "looming", sounds have been found across ages and species, thereby implicating the potential of their evolutionary origin and universal basis. The human auditory system is well-developed at birth, yet spatial hearing abilities further develop with age. To disentangle the speculated inborn, evolutionary component of the auditory looming bias from its learned counterpart, we collected high-density electroencephalographic data across human adults and newborns. As distance-motion cues we manipulated either the sound's intensity or spectral shape, which is pinna-induced and thus prenatally inaccessible. Through cortical source localisation we demonstrated the emergence of the bias in both age groups at the level of Heschl's gyrus. Adults exhibited the bias in both attentive and inattentive states; yet differences in amplitude and latency appeared based on attention and cue type. Contrary to the adults, in newborns the bias was elicited only through manipulations of intensity and not spectral cues. We conclude that the looming bias comprises innate components while flexibly incorporating the spatial cues acquired through lifelong exposure.

The emergence of the multisensory brain: From the womb to the first steps

The becoming of the human being is a multisensory process that starts in the womb. By integrating spontaneous neuronal activity with inputs from the external world, the developing brain learns to make sense of itself through multiple sensory experiences. Over the past ten years, advances in neuroimaging and electrophysiological techniques have allowed the exploration of the neural correlates of multisensory processing in the newborn and infant brain, thus adding an important piece of information to behavioral evidence of early sensitivity to multisensory events. Here, we review recent behavioral and neuroimaging findings to document the origins and early development of multisensory processing, particularly showing that the human brain appears naturally tuned to multisensory events at birth, which requires multisensory experience to fully mature. We conclude the review by highlighting the potential uses and benefits of multisensory interventions in promoting healthy development by discussing emerging studies in preterm infants.

Newborns' perception of approach and withdrawal from biological movement: A closeness story

Since birth, infants discriminate the biological motion (BM) revealed by point-light displays (PLDs). To date, no studies have explored whether newborns differentiate BM that approaches rather than withdraws from them. Yet, approach and withdrawal are two fundamental motivations in the socio-emotional world, key to developing empathy and prosocial behavior. Through a looking-behavior paradigm, we demonstrated that a few hours after birth, a human figure approaching attracted more visual attention than a human figure receding, showing that newborns are attuned to PLDs of others moving toward rather than walking away from them. Further, a withdrawing body appears to be less attractive than withdrawing scrambled points. Altogether, these observations support the existence of an early predisposition toward social closeness that might have its roots in an evolutionary perspective.

Neural Integration of Audiovisual Sensory Inputs in Macaque Amygdala and Adjacent Regions

Integrating multisensory inputs to generate accurate perception and guide behavior is among the most critical functions of the brain. Subcortical regions such as the amygdala are involved in sensory processing including vision and audition, yet their roles in multisensory integration remain unclear. In this study, we systematically investigated the function of neurons in the amygdala and adjacent regions in integrating audiovisual sensory inputs using a semi-chronic multi-electrode array and multiple combinations of audiovisual stimuli. From a sample of 332 neurons, we showed the diverse response patterns to audiovisual stimuli and the neural characteristics of bimodal over unimodal modulation, which could be classified into four types with differentiated regional origins. Using the hierarchical clustering method, neurons were further clustered into five groups and associated with different integrating functions and sub-regions. Finally, regions distinguishing congruent and incongruent bimodal sensory inputs were identified. Overall, visual processing dominates audiovisual integration in the amygdala and adjacent regions. Our findings shed new light on the neural mechanisms of multisensory integration in the primate brain.

Visual objects approaching the body modulate subsequent somatosensory processing at 4 months of age

We asked whether, in the first year of life, the infant brain can support the dynamic crossmodal interactions between vision and somatosensation that are required to represent peripersonal space. Infants aged 4 (n = 20, 9 female) and 8 (n = 20, 10 female) months were presented with a visual object that moved towards their body or receded away from it. This was presented in the bottom half of the screen and not fixated upon by the infants, who were instead focusing on an attention getter at the top of the screen. The visual moving object then disappeared and was followed by a vibrotactile stimulus occurring later in time and in a different location in space (on their hands). The 4-month-olds' somatosensory evoked potentials (SEPs) were enhanced when tactile stimuli were preceded by unattended approaching visual motion, demonstrating that the dynamic visual-somatosensory cortical interactions underpinning representations of the body and peripersonal space begin early in the first year of life. Within the 8-month-olds' sample, SEPs were increasingly enhanced by (unexpected) tactile stimuli following receding visual motion as age in days increased, demonstrating changes in the neural underpinnings of the representations of peripersonal space across the first year of life.

Abstract representations of small sets in newborns

From the very first days of life, newborns are not tied to represent narrow, modality- and object-specific aspects of their environment. Rather, they sometimes react to abstract properties shared by stimuli of very different nature, such as approximate numerosity or magnitude. As of now, however, there is no evidence that newborns possess abstract representations that apply to small sets: in particular, while newborns can match large approximate numerosities across senses, this ability does not extend to small numerosities. In two experiments, we presented newborn infants (N = 64, age 17 to 98 h) with patterned sets AB or ABB simultaneously in the auditory and visual modalities. Auditory patterns were presented as periodic sequences of sounds (AB: triangle-drum-triangle-drum-triangle-drum …; ABB: triangle-drum-drum-triangle-drum-drum-triangle-drum-drum …), and visual patterns as arrays of 2 or 3 shapes (AB: circle-diamond; ABB: circle-diamond-diamond). In both experiments, we found that participants reacted and looked longer when the patterns matched across the auditory and visual modalities – provided that the first stimulus they received was congruent. These findings uncover the existence of yet another type of abstract representations at birth, applying to small sets. As such, they bolster the hypothesis that newborns are endowed with the capacity to represent their environment in broad strokes, in terms of its most abstract properties. This capacity for abstraction could later serve as a scaffold for infants to learn about the particular entities surrounding them.

•

Newborns were presented with auditory and visual patterns (AB vs. ABB).

•

Participants reacted when the patterns presented were congruent across modalities.

•

Newborns possess abstract representations applying to small sets.

•

These representations may encode numerosity and/or repetitions.

Image statistics determine the integration of visual cues to motion-in-depth

Motion-in-depth perception is critical in enabling animals to avoid hazards and respond to potential threats. For humans, important visual cues for motion-in-depth include changing disparity (CD) and changing image size (CS). The interpretation and integration of these cues depends upon multiple scene parameters, such as distance moved, object size and viewing distance, posing a significant computational challenge. We show that motion-in-depth cue integration depends upon sensitivity to the joint probabilities of the scene parameters determining these signals, and on the probability of CD and CS signals co-occurring. Models that took these factors into account predicted human performance in speed-in-depth and cue conflict discrimination tasks, where standard linear integration models could not. These results suggest that cue integration is affected by both the uncertainty of sensory signals and the mapping of those signals to real-world properties. Evidence of a role for such mappings demonstrates the importance of scene and image statistics to the processes underpinning cue integration and the perception of motion-in-depth.

Face in collision: emotional looming stimuli modulate interpersonal space across development and gender

Basic visual functions have evolved to allow for rapid detection of dynamic stimuli in our surrounding environment. In particular, looming stimuli are of relevance because they are expected to enter the individual’s interpersonal space representing a potential threat. Different studies showed that emotions can modulate the perception of visual looming stimuli and the borders of interpersonal space, defined as the area around the body that individuals maintain between themselves and others during social interactions. Here, we investigated how emotions modulate the perception and the physiological correlates of interpersonal space and whether such indexes change across age and gender. Children and adults were asked to quickly react to emotional looming stimuli while measuring their skin conductance response (SCR). We found that emotional looming stimuli shrink the borders of interpersonal space of males more than females, and that this pattern does not change with age. In addition, adults reacted faster to angry than happy and neutral faces, which is in line with the notion that threatening stimuli capture attention more quickly than other types of emotional stimuli. However, this was not observed in children, suggesting that experience with negative stimuli, rather than the evolutionary meaning they possess, may influence the boundaries of interpersonal space. Overall, our study suggests that interpersonal space is modulated by emotions, but this appears to be modulated by gender and age: while behavioural responses to emotional looming stimuli refine with age, physiological responses are adult-like as early as 5 years of age.

Labels and object categorization in six- and nine-month-olds: tracking labels across varying carrier phrases

Language shapes object categorization in infants. This starts as a general enhanced attentional effect of language, which narrows to a specific link between labels and categories by twelve months. The current experiments examined this narrowing effect by investigating when infants track a consistent label across varied input. Six-month-old infants (N = 48) were familiarized to category exemplars, each presented with the exact same labeling phrase or the same label in different phrases. Evidence of object categorization at test was only found with the same phrase, suggesting that infants were not tracking the label's consistency, but rather that of the entire input. Nine-month-olds (N = 24) did show evidence of categorization across the varied phrases, suggesting that they were tracking the consistent label across the varied input.

Neural Mechanisms Underlying the Auditory Looming Bias

Our auditory system constantly keeps track of our environment, informing us about our surroundings and warning us of potential threats. The auditory looming bias is an early perceptual phenomenon, reflecting higher alertness of listeners to approaching auditory objects, rather than to receding ones. Experimentally, this sensation has been elicited by using both intensity-varying stimuli, as well as spectrally varying stimuli with constant intensity. Following the intensity-based approach, recent research delving into the cortical mechanisms underlying the looming bias argues for top-down signaling from the prefrontal cortex to the auditory cortex in order to prioritize approaching over receding sonic motion. We here test the generalizability of that finding to spectrally induced looms by re-analyzing previously published data. Our results indicate the promoted top-down projection but at time points slightly preceding the motion onset and thus considered to reflect a bias driven by anticipation. At time points following the motion onset, our findings show a bottom-up bias along the dorsal auditory pathway directed toward the prefrontal cortex.

Spatial tuning of electrophysiological responses to multisensory stimuli reveals a primitive coding of the body boundaries in newborns

The ability to identify our own body and its boundaries is crucial for survival. Ideally, the sooner we learn to discriminate external stimuli occurring close to our body from those occurring far from it, the better (and safer) we may interact with the sensory environment. However, when this mechanism emerges within ontogeny is unknown. Is it something acquired throughout infancy, or is it already present soon after birth? The presence of a spatial modulation of multisensory integration (MSI) is considered a hallmark of a functioning representation of the body position in space. Here, we investigated whether MSI is present and spatially organized in 18- to 92-h-old newborns. We compared electrophysiological responses to tactile stimulation when concurrent auditory events were delivered close to, as opposed to far from, the body in healthy newborns and in a control group of adult participants. In accordance with previous studies, adult controls showed a clear spatial modulation of MSI, with greater superadditive responses for multisensory stimuli close to the body. In newborns, we demonstrated the presence of a genuine electrophysiological pattern of MSI, with older newborns showing a larger MSI effect. Importantly, as for adults, multisensory superadditive responses were modulated by the proximity to the body. This finding may represent the electrophysiological mechanism responsible for a primitive coding of bodily self boundaries, thus suggesting that even just a few hours after birth, human newborns identify their own body as a distinct entity from the environment.

Direct Structural Connections between Auditory and Visual Motion-Selective Regions in Humans

In humans, the occipital middle-temporal region (hMT⁺/V5) specializes in the processing of visual motion, while the planum temporale (hPT) specializes in auditory motion processing. It has been hypothesized that these regions might communicate directly to achieve fast and optimal exchange of multisensory motion information. Here we investigated, for the first time in humans (male and female), the presence of direct white matter connections between visual and auditory motion-selective regions using a combined fMRI and diffusion MRI approach. We found evidence supporting the potential existence of direct white matter connections between individually and functionally defined hMT⁺/V5 and hPT. We show that projections between hMT⁺/V5 and hPT do not overlap with large white matter bundles, such as the inferior longitudinal fasciculus and the inferior frontal occipital fasciculus. Moreover, we did not find evidence suggesting the presence of projections between the fusiform face area and hPT, supporting the functional specificity of hMT⁺/V5-hPT connections. Finally, the potential presence of hMT⁺/V5-hPT connections was corroborated in a large sample of participants (n = 114) from the human connectome project. Together, this study provides a first indication for potential direct occipitotemporal projections between hMT⁺/V5 and hPT, which may support the exchange of motion information between functionally specialized auditory and visual regions.SIGNIFICANCE STATEMENT Perceiving and integrating moving signal across the senses is arguably one of the most important perceptual skills for the survival of living organisms. In order to create a unified representation of movement, the brain must therefore integrate motion information from separate senses. Our study provides support for the potential existence of direct connections between motion-selective regions in the occipital/visual (hMT⁺/V5) and temporal/auditory (hPT) cortices in humans. This connection could represent the structural scaffolding for the rapid and optimal exchange and integration of multisensory motion information. These findings suggest the existence of computationally specific pathways that allow information flow between areas that share a similar computational goal.

Visual stimulus-specific habituation of innate defensive behaviour in mice

Innate defensive responses such as freezing or escape are essential for animal survival. Mice show defensive behaviour to stimuli sweeping overhead, like a bird cruising the sky. Here, we tested this in young male mice and found that mice reduced their defensive freezing after sessions with a stimulus passing overhead repeatedly. This habituation is stimulus specific, as mice freeze again to a novel shape. Habituation occurs regardless of the visual field location of the repeated stimulus. The mice generalized over a range of sizes and shapes, but distinguished objects when they differed in both size and shape. Innate visual defensive responses are thus strongly influenced by previous experience as mice learn to ignore specific stimuli.

Discrimination of ordinal relationships in temporal sequences by 4-month-old infants

The ability to discriminate the ordinal information embedded in magnitude-based sequences has been shown in 4-month-old infants, both for numerical and size-based sequences. At this early age, however, this ability is confined to increasing sequences, with infants failing to extract and represent decreasing order. Here we investigate whether the ability to represent order extends to duration-based sequences in 4-month-old infants, and whether it also shows the asymmetry signature previously observed for number and size. Infants were tested in an order discrimination task in which they were habituated to either increasing or decreasing variations in temporal duration, and were then tested with novel sequences composed of new temporal items whose durations varied following the familiar and the novel orders in alternation. Across three experiments, we manipulated the duration of the single temporal items and therefore of the whole sequences, which resulted in imposing more or less constraints on infants' working memory, or general processing capacities. Results showed that infants failed at discriminating the ordinal direction in temporal sequences when the sequences had an overall long duration (Experiment 1), but succeeded when the duration of the sequences was shortened (Experiments 2 and 3). Moreover, there was no sign of the asymmetry signature previously reported for number and size, as successful discrimination was present for infants habituated to both increasing and decreasing sequences. These results suggest that sensitivity to temporal order is present very early in development, and that its functional properties are not shared with other magnitude dimensions, such as size and number.