Steady state visual evoked potentials reveal a signature of the pitch-size crossmodal association in visual cortex

Crossmodal correspondence of elevation/pitch and size/pitch is driven by real-world features

Crossmodal correspondences are consistent associations between sensory features from different modalities, with some theories suggesting they may either reflect environmental correlations or stem from innate neural structures. This study investigates this question by examining whether retinotopic or representational features of stimuli induce crossmodal congruency effects. Participants completed an auditory pitch discrimination task paired with visual stimuli varying in their sensory (retinotopic) or representational (scene integrated) nature, for both the elevation/pitch and size/pitch correspondences. Results show that only representational visual stimuli produced crossmodal congruency effects on pitch discrimination. These results support an environmental statistics hypothesis, suggesting crossmodal correspondences rely on real-world features rather than on sensory representations.

Involvement of the superior colliculi in crossmodal correspondences

There is an increasing body of evidence suggesting that there are low-level perceptual processes involved in crossmodal correspondences. In this study, we investigate the involvement of the superior colliculi in three basic crossmodal correspondences: elevation/pitch, lightness/pitch, and size/pitch. Using a psychophysical design, we modulate visual input to the superior colliculus to test whether the superior colliculus is required for behavioural crossmodal congruency effects to manifest in an unspeeded multisensory discrimination task. In the elevation/pitch task, superior colliculus involvement is required for a behavioural elevation/pitch congruency effect to manifest in the task. In the lightness/pitch and size/pitch task, we observed a behavioural elevation/pitch congruency effect regardless of superior colliculus involvement. These results suggest that the elevation/pitch correspondence may be processed differently to other low-level crossmodal correspondences. The implications of a distributed model of crossmodal correspondence processing in the brain are discussed.

Communicative signals during joint attention promote neural processes of infants and caregivers

Communicative signals such as eye contact increase infants' brain activation to visual stimuli and promote joint attention. Our study assessed whether communicative signals during joint attention enhance infant-caregiver dyads' neural responses to objects, and their neural synchrony. To track mutual attention processes, we applied rhythmic visual stimulation (RVS), presenting images of objects to 12-month-old infants and their mothers (n = 37 dyads), while we recorded dyads' brain activity (i.e., steady-state visual evoked potentials, SSVEPs) with electroencephalography (EEG) hyperscanning. Within dyads, mothers either communicatively showed the images to their infant or watched the images without communicative engagement. Communicative cues increased infants' and mothers' SSVEPs at central-occipital-parietal, and central electrode sites, respectively. Infants showed significantly more gaze behaviour to images during communicative engagement. Dyadic neural synchrony (SSVEP amplitude envelope correlations, AECs) was not modulated by communicative cues. Taken together, maternal communicative cues in joint attention increase infants' neural responses to objects, and shape mothers' own attention processes. We show that communicative cues enhance cortical visual processing, thus play an essential role in social learning. Future studies need to elucidate the effect of communicative cues on neural synchrony during joint attention. Finally, our study introduces RVS to study infant-caregiver neural dynamics in social contexts.