Visual response properties and visuotopic representation in the newborn monkey superior colliculus

The brain can develop conflicting multisensory principles to guide behavior

Midbrain multisensory neurons undergo a significant postnatal transition in how they process cross-modal (e.g. visual-auditory) signals. In early stages, signals derived from common events are processed competitively; however, at later stages they are processed cooperatively such that their salience is enhanced. This transition reflects adaptation to cross-modal configurations that are consistently experienced and become informative about which correspond to common events. Tested here was the assumption that overt behaviors follow a similar maturation. Cats were reared in omnidirectional sound thereby compromising the experience needed for this developmental process. Animals were then repeatedly exposed to different configurations of visual and auditory stimuli (e.g. spatiotemporally congruent or spatially disparate) that varied on each side of space and their behavior was assessed using a detection/localization task. Animals showed enhanced performance to stimuli consistent with the experience provided: congruent stimuli elicited enhanced behaviors where spatially congruent cross-modal experience was provided, and spatially disparate stimuli elicited enhanced behaviors where spatially disparate cross-modal experience was provided. Cross-modal configurations not consistent with experience did not enhance responses. The presumptive benefit of such flexibility in the multisensory developmental process is to sensitize neural circuits (and the behaviors they control) to the features of the environment in which they will function. These experiments reveal that these processes have a high degree of flexibility, such that two (conflicting) multisensory principles can be implemented by cross-modal experience on opposite sides of space even within the same animal.

Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression

Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

Correlated variability in primate superior colliculus depends on functional class

Correlated variability in neuronal activity (spike count correlations, r_SC) can constrain how information is read out from populations of neurons. Traditionally, r_SC is reported as a single value summarizing a brain area. However, single values, like summary statistics, stand to obscure underlying features of the constituent elements. We predict that in brain areas containing distinct neuronal subpopulations, different subpopulations will exhibit distinct levels of r_SC that are not captured by the population r_SC. We tested this idea in macaque superior colliculus (SC), a structure containing several functional classes (i.e., subpopulations) of neurons. We found that during saccade tasks, different functional classes exhibited differing degrees of r_SC. "Delay class" neurons displayed the highest r_SC, especially during saccades that relied on working memory. Such dependence of r_SC on functional class and cognitive demand underscores the importance of taking functional subpopulations into account when attempting to model or infer population coding principles.

Multisensory Integration in Caenorhabditis elegans in Comparison to Mammals

Multisensory integration refers to sensory inputs from different sensory modalities being processed simultaneously to produce a unitary output. Surrounded by stimuli from multiple modalities, animals utilize multisensory integration to form a coherent and robust representation of the complex environment. Even though multisensory integration is fundamentally essential for animal life, our understanding of the underlying mechanisms, especially at the molecular, synaptic and circuit levels, remains poorly understood. The study of sensory perception in Caenorhabditis elegans has begun to fill this gap. We have gained a considerable amount of insight into the general principles of sensory neurobiology owing to C. elegans’ highly sensitive perceptions, relatively simple nervous system, ample genetic tools and completely mapped neural connectome. Many interesting paradigms of multisensory integration have been characterized in C. elegans, for which input convergence occurs at the sensory neuron or the interneuron level. In this narrative review, we describe some representative cases of multisensory integration in C. elegans, summarize the underlying mechanisms and compare them with those in mammalian systems. Despite the differences, we believe C. elegans is able to provide unique insights into how processing and integrating multisensory inputs can generate flexible and adaptive behaviors. With the emergence of whole brain imaging, the ability of C. elegans to monitor nearly the entire nervous system may be crucial for understanding the function of the brain as a whole.

On the relationship between maps and domains in inferotemporal cortex

How does the brain encode information about the environment? Decades of research have led to the pervasive notion that the object-processing pathway in primate cortex consists of multiple areas that are each specialized to process different object categories (such as faces, bodies, hands, non-face objects and scenes). The anatomical consistency and modularity of these regions have been interpreted as evidence that these regions are innately specialized. Here, we propose that ventral-stream modules do not represent clusters of circuits that each evolved to process some specific object category particularly important for survival, but instead reflect the effects of experience on a domain-general architecture that evolved to be able to adapt, within a lifetime, to its particular environment. Furthermore, we propose that the mechanisms underlying the development of domains are both evolutionarily old and universal across cortex. Topographic maps are fundamental, governing the development of specializations across systems, providing a framework for brain organization.

Successful visually guided eye movements following sight restoration after congenital cataracts

Sensitive periods have previously been identified for several human visual system functions. Yet, it is unknown to what degree the development of visually guided oculomotor control depends on early visual experience-for example, whether and to what degree humans whose sight was restored after a transient period of congenital visual deprivation are able to conduct visually guided eye movements. In the present study, we developed new calibration and analysis techniques for eye tracking data contaminated with pervasive nystagmus, which is typical for this population. We investigated visually guided eye movements in sight recovery individuals with long periods of visual pattern deprivation (3-36 years) following birth due to congenital, dense, total, bilateral cataracts. As controls we assessed (1) individuals with nystagmus due to causes other than cataracts, (2) individuals with developmental cataracts after cataract removal, and (3) individuals with normal vision. Congenital cataract reversal individuals were able to perform visually guided gaze shifts, even when their blindness had lasted for decades. The typical extensive nystagmus of this group distorted eye movement trajectories, but measures of latency and accuracy were as expected from their prevailing nystagmus-that is, not worse than in the nystagmus control group. To the best of our knowledge, the present quantitative study is the first to investigate the characteristics of oculomotor control in congenital cataract reversal individuals, and it indicates a remarkable effectiveness of visually guided eye movements despite long-lasting periods of visual deprivation.

Decoupling of Early V5 Motion Processing from Visual Awareness: A Matter of Velocity as Revealed by Transcranial Magnetic Stimulation

Motion information can reach V5/MT through two parallel routes: one conveying information at early latencies through a direct subcortical route and the other reaching V5 later via recurrent projections through V1. Here, we tested the hypothesis that input via the faster direct pathway depends on motion characteristics. To this end, we presented motion stimuli to healthy human observers at different velocities (4.4°/sec vs. 23°/sec) with static stimuli as controls while applying transcranial magnetic stimulation (TMS) pulses over V5 or V1. We probed for TMS interference with objective (two-alternative forced choice [2AFC]) and subjective (awareness) measures of motion processing at six TMS delays from stimulus onset (poststimulus window covered: ∼27–160 msec). Our results for V5–TMS showed earlier interference with objective performance for fast motion (53.3 msec) than slow motion (80 msec) stimuli. Importantly, TMS-induced decreases in objective measures of motion processing did correlate with decreases in subjective measures for slow but not fast motion stimuli. Moreover, V1–TMS induced a temporally unspecific interference with visual processing as it impaired the processing of both motion and static stimuli at the same delays. These results are in accordance with fast moving stimuli reaching V5 through a different route than slow moving stimuli. The differential latencies and coupling to awareness suggest distinct involvement of a direct (i.e., colliculo-extrastriate) connection bypassing V1 depending on stimulus velocity (fast vs. slow). Implication of a direct pathway in the early processing of fast motion may have evolved through its behavioral relevance.

From multisensory integration in peripersonal space to bodily self-consciousness: from statistical regularities to statistical inference

Integrating information across sensory systems is a critical step toward building a cohesive representation of the environment and one's body, and as illustrated by numerous illusions, scaffolds subjective experience of the world and self. In the last years, classic principles of multisensory integration elucidated in the subcortex have been translated into the language of statistical inference understood by the neocortical mantle. Most importantly, a mechanistic systems-level description of multisensory computations via probabilistic population coding and divisive normalization is actively being put forward. In parallel, by describing and understanding bodily illusions, researchers have suggested multisensory integration of bodily inputs within the peripersonal space as a key mechanism in bodily self-consciousness. Importantly, certain aspects of bodily self-consciousness, although still very much a minority, have been recently casted under the light of modern computational understandings of multisensory integration. In doing so, we argue, the field of bodily self-consciousness may borrow mechanistic descriptions regarding the neural implementation of inference computations outlined by the multisensory field. This computational approach, leveraged on the understanding of multisensory processes generally, promises to advance scientific comprehension regarding one of the most mysterious questions puzzling humankind, that is, how our brain creates the experience of a self in interaction with the environment.

Development of the Mechanisms Governing Midbrain Multisensory Integration

The ability to integrate information across multiple senses enhances the brain's ability to detect, localize, and identify external events. This process has been well documented in single neurons in the superior colliculus (SC), which synthesize concordant combinations of visual, auditory, and/or somatosensory signals to enhance the vigor of their responses. This increases the physiological salience of crossmodal events and, in turn, the speed and accuracy of SC-mediated behavioral responses to them. However, this capability is not an innate feature of the circuit and only develops postnatally after the animal acquires sufficient experience with covariant crossmodal events to form links between their modality-specific components. Of critical importance in this process are tectopetal influences from association cortex. Recent findings suggest that, despite its intuitive appeal, a simple generic associative rule cannot explain how this circuit develops its ability to integrate those crossmodal inputs to produce enhanced multisensory responses. The present neurocomputational model explains how this development can be understood as a transition from a default state in which crossmodal SC inputs interact competitively to one in which they interact cooperatively. Crucial to this transition is the operation of a learning rule requiring coactivation among tectopetal afferents for engagement. The model successfully replicates findings of multisensory development in normal cats and cats of either sex reared with special experience. In doing so, it explains how the cortico-SC projections can use crossmodal experience to craft the multisensory integration capabilities of the SC and adapt them to the environment in which they will be used.SIGNIFICANCE STATEMENT The brain's remarkable ability to integrate information across the senses is not present at birth, but typically develops in early life as experience with crossmodal cues is acquired. Recent empirical findings suggest that the mechanisms supporting this development must be more complex than previously believed. The present work integrates these data with what is already known about the underlying circuit in the midbrain to create and test a mechanistic model of multisensory development. This model represents a novel and comprehensive framework that explains how midbrain circuits acquire multisensory experience and reveals how disruptions in this neurotypic developmental trajectory yield divergent outcomes that will affect the multisensory processing capabilities of the mature brain.

Audiovisual Simultaneity Judgment and Rapid Recalibration throughout the Lifespan

Multisensory interactions are well established to convey an array of perceptual and behavioral benefits. One of the key features of multisensory interactions is the temporal structure of the stimuli combined. In an effort to better characterize how temporal factors influence multisensory interactions across the lifespan, we examined audiovisual simultaneity judgment and the degree of rapid recalibration to paired audiovisual stimuli (Flash-Beep and Speech) in a sample of 220 participants ranging from 7 to 86 years of age. Results demonstrate a surprisingly protracted developmental time-course for both audiovisual simultaneity judgment and rapid recalibration, with neither reaching maturity until well into adolescence. Interestingly, correlational analyses revealed that audiovisual simultaneity judgments (i.e., the size of the audiovisual temporal window of simultaneity) and rapid recalibration significantly co-varied as a function of age. Together, our results represent the most complete description of age-related changes in audiovisual simultaneity judgments to date, as well as being the first to describe changes in the degree of rapid recalibration as a function of age. We propose that the developmental time-course of rapid recalibration scaffolds the maturation of more durable audiovisual temporal representations.

Multisensory integration and child neurodevelopment

A considerable number of cognitive processes depend on the integration of multisensory information. The brain integrates this information, providing a complete representation of our surrounding world and giving us the ability to react optimally to the environment. Infancy is a period of great changes in brain structure and function that are reflected by the increase of processing capacities of the developing child. However, it is unclear if the optimal use of multisensory information is present early in childhood or develops only later, with experience. The first part of this review has focused on the typical development of multisensory integration (MSI). We have described the two hypotheses on the developmental process of MSI in neurotypical infants and children, and have introduced MSI and its neuroanatomic correlates. The second section has discussed the neurodevelopmental trajectory of MSI in cognitively-challenged infants and children. A few studies have brought to light various difficulties to integrate sensory information in children with a neurodevelopmental disorder. Consequently, we have exposed certain possible neurophysiological relationships between MSI deficits and neurodevelopmental disorders, especially dyslexia and attention deficit disorder with/without hyperactivity.

The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities

Behavior, perception and cognition are strongly shaped by the synthesis of information across the different sensory modalities. Such multisensory integration often results in performance and perceptual benefits that reflect the additional information conferred by having cues from multiple senses providing redundant or complementary information. The spatial and temporal relationships of these cues provide powerful statistical information about how these cues should be integrated or "bound" in order to create a unified perceptual representation. Much recent work has examined the temporal factors that are integral in multisensory processing, with many focused on the construct of the multisensory temporal binding window - the epoch of time within which stimuli from different modalities is likely to be integrated and perceptually bound. Emerging evidence suggests that this temporal window is altered in a series of neurodevelopmental disorders, including autism, dyslexia and schizophrenia. In addition to their role in sensory processing, these deficits in multisensory temporal function may play an important role in the perceptual and cognitive weaknesses that characterize these clinical disorders. Within this context, focus on improving the acuity of multisensory temporal function may have important implications for the amelioration of the "higher-order" deficits that serve as the defining features of these disorders.

Development of multisensory integration from the perspective of the individual neuron

The ability to use cues from multiple senses in concert is a fundamental aspect of brain function. It maximizes the brain’s use of the information available to it at any given moment and enhances the physiological salience of external events. Because each sense conveys a unique perspective of the external world, synthesizing information across senses affords computational benefits that cannot otherwise be achieved. Multisensory integration not only has substantial survival value but can also create unique experiences that emerge when signals from different sensory channels are bound together. However, neurons in a newborn’s brain are not capable of multisensory integration, and studies in the midbrain have shown that the development of this process is not predetermined. Rather, its emergence and maturation critically depend on cross-modal experiences that alter the underlying neural circuit in such a way that optimizes multisensory integrative capabilities for the environment in which the animal will function.

Subcortical connections of area V4 in the macaque

Area V4 has numerous, topographically organized connections with multiple cortical areas, some of which are important for spatially organized visual processing, and others which seem important for spatial attention. Although the topographic organization of V4's connections with other cortical areas has been established, the detailed topography of its connections with subcortical areas is unclear. We therefore injected retrograde and anterograde tracers in different topographical regions of V4 in nine macaques to determine the organization of its subcortical connections. The injection sites included representations ranging from the fovea to far peripheral eccentricities in both the upper and lower visual fields. The topographically organized connections of V4 included bidirectional connections with four subdivisions of the pulvinar, two subdivisions of the claustrum, and the interlaminar portions of the lateral geniculate nucleus, and efferent projections to the superficial and intermediate layers of the superior colliculus, the thalamic reticular nucleus, and the caudate nucleus. All of these structures have a possible role in spatial attention. The nontopographic, or converging, connections included bidirectional connections with the lateral nucleus of the amygdala, afferent inputs from the dorsal raphe, median raphe, locus coeruleus, ventral tegmentum and nucleus basalis of Meynert, and efferent projections to the putamen. Any role of these structures in attention may be less spatially specific.

Subcortical projections of area V2 in the macaque

To investigate the subcortical efferent connections of visual area V2, we injected tritiated amino acids under electrophysiological control into 15 V2 sites in 14 macaques. The injection sites included the fovea representation as well as representations ranging from central to far peripheral eccentricities in both the upper and lower visual fields. The results indicated that V2 projects topographically to different portions of the inferior and lateral pulvinar and to the superficial and intermediate layers of the superior colliculus. Within the pulvinar, the V2 projections terminated in fields P1, P2, and P4, with the strongest projection being in P2. Central visual field injections in V2 labeled projection zones in P1 and P2, whereas peripheral field injections labeled P1, P2, and P4. No projections were found in P3. Both central and peripheral field injections in V2 projected topographically to the superficial and intermediate layers of the superior colliculus. Projections from V2 to the pulvinar and the superior colliculus constituted cortical-subcortical loops through which circuits serving spatial attention are activated.

Effect of microstimulation of the superior colliculus on visual space attention

We investigated the effect of microstimulation of the superficial layers of the superior colliculus (SC) on the performance of animals in a peripheral detection paradigm while maintaining fixation. In a matching-to-sample paradigm, a sample stimulus was presented at one location followed by a brief test stimulus at that (relevant) location and a distractor at another (irrelevant) location. While maintaining fixation, the monkey indicated whether the sample and the test stimulus matched, ignoring the distractor. The relevant and irrelevant locations were switched from trial to trial. Cells in the superficial layers of SC gave enhanced responses when the attended test stimulus was inside the receptive field compared with when the (physically identical) distractor was inside the field. These effects were found only in an "automatic" attentional cueing paradigm, in which a peripheral stimulus explicitly cued the animal as to the relevant location in the receptive field. No attentional effects were found with block of trials. The transient enhancement to the attended stimulus was observed at the onset and not at the offset of the stimulus. Electrical stimulation at the site corresponding to the irrelevant distractor location in the SC causes it to gain control over attention, causing impaired performance of the task at the relevant location. Stimulation at unattended sites without the presence of a distractor stimulus causes little or no impairment in performance. The effect of stimulation decays with successive stimulations. The animals learn to ignore the stimulation unless the parameters of the task are varied.

Convergent approaches toward the study of multisensory perception

Classical analytical approaches for examining multisensory processing in individual neurons have relied heavily on changes in mean firing rate to assess the presence and magnitude of multisensory interaction. However, neurophysiological studies within individual sensory systems have illustrated that important sensory and perceptual information is encoded in forms that go beyond these traditional spike-based measures. Here we review analytical tools as they are used within individual sensory systems (auditory, somatosensory, and visual) to advance our understanding of how sensory cues are effectively integrated across modalities (e.g., audiovisual cues facilitating speech processing). Specifically, we discuss how methods used to assess response variability (Fano factor, or FF), local field potentials (LFPs), current source density (CSD), oscillatory coherence, spike synchrony, and receiver operating characteristics (ROC) represent particularly promising tools for understanding the neural encoding of multisensory stimulus features. The utility of each approach and how it might optimally be applied toward understanding multisensory processing is placed within the context of exciting new data that is just beginning to be generated. Finally, we address how underlying encoding mechanisms might shape-and be tested alongside with-the known behavioral and perceptual benefits that accompany multisensory processing.

Adult plasticity of spatiotemporal receptive fields of multisensory superior colliculus neurons following early visual deprivation

PURPOSE

Previous work has established that the integrative capacity of multisensory neurons in the superior colliculus (SC) matures over a protracted period of postnatal life (Wallace and Stein, 1997), and that the development of normal patterns of multisensory integration depends critically on early sensory experience (Wallace et al., 2004). Although these studies demonstrated the importance of early sensory experience in the creation of mature multisensory circuits, it remains unknown whether the reestablishment of sensory experience in adulthood can reverse these effects and restore integrative capacity.

METHODS

The current study tested this hypothesis in cats that were reared in absolute darkness until adulthood and then returned to a normal housing environment for an equivalent period of time. Single unit extracellular recordings targeted multisensory neurons in the deep layers of the SC, and analyses were focused on both conventional measures of multisensory integration and on more recently developed methods designed to characterize spatiotemporal receptive fields (STRF).

RESULTS

Analysis of the STRF structure and integrative capacity of multisensory SC neurons revealed significant modifications in the temporal response dynamics of multisensory responses (e.g., discharge durations, peak firing rates, and mean firing rates), as well as significant changes in rates of spontaneous activation and degrees of multisensory integration.

CONCLUSIONS

These results emphasize the importance of early sensory experience in the establishment of normal multisensory processing architecture and highlight the limited plastic potential of adult multisensory circuits.

Perceptual training narrows the temporal window of multisensory binding

The brain's ability to bind incoming auditory and visual stimuli depends critically on the temporal structure of this information. Specifically, there exists a temporal window of audiovisual integration within which stimuli are highly likely to be bound together and perceived as part of the same environmental event. Several studies have described the temporal bounds of this window, but few have investigated its malleability. Here, the plasticity in the size of this temporal window was investigated using a perceptual learning paradigm in which participants were given feedback during a two-alternative forced choice (2-AFC) audiovisual simultaneity judgment task. Training resulted in a marked (i.e., approximately 40%) narrowing in the size of the window. To rule out the possibility that this narrowing was the result of changes in cognitive biases, a second experiment using a two-interval forced choice (2-IFC) paradigm was undertaken during which participants were instructed to identify a simultaneously presented audiovisual pair presented within one of two intervals. The 2-IFC paradigm resulted in a narrowing that was similar in both degree and dynamics to that using the 2-AFC approach. Together, these results illustrate that different methods of multisensory perceptual training can result in substantial alterations in the circuits underlying the perception of audiovisual simultaneity. These findings suggest a high degree of flexibility in multisensory temporal processing and have important implications for interventional strategies that may be used to ameliorate clinical conditions (e.g., autism, dyslexia) in which multisensory temporal function may be impaired.

Superadditivity in multisensory integration: putting the computation in context

Single-neuron studies have highlighted dramatic enhancements in neural activity consequent to multisensory integration. Most notable are 'superadditive' enhancements in which the multisensory response exceeds the sum of those evoked by the modality-specific stimulus components individually. Although all multisensory enhancements may have perceptual/behavioral consequences, superadditivity, which suggests a nonlinear combination of modality-specific influences, seems to have had a disproportionate influence within the multisensory literature. This influence has been reinforced by the increasing application of noninvasive techniques such as functional imaging and event-related potential recording, which depend on response nonlinearities to demonstrate underlying multisensory processes. In promoting the idea that many multisensory behaviors may not rely on superadditivity, we consider more recent single-neuron studies that place its incidence in context.

Most superficial sublamina of rat superior colliculus: neuronal response properties and correlates with perceptual figure-ground segregation

The uppermost layer (stratum griseum superficiale, SGS) of the superior colliculus (SC) provides an important gateway from the retina to the visual extrastriate and visuomotor systems. The majority of attention has been given to the role of this "visual" SC in saccade generation and target selection and it is generally considered to be less important in visual perception. We have found, however, that in the rat SGS1, the most superficial division of the SGS, the neurons perform very sophisticated analysis of visual information. First, in studying their responses with a variety of flashing stimuli we found that the neurons respond not to brightness changes per se, but to the appearance and/or disappearance of visual shapes in their receptive fields (RFs). Contrary to conventional RFs of neurons at the early stages of visual processing, the RFs in SGS1 cannot be described in terms of fixed spatial distribution of excitatory and inhibitory inputs. Second, SGS1 neurons showed robust orientation tuning to drifting gratings and orientation-specific modulation of the center response from surround. These are features previously seen only in visual cortical neurons and are considered to be involved in "contour" perception and figure-ground segregation. Third, responses of SGS1 neurons showed complex dynamics; typically the response tuning became progressively sharpened with repetitive grating periods. We conclude that SGS1 neurons are involved in considerably more complex analysis of retinal input than was previously thought. SGS1 may participate in early stages of figure-ground segregation and have a role in low-resolution nonconscious vision as encountered after visual decortication.

Spatio-temporal receptive field properties of cells in the rat superior colliculus

Although the rat is widely used in neurobehavioural research, the spatio-temporal receptive field properties of neurons in superficial layers of the superior colliculus are relatively unknown. Extracellular recordings were carried out in anesthetized Long Evans rats. Neurons in these layers had simple-like and complex-like receptive fields (RFs). Most cells (67%) had RFs showing band-pass and low-pass spatial frequency (SF) tuning profiles. Spatial band-pass profiles showed low optimal SF (mean=0.03 c/deg), low spatial resolution (mean=0.18 c/deg) and large spatial bandwidths (mean=2.3 octaves). More than two-thirds of the RFs (71%) were selective to orientation and only 11% were clearly direction selective. Nearly two-thirds of cells (68%) had band-pass temporal frequency (TF) tuning profiles with narrow bandwidths (mean=1.7 oct.) whereas the others showed low-pass TF tuning profiles. Temporal band-pass profiles had low optimal TFs (mean=3.5 c/s). Although some cells showed relatively low contrast thresholds (6%), most cells only responded to high contrast values (mean=38.2%). These results show that the spatial resolution of collicular cells is poor and that they respond mainly to highly contrasted moving stimuli.

Multisensory integration for timing engages different brain networks

How does the brain integrate information from different senses into a unitary percept? What factors influence such multisensory integration? Using a rhythmic behavioral paradigm and functional magnetic resonance imaging, we identified networks of brain regions for perceptions of physically synchronous and asynchronous auditory-visual events. Measures of behavioral performance revealed the existence of three distinct perceptual states. Perception of asynchrony activated a network of the primary sensory, prefrontal, and inferior parietal cortices, perception of synchrony disengaged the inferior parietal cortex and further recruited the superior colliculus, and when no clear percept was established, only the residual areas comprised of prefrontal and sensory areas were active. These results indicate that distinct percepts arise within specific brain sub-networks, the components of which are differentially engaged and disengaged depending on the timing of environmental signals.

Multisensory orientation behavior is disrupted by neonatal cortical ablation

The integration of visual and auditory information can significantly amplify the sensory responses of superior colliculus (SC) neurons and the behaviors that depend on them. This response amplification depends on the development of SC inputs that are derived from two regions of cortex: the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS). Neonatal ablation of these cortico-collicular areas has been shown to disrupt the development of the multisensory enhancement capabilities of SC neurons and the present results demonstrate that it also precludes the development of the normal multisensory enhancements in orientation behavior. Animals with neonatal ablation of AES and rLS were tested at maturity and found unable to benefit from the combination of visual and auditory cues in their efforts to localize targets in contralesional space. In contrast, their ipsilesional multisensory orientation capabilities were indistinguishable from those of normal animals. However, when only one of these cortical areas was removed during early life, later behavioral consequences were negligible. Whether similar compensatory processes would occur in adult animals remains to be determined. These observations, coupled with those from previous studies, also suggest that a surprisingly high proportion of SC neurons capable of multisensory integration must be present for orientation behavior benefits to be realized. Compensatory mechanisms can achieve this if early lesions spare either AES or rLS, but even the impressive plasticity of the neonatal brain cannot compensate for the early loss of both of them.

Neonatal cortical ablation disrupts multisensory development in superior colliculus

The ability of cat superior colliculus (SC) neurons to synthesize information from different senses depends on influences from two areas of the cortex: the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS). Reversibly deactivating the inputs to the SC from either of these areas in normal adults severely compromises this ability and the SC-mediated behaviors that depend on it. In this study, we found that removal of these areas in neonatal animals precluded the normal development of multisensory SC processes. At maturity there was a substantial decrease in the incidence of multisensory neurons, and those multisensory neurons that did develop were highly abnormal. Their cross-modal receptive field register was severely compromised, as was their ability to integrate cross-modal stimuli. Apparently, despite the impressive plasticity of the neonatal brain, it cannot compensate for the early loss of these cortices. Surprisingly, however, neonatal removal of either AES or rLS had comparatively minor consequences on these properties. At maturity multisensory SC neurons were quite common: they developed the characteristic spatial register among their unisensory receptive fields and exhibited normal adult-like multisensory integration. These observations suggest that during early ontogeny, when the multisensory properties of SC neurons are being crafted, AES and rLS may have the ability to compensate for the loss of one another's cortico-collicular influences so that normal multisensory processes can develop in the SC.

Neural mechanisms of stimulus velocity tuning in the superior colliculus

Superior colliculus (SC)-mediated control of visuomotor behavior depends on neuronal selectivity for stimulus velocity. However, the mechanism responsible for velocity tuning in SC neurons is unclear. It was shown in a previous study of anesthetized, decorticate hamsters that the number and distribution of feed-forward retinal inputs are not critical for velocity tuning. Here the alternate hypothesis that inhibition from the surround determines velocity tuning of SC neurons was tested. Surround inhibition was present in 65% (43/66) of SC neurons recorded in the superficial gray layer. Neurons within this group that were selective for slowly moving stimuli exhibited spatially asymmetric surround inhibition, and their velocity tuning arose by preferential suppression of responses to rapidly moving stimuli. In the other 35% (23/66) of SC neurons recorded, surround inhibition was weak or absent and did not play a role in velocity tuning. Most neurons with surround inhibition were nonselective for the duration of stationary flashed stimuli, whereas neurons without surround inhibition were selective for stimulus duration. The majority of neurons that preferred intermediate or rapidly moving stimuli exhibited spatially symmetric surround inhibition. In these neurons, occluding the surround reduced velocity selectivity by enhancing responses to slowly moving stimuli. Based on these data, a model is proposed suggesting spatiotemporal interactions between inhibition and excitation that could underlie velocity tuning.

Dynamic response properties of visual neurons and context-dependent surround effects on receptive fields in the tectum of the salamander Plethodon shermani

Neuronal responses to complex prey-like stimuli and rectangles were investigated in the tectum of the salamander Plethodon shermani using extracellular single-cell recording. Cricket dummies differing in size, contrast or movement pattern or a rectangle were moved singly through the excitatory receptive field of a neuron. Paired presentations were performed, in which a reference stimulus was moved inside and the different cricket dummies or the rectangle outside the excitatory receptive field. Visual object recognition involves much more complex spatial and temporal processing than previously assumed in amphibians. This concerns significant changes in absolute number of spikes, temporal discharge pattern, and receptive field size. At single presentation of stimuli, the number of discharges was significantly changed compared with the reference stimulus, and in the majority of neurons the temporal pattern of discharges was changed in addition. At paired presentation of stimuli, neurons mainly revealed a significant decrease in average spike number and a reduction of excitatory receptive field size to presentation of the reference stimulus inside the excitatory receptive field, when a large-sized cricket stimulus or the rectangle was located outside the excitatory receptive field. This inhibition was significantly greater for the large-sized cricket stimulus than for the rectangle, and indicates the biological relevance of the prey-like stimulus in object selection. The response properties of tectal neurons at single or paired presentation of stimuli indicate that tectal neurons integrate information across a much larger part of visual space than covered by the excitatory receptive field. The spike number of a tectal neuron and the spatio-temporal extent of its excitatory receptive field are not fixed but depend on the context, i.e. the stimulus type and combination. This dynamic processing corresponds with the selection of the stimuli in the visual orienting behavior of Plethodon investigated in a previous study, and we assume that tectal processing is modulated by top down processes as well as feedback circuitries.

The integration of auditory and visual motion signals at threshold

To interpret our environment, we integrate information from all our senses. For moving objects, auditory and visual motion signals are correlated and provide information about the speed and the direction of the moving object. We investigated at what level the auditory and the visual modalities interact and whether the human brain integrates only motion signals that are ecologically valid. We found that the sensitivity for identifying motion was improved when motion signals were provided in both modalities. This improvement in sensitivity can be explained by probability summation. That is, auditory and visual stimuli are combined at a decision level, after the stimuli have been processed independently in the auditory and the visual pathways. Furthermore, this integration is direction blind and is not restricted to ecologically valid motion signals.

Functional connectivity between the superficial and deeper layers of the superior colliculus: an anatomical substrate for sensorimotor integration

The superior colliculus (SC) transforms both visual and nonvisual sensory signals into motor commands that control orienting behavior. Although the afferent and efferent connections of this midbrain nucleus have been well characterized, little is know about the intrinsic circuitry involved in sensorimotor integration. Transmission of visual signals from the superficial (sSC) to the deeper layers (dSC) of the SC has been implicated in both the triggering of orienting movements and the activity-dependent processes that align maps of different sensory modalities during development. However, evidence for the synaptic connectivity appropriate for these functions is lacking. In this study, we used a variety of anatomical and physiological methods to examine the functional organization of the sSC-dSC pathway in juvenile and adult ferrets. Axonal tracing in adult ferrets showed that, as in other species, sSC neurons project topographically to the dSC, providing a route for the transmission of visual signals to the multisensory output layers of the SC. We found that sSC axons terminate on dSC neurons that stain prominently for the NR1 subunit of the NMDA receptor, a subpopulation of which were identified as tectoreticulospinal projection neurons. We also show that the sSC-dSC pathway is topographically organized and mediated by monosynaptic excitatory synapses even before eye opening in young ferrets, suggesting that visual signals routed via the sSC may influence the activity of dSC neurons before the emergence of their multisensory response properties. These findings indicate that superficial- to deep-layer projections provide spatially ordered visual signals, both during development and into adulthood, directly to SC neurons that are involved in coordinating sensory inputs with motor outputs.

Learning innate face preferences

Newborn humans preferentially orient to facelike patterns at birth, but months of experience with faces are required for full face processing abilities to develop. Several models have been proposed for how the interaction of genetic and environmental influences can explain these data. These models generally assume that the brain areas responsible for newborn orienting responses are not capable of learning and are physically separate from those that later learn from real faces. However, it has been difficult to reconcile these models with recent discoveries of face learning in newborns and young infants. We propose a general mechanism by which genetically specified and environment-driven preferences can coexist in the same visual areas. In particular, newborn face orienting may be the result of prenatal exposure of a learning system to internally generated input patterns, such as those found in PGO waves during REM sleep. Simulating this process with the HLISSOM biological model of the visual system, we demonstrate that the combination of learning and internal patterns is an efficient way to specify and develop circuitry for face perception. This prenatal learning can account for the newborn preferences for schematic and photographic images of faces, providing a computational explanation for how genetic influences interact with experience to construct a complex adaptive system.

Interhemispheric neural summation in the split brain with symmetrical and asymmetrical displays

The present study, investigates interhemispheric integration in the split brain. Four split-brained, two acallosal and 14 normal subjects carried out a simple reaction time task in which they responded to stimuli presented either singly in the left visual field, singly in the right visual field, or simultaneously in both visual fields. Stimuli were white against a black background and bilateral stimuli were either symmetrical or asymmetrical around the central vertical meridian. For unilateral stimuli, the difference in response time (RT) between crossed and uncrossed hand-field combinations (crossed-uncrossed difference, or CUD) measured interhemispheric transfer time. RTs to bilateral and unilateral stimulus displays were compared to provide a measure of redundancy gain (RG). Normal subjects exhibited small CUDs and RGs. Split-brained and acallosal subjects were found to have much longer CUDs, and to show enhanced RGs which could not be explained by a probability (race) model, implying subcortical neural summation. This summation did not depend on the preservation of symmetry, suggesting that it may not occur at the retinotopically organized superior colliculus, but at another site such as the pons or reticular formation.

Sensory and multisensory responses in the newborn monkey superior colliculus

Superior colliculus (SC) neurons have the ability to synthesize information from different sensory modalities, resulting in enhancements (or depressions) of their activity. This physiological capacity is, in turn, closely tied to changes in overt attentive and orientation responses. The present study shows that, in contrast to more altricial species, many deep layer SC neurons in the rhesus monkey are multisensory at birth. Such neurons can respond to stimuli from different sensory modalities, and all convergence patterns seen in the adult are represented. Nevertheless, these neurons cannot yet synthesize their multisensory inputs. Rather, they respond to combinations of cross-modal stimuli much like they respond to their individual modality-specific components. This immature property of multisensory neurons is in contrast to many of the surprisingly sophisticated modality-specific response properties of these neurons and of their modality-specific neighbors. Thus, although deep SC neurons in the newborn have longer latencies and larger receptive fields than their adult counterparts, they are already highly active and are distributed in the typical adult admixture of visual, auditory, somatosensory, and multisensory neurons. Furthermore, the receptive fields of these neurons are already ordered into well organized topographic maps, and the different receptive fields of the same multisensory neurons show a good degree of cross-modal spatial register. These data, coupled with those from cat, suggest that the capacity to synthesize multisensory information does not simply appear in SC neurons at a prescribed maturational stage but rather develops only after substantial experience with cross-modal cues.

Cross-modal integration of auditory and visual motion signals

Real-world moving objects are usually defined by correlated information in multiple sensory modalities such as vision and hearing. The aim of our study was to assess whether simultaneous auditory supra-threshold motion introduces a bias or affects the sensitivity in a visual motion detection task. We demonstrate a bias in the perceived direction of visual motion that is consistent with the direction of the auditory motion (audio-visual motion capture). This bias effect is robust and occurs even if the auditory and visual motion signals come from different locations or move at different speeds. We also show that visual motion detection thresholds are higher for consistent auditory motion than for inconsistent motion, provided the stimuli move at the same speed and are co-localised.

Molecular evidence for the early specification of presumptive functional domains in the embryonic primate cerebral cortex

To identify molecules that may play a role in the initiation of cerebral cortical area formation, we examined the expression of the Eph receptors and their ligands, the ephrins, during primate corticogenesis. We selected the macaque monkey neocortex because of its clear areal subdivisions, large surface area, protracted development (gestation = 165 d), and similarity to the human brain. In situ hybridizations, performed at early [embryonic day 65 (E65)], middle (E80), and late (E95) stages of cortical development, revealed that EphA system family members are expressed in distinct gradients and laminar and areal domains in the embryonic neocortex. Indeed, several regionally restricted molecular patterns are already apparent within the cortical plate at E65, before the formation of thalamocortical connections, suggesting that the initial expression of some EphA system members is regulated by programs intrinsic to cortical cells. For example, EphA3, EphA6, and EphA7 are all selectively expressed within the presumptive visual cortex. However, although EphA6 and EphA7 are present throughout this region, EphA3 is only expressed in the prospective extrastriate cortex, suggesting that cortical cells harbor functional biases that may influence the formation of appropriate synaptic connections. Although several patterns of early gene expression are stable (e.g., EphA3, EphA4, and EphA6), others change as development proceeds (e.g., EphA5, EphA7, ephrin-A2, ephrin-A3, and ephrin-A5), perhaps responding to extrinsic cues. Thus, at E95, after connections between the cortical plate and thalamus have formed, receptor subtypes EphA3, EphA5, EphA6, and EphA7 and the ligand ephrin-A5 are expressed in posterior regions, whereas EphA4 and ephrin-A2 and ephrin-A3 are either uniformly distributed or anteriorly biased. Taken together, our results demonstrate molecular distinctions among cells of the embryonic primate neocortex, revealing hitherto unrecognized compartmentalization early in corticogenesis.

Maturation of visual receptive field properties in the rat superior colliculus

Visually responsive neurons were recorded in the superficial layers of rat superior colliculus from postnatal day 12 to 28. Receptive field properties such as size, type (ON, OFF, ON-OFF and motion sensitive) and direction selectivity were analyzed to disclose changes during maturation. Although some aspects of sensory properties are modified during development (latency, receptive field sizes, and proportions of receptive field types), a high level of sophistication is also present in young animals even before eyelid opening. For instance, direction selective and direction biased cells, which require complex synaptic relations, are already observed when the first light evoked responses emerge in the superior colliculus (P13), strongly suggesting that this property develops without visual experience. Furthermore, direction selectivity is present in the colliculus prior to the appearance of visually evoked activity in the cortex. This indicates that direction selectivity can not be attributable to incoming cortical afferents. This study provides the first direct evidence that, unlike the cat, the rat's cortico-tectal pathway is only weakly involved in the establishment of direction selectivity in collicular neurons.

Signals from the superficial layers of the superior colliculus enable the development of the auditory space map in the deeper layers

We have examined whether the superficial layers of the superior colliculus (SC) provide the source of visual signals that guide the development of the auditory space map in the deeper layers. Anatomical tracing experiments with fluorescent microspheres revealed that a retinotopic map is present in the newborn ferret SC. Aspiration of the caudal region of the superficial layers of the right SC on postnatal day 0 did not cause a reorganization of this projection. Consequently, recordings made when the animals were mature showed that visual units in the remaining superficial layers in rostral SC had receptive fields that spanned a restricted region of anterior space. Auditory units recorded beneath the remaining superficial layers were tuned to corresponding anterior locations. Both the superficial layer visual map and the deeper layer auditory map were normal in the left, unoperated SC. The majority of auditory units recorded throughout the deeper layers ventral to the superficial layer lesion were also tuned to single sound directions. In this region of the SC, however, we observed much greater scatter in the distribution of preferred sound directions and a significant increase in the proportion of units with spatially ambiguous responses. The auditory representation was degraded, although many of these units were also visually responsive. Equivalent lesions of the superficial layers made in adult ferrets did not alter the topographic order in the auditory representation, suggesting that visual activity in these layers may be involved in aligning the different sensory maps in the developing SC.