Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane

Neural latencies across auditory cortex of macaque support a dorsal stream supramodal timing advantage in primates

Sensory systems across the brain are specialized for their input, yet some principles of neural organization are conserved across modalities. The pattern of anatomical connections from the primate auditory cortex to the temporal, parietal, and prefrontal lobes suggests a possible division into dorsal and ventral auditory processing streams, with the dorsal stream originating from more caudal areas of the auditory cortex, and the ventral stream originating from more rostral areas. These streams are hypothesized to be analogous to the well-established dorsal and ventral streams of visual processing. In the visual system, the dorsal processing stream shows substantially faster neural response latencies than does the ventral stream. However, the relative timing of putative dorsal and ventral stream processing has yet to be explored in other sensory modalities. Here, we compare distributions of neural response latencies from 10 different areas of macaque auditory cortex, confirmed by individual anatomical reconstructions, to determine whether a similar timing advantage is found for the hypothesized dorsal auditory stream. Across three varieties of auditory stimuli (clicks, noise, and pure tones), we find that latencies increase with hierarchical level, as predicted by anatomical connectivity. Critically, we also find a pronounced timing differential along the caudal-to-rostral axis within the same hierarchical level, with caudal (dorsal stream) latencies being faster than rostral (ventral stream) latencies. This observed timing differential mirrors that found for the dorsal stream of the visual system, suggestive of a common timing advantage for the dorsal stream across sensory modalities.

Unravelling cerebellar pathways with high temporal precision targeting motor and extensive sensory and parietal networks

Increasing evidence has implicated the cerebellum in providing forward models of motor plants predicting the sensory consequences of actions. Assuming that cerebellar input to the cerebral cortex contributes to the cerebro-cortical processing by adding forward model signals, we would expect to find projections emphasising motor and sensory cortical areas. However, this expectation is only partially met by studies of cerebello-cerebral connections. Here we show that by electrically stimulating the cerebellar output and imaging responses with functional magnetic resonance imaging, evoked blood oxygen level-dependant activity is observed not only in the classical cerebellar projection target, the primary motor cortex, but also in a number of additional areas in insular, parietal and occipital cortex, including sensory cortical representations. Further probing of the responses reveals a projection system that has been optimized to mediate fast and temporarily precise information. In conclusion, both the topography of the stimulation effects and its emphasis on temporal precision are in full accordance with the concept of cerebellar forward model information modulating cerebro-cortical processing.

Cross-modal recruitment of primary visual cortex by auditory stimuli in the nonhuman primate brain: a molecular mapping study

Recent studies suggest that exposure to only one component of audiovisual events can lead to cross-modal cortical activation. However, it is not certain whether such crossmodal recruitment can occur in the absence of explicit conditioning, semantic factors, or long-term associations. A recent study demonstrated that crossmodal cortical recruitment can occur even after a brief exposure to bimodal stimuli without semantic association. In addition, the authors showed that the primary visual cortex is under such crossmodal influence. In the present study, we used molecular activity mapping of the immediate early gene zif268. We found that animals, which had previously been exposed to a combination of auditory and visual stimuli, showed increased number of active neurons in the primary visual cortex when presented with sounds alone. As previously implied, this crossmodal activation appears to be the result of implicit associations of the two stimuli, likely driven by their spatiotemporal characteristics; it was observed after a relatively short period of exposure (~45 min) and lasted for a relatively long period after the initial exposure (~1 day). These results suggest that the previously reported findings may be directly rooted in the increased activity of the neurons occupying the primary visual cortex.

Thalamic connections of auditory cortex in marmoset monkeys: lateral belt and parabelt regions

The primate auditory cortex is comprised of a core region of three primary areas, surrounded by a belt region of secondary areas and a parabelt region lateral to the belt. The main sources of thalamocortical inputs to the auditory cortex are the medial geniculate complex (MGC), medial pulvinar (PM), and several adjoining nuclei in the posterior thalamus. The distribution of inputs varies topographically by cortical area and thalamic nucleus, but in a manner that has not been fully characterized in primates. In this study, the thalamocortical connections of the lateral belt and parabelt were determined by placing retrograde tracer injections into various areas of these regions in the marmoset monkey. Both regions received projections from the medial (MGm) and posterodorsal (MGpd) divisions of the medial geniculate complex (MGC); however, labeled cells in the anterodorsal (MGad) division were present only from injections into the caudal belt. Thalamic inputs to the lateral belt appeared to come mainly from the MGC, whereas the parabelt also received a strong projection from the PM, consistent with its position as a later stage of auditory cortical processing. The results of this study also indicate that the organization of the marmoset auditory cortex is similar to other primates.

Activity related to perceptual judgment and action in primary auditory cortex

Recent evidence is reshaping the view of primary auditory cortex (A1) from a unisensory area to one more involved in dynamically integrating multisensory- and task-related information. We found A1 single- (SU) and multiple-unit (MU) activity correlated with macaques' choices in an amplitude modulation (AM) discrimination task. Animals were trained to discriminate AM noise from unmodulated noise by releasing a lever for AM noise and holding down the lever for unmodulated noise. Activity for identical stimuli was compared between trials where the animals reported AM and trials where they did not. We found 47.4% of MUs and 22.8% of SUs significantly increased firing shortly before the animal's behavioral response to report AM when compared to the equivalent time period on trials where AM was not reported. Activity was also linked to lever release in a different task context, suggesting A1 modulation by somatosensory, or efference copy, input. When spikes were counted only during the stimulus, 19.6% of MUs and 13.8% of SUs increased firing rate when animals reported AM compared to when they did not, suggesting an attentional effect, or that A1 activity can be used by higher decision areas, or that such areas provide feedback to A1. Activity associated with AM reporting was correlated with a unit's AM sensitivity, suggesting AM sensitive neurons' involvement in task performance. A1 neurons' phase locking to AM correlated more weakly (compared to firing rate) with the animals' report of AM, suggesting a preferential role for rate-codes in A1 for this AM discrimination task.

Effects of parietal TMS on visual and auditory processing at the primary cortical level -- a concurrent TMS-fMRI study

Accumulating evidence suggests that multisensory interactions emerge already at the primary cortical level. Specifically, auditory inputs were shown to suppress activations in visual cortices when presented alone but amplify the blood oxygen level-dependent (BOLD) responses to concurrent visual inputs (and vice versa). This concurrent transcranial magnetic stimulation-functional magnetic resonance imaging (TMS-fMRI) study applied repetitive TMS trains at no, low, and high intensity over right intraparietal sulcus (IPS) and vertex to investigate top-down influences on visual and auditory cortices under 3 sensory contexts: visual, auditory, and no stimulation. IPS-TMS increased activations in auditory cortices irrespective of sensory context as a result of direct and nonspecific auditory TMS side effects. In contrast, IPS-TMS modulated activations in the visual cortex in a state-dependent fashion: it deactivated the visual cortex under no and auditory stimulation but amplified the BOLD response to visual stimulation. However, only the response amplification to visual stimulation was selective for IPS-TMS, while the deactivations observed for IPS- and Vertex-TMS resulted from crossmodal deactivations induced by auditory activity to TMS sounds. TMS to IPS may increase the responses in visual (or auditory) cortices to visual (or auditory) stimulation via a gain control mechanism or crossmodal interactions. Collectively, our results demonstrate that understanding TMS effects on (uni)sensory processing requires a multisensory perspective.

Phoneme and word recognition in the auditory ventral stream

Spoken word recognition requires complex, invariant representations. Using a meta-analytic approach incorporating more than 100 functional imaging experiments, we show that preference for complex sounds emerges in the human auditory ventral stream in a hierarchical fashion, consistent with nonhuman primate electrophysiology. Examining speech sounds, we show that activation associated with the processing of short-timescale patterns (i.e., phonemes) is consistently localized to left mid-superior temporal gyrus (STG), whereas activation associated with the integration of phonemes into temporally complex patterns (i.e., words) is consistently localized to left anterior STG. Further, we show left mid- to anterior STG is reliably implicated in the invariant representation of phonetic forms and that this area also responds preferentially to phonetic sounds, above artificial control sounds or environmental sounds. Together, this shows increasing encoding specificity and invariance along the auditory ventral stream for temporally complex speech sounds.

Audiotactile interactions in temporal perception

In the present review, we focus on how commonalities in the ontogenetic development of the auditory and tactile sensory systems may inform the interplay between these signals in the temporal domain. In particular, we describe the results of behavioral studies that have investigated temporal resolution (in temporal order, synchrony/asynchrony, and simultaneity judgment tasks), as well as temporal numerosity perception, and similarities in the perception of frequency across touch and hearing. The evidence reviewed here highlights features of audiotactile temporal perception that are distinctive from those seen for other pairings of sensory modalities. For instance, audiotactile interactions are characterized in certain tasks (e.g., temporal numerosity judgments) by a more balanced reciprocal influence than are other modality pairings. Moreover, relative spatial position plays a different role in the temporal order and temporal recalibration processes for audiotactile stimulus pairings than for other modality pairings. The effect exerted by both the spatial arrangement of stimuli and attention on temporal order judgments is described. Moreover, a number of audiotactile interactions occurring during sensory-motor synchronization are highlighted. We also look at the audiotactile perception of rhythm and how it may be affected by musical training. The differences emerging from this body of research highlight the need for more extensive investigation into audiotactile temporal interactions. We conclude with a brief overview of some of the key issues deserving of further research in this area.

Thalamic influences on multisensory integration

In everyday life our brain often receives information about events and objects in the real world via several sensory modalities, because natural objects often stimulate more than one sense. These different types of information are processed in our brain along different sensory-specific pathways, but are finally integrated into a unified percept. During the last years, studies provided compelling evidence that the neural basis of multisensory integration is not restricted to higher association areas of the cortex, but can already occur at low-level stages of sensory cortical processing and even in subcortical structures. In this article we will review the potential role of several thalamic structures in multisensory interplay and discuss their extensive anatomical connections with sensory-specific and multisensory cortical structures. We conclude that sensory-specific thalamic structures may act as a crucial processing node of multisensory interplay in addition to their traditional role as sensory relaying structure.

Current perspectives and methods in studying neural mechanisms of multisensory interactions

In the past decade neuroscience has witnessed major advances in the field of multisensory interactions. A large body of research has revealed several new types of cross-sensory interactions. In addition, multisensory interactions have been reported at temporal and spatial system levels previously thought of as strictly unimodal. We review the findings that have led to the current broad consensus that most, if not all, higher, as well as lower level neural processes are in some form multisensory. We continue by outlining the progress that has been made in identifying the functional significance of different types of interactions, for example, in subserving stimulus binding and enhancement of perceptual certainty. Finally, we provide a critical introduction to cutting edge methods from bayes optimal integration to multivoxel pattern analysis as applied to multisensory research at different system levels.

Multisensory facilitation of behavior in monkeys: effects of stimulus intensity

Multisensory stimuli can improve performance, facilitating RTs on sensorimotor tasks. This benefit is referred to as the redundant signals effect (RSE) and can exceed predictions on the basis of probability summation, indicative of integrative processes. Although an RSE exceeding probability summation has been repeatedly observed in humans and nonprimate animals, there are scant and inconsistent data from nonhuman primates performing similar protocols. Rather, existing paradigms have instead focused on saccadic eye movements. Moreover, the extant results in monkeys leave unresolved how stimulus synchronicity and intensity impact performance. Two trained monkeys performed a simple detection task involving arm movements to auditory, visual, or synchronous auditory-visual multisensory pairs. RSEs in excess of predictions on the basis of probability summation were observed and thus forcibly follow from neural response interactions. Parametric variation of auditory stimulus intensity revealed that in both animals, RT facilitation was limited to situations where the auditory stimulus intensity was below or up to 20 dB above perceptual threshold, despite the visual stimulus always being suprathreshold. No RT facilitation or even behavioral costs were obtained with auditory intensities 30-40 dB above threshold. The present study demonstrates the feasibility and the suitability of behaving monkeys for investigating links between psychophysical and neurophysiologic instantiations of multisensory interactions.

VGLUT1 and VGLUT2 mRNA expression in the primate auditory pathway

The vesicular glutamate transporters (VGLUTs) regulate the storage and release of glutamate in the brain. In adult animals, the VGLUT1 and VGLUT2 isoforms are widely expressed and differentially distributed, suggesting that neural circuits exhibit distinct modes of glutamate regulation. Studies in rodents suggest that VGLUT1 and VGLUT2 mRNA expression patterns are partly complementary, with VGLUT1 expressed at higher levels in the cortex and VGLUT2 prominent subcortically, but with overlapping distributions in some nuclei. In primates, VGLUT gene expression has not been previously studied in any part of the brain. The purposes of the present study were to document the regional expression of VGLUT1 and VGLUT2 mRNA in the auditory pathway through A1 in the cortex, and to determine whether their distributions are comparable to rodents. In situ hybridization with antisense riboprobes revealed that VGLUT2 was strongly expressed by neurons in the cerebellum and most major auditory nuclei, including the dorsal and ventral cochlear nuclei, medial and lateral superior olivary nuclei, central nucleus of the inferior colliculus, sagulum, and all divisions of the medial geniculate. VGLUT1 was densely expressed in the hippocampus and ventral cochlear nuclei, and at reduced levels in other auditory nuclei. In the auditory cortex, neurons expressing VGLUT1 were widely distributed in layers II-VI of the core, belt and parabelt regions. VGLUT2 was expressed most strongly by neurons in layers IIIb and IV, weakly by neurons in layers II-IIIa, and at very low levels in layers V-VI. The findings indicate that VGLUT2 is strongly expressed by neurons at all levels of the subcortical auditory pathway, and by neurons in the middle layers of the cortex, whereas VGLUT1 is strongly expressed by most if not all glutamatergic neurons in the auditory cortex and at variable levels among auditory subcortical nuclei. These patterns imply that VGLUT2 is the main vesicular glutamate transporter in subcortical and thalamocortical (TC) circuits, whereas VGLUT1 is dominant in corticocortical (CC) and corticothalamic (CT) systems of projections. The results also suggest that VGLUT mRNA expression patterns in primates are similar to rodents, and establish a baseline for detailed studies of these transporters in selected circuits of the auditory system.

Onset timing of cross-sensory activations and multisensory interactions in auditory and visual sensory cortices

Here we report early cross-sensory activations and audiovisual interactions at the visual and auditory cortices using magnetoencephalography (MEG) to obtain accurate timing information. Data from an identical fMRI experiment were employed to support MEG source localization results. Simple auditory and visual stimuli (300-ms noise bursts and checkerboards) were presented to seven healthy humans. MEG source analysis suggested generators in the auditory and visual sensory cortices for both within-modality and cross-sensory activations. fMRI cross-sensory activations were strong in the visual but almost absent in the auditory cortex; this discrepancy with MEG possibly reflects the influence of acoustical scanner noise in fMRI. In the primary auditory cortices (Heschl's gyrus) the onset of activity to auditory stimuli was observed at 23 ms in both hemispheres, and to visual stimuli at 82 ms in the left and at 75 ms in the right hemisphere. In the primary visual cortex (Calcarine fissure) the activations to visual stimuli started at 43 ms and to auditory stimuli at 53 ms. Cross-sensory activations thus started later than sensory-specific activations, by 55 ms in the auditory cortex and by 10 ms in the visual cortex, suggesting that the origins of the cross-sensory activations may be in the primary sensory cortices of the opposite modality, with conduction delays (from one sensory cortex to another) of 30-35 ms. Audiovisual interactions started at 85 ms in the left auditory, 80 ms in the right auditory and 74 ms in the visual cortex, i.e., 3-21 ms after inputs from the two modalities converged.

Separate mechanisms for audio-tactile pitch and loudness interactions

A major goal in perceptual neuroscience is to understand how signals from different sensory modalities are combined to produce stable and coherent representations. We previously investigated interactions between audition and touch, motivated by the fact that both modalities are sensitive to environmental oscillations. In our earlier study, we characterized the effect of auditory distractors on tactile frequency and intensity perception. Here, we describe the converse experiments examining the effect of tactile distractors on auditory processing. Because the two studies employ the same psychophysical paradigm, we combined their results for a comprehensive view of how auditory and tactile signals interact and how these interactions depend on the perceptual task. Together, our results show that temporal frequency representations are perceptually linked regardless of the attended modality. In contrast, audio-tactile loudness interactions depend on the attended modality: Tactile distractors influence judgments of auditory intensity, but judgments of tactile intensity are impervious to auditory distraction. Lastly, we show that audio-tactile loudness interactions depend critically on stimulus timing, while pitch interactions do not. These results reveal that auditory and tactile inputs are combined differently depending on the perceptual task. That distinct rules govern the integration of auditory and tactile signals in pitch and loudness perception implies that the two are mediated by separate neural mechanisms. These findings underscore the complexity and specificity of multisensory interactions.

Anatomical pathways for auditory memory in primates

Episodic memory or the ability to store context-rich information about everyday events depends on the hippocampal formation (entorhinal cortex, subiculum, presubiculum, parasubiculum, hippocampus proper, and dentate gyrus). A substantial amount of behavioral-lesion and anatomical studies have contributed to our understanding of the organization of how visual stimuli are retained in episodic memory. However, whether auditory memory is organized similarly is still unclear. One hypothesis is that, like the “visual ventral stream” for which the connections of the inferior temporal gyrus with the perirhinal cortex are necessary for visual recognition in monkeys, direct connections between the auditory association areas of the superior temporal gyrus and the hippocampal formation and with the parahippocampal region (temporal pole, perhirinal, and posterior parahippocampal cortices) might also underlie recognition memory for sounds. Alternatively, the anatomical organization of memory could be different in audition. This alternative “indirect stream” hypothesis posits that, unlike the visual association cortex, the majority of auditory information makes one or more synapses in intermediate, polymodal areas, where they may integrate information from other sensory modalities, before reaching the medial temporal memory system. This review considers anatomical studies that can support either one or both hypotheses – focusing on anatomical studies on the primate brain, primarily in macaque monkeys, that have reported not only direct auditory association connections with medial temporal areas, but, importantly, also possible indirect pathways for auditory information to reach the medial temporal lobe memory system.

Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources

Current models of brain organization include multisensory interactions at early processing stages and within low-level, including primary, cortices. Embracing this model with regard to auditory-visual (AV) interactions in humans remains problematic. Controversy surrounds the application of an additive model to the analysis of event-related potentials (ERPs), and conventional ERP analysis methods have yielded discordant latencies of effects and permitted limited neurophysiologic interpretability. While hemodynamic imaging and transcranial magnetic stimulation studies provide general support for the above model, the precise timing, superadditive/subadditive directionality, topographic stability, and sources remain unresolved. We recorded ERPs in humans to attended, but task-irrelevant stimuli that did not require an overt motor response, thereby circumventing paradigmatic caveats. We applied novel ERP signal analysis methods to provide details concerning the likely bases of AV interactions. First, nonlinear interactions occur at 60-95 ms after stimulus and are the consequence of topographic, rather than pure strength, modulations in the ERP. AV stimuli engage distinct configurations of intracranial generators, rather than simply modulating the amplitude of unisensory responses. Second, source estimations (and statistical analyses thereof) identified primary visual, primary auditory, and posterior superior temporal regions as mediating these effects. Finally, scalar values of current densities in all of these regions exhibited functionally coupled, subadditive nonlinear effects, a pattern increasingly consistent with the mounting evidence in nonhuman primates. In these ways, we demonstrate how neurophysiologic bases of multisensory interactions can be noninvasively identified in humans, allowing for a synthesis across imaging methods on the one hand and species on the other.

Connections of auditory and visual cortex in the prairie vole (Microtus ochrogaster): evidence for multisensory processing in primary sensory areas

In prairie voles, primary sensory areas are dominated by neurons that respond to one sensory modality, but some neurons also respond to stimulation of other modalities. To reveal the anatomical substrate for these multimodal responses, we examined the connections of the primary auditory area + the anterior auditory field (A1 + AAF), the temporal anterior area (TA), and the primary visual area (V1). A1 + AAF had intrinsic connections and connections with TA, multimodal cortex (MM), V1, and primary somatosensory area (S1). TA had intrinsic connections and connections with A1 + AAF, MM, and V2. Callosal connections were observed in homotopic locations in auditory cortex for both fields. A1 + AAF and TA receive thalamic input primarily from divisions of the medial geniculate nucleus but also from the lateral geniculate nucleus (LGd), the lateral posterior nucleus, and the ventral posterior nucleus (VP). V1 had dense intrinsic connections and connections with V2, MM, auditory cortex, pyriform cortex (Pyr), and, in some cases, somatosensory cortex. V1 had interhemispheric connections with V1, V2, MM, S1, and Pyr and received thalamic input from LGd and VP. Our results indicate that multisensory integration occurs in primary sensory areas of the prairie vole cortex, and this may be related to behavioral specializations associated with its niche.

Lipreading and covert speech production similarly modulate human auditory-cortex responses to pure tones

Watching the lips of a speaker enhances speech perception. At the same time, the 100 ms response to speech sounds is suppressed in the observer's auditory cortex. Here, we used whole-scalp 306-channel magnetoencephalography (MEG) to study whether lipreading modulates human auditory processing already at the level of the most elementary sound features, i.e., pure tones. We further envisioned the temporal dynamics of the suppression to tell whether the effect is driven by top-down influences. Nineteen subjects were presented with 50 ms tones spanning six octaves (125-8000 Hz) (1) during "lipreading," i.e., when they watched video clips of silent articulations of Finnish vowels /a/, /i/, /o/, and /y/, and reacted to vowels presented twice in a row; (2) during a visual control task; (3) during a still-face passive control condition; and (4) in a separate experiment with a subset of nine subjects, during covert production of the same vowels. Auditory-cortex 100 ms responses (N100m) were equally suppressed in the lipreading and covert-speech-production tasks compared with the visual control and baseline tasks; the effects involved all frequencies and were most prominent in the left hemisphere. Responses to tones presented at different times with respect to the onset of the visual articulation showed significantly increased N100m suppression immediately after the articulatory gesture. These findings suggest that the lipreading-related suppression in the auditory cortex is caused by top-down influences, possibly by an efference copy from the speech-production system, generated during both own speech and lipreading.

The behavioral relevance of multisensory neural response interactions

Sensory information can interact to impact perception and behavior. Foods are appreciated according to their appearance, smell, taste and texture. Athletes and dancers combine visual, auditory, and somatosensory information to coordinate their movements. Under laboratory settings, detection and discrimination are likewise facilitated by multisensory signals. Research over the past several decades has shown that the requisite anatomy exists to support interactions between sensory systems in regions canonically designated as exclusively unisensory in their function and, more recently, that neural response interactions occur within these same regions, including even primary cortices and thalamic nuclei, at early post-stimulus latencies. Here, we review evidence concerning direct links between early, low-level neural response interactions and behavioral measures of multisensory integration.

Auditory spatial and object processing in the human planum temporale: no evidence for selectivity

Although it is generally acknowledged that at least two processing streams exist in the primate cortical auditory system, the function of the posterior dorsal stream is a topic of much debate. Recent studies have reported selective activation to auditory spatial change in portions of the human planum temporale (PT) relative to nonspatial stimuli such as pitch changes or complex acoustic patterns. However, previous work has suggested that the PT may be sensitive to another kind of nonspatial variable, namely, the number of auditory objects simultaneously presented in the acoustic signal. The goal of the present fMRI experiment was to assess whether any portion of the PT showed spatial selectivity relative to manipulations of the number of auditory objects presented. Spatially sensitive regions in the PT were defined by comparing activity associated with listening to an auditory object (speech from a single talker) that changed location with one that remained stationary. Activity within these regions was then examined during a nonspatial manipulation: increasing the number of objects (talkers) from one to three. The nonspatial manipulation modulated activity within the "spatial" PT regions. No region within the PT was found to be selective for spatial or object processing. We suggest that previously documented spatial sensitivity in the PT reflects auditory source separation using spatial cues rather than spatial processing per se.

Wernicke's area homologue in chimpanzees (Pan troglodytes) and its relation to the appearance of modern human language

Human language is distinctive compared with the communication systems of other species. Yet, several questions concerning its emergence and evolution remain unresolved. As a means of evaluating the neuroanatomical changes relevant to language that accompanied divergence from the last common ancestor of chimpanzees, bonobos and humans, we defined the cytoarchitectonic boundaries of area Tpt, a component of Wernicke's area, in 12 common chimpanzee brains and used design-based stereologic methods to estimate regional volumes, total neuron number and neuron density. In addition, we created a probabilistic map of the location of area Tpt in a template chimpanzee brain coordinate space. Our results show that chimpanzees display significant population-level leftward asymmetry of area Tpt in terms of neuron number, with volume asymmetry approaching significance. Furthermore, asymmetry in the number of neurons in area Tpt was positively correlated with asymmetry of neuron numbers in Brodmann's area 45, a component of Broca's frontal language region. Our findings support the conclusion that leftward asymmetry of Wernicke's area originated prior to the appearance of modern human language and before our divergence from the last common ancestor. Moreover, this study provides the first evidence of covariance between asymmetry of anterior and posterior cortical regions that in humans are important to language and other higher order cognitive functions.

Information flow in the auditory cortical network

Auditory processing in the cerebral cortex is comprised of an interconnected network of auditory and auditory-related areas distributed throughout the forebrain. The nexus of auditory activity is located in temporal cortex among several specialized areas, or fields, that receive dense inputs from the medial geniculate complex. These areas are collectively referred to as auditory cortex. Auditory activity is extended beyond auditory cortex via connections with auditory-related areas elsewhere in the cortex. Within this network, information flows between areas to and from countless targets, but in a manner that is characterized by orderly regional, areal and laminar patterns. These patterns reflect some of the structural constraints that passively govern the flow of information at all levels of the network. In addition, the exchange of information within these circuits is dynamically regulated by intrinsic neurochemical properties of projecting neurons and their targets. This article begins with an overview of the principal circuits and how each is related to information flow along major axes of the network. The discussion then turns to a description of neurochemical gradients along these axes, highlighting recent work on glutamate transporters in the thalamocortical projections to auditory cortex. The article concludes with a brief discussion of relevant neurophysiological findings as they relate to structural gradients in the network.

The multisensory roles for auditory cortex in primate vocal communication

Primate vocal communication is a fundamentally multisensory behavior and this will be reflected in the different roles brain regions play in mediating it. Auditory cortex is illustrative, being influenced, I will argue, by the visual, somatosensory, proprioceptive and motor modalities during vocal communication. It is my intention that the data reviewed here suggest that investigating auditory cortex through the lens of a specific behavior may lead to a much clearer picture of its functions and dynamic organization. One possibility is that, beyond its tonotopic and cytoarchitectural organization, the auditory cortex may be organized according to ethologically-relevant actions. Such action-specific representations would be overlayed on top of traditional mapping schemes and would help mediate motor and multisensory processes related to a particular type of behavior.

Crossmodal recruitment of primary visual cortex following brief exposure to bimodal audiovisual stimuli

Several lines of evidence suggest that exposure to only one component of typically audiovisual events can lead to crossmodal cortical activation. These effects are likely explained by long-term associations formed between the auditory and visual components of such events. It is not certain whether such crossmodal recruitment can occur in the absence of explicit conditioning, semantic factors, or long-term association; nor is it clear whether primary sensory cortices can be recruited in such paradigms. In the present study we tested the hypothesis that crossmodal cortical recruitment would occur even after a brief exposure to bimodal stimuli without semantic association. We used positron emission tomography, and an apparatus allowing presentation of spatially and temporally congruous audiovisual stimuli (noise bursts and light flashes). When presented with only the auditory or visual components of the bimodal stimuli, naïve subjects showed only modality-specific cortical activation, as expected. However, subjects who had previously been exposed to the audiovisual stimuli showed increased cerebral blood flow in the primary visual cortex when presented with sounds alone. Functional connectivity analysis suggested that the auditory cortex was the source of visual cortex activity. This crossmodal activation appears to be the result of implicit associations of the two stimuli, likely driven by their spatiotemporal characteristics; it was observed after a relatively short period of exposure (approximately 45 min), and lasted for a relatively long period after the initial exposure (approximately 1 day). The findings indicate that auditory and visual cortices interact with one another to a larger degree than typically assumed.

Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey

Studies in humans and monkeys report widespread multisensory interactions at or near primary visual and auditory areas of neocortex. The range and scale of these effects has prompted increased interest in interconnectivity between the putatively "unisensory" cortices at lower hierarchical levels. Recent anatomical tract-tracing studies have revealed direct projections from auditory cortex to primary visual area (V1) and secondary visual area (V2) that could serve as a substrate for auditory influences over low-level visual processing. To better understand the significance of these connections, we looked for reciprocal projections from visual cortex to caudal auditory cortical areas in macaque monkeys. We found direct projections from area prostriata and the peripheral visual representations of area V2. Projections were more abundant after injections of temporoparietal area and caudal parabelt than after injections of caudal medial belt and the contiguous areas near the fundus of the lateral sulcus. Only one injection was confined to primary auditory cortex (area A1) and did not demonstrate visual connections. The projections from visual areas originated mainly from infragranular layers, suggestive of a "feedback"-type projection. The selective localization of these connections to peripheral visual areas and caudal auditory cortex suggests that they are involved in spatial localization.

Integration of auditory and vibrotactile stimuli: effects of phase and stimulus-onset asynchrony

The perceptual integration of 250 Hz, 500 ms vibrotactile and auditory tones was studied in detection experiments as a function of (1) relative phase and (2) temporal asynchrony of the tone pulses. Vibrotactile stimuli were delivered through a single-channel vibrator to the left middle fingertip and auditory stimuli were presented diotically through headphones in a background of 50 dB sound pressure level broadband noise. The vibrotactile and auditory stimulus levels used each yielded 63%-77%-correct unimodal detection performance in a 2-I, 2-AFC task. Results for combined vibrotactile and auditory detection indicated that (1) performance improved for synchronous presentation, (2) performance was not affected by the relative phase of the auditory and tactile sinusoidal stimuli, and (3) performance for non-overlapping stimuli improved only if the tactile stimulus preceded the auditory. The results are generally more consistent with a "Pythagorean Sum" model than with either an "Algebraic Sum" or an "Optimal Single-Channel" Model of perceptual integration. Thus, certain combinations of auditory and tactile signals result in significant integrative effects. The lack of phase effect suggests an envelope rather than fine-structure operation for integration. The effects of asynchronous presentation of the auditory and tactile stimuli are consistent with time constants deduced from single-modality masking experiments.

Multisensory integration of sounds and vibrotactile stimuli in processing streams for "what" and "where"

The segregation between cortical pathways for the identification and localization of objects is thought of as a general organizational principle in the brain. Yet, little is known about the unimodal versus multimodal nature of these processing streams. The main purpose of the present study was to test whether the auditory and tactile dual pathways converged into specialized multisensory brain areas. We used functional magnetic resonance imaging (fMRI) to compare directly in the same subjects the brain activation related to localization and identification of comparable auditory and vibrotactile stimuli. Results indicate that the right inferior frontal gyrus (IFG) and both left and right insula were more activated during identification conditions than during localization in both touch and audition. The reverse dissociation was found for the left and right inferior parietal lobules (IPL), the left superior parietal lobule (SPL) and the right precuneus-SPL, which were all more activated during localization conditions in the two modalities. We propose that specialized areas in the right IFG and the left and right insula are multisensory operators for the processing of stimulus identity whereas parts of the left and right IPL and SPL are specialized for the processing of spatial attributes independently of sensory modality.

Multisensory functional magnetic resonance imaging: a future perspective

Advances in functional magnetic resonance imaging (fMRI) technology and analytic tools provide a powerful approach to unravel how the human brain combines the different sensory systems. In this perspective, we outline promising future directions of fMRI to make optimal use of its strengths in multisensory research, and to meet its weaker sides by combining it with other imaging modalities and computational modeling.

Altered volume and hemispheric asymmetry of the superficial cortical layers in the schizophrenia planum temporale

In vivo structural MRI studies in schizophrenia auditory cerebral cortex have reported smaller volumes and, less consistently, have reported altered hemispheric asymmetry of volumes. We used autopsy brains from 19 schizophrenia and 18 nonpsychiatric male subjects to measure the volume asymmetry of the planum temporal (PT). We then used the most recently autopsied 11 schizophrenia and 10 nonpsychiatric brains to measure the widths and fractional volumes of the upper (I-III) and lower (IV-VI) layers. Measurements of whole PT gray matter volumes did not show significant changes in schizophrenia. Nevertheless, laminar volume measurements revealed that the upper layers of the PT comprise a smaller fraction of the total cortex in schizophrenia than in nonpsychiatric brains. Subdivision of the PT showed that this change was especially prominent caudally, beyond Heschl's gyrus, whereas similar but less pronounced changes were found in the rostral PT and Heschl's gyrus. Complementary measures of laminar widths showed that the altered fractional volume in the caudal left PT was due mainly to approximately 8% thinner upper layers. However, the caudal right PT had a different profile, with thicker lower layers and comparatively unchanged upper layers. Thus, in the present study, laminar measurements provided a more sensitive method for detecting changes than measurement of whole PT volumes. Besides findings in schizophrenia, our cortical width measurements revealed normal hemispheric asymmetries consistent with previous reports. In schizophrenia, the thinner upper layers of the caudal PT suggest disrupted corticocortical processing, possibly affecting the multisensory integration and phonetic processing of this region.

Multisensory connections of monkey auditory cerebral cortex

Functional studies have demonstrated multisensory responses in auditory cortex, even in the primary and early auditory association areas. The features of somatosensory and visual responses in auditory cortex suggest that they are involved in multiple processes including spatial, temporal and object-related perception. Tract tracing studies in monkeys have demonstrated several potential sources of somatosensory and visual inputs to auditory cortex. These include potential somatosensory inputs from the retroinsular (RI) and granular insula (Ig) cortical areas, and from the thalamic posterior (PO) nucleus. Potential sources of visual responses include peripheral field representations of areas V2 and prostriata, as well as the superior temporal polysensory area (STP) in the superior temporal sulcus, and the magnocellular medial geniculate thalamic nucleus (MGm). Besides these sources, there are several other thalamic, limbic and cortical association structures that have multisensory responses and may contribute cross-modal inputs to auditory cortex. These connections demonstrated by tract tracing provide a list of potential inputs, but in most cases their significance has not been confirmed by functional experiments. It is possible that the somatosensory and visual modulation of auditory cortex are each mediated by multiple extrinsic sources.

Visual influences on ferret auditory cortex

Multisensory neurons are now known to be widespread in low-level regions of the cortex usually thought of as being responsible for modality-specific processing. The auditory cortex provides a particularly striking example of this, exhibiting responses to both visual and somatosensory stimulation. Single-neuron recording studies in ferrets have shown that each of auditory fields that have been characterized using physiological and anatomical criteria also receives visual inputs, with the incidence of visually-sensitive neurons ranging from 15% to 20% in the primary areas to around 50% or more in higher-level areas. Although some neurons exhibit spiking responses to visual stimulation, these inputs often have subthreshold influences that modulate the responses of the cortical neurons to sound. Insights into the possible role played by the visual inputs can be obtained by examining their sources of origin and the way in which they alter the processing capabilities of neurons in the auditory cortex. These studies suggest that one of the functions of the visual input to auditory cortex is to sharpen the relatively imprecise spatial coding typically found there. Because the extent to which this happens varies between cortical fields, the investigation of multisensory interactions can also help in understanding their relative contributions to auditory perception.