Resolving multisensory and attentional influences across cortical depth in sensory cortices

On the influence of the vascular architecture on Gradient Echo and Spin Echo BOLD fMRI signals across cortical depth: a simulation approach based on realistic 3D vascular networks

GE-BOLD contrast stands out as the predominant technique in functional MRI experiments for its high sensitivity and straightforward implementation. GE-BOLD exhibits rather similar sensitivity to vessels independent of their size at submillimeter resolution studies like those examining cortical columns and laminae. However, the presence of nonspecific macrovascular contributions poses a challenge to accurately isolate neuronal activity. SE-BOLD increases specificity towards small vessels, thereby enhancing its specificity to neuronal activity, due to the effective suppression of extravascular contributions caused by macrovessels with its refocusing pulse. However, even SE-BOLD measurements may not completely remove these macrovascular contributions. By simulating hemodynamic signals across cortical depth, we gain insights into vascular contributions to the laminar BOLD signal. In this study, we employed four realistic 3D vascular models to simulate oxygen saturation states in various vascular compartments, aiming to characterize both intravascular and extravascular contributions to GE and SE signals, and corresponding BOLD signal changes, across cortical depth at 7T. Simulations suggest that SE-BOLD cannot completely reduce the macrovascular contribution near the pial surface. Simulations also show that both the specificity and signal amplitude of BOLD signals at 7T depend on the spatial arrangement of large vessels throughout cortical depth and on the pial surface.

Neuronal Modeling of Cross-Sensory Visual Evoked Magnetoencephalography Responses in the Auditory Cortex

Previous studies have demonstrated that auditory cortex activity can be influenced by cross-sensory visual inputs. Intracortical laminar recordings in nonhuman primates have suggested a feedforward (FF) type profile for auditory evoked but feedback (FB) type for visual evoked activity in the auditory cortex. To test whether cross-sensory visual evoked activity in the auditory cortex is associated with FB inputs also in humans, we analyzed magnetoencephalography (MEG) responses from eight human subjects (six females) evoked by simple auditory or visual stimuli. In the estimated MEG source waveforms for auditory cortex regions of interest, auditory evoked response showed peaks at 37 and 90 ms and visual evoked response at 125 ms. The inputs to the auditory cortex were modeled through FF- and FB-type connections targeting different cortical layers using the Human Neocortical Neurosolver (HNN), which links cellular- and circuit-level mechanisms to MEG signals. HNN modeling suggested that the experimentally observed auditory response could be explained by an FF input followed by an FB input, whereas the cross-sensory visual response could be adequately explained by just an FB input. Thus, the combined MEG and HNN results support the hypothesis that cross-sensory visual input in the auditory cortex is of FB type. The results also illustrate how the dynamic patterns of the estimated MEG source activity can provide information about the characteristics of the input into a cortical area in terms of the hierarchical organization among areas.

Visual Stimuli Modulate Local Field Potentials But Drive No High-Frequency Activity in Human Auditory Cortex

Neuroimaging studies suggest cross-sensory visual influences in human auditory cortices (ACs). Whether these influences reflect active visual processing in human ACs, which drives neuronal firing and concurrent broadband high-frequency activity (BHFA; >70 Hz), or whether they merely modulate sound processing is still debatable. Here, we presented auditory, visual, and audiovisual stimuli to 16 participants (7 women, 9 men) with stereo-EEG depth electrodes implanted near ACs for presurgical monitoring. Anatomically normalized group analyses were facilitated by inverse modeling of intracranial source currents. Analyses of intracranial event-related potentials (iERPs) suggested cross-sensory responses to visual stimuli in ACs, which lagged the earliest auditory responses by several tens of milliseconds. Visual stimuli also modulated the phase of intrinsic low-frequency oscillations and triggered 15-30 Hz event-related desynchronization in ACs. However, BHFA, a putative correlate of neuronal firing, was not significantly increased in ACs after visual stimuli, not even when they coincided with auditory stimuli. Intracranial recordings demonstrate cross-sensory modulations, but no indication of active visual processing in human ACs.

Neuronal modeling of magnetoencephalography responses in auditory cortex to auditory and visual stimuli

Previous studies have demonstrated that auditory cortex activity can be influenced by crosssensory visual inputs. Intracortical recordings in non-human primates (NHP) have suggested a bottom-up feedforward (FF) type laminar profile for auditory evoked but top-down feedback (FB) type for cross-sensory visual evoked activity in the auditory cortex. To test whether this principle applies also to humans, we analyzed magnetoencephalography (MEG) responses from eight human subjects (six females) evoked by simple auditory or visual stimuli. In the estimated MEG source waveforms for auditory cortex region of interest, auditory evoked responses showed peaks at 37 and 90 ms and cross-sensory visual responses at 125 ms. The inputs to the auditory cortex were then modeled through FF and FB type connections targeting different cortical layers using the Human Neocortical Neurosolver (HNN), which consists of a neocortical circuit model linking the cellular- and circuit-level mechanisms to MEG. The HNN models suggested that the measured auditory response could be explained by an FF input followed by an FB input, and the crosssensory visual response by an FB input. Thus, the combined MEG and HNN results support the hypothesis that cross-sensory visual input in the auditory cortex is of FB type. The results also illustrate how the dynamic patterns of the estimated MEG/EEG source activity can provide information about the characteristics of the input into a cortical area in terms of the hierarchical organization among areas.

Older adults preserve audiovisual integration through enhanced cortical activations, not by recruiting new regions

Effective interactions with the environment rely on the integration of multisensory signals: Our brains must efficiently combine signals that share a common source, and segregate those that do not. Healthy ageing can change or impair this process. This functional magnetic resonance imaging study assessed the neural mechanisms underlying age differences in the integration of auditory and visual spatial cues. Participants were presented with synchronous audiovisual signals at various degrees of spatial disparity and indicated their perceived sound location. Behaviourally, older adults were able to maintain localisation accuracy. At the neural level, they integrated auditory and visual cues into spatial representations along dorsal auditory and visual processing pathways similarly to their younger counterparts but showed greater activations in a widespread system of frontal, temporal, and parietal areas. According to multivariate Bayesian decoding, these areas encoded critical stimulus information beyond that which was encoded in the brain areas commonly activated by both groups. Surprisingly, however, the boost in information provided by these areas with age-related activation increases was comparable across the 2 age groups. This dissociation-between comparable information encoded in brain activation patterns across the 2 age groups, but age-related increases in regional blood-oxygen-level-dependent responses-contradicts the widespread notion that older adults recruit new regions as a compensatory mechanism to encode task-relevant information. Instead, our findings suggest that activation increases in older adults reflect nonspecific or modulatory mechanisms related to less efficient or slower processing, or greater demands on attentional resources.

Evaluating the effect of denoising submillimeter auditory fMRI data with NORDIC

Functional magnetic resonance imaging (fMRI) has emerged as an essential tool for exploring human brain function. Submillimeter fMRI, in particular, has emerged as a tool to study mesoscopic computations. The inherently low signal-to-noise ratio (SNR) at submillimeter resolutions warrants the use of denoising approaches tailored at reducing thermal noise - the dominant contributing noise component in high resolution fMRI. NORDIC PCA is one of such approaches, and has been benchmarked against other approaches in several applications. Here, we investigate the effects that two versions of NORDIC denoising have on auditory submillimeter data. As investigating auditory functional responses poses unique challenges, we anticipated that the benefit of this technique would be especially pronounced. Our results show that NORDIC denoising improves the detection sensitivity and the reliability of estimates in submillimeter auditory fMRI data. These effects can be explained by the reduction of the noise-induced signal variability. However, we also observed a reduction in the average response amplitude (percent signal), which may suggest that a small amount of signal was also removed. We conclude that, while evaluating the effects of the signal reduction induced by NORDIC may be necessary for each application, using NORDIC in high resolution auditory fMRI studies may be advantageous because of the large reduction in variability of the estimated responses.

Multisensory Integration and Causal Inference in Typical and Atypical Populations

Multisensory perception is critical for effective interaction with the environment, but human responses to multisensory stimuli vary across the lifespan and appear changed in some atypical populations. In this review chapter, we consider multisensory integration within a normative Bayesian framework. We begin by outlining the complex computational challenges of multisensory causal inference and reliability-weighted cue integration, and discuss whether healthy young adults behave in accordance with normative Bayesian models. We then compare their behaviour with various other human populations (children, older adults, and those with neurological or neuropsychiatric disorders). In particular, we consider whether the differences seen in these groups are due only to changes in their computational parameters (such as sensory noise or perceptual priors), or whether the fundamental computational principles (such as reliability weighting) underlying multisensory perception may also be altered. We conclude by arguing that future research should aim explicitly to differentiate between these possibilities.

Multilayer Network Analysis across Cortical Depths in Resting-State 7T fMRI

In graph theory, "multilayer networks" represent systems involving several interconnected topological levels. A neuroscience example is the hierarchy of connections between different cortical depths or "lamina". This hierarchy is becoming non-invasively accessible in humans using ultra-high-resolution functional MRI (fMRI). Here, we applied multilayer graph theory to examine functional connectivity across different cortical depths in humans, using 7T fMRI (1-mm3 voxels; 30 participants). Blood oxygenation level dependent (BOLD) signals were derived from five depths between the white matter and pial surface. We then compared networks where the inter-regional connections were limited to a single cortical depth only ("layer-by-layer matrices") to those considering all possible connections between regions and cortical depths ("multilayer matrix"). We utilized global and local graph theory features that quantitatively characterize network attributes such as network composition, nodal centrality, path-based measures, and hub segregation. Detecting functional differences between cortical depths was improved using multilayer connectomics compared to the layer-by-layer versions. Superficial aspects of the cortex dominated information transfer and deeper aspects clustering. These differences were largest in frontotemporal and limbic brain regions. fMRI functional connectivity across different cortical depths may contain neurophysiologically relevant information. Multilayer connectomics could provide a methodological framework for studies on how information flows across this hierarchy.

Functional neuroimaging as a catalyst for integrated neuroscience

Functional magnetic resonance imaging (fMRI) enables non-invasive access to the awake, behaving human brain. By tracking whole-brain signals across a diverse range of cognitive and behavioural states or mapping differences associated with specific traits or clinical conditions, fMRI has advanced our understanding of brain function and its links to both normal and atypical behaviour. Despite this headway, progress in human cognitive neuroscience that uses fMRI has been relatively isolated from rapid advances in other subdomains of neuroscience, which themselves are also somewhat siloed from one another. In this Perspective, we argue that fMRI is well-placed to integrate the diverse subfields of systems, cognitive, computational and clinical neuroscience. We first summarize the strengths and weaknesses of fMRI as an imaging tool, then highlight examples of studies that have successfully used fMRI in each subdomain of neuroscience. We then provide a roadmap for the future advances that will be needed to realize this integrative vision. In this way, we hope to demonstrate how fMRI can help usher in a new era of interdisciplinary coherence in neuroscience.

Visuospatial attention revamps cortical processing of sound amid audiovisual uncertainty

Selective attentional biases arising from one sensory modality manifest in others. The effects of visuospatial attention, important in visual object perception, are unclear in the auditory domain during audiovisual (AV) scene processing. We investigate temporal and spatial factors that underlie such transfer neurally. Auditory encoding of random tone pips in AV scenes was addressed via a temporal response function model (TRF) of participants' electroencephalogram (N = 30). The spatially uninformative pips were associated with spatially distributed visual contrast reversals ("flips"), through asynchronous probabilistic AV temporal onset distributions. Participants deployed visuospatial selection on these AV stimuli to perform a task. A late (~300 ms) cross-modal influence over the neural representation of pips was found in the original and a replication study (N = 21). Transfer depended on selected visual input being (i) presented during or shortly after a related sound, in relatively limited temporal distributions (<165 ms); (ii) positioned across limited (1:4) visual foreground to background ratios. Neural encoding of auditory input, as a function of visual input, was largest at visual foreground quadrant sectors and lowest at locations opposite to the target. The results indicate that ongoing neural representations of sounds incorporate visuospatial attributes for auditory stream segregation, as cross-modal transfer conveys information that specifies the identity of multisensory signals. A potential mechanism is by enhancing or recalibrating the tuning properties of the auditory populations that represent them as objects. The results account for the dynamic evolution under visual attention of multisensory integration, specifying critical latencies at which relevant cortical networks operate.

Layer-fMRI VASO with short stimuli and event-related designs at 7 T

Layers and columns are the dominant processing units in the human (neo)cortex at the mesoscopic scale. While the blood oxygenation dependent (BOLD) signal has a high detection sensitivity, it is biased towards unwanted signals from large draining veins at the cortical surface. The additional fMRI contrast of vascular space occupancy (VASO) has the potential to augment the neuroscientific interpretability of layer-fMRI results by means of capturing complementary information of locally specific changes in cerebral blood volume (CBV). Specifically, VASO is not subject to unwanted sensitivity amplifications of large draining veins. Because of constrained sampling efficiency, it has been mainly applied in combination with efficient block task designs and long trial durations. However, to study cognitive processes in neuroscientific contexts, or probe vascular reactivity, short stimulation periods are often necessary. Here, we developed a VASO acquisition procedure with a short acquisition period and sub-millimeter resolution. During visual event-related stimulation, we show reliable responses in visual cortices within a reasonable number of trials (∼20). Furthermore, the short TR and high spatial specificity of our VASO implementation enabled us to show differences in laminar reactivity and onset times. Finally, we explore the generalizability to a different stimulus modality (somatosensation). With this, we showed that CBV-sensitive VASO provides the means to capture layer-specific haemodynamic responses with high spatio-temporal resolution and is able to be used with event-related paradigms.

Ultra-High Field Imaging of Human Visual Cognition

Functional magnetic resonance imaging (fMRI), the key methodology for mapping the functions of the human brain in a noninvasive manner, is limited by low temporal and spatial resolution. Recent advances in ultra-high field (UHF) fMRI provide a mesoscopic (i.e., submillimeter resolution) tool that allows us to probe laminar and columnar circuits, distinguish bottom-up versus top-down pathways, and map small subcortical areas. We review recent work demonstrating that UHF fMRI provides a robust methodology for imaging the brain across cortical depths and columns that provides insights into the brain's organization and functions at unprecedented spatial resolution, advancing our understanding of the fine-scale computations and interareal communication that support visual cognition.

Anterior cingulate glutamate levels associate with functional activation and connectivity during sensory integration in schizophrenia: a multimodal 1H-MRS and fMRI study

BACKGROUND

Glutamatergic dysfunction has been implicated in sensory integration deficits in schizophrenia, yet how glutamatergic function contributes to behavioural impairments and neural activities of sensory integration remains unknown.

METHODS

Fifty schizophrenia patients and 43 healthy controls completed behavioural assessments for sensory integration and underwent magnetic resonance spectroscopy (MRS) for measuring the anterior cingulate cortex (ACC) glutamate levels. The correlation between glutamate levels and behavioural sensory integration deficits was examined in each group. A subsample of 20 pairs of patients and controls further completed an audiovisual sensory integration functional magnetic resonance imaging (fMRI) task. Blood Oxygenation Level Dependent (BOLD) activation and task-dependent functional connectivity (FC) were assessed based on fMRI data. Full factorial analyses were performed to examine the Group-by-Glutamate Level interaction effects on fMRI measurements (group differences in correlation between glutamate levels and fMRI measurements) and the correlation between glutamate levels and fMRI measurements within each group.

RESULTS

We found that schizophrenia patients exhibited impaired sensory integration which was positively correlated with ACC glutamate levels. Multimodal analyses showed significantly Group-by-Glutamate Level interaction effects on BOLD activation as well as task-dependent FC in a 'cortico-subcortical-cortical' network (including medial frontal gyrus, precuneus, ACC, middle cingulate gyrus, thalamus and caudate) with positive correlations in patients and negative in controls.

CONCLUSIONS

Our findings indicate that ACC glutamate influences neural activities in a large-scale network during sensory integration, but the effects have opposite directionality between schizophrenia patients and healthy people. This implicates the crucial role of glutamatergic system in sensory integration processing in schizophrenia.

Crossmodal interactions in human learning and memory

Most studies of memory and perceptual learning in humans have employed unisensory settings to simplify the study paradigm. However, in daily life we are often surrounded by complex and cluttered scenes made up of many objects and sources of sensory stimulation. Our experiences are, therefore, highly multisensory both when passively observing the world and when acting and navigating. We argue that human learning and memory systems are evolved to operate under these multisensory and dynamic conditions. The nervous system exploits the rich array of sensory inputs in this process, is sensitive to the relationship between the sensory inputs, and continuously updates sensory representations, and encodes memory traces based on the relationship between the senses. We review some recent findings that demonstrate a range of human learning and memory phenomena in which the interactions between visual and auditory modalities play an important role, and suggest possible neural mechanisms that can underlie some surprising recent findings. We outline open questions as well as directions of future research to unravel human perceptual learning and memory.

Cerebral blood volume sensitive layer-fMRI in the human auditory cortex at 7T: Challenges and capabilities

The development of ultra high field fMRI signal readout strategies and contrasts has led to the possibility of imaging the human brain in vivo and non-invasively at increasingly higher spatial resolutions of cortical layers and columns. One emergent layer-fMRI acquisition method with increasing popularity is the cerebral blood volume sensitive sequence named vascular space occupancy (VASO). This approach has been shown to be mostly sensitive to locally-specific changes of laminar microvasculature, without unwanted biases of trans-laminar draining veins. Until now, however, VASO has not been applied in the technically challenging cortical area of the auditory cortex. Here, we describe the main challenges we encountered when developing a VASO protocol for auditory neuroscientific applications and the solutions we have adopted. With the resulting protocol, we present preliminary results of laminar responses to sounds and as a proof of concept for future investigations, we map the topographic representation of frequency preference (tonotopy) in the auditory cortex.

Cortical depth profiles of auditory and visual 7 T functional MRI responses in human superior temporal areas

Invasive neurophysiological studies in nonhuman primates have shown different laminar activation profiles to auditory vs. visual stimuli in auditory cortices and adjacent polymodal areas. Means to examine the underlying feedforward vs. feedback type influences noninvasively have been limited in humans. Here, using 1-mm isotropic resolution 3D echo-planar imaging at 7 T, we studied the intracortical depth profiles of functional magnetic resonance imaging (fMRI) blood oxygenation level dependent (BOLD) signals to brief auditory (noise bursts) and visual (checkerboard) stimuli. BOLD percent-signal-changes were estimated at 11 equally spaced intracortical depths, within regions-of-interest encompassing auditory (Heschl's gyrus, Heschl's sulcus, planum temporale, and posterior superior temporal gyrus) and polymodal (middle and posterior superior temporal sulcus) areas. Effects of differing BOLD signal strengths for auditory and visual stimuli were controlled via normalization and statistical modeling. The BOLD depth profile shapes, modeled with quadratic regression, were significantly different for auditory vs. visual stimuli in auditory cortices, but not in polymodal areas. The different depth profiles could reflect sensory-specific feedforward versus cross-sensory feedback influences, previously shown in laminar recordings in nonhuman primates. The results suggest that intracortical BOLD profiles can help distinguish between feedforward and feedback type influences in the human brain. Further experimental studies are still needed to clarify how underlying signal strength influences BOLD depth profiles under different stimulus conditions.

Layer-specific correlates of detected and undetected auditory targets during attention

In everyday life, the processing of acoustic information allows us to react to subtle changes in the auditory scene. Yet even when closely attending to sounds in the context of a task, we occasionally miss task-relevant features. The neural computations that underlie our ability to detect behavioral relevant sound changes are thought to be grounded in both feedforward and feedback processes within the auditory hierarchy. Here, we assessed the role of feedforward and feedback contributions in primary and non-primary auditory areas during behavioral detection of target sounds using submillimeter spatial resolution functional magnetic resonance imaging (fMRI) at high-fields (7 T) in humans. We demonstrate that the successful detection of subtle temporal shifts in target sounds leads to a selective increase of activation in superficial layers of primary auditory cortex (PAC). These results indicate that feedback signals reaching as far back as PAC may be relevant to the detection of targets in the auditory scene.

Laminar functional magnetic resonance imaging in vision research

Magnetic resonance imaging (MRI) scanners at ultra-high magnetic fields have become available to use in humans, thus enabling researchers to investigate the human brain in detail. By increasing the spatial resolution, ultra-high field MR allows both structural and functional characterization of cortical layers. Techniques that can differentiate cortical layers, such as histological studies and electrode-based measurements have made critical contributions to the understanding of brain function, but these techniques are invasive and thus mainly available in animal models. There are likely to be differences in the organization of circuits between humans and even our closest evolutionary neighbors. Thus research on the human brain is essential. Ultra-high field MRI can observe differences between cortical layers, but is non-invasive and can be used in humans. Extensive previous literature has shown that neuronal connections between brain areas that transmit feedback and feedforward information terminate in different layers of the cortex. Layer-specific functional MRI (fMRI) allows the identification of layer-specific hemodynamic responses, distinguishing feedback and feedforward pathways. This capability has been particularly important for understanding visual processing, as it has allowed researchers to test hypotheses concerning feedback and feedforward information in visual cortical areas. In this review, we provide a general overview of successful ultra-high field MRI applications in vision research as examples of future research.

The promise of layer-specific neuroimaging for testing predictive coding theories of psychosis

Predictive coding potentially provides an explanatory model for understanding the neurocognitive mechanisms of psychosis. It proposes that cognitive processes, such as perception and inference, are implemented by a hierarchical system, with the influence of each level being a function of the estimated precision of beliefs at that level. However, predictive coding models of psychosis are insufficiently constrained-any phenomenon can be explained in multiple ways by postulating different changes to precision at different levels of processing. One reason for the lack of constraint in these models is that the core processes are thought to be implemented by the function of specific cortical layers, and the technology to measure layer specific neural activity in humans has until recently been lacking. As a result, our ability to constrain the models with empirical data has been limited. In this review we provide a brief overview of predictive processing models of psychosis and then describe the potential for newly developed, layer specific neuroimaging techniques to test and thus constrain these models. We conclude by discussing the most promising avenues for this research as well as the technical and conceptual challenges which may limit its application.

The role of alpha oscillations in temporal binding within and across the senses

An intriguing notion in cognitive neuroscience posits that alpha oscillations mould how the brain parses the constant influx of sensory signals into discrete perceptual events. Yet, the evidence is controversial and the underlying neural mechanism unclear. Further, it is unknown whether alpha oscillations influence observers’ perceptual sensitivity (i.e. temporal resolution) or their top-down biases to bind signals within and across the senses. Combining EEG, psychophysics and signal detection theory, this multi-day study rigorously assessed the impact of alpha frequency on temporal binding of signals within and across the senses. In a series of two-flash discrimination experiments twenty human observers were presented with one or two flashes together with none, one or two sounds. Our results provide robust evidence that pre-stimulus alpha frequency as a dynamic neural state and an individual’s trait index does not influence observers’ perceptual sensitivity or bias for two-flash discrimination in any of the three sensory contexts. These results challenge the notion that alpha oscillations have a profound impact on how observers parse sensory inputs into discrete perceptual events.

Effect of visual input on syllable parsing in a computational model of a neural microcircuit for speech processing

Objective.Seeing a person talking can help us understand them, particularly in a noisy environment. However, how the brain integrates the visual information with the auditory signal to enhance speech comprehension remains poorly understood.Approach.Here we address this question in a computational model of a cortical microcircuit for speech processing. The model consists of an excitatory and an inhibitory neural population that together create oscillations in the theta frequency range. When stimulated with speech, the theta rhythm becomes entrained to the onsets of syllables, such that the onsets can be inferred from the network activity. We investigate how well the obtained syllable parsing performs when different types of visual stimuli are added. In particular, we consider currents related to the rate of syllables as well as currents related to the mouth-opening area of the talking faces.Main results.We find that currents that target the excitatory neuronal population can influence speech comprehension, both boosting it or impeding it, depending on the temporal delay and on whether the currents are excitatory or inhibitory. In contrast, currents that act on the inhibitory neurons do not impact speech comprehension significantly.Significance.Our results suggest neural mechanisms for the integration of visual information with the acoustic information in speech and make experimentally-testable predictions.

Ultra-High-Field Neuroimaging Reveals Fine-Scale Processing for 3D Perception

Binocular disparity provides critical information about three-dimensional (3D) structures to support perception and action. In the past decade significant progress has been made in uncovering human brain areas engaged in the processing of binocular disparity signals. Yet, the fine-scale brain processing underlying 3D perception remains unknown. Here, we use ultra-high-field (7T) functional imaging at submillimeter resolution to examine fine-scale BOLD fMRI signals involved in 3D perception. In particular, we sought to interrogate the local circuitry involved in disparity processing by sampling fMRI responses at different positions relative to the cortical surface (i.e., across cortical depths corresponding to layers). We tested for representations related to 3D perception by presenting participants (male and female, N = 8) with stimuli that enable stable stereoscopic perception [i.e., correlated random dot stereograms (RDS)] versus those that do not (i.e., anticorrelated RDS). Using multivoxel pattern analysis (MVPA), we demonstrate cortical depth-specific representations in areas V3A and V7 as indicated by stronger pattern responses for correlated than for anticorrelated stimuli in upper rather than deeper layers. Examining informational connectivity, we find higher feedforward layer-to-layer connectivity for correlated than anticorrelated stimuli between V3A and V7. Further, we observe disparity-specific feedback from V3A to V1 and from V7 to V3A. Our findings provide evidence for the role of V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures.SIGNIFICANCE STATEMENT Binocular vision plays a significant role in supporting our interactions with the surrounding environment. The fine-scale neural mechanisms that underlie the brain's skill in extracting 3D structures from binocular signals are poorly understood. Here, we capitalize on recent advances in ultra-high-field functional imaging to interrogate human brain circuits involved in 3D perception at submillimeter resolution. We provide evidence for the role of area V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures from binocular signals. These fine-scale measurements help bridge the gap between animal neurophysiology and human fMRI studies investigating cross-scale circuits, from micro circuits to global brain networks for 3D perception.

LayNii: A software suite for layer-fMRI

High-resolution fMRI in the sub-millimeter regime allows researchers to resolve brain activity across cortical layers and columns non-invasively. While these high-resolution data make it possible to address novel questions of directional information flow within and across brain circuits, the corresponding data analyses are challenged by MRI artifacts, including image blurring, image distortions, low SNR, and restricted coverage. These challenges often result in insufficient spatial accuracy of conventional analysis pipelines. Here we introduce a new software suite that is specifically designed for layer-specific functional MRI: LayNii. This toolbox is a collection of command-line executable programs written in C/C++ and is distributed opensource and as pre-compiled binaries for Linux, Windows, and macOS. LayNii is designed for layer-fMRI data that suffer from SNR and coverage constraints and thus cannot be straightforwardly analyzed in alternative software packages. Some of the most popular programs of LayNii contain 'layerification' and columnarization in the native voxel space of functional data as well as many other layer-fMRI specific analysis tasks: layer-specific smoothing, model-based vein mitigation of GE-BOLD data, quality assessment of artifact dominated sub-millimeter fMRI, as well as analyses of VASO data.

Topographical and laminar distribution of audiovisual processing within human planum temporale

The brain is capable of integrating signals from multiple sensory modalities. Such multisensory integration can occur in areas that are commonly considered unisensory, such as planum temporale (PT) representing the auditory association cortex. However, the roles of different afferents (feedforward vs. feedback) to PT in multisensory processing are not well understood. Our study aims to understand that by examining laminar activity patterns in different topographical subfields of human PT under unimodal and multisensory stimuli. To this end, we adopted an advanced mesoscopic (sub-millimeter) fMRI methodology at 7 T by acquiring BOLD (blood-oxygen-level-dependent contrast, which has higher sensitivity) and VAPER (integrated blood volume and perfusion contrast, which has superior laminar specificity) signal concurrently, and performed all analyses in native fMRI space benefiting from an identical acquisition between functional and anatomical images. We found a division of function between visual and auditory processing in PT and distinct feedback mechanisms in different subareas. Specifically, anterior PT was activated more by auditory inputs and received feedback modulation in superficial layers. This feedback depended on task performance and likely arose from top-down influences from higher-order multimodal areas. In contrast, posterior PT was preferentially activated by visual inputs and received visual feedback in both superficial and deep layers, which is likely projected directly from the early visual cortex. Together, these findings provide novel insights into the mechanism of multisensory interaction in human PT at the mesoscopic spatial scale.

Visual Signals in the Mammalian Auditory System

Coordination between different sensory systems is a necessary element of sensory processing. Where and how signals from different sense organs converge onto common neural circuitry have become topics of increasing interest in recent years. In this article, we focus specifically on visual-auditory interactions in areas of the mammalian brain that are commonly considered to be auditory in function. The auditory cortex and inferior colliculus are two key points of entry where visual signals reach the auditory pathway, and both contain visual- and/or eye movement-related signals in humans and other animals. The visual signals observed in these auditory structures reflect a mixture of visual modulation of auditory-evoked activity and visually driven responses that are selective for stimulus location or features. These key response attributes also appear in the classic visual pathway but may play a different role in the auditory pathway: to modify auditory rather than visual perception. Finally, while this review focuses on two particular areas of the auditory pathway where this question has been studied, robust descending as well as ascending connections within this pathway suggest that undiscovered visual signals may be present at other stages as well. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The Role of Unimodal Feedback Pathways in Gender Perception During Activation of Voice and Face Areas

Cross-modal effects provide a model framework for investigating hierarchical inter-areal processing, particularly, under conditions where unimodal cortical areas receive contextual feedback from other modalities. Here, using complementary behavioral and brain imaging techniques, we investigated the functional networks participating in face and voice processing during gender perception, a high-level feature of voice and face perception. Within the framework of a signal detection decision model, Maximum likelihood conjoint measurement (MLCM) was used to estimate the contributions of the face and voice to gender comparisons between pairs of audio-visual stimuli in which the face and voice were independently modulated. Top–down contributions were varied by instructing participants to make judgments based on the gender of either the face, the voice or both modalities (N = 12 for each task). Estimated face and voice contributions to the judgments of the stimulus pairs were not independent; both contributed to all tasks, but their respective weights varied over a 40-fold range due to top–down influences. Models that best described the modal contributions required the inclusion of two different top–down interactions: (i) an interaction that depended on gender congruence across modalities (i.e., difference between face and voice modalities for each stimulus); (ii) an interaction that depended on the within modalities’ gender magnitude. The significance of these interactions was task dependent. Specifically, gender congruence interaction was significant for the face and voice tasks while the gender magnitude interaction was significant for the face and stimulus tasks. Subsequently, we used the same stimuli and related tasks in a functional magnetic resonance imaging (fMRI) paradigm (N = 12) to explore the neural correlates of these perceptual processes, analyzed with Dynamic Causal Modeling (DCM) and Bayesian Model Selection. Results revealed changes in effective connectivity between the unimodal Fusiform Face Area (FFA) and Temporal Voice Area (TVA) in a fashion that paralleled the face and voice behavioral interactions observed in the psychophysical data. These findings explore the role in perception of multiple unimodal parallel feedback pathways.

Visual Influences on Auditory Behavioral, Neural, and Perceptual Processes: A Review

In a naturalistic environment, auditory cues are often accompanied by information from other senses, which can be redundant with or complementary to the auditory information. Although the multisensory interactions derived from this combination of information and that shape auditory function are seen across all sensory modalities, our greatest body of knowledge to date centers on how vision influences audition. In this review, we attempt to capture the state of our understanding at this point in time regarding this topic. Following a general introduction, the review is divided into 5 sections. In the first section, we review the psychophysical evidence in humans regarding vision's influence in audition, making the distinction between vision's ability to enhance versus alter auditory performance and perception. Three examples are then described that serve to highlight vision's ability to modulate auditory processes: spatial ventriloquism, cross-modal dynamic capture, and the McGurk effect. The final part of this section discusses models that have been built based on available psychophysical data and that seek to provide greater mechanistic insights into how vision can impact audition. The second section reviews the extant neuroimaging and far-field imaging work on this topic, with a strong emphasis on the roles of feedforward and feedback processes, on imaging insights into the causal nature of audiovisual interactions, and on the limitations of current imaging-based approaches. These limitations point to a greater need for machine-learning-based decoding approaches toward understanding how auditory representations are shaped by vision. The third section reviews the wealth of neuroanatomical and neurophysiological data from animal models that highlights audiovisual interactions at the neuronal and circuit level in both subcortical and cortical structures. It also speaks to the functional significance of audiovisual interactions for two critically important facets of auditory perception-scene analysis and communication. The fourth section presents current evidence for alterations in audiovisual processes in three clinical conditions: autism, schizophrenia, and sensorineural hearing loss. These changes in audiovisual interactions are postulated to have cascading effects on higher-order domains of dysfunction in these conditions. The final section highlights ongoing work seeking to leverage our knowledge of audiovisual interactions to develop better remediation approaches to these sensory-based disorders, founded in concepts of perceptual plasticity in which vision has been shown to have the capacity to facilitate auditory learning.

A Distinct Population of L6 Neurons in Mouse V1 Mediate Cross-Callosal Communication

Through the corpus callosum, interhemispheric communication is mediated by callosal projection (CP) neurons. Using retrograde labeling, we identified a population of layer 6 (L6) excitatory neurons as the main conveyer of transcallosal information in the monocular zone of the mouse primary visual cortex (V1). Distinct from L6 corticothalamic (CT) population, V1 L6 CP neurons contribute to an extensive reciprocal network across multiple sensory cortices over two hemispheres. Receiving both local and long-range cortical inputs, they encode orientation, direction, and receptive field information, while are also highly spontaneous active. The spontaneous activity of L6 CP neurons exhibits complex relationships with brain states and stimulus presentation, distinct from the spontaneous activity patterns of the CT population. The anatomical and functional properties of these L6 CP neurons enable them to broadcast visual and nonvisual information across two hemispheres, and thus may play a role in regulating and coordinating brain-wide activity events.

Perceptual Inference, Learning, and Attention in a Multisensory World

Adaptive behavior in a complex, dynamic, and multisensory world poses some of the most fundamental computational challenges for the brain, notably inference, decision-making, learning, binding, and attention. We first discuss how the brain integrates sensory signals from the same source to support perceptual inference and decision-making by weighting them according to their momentary sensory uncertainties. We then show how observers solve the binding or causal inference problem-deciding whether signals come from common causes and should hence be integrated or else be treated independently. Next, we describe the multifarious interplay between multisensory processing and attention. We argue that attentional mechanisms are crucial to compute approximate solutions to the binding problem in naturalistic environments when complex time-varying signals arise from myriad causes. Finally, we review how the brain dynamically adapts multisensory processing to a changing world across multiple timescales.

New insights on the ventral attention network: Active suppression and involuntary recruitment during a bimodal task

Detection of unexpected, yet relevant events is essential in daily life. fMRI studies have revealed the involvement of the ventral attention network (VAN), including the temporo-parietal junction (TPJ), in such process. In this MEG study with 34 participants (17 women), we used a bimodal (visual/auditory) attention task to determine the neuronal dynamics associated with suppression of the activity of the VAN during top-down attention and its recruitment when information from the unattended sensory modality is involuntarily integrated. We observed an anticipatory power increase of alpha/beta oscillations (12-20 Hz, previously associated with functional inhibition) in the VAN following a cue indicating the modality to attend. Stronger VAN power increases were associated with better task performance, suggesting that the VAN suppression prevents shifting attention to distractors. Moreover, the TPJ was synchronized with the frontal eye field in that frequency band, indicating that the dorsal attention network (DAN) might participate in such suppression. Furthermore, we found a 12-20 Hz power decrease and enhanced synchronization, in both the VAN and DAN, when information between sensory modalities was congruent, suggesting an involvement of these networks when attention is involuntarily enhanced due to multisensory integration. Our results show that effective multimodal attentional allocation includes the modulation of the VAN and DAN through upper-alpha/beta oscillations. Altogether these results indicate that the suppressing role of alpha/beta oscillations might operate beyond sensory regions.

Fine-scale computations for adaptive processing in the human brain

Adapting to the environment statistics by reducing brain responses to repetitive sensory information is key for efficient information processing. Yet, the fine-scale computations that support this adaptive processing in the human brain remain largely unknown. Here, we capitalise on the sub-millimetre resolution of ultra-high field imaging to examine functional magnetic resonance imaging signals across cortical depth and discern competing hypotheses about the brain mechanisms (feedforward vs. feedback) that mediate adaptive processing. We demonstrate layer-specific suppressive processing within visual cortex, as indicated by stronger BOLD decrease in superficial and middle than deeper layers for gratings that were repeatedly presented at the same orientation. Further, we show altered functional connectivity for adaptation: enhanced feedforward connectivity from V1 to higher visual areas, short-range feedback connectivity between V1 and V2, and long-range feedback occipito-parietal connectivity. Our findings provide evidence for a circuit of local recurrent and feedback interactions that mediate rapid brain plasticity for adaptive information processing.

Auditory information enhances post-sensory visual evidence during rapid multisensory decision-making

Despite recent progress in understanding multisensory decision-making, a conclusive mechanistic account of how the brain translates the relevant evidence into a decision is lacking. Specifically, it remains unclear whether perceptual improvements during rapid multisensory decisions are best explained by sensory (i.e., ‘Early’) processing benefits or post-sensory (i.e., ‘Late’) changes in decision dynamics. Here, we employ a well-established visual object categorisation task in which early sensory and post-sensory decision evidence can be dissociated using multivariate pattern analysis of the electroencephalogram (EEG). We capitalize on these distinct neural components to identify when and how complementary auditory information influences the encoding of decision-relevant visual evidence in a multisensory context. We show that it is primarily the post-sensory, rather than the early sensory, EEG component amplitudes that are being amplified during rapid audiovisual decision-making. Using a neurally informed drift diffusion model we demonstrate that a multisensory behavioral improvement in accuracy arises from an enhanced quality of the relevant decision evidence, as captured by the post-sensory EEG component, consistent with the emergence of multisensory evidence in higher-order brain areas.

A conclusive account on how the brain translates audiovisual evidence into a rapid decision is still lacking. Here, using a neurally-informed modelling approach, the authors show that sounds amplify visual evidence later in the decision process, in line with higher-order multisensory effects.

Higher and deeper: Bringing layer fMRI to association cortex

Recent advances in fMRI have enabled non-invasive measurements of brain function in awake, behaving humans at unprecedented spatial resolutions, allowing us to separate activity in distinct cortical layers. While most layer fMRI studies to date have focused on primary cortices, we argue that the next big steps forward in our understanding of cognition will come from expanding this technology into higher-order association cortex, to characterize depth-dependent activity during increasingly sophisticated mental processes. We outline phenomena and theories ripe for investigation with layer fMRI, including perception and imagery, selective attention, and predictive coding. We discuss practical and theoretical challenges to cognitive applications of layer fMRI, including localizing regions of interest in the face of substantial anatomical heterogeneity across individuals, designing appropriate task paradigms within the confines of acquisition parameters, and generating hypotheses for higher-order brain regions where the laminar circuitry is less well understood. We consider how applying layer fMRI in association cortex may help inform computational models of brain function as well as shed light on consciousness and mental illness, and issue a call to arms to our fellow methodologists and neuroscientists to bring layer fMRI to this next frontier.

Recurrent Processing Drives Perceptual Plasticity

Learning and experience are critical for translating ambiguous sensory information from our environments to perceptual decisions. Yet evidence on how training molds the adult human brain remains controversial, as fMRI at standard resolution does not allow us to discern the finer scale mechanisms that underlie sensory plasticity. Here, we combine ultra-high-field (7T) functional imaging at sub-millimeter resolution with orientation discrimination training to interrogate experience-dependent plasticity across cortical depths that are known to support dissociable brain computations. We demonstrate that learning alters orientation-specific representations in superficial rather than middle or deeper V1 layers, consistent with recurrent plasticity mechanisms via horizontal connections. Further, learning increases feedforward rather than feedback layer-to-layer connectivity in occipito-parietal regions, suggesting that sensory plasticity gates perceptual decisions. Our findings reveal finer scale plasticity mechanisms that re-weight sensory signals to inform improved decisions, bridging the gap between micro- and macro-circuits of experience-dependent plasticity.

•

Discrimination training alters orientation representations in superficial V1 layers

•

Orientation-specific V1 plasticity is independent of task context

•

Discrimination training alters orientation representations in middle IPS layers

•

Learning enhances feedforward connectivity from visual to parietal cortex

Judging Relative Onsets and Offsets of Audiovisual Events

This study assesses the fidelity with which people can make temporal order judgments (TOJ) between auditory and visual onsets and offsets. Using an adaptive staircase task administered to a large sample of young adults, we find that the ability to judge temporal order varies widely among people, with notable difficulty created when auditory events closely follow visual events. Those findings are interpretable within the context of an independent channels model. Visual onsets and offsets can be difficult to localize in time when they occur within the temporal neighborhood of sound onsets or offsets.

Cortical laminar resting-state signal fluctuations scale with the hypercapnic blood oxygenation level-dependent response

Calibrated functional magnetic resonance imaging can remove unwanted sources of signal variability in the blood oxygenation level‐dependent (BOLD) response. This is achieved by scaling, using information from a perfusion‐sensitive scan during a purely vascular challenge, typically induced by a gas manipulation or a breath‐hold task. In this work, we seek for a validation of the use of the resting‐state fluctuation amplitude (RSFA) as a scaling factor to remove vascular contributions from the BOLD response. Given the peculiarity of depth‐dependent vascularization in gray matter, BOLD and vascular space occupancy (VASO) data were acquired at submillimeter resolution and averaged across cortical laminae. RSFA from the primary motor cortex was, thus, compared to the amplitude of hypercapnia‐induced signal changes (tSD_hc) and with the M factor of the Davis model on a laminar level. High linear correlations were observed for RSFA and tSD_hc (R² = 0.92 ± 0.06) and somewhat reduced for RSFA and M (R² = 0.62 ± 0.19). Laminar profiles of RSFA‐normalized BOLD signal changes yielded good agreement with corresponding VASO profiles. Overall, this suggests that RSFA contains strong vascular components and is also modulated by baseline quantities contained in the M factor. We conclude that RSFA may replace the scaling factor tSD_hc for normalizing the laminar BOLD response.