The spatiotemporal frequency tuning of LGN receptive field facilitates neural discrimination of natural stimuli

Spectral-temporal processing of naturalistic sounds in monkeys and humans

Human speech and vocalizations in animals are rich in joint spectrotemporal (S-T) modulations, wherein acoustic changes in both frequency and time are functionally related. In principle, the primate auditory system could process these complex dynamic sounds based on either an inseparable representation of S-T features or, alternatively, a separable representation. The separability hypothesis implies an independent processing of spectral and temporal modulations. We collected comparative data on the S-T hearing sensitivity in humans and macaque monkeys to a wide range of broadband dynamic spectrotemporal ripple stimuli employing a yes-no signal-detection task. Ripples were systematically varied, as a function of density (spectral modulation frequency), velocity (temporal modulation frequency), or modulation depth, to cover a listener's full S-T modulation sensitivity, derived from a total of 87 psychometric ripple detection curves. Audiograms were measured to control for normal hearing. Determined were hearing thresholds, reaction time distributions, and S-T modulation transfer functions (MTFs), both at the ripple detection thresholds and at suprathreshold modulation depths. Our psychophysically derived MTFs are consistent with the hypothesis that both monkeys and humans employ analogous perceptual strategies: S-T acoustic information is primarily processed separable. Singular value decomposition (SVD), however, revealed a small, but consistent, inseparable spectral-temporal interaction. Finally, SVD analysis of the known visual spatiotemporal contrast sensitivity function (CSF) highlights that human vision is space-time inseparable to a much larger extent than is the case for S-T sensitivity in hearing. Thus, the specificity with which the primate brain encodes natural sounds appears to be less strict than is required to adequately deal with natural images.NEW & NOTEWORTHY We provide comparative data on primate audition of naturalistic sounds comprising hearing thresholds, reaction time distributions, and spectral-temporal modulation transfer functions. Our psychophysical experiments demonstrate that auditory information is primarily processed in a spectral-temporal-independent manner by both monkeys and humans. Singular value decomposition of known visual spatiotemporal contrast sensitivity, in comparison to our auditory spectral-temporal sensitivity, revealed a striking contrast in how the brain encodes natural sounds as opposed to natural images, as vision appears to be space-time inseparable.

The dorsal lateral geniculate nucleus and the pulvinar as essential partners for visual cortical functions

In most neuroscience textbooks, the thalamus is presented as a structure that relays sensory signals from visual, auditory, somatosensory, and gustatory receptors to the cerebral cortex. But the function of the thalamic nuclei goes beyond the simple transfer of information. This is especially true for the second-order nuclei, but also applies to first-order nuclei. First order thalamic nuclei receive information from the periphery, like the dorsal lateral geniculate nucleus (dLGN), which receives a direct input from the retina. In contrast, second order thalamic nuclei, like the pulvinar, receive minor or no input from the periphery, with the bulk of their input derived from cortical areas. The dLGN refines the information received from the retina by temporal decorrelation, thereby transmitting the most "relevant" signals to the visual cortex. The pulvinar is closely linked to virtually all visual cortical areas, and there is growing evidence that it is necessary for normal cortical processing and for aspects of visual cognition. In this article, we will discuss what we know and do not know about these structures and propose some thoughts based on the knowledge gained during the course of our careers. We hope that these thoughts will arouse curiosity about the visual thalamus and its important role, especially for the next generation of neuroscientists.

Evidence for an A-Modal Number Sense: Numerosity Adaptation Generalizes Across Visual, Auditory, and Tactile Stimuli

Humans and other species share a perceptual mechanism dedicated to the representation of approximate quantities that allows to rapidly and reliably estimate the numerosity of a set of objects: an Approximate Number System (ANS). Numerosity perception shows a characteristic shared by all primary visual features: it is susceptible to adaptation. As a consequence of prolonged exposure to a large/small quantity (“adaptor”), the apparent numerosity of a subsequent (“test”) stimulus is distorted yielding a robust under- or over-estimation, respectively. Even if numerosity adaptation has been reported across several sensory modalities (vision, audition, and touch), suggesting the idea of a central and a-modal numerosity processing system, evidence for cross-modal effects are limited to vision and audition, two modalities that are known to preferentially encode sensory stimuli in an external coordinate system. Here we test whether numerosity adaptation for visual and auditory stimuli also distorts the perceived numerosity of tactile stimuli (and vice-versa) despite touch being a modality primarily coded in an internal (body-centered) reference frame. We measured numerosity discrimination of stimuli presented sequentially after adaptation to series of either few (around 2 Hz; low adaptation) or numerous (around 8 Hz; high adaptation) impulses for all possible combinations of visual, auditory, or tactile adapting and test stimuli. In all cases, adapting to few impulses yielded a significant overestimation of the test numerosity with the opposite occurring as a consequence of adaptation to numerous stimuli. The overall magnitude of adaptation was robust (around 30%) and rather similar for all sensory modality combinations. Overall, these findings support the idea of a truly generalized and a-modal mechanism for numerosity representation aimed to process numerical information independently from the sensory modality of the incoming signals.

Do Humans and Deep Convolutional Neural Networks Use Visual Information Similarly for the Categorization of Natural Scenes?

The investigation of visual categorization has recently been aided by the introduction of deep convolutional neural networks (CNNs), which achieve unprecedented accuracy in picture classification after extensive training. Even if the architecture of CNNs is inspired by the organization of the visual brain, the similarity between CNN and human visual processing remains unclear. Here, we investigated this issue by engaging humans and CNNs in a two-class visual categorization task. To this end, pictures containing animals or vehicles were modified to contain only low/high spatial frequency (HSF) information, or were scrambled in the phase of the spatial frequency spectrum. For all types of degradation, accuracy increased as degradation was reduced for both humans and CNNs; however, the thresholds for accurate categorization varied between humans and CNNs. More remarkable differences were observed for HSF information compared to the other two types of degradation, both in terms of overall accuracy and image-level agreement between humans and CNNs. The difficulty with which the CNNs were shown to categorize high-passed natural scenes was reduced by picture whitening, a procedure which is inspired by how visual systems process natural images. The results are discussed concerning the adaptation to regularities in the visual environment (scene statistics); if the visual characteristics of the environment are not learned by CNNs, their visual categorization may depend only on a subset of the visual information on which humans rely, for example, on low spatial frequency information.

Toward New Modalities in VEP-Based BCI Applications Using Dynamical Stimuli: Introducing Quasi-Periodic and Chaotic VEP-Based BCI

Visual evoked potentials (VEPs) to periodic stimuli are commonly used in brain computer interfaces for their favorable properties such as high target identification accuracy, less training time, and low surrounding target interference. Conventional periodic stimuli can lead to subjective visual fatigue due to continuous and high contrast stimulation. In this study, we compared quasi-periodic and chaotic complex stimuli to common periodic stimuli for use with VEP-based brain computer interfaces (BCIs). Canonical correlation analysis (CCA) and coherence methods were used to evaluate the performance of the three stimulus groups. Subjective fatigue caused by the presented stimuli was evaluated by the Visual Analogue Scale (VAS). Using CCA with the M2 template approach, target identification accuracy was highest for the chaotic stimuli (M = 86.8, SE = 1.8) compared to the quasi-periodic (M = 78.1, SE = 2.6, p = 0.008) and periodic (M = 64.3, SE = 1.9, p = 0.0001) stimulus groups. The evaluation of fatigue rates revealed that the chaotic stimuli caused less fatigue compared to the quasi-periodic (p = 0.001) and periodic (p = 0.0001) stimulus groups. In addition, the quasi-periodic stimuli led to lower fatigue rates compared to the periodic stimuli (p = 0.011). We conclude that the target identification results were better for the chaotic group compared to the other two stimulus groups with CCA. In addition, the chaotic stimuli led to a less subjective visual fatigue compared to the periodic and quasi-periodic stimuli and can be suitable for designing new comfortable VEP-based BCIs.

High-gamma oscillations precede visual steady-state responses: A human electrocorticography study

The robust steady-state cortical activation elicited by flickering visual stimulation has been exploited by a wide range of scientific studies. As the fundamental neural response inherits the spectral properties of the gazed flickering, the paradigm has been used to chart cortical characteristics and their relation to pathologies. However, despite its widespread adoption, the underlying neural mechanisms are not well understood. Here, we show that the fundamental response is preceded by high-gamma (55-125 Hz) oscillations which are also synchronised to the gazed frequency. Using a subdural recording of the primary and associative visual cortices of one human subject, we demonstrate that the latencies of the high-gamma and fundamental components are highly correlated on a single-trial basis albeit that the latter is consistently delayed by approximately 55 ms. These results corroborate previous reports that top-down feedback projections are involved in the generation of the fundamental response, but, in addition, we show that trial-to-trial variability in fundamental latency is paralleled by a highly similar variability in high-gamma latency. Pathology- or paradigm-induced alterations in steady-state responses could thus originate either from deviating visual gamma responses or from aberrations in the neural feedback mechanism. Experiments designed to tease apart the two processes are expected to provide deeper insights into the studied paradigm.

Introducing chaotic codes for the modulation of code modulated visual evoked potentials (c-VEP) in normal adults for visual fatigue reduction

Code modulated Visual Evoked Potentials (c-VEP) based BCI studies usually employ m-sequences as a modulating codes for their broadband spectrum and correlation property. However, subjective fatigue of the presented codes has been a problem. In this study, we introduce chaotic codes containing broadband spectrum and similar correlation property. We examined whether the introduced chaotic codes could be decoded from EEG signals and also compared the subjective fatigue level with m-sequence codes in normal subjects. We generated chaotic code from one-dimensional logistic map and used it with conventional 31-bit m-sequence code. In a c-VEP based study in normal subjects (n = 44, 21 females) we presented these codes visually and recorded EEG signals from the corresponding codes for their four lagged versions. Canonical correlation analysis (CCA) and spatiotemporal beamforming (STB) methods were used for target identification and comparison of responses. Additionally, we compared the subjective self-declared fatigue using VAS caused by presented m-sequence and chaotic codes. The introduced chaotic code was decoded from EEG responses with CCA and STB methods. The maximum total accuracy values of 93.6 ± 11.9% and 94 ± 14.4% were achieved with STB method for chaotic and m-sequence codes for all subjects respectively. The achieved accuracies in all subjects were not significantly different in m-sequence and chaotic codes. There was significant reduction in subjective fatigue caused by chaotic codes compared to the m-sequence codes. Both m-sequence and chaotic codes were similar in their accuracies as evaluated by CCA and STB methods. The chaotic codes significantly reduced subjective fatigue compared to the m-sequence codes.

Mapping the primate lateral geniculate nucleus: a review of experiments and methods

Mapping neuronal responses in the lateral geniculate nucleus (LGN) is key to understanding how visual information is processed in the brain. This paper focuses on our current knowledge of the dynamics the receptive field (RF) as broken down into the classical receptive field (CRF) and the extra-classical receptive field (ECRF) in primate LGN. CRFs in the LGN are known to be similar to those in the retinal ganglion cell layer in terms of both spatial and temporal characteristics, leading to the standard interpretation of the LGN as a relay center from retina to primary visual cortex. ECRFs have generally been found to be large and inhibitory, with some differences in magnitude between the magno-, parvo-, and koniocellular pathways. The specific contributions of the retina, thalamus, and visual cortex to LGN ECRF properties are presently unknown. Some reports suggest a retinal origin for extra-classical suppression based on latency arguments and other reports have suggested a thalamic origin for extra-classical suppression. This issue is complicated by the use of anesthetized animals, where cortical activity is likely to be altered. Thus further study of LGN ECRFs is warranted to reconcile these discrepancies. Producing descriptions of RF properties of LGN neurons could be enhanced by employing preferred naturalistic stimuli. Although there has been significant work in cats with natural scene stimuli and noise that statistically imitates natural scenes, we highlight a need for similar data from primates. Obtaining these data may be aided by recent advancements in experimental and analytical techniques that permit the efficient study of nonlinear RF characteristics in addition to traditional linear factors. In light of the reviewed topics, we conclude by suggesting experiments to more clearly elucidate the spatial and temporal structure of ECRFs of primate LGN neurons.

Spatiotemporal receptive fields of cells in V1 are optimally shaped for stimulus velocity estimation

In recent literature, particularly interesting stimulus velocity-selective behaviors were found in the response properties of neurons belonging to the primary visual cortex (V1). In this work, 93 simple and complex cell receptive fields were obtained from the recordings of different experiments made on cats (DeAngelis, Blanche, Touryan) with reverse correlation and the spike-triggered covariance methods and then fitted with a three-dimensional Gabor model, so that cells are seen as minimizers of the Heisenberg uncertainty principle over both space and time. Analysis of the model parameters' cortical distribution suggests that V1 is spatiotemporally organized to maximize the resolution on the stimulus velocity measure.