Efficient Temporal Coding in the Early Visual System: Existing Evidence and Future Directions

Signatures of hierarchical temporal processing in the mouse visual system

A core challenge for the brain is to process information across various timescales. This could be achieved by a hierarchical organization of temporal processing through intrinsic mechanisms (e.g., recurrent coupling or adaptation), but recent evidence from spike recordings of the rodent visual system seems to conflict with this hypothesis. Here, we used an optimized information-theoretic and classical autocorrelation analysis to show that information- and correlation timescales of spiking activity increase along the anatomical hierarchy of the mouse visual system under visual stimulation, while information-theoretic predictability decreases. Moreover, intrinsic timescales for spontaneous activity displayed a similar hierarchy, whereas the hierarchy of predictability was stimulus-dependent. We could reproduce these observations in a basic recurrent network model with correlated sensory input. Our findings suggest that the rodent visual system employs intrinsic mechanisms to achieve longer integration for higher cortical areas, while simultaneously reducing predictability for an efficient neural code.

Adaptation of the inferior temporal neurons and efficient visual processing

Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.

Gaming expertise induces meso‑scale brain plasticity and efficiency mechanisms as revealed by whole-brain modeling

Video games are a valuable tool for studying the effects of training and neural plasticity on the brain. However, the underlying mechanisms related to plasticity-associated brain structural changes and their impact on brain dynamics are unknown. Here, we used a semi-empirical whole-brain model to study structural neural plasticity mechanisms linked to video game expertise. We hypothesized that video game expertise is associated with neural plasticity-mediated changes in structural connectivity that manifest at the meso‑scale level, resulting in a more segregated functional network topology. To test this hypothesis, we combined structural connectivity data of StarCraft II video game players (VGPs, n = 31) and non-players (NVGPs, n = 31), with generic fMRI data from the Human Connectome Project and computational models, to generate simulated fMRI recordings. Graph theory analysis on simulated data was performed during both resting-state conditions and external stimulation. VGPs' simulated functional connectivity was characterized by a meso‑scale integration, with increased local connectivity in frontal, parietal, and occipital brain regions. The same analyses at the level of structural connectivity showed no differences between VGPs and NVGPs. Regions that increased their connectivity strength in VGPs are known to be involved in cognitive processes crucial for task performance such as attention, reasoning, and inference. In-silico stimulation suggested that differences in FC between VGPs and NVGPs emerge in noisy contexts, specifically when the noisy level of stimulation is increased. This indicates that the connectomes of VGPs may facilitate the filtering of noise from stimuli. These structural alterations drive the meso‑scale functional changes observed in individuals with gaming expertise. Overall, our work sheds light on the mechanisms underlying structural neural plasticity triggered by video game experiences.

Learned response dynamics reflect stimulus timing and encode temporal expectation violations in superficial layers of mouse V1

The ability to recognize ordered event sequences is a fundamental component of sensory cognition and underlies the capacity to generate temporally specific expectations of future events based on previous experience. Various lines of evidence suggest that the primary visual cortex participates in some form of predictive processing, but many details remain ambiguous. Here we use two-photon calcium imaging in layer 2/3 (L2/3) of the mouse primary visual cortex (V1) to study changes to neural activity under a multi-day sequence learning paradigm with respect to prediction error responses, stimulus encoding, and time. We find increased neural activity at the time an expected, but omitted, stimulus would have occurred but no significant prediction error responses following an unexpected stimulus substitution. Sequence representations became sparser and less correlated with training, although these changes had no effect on decoding accuracy of stimulus identity or timing. Additionally, we find that experience modifies the temporal structure of stimulus responses to produce a bias towards predictive stimulus-locked activity. Finally, we find significant temporal structure during intersequence rest periods that was largely unchanged by training.

Gaming expertise induces meso-scale brain plasticity and efficiency mechanisms as revealed by whole-brain modeling Gaming expertise, neuroplasticity and functional dynamics

Video games are a valuable tool for studying the effects of training and neural plasticity on the brain. However, the underlaying mechanisms related to plasticity-induced brain structural changes and their impact in brain dynamics are unknown. Here, we used a semi-empirical whole-brain model to study structural neural plasticity mechanisms linked to video game expertise. We hypothesized that video game expertise is associated with neural plasticity-mediated changes in structural connectivity that manifest at the meso-scale level, resulting in a more segregated functional network topology. To test this hypothesis, we combined structural connectivity data of StarCraft II video game players (VGPs, n = 31) and non-players (NVGPs, n = 31), with generic fMRI data from the Human Connectome Project and computational models, with the aim of generating simulated fMRI recordings. Graph theory analysis on simulated data was performed during both resting-state conditions and external stimulation. VGPs' simulated functional connectivity was characterized by a meso-scale integration, with increased local connectivity in frontal, parietal and occipital brain regions. The same analyses at the level of structural connectivity showed no differences between VGPs and NVGPs. Regions that increased their connectivity strength in VGPs are known to be involved in cognitive processes crucial for task performance such as attention, reasoning, and inference. In-silico stimulation suggested that differences in FC between VGPs and NVGPs emerge in noisy contexts, specifically when the noisy level of stimulation is increased. This indicates that the connectomes of VGPs may facilitate the filtering of noise from stimuli. These structural alterations drive the meso-scale functional changes observed in individuals with gaming expertise. Overall, our work sheds light into the mechanisms underlying structural neural plasticity triggered by video game experiences.

Signal processing in the vagus nerve: Hypotheses based on new genetic and anatomical evidence

Each organism must regulate its internal state in a metabolically efficient way as it interacts in space and time with an ever-changing and only partly predictable world. Success in this endeavor is largely determined by the ongoing communication between brain and body, and the vagus nerve is a crucial structure in that dialogue. In this review, we introduce the novel hypothesis that the afferent vagus nerve is engaged in signal processing rather than just signal relay. New genetic and structural evidence of vagal afferent fiber anatomy motivates two hypotheses: (1) that sensory signals informing on the physiological state of the body compute both spatial and temporal viscerosensory features as they ascend the vagus nerve, following patterns found in other sensory architectures, such as the visual and olfactory systems; and (2) that ascending and descending signals modulate one another, calling into question the strict segregation of sensory and motor signals, respectively. Finally, we discuss several implications of our two hypotheses for understanding the role of viscerosensory signal processing in predictive energy regulation (i.e., allostasis) as well as the role of metabolic signals in memory and in disorders of prediction (e.g., mood disorders).

Expectation violations produce error signals in mouse V1

Repeated exposure to visual sequences changes the form of evoked activity in the primary visual cortex (V1). Predictive coding theory provides a potential explanation for this, namely that plasticity shapes cortical circuits to encode spatiotemporal predictions and that subsequent responses are modulated by the degree to which actual inputs match these expectations. Here we use a recently developed statistical modeling technique called Model-Based Targeted Dimensionality Reduction (MbTDR) to study visually evoked dynamics in mouse V1 in the context of an experimental paradigm called "sequence learning." We report that evoked spiking activity changed significantly with training, in a manner generally consistent with the predictive coding framework. Neural responses to expected stimuli were suppressed in a late window (100-150 ms) after stimulus onset following training, whereas responses to novel stimuli were not. Substituting a novel stimulus for a familiar one led to increases in firing that persisted for at least 300 ms. Omitting predictable stimuli in trained animals also led to increased firing at the expected time of stimulus onset. Finally, we show that spiking data can be used to accurately decode time within the sequence. Our findings are consistent with the idea that plasticity in early visual circuits is involved in coding spatiotemporal information.

Optogenetic manipulation of inhibitory interneurons can be used to validate a model of spatiotemporal sequence learning

The brain uses temporal information to link discrete events into memory structures supporting recognition, prediction, and a wide variety of complex behaviors. It is still an open question how experience-dependent synaptic plasticity creates memories including temporal and ordinal information. Various models have been proposed to explain how this could work, but these are often difficult to validate in a living brain. A recent model developed to explain sequence learning in the visual cortex encodes intervals in recurrent excitatory synapses and uses a learned offset between excitation and inhibition to generate precisely timed "messenger" cells that signal the end of an instance of time. This mechanism suggests that the recall of stored temporal intervals should be particularly sensitive to the activity of inhibitory interneurons that can be easily targeted in vivo with standard optogenetic tools. In this work we examined how simulated optogenetic manipulations of inhibitory cells modifies temporal learning and recall based on these mechanisms. We show that disinhibition and excess inhibition during learning or testing cause characteristic errors in recalled timing that could be used to validate the model in vivo using either physiological or behavioral measurements.

On temporal scale-free non-periodic stimulation and its mechanisms as an infinite improbability drive of the brain's functional connectogram

Rationalized development of electrical stimulation (ES) therapy is of paramount importance. Not only it will foster new techniques and technologies with increased levels of safety, efficacy, and efficiency, but it will also facilitate the translation from basic research to clinical practice. For such endeavor, design of new technologies must dialogue with state-of-the-art neuroscientific knowledge. By its turn, neuroscience is transitioning-a movement started a couple of decades earlier-into adopting a new conceptual framework for brain architecture, in which time and thus temporal patterns plays a central role in the neuronal representation of sampled data from the world. This article discusses how neuroscience has evolved to understand the importance of brain rhythms in the overall functional architecture of the nervous system and, consequently, that neuromodulation research should embrace this new conceptual framework. Based on such support, we revisit the literature on standard (fixed-frequency pulsatile stimuli) and mostly non-standard patterns of ES to put forward our own rationale on how temporally complex stimulation schemes may impact neuromodulation strategies. We then proceed to present a low frequency, on average (thus low energy), scale-free temporally randomized ES pattern for the treatment of experimental epilepsy, devised by our group and termed NPS (Non-periodic Stimulation). The approach has been shown to have robust anticonvulsant effects in different animal models of acute and chronic seizures (displaying dysfunctional hyperexcitable tissue), while also preserving neural function. In our understanding, accumulated mechanistic evidence suggests such a beneficial mechanism of action may be due to the natural-like characteristic of a scale-free temporal pattern that may robustly compete with aberrant epileptiform activity for the recruitment of neural circuits. Delivering temporally patterned or random stimuli within specific phases of the underlying oscillations (i.e., those involved in the communication within and across brain regions) could both potentiate and disrupt the formation of neuronal assemblies with random probability. The usage of infinite improbability drive here is obviously a reference to the "The Hitchhiker's Guide to the Galaxy" comedy science fiction classic, written by Douglas Adams. The parallel is that dynamically driving brain functional connectogram, through neuromodulation, in a manner that would not favor any specific neuronal assembly and/or circuit, could re-stabilize a system that is transitioning to fall under the control of a single attractor. We conclude by discussing future avenues of investigation and their potentially disruptive impact on neurotechnology, with a particular interest in NPS implications in neural plasticity, motor rehabilitation, and its potential for clinical translation.

The influence of stimulus history on directional coding in the monarch butterfly brain

The central complex is a brain region in the insect brain that houses a neural network specialized to encode directional information. Directional coding has traditionally been investigated with compass cues that revolve in full rotations and at constant angular velocities around the insect's head. However, these stimulus conditions do not fully simulate an insect's sensory perception of compass cues during navigation. In nature, an insect flight is characterized by abrupt changes in moving direction as well as constant changes in velocity. The influence of such varying cue dynamics on compass coding remains unclear. We performed long-term tetrode recordings from the brain of monarch butterflies to study how central complex neurons respond to different stimulus velocities and directions. As these butterflies derive directional information from the sun during migration, we measured the neural response to a virtual sun. The virtual sun was either presented as a spot that appeared at random angular positions or was rotated around the butterfly at different angular velocities and directions. By specifically manipulating the stimulus velocity and trajectory, we dissociated the influence of angular velocity and direction on compass coding. While the angular velocity substantially affected the tuning directedness, the stimulus trajectory influenced the shape of the angular tuning curve. Taken together, our results suggest that the central complex flexibly adjusts its directional coding to the current stimulus dynamics ensuring a precise compass even under highly demanding conditions such as during rapid flight maneuvers.

Toward a personalized closed-loop stimulation of the visual cortex: Advances and challenges

Current cortical visual prosthesis approaches are primarily unidirectional and do not consider the feed-back circuits that exist in just about every part of the nervous system. Herein, we provide a brief overview of some recent developments for better controlling brain stimulation and present preliminary human data indicating that closed-loop strategies could considerably enhance the effectiveness, safety, and long-term stability of visual cortex stimulation. We propose that the development of improved closed-loop strategies may help to enhance our capacity to communicate with the brain.