The role of adaptation in neural coding

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.

Temporal dynamics of short-term neural adaptation across human visual cortex

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses between lower and higher visual brain areas and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Temporal dynamics of short-term neural adaptation across human visual cortex

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses across the human visual hierarchy and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Inferring neural dynamics of memory during naturalistic social communication

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present 'Natural Continuation' (NC)-a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

Sensory experience selectively reorganizes the late component of evoked responses

In response to sensory stimulation, the cortex exhibits an early transient response followed by late and slower activation. Recent studies suggest that the early component represents features of the stimulus while the late component is associated with stimulus perception. Although very informative, these studies only focus on the amplitude of the evoked responses to study its relationship with sensory perception. In this work, we expand upon the study of how patterns of evoked and spontaneous activity are modified by experience at the mesoscale level using voltage and extracellular glutamate transient recordings over widespread regions of mouse dorsal neocortex. We find that repeated tactile or auditory stimulation selectively modifies the spatiotemporal patterns of cortical activity, mainly of the late evoked response in anesthetized mice injected with amphetamine and also in awake mice. This modification lasted up to 60 min and results in an increase in the amplitude of the late response after repeated stimulation and in an increase in the similarity between the spatiotemporal patterns of the late early evoked response. This similarity increase occurs only for the evoked responses of the sensory modality that received the repeated stimulation. Thus, this selective long-lasting spatiotemporal modification of the cortical activity patterns might provide evidence that evoked responses are a cortex-wide phenomenon. This work opens new questions about how perception-related cortical activity changes with sensory experience across the cortex.

High-frequency neuronal signal better explains multi-phase BOLD response

Visual stimulation-evoked blood-oxygen-level dependent (BOLD) responses can exhibit more complex temporal dynamics than a simple monophasic response. For instance, BOLD responses sometimes include a phase of positive response followed by a phase of post-stimulus undershoot. Whether the BOLD response during these phases reflects the underlying neuronal signal fluctuations or is contributed by non-neuronal physiological factors remains elusive. When presenting blocks of sustained (i.e. DC) light ON-OFF stimulations to unanesthetized rats, we observed that the response following a decrease in illumination (i.e. OFF stimulation-evoked BOLD response) in the visual cortices displayed reproducible multiple phases, including an initial positive BOLD response, followed by an undershoot and then an overshoot before the next ON trial. This multi-phase BOLD response did not result from the entrainment of the periodic stimulation structure. When we measured the neural correlates of these responses, we found that the high-frequency band from the LFP power (300 - 3000 Hz, multi-unit activity (MUA)), but not the power in the gamma band (30 - 100 Hz) exhibited the same multiphasic dynamics as the BOLD signal. This study suggests that the post-stimulus phases of the BOLD response can be better explained by the high-frequency neuronal signal.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Modelling mouse auditory response dynamics along a continuum of consciousness using a deep recurrent neural network

Objective Understanding neurophysiological changes that accompany transitions between anaesthetized and conscious states is a key objective of anesthesiology and consciousness science. This study aimed to characterize the dynamics of auditory-evoked potential morphology in mice along a continuum of consciousness. Approach Epidural field potentials were recorded from above the primary auditory cortices of two groups of laboratory mice: urethane-anaesthetized (A, n = 14) and conscious (C, n = 17). Both groups received auditory stimulation in the form of a repeated pure-tone stimulus, before and after receiving 10 mg/kg i.p. ketamine (AK and CK). Evoked responses were then ordered by ascending sample entropy into AK, A, CK, and C, considered to reflect physiological correlates of awareness. These data were used to train a recurrent neural network (RNN) with an input parameter encoding state. Model outputs were compared with grand-average event-related potential (ERP) waveforms. Subsequently, the state parameter was varied to simulate changes in the ERP that occur during transitions between states, and relationships with dominant peak amplitudes were quantified. Main results The RNN synthesized output waveforms that were in close agreement with grand-average ERPs for each group (r2 > 0.9, p < 0.0001). Varying the input state parameter generated model outputs reflecting changes in ERP morphology predicted to occur between states. Positive peak amplitudes within 25 to 50 ms, and negative peak amplitudes within 50 to 75 ms post-stimulus-onset, were found to display a sigmoidal characteristic during the transition from anaesthetized to conscious states. In contrast, negative peak amplitudes within 0 to 25 ms displayed greater linearity. Significance This study demonstrates a method for modelling changes in ERP morphology that accompany transitions between states of consciousness using a RNN. In future studies, this approach may be applied to human data to support the clinical use of ERPs to predict transition to consciousness.

Explaining Orientation Adaptation in V1 by Updating the State of a Spatial Model

In this work, we extend an influential statistical model based on the spatial classical receptive field (CRF) and non-classical receptive field (nCRF) interactions (Coen-Cagli et al., 2012) to explain the typical orientation adaptation effects observed in V1. If we assume that the temporal adaptation modifies the “state” of the model, the spatial statistical model can explain all of the orientation adaptation effects in the context of neuronal output using small and large grating observed in neurophysiological experiments in V1. The “state” of the model represents the internal parameters such as the prior and the covariance trained on a mixed dataset that totally determine the response of the model. These two parameters, respectively, reflect the probability of the orientation component and the connectivity among neurons between CRF and nCRF. Specifically, we have two key findings: First, neural adapted results using a small grating that just covers the CRF can be predicted by the change of the prior of our model. Second, the change of the prior can also predict most of the observed results using a large grating that covers both CRF and nCRF of a neuron. However, the prediction of the novel attractive adaptation using large grating covering both CRF and nCRF also necessitates the involvement of a connectivity change of the center-surround RFs. In addition, our paper contributes a new prior-based winner-take-all (WTA) working mechanism derived from the statistical-based model to explain why and how all of these orientation adaptation effects can be predicted by relying on this spatial model without modifying its structure, a novel application of the spatial model. The research results show that adaptation may link time and space by changing the “state” of the neural system according to a specific adaptor. Furthermore, different forms of stimulus used for adaptation can cause various adaptation effects, such as an a priori shift or a connectivity change, depending on the specific stimulus size.

Efficient spline regression for neural spiking data

Point process generalized linear models (GLMs) provide a powerful tool for characterizing the coding properties of neural populations. Spline basis functions are often used in point process GLMs, when the relationship between the spiking and driving signals are nonlinear, but common choices for the structure of these spline bases often lead to loss of statistical power and numerical instability when the signals that influence spiking are bounded above or below. In particular, history dependent spike train models often suffer these issues at times immediately following a previous spike. This can make inferences related to refractoriness and bursting activity more challenging. Here, we propose a modified set of spline basis functions that assumes a flat derivative at the endpoints and show that this limits the uncertainty and numerical issues associated with cardinal splines. We illustrate the application of this modified basis to the problem of simultaneously estimating the place field and history dependent properties of a set of neurons from the CA1 region of rat hippocampus, and compare it with the other commonly used basis functions. We have made code available in MATLAB to implement spike train regression using these modified basis functions.

A precise and adaptive neural mechanism for predictive temporal processing in the frontal cortex

The theory of predictive processing posits that the brain computes expectations to process information predictively. Empirical evidence in support of this theory, however, is scarce and largely limited to sensory areas. Here, we report a precise and adaptive mechanism in the frontal cortex of non-human primates consistent with predictive processing of temporal events. We found that the speed of neural dynamics is precisely adjusted according to the average time of an expected stimulus. This speed adjustment, in turn, enables neurons to encode stimuli in terms of deviations from expectation. This lawful relationship was evident across multiple experiments and held true during learning: when temporal statistics underwent covert changes, neural responses underwent predictable changes that reflected the new mean. Together, these results highlight a precise mathematical relationship between temporal statistics in the environment and neural activity in the frontal cortex that may serve as a mechanism for predictive temporal processing.

Stimulus Driven Functional Transformations in the Early Olfactory System

Olfactory stimuli are encountered across a wide range of odor concentrations in natural environments. Defining the neural computations that support concentration invariant odor perception, odor discrimination, and odor-background segmentation across a wide range of stimulus intensities remains an open question in the field. In principle, adaptation could allow the olfactory system to adjust sensory representations to the current stimulus conditions, a well-known process in other sensory systems. However, surprisingly little is known about how adaptation changes olfactory representations and affects perception. Here we review the current understanding of how adaptation impacts processing in the first two stages of the vertebrate olfactory system, olfactory receptor neurons (ORNs), and mitral/tufted cells.

Spike frequency adaptation supports network computations on temporally dispersed information

For solving tasks such as recognizing a song, answering a question, or inverting a sequence of symbols, cortical microcircuits need to integrate and manipulate information that was dispersed over time during the preceding seconds. Creating biologically realistic models for the underlying computations, especially with spiking neurons and for behaviorally relevant integration time spans, is notoriously difficult. We examine the role of spike frequency adaptation in such computations and find that it has a surprisingly large impact. The inclusion of this well-known property of a substantial fraction of neurons in the neocortex - especially in higher areas of the human neocortex - moves the performance of spiking neural network models for computations on network inputs that are temporally dispersed from a fairly low level up to the performance level of the human brain.

Remembering the Past to See the Future

In addition to the role that our visual system plays in determining what we are seeing right now, visual computations contribute in important ways to predicting what we will see next. While the role of memory in creating future predictions is often overlooked, efficient predictive computation requires the use of information about the past to estimate future events. In this article, we introduce a framework for understanding the relationship between memory and visual prediction and review the two classes of mechanisms that the visual system relies on to create future predictions. We also discuss the principles that define the mapping from predictive computations to predictive mechanisms and how downstream brain areas interpret the predictive signals computed by the visual system. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Neural activity temporal pattern dictates long-range propagation targets

Brain possesses a complex spatiotemporal architecture for efficient information processing and computing. However, it remains unknown how neural signal propagates to its intended targets brain-wide. Using optogenetics and functional MRI, we arbitrarily initiated various discrete neural activity pulse trains with different temporal patterns and revealed their distinct long-range propagation targets within the well-defined, topographically organized somatosensory thalamo-cortical circuit. We further observed that such neural activity propagation over long range could modulate brain-wide sensory functions. Electrophysiological analysis indicated that distinct propagation pathways arose from system level neural adaptation and facilitation in response to the neural activity temporal characteristics. Together, our findings provide fundamental insights into the long-range information transfer and processing. They directly support that temporal coding underpins the whole brain functional architecture in presence of the vast and relatively static anatomical architecture.

Capturing Multiple Timescales of Adaptation to Second-Order Statistics With Generalized Linear Models: Gain Scaling and Fractional Differentiation

Single neurons can dynamically change the gain of their spiking responses to take into account shifts in stimulus variance. Moreover, gain adaptation can occur across multiple timescales. Here, we examine the ability of a simple statistical model of spike trains, the generalized linear model (GLM), to account for these adaptive effects. The GLM describes spiking as a Poisson process whose rate depends on a linear combination of the stimulus and recent spike history. The GLM successfully replicates gain scaling observed in Hodgkin-Huxley simulations of cortical neurons that occurs when the ratio of spike-generating potassium and sodium conductances approaches one. Gain scaling in the GLM depends on the length and shape of the spike history filter. Additionally, the GLM captures adaptation that occurs over multiple timescales as a fractional derivative of the stimulus envelope, which has been observed in neurons that include long timescale afterhyperpolarization conductances. Fractional differentiation in GLMs requires long spike history that span several seconds. Together, these results demonstrate that the GLM provides a tractable statistical approach for examining single-neuron adaptive computations in response to changes in stimulus variance.

Dynamic representations in networked neural systems

A group of neurons can generate patterns of activity that represent information about stimuli; subsequently, the group can transform and transmit activity patterns across synapses to spatially distributed areas. Recent studies in neuroscience have begun to independently address the two components of information processing: the representation of stimuli in neural activity and the transmission of information in networks that model neural interactions. Yet only recently are studies seeking to link these two types of approaches. Here we briefly review the two separate bodies of literature; we then review the recent strides made to address this gap. We continue with a discussion of how patterns of activity evolve from one representation to another, forming dynamic representations that unfold on the underlying network. Our goal is to offer a holistic framework for understanding and describing neural information representation and transmission while revealing exciting frontiers for future research.