Coding Principles in Adaptation

Delayed Accumulation of Inhibitory Input Explains Gamma Frequency Variation with Changing Contrast in an Inhibition Stabilized Network

Gamma rhythm (30-70 Hz), thought to represent the interactions between excitatory and inhibitory populations, can be induced by presenting achromatic gratings in the primary visual cortex (V1) and is sensitive to stimulus properties such as size and contrast. In addition, gamma occurs in short bursts and shows a "frequency falloff" effect where its peak frequency is high after stimulus onset and slowly decreases to a steady state. Recently, these size-contrast properties and temporal characteristics were replicated in a self-oscillating Wilson-Cowan (WC) model operating as an inhibition stabilized network (ISN), stimulated by Ornstein-Uhlenbeck (OU) type inputs. In particular, frequency falloff was explained by delayed and slowly accumulated inputs arriving at local inhibitory populations. We hypothesized that if the stimulus is preceded by another higher contrast stimulus, frequency falloff could be abolished or reversed, since the excessive inhibition will now take more time to dissipate. We presented gratings at different contrasts consecutively to two female monkeys while recording gamma using microelectrode arrays in V1 and confirmed this prediction. Further, this model also replicated a characteristic pattern of gamma frequency modulation to counter-phasing stimuli as reported previously. These phenomena were also replicated by an ISN model subject to slow adaptation in feedforward excitatory input. Thus, ISN model with delayed surround input or adapted feedforward input replicates gamma frequency responses to time-varying contrasts.

Hierarchical representations of relative numerical magnitudes in the human frontoparietal cortex

The ability to estimate numerical magnitude is essential for decision-making and is thought to underlie arithmetic skills. In humans, neural populations in the frontoparietal regions are tuned to represent numerosity. However, it remains unclear whether their response properties are fixed to a specific numerosity (i.e., absolute code) or dynamically scaled according to the range of numerosities relevant to the context (i.e., relative code). Here, using functional magnetic resonance imaging combined with multivariate pattern analysis, we uncover evidence that representations of relative numerosity coding emerge gradually as visual information processing advances in the frontoparietal regions. In contrast, the early sensory areas predominantly exhibit absolute coding. These findings indicate a hierarchical organization of relative numerosity representations that adapt their response properties according to the context. Our results highlight the existence of a context-dependent optimization mechanism in numerosity representation, enabling the efficient processing of infinite magnitude information with finite neural resources.

Adaptation to visual sparsity enhances responses to isolated stimuli

Sensory systems adapt their response properties to the statistics of their inputs. For instance, visual systems adapt to low-order statistics like mean and variance to encode stimuli efficiently or to facilitate specific downstream computations. However, it remains unclear how other statistical features affect sensory adaptation. Here, we explore how Drosophila's visual motion circuits adapt to stimulus sparsity, a measure of the signal's intermittency not captured by low-order statistics alone. Early visual neurons in both ON and OFF pathways alter their responses dramatically with stimulus sparsity, responding positively to both light and dark sparse stimuli but linearly to dense stimuli. These changes extend to downstream ON and OFF direction-selective neurons, which are activated by sparse stimuli of both polarities but respond with opposite signs to light and dark regions of dense stimuli. Thus, sparse stimuli activate both ON and OFF pathways, recruiting a larger fraction of the circuit and potentially enhancing the salience of isolated stimuli. Overall, our results reveal visual response properties that increase the fraction of the circuit responding to sparse, isolated stimuli.

In Search of Transcriptomic Correlates of Neuronal Firing-Rate Adaptation across Subtypes, Regions and Species: A Patch-seq Analysis

Can the transcriptomic profile of a neuron predict its physiological properties? Using a Patch-seq dataset of the primary visual cortex, we addressed this question by focusing on spike rate adaptation (SRA), a well-known phenomenon that depends on small conductance calcium (Ca)-dependent potassium (SK) channels. We first show that in parvalbumin-expressing (PV) and somatostatin-expressing (SST) interneurons (INs), expression levels of genes encoding the ion channels underlying action potential generation are correlated with the half-width (HW) of spikes. Surprisingly, the SK encoding gene is not correlated with the degree of SRA (dAdap). Instead, genes that encode proteins upstream from the SK current are correlated with dAdap, a finding validated by a different dataset from the mouse's primary motor cortex that includes pyramidal cells and interneurons, as well as physiological datasets from multiple regions of macaque monkeys. Finally, we construct a minimal model to reproduce observed heterogeneity across cells, with testable predictions.

Neural networks with optimized single-neuron adaptation uncover biologically plausible regularization

Neurons in the brain have rich and adaptive input-output properties. Features such as heterogeneous f-I curves and spike frequency adaptation are known to place single neurons in optimal coding regimes when facing changing stimuli. Yet, it is still unclear how brain circuits exploit single-neuron flexibility, and how network-level requirements may have shaped such cellular function. To answer this question, a multi-scaled approach is needed where the computations of single neurons and neural circuits must be considered as a complete system. In this work, we use artificial neural networks to systematically investigate single-neuron input-output adaptive mechanisms, optimized in an end-to-end fashion. Throughout the optimization process, each neuron has the liberty to modify its nonlinear activation function parametrized to mimic f-I curves of biological neurons, either by learning an individual static function or via a learned and shared adaptation mechanism to modify activation functions in real-time during a task. We find that such adaptive networks show much-improved robustness to noise and changes in input statistics. Using tools from dynamical systems theory, we analyze the role of these emergent single-neuron properties and argue that neural diversity and adaptation play an active regularization role, enabling neural circuits to optimally propagate information across time. Finally, we outline similarities between these optimized solutions and known coding strategies found in biological neurons, such as gain scaling and fractional order differentiation/integration.

Recent Visual Experience Reshapes V4 Neuronal Activity and Improves Perceptual Performance

Recent visual experience heavily influences our visual perception, but how neuronal activity is reshaped to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while two male rhesus macaque monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in the mid-level visual cortex.

Bifurcation enhances temporal information encoding in the olfactory periphery

Living systems continually respond to signals from the surrounding environment. Survival requires that their responses adapt quickly and robustly to the changes in the environment. One particularly challenging example is olfactory navigation in turbulent plumes, where animals experience highly intermittent odor signals while odor concentration varies over many length- and timescales. Here, we show theoretically that Drosophila olfactory receptor neurons (ORNs) can exploit proximity to a bifurcation point of their firing dynamics to reliably extract information about the timing and intensity of fluctuations in the odor signal, which have been shown to be critical for odor-guided navigation. Close to the bifurcation, the system is intrinsically invariant to signal variance, and information about the timing, duration, and intensity of odor fluctuations is transferred efficiently. Importantly, we find that proximity to the bifurcation is maintained by mean adaptation alone and therefore does not require any additional feedback mechanism or fine-tuning. Using a biophysical model with calcium-based feedback, we demonstrate that this mechanism can explain the measured adaptation characteristics of Drosophila ORNs.

Coordinated changes in a cortical circuit sculpt effects of novelty on neural dynamics

Recent studies have found dramatic cell-type-specific responses to stimulus novelty, highlighting the importance of analyzing the cortical circuitry at this granularity to understand brain function. Although initial work characterized activity by cell type, the alterations in cortical circuitry due to interacting novelty effects remain unclear. We investigated circuit mechanisms underlying the observed neural dynamics in response to novel stimuli using a large-scale public dataset of electrophysiological recordings in behaving mice and a population network model. The model was constrained by multi-patch synaptic physiology and electron microscopy data. We found generally weaker connections under novel stimuli, with shifts in the balance between somatostatin (SST) and vasoactive intestinal polypeptide (VIP) populations and increased excitatory influences on parvalbumin (PV) and SST populations. These findings systematically characterize how cortical circuits adapt to stimulus novelty.

Localized and Long-Lasting Adaptation in Dragonfly Target-Detecting Neurons

Some visual neurons in the dragonfly (Hemicordulia tau) optic lobe respond to small, moving targets, likely underlying their fast pursuit of prey and conspecifics. In response to repetitive targets presented at short intervals, the spiking activity of these "small target motion detector" (STMD) neurons diminishes over time. Previous experiments limited this adaptation by including intertrial rest periods of varying durations. However, the characteristics of this effect have never been quantified. Here, using extracellular recording techniques lasting for several hours, we quantified both the spatial and temporal properties of STMD adaptation. We found that the time course of adaptation was variable across STMD units. In any one STMD, a repeated series led to more rapid adaptation, a minor accumulative effect more akin to habituation. Following an adapting stimulus, responses recovered quickly, though the rate of recovery decreased nonlinearly over time. We found that the region of adaptation is highly localized, with targets displaced by ∼2.5° eliciting a naive response. Higher frequencies of target stimulation converged to lower levels of sustained response activity. We determined that adaptation itself is a target-tuned property, not elicited by moving bars or luminance flicker. As STMD adaptation is a localized phenomenon, dependent on recent history, it is likely to play an important role in closed-loop behavior where a target is foveated in a localized region for extended periods of the pursuit duration.

Adapting and facilitating responses in mouse somatosensory cortex are dynamic and shaped by experience

Sensory adaptation is the process whereby brain circuits adjust neuronal activity in response to redundant sensory stimuli. Although sensory adaptation has been extensively studied for individual neurons on timescales of tens of milliseconds to a few seconds, little is known about it over longer timescales or at the population level. We investigated population-level adaptation in the barrel field of the mouse somatosensory cortex (S1BF) using in vivo two-photon calcium imaging and Neuropixels recordings in awake mice. Among stimulus-responsive neurons, we found both adapting and facilitating neurons, which decreased or increased their firing, respectively, with repetitive whisker stimulation. The former outnumbered the latter by 2:1 in layers 2/3 and 4; hence, the overall population response of mouse S1BF was slightly adapting. We also discovered that population adaptation to one stimulus frequency (5 Hz) does not necessarily generalize to a different frequency (12.5 Hz). Moreover, responses of individual neurons to repeated rounds of stimulation over tens of minutes were strikingly heterogeneous and stochastic, such that their adapting or facilitating response profiles were not stable across time. Such representational drift was particularly striking when recording longitudinally across 8-9 days, as adaptation profiles of most whisker-responsive neurons changed drastically from one day to the next. Remarkably, repeated exposure to a familiar stimulus paradoxically shifted the population away from strong adaptation and toward facilitation. Thus, the adapting vs. facilitating response profile of S1BF neurons is not a fixed property of neurons but rather a highly dynamic feature that is shaped by sensory experience across days.

Recent visual experience reshapes V4 neuronal activity and improves perceptual performance

Recent visual experience heavily influences our visual perception, but how this is mediated by the reshaping of neuronal activity to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in mid-level visual cortex.

A dynamic scale-mixture model of motion in natural scenes

Some of the most important tasks of visual and motor systems involve estimating the motion of objects and tracking them over time. Such systems evolved to meet the behavioral needs of the organism in its natural environment, and may therefore be adapted to the statistics of motion it is likely to encounter. By tracking the movement of individual points in movies of natural scenes, we begin to identify common properties of natural motion across scenes. As expected, objects in natural scenes move in a persistent fashion, with velocity correlations lasting hundreds of milliseconds. More subtly, but crucially, we find that the observed velocity distributions are heavy-tailed and can be modeled as a Gaussian scale-mixture. Extending this model to the time domain leads to a dynamic scale-mixture model, consisting of a Gaussian process multiplied by a positive scalar quantity with its own independent dynamics. Dynamic scaling of velocity arises naturally as a consequence of changes in object distance from the observer, and may approximate the effects of changes in other parameters governing the motion in a given scene. This modeling and estimation framework has implications for the neurobiology of sensory and motor systems, which need to cope with these fluctuations in scale in order to represent motion efficiently and drive fast and accurate tracking behavior.

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

Memristor-based adaptive neuromorphic perception in unstructured environments

Efficient operation of control systems in robotics or autonomous driving targeting real-world navigation scenarios requires perception methods that allow them to understand and adapt to unstructured environments with good accuracy, adaptation, and generality, similar to humans. To address this need, we present a memristor-based differential neuromorphic computing, perceptual signal processing, and online adaptation method providing neuromorphic style adaptation to external sensory stimuli. The adaptation ability and generality of this method are confirmed in two application scenarios: object grasping and autonomous driving. In the former, a robot hand realizes safe and stable grasping through fast ( ~ 1 ms) adaptation based on the tactile object features with a single memristor. In the latter, decision-making information of 10 unstructured environments in autonomous driving is extracted with an accuracy of 94% with a 40×25 memristor array. By mimicking human low-level perception mechanisms, the electronic neuromorphic circuit-based method achieves real-time adaptation and high-level reactions to unstructured environments.

Detection of a temporal salient object benefits from visual stimulus-specific adaptation in avian midbrain inhibitory nucleus

Food and predators are the most noteworthy objects for the basic survival of wild animals, and both are often deviant in both spatial and temporal domains and quickly attract an animal's attention. Although stimulus-specific adaptation (SSA) is considered a potential neural basis of salient sound detection in the temporal domain, related research on visual SSA is limited and its relationship with temporal saliency is uncertain. The avian nucleus isthmi pars magnocellularis (Imc), which is central to midbrain selective attention network, is an ideal site to investigate the neural correlate of visual SSA and detection of a salient object in the time domain. Here, the constant order paradigm was applied to explore the visual SSA in the Imc of pigeons. The results showed that the firing rates of Imc neurons gradually decrease with repetitions of motion in the same direction, but recover when a motion in a deviant direction is presented, implying visual SSA to the direction of a moving object. Furthermore, enhanced response for an object moving in other directions that were not presented ever in the paradigm is also observed. To verify the neural mechanism underlying these phenomena, we introduced a neural computation model involving a recoverable synaptic change with a "center-surround" pattern to reproduce the visual SSA and temporal saliency for the moving object. These results suggest that the Imc produces visual SSA to motion direction, allowing temporal salient object detection, which may facilitate the detection of the sudden appearance of a predator.

Enhanced glucose utilization of skeletal muscle after 4 weeks of intermittent hypoxia in a mouse model of type 2 diabetes

BACKGROUND

Intermittent hypoxia intervention (IHI) has been shown to reduces blood glucose and improves insulin resistance in type 2 diabetes (T2D) and has been suggested as a complementary or alternative intervention to exercise for individuals with limited mobility. Previous research on IHI has assessed cellular glucose uptake rather than utilization. The purpose of this study was to determine the effect of a 4-week IHI, with or without an aerobic exercise, on skeletal muscle glucose utilization as indicated by the changes in pyruvate, lactate, NAD+, and NADH, using a mouse model of diet-induced T2D. In addition, the effects of one exposure to hypoxia (acute) and of a 4-week IHI (chronic) were compared to explore their relationship.

METHODS

C57BL/6J mice were randomly assigned to normal control and high-fat-diet groups, and the mice that developed diet-induced diabetes were assigned to diabetes control, and intervention groups with 1 hour (acute) or 4 weeks (1 hour/day, 6 days/week) exposure to a hypoxic envrionment (0.15 FiO2), exercise (treadmill run) in normoxia, and exercise in hypoxia, respectively, with N = 7 in each group. The effects of the interventions on concentrations of fasting blood glucose, muscle glucose, GLUT4, lactate, pyruvate, nicotinamide adenine dinucleotide (NAD+), and NADH were measured, and statistically compared between the groups.

RESULTS

Compared with diabetes control group, the mice treated in the hypoxic environment for 4 weeks showed a significantly higher pyruvate levels and lower lactate/pyruvate ratios in the quadriceps muscle, and the mice exposed to hypoxia without or with aerobic exercise for either for 4 weeks or just 1 hour showed higher NAD+ levels and lower NADH/NAD+ ratios.

CONCLUSIONS

Exposure to moderate hypoxia for either one bout or 4 weeks significantly increased the body's mitochondrial NAD cyclethe in diabetic mice even in the absence of aerobic exercise. The hypoxia and exercise interventions exhibited synergistic effects on glycolysis. These findings provide mechanistic insights into the effects of IHI in respect of the management of hyperglycemia.

A power law describes the magnitude of adaptation in neural populations of primary visual cortex

How do neural populations adapt to the time-varying statistics of sensory input? We used two-photon imaging to measure the activity of neurons in mouse primary visual cortex adapted to different sensory environments, each defined by a distinct probability distribution over a stimulus set. We find that two properties of adaptation capture how the population response to a given stimulus, viewed as a vector, changes across environments. First, the ratio between the response magnitudes is a power law of the ratio between the stimulus probabilities. Second, the response direction to a stimulus is largely invariant. These rules could be used to predict how cortical populations adapt to novel, sensory environments. Finally, we show how the power law enables the cortex to preferentially signal unexpected stimuli and to adjust the metabolic cost of its sensory representation to the entropy of the environment.

Interactions between specialized gain control mechanisms in olfactory processing

Gain control is a process that adjusts a system's sensitivity when input levels change. Neural systems contain multiple mechanisms of gain control, but we do not understand why so many mechanisms are needed or how they interact. Here, we investigate these questions in the Drosophila antennal lobe, where we identify several types of inhibitory interneurons with specialized gain control functions. We find that some interneurons are nonspiking, with compartmentalized calcium signals, and they specialize in intra-glomerular gain control. Conversely, we find that other interneurons are recruited by strong and widespread network input; they specialize in global presynaptic gain control. Using computational modeling and optogenetic perturbations, we show how these mechanisms can work together to improve stimulus discrimination while also minimizing temporal distortions in network activity. Our results demonstrate how the robustness of neural network function can be increased by interactions among diverse and specialized mechanisms of gain control.

Distinct value computations support rapid sequential decisions

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Coordinated changes in a cortical circuit sculpt effects of novelty on neural dynamics

Recent studies have found dramatic cell-type specific responses to stimulus novelty, highlighting the importance of analyzing the cortical circuitry at the cell-type specific level of granularity to understand brain function. Although initial work classified and characterized activity for each cell type, the specific alterations in cortical circuitry-particularly when multiple novelty effects interact-remain unclear. To address this gap, we employed a large-scale public dataset of electrophysiological recordings in the visual cortex of awake, behaving mice using Neuropixels probes and designed population network models to investigate the observed changes in neural dynamics in response to a combination of distinct forms of novelty. The model parameters were rigorously constrained by publicly available structural datasets, including multi-patch synaptic physiology and electron microscopy data. Our systematic optimization approach identified tens of thousands of model parameter sets that replicate the observed neural activity. Analysis of these solutions revealed generally weaker connections under novel stimuli, as well as a shift in the balance e between SST and VIP populations. Along with this, PV and SST populations experienced overall more excitatory influences compared to excitatory and VIP populations. Our results also highlight the role of VIP neurons in multiple aspects of visual stimulus processing and altering gain and saturation dynamics under novel conditions. In sum, our findings provide a systematic characterization of how the cortical circuit adapts to stimulus novelty by combining multiple rich public datasets.

Input-specific synaptic depression shapes temporal integration in mouse visual cortex

Efficient sensory processing requires the nervous system to adjust to ongoing features of the environment. In primary visual cortex (V1), neuronal activity strongly depends on recent stimulus history. Existing models can explain effects of prolonged stimulus presentation but remain insufficient for explaining effects observed after shorter durations commonly encountered under natural conditions. We investigated the mechanisms driving adaptation in response to brief (100 ms) stimuli in L2/3 V1 neurons by performing in vivo whole-cell recordings to measure membrane potential and synaptic inputs. We find that rapid adaptation is generated by stimulus-specific suppression of excitatory and inhibitory synaptic inputs. Targeted optogenetic experiments reveal that these synaptic effects are due to input-specific short-term depression of transmission between layers 4 and 2/3. Thus, brief stimulus presentation engages a distinct adaptation mechanism from that previously reported in response to prolonged stimuli, enabling flexible control of sensory encoding across a wide range of timescales.

Distinct value computations support rapid sequential decisions

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Perspectives on adaptive dynamical systems

Adaptivity is a dynamical feature that is omnipresent in nature, socio-economics, and technology. For example, adaptive couplings appear in various real-world systems, such as the power grid, social, and neural networks, and they form the backbone of closed-loop control strategies and machine learning algorithms. In this article, we provide an interdisciplinary perspective on adaptive systems. We reflect on the notion and terminology of adaptivity in different disciplines and discuss which role adaptivity plays for various fields. We highlight common open challenges and give perspectives on future research directions, looking to inspire interdisciplinary approaches.

Neural Evidence for Boundary Updating as the Source of the Repulsive Bias in Classification

Binary classification, an act of sorting items into two classes by setting a boundary, is biased by recent history. One common form of such bias is repulsive bias, a tendency to sort an item into the class opposite to its preceding items. Sensory-adaptation and boundary-updating are considered as two contending sources of the repulsive bias, yet no neural support has been provided for either source. Here, we explored human brains of both men and women, using functional magnetic resonance imaging (fMRI), to find such support by relating the brain signals of sensory-adaptation and boundary-updating to human classification behavior. We found that the stimulus-encoding signal in the early visual cortex adapted to previous stimuli, yet its adaptation-related changes were dissociated from current choices. Contrastingly, the boundary-representing signals in the inferior-parietal and superior-temporal cortices shifted to previous stimuli and covaried with current choices. Our exploration points to boundary-updating, rather than sensory-adaptation, as the origin of the repulsive bias in binary classification.SIGNIFICANCE STATEMENT Many animal and human studies on perceptual decision-making have reported an intriguing history effect called "repulsive bias," a tendency to classify an item as the opposite class of its previous item. Regarding the origin of repulsive bias, two contending ideas have been proposed: "bias in stimulus representation because of sensory adaptation" versus "bias in class-boundary setting because of belief updating." By conducting model-based neuroimaging experiments, we verified their predictions about which brain signal should contribute to the trial-to-trial variability in choice behavior. We found that the brain signal of class boundary, but not stimulus representation, contributed to the choice variability associated with repulsive bias. Our study provides the first neural evidence supporting the boundary-based hypothesis of repulsive bias.

Mechanisms of human dynamic object recognition revealed by sequential deep neural networks

Humans can quickly recognize objects in a dynamically changing world. This ability is showcased by the fact that observers succeed at recognizing objects in rapidly changing image sequences, at up to 13 ms/image. To date, the mechanisms that govern dynamic object recognition remain poorly understood. Here, we developed deep learning models for dynamic recognition and compared different computational mechanisms, contrasting feedforward and recurrent, single-image and sequential processing as well as different forms of adaptation. We found that only models that integrate images sequentially via lateral recurrence mirrored human performance (N = 36) and were predictive of trial-by-trial responses across image durations (13-80 ms/image). Importantly, models with sequential lateral-recurrent integration also captured how human performance changes as a function of image presentation durations, with models processing images for a few time steps capturing human object recognition at shorter presentation durations and models processing images for more time steps capturing human object recognition at longer presentation durations. Furthermore, augmenting such a recurrent model with adaptation markedly improved dynamic recognition performance and accelerated its representational dynamics, thereby predicting human trial-by-trial responses using fewer processing resources. Together, these findings provide new insights into the mechanisms rendering object recognition so fast and effective in a dynamic visual world.

A power law of cortical adaptation

How do neural populations adapt to the time-varying statistics of sensory input? To investigate, we measured the activity of neurons in primary visual cortex adapted to different environments, each associated with a distinct probability distribution over a stimulus set. Within each environment, a stimulus sequence was generated by independently sampling form its distribution. We find that two properties of adaptation capture how the population responses to a given stimulus, viewed as vectors, are linked across environments. First, the ratio between the response magnitudes is a power law of the ratio between the stimulus probabilities. Second, the response directions are largely invariant. These rules can be used to predict how cortical populations adapt to novel, sensory environments. Finally, we show how the power law enables the cortex to preferentially signal unexpected stimuli and to adjust the metabolic cost of its sensory representation to the entropy of the environment.

The influence of stimulus history on directional coding in the monarch butterfly brain

The central complex is a brain region in the insect brain that houses a neural network specialized to encode directional information. Directional coding has traditionally been investigated with compass cues that revolve in full rotations and at constant angular velocities around the insect's head. However, these stimulus conditions do not fully simulate an insect's sensory perception of compass cues during navigation. In nature, an insect flight is characterized by abrupt changes in moving direction as well as constant changes in velocity. The influence of such varying cue dynamics on compass coding remains unclear. We performed long-term tetrode recordings from the brain of monarch butterflies to study how central complex neurons respond to different stimulus velocities and directions. As these butterflies derive directional information from the sun during migration, we measured the neural response to a virtual sun. The virtual sun was either presented as a spot that appeared at random angular positions or was rotated around the butterfly at different angular velocities and directions. By specifically manipulating the stimulus velocity and trajectory, we dissociated the influence of angular velocity and direction on compass coding. While the angular velocity substantially affected the tuning directedness, the stimulus trajectory influenced the shape of the angular tuning curve. Taken together, our results suggest that the central complex flexibly adjusts its directional coding to the current stimulus dynamics ensuring a precise compass even under highly demanding conditions such as during rapid flight maneuvers.

Adaptation in auditory processing

Adaptation is an essential feature of auditory neurons, which reduces their responses to unchanging and recurring sounds and allows their response properties to be matched to the constantly changing statistics of sounds that reach the ears. As a consequence, processing in the auditory system highlights novel or unpredictable sounds and produces an efficient representation of the vast range of sounds that animals can perceive by continually adjusting the sensitivity and, to a lesser extent, the tuning properties of neurons to the most commonly encountered stimulus values. Together with attentional modulation, adaptation to sound statistics also helps to generate neural representations of sound that are tolerant to background noise and therefore plays a vital role in auditory scene analysis. In this review, we consider the diverse forms of adaptation that are found in the auditory system in terms of the processing levels at which they arise, the underlying neural mechanisms, and their impact on neural coding and perception. We also ask what the dynamics of adaptation, which can occur over multiple timescales, reveal about the statistical properties of the environment. Finally, we examine how adaptation to sound statistics is influenced by learning and experience and changes as a result of aging and hearing loss.

Input-specific synaptic depression shapes temporal integration in mouse visual cortex

Efficient sensory processing requires the nervous system to adjust to ongoing features of the environment. In primary visual cortex (V1), neuronal activity strongly depends on recent stimulus history. Existing models can explain effects of prolonged stimulus presentation, but remain insufficient for explaining effects observed after shorter durations commonly encountered under natural conditions. We investigated the mechanisms driving adaptation in response to brief (100 ms) stimuli in L2/3 V1 neurons by performing in vivo whole-cell recordings to measure membrane potential and synaptic inputs. We find that rapid adaptation is generated by stimulus-specific suppression of excitatory and inhibitory synaptic inputs. Targeted optogenetic experiments reveal that these synaptic effects are due to input-specific short-term depression of transmission between layers 4 and 2/3. Thus, distinct mechanisms are engaged following brief and prolonged stimulus presentation and together enable flexible control of sensory encoding across a wide range of time scales.

Adaptive coding across visual features during free-viewing and fixation conditions

Theoretical studies have long proposed that adaptation allows the brain to effectively use the limited response range of sensory neurons to encode widely varying natural inputs. However, despite this influential view, experimental studies have exclusively focused on how the neural code adapts to a range of stimuli lying along a single feature axis, such as orientation or contrast. Here, we performed electrical recordings in macaque visual cortex (area V4) to reveal significant adaptive changes in the neural code of single cells and populations across multiple feature axes. Both during free viewing and passive fixation, populations of cells improved their ability to encode image features after rapid exposure to stimuli lying on orthogonal feature axes even in the absence of initial tuning to these stimuli. These results reveal a remarkable adaptive capacity of visual cortical populations to improve network computations relevant for natural viewing despite the modularity of the functional cortical architecture.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Attractive and repulsive effects of sensory history concurrently shape visual perception

BACKGROUND

Sequential effects of environmental stimuli are ubiquitous in most behavioral tasks involving magnitude estimation, memory, decision making, and emotion. The human visual system exploits continuity in the visual environment, which induces two contrasting perceptual phenomena shaping visual perception. Previous work reported that perceptual estimation of a stimulus may be influenced either by attractive serial dependencies or repulsive aftereffects, with a number of experimental variables suggested as factors determining the direction and magnitude of sequential effects. Recent studies have theorized that these two effects concurrently arise in perceptual processing, but empirical evidence that directly supports this hypothesis is lacking, and it remains unclear whether and how attractive and repulsive sequential effects interact in a trial. Here we show that the two effects concurrently modulate estimation behavior in a typical sequence of perceptual tasks.

RESULTS

We first demonstrate that observers' estimation error as a function of both the previous stimulus and response cannot be fully described by either attractive or repulsive bias but is instead well captured by a summation of repulsion from the previous stimulus and attraction toward the previous response. We then reveal that the repulsive bias is centered on the observer's sensory encoding of the previous stimulus, which is again repelled away from its own preceding trial, whereas the attractive bias is centered precisely on the previous response, which is the observer's best prediction about the incoming stimuli.

CONCLUSIONS

Our findings provide strong evidence that sensory encoding is shaped by dynamic tuning of the system to the past stimuli, inducing repulsive aftereffects, and followed by inference incorporating the prediction from the past estimation, leading to attractive serial dependence.

Nonequilibrium dynamics of adaptation in sensory systems

Adaptation is used by biological sensory systems to respond to a wide range of environmental signals, by adapting their response properties to the statistics of the stimulus in order to maximize information transmission. We derive rules of optimal adaptation to changes in the mean and variance of a continuous stimulus in terms of Bayesian filters and map them onto stochastic equations that couple the state of the environment to an internal variable controlling the response function. We calculate numerical and exact results for the speed and accuracy of adaptation and its impact on information transmission. We find that, in the regime of efficient adaptation, the speed of adaptation scales sublinearly with the rate of change of the environment. Finally, we exploit the mathematical equivalence between adaptation and stochastic thermodynamics to quantitatively relate adaptation to the irreversibility of the adaptation time course, defined by the rate of entropy production. Our results suggest a means to empirically quantify adaptation in a model-free and nonparametric way.

Efficient Temporal Coding in the Early Visual System: Existing Evidence and Future Directions

While it is universally accepted that the brain makes predictions, there is little agreement about how this is accomplished and under which conditions. Accurate prediction requires neural circuits to learn and store spatiotemporal patterns observed in the natural environment, but it is not obvious how such information should be stored, or encoded. Information theory provides a mathematical formalism that can be used to measure the efficiency and utility of different coding schemes for data transfer and storage. This theory shows that codes become efficient when they remove predictable, redundant spatial and temporal information. Efficient coding has been used to understand retinal computations and may also be relevant to understanding more complicated temporal processing in visual cortex. However, the literature on efficient coding in cortex is varied and can be confusing since the same terms are used to mean different things in different experimental and theoretical contexts. In this work, we attempt to provide a clear summary of the theoretical relationship between efficient coding and temporal prediction, and review evidence that efficient coding principles explain computations in the retina. We then apply the same framework to computations occurring in early visuocortical areas, arguing that data from rodents is largely consistent with the predictions of this model. Finally, we review and respond to criticisms of efficient coding and suggest ways that this theory might be used to design future experiments, with particular focus on understanding the extent to which neural circuits make predictions from efficient representations of environmental statistics.

Opposite forms of adaptation in mouse visual cortex are controlled by distinct inhibitory microcircuits

Sensory processing in the cortex adapts to the history of stimulation but the mechanisms are not understood. Imaging the primary visual cortex of mice we find here that an increase in stimulus contrast is not followed by a simple decrease in gain of pyramidal cells; as many cells increase gain to improve detection of a subsequent decrease in contrast. Depressing and sensitizing forms of adaptation also occur in different types of interneurons (PV, SST and VIP) and the net effect within individual pyramidal cells reflects the balance of PV inputs, driving depression, and a subset of SST interneurons driving sensitization. Changes in internal state associated with locomotion increase gain across the population of pyramidal cells while maintaining the balance between these opposite forms of plasticity, consistent with activation of both VIP->SST and SST->PV disinhibitory pathways. These results reveal how different inhibitory microcircuits adjust the gain of pyramidal cells signalling changes in stimulus strength.

The authors describe the role of inhibitory microcircuits in the visual cortex of mice in adaptation to contrast. They show how external stimuli and internal state interact to adjust processing in the visual cortex.

Explaining Orientation Adaptation in V1 by Updating the State of a Spatial Model

In this work, we extend an influential statistical model based on the spatial classical receptive field (CRF) and non-classical receptive field (nCRF) interactions (Coen-Cagli et al., 2012) to explain the typical orientation adaptation effects observed in V1. If we assume that the temporal adaptation modifies the “state” of the model, the spatial statistical model can explain all of the orientation adaptation effects in the context of neuronal output using small and large grating observed in neurophysiological experiments in V1. The “state” of the model represents the internal parameters such as the prior and the covariance trained on a mixed dataset that totally determine the response of the model. These two parameters, respectively, reflect the probability of the orientation component and the connectivity among neurons between CRF and nCRF. Specifically, we have two key findings: First, neural adapted results using a small grating that just covers the CRF can be predicted by the change of the prior of our model. Second, the change of the prior can also predict most of the observed results using a large grating that covers both CRF and nCRF of a neuron. However, the prediction of the novel attractive adaptation using large grating covering both CRF and nCRF also necessitates the involvement of a connectivity change of the center-surround RFs. In addition, our paper contributes a new prior-based winner-take-all (WTA) working mechanism derived from the statistical-based model to explain why and how all of these orientation adaptation effects can be predicted by relying on this spatial model without modifying its structure, a novel application of the spatial model. The research results show that adaptation may link time and space by changing the “state” of the neural system according to a specific adaptor. Furthermore, different forms of stimulus used for adaptation can cause various adaptation effects, such as an a priori shift or a connectivity change, depending on the specific stimulus size.

Inhibition, but not excitation, recovers from partial cone loss with greater spatiotemporal integration, synapse density, and frequency

Neural circuits function in the face of changing inputs, either caused by normal variation in stimuli or by cell death. To maintain their ability to perform essential computations with partial inputs, neural circuits make modifications. Here, we study the retinal circuit’s responses to changes in light stimuli or in photoreceptor inputs by inducing partial cone death in the mature mouse retina. Can the retina withstand or recover from input loss? We find that the excitatory pathways exhibit functional loss commensurate with cone death and with some aspects predicted by partial light stimulation. However, inhibitory pathways recover functionally from lost input by increasing spatiotemporal integration in a way that is not recapitulated by partially stimulating the control retina. Anatomically, inhibitory synapses are upregulated on secondary bipolar cells and output ganglion cells. These findings demonstrate the greater capacity for inhibition, compared with excitation, to modify spatiotemporal processing with fewer cone inputs.

Lee et al. find partial cone loss triggers inhibition, but not excitation, to increase spatiotemporal integration, recover contrast gain, and increase synaptic release onto retinal ganglion cells. Natural images filtered by cone-loss receptive fields perceptually match those of controls. Thus, inhibition compensates for fewer cones to potentially preserve perception.

Efficient coding in the economics of human brain connectomics

In systems neuroscience, most models posit that brain regions communicate information under constraints of efficiency. Yet, evidence for efficient communication in structural brain networks characterized by hierarchical organization and highly connected hubs remains sparse. The principle of efficient coding proposes that the brain transmits maximal information in a metabolically economical or compressed form to improve future behavior. To determine how structural connectivity supports efficient coding, we develop a theory specifying minimum rates of message transmission between brain regions to achieve an expected fidelity, and we test five predictions from the theory based on random walk communication dynamics. In doing so, we introduce the metric of compression efficiency, which quantifies the trade-off between lossy compression and transmission fidelity in structural networks. In a large sample of youth (n = 1,042; age 8-23 years), we analyze structural networks derived from diffusion-weighted imaging and metabolic expenditure operationalized using cerebral blood flow. We show that structural networks strike compression efficiency trade-offs consistent with theoretical predictions. We find that compression efficiency prioritizes fidelity with development, heightens when metabolic resources and myelination guide communication, explains advantages of hierarchical organization, links higher input fidelity to disproportionate areal expansion, and shows that hubs integrate information by lossy compression. Lastly, compression efficiency is predictive of behavior-beyond the conventional network efficiency metric-for cognitive domains including executive function, memory, complex reasoning, and social cognition. Our findings elucidate how macroscale connectivity supports efficient coding and serve to foreground communication processes that utilize random walk dynamics constrained by network connectivity.

Novel stimuli evoke excess activity in the mouse primary visual cortex

Rapid detection and processing of stimulus novelty are key elements of adaptive behavior. Predictive coding theories postulate that novel stimuli should be encoded differently from familiar stimuli. Here, we show that the majority of neurons in layer 2/3 of the mouse primary visual cortex exhibit a significant excess response to novel visual stimuli. The distinction between novel and familiar images developed rapidly, requiring only a few repeated presentations. We show that this phenomenon can be described by a model of cascading adaptation. This ubiquitous mechanism makes it likely that similar computations could be carried out in many brain areas.

To explore how neural circuits represent novel versus familiar inputs, we presented mice with repeated sets of images with novel images sparsely substituted. Using two-photon calcium imaging to record from layer 2/3 neurons in the mouse primary visual cortex, we found that novel images evoked excess activity in the majority of neurons. This novelty response rapidly emerged, arising with a time constant of 2.6 ± 0.9 s. When a new image set was repeatedly presented, a majority of neurons had similarly elevated activity for the first few presentations, which decayed to steady state with a time constant of 1.4 ± 0.4 s. When we increased the number of images in the set, the novelty response’s amplitude decreased, defining a capacity to store ∼15 familiar images under our conditions. These results could be explained quantitatively using an adaptive subunit model in which presynaptic neurons have individual tuning and gain control. This result shows that local neural circuits can create different representations for novel versus familiar inputs using generic, widely available mechanisms.

A transcriptional rheostat couples past activity to future sensory responses

Animals traversing different environments encounter both stable background stimuli and novel cues, which are thought to be detected by primary sensory neurons and then distinguished by downstream brain circuits. Here, we show that each of the ∼1,000 olfactory sensory neuron (OSN) subtypes in the mouse harbors a distinct transcriptome whose content is precisely determined by interactions between its odorant receptor and the environment. This transcriptional variation is systematically organized to support sensory adaptation: expression levels of more than 70 genes relevant to transforming odors into spikes continuously vary across OSN subtypes, dynamically adjust to new environments over hours, and accurately predict acute OSN-specific odor responses. The sensory periphery therefore separates salient signals from predictable background via a transcriptional rheostat whose moment-to-moment state reflects the past and constrains the future; these findings suggest a general model in which structured transcriptional variation within a cell type reflects individual experience.

Prediction under uncertainty: Dissociating sensory from cognitive expectations in highly uncertain musical contexts

Predictive models in the brain rely on the continuous extraction of regularities from the environment. These models are thought to be updated by novel information, as reflected in prediction error responses such as the mismatch negativity (MMN). However, although in real life individuals often face situations in which uncertainty prevails, it remains unclear whether and how predictive models emerge in high-uncertainty contexts. Recent research suggests that uncertainty affects the magnitude of MMN responses in the context of music listening. However, musical predictions are typically studied with MMN stimulation paradigms based on Western tonal music, which are characterized by relatively high predictability. Hence, we developed an MMN paradigm to investigate how the high uncertainty of atonal music modulates predictive processes as indexed by the MMN and behavior. Using MEG in a group of 20 subjects without musical training, we demonstrate that the magnetic MMN in response to pitch, intensity, timbre, and location deviants is evoked in both tonal and atonal melodies, with no significant differences between conditions. In contrast, in a separate behavioral experiment involving 39 non-musicians, participants detected pitch deviants more accurately and rated confidence higher in the tonal than in the atonal musical context. These results indicate that contextual tonal uncertainty modulates processing stages in which conscious awareness is involved, although deviants robustly elicit low-level pre-attentive responses such as the MMN. The achievement of robust MMN responses, despite high tonal uncertainty, is relevant for future studies comparing groups of listeners' MMN responses to increasingly ecological music stimuli.

Flexible filtering by neural inputs supports motion computation across states and stimuli

Sensory systems flexibly adapt their processing properties across a wide range of environmental and behavioral conditions. Such variable processing complicates attempts to extract a mechanistic understanding of sensory computations. This is evident in the highly constrained, canonical Drosophila motion detection circuit, where the core computation underlying direction selectivity is still debated despite extensive studies. Here we measured the filtering properties of neural inputs to the OFF motion-detecting T5 cell in Drosophila. We report state- and stimulus-dependent changes in the shape of these signals, which become more biphasic under specific conditions. Summing these inputs within the framework of a connectomic-constrained model of the circuit demonstrates that these shapes are sufficient to explain T5 responses to various motion stimuli. Thus, our stimulus- and state-dependent measurements reconcile motion computation with the anatomy of the circuit. These findings provide a clear example of how a basic circuit supports flexible sensory computation.

A Correspondence Between Normalization Strategies in Artificial and Biological Neural Networks

A fundamental challenge at the interface of machine learning and neuroscience is to uncover computational principles that are shared between artificial and biological neural networks. In deep learning, normalization methods such as batch normalization, weight normalization, and their many variants help to stabilize hidden unit activity and accelerate network training, and these methods have been called one of the most important recent innovations for optimizing deep networks. In the brain, homeostatic plasticity represents a set of mechanisms that also stabilize and normalize network activity to lie within certain ranges, and these mechanisms are critical for maintaining normal brain function. In this article, we discuss parallels between artificial and biological normalization methods at four spatial scales: normalization of a single neuron's activity, normalization of synaptic weights of a neuron, normalization of a layer of neurons, and normalization of a network of neurons. We argue that both types of methods are functionally equivalent-that is, both push activation patterns of hidden units toward a homeostatic state, where all neurons are equally used-and we argue that such representations can improve coding capacity, discrimination, and regularization. As a proof of concept, we develop an algorithm, inspired by a neural normalization technique called synaptic scaling, and show that this algorithm performs competitively against existing normalization methods on several data sets. Overall, we hope this bidirectional connection will inspire neuroscientists and machine learners in three ways: to uncover new normalization algorithms based on established neurobiological principles; to help quantify the trade-offs of different homeostatic plasticity mechanisms used in the brain; and to offer insights about how stability may not hinder, but may actually promote, plasticity.

The generation of cortical novelty responses through inhibitory plasticity

Animals depend on fast and reliable detection of novel stimuli in their environment. Neurons in multiple sensory areas respond more strongly to novel in comparison to familiar stimuli. Yet, it remains unclear which circuit, cellular, and synaptic mechanisms underlie those responses. Here, we show that spike-timing-dependent plasticity of inhibitory-to-excitatory synapses generates novelty responses in a recurrent spiking network model. Inhibitory plasticity increases the inhibition onto excitatory neurons tuned to familiar stimuli, while inhibition for novel stimuli remains low, leading to a network novelty response. The generation of novelty responses does not depend on the periodicity but rather on the distribution of presented stimuli. By including tuning of inhibitory neurons, the network further captures stimulus-specific adaptation. Finally, we suggest that disinhibition can control the amplification of novelty responses. Therefore, inhibitory plasticity provides a flexible, biologically plausible mechanism to detect the novelty of bottom-up stimuli, enabling us to make experimentally testable predictions.

Normative Principles for Decision-Making in Natural Environments

The decisions we make are shaped by a lifetime of learning. Past experience guides the way that we encode information in neural systems for perception and valuation, and determines the information we retrieve when making decisions. Distinct literatures have discussed how lifelong learning and local context shape decisions made about sensory signals, propositional information, or economic prospects. Here, we build bridges between these literatures, arguing for common principles of adaptive rationality in perception, cognition, and economic choice. We discuss how a single common framework, based on normative principles of efficient coding and Bayesian inference, can help us understand a myriad of human decision biases, including sensory illusions, adaptive aftereffects, choice history biases, central tendency effects, anchoring effects, contrast effects, framing effects, congruency effects, reference-dependent valuation, nonlinear utility functions, and discretization heuristics. We describe a simple computational framework for explaining these phenomena. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Impact of Photoreceptor Loss on Retinal Circuitry

Our sense of sight relies on photoreceptors, which transduce photons into the nervous system's electrochemical interpretation of the visual world. These precious photoreceptors can be disrupted by disease, injury, and aging. Once photoreceptors start to die, but before blindness occurs, the remaining retinal circuitry can withstand, mask, or exacerbate the photoreceptor deficit and potentially be receptive to newfound therapies for vision restoration. To maximize the retina's receptivity to therapy, one must understand the conditions that influence the state of the remaining retina. In this review, we provide an overview of the retina's structure and function in health and disease. We analyze a collection of observations on photoreceptor disruption and generate a predictive model to identify parameters that influence the retina's response. Finally, we speculate on whether the retina, with its remarkable capacity to function over light levels spanning nine orders of magnitude, uses these same adaptational mechanisms to withstand and perhaps mask photoreceptor loss.

Semantic Predictability Facilitates Comprehension of Degraded Speech in a Graded Manner

Previous studies have shown that at moderate levels of spectral degradation, semantic predictability facilitates language comprehension. It is argued that when speech is degraded, listeners have narrowed expectations about the sentence endings; i.e., semantic prediction may be limited to only most highly predictable sentence completions. The main objectives of this study were to (i) examine whether listeners form narrowed expectations or whether they form predictions across a wide range of probable sentence endings, (ii) assess whether the facilitatory effect of semantic predictability is modulated by perceptual adaptation to degraded speech, and (iii) use and establish a sensitive metric for the measurement of language comprehension. For this, we created 360 German Subject-Verb-Object sentences that varied in semantic predictability of a sentence-final target word in a graded manner (high, medium, and low) and levels of spectral degradation (1, 4, 6, and 8 channels noise-vocoding). These sentences were presented auditorily to two groups: One group (n =48) performed a listening task in an unpredictable channel context in which the degraded speech levels were randomized, while the other group (n =50) performed the task in a predictable channel context in which the degraded speech levels were blocked. The results showed that at 4 channels noise-vocoding, response accuracy was higher in high-predictability sentences than in the medium-predictability sentences, which in turn was higher than in the low-predictability sentences. This suggests that, in contrast to the narrowed expectations view, comprehension of moderately degraded speech, ranging from low- to high- including medium-predictability sentences, is facilitated in a graded manner; listeners probabilistically preactivate upcoming words from a wide range of semantic space, not limiting only to highly probable sentence endings. Additionally, in both channel contexts, we did not observe learning effects; i.e., response accuracy did not increase over the course of experiment, and response accuracy was higher in the predictable than in the unpredictable channel context. We speculate from these observations that when there is no trial-by-trial variation of the levels of speech degradation, listeners adapt to speech quality at a long timescale; however, when there is a trial-by-trial variation of the high-level semantic feature (e.g., sentence predictability), listeners do not adapt to low-level perceptual property (e.g., speech quality) at a short timescale.

Spike frequency adaptation supports network computations on temporally dispersed information

For solving tasks such as recognizing a song, answering a question, or inverting a sequence of symbols, cortical microcircuits need to integrate and manipulate information that was dispersed over time during the preceding seconds. Creating biologically realistic models for the underlying computations, especially with spiking neurons and for behaviorally relevant integration time spans, is notoriously difficult. We examine the role of spike frequency adaptation in such computations and find that it has a surprisingly large impact. The inclusion of this well-known property of a substantial fraction of neurons in the neocortex - especially in higher areas of the human neocortex - moves the performance of spiking neural network models for computations on network inputs that are temporally dispersed from a fairly low level up to the performance level of the human brain.

Remembering the Past to See the Future

In addition to the role that our visual system plays in determining what we are seeing right now, visual computations contribute in important ways to predicting what we will see next. While the role of memory in creating future predictions is often overlooked, efficient predictive computation requires the use of information about the past to estimate future events. In this article, we introduce a framework for understanding the relationship between memory and visual prediction and review the two classes of mechanisms that the visual system relies on to create future predictions. We also discuss the principles that define the mapping from predictive computations to predictive mechanisms and how downstream brain areas interpret the predictive signals computed by the visual system. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Efficient and adaptive sensory codes

The ability to adapt to changes in stimulus statistics is a hallmark of sensory systems. Here, we developed a theoretical framework that can account for the dynamics of adaptation from an information processing perspective. We use this framework to optimize and analyze adaptive sensory codes, and we show that codes optimized for stationary environments can suffer from prolonged periods of poor performance when the environment changes. To mitigate the adversarial effects of these environmental changes, sensory systems must navigate tradeoffs between the ability to accurately encode incoming stimuli and the ability to rapidly detect and adapt to changes in the distribution of these stimuli. We derive families of codes that balance these objectives, and we demonstrate their close match to experimentally observed neural dynamics during mean and variance adaptation. Our results provide a unifying perspective on adaptation across a range of sensory systems, environments, and sensory tasks.