Internally Generated Predictions Enhance Neural and Behavioral Detection of Sensory Stimuli in an Electric Fish

Cerebellar Purkinje cells in male macaques combine sensory and motor information to predict the sensory consequences of active self-motion

Accurate perception and behavior rely on distinguishing sensory signals arising from unexpected events from those originating from our own voluntary actions. In the vestibular system, sensory input that is the consequence of active self-motion is canceled early at the first central stage of processing to ensure postural and perceptual stability. However, the source of the required cancellation signal was unknown. Here, we show that the cerebellum combines sensory and motor-related information to predict the sensory consequences of active self-motion. Recordings during attempted but unrealized head movements in two male rhesus monkeys, revealed that the motor-related signals encoded by anterior vermis Purkinje cells explain their altered sensitivity to active versus passive self-motion. Further, a model combining responses from ~40 Purkinje cells accounted for the cancellation observed in early vestibular pathways. These findings establish how cerebellar Purkinje cells predict sensory outcomes of self-movements, resolving a long-standing issue of sensory signal suppression during self-motion.

Active sensing: Eavesdropping on your neighbor to locate prey

When animals using active sensing, e.g., sonar or an electric organ discharge, cooperate while foraging, the emitted sound or electric field is available to neighboring conspecifics. Experimental and modelling studies have shown that an electric fish can use the discharge of neighbors to extend their own electrosensory prey detection range.

Collective sensing in electric fish

A number of organisms, including dolphins, bats and electric fish, possess sophisticated active sensory systems that use self-generated signals (for example, acoustic or electrical emissions) to probe the environment1,2. Studies of active sensing in social groups have typically focused on strategies for minimizing interference from conspecific emissions2-4. However, it is well known from engineering that multiple spatially distributed emitters and receivers can greatly enhance environmental sensing (for example, multistatic radar and sonar)5-8. Here we provide evidence from modelling, neural recordings and behavioural experiments that the African weakly electric fish Gnathonemus petersii utilizes the electrical pulses of conspecifics to extend its electrolocation range, discriminate objects and increase information transmission. These results provide evidence for a new, collective mode of active sensing in which individual perception is enhanced by the energy emissions of nearby group members.

Collective Sensing in Electric Fish

A number of organisms, including dolphins, bats, and electric fish, possess sophisticated active sensory systems that use self-generated signals (e.g. acoustic or electrical emissions) to probe the environment1,2. Studies of active sensing in social groups have typically focused on strategies for minimizing interference from conspecific emissions2-4. However, it is well-known from engineering that multiple spatially distributed emitters and receivers can greatly enhance environmental sensing (e.g. multistatic radar and sonar)5-8. Here we provide evidence from modeling, neural recordings, and behavioral experiments that the African weakly electric fish Gnathonemus petersii utilizes the electrical pulses of conspecifics to extend electrolocation range, discriminate objects, and increase information transmission. These results suggest a novel, collective mode of active sensing in which individual perception is enhanced by the energy emissions of nearby group members.

An internal model for canceling self-generated sensory input in freely behaving electric fish

Internal models that predict the sensory consequences of motor actions are vital for sensory, motor, and cognitive functions. However, the relationship between motor action and sensory input is complex, often varying from one moment to another depending on the state of the animal and the environment. The neural mechanisms for generating predictions under such challenging, real-world conditions remain largely unknown. Using novel methods for underwater neural recording, a quantitative analysis of unconstrained behavior, and computational modeling, we provide evidence for an unexpectedly sophisticated internal model at the first stage of active electrosensory processing in mormyrid fish. Closed-loop manipulations reveal that electrosensory lobe neurons are capable of simultaneously learning and storing multiple predictions of the sensory consequences of motor commands specific to different sensory states. These results provide mechanistic insights into how internal motor signals and information about the sensory environment are combined within a cerebellum-like circuitry to predict the sensory consequences of natural behavior.

A mechanism for differential control of axonal and dendritic spiking underlying learning in a cerebellum-like circuit

In addition to the action potentials used for axonal signaling, many neurons generate dendritic "spikes" associated with synaptic plasticity. However, in order to control both plasticity and signaling, synaptic inputs must be able to differentially modulate the firing of these two spike types. Here, we investigate this issue in the electrosensory lobe (ELL) of weakly electric mormyrid fish, where separate control over axonal and dendritic spikes is essential for the transmission of learned predictive signals from inhibitory interneurons to the output stage of the circuit. Through a combination of experimental and modeling studies, we uncover a novel mechanism by which sensory input selectively modulates the rate of dendritic spiking by adjusting the amplitude of backpropagating axonal action potentials. Interestingly, this mechanism does not require spatially segregated synaptic inputs or dendritic compartmentalization but relies instead on an electrotonically distant spike initiation site in the axon-a common biophysical feature of neurons.

Neural response and representation: Facial expressions in scenes

Previous studies have shown that the brain generates expectations based on scenes, which affects facial expression recognition. However, although facial expressions are known to interact with perception, the mechanism underlying this interaction remains poorly understood. Here, we used frequency labeling and decoding techniques to reveal the effects of scene-based expectation on the amplitude and representational strength of neural activity. We also reduced the relative reliability between expectation and sensory input by blurring facial expressions to further investigate the effects of this relative reliability on the pattern of neural activation and representation. Participants viewed emotional changes in unblurred or blurred facial expressions, which flickered at a rate of 6 Hz within a scene. We found that facial expressions that were congruent with the emotional significance of the scene elicited a larger steady-state visual evoked potential amplitude than did facial expressions that were incongruent with the emotional significance of a scene, in both unblurred and blurred conditions. We also found that expected facial expression representations were stronger than unexpected representations during the unblurred condition. In the blurred condition, unexpected representations were stronger than expected representations. Taken together, these results suggested that facial expression processing in the visual cortex is modulated by top-down signals. The relative reliability of expectation and sensory input moderated the influence of a scene on facial expression representation. Furthermore, our study showed that neural activation amplitudes did not correspond to representational strength.

Cancelling cancellation? Sensorimotor control, agency, and prediction

For decades, classic theories of action control and action awareness have been built around the idea that the brain predictively 'cancels' expected action outcomes from perception. However, recent research casts doubt over this basic premise. What do these new findings mean for classic accounts of action? Should we now 'cancel' old data, theories and approaches generated under this idea? In this paper, we argue 'No'. While doubts about predictive cancellation may urge us to fundamentally rethink how predictions shape perception, the wider pyramid using these ideas to explain action control and agentic experiences can remain largely intact. Some adaptive functions assigned to predictive cancellation can be achieved through quasi-predictive processes, that influence perception without actively tracking the probabilistic structure of the environment. Other functions may rely upon truly predictive processes, but not require that these predictions cancel perception. Appreciating the role of these processes may help us to move forward in explaining how agents optimise their interactions with the external world, even if predictive cancellation is cancelled from theory.

Entorhinal cortex directs learning-related changes in CA1 representations

Learning-related changes in brain activity are thought to underlie adaptive behaviours1,2. For instance, the learning of a reward site by rodents requires the development of an over-representation of that location in the hippocampus3–6. How this learning-related change occurs remains unknown. Here we recorded hippocampal CA1 population activity as mice learned a reward location on a linear treadmill. Physiological and pharmacological evidence suggests that the adaptive over-representation required behavioural timescale synaptic plasticity (BTSP)7. BTSP is known to be driven by dendritic voltage signals that we proposed were initiated by input from entorhinal cortex layer 3 (EC3). Accordingly, the CA1 over-representation was largely removed by optogenetic inhibition of EC3 activity. Recordings from EC3 neurons revealed an activity pattern that could provide an instructive signal directing BTSP to generate the over-representation. Consistent with this function, our observations show that exposure to a second environment possessing a prominent reward-predictive cue resulted in both EC3 activity and CA1 place field density that were more elevated at the cue than at the reward. These data indicate that learning-related changes in the hippocampus are produced by synaptic plasticity directed by an instructive signal from the EC3 that seems to be specifically adapted to the behaviourally relevant features of the environment.

Action Enhances Predicted Touch

It is widely believed that predicted tactile action outcomes are perceptually attenuated. The present experiments determined whether predictive mechanisms necessarily generate attenuation or, instead, can enhance perception-as typically observed in sensory cognition domains outside of action. We manipulated probabilistic expectations in a paradigm often used to demonstrate tactile attenuation. Adult participants produced actions and subsequently rated the intensity of forces on a static finger. Experiment 1 confirmed previous findings that action outcomes are perceived less intensely than passive stimulation but demonstrated more intense perception when active finger stimulation was removed. Experiments 2 and 3 manipulated prediction explicitly and found that expected touch during action is perceived more intensely than unexpected touch. Computational modeling suggested that expectations increase the gain afforded to expected tactile signals. These findings challenge a central tenet of prominent motor control theories and demonstrate that sensorimotor predictions do not exhibit a qualitatively distinct influence on tactile perception.

Communication with self, friends and foes in active-sensing animals

Animals that rely on electrolocation and echolocation for navigation and prey detection benefit from sensory systems that can operate in the dark, allowing them to exploit sensory niches with few competitors. Active sensing has been characterized as a highly specialized form of communication, whereby an echolocating or electrolocating animal serves as both the sender and receiver of sensory information. This characterization inspires a framework to explore the functions of sensory channels that communicate information with the self and with others. Overlapping communication functions create challenges for signal privacy and fidelity by leaving active-sensing animals vulnerable to eavesdropping, jamming and masking. Here, we present an overview of active-sensing systems used by weakly electric fish, bats and odontocetes, and consider their susceptibility to heterospecific and conspecific jamming signals and eavesdropping. Susceptibility to interference from signals produced by both conspecifics and prey animals reduces the fidelity of electrolocation and echolocation for prey capture and foraging. Likewise, active-sensing signals may be eavesdropped, increasing the risk of alerting prey to the threat of predation or the risk of predation to the sender, or drawing competition to productive foraging sites. The evolutionary success of electrolocating and echolocating animals suggests that they effectively counter the costs of active sensing through rich and diverse adaptive behaviors that allow them to mitigate the effects of competition for signal space and the exploitation of their signals.

Achieving stable dynamics in neural circuits

The brain consists of many interconnected networks with time-varying, partially autonomous activity. There are multiple sources of noise and variation yet activity has to eventually converge to a stable, reproducible state (or sequence of states) for its computations to make sense. We approached this problem from a control-theory perspective by applying contraction analysis to recurrent neural networks. This allowed us to find mechanisms for achieving stability in multiple connected networks with biologically realistic dynamics, including synaptic plasticity and time-varying inputs. These mechanisms included inhibitory Hebbian plasticity, excitatory anti-Hebbian plasticity, synaptic sparsity and excitatory-inhibitory balance. Our findings shed light on how stable computations might be achieved despite biological complexity. Crucially, our analysis is not limited to analyzing the stability of fixed geometric objects in state space (e.g points, lines, planes), but rather the stability of state trajectories which may be complex and time-varying.

Corollary Discharge Versus Efference Copy: Distinct Neural Signals in Speech Preparation Differentially Modulate Auditory Responses

Actions influence sensory processing in a complex way to shape behavior. For example, during actions, a copy of motor signals-termed "corollary discharge" (CD) or "efference copy" (EC)-can be transmitted to sensory regions and modulate perception. However, the sole inhibitory function of the motor copies is challenged by mixed empirical observations as well as multifaceted computational demands for behaviors. We hypothesized that the content in the motor signals available at distinct stages of actions determined the nature of signals (CD vs. EC) and constrained their modulatory functions on perceptual processing. We tested this hypothesis using speech in which we could precisely control and quantify the course of action. In three electroencephalography (EEG) experiments using a novel delayed articulation paradigm, we found that preparation without linguistic contents suppressed auditory responses to all speech sounds, whereas preparing to speak a syllable selectively enhanced the auditory responses to the prepared syllable. A computational model demonstrated that a bifurcation of motor signals could be a potential algorithm and neural implementation to achieve the distinct functions in the motor-to-sensory transformation. These results suggest that distinct motor signals are generated in the motor-to-sensory transformation and integrated with sensory input to modulate perception.

Active Sensing: Constancy Requires Change

We constantly generate movements in order to enhance our ability to perceive the external environment. New research on electric fish has used augmented reality to demonstrate that animals dynamically regulate their movements to maintain variability in their sensory input.

A cerebellum-like circuit in the lateral line system of fish cancels mechanosensory input associated with its own movements

An animal's own movement exerts a profound impact on sensory input to its nervous system. Peripheral sensory receptors do not distinguish externally generated stimuli from stimuli generated by an animal's own behavior (reafference) - although the animal often must. One way that nervous systems can solve this problem is to provide movement-related signals (copies of motor commands and sensory feedback) to sensory systems, which can then be used to generate predictions that oppose or cancel out sensory responses to reafference. Here, we studied the use of movement-related signals to generate sensory predictions in the lateral line medial octavolateralis nucleus (MON) of the little skate. In the MON, mechanoreceptive afferents synapse on output neurons that also receive movement-related signals from central sources, via a granule cell parallel fiber system. This parallel fiber system organization is characteristic of a set of so-called cerebellum-like structures. Cerebellum-like structures have been shown to support predictive cancellation of reafference in the electrosensory systems of fish and the auditory system of mice. Here, we provide evidence that the parallel fiber system in the MON can generate predictions that are negative images of (and therefore cancel) sensory input associated with respiratory and fin movements. The MON, found in most aquatic vertebrates, is probably one of the most primitive cerebellum-like structures and a starting point for cerebellar evolution. The results of this study contribute to a growing body of work that uses an evolutionary perspective on the vertebrate cerebellum to understand its functional diversity in animal behavior.

Synaptic Plasticity Forms and Functions

Synaptic plasticity, the activity-dependent change in neuronal connection strength, has long been considered an important component of learning and memory. Computational and engineering work corroborate the power of learning through the directed adjustment of connection weights. Here we review the fundamental elements of four broadly categorized forms of synaptic plasticity and discuss their functional capabilities and limitations. Although standard, correlation-based, Hebbian synaptic plasticity has been the primary focus of neuroscientists for decades, it is inherently limited. Three-factor plasticity rules supplement Hebbian forms with neuromodulation and eligibility traces, while true supervised types go even further by adding objectives and instructive signals. Finally, a recently discovered hippocampal form of synaptic plasticity combines the above elements, while leaving behind the primary Hebbian requirement. We suggest that the effort to determine the neural basis of adaptive behavior could benefit from renewed experimental and theoretical investigation of more powerful directed types of synaptic plasticity.

Bottom-up and top-down neural signatures of disordered multi-talker speech perception in adults with normal hearing

In social settings, speech waveforms from nearby speakers mix together in our ear canals. Normally, the brain unmixes the attended speech stream from the chorus of background speakers using a combination of fast temporal processing and cognitive active listening mechanisms. Of >100,000 patient records,~10% of adults visited our clinic because of reduced hearing, only to learn that their hearing was clinically normal and should not cause communication difficulties. We found that multi-talker speech intelligibility thresholds varied widely in normal hearing adults, but could be predicted from neural phase-locking to frequency modulation (FM) cues measured with ear canal EEG recordings. Combining neural temporal fine structure processing, pupil-indexed listening effort, and behavioral FM thresholds accounted for 78% of the variability in multi-talker speech intelligibility. The disordered bottom-up and top-down markers of poor multi-talker speech perception identified here could inform the design of next-generation clinical tests for hidden hearing disorders.

Task-Related Sensorimotor Adjustments Increase the Sensory Range in Electrolocation

Perception and motor control traditionally are studied separately. However, motor activity can serve as a scaffold to shape the sensory flow. This tight link between motor actions and sensing is particularly evident in active sensory systems. Here, we investigate how the weakly electric mormyrid fish Gnathonemus petersii of undetermined sex structure their sensing and motor behavior while learning a perceptual task. We find systematic adjustments of the motor behavior that correlate with an increased performance. Using a model to compute the electrosensory input, we show that these behavioral adjustments improve the sensory input. As we find low neuronal detection thresholds at the level of medullary electrosensory neurons, it seems that the behavior-driven improvements of the sensory input are highly suitable to overcome the sensory limitations, thereby increasing the sensory range. Our results show that motor control is an active component of sensory learning, demonstrating that a detailed understanding of contribution of motor actions to sensing is needed to understand even seemingly simple behaviors.SIGNIFICANCE STATEMENT Motor-guided sensation and perception are intertwined, with motor behavior serving as a scaffold to shape the sensory input. We characterized how the weakly electric mormyrid fish Gnathonemus petersii, as it learns a perceptual task, restructures its sensorimotor behavior. We find that systematic adjustments of the motor behavior correlate with increased performance and a shift of the sensory attention of the animal. Analyzing the afferent electrosensory input shows that a significant gain in information results from these sensorimotor adjustments. Our results show that motor control can be an active component of sensory learning. Researching the sensory corollaries of motor control thus can be crucial to understand sensory sensation and perception under naturalistic conditions.

Continual Learning in a Multi-Layer Network of an Electric Fish

Distributing learning across multiple layers has proven extremely powerful in artificial neural networks. However, little is known about how multi-layer learning is implemented in the brain. Here, we provide an account of learning across multiple processing layers in the electrosensory lobe (ELL) of mormyrid fish and report how it solves problems well known from machine learning. Because the ELL operates and learns continuously, it must reconcile learning and signaling functions without switching its mode of operation. We show that this is accomplished through a functional compartmentalization within intermediate layer neurons in which inputs driving learning differentially affect dendritic and axonal spikes. We also find that connectivity based on learning rather than sensory response selectivity assures that plasticity at synapses onto intermediate-layer neurons is matched to the requirements of output neurons. The mechanisms we uncover have relevance to learning in the cerebellum, hippocampus, and cerebral cortex, as well as in artificial systems.

Novel Functions of Feedback in Electrosensory Processing

Environmental signals act as input and are processed across successive stages in the brain to generate a meaningful behavioral output. However, a ubiquitous observation is that descending feedback projections from more central to more peripheral brain areas vastly outnumber ascending feedforward projections. Such projections generally act to modify how sensory neurons respond to afferent signals. Recent studies in the electrosensory system of weakly electric fish have revealed novel functions for feedback pathways in that their transformation of the afferent input generates neural firing rate responses to sensory signals mediating perception and behavior. In this review, we focus on summarizing these novel and recently uncovered functions and put them into context by describing the more "classical" functions of feedback in the electrosensory system. We further highlight the parallels between the electrosensory system and other systems as well as outline interesting future directions.

Learning to Expect: Predicting Sounds During Movement Is Related to Sensorimotor Association During Listening

Sensory experiences, such as sound, often result from our motor actions. Over time, repeated sound-producing performance can generate sensorimotor associations. However, it is not clear how sensory and motor information are associated. Here, we explore if sensory prediction is associated with the formation of sensorimotor associations during a learning task. We recorded event-related potentials (ERPs) while participants produced index and little finger-swipes on a bespoke device, generating novel sounds. ERPs were also obtained as participants heard those sounds played back. Peak suppression was compared to assess sensory prediction. Additionally, transcranial magnetic stimulation (TMS) was used during listening to generate finger-motor evoked potentials (MEPs). MEPs were recorded before and after training upon hearing these sounds, and then compared to reveal sensorimotor associations. Finally, we explored the relationship between these components. Results demonstrated that an increased positive-going peak (e.g., P2) and a suppressed negative-going peak (e.g., N2) were recorded during action, revealing some sensory prediction outcomes (P2: p = 0.050, ηp2 = 0.208; N2: p = 0.001, ηp2 = 0.474). Increased MEPs were also observed upon hearing congruent sounds compared with incongruent sounds (i.e., associated to a finger), demonstrating precise sensorimotor associations that were not present before learning (Index finger: p < 0.001, ηp2 = 0.614; Little finger: p < 0.001, ηp2 = 0.529). Consistent with our broad hypotheses, a negative association between the MEPs in one finger during listening and ERPs during performance of the other was observed (Index finger MEPs and Fz N1 action ERPs; r = −0.655, p = 0.003). Overall, data suggest that predictive mechanisms are associated with the fine-tuning of sensorimotor associations.

Corollary Discharge Signals in the Cerebellum

The cerebellum is known to make movements fast, smooth, and accurate. Many hypotheses emphasize the role of the cerebellum in computing learned predictions important for sensorimotor calibration and feedforward control of movements. Hypotheses of the computations performed by the cerebellum in service of motor control borrow heavily from control systems theory, with models that frequently invoke copies of motor commands, called corollary discharge. This review describes evidence for corollary discharge inputs to the cerebellum and highlights the hypothesized roles for this information in cerebellar motor-related computations. Insights into the role of corollary discharge in motor control, described here, are intended to inform the exciting but still untested roles of corollary discharge in cognition, perception, and thought control relevant in psychiatric disorders.

Generalization of learned responses in the mormyrid electrosensory lobe

Appropriate generalization of learned responses to new situations is vital for adaptive behavior. We provide a circuit-level account of generalization in the electrosensory lobe (ELL) of weakly electric mormyrid fish. Much is already known in this system about a form of learning in which motor corollary discharge signals cancel responses to the uninformative input evoked by the fish's own electric pulses. However, for this cancellation to be useful under natural circumstances, it must generalize accurately across behavioral regimes, specifically different electric pulse rates. We show that such generalization indeed occurs in ELL neurons, and develop a circuit-level model explaining how this may be achieved. The mechanism involves regularized synaptic plasticity and an approximate matching of the temporal dynamics of motor corollary discharge and electrosensory inputs. Recordings of motor corollary discharge signals in mossy fibers and granule cells provide direct evidence for such matching.