Perception as a closed-loop convergence process

Sound-seeking before and after hearing loss in mice

How we move our bodies affects how we perceive sound. For instance, head movements help us to better localize the source of a sound and to compensate for asymmetric hearing loss. However, many auditory experiments are designed to restrict head and body movements. To study the role of movement in hearing, we developed a behavioral task called sound-seeking that rewarded freely moving mice for tracking down an ongoing sound source. Over the course of learning, mice more efficiently navigated to the sound. Next, we asked how sound-seeking was affected by hearing loss induced by surgical removal of the malleus from the middle ear. After bilateral hearing loss sound-seeking performance drastically declined and did not recover. In striking contrast, after unilateral hearing loss mice were only transiently impaired and then recovered their sound-seek ability over about a week. Throughout recovery, unilateral mice increasingly relied on a movement strategy of sequentially checking potential locations for the sound source. In contrast, the startle reflex (an innate auditory behavior) was preserved after unilateral hearing loss and abolished by bilateral hearing loss without recovery over time. In sum, mice compensate with body movement for permanent unilateral damage to the peripheral auditory system. Looking forward, this paradigm provides an opportunity to examine how movement enhances perception and enables resilient adaptation to sensory disorders.

The neuron as a direct data-driven controller

In the quest to model neuronal function amid gaps in physiological data, a promising strategy is to develop a normative theory that interprets neuronal physiology as optimizing a computational objective. This study extends current normative models, which primarily optimize prediction, by conceptualizing neurons as optimal feedback controllers. We posit that neurons, especially those beyond early sensory areas, steer their environment toward a specific desired state through their output. This environment comprises both synaptically interlinked neurons and external motor sensory feedback loops, enabling neurons to evaluate the effectiveness of their control via synaptic feedback. To model neurons as biologically feasible controllers which implicitly identify loop dynamics, infer latent states, and optimize control we utilize the contemporary direct data-driven control (DD-DC) framework. Our DD-DC neuron model explains various neurophysiological phenomena: the shift from potentiation to depression in spike-timing-dependent plasticity with its asymmetry, the duration and adaptive nature of feedforward and feedback neuronal filters, the imprecision in spike generation under constant stimulation, and the characteristic operational variability and noise in the brain. Our model presents a significant departure from the traditional, feedforward, instant-response McCulloch-Pitts-Rosenblatt neuron, offering a modern, biologically informed fundamental unit for constructing neural networks.

A sensory-motor theory of the neocortex

Recent neurophysiological and neuroanatomical studies suggest a close interaction between sensory and motor processes across the neocortex. Here, I propose that the neocortex implements active predictive coding (APC): each cortical area estimates both latent sensory states and actions (including potentially abstract actions internal to the cortex), and the cortex as a whole predicts the consequences of actions at multiple hierarchical levels. Feedback from higher areas modulates the dynamics of state and action networks in lower areas. I show how the same APC architecture can explain (1) how we recognize an object and its parts using eye movements, (2) why perception seems stable despite eye movements, (3) how we learn compositional representations, for example, part-whole hierarchies, (4) how complex actions can be planned using simpler actions, and (5) how we form episodic memories of sensory-motor experiences and learn abstract concepts such as a family tree. I postulate a mapping of the APC model to the laminar architecture of the cortex and suggest possible roles for cortico-cortical and cortico-subcortical pathways.

Neural representation in active inference: Using generative models to interact with-and understand-the lived world

This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.

Neural dynamics of predictive timing and motor engagement in music listening

Why do humans spontaneously dance to music? To test the hypothesis that motor dynamics reflect predictive timing during music listening, we created melodies with varying degrees of rhythmic predictability (syncopation) and asked participants to rate their wanting-to-move (groove) experience. Degree of syncopation and groove ratings are quadratically correlated. Magnetoencephalography data showed that, while auditory regions track the rhythm of melodies, beat-related 2-hertz activity and neural dynamics at delta (1.4 hertz) and beta (20 to 30 hertz) rates in the dorsal auditory pathway code for the experience of groove. Critically, the left sensorimotor cortex coordinates these groove-related delta and beta activities. These findings align with the predictions of a neurodynamic model, suggesting that oscillatory motor engagement during music listening reflects predictive timing and is effected by interaction of neural dynamics along the dorsal auditory pathway.

Neurocomputational mechanisms underlying perception and sentience in the neocortex

The basis for computation in the brain is the quantum threshold of "soliton," which accompanies the ion changes of the action potential, and the refractory membrane at convergences. Here, we provide a logical explanation from the action potential to a neuronal model of the coding and computation of the retina. We also explain how the visual cortex operates through quantum-phase processing. In the small-world network, parallel frequencies collide into definable patterns of distinct objects. Elsewhere, we have shown how many sensory cells are meanly sampled from a single neuron and that convergences of neurons are common. We also demonstrate, using the threshold and refractory period of a quantum-phase pulse, that action potentials diffract across a neural network due to the annulment of parallel collisions in the phase ternary computation (PTC). Thus, PTC applied to neuron convergences results in a collective mean sampled frequency and is the only mathematical solution within the constraints of the brain neural networks (BNN). In the retina and other sensory areas, we discuss how this information is initially coded and then understood in terms of network abstracts within the lateral geniculate nucleus (LGN) and visual cortex. First, by defining neural patterning within a neural network, and then in terms of contextual networks, we demonstrate that the output of frequencies from the visual cortex contains information amounting to abstract representations of objects in increasing detail. We show that nerve tracts from the LGN provide time synchronization to the neocortex (defined as the location of the combination of connections of the visual cortex, motor cortex, auditory cortex, etc.). The full image is therefore combined in the neocortex with other sensory modalities so that it receives information about the object from the eye and all the abstracts that make up the object. Spatial patterns in the visual cortex are formed from individual patterns illuminating the retina, and memory is encoded by reverberatory loops of computational action potentials (CAPs). We demonstrate that a similar process of PTC may take place in the cochlea and associated ganglia, as well as ascending information from the spinal cord, and that this function should be considered universal where convergences of neurons occur.

Sound-seeking before and after hearing loss in mice

How we move our bodies affects how we perceive sound. For instance, we can explore an environment to seek out the source of a sound and we can use head movements to compensate for hearing loss. How we do this is not well understood because many auditory experiments are designed to limit head and body movements. To study the role of movement in hearing, we developed a behavioral task called sound-seeking that rewarded mice for tracking down an ongoing sound source. Over the course of learning, mice more efficiently navigated to the sound. We then asked how auditory behavior was affected by hearing loss induced by surgical removal of the malleus from the middle ear. An innate behavior, the auditory startle response, was abolished by bilateral hearing loss and unaffected by unilateral hearing loss. Similarly, performance on the sound-seeking task drastically declined after bilateral hearing loss and did not recover. In striking contrast, mice with unilateral hearing loss were only transiently impaired on sound-seeking; over a recovery period of about a week, they regained high levels of performance, increasingly reliant on a different spatial sampling strategy. Thus, even in the face of permanent unilateral damage to the peripheral auditory system, mice recover their ability to perform a naturalistic sound-seeking task. This paradigm provides an opportunity to examine how body movement enables better hearing and resilient adaptation to sensory deprivation.

The Motor of Time: Coupling Action to Temporally Predictable Events Heightens Perception

Timing and motor function share neural circuits and dynamics, which underpin their close and synergistic relationship. For instance, the temporal predictability of a sensory event optimizes motor responses to that event. Knowing when an event is likely to occur lowers response thresholds, leading to faster and more efficient motor behavior though in situations of response conflict can induce impulsive and inappropriate responding. In turn, through a process of active sensing, coupling action to temporally predictable sensory input enhances perceptual processing. Action not only hones perception of the event's onset or duration, but also boosts sensory processing of its non-temporal features such as pitch or shape. The effects of temporal predictability on motor behavior and sensory processing involve motor and left parietal cortices and are mediated by changes in delta and beta oscillations in motor areas of the brain.

Adaptive oscillators support Bayesian prediction in temporal processing

Humans excel at predictively synchronizing their behavior with external rhythms, as in dance or music performance. The neural processes underlying rhythmic inferences are debated: whether predictive perception relies on high-level generative models or whether it can readily be implemented locally by hard-coded intrinsic oscillators synchronizing to rhythmic input remains unclear and different underlying computational mechanisms have been proposed. Here we explore human perception for tone sequences with some temporal regularity at varying rates, but with considerable variability. Next, using a dynamical systems perspective, we successfully model the participants behavior using an adaptive frequency oscillator which adjusts its spontaneous frequency based on the rate of stimuli. This model better reflects human behavior than a canonical nonlinear oscillator and a predictive ramping model-both widely used for temporal estimation and prediction-and demonstrate that the classical distinction between absolute and relative computational mechanisms can be unified under this framework. In addition, we show that neural oscillators may constitute hard-coded physiological priors-in a Bayesian sense-that reduce temporal uncertainty and facilitate the predictive processing of noisy rhythms. Together, the results show that adaptive oscillators provide an elegant and biologically plausible means to subserve rhythmic inference, reconciling previously incompatible frameworks for temporal inferential processes.

Autogenous cerebral processes: an invitation to look at the brain from inside out

While external stimulation can reliably trigger neuronal activity, cerebral processes can operate independently from the environment. In this study, we conceptualize autogenous cerebral processes (ACPs) as intrinsic operations of the brain that exist on multiple scales and can influence or shape stimulus responses, behavior, homeostasis, and the physiological state of an organism. We further propose that the field should consider exploring to what extent perception, arousal, behavior, or movement, as well as other cognitive functions previously investigated mainly regarding their stimulus-response dynamics, are ACP-driven.

Sensorimotor Integration Supporting Perception Requires Syngap1 Expression in Cortex

Perception, a cognitive construct, emerges through sensorimotor integration (SMI). The molecular and cellular mechanisms that shape SMI within circuits that promote cognition are poorly understood. Here, we demonstrate that expression of the autism/intellectual disability gene, Syngap1, in mouse cortical excitatory neurons promotes touch sensitivity required to elicit perceptual behaviors. Cortical Syngap1 expression enabled touch-induced feedback signals within sensorimotor loops by assembling circuits that support tactile sensitivity. These circuits also encoded correlates of attention that promoted self-generated whisker movements underlying purposeful and sustained object exploration. As Syngap1 deficient animals explored objects with whiskers, relatively weak touch signals were integrated with relatively strong motor signals. This produced a signal-to-noise deficit consistent with impaired tactile sensitivity, reduced tactile exploration, and weak tactile learning. Thus, Syngap1 expression in cortex promotes tactile perception by assembling circuits that integrate touch and whisker motor signals. Deficient Syngap1 expression likely contributes to cognitive impairment through abnormal top-down SMI.

Thalamocortical loops as temporal demodulators across senses

Sensory information is coded in space and in time. The organization of neuronal activity in space maintains straightforward relationships with the spatial organization of the perceived environment. In contrast, the temporal organization of neuronal activity is not trivially related to external features due to sensor motion. Still, the temporal organization shares similar principles across sensory modalities. Likewise, thalamocortical circuits exhibit common features across senses. Focusing on touch, vision, and audition, we review their shared coding principles and suggest that thalamocortical systems include circuits that allow analogous recoding mechanisms in all three senses. These thalamocortical circuits constitute oscillations-based phase-locked loops, that translate temporally-coded sensory information to rate-coded cortical signals, signals that can integrate information across sensory and motor modalities. The loop also allows predictive locking to the onset of future modulations of the sensory signal. The paper thus suggests a theoretical framework in which a common thalamocortical mechanism implements temporal demodulation across senses.

Monosynaptic trans-collicular pathways link mouse whisker circuits to integrate somatosensory and motor cortical signals

The superior colliculus (SC), a conserved midbrain node with extensive long-range connectivity throughout the brain, is a key structure for innate behaviors. Descending cortical pathways are increasingly recognized as central control points for SC-mediated behaviors, but how cortico-collicular pathways coordinate SC activity at the cellular level is poorly understood. Moreover, despite the known role of the SC as a multisensory integrator, the involvement of the SC in the somatosensory system is largely unexplored in comparison to its involvement in the visual and auditory systems. Here, we mapped the connectivity of the whisker-sensitive region of the SC in mice with trans-synaptic and intersectional tracing tools and in vivo electrophysiology. The results reveal a novel trans-collicular connectivity motif in which neurons in motor- and somatosensory cortices impinge onto the brainstem-SC-brainstem sensory-motor arc and onto SC-midbrain output pathways via only one synapse in the SC. Intersectional approaches and optogenetically assisted connectivity quantifications in vivo reveal convergence of motor and somatosensory cortical input on individual SC neurons, providing a new framework for sensory-motor integration in the SC. More than a third of the cortical recipient neurons in the whisker SC are GABAergic neurons, which include a hitherto unknown population of GABAergic projection neurons targeting thalamic nuclei and the zona incerta. These results pinpoint a whisker region in the SC of mice as a node for the integration of somatosensory and motor cortical signals via parallel excitatory and inhibitory trans-collicular pathways, which link cortical and subcortical whisker circuits for somato-motor integration.

Ausubel's meaningful learning re-visited

This review provides a critique of David Ausubel's theory of meaningful learning and the use of advance organizers in teaching. It takes into account the developments in cognition and neuroscience which have taken place in the 50 or so years since he advanced his ideas, developments which challenge our understanding of cognitive structure and the recall of prior learning. These include (i) how effective questioning to ascertain previous knowledge necessitates in-depth Socratic dialogue; (ii) how many findings in cognition and neuroscience indicate that memory may be non-representational, thereby affecting our interpretation of student recollections; (iii) the now recognised dynamism of memory; (iv) usefully regarding concepts as abilities or simulators and skills; (v) acknowledging conscious and unconscious memory and imagery; (vi) how conceptual change involves conceptual coexistence and revision; (vii) noting linguistic and neural pathways as a result of experience and neural selection; and (viii) recommending that wider concepts of scaffolding should be adopted, particularly given the increasing focus on collaborative learning in a technological world.

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

The channel capacity of multilevel linguistic features constrains speech comprehension

Humans are expert at processing speech but how this feat is accomplished remains a major question in cognitive neuroscience. Capitalizing on the concept of channel capacity, we developed a unified measurement framework to investigate the respective influence of seven acoustic and linguistic features on speech comprehension, encompassing acoustic, sub-lexical, lexical and supra-lexical levels of description. We show that comprehension is independently impacted by all these features, but at varying degrees and with a clear dominance of the syllabic rate. Comparing comprehension of French words and sentences further reveals that when supra-lexical contextual information is present, the impact of all other features is dramatically reduced. Finally, we estimated the channel capacity associated with each linguistic feature and compared them with their generic distribution in natural speech. Our data reveal that while acoustic modulation, syllabic and phonemic rates unfold respectively at 5, 5, and 12 Hz in natural speech, they are associated with independent processing bottlenecks whose channel capacity are of 15, 15 and 35 Hz, respectively, as suggested by neurophysiological theories. They moreover point towards supra-lexical contextual information as the feature limiting the flow of natural speech. Overall, this study reveals how multilevel linguistic features constrain speech comprehension.

Latency shortening with enhanced sparseness and responsiveness in V1 during active visual sensing

In natural vision, neuronal responses to visual stimuli occur due to self-initiated eye movements. Here, we compare single-unit activity in the primary visual cortex (V1) of non-human primates to flashed natural scenes (passive vision condition) to when they freely explore the images by self-initiated eye movements (active vision condition). Active vision enhances the number of neurons responding, and the response latencies become shorter and less variable across neurons. The increased responsiveness and shortened latency during active vision were not explained by increased visual contrast. While the neuronal activities in all layers of V1 show enhanced responsiveness and shortened latency, a significant increase in lifetime sparseness during active vision is observed only in the supragranular layer. These findings demonstrate that the neuronal responses become more distinct in active vision than passive vision, interpreted as consequences of top-down predictive mechanisms.

Idiosyncratic selection of active touch for shape perception

Hand movements are essential for tactile perception of objects. However, the specific functions served by active touch strategies, and their dependence on physiological parameters, are unclear and understudied. Focusing on planar shape perception, we tracked at high resolution the hands of 11 participants during shape recognition task. Two dominant hand movement strategies were identified: contour following and scanning. Contour following movements were either tangential to the contour or oscillating perpendicular to it. Scanning movements crossed between distant parts of the shapes’ contour. Both strategies exhibited non-uniform coverage of the shapes’ contours. Idiosyncratic movement patterns were specific to the sensed object. In a second experiment, we have measured the participants’ spatial and temporal tactile thresholds. Significant portions of the variations in hand speed and in oscillation patterns could be explained by the idiosyncratic thresholds. Using data-driven simulations, we show how specific strategy choices may affect receptors activation. These results suggest that motion strategies of active touch adapt to both the sensed object and to the perceiver’s physiological parameters.

Active inference through whiskers

Rodents use whisking to probe actively their environment and to locate objects in space, hence providing a paradigmatic biological example of active sensing. Numerous studies show that the control of whisking has anticipatory aspects. For example, rodents target their whisker protraction to the distance at which they expect objects, rather than just reacting fast to contacts with unexpected objects. Here we characterize the anticipatory control of whisking in rodents as an active inference process. In this perspective, the rodent is endowed with a prior belief that it will touch something at the end of the whisker protraction, and it continuously modulates its whisking amplitude to minimize (proprioceptive and somatosensory) prediction errors arising from an unexpected whisker-object contact, or from a lack of an expected contact. We will use the model to qualitatively reproduce key empirical findings about the ways rodents modulate their whisker amplitude during exploration and the scanning of (expected or unexpected) objects. Furthermore, we will discuss how the components of active inference model can in principle map to the neurobiological circuits of rodent whisking.

Oculo-retinal dynamics can explain the perception of minimal recognizable configurations

Natural vision is a dynamic and continuous process. Under natural conditions, visual object recognition typically involves continuous interactions between ocular motion and visual contrasts, resulting in dynamic retinal activations. In order to identify the dynamic variables that participate in this process and are relevant for image recognition, we used a set of images that are just above and below the human recognition threshold and whose recognition typically requires >2 s of viewing. We recorded eye movements of participants while attempting to recognize these images within trials lasting 3 s. We then assessed the activation dynamics of retinal ganglion cells resulting from ocular dynamics using a computational model. We found that while the saccadic rate was similar between recognized and unrecognized trials, the fixational ocular speed was significantly larger for unrecognized trials. Interestingly, however, retinal activation level was significantly lower during these unrecognized trials. We used retinal activation patterns and oculomotor parameters of each fixation to train a binary classifier, classifying recognized from unrecognized trials. Only retinal activation patterns could predict recognition, reaching 80% correct classifications on the fourth fixation (on average, ∼2.5 s from trial onset). We thus conclude that the information that is relevant for visual perception is embedded in the dynamic interactions between the oculomotor sequence and the image. Hence, our results suggest that ocular dynamics play an important role in recognition and that understanding the dynamics of retinal activation is crucial for understanding natural vision.

Sensorimotor strategies and neuronal representations for shape discrimination

Humans and other animals can identify objects by active touch, requiring the coordination of exploratory motion and tactile sensation. Both the motor strategies and neural representations employed could depend on the subject's goals. We developed a shape discrimination task that challenged head-fixed mice to discriminate concave from convex shapes. Behavioral decoding revealed that mice did this by comparing contacts across whiskers. In contrast, a separate group of mice performing a shape detection task simply summed up contacts over whiskers. We recorded populations of neurons in the barrel cortex, which processes whisker input, and found that individual neurons across the cortical layers encoded touch, whisker motion, and task-related signals. Sensory representations were task-specific: during shape discrimination, but not detection, neurons responded most to behaviorally relevant whiskers, overriding somatotopy. Thus, sensory cortex employs task-specific representations compatible with behaviorally relevant computations.

Spatial integration during active tactile sensation drives orientation perception

Active haptic sensation is critical for object identification, but its neural circuit basis is poorly understood. We combined optogenetics, two-photon imaging, and high-speed behavioral tracking in mice solving a whisker-based object orientation discrimination task. We found that orientation discrimination required animals to summate input from multiple whiskers specifically along the whisker arc. Animals discriminated the orientation of the stimulus per se as their performance was invariant to the location of the presented stimulus. Populations of barrel cortex neurons summated across whiskers to encode each orientation. Finally, acute optogenetic inactivation of the barrel cortex and cell-type-specific optogenetic suppression of layer 4 excitatory neurons degraded performance, implying that infragranular layers alone are not sufficient to solve the task. These data suggest that spatial summation over an active haptic array generates representations of an object's orientation, which may facilitate encoding of complex three-dimensional objects during active exploration.

Sniff-synchronized, gradient-guided olfactory search by freely moving mice

For many organisms, searching for relevant targets such as food or mates entails active, strategic sampling of the environment. Finding odorous targets may be the most ancient search problem that motile organisms evolved to solve. While chemosensory navigation has been well characterized in microorganisms and invertebrates, spatial olfaction in vertebrates is poorly understood. We have established an olfactory search assay in which freely moving mice navigate noisy concentration gradients of airborne odor. Mice solve this task using concentration gradient cues and do not require stereo olfaction for performance. During task performance, respiration and nose movement are synchronized with tens of milliseconds precision. This synchrony is present during trials and largely absent during inter-trial intervals, suggesting that sniff-synchronized nose movement is a strategic behavioral state rather than simply a constant accompaniment to fast breathing. To reveal the spatiotemporal structure of these active sensing movements, we used machine learning methods to parse motion trajectories into elementary movement motifs. Motifs fall into two clusters, which correspond to investigation and approach states. Investigation motifs lock precisely to sniffing, such that the individual motifs preferentially occur at specific phases of the sniff cycle. The allocentric structure of investigation and approach indicates an advantage to sampling both sides of the sharpest part of the odor gradient, consistent with a serial-sniff strategy for gradient sensing. This work clarifies sensorimotor strategies for mouse olfactory search and guides ongoing work into the underlying neural mechanisms.

Integrative Neuroscience of Paramecium, a "Swimming Neuron"

Paramecium is a unicellular organism that swims in fresh water by beating thousands of cilia. When it is stimulated (mechanically, chemically, optically, thermally…), it often swims backward then turns and swims forward again. This "avoiding reaction" is triggered by a calcium-based action potential. For this reason, some authors have called Paramecium a "swimming neuron." This review summarizes current knowledge about the physiological basis of behavior of Paramecium.

Masking interferes with haptic texture perception from sequential exploratory movements

Haptic texture perception is based on sensory information sequentially gathered during several lateral movements ("strokes"). In this process, sensory information of earlier strokes must be preserved in a memory system. We investigated whether this system may be a haptic sensory memory. In the first experiment, participants performed three strokes across each of two textures in a frequency discrimination task. Between the strokes over the first texture, participants explored an intermediate area, which presented either a mask (high-energy tactile pattern) or minimal stimulation (low-energy smooth surface). Perceptual precision was significantly lower with the mask compared with a three-strokes control condition without an intermediate area, approaching performance in a one-stroke-control condition. In contrast, precision in the minimal stimulation condition was significantly better than in the one-stroke control condition and similar to the three-strokes control condition. In a second experiment, we varied the number of strokes across the first stimulus (one, three, five, or seven strokes) and either presented no masking or repeated masking after each stroke. Again, masking between the strokes decreased perceptual precision relative to the control conditions without masking. Precision effects of masking over different numbers of strokes were fit by a proven model on haptic serial integration (Lezkan & Drewing, Attention, Perception, & Psychophysics 80(1): 177-192, 2018b) that modeled masking by repeated disturbances in the ongoing integration. Taken together, results suggest that masking impedes the processes of haptic information preservation and integration. We conclude that a haptic sensory memory, which is comparable to iconic memory in vision, is used for integrating sequentially gathered sensory information.

Active sensory substitution allows fast learning via effective motor-sensory strategies

We examined the development of new sensing abilities in adults by training participants to perceive remote objects through their fingers. Using an Active-Sensing based sensory Substitution device (ASenSub), participants quickly learned to perceive fast via the new modality and preserved their high performance for more than 20 months. Both sighted and blind participants exhibited almost complete transfer of performance from 2D images to novel 3D physical objects. Perceptual accuracy and speed using the ASenSub were, on average, 300% and 600% better than previous reports for 2D images and 3D objects. This improvement is attributed to the ability of the participants to employ their own motor-sensory strategies. Sighted participants dominant strategy was based on motor-sensory convergence on the most informative regions of objects, similarly to fixation patterns in vision. Congenitally, blind participants did not show such a tendency, and many of their exploratory procedures resembled those observed with natural touch.

Unlimited Associative Learning and the origins of consciousness: a primer and some predictions

Over the past two decades, Ginsburg and Jablonka have developed a novel approach to studying the evolutionary origins of consciousness: the Unlimited Associative Learning (UAL) framework. The central idea is that there is a distinctive type of learning that can serve as a transition marker for the evolutionary transition from non-conscious to conscious life. The goal of this paper is to stimulate discussion of the framework by providing a primer on its key claims (Part I) and a clear statement of its main empirical predictions (Part II).

Visual detection is locked to the internal dynamics of cortico-motor control

Movements overtly sample sensory information, making sensory analysis an active-sensing process. In this study, we show that visual information sampling is not just locked to the (overt) movement dynamics but to the internal (covert) dynamics of cortico-motor control. We asked human participants to perform continuous isometric contraction while detecting unrelated and unpredictable near-threshold visual stimuli. The motor output (force) shows zero-lag coherence with brain activity (recorded via electroencephalography) in the beta-band, as previously reported. In contrast, cortical rhythms in the alpha-band systematically forerun the motor output by 200 milliseconds. Importantly, visual detection is facilitated when cortico-motor alpha (not beta) synchronization is enhanced immediately before stimulus onset, namely, at the optimal phase relationship for sensorimotor communication. These findings demonstrate an ongoing coupling between visual sampling and motor control, suggesting the operation of an internal and alpha-cycling visuomotor loop.

Closed loop motor-sensory dynamics in human vision

Vision is obtained with a continuous motion of the eyes. The kinematic analysis of eye motion, during any visual or ocular task, typically reveals two (kinematic) components: saccades, which quickly replace the visual content in the retinal fovea, and drifts, which slowly scan the image after each saccade. While the saccadic exchange of regions of interest (ROIs) is commonly considered to be included in motor-sensory closed-loops, it is commonly assumed that drifts function in an open-loop manner, that is, independent of the concurrent visual input. Accordingly, visual perception is assumed to be based on a sequence of open-loop processes, each initiated by a saccade-triggered retinal snapshot. Here we directly challenged this assumption by testing the dependency of drift kinematics on concurrent visual inputs using real-time gaze-contingent-display. Our results demonstrate a dependency of the trajectory on the concurrent visual input, convergence of speed to condition-specific values and maintenance of selected drift-related motor-sensory controlled variables, all strongly indicative of drifts being included in a closed-loop brain-world process, and thus suggesting that vision is inherently a closed-loop process.

On the cognitive bases of illusionism

Cognitive scientists have paid very little attention to magic as a distinctly human activity capable of creating situations that are considered impossible because they violate expectations and conclude with the apparent transgression of well-established cognitive and natural laws. This illusory experience of the "impossible" entails a very particular cognitive dissonance that is followed by a subjective and complex "magical experience". Here, from a perspective inspired by visual neuroscience and ecological cognition, we propose a set of seven fundamental cognitive phenomena (from attention and perception to memory and decision-making) plus a previous pre-sensory stage that magicians interfere with during the presentation of their effects. By doing so, and using as an example the deconstruction of a classic trick, we show how magic offers novel and powerful insights to study human cognition. Furthermore, live magic performances afford to do so in tasks that are more ecological and context-dependent than those usually exploited in artificial laboratory settings. We thus believe that some of the mysteries of how the brain works may be trapped in the split realities present in every magic effect.

Learning steers the ontogeny of an efficient hunting sequence in zebrafish larvae

Goal-directed behaviors may be poorly coordinated in young animals but, with age and experience, behavior progressively adapts to efficiently exploit the animal’s ecological niche. How experience impinges on the developing neural circuits of behavior is an open question. We have conducted a detailed study of the effects of experience on the ontogeny of hunting behavior in larval zebrafish. We report that larvae with prior experience of live prey consume considerably more prey than naive larvae. This is mainly due to increased capture success and a modest increase in hunt rate. We demonstrate that the initial turn to prey and the final capture manoeuvre of the hunting sequence were jointly modified by experience and that modification of these components predicted capture success. Our findings establish an ethologically relevant paradigm in zebrafish for studying how the brain is shaped by experience to drive the ontogeny of efficient behavior.

Movement-Related Signals in Sensory Areas: Roles in Natural Behavior

Recent studies have demonstrated prominent and widespread movement-related signals in the brain of head-fixed mice, even in primary sensory areas. However, it is still unknown what role these signals play in sensory processing. Why are these sensory areas 'contaminated' by movement signals? During natural behavior, animals actively acquire sensory information as they move through the environment and use this information to guide ongoing actions. In this context, movement-related signals could allow sensory systems to predict self-induced sensory changes and extract additional information about the environment. In this review we summarize recent findings on the presence of movement-related signals in sensory areas and discuss how their study, in the context of natural freely moving behaviors, could advance models of sensory processing.

Anatomically and functionally distinct thalamocortical inputs to primary and secondary mouse whisker somatosensory cortices

Subdivisions of mouse whisker somatosensory thalamus project to cortex in a region-specific and layer-specific manner. However, a clear anatomical dissection of these pathways and their functional properties during whisker sensation is lacking. Here, we use anterograde trans-synaptic viral vectors to identify three specific thalamic subpopulations based on their connectivity with brainstem. The principal trigeminal nucleus innervates ventral posterior medial thalamus, which conveys whisker-selective tactile information to layer 4 primary somatosensory cortex that is highly sensitive to self-initiated movements. The spinal trigeminal nucleus innervates a rostral part of the posterior medial (POm) thalamus, signaling whisker-selective sensory information, as well as decision-related information during a goal-directed behavior, to layer 4 secondary somatosensory cortex. A caudal part of the POm, which apparently does not receive brainstem input, innervates layer 1 and 5A, responding with little whisker selectivity, but showing decision-related modulation. Our results suggest the existence of complementary segregated information streams to somatosensory cortices.

Spontaneous and stimulus-driven rhythmic behaviors in ADHD adults and controls

Many aspects of human behavior are inherently rhythmic, requiring production of rhythmic motor actions as well as synchronizing to rhythms in the environment. It is well-established that individuals with ADHD exhibit deficits in temporal estimation and timing functions, which may impact their ability to accurately produce and interact with rhythmic stimuli. In the current study we seek to understand the specific aspects of rhythmic behavior that are implicated in ADHD. We specifically ask whether they are attributed to imprecision in the internal generation of rhythms or to reduced acuity in rhythm perception. We also test key predictions of the Preferred Period Hypothesis, which suggests that both perceptual and motor rhythmic behaviors are biased towards a specific personal 'default' tempo. To this end, we tested several aspects of rhythmic behavior and the correspondence between them, including spontaneous motor tempo (SMT), preferred auditory perceptual tempo (PPT) and synchronization-continuations tapping in a broad range of rhythms, from sub-second to supra-second intervals. Moreover, we evaluate the intra-subject consistency of rhythmic preferences, as a means for testing the reality and reliability of personal 'default-rhythms'. We used a modified operational definition for assessing SMT and PPT, instructing participants to tap or calibrate the rhythms most comfortable for them to count along with, to avoid subjective interpretations of the task. Our results shed new light on the specific aspect of rhythmic deficits implicated in ADHD adults. We find that individuals with ADHD are primarily challenged in producing and maintaining isochronous self-generated motor rhythms, during both spontaneous and memory-paced tapping. However, they nonetheless exhibit good flexibility for synchronizing to a broad range of external rhythms, suggesting that auditory-motor entrainment for simple rhythms is preserved in ADHD, and that the presence of an external pacer allows overcoming their inherent difficulty in self-generating isochronous motor rhythms. In addition, both groups showed optimal memory-paced tapping for rhythms near their 'counting-based' SMT and PPT, which were slightly faster in the ADHD group. This is in line with the predictions of the Preferred Period Hypothesis, indicating that at least for this well-defined rhythmic behavior (i.e., counting), individuals tend to prefer similar time-scales in both motor production and perceptual evaluation.

Defining "active sensing" through an analysis of sensing energetics: homeoactive and alloactive sensing

The term "active sensing" has been defined in multiple ways. Most strictly, the term refers to sensing that uses self-generated energy to sample the environment (e.g., echolocation). More broadly, the definition includes all sensing that occurs when the sensor is moving (e.g., tactile stimuli obtained by an immobile versus moving fingertip) and, broader still, includes all sensing guided by attention or intent (e.g., purposeful eye movements). The present work offers a framework to help disambiguate aspects of the "active sensing" terminology and reveals properties of tactile sensing unique among all modalities. The framework begins with the well-described "sensorimotor loop," which expresses the perceptual process as a cycle involving four subsystems: environment, sensor, nervous system, and actuator. Using system dynamics, we examine how information flows through the loop. This "sensory-energetic loop" reveals two distinct sensing mechanisms that subdivide active sensing into homeoactive and alloactive sensing. In homeoactive sensing, the animal can change the state of the environment, while in alloactive sensing the animal can alter only the sensor's configurational parameters and thus the mapping between input and output. Given these new definitions, examination of the sensory-energetic loop helps identify two unique characteristics of tactile sensing: 1) in tactile systems, alloactive and homeoactive sensing merge to a mutually controlled sensing mechanism, and 2) tactile sensing may require fundamentally different predictions to anticipate reafferent input. We expect this framework may help resolve ambiguities in the active sensing community and form a basis for future theoretical and experimental work regarding alloactive and homeoactive sensing.

Predictive whisker kinematics reveal context-dependent sensorimotor strategies

Animals actively move their sensory organs in order to acquire sensory information. Some rodents, such as mice and rats, employ cyclic scanning motions of their facial whiskers to explore their proximal surrounding, a behavior known as whisking. Here, we investigated the contingency of whisking kinematics on the animal's behavioral context that arises from both internal processes (attention and expectations) and external constraints (available sensory and motor degrees of freedom). We recorded rat whisking at high temporal resolution in 2 experimental contexts-freely moving or head-fixed-and 2 spatial sensory configurations-a single row or 3 caudal whiskers on each side of the snout. We found that rapid sensorimotor twitches, called pumps, occurring during free-air whisking carry information about the rat's upcoming exploratory direction, as demonstrated by the ability of these pumps to predict consequent head and body locomotion. Specifically, pump behavior during both voluntary motionlessness and imposed head fixation exposed a backward redistribution of sensorimotor exploratory resources. Further, head-fixed rats employed a wide range of whisking profiles to compensate for the loss of head- and body-motor degrees of freedom. Finally, changing the number of intact vibrissae available to a rat resulted in an alteration of whisking strategy consistent with the rat actively reallocating its remaining resources. In sum, this work shows that rats adapt their active exploratory behavior in a homeostatic attempt to preserve sensorimotor coverage under changing environmental conditions and changing sensory capacities, including those imposed by various laboratory conditions.

The Life of Behavior

Neuroscience needs behavior. However, it is daunting to render the behavior of organisms intelligible without suppressing most, if not all, references to life. When animals are treated as passive stimulus-response, disembodied and identical machines, the life of behavior perishes. Here, we distill three biological principles (materiality, agency, and historicity), spell out their consequences for the study of animal behavior, and illustrate them with various examples from the literature. We propose to put behavior back into context, with the brain in a species-typical body and with the animal's body situated in the world; stamp Newtonian time with nested ontogenetic and phylogenetic processes that give rise to individuals with their own histories; and supplement linear cause-and-effect chains and information processing with circular loops of purpose and meaning. We believe that conceiving behavior in these ways is imperative for neuroscience.

Task-Related Sensorimotor Adjustments Increase the Sensory Range in Electrolocation

Perception and motor control traditionally are studied separately. However, motor activity can serve as a scaffold to shape the sensory flow. This tight link between motor actions and sensing is particularly evident in active sensory systems. Here, we investigate how the weakly electric mormyrid fish Gnathonemus petersii of undetermined sex structure their sensing and motor behavior while learning a perceptual task. We find systematic adjustments of the motor behavior that correlate with an increased performance. Using a model to compute the electrosensory input, we show that these behavioral adjustments improve the sensory input. As we find low neuronal detection thresholds at the level of medullary electrosensory neurons, it seems that the behavior-driven improvements of the sensory input are highly suitable to overcome the sensory limitations, thereby increasing the sensory range. Our results show that motor control is an active component of sensory learning, demonstrating that a detailed understanding of contribution of motor actions to sensing is needed to understand even seemingly simple behaviors.SIGNIFICANCE STATEMENT Motor-guided sensation and perception are intertwined, with motor behavior serving as a scaffold to shape the sensory input. We characterized how the weakly electric mormyrid fish Gnathonemus petersii, as it learns a perceptual task, restructures its sensorimotor behavior. We find that systematic adjustments of the motor behavior correlate with increased performance and a shift of the sensory attention of the animal. Analyzing the afferent electrosensory input shows that a significant gain in information results from these sensorimotor adjustments. Our results show that motor control can be an active component of sensory learning. Researching the sensory corollaries of motor control thus can be crucial to understand sensory sensation and perception under naturalistic conditions.

Generalization of Object Localization From Whiskers to Other Body Parts in Freely Moving Rats

Rats can be trained to associate relative spatial locations of objects with the spatial location of rewards. Here we ask whether rats can localize static silent objects with other body parts in the dark, and if so with what resolution. We addressed these questions in trained rats, whose interactions with the objects were tracked at high-resolution before and after whisker trimming. We found that rats can use other body parts, such as trunk and ears, to localize objects. Localization resolution with non-whisking body parts (henceforth, ‘body’) was poorer than that obtained with whiskers, even when left with a single whisker at each side. Part of the superiority of whiskers was obtained via the use of multiple contacts. Transfer from whisker to body localization occurred within one session, provided that body contacts with the objects occurred before whisker trimming, or in the next session otherwise. This transfer occurred whether temporal cues were used for discrimination or when discrimination was based on spatial cues alone. Rats’ decision in each trial was based on the sensory cues acquired in that trial and on decisions and reward locations in previous trials. When sensory cues were acquired by body contacts, rat decisions relied more on the reward location in previous trials. Overall, the results suggest that rats can generalize the idea of relative object location across different body parts, while preferring to rely on whiskers-based localization, which occurs earlier and conveys higher resolution.

Getting Back Into the Loop: The Perceptual-Motor Determinants of Successful Transitions out of Automated Driving

OBJECTIVE

To present a structured, narrative review highlighting research into human perceptual-motor coordination that can be applied to automated vehicle (AV)-human transitions.

BACKGROUND

Manual control of vehicles is made possible by the coordination of perceptual-motor behaviors (gaze and steering actions), where active feedback loops enable drivers to respond rapidly to ever-changing environments. AVs will change the nature of driving to periods of monitoring followed by the human driver taking over manual control. The impact of this change is currently poorly understood.

METHOD

We outline an explanatory framework for understanding control transitions based on models of human steering control. This framework can be summarized as a perceptual-motor loop that requires (a) calibration and (b) gaze and steering coordination. A review of the current experimental literature on transitions is presented in the light of this framework.

RESULTS

The success of transitions are often measured using reaction times, however, the perceptual-motor mechanisms underpinning steering quality remain relatively unexplored.

CONCLUSION

Modeling the coordination of gaze and steering and the calibration of perceptual-motor control will be crucial to ensure safe and successful transitions out of automated driving.

APPLICATION

This conclusion poses a challenge for future research on AV-human transitions. Future studies need to provide an understanding of human behavior that will be sufficient to capture the essential characteristics of drivers reengaging control of their vehicle. The proposed framework can provide a guide for investigating specific components of human control of steering and potential routes to improving manual control recovery.

Real-time contextual feedback for close-loop control of navigation

OBJECTIVE

Close-loop control of brain and behavior will benefit from real-time detection of behavioral events to enable low-latency communication with peripheral devices. In animal experiments, this is typically achieved by using sparsely distributed (embedded) sensors that detect animal presence in select regions of interest. High-speed cameras provide high-density sampling across large arenas, capturing the richness of animal behavior, however, the image processing bottleneck prohibits real-time feedback in the context of rapidly evolving behaviors.

APPROACH

Here we developed an open-source software, named PolyTouch, to track animal behavior in large arenas and provide rapid close-loop feedback in ~5.7 ms, ie. average latency from the detection of an event to analog stimulus delivery, e.g. auditory tone, TTL pulse, when tracking a single body. This stand-alone software is written in JAVA. The included wrapper for MATLAB provides experimental flexibility for data acquisition, analysis and visualization.

MAIN RESULTS

As a proof-of-principle application we deployed the PolyTouch for place awareness training. A user-defined portion of the arena was used as a virtual target; visit (or approach) to the target triggered auditory feedback. We show that mice develop awareness to virtual spaces, tend to stay shorter and move faster when they reside in the virtual target zone if their visits are coupled to relatively high stimulus intensity (⩾49 dB). Thus, close-loop presentation of perceived aversive feedback is sufficient to condition mice to avoid virtual targets within the span of a single session (~20 min).

SIGNIFICANCE

Neuromodulation techniques now allow control of neural activity in a cell-type specific manner in spiking resolution. Using animal behavior to drive closed-loop control of neural activity would help to address the neural basis of behavioral state and environmental context-dependent information processing in the brain.

The Sensorimotor Basis of Whisker-Guided Anteroposterior Object Localization in Head-Fixed Mice

Active tactile perception combines directed motion with sensory signals to generate mental representations of objects in space. Competing models exist for how mice use these signals to determine the precise location of objects along their face. We tested six of these models using behavioral manipulations and statistical learning in head-fixed mice. Trained mice used a whisker to locate a pole in a continuous range of locations along the anteroposterior axis. Mice discriminated locations to ≤0.5 mm (<2°) resolution. Their motor program was noisy, adaptive to touch, and directed to the rewarded range. This exploration produced several sets of sensorimotor features that could discriminate location. Integration of two features, touch count and whisking midpoint at touch, was the simplest model that explained behavior best. These results show how mice locate objects at hyperacute resolution using a learned motor strategy and minimal set of mentally accessible sensorimotor features.

The Energy Homeostasis Principle: Neuronal Energy Regulation Drives Local Network Dynamics Generating Behavior

A major goal of neuroscience is understanding how neurons arrange themselves into neural networks that result in behavior. Most theoretical and experimental efforts have focused on a top-down approach which seeks to identify neuronal correlates of behaviors. This has been accomplished by effectively mapping specific behaviors to distinct neural patterns, or by creating computational models that produce a desired behavioral outcome. Nonetheless, these approaches have only implicitly considered the fact that neural tissue, like any other physical system, is subjected to several restrictions and boundaries of operations. Here, we proposed a new, bottom-up conceptual paradigm: The Energy Homeostasis Principle, where the balance between energy income, expenditure, and availability are the key parameters in determining the dynamics of neuronal phenomena found from molecular to behavioral levels. Neurons display high energy consumption relative to other cells, with metabolic consumption of the brain representing 20% of the whole-body oxygen uptake, contrasting with this organ representing only 2% of the body weight. Also, neurons have specialized surrounding tissue providing the necessary energy which, in the case of the brain, is provided by astrocytes. Moreover, and unlike other cell types with high energy demands such as muscle cells, neurons have strict aerobic metabolism. These facts indicate that neurons are highly sensitive to energy limitations, with Gibb's free energy dictating the direction of all cellular metabolic processes. From this activity, the largest energy, by far, is expended by action potentials and post-synaptic potentials; therefore, plasticity can be reinterpreted in terms of their energy context. Consequently, neurons, through their synapses, impose energy demands over post-synaptic neurons in a close loop-manner, modulating the dynamics of local circuits. Subsequently, the energy dynamics end up impacting the homeostatic mechanisms of neuronal networks. Furthermore, local energy management also emerges as a neural population property, where most of the energy expenses are triggered by sensory or other modulatory inputs. Local energy management in neurons may be sufficient to explain the emergence of behavior, enabling the assessment of which properties arise in neural circuits and how. Essentially, the proposal of the Energy Homeostasis Principle is also readily testable for simple neuronal networks.

Rats Generate Vibrissal Sensory Evidence until Boundary Crossing Triggers a Decision

Behaviors in which primates collect externally generated streams of sensory evidence, such as judgment of random dot motion direction, are explained by a bounded integration decision model. Does this model extend to rodents, and does it account for behavior in which the motor system generates evidence through interactions with the environment? In this study, rats palpated surfaces to identify the texture before them, showing marked trial-to-trial variability in the number of touches prior to expressing their choice. By high-speed video, we tracked whisker kinematic features and characterized how they encoded the contacted texture. Next, we quantified the evidence for each candidate texture transmitted on each touch by the specified whisker kinematic features. The instant of choice was well fit by modeling the brain as an integrator that gives the greatest weight to vibrissal evidence on first touch and exponentially less weight to evidence on successive touches; according to this model, the rat makes a decision when the accumulated quantity of evidence for one texture reaches a boundary. In summary, evidence appears to be accumulated within the brain until sufficient to support a well-grounded choice. These findings extend the framework of bounded sensory integration from primates to rodents and from passively received evidence to evidence that is actively generated by the sensorimotor system.

Posterior thalamic nucleus axon terminals have different structure and functional impact in the motor and somatosensory vibrissal cortices

Rodents extract information about nearby objects from the movement of their whiskers through dynamic computations that are carried out by a network of forebrain structures that includes the thalamus and the primary sensory (S1BF) and motor (M1wk) whisker cortices. The posterior nucleus (Po), a higher order thalamic nucleus, is a key hub of this network, receiving cortical and brainstem sensory inputs and innervating both motor and sensory whisker-related cortical areas. In a recent study in rats, we showed that Po inputs differently impact sensory processing in S1BF and M1wk. Here, in C57BL/6 mice, we measured Po synaptic bouton layer distribution and size, compared cortical unit response latencies to "in vivo" Po activation, and pharmacologically examined the glutamatergic receptor mechanisms involved. We found that, in S1BF, a large majority (56%) of Po axon varicosities are located in layer (L)5a and only 12% in L2-L4, whereas in M1wk this proportion is inverted to 18% and 55%, respectively. Light and electron microscopic measurements showed that Po synaptic boutons in M1wk layers 3-4 are significantly larger (~ 50%) than those in S1BF L5a. Electrical Po stimulation elicits different area-specific response patterns. In S1BF, responses show weak or no facilitation, and involve both ionotropic and metabotropic glutamate receptors, whereas in M1wk, unit responses exhibit facilitation to repetitive stimulation and involve ionotropic NMDA glutamate receptors. Because of the different laminar distribution of axon terminals, synaptic bouton size and receptor mechanisms, the impact of Po signals on M1wk and S1BF, although simultaneous, is likely to be markedly different.