SLoN: a spiking looming perception network exploiting neural encoding and processing in ON/OFF channels

Looming perception, the ability to sense approaching objects, is crucial for the survival of humans and animals. After hundreds of millions of years of evolutionary development, biological entities have evolved efficient and robust looming perception visual systems. However, current artificial vision systems fall short of such capabilities. In this study, we propose a novel spiking neural network for looming perception that mimics biological vision to communicate motion information through action potentials or spikes, providing a more realistic approach than previous artificial neural networks based on sum-then-activate operations. The proposed spiking looming perception network (SLoN) comprises three core components. Neural encoding, known as phase coding, transforms video signals into spike trains, introducing the concept of phase delay to depict the spatial-temporal competition between phasic excitatory and inhibitory signals shaping looming selectivity. To align with biological substrates where visual signals are bifurcated into parallel ON/OFF channels encoding brightness increments and decrements separately to achieve specific selectivity to ON/OFF-contrast stimuli, we implement eccentric down-sampling at the entrance of ON/OFF channels, mimicking the foveal region of the mammalian receptive field with higher acuity to motion, computationally modeled with a leaky integrate-and-fire (LIF) neuronal network. The SLoN model is deliberately tested under various visual collision scenarios, ranging from synthetic to real-world stimuli. A notable achievement is that the SLoN selectively spikes for looming features concealed in visual streams against other categories of movements, including translating, receding, grating, and near misses, demonstrating robust selectivity in line with biological principles. Additionally, the efficacy of the ON/OFF channels, the phase coding with delay, and the eccentric visual processing are further investigated to demonstrate their effectiveness in looming perception. The cornerstone of this study rests upon showcasing a new paradigm for looming perception that is more biologically plausible in light of biological motion perception.

Gamma amplitude is coupled to opposed hippocampal theta-phase states during the encoding and retrieval of episodic memories in humans

Computational models and in vivo studies in rodents suggest that the emergence of gamma activity (40-140 Hz) during memory encoding and retrieval is coupled to opposed-phase states of the underlying hippocampal theta rhythm (4-9 Hz).^{1,2,3,4,5,6,7,8,9,10} However, direct evidence for whether human hippocampal gamma-modulated oscillatory activity in memory processes is coupled to opposed-phase states of the ongoing theta rhythm remains elusive. Here, we recorded local field potentials (LFPs) directly from the hippocampus of 10 patients with epilepsy, using depth electrodes. We used a memory encoding and retrieval task whereby trial unique sequences of pictures depicting real-life episodes were presented, and 24 h later, participants were asked to recall them upon the appearance of the first picture of the encoded episodic sequence. We found theta-to-gamma cross-frequency coupling that was specific to the hippocampus during both the encoding and retrieval of episodic memories. We also revealed that gamma was coupled to opposing theta phases during both encoding and recall processes. Additionally, we observed that the degree of theta-gamma phase opposition between encoding and recall was associated with participants' memory performance, so gamma power was modulated by theta phase for both remembered and forgotten trials, although only for remembered trials the dominant theta phase was different for encoding and recall trials. The current results offer direct empirical evidence in support of hippocampal theta-gamma phase opposition models in human long-term memory and provide fundamental insights into mechanistic predictions derived from computational and animal work, thereby contributing to establishing similarities and differences across species.

An oscillatory pipelining mechanism supporting previewing during visual exploration and reading

Humans have a remarkable ability to efficiently explore visual scenes and text using eye movements. Humans typically make eye movements (saccades) every ~250 ms. Since saccade initiation and execution take 100 ms, this leaves only ~150 ms to recognize the fixated object (or word) while simultaneously previewing candidates for the next saccade goal. We propose a pipelining mechanism where serial processing occurs within a specific brain region, whereas parallel processing occurs across different brain regions. The mechanism is timed by alpha oscillations that coordinate the saccades, visual recognition, and previewing in the cortical hierarchy. Consequently, the neuronal mechanism supporting natural vision and saccades must be studied in unison to uncover the brain mechanisms supporting visual exploration and reading.

Ripple band phase precession of place cell firing during replay

Neuronal “replay,” in which place cell firing during rest recapitulates recently experienced trajectories, is thought to mediate the transmission of information from hippocampus to neocortex, but the mechanism for this transmission is unknown. Here, we show that replay uses a phase code to represent spatial trajectories by the phase of firing relative to the 150- to 250-Hz “ripple” oscillations that accompany replay events. This phase code is analogous to the theta phase precession of place cell firing during navigation, in which place cells fire at progressively earlier phases of the 6- to 12-Hz theta oscillation as their place field is traversed, providing information about self-location that is additional to the rate code and a necessary precursor of replay. Thus, during replay, each ripple cycle contains a “forward sweep” of decoded locations along the recapitulated trajectory. Our results indicate a novel encoding of trajectory information during replay and implicates phase coding as a general mechanism by which the hippocampus transmits experienced and replayed sequential information to downstream targets.

•

Place cells fire at successively earlier ripple band phases during replay

•

Ripple band firing phase during replay encodes location within the place field

•

This produces forward sweeps of place cell activity during each ripple cycle

Neural Coding in Spiking Neural Networks: A Comparative Study for Robust Neuromorphic Systems

Various hypotheses of information representation in brain, referred to as neural codes, have been proposed to explain the information transmission between neurons. Neural coding plays an essential role in enabling the brain-inspired spiking neural networks (SNNs) to perform different tasks. To search for the best coding scheme, we performed an extensive comparative study on the impact and performance of four important neural coding schemes, namely, rate coding, time-to-first spike (TTFS) coding, phase coding, and burst coding. The comparative study was carried out using a biological 2-layer SNN trained with an unsupervised spike-timing-dependent plasticity (STDP) algorithm. Various aspects of network performance were considered, including classification accuracy, processing latency, synaptic operations (SOPs), hardware implementation, network compression efficacy, input and synaptic noise resilience, and synaptic fault tolerance. The classification tasks on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets were applied in our study. For hardware implementation, area and power consumption were estimated for these coding schemes, and the network compression efficacy was analyzed using pruning and quantization techniques. Different types of input noise and noise variations in the datasets were considered and applied. Furthermore, the robustness of each coding scheme to the non-ideality-induced synaptic noise and fault in analog neuromorphic systems was studied and compared. Our results show that TTFS coding is the best choice in achieving the highest computational performance with very low hardware implementation overhead. TTFS coding requires 4x/7.5x lower processing latency and 3.5x/6.5x fewer SOPs than rate coding during the training/inference process. Phase coding is the most resilient scheme to input noise. Burst coding offers the highest network compression efficacy and the best overall robustness to hardware non-idealities for both training and inference processes. The study presented in this paper reveals the design space created by the choice of each coding scheme, allowing designers to frame each scheme in terms of its strength and weakness given a designs' constraints and considerations in neuromorphic systems.

Two septal-entorhinal GABAergic projections differentially control coding properties of spatially tuned neurons in the medial entorhinal cortex

Septal parvalbumin-expressing (PV+) and calbindin-expressing (CB+) projections inhibit low-threshold and fast-spiking interneurons, respectively, in the medial entorhinal cortex (MEC). We investigate how the two inputs control neuronal activity in the MEC in freely moving mice. Stimulation of PV+ and CB+ terminals causes disinhibition of spatially tuned MEC neurons, but exerts differential effects on temporal coding and burst firing. Thus, recruitment of PV+ projections disrupts theta-rhythmic firing of MEC neurons, while stimulation of CB+ projections increases burst firing of grid cells and enhances phase precession in a cell-type-specific manner. Inactivation of septal PV+ or CB+ neurons differentially affects context, reference, and working memory. Together, our results reveal how specific connectivity of septal GABAergic projections with MEC interneurons translates into differential modulation of MEC neuronal coding.

A Model of Memory Linking Time to Space

The storage of temporally precise spike patterns can be realized by a single neuron. A spiking neural network (SNN) model is utilized to demonstrate the ability to precisely recall a spike pattern after presenting a single input. We show by using a simulation study that the temporal properties of input patterns can be transformed into spatial patterns of local dendritic spikes. The localization of time-points of spikes is facilitated by phase-shift of the subthreshold membrane potential oscillations (SMO) in the dendritic branches, which modifies their excitability. In reference to the points in time of the arriving input, the dendritic spikes are triggered in different branches. To store spatially distributed patterns, two unsupervised learning mechanisms are utilized. Either synaptic weights to the branches, spatial representation of the temporal input pattern, are enhanced by spike-timing-dependent plasticity (STDP) or the oscillation power of SMOs in spiking branches is increased by dendritic spikes. For retrieval, spike bursts activate stored spatiotemporal patterns in dendritic branches, which reactivate the original somatic spike patterns. The simulation of the prototypical model demonstrates the principle, how linking time to space enables the storage of temporal features of an input. Plausibility, advantages, and some variations of the proposed model are also discussed.

Neural Coding of Thermal Preferences in the Nematode Caenorhabditis elegans

Animals are capable to modify sensory preferences according to past experiences. Surrounded by ever-changing environments, they continue assigning a hedonic value to a sensory stimulus. It remains to be elucidated however how such alteration of sensory preference is encoded in the nervous system. Here we show that past experiences alter temporal interaction between the calcium responses of sensory neurons and their postsynaptic interneurons in the nematode Caenorhabditis elegans. C. elegans exhibits thermotaxis, in which its temperature preference is modified by the past feeding experience: well-fed animals are attracted toward their past cultivation temperature on a thermal gradient, whereas starved animals lose that attraction. By monitoring calcium responses simultaneously from both AFD thermosensory neurons and their postsynaptic AIY interneurons in well-fed and starved animals under time-varying thermal stimuli, we found that past feeding experiences alter phase shift between AFD and AIY calcium responses. Furthermore, the difference in neuronal activities between well-fed and starved animals observed here are able to explain the difference in the behavioral output on a thermal gradient between well-fed and starved animals. Although previous studies have shown that C. elegans executes thermotaxis by regulating amplitude or frequency of the AIY response, our results proposed a new mechanism by which thermal preference is encoded by phase shift between AFD and AIY activities. Given these observations, thermal preference is likely to be computed on synapses between AFD and AIY neurons. Such a neural strategy may enable animals to enrich information processing within defined connectivity via dynamic alterations of synaptic communication.

Cortical Coding of Whisking Phase during Surface Whisking

In rodent whisker sensation, whisker position signals, including whisking phase, are integrated with touch signals to enable spatially accurate tactile perception, but other functions of phase coding are unclear. We investigate how phase coding affects the neural coding of surface features during surface whisking. In mice performing rough-smooth discrimination, S1 units exhibit much stronger phase tuning during surface whisking than in prior studies of whisking in air. Among putative pyramidal cells, preferred phase tiles phase space, but protraction phases are strongly over-represented. Fast-spiking units are nearly all protraction tuned. This protraction bias increases the coding of stick-slip whisker events during protraction, suggesting that surface features are preferentially encoded during protraction. Correspondingly, protraction-tuned units encode rough-smooth texture better than retraction-tuned units and encode the precise spatial location of surface ridges with higher acuity. This suggests that protraction is the main information-gathering phase for high-resolution surface features, with phase coding organized to support this function.

Combined Phase-Rate Coding by Persistently Active Neurons as a Mechanism for Maintaining Multiple Items in Working Memory in Humans

Maintaining multiple items in working memory (WM) is central to human behavior. Persistently active neurons are thought to be a mechanism to maintain WMs, but it remains unclear how such activity is coordinated when multiple items are kept in memory. We show that memoranda-selective persistently active neurons in the human medial temporal lobe phase lock to ongoing slow-frequency (1-7 Hz) oscillations during WM maintenance. The properties of phase locking are dependent on memory content and load. During high memory loads, the phase of the oscillatory activity to which neurons phase lock provides information about memory content not available in the firing rate of the neurons. We provide a computational model that reveals that inhibitory-feedback-mediated competition between multiple persistently active neurons reproduces this phenomenon. This work reveals a mechanism for the active maintenance of multiple items in WM that relies on persistently active neurons whose activation is orchestrated by oscillatory activity.

Neural Entrainment and Attentional Selection in the Listening Brain

The streams of sounds we typically attend to abound in acoustic regularities. Neural entrainment is seen as an important mechanism that the listening brain exploits to attune to these regularities and to enhance the representation of attended sounds. We delineate the neurophysiology underlying this mechanism and review entrainment alongside its more pragmatic signature, often called 'speech tracking'. The latter has become a popular analytical approach to trace the reflection of acoustic and linguistic information at different levels of granularity, from neurophysiology to neuroimaging. As we discuss, the concept of entrainment offers both a putative neurophysiological mechanism for selective listening and a versatile window onto the neural basis of hearing and speech comprehension.

Modulation of local field potentials and neuronal activity in primate hippocampus during saccades

Primates use saccades to gather information about objects and their relative spatial arrangement, a process essential for visual perception and memory. It has been proposed that signals linked to saccades reset the phase of local field potential (LFP) oscillations in the hippocampus, providing a temporal window for visual signals to activate neurons in this region and influence memory formation. We investigated this issue by measuring hippocampal LFPs and spikes in two macaques performing different tasks with unconstrained eye movements. We found that LFP phase clustering (PC) in the alpha/beta (8-16 Hz) frequencies followed foveation onsets, while PC in frequencies lower than 8 Hz followed spontaneous saccades, even on a homogeneous background. Saccades to a solid grey background were not followed by increases in local neuronal firing, whereas saccades toward appearing visual stimuli were. Finally, saccade parameters correlated with LFPs phase and amplitude: saccade direction correlated with delta (≤4 Hz) phase, and saccade amplitude with theta (4-8 Hz) power. Our results suggest that signals linked to saccades reach the hippocampus, producing synchronization of delta/theta LFPs without a general activation of local neurons. Moreover, some visual inputs co-occurring with saccades produce LFP synchronization in the alpha/beta bands and elevated neuronal firing. Our findings support the hypothesis that saccade-related signals enact sensory input-dependent plasticity and therefore memory formation in the primate hippocampus.

Regulation of gamma-frequency oscillation by feedforward inhibition: A computational modeling study

Throughout the brain, reciprocally connected excitatory and inhibitory neurons interact to produce gamma-frequency oscillations. The emergent gamma rhythm synchronizes local neural activity and helps to select which cells should fire in each cycle. We previously found that such excitation-inhibition microcircuits, however, have a potentially significant caveat: the frequency of the gamma oscillation and the level of selection (i.e., the percentage of cells that are allowed to fire) vary with the magnitude of the input signal. In networks with varying levels of brain activity, such a feature may produce undesirable instability on the time and spatial structure of the neural signal with a potential for adversely impacting important neural processing mechanisms. Here we propose that feedforward inhibition solves the latter instability problem of the excitation-inhibition microcircuit. Using computer simulations, we show that the feedforward inhibitory signal reduces the dependence of both the frequency of population oscillation and the level of selection on the magnitude of the input excitation. Such a mechanism can produce stable gamma oscillations with its frequency determined only by the properties of the feedforward network, as observed in the hippocampus. As feedforward and feedback inhibition motifs commonly appear together in the brain, we hypothesize that their interaction underlies a robust implementation of general computational principles of neural processing involved in several cognitive tasks, including the formation of cell assemblies and the routing of information between brain areas.

Detection of Multiway Gamma Coordination Reveals How Frequency Mixing Shapes Neural Dynamics

A principle of communication technology, frequency mixing, describes how novel oscillations are generated when rhythmic inputs converge on a nonlinearly activating target. As expected given that neurons are nonlinear integrators, it was demonstrated that neuronal networks exhibit mixing in response to imposed oscillations of known frequencies. However, determining when mixing occurs in spontaneous conditions, where weaker or more variable rhythms prevail, has remained impractical. Here, we show that, by exploiting the predicted phase (rather than frequency) relationships between oscillations, the contributions of mixing can be readily identified, even in small samples of noisy data. Assessment of extracellular data using this approach revealed that frequency mixing is widely expressed in a state- and region-dependent manner and that oscillations emerging from mixing entrain unit activity. Frequency mixing is thus intrinsic to the structure of neural activity and contributes importantly to neural dynamics.

Spike Time Synchrony in the Absence of Continuous Oscillations

Eliav et al., (2018) recently reported hippocampal-entorhinal spiking in bats occurring preferentially at specific phases of nonrhythmic extracellular voltage fluctuations. This disentanglement of phase coding from continuous oscillations raises new questions about the importance of rhythms for neuronal coordination.

Space and Time: The Hippocampus as a Sequence Generator

Neural computations are often compared to instrument-measured distance or duration, and such relationships are interpreted by a human observer. However, neural circuits do not depend on human-made instruments but perform computations relative to an internally defined rate-of-change. While neuronal correlations with external measures, such as distance or duration, can be observed in spike rates or other measures of neuronal activity, what matters for the brain is how such activity patterns are utilized by downstream neural observers. We suggest that hippocampal operations can be described by the sequential activity of neuronal assemblies and their internally defined rate of change without resorting to the concept of space or time.

Transformation of a Spatial Map across the Hippocampal-Lateral Septal Circuit

The hippocampus constructs a map of the environment. How this "cognitive map" is utilized by other brain regions to guide behavior remains unexplored. To examine how neuronal firing patterns in the hippocampus are transmitted and transformed, we recorded neurons in its principal subcortical target, the lateral septum (LS). We observed that LS neurons carry reliable spatial information in the phase of action potentials, relative to hippocampal theta oscillations, while the firing rates of LS neurons remained uninformative. Furthermore, this spatial phase code had an anatomical microstructure within the LS and was bound to the hippocampal spatial code by synchronous gamma frequency cell assemblies. Using a data-driven model, we show that rate-independent spatial tuning arises through the dynamic weighting of CA1 and CA3 cell assemblies. Our findings demonstrate that transformation of the hippocampal spatial map depends on higher-order theta-dependent neuronal sequences. VIDEO ABSTRACT.

Mechanosensory-Based Phase Coding of Odor Identity in the Olfactory Bulb

Mitral and tufted (M/T) cells in the olfactory bulb produce rich temporal patterns of activity in response to different odors. However, it remains unknown how these temporal patterns are generated and how they are utilized in olfaction. Here we show that temporal patterning effectively discriminates between the two sensory modalities detected by olfactory sensory neurons (OSNs): odor and airflow-driven mechanical signals. Sniff-induced mechanosensation generates glomerulus-specific oscillatory activity in M/T cells, whose phase was invariant across airflow speed. In contrast, odor stimulation caused phase shifts (phase coding). We also found that odor-evoked phase shifts are concentration invariant and stable across multiple sniff cycles, contrary to the labile nature of rate coding. The loss of oscillatory mechanosensation impaired the precision and stability of phase coding, demonstrating its role in olfaction. We propose that phase, not rate, coding is a robust encoding strategy of odor identity and is ensured by airflow-induced mechanosensation in OSNs.

Phase-amplitude coupling supports phase coding in human ECoG

Prior studies have shown that high-frequency activity (HFA) is modulated by the phase of low-frequency activity. This phenomenon of phase-amplitude coupling (PAC) is often interpreted as reflecting phase coding of neural representations, although evidence for this link is still lacking in humans. Here, we show that PAC indeed supports phase-dependent stimulus representations for categories. Six patients with medication-resistant epilepsy viewed images of faces, tools, houses, and scenes during simultaneous acquisition of intracranial recordings. Analyzing 167 electrodes, we observed PAC at 43% of electrodes. Further inspection of PAC revealed that category specific HFA modulations occurred at different phases and frequencies of the underlying low-frequency rhythm, permitting decoding of categorical information using the phase at which HFA events occurred. These results provide evidence for categorical phase-coded neural representations and are the first to show that PAC coincides with phase-dependent coding in the human brain.

HCN channels enhance spike phase coherence and regulate the phase of spikes and LFPs in the theta-frequency range

What are the implications for the existence of subthreshold ion channels, their localization profiles, and plasticity on local field potentials (LFPs)? Here, we assessed the role of hyperpolarization-activated cyclic-nucleotide-gated (HCN) channels in altering hippocampal theta-frequency LFPs and the associated spike phase. We presented spatiotemporally randomized, balanced theta-modulated excitatory and inhibitory inputs to somatically aligned, morphologically realistic pyramidal neuron models spread across a cylindrical neuropil. We computed LFPs from seven electrode sites and found that the insertion of an experimentally constrained HCN-conductance gradient into these neurons introduced a location-dependent lead in the LFP phase without significantly altering its amplitude. Further, neurons fired action potentials at a specific theta phase of the LFP, and the insertion of HCN channels introduced large lags in this spike phase and a striking enhancement in neuronal spike-phase coherence. Importantly, graded changes in either HCN conductance or its half-maximal activation voltage resulted in graded changes in LFP and spike phases. Our conclusions on the impact of HCN channels on LFPs and spike phase were invariant to changes in neuropil size, to morphological heterogeneity, to excitatory or inhibitory synaptic scaling, and to shifts in the onset phase of inhibitory inputs. Finally, we selectively abolished the inductive lead in the impedance phase introduced by HCN channels without altering neuronal excitability and found that this inductive phase lead contributed significantly to changes in LFP and spike phase. Our results uncover specific roles for HCN channels and their plasticity in phase-coding schemas and in the formation and dynamic reconfiguration of neuronal cell assemblies.

Do gamma oscillations play a role in cerebral cortex?

Gamma rhythm (which has a center frequency between 30 and 80 Hz) is modulated by cognitive mechanisms such as attention and memory, and has been hypothesized to play a role in mediating these processes by supporting communication channels between cortical areas or encoding information in its phase. We highlight several issues related to gamma rhythms, such as low and inconsistent power, its dependence on low-level stimulus features, problems due to conduction delays, and contamination due to spike-related activity that makes accurate estimation of gamma phase difficult. Gamma rhythm could be a potentially useful signature of excitation-inhibition interactions in the brain, but whether it also provides a mechanism for information processing or coding remains an open question.

Spatial computation with gamma oscillations

Gamma oscillations in cortex have been extensively studied with relation to behavior in both humans and animal models; however, their computational role in the processing of behaviorally relevant signals is still not clear. One oft-overlooked characteristic of gamma oscillations is their spatial distribution over the cortical space and the computational consequences of such an organization. Here, we advance the proposal that the spatial organization of gamma oscillations is of major importance for their function. The interaction of specific spatial distributions of oscillations with the functional topography of cortex enables select amplification of neuronal signals, which supports perceptual and cognitive processing.

Rhythmic fluctuations in evidence accumulation during decision making in the human brain

Categorical choices are preceded by the accumulation of sensory evidence in favor of one action or another. Current models describe evidence accumulation as a continuous process occurring at a constant rate, but this view is inconsistent with accounts of a psychological refractory period during sequential information processing. During multisample perceptual categorization, we found that the neural encoding of momentary evidence in human electrical brain signals and its subsequent impact on choice fluctuated rhythmically according to the phase of ongoing parietal delta oscillations (1-3 Hz). By contrast, lateralized beta-band power (10-30 Hz) overlying human motor cortex encoded the integrated evidence as a response preparation signal. These findings draw a clear distinction between central and motor stages of perceptual decision making, with successive samples of sensory evidence competing to pass through a serial processing bottleneck before being mapped onto action.

Functional micro-organization of primary visual cortex: receptive field analysis of nearby neurons

It is well established that multiple stimulus dimensions (e.g., orientation and spatial frequency) are mapped onto the surface of striate cortex. However, the detailed organization of neurons within a local region of striate cortex remains unclear. Within a vertical column, do all neurons have the same response selectivities? And if not, how do they most commonly differ and why? To address these questions, we recorded from nearby pairs of simple cells and made detailed spatiotemporal maps of their receptive fields. From these maps, we extracted and analyzed a variety of response metrics. Our results provide new insights into the local organization of striate cortex. First, we show that nearby neurons seldom have very similar receptive fields, when these fields are characterized in space and time. Thus, there may be less redundancy within a column than previously thought. Moreover, we show that correlated discharge increases with receptive field similarity; thus, the local dissimilarity between neurons may allow for noise reduction by response pooling. Second, we show that several response variables are clustered within striate cortex, including some that have not received much attention such as response latency and temporal frequency. We also demonstrate that other parameters are not clustered, including the spatial phase (or symmetry) of the receptive field. Third, we show that spatial phase is the single parameter that accounts for most of the difference between receptive fields of nearby neurons. We consider the implications of this local diversity of spatial phase for population coding and construction of higher-order receptive fields.