Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

It is widely believed the brain-inspired spiking neural networks have the capability of processing temporal information owing to their dynamic attributes. However, how to understand what kind of mechanisms contributing to the learning ability and exploit the rich dynamic properties of spiking neural networks to satisfactorily solve complex temporal computing tasks in practice still remains to be explored. In this article, we identify the importance of capturing the multi-timescale components, based on which a multi-compartment spiking neural model with temporal dendritic heterogeneity, is proposed. The model enables multi-timescale dynamics by automatically learning heterogeneous timing factors on different dendritic branches. Two breakthroughs are made through extensive experiments: the working mechanism of the proposed model is revealed via an elaborated temporal spiking XOR problem to analyze the temporal feature integration at different levels; comprehensive performance benefits of the model over ordinary spiking neural networks are achieved on several temporal computing benchmarks for speech recognition, visual recognition, electroencephalogram signal recognition, and robot place recognition, which shows the best-reported accuracy and model compactness, promising robustness and generalization, and high execution efficiency on neuromorphic hardware. This work moves neuromorphic computing a significant step toward real-world applications by appropriately exploiting biological observations.

A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time

This article presents an overview of a theory for performing temporal smoothing on temporal signals in such a way that: (i) temporally smoothed signals at coarser temporal scales are guaranteed to constitute simplifications of corresponding temporally smoothed signals at any finer temporal scale (including the original signal) and (ii) the temporal smoothing process is both time-causal and time-recursive, in the sense that it does not require access to future information and can be performed with no other temporal memory buffer of the past than the resulting smoothed temporal scale-space representations themselves. For specific subsets of parameter settings for the classes of linear and shift-invariant temporal smoothing operators that obey this property, it is shown how temporal scale covariance can be additionally obtained, guaranteeing that if the temporal input signal is rescaled by a uniform temporal scaling factor, then also the resulting temporal scale-space representations of the rescaled temporal signal will constitute mere rescalings of the temporal scale-space representations of the original input signal, complemented by a shift along the temporal scale dimension. The resulting time-causal limit kernel that obeys this property constitutes a canonical temporal kernel for processing temporal signals in real-time scenarios when the regular Gaussian kernel cannot be used, because of its non-causal access to information from the future, and we cannot additionally require the temporal smoothing process to comprise a complementary memory of the past beyond the information contained in the temporal smoothing process itself, which in this way also serves as a multi-scale temporal memory of the past. We describe how the time-causal limit kernel relates to previously used temporal models, such as Koenderink's scale-time kernels and the ex-Gaussian kernel. We do also give an overview of how the time-causal limit kernel can be used for modelling the temporal processing in models for spatio-temporal and spectro-temporal receptive fields, and how it more generally has a high potential for modelling neural temporal response functions in a purely time-causal and time-recursive way, that can also handle phenomena at multiple temporal scales in a theoretically well-founded manner. We detail how this theory can be efficiently implemented for discrete data, in terms of a set of recursive filters coupled in cascade. Hence, the theory is generally applicable for both: (i) modelling continuous temporal phenomena over multiple temporal scales and (ii) digital processing of measured temporal signals in real time. We conclude by stating implications of the theory for modelling temporal phenomena in biological, perceptual, neural and memory processes by mathematical models, as well as implications regarding the philosophy of time and perceptual agents. Specifically, we propose that for A-type theories of time, as well as for perceptual agents, the notion of a non-infinitesimal inner temporal scale of the temporal receptive fields has to be included in representations of the present, where the inherent nonzero temporal delay of such time-causal receptive fields implies a need for incorporating predictions from the actual time-delayed present in the layers of a perceptual hierarchy, to make it possible for a representation of the perceptual present to constitute a representation of the environment with timing properties closer to the actual present.

Consciousness, Memory, and the Human Self: Commentary on "Consciousness as a Memory System" by Budson et al (2022)

Philosophical theories have attempted to shed light on the intricate relationships between consciousness and memory since long before this became a major theme in psychology and neuroscience. In the December 2022 issue of Cognitive and Behavioral Neurology , Budson, Richman, and Kensinger (2022) introduced a comprehensive theoretical framework pertaining to the origins of consciousness in relation to the memory system, its implications on our real-time perception of the world, and the neuroanatomical correlates underlying these phenomena. Throughout their paper, Budson et al (2022) focus on their theory's explanatory value regarding several clinical syndromes and experimental findings. In this commentary, we first summarize the theory presented by Budson and colleagues (2022). Then, we suggest a complementary approach of studying the relationships between consciousness and memory through the concept of the human self and its protracted representation through time (so-called mental time travel). Finally, we elaborate on Budson and colleagues' (2022) neuroanatomical explanation to their theory and suggest that adding the concepts of brain networks and cortical gradients may contribute to their theory's interpretability.

Mental travel in the person domain

"Mental travel" is a cognitive concept embodying the human capacity to intentionally disengage from the here and now, and mentally experience the world from different perspectives. We explored how individuals mentally "travel" to the point of view (POV) of other people in varying levels of personal closeness and from these perspectives process these people's social network. Under fMRI, participants were asked to "project" themselves to the POVs of four different people: a close other, a nonclose other, a famous-person, and their own-self, and rate the level of affiliation (closeness) to different individuals in the respective social network. Participants were always faster making judgments from their own POV compared with other POVs (self-projection effect) and for people who were personally closer to their adopted POV (social-distance effect). Brain activity at the medial prefrontal and anterior cingulate cortex in the self-POV was higher, compared with all other conditions. Activity at the right temporoparietal junction and medial parietal cortex was found to distinguish between the personally related (self, close, and nonclose others) and unrelated (famous-person) people. No difference was found between mental travel to the POVs of close and nonclose others. Regardless of POV, the precuneus, anterior cingulate cortex, prefrontal cortex, and temporoparietal junction distinguished between close and distant individuals within the different social networks. Representational similarity analysis implicated the left retrosplenial cortex as crucial for social distance processing across all POVs. These distinctions suggest several constraints regarding our ability to adopt others' POV and process not only ours but also other people's social networks and stress the importance of proximity in social cognition.NEW & NOTEWORTHY Mental-travel, the ability to mentally imagine oneself in a different place and time, is a fundamental cognitive concept. Investigation of mental-travel in the social domain under fMRI revealed that a network of brain regions, largely overlapping the default-mode-network, is responsible for "traveling" to points of view of different others; moreover, this network distinguishes between closer and less-close others, suggesting that mental-travel is a rich dynamical process, encompassing individuals in different proximities and these individuals' social network.

Organization of directed functional connectivity among nodes of ventral attention network reveals the common network mechanisms underlying saliency processing across distinct spatial and spatio-temporal scales

Previous neuroimaging studies have extensively evaluated the structural and functional connectivity of the Ventral Attention Network (VAN) and its role in reorienting attention in the presence of a salient (pop-out) stimulus. However, a detailed understanding of the "directed" functional connectivity within the VAN during the process of reorientation remains elusive. Functional magnetic resonance imaging (fMRI) studies have not adequately addressed this issue due to a lack of appropriate temporal resolution required to capture this dynamic process. The present study investigates the neural changes associated with processing salient distractors operating at a slow and a fast time scale using custom-designed experiment involving visual search on static images and dynamic motion tracking, respectively. We recorded high-density scalp electroencephalography (EEG) from healthy human volunteers, obtained saliency-specific behavioral and spectral changes during the tasks, localized the sources underlying the spectral power modulations with individual-specific structural MRI scans, reconstructed the waveforms of the sources and finally, investigated the causal relationships between the sources using spectral Granger-Geweke Causality (GGC). We found that salient stimuli processing, across tasks with varying spatio-temporal complexities, involves a characteristic modulation in the alpha frequency band which is executed primarily by the nodes of the VAN constituting the temporo-parietal junction (TPJ), the insula and the lateral prefrontal cortex (lPFC). The directed functional connectivity results further revealed the presence of bidirectional interactions among prominent nodes of right-lateralized VAN, corresponding only to the trials with saliency. Thus, our study elucidates the invariant network mechanisms for processing saliency in visual attention tasks across diverse time-scales.