Principles of Intelligence: On Evolutionary Logic of the Brain

Reconfigurable Neuromorphic Computing with 2D Material Heterostructures for Versatile Neural Information Processing

Reconfigurable neuromorphic computing holds promise for advancing energy-efficient neural network implementation and functional versatility. Previous work has focused on emulating specific neural functions rather than an integrated approach. We propose an all two-dimensional (2D) material-based heterostructure capable of performing multiple neuromorphic operations by reconfiguring output terminals in response to stimuli. Specifically, our device can synergistically emulate the key neural elements of the synapse, neuron, and dendrite, which play important and interrelated roles in information processing. Dendrites, the branches that receive and transmit presynaptic action potentials, possess the ability to nonlinearly integrate and filter incoming signals. The proposed heterostructure allows reconfiguration between different operation modes, demonstrating its potential for diverse computing tasks. As a proof of concept, we show that the device can perform basic Boolean logic functions. This highlights its applicability to complex neural-network-based information processing problems. Our integrated neuromorphic approach may advance the development of versatile, low-power neuromorphic hardware.

A Vector Space Model for Neural Network Functions: Inspirations From Similarities Between the Theory of Connectivity and the Logarithmic Time Course of Word Production

The present report examines the coinciding results of two study groups each presenting a power-of-two function to describe network structures underlying perceptual processes in one case and word production during verbal fluency tasks in the other. The former is theorized as neural cliques organized according to the function N = 2 ⁱ - 1, whereas the latter assumes word conglomerations thinkable as tuples following the function N = 2 ⁱ . Both theories assume the innate optimization of energy efficiency to cause the specific connectivity structure. The vast resemblance between both formulae motivated the development of a common formulation. This was obtained by using a vector space model, in which the configuration of neural cliques or connected words is represented by a N-dimensional state vector. A further analysis of the model showed that the entire time course of word production could be derived using basically one single minimal transformation-matrix. This again seems in line with the principle of maximum energy efficiency.

Power-efficient neural network with artificial dendrites

In the nervous system, dendrites, branches of neurons that transmit signals between synapses and soma, play a critical role in processing functions, such as nonlinear integration of postsynaptic signals. The lack of these critical functions in artificial neural networks compromises their performance, for example in terms of flexibility, energy efficiency and the ability to handle complex tasks. Here, by developing artificial dendrites, we experimentally demonstrate a complete neural network fully integrated with synapses, dendrites and soma, implemented using scalable memristor devices. We perform a digit recognition task and simulate a multilayer network using experimentally derived device characteristics. The power consumption is more than three orders of magnitude lower than that of a central processing unit and 70 times lower than that of a typical application-specific integrated circuit chip. This network, equipped with functional dendrites, shows the potential of substantial overall performance improvement, for example by extracting critical information from a noisy background with significantly reduced power consumption and enhanced accuracy.

Quantum-like behavior without quantum physics III : Logic and memory

We employ some of the machinery developed in previous work to investigate the inferential and memory functions of quantum-like neural networks. We set up a logical apparatus to implement this in the form of a Gentzen sequent calculus which codifies some of the combinatory rules for the state spaces of the neuronal networks introduced earlier. We discuss memory storage in this context and along the way find formal proof that synchronicity promotes binding and storage. These results lead to an algorithmic fragment in calculus that simulates the memory function known as pattern completion. This claim is tested by noting that the failure of certain steps in the algorithm leads to memory deficits essentially identical to those found in such pathologies as Alzheimer's dementia, schizophrenia, and certain forms of autism. Moreover, a specific "power-of-two" wiring architecture and computational logic, which have been postulated and observed across many brain circuits, emerge spontaneously from our model. We draw conclusions concerning the possible nature of such mental processes qua computations.

Systematic Analysis of Pigeons' Discrimination of Pixelated Stimuli: A Hierarchical Pattern Recognition System Is Not Identifiable

Pigeons learned to discriminate two different patterns displayed with miniature light-emitting diode arrays. They were then tested with 84 interspersed, non-reinforced degraded pattern pairs. Choices ranged between 100% and 50% for one or other of the patterns. Stimuli consisting of few pixels yielded low choice scores whereas those consisting of many pixels yielded a broad range of scores. Those patterns with a high number of pixels coinciding with those of the rewarded training stimulus were preferred and those with a high number of pixels coinciding with the non-rewarded training pattern were avoided; a discrimination index based on this correlated 0.74 with the pattern choices. Pixels common to both training patterns had a minimal influence. A pixel-by-pixel analysis revealed that eight pixels of one pattern and six pixels of the other pattern played a prominent role in the pigeons’ choices. These pixels were disposed in four and two clusters of neighbouring locations. A summary index calculated on this basis still only yielded a weak 0.73 correlation. The individual pigeons’ data furthermore showed that these clusters were a mere averaging mirage. The pigeons’ performance depends on deep learning in a midbrain-based multimillion synapse neuronal network. Pixelated visual patterns should be helpful when simulating perception of patterns with artificial networks.

O-GlcNAc cycling in the developing, adult and geriatric brain

Hundreds of proteins in the nervous system are modified by the monosaccharide O-GlcNAc. A single protein is often O-GlcNAcylated on several amino acids and the modification of a single site can play a crucial role for the function of the protein. Despite its complexity, only two enzymes add and remove O-GlcNAc from proteins, O-GlcNAc transferase (OGT) and O-GlcNAcase (OGA). Global and local regulation of these enzymes make it possible for O-GlcNAc to coordinate multiple cellular functions at the same time as regulating specific pathways independently from each other. If O-GlcNAcylation is disrupted, metabolic disorder or intellectual disability may ensue, depending on what neurons are affected. O-GlcNAc's promise as a clinical target for developing drugs against neurodegenerative diseases has been recognized for many years. Recent literature puts O-GlcNAc in the forefront among mechanisms that can help us better understand how neuronal circuits integrate diverse incoming stimuli such as fluctuations in nutrient supply, metabolic hormones, neuronal activity and cellular stress. Here the functions of O-GlcNAc in the nervous system are reviewed.

Neural Code-Neural Self-information Theory on How Cell-Assembly Code Rises from Spike Time and Neuronal Variability

A major stumbling block to cracking the real-time neural code is neuronal variability - neurons discharge spikes with enormous variability not only across trials within the same experiments but also in resting states. Such variability is widely regarded as a noise which is often deliberately averaged out during data analyses. In contrast to such a dogma, we put forth the Neural Self-Information Theory that neural coding is operated based on the self-information principle under which variability in the time durations of inter-spike-intervals (ISI), or neuronal silence durations, is self-tagged with discrete information. As the self-information processor, each ISI carries a certain amount of information based on its variability-probability distribution; higher-probability ISIs which reflect the balanced excitation-inhibition ground state convey minimal information, whereas lower-probability ISIs which signify rare-occurrence surprisals in the form of extremely transient or prolonged silence carry most information. These variable silence durations are naturally coupled with intracellular biochemical cascades, energy equilibrium and dynamic regulation of protein and gene expression levels. As such, this silence variability-based self-information code is completely intrinsic to the neurons themselves, with no need for outside observers to set any reference point as typically used in the rate code, population code and temporal code models. Moreover, temporally coordinated ISI surprisals across cell population can inherently give rise to robust real-time cell-assembly codes which can be readily sensed by the downstream neural clique assemblies. One immediate utility of this self-information code is a general decoding strategy to uncover a variety of cell-assembly patterns underlying external and internal categorical or continuous variables in an unbiased manner.

Brain Computation Is Organized via Power-of-Two-Based Permutation Logic

There is considerable scientific interest in understanding how cell assemblies—the long-presumed computational motif—are organized so that the brain can generate intelligent cognition and flexible behavior. The Theory of Connectivity proposes that the origin of intelligence is rooted in a power-of-two-based permutation logic (N = 2ⁱ–1), producing specific-to-general cell-assembly architecture capable of generating specific perceptions and memories, as well as generalized knowledge and flexible actions. We show that this power-of-two-based permutation logic is widely used in cortical and subcortical circuits across animal species and is conserved for the processing of a variety of cognitive modalities including appetitive, emotional and social information. However, modulatory neurons, such as dopaminergic (DA) neurons, use a simpler logic despite their distinct subtypes. Interestingly, this specific-to-general permutation logic remained largely intact although NMDA receptors—the synaptic switch for learning and memory—were deleted throughout adulthood, suggesting that the logic is developmentally pre-configured. Moreover, this computational logic is implemented in the cortex via combining a random-connectivity strategy in superficial layers 2/3 with nonrandom organizations in deep layers 5/6. This randomness of layers 2/3 cliques—which preferentially encode specific and low-combinatorial features and project inter-cortically—is ideal for maximizing cross-modality novel pattern-extraction, pattern-discrimination and pattern-categorization using sparse code, consequently explaining why it requires hippocampal offline-consolidation. In contrast, the nonrandomness in layers 5/6—which consists of few specific cliques but a higher portion of more general cliques projecting mostly to subcortical systems—is ideal for feedback-control of motivation, emotion, consciousness and behaviors. These observations suggest that the brain’s basic computational algorithm is indeed organized by the power-of-two-based permutation logic. This simple mathematical logic can account for brain computation across the entire evolutionary spectrum, ranging from the simplest neural networks to the most complex.

512-Channel and 13-Region Simultaneous Recordings Coupled with Optogenetic Manipulation in Freely Behaving Mice

The development of technologies capable of recording both single-unit activity and local field potentials (LFPs) over a wide range of brain circuits in freely behaving animals is the key to constructing brain activity maps. Although mice are the most popular mammalian genetic model, in vivo neural recording has been traditionally limited to smaller channel count and fewer brain structures because of the mouse’s small size and thin skull. Here, we describe a 512-channel tetrode system that allows us to record simultaneously over a dozen cortical and subcortical structures in behaving mice. This new technique offers two major advantages – namely, the ultra-low cost and the do-it-yourself flexibility for targeting any combination of many brain areas. We show the successful recordings of both single units and LFPs from 13 distinct neural circuits of the mouse brain, including subregions of the anterior cingulate cortices, retrosplenial cortices, somatosensory cortices, secondary auditory cortex, hippocampal CA1, dentate gyrus, subiculum, lateral entorhinal cortex, perirhinal cortex, and prelimbic cortex. This 512-channel system can also be combined with Cre-lox neurogenetics and optogenetics to further examine interactions between genes, cell types, and circuit dynamics across a wide range of brain structures. Finally, we demonstrate that complex stimuli – such as an earthquake and fear-inducing foot-shock – trigger firing changes in all of the 13 brain regions recorded, supporting the notion that neural code is highly distributed. In addition, we show that localized optogenetic manipulation in any given brain region could disrupt network oscillations and caused changes in single-unit firing patterns in a brain-wide manner, thereby raising the cautionary note of the interpretation of optogenetically manipulated behaviors.

Theory of Connectivity: Nature and Nurture of Cell Assemblies and Cognitive Computation

Richard Semon and Donald Hebb are among the firsts to put forth the notion of cell assembly—a group of coherently or sequentially-activated neurons—to represent percept, memory, or concept. Despite the rekindled interest in this century-old idea, the concept of cell assembly still remains ill-defined and its operational principle is poorly understood. What is the size of a cell assembly? How should a cell assembly be organized? What is the computational logic underlying Hebbian cell assemblies? How might Nature vs. Nurture interact at the level of a cell assembly? In contrast to the widely assumed randomness within the mature but naïve cell assembly, the Theory of Connectivity postulates that the brain consists of the developmentally pre-programmed cell assemblies known as the functional connectivity motif (FCM). Principal cells within such FCM is organized by the power-of-two-based mathematical principle that guides the construction of specific-to-general combinatorial connectivity patterns in neuronal circuits, giving rise to a full range of specific features, various relational patterns, and generalized knowledge. This pre-configured canonical computation is predicted to be evolutionarily conserved across many circuits, ranging from these encoding memory engrams and imagination to decision-making and motor control. Although the power-of-two-based wiring and computational logic places a mathematical boundary on an individual’s cognitive capacity, the fullest intellectual potential can be brought about by optimized nature and nurture. This theory may also open up a new avenue to examining how genetic mutations and various drugs might impair or improve the computational logic of brain circuits.