Extending stability through hierarchical clusters in echo state networks

Evolving Local Plasticity Rules for Synergistic Learning in Echo State Networks

Existing synaptic plasticity rules for optimizing the connections between neurons within the reservoir of echo state networks (ESNs) remain to be global in that the same type of plasticity rule with the same parameters is applied to all neurons. However, this is biologically implausible and practically inflexible for learning the structures in the input signals, thereby limiting the learning performance of ESNs. In this paper, we propose to use local plasticity rules that allow different neurons to use different types of plasticity rules and different parameters, which are achieved by optimizing the parameters of the local plasticity rules using the evolution strategy (ES) with covariance matrix adaptation (CMA-ES). We show that evolving neural plasticity will result in a synergistic learning of different plasticity rules, which plays an important role in improving the learning performance. Meanwhile, we show that the local plasticity rules can effectively alleviate synaptic interferences in learning the structure in sensory inputs. The proposed local plasticity rules are compared with a number of the state-of-the-art ESN models and the canonical ESN using a global plasticity rule on a set of widely used prediction and classification benchmark problems to demonstrate its competitive learning performance.

Inferring general relations between network characteristics from specific network ensembles

Different network models have been suggested for the topology underlying complex interactions in natural systems. These models are aimed at replicating specific statistical features encountered in real-world networks. However, it is rarely considered to which degree the results obtained for one particular network class can be extrapolated to real-world networks. We address this issue by comparing different classical and more recently developed network models with respect to their ability to generate networks with large structural variability. In particular, we consider the statistical constraints which the respective construction scheme imposes on the generated networks. After having identified the most variable networks, we address the issue of which constraints are common to all network classes and are thus suitable candidates for being generic statistical laws of complex networks. In fact, we find that generic, not model-related dependencies between different network characteristics do exist. This makes it possible to infer global features from local ones using regression models trained on networks with high generalization power. Our results confirm and extend previous findings regarding the synchronization properties of neural networks. Our method seems especially relevant for large networks, which are difficult to map completely, like the neural networks in the brain. The structure of such large networks cannot be fully sampled with the present technology. Our approach provides a method to estimate global properties of under-sampled networks in good approximation. Finally, we demonstrate on three different data sets (C. elegans neuronal network, R. prowazekii metabolic network, and a network of synonyms extracted from Roget’s Thesaurus) that real-world networks have statistical relations compatible with those obtained using regression models.

Modular and hierarchically modular organization of brain networks

Brain networks are increasingly understood as one of a large class of information processing systems that share important organizational principles in common, including the property of a modular community structure. A module is topologically defined as a subset of highly inter-connected nodes which are relatively sparsely connected to nodes in other modules. In brain networks, topological modules are often made up of anatomically neighboring and/or functionally related cortical regions, and inter-modular connections tend to be relatively long distance. Moreover, brain networks and many other complex systems demonstrate the property of hierarchical modularity, or modularity on several topological scales: within each module there will be a set of sub-modules, and within each sub-module a set of sub-sub-modules, etc. There are several general advantages to modular and hierarchically modular network organization, including greater robustness, adaptivity, and evolvability of network function. In this context, we review some of the mathematical concepts available for quantitative analysis of (hierarchical) modularity in brain networks and we summarize some of the recent work investigating modularity of structural and functional brain networks derived from analysis of human neuroimaging data.