A neuromorphic model of spatial lookahead planning

Foundations of human spatial problem solving

Despite great strides in both machine learning and neuroscience, we do not know how the human brain solves problems in the general sense. We approach this question by drawing on the framework of engineering control theory. We demonstrate a computational neural model with only localist learning laws that is able to find solutions to arbitrary problems. The model and humans perform a multi-step task with arbitrary and changing starting and desired ending states. Using a combination of computational neural modeling, human fMRI, and representational similarity analysis, we show here that the roles of a number of brain regions can be reinterpreted as interacting mechanisms of a control theoretic system. The results suggest a new set of functional perspectives on the orbitofrontal cortex, hippocampus, basal ganglia, anterior temporal lobe, lateral prefrontal cortex, and visual cortex, as well as a new path toward artificial general intelligence.

The Versatile Wayfinder: Prefrontal Contributions to Spatial Navigation

The prefrontal cortex (PFC) supports decision-making, goal tracking, and planning. Spatial navigation is a behavior that taxes these cognitive processes, yet the role of the PFC in models of navigation has been largely overlooked. In humans, activity in dorsolateral PFC (dlPFC) and ventrolateral PFC (vlPFC) during detours, reveal a role in inhibition and replanning. Dorsal anterior cingulate cortex (dACC) is implicated in planning and spontaneous internally-generated changes of route. Orbitofrontal cortex (OFC) integrates representations of the environment with the value of actions, providing a 'map' of possible decisions. In rodents, medial frontal areas interact with hippocampus during spatial decisions and switching between navigation strategies. In reviewing these advances, we provide a framework for how different prefrontal regions may contribute to different stages of navigation.

Accelerated Exploration for Long-Term Urban Water Infrastructure Planning through Machine Learning

In this study, the neural network method (Multi-Layer Perceptron, MLP) was integrated with an explorative model, to study the feasibility of using machine learning to reduce the exploration time but providing the same support in long-term water system adaptation planning. The specific network structure and training pattern were determined through a comprehensive statistical trial-and-error (considering the distribution of errors). The network was applied to the case study in Scotchman’s Creek, Melbourne. The network was trained with the first 10% of the exploration data, validated with the following 5% and tested on the rest. The overall root-mean-square-error between the entire observed data and the predicted data is 10.5722, slightly higher than the validation result (9.7961), suggesting that the proposed trial-and-error method is reliable. The designed MLP showed good performance dealing with spatial randomness from decentralized strategies. The adoption of MLP-supported planning may overestimate the performance of candidate urban water systems. By adopting the safety coefficient, a multiplicator or exponent calculated by observed data and predicted data in the validation process, the overestimation problem can be controlled in an acceptable range and have few impacts on final decision making.

Rapid, parallel path planning by propagating wavefronts of spiking neural activity

Efficient path planning and navigation is critical for animals, robotics, logistics and transportation. We study a model in which spatial navigation problems can rapidly be solved in the brain by parallel mental exploration of alternative routes using propagating waves of neural activity. A wave of spiking activity propagates through a hippocampus-like network, altering the synaptic connectivity. The resulting vector field of synaptic change then guides a simulated animal to the appropriate selected target locations. We demonstrate that the navigation problem can be solved using realistic, local synaptic plasticity rules during a single passage of a wavefront. Our model can find optimal solutions for competing possible targets or learn and navigate in multiple environments. The model provides a hypothesis on the possible computational mechanisms for optimal path planning in the brain, at the same time it is useful for neuromorphic implementations, where the parallelism of information processing proposed here can fully be harnessed in hardware.

The mixed instrumental controller: using value of information to combine habitual choice and mental simulation

Instrumental behavior depends on both goal-directed and habitual mechanisms of choice. Normative views cast these mechanisms in terms of model-free and model-based methods of reinforcement learning, respectively. An influential proposal hypothesizes that model-free and model-based mechanisms coexist and compete in the brain according to their relative uncertainty. In this paper we propose a novel view in which a single Mixed Instrumental Controller produces both goal-directed and habitual behavior by flexibly balancing and combining model-based and model-free computations. The Mixed Instrumental Controller performs a cost-benefits analysis to decide whether to chose an action immediately based on the available "cached" value of actions (linked to model-free mechanisms) or to improve value estimation by mentally simulating the expected outcome values (linked to model-based mechanisms). Since mental simulation entails cognitive effort and increases the reward delay, it is activated only when the associated "Value of Information" exceeds its costs. The model proposes a method to compute the Value of Information, based on the uncertainty of action values and on the distance of alternative cached action values. Overall, the model by default chooses on the basis of lighter model-free estimates, and integrates them with costly model-based predictions only when useful. Mental simulation uses a sampling method to produce reward expectancies, which are used to update the cached value of one or more actions; in turn, this updated value is used for the choice. The key predictions of the model are tested in different settings of a double T-maze scenario. Results are discussed in relation with neurobiological evidence on the hippocampus - ventral striatum circuit in rodents, which has been linked to goal-directed spatial navigation.

VIRTUAL INSTRUMENTATION BASED SYSTEMS FOR REAL-TIME PATH PLANNING OF MOBILE ROBOTS USING BIO-INSPIRED NEURAL NETWORKS

In this paper, novel virtual instrumentation based systems for real-time collision-free path planning and tracking control of mobile robots are proposed. The developed virtual instruments are computationally simple and efficient in comparison to other approaches, which act as a new soft-computing platform to implement a biologically-inspired neural network. This neural network is topologically arranged with only local lateral connections among neurons. The dynamics of each neuron is described by a shunting equation with both excitatory and inhibitory connections. The neural network requires no off-line training or on-line learning, which is capable of planning a comfortable trajectory to the target without suffering from neither the too close nor the too far problems. LabVIEW is chosen as the software platform to build the proposed virtual instrumentation systems, as it is one of the most important industrial platforms. We take the initiative to develop the first neuro-dynamic application in LabVIEW. The developed virtual instruments could be easily used as educational and research tools for studying various robot path planning and tracking situations that could be easily understood and analyzed step by step. The effectiveness and efficiency of the developed virtual instruments are demonstrated through simulation and comparison studies.

A computational model of fMRI activity in the intraparietal sulcus that supports visual working memory

A computational model was developed to explain a pattern of results of fMRI activation in the intraparietal sulcus (IPS) supporting visual working memory for multiobject scenes. The model is based on the hypothesis that dendrites of excitatory neurons are major computational elements in the cortical circuit. Dendrites enable formation of a competitive queue that exhibits a gradient of activity values for nodes encoding different objects, and this pattern is stored in working memory. In the model, brain imaging data are interpreted as a consequence of blood flow arising from dendritic processing. Computer simulations showed that the model successfully simulates data showing the involvement of inferior IPS in object individuation and spatial grouping through representation of objects' locations in space, along with the involvement of superior IPS in object identification through representation of a set of objects' features. The model exhibits a capacity limit due to the limited dynamic range for nodes and the operation of lateral inhibition among them. The capacity limit is fixed in the inferior IPS regardless of the objects' complexity, due to the normalization of lateral inhibition, and variable in the superior IPS, due to the different encoding demands for simple and complex shapes. Systematic variation in the strength of self-excitation enables an understanding of the individual differences in working memory capacity. The model offers several testable predictions regarding the neural basis of visual working memory.