Value and reward based learning in neurorobots

Anxiety predicts impulsive-compulsive behaviours in Parkinson's disease: Clinical relevance and theoretical implications

Patients with Parkinson's disease (PD) often present symptoms of anxiety, depression and apathy. These negative affect manifestations have been recently associated with the presence of impulsive compulsive behaviours (ICBs). However, their relation with the use of dopamine replacement therapy (DRT), a renewed risk factor for ICBs, is still not fully understood. Elucidating the role of these different ICBs predictors in PD could inform both prevention/intervention recommendations as well as theoretical models. In the present study, we have analysed data collected in 417 PD patients, 50 patients with Parkinsonian symptoms but with scan without evidence of dopaminergic deficit (SWEDD), and 185 healthy controls (HC). We examined each patient's clinical profile over a two-year time window, investigating the role of both negative affect and DRT on ICBs. Results confirmed the presence of higher levels of anxiety in both the clinical groups, and of higher level of ICBs in SWEDD patients, respect to both PD and HC. Mixed model analyses revealed a statistically significant association between anxiety and ICBs in the SWEDD patients who did not take any DRT. Findings suggest the independence between anxiety and DRT in ICBs development, and provide new evidence for the motivational opponency theoretical framework.

Hedonic quality or reward? A study of basic pleasure in homeostasis and decision making of a motivated autonomous robot

We present a robot architecture and experiments to investigate some of the roles that pleasure plays in the decision making (action selection) process of an autonomous robot that must survive in its environment. We have conducted three sets of experiments to assess the effect of different types of pleasure-related versus unrelated to the satisfaction of physiological needs-under different environmental circumstances. Our results indicate that pleasure, including pleasure unrelated to need satisfaction, has value for homeostatic management in terms of improved viability and increased flexibility in adaptive behavior.

Learning touch preferences with a tactile robot using dopamine modulated STDP in a model of insular cortex

Neurorobots enable researchers to study how behaviors are produced by neural mechanisms in an uncertain, noisy, real-world environment. To investigate how the somatosensory system processes noisy, real-world touch inputs, we introduce a neurorobot called CARL-SJR, which has a full-body tactile sensory area. The design of CARL-SJR is such that it encourages people to communicate with it through gentle touch. CARL-SJR provides feedback to users by displaying bright colors on its surface. In the present study, we show that CARL-SJR is capable of learning associations between conditioned stimuli (CS; a color pattern on its surface) and unconditioned stimuli (US; a preferred touch pattern) by applying a spiking neural network (SNN) with neurobiologically inspired plasticity. Specifically, we modeled the primary somatosensory cortex, prefrontal cortex, striatum, and the insular cortex, which is important for hedonic touch, to process noisy data generated directly from CARL-SJR's tactile sensory area. To facilitate learning, we applied dopamine-modulated Spike Timing Dependent Plasticity (STDP) to our simulated prefrontal cortex, striatum, and insular cortex. To cope with noisy, varying inputs, the SNN was tuned to produce traveling waves of activity that carried spatiotemporal information. Despite the noisy tactile sensors, spike trains, and variations in subject hand swipes, the learning was quite robust. Further, insular cortex activities in the incremental pathway of dopaminergic reward system allowed us to control CARL-SJR's preference for touch direction without heavily pre-processed inputs. The emerged behaviors we found in this model match animal's behaviors wherein they prefer touch in particular areas and directions. Thus, the results in this paper could serve as an explanation on the underlying neural mechanisms for developing tactile preferences and hedonic touch.

Operant conditioning: a minimal components requirement in artificial spiking neurons designed for bio-inspired robot's controller

In this paper, we investigate the operant conditioning (OC) learning process within a bio-inspired paradigm, using artificial spiking neural networks (ASNN) to act as robot brain controllers. In biological agents, OC results in behavioral changes learned from the consequences of previous actions, based on progressive prediction adjustment from rewarding or punishing signals. In a neurorobotics context, virtual and physical autonomous robots may benefit from a similar learning skill when facing unknown and unsupervised environments. In this work, we demonstrate that a simple invariant micro-circuit can sustain OC in multiple learning scenarios. The motivation for this new OC implementation model stems from the relatively complex alternatives that have been described in the computational literature and recent advances in neurobiology. Our elementary kernel includes only a few crucial neurons, synaptic links and originally from the integration of habituation and spike-timing dependent plasticity as learning rules. Using several tasks of incremental complexity, our results show that a minimal neural component set is sufficient to realize many OC procedures. Hence, with the proposed OC module, designing learning tasks with an ASNN and a bio-inspired robot context leads to simpler neural architectures for achieving complex behaviors.