Integrating unsupervised and reinforcement learning in human categorical perception: A computational model

Bridging flexible goal-directed cognition and consciousness: The Goal-Aligning Representation Internal Manipulation theory

Goal-directed manipulation of internal representations is a key element of human flexible behaviour, while consciousness is commonly associated with higher-order cognition and human flexibility. Current perspectives have only partially linked these processes, thus preventing a clear understanding of how they jointly generate flexible cognition and behaviour. Moreover, these limitations prevent an effective exploitation of this knowledge for technological scopes. We propose a new theoretical perspective that extends our 'three-component theory of flexible cognition' toward higher-order cognition and consciousness, based on the systematic integration of key concepts from Cognitive Neuroscience and AI/Robotics. The theory proposes that the function of conscious processes is to support the alignment of representations with multi-level goals. This higher alignment leads to more flexible and effective behaviours. We analyse here our previous model of goal-directed flexible cognition (validated with more than 20 human populations) as a starting GARIM-inspired model. By bridging the main theories of consciousness and goal-directed behaviour, the theory has relevant implications for scientific and technological fields. In particular, it contributes to developing new experimental tasks and interpreting clinical evidence. Finally, it indicates directions for improving machine learning and robotics systems and for informing real-world applications (e.g., in digital-twin healthcare and roboethics).

Simulated operant reflex conditioning environment reveals effects of feedback parameters

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, biological variability, and reward threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), biological variability (low, high), and reward threshold (easy, moderate, difficult) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

A computational model of inner speech supporting flexible goal-directed behaviour in Autism

Experimental and computational studies propose that inner speech boosts categorisation skills and executive functions, making human behaviour more focused and flexible. In addition, many clinical studies highlight a relationship between poor inner-speech and an executive impairment in autism spectrum condition (ASC), but contrasting findings are reported. Here we directly investigate the latter issue through a previously implemented and validated computational model of the Wisconsin Cards Sorting Tests. In particular, the model was applied to explore potential individual differences in cognitive flexibility and inner speech contribution in autistic and neurotypical participants. Our model predicts that the use of inner-speech could increase along the life-span of neurotypical participants but would be reduced in autistic ones. Although we found more attentional failures (i.e., wrong behavioural rule switches) in autistic children/teenagers and more perseverative behaviours in autistic young/older adults, only autistic children and older adults exhibited a lower performance (i.e., fewer consecutive correct rule switches) than matched control groups. Overall, our results corroborate the idea that the reduced use of inner speech could represent a disadvantage for autistic children and autistic older adults. Moreover, the results suggest that cognitive-behavioural therapies should focus on developing inner speech skills in autistic children as this could provide cognitive support throughout their whole life span.

Category learning in a recurrent neural network with reinforcement learning

It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.