Differences of Training Structures on Stimulus Class Formation in Computational Agents

Stimulus Equivalence (SE) is a behavioural phenomenon in which organisms respond functionally to stimuli without explicit training. SE provides a framework in the experimental analysis of behaviour to study language, symbolic behaviour, and cognition. It is also a frequently discussed matter in interdisciplinary research, linking behaviour analysis with linguistics and neuroscience. Previous research has attempted to replicate SE with computational agents, mostly based on Artificial Neural Network (ANN) models. The aim of this paper was to analyse the effect of three Training Structures (TSs) on stimulus class formation in a simulation with ANNs as computational agents performing a classification task, in a matching-to-sample procedure. Twelve simulations were carried out as a product of the implementation of four ANN architectures on the three TSs. SE was not achieved, but two agents showed an emergent response on half of the transitivity test pairs on linear sequence TSs and reflexivity on one member of the class. The results suggested that an ANN with a large enough number of units in a hidden layer can perform a limited number of emergent relations within specific experimental conditions: reflexivity on B and transitivity on AC, when pairs AB and BC are trained on a three-member stimulus class and tested in a classification task. Reinforcement learning is proposed as the framework for further simulations.

Computational models of stimulus equivalence: An intersection for the study of symbolic behavior

Stimulus equivalence is a central paradigm in the analysis of symbolic behavior, language, and cognition. It describes emergent relations between stimuli that were not explicitly trained and cannot be explained by primary stimulus generalization. In recent years, researchers have developed computational models to simulate the learning of equivalence relations. These models have been used to address primary theoretical and methodological issues in this field, such as exploring the underlying mechanisms that explain emergent equivalence relations and analyzing the effects of training and testing protocols on equivalence outcomes. Nonetheless, although these models build upon general learning principles, their operation is usually obscure for nonmodelers, and in the field of stimulus equivalence computational models have been developed with a variety of approaches, architectures, and algorithms that make it difficult to understand the scope and contributions of these tools. In this paper, we present the state of the art in computational modeling of stimulus equivalence. We seek to provide concise and accessible descriptions of the models' functioning and operation, highlight their main theoretical and methodological contributions, identify the existing software available for researchers to run experiments, and suggest future directions in the emergent field of computational modeling of stimulus equivalence.

How a Minimal Learning Agent can Infer the Existence of Unobserved Variables in a Complex Environment

According to a mainstream position in contemporary cognitive science and philosophy, the use of abstract compositional concepts is amongst the most characteristic indicators of meaningful deliberative thought in an organism or agent. In this article, we show how the ability to develop and utilise abstract conceptual structures can be achieved by a particular kind of learning agent. More specifically, we provide and motivate a concrete operational definition of what it means for these agents to be in possession of abstract concepts, before presenting an explicit example of a minimal architecture that supports this capability. We then proceed to demonstrate how the existence of abstract conceptual structures can be operationally useful in the process of employing previously acquired knowledge in the face of new experiences, thereby vindicating the natural conjecture that the cognitive functions of abstraction and generalisation are closely related.

Enhanced Equivalence Projective Simulation: A Framework for Modeling Formation of Stimulus Equivalence Classes

Formation of stimulus equivalence classes has been recently modeled through equivalence projective simulation (EPS), a modified version of a projective simulation (PS) learning agent. PS is endowed with an episodic memory that resembles the internal representation in the brain and the concept of cognitive maps. PS flexibility and interpretability enable the EPS model and, consequently the model we explore in this letter, to simulate a broad range of behaviors in matching-to-sample experiments. The episodic memory, the basis for agent decision making, is formed during the training phase. Derived relations in the EPS model that are not trained directly but can be established via the network's connections are computed on demand during the test phase trials by likelihood reasoning. In this letter, we investigate the formation of derived relations in the EPS model using network enhancement (NE), an iterative diffusion process, that yields an offline approach to the agent decision making at the testing phase. The NE process is applied after the training phase to denoise the memory network so that derived relations are formed in the memory network and retrieved during the testing phase. During the NE phase, indirect relations are enhanced, and the structure of episodic memory changes. This approach can also be interpreted as the agent's replay after the training phase, which is in line with recent findings in behavioral and neuroscience studies. In comparison with EPS, our model is able to model the formation of derived relations and other features such as the nodal effect in a more intrinsic manner. Decision making in the test phase is not an ad hoc computational method, but rather a retrieval and update process of the cached relations from the memory network based on the test trial. In order to study the role of parameters on agent performance, the proposed model is simulated and the results discussed through various experimental settings.

Balanced difficulty task finder: an adaptive recommendation method for learning tasks based on the concept of state of flow

An adaptive task difficulty assignment method which we reckon as balanced difficulty task finder (BDTF) is proposed in this paper. The aim is to recommend tasks to a learner using a trade-off between skills of the learner and difficulty of the tasks such that the learner experiences a state of flow during the learning. Flow is a mental state that psychologists refer to when someone is completely immersed in an activity. Flow state is a multidisciplinary field of research and has been studied not only in psychology, but also neuroscience, education, sport, and games. The idea behind this paper is to try to achieve a flow state in a similar way as Elo’s chess skill rating (Glickman in Am Chess J 3:59–102) and TrueSkill (Herbrich et al. in Advances in neural information processing systems, 2006) for matching game players, where “matched players” should possess similar capabilities and skills in order to maintain the level of motivation and involvement in the game. The BDTF draws analogy between choosing an appropriate opponent or appropriate game level and automatically choosing an appropriate difficulty level of a learning task. This method, as an intelligent tutoring system, could be used in a wide range of applications from online learning environments and e-learning, to learning and remembering techniques in traditional methods such as adjusting delayed matching to sample and spaced retrieval training that can be used for people with memory problems such as people with dementia.