Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning

A hallmark of human intelligence, but challenging for reinforcement learning (RL) agents, is the ability to compositionally generalise, that is, to recompose familiar knowledge components in novel ways to solve new problems. For instance, when navigating in a city, one needs to know the location of the destination and how to operate a vehicle to get there, whether it be pedalling a bike or operating a car. In RL, these correspond to the reward function and transition function, respectively. To compositionally generalize, these two components need to be transferable independently of each other: multiple modes of transport can reach the same goal, and any given mode can be used to reach multiple destinations. Yet there are also instances where it can be helpful to learn and transfer entire structures, jointly representing goals and transitions, particularly whenever these recur in natural tasks (e.g., given a suggestion to get ice cream, one might prefer to bike, even in new towns). Prior theoretical work has explored how, in model-based RL, agents can learn and generalize task components (transition and reward functions). But a satisfactory account for how a single agent can simultaneously satisfy the two competing demands is still lacking. Here, we propose a hierarchical RL agent that learns and transfers individual task components as well as entire structures (particular compositions of components) by inferring both through a non-parametric Bayesian model of the task. It maintains a factorised representation of task components through a hierarchical Dirichlet process, but it also represents different possible covariances between these components through a standard Dirichlet process. We validate our approach on a variety of navigation tasks covering a wide range of statistical correlations between task components and show that it can also improve generalisation and transfer in more complex, hierarchical tasks with goal/subgoal structures. Finally, we end with a discussion of our work including how this clustering algorithm could conceivably be implemented by cortico-striatal gating circuits in the brain.

On the origin of mental representations

The symbol grounding problem raises its head in the fields of the philosophy of AI, philosophy of psychology and philosophy of cognitive sciences. The solution to the symbol grounding problem must account for the genesis of mental representations in the world. It has to offer a strategy for grounding mental representations in the objective domain. Orthodox representationalist theories do not provide a satisfactory reply to the symbol grounding problem. On the other hand, there are embodied-enactivist approaches that dissolve the problem but only at the cost of representations and internal phenomenal states. The code model of biosemiotics provides a biologically viable (i.e., mechanistic) venue for developing a new solution to the problem. For the same reason, it could reconcile representationalism to the embodied approach.

Comparative Analysis of the WISC between Two ADHD Subgroups

OBJECTIVE

The prevalence of attention deficit/hyperactivity disorder (ADHD) in school-age children is 7.2%, and ADHD is divided into clinical subtypes.

METHODS

The current study explored whether specific cognitive profiles as assessed using the Wechsler Intelligence Scale for Children (WISC)-IV could be obtained for each clinical ADHD subtype (ADHD-Inattentive type and ADHD-Combined type) and investigated the correlation between WISC scores and parental age at their children's birth or birthweight. The enrolled sample comprised 12 ADHD-I and 15 ADHD-C subjects.

RESULTS

An impaired Processing Speed Index was found in ADHD-I. The age of the father at the child's birth and birthweight positively correlated with the full scale intelligence quotient (FSIQ) score in the WISC assessment.

CONCLUSION

Inattentiveness within the behaviors of the children with ADHD-I is partly due to the impaired processing speed, therefore effective support for ADHD will be conducted if educator decreases their speaking speed. Since biological basis of ADHD is still largely unknown, future studies using both psychological and biological methods will reveal the etiology of ADHD. These scientific assessments will provide information for more effective approaches in the care of children with ADHD.

Frontal Lobe Dysfunction in a Depressed Patient Who Survived a Suicide Attempt by Jumping from the Bridge on the Han River

Suicide attempts at the Han river are rapidly increasing, which are 4.11 times from 2005 to 2015, whereas the rate of completed suicide in South Korea increased 1.07 times during the same period. However, few studies have been conducted on the issue because many suicide attempters were seriously injured after a fall in the Han river. We present a case of a patient with major depressive disorder (MDD) who attempted suicide and minimally injured after jumping from the bridge at the Han river. We could assess his psychological and neurocognitive functions before and immediately after his attempt. From this case, we can identify that higher cognitive aspect of executive dysfunction, especially in the frontal domain of selective attention and inhibition, may be associated with his suicide attempt. In conclusion, we suggest psychiatric treatments for cognitive impulsiveness and safety barriers at the bridge to prevent suicide attempts of patients with MDD.

Agonistic and antagonistic interaction in speed/accuracy tradeoff: a delta-lognormal perspective

This paper reports the results of a model-based analysis of movements gathered in a 4×4 experimental design of speed/accuracy tradeoffs with variable target distances and width. Our study was performed on a large (120 participants) and varied sample (both genders, wide age range, various health conditions). The delta-lognormal equation was used for data modeling to investigate the interaction between the output of the agonist and the antagonist neuromuscular systems. Empirical observations show that the subjects must correlate more tightly the impulse commands sent to both neuromuscular systems in order to achieve good performances as the difficulty of the task increases whereas the correlation in the timing of the neuromuscular action co-varies with the size of the geometrical properties of the task. These new phenomena are discussed under the paradigm provided by the Kinematic Theory and new research hypotheses are proposed for further investigation of the speed/accuracy tradeoffs.