Learning with incomplete information and the mathematical structure behind it

Isolating the delay component of impulsive choice in adolescent rats

Impulsive choice—the preference for small immediate rewards over larger delayed rewards—has been linked to various psychological conditions ranging from behavioral disorders to addiction. These links highlight the critical need to dissect the various components of this multifaceted behavioral trait. Delay discounting tasks allow researchers to study an important factor of this behavior: how the subjective value of a rewards changes over a delay period. However, existing methods of delay discounting include a confound of different reward sizes within the procedure. Here we present a new approach of using a single constant reward size to assess delay discounting. A complementary approach could hold delay constant and assess the utility of changing quantities of a reward. Isolating these behavioral components can advance our ability to explore the behavioral complexity of impulsive choice. We present in detail the methods for isolating delay, and further capitalize on this method by pairing it with a standard peak interval task to test whether individual variation in delay discounting can be explained by differences in perception of time in male and female adolescent rats. We find that rats that were more precise in discriminating time intervals were also less impulsive in their choice. Our data suggest that differences in timing and delay discounting are not causally related, but instead are more likely influenced by a common factor. Further, the mean-level change in our measure between post-natal day 28 and 42 suggests this test may be capturing a developmental change in this factor. In summary, this new method of isolating individual components of impulsive choice (delay or quantity) can be efficiently applied in either adolescent or adult animal models and may help elucidate the mechanisms underlying impulsivity and its links to psychological disorders.

Learning with incomplete information in the committee machine

We study the problem of learning with incomplete information in a student-teacher setup for the committee machine. The learning algorithm combines unsupervised Hebbian learning of a series of associations with a delayed reinforcement step, in which the set of previously learnt associations is partly and indiscriminately unlearnt, to an extent that depends on the success rate of the student on these previously learnt associations. The relevant learning parameter lambda represents the strength of Hebbian learning. A coarse-grained analysis of the system yields a set of differential equations for overlaps of student and teacher weight vectors, whose solutions provide a complete description of the learning behavior. It reveals complicated dynamics showing that perfect generalization can be obtained if the learning parameter exceeds a threshold lambda ( c ), and if the initial value of the overlap between student and teacher weights is non-zero. In case of convergence, the generalization error exhibits a power law decay as a function of the number of examples used in training, with an exponent that depends on the parameter lambda. An investigation of the system flow in a subspace with broken permutation symmetry between hidden units reveals a bifurcation point lambda* above which perfect generalization does not depend on initial conditions. Finally, we demonstrate that cases of a complexity mismatch between student and teacher are optimally resolved in the sense that an over-complex student can emulate a less complex teacher rule, while an under-complex student reaches a state which realizes the minimal generalization error compatible with the complexity mismatch.