Learning in multilevel games with incomplete information. II

Learning in multilevel games with incomplete information. I

A model is presented of learning automata playing stochastic games at two levels. The high level represents the choice of the game environment and corresponds to a group decision. The low level represents the choice of action within the selected game environment. Both of these decision processes are affected by delays in the information state due to inherent latencies or to the delayed broadcast of state changes. Analysis of the intrinsic properties of this Markov process is presented along with simulated iterative behavior and expected iterative behavior. The results show that simulation agrees with expected behavior for small step lengths in the iterative map. A Feigenbaum diagram and numerical computation of the Lyapunov exponents show that, for very small penalty parameters, the system exhibits chaotic behavior.

A relative reward-strength algorithm for the hierarchical structure learning automata operating in the general nonstationary multiteacher environment

A new learning algorithm for the hierarchical structure learning automata (HSLA) operating in the nonstationary multiteacher environment (NME) is proposed. The proposed algorithm is derived by extending the original relative reward-strength algorithm to be utilized in the HSLA operating in the general NME. It is shown that the proposed algorithm ensures convergence with probability 1 to the optimal path under a certain type of the NME. Several computer-simulation results, which have been carried out in order to compare the relative performance of the proposed algorithm in some NMEs against those of the two of the fastest algorithms today, confirm the effectiveness of the proposed algorithm.

An application of a computational ecology model to a routing method in computer networks

The paper proposes a network routing method based on a computational ecology model. The computational ecology model is a mathematical model proposed by B.A. Huberman and T. Hogg (1988), which represents a macro action of multi-agent systems. We formulate routing on a computer network as a resource allocation problem, where packets and links are regarded as agents and resources, respectively. Then, we apply an extended computational ecology model for this problem. Agents conflict so as to get more payoffs from links. As a result, they get the same payoffs, and a good resource allocation is achieved. In each node, each packet selects a link according to the selection rate decided through conflicts, and routing is accomplished autonomously with adaptability on the computer network. Moreover, we improve fault-tolerance of the system by local information exchanges. Finally, we examine the efficiency of the proposed method by computer simulation.