A kernel-based perceptron with dynamic memory

A Note on the Unification of Adaptive Online Learning

In online convex optimization, adaptive algorithms, which can utilize the second-order information of the loss function's (sub)gradient, have shown improvements over standard gradient methods. This paper presents a framework Follow the Bregman Divergence Leader that unifies various existing adaptive algorithms from which new insights are revealed. Under the proposed framework, two simple adaptive online algorithms with improvable performance guarantee are derived. Furthermore, a general equation derived from a matrix analysis generalizes the adaptive learning to nonlinear case with kernel trick.

Simple randomized algorithms for online learning with kernels

In online learning with kernels, it is vital to control the size (budget) of the support set because of the curse of kernelization. In this paper, we propose two simple and effective stochastic strategies for controlling the budget. Both algorithms have an expected regret that is sublinear in the horizon. Experimental results on a number of benchmark data sets demonstrate encouraging performance in terms of both efficacy and efficiency.

Incremental Learning on a Budget and its Application to Quick Maximum Power Point Tracking of Photovoltaic Systems

Recent improvements in embedded systems has enabled learning algorithms to provide realistic solutions for system identification problems. Existing learning algorithms, however, continue to have limitations in learning on embedded systems, where physical memory space is constrained. To overcome this problem, we propose a Limited General Regression Neural Network (LGRNN), which is a variation of general regression neural network proposed by Specht or of simplified fuzzy inference systems. The LGRNN continues incremental learning even if the number of instances exceeds the maximum number of kernels in the LGRNN. We demonstrate LGRNN advantages by comparing it to other kernel-based perceptron learning methods. We also propose a light-weighted LGRNN algorithm, -LGRNNLight- for reducing computational complexity. As an example of its application, we present a Maximum Power Point Tracking (MPPT) microconverter for photovoltaic power generation systems. MPPT is essential for improving the efficiency of renewable energy systems. Although various techniques exist that can realize MPPT, few techniques are able to realize quick control using conventional circuit design. The LGRNN enables the MPPT converter to be constructed at low cost using the conventional combination of a chopper circuit and microcomputer control. The LGRNN learns the Maximum Power Point (MPP) found by Perturb and Observe (P&O), and immediately sets the converter reference voltage after a sudden irradiation change. By using this strategy, the MPPT quickly responds without a predetermination of parameters. The experimental results suggest that, after learning, the proposed converter controls a chopper circuit within 14 ms after a sudden irradiation change. This rapid response property is suitable for efficient power generation, even under shadow flicker conditions that often occur in solar panels located near large wind turbines.