Distributed learning for feedforward neural networks with random weights using an event-triggered communication scheme

A distributed semi-supervised learning algorithm based on manifold regularization using wavelet neural network

This paper aims to propose a distributed semi-supervised learning (D-SSL) algorithm to solve D-SSL problems, where training samples are often extremely large-scale and located on distributed nodes over communication networks. Training data of each node consists of labeled and unlabeled samples whose output values or labels are unknown. These nodes communicate in a distributed way, where each node has only access to its own data and can only exchange local information with its neighboring nodes. In some scenarios, these distributed data cannot be processed centrally. As a result, D-SSL problems cannot be centrally solved by using traditional semi-supervised learning (SSL) algorithms. The state-of-the-art D-SSL algorithm, denoted as Distributed Laplacian Regularization Least Square (D-LapRLS), is a kernel based algorithm. It is essential for the D-LapRLS algorithm to estimate the global Euclidian Distance Matrix (EDM) with respect to total samples, which is time-consuming especially when the scale of training data is large. In order to solve D-SSL problems and overcome the common drawback of kernel based D-SSL algorithms, we propose a novel Manifold Regularization (MR) based D-SSL algorithm using Wavelet Neural Network (WNN) and Zero-Gradient-Sum (ZGS) distributed optimization strategy. Accordingly, each node is assigned an individual WNN with the same basis functions. In order to initialize the proposed D-SSL algorithm, we propose a centralized MR based SSL algorithm using WNN. We denote the proposed SSL and D-SSL algorithms as Laplacian WNN (LapWNN) and distributed LapWNN (D-LapWNN), respectively. The D-LapWNN algorithm works in a fully distributed fashion by using ZGS strategy, whose convergence is guaranteed by the Lyapunov method. During the learning process, each node only exchanges local coefficients with its neighbors rather than raw data. It means that the D-LapWNN algorithm is a privacy preserving method. At last, several illustrative simulations are presented to show the efficiency and advantage of the proposed algorithm.

Distributed semi-supervised learning algorithm based on extreme learning machine over networks using event-triggered communication scheme

In this paper, we propose a distributed semi-supervised learning (DSSL) algorithm based on the extreme learning machine (ELM) algorithm over communication network using the event-triggered (ET) communication scheme. In DSSL problems, training data consisting of labeled and unlabeled samples are distributed over a communication network. Traditional semi-supervised learning (SSL) algorithms cannot be used to solve DSSL problems. The proposed algorithm, denoted as ET-DSS-ELM, is based on the semi-supervised ELM (SS-ELM) algorithm, the zero gradient sum (ZGS) distributed optimization strategy and the ET communication scheme. Correspondingly, the SS-ELM algorithm is used to calculate the local initial value, the ZGS strategy is used to calculate the globally optimal value and the ET scheme is used to reduce communication times during the learning process. According to the ET scheme, each node over the communication network broadcasts its updated information only when the event occurs. Therefore, the proposed ET-DSS-ELM algorithm not only takes the advantages of traditional DSSL algorithms, but also saves network resources by reducing communication times. The convergence of the proposed ET-DSS-ELM algorithm is guaranteed by using the Lyapunov method. Finally, some simulations are given to show the efficiency of the proposed algorithm.