1
|
Optimal echo state network parameters based on behavioural spaces. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.06.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
2
|
Deep reservoir calculation model and its application in the field of temperature and humidity prediction. APPL INTELL 2022. [DOI: 10.1007/s10489-022-03685-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
3
|
Wang H, Liu Y, Lu P, Luo Y, Wang D, Xu X. Echo state network with logistic mapping and bias dropout for time series prediction. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.03.018] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
4
|
|
5
|
Multivariate time series forecasting method based on nonlinear spiking neural P systems and non-subsampled shearlet transform. Neural Netw 2022; 152:300-310. [DOI: 10.1016/j.neunet.2022.04.030] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2022] [Revised: 04/01/2022] [Accepted: 04/28/2022] [Indexed: 11/23/2022]
|
6
|
Statistical Analysis on Random Matrices of Echo State Network in PEMFC System’s Lifetime Prediction. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12073421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
Abstract
The data-driven method of echo state network (ESN) has been successfully used in the proton exchange membrane fuel cell (PEMFC) system’s lifetime prediction area. Nevertheless, the uncertainties of the randomly generated input and internal weight matrices in ESN have not been reported yet. In view of this, an ensemble ESN structure is proposed in this paper to analyze the effects of random matrices from a statistical point of view. For each ESN, the particle swarm optimization (PSO) method is utilized to optimize the hyperparameters of the leaking rate, spectral radius, and regularization coefficient. The statistical results of each ensemble ESN are analyzed from 100 repeated tests whose weight matrices are generated randomly. The mean value of the ensemble ESN and a confidence interval (CI) of 95% are given during the long-term lifetime prediction. The effects of two different distribution shapes, i.e., uniform distribution and Gaussian distribution, are fully compared. Finally, the effects of the ensemble structure and two different distribution shapes are tested by three experimental datasets under steady-state, quasi-dynamic, and full dynamic operating conditions.
Collapse
|
7
|
Lemos T, Campos LF, Melo A, Clavijo N, Soares R, Câmara M, Feital T, Anzai T, Pinto JC. Echo State Network Based Soft Sensor for Monitoring and Fault Detection of Industrial Processes. Comput Chem Eng 2021. [DOI: 10.1016/j.compchemeng.2021.107512] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
8
|
Ji Y, Chen Z, Shen Y, Yang K, Wang Y, Cui J. An RUL prediction approach for lithium-ion battery based on SADE-MESN. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107195] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
9
|
Racca A, Magri L. Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics. Neural Netw 2021; 142:252-268. [PMID: 34034072 DOI: 10.1016/j.neunet.2021.05.004] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 05/03/2021] [Accepted: 05/06/2021] [Indexed: 11/26/2022]
Abstract
An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network's performance. The overarching aim of this paper is to improve the robustness in the selection of hyperparameters in Echo State Networks for the time-accurate prediction of chaotic solutions. We define the robustness of a validation strategy as its ability to select hyperparameters that perform consistently between validation and test sets. The goal is three-fold. First, we investigate routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional grid search for optimal hyperparameter selection. Numerical tests are performed on prototypical nonlinear systems that have chaotic and quasiperiodic solutions, such as the Lorenz and Lorenz-96 systems, and the Kuznetsov oscillator. Both model-free and model-informed Echo State Networks are analysed. By comparing the networks' performance in learning chaotic (unpredictable) versus quasiperiodic (predictable) solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled - they are based on chaos theory such as the Lyapunov time - they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems.
Collapse
Affiliation(s)
- Alberto Racca
- Department of Engineering, University of Cambridge, Trumpington Street, Cambridge CB2 1PZ, UK
| | - Luca Magri
- Department of Engineering, University of Cambridge, Trumpington Street, Cambridge CB2 1PZ, UK; Aeronautics Department, Imperial College London, Exhibition Rd, London, SW7 2AZ, UK; The Alan Turing Institute, 96 Euston Road, London, England, NW1 2DB, UK; Institute for Advanced Study, Technical University of Munich, Lichtenbergstrasse 2a, 85748 Garching, Germany(1).
| |
Collapse
|
10
|
Hadikhani P, Hadikhani P. An adaptive hybrid algorithm for social networks to choose groups with independent members. EVOLUTIONARY INTELLIGENCE 2020. [DOI: 10.1007/s12065-020-00384-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
11
|
Öztürk MM, Cankaya İA, İpekçi D. Optimizing echo state network through a novel fisher maximization based stochastic gradient descent. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.07.034] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
12
|
Yao X, Wang Z. Fractional Order Echo State Network for Time Series Prediction. Neural Process Lett 2020. [DOI: 10.1007/s11063-020-10267-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
13
|
Yang C, Zhu X, Qiao J, Nie K. Forward and backward input variable selection for polynomial echo state networks. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.02.034] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
14
|
Nguyen HM, Kalra G, Jun T, Kim D. Chaotic Time Series Prediction Using a Novel Echo State Network Model with Input Reconstruction, Bayesian Ridge Regression and Independent Component Analysis. INT J PATTERN RECOGN 2020. [DOI: 10.1142/s0218001420510088] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
This paper presents a novel Echo State Network (ESN) model for chaotic time series prediction, which consists of three steps including input reconstruction, dimensionality reduction and regression. First, phase-space reconstruction is used to reconstruct the original ‘attractor’ of the input time series. Then, Independent Component Analysis (ICA) is used to identify independent components, reduce dimensionality and overcome multicollinearity problem of the reconstructed input matrix. Finally, Bayesian Ridge Regression provides accurate predictions thanks to its regularization effect to avoid over-fitting and its robustness to noise owing to its probabilistic strategy. Our experimental results show that our model significantly outperforms other ESN models in predicting both artificial and real-world chaotic time series.
Collapse
Affiliation(s)
- Hoang Minh Nguyen
- School of Computing, Korea Advanced Institute of Science and Technology 291 Daehak-ro, Yuseong-gu, Daejeon 34141, South Korea
| | - Gaurav Kalra
- School of Computing, Korea Advanced Institute of Science and Technology 291 Daehak-ro, Yuseong-gu, Daejeon 34141, South Korea
| | - Taejoon Jun
- School of Computing, Korea Advanced Institute of Science and Technology 291 Daehak-ro, Yuseong-gu, Daejeon 34141, South Korea
| | - Daeyoung Kim
- School of Computing, Korea Advanced Institute of Science and Technology 291 Daehak-ro, Yuseong-gu, Daejeon 34141, South Korea
| |
Collapse
|
15
|
Liu J, Sun T, Luo Y, Yang S, Cao Y, Zhai J. Echo state network optimization using binary grey wolf algorithm. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2019.12.069] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
16
|
Howard N, Chouikhi N, Adeel A, Dial K, Howard A, Hussain A. BrainOS: A Novel Artificial Brain-Alike Automatic Machine Learning Framework. Front Comput Neurosci 2020; 14:16. [PMID: 32194389 PMCID: PMC7063840 DOI: 10.3389/fncom.2020.00016] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2019] [Accepted: 02/10/2020] [Indexed: 11/28/2022] Open
Abstract
Human intelligence is constituted by a multitude of cognitive functions activated either directly or indirectly by external stimuli of various kinds. Computational approaches to the cognitive sciences and to neuroscience are partly premised on the idea that computational simulations of such cognitive functions and brain operations suspected to correspond to them can help to further uncover knowledge about those functions and operations, specifically, how they might work together. These approaches are also partly premised on the idea that empirical neuroscience research, whether following on from such a simulation (as indeed simulation and empirical research are complementary) or otherwise, could help us build better artificially intelligent systems. This is based on the assumption that principles by which the brain seemingly operate, to the extent that it can be understood as computational, should at least be tested as principles for the operation of artificial systems. This paper explores some of the principles of the brain that seem to be responsible for its autonomous, problem-adaptive nature. The brain operating system (BrainOS) explicated here is an introduction to ongoing work aiming to create a robust, integrated model, combining the connectionist paradigm underlying neural networks and the symbolic paradigm underlying much else of AI. BrainOS is an automatic approach that selects the most appropriate model based on the (a) input at hand, (b) prior experience (a history of results of prior problem solving attempts), and (c) world knowledge (represented in the symbolic way and used as a means to explain its approach). It is able to accept diverse and mixed input data types, process histories and objectives, extract knowledge and infer a situational context. BrainOS is designed to be efficient through its ability to not only choose the most suitable learning model but to effectively calibrate it based on the task at hand.
Collapse
Affiliation(s)
- Newton Howard
- Etats-Unis, Department of Neurosurgery, Nuffield Department of Surgical Sciences, John Radcliffe Hospital, Oxford, United Kingdom
| | - Naima Chouikhi
- REGIM-Lab: REsearch Groups in Intelligent Machines, National Engineering School of Sfax (ENIS), University of Sfax, Sfax, Tunisia
| | - Ahsan Adeel
- School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, United Kingdom.,School of Mathematics and Computer Science, University of Wolverhampton, Wolverhampton, United Kingdom
| | - Katelyn Dial
- Howard Brain Sciences Foundation, Providence, RI, United States
| | - Adam Howard
- Howard Brain Sciences Foundation, Providence, RI, United States
| | - Amir Hussain
- School of Computing, Edinburgh Napier University, Edinburgh, United Kingdom
| |
Collapse
|
17
|
Optimizing Deep Belief Echo State Network with a Sensitivity Analysis Input Scaling Auto-Encoder algorithm. Knowl Based Syst 2020. [DOI: 10.1016/j.knosys.2019.105257] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
18
|
Liu J, Sun T, Luo Y, Yang S, Cao Y, Zhai J. An echo state network architecture based on quantum logic gate and its optimization. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2019.09.002] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
19
|
|
20
|
Li B, Wei X, Li C, Ding S. Improvement effect of improved particle algorithm on BP neural network cell detection model. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2019. [DOI: 10.3233/jifs-179211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Bin Li
- School of Management, HeFei University of Technology, Hefei, China
- Bengbu Medical College, Bengbu, China
| | - Xing Wei
- Bengbu Medical College, Bengbu, China
| | - Chao Li
- Bengbu Medical College, Bengbu, China
| | - Shuai Ding
- School of Management, HeFei University of Technology, Hefei, China
| |
Collapse
|
21
|
Optimizing simple deterministically constructed cycle reservoir network with a Redundant Unit Pruning Auto-Encoder algorithm. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.05.035] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
22
|
Guo SS, Wang JS, Ma XX. Improved Bat Algorithm Based on Multipopulation Strategy of Island Model for Solving Global Function Optimization Problem. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:6068743. [PMID: 31531009 PMCID: PMC6721339 DOI: 10.1155/2019/6068743] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2019] [Revised: 07/03/2019] [Accepted: 07/08/2019] [Indexed: 11/17/2022]
Abstract
The bat algorithm (BA) is a heuristic algorithm that globally optimizes by simulating the bat echolocation behavior. In order to improve the search performance and further improve the convergence speed and optimization precision of the bat algorithm, an improved algorithm based on chaotic map is introduced, and the improved bat algorithm of Levy flight search strategy and contraction factor is proposed. The optimal chaotic map operator is selected based on the simulation experiments results. Then, a multipopulation parallel bat algorithm based on the island model is proposed. Finally, the typical test functions are used to carry out the simulation experiments. The simulation results show that the proposed improved algorithm can effectively improve the convergence speed and optimization accuracy.
Collapse
Affiliation(s)
- Sha-Sha Guo
- School of Electronic and Information Engineering, University of Science & Technology Liaoning, Anshan 114044, China
| | - Jie-Sheng Wang
- School of Electronic and Information Engineering, University of Science & Technology Liaoning, Anshan 114044, China
- School of International Finance and Banking, University of Science & Technology Liaoning, Anshan 114044, China
| | - Xiao-Xu Ma
- School of Electronic and Information Engineering, University of Science & Technology Liaoning, Anshan 114044, China
| |
Collapse
|
23
|
Bala A, Ismail I, Ibrahim R, Sait SM, Salami HO. Prediction Using Cuckoo Search Optimized Echo State Network. ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING 2019. [DOI: 10.1007/s13369-019-04008-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
24
|
Yang C, Qiao J, Ahmad Z, Nie K, Wang L. Online sequential echo state network with sparse RLS algorithm for time series prediction. Neural Netw 2019; 118:32-42. [PMID: 31228722 DOI: 10.1016/j.neunet.2019.05.006] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2019] [Revised: 04/22/2019] [Accepted: 05/12/2019] [Indexed: 11/19/2022]
Abstract
Recently, the echo state networks (ESNs) have been widely used for time series prediction. To meet the demand of actual applications and avoid the overfitting issue, the online sequential ESN with sparse recursive least squares (OSESN-SRLS) algorithm is proposed. Firstly, the ℓ0 and ℓ1 norm sparsity penalty constraints of output weights are separately employed to control the network size. Secondly, the sparse recursive least squares (SRLS) algorithm and the subgradients technique are combined to estimate the output weight matrix. Thirdly, an adaptive selection mechanism for the ℓ0 or ℓ1 norm regularization parameter is designed. With the selected regularization parameter, it is proved that the developed SRLS shows comparable or better performance than the regular RLS. Furthermore, the convergence of OSESN-SRLS is theoretically analyzed to guarantee its effectiveness. Simulation results illustrate that the proposed OSESN-SRLS always outperforms other existing ESNs in terms of estimation accuracy and network compactness.
Collapse
Affiliation(s)
- Cuili Yang
- Faculty of Information Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, PR China
| | - Junfei Qiao
- Faculty of Information Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, PR China.
| | - Zohaib Ahmad
- Faculty of Information Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, PR China
| | - Kaizhe Nie
- Faculty of Information Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, PR China
| | - Lei Wang
- Faculty of Information Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, PR China
| |
Collapse
|
25
|
|
26
|
Lv SX, Peng L, Wang L. Stacked autoencoder with echo-state regression for tourism demand forecasting using search query data. Appl Soft Comput 2018. [DOI: 10.1016/j.asoc.2018.08.024] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
|
27
|
|
28
|
Jiang S, Chin KS, Qu G, Tsui KL. An integrated machine learning framework for hospital readmission prediction. Knowl Based Syst 2018. [DOI: 10.1016/j.knosys.2018.01.027] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
29
|
Optimized Parameter Settings of Binary Bat Algorithm for Solving Function Optimization Problems. JOURNAL OF ELECTRICAL AND COMPUTER ENGINEERING 2018. [DOI: 10.1155/2018/3847951] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The bat algorithm (BA) is a new bionic intelligent optimization algorithm to simulate the foraging behavior and the echolocation principle of the bats. The parameter initialization of the discussed binary bat algorithm (BBA) has important influence on the convergence speed, convergence precision, and good global searching ability of the BBA. The convergence speed and algorithm searching precision are determined by the pulse of loudness and pulse rate. The simulation experiments are carried out by using the six typical test functions to discuss this influence. The simulation results show that the convergence speed of the BBA is relatively sensitive to the setting of the algorithm parameters. The convergence precision reduces when increasing the rate of bat transmitted pulse alone and the convergence speed increases the launch loudness alone. The proper combination of BBA parameters (the rate of bat transmitted pulse and the launch loudness) can flexibly improve the algorithm’s convergence velocity and improve the accuracy of the searched solutions.
Collapse
|
30
|
|
31
|
Empowering particle swarm optimization algorithm using multi agents’ capability: A holonic approach. Knowl Based Syst 2017. [DOI: 10.1016/j.knosys.2017.08.023] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
32
|
|
33
|
Seasonal SVR with FOA algorithm for single-step and multi-step ahead forecasting in monthly inbound tourist flow. Knowl Based Syst 2016. [DOI: 10.1016/j.knosys.2016.07.023] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
34
|
Meng Z, Pan JS. Monkey King Evolution: A new memetic evolutionary algorithm and its application in vehicle fuel consumption optimization. Knowl Based Syst 2016. [DOI: 10.1016/j.knosys.2016.01.009] [Citation(s) in RCA: 56] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
35
|
Cui H, Feng J, Guo J, Wang T. A novel single multiplicative neuron model trained by an improved glowworm swarm optimization algorithm for time series prediction. Knowl Based Syst 2015. [DOI: 10.1016/j.knosys.2015.07.032] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|