1
|
Two phase learning technique in modular neural network for pattern classification of handwritten Hindi alphabets. MACHINE LEARNING WITH APPLICATIONS 2021. [DOI: 10.1016/j.mlwa.2021.100174] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
2
|
Xing YL, Sun H, Feng GH, Shen FR, Zhao J. Artificial Evolution Network: A Computational Perspective on the Expansibility of the Nervous System. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021; 32:2180-2194. [PMID: 32584773 DOI: 10.1109/tnnls.2020.3002556] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Neurobiologists recently found the brain can use sudden emerged channels to process information. Based on this finding, we put forward a question whether we can build a computation model that is able to integrate a sudden emerged new type of perceptual channel into itself in an online way. If such a computation model can be established, it will introduce a channel-free property to the computation model and meanwhile deepen our understanding about the extendibility of the brain. In this article, a biologically inspired neural network named artificial evolution (AE) network is proposed to handle the problem. When a new perceptual channel emerges, the neurons in the network can grow new connections to connect the emerged channel according to the Hebb rule. In this article, we design a sensory channel expansion experiment to test the AE network. The experimental results demonstrate that the AE network can handle the sudden emerged perceptual channels effectively.
Collapse
|
3
|
Lim DH, Wu S, Zhao R, Lee JH, Jeong H, Shi L. Spontaneous sparse learning for PCM-based memristor neural networks. Nat Commun 2021; 12:319. [PMID: 33436611 PMCID: PMC7803975 DOI: 10.1038/s41467-020-20519-z] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2020] [Accepted: 11/27/2020] [Indexed: 01/29/2023] Open
Abstract
Neural networks trained by backpropagation have achieved tremendous successes on numerous intelligent tasks. However, naïve gradient-based training and updating methods on memristors impede applications due to intrinsic material properties. Here, we built a 39 nm 1 Gb phase change memory (PCM) memristor array and quantified the unique resistance drift effect. On this basis, spontaneous sparse learning (SSL) scheme that leverages the resistance drift to improve PCM-based memristor network training is developed. During training, SSL regards the drift effect as spontaneous consistency-based distillation process that reinforces the array weights at the high-resistance state continuously unless the gradient-based method switches them to low resistance. Experiments show that the SSL not only helps the convergence of network with better performance and sparsity controllability without additional computation in handwritten digit classification. This work promotes the learning algorithms with the intrinsic properties of memristor devices, opening a new direction for development of neuromorphic computing chips.
Collapse
Affiliation(s)
- Dong-Hyeok Lim
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, 100084, Beijing, China
- Department of Materials Science and Technology, Center for Future Semiconductor Technology, UNIST, 44919, Ulsan, South Korea
| | - Shuang Wu
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, 100084, Beijing, China
| | - Rong Zhao
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, 100084, Beijing, China
| | - Jung-Hoon Lee
- Department of Electronic Engineering, Center for Brain Inspired Computing Research, Tsinghua University, 100084, Beijing, China
| | - Hongsik Jeong
- Department of Electronic Engineering, Center for Brain Inspired Computing Research, Tsinghua University, 100084, Beijing, China.
- Department of Materials Science and Technology, Center for Future Semiconductor Technology, UNIST, 44919, Ulsan, South Korea.
| | - Luping Shi
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, 100084, Beijing, China.
| |
Collapse
|
4
|
Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence. Neural Netw 2019; 121:208-228. [PMID: 31574412 DOI: 10.1016/j.neunet.2019.08.033] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2018] [Revised: 05/12/2019] [Accepted: 08/29/2019] [Indexed: 11/21/2022]
Abstract
This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning, namely the distributed dual vigilance fuzzy ART (DDVFA). DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with distributed higher-order activation and match functions and a dual vigilance mechanism. Together, these allow DDVFA to perform unsupervised modularization, create multi-prototype cluster representations, retrieve arbitrarily-shaped clusters, and reduce category proliferation. Another important contribution is the reduction of order-dependence, an issue that affects any agglomerative clustering method. This paper demonstrates two approaches for mitigating order-dependence: pre-processing using visual assessment of cluster tendency (VAT) or post-processing using a novel Merge ART module. The former is suitable for batch processing, whereas the latter also works for online learning. Experimental results in online mode carried out on 30 benchmark data sets show that DDVFA cascaded with Merge ART statistically outperformed the best other ART-based systems when samples were randomly presented. Conversely, they were found to be statistically equivalent in offline mode when samples were pre-processed using VAT. Remarkably, performance comparisons to non-ART-based clustering algorithms show that DDVFA (which learns incrementally) was also statistically equivalent to the non-incremental (offline) methods of density-based spatial clustering of applications with noise (DBSCAN), single linkage hierarchical agglomerative clustering (SL-HAC), and k-means, while retaining the appealing properties of ART. Links to the source code and data are provided. Considering the algorithm's simplicity, online learning capability, and performance, it is an ideal choice for many agglomerative clustering applications.
Collapse
|
5
|
|
6
|
Xing YL, Shi XF, Shen FR, Zhao JX, Pan JX, Tan AH. Perception Coordination Network: A Neuro Framework for Multimodal Concept Acquisition and Binding. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019; 30:1104-1118. [PMID: 30137016 DOI: 10.1109/tnnls.2018.2861680] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
To simulate the concept acquisition and binding of different senses in the brain, a biologically inspired neural network model named perception coordination network (PCN) is proposed. It is a hierarchical structure, which is functionally divided into the primary sensory area (PSA), the primary sensory association area (SAA), and the higher order association area (HAA). The PSA contains feature neurons which respond to many elementary features, e.g., colors, shapes, syllables, and basic flavors. The SAA contains primary concept neurons which combine the elementary features in the PSA to represent unimodal concept of objects, e.g., the image of an apple, the Chinese word "[píng guǒ]" which names the apple, and the taste of the apple. The HAA contains associated neurons which connect the primary concept neurons of several PSA, e.g., connects the image, the taste, and the name of an apple. It means that the associated neurons have a multimodal response mode. Therefore, this area executes multisensory integration. PCN is an online incremental learning system, it is able to continuously acquire and bind multimodality concepts in an online way. The experimental results suggest that PCN is able to handle the multimodal concept acquisition and binding effectively.
Collapse
|
7
|
Eshraghian JK, Baek S, Kim JH, Iannella N, Cho K, Goo YS, Iu HHC, Kang SM, Eshraghian K. Formulation and Implementation of Nonlinear Integral Equations to Model Neural Dynamics Within the Vertebrate Retina. Int J Neural Syst 2018; 28:1850004. [PMID: 29631506 DOI: 10.1142/s0129065718500041] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Existing computational models of the retina often compromise between the biophysical accuracy and a hardware-adaptable methodology of implementation. When compared to the current modes of vision restoration, algorithmic models often contain a greater correlation between stimuli and the affected neural network, but lack physical hardware practicality. Thus, if the present processing methods are adapted to complement very-large-scale circuit design techniques, it is anticipated that it will engender a more feasible approach to the physical construction of the artificial retina. The computational model presented in this research serves to provide a fast and accurate predictive model of the retina, a deeper understanding of neural responses to visual stimulation, and an architecture that can realistically be transformed into a hardware device. Traditionally, implicit (or semi-implicit) ordinary differential equations (OES) have been used for optimal speed and accuracy. We present a novel approach that requires the effective integration of different dynamical time scales within a unified framework of neural responses, where the rod, cone, amacrine, bipolar, and ganglion cells correspond to the implemented pathways. Furthermore, we show that adopting numerical integration can both accelerate retinal pathway simulations by more than 50% when compared with traditional ODE solvers in some cases, and prove to be a more realizable solution for the hardware implementation of predictive retinal models.
Collapse
Affiliation(s)
- Jason K Eshraghian
- 1 School of Electrical, Electronic and Computer Engineering, University of Western Australia, Crawley, Australia
| | - Seungbum Baek
- 2 College of Electrical and Computer Engineering, Chungbuk National University, Cheongju, Republic of Korea
| | - Jun-Ho Kim
- 2 College of Electrical and Computer Engineering, Chungbuk National University, Cheongju, Republic of Korea
| | | | - Kyoungrok Cho
- 2 College of Electrical and Computer Engineering, Chungbuk National University, Cheongju, Republic of Korea
| | - Yong Sook Goo
- 4 Department of Physiology, School of Medicine, Chungbuk National University, Cheongju, Republic of Korea
| | - Herbert H C Iu
- 1 School of Electrical, Electronic and Computer Engineering, University of Western Australia, Crawley, Australia
| | - Sung-Mo Kang
- 5 Department of Electrical Engineering, University of California, Santa Cruz, Santa Cruz, USA
| | | |
Collapse
|
8
|
Pande S, Morgan F, Krewer F, Harkin J, McDaid L, McGinley B. Rapid application prototyping for hardware modular spiking neural network architectures. Neural Comput Appl 2017. [DOI: 10.1007/s00521-015-2136-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
9
|
Turner AP, Caves LSD, Stepney S, Tyrrell AM, Lones MA. Artificial Epigenetic Networks: Automatic Decomposition of Dynamical Control Tasks Using Topological Self-Modification. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2017; 28:218-230. [PMID: 26742145 DOI: 10.1109/tnnls.2015.2497142] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
This paper describes the artificial epigenetic network, a recurrent connectionist architecture that is able to dynamically modify its topology in order to automatically decompose and solve dynamical problems. The approach is motivated by the behavior of gene regulatory networks, particularly the epigenetic process of chromatin remodeling that leads to topological change and which underlies the differentiation of cells within complex biological organisms. We expected this approach to be useful in situations where there is a need to switch between different dynamical behaviors, and do so in a sensitive and robust manner in the absence of a priori information about problem structure. This hypothesis was tested using a series of dynamical control tasks, each requiring solutions that could express different dynamical behaviors at different stages within the task. In each case, the addition of topological self-modification was shown to improve the performance and robustness of controllers. We believe this is due to the ability of topological changes to stabilize attractors, promoting stability within a dynamical regime while allowing rapid switching between different regimes. Post hoc analysis of the controllers also demonstrated how the partitioning of the networks could provide new insights into problem structure.
Collapse
|
10
|
Wang Y, Gu J, Zhou Z, Wang Z. Diarrhoea outpatient visits prediction based on time series decomposition and multi-local predictor fusion. Knowl Based Syst 2015. [DOI: 10.1016/j.knosys.2015.08.013] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
11
|
Sánchez D, Melin P, Castillo O. Optimization of modular granular neural networks using a hierarchical genetic algorithm based on the database complexity applied to human recognition. Inf Sci (N Y) 2015. [DOI: 10.1016/j.ins.2015.02.020] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
12
|
Qiao J, Zhang Z, Bo Y. An online self-adaptive modular neural network for time-varying systems. Neurocomputing 2014. [DOI: 10.1016/j.neucom.2012.09.038] [Citation(s) in RCA: 35] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
13
|
Asadi S, Shahrabi J, Abbaszadeh P, Tabanmehr S. A new hybrid artificial neural networks for rainfall–runoff process modeling. Neurocomputing 2013. [DOI: 10.1016/j.neucom.2013.05.023] [Citation(s) in RCA: 81] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
14
|
Developing a hybrid intelligent model for forecasting problems: Case study of tourism demand time series. Knowl Based Syst 2013. [DOI: 10.1016/j.knosys.2013.01.014] [Citation(s) in RCA: 70] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
15
|
Pande S, Morgan F, Cawley S, Bruintjes T, Smit G, McGinley B, Carrillo S, Harkin J, McDaid L. Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network. Neural Process Lett 2013. [DOI: 10.1007/s11063-012-9274-5] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
16
|
Zarandi MHF, Hadavandi E, Turksen IB. A hybrid fuzzy intelligent agent-based system for stock price prediction. INT J INTELL SYST 2012. [DOI: 10.1002/int.21554] [Citation(s) in RCA: 35] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
17
|
Watanabe T, Seki H. Modeling Approach Based on Modular Fuzzy Model. JOURNAL OF ADVANCED COMPUTATIONAL INTELLIGENCE AND INTELLIGENT INFORMATICS 2012. [DOI: 10.20965/jaciii.2012.p0653] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Fuzzy modeling is one of the most important techniques for nonlinear modeling. SIRMs (Single Input Rule Modules) has been studied as a useful modeling method for real-life applications such as control and pattern recognition. Although the SIRMs is a practical modeling approach based on fuzzy reasoning, its performance is adversely affected by high-dimensional or complicated characteristics of the problems. The modular fuzzy model is an extension of the SIRMs for overcoming such a performance problem. In this paper, we study a modeling approach based on the modular fuzzy model by extending the SIRMs architecture. We show that the construction of error objective functions for modeling the modular fuzzy model and the SIRMs affects the prediction performance of the model. Through numerical experiments on modeling problems and reinforcement learning problems, we study the model construction based on the error objective functions. We find that the error objective function should be selected according to the number of dimensions of projection in the modular fuzzy model.
Collapse
|
18
|
FERGUSON BRENT, GHOSH RANADHIR, YEARWOOD JOHN. MODULAR NEURAL NETWORK DESIGN FOR THE PROBLEM OF ALPHABETIC CHARACTER RECOGNITION. INT J PATTERN RECOGN 2011. [DOI: 10.1142/s0218001405004009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
This paper reports on an experimental approach to find a modularized artificial neural network solution for the UCI letters recognition problem. Our experiments have been carried out in two parts. We investigate directed task decomposition using expert knowledge and clustering approaches to find the subtasks for the modules of the network. We next investigate processes to combine the modules effectively in a single decision process. After having found suitable modules through task decomposition we have found through further experimentation that when the modules are combined with decision tree supervision, their functional error is reduced significantly to improve their combination through the decision process that has been implemented as a small multilayered perceptron. The experiments conclude with a modularized neural network design for this classification problem that has increased learning and generalization characteristics. The test results for this network are markedly better than a single or stand alone network that has a fully connected topology.
Collapse
Affiliation(s)
- BRENT FERGUSON
- University of Ballarat, PO Box 663, Ballarat, Victoria 3353, Australia
| | - RANADHIR GHOSH
- University of Ballarat, PO Box 663, Ballarat, Victoria 3353, Australia
| | - JOHN YEARWOOD
- University of Ballarat, PO Box 663, Ballarat, Victoria 3353, Australia
| |
Collapse
|
19
|
KUMAR RAJEEV. ANCHOR — A CONNECTIONIST ARCHITECTURE FOR PARTITIONING FEATURE SPACES AND HIERARCHICAL NESTING OF NEURAL NETS. INT J ARTIF INTELL T 2011. [DOI: 10.1142/s0218213000000252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
We present a novel connectionist architecture for handling arbitrarily complex neural computations. The architecture — which we call an Artificial Neural Network Compiler for Hierarchical ORganisation (ANCHOR) — facilitates network hierarchy and the partitioning of the input space into a number of small, simpler sub-mappings. This strategy is a mix of both a modular approach for reduction of the problem complexity, and an ensemble-based approach for increasing prediction accuracy. ANCHOR is implemented around the concept of a Superneuron which is a generalised view of a neuron-processing element; a superneuron is a single (or higher-order) perceptron which can be trained individually with the support of multiple learning algorithms. The indistinguishability between a superneuron, and a neuron is employed in hierarchical nesting of superneurons, up to (theoretically) infinite depth, within other superneurons as well as linear or tree-structured cascading. Hierarchical decomposition of simple boolean functions has been demonstrated as proof-of-concept. It is argued that this strategy of data-partitioning for subsequent mapping onto a hierarchical classifier reduces the learning complexity and offers better generalisation.
Collapse
Affiliation(s)
- RAJEEV KUMAR
- Department of Computer Science & Information Systems, Birla Institute of Technology & Science, Pilani – 333 031, India
| |
Collapse
|
20
|
Fu HC, Lee YP, Chiang CC, Pao HT. Divide-and-conquer learning and modular perceptron networks. ACTA ACUST UNITED AC 2010; 12:250-63. [PMID: 18244382 DOI: 10.1109/72.914522] [Citation(s) in RCA: 27] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
A novel modular perceptron network (MPN) and divide-and-conquer learning (DCL) schemes for the design of modular neural networks are proposed. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region into two easier to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We evaluated and compared the proposed MPN with several representative neural networks on the two-spirals problem and real-world dataset. The MPN achieved better weight learning performance by requiring much less data presentations during the network training phases, and better generalization performance, and less processing time during the retrieving phase.
Collapse
Affiliation(s)
- H C Fu
- Department of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu, Taiwan 300, R.O.C
| | | | | | | |
Collapse
|
21
|
Chris Tseng H, Almogahed B. Modular neural networks with applications to pattern profiling problems. Neurocomputing 2009. [DOI: 10.1016/j.neucom.2008.10.020] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
22
|
Cooperative recurrent modular neural networks for constrained optimization: a survey of models and applications. Cogn Neurodyn 2008; 3:47-81. [PMID: 19003467 DOI: 10.1007/s11571-008-9036-2] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2007] [Accepted: 11/27/2007] [Indexed: 10/22/2022] Open
Abstract
Constrained optimization problems arise in a wide variety of scientific and engineering applications. Since several single recurrent neural networks when applied to solve constrained optimization problems for real-time engineering applications have shown some limitations, cooperative recurrent neural network approaches have been developed to overcome drawbacks of these single recurrent neural networks. This paper surveys in details work on cooperative recurrent neural networks for solving constrained optimization problems and their engineering applications, and points out their standing models from viewpoint of both convergence to the optimal solution and model complexity. We provide examples and comparisons to shown advantages of these models in the given applications.
Collapse
|
23
|
|
24
|
An optimized modular neural network controller based on environment classification and selective sensor usage for mobile robot reactive navigation. Neural Comput Appl 2007. [DOI: 10.1007/s00521-006-0079-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
25
|
Wedge D, Ingram D, McLean D, Mingham C, Bandar Z. On global-local artificial neural networks for function approximation. IEEE TRANSACTIONS ON NEURAL NETWORKS 2006; 17:942-952. [PMID: 16856657 DOI: 10.1109/tnn.2006.875972] [Citation(s) in RCA: 34] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
We present a hybrid radial basis function (RBF) sigmoid neural network with a three-step training algorithm that utilizes both global search and gradient descent training. The algorithm used is intended to identify global features of an input-output relationship before adding local detail to the approximating function. It aims to achieve efficient function approximation through the separate identification of aspects of a relationship that are expressed universally from those that vary only within particular regions of the input space. We test the effectiveness of our method using five regression tasks; four use synthetic datasets while the last problem uses real-world data on the wave overtopping of seawalls. It is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways: lower mean square errors are often achievable using fewer hidden neurons and with less need for regularization. Our global-local artificial neural network (GL-ANN) is also seen to compare favorably with both perceptron radial basis net and regression tree derived RBFs. A number of issues concerning the training of GL-ANNs are discussed: the use of regularization, the inclusion of a gradient descent optimization step, the choice of RBF spreads, model selection, and the development of appropriate stopping criteria.
Collapse
|
26
|
Chalup SK. Incremental learning in biological and machine learning systems. Int J Neural Syst 2002; 12:447-65. [PMID: 12528196 DOI: 10.1142/s0129065702001308] [Citation(s) in RCA: 15] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2002] [Revised: 09/23/2002] [Accepted: 09/25/2002] [Indexed: 11/18/2022]
Abstract
Incremental learning concepts are reviewed in machine learning and neurobiology. They are identified in evolution, neurodevelopment and learning. A timeline of qualitative axon, neuron and synapse development summarizes the review on neurodevelopment. A discussion of experimental results on data incremental learning with recurrent artificial neural networks reveals that incremental learning often seems to be more efficient or powerful than standard learning but can produce unexpected side effects. A characterization of incremental learning is proposed which takes the elaborated biological and machine learning concepts into account.
Collapse
Affiliation(s)
- Stephan K Chalup
- School of Electrical Engineering and Computer Science, The University of Newcastle, Australia.
| |
Collapse
|