Generalization and specialization in artificial neural networks

Phenotypic Switching Resulting From Developmental Plasticity: Fixed or Reversible?

The prevalent view of developmental phenotypic switching holds that phenotype modifications occurring during critical windows of development are "irreversible" - that is, once produced by environmental perturbation, the consequent juvenile and/or adult phenotypes are indelibly modified. Certainly, many such changes appear to be non-reversible later in life. Yet, whether animals with switched phenotypes during early development are unable to return to a normal range of adult phenotypes, or whether they do not experience the specific environmental conditions necessary for them to switch back to the normal range of adult phenotypes, remains an open question. Moreover, developmental critical windows are typically brief, early periods punctuating a much longer period of overall development. This leaves open additional developmental time for reversal (correction) of a switched phenotype resulting from an adverse environment early in development. Such reversal could occur from right after the critical window "closes," all the way into adulthood. In fact, examples abound of the capacity to return to normal adult phenotypes following phenotypic changes enabled by earlier developmental plasticity. Such examples include cold tolerance in the fruit fly, developmental switching of mouth formation in a nematode, organization of the spinal cord of larval zebrafish, camouflage pigmentation formation in larval newts, respiratory chemosensitivity in frogs, temperature-metabolism relations in turtles, development of vascular smooth muscle and kidney tissue in mammals, hatching/birth weight in numerous vertebrates,. More extreme cases of actual reversal (not just correction) occur in invertebrates (e.g., hydrozoans, barnacles) that actually 'backtrack' along normal developmental trajectories from adults back to earlier developmental stages. While developmental phenotypic switching is often viewed as a permanent deviation from the normal range of developmental plans, the concept of developmental phenotypic switching should be expanded to include sufficient plasticity allowing subsequent correction resulting in the normal adult phenotype.

The superior fault tolerance of artificial neural network training with a fault/noise injection-based genetic algorithm

Artificial neural networks (ANNs) are powerful computational tools that are designed to replicate the human brain and adopted to solve a variety of problems in many different fields. Fault tolerance (FT), an important property of ANNs, ensures their reliability when significant portions of a network are lost. In this paper, a fault/noise injection-based (FIB) genetic algorithm (GA) is proposed to construct fault-tolerant ANNs. The FT performance of an FIB-GA was compared with that of a common genetic algorithm, the back-propagation algorithm, and the modification of weights algorithm. The FIB-GA showed a slower fitting speed when solving the exclusive OR (XOR) problem and the overlapping classification problem, but it significantly reduced the errors in cases of single or multiple faults in ANN weights or nodes. Further analysis revealed that the fit weights showed no correlation with the fitting errors in the ANNs constructed with the FIB-GA, suggesting a relatively even distribution of the various fitting parameters. In contrast, the output weights in the training of ANNs implemented with the use the other three algorithms demonstrated a positive correlation with the errors. Our findings therefore indicate that a combination of the fault/noise injection-based method and a GA is capable of introducing FT to ANNs and imply that the distributed ANNs demonstrate superior FT performance.