Universal Approximation Using Radial-Basis-Function Networks

Artificial neural networks as a useful tool to predict the risk level of Betula pollen in the air

An increasing percentage of the European population suffers from allergies to pollen. The study of the evolution of air pollen concentration supplies prior knowledge of the levels of pollen in the air, which can be useful for the prevention and treatment of allergic symptoms, and the management of medical resources. The symptoms of Betula pollinosis can be associated with certain levels of pollen in the air. The aim of this study was to predict the risk of the concentration of pollen exceeding a given level, using previous pollen and meteorological information, by applying neural network techniques. Neural networks are a widespread statistical tool useful for the study of problems associated with complex or poorly understood phenomena. The binary response variable associated with each level requires a careful selection of the neural network and the error function associated with the learning algorithm used during the training phase. The performance of the neural network with the validation set showed that the risk of the pollen level exceeding a certain threshold can be successfully forecasted using artificial neural networks. This prediction tool may be implemented to create an automatic system that forecasts the risk of suffering allergic symptoms.

High-Speed Face Recognition Based on Discrete Cosine Transform and RBF Neural Networks

In this paper, an efficient method for high-speed face recognition based on the discrete cosine transform (DCT), the Fisher's linear discriminant (FLD) and radial basis function (RBF) neural networks is presented. First, the dimensionality of the original face image is reduced by using the DCT and the large area illumination variations are alleviated by discarding the first few low-frequency DCT coefficients. Next, the truncated DCT coefficient vectors are clustered using the proposed clustering algorithm. This process makes the subsequent FLD more efficient. After implementing the FLD, the most discriminating and invariant facial features are maintained and the training samples are clustered well. As a consequence, further parameter estimation for the RBF neural networks is fulfilled easily which facilitates fast training in the RBF neural networks. Simulation results show that the proposed system achieves excellent performance with high training and recognition speed, high recognition rate as well as very good illumination robustness.

On the capabilities of higher-order neurons: a radial basis function approach

Higher-order neurons with k monomials in n variables are shown to have Vapnik-Chervonenkis (VC) dimension at least nk + 1. This result supersedes the previously known lower bound obtained via k-term monotone disjunctive normal form (DNF) formulas. Moreover, it implies that the VC dimension of higher-order neurons with k monomials is strictly larger than the VC dimension of k-term monotone DNF. The result is achieved by introducing an exponential approach that employs gaussian radial basis function neural networks for obtaining classifications of points in terms of higher-order neurons.

Neural Network-Based Adaptive Dynamic Surface Control for a Class of Uncertain Nonlinear Systems in Strict-Feedback Form

The dynamic surface control (DSC) technique was developed recently by Swaroop et al. This technique simplified the backstepping design for the control of nonlinear systems in strict-feedback form by overcoming the problem of "explosion of complexity." It was later extended to adaptive backstepping design for nonlinear systems with linearly parameterized uncertainty. In this paper, by incorporating this design technique into a neural network based adaptive control design framework, we have developed a backstepping based control design for a class of nonlinear systems in strict-feedback form with arbitrary uncertainty. Our development is able to eliminate the problem of "explosion of complexity" inherent in the existing method. In addition, a stability analysis is given which shows that our control law can guarantee the uniformly ultimate boundedness of the solution of the closed-loop system, and make the tracking error arbitrarily small.

Hidden Space Support Vector Machines

Hidden space support vector machines (HSSVMs) are presented in this paper. The input patterns are mapped into a high-dimensional hidden space by a set of hidden nonlinear functions and then the structural risk is introduced into the hidden space to construct HSSVMs. Moreover, the conditions for the nonlinear kernel function in HSSVMs are more relaxed, and even differentiability is not required. Compared with support vector machines (SVMs), HSSVMs can adopt more kinds of kernel functions because the positive definite property of the kernel function is not a necessary condition. The performance of HSSVMs for pattern recognition and regression estimation is also analyzed. Experiments on artificial and real-world domains confirm the feasibility and the validity of our algorithms.

Multiscale approximation with hierarchical radial basis functions networks

An approximating neural model, called hierarchical radial basis function (HRBF) network, is presented here. This is a self-organizing (by growing) multiscale version of a radial basis function (RBF) network. It is constituted of hierarchical layers, each containing a Gaussian grid at a decreasing scale. The grids are not completely filled, but units are inserted only where the local error is over threshold. This guarantees a uniform residual error and the allocation of more units with smaller scales where the data contain higher frequencies. Only local operations, which do not require any iteration on the data, are required; this allows to construct the network in quasi-real time. Through harmonic analysis, it is demonstrated that, although a HRBF cannot be reduced to a traditional wavelet-based multiresolution analysis (MRA), it does employ Riesz bases and enjoys asymptotic approximation properties for a very large class of functions. HRBF networks have been extensively applied to the reconstruction of three-dimensional (3-D) models from noisy range data. The results illustrate their power in denoising the original data, obtaining an effective multiscale reconstruction of better quality than that obtained by MRA.

When is approximation by Gaussian networks necessarily a linear process?

Let s > or = 1 be an integer. A Gaussian network is a function on Rs of the form [Formula: see text]. The minimal separation among the centers, defined by (1/2) min(1 < or = j not = k < or = N) [Formula: see text], is an important characteristic of the network that determines the stability of interpolation by Gaussian networks, the degree of approximation by such networks, etc. Let (within this abstract only) the set of all Gaussian networks with minimal separation exceeding 1/m be denoted by Gm. We prove that for functions [Formula: see text] such that [Formula: see text], if the degree of L2(nonlinear) approximation of [Formula: see text] from Gm is [Formula: see text] then necessarily the degree of approximation of [Formula: see text] by (rectangular) partial sums of degree m2 of the Hermite expansion of [Formula: see text] is also [Formula: see text]. Moreover, Gaussian networks in Gm having fixed centers in a ball of radius [Formula: see text] and coefficients being linear functionals of [Formula: see text] can be constructed to yield the same degree of approximation. Similar results are proved for the Lp norms, 1 < or = p < or =[Formula: see text] but with the condition that the number of neurons N, should satisfy logN = [Formula: see text](m2).

Artificial neural networks for predictive modeling in prostate cancer

Artificial neural networks (ANNs) represent a relatively new methodology for predictive modeling in medicine. ANNs, a form of artificial intelligence loosely based on the brain, have a demonstrated ability to learn complex and subtle relationships between variables in medical applications. In contrast with traditional statistical techniques, ANNs are capable of automatically resolving these relationships without the need for a priori assumptions about the nature of the interactions between variables. As with any technique, ANNs have limitations and potential drawbacks. This article provides an overview of the theoretical basis of ANNs, how they function, their strengths and limitations, and examples of how ANNs have been used to develop predictive models for the management of prostate cancer.

Adaptive Neural Control of Uncertain MIMO Nonlinear Systems

In this paper, adaptive neural control schemes are proposed for two classes of uncertain multi-input/multi-output (MIMO) nonlinear systems in block-triangular forms. The MIMO systems consist of interconnected subsystems, with couplings in the forms of unknown nonlinearities and/or parametric uncertainties in the input matrices, as well as in the system interconnections without any bounding restrictions. Using the block-triangular structure properties, the stability analyses of the closed-loop MIMO systems are shown in a nested iterative manner for all the states. By exploiting the special properties of the affine terms of the two classes of MIMO systems, the developed neural control schemes avoid the controller singularity problem completely without using projection algorithms. Semiglobal uniform ultimate boundedness (SGUUB) of all the signals in the closed-loop of MIMO nonlinear systems is achieved. The outputs of the systems are proven to converge to a small neighborhood of the desired trajectories. The control performance of the closed-loop system is guaranteed by suitably choosing the design parameters. The proposed schemes offer systematic design procedures for the control of the two classes of uncertain MIMO nonlinear systems. Simulation results are presented to show the effectiveness of the approach.

Activation Functions Defined on Higher-Dimensional Spaces for Approximation on Compact Sets with and without Scaling

Let g be a slowly increasing function of locally bounded variation defined on Rc, 1 ≤c≤d. We investigate when g can be an activation function of the hidden-layer units of three-layer neural networks that approximate continuous functions on compact sets. If the support of the Fourier transform of g includes a converging sequence of points with distinct distances from the origin, it can be an activation function without scaling. If and only if the support of its Fourier transform includes a point other than the origin, it can be an activation function with scaling. We also look for a condition on which an activation function can be used for approximation without rotation. Any nonpolynomial functions can be activation functions with scaling, and many familiar functions, such as sigmoid functions and radial basis functions, can be activation functions without scaling. With or without scaling, some of them defined on Rd can be used without rotation even if they are not spherically symmetric.

Relaxed conditions for radial-basis function networks to be universal approximators

In this paper, we investigate the universal approximation property of Radial Basis Function (RBF) networks. We show that RBFs are not required to be integrable for the REF networks to be universal approximators. Instead, RBF networks can uniformly approximate any continuous function on a compact set provided that the radial basis activation function is continuous almost everywhere, locally essentially bounded, and not a polynomial. The approximation in L(p)(micro)(1 < or = p < infinity) space is also discussed. Some experimental results are reported to illustrate our findings.

Source separation in astrophysical maps using independent factor analysis

A microwave sky map results from a combination of signals from various astrophysical sources, such as cosmic microwave background radiation, synchrotron radiation and galactic dust radiation. To derive information about these sources, one needs to separate them from the measured maps on different frequency channels. Our insufficient knowledge of the weights to be given to the individual signals at different frequencies makes this a difficult task. Recent work on the problem led to only limited success due to ignoring the noise and to the lack of a suitable statistical model for the sources. In this paper, we derive the statistical distribution of some source realizations, and check the appropriateness of a Gaussian mixture model for them. A source separation technique, namely, independent factor analysis, has been suggested recently in the literature for Gaussian mixture sources in the presence of noise. This technique employs a three layered neural network architecture which allows a simple, hierarchical treatment of the problem. We modify the algorithm proposed in the literature to accommodate for space-varying noise and test its performance on simulated astrophysical maps. We also compare the performances of an expectation-maximization and a simulated annealing learning algorithm in estimating the mixture matrix and the source model parameters. The problem with expectation-maximization is that it does not ensure global optimization, and thus the choice of the starting point is a critical task. Indeed, we did not succeed to reach good solutions for random initializations of the algorithm. Conversely, our experiments with simulated annealing yielded initialization-independent results. The mixing matrix and the means and coefficients in the source model were estimated with a good accuracy while some of the variances of the components in the mixture model were not estimated satisfactorily.

Hierarchical radial basis function networks and local polynomial un-warping for X-ray image intensifier distortion correction: a comparison with global techniques

Global polynomial (GP) methods have been widely used to correct geometric image distortion of small-size (up to 30 cm) X-ray image intensifiers (XRIIs). This work confirms that this kind of approach is suitable for 40 cm XRIIs (now increasingly used). Nonetheless, two local methods, namely 3rd-order local un-warping polynomials (LUPs) and hierarchical radial basis function (HRBF) networks are proposed as alternative solutions. Extensive experimental tests were carried out to compare these methods with classical low-order local polynomial and GP techniques, in terms of residual error (RMSE) measured at points not used for parameter estimation. Simulations showed that the LUP and HRBF methods had accuracies comparable with that attained using GP methods. In detail, the LUP method (0.353 microm) performed worse than HRBF (0.348 microm) only for small grid spacing (15 x 15 control points); the accuracy of both HRBF (0.157 microm) and LUP (0.160 microm) methods was little affected by local distortions (30 x 30 control points); weak local distortions made the GP method poorer (0.320 microm). Tests on real data showed that LUP and HRBF had accuracies comparable with that of GP for both 30 cm (GP: 0.238 microm; LUP: 0.240 microm; HRBF: 0.238 microm) and 40 cm (GP: 0.164 microm; LUP: 0.164 microm; HRBF: 0.164 microm) XRIIs. The LUP-based distortion correction was implemented in real time for image correction in digital tomography applications.

Indexed families of functionals and gaussian radial basis functions

We report on results concerning the capabilities of gaussian radial basis function networks in the setting of inner product spaces that need not be finite dimensional. Specifically, we show that important indexed families of functionals can be uniformly approximated, with the approximation uniform also with respect to the index. Applications are described concerning the classification of signals and the synthesis of reconfigurable classifiers.

Descartes' rule of signs for radial basis function neural networks

We establish versions of Descartes' rule of signs for radial basis function (RBF) neural networks. The RBF rules of signs provide tight bounds for the number of zeros of univariate networks with certain parameter restrictions. Moreover, they can be used to infer that the Vapnik-Chervonenkis (VC) dimension and pseudodimension of these networks are no more than linear. This contrasts with previous work showing that RBF neural networks with two or more input nodes have superlinear VC dimension. The rules also give rise to lower bounds for network sizes, thus demonstrating the relevance of network parameters for the complexity of computing with RBF neural networks.

A New Color 3D SFS Methodology Using Neural-Based Color Reflectance Models and Iterative Recursive Method

In this article, a new methodology for color shape from shading (SFS) problem is proposed. The problem of color SFS refers to the well-known fact that most real objects usually contain mixtures of diffuse and specular color reflections and are affected by the multicolored interreflection under unknown reflectivity. In this article, these limitations are addressed, and a new color SFS methodology is proposed. The proposed approach focuses on two main parts. First, a generalized neural-based color reflectance model is developed. Second, an iterative recursive method is developed to reconstruct a multicolor 3D surface. Experimental results on synthetic-colored objects and real-colored objects were performed to demonstrate the performance of the proposed methodology.

Distortion correction for x-ray image intensifiers: local unwarping polynomials and RBF neural networks

In this paper we present two novel techniques, namely a local unwarping polynomial (LUP) and a hierarchical radial basis function (HRBF) network, to correct geometric distortions in XRII images. The two techniques have been implemented and compared, in terms of residual error measured at control and intermediate points, with local and global methods reported in the previous literature. In particular, LUP rests on a locally optimized 3rd degree polynomial applied within each quadrilateral cell on the rectilinear calibration grid of points. HRBF, based on a feed-forward neural network paradigm, is constituted by a set of hierarchical layers at increasing cut-off frequency, each characterized by a set of Gaussian functions. Extensive experiments have been performed both on simulated and real data. In simulation, we tested the effect of pincushion, sigmoidal and local distortions, along with the number of calibration points. Provided that a sufficient number of cells of the calibration grid is available, the obtained accuracy for both LUP and HRBF is comparable to or better than that of global polynomial technique. Tests on real data, carried out by using two different (12 in. and 16 in.) XRIIs, showed that the global polynomial accuracy (0.16+/-0.08 pixels) is slightly worse than that of LUP (0.07+/-0.05 pixels) and HRBF (0.08+/-0.04 pixels). The effects of the discontinuity at the border of the local areas and the decreased accuracy at intermediate points, typical of local techniques, have been proved to be smoothed for both LUP and HRBF.

Neural networks with local receptive fields and superlinear VC dimension

Local receptive field neurons comprise such well-known and widely used unit types as radial basis function (RBF) neurons and neurons with center-surround receptive field. We study the Vapnik-Chervonenkis (VC) dimension of feedforward neural networks with one hidden layer of these units. For several variants of local receptive field neurons, we show that the VC dimension of these networks is superlinear. In particular, we establish the bound Omega(W log k) for any reasonably sized network with W parameters and k hidden nodes. This bound is shown to hold for discrete center-surround receptive field neurons, which are physiologically relevant models of cells in the mammalian visual system, for neurons computing a difference of gaussians, which are popular in computational vision, and for standard RBF neurons, a major alternative to sigmoidal neurons in artificial neural networks. The result for RBF neural networks is of particular interest since it answers a question that has been open for several years. The results also give rise to lower bounds for networks with fixed input dimension. Regarding constants, all bounds are larger than those known thus far for similar architectures with sigmoidal neurons. The superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here.