Universal Approximation Using Radial-Basis-Function Networks

Systematic Boolean Satisfiability Programming in Radial Basis Function Neural Network

Radial Basis Function Neural Network (RBFNN) is a class of Artificial Neural Network (ANN) that contains hidden layer processing units (neurons) with nonlinear, radially symmetric activation functions. Consequently, RBFNN has extensively suffered from significant computational error and difficulties in approximating the optimal hidden neuron, especially when dealing with Boolean Satisfiability logical rule. In this paper, we present a comprehensive investigation of the potential effect of systematic Satisfiability programming as a logical rule, namely 2 Satisfiability (2SAT) to optimize the output weights and parameters in RBFNN. The 2SAT logical rule has extensively applied in various disciplines, ranging from industrial automation to the complex management system. The core impetus of this study is to investigate the effectiveness of 2SAT logical rule in reducing the computational burden for RBFNN by obtaining the parameters in RBFNN. The comparison is made between RBFNN and the existing method, based on the Hopfield Neural Network (HNN) in searching for the optimal neuron state by utilizing different numbers of neurons. The comparison was made with the HNN as a benchmark to validate the final output of our proposed RBFNN with 2SAT logical rule. Note that the final output in HNN is represented in terms of the quality of the final states produced at the end of the simulation. The simulation dynamic was carried out by using the simulated data, randomly generated by the program. In terms of 2SAT logical rule, simulation revealed that RBFNN has two advantages over HNN model: RBFNN can obtain the correct final neuron state with the lowest error and does not require any approximation for the number of hidden layers. Furthermore, this study provides a new paradigm in the field feed-forward neural network by implementing a more systematic propositional logic rule.

A semi-holographic hyperdimensional representation system for hardware-friendly cognitive computing

One of the main, long-term objectives of artificial intelligence is the creation of thinking machines. To that end, substantial effort has been placed into designing cognitive systems; i.e. systems that can manipulate semantic-level information. A substantial part of that effort is oriented towards designing the mathematical machinery underlying cognition in a way that is very efficiently implementable in hardware. In this work, we propose a 'semi-holographic' representation system that can be implemented in hardware using only multiplexing and addition operations, thus avoiding the need for expensive multiplication. The resulting architecture can be readily constructed by recycling standard microprocessor elements and is capable of performing two key mathematical operations frequently used in cognition, superposition and binding, within a budget of below 6 pJ for 64-bit operands. Our proposed 'cognitive processing unit' is intended as just one (albeit crucial) part of much larger cognitive systems where artificial neural networks of all kinds and associative memories work in concord to give rise to intelligence. This article is part of the theme issue 'Harmonizing energy-autonomous computing and intelligence'.

Leveraging effect size distributions to improve polygenic risk scores derived from summary statistics of genome-wide association studies

Genetic risk prediction is an important problem in human genetics, and accurate prediction can facilitate disease prevention and treatment. Calculating polygenic risk score (PRS) has become widely used due to its simplicity and effectiveness, where only summary statistics from genome-wide association studies are needed in the standard method. Recently, several methods have been proposed to improve standard PRS by utilizing external information, such as linkage disequilibrium and functional annotations. In this paper, we introduce EB-PRS, a novel method that leverages information for effect sizes across all the markers to improve prediction accuracy. Compared to most existing genetic risk prediction methods, our method does not need to tune parameters nor external information. Real data applications on six diseases, including asthma, breast cancer, celiac disease, Crohn's disease, Parkinson's disease and type 2 diabetes show that EB-PRS achieved 307.1%, 42.8%, 25.5%, 3.1%, 74.3% and 49.6% relative improvements in terms of predictive r2 over standard PRS method with optimally tuned parameters. Besides, compared to LDpred that makes use of LD information, EB-PRS also achieved 37.9%, 33.6%, 8.6%, 36.2%, 40.6% and 10.8% relative improvements. We note that our method is not the first method leveraging effect size distributions. Here we first justify our method by presenting theoretical optimal property over existing methods in this class of methods, and substantiate our theoretical result with extensive simulation results. The R-package EBPRS that implements our method is available on CRAN.

Neural Network-Based Distributed Cooperative Learning Control for Multiagent Systems via Event-Triggered Communication

In this paper, an event-based distributed cooperative learning (DCL) law is proposed for a group of adaptive neural control systems. The plants to be controlled have identical structures, but reference signals for each plant are different. During control process, each agent intermittently broadcasts its neural network (NN) weight estimation to its neighboring agents under an event-triggered condition that is only based on its own estimated NN weights. If communication topology is connected and undirected, the NN weights of all neural control systems can converge to a small neighborhood of their optimal values. The generalization ability of NNs is guaranteed in the event-triggered context, that is, the approximation domain of each NN is the union of all system trajectories. Furthermore, a strictly positive lower bound on the interevent intervals is also guaranteed to avoid the Zeno behavior. Finally, a numerical example is given to illustrate the effectiveness of the proposed learning law.

Tackling environmental challenges in pollution controls using artificial intelligence: A review

This review presents the developments in artificial intelligence technologies for environmental pollution controls. A number of AI approaches, which start with the reliable mapping of nonlinear behavior between inputs and outputs in chemical and biological processes in terms of prediction models to the emerging optimization and control algorithms that study the pollutants removal processes and intelligent control systems, have been developed for environmental clean-ups. The characteristics, advantages and limitations of AI methods, including single and hybrid AI methods, were overviewed. Hybrid AI methods exhibited synergistic effects, but with computational heaviness. The up-to-date review summarizes i) Various artificial neural networks employed in wastewater degradation process for the prediction of removal efficiency of pollutants and the search of optimizing experimental conditions; ii) Evaluation of fuzzy logic used for intelligent control of aerobic stage of wastewater treatment process; iii) AI-aided soft-sensors for precisely on-line/off-line estimation of hard-to-measure parameters in wastewater treatment plants; iv) Single and hybrid AI methods applied to estimate pollutants concentrations and design monitoring and early-warning systems for both aquatic and atmospheric environments; v) AI modelings of short-term, mid-term and long-term solid waste generations, and various ANNs for solid waste recycling and reduction. Finally, the future challenges of AI-based models employed in the environmental fields are discussed and proposed.

Multi-view ensemble learning with empirical kernel for heart failure mortality prediction

Heart failure (HF) refers to the heart's inability to pump sufficient blood to maintain the body's needs, which has a very serious impact on human health. In recent years, the prevalence of HF has remained high. This paper proposes a multi-view ensemble learning algorithm based on empirical kernel mapping called MVE-EK, which predicts the mortality of patient through hospital records. Multi-view ensemble learning can take advantage of the consistency and complementarity of different views. The MVE-EK first divides the patient's features into multiple views and then divides the samples of each view to multiple subsets through under sampling, which can reduce the imbalance rate of the original dataset and obtain some relatively balanced subsets. Each subset is mapped into kernel space by empirical kernel mapping, which can map samples from linearly inseparable spaces to linearly separable spaces. Finally, the multi-view ensemble learning is performed by the designed loss of acquaintance between views. The effectiveness of the algorithm is verified on the three datasets of HF patient in the real world. The performance of the algorithm is better than other comparison algorithms. The datasets are collected from Shanghai Shuguang Hospital and involve 10 203 hospitalization records for 4682 HF patients between March 2009 and April 2016. The prediction information provided by the algorithm can assist the clinician in providing a more personalized treatment plan for patients with HF.

Computational complexity comparison of feedforward/radial basis function/recurrent neural network-based equalizer for a 50-Gb/s PAM4 direct-detection optical link

The computational complexity and system bit-error-rate (BER) performance of four types of neural-network-based nonlinear equalizers are analyzed for a 50-Gb/s pulse amplitude modulation (PAM)-4 direct-detection (DD) optical link. The four types are feedforward neural networks (F-NN), radial basis function neural networks (RBF-NN), auto-regressive recurrent neural networks (AR-RNN) and layer-recurrent neural networks (L-RNN). Numerical results show that, for a fixed BER threshold, the AR-RNN-based equalizers have the lowest computational complexity. Amongst all the nonlinear NN-based equalizers with the same number of inputs and hidden neurons, F-NN-based equalizers have the lowest computational complexity while the AR-RNN-based equalizers exhibit the best BER performance. Compared with F-NN or RNN, RBF-NN tends to require more hidden neurons with the increase of the number of inputs, making it not suitable for long fiber transmission distance. We also demonstrate that only a few tens of multiplications per symbol are needed for NN-based equalizers to guarantee a good BER performance. This relatively low computational complexity signifies that various NN-based equalizers can be potentially implemented in real time. More broadly, this paper provides guidelines for selecting a suitable NN-based equalizer based on BER and computational complexity requirements.

Neural Network-Based Adaptive Consensus Control for a Class of Nonaffine Nonlinear Multiagent Systems With Actuator Faults

In this paper, the consensus problem is investigated for a class of nonaffine nonlinear multiagent systems (MASs) with actuator faults of partial loss of effectiveness fault and biased fault. To deal with the control difficulty caused by the nonaffine dynamics, a neural network (NN)-based adaptive consensus protocol is developed based on the Lyapunov analysis. The neuron input of the NN uses both the state information and the consensus error information. In addition, the negative feedback term of the NN weight update law is multiplied by an absolute value of the consensus error, which is helpful in improving the consensus accuracy. With the developed adaptive NN consensus protocol, semiglobal consensus with a bounded residual consensus error of the MAS is achieved, and the bounded NN weight matrix is guaranteed. Finally, simulation results show that the developed adaptive NN consensus protocol has advantages of fast convergence rate and good consensus accuracy and has the capability of rapid response with respect to the actuator faults.

Robust Tracking of a Cost-Effective Micro-Stereolithography System Based on a Compliant Nanomanipulator

Micro-stereolithography (MSL) has emerged as a promising and challenging technique in micro-/nano-scale additive manufacturing. Besides the requirement of the light source, the motion system requires ultra-high-precision tracking capability to reach the right location for every solidification event. To achieve single-digit micron feature size of the fabrication, we propose a robust control strategy to support a self-developed cost-effective MSL prototype based on a compliant nanomanipulator and a blue light-emitting diode (LED) module. In particular, the nonlinearity and parameter-variation of the compliant manipulator are dealt with by a robust radial basis function (RBF)-based neural network, and the repetitive control (RC) is innovatively integrated with RBF to improve the tracking performance of a closed pattern. Various simulations and real-time experiments are conducted to validate the proposed control strategy. The fabrication of a closed pattern will not begin by turning on the laser source until the tracking error reaches submicrons, and the fabrication results demonstrate that the cost-effective MSL system is capable of fabricating 2.5 µm feature size in a 0.5 mm working range.

A comparative evaluation of the generalised predictive ability of eight machine learning algorithms across ten clinical metabolomics data sets for binary classification

INTRODUCTION

Metabolomics is increasingly being used in the clinical setting for disease diagnosis, prognosis and risk prediction. Machine learning algorithms are particularly important in the construction of multivariate metabolite prediction. Historically, partial least squares (PLS) regression has been the gold standard for binary classification. Nonlinear machine learning methods such as random forests (RF), kernel support vector machines (SVM) and artificial neural networks (ANN) may be more suited to modelling possible nonlinear metabolite covariance, and thus provide better predictive models.

OBJECTIVES

We hypothesise that for binary classification using metabolomics data, non-linear machine learning methods will provide superior generalised predictive ability when compared to linear alternatives, in particular when compared with the current gold standard PLS discriminant analysis.

METHODS

We compared the general predictive performance of eight archetypal machine learning algorithms across ten publicly available clinical metabolomics data sets. The algorithms were implemented in the Python programming language. All code and results have been made publicly available as Jupyter notebooks.

RESULTS

There was only marginal improvement in predictive ability for SVM and ANN over PLS across all data sets. RF performance was comparatively poor. The use of out-of-bag bootstrap confidence intervals provided a measure of uncertainty of model prediction such that the quality of metabolomics data was observed to be a bigger influence on generalised performance than model choice.

CONCLUSION

The size of the data set, and choice of performance metric, had a greater influence on generalised predictive performance than the choice of machine learning algorithm.

On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation

We propose a novel family of connectionist models based on kernel machines and consider the problem of learning layer by layer a compositional hypothesis class (i.e., a feedforward, multilayer architecture) in a supervised setting. In terms of the models, we present a principled method to "kernelize" (partly or completely) any neural network (NN). With this method, we obtain a counterpart of any given NN that is powered by kernel machines instead of neurons. In terms of learning, when learning a feedforward deep architecture in a supervised setting, one needs to train all the components simultaneously using backpropagation (BP) since there are no explicit targets for the hidden layers (Rumelhart, Hinton, & Williams, 1986). We consider without loss of generality the two-layer case and present a general framework that explicitly characterizes a target for the hidden layer that is optimal for minimizing the objective function of the network. This characterization then makes possible a purely greedy training scheme that learns one layer at a time, starting from the input layer. We provide instantiations of the abstract framework under certain architectures and objective functions. Based on these instantiations, we present a layer-wise training algorithm for an l-layer feedforward network for classification, where l≥2 can be arbitrary. This algorithm can be given an intuitive geometric interpretation that makes the learning dynamics transparent. Empirical results are provided to complement our theory. We show that the kernelized networks, trained layer-wise, compare favorably with classical kernel machines as well as other connectionist models trained by BP. We also visualize the inner workings of the greedy kernelized models to validate our claim on the transparency of the layer-wise algorithm.

Chattering Free Sliding Mode Control with Observer Based Adaptive Radial Basis Function Neural Network for Temperature Tracking in a Fixed Bed Reactor

In this paper, a Chattering Free Sliding Mode Control (CFSMC) with observer based adaptive Radial Basis Function Neural Network (RBFNN) has been designed for first-order transfer function model of temperature trajectory in a fixed bed reactor. The steady-state behavior and effect of different operating parameters such as feed velocity and temperature on the operation of the fixed bed reactor have been discussed. Due to RBFNN’s capability to map the nonlinear dynamics online through self-learning ability, it is combined with CFSMC to reduce the chattering behavior. The adaptive RBFNN has been used to approximate the nonlinear dynamic behavior of the fixed bed reactor. To predict the states of the system, high gain observer based on adaptive RBFNN has been used. Design parameter of the observer has been estimated using Hurwitz polynomial. The effect of neuron number on the mapping error and the effect of space discretization step on modeling error have also been discussed. To decrease the chattering generated by the Sliding Mode Controller (SMC) in the temperature trajectory tracking, an equivalent control term is neglected from the final controller. It has two main advantages: one is the reduction in chattering behavior which is the main drawback of SMC and the second is the reduction of the high gain requirement. The SMC is used to overcome against external disturbance, load variation, variation in key parameters and model mismatch. To make the simulation realistic, constraints have been applied to control input and input rate. For guaranteeing the system stability, Lyapunov theorem has been applied. To show the suitability of the hybrid controller, a comparison has been carried out between the hybrid and PID controller. To quantify the performance, Integral Time Weighted Absolute Error (ITAE) has been estimated. Under the condition of existing model errors and external disturbances, simulation study of the control of the fixed bed reactor shows that the hybrid control algorithm consisting of sliding mode control and observer-based adaptive RBFNN performs well both for tracking the temperature trajectory and reducing the chattering.

Objective classification of psychomotor laparoscopic skills of surgeons based on three different approaches

BACKGROUND

The determination of surgeons' psychomotor skills in minimally invasive surgery techniques is one of the major concerns of the programs of surgical training in several hospitals. Therefore, it is important to assess and classify objectively the level of experience of surgeons and residents during their training process. The aim of this study was to investigate three classification methods for establishing automatically the level of surgical competence of the surgeons based on their psychomotor laparoscopic skills.

METHODS

A total of 43 participants, divided into an experienced surgeons group with ten experts (> 100 laparoscopic procedures performed) and non-experienced surgeons group with 24 residents and nine medical students (< 10 laparoscopic procedures performed), performed three tasks in the EndoViS training system. Motion data of the instruments were captured with a video-tracking system built into the EndoViS simulator and analyzed using 13 motion analysis parameters (MAPs). Radial basis function networks (RBFNets), K-star (K*), and random forest (RF) were used for classifying surgeons based on the MAPs' scores of all participants. The performance of the three classifiers was examined using hold-out and leave-one-out validation techniques.

RESULTS

For all three tasks, the K-star method was superior in terms of accuracy and AUC in both validation techniques. The mean accuracy of the classifiers was 93.33% for K-star, 87.58% for RBFNets, and 84.85% for RF in hold-out validation, and 91.47% for K-star, 89.92% for RBFNets, and 83.72% for RF in leave-one-out cross-validation.

CONCLUSIONS

The three proposed methods demonstrated high performance in the classification of laparoscopic surgeons, according to their level of psychomotor skills. Together with motion analysis and three laparoscopic tasks of the Fundamental Laparoscopic Surgery Program, these classifiers provide a means for objectively classifying surgical competence of the surgeons for existing laparoscopic box trainers.

Radial Basis Functions Intended to Determine the Upper Bound of Absolute Dynamic Error at the Output of Voltage-Mode Accelerometers

In this paper, we propose using the radial basis functions (RBF) to determine the upper bound of absolute dynamic error (UAE) at the output of a voltage-mode accelerometer. Such functions can be obtained as a result of approximating the error values determined for the assumed-in-advance parameter variability associated with the mathematical model of an accelerometer. This approximation was carried out using the radial basis function neural network (RBF-NN) procedure for a given number of the radial neurons. The Monte Carlo (MC) method was also applied to determine the related error when considering the uncertainties associated with the parameters of an accelerometer mathematical model. The upper bound of absolute dynamic error can be a quality ratio for comparing the errors produced by different types of voltage-mode accelerometers that have the same operational frequency bandwidth. Determination of the RBFs was performed by applying the Python-related scientific packages, while the calculations related both to the UAE and the MC method were carried out using the MathCad program. Application of the RBFs represent a new approach for determining the UAE. These functions allow for the easy and quick determination of the value of such errors.

Compact Hardware Synthesis of Stochastic Spiking Neural Networks

Spiking neural networks (SNN) are able to emulate real neural behavior with high confidence due to their bio-inspired nature. Many designs have been proposed for the implementation of SNN in hardware, although the realization of high-density and biologically-inspired SNN is currently a complex challenge of high scientific and technical interest. In this work, we propose a compact digital design for the implementation of high-volume SNN that considers the intrinsic stochastic processes present in biological neurons and enables high-density hardware implementation. The proposed stochastic SNN model (SSNN) is compared with previous SSNN models, achieving a higher processing speed. We also show how the proposed model can be scaled to high-volume neural networks trained by using back propagation and applied to a pattern classification task. The proposed model achieves better results compared with other recently-published SNN models configured with unsupervised STDP learning.

Modeling and uncertainty analysis of seawater intrusion based on surrogate models

When using a simulation model to study seawater intrusion (SI), uncertainty in the parameters directly affects the results. The impact of the rise in sea levels due to global warming on SI cannot be ignored. In this paper, the Monte Carlo method is used to analyze the uncertainty in modeling SI. To reduce the computational cost of the repeated invocation of the simulation model as well as time, a surrogate model is established using a radial basis function (RBF)-based neural network method. To enhance the accuracy of the substitution model, input samples are sampled using the Latin hypercube sampling (LHS) method. The results of uncertainty analysis had a high reference value and show the following: (1) The surrogate model created using the RBF method can significantly reduce computational cost and save at least 95% of the time needed for the repeated invocation of the simulation model while maintaining high accuracy. (2) Uncertainty in the parameters and the magnitude of the rise in sea levels have a significant impact on SI. The results of prediction were thus highly uncertain. In practice, it is necessary to quantify uncertainty to provide more intuitive predictions.

Bayesian Weight Decay on Bounded Approximation for Deep Convolutional Neural Networks

This paper determines the weight decay parameter value of a deep convolutional neural network (CNN) that yields a good generalization. To obtain such a CNN in practice, numerical trials with different weight decay values are needed. However, the larger the CNN architecture is, the higher is the computational cost of the trials. To address this problem, this paper formulates an analytical solution for the decay parameter through a proposed objective function in conjunction with Bayesian probability distributions. For computational efficiency, a novel method to approximate this solution is suggested. This method uses a small amount of information in the Hessian matrix. Theoretically, the approximate solution is guaranteed by a provable bound and is obtained by a proposed algorithm, where its time complexity is linear in terms of both the depth and width of the CNN. The bound provides a consistent result for the proposed learning scheme. By reducing the computational cost of determining the decay value, the approximation allows for the fast investigation of a deep CNN (DCNN) which yields a small generalization error. Experimental results show that our assumption verified with different DCNNs is suitable for real-world image data sets. In addition, the proposed method significantly reduces the time cost of learning with setting the weight decay parameter while achieving good classification performances.

Concepts of Artificial Intelligence for Computer-Assisted Drug Discovery

Artificial intelligence (AI), and, in particular, deep learning as a subcategory of AI, provides opportunities for the discovery and development of innovative drugs. Various machine learning approaches have recently (re)emerged, some of which may be considered instances of domain-specific AI which have been successfully employed for drug discovery and design. This review provides a comprehensive portrayal of these machine learning techniques and of their applications in medicinal chemistry. After introducing the basic principles, alongside some application notes, of the various machine learning algorithms, the current state-of-the art of AI-assisted pharmaceutical discovery is discussed, including applications in structure- and ligand-based virtual screening, de novo drug design, physicochemical and pharmacokinetic property prediction, drug repurposing, and related aspects. Finally, several challenges and limitations of the current methods are summarized, with a view to potential future directions for AI-assisted drug discovery and design.

Improving clinical refractive results of cataract surgery by machine learning

AIM

To evaluate the potential of the Support Vector Machine Regression model (SVM-RM) and Multilayer Neural Network Ensemble model (MLNN-EM) to improve the intraocular lens (IOL) power calculation for clinical workflow.

BACKGROUND

Current IOL power calculation methods are limited in their accuracy with the possibility of decreased accuracy especially in eyes with an unusual ocular dimension. In case of an improperly calculated power of the IOL in cataract or refractive lens replacement surgery there is a risk of re-operation or further refractive correction. This may create potential complications and discomfort for the patient.

METHODS

A dataset containing information about 2,194 eyes was obtained using data mining process from the Electronic Health Record (EHR) system database of the Gemini Eye Clinic. The dataset was optimized and split into the selection set (used in the design for models and training), and the verification set (used in the evaluation). The set of mean prediction errors (PEs) and the distribution of predicted refractive errors were evaluated for both models and clinical results (CR).

RESULTS

Both models performed significantly better for the majority of the evaluated parameters compared with the CR. There was no significant difference between both evaluated models. In the ±0.50 D PE category both SVM-RM and MLNN-EM were slightly better than the Barrett Universal II formula, which is often presented as the most accurate calculation formula.

CONCLUSION

In comparison to the current clinical method, both SVM-RM and MLNN-EM have achieved significantly better results in IOL calculations and therefore have a strong potential to improve clinical cataract refractive outcomes.

Fast Multi-Objective Antenna Optimization Based on RBF Neural Network Surrogate Model Optimized by Improved PSO Algorithm

In this paper, a radial basis function neural network (RBFNN) surrogate model optimized by an improved particle swarm optimization (PSO) algorithm is developed to reduce the computation cost of traditional antenna design methods which rely on high-fidelity electromagnetic (EM) simulations. Considering parameters adjustment and update mechanism simultaneously, two modifications are proposed in this improved PSO. First, time-varying learning factors are designed to balance exploration and exploitation ability of particles in the search space. Second, the local best information is added to the updating process of particles except for personal and global best information for better population diversity. The improved PSO is applied to train RBFNN for determining optimal network parameters. As a result, the constructed improved PSO-RBFNN model can be used as a surrogate model for antenna performance prediction with better network generalization capability. By integrating the improved PSO-RBFNN surrogate model with multi-objective evolutionary algorithms (MOEAs), a fast multi-objective antenna optimization framework for multi-parameter antenna structures is then established. Finally, a Pareto-optimal planar miniaturized multiband antenna design is presented, demonstrating that the proposed model provides better prediction performance and considerable computational savings compared to those previously published approaches.

Cholesky Factorization Based Online Sequential Extreme Learning Machines with Persistent Regularization and Forgetting Factor

The online sequential extreme learning machine with persistent regularization and forgetting factor (OSELM-PRFF) can avoid potential singularities or ill-posed problems of online sequential regularized extreme learning machines with forgetting factors (FR-OSELM), and is particularly suitable for modelling in non-stationary environments. However, existing algorithms for OSELM-PRFF are time-consuming or unstable in certain paradigms or parameters setups. This paper presents a novel algorithm for OSELM-PRFF, named “Cholesky factorization based” OSELM-PRFF (CF-OSELM-PRFF), which recurrently constructs an equation for extreme learning machine and efficiently solves the equation via Cholesky factorization during every cycle. CF-OSELM-PRFF deals with timeliness of samples by forgetting factor, and the regularization term in its cost function works persistently. CF-OSELM-PRFF can learn data one-by-one or chunk-by-chunk with a fixed or varying chunk size. Detailed performance comparisons between CF-OSELM-PRFF and relevant approaches are carried out on several regression problems. The numerical simulation results show that CF-OSELM-PRFF demonstrates higher computational efficiency than its counterparts, and can yield stable predictions.

Extreme learning machine Cox model for high-dimensional survival analysis

Some interesting recent studies have shown that neural network models are useful alternatives in modeling survival data when the assumptions of a classical parametric or semiparametric survival model such as the Cox (1972) model are seriously violated. However, to the best of our knowledge, the plausibility of adapting the emerging extreme learning machine (ELM) algorithm for single-hidden-layer feedforward neural networks to survival analysis has not been explored. In this paper, we present a kernel ELM Cox model regularized by an L0 -based broken adaptive ridge (BAR) penalization method. Then, we demonstrate that the resulting method, referred to as ELMCoxBAR, can outperform some other state-of-art survival prediction methods such as L1 - or L2 -regularized Cox regression, random survival forest with various splitting rules, and boosted Cox model, in terms of its predictive performance using both simulated and real world datasets. In addition to its good predictive performance, we illustrate that the proposed method has a key computational advantage over the above competing methods in terms of computation time efficiency using an a real-world ultra-high-dimensional survival data.

On-line prediction of ferrous ion concentration in goethite process based on self-adjusting structure RBF neural network

Outlet ferrous ion concentration is an essential indicator to manipulate the goethite process in the zinc hydrometallurgy plant. However, it cannot be measured on-line, which leads to the delay of this feedback information. In this study, a self-adjusting structure radial basis function neural network (SAS-RBFNN) is developed to predict the outlet ferrous ion concentration on-line. First, a supervised cluster algorithm is proposed to initialize the RBFNN. Then, the network structure is adjusted by the developed self-adjusting structure mechanism. This mechanism can merge or divide the hidden neurons according to the distance of the clusters to achieve the adaptability of the RBFNN. Finally, the connection weights are determined by the gradient-based algorithm. The convergence of the SAS-RBFNN is analyzed by the Lyapunov criterion. A simulation for a benchmark problem shows the effectiveness of the proposed network. The SAS-RBFNN is then applied to predict the outlet ferrous ion concentration in the goethite process. The results demonstrate that this network can provide a more accurate prediction than the mathematical model, even under the fluctuating production condition.

Classification of Special Days in Short-Term Load Forecasting: The Spanish Case Study

Short-Term Load Forecasting is a very relevant aspect in managing, operating or participating an electric system. From system operators to energy producers and retailers knowing the electric demand in advance with high accuracy is a key feature for their business. The load series of a given system presents highly repetitive daily, weekly and yearly patterns. However, other factors like temperature or social events cause abnormalities in this otherwise periodic behavior. In order to develop an effective load forecasting system, it is necessary to understand and model these abnormalities because, in many cases, the higher forecasting error typical of these special days is linked to the larger part of the losses related to load forecasting. This paper focuses on the effect that several types of special days have on the load curve and how important it is to model these behaviors in detail. The paper analyzes the Spanish national system and it uses linear regression to model the effect that social events like holidays or festive periods have on the load curve. The results presented in this paper show that a large classification of events is needed in order to accurately model all the events that may occur in a 7-year period.

The present and future of deep learning in radiology

The advent of Deep Learning (DL) is poised to dramatically change the delivery of healthcare in the near future. Not only has DL profoundly affected the healthcare industry it has also influenced global businesses. Within a span of very few years, advances such as self-driving cars, robots performing jobs that are hazardous to human, and chat bots talking with human operators have proved that DL has already made large impact on our lives. The open source nature of DL and decreasing prices of computer hardware will further propel such changes. In healthcare, the potential is immense due to the need to automate the processes and evolve error free paradigms. The sheer quantum of DL publications in healthcare has surpassed other domains growing at a very fast pace, particular in radiology. It is therefore imperative for the radiologists to learn about DL and how it differs from other approaches of Artificial Intelligence (AI). The next generation of radiology will see a significant role of DL and will likely serve as the base for augmented radiology (AR). Better clinical judgement by AR will help in improving the quality of life and help in life saving decisions, while lowering healthcare costs. A comprehensive review of DL as well as its implications upon the healthcare is presented in this review. We had analysed 150 articles of DL in healthcare domain from PubMed, Google Scholar, and IEEE EXPLORE focused in medical imagery only. We have further examined the ethic, moral and legal issues surrounding the use of DL in medical imaging.

Optimal Control of Propagating Fronts by Using Level Set Methods and Neural Approximations

We address the optimal control of level sets associated with the solution of the normal flow equation. The problem consists in finding the normal velocity to the front described by a certain level set in such a way to minimize a given cost functional. First, the considered problem is shown to admit a solution on a suitable space of functions. Then, since in general it is difficult to solve it analytically, an approximation scheme that relies on the extended Ritz method is proposed to find suboptimal solutions. Specifically, the control law is forced to take on a neural structure depending nonlinearly on a finite number of parameters to be tuned, i.e., the neural weights. The selection of the optimal weights is performed with two different approaches. The first one employs classical line-search descent methods, while the second one is based on a quasi-Newton optimization that can be regarded as neural learning based on the extended Kalman filter. Compared with line-search methods, such an approach reveals to be successful with a reduced computational effort and an increased robustness with respect to the trapping into local minima, as confirmed by simulations in both two and three dimensions.

Local Kernels that Approximate Bayesian Regularization and Proximal Operators

In this work, we broadly connect kernel-based filtering (e.g. approaches such as the bilateral filter and nonlocal means, but also many more) with general variational formulations of Bayesian regularized least squares, and the related concept of proximal operators. Variational/Bayesian/proximal formulations often result in optimization problems that do not have closed-form solutions, and therefore typically require global iterative solutions. Our main contribution here is to establish how one can approximate the solution of the resulting global optimization problems using locally adaptive filters with specific kernels. Our results are valid for small regularization strength (i.e. weak noise) but the approach is powerful enough to be useful for a wide range of applications because we expose how to derive a "kernelized" solution to these problems that approximates the global solution in one shot, using only local operations. As another side benefit in the reverse direction, given a local data-adaptive filter constructed with a particular choice of kernel, we enable the interpretation of such filters in the variational/Bayesian/proximal framework.