A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression

Ensemble Deep Random Vector Functional Link Network Using Privileged Information for Alzheimer's Disease Diagnosis

Alzheimer's disease (AD) is a progressive brain disorder. Machine learning models have been proposed for the diagnosis of AD at early stage. Recently, deep learning architectures have received quite a lot attention. Most of the deep learning architectures suffer from the issues of local minima, slow convergence and sensitivity to learning rate. To overcome these issues, non-iterative learning based deep randomized models especially random vector functional link network (RVFL) with direct links have proven to be successful. However, deep RVFL and its ensemble models are trained only on normal samples. In this paper, deep RVFL and its ensembles are enabled to incorporate privileged information, as the standard RVFL model and its deep models are unable to use privileged information. To fill this gap, we have incorporated learning using privileged information (LUPI) in deep RVFL model, and propose deep RVFL with LUPI framework (dRVFL+). Privileged information is available while training the models. As RVFL is an unstable classifier, we propose ensemble deep RVFL+ with LUPI framework (edRVFL+) which exploits the LUPI as well as the diversity among the base leaners for better classification. Unlike traditional ensemble approach wherein multiple base learners are trained, the proposed edRVFL+ model optimises a single network and generates an ensemble via optimization at different levels of random projections of the data. Both dRVFL+ and edRVFL+ efficiently utilise the privileged information which results in better generalization performance. In LUPI framework, half of the available features are used as normal features and rest as the privileged features. However, we propose a novel approach for generating the privileged information. We utilise different activation functions while processing the normal and privileged information in the proposed deep architectures. To the best of our knowledge, this is first time that a separate privileged information is generated. The proposed dRVFL+ and edRVFL+ models are employed for the diagnosis of Alzheimer's disease. Experimental results demonstrate the superiority of the proposed dRVFL+ and edRVFL+ models over baseline models. Thus, the proposed edRVFL+ model can be utilised in clinical setting for the diagnosis of AD.

Assessing energy consumption and economic growth interrelations in Asia-Pacific: A multivariate approach with panel FMOLS and bootstrap Granger causality tests

This study investigates the cointegration and causal relationship between energy consumption and economic growth using data from 16 Asian and Pacific countries from 1970 to 2010. The expanded production function is used in this investigation; this function considers not only labor but also financial resources. This study investigates whether or not a rise in energy demand is associated with a healthy economy. Human capital, in addition to material and labor resources, is taken into account by this operation. One of the first studies to adopt a multivariate method and add human capital was undertaken on the energy-growth nexus. Using the panel unit root and cointegration tests, this study confirms the existence of a long-run cointegrating connection between these variables. These studies recognize the presence of cross-sectional interdependence, according to specific reports. The significance of considering the interconnection of various countries is confirmed by comparing estimates from panel heterogeneous fully modified ordinary least squares (FMOLS) models with those from unceasingly efficient and fully modified models. Nonetheless, the bootstrap panel Granger causality test findings demonstrate that economic growth is a causal factor in rising energy consumption in the region, indicating that the relationship is not constant across countries.

A regularized orthogonal activated inverse-learning neural network for regression and classification with outliers

A novel regularized orthogonal activated inverse-learning (ROAIL) neural network is proposed and investigated for reducing the impact of outliers in regression and classification fields. The proposed ROAIL network does not require extensive iterative computations. Instead, it can achieve the desired results with a single step of computation, allowing for the efficient acquisition of network weights. By extending the Gegenbauer polynomials to a multi-variate version, and integrating the ℓ2 regularization and Welsch loss function into the orthogonal activated inverse-learning framework, two forms of ROAIL are obtained, i.e., ℓ2 norm ROAIL (ℓ2-ROAIL) and Welsch-ROAIL (W-ROAIL). ℓ2-ROAIL neural network is proposed to minimize the empirical and structural risk simultaneously since taking the structural risk as a part of loss function can effectively reduce the complexity of the model and thus improve the generalization ability. W-ROAIL neural network further improves the robustness of the ℓ2-ROAIL neural network by replacing the original two-norm in loss function with Welsch function. The Welsch function can determine the weights of each sample according to its output error, and influence of outliers could be weakened since the weights of outliers would be reduced. Both regression and classification experiments show that W-ROAIL neural network has strong ability to suppress the influence of outliers.

Online dynamic ensemble deep random vector functional link neural network for forecasting

This paper proposes a three-stage online deep learning model for time series based on the ensemble deep random vector functional link (edRVFL). The edRVFL stacks multiple randomized layers to enhance the single-layer RVFL's representation ability. Each hidden layer's representation is utilized for training an output layer, and the ensemble of all output layers forms the edRVFL's output. However, the original edRVFL is not designed for online learning, and the randomized nature of the features is harmful to extracting meaningful temporal features. In order to address the limitations and extend the edRVFL to an online learning mode, this paper proposes a dynamic edRVFL consisting of three online components, the online decomposition, the online training, and the online dynamic ensemble. First, an online decomposition is utilized as a feature engineering block for the edRVFL. Then, an online learning algorithm is designed to learn the edRVFL. Finally, an online dynamic ensemble method, which can measure the change in the distribution, is proposed for aggregating all layers' outputs. This paper evaluates and compares the proposed model with state-of-the-art methods on sixteen time series.

Two-step approach based multi-objective groundwater remediation using enhanced random vector functional link integrated with evolutionary marine predator algorithm

We here propose a two-step approach-based simulation-optimization model for multi-objective groundwater remediation using enhanced random vector functional link (ERVFL) and evolutionary marine predator algorithm (EMPA). In this study, groundwater flow and solute transport models are developed using MODFLOW and MT3DMS. The ERVFL network is used to approximate the flow and transport models, enhancing the computational performance. This study also improves the robustness of the ERVFL network using a kernel density estimator (KDE) based weighted least square approach. We further develop the EMPA by modifying the marine predator algorithm (MPA) using elite opposition-based learning, biological evolution operators, and elimination mechanisms. In the multi-objective version of EMPA, the non-dominated/Pareto-optimal solutions are stored in an external repository using an archive controller and adaptive grid mechanism to promote better convergence and diversity of the Pareto front. The proposed methodologies are applied for multi-objective groundwater remediation of a hypothetical unconfined aquifer based on the two-step method. The first step directly integrates flow and transport models with EMPA and finds the optimal locations of pumping wells by minimizing the percent of contaminant mass remaining in the aquifer. In the second step, the ERVL-based proxy model is integrated with EMPA and used for multi-objective optimization while explicitly using the pumping well locations obtained in the first step. The multi-objective optimization generates a Pareto-optimal solution representing the relationship between the rate of pumping and the amount of contaminant mass in the aquifer. Further analyses show a significant advantage of the two-step approach over a traditional method for multi-objective groundwater remediation.

Theoretical bounds of generalization error for generalized extreme learning machine and random vector functional link network

Ensuring the prediction accuracy of a learning algorithm on a theoretical basis is crucial and necessary for building the reliability of the learning algorithm. This paper analyzes prediction error obtained through the least square estimation in the generalized extreme learning machine (GELM), which applies the limiting behavior of the Moore-Penrose generalized inverse (M-P GI) to the output matrix of ELM. ELM is the random vector functional link (RVFL) network without direct input to output links Specifically, we analyze tail probabilities associated with upper and lower bounds to the error expressed by norms. The analysis employs the concepts of the L₂ norm, the Frobenius norm, the stable rank, and the M-P GI. The coverage of theoretical analysis extends to the RVFL network. In addition, a criterion for more precise bounds of prediction errors that may give stochastically better network environments is provided. The analysis is applied to simple examples and large-size datasets to illustrate the procedure and verify the analysis and execution speed with big data. Based on this study, we can immediately obtain the upper and lower bounds of prediction errors and their associated tail probabilities through matrices calculations appearing in the GELM and RVFL. This analysis provides criteria for the reliability of the learning performance of a network in real-time and for network structure that enables obtaining better performance reliability. This analysis can be applied in various areas where the ELM and RVFL are adopted. The proposed analytical method will guide the theoretical analysis of errors occurring in DNNs, which employ a gradient descent algorithm.

SAFNet: A deep spatial attention network with classifier fusion for breast cancer detection

Breast cancer is a top dangerous killer for women. An accurate early diagnosis of breast cancer is the primary step for treatment. A novel breast cancer detection model called SAFNet is proposed based on ultrasound images and deep learning. We employ a pre-trained ResNet-18 embedded with the spatial attention mechanism as the backbone model. Three randomized network models are trained for prediction in the SAFNet, which are fused by majority voting to produce more accurate results. A public ultrasound image dataset is utilized to evaluate the generalization ability of our SAFNet using 5-fold cross-validation. The simulation experiments reveal that the SAFNet can produce higher classification results compared with four existing breast cancer classification methods. Therefore, our SAFNet is an accurate tool to detect breast cancer that can be applied in clinical diagnosis.

Random vector functional link with ε-insensitive Huber loss function for biomedical data classification

BACKGROUND AND OBJECTIVE

Biomedical data classification has been a trending topic among researchers during the last decade. Biomedical datasets may contain several features noises. Hence, the conventional machine learning model cannot efficiently handle the presence of noise in datasets. Among the several machine learning model, the random vector functional link (RVFL) is one of the most popular and efficient models for task related to both classification and regression. Despite its excellent classification performance, its performance degrades while dealing with the datasets with noise. Researchers are searching for powerful models to minimize the influence of noise in datasets. Therefore, to enhance the classification ability of RVFL on noisy datasets, this paper suggests a novel random vector functional link with ε-insensitive Huber loss function (ε-HRVFL) for biomedical data classification problems.

METHODS

The optimization problem of ε-HRVFL is reformulated as strongly convex minimization problems with a simple function iterative approach to find solutions. To have a better understanding of the scope of the biomedical data classification problem and potential solutions, we conducted experiments with three different types of label noise in biomedical datasets as well as a few non-biomedical datasets. The classification accuracy of the proposed ε-HRVFL model is compared statistically using Friedman test with the support vector machine, extreme learning machine with radial basis function (RBF) and sigmoid activation functions and RVFL with RBF and sigmoid activation functions.

RESULTS

For non-biomedical datasets, the proposed model showed the highest accuracy of 98.1332%. Moreover, for the biomedical datasets, the proposed model showed the best accuracy of 96.5229%. The proposed ε-HRVFL model with sigmoid activation function reveals the best mean ranks among the reported classifiers for both, biomedical and non-biomedical datasets.

CONCLUSION

Numerical results show the applicability of the proposed ε-HRVFL model. In future, the proposed ε-HRVFL can be developed to solve multiclass biomedical data classification problems. Moreover, ε-insensitive asymmetric Huber loss function based RVFL model can be developed for dealing more efficiently with these noisy biomedical datasets.

A Novel Hybrid NN-ABPE-Based Calibration Method for Improving Accuracy of Lateration Positioning System

Positioning systems based on the lateration method utilize distance measurements and the knowledge of the location of the beacons to estimate the position of the target object. Although most of the global positioning techniques rely on beacons whose locations are known a priori, miscellaneous factors and disturbances such as obstacles, reflections, signal propagation speed, the orientation of antennas, measurement offsets of the beacons hardware, electromagnetic noise, or delays can affect the measurement accuracy. In this paper, we propose a novel hybrid calibration method based on Neural Networks (NN) and Apparent Beacon Position Estimation (ABPE) to improve the accuracy of a lateration positioning system. The main idea of the proposed method is to use a two-step position correction pipeline that first performs the ABPE step to estimate the perceived positions of the beacons that are used in the standard position estimation algorithm and then corrects these initial estimates by filtering them with a multi-layer feed-forward neural network in the second step. In order to find an optimal neural network, 16 NN architectures with 10 learning algorithms and 12 different activation functions for hidden layers were implemented and tested in the MATLAB environment. The best training outcomes for NNs were then employed in two real-world indoor scenarios: without and with obstacles. With the aim to validate the proposed methodology in a scenario where a fast set-up of the system is desired, we tested eight different uniform sampling patterns to establish the influence of the number of the training samples on the accuracy of the system. The experimental results show that the proposed hybrid NN-ABPE method can achieve a high level of accuracy even in scenarios when a small number of calibration reference points are measured.

A novel two-step approach for optimal groundwater remediation by coupling extreme learning machine with evolutionary hunting strategy based metaheuristics

We propose a simulation-optimization (SO) model based on a novel two-step strategy for the optimal design of groundwater remediation systems. The SO models are developed by coupling simulation models directly or through the extreme learning machine (ELM) with evolutionary hunting strategy based metaheuristics (EHSMs). In the first step, EHSMs with a combinatorial optimization technique are used to obtain optimal pumping locations by minimizing the percentage of contaminant mass that remained in the aquifer while keeping the pumping strategy as constant. In the second step, the optimal pumping locations are directly used as input, and a composite function is employed to minimize the sum of the water extraction rates and the percentage of extracted contaminant mass by constraining hydraulic heads and contaminant concentrations. The performance of the two-step strategy is found to be slightly better and computationally more efficient than the alternate approach. Moreover, various statistical measures suggest the superiority of EHSMs over other metaheuristics for groundwater remediation.

Extreme learning machine versus classical feedforward network

Our research is devoted to answering whether randomisation-based learning can be fully competitive with the classical feedforward neural networks trained using backpropagation algorithm for classification and regression tasks. We chose extreme learning as an example of randomisation-based networks. The models were evaluated in reference to training time and achieved efficiency. We conducted an extensive comparison of these two methods for various tasks in two scenarios: $$\bullet$$ ∙ using comparable network capacity and $$\bullet$$ ∙ using network architectures tuned for each model. The comparison was conducted on multiple datasets from public repositories and some artificial datasets created for this research. Overall, the experiments covered more than 50 datasets. Suitable statistical tests supported the results. They confirm that for relatively small datasets, extreme learning machines (ELM) are better than networks trained by the backpropagation algorithm. But for demanding image datasets, like ImageNet, ELM is not competitive to modern networks trained by backpropagation; therefore, in order to properly address current practical needs in pattern recognition entirely, ELM needs further development. Based on our experience, we postulate to develop smart algorithms for the inverse matrix calculation, so that determining weights for challenging datasets becomes feasible and memory efficient. There is a need to create specific mechanisms to avoid keeping the whole dataset in memory to compute weights. These are the most problematic elements in ELM processing, establishing the main obstacle in the widespread ELM application.

Analyzing and Improving the Performance of a Particulate Matter Low Cost Air Quality Monitoring Device

Air quality (AQ) in urban areas is deteriorating, thus having negative effects on people’s everyday lives. Official air quality monitoring stations provide the most reliable information, but do not always depict air pollution levels at scales reflecting human activities. They also have a high cost and therefore are limited in number. This issue can be addressed by deploying low cost AQ monitoring devices (LCAQMD), though their measurements are of far lower quality. In this paper we study the correlation of air pollution levels reported by such a device and by a reference station for particulate matter, ozone and nitrogen dioxide in Thessaloniki, Greece. On this basis, a corrective factor is modeled via seven machine learning algorithms in order to improve the quality of measurements for the LCAQMD against reference stations, thus leading to its on-field computational improvement. We show that our computational intelligence approach can improve the performance of such a device for PM10 under operational conditions.

Investigation of esthetic evaluation and its influencing factors for a tunnel portal based on dynamic vision

With the development of modern cities, roads, and landscapes, it is becoming increasingly important for infrastructure such as tunnels to provide an esthetically pleasing experience. In this respect, it is necessary to conduct studies that consider the esthetic design of tunnel portals using esthetics research. Regarding the esthetic evaluation of tunnel portals, this paper fully considers the dynamic visual effect from the driver’s perspective. This study combines the use of Blender, SpeedTree Modeler Cinema, Adobe Photoshop CS6, and other software for secondary development. These programs are connected to the driving simulation platform Euro Truck Simulator 2 (which is equipped with a driving simulator) to construct a set of driving simulation tests that enable the esthetic evaluation of a tunnel portal. The Banlun Tunnel on the Funing–Longliu Expressway in Yunnan Province, China, is used as a case study, and four impact factors that vary significantly in esthetic design are included: the linearity, color, greening and texture of the portal. Using an orthogonal experimental design, the influence of the esthetic degree was simulated and evaluated, and the order of sensitivity to esthetic factors of a headwall tunnel portal was sequentially determined as follows: the portal texture exerts the maximum impact on the beauty degree of the headwall portal, followed by the portal greening and the portal color, while the portal linearity exerts the minimum impact. The results show that the developed driving simulation test system can be used to determine the sensitivity of esthetic factors for a tunnel portal and obtain an optimal collocation of esthetic factors on different levels; hence, it provides feedback for use in designing the optimum esthetic tunnel portal. This test system can be used as a reference when conducting future evaluations and studies on tunnel portal esthetics.