A complete expected improvement criterion for Gaussian process assisted highly constrained expensive optimization

Evolving Dual-Threshold Bienenstock-Cooper-Munro Learning Rules in Echo State Networks

The strengthening and the weakening of synaptic strength in existing Bienenstock-Cooper-Munro (BCM) learning rule are determined by a long-term potentiation (LTP) sliding modification threshold and the afferent synaptic activities. However, synaptic long-term depression (LTD) even affects low-active synapses during the induction of synaptic plasticity, which may lead to information loss. Biological experiments have found another LTD threshold that can induce either potentiation or depression or no change, even at the activated synapses. In addition, existing BCM learning rules can only select a set of fixed rule parameters, which is biologically implausible and practically inflexible to learn the structural information of input signals. In this article, an evolved dual-threshold BCM learning rule is proposed to regulate the reservoir internal connection weights of the echo-state-network (ESN), which can contribute to alleviating information loss and enhancing learning performance by introducing different optimal LTD thresholds for different postsynaptic neurons. Our experimental results show that the evolved dual-threshold BCM learning rule can result in the synergistic learning of different plasticity rules, effectively improving the learning performance of an ESN in comparison with existing neural plasticity learning rules and some state-of-the-art ESN variants on three widely used benchmark tasks and the prediction of an esterification process.

On Nonstationary Gaussian Process Model for Solving Data-Driven Optimization Problems

In data-driven evolutionary optimization, most existing Gaussian processes (GPs)-assisted evolutionary algorithms (EAs) adopt stationary GPs (SGPs) as surrogate models, which might be insufficient for solving most optimization problems. This article finds that GPs in the optimization problems are nonstationary with great probability. We propose to employ a nonstationary GP (NSGP) surrogate model for data-driven evolutionary optimization, where the mean of the NSGP is allowed to vary with the decision variables, while its residue variance follows an SGP. In this article, the nonstationarity of GPs in the tested functions is theoretically analyzed. In addition, this article constructs an NSGP where the SGP is a degenerate case. Performance comparisons of the NSGP with the SGP and the NSGP-assisted EA (NSGP-MAEA) with the SGP-assisted EA (SGP-MAEA) are carried out on a set of benchmark problems and an antenna design problem. These comparison results demonstrate the competitiveness of the NSGP model.

Data-driven Harris Hawks constrained optimization for computationally expensive constrained problems

Aiming at the constrained optimization problem where function evaluation is time-consuming, this paper proposed a novel algorithm called data-driven Harris Hawks constrained optimization (DHHCO). In DHHCO, Kriging models are utilized to prospect potentially optimal areas by leveraging computationally expensive historical data during optimization. Three powerful strategies are, respectively, embedded into different phases of conventional Harris Hawks optimization (HHO) to generate diverse candidate sample data for exploiting around the existing sample data and exploring uncharted region. Moreover, a Kriging-based data-driven strategy composed of data-driven population construction and individual selection strategy is presented, which fully mines and utilizes the potential available information in the existing sample data. DHHCO inherits and develops HHO's offspring updating mechanism, and meanwhile exerts the prediction ability of Kriging, reduces the number of expensive function evaluations, and provides new ideas for data-driven constraint optimization. Comprehensive experiments have been conducted on 13 benchmark functions and a real-world expensive optimization problem. The experimental results suggest that the proposed DHHCO can achieve quite competitive performance compared with six representative algorithms and can find the near global optimum with 200 function evaluations for most examples. Moreover, DHHCO is applied to the structural optimization of the internal components of the real underwater vehicle, and the final satisfactory weight reduction effect is more than 18%.

Globalized simulation-driven miniaturization of microwave circuits by means of dimensionality-reduced constrained surrogates

Small size has become a crucial prerequisite in the design of modern microwave components. Miniaturized devices are essential for a number of application areas, including wireless communications, 5G/6G technology, wearable devices, or the internet of things. Notwithstanding, size reduction generally degrades the electrical performance of microwave systems. Therefore, trade-off solutions have to be sought that represent acceptable compromises between the ability to meet the design targets and physical compactness. From an optimization perspective, this poses a constrained task, which is computationally expensive because a reliable evaluation of microwave components has to rely on full-wave electromagnetic analysis. Furthermore, due to its constrained nature, size reduction is a multimodal problem, i.e., the results are highly dependent on the initial design. Thus, utilization of global search algorithms is advisable in principle, yet, often undoable in practice because of the associated computational expenses, especially when using nature-inspired procedures. This paper introduces a novel technique for globalized miniaturization of microwave components. Our technique starts by identifying the feasible region boundary, and by constructing a dimensionality-reduced surrogate model therein. Global optimization of the metamodel is followed by EM-driven local tuning. Application of the domain-confined surrogate ensures low cost of the entire procedure, further reduced by the incorporation of variable-fidelity EM simulations. Our framework is validated using two microstrip couplers, and compared to nature-inspired optimization, as well as gradient-based size reduction. The results indicate superior miniaturization rates and low running cost, which make the presented algorithm a potential candidate for efficient simulation-based design of compact structures.

Learning to Ask: Conversational Product Search via Representation Learning

Online shopping platforms, such as Amazon and AliExpress, are increasingly prevalent in the society, helping customers purchase products conveniently. With recent progress on natural language processing, researchers and practitioners shift their focus from traditional product search to conversational product search. Conversational product search enables user-machine conversations and through them collects explicit user feedback that allows to actively clarify the users’ product preferences. Therefore, prospective research on an intelligent shopping assistant via conversations is indispensable. Existing publications on conversational product search either model conversations independently from users, queries, and products or lead to a vocabulary mismatch. In this work, we propose a new conversational product search model, ConvPS, to assist users to locate desirable items. The model is first trained to jointly learn the semantic representations of user, query, item, and conversation via a unified generative framework. After learning these representations, they are integrated to retrieve the target items in the latent semantic space. Meanwhile, we propose a set of greedy and explore-exploit strategies to learn to ask the user a sequence of high-performance questions for conversations. Our proposed ConvPS model can naturally integrate the representation learning of the user, query, item, and conversation into a unified generative framework, which provides a promising avenue for constructing accurate and robust conversational product search systems that are flexible and adaptive. Experimental results demonstrate that our ConvPS model significantly outperforms state-of-the-art baselines.

A Comparative Study of Infill Sampling Criteria for Computationally Expensive Constrained Optimization Problems

Engineering optimization problems often involve computationally expensive black-box simulations of underlying physical phenomena. This paper compares the performance of four constrained optimization algorithms relying on a Gaussian process model and an infill sampling criterion under the framework of Bayesian optimization. The four infill sampling criteria include expected feasible improvement (EFI), constrained expected improvement (CEI), stepwise uncertainty reduction (SUR), and augmented Lagrangian (AL). Numerical tests were rigorously performed on a benchmark set consisting of nine constrained optimization problems with features commonly found in engineering, as well as a constrained structural engineering design optimization problem. Based upon several measures including statistical analysis, our results suggest that, overall, the EFI and CEI algorithms are significantly more efficient and robust than the other two methods, in the sense of providing the most improvement within a very limited number of objective and constraint function evaluations, and also in the number of trials for which a feasible solution could be located.

An Entropy Weight-Based Lower Confidence Bounding Optimization Approach for Engineering Product Design

The optimization design of engineering products involving computationally expensive simulation is usually a time-consuming or even prohibitive process. As a promising way to relieve computational burden, adaptive Kriging-based design optimization (AKBDO) methods have been widely adopted due to their excellent ability for global optimization under limited computational resource. In this paper, an entropy weight-based lower confidence bounding approach (EW-LCB) is developed to objectively make a trade-off between the global exploration and the local exploitation in the adaptive optimization process. In EW-LCB, entropy theory is used to measure the degree of the variation of the predicted value and variance of the Kriging model, respectively. Then, an entropy weight function is proposed to allocate the weights of exploration and exploitation objectively and adaptively based on the values of information entropy. Besides, an index factor is defined to avoid the sequential process falling into the local regions, which is associated with the frequencies of the current optimal solution. To demonstrate the effectiveness of the proposed EW- LCB method, several numerical examples with different dimensions and complexities and the lightweight optimization design problem of an underwater vehicle base are utilized. Results show that the proposed approach is competitive compared with state-of-the-art AKBDO methods considering accuracy, efficiency, and robustness.