Parallel asynchronous particle swarm optimization

System-Specific Parameter Optimization for Nonpolarizable and Polarizable Force Fields

The accuracy of classical force fields (FFs) has been shown to be limited for the simulation of cation-protein systems despite their importance in understanding the processes of life. Improvements can result from optimizing the parameters of classical FFs or by extending the FF formulation by terms describing charge transfer (CT) and polarization (POL) effects. In this work, we introduce our implementation of the CTPOL model in OpenMM, which extends the classical additive FF formula by adding CT and POL. Furthermore, we present an open-source parametrization tool, called FFAFFURR, that enables the (system-specific) parametrization of OPLS-AA and CTPOL models. The performance of our workflow was evaluated by its ability to reproduce quantum chemistry energies and by molecular dynamics simulations of a zinc-finger protein.

Adaptive Filtering: Issues, Challenges, and Best-Fit Solutions Using Particle Swarm Optimization Variants

Adaptive equalization is crucial in mitigating distortions and compensating for frequency response variations in communication systems. It aims to enhance signal quality by adjusting the characteristics of the received signal. Particle swarm optimization (PSO) algorithms have shown promise in optimizing the tap weights of the equalizer. However, there is a need to enhance the optimization capabilities of PSO further to improve the equalization performance. This paper provides a comprehensive study of the issues and challenges of adaptive filtering by comparing different variants of PSO and analyzing the performance by combining PSO with other optimization algorithms to achieve better convergence, accuracy, and adaptability. Traditional PSO algorithms often suffer from high computational complexity and slow convergence rates, limiting their effectiveness in solving complex optimization problems. To address these limitations, this paper proposes a set of techniques aimed at reducing the complexity and accelerating the convergence of PSO.

Toward an Ideal Particle Swarm Optimizer for Multidimensional Functions

The Particle Swarm Optimization (PSO) method is a global optimization technique based on the gradual evolution of a population of solutions called particles. The method evolves the particles based on both the best position of each of them in the past and the best position of the whole. Due to its simplicity, the method has found application in many scientific areas, and for this reason, during the last few years, many modifications have been presented. This paper introduces three modifications to the method that aim to reduce the required number of function calls while maintaining the accuracy of the method in locating the global minimum. These modifications affect important components of the method, such as how fast the particles change or even how the method is terminated. The above modifications were tested on a number of known universal optimization problems from the relevant literature, and the results were compared with similar techniques.

Virtual Populations for Quantitative Systems Pharmacology Models

Quantitative systems pharmacology (QSP) places an emphasis on dynamic systems modeling, incorporating considerations from systems biology modeling and pharmacodynamics. The goal of QSP is often to quantitatively predict the effects of clinical therapeutics, their combinations, and their doses on clinical biomarkers and endpoints. In order to achieve this goal, strategies for incorporating clinical data into model calibration are critical. Virtual population (VPop) approaches facilitate model calibration while faced with challenges encountered in QSP model application, including modeling a breadth of clinical therapies, biomarkers, endpoints, utilizing data of varying structure and source, capturing observed clinical variability, and simulating with models that may require more substantial computational time and resources than often found in pharmacometrics applications. VPops are frequently developed in a process that may involve parameterization of isolated pathway models, integration into a larger QSP model, incorporation of clinical data, calibration, and quantitative validation that the model with the accompanying, calibrated VPop is suitable to address the intended question or help with the intended decision. Here, we introduce previous strategies for developing VPops in the context of a variety of therapeutic and safety areas: metabolic disorders, drug-induced liver injury, autoimmune diseases, and cancer. We introduce methodological considerations, prior work for sensitivity analysis and VPop algorithm design, and potential areas for future advancement. Finally, we give a more detailed application example of a VPop calibration algorithm that illustrates recent progress and many of the methodological considerations. In conclusion, although methodologies have varied, VPop strategies have been successfully applied to give valid clinical insights and predictions with the assistance of carefully defined and designed calibration and validation strategies. While a uniform VPop approach for all potential QSP applications may be challenging given the heterogeneity in use considerations, we anticipate continued innovation will help to drive VPop application for more challenging cases of greater scale while developing new rigorous methodologies and metrics.

Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation

Federated learning obtains a central model on the server by aggregating models trained locally on clients. As a result, federated learning does not require clients to upload their data to the server, thereby preserving the data privacy of the clients. One challenge in federated learning is to reduce the client-server communication since the end devices typically have very limited communication bandwidth. This article presents an enhanced federated learning technique by proposing an asynchronous learning strategy on the clients and a temporally weighted aggregation of the local models on the server. In the asynchronous learning strategy, different layers of the deep neural networks (DNNs) are categorized into shallow and deep layers, and the parameters of the deep layers are updated less frequently than those of the shallow layers. Furthermore, a temporally weighted aggregation strategy is introduced on the server to make use of the previously trained local models, thereby enhancing the accuracy and convergence of the central model. The proposed algorithm is empirically on two data sets with different DNNs. Our results demonstrate that the proposed asynchronous federated deep learning outperforms the baseline algorithm both in terms of communication cost and model accuracy.

Particle Swarm Optimisation: A Historical Review Up to the Current Developments

The Particle Swarm Optimisation (PSO) algorithm was inspired by the social and biological behaviour of bird flocks searching for food sources. In this nature-based algorithm, individuals are referred to as particles and fly through the search space seeking for the global best position that minimises (or maximises) a given problem. Today, PSO is one of the most well-known and widely used swarm intelligence algorithms and metaheuristic techniques, because of its simplicity and ability to be used in a wide range of applications. However, in-depth studies of the algorithm have led to the detection and identification of a number of problems with it, especially convergence problems and performance issues. Consequently, a myriad of variants, enhancements and extensions to the original version of the algorithm, developed and introduced in the mid-1990s, have been proposed, especially in the last two decades. In this article, a systematic literature review about those variants and improvements is made, which also covers the hybridisation and parallelisation of the algorithm and its extensions to other classes of optimisation problems, taking into consideration the most important ones. These approaches and improvements are appropriately summarised, organised and presented, in order to allow and facilitate the identification of the most appropriate PSO variant for a particular application.

The active selfish herd

The selfish herd hypothesis provides an explanation for group aggregation via the selfish avoidance of predators. Conceptually, and as was first proposed, this movement should aim to minimise the danger domain of each individual. Whilst many reasonable proxies have been proposed, none have directly sought to reduce the danger domain. In this work we present a two dimensional stochastic model that actively optimises these domains. The individuals' dynamics are determined by sampling the space surrounding them and moving to achieve the largest possible domain reduction. Two variants of this idea are investigated with sampling occurring either locally or globally. We simulate our models and two of the previously proposed benchmark selfish herd models: k-nearest neighbours (kNN); and local crowded horizon (LCH). The resulting positions are analysed to determine the benefit to the individual and the group's ability to form a compact group. To do this, the group level metric of packing fraction and individual level metric of domain size are observed over time for a range of noise levels. With these measures we show a clear stratification of the four models when noise is not included. kNN never resulted in centrally compacted herd, while the local active selfish model and LCH did so with varying levels of success. The most centralised groups were achieved with our global active selfish herd model. The inclusion of noise improved aggregation in all models. This was particularly so with the local active selfish model with a change to ordering of performance so that it marginally outperformed LCH in aggregation. By more closely following Hamilton's original conception and aligning the individual's goal of a reduced danger domain with the movement it makes increased cohesion is observed, thus confirming his hypothesis, however, these findings are dependent on noise. Moreover, many features originally conjectured by Hamilton are also observed in our simulations.

Resource Allocation in the Cognitive Radio Network-Aided Internet of Things for the Cyber-Physical-Social System: An Efficient Jaya Algorithm

Currently, there is a growing demand for the use of communication network bandwidth for the Internet of Things (IoT) within the cyber-physical-social system (CPSS), while needing progressively more powerful technologies for using scarce spectrum resources. Then, cognitive radio networks (CRNs) as one of those important solutions mentioned above, are used to achieve IoT effectively. Generally, dynamic resource allocation plays a crucial role in the design of CRN-aided IoT systems. Aiming at this issue, orthogonal frequency division multiplexing (OFDM) has been identified as one of the successful technologies, which works with a multi-carrier parallel radio transmission strategy. In this article, through the use of swarm intelligence paradigm, a solution approach is accordingly proposed by employing an efficient Jaya algorithm, called PA-Jaya, to deal with the power allocation problem in cognitive OFDM radio networks for IoT. Because of the algorithm-specific parameter-free feature in the proposed PA-Jaya algorithm, a satisfactory computational performance could be achieved in the handling of this problem. For this optimization problem with some constraints, the simulation results show that compared with some popular algorithms, the efficiency of spectrum utilization could be further improved by using PA-Jaya algorithm with faster convergence speed, while maximizing the total transmission rate.

Enhanced Particle Swarm Optimization Algorithm: Efficient Training of ReaxFF Reactive Force Fields

Particle swarm optimization (PSO) is a powerful metaheuristic population-based global optimization algorithm. However, when it is applied to nonseparable objective functions, its performance on multimodal landscapes is significantly degraded. Here we show that a significant improvement in the search quality and efficiency on multimodal functions can be achieved by enhancing the basic rotation-invariant PSO algorithm with isotropic Gaussian mutation operators. The new algorithm demonstrates superior performance across several nonlinear, multimodal benchmark functions compared with the rotation-invariant PSO algorithm and the well-established simulated annealing and sequential one-parameter parabolic interpolation methods. A search for the optimal set of parameters for the dispersion interaction model in the ReaxFF- lg reactive force field was carried out with respect to accurate DFT-TS calculations. The resulting optimized force field accurately describes the equations of state of several high-energy molecular crystals where such interactions are of crucial importance. The improved algorithm also presents better performance compared to a genetic algorithm optimization method in the optimization of the parameters of a ReaxFF- lg correction model. The computational framework is implemented in a stand-alone C++ code that allows the straightforward development of ReaxFF reactive force fields.

Heterogeneous computing for epidemiological model fitting and simulation

Background

Over the last years, substantial effort has been put into enhancing our arsenal in fighting epidemics from both technological and theoretical perspectives with scientists from different fields teaming up for rapid assessment of potentially urgent situations. This paper focusses on the computational aspects of infectious disease models and applies commonly available graphics processing units (GPUs) for the simulation of these models. However, fully utilizing the resources of both CPUs and GPUs requires a carefully balanced heterogeneous approach.

Results

The contribution of this paper is twofold. First, an efficient GPU implementation for evaluating a small-scale ODE model; here, the basic S(usceptible)-I(nfected)-R(ecovered) model, is discussed. Second, an asynchronous particle swarm optimization (PSO) implementation is proposed where batches of particles are sent asynchronously from the host (CPU) to the GPU for evaluation. The ultimate goal is to infer model parameters that enable the model to correctly describe observed data. The particles of the PSO algorithm are candidate parameters of the model; finding the right one is a matter of optimizing the likelihood function which quantifies how well the model describes the observed data. By employing a heterogeneous approach, in which both CPU and GPU are kept busy with useful work, speedups of 10 to 12 times can be achieved on a moderate machine with a high-end consumer GPU as compared to a high-end system with 32 CPU cores.

Conclusions

Utilizing GPUs for parameter inference can bring considerable increases in performance using average host systems with high-end consumer GPUs. Future studies should evaluate the benefit of using newer CPU and GPU architectures as well as applying this method to more complex epidemiological scenarios.

Hierarchical particle swarm optimizer for minimizing the non-convex potential energy of molecular structure

The stable conformation of a molecule is greatly important to uncover the secret of its properties and functions. Generally, the conformation of a molecule will be the most stable when it is of the minimum potential energy. Accordingly, the determination of the conformation can be solved in the optimization framework. It is, however, not an easy task to achieve the only conformation with the lowest energy among all the potential ones because of the high complexity of the energy landscape and the exponential computation increasing with molecular size. In this paper, we develop a hierarchical and heterogeneous particle swarm optimizer (HHPSO) to deal with the problem in the minimization of the potential energy. The proposed method is evaluated over a scalable simplified molecular potential energy function with up to 200 degrees of freedom and a realistic energy function of pseudo-ethane molecule. The experimental results are compared with other six PSO variants and four genetic algorithms. The results show HHPSO is significantly better than the compared PSOs with p-value less than 0.01277 over molecular potential energy function.

Complex metabolic network of 1,3-propanediol transport mechanisms and its system identification via biological robustness

The bioconversion of glycerol to 1,3-propanediol (1,3-PD) by Klebsiella pneumoniae (K. pneumoniae) can be characterized by an intricate metabolic network of interactions among biochemical fluxes, metabolic compounds, key enzymes and genetic regulation. Since there are some uncertain factors in the fermentation, especially the transport mechanisms of 1,3-PD across cell membrane, the metabolic network contains multiple possible metabolic systems. Considering the genetic regulation of dha regulon and inhibition of 3-hydroxypropionaldehyde to the growth of cells, we establish a 14-dimensional nonlinear hybrid dynamical system aiming to determine the most possible metabolic system and the corresponding optimal parameter. The existence, uniqueness and continuity of solutions are discussed. Taking the robustness index of the intracellular substances together as a performance index, a system identification model is proposed, in which 1,395 continuous variables and 90 discrete variables are involved. The identification problem is decomposed into two subproblems and a parallel particle swarm optimization procedure is constructed to solve them. Numerical results show that it is most possible that 1,3-PD passes the cell membrane by active transport coupled with passive diffusion.

Asynchronous master-slave parallelization of differential evolution for multi-objective optimization

In this paper, we present AMS-DEMO, an asynchronous master-slave implementation of DEMO, an evolutionary algorithm for multi-objective optimization. AMS-DEMO was designed for solving time-intensive problems efficiently on both homogeneous and heterogeneous parallel computer architectures. The algorithm is used as a test case for the asynchronous master-slave parallelization of multi-objective optimization that has not yet been thoroughly investigated. Selection lag is identified as the key property of the parallelization method, which explains how its behavior depends on the type of computer architecture and the number of processors. It is arrived at analytically and from the empirical results. AMS-DEMO is tested on a benchmark problem and a time-intensive industrial optimization problem, on homogeneous and heterogeneous parallel setups, providing performance results for the algorithm and an insight into the parallelization method. A comparison is also performed between AMS-DEMO and generational master-slave DEMO to demonstrate how the asynchronous parallelization method enhances the algorithm and what benefits it brings compared to the synchronous method.

Limitations of parallel global optimization for large-scale human movement problems

Global optimization algorithms (e.g., simulated annealing, genetic, and particle swarm) have been gaining popularity in biomechanics research, in part due to advances in parallel computing. To date, such algorithms have only been applied to small- or medium-scale optimization problems (<100 design variables). This study evaluates the applicability of a parallel particle swarm global optimization algorithm to large-scale human movement problems. The evaluation was performed using two large-scale (660 design variables) optimization problems that utilized a dynamic, 27 degree-of-freedom, full-body gait model to predict new gait motions from a nominal gait motion. Both cost functions minimized a quantity that reduced the external knee adduction torque. The first one minimized footpath errors corresponding to an increased toe out angle of 15 degrees, while the second one minimized the knee adduction torque directly without changing the footpath. Constraints on allowable changes in trunk orientation, joint angles, joint torques, centers of pressure, and ground reactions were handled using a penalty method. For both problems, a single run with a gradient-based nonlinear least squares algorithm found a significantly better solution than did 10 runs with the global particle swarm algorithm. Due to the penalty terms, the physically realistic gradient-based solutions were located within a narrow "channel" in design space that was difficult to enter without gradient information. Researchers should exercise caution when extrapolating the performance of parallel global optimizers to human movement problems with hundreds of design variables, especially when penalty terms are included in the cost function.