High Passivation Performance of Cat-CVD i-a-Si:H Derived from Bayesian Optimization with Practical Constraints

High-quality passivation with intrinsic hydrogenated amorphous Si (i-a-Si:H) is essential for achieving high-efficiency Si heterojunction (SHJ) solar cells. The formation of i-a-Si:H with a high passivation quality requires strict control of the hydrogen content and film density. In this study, we report the effective discovery of i-a-Si:H deposition conditions through catalytic chemical vapor deposition using Bayesian optimization (BO) to maximize the passivation performance. Another contribution of this study to materials science is the establishment of a practical BO scheme consisting of several prediction models in order to account for the practical constraints. By applying the BO scheme, effective minority carrier lifetime (τeff) is maximized within the deposition condition range, while being constrained by the i-a-Si:H thickness and the capabilities of the experimental setup. We achieved a high passivation performance of τeff > 2.6 ms with only 8 cycles in BO, starting with 14 initial samples. Within the investigated range, the deposition conditions were further explored over 20 cycles. The BO provided not only optimal deposition conditions but also scientific knowledge. Contour plots of the predicted τeff values obtained through the BO process demonstrated that there is a band-like high τeff condition in the parameter space between the substrate temperature and SiH4 flow rate. The high void fraction and epitaxial growth were inhibited by controlling the substrate temperature and SiH4 flow rate, resulting in a high passivation quality. This indicates that the combination of the SiH4 flow rate and substrate temperature parameters is crucial to passivation quality. These results can be applied to determine the deposition conditions for a good a-Si:H layer without a high void fraction or epitaxial growth. The research methods shown in this study, practical BO scheme, and further analysis based on the optimized results will be also useful to optimize and analyze the process conditions of semiconductor processes including plasma-enhanced chemical vapor deposition for SHJ solar cells.

Autonomous Electron Tomography Reconstruction with Machine Learning

Modern electron tomography has progressed to higher resolution at lower doses by leveraging compressed sensing (CS) methods that minimize total variation (TV). However, these sparsity-emphasized reconstruction algorithms introduce tunable parameters that greatly influence the reconstruction quality. Here, Pareto front analysis shows that high-quality tomograms are reproducibly achieved when TV minimization is heavily weighted. However, in excess, CS tomography creates overly smoothed three-dimensional (3D) reconstructions. Adding momentum to the gradient descent during reconstruction reduces the risk of over-smoothing and better ensures that CS is well behaved. For simulated data, the tedious process of tomography parameter selection is efficiently solved using Bayesian optimization with Gaussian processes. In combination, Bayesian optimization with momentum-based CS greatly reduces the required compute time-an 80% reduction was observed for the 3D reconstruction of SrTiO3 nanocubes. Automated parameter selection is necessary for large-scale tomographic simulations that enable the 3D characterization of a wider range of inorganic and biological materials.

High-Throughput Synthesis and Machine Learning Assisted Design of Photodegradable Hydrogels

Due to the large chemical space, the design of functional and responsive soft materials poses many challenges but also offers a wide range of opportunities in terms of the scope of possible properties. Herein, an experimental workflow for miniaturized combinatorial high-throughput screening of functional hydrogel libraries is reported. The data created from the analysis of the photodegradation process of more than 900 different types of hydrogel pads are used to train a machine learning model for automated decision making. Through iterative model optimization based on Bayesian optimization, a substantial improvement in response properties is achieved and thus expanded the scope of material properties obtainable within the chemical space of hydrogels in the study. It is therefore demonstrated that the potential of combining miniaturized high-throughput experiments with smart optimization algorithms for cost and time efficient optimization of materials properties.

Exploring Hydrogen Storage Capacity in Metal-Organic Frameworks: A Bayesian Optimization Approach

Metal-organic Frameworks (MOFs) can be employed for gas storage, capture, and sensing. Finding the MOF with the best adsorption property from a large database is usual for adsorption calculations. In high-throughput computational research, the expense of computing thermodynamic quantities limits the finding of MOFs for separations and storage. In this work, we demonstrate the usefulness of Bayesian optimization (BO) for estimating the H2 uptake capability of MOFs by using an existing dataset containing 98000 real and hypothetical MOFs. We demonstrate that in order to recover the best candidate MOFs, less than 0.027 % of the database needs to be screened using the BO method. This allows future adsorption experiments on a small sample of MOFs to be undertaken with minimal experimental effort by effectively screening MOF databases. In addition, the presented BO can provide comprehensible material design insights, and the framework will be transferable to optimizing other target properties. We also suggest using Particle Swarm Optimisation (PSO), a swarm intelligence technique in artificial intelligence, to estimate MOFs' H2 uptake potential to achieve results comparable to BO. In addition, we implement a novel modification of PSO called Evolutionary-PSO (EPSO) to compare and find interesting outcomes.

A Customized Bayesian Algorithm to Optimize Enzyme-Catalyzed Reactions

Design of experiments (DoE) plays an important role in optimizing the catalytic performance of chemical reactions. The most commonly used DoE relies on the response surface methodology (RSM) to model the variable space of experimental conditions with the fewest number of experiments. However, the RSM leads to an exponential increase in the number of required experiments as the number of variables increases. Herein we describe a Bayesian optimization algorithm (BOA) to optimize the continuous parameters (e.g., temperature, reaction time, reactant and enzyme concentrations, etc.) of enzyme-catalyzed reactions with the aim of maximizing performance. Compared to existing Bayesian optimization methods, we propose an improved algorithm that leads to better results under limited resources and time for experiments. To validate the versatility of the BOA, we benchmarked its performance with biocatalytic C-C bond formation and amination for the optimization of the turnover number. Gratifyingly, up to 80% improvement compared to RSM and up to 360% improvement vs previous Bayesian optimization algorithms were obtained. Importantly, this strategy enabled simultaneous optimization of both the enzyme's activity and selectivity for cross-benzoin condensation.

Application of convolutional neural networks based on Bayesian optimization to landslide susceptibility mapping of transmission tower foundation

The stability of tower foundation slopes is an important factor to maintain the operation of a power system. However, it is time-consuming and expensive to evaluate tower foundation slopes one by one due to the large area. The aim of this study is to investigate the performance of CNNs with different architectures and training options for transmission tower foundation landslide spatial prediction (LSP) by Bayesian optimization. Accordingly, fourteen influencing factors related to landslide evaluated by gain ratio technique are considered and 424 historical landslide locations in Luoding and Xinyi Counties (Guangdong Province, China) are randomly divided into 80% for training and 20% for testing the CNNs. The CNN performances are investigated by permutating and combining different numbers of convolutional layers, pooling layers and learning rate strategy. In 59 Bayesian optimized cases, three conclusions are drawn: (a) the CNNs yielded the best result with 3 convolution layers, (b) the CNN without a pooling layer performs best, and (c) a piece-wise decay learning rate strategy yields better performance. Meanwhile, the excellent performance of the CNN obtained by Bayesian optimization (CNNB) has also been validated by comparisons with gravitational search optimization algorithm and other landslide spatial models, which indicates that CNNB can be applied to generate the susceptibility maps for locating transmission tower foundations in high landslide susceptibility zones and reducing the impact of landslides on power supply by taking measures in advance.

Implementing robotics and artificial intelligence

An automated platform for cell culture combines robotics and artificial intelligence to optimize cell culture protocols and reliably produce specific cell types that could be used for regenerative medicine treatments.

Robotic search for optimal cell culture in regenerative medicine

Induced differentiation is one of the most experience- and skill-dependent experimental processes in regenerative medicine, and establishing optimal conditions often takes years. We developed a robotic AI system with a batch Bayesian optimization algorithm that autonomously induces the differentiation of induced pluripotent stem cell-derived retinal pigment epithelial (iPSC-RPE) cells. From 200 million possible parameter combinations, the system performed cell culture in 143 different conditions in 111 days, resulting in 88% better iPSC-RPE production than that obtained by the pre-optimized culture in terms of the pigmentation scores. Our work demonstrates that the use of autonomous robotic AI systems drastically accelerates systematic and unbiased exploration of experimental search space, suggesting immense use in medicine and research.

pSuc-FFSEA: Predicting Lysine Succinylation Sites in Proteins Based on Feature Fusion and Stacking Ensemble Algorithm

Being a new type of widespread protein post-translational modifications discovered in recent years, succinylation plays a key role in protein conformational regulation and cellular function regulation. Numerous studies have shown that succinylation modifications are closely associated with the development of many diseases. In order to gain insight into the mechanism of succinylation, it is vital to identify lysine succinylation sites. However, experimental identification of succinylation sites is time-consuming and laborious, and traditional identification tools are unable to meet the rapid growth of datasets. Therefore, to solve this problem, we developed a new predictor named pSuc-FFSEA, which can predict succinylation sites in protein sequences by feature fusion and stacking ensemble algorithm. Specifically, the sequence information and physicochemical properties were first extracted using EBGW, One-Hot, continuous bag-of-words, chaos game representation, and AAF_DWT. Following that, feature selection was performed, which applied LASSO to select the optimal subset of features for the classifier, and then, stacking ensemble classifier was designed using two-layer stacking ensemble, selecting three classifiers, SVM, broad learning system and LightGBM classifier, as the base classifiers of the first layer, using logistic regression classifier as the meta classifier of the second layer. In order to further improve the model prediction accuracy and reduce the computational effort, bayesian optimization algorithm and grid search algorithm were utilized to optimize the hyperparameters of the classifier. Finally, the results of rigorous 10-fold cross-validation indicated our predictor showed excellent robustness and performed better than the previous prediction tools, which achieved an average prediction accuracy of 0.7773 ± 0.0120. Besides, for the convenience of the most experimental scientists, a user-friendly and comprehensive web-server for pSuc-FFSEA has been established at https://bio.cangmang.xyz/pSuc-FFSEA, by which one can easily obtain the expected data and results without going through the complicated mathematics.

Handling EEG artifacts and searching individually optimal experimental parameter in real time: a system development and demonstration

Objective.Neuroadaptive paradigms that systematically assess event-related potential (ERP) features across many different experimental parameters have the potential to improve the generalizability of ERP findings and may help to accelerate ERP-based biomarker discovery by identifying the exact experimental conditions for which ERPs differ most for a certain clinical population. Obtaining robust and reliable ERPs online is a prerequisite for ERP-based neuroadaptive research. One of the key steps involved is to correctly isolate electroencephalography artifacts in real time because they contribute a large amount of variance that, if not removed, will greatly distort the ERP obtained. Another key factor of concern is the computational cost of the online artifact handling method. This work aims to develop and validate a cost-efficient system to support ERP-based neuroadaptive research.Approach.We developed a simple online artifact handling method, single trial PCA-based artifact removal (SPA), based on variance distribution dichotomies to distinguish between artifacts and neural activity. We then applied this method in an ERP-based neuroadaptive paradigm in which Bayesian optimization was used to search individually optimal inter-stimulus-interval (ISI) that generates ERP with the highest signal-to-noise ratio.Main results.SPA was compared to other offline and online algorithms. The results showed that SPA exhibited good performance in both computational efficiency and preservation of ERP pattern. Based on SPA, the Bayesian optimization procedure was able to quickly find individually optimal ISI.Significance.The current work presents a simple yet highly cost-efficient method that has been validated in its ability to extract ERP, preserve ERP effects, and better support ERP-based neuroadaptive paradigm.

Forecast of the COVID-19 Epidemic Based on RF-BOA-LightGBM

In this paper, we utilize the Internet big data tool, namely Baidu Index, to predict the development trend of the new coronavirus pneumonia epidemic to obtain further data. By selecting appropriate keywords, we can collect the data of COVID-19 cases in China between 1 January 2020 and 1 April 2020. After preprocessing the data set, the optimal sub-data set can be obtained by using random forest feature selection method. The optimization results of the seven hyperparameters of the LightGBM model by grid search, random search and Bayesian optimization algorithms are compared. The experimental results show that applying the data set obtained from the Baidu Index to the Bayesian-optimized LightGBM model can better predict the growth of the number of patients with new coronary pneumonias, and also help people to make accurate judgments to the development trend of the new coronary pneumonia.

Intelligent Fault Diagnosis of Rotary Machinery by Convolutional Neural Network with Automatic Hyper-Parameters Tuning Using Bayesian Optimization

Intelligent fault diagnosis can be related to applications of machine learning theories to machine fault diagnosis. Although there is a large number of successful examples, there is a gap in the optimization of the hyper-parameters of the machine learning model, which ultimately has a major impact on the performance of the model. Machine learning experts are required to configure a set of hyper-parameter values manually. This work presents a convolutional neural network based data-driven intelligent fault diagnosis technique for rotary machinery which uses model with optimized hyper-parameters and network structure. The proposed technique input raw three axes accelerometer signal as high definition 1-D data into deep learning layers with optimized hyper-parameters. Input is consisted of wide 12,800 × 1 × 3 vibration signal matrix. Model learning phase includes Bayesian optimization that optimizes hyper-parameters of the convolutional neural network. Finally, by using a Convolutional Neural Network (CNN) model with optimized hyper-parameters, classification in one of the 8 different machine states and 2 rotational speeds can be performed. This study accomplished the effective classification of different rotary machinery states in different rotational speeds using optimized convolutional artificial neural network for classification of raw three axis accelerometer signal input. Overall classification accuracy of 99.94% on evaluation set is obtained with the CNN model based on 19 layers. Additionally, more data are collected on the same machine with altered bearings to test the model for overfitting. Result of classification accuracy of 100% on second evaluation set has been achieved, proving the potential of using the proposed technique.

Automating proton treatment planning with beam angle selection using Bayesian optimization

PURPOSE

To present a fully automated treatment planning process for proton therapy including beam angle selection using a novel Bayesian optimization approach and previously developed constrained hierarchical fluence optimization method.

METHODS

We adapted our in-house automated intensity modulated radiation therapy (IMRT) treatment planning system, which is based on constrained hierarchical optimization and referred to as ECHO (expedited constrained hierarchical optimization), for proton therapy. To couple this to beam angle selection, we propose using a novel Bayesian approach. By integrating ECHO with this Bayesian beam selection approach, we obtain a fully automated treatment planning framework including beam angle selection. Bayesian optimization is a global optimization technique which only needs to search a small fraction of the search space for slowly varying objective functions (i.e., smooth functions). Expedited constrained hierarchical optimization is run for some initial beam angle candidates and the resultant treatment plan for each beam configuration is rated using a clinically relevant treatment score function. Bayesian optimization iteratively predicts the treatment score for not-yet-evaluated candidates to find the best candidate to be optimized next with ECHO. We tested this technique on five head-and-neck (HN) patients with two coplanar beams. In addition, tests were performed with two noncoplanar and three coplanar beams for two patients.

RESULTS

For the two coplanar configurations, the Bayesian optimization found the optimal beam configuration after running ECHO for, at most, 4% of all potential configurations (23 iterations) for all patients (range: 2%-4%). Compared with the beam configurations chosen by the planner, the optimal configurations reduced the mandible maximum dose by 6.6 Gy and high dose to the unspecified normal tissues by 3.8 Gy, on average. For the two noncoplanar and three coplanar beam configurations, the algorithm converged after 45 iterations (examining <1% of all potential configurations).

CONCLUSIONS

A fully automated and efficient treatment planning process for proton therapy, including beam angle optimization was developed. The algorithm automatically generates high-quality plans with optimal beam angle configuration by combining Bayesian optimization and ECHO. As the Bayesian optimization is capable of handling complex nonconvex functions, the treatment score function which is used in the algorithm to evaluate the dose distribution corresponding to each beam configuration can contain any clinically relevant metric.

Exploring Successful Parameter Region for Coarse-Grained Simulation of Biomolecules by Bayesian Optimization and Active Learning

Accompanied with an increase of revealed biomolecular structures owing to advancements in structural biology, the molecular dynamics (MD) approach, especially coarse-grained (CG) MD suitable for macromolecules, is becoming increasingly important for elucidating their dynamics and behavior. In fact, CG-MD simulation has succeeded in qualitatively reproducing numerous biological processes for various biomolecules such as conformational changes and protein folding with reasonable calculation costs. However, CG-MD simulations strongly depend on various parameters, and selecting an appropriate parameter set is necessary to reproduce a particular biological process. Because exhaustive examination of all candidate parameters is inefficient, it is important to identify successful parameters. Furthermore, the successful region, in which the desired process is reproducible, is essential for describing the detailed mechanics of functional processes and environmental sensitivity and robustness. We propose an efficient search method for identifying the successful region by using two machine learning techniques, Bayesian optimization and active learning. We evaluated its performance using F1-ATPase, a biological rotary motor, with CG-MD simulations. We successfully identified the successful region with lower computational costs (12.3% in the best case) without sacrificing accuracy compared to exhaustive search. This method can accelerate not only parameter search but also biological discussion of the detailed mechanics of functional processes and environmental sensitivity based on MD simulation studies.

Bayesian Optimization Based on K-Optimality

Bayesian optimization (BO) based on the Gaussian process (GP) surrogate model has attracted extensive attention in the field of optimization and design of experiments (DoE). It usually faces two problems: the unstable GP prediction due to the ill-conditioned Gram matrix of the kernel and the difficulty of determining the trade-off parameter between exploitation and exploration. To solve these problems, we investigate the K-optimality, aiming at minimizing the condition number. Firstly, the Sequentially Bayesian K-optimal design (SBKO) is proposed to ensure the stability of the GP prediction, where the K-optimality is given as the acquisition function. We show that the SBKO reduces the integrated posterior variance and maximizes the hyper-parameters’ information gain simultaneously. Secondly, a K-optimal enhanced Bayesian Optimization (KO-BO) approach is given for the optimization problems, where the K-optimality is used to define the trade-off balance parameters which can be output automatically. Specifically, we focus our study on the K-optimal enhanced Expected Improvement algorithm (KO-EI). Numerical examples show that the SBKO generally outperforms the Monte Carlo, Latin hypercube sampling, and sequential DoE approaches by maximizing the posterior variance with the highest precision of prediction. Furthermore, the study of the optimization problem shows that the KO-EI method beats the classical EI method due to its higher convergence rate and smaller variance.