Wearable IoT Smart-Log Patch: An Edge Computing-Based Bayesian Deep Learning Network System for Multi Access Physical Monitoring System

According to the survey on various health centres, smart log-based multi access physical monitoring system determines the health conditions of humans and their associated problems present in their lifestyle. At present, deficiency in significant nutrients leads to deterioration of organs, which creates various health problems, particularly for infants, children, and adults. Due to the importance of a multi access physical monitoring system, children and adolescents’ physical activities should be continuously monitored for eliminating difficulties in their life using a smart environment system. Nowadays, in real-time necessity on multi access physical monitoring systems, information requirements and the effective diagnosis of health condition is the challenging task in practice. In this research, wearable smart-log patch with Internet of Things (IoT) sensors has been designed and developed with multimedia technology. Further, the data computation in that smart-log patch has been analysed using edge computing on Bayesian deep learning network (EC-BDLN), which helps to infer and identify various physical data collected from the humans in an accurate manner to monitor their physical activities. Then, the efficiency of this wearable IoT system with multimedia technology is evaluated using experimental results and discussed in terms of accuracy, efficiency, mean residual error, delay, and less energy consumption. This state-of-the-art smart-log patch is considered as one of evolutionary research in health checking of multi access physical monitoring systems with multimedia technology.

Compensating for visibility artefacts in photoacoustic imaging with a deep learning approach providing prediction uncertainties

Conventional photoacoustic imaging may suffer from the limited view and bandwidth of ultrasound transducers. A deep learning approach is proposed to handle these problems and is demonstrated both in simulations and in experiments on a multi-scale model of leaf skeleton. We employed an experimental approach to build the training and the test sets using photographs of the samples as ground truth images. Reconstructions produced by the neural network show a greatly improved image quality as compared to conventional approaches. In addition, this work aimed at quantifying the reliability of the neural network predictions. To achieve this, the dropout Monte-Carlo procedure is applied to estimate a pixel-wise degree of confidence on each predicted picture. Last, we address the possibility to use transfer learning with simulated data in order to drastically limit the size of the experimental dataset.

Probabilistic Damage Detection of a Steel Truss Bridge Model by Optimally Designed Bayesian Neural Network

Excellent pattern matching capability makes artificial neural networks (ANNs) a very promising approach for vibration-based structural health monitoring (SHM). The proper design of the network architecture with the suitable complexity is vital to the ANN-based structural damage detection. In addition to the number of hidden neurons, the type of transfer function used in the hidden layer cannot be neglected for the ANN design. Neural network learning can be further presented in the framework of Bayesian statistics, but the issues of selection for the hidden layer transfer function with respect to the Bayesian neural network has not yet been reported in the literature. In addition, most of the research works in the literature for addressing the predictive distribution of neural network output is only for a single target variable, while multiple target variables are rarely involved. In the present paper, for the purpose of probabilistic structural damage detection, Bayesian neural networks with multiple target variables are optimally designed, and the selection of the number of neurons, and the transfer function in the hidden layer, are carried out simultaneously to achieve a neural network architecture with suitable complexity. Furthermore, the nonlinear network function can be approximately linear by assuming the posterior distribution of network parameters is a sufficiently narrow Gaussian, and then the input-dependent covariance matrix of the predictive distribution of network output can be obtained with the Gaussian assumption for the situation of multiple target variables. Structural damage detection is conducted for a steel truss bridge model to verify the proposed method through a set of numerical case studies.

Enabling Training of Neural Networks on Noisy Hardware

Deep neural networks (DNNs) are typically trained using the conventional stochastic gradient descent (SGD) algorithm. However, SGD performs poorly when applied to train networks on non-ideal analog hardware composed of resistive device arrays with non-symmetric conductance modulation characteristics. Recently we proposed a new algorithm, the Tiki-Taka algorithm, that overcomes this stringent symmetry requirement. Here we build on top of Tiki-Taka and describe a more robust algorithm that further relaxes other stringent hardware requirements. This more robust second version of the Tiki-Taka algorithm (referred to as TTv2) 1. decreases the number of device conductance states requirement from 1000s of states to only 10s of states, 2. increases the noise tolerance to the device conductance modulations by about 100x, and 3. increases the noise tolerance to the matrix-vector multiplication performed by the analog arrays by about 10x. Empirical simulation results show that TTv2 can train various neural networks close to their ideal accuracy even at extremely noisy hardware settings. TTv2 achieves these capabilities by complementing the original Tiki-Taka algorithm with lightweight and low computational complexity digital filtering operations performed outside the analog arrays. Therefore, the implementation cost of TTv2 compared to SGD and Tiki-Taka is minimal, and it maintains the usual power and speed benefits of using analog hardware for training workloads. Here we also show how to extract the neural network from the analog hardware once the training is complete for further model deployment. Similar to Bayesian model averaging, we form analog hardware compatible averages over the neural network weights derived from TTv2 iterates. This model average then can be transferred to another analog or digital hardware with notable improvements in test accuracy, transcending the trained model itself. In short, we describe an end-to-end training and model extraction technique for extremely noisy crossbar-based analog hardware that can be used to accelerate DNN training workloads and match the performance of full-precision SGD.

A hybrid framework for improving uncertainty quantification in deep learning-based QSAR regression modeling

Reliable uncertainty quantification for statistical models is crucial in various downstream applications, especially for drug design and discovery where mistakes may incur a large amount of cost. This topic has therefore absorbed much attention and a plethora of methods have been proposed over the past years. The approaches that have been reported so far can be mainly categorized into two classes: distance-based approaches and Bayesian approaches. Although these methods have been widely used in many scenarios and shown promising performance with their distinct superiorities, being overconfident on out-of-distribution examples still poses challenges for the deployment of these techniques in real-world applications. In this study we investigated a number of consensus strategies in order to combine both distance-based and Bayesian approaches together with post-hoc calibration for improved uncertainty quantification in QSAR (Quantitative Structure-Activity Relationship) regression modeling. We employed a set of criteria to quantitatively assess the ranking and calibration ability of these models. Experiments based on 24 bioactivity datasets were designed to make critical comparison between the model we proposed and other well-studied baseline models. Our findings indicate that the hybrid framework proposed by us can robustly enhance the model ability of ranking absolute errors. Together with post-hoc calibration on the validation set, we show that well-calibrated uncertainty quantification results can be obtained in domain shift settings. The complementarity between different methods is also conceptually analyzed.

Machine Learning Derived Lifting Techniques and Pain Self-Efficacy in People with Chronic Low Back Pain

This paper proposes an innovative methodology for finding how many lifting techniques people with chronic low back pain (CLBP) can demonstrate with camera data collected from 115 participants. The system employs a feature extraction algorithm to calculate the knee, trunk and hip range of motion in the sagittal plane, Ward’s method, a combination of K-means and Ensemble clustering method for classification algorithm, and Bayesian neural network to validate the result of Ward’s method and the combination of K-means and Ensemble clustering method. The classification results and effect size show that Ward clustering is the optimal method where precision and recall percentages of all clusters are above 90, and the overall accuracy of the Bayesian Neural Network is 97.9%. The statistical analysis reported a significant difference in the range of motion of the knee, hip and trunk between each cluster, F (9, 1136) = 195.67, p < 0.0001. The results of this study suggest that there are four different lifting techniques in people with CLBP. Additionally, the results show that even though the clusters demonstrated similar pain levels, one of the clusters, which uses the least amount of trunk and the most knee movement, demonstrates the lowest pain self-efficacy.

Left ventricle quantification with sample-level confidence estimation via Bayesian neural network

Quantification of cardiac left ventricle has become a hot topic due to its great significance in clinical practice. Many efforts have been devoted to LV quantification and obtained promising performance with the help of various deep neural networks when validated on a group of samples. However, none of them can provide sample-level confidence of the results, i.e., how reliable is the prediction result for one single sample, which would help clinicians make decisions of whether or not to accept the automatic results. In this paper, we achieve this by introducing the uncertainty analysis theory into our LV quantification network. Two types of uncertainty, Model Uncertainty, and Data Uncertainty are analyzed for the quantification performance and contribute to the sample-level confidence. Experiments with data of 145 subjects validate that our method not only improved the quantification performance with an uncertainty-weighted regression loss but also is capable of providing for each sample the confidence level of the estimation results for clinicians' further consideration.

Winsorization for Robust Bayesian Neural Networks

With the advent of big data and the popularity of black-box deep learning methods, it is imperative to address the robustness of neural networks to noise and outliers. We propose the use of Winsorization to recover model performances when the data may have outliers and other aberrant observations. We provide a comparative analysis of several probabilistic artificial intelligence and machine learning techniques for supervised learning case studies. Broadly, Winsorization is a versatile technique for accounting for outliers in data. However, different probabilistic machine learning techniques have different levels of efficiency when used on outlier-prone data, with or without Winsorization. We notice that Gaussian processes are extremely vulnerable to outliers, while deep learning techniques in general are more robust.

A Bayesian Deep Neural Network for Safe Visual Servoing in Human-Robot Interaction

Safety is an important issue in human–robot interaction (HRI) applications. Various research works have focused on different levels of safety in HRI. If a human/obstacle is detected, a repulsive action can be taken to avoid the collision. Common repulsive actions include distance methods, potential field methods, and safety field methods. Approaches based on machine learning are less explored regarding the selection of the repulsive action. Few research works focus on the uncertainty of the data-based approaches and consider the efficiency of the executing task during collision avoidance. In this study, we describe a system that can avoid collision with human hands while the robot is executing an image-based visual servoing (IBVS) task. We use Monte Carlo dropout (MC dropout) to transform a deep neural network (DNN) to a Bayesian DNN, and learn the repulsive position for hand avoidance. The Bayesian DNN allows IBVS to converge faster than the opposite repulsive pose. Furthermore, it allows the robot to avoid undesired poses that the DNN cannot avoid. The experimental results show that Bayesian DNN has adequate accuracy and can generalize well on unseen data. The predictive interval coverage probability (PICP) of the predictions along x, y, and z directions are 0.84, 0.94, and 0.95, respectively. In the space which is unseen in the training data, the Bayesian DNN is also more robust than a DNN. We further implement the system on a UR10 robot, and test the robustness of the Bayesian DNN and the IBVS convergence speed. Results show that the Bayesian DNN can avoid the poses out of the reach range of the robot and it lets the IBVS task converge faster than the opposite repulsive pose.¹

RAMP: response-aware multi-task learning with contrastive regularization for cancer drug response prediction

The accurate prediction of cancer drug sensitivity according to the multiomics profiles of individual patients is crucial for precision cancer medicine. However, the development of prediction models has been challenged by the complex crosstalk of input features and the resistance-dominant drug response information contained in public databases. In this study, we propose a novel multidrug response prediction framework, response-aware multitask prediction (RAMP), via a Bayesian neural network and restrict it by soft-supervised contrastive regularization. To utilize network embedding vectors as representation learning features for heterogeneous networks, we harness response-aware negative sampling, which applies cell line-drug response information to the training of network embeddings. RAMP overcomes the prediction accuracy limitation induced by the imbalance of trained response data based on the comprehensive selection and utilization of drug response features. When trained on the Genomics of Drug Sensitivity in Cancer dataset, RAMP achieved an area under the receiver operating characteristic curve > 89%, an area under the precision-recall curve > 59% and an $\textrm{F}_1$ score > 52% and outperformed previously developed methods on both balanced and imbalanced datasets. Furthermore, RAMP predicted many missing drug responses that were not included in the public databases. Our results showed that RAMP will be suitable for the high-throughput prediction of cancer drug sensitivity and will be useful for guiding cancer drug selection processes. The Python implementation for RAMP is available at https://github.com/hvcl/RAMP.

Large-scale spatiotemporal deep learning predicting urban residential indoor PM2.5 concentration

Indoor PM2.5 pollution is one of the leading causes of death and disease worldwide. As monitoring indoor PM2.5 concentrations on a large scale is challenging, it is urgent to assess population-level exposure and related health risks to develop an easy-to-use and generalized model to predict indoor PM2.5 concentrations and spatiotemporal variations at the global level. Existing machine learning models of indoor PM2.5 are prone to deliver single-point predictions, and their input strategies are not widely applicable. Here, we developed a Bayesian neural network (BNN) model for predicting the distribution of daily average urban residential PM2.5 concentration based on multiple data sources available from nationwide comprehensive sensor-monitoring records in China. The BNN model showed good performance with a 10-fold cross-validation R2 of 0.70, mean-absolute-error of 9.45 μg/m3, root-mean-square error of 13.3 μg/m3, and 95 % prediction interval coverage of 85 %. To demonstrate the application process, this model was applied to predict indoor PM2.5 concentrations on a large spatiotemporal scale. Our modeled population-weighted annual indoor PM2.5 concentration for China in 2019 was 22.8 μg/m3, far exceeding the WHO standard. The validity of the model at the population level can be further bolstered, making it valuable for assessing and managing indoor air pollution-related health risks.

Predictions of bitcoin prices through machine learning based frameworks

The high volatility of an asset in financial markets is commonly seen as a negative factor. However short-term trades may entail high profits if traders open and close the correct positions. The high volatility of cryptocurrencies, and in particular of Bitcoin, is what made cryptocurrency trading so profitable in these last years. The main goal of this work is to compare several frameworks each other to predict the daily closing Bitcoin price, investigating those that provide the best performance, after a rigorous model selection by the so-called k-fold cross validation method. We evaluated the performance of one stage frameworks, based only on one machine learning technique, such as the Bayesian Neural Network, the Feed Forward and the Long Short Term Memory Neural Networks, and that of two stages frameworks formed by the neural networks just mentioned in cascade to Support Vector Regression. Results highlight higher performance of the two stages frameworks with respect to the correspondent one stage frameworks, but for the Bayesian Neural Network. The one stage framework based on Bayesian Neural Network has the highest performance and the order of magnitude of the mean absolute percentage error computed on the predicted price by this framework is in agreement with those reported in recent literature works.

Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability

Bayesian neural network (BNN) allows for uncertainty quantification in prediction, offering an advantage over regular neural networks that has not been explored in the differential privacy (DP) framework. We fill this important gap by leveraging recent development in Bayesian deep learning and privacy accounting to offer a more precise analysis of the trade-off between privacy and accuracy in BNN. We propose three DP-BNNs that characterize the weight uncertainty for the same network architecture in distinct ways, namely DP-SGLD (via the noisy gradient method), DP-BBP (via changing the parameters of interest) and DP-MC Dropout (via the model architecture). Interestingly, we show a new equivalence between DP-SGD and DP-SGLD, implying that some non-Bayesian DP training naturally allows for uncertainty quantification. However, the hyperparameters such as learning rate and batch size, can have different or even opposite effects in DP-SGD and DP-SGLD. Extensive experiments are conducted to compare DP-BNNs, in terms of privacy guarantee, prediction accuracy, uncertainty quantification, calibration, computation speed, and generalizability to network architecture. As a result, we observe a new tradeoff between the privacy and the reliability. When compared to non-DP and non-Bayesian approaches, DP-SGLD is remarkably accurate under strong privacy guarantee, demonstrating the great potential of DP-BNN in real-world tasks.

Prediction of Shooting Events in Soccer Videos Using Complete Bipartite Graphs and Players' Spatial-Temporal Relations

In soccer, quantitatively evaluating the performance of players and teams is essential to improve tactical coaching and players' decision-making abilities. To achieve this, some methods use predicted probabilities of shoot event occurrences to quantify player performances, but conventional shoot prediction models have not performed well and have failed to consider the reliability of the event probability. This paper proposes a novel method that effectively utilizes players' spatio-temporal relations and prediction uncertainty to predict shoot event occurrences with greater accuracy and robustness. Specifically, we represent players' relations as a complete bipartite graph, which effectively incorporates soccer domain knowledge, and capture latent features by applying a graph convolutional recurrent neural network (GCRNN) to the constructed graph. Our model utilizes a Bayesian neural network to predict the probability of shoot event occurrence, considering spatio-temporal relations between players and prediction uncertainty. In our experiments, we confirmed that the proposed method outperformed several other methods in terms of prediction performance, and we found that considering players' distances significantly affects the prediction accuracy.

Enhanced probabilistic prediction of pavement deterioration using Bayesian neural networks and cuckoo search optimization

The predictive performance of probabilistic pavement condition deterioration is critical for effective maintenance and rehabilitation decisions. Currently, numerous improved models exist, but few rely on probabilistic models to improve pavement deterioration prediction. Therefore, this study proposed an improved probabilistic model for pavement deterioration prediction based on the coupling of Bayesian neural network (BNN) and cuckoo search (CS) algorithm. The model prediction performance is evaluated against two metrics: determination coefficient (R2) and standard deviation (stability). Finally, based on the data from the pavement management system in Shanxi Province, it was verified that the CS-BNN model outperforms the genetic algorithm-BNN, particle swarm optimization-BNN, and BNN models in terms of the two metrics. Sensitivity analysis further confirms the robustness of the CS-BNN model. The findings indicate that the CS-BNN model provides more reliable predictions with lower uncertainty, aiding road engineers in optimizing maintenance schedules and costs.

Multi-energy CT material decomposition using Bayesian deep convolutional neural network with explicit penalty of uncertainty and bias

Convolutional neural network (CNN)-based material decomposition has the potential to improve image quality (visual appearance) and quantitative accuracy of material maps. Most methods use deterministic CNNs with mean-square-error loss to provide point-estimates of mass densities. Point estimates can be over-confident as the reliability of CNNs is frequently compromised by bias and two major uncertainties - data and model uncertainties originating from noise in inputs and train-test data dissimilarity, respectively. Also, mean-square-error lacks explicit control of uncertainty and bias. To tackle these problems, a Bayesian dual-task CNN (BDT-CNN) with explicit penalization of uncertainty and bias was developed. It is a probabilistic CNN that concurrently conducts material classification and quantification and allows for pixel-wise modeling of bias, data uncertainty, and model uncertainty. CNN was trained with images of physical and simulated tissue-mimicking inserts at varying mass densities. Hydroxyapatite (nominal density 400mg/cc) and blood (nominal density 1095mg/cc) inserts were placed in different-sized body phantoms (30 - 45cm) and used to evaluate mean-absolute-bias (MAB) in predicted mass densities across different images at routine- and half-routine-dose. Patient CT exams were collected to assess generalizability of BDT-CNN in the presence of anatomical background. Noise insertion was used to simulate patient exams at half- and quarter-routine-dose. The deterministic dual-task CNN was used as baseline. In phantoms, BDT-CNN improved consistency of insert delineation, especially edges, and reduced overall bias (average MAB for hydroxyapatite: BDT-CNN 5.4mgHA/cc, baseline 11.0mgHA/cc and blood: BDT-CNN 8.9mgBlood/cc, baseline 14.0mgBlood/cc). In patient images, BDT-CNN improved detail preservation, lesion conspicuity, and structural consistency across different dose levels.

Analysis of Biomechanical Characteristics of Bone Tissues Using a Bayesian Neural Network: A Narrative Review

Background: Bone elasticity is one of the most important biomechanical parameters of the skeleton. It varies markedly with age, anatomical zone, bone type (cortical or trabecular) and bone marrow status. Methods: This review presents the result of a systematic review and analysis of 495 experimental and analytical papers on the elastic properties of bone tissue. The bone characteristics of hip, shoulder, skull, vertebrae as a function of the factors of age (young and old), sex (male and female), presence/absence of bone marrow and different test methods are examined. The Bayesian neural network (BNN) was used to estimate the uncertainty in some skeletal parameters (age, sex, and body mass index) in predicting bone elastic modulus. Results: It was found that the modulus of elasticity of cortical bone in young people is in the range of 10-30 GPa (depending on the type of bone), and with increasing age, this slightly decreases to 10-25 GPa, while trabecular tissue varies from 0.2 to 5 GPa and reacts more acutely to osteoporosis. Bone marrow, according to several studies, is able to partially increase stiffness under impact loading, but its contribution is minimal under slow deformations. Conclusions: BNN confirmed high variability, supplementing the predictions with confidence intervals and allowed the formation of equations for the calculation of bone tissue elastic modulus for the subsequent selection of the recommended elastic modulus of the finished implant, taking into account the biomechanical characteristics of bone tissue depending on age (young and old), sex (men and women) and anatomical zones of the human skeleton.

A factorial-analysis-based Bayesian neural network method for quantifying China's CO2 emissions under dual-carbon target

Energy-structure transformation and CO2-emission reduction are becoming particularly urgent for China and many other countries. Development of effective methods that are capable of quantifying and predicting CO2 emissions to achieve carbon neutrality is desired. This study advances a factorial-analysis-based Bayesian neural network (abbreviated as FABNN) method to reflect the complex relationship between inputs and outputs as well as reveal the individual and interactive effects of multiple factors affecting CO2 emissions. FABNN is then applied to analyzing CO2 emissions of China (abbreviated as CEC), where multiple factors involve in energy (e.g., the consumption of natural gas, CONG), economic (e.g., Gross domestic product, GDP) and social (e.g., the rate of urbanization, ROU) aspects are investigated and 512 scenarios are designed to achieve the national dual carbon targets (i.e., carbon peak before 2030 and carbon neutrality by 2060). Comparing to the conventional machine learning methods, FABNN performs better in calibration and validation results, indicating that FABNN is suitable for CEC simulation and prediction. Results disclose that the top three factors affecting CEC under the dual‑carbon target are GDP, CONG, and ROU; energy, economic and social contributions are 43.5 %, 34.6 % and 21.9 %, respectively. CEC reaches its carbon peak during 2027-2032 and achieve carbon neutrality during 2053-2057 under all scenarios. Under the optimal scenario (S195), the CO2-emission reduction potential is about 772.2 million tonnes and the consumptions of coal, petroleum and natural gas can be respectively reduced by 3.1 %, 9.9 % and 23.0 % compared to the worst scenario (S466). The results can provide solid support for national energy-structure transformation and CO2-emission reduction to achieve carbon-peak and carbon-neutrality targets.

Development and evaluation of a robotic system for lumbar puncture and epidural steroid injection

Introduction

Lumbar puncture is an important medical procedure for various diagnostics and therapies, but it can be hazardous due to individual variances in subcutaneous soft tissue, especially in the elderly and obese. Our research describes a novel robot-assisted puncture system that automatically controls and maintains the probe at the target tissue layer through a process of tissue recognition.

Methods

The system comprises a robotic system and a master computer. The robotic system is constructed based on a probe consisting of a pair of concentric electrodes. From the probe, impedance spectroscopy measures bio-impedance signals and transforms them into spectra that are communicated to the master computer. The master computer uses a Bayesian neural network to classify the bio-impedance spectra as corresponding to different soft tissues. By feeding the bio-impedance spectra of unknown tissues into the Bayesian neural network, we can determine their categories. Based on the recognition results, the master computer controls the motion of the robotic system.

Results

The proposed system is demonstrated on a realistic phantom made of ex vivo tissues to simulate the spinal environment. The findings indicate that the technology has the potential to increase the precision and security of lumbar punctures and associated procedures.

Discussion

In addition to lumbar puncture, the robotic system is suitable for related puncture operations such as discography, radiofrequency ablation, facet joint injection, and epidural steroid injection, as long as the required tissue recognition features are available. These operations can only be carried out once the puncture needle and additional instruments reach the target tissue layer, despite their ensuing processes being distinct.

Bayesian neural networks for probabilistic modeling of thermal dynamics in multiscale tissue engineering scaffolds

Multiscale tissue engineering integrated with thermal dynamics exhibits a critical role as scaffolds in maintaining the structural integrity and functionality in fabrication applications. Thermal dynamics influence the properties such as thermal conductivity, the capacity of heat, and thermal expansion of materials to maintain the scaffold stability in various physiological conditions. However, the scaffold's heat distribution in a uniform manner is varied due to variations in pore size and geometrics. Additionally, variation in scales affects the thermal gradient impacts on the cell growth and integrity. This paper proposes the 3D Scaffolds Probabilistic Weighted Bayesian Neural Network (3D-SP-WBNN) for tissue engineering with a multiscale scaffold model. It uses the 3D scaffolds for the multiscale design estimation. The weighted Probabilistic Bayesian Neural Network model is employed for estimating features in tissues and evaluating the cell growth and proliferation. Thermal gradients measured were in between 1 °C and 4 °C for human bone, skin, and cartilage tissues. For low and moderate temperatures of 1 °C and 2 °C, the cell proliferation rates in cartilage tissues were 15-20 % per day. The scaffold design uses the Hybrid Hydrogel/PCL composite to achieve a higher proliferation rate of 25-35 % per day. The estimated forecasting achieves an accuracy range of 82-96 % for different cell densities and thermal conditions.

Learning Sparse Deep Neural Networks with a Spike-and-Slab Prior

Deep learning has achieved great successes in many machine learning tasks. However, the deep neural networks (DNNs) are often severely over-parameterized, making them computationally expensive, memory intensive, less interpretable and mis-calibrated. We study sparse DNNs under the Bayesian framework: we establish posterior consistency and structure selection consistency for Bayesian DNNs with a spike-and-slab prior, and illustrate their performance using examples on high-dimensional nonlinear variable selection, large network compression and model calibration. Our numerical results indicate that sparsity is essential for improving the prediction accuracy and calibration of the DNN.

Federated learning meets Bayesian neural network: Robust and uncertainty-aware distributed variational inference

Federated Learning (FL) is a popular framework for data privacy protection in distributed machine learning. However, current FL faces some several problems and challenges, including the limited amount of client data and data heterogeneity. These lead to models trained on clients prone to drifting and overfitting, such that we just obtain suboptimal performance of the aggregated model. To tackle the aforementioned problems, we introduce a novel approach explicitly integrating Bayesian neural networks (BNNs) into the FL framework. The proposed approach is able to enhance the robustness. We refer to this approach as FedUAB, standing for FL with uncertainty-aware BNNs. In the FedUAB algorithm, each FL client independently trains a BNN using the Bayes by backprop algorithm. The weights of approximating model are modeled as Gaussian distributions, which mitigates the overfitting issue and also ensures better data privacy. Besides, we apply novel methods to overcome other key challenges in the fusion of BNNs and FL, such as selecting an optimal prior distribution, aggregating weights characterized by Gaussian forms across multiple clients, and rigorously managing weights variances. In the simulation of a FL environment, FedUAB demonstrated superior performance with both its server-side global model and client-side personalized models, outperforming traditional FL and other Bayesian FL methods. Moreover, it possesses the capability to quantify and decompose uncertainties. We have open-sourced our project at https://github.com/lpf111222/FedUAB/.

A Design Methodology for Fault-Tolerant Neuromorphic Computing Using Bayesian Neural Network

Memristor crossbar arrays are a promising platform for neuromorphic computing. In practical scenarios, the synapse weights represented by the memristors for the underlying system are subject to process variations, in which the programmed weight when read out for inference is no longer deterministic but a stochastic distribution. It is therefore highly desired to learn the weight distribution accounting for process variations, to ensure the same inference performance in memristor crossbar arrays as the design value. In this paper, we introduce a design methodology for fault-tolerant neuromorphic computing using a Bayesian neural network, which combines the variational Bayesian inference technique with a fault-aware variational posterior distribution. The proposed framework based on Bayesian inference incorporates the impacts of memristor deviations into algorithmic training, where the weight distributions of neural networks are optimized to accommodate uncertainties and minimize inference degradation. The experimental results confirm the capability of the proposed methodology to tolerate both process variations and noise, while achieving more robust computing in memristor crossbar arrays.

Achieving well-informed decision-making in drug discovery: a comprehensive calibration study using neural network-based structure-activity models

In the drug discovery process, where experiments can be costly and time-consuming, computational models that predict drug-target interactions are valuable tools to accelerate the development of new therapeutic agents. Estimating the uncertainty inherent in these neural network predictions provides valuable information that facilitates optimal decision-making when risk assessment is crucial. However, such models can be poorly calibrated, which results in unreliable uncertainty estimates that do not reflect the true predictive uncertainty. In this study, we compare different metrics, including accuracy and calibration scores, used for model hyperparameter tuning to investigate which model selection strategy achieves well-calibrated models. Furthermore, we propose to use a computationally efficient Bayesian uncertainty estimation method named HMC Bayesian Last Layer (HBLL), which generates Hamiltonian Monte Carlo (HMC) trajectories to obtain samples for the parameters of a Bayesian logistic regression fitted to the hidden layer of the baseline neural network. We report that this approach improves model calibration and achieves the performance of common uncertainty quantification methods by combining the benefits of uncertainty estimation and probability calibration methods. Finally, we show that combining post hoc calibration method with well-performing uncertainty quantification approaches can boost model accuracy and calibration.

scCrab: A Reference-Guided Cancer Cell Identification Method based on Bayesian Neural Networks

Cancer is a significant global public health concern, where early detection can greatly enhance curative outcomes. Therefore, the identification of cancer cells holds significant importance as the primary method for cancer diagnosis. The advancement of single-cell RNA sequencing (scRNA-seq) technology has made it possible to address the problem of cancer cell identification at the single-cell level more efficiently with computational methods, as opposed to the time-consuming and less reproducible manual identification methods. However, existing computational methods have shown suboptimal identification performance and a lack of capability to incorporate external reference data as prior information. Here, we propose scCrab, a reference-guided automatic cancer cell identification method, which performs ensemble learning based on a Bayesian neural network (BNN) with multi-head self-attention mechanisms and a linear regression model. Through a series of experiments on various datasets, we systematically validated the superior performance of scCrab in both intra- and inter-dataset predictions. Besides, we demonstrated the robustness of scCrab to dropout rate and sample size, and conducted ablation experiments to investigate the contributions of each component in scCrab. Furthermore, as a dedicated model for cancer cell identification, scCrab effectively captures cancer-related biological significance during the identification process.