Empirical validation of Conformal Prediction for trustworthy skin lesions classification

BACKGROUND AND OBJECTIVE

Uncertainty quantification is a pivotal field that contributes to realizing reliable and robust systems. It becomes instrumental in fortifying safe decisions by providing complementary information, particularly within high-risk applications. existing studies have explored various methods that often operate under specific assumptions or necessitate substantial modifications to the network architecture to effectively account for uncertainties. The objective of this paper is to study Conformal Prediction, an emerging distribution-free uncertainty quantification technique, and provide a comprehensive understanding of the advantages and limitations inherent in various methods within the medical imaging field.

METHODS

In this study, we developed Conformal Prediction, Monte Carlo Dropout, and Evidential Deep Learning approaches to assess uncertainty quantification in deep neural networks. The effectiveness of these methods is evaluated using three public medical imaging datasets focused on detecting pigmented skin lesions and blood cell types.

RESULTS

The experimental results demonstrate a significant enhancement in uncertainty quantification with the utilization of the Conformal Prediction method, surpassing the performance of the other two methods. Furthermore, the results present insights into the effectiveness of each uncertainty method in handling Out-of-Distribution samples from domain-shifted datasets. Our code is available at: github.com/jfayyad/ConformalDx.

CONCLUSIONS

Our conclusion highlights a robust and consistent performance of conformal prediction across diverse testing conditions. This positions it as the preferred choice for decision-making in safety-critical applications.

Towards CO2 valorization in a multi remote renewable energy hub framework with uncertainty quantification

In this paper, we propose a multi-RREH (Remote Renewable Energy Hub) based optimization framework. This framework allows a valorization of CO2 using carbon capture technologies. This valorization is grounded on the idea that CO2 gathered from the atmosphere or post combustion can be combined with hydrogen to produce synthetic methane. The hydrogen is obtained from water electrolysis using renewable energy. Such renewable energy is generated in RREH, which are locations where RE is cheap and abundant (e.g., solar PV in the Sahara Desert, or wind in Greenland). We instantiate our framework on a case study focusing on Belgium and 2 RREH, and we conduct a techno-economic analysis under uncertainty. This analysis highlights, among others, the interest in capturing CO2 via Post Combustion Carbon Capture (PCCC) rather than only through Direct Air Capture (DAC) for methane synthesis in RREH. By doing so, a notable reduction of 10% is observed in the total cost of the system under our reference scenario. In addition, we use our framework to derive a carbon price threshold above which carbon capture technologies may start playing a pivotal role in the decarbonation process of our industries.

Modeling correlated uncertainties in stochastic compartmental models

We consider compartmental models of communicable disease with uncertain contact rates. Stochastic fluctuations are often added to the contact rate to account for uncertainties. White noise, which is the typical choice for the fluctuations, leads to significant underestimation of the disease severity. Here, starting from reasonable assumptions on the social behavior of individuals, we model the contacts as a Markov process which takes into account the temporal correlations present in human social activities. Consequently, we show that the mean-reverting Ornstein-Uhlenbeck (OU) process is the correct model for the stochastic contact rate. We demonstrate the implication of our model on two examples: a Susceptibles-Infected-Susceptibles (SIS) model and a Susceptibles-Exposed-Infected-Removed (SEIR) model of the COVID-19 pandemic and compare the results to the available US data from the Johns Hopkins University database. In particular, we observe that both compartmental models with white noise uncertainties undergo transitions that lead to the systematic underestimation of the spread of the disease. In contrast, modeling the contact rate with the OU process significantly hinders such unrealistic noise-induced transitions. For the SIS model, we derive its stationary probability density analytically, for both white and correlated noise. This allows us to give a complete description of the model's asymptotic behavior as a function of its bifurcation parameters, i.e., the basic reproduction number, noise intensity, and correlation time. For the SEIR model, where the probability density is not available in closed form, we study the transitions using Monte Carlo simulations. Our modeling approach can be used to quantify uncertain parameters in a broad range of biological systems.

Modelling lower-limb peripheral arterial disease using clinically available datasets: impact of inflow boundary conditions on hemodynamic indices for restenosis prediction

BACKGROUND AND OBJECTIVES

The integration of hemodynamic markers as risk factors in restenosis prediction models for lower-limb peripheral arteries is hindered by fragmented clinical datasets. Computed tomography (CT) scans enable vessel geometry reconstruction and can be obtained at different times than the Doppler ultrasound (DUS) images, which provide information on blood flow velocity. Computational fluid dynamics (CFD) simulations allow the computation of near-wall hemodynamic indices, whose accuracy depends on the prescribed inlet boundary condition (BC), derived from the DUS images. This study aims to: (i) investigate the impact of different DUS-derived velocity waveforms on CFD results; (ii) test whether the same vessel areas, subjected to altered hemodynamics, can be detected independently of the applied inlet BC; (iii) suggest suitable DUS images to obtain reliable CFD results.

METHODS

CFD simulations were conducted on three patients treated with bypass surgery, using patient-specific DUS-derived inlet BCs recorded at either the same or different time points than the CT scan. The impact of the chosen inflow condition on bypass hemodynamics was assessed in terms of wall shear stress (WSS)-derived quantities. Patient-specific critical thresholds for the hemodynamic indices were applied to identify critical luminal areas and compare the results with a reference obtained with a DUS image acquired in close temporal proximity to the CT scan.

RESULTS

The main findings indicate that: (i) DUS-derived inlet velocity waveforms acquired at different time points than the CT scan led to statistically significantly different CFD results (p<0.001); (ii) the same luminal surface areas, exposed to low time-averaged WSS, could be identified independently of the applied inlet BCs; (iii) similar outcomes were observed for the other hemodynamic indices if the prescribed inlet velocity waveform had the same shape and comparable systolic acceleration time to the one recorded in close temporal proximity to the CT scan.

CONCLUSIONS

Despite a lack of standardised data collection for diseased lower-limb peripheral arteries, an accurate estimation of luminal areas subjected to altered near-wall hemodynamics is possible independently of the applied inlet BC. This holds if the applied inlet waveform shares some characteristics - derivable from the DUS report - as one matching the acquisition time of the CT scan.

Age and sex-dependent sensitivity analysis of a common carotid artery model

The common carotid artery (CCA) is an accessible and informative site for assessing cardiovascular function which makes it a prime candidate for clinically relevant computational modelling. The interpretation of supplemental information possible through modelling is encumbered by measurement uncertainty and population variability in model parameters. The distribution of model parameters likely depends on the specific sub-population of interest and delineation based on sex, age or health status may correspond to distinct ranges of typical parameter values. To assess this impact in a 1D-CCA-model, we delineated specific sub-populations based on age, sex and health status and carried out uncertainty quantification and sensitivity analysis for each sub-population. We performed a structured literature review to characterize sub-population-specific variabilities for eight model parameters without consideration of health status; variations for a healthy sub-populations were based on previously established references values. The variabilities of diameter and distensibility found in the literature review differed from those previously established in a healthy population. Model diameter change and pulse pressure were most sensitive to variations in distensibility, while pressure was most sensitive to resistance in the Windkessel model for all groups. Uncertainties were lower when variabilities were based on a healthy sub-population; however, the qualitative distribution of sensitivity indices was largely similar between the healthy and general population. Average sensitivity of the pressure waveform showed a moderate dependence on age with decreasing sensitivity to distal resistance and increasing sensitivity to distensibility and diameter. The female population was less sensitive to variations in diameter but more sensitive to distensibility coefficient than the male population. Overall, as hypothesized input variabilities differed between sub-populations and resulted in distinct uncertainties and sensitivities of the 1D-CCA-model outputs, particularly over age for the pressure waveform and between males and females for pulse pressure.

A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods

The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the results. In this review, we offer a comprehensive overview of the prevailing methods proposed to quantify the uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models.

Uncertainty quantification of the lattice Boltzmann method focussing on studies of human-scale vascular blood flow

Uncertainty quantification is becoming a key tool to ensure that numerical models can be sufficiently trusted to be used in domains such as medical device design. Demonstration of how input parameters impact the quantities of interest generated by any numerical model is essential to understanding the limits of its reliability. With the lattice Boltzmann method now a widely used approach for computational fluid dynamics, building greater understanding of its numerical uncertainty characteristics will support its further use in science and industry. In this study we apply an in-depth uncertainty quantification study of the lattice Boltzmann method in a canonical bifurcating geometry that is representative of the vascular junctions present in arterial and venous domains. These campaigns examine how quantities of interest-pressure and velocity along the central axes of the bifurcation-are influenced by the algorithmic parameters of the lattice Boltzmann method and the parameters controlling the values imposed at inlet velocity and outlet pressure boundary conditions. We also conduct a similar campaign on a set of personalised vessels to further illustrate the application of these techniques. Our work provides insights into how input parameters and boundary conditions impact the velocity and pressure distributions calculated in a simulation and can guide the choices of such values when applied to vascular studies of patient specific geometries. We observe that, from an algorithmic perspective, the number of time steps and the size of the grid spacing are the most influential parameters. When considering the influence of boundary conditions, we note that the magnitude of the inlet velocity and the mean pressure applied within sinusoidal pressure outlets have the greatest impact on output quantities of interest. We also observe that, when comparing the magnitude of variation imposed in the input parameters with that observed in the output quantities, this variability is particularly magnified when the input velocity is altered. This study also demonstrates how open-source toolkits for validation, verification and uncertainty quantification can be applied to numerical models deployed on high-performance computers without the need for modifying the simulation code itself. Such an ability is key to the more widespread adoption of the analysis of uncertainty in numerical models by significantly reducing the complexity of their execution and analysis.

Quantifying uncertainty: Air quality forecasting based on dynamic spatial-temporal denoising diffusion probabilistic model

Air pollution constitutes a substantial peril to human health, thereby catalyzing the evolution of an array of air quality prediction models. These models span from mechanistic and statistical strategies to machine learning methodologies. The burgeoning field of deep learning has given rise to a plethora of advanced models, which have demonstrated commendable performance. However, previous investigations have overlooked the salience of quantifying prediction uncertainties and potential future interconnections among air monitoring stations. Moreover, prior research typically utilized static predetermined spatial relationships, neglecting dynamic dependencies. To address these limitations, we propose a model named Dynamic Spatial-Temporal Denoising Diffusion Probabilistic Model (DST-DDPM) for air quality prediction. Our model is underpinned by the renowned denoising diffusion model, aiding us in discerning indeterminacy. In order to encapsulate dynamic patterns, we design a dynamic context encoder to generate dynamic adjacency matrices, whilst maintaining static spatial information. Furthermore, we incorporate a spatial-temporal denoising model to concurrently learn both spatial and temporal dependencies. Authenticating our model's performance using a real-world dataset collected in Beijing, the outcomes indicate that our model eclipses other baseline models in terms of both short-term and long-term predictions by 1.36% and 11.62% respectively. Finally, we conduct a case study to exhibit our model's capacity to quantify uncertainties.

Multielement simultaneous quantitative analysis of trace elements in stainless steel via full spectrum laser-induced breakdown spectroscopy

Laser-Induced Breakdown Spectroscopy (LIBS) instruments are increasingly recognized as valuable tools for detecting trace metal elements due to their simplicity, rapid detection, and ability to perform simultaneous multi-element analysis. Traditional LIBS modeling often relies on empirical or machine learning-based feature band selection to establish quantitative models. In this study, we introduce a novel approach-simultaneous multi-element quantitative analysis based on the entire spectrum, which enhances model establishment efficiency and leverages the advantages of LIBS. By logarithmically processing the spectra and quantifying the cognitive uncertainty of the model, we achieved remarkable predictive performance (R2) for trace elements Mn, Mo, Cr, and Cu (0.9876, 0.9879, 0.9891, and 0.9841, respectively) in stainless steel. Our multi-element model shares features and parameters during the learning process, effectively mitigating the impact of matrix effects and self-absorption. Additionally, we introduce a cognitive error term to quantify the cognitive uncertainty of the model. The results suggest that our approach has significant potential in the quantitative analysis of trace elements, providing a reliable data processing method for efficient and accurate multi-task analysis in LIBS. This methodology holds promising applications in the field of LIBS quantitative analysis.

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

Multilevel Monte Carlo Methods for Stochastic Convection-Diffusion Eigenvalue Problems

We develop new multilevel Monte Carlo (MLMC) methods to estimate the expectation of the smallest eigenvalue of a stochastic convection-diffusion operator with random coefficients. The MLMC method is based on a sequence of finite element (FE) discretizations of the eigenvalue problem on a hierarchy of increasingly finer meshes. For the discretized, algebraic eigenproblems we use both the Rayleigh quotient (RQ) iteration and implicitly restarted Arnoldi (IRA), providing an analysis of the cost in each case. By studying the variance on each level and adapting classical FE error bounds to the stochastic setting, we are able to bound the total error of our MLMC estimator and provide a complexity analysis. As expected, the complexity bound for our MLMC estimator is superior to plain Monte Carlo. To improve the efficiency of the MLMC further, we exploit the hierarchy of meshes and use coarser approximations as starting values for the eigensolvers on finer ones. To improve the stability of the MLMC method for convection-dominated problems, we employ two additional strategies. First, we consider the streamline upwind Petrov-Galerkin formulation of the discrete eigenvalue problem, which allows us to start the MLMC method on coarser meshes than is possible with standard FEs. Second, we apply a homotopy method to add stability to the eigensolver for each sample. Finally, we present a multilevel quasi-Monte Carlo method that replaces Monte Carlo with a quasi-Monte Carlo (QMC) rule on each level. Due to the faster convergence of QMC, this improves the overall complexity. We provide detailed numerical results comparing our different strategies to demonstrate the practical feasibility of the MLMC method in different use cases. The results support our complexity analysis and further demonstrate the superiority over plain Monte Carlo in all cases.

Establishing finite element model credibility of a pedicle screw system under compression-bending: An end-to-end example of the ASME V&V 40 standard

Computational modeling and simulation (CM&S) is a key tool in medical device design, development, and regulatory approval. For example, finite element analysis (FEA) is widely used to understand the mechanical integrity and durability of orthopaedic implants. The ASME V&V 40 standard and supporting FDA guidance provide a framework for establishing model credibility, enabling deeper reliance on CM&S throughout the total product lifecycle. Examples of how to apply the principles outlined in the ASME V&V 40 standard are important to facilitating greater adoption by the medical device community, but few published examples are available that demonstrate best practices. Therefore, this paper outlines an end-to-end (E2E) example of the ASME V&V 40 standard applied to an orthopaedic implant. The objective of this study was to illustrate how to establish the credibility of a computational model intended for use as part of regulatory evaluation. In particular, this study focused on whether a design change to a spinal pedicle screw construct (specifically, the addition of a cannulation to an existing non-cannulated pedicle screw) would compromise the rod-screw construct mechanical performance. This question of interest (?OI) was addressed by establishing model credibility requirements according to the ASME V&V 40 standard. Experimental testing to support model validation was performed using spinal rods and non-cannulated pedicle screw constructs made with medical grade titanium (Ti-6Al-4V ELI). FEA replicating the experimental tests was performed by three independent modelers and validated through comparisons of common mechanical properties such as stiffness and yield force. The validated model was then used to simulate F1717 compression-bending testing on the new cannulated pedicle screw design to answer the ?OI, without performing any additional experimental testing. This E2E example provides a realistic scenario for the application of the ASME V&V 40 standard to orthopedic medical device applications.

Photometric stereo data for the validation of a structural health monitoring test rig

Photometric stereo uses images of objects illuminated from various directions to calculate surface normals which can be used to generate 3D meshes of the object. Such meshes can be used by engineers to estimate damage of a concrete surface, or track damage progression over time to inform maintenance decisions. This dataset [1] was collected to quantify the uncertainty in a photometric stereo test rig through both the comparison with a well characterised method (coordinate measurement machine) and experiment virtualisation. Data was collected for 9 real objects using both the test rig and the coordinate measurement machine. These objects range from clay statues to damaged concrete slabs. Furthermore, synthetic data for 12 objects was created via virtual renders generated using Blender (3D software) [2]. The two methods of data generation allowed the decoupling of the physical rig (used to light and photograph objects) and the photometric stereo algorithm (used to convert images and lighting information into 3D meshes). This data can allow users to: test their own photometric stereo algorithms, with specialised data created for structural health monitoring applications; provide an industrially relevant case study to develop and test uncertainty quantification methods on test rigs for structural health monitoring of concrete; or develop data processing methodologies for the alignment of scaled, translated, and rotated data.

Uncertainty-aware image classification on 3D CT lung

Early detection is crucial for lung cancer to prolong the patient's survival. Existing model architectures used in such systems have shown promising results. However, they lack reliability and robustness in their predictions and the models are typically evaluated on a single dataset, making them overconfident when a new class is present. With the existence of uncertainty, uncertain images can be referred to medical experts for a second opinion. Thus, we propose an uncertainty-aware framework that includes three phases: data preprocessing and model selection and evaluation, uncertainty quantification (UQ), and uncertainty measurement and data referral for the classification of benign and malignant nodules using 3D CT images. To quantify the uncertainty, we employed three approaches; Monte Carlo Dropout (MCD), Deep Ensemble (DE), and Ensemble Monte Carlo Dropout (EMCD). We evaluated eight different deep learning models consisting of ResNet, DenseNet, and the Inception network family, all of which achieved average F1 scores above 0.832, and the highest average value of 0.845 was obtained using InceptionResNetV2. Furthermore, incorporating the UQ demonstrated significant improvement in the overall model performance. Upon evaluation of the uncertainty estimate, MCD outperforms the other UQ models except for the metric, URecall, where DE and EMCD excel, implying that they are better at identifying incorrect predictions with higher uncertainty levels, which is vital in the medical field. Finally, we show that using a threshold for data referral can greatly improve the performance further, increasing the accuracy up to 0.959.

Neural orientation distribution fields for estimation and uncertainty quantification in diffusion MRI

Inferring brain connectivity and structure in-vivo requires accurate estimation of the orientation distribution function (ODF), which encodes key local tissue properties. However, estimating the ODF from diffusion MRI (dMRI) signals is a challenging inverse problem due to obstacles such as significant noise, high-dimensional parameter spaces, and sparse angular measurements. In this paper, we address these challenges by proposing a novel deep-learning based methodology for continuous estimation and uncertainty quantification of the spatially varying ODF field. We use a neural field (NF) to parameterize a random series representation of the latent ODFs, implicitly modeling the often ignored but valuable spatial correlation structures in the data, and thereby improving efficiency in sparse and noisy regimes. An analytic approximation to the posterior predictive distribution is derived which can be used to quantify the uncertainty in the ODF estimate at any spatial location, avoiding the need for expensive resampling-based approaches that are typically employed for this purpose. We present empirical evaluations on both synthetic and real in-vivo diffusion data, demonstrating the advantages of our method over existing approaches.

Modeling county level COVID-19 transmission in the greater St. Louis area: Challenges of uncertainty and identifiability when fitting mechanistic models to time-varying processes

We use a compartmental model with a time-varying transmission parameter to describe county level COVID-19 transmission in the greater St. Louis area of Missouri and investigate the challenges in fitting such a model to time-varying processes. We fit this model to synthetic and real confirmed case and hospital discharge data from May to December 2020 and calculate uncertainties in the resulting parameter estimates. We also explore non-identifiability within the estimated parameter set. We find that the death rate of infectious non-hospitalized individuals, the testing parameter and the initial number of exposed individuals are not identifiable based on an investigation of correlation coefficients between pairs of parameter estimates. We also explore how this non-identifiability ties back into uncertainties in the estimated parameters and find that it inflates uncertainty in the estimates of our time-varying transmission parameter. However, we do find that R0 is not highly affected by non-identifiability of its constituent components and the uncertainties associated with the quantity are smaller than those of the estimated parameters. Parameter values estimated from data will always be associated with some uncertainty and our work highlights the importance of conducting these analyses when fitting such models to real data. Exploring identifiability and uncertainty is crucial in revealing how much we can trust the parameter estimates.

Relationship between prediction accuracy and uncertainty in compound potency prediction using deep neural networks and control models

The assessment of prediction variance or uncertainty contributes to the evaluation of machine learning models. In molecular machine learning, uncertainty quantification is an evolving area of research where currently no standard approaches or general guidelines are available. We have carried out a detailed analysis of deep neural network variants and simple control models for compound potency prediction to study relationships between prediction accuracy and uncertainty. For comparably accurate predictions obtained with models of different complexity, highly variable prediction uncertainties were detected using different metrics. Furthermore, a strong dependence of prediction characteristics and uncertainties on potency levels of test compounds was observed, often leading to over- or under-confident model decisions with respect to the expected variance of predictions. Moreover, neural network models responded very differently to training set modifications. Taken together, our findings indicate that there is only little, if any correlation between compound potency prediction accuracy and uncertainty, especially for deep neural network models, when predictions are assessed on the basis of currently used metrics for uncertainty quantification.

Cryo-forum: A framework for orientation recovery with uncertainty measure with the application in cryo-EM image analysis

In single-particle cryo-electron microscopy (cryo-EM), efficient determination of orientation parameters for particle images poses a significant challenge yet is crucial for reconstructing 3D structures. This task is complicated by the high noise levels in the datasets, which often include outliers, necessitating several time-consuming 2D clean-up processes. Recently, solutions based on deep learning have emerged, offering a more streamlined approach to the traditionally laborious task of orientation estimation. These solutions employ amortized inference, eliminating the need to estimate parameters individually for each image. However, these methods frequently overlook the presence of outliers and may not adequately concentrate on the components used within the network. This paper introduces a novel method using a 10-dimensional feature vector for orientation representation, extracting orientations as unit quaternions with an accompanying uncertainty metric. Furthermore, we propose a unique loss function that considers the pairwise distances between orientations, thereby enhancing the accuracy of our method. Finally, we also comprehensively evaluate the design choices in constructing the encoder network, a topic that has not received sufficient attention in the literature. Our numerical analysis demonstrates that our methodology effectively recovers orientations from 2D cryo-EM images in an end-to-end manner. Notably, the inclusion of uncertainty quantification allows for direct clean-up of the dataset at the 3D level. Lastly, we package our proposed methods into a user-friendly software suite named cryo-forum, designed for easy access by developers.

Uncertainty-based saltwater intrusion prediction using integrated Bayesian machine learning modeling (IBMLM) in a deep aquifer

Data-driven machine learning approaches are promising to substitute physically based groundwater numerical models and capture input-output relationships for reducing computational burden. But the performance and reliability are strongly influenced by different sources of uncertainty. Conventional researches generally rely on a stand-alone machine learning surrogate approach and fail to account for errors in model outputs resulting from structural deficiencies. To overcome this issue, this study proposes a flexible integrated Bayesian machine learning modeling (IBMLM) method to explicitly quantify uncertainties originating from structures and parameters of machine learning surrogate models. An Expectation-Maximization (EM) algorithm is combined with Bayesian model averaging (BMA) to find out maximum likelihood and construct posterior predictive distribution. Three machine learning approaches representing different model complexity are incorporated in the framework, including artificial neural network (ANN), support vector machine (SVM) and random forest (RF). The proposed IBMLM method is demonstrated in a field-scale real-world "1500-foot" sand aquifer, Baton Rouge, USA, where overexploitation caused serious saltwater intrusion (SWI) issues. This study adds to the understanding of how chloride concentration transport responds to multi-dimensional extraction-injection remediation strategies in a sophisticated saltwater intrusion model. Results show that most IBMLM exhibit r values above 0.98 and NSE values above 0.93, both slightly higher than individual machine learning, confirming that the IBMLM is well established to provide better model predictions than individual machine learning models, while maintaining the advantage of high computing efficiency. The IBMLM is found useful to predict saltwater intrusion without running the physically based numerical simulation model. We conclude that an explicit consideration of machine learning model structure uncertainty along with parameters improves accuracy and reliability of predictions, and also corrects uncertainty bounds. The applicability of the IBMLM framework can be extended in regions where a physical hydrogeologic model is difficult to build due to lack of subsurface information.

Uncertainty quantification of the wall thickness and stiffness in an idealized dissected aorta

Personalized treatment informed by computational models has the potential to markedly improve the outcome for patients with a type B aortic dissection. However, existing computational models of dissected walls significantly simplify the characteristic false lumen, tears and/or material behavior. Moreover, the patient-specific wall thickness and stiffness cannot be accurately captured non-invasively in clinical practice, which inevitably leads to assumptions in these wall models. It is important to evaluate the impact of the corresponding uncertainty on the predicted wall deformations and stress, which are both key outcome indicators for treatment optimization. Therefore, a physiology-inspired finite element framework was proposed to model the wall deformation and stress of a type B aortic dissection at diastolic and systolic pressure. Based on this framework, 300 finite element analyses, sampled with a Latin hypercube, were performed to assess the global uncertainty, introduced by 4 uncertain wall thickness and stiffness input parameters, on 4 displacement and stress output parameters. The specific impact of each input parameter was estimated using Gaussian process regression, as surrogate model of the finite element framework, and a δ moment-independent analysis. The global uncertainty analysis indicated minor differences between the uncertainty at diastolic and systolic pressure. For all output parameters, the 4th quartile contained the major fraction of the uncertainty. The parameter-specific uncertainty analysis elucidated that the material stiffness and relative thickness of the dissected membrane were the respective main determinants of the wall deformation and stress. The uncertainty analysis provides insight into the effect of uncertain wall thickness and stiffness parameters on the predicted deformation and stress. Moreover, it emphasizes the need for probabilistic rather than deterministic predictions for clinical decision making in aortic dissections.

Model-based dimensionality reduction for single-cell RNA-seq using generalized bilinear models

Dimensionality reduction is a critical step in the analysis of single-cell RNA-seq (scRNA-seq) data. The standard approach is to apply a transformation to the count matrix followed by principal components analysis (PCA). However, this approach can induce spurious heterogeneity and mask true biological variability. An alternative approach is to directly model the counts, but existing methods tend to be computationally intractable on large datasets and do not quantify uncertainty in the low-dimensional representation. To address these problems, we develop scGBM, a novel method for model-based dimensionality reduction of scRNA-seq data using a Poisson bilinear model. We introduce a fast estimation algorithm to fit the model using iteratively reweighted singular value decompositions, enabling the method to scale to datasets with millions of cells. Furthermore, scGBM quantifies the uncertainty in each cell's latent position and leverages these uncertainties to assess the confidence associated with a given cell clustering. On real and simulated single-cell data, we find that scGBM produces low-dimensional embeddings that better capture relevant biological information while removing unwanted variation.

Intrinsic and extrinsic techniques for quantification uncertainty of an interpretable GRU deep learning model used to predict atmospheric total suspended particulates (TSP) in Zabol, Iran during the dusty period of 120-days wind

Total suspended particulates (TSP), as a key pollutant, is a serious threat for air quality, climate, ecosystems and human health. Therefore, measurements, prediction and forecasting of TSP concentrations are necessary to mitigate their negative effects. This study applies the gated recurrent unit (GRU) deep learning model to predict TSP concentrations in Zabol, Iran, during the dust period of the 120-day wind (3 June - 4 October 2014). Three uncertainty quantification (UQ) techniques consisting of the blackbox metamodel, heteroscedastic regression and infinitesimal jackknife were applied to quantify the uncertainty associated with GRU model. Permutation feature importance measure (PFIM), based on the game theory, was employed for the interpretability of the predictive model's outputs. A total of 80 TSP samples were collected and were randomly divided as training (70%) and validation (30%) datasets, while eight variables were used in the TSP prediction model. Our findings showed that GRU performed very well for TSP prediction (with r and Nash Sutcliffe coefficient (NSC) values above 0.99 for both datasets, and RMSE of 57 μg m-3 and 73 μg m-3 for training and validation datasets, respectively). Among the three UQ techniques, the infinitesimal jackknife was the most accurate one, while all the observed and predicted TSP values fell within the continence limitation estimated by the model. PFIM plots showed that wind speed and air humidity were the most and least important variables, respectively, impacting the predictive model's outputs. This is the first attempt of using an interpretable DL model for TSP prediction modelling, recommending that future research should involve aspects of uncertainty and interpretability of the predictive models. Overall, UQ and interpretability techniques have a key role in reducing the impact of uncertainties during optimization and decision making, resulting in better understanding of sophisticated mechanisms related to the predictive model.

Experimental vibration dataset collected of a beam reinforced with masses under different health conditions

Vibration signals extracted from structures across diverse health conditions have become indispensable for monitoring structural integrity. These datasets represent a resource for real-time condition monitoring, enabling the precise detection and diagnosis of system anomalies. This paper aims to enrich the scientific community's database on structural dynamics and experimental methodologies pertinent to system modelling. Leveraging experimental measurements obtained from mass-reinforced beams, these datasets validate numerical models, refine identification techniques, quantify uncertainties, and continuously foster machine learning algorithms' evolution to monitor structural integrity. Furthermore, the beam dataset is data-driven and can be used to develop and test innovative structural health monitoring strategies, specifically identifying damages and anomalies within intricate structural frameworks. Supplemental datasets like Mass-position and damage index introduce parametric uncertainty into experimental and damage identification metrics. Thereby offering valuable insights to elevate the efficacy of monitoring and control techniques. These comprehensive tests also encapsulate paramedic uncertainty, providing robust support for applications in uncertainty quantification, stochastic modelling, and supervised and unsupervised machine learning methodologies.